CN113834425A

CN113834425A - Method and device for three-dimensional measurement of objects in a central logistics repository

Info

Publication number: CN113834425A
Application number: CN202010579161.6A
Authority: CN
Inventors: 席鹏
Original assignee: BMW Brilliance Automotive Ltd
Current assignee: BMW Brilliance Automotive Ltd
Priority date: 2020-06-23
Filing date: 2020-06-23
Publication date: 2021-12-24
Anticipated expiration: 2040-06-23
Also published as: CN113834425B

Abstract

The present application relates to a method and apparatus for three-dimensional measurement of objects in a central logistics repository. There is provided a three-dimensional measuring device for a central logistics repository, comprising a frame, a pan-tilt, a pair of cameras, one of the cameras being a colour camera to facilitate plane acquisition of a target object and the other being a depth camera to facilitate depth acquisition of the target object, wherein the cameras are movably mounted to the frame by the pan-tilt, wherein the three-dimensional measuring device is configured such that: the pan/tilt head can remain stationary, so that the camera fixed to the pan/tilt head remains stationary, facilitating the acquisition of the camera; the pan/tilt head is movable according to a control, so that a camera fixed to the pan/tilt head moves therewith, wherein the pan/tilt head is configured to be rotatable in at least two dimensions. In particular, the three-dimensional measuring device is configured to implement the three-dimensional measuring method for the central logistics repository according to the invention.

Description

Method and device for three-dimensional measurement of objects in a central logistics repository

Technical Field

The invention relates to a method and a device for three-dimensional measurement of objects, in particular large objects, in a central logistics repository.

Background

The central logistics repository is an important link from the external production repository to distribution to downstream distributors. In the site of the central logistics warehouse, three-dimensional measurement is needed for objects, particularly large objects, for example, the length, width and height of the objects are measured. The size of a large object may be of the order of about 1m, or even larger, compared to, for example, a small object having a size of the order of about 30 cm.

In conventional measuring methods and devices, the measuring device is fixed after a one-time adjustment relative to the working environment has been carried out during installation. In the case of a measuring device comprising a camera, the adjustment here may involve, for example, determining the directionality of the camera plane, for example the vertical directionality in the case of a lateral measurement. The target object is then subjected to three-dimensional measurements at a relatively fixed distance.

However, due to differences in measurement principles, there are differences in measurement accuracy and different complexities in processing and storing data from cameras, for example, and some existing solutions may not be suitable for convenience and economy requirements in a central logistics repository scenario. Furthermore, due to the possible inherent defects of the camera and the wear that occurs over time, the initially fixed measuring device is no longer able to perform an accurate detection, and therefore it is not possible to accurately measure the three-dimensional dimensions of the object. In some cases, it may be that the measurement device is internally self-compensating, but such internal self-compensation may not be sufficient to limit the error to the desired range. But readjusting and re-fixing the measuring device again manually to release the fixing would be time consuming and laborious.

Disclosure of Invention

Therefore, there is a need to provide a method and apparatus for three-dimensional measurement of objects, particularly large objects, for a central logistics repository that can have good automation, expanded measurement conditions, improved measurement accuracy and reduced maintenance costs.

According to some aspects of the present invention, there is provided a three-dimensional measuring device for a central logistics repository, comprising a rack, a pan-tilt, a pair of cameras, one of the cameras being a color camera to facilitate plane acquisition of a target object and the other being a depth camera to facilitate depth acquisition of the target object, wherein the cameras are movably mounted to the rack by the pan-tilt, wherein the three-dimensional measuring device is configured such that: the pan/tilt head can remain stationary, so that the camera fixed to the pan/tilt head remains stationary, facilitating the acquisition of the camera; the pan/tilt head is movable according to a control, so that a camera fixed to the pan/tilt head moves therewith, wherein the pan/tilt head is configured to be rotatable in at least two dimensions. In particular, the three-dimensional measuring device is configured to implement the three-dimensional measuring method for the central logistics repository according to the invention.

According to some embodiments of the invention, the cameras are mounted in a manner that the preferably coplanar camera planes are substantially vertically above and below. In other words, the camera will acquire the measurement area from above towards below. Thus, in this case, the rotation of the head in at least two dimensions will correspond approximately to four directions of rotation (front-back, left-right) with respect to the vertical. This rotation will adjust the bi-level of the camera plane. Preferably, the camera plane is positioned at a height of about 2.5-3m from the measurement area where the target object is positioned. This orientation and/or position arrangement is very advantageous for a central logistics warehouse scenario, as the target object may arrive at the measurement area in a certain direction and leave the measurement area after the measurement is completed, and this arrangement is not prone to interfere with logistics transportation and to interference from the surrounding environment.

According to some embodiments of the invention, the frame comprises a vertical column and a horizontal column connected substantially perpendicular to the vertical column, wherein the pan-tilt head is mounted to the horizontal column. Preferably, the bottom end of the upright is fixed to the support panel or directly to the ground, and the inner end of the cross-post is fixed to the top end of the upright, wherein the pan-tilt head is mounted to the outer end of the cross-post. In this way, the arrangement of the uprights and cross-posts of the frame and the connection of the head to the frame bring about a simple but sturdy support structure.

According to some embodiments of the invention, the head comprises a connecting portion, a first dimension handling portion, a second dimension handling portion, wherein the connecting portion is connected to the frame, preferably to the crosspiece of the frame, in particular to the outer end thereof, and the first dimension handling portion is connected to both the second dimension handling portion and the connecting portion. Accordingly, the camera is connected to the second dimension manipulation section. In practice, the first dimension manipulation part, the second dimension manipulation part, the camera form a cascaded relationship, which means that operation of the first dimension manipulation part results in a common movement of the second dimension manipulation part together with the camera, and operation of the second dimension manipulation part results in a movement of the camera. The rotation of the pan/tilt head in at least two dimensions is also distributed to a first dimension manipulation section that realizes two of the above four rotation directions (forward and reverse rotation along one axis) and a second dimension manipulation section that realizes the other two of the above four rotation directions (forward and reverse rotation along the other axis). In particular, the two axes are substantially orthogonal to each other and both are substantially parallel to the camera plane.

According to some embodiments of the invention, the connecting portion comprises a connecting pinch plate layer. The first dimension manipulation section includes: a first steering engine; the shell of the first steering engine is fixed on the first clamping plate layer; and the rotation output end of the first steering engine is connected to the rotating layer. The second dimension manipulation section includes: a second steering engine; the shell of the second steering engine is fixed on the second clamping plate layer; and the rotation output end of the second steering engine is connected to the swing arm part. The first clamping plate layer is fixed to the connecting clamping plate layer, the second clamping plate layer is fixed to the rotating layer, and the camera is mounted on the swing arm. Preferably, the rotational outputs of the first and second steering engines are substantially orthogonal with their axes. Preferably, the connecting clamping plate layer, the first clamping plate layer, the rotating layer and the second clamping plate layer are kept basically parallel. In this way, a multi-layer arrangement of the head structure is formed, which not only can be easily adapted to the dimensions of the steering engine and/or the camera etc., but also provides a relatively rigid construction, ensuring stability of the steering in two dimensions.

According to some embodiments of the invention, the color camera comprises an area-array (i.e., two-dimensional) and/or wide-angle camera, and/or the depth camera comprises a structured-light based depth camera.

According to some embodiments of the invention, the three-dimensional measurement device is configured to at least partially accept remote manipulation.

According to some aspects of the present invention, there is provided a three-dimensional measurement method for a central logistics repository, comprising an adjustment step and a measurement step, wherein the measurement step comprises: preparing to carry out three-dimensional measurement, respectively carrying out plane acquisition and depth acquisition on a target object by utilizing a color camera and a depth camera which are fixedly arranged on a holder, and calculating the three-dimensional size of the target object based on the acquired information; wherein the adjusting step comprises rotating a pan/tilt head to which the color camera and the depth camera are fixed in at least two dimensions so as to reach the adjusted reference position; wherein the adjusting step comprises an adjustment using a standard, the adjustment using a standard comprising the acts of: rotating the pan-tilt in a first dimension and a second dimension, and confirming a position reaching a minimum measurement height from a measurement area based on the acquisition of the depth camera; starting from this position, changing the position of the head in at least one of the first dimension and the second dimension; determining a three-dimensional measurement dimension of a standard placed in the measurement area; repeating the substeps of changing the position of the head and determining the three-dimensionally measured dimension to establish the adjusted reference position based on at least a loss function characterizing the difference between the three-dimensionally measured dimension and the actual dimension of the standard or a derivative function of the gradient thereof. Preferably, the standard is directly under the camera and/or is a standard cube of 1mx1mx1 m.

According to some embodiments of the invention, the adjusted reference position is established in case the value of the loss function or a derivative function of its gradient is within a predetermined range, thereby completing the adjustment with the standard. Preferably, in the case where the adjusted reference position is established in the adjustment with the standard, a first current angle and a second current angle at which the head is located in the first dimension and the second dimension are recorded. The first current angle and the second current angle are of significance here, for example, for the following reference-free adjustment.

According to some embodiments of the invention, wherein after a given number of repetitions or adjustment time, when the value of the loss function or a derivative of its gradient is outside a predetermined range, the adjustment with the standard is terminated and/or a manual adjustment is made by notification. In other words, the adjustment with the standard is preferably not continued endlessly.

According to some embodiments of the invention, the pan-tilt head is caused to change the defined step in one of the first and second dimensions at a time. Thus, the head changes in only one degree of freedom at a time. Preferably, the step is 1/4096 degrees. In this way, the head can be moved, for example by a single movement, with the minimum permissible rotational amplitude, it being possible to advantageously avoid overshooting.

According to some embodiments of the invention, after each pan-tilt change position, the value of the loss function or of the derivative of its gradient before the current change is compared with the current value, and if the current change causes said value to rise, the next change causes the pan-tilt to move at least two steps along the same dimension but in the opposite direction to the current change. Thus, by comparison, it is possible to clearly confirm whether the adjustment of the head in a given dimension is effective.

According to some embodiments of the invention, each pan-tilt repositioning is alternated in a first dimension and in a second dimension, in case the value of the loss function or of a derivative of its gradient has a non-ascending trend. Thus, by alternating, it is ensured that the head can be effectively adjusted in both dimensions.

According to some embodiments of the invention, the loss function is a weighted quadratic distance of the actual dimension of the standard and the three-dimensionally measured dimension, wherein the weighting factor is positive. It is particularly preferred that the derivative function of the gradient of the loss function comprises or is the sum of the squares of the variables of each dimension of the gradient of the loss function. Thus, such a derivative function may be used in the above-described steps of the adjustment using the standard. The applicant has noted that this allows to judge more significantly the correctness of the adjustment direction and possibly to avoid local oscillations of the resulting value.

According to some embodiments of the invention, the adjusting step further comprises a reference-free adjustment comprising the acts of: rotating the pan-tilt along a first dimension, confirming a first position reaching a minimum of a measured height from the measurement area based on the acquisition of the depth camera, thereby determining a first angle at which the pan-tilt is associated with the first dimension; the pan/tilt head is rotated along a second dimension, and a second position that reaches a minimum of the measured height from the measurement area is identified based on the acquisition of the depth camera, thereby determining a second angle of the pan/tilt head associated with the second dimension. Preferably, the rotational movement of the head in the first and second dimensions is in the range of 0 ° to 180 °. Preferably, the difference of the determined associated angle from the predetermined angle is compared (e.g. to determine if different or a threshold is exceeded) in order to establish an adjusted reference position.

According to some embodiments of the invention, the adjustment without reference and the adjustment with the standard may be performed independently or may be performed in a coupled manner. Preferably, the adjustment without reference is performed such that, in the case where the adjusted reference position is established in the adjustment without reference prior to the adjustment with the standard, the subsequent adjustment with the standard is not performed any more. On the other hand, in the case where the correlation angle is different from the predetermined angle or the difference exceeds the threshold value in the adjustment without the reference object, the adjustment using the standard is performed instead. In this way, it is possible to switch to an adjustment using a standard after the determination of the angular difference along one dimension, so that in the adjustment using a standard, the pan/tilt head may only need to reach the position of the minimum measured height from the measurement region in the other dimension, since the position of the minimum measured height from the measurement region in the one dimension was reached before the determination of the angular difference in the one dimension. Preferably, the first predetermined angle, the second predetermined angle are initially defined angles and/or are updated by a first current angle and a second current angle at which the head is located in the first dimension and the second dimension, which are recorded in the case of an adjusted reference position being established in the adjustment with the standard. That is, the two angles of the pan and tilt head associated with the two dimensions that characterize the adjusted reference position in the adjustment with the standard may be used to update the predetermined angle used in the adjustment without the reference.

According to some embodiments of the invention, the adjusting step may have one or more of the following situations: before the target object is measured; after evaluating and calculating the three-dimensional size of the target object; including in the sub-step of preparing for three-dimensional measurements.

According to some embodiments of the present invention, after plane acquisition of a target object using a color camera, a pixel region of the target object is determined based on a result of the two-dimensional acquisition; after the target object is subjected to depth acquisition by using a depth camera, determining the zoom ratio of the target object based on the result of the depth acquisition; based on the result of the determination, the three-dimensional size of the target object is calculated.

According to some embodiments of the invention, when the thickness of the target object is less than 3%, 2%, 1% of the height of the camera plane defined by the camera from the measurement area where the target object is located, the planar two-dimensional size calculation of the target object is decoupled from the height one-dimensional size calculation when calculating the three-dimensional size of the target object.

According to some embodiments of the invention, at least some steps of the method are manipulated remotely.

According to some aspects of the present invention, there is provided a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements a three-dimensional measurement method for a central logistics repository according to the present invention.

Drawings

FIG. 1 illustrates an exemplary block schematic diagram of a three-dimensional measurement device according to some embodiments of the invention.

FIG. 2 illustrates an exemplary partial schematic view of a three-dimensional measurement device according to some embodiments of the invention.

FIG. 3 illustrates an exemplary exploded schematic view of a three-dimensional measurement device according to some embodiments of the invention.

Fig. 4A-4C illustrate plan views of exemplary holders for three-dimensional measurement devices according to some embodiments of the present invention.

5A-5C illustrate plan views of exemplary steering engines of three-dimensional measurement devices according to some embodiments of the invention.

FIG. 6 shows a flow chart of the measurement steps of a three-dimensional measurement method according to some embodiments of the invention.

Fig. 7 shows an exemplary illustration of the measurement principle of the three-dimensional measurement method according to some embodiments of the invention.

FIG. 8 illustrates a flow chart of the adjustment steps of a three-dimensional measurement method according to some embodiments of the invention.

FIG. 9 illustrates a flow chart of the adjustment steps of a three-dimensional measurement method according to some embodiments of the invention.

FIG. 10 illustrates a flow chart of the adjustment steps of a three-dimensional measurement method according to some embodiments of the invention.

Detailed Description

The present invention is described with reference to the accompanying drawings, in which certain embodiments of the invention are shown. This invention may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth and described herein; rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art. It will also be appreciated that the embodiments disclosed herein may be combined in any manner and/or combination to provide many additional embodiments.

Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. The terminology used in the description above is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention.

For purposes of illustration, the same reference numbers will be used in the drawings to refer to the same or like modules, units, and/or components, and the parts will not necessarily be to scale.

In conventional measurement methods and devices, maintenance is usually manually operated and measurement environments are often limited to specific areas, and generally speaking, the measurement accuracy is in the order of ± 5% error. According to the present invention, a three-dimensional measurement method and apparatus for a central logistics repository are provided, which can be automatically adjusted, maintained via remote control, and have good adaptability to the measurement environment, and the measurement accuracy can reach ± 2% or even better error magnitude through testing.

Referring to fig. 1, an exemplary block schematic diagram of a three-dimensional measurement device 1 according to some embodiments of the invention is shown. As shown in the drawing, the three-dimensional measuring device 1 for a central logistics repository includes a rack 10, a pan-tilt 20, and a pair of

cameras

30a, 30 b. The pair of

cameras

30a, 30b are movably mounted to the gantry 10 by the pan and tilt head 20. The three-dimensional measurement apparatus 1 is configured such that: the head 20 can remain stationary, so that the

cameras

30a, 30b fixed to the head 20 remain stationary, to facilitate the acquisition of the

cameras

30a, 30 b; the head 20 is movable according to a control, and therefore the

cameras

30a, 30b fixed to the head 20 move therewith, wherein the head 20 is configured to be rotatable in at least two dimensions. In particular, the three-dimensional measuring device 1 is configured to implement the three-dimensional measuring method for the central logistics repository according to the present invention, as described in detail below. In the present invention, the pair of

cameras

30a, 30b may be a color camera and a depth camera, respectively. The color camera is configured to perform plane acquisition on a target object, and the depth camera is configured to perform depth acquisition on the target object. The three-dimensional measurement device 1 may also include other components including, but not limited to, a power module, a control module, a communication module, and the like.

Referring to fig. 2, an exemplary partial schematic view of a three-dimensional measurement device 1 according to some embodiments of the invention is shown. In practice, the figure shows a particular orientation. In particular, the

cameras

30a, 30b are mounted with the preferably coplanar camera planes 300 facing substantially vertically downward. Thus, the

cameras

30a, 30b will acquire the measurement region R from above towards below. Accordingly, in the illustrated embodiment, rotation of the head 20 in at least two dimensions will generally correspond to four rotational directions relative to vertical (e.g., two directions into and out of the page). This rotation will adjust the bi-level of the camera plane. Although the measurement region R is shown as belonging to a portion of the ground 2, in other embodiments not shown, the measurement region R may be arranged differently, as a non-limiting example, it may belong to a portion of a platform located above the ground 2. Preferably, the camera plane 300 is positioned at a height H of about 2.5-3m from the measurement region R where the target object is positioned.

Referring to fig. 3, an exemplary exploded schematic view of a three-dimensional measurement device 1 according to some embodiments of the invention is shown. The configuration of the gantry 10, the pan and tilt head 20, respectively, is described below, with the

cameras

30a, 30b not shown for clarity, which is described in detail below.

The frame 10 comprises a vertical column 101 and a horizontal column 102, wherein the vertical column 101 and the horizontal column 102 are connected substantially vertically. The head 20 will be mounted to the cross-post 102. In the illustrated embodiment, the bottom end of the mast 101 is secured to the support panel 103 and the inner end of the cross-post 102 is secured to the top end of the mast 101. The head 20 will be mounted to the outer end of the cross-post 102. It should be noted that the uprights 101, the crosspieces 102 and the head 20 may also have other forms of connection, connection positions, etc., and that the support panel 103 may have other shapes than the rectangular one illustrated, by way of non-limiting example, or the uprights 102 may be fixed directly to the ground 2 without using the support panel 103, or the uprights 101 and the crosspieces 102 may be connected in their intermediate, rather than end, positions.

The pan/tilt head 20 includes a connecting portion 200, a first dimension manipulating portion 201, and a second dimension manipulating portion 202, which are arranged from the left side to the right side in the drawing. The connecting portion 200 is connected to the frame 10, i.e. the head 20 is connected to the frame 10 by its connecting portion 200. In the illustrated embodiment, the attachment portion 200 is attached to the cross-post 102 of the stand 10, specifically the outer end of the cross-post 102. The first-dimension manipulation section 201 is connected to both the second-dimension manipulation section 202 and the connection section 200. Accordingly, the

cameras

30a, 30b, not shown, are connected to the second-dimension manipulation section 202. In this way, the first-dimension manipulation section 201, the second-dimension manipulation section 202, and the

cameras

30a, 30b form a cascade relationship, and therefore the operation of the first-dimension manipulation section 201 causes the second-dimension manipulation section 202 to move together with the

cameras

30a, 30b, and the operation of the second-dimension manipulation section 202 causes the

cameras

30a, 30b to move. The rotation of the pan/tilt head 20 in at least two dimensions will be assigned to the first dimension manipulation part 201 and the second dimension manipulation part 202. Specifically, the first-dimension manipulation section 201 realizes two of the above-described four rotational directions (for example, forward and reverse rotations into and out of the paper), and the second-dimension manipulation section 202 realizes the other two of the above-described four rotational directions (forward and reverse rotations along the paper). Therefore, the dimensions manipulated by the first-dimension manipulation part 201 and the second-dimension manipulation part 202 are substantially orthogonal to each other, and the axis of rotation is substantially parallel to the camera plane 300.

The detailed configuration of each part of the pan/tilt head 20 is described below.

The connection portion 200 includes a connection pinch plate layer 2000. In the illustrated embodiment, the connecting clip layer 2000 is formed in a rectangular shape, but may have other suitable shapes. The connecting clamping plate layer 2000 is provided with holes 2000-1 (only some of which are shown) therein, for example for a fixed connection with the cross-post 102 and/or a fixed connection with the first dimension handling portion 201.

The first-dimension manipulation portion 201 includes a first steering engine 2010, a first clamp plate layer 2011, and a rotation layer 2012. The housing of first steering engine 2010 is secured to first clamp plate layer 2011 and the rotational output of first steering engine 2010 is connected to rotating layer 2012. In the illustrated example, the first chuck layer 2011 and the rotation layer 2012 are formed in a circular shape, but may have other suitable shapes. The first clamp plate layer 2011 is secured to the connecting clamp plate layer 2000, preferably by connecting rods 2011-1, particularly in alignment with at least some of the holes 2000-1. Holes 2012-1 (only some of which are shown) are provided in the spin layer 2012, e.g., for fixed connection to the rotational output and/or to the second dimension manipulation portion 202.

The second-dimension manipulation section 202 includes a second steering engine 2020, a second clamping plate layer 2021, and a rocker arm portion 2012. The housing of the second steering engine 2020 is fixed to the second clamping plate layer 2021, and the rotation output of the second steering engine 2020 is connected to the rocker arm portion 2012. In the illustrated example, the second clamp plate layer 2021 is formed in a circular shape and the rocker arm portion 2012 is formed in a C-shape, but may have other suitable shapes. The second clamping ply 2021 is secured to the rotating layer 2012, preferably by connecting bars 2021-1, particularly in alignment with at least some of the holes 2012-1.

Cameras

30a, 30b, not shown, would be mounted to the swing arm 2012. Preferably, the rotational outputs of the first steering engine 2010 and the second steering engine 2020 are substantially orthogonal with their axes. In fact, these two axes may correspond to one axis of the first-dimension manipulation part 201 and the other axis of the second-dimension manipulation part 202 described above.

Preferably, the connecting clamp layer 2000, the first clamp layer 2011, the rotating layer 2012, and the second clamp layer 2021 remain substantially parallel and are preferably substantially aligned extending to the right about respective central longitudinal axes (substantially from the left as shown). In this way, a multi-tiered pan-tilt structure is formed that is compact and stable.

4A-4C, plan views of an exemplary pan and tilt head 20 of a three-dimensional measurement device 1 are shown, according to some embodiments of the present invention.

An exemplary side view of the head 20 can be seen in fig. 4A-4B. In particular, the steering engine 2020 of the second manipulation section 202 of the head 20 is indirectly mounted to the second clamping plate layer 2021 via the intermediate bracket 2023. From figure 4C, the connection aperture 2011-2 of the first clip layer 2011 can be seen, which receives one end of the connecting bar 2011-1, for example. The first clamping plate layer 2011 is further provided with a substantially rectangular opening 2011-3 through which the housing of the first steering engine 2010 is to be partially passed and secured. It is to be noted that only possible examples are shown here, and that the various parts of the head 20 may be connected directly or indirectly in a suitable manner without departing from the scope and idea of the present invention.

Referring to fig. 5A-5C, plan views of

exemplary steering engines

2010, 2020 of three-dimensional measurement devices 1 are shown, according to some embodiments of the invention. In particular, steering

engines

2010, 2020 include or are used in conjunction with position/angle detectors (e.g., potentiometers, encoders, etc.) to give information about the steering engine rotational position/angle. The mating relationship of the

exemplary steering engines

2010, 2020 and the first clamping plate layer 2011 of fig. 4C above is seen in fig. 5C. It should be noted that only possible examples are shown here, and that the

steering engines

2010, 2020 may have other configurations without departing from the scope and spirit of the present invention.

It is preferred in the present invention that the color camera comprises an area-array (i.e. two-dimensional) and/or wide-angle camera, and/or that the depth camera comprises a structured-light based depth camera. First, this configuration avoids the use of laser scanning methods and apparatus to increase equipment costs. Moreover, structured light has clear advantages in the context of the application of the invention compared to other possible depth measurement techniques.

The depth measurement technology for the structured light is generally suitable for being used in a short-distance scene (0.3m-5m), and the precision is high. Within 1m, the precision can be 1mm, and the method can be even used for 3D face recognition. The distance error between 1m and 5m can be controlled in millimeter level. The resolution ratio is higher, and the ambient light influence is little, and the power consumption is low. Although the distance error increases rapidly and the accuracy is poor, this disadvantage is not highlighted in view of the preferred measuring distance and measuring object of the present invention.

For TOF (time of flight) depth measurement techniques, the error is calculated as a percentage, e.g. around 2%, so that at far distances the accuracy is much higher and the response time is fast compared to other solutions. However, in the current TOF depth measurement technology, the device resolution is low, and most of the TOF depth measurement technology is QVGA. The accuracy error of the percentage is relatively poor at close distances and is therefore not suitable.

For the depth measurement technology of binocular vision, the precision is high, the resolution is high, but the depth measurement technology is easily influenced by illumination, the algorithm difficulty is relatively complex, the response speed is slow, the frame rate is low, and the system resource requirement is large, so that the depth measurement technology is not suitable.

In the above apparatus, the apparatus is configured to at least partially accept remote manipulation.

Referring to fig. 6, a flow chart illustrating the measurement steps of a three-dimensional measurement method according to some embodiments of the present invention is shown. Preferably, the three-dimensional measuring device 1 according to the invention is configured to perform the procedure as shown in fig. 6, essentially comprising the following steps: 601, preparing to carry out three-dimensional measurement; 602, performing plane acquisition and depth acquisition on a target object by using a color camera and a depth camera respectively; 603, calculating the three-dimensional size of the target object based on the acquired information. It is noted that although the color camera and depth camera based acquisitions are described together herein, the acquisition process may be performed serially, separately, i.e., with the acquisition of one camera following the acquisition of the other camera. Optionally, the target object is evaluated after its three-dimensional dimensions are calculated, for example manually.

Referring to fig. 7, an exemplary illustration of the measurement principle of a three-dimensional measurement method according to some embodiments of the invention is shown.

Referring to fig. 7, the camera plane is substantially vertically oriented downward with its origin indicated by reference numeral O, and the object has an apex A, D, E, F (e.g., a generally cubic shape) and is placed within the measurement region, wherein the distance (e.g., height) from the camera plane (e.g., origin O) to the measurement region is H, the distance (e.g., height) from the camera plane (e.g., origin O) to the upper surface of the object is H, and the pixel size of the measurement region is L (accordingly, the actual size of the measurement region is previously given or calibrated to be L)_real) The pixel size of the object is the line segment BC. In practice, the real pixel size of the object is the line segment AD. When measuring the dimension of the object as shown in the figure, the following calculation is performed:

wherein, can be

Defined as the height ratio (zoom ratio).

When the height of the object is high, the pixel sizes of the line segments BE, FC characterizing the deviations are not negligible, and therefore the dimensional size of the object is calculated as above. On the other hand, when the height of the object is low, the line segments BE, FC characterizing the deviation may BE negligible, in other words, the scaling ratio is less than 1 but close to 1, and then the dimension of the object may BE calculated as follows:

therefore, when calculating the three-dimensional size of the target object, for example, when the target object is a thin plate type (e.g., the thickness is less than 3%, 2%, 1% of the height H), the calculation of the two-dimensional size of the target object in the plane is decoupled from the calculation of the one-dimensional size of the height.

In practice, the above-described characterization (pixel size, actual size, etc.) regarding the measurement region R may utilize reticles arranged in a predetermined pattern within the measurement region R, such as a reticle frame of known dimensions. The pixel area of the target object is determined based on the results of the two-dimensional acquisition, and thus the pixel size of the target object can be determined. Similarly, the pixel area of the reticle (e.g., the reticle frame) is determined based on the results of the two-dimensional acquisition, and thus the characterization of the predetermined pattern of the reticle on the pixels (e.g., the pixel size of the reticle frame) can be determined. Thus, the above calculation formula is obtained.

It is noted that although the calculation of the dimensions of one planar dimension of the object has been described above in terms of the exemplary illustration of fig. 7, it is clear that the same concept is also applicable to the calculation of the dimensions of another planar dimension orthogonal thereto, and possibly of other non-orthogonal planar dimensions.

Theoretically, the length, width and height information of the object can be obtained by performing scatter modeling based on the collected information of the depth camera. For example, (3D) structured light technology itself can be used for stereo vision within 5m close range accuracy requirements at the millimeter level. However, in consideration of the comprehensive application environment and conditions of the present invention, it is not only necessary to consider the measurement of the object in three dimensions, but also to consider factors such as convenience in data processing in practical use. Such modeling, which is commonly done, lacks intuitiveness and is not conducive to archiving. In contrast, measurement calculations as described above are particularly advantageous based on the fact that the depth camera mainly obtains height information (thereby obtaining a so-called zoom ratio), in combination with information based on the two-dimensional camera, the acquired results can be visually observed, recording is facilitated, and data processing can be simple and convenient. This is different from the usual concept.

According to some embodiments of the invention, a depth camera is used only when measuring height (e.g., a two-dimensional camera may not be able to obtain height information). Then in the case where the depth camera includes a structured-light based depth camera, the structured light is used only to measure depth/height (e.g., of the object, of the measurement area). Of course, suitable processing of the results based on the two-dimensional acquisition is well known to those skilled in the art, including but not limited to perspective correction, perspective compensation, binarization, edge recognition, and the like.

In the present invention, the three-dimensional measurement method comprises an adjustment step comprising the pan/tilt head 20, which is concomitantly fixed thereto by rotating the

cameras

30a, 30b, so as to reach the adjusted reference position. In this process, the

cameras

30a, 30b move with the head 10, and therefore the orientation of the camera plane 300 changes. Preferably, the three-dimensional measuring device 1 according to the invention is configured to perform said adjusting step. Preferably, the adjustment may be performed before the target object is acquired (and thus may be included in step 601) or may be initiated after the evaluation of the three-dimensional size of the target object, for example, manually initiated.

The adjusting step comprises implementing one or more of the adjusting protocols: with adjustment of the standard and/or adjustment without a reference.

Referring to fig. 8, a flow chart illustrating the adjustment steps of a three-dimensional measurement method according to some embodiments of the present invention is shown. Preferably, the three-dimensional measuring device 1 according to the invention is configured to perform the adjusting step herein. The adjusting step here essentially comprises the following steps: rotating 801 the head 20 along a first dimension (e.g. in the range 0 ° -180 °), confirming a first position reaching a minimum of the measured height from the measurement region R based on the acquisition of the depth camera, thereby determining a first angle of the head 20 associated with the first dimension; comparing 802 the difference of the first angle to a first predetermined angle; 803, rotating the head 20 along a second dimension (e.g., in the range of 0 ° -180 °), confirming a second position reaching a minimum of the measured height from the measurement region R based on the acquisition of the depth camera, thereby determining a second angle of the head 20 associated with the second dimension; 804, comparing the difference of the second angle and the second preset angle; an adjusted reference position is established 805. In the illustrated embodiment, while the comparison is described as being performed after each determination, it is to be understood that the comparison may be performed together after all determinations are made. Here, the first and second predetermined angles may be angles defined when the three-dimensional measuring apparatus 1 is initially installed, or may be angles obtained by adjustment using a standard member as described below. When the difference from the predetermined angle is within a predetermined range, the adjustment without reference is considered to be completed. However, such reference-free adjustment may not be satisfactory when the difference from the predetermined angle is outside the predetermined range. It is then possible that the adjustment without reference is terminated early (e.g. rotation and/or comparison with respect to the other dimension is no longer performed) and/or that instead an adjustment with a standard is performed in case a negative result is obtained with respect to the comparison with respect to one dimension.

Referring to fig. 9, a flow chart illustrating the adjustment steps of a three-dimensional measurement method according to some embodiments of the invention is shown. Preferably, the three-dimensional measuring device 1 according to the invention is configured to perform the adjusting step herein. The adjusting step here essentially comprises the following steps: a stage 901 for bringing the pan/tilt head 20 to a position of minimum measurement height from the measurement region R in both the first dimension and the second dimension, thereby determining a coarse position of the pan/tilt head (for example, a coarse horizontal position of a camera plane in a case where the camera is substantially vertically above and below the camera plane); starting from this coarse position, changing the position of the head in at least one of the first and second dimensions (e.g. by changing the angle, in particular the operation of the steering engine); determining 903 a three-dimensional measurement dimension of a standard (e.g., a standard cube of 1mx1mx1m) placed in the measurement region R (e.g., directly below the camera); 902, 903 are repeated to establish 904 an adjusted reference position based at least on a loss function characterizing the difference in the three-dimensional measured dimension and the actual dimension of the standard or a derivative function of the gradient thereof. For example, the adjusted reference position is established in the case where the value of the loss function or a derivative function of its gradient is within a predetermined range, and the adjustment using the standard is considered to be completed. However, such a standard adjustment may not be sufficient when the value of the loss function or a derivative of its gradient is outside a predetermined range. It is then possible to terminate the adjustment with the standard element and/or to notify a manual adjustment in case negative results are still obtained a certain number of times or for a certain time.

It is noted that the action of reaching the coarse position in the above-described adjustment with the standard is found to be similar to the turning action in the above-described adjustment without reference, but the comparison action in the adjustment without reference is no longer necessary. Indeed, according to some embodiments of the invention, the two adjustments may be coupled at least in some steps. Of course, in other embodiments, each adjustment may be performed independently.

According to some embodiments of the invention, the pan-tilt is caused to change a defined minimum rotation amplitude (step) in one of the first and second dimensions at a time, for example 1/4096 degrees.

According to some embodiments of the invention, after each pan-tilt change position, the value of the loss function or of the derivative of its gradient before the current change is compared with the current one, and if the current change does not allow said value to be reduced (and possibly kept unchanged), the next change moves the pan-tilt by at least two steps along the same dimension but in the opposite direction to the current change.

According to some embodiments of the invention, each pan-tilt repositioning is alternated in a first dimension and in a second dimension, with a correct decrease in the value of the loss function or of a derivative of its gradient.

For example, in one non-limiting example, the pan/tilt head may move in the following pattern (where two directions for the first dimension are denoted by w +, w-, and two directions for the second dimension are denoted by l +, l-):

w->l+>l->l->w->l->w->l->w-

wherein l + followed by two l-embodies the process of changing the commutation as described above.

According to some embodiments of the invention, the loss function is preferably:

loss＝a₁*(x-x₀)²+a₂*(y-y₀)²+a₃*(z-z₀)²

wherein, a₁-a₃Is a positive coefficient, x₀，y₀，z₀Is the actual size of the standard (e.g., 1mx1mx1m) and x, y, z are three-dimensionally measured sizes.

The gradient of the loss function can be calculated as:

then in the case of the preferred loss function described above, the gradient can be calculated as:

according to some embodiments of the invention, it is particularly preferred to use the sum of the squares of the variables of each dimension of the gradient of the loss function (i.e. the so-called derivative function of the gradient of the loss function) in step 904.

Of course, other forms of loss functions and gradients thereof are known to those skilled in the art.

Referring to fig. 10, a flow chart illustrating the adjustment steps of a three-dimensional measurement method according to some embodiments of the present invention is shown. Preferably, the three-dimensional measuring device 1 according to the invention is configured to perform the adjusting step herein. The adjustment steps here basically include no reference adjustment 1001, manual adjustment with standard adjustment 1002 and possible notification 1003. The reference-free conditioning 1001 comprises steps 1001-1 to 1001-5 similar to steps 801-805. 1000 represents establishing an adjusted reference position. As shown, when the comparison steps 1001-2, 1001-4 have a negative result, the adjustment 1002 using the standard is turned to. Here, the adjustment 1002 with the standard does not show a specific step, but it is clear that, with regard to the determination of the coarse position (reference step 901), the head 20 has in fact reached a position in one or both of the two dimensions at which the minimum of the measured height from the measuring region R is reached. Thus, with respect to the determination of the coarse position, the head 20 may no longer need to be rotated or only need to reach a position in the remaining dimension at which the minimum of the measured height from the measurement area is reached. Next, if a positive result is obtained with the adjustment 1002 of the standard, an adjusted reference position can be established 1000. Preferably, the predetermined angle used in the adjustment without reference can be updated to two angles associated with two dimensions by the head 20 characterizing the adjusted reference position in the adjustment of the standard. If a negative result is obtained with the standard adjustment 1002, it indicates that the adjustments have all failed. According to some embodiments of the invention, the method may include notifying a manual adjustment, for example, for maintenance in the event that the adjustment may still not meet the intended purpose.

In the above method, at least some steps of the method are manipulated remotely, e.g. the above measurements and/or adjustments may all be manipulated remotely.

It is noted that the method steps do not have to be performed in the depicted sequence without departing from the scope and spirit of the present invention.

By utilizing the three-dimensional measuring device and the three-dimensional measuring method for the central logistics warehouse, the measuring precision is improved by about three percentage points, the device is not limited due to self-adjustment, flexibly adapts to the actual measuring environment, preferably adopts the structured light technology to be superior to technologies such as TOF (time of flight), binocular and the like, and also provides the possibility of remote control.

The foregoing is illustrative of the present invention and is not to be construed as limiting thereof. Although exemplary embodiments of this invention have been described, those skilled in the art will readily appreciate that many modifications are possible in the exemplary embodiments without materially departing from the novel teachings and advantages of this invention. Accordingly, all such modifications are intended to be included within the scope of this invention as defined in the claims. The invention is defined by the following claims, with equivalents of the claims to be included therein.

Claims

1. A three-dimensional measurement method for a central logistics warehouse comprises an adjusting step and a measuring step, wherein the measuring step comprises the following steps:

is ready for a three-dimensional measurement to be made,

the color camera and the depth camera which are fixedly arranged on the holder are utilized to respectively carry out plane acquisition and depth acquisition on the target object,

calculating the three-dimensional size of the target object based on the acquired information;

wherein the adjusting step comprises rotating a pan/tilt head to which the color camera and the depth camera are fixed in at least two dimensions so as to reach the adjusted reference position;

wherein the adjusting step comprises an adjustment using a standard, the adjustment using a standard comprising the acts of:

a. rotating the pan-tilt in a first dimension and a second dimension, and confirming a position reaching a minimum measurement height from a measurement area based on the acquisition of the depth camera;

b. starting from this position, changing the position of the head in at least one of the first dimension and the second dimension;

c. determining a three-dimensional measurement dimension of a standard placed in the measurement area;

d. repeating the substeps b, c of changing the position of the head and determining the three-dimensional measurement, so as to establish the adjusted reference position based at least on a loss function characterizing the difference between the three-dimensional measurement and the actual size of the standard or a derivative function of the gradient thereof.

2. The three-dimensional measurement method according to claim 1, wherein the adjusted reference position is established in a case where a value of the loss function or a derivative function of a gradient thereof is within a predetermined range, whereby the adjustment using the standard is completed.

3. The three-dimensional measurement method according to claim 2, wherein, in a case where the adjusted reference position is established in the adjustment using the standard, a first current angle and a second current angle at which the pan-tilt head is located in the first dimension and the second dimension are recorded.

4. The three-dimensional measurement method according to claim 1, wherein, after a given number of repetitions or adjustment time, when the value of the loss function or a derivative function of its gradient is outside a predetermined range, the adjustment using the standard is terminated, and/or manual adjustment is performed by notification.

5. The three-dimensional measurement method of any one of claims 1-4, wherein the pan-tilt is changed by a defined step in one of the first and second dimensions at a time.

6. The three-dimensional measurement method of claim 5, wherein the step is 1/4096 degrees.

7. The three-dimensional measurement method according to claim 5, wherein after each pan/tilt head change position, the value of the loss function or a derivative of its gradient before the current change is compared with the current value, and if the current change causes the value to rise, the next change moves the pan/tilt head by at least two steps in the same dimension but in the opposite direction to the current change.

8. The three-dimensional measurement method according to claim 7, wherein each pan-tilt position change is performed alternately in the first dimension and the second dimension in a case where the value of the loss function or a derivative function of its gradient has a non-ascending trend.

9. The three-dimensional measurement method according to any one of claims 1 to 4, wherein the loss function is defined as:

loss＝a₁*(x-x₀)²+a₂*(y-y₀)²+a₃*(z-z₀)²

wherein, a₁-a₃Is a positive coefficient, x₀，y₀，z₀Is the actual size of the standard and x, y, z are three-dimensional measurements.

10. The three-dimensional measurement method according to any one of claims 1 to 4, wherein the gradient of the loss function is:

wherein in step d is used the sum of the squares of the variables of each dimension of the gradient of the loss function.

11. The three-dimensional measurement method of any one of claims 1-4, wherein the adjusting step further comprises a reference-free adjustment, the reference-free adjustment comprising the acts of:

A. rotating the pan-tilt along a first dimension, confirming a first position reaching a minimum of a measured height from the measurement area based on the acquisition of the depth camera, thereby determining a first angle at which the pan-tilt is associated with the first dimension;

B. comparing the difference between the first angle and the first predetermined angle;

C. rotating the pan/tilt head along a second dimension, identifying a second position that reaches a minimum of the measured height from the measurement area based on the acquisition of the depth camera, thereby determining a second angle of the pan/tilt head associated with the second dimension;

D. comparing the difference between the second angle and a second predetermined angle;

E. when the difference from the predetermined angle is within a predetermined range, an adjusted reference position is established, thereby completing the adjustment without the reference object.

12. The three-dimensional measurement method according to claim 11, wherein the adjustment without reference is performed such that, in a case where the adjusted reference position is established in the adjustment without reference before the adjustment with the standard, the subsequent adjustment with the standard is not performed any more.

13. The three-dimensional measurement method according to claim 11, wherein when the difference from the predetermined angle in any one of steps B and D exceeds a predetermined range, the adjustment without the reference object is terminated early, and/or the adjustment with the standard is instead performed, wherein the pan/tilt head is brought to the position of the minimum value of the measured height from the measurement area in the dimension which has not yet reached the position of the minimum value of the measured height from the measurement area, so as to reach the same state as performing step a, and thereby the remaining steps of the adjustment with the standard are performed.

14. The three-dimensional measurement method according to claim 11, wherein the first predetermined angle, the second predetermined angle are initially defined angles, and/or are updated by first and second current angles at which the pan-tilt is located in the first and second dimensions, which are recorded in the case where the adjusted reference position is established in the adjustment with the standard.

15. The three-dimensional measurement method according to any one of claims 1-4, wherein the adjusting step is performed before the target object is measured and/or is initiated after the evaluation of the three-dimensional dimensions of the calculated target object and/or is included in the sub-step ready for three-dimensional measurement.

16. The three-dimensional measurement method according to any one of claims 1 to 4, wherein, after planar acquisition of the target object by the color camera, a pixel region of the target object is determined based on a result of the two-dimensional acquisition; after the target object is subjected to depth acquisition by using a depth camera, determining the zoom ratio of the target object based on the result of the depth acquisition; based on the result of the determination, the three-dimensional size of the target object is calculated.

17. The three-dimensional measurement method of any one of claims 1-4, wherein, when the thickness of the target object is less than 3% of the height of the camera plane defined by the camera from the measurement area where the target object is located, the planar two-dimensional calculations of the target object are decoupled from the height one-dimensional calculations in calculating the three-dimensional dimensions of the target object.

18. The three-dimensional measurement method according to any one of claims 1-4, wherein at least some steps of the method are manipulated remotely.

19. The three-dimensional measurement method of any one of claims 1-4, wherein the color camera comprises an area-array and/or wide-angle camera, and/or the depth camera comprises a structured-light based depth camera.

20. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out a three-dimensional measurement method for a central logistics repository according to any one of claims 1-19.

21. A three-dimensional measurement device for a central logistics repository, comprising a frame, a pan-tilt, a pair of cameras, one of the cameras being a color camera to facilitate planar acquisition of a target object and the other being a depth camera to facilitate depth acquisition of the target object, wherein the cameras are movably mounted to the frame by the pan-tilt, wherein the three-dimensional measurement device is configured such that: the pan/tilt head can remain stationary, so that the camera fixed to the pan/tilt head remains stationary, facilitating the acquisition of the camera; the pan/tilt head is movable according to a control, so that a camera fixed to the pan/tilt head moves therewith, wherein the pan/tilt head is configured to be rotatable in at least two dimensions,

the three-dimensional measurement device is configured to implement the three-dimensional measurement method for a central logistics repository of any one of claims 1-19.

22. The three-dimensional measuring device of claim 21, wherein the camera is mounted with the camera plane substantially vertically facing downwards and/or the camera plane is positioned at a height of 2.5-3m from the measuring area where the target object is positioned.

23. The three-dimensional measuring device of claim 21, wherein the frame comprises a vertical column and a cross-column connected substantially perpendicular to the vertical column, wherein the pan and tilt head is mounted to the cross-column.

24. The three-dimensional measurement device of claim 21, wherein the stage comprises a connection portion, a first dimension manipulation portion, and a second dimension manipulation portion, wherein the connection portion is connected to the frame, the first dimension manipulation portion is connected to both the second dimension manipulation portion and the connection portion, and the camera is connected to the second dimension manipulation portion, wherein the connection portion comprises a connection clamping plate layer,

the first dimension manipulation section includes: a first steering engine, a first clamp plate layer and a rotating layer, wherein the shell of the first steering engine is fixed on the first clamp plate layer, the rotating output end of the first steering engine is connected to the rotating layer,

the second dimension manipulation section includes: the shell of the second steering engine is fixed to the second clamp plate layer; the rotation output end of the second steering engine is connected to the swing arm part,

a first clamping plate layer is fixed to the connecting clamping plate layer, a second clamping plate layer is fixed to the rotating layer, and a camera is mounted on the swing arm.

25. The three-dimensional measurement device of any one of claims 21-24, wherein the color camera comprises an area-array and/or wide-angle camera, and/or the depth camera comprises a structured-light based depth camera.

26. The three-dimensional measurement device of any of claims 21-24, wherein the three-dimensional measurement device is configured to at least partially accept remote manipulation.