CN114396875A - Rectangular parcel volume measurement method based on vertical shooting of depth camera - Google Patents

Rectangular parcel volume measurement method based on vertical shooting of depth camera Download PDF

Info

Publication number
CN114396875A
CN114396875A CN202210052053.2A CN202210052053A CN114396875A CN 114396875 A CN114396875 A CN 114396875A CN 202210052053 A CN202210052053 A CN 202210052053A CN 114396875 A CN114396875 A CN 114396875A
Authority
CN
China
Prior art keywords
depth
package
rectangular
pixel
points
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202210052053.2A
Other languages
Chinese (zh)
Other versions
CN114396875B (en
Inventor
单建华
于斐斐
孙宇昊
丁星宇
吕权权
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Anhui University of Technology AHUT
Original Assignee
Anhui University of Technology AHUT
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Anhui University of Technology AHUT filed Critical Anhui University of Technology AHUT
Priority to CN202210052053.2A priority Critical patent/CN114396875B/en
Publication of CN114396875A publication Critical patent/CN114396875A/en
Application granted granted Critical
Publication of CN114396875B publication Critical patent/CN114396875B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01B—MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
    • G01B11/00—Measuring arrangements characterised by the use of optical techniques
    • G01B11/002—Measuring arrangements characterised by the use of optical techniques for measuring two or more coordinates
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
    • Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
    • Y02P90/30—Computing systems specially adapted for manufacturing

Landscapes

  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Image Processing (AREA)
  • Length Measuring Devices By Optical Means (AREA)

Abstract

The invention discloses a rectangular parcel volume measurement method based on vertical shooting of a depth camera, and relates to the technical field of parcel volume measurement. The measuring method of the invention comprises the steps of obtaining a regular depth map of a package by using a depth camera which shoots vertically; selecting a threshold according to depth gradient information in the neighborhood of part of randomly selected pixels in the depth map, and respectively segmenting a plane area and an edge area; extracting a minimum external rectangle from the plane area, removing the minimum external rectangle with an undersize, and obtaining a minimum external rectangle area of the upper surface of the package according to the principle that the upper surface of the package is closest to the camera; then searching boundary pixel points in the minimum circumscribed rectangular area; obtaining four boundary lines, calculating four intersection points of the four boundary lines, projecting the four intersection points to the package, and calculating the distance between the adjacent intersection points after projection; and calculating the volume of the package by taking the difference between the depth distance value near the outer side of the minimum circumscribed rectangular region and the internal depth distance value as the package height.

Description

Rectangular parcel volume measurement method based on vertical shooting of depth camera
Technical Field
The invention relates to the technical field of parcel volume measurement, in particular to a rectangular parcel volume measurement method based on vertical shooting of a depth camera.
Background
With the rapid growth of the internet and electronic commerce in recent years, logistics has shifted from traditional human-intensive to technology-intensive intelligent logistics. Volume data of parcels in logistics is very important in intelligent logistics, and is more difficult to obtain relative to other data such as weight, and therefore, the method has important research value in how to accurately and conveniently obtain the volume of a single parcel or a plurality of parcels stacked together.
Patent CN100570275C proposes a volume measurement device based on images and a measurement method thereof, which adopts three cameras to perform three-dimensional reconstruction on a measured object, and the calibration process is complex and the operation is cumbersome.
Patent CN104501718A provides a parcel volume measuring device based on vision, adopts the mode that light curtain sensing device and camera combine together to measure the object volume, has adopted a plurality of sensing equipment, and the installation is complicated, and the cost is higher.
Patent CN109961468A proposes a volume measurement method, device and storage medium based on binocular vision, which needs dense depth based on full map, and most area information is useless when processing depth map, consuming resources and not high in economy.
Patent CN109029618A provides a monocular vision packing box volume measurement method, and a method for measuring volume by using a monocular camera and a single-point distance sensor detects and segments a package contour based on a monocular two-dimensional image, is easily affected by the texture of the upper surface of the package, and has a certain problem in measurement accuracy.
Disclosure of Invention
1. Technical problem to be solved by the invention
Aiming at the problems that the manual measurement efficiency of the package volume measurement in the intelligent logistics of the logistics industry in the prior art is low, the three-dimensional reconstruction calibration process of the package by using multiple cameras is complex, and the operation is complicated, the invention provides the rectangular package volume measurement method based on the vertical shooting of the depth camera, which can completely replace the manual completion of the package size measurement, effectively improve the measurement precision and improve the operation effect.
2. Technical scheme
In order to achieve the purpose, the technical scheme provided by the invention is as follows:
a rectangular parcel volume measurement method based on vertical shooting of a depth camera comprises the following steps:
s1, acquiring a regular depth map of the package by using a depth camera for vertical shooting;
s2, selecting a threshold according to depth gradient information in the neighborhood of part of randomly selected pixels in the depth map, and respectively segmenting a plane area and an edge area;
s3, extracting the minimum circumscribed rectangle from the plane area, removing the minimum circumscribed rectangle with an undersize size, and then obtaining the minimum circumscribed rectangle area of the upper surface of the package according to the principle that the upper surface of the package is closest to the camera;
s4, searching boundary pixel points in the minimum circumscribed rectangular area;
s5, performing linear fitting on the boundary pixel points to obtain four boundary lines, calculating four intersection points of the four boundary lines, projecting the four intersection points to the upper surface of the package, and calculating the distance between the adjacent projected intersection points to obtain the length and the width of the package;
and S6, calculating the volume of the parcel by taking the difference between the depth distance value near the outer side of the minimum circumscribed rectangular area and the internal depth distance value as the height of the parcel.
In a further technical solution, the threshold calculation in step S2 specifically includes:
A. randomly selecting a% pixel points of the total pixels, calculating the absolute value of the depth distance difference value of each selected pixel point and 8 pixel points which are separated from the neighborhood of the selected pixel point by b pixel distances to obtain an array, wherein the T1 th percentile of the array is a first segmentation threshold, the T2 th percentile is a second segmentation threshold, and the T1 is smaller than the T2;
B. traversing all pixel points in the depth map, calculating the absolute value of the depth distance difference value of the current pixel point and 8 pixel points which are separated from the neighborhood of the current pixel point by b pixel distances, wherein if the absolute value is smaller than a first segmentation threshold, the current pixel point is a plane point, and if the absolute value is larger than a second segmentation threshold, the current pixel point is an edge point;
C. all plane points constitute a plane area and all edge points constitute an edge area.
In a further technical scheme, in step S3, the method specifically includes:
A. extracting a minimum circumscribed rectangle from the plane area, and removing the minimum circumscribed rectangle with the side length less than d pixels;
B. and calculating the median of the depth values of all pixel points in the contour corresponding to each minimum external rectangle to obtain the median array of the contours corresponding to all the minimum external rectangles, and then taking the minimum external rectangle corresponding to the minimum value of the median array as the minimum external rectangle area of the upper surface of the package according to the principle that the upper surface of the package is closest to the camera.
In a further technical scheme, in step S4, the method specifically includes:
A. obtaining four vertex pixel coordinates P1P2P3P4 of a minimum circumscribed rectangle corresponding to the upper surface of the package, wherein the side lengths are respectively P1P 2-L1, and P1P 4-L2;
B. respectively taking midpoints M1 and M2 of opposite sides P1P4 and P2P3, respectively finding a first edge point in the range of e x L2 along the positive and negative directions of a vector P1P2 of each pixel point P on the line segment M1M2, wherein e is larger than 0.5; if the edge point is found, the edge point is a boundary pixel point of the pixel point p; if not, the pixel point p has no corresponding boundary pixel point; thus, boundary pixel point sets corresponding to the edge P1P2 and the edge P3P4 are obtained;
C. similarly, the boundary pixel point sets corresponding to the edge P1P4 and the edge P2P3 can also be obtained.
In a further technical scheme, in step S5, the method specifically includes:
A. respectively performing linear fitting on the boundary pixel point set corresponding to each edge to obtain linear equations of four boundary lines;
B. the intersection point of two adjacent straight lines is used as the vertex of the rectangle wrapping the upper surface, so that four vertexes are obtained;
C. projecting the four vertexes to the upper surface of the package, and obtaining the length and the width of the package according to the distance of the projected vertexes;
in a further technical scheme, in step S6, the method specifically includes:
A. according to the pixel coordinates P1P2P3P4 of four vertexes of the minimum circumscribed rectangle corresponding to the upper surface of the package, respectively taking points Q1 and Q2 at a distance f X L2 from points P1 and P2 along the direction of a vector P4P1 to obtain a rectangular region P1P2Q2Q1 corresponding to the side P1P2, and similarly obtaining rectangular regions corresponding to the other three sides;
B. obtaining depth values of all plane points in the four rectangular areas, taking the Tth percentile of the depth values, namely the depth distance value near the outer side of the minimum external rectangular area corresponding to the upper surface of the package, and obtaining the distance value of the package support surface from the camera because the outer side of the rectangular area is the package support surface;
C. obtaining depth values of all plane points in the rectangular area P1P2P3P4, and taking the median of the depth values to obtain a depth distance value inside the minimum circumscribed rectangular area, which is the distance value of the upper surface distance camera;
D. the difference between the distance value of the upper surface of the parcel from the camera and the distance value of the parcel supporting surface from the camera is the height of the parcel.
In a further technical scheme, in step S5, four vertices are projected onto the upper surface of the package, and are transformed by the following formula:
Figure BDA0003474589180000031
wherein, (C, R) is two-dimensional pixel coordinate of boundary pixel, (x, y, z) is three-dimensional coordinate of boundary pixel after projection under camera coordinate system, fxAnd fyIs the effective focal length, (C)0，R0) And Z is the distance value from the upper surface of the package to the camera as the pixel datum point.
In the step S6, f is 0.2-0.4; and the Tth percentile is 70-80 th percentile.
3. Advantageous effects
Compared with the prior art, the technical scheme provided by the invention has the following beneficial effects:
(1) according to the rectangular parcel volume measurement method based on vertical shooting of the depth camera, the object to be measured is extracted through the depth information acquired by the depth camera, the two-dimensional image coordinate of the object to be measured is converted into the three-dimensional space coordinate, and then volume measurement is carried out on parcels without calibration; and extracting a plane region and an edge region respectively based on a threshold value by using the difference of the depth distance between each adjacent pixel point in the depth map. Extracting a minimum circumscribed rectangle from the plane area, applying the minimum circumscribed rectangle to a scene with a plurality of interfering objects, and ensuring that all packages are detected;
(2) according to the rectangular parcel volume measurement method based on vertical shooting of the depth camera, the minimum external rectangular area of the upper surface of a parcel is obtained by the principle that the distance between the parcel and the camera is the nearest, so that the method can adapt to complex scenes; the rectangular area is used for searching boundary points of the package from inside to outside and fitting a linear equation of the four boundary lines, and the intersection point of the four linear equations is calculated to obtain the length and width of the package, so that the package is high in precision, stable and reliable;
(3) according to the rectangular parcel volume measurement method based on vertical shooting of the depth camera, the parcel height value is obtained by respectively calculating the median values of the depth distance values of the plane points inside and outside the minimum circumscribed rectangular area of a parcel, and the method is high in precision, stable and reliable;
(4) according to the rectangular parcel volume measurement method based on vertical shooting of the depth camera, two-dimensional depth data rather than three-dimensional point cloud data are processed in the whole step process, and an algorithm with high time complexity does not exist, so that the real-time performance is good;
(4) the rectangular parcel volume measurement method based on vertical shooting of the depth camera is based on Visual Studio 2019 software, utilizes a third-party library PCL and OpenCV, adopts C + + language codes to realize the whole implementation case system, runs in a windows10 system with a CPU AMD-R7-5800H, processes regular depth images with 640x480 resolution acquired by a single depth camera, and has the calculation time within 8ms and the length, width and high-precision errors within 3 mm.
Drawings
FIG. 1 is a schematic flow chart of the steps of the present invention;
FIG. 2 is a schematic diagram of a depth camera mounting location configuration involved in an embodiment of the present invention;
FIG. 3 is a schematic diagram of finding boundary points on the upper surface of a package in accordance with an embodiment of the present invention;
FIG. 4 is a schematic diagram of finding out a point on an outer plane of an upper surface of a package in accordance with an embodiment of the present invention;
FIG. 5(a) is a schematic structural diagram of a single parcel in a visual camera ray projection according to embodiment 2 of the present invention;
FIG. 5(b) is a graph of the measurement of a single parcel point cloud and the effect of the boundary in embodiment 2 of the present invention;
fig. 6(a) is a schematic structural diagram of a visual camera ray projection of a parcel in a complex scene according to embodiment 3 of the present invention;
fig. 6(b) is a graph of the point cloud and the boundary effect of the parcel in the measurement complex scene in embodiment 3 of the present invention.
In the figure: 1. a depth camera; 2. wrapping; 21. an upper surface; 22. a side surface; 23. a support surface.
Detailed Description
For a further understanding of the invention, reference should be made to the following detailed description taken in conjunction with the accompanying drawings.
Example 1
The rectangular parcel volume measurement method based on vertical shooting by a depth camera in the embodiment, as shown in fig. 1, includes the following steps:
s1, acquiring a regular depth image of the parcel 2 by using the depth camera 1 which vertically shoots;
as shown in fig. 2, the depth camera 1 is disposed right above the parcel 2, and the shooting direction thereof is perpendicular to the parcel 2;
the data processing control module in the depth camera 1 controls the depth camera to be turned on and off, and a depth image is obtained.
The depth camera 1 obtains depth images, and the depth images are sequentially arranged in a regular manner in a row-by-row manner to form a regular depth image, and the resolution is selected to be 640 × 480.
S2, selecting a threshold value according to depth gradient information in the neighborhood of part of randomly selected pixel points in the depth map, and respectively segmenting a plane area and an edge area;
the depth camera cannot shoot the side face 22 of the package, so the difference of the depth values of the pixel points in the side face 22 of the package is large, and the depth camera shoots the upper surface 21 and the supporting surface 23 of the package in the front face, so the difference of the depth values of the pixel points in the two plane areas is small;
wherein the upper surface 21 of the rectangular parcel is approximately parallel to the support surface 23, the two planes belong to the plane area, and the parcel side 22 between the two planes belongs to the edge area;
randomly selecting 10-20% of pixel points of a total pixel to determine a reference threshold, reflecting the depth distance difference condition of the whole pixel points, saving the operation time, calculating the absolute value of the depth distance difference of 8 pixel points which are 3-6 pixel distances away from each selected pixel point in the neighborhood of the selected pixel point to obtain an array, wherein the value range of 3-6 is to avoid the influence of noise points in a short distance and avoid the invalid depth distance difference of points in a long distance, taking the 85-90 th percentile as a first segmentation threshold, taking the 96-98 th percentile as a second segmentation threshold, and taking the two values as most of all the pixel points of a depth map based on the obtained package as a plane area and the small part as an edge area, wherein the depth distance difference of the pixel points in the plane area is small, and the depth distance difference of the edge area is large;
traversing all pixel points in the depth map, and calculating the absolute value of the depth distance difference value of 8 pixel points which are 3-6 pixel distances away from the current pixel point and the neighborhood of the current pixel point (wherein, the value range of 3-6 is the same as the threshold setting process, and the segmentation is guaranteed to be effective);
all plane points constitute a plane area and all edge points constitute an edge area.
S3, extracting the minimum circumscribed rectangle from the plane area, removing the minimum circumscribed rectangle with an undersize size, and then obtaining the minimum circumscribed rectangle area of the upper surface of the package according to the principle that the upper surface of the package is closest to the camera; extracting a minimum circumscribed rectangle for the plane area;
optionally, a findContours function in OpenCV is used for extracting a minimum circumscribed rectangle for a plane region in a depth map, and a RETR _ LIST parameter is used in the findContours function for extracting a boundary point of an outline;
according to the fact that the length value of each side of the minimum circumscribed rectangle needs to be larger than 20 pixels (the side length of a package generally needs to have a certain length, and the rectangle which is smaller than the length of 20 pixels and is regarded as a rectangle generated by noise influence needs to be removed), the minimum circumscribed rectangle with an excessively small size is removed;
and calculating the median of the depth value sequences of all pixel points in the contour corresponding to each minimum circumscribed rectangle to obtain a median array of all the contours, and then taking the minimum circumscribed rectangle corresponding to the minimum value of the median array as the minimum circumscribed rectangle area of the upper surface of the package according to the principle that the upper surface of the package is closest to the camera.
S4, searching boundary pixel points in the minimum circumscribed rectangular area;
specifically, as shown in fig. 3, four vertex pixel coordinates P1P2P3P4 of a minimum circumscribed rectangle corresponding to the upper surface of the package are obtained, where the length of each side is P1P2 — L1, and P1P4 — L2. The midpoints M1 and M2 of opposite sides P1P4 and P2P3 are taken respectively. For each pixel point P on the segment M1M2, a first edge point is found within a range of (0.55-0.75) × L2 along the positive and negative directions of the vector P1P2 (a boundary pixel point is found from the central line of the rectangular frame to the outside, and the boundary pixel point is generally found at a position of 0.5 × side length thereof, which is slightly larger than 0.5 × side length, so as to ensure that all boundary points are found). If the edge point is found, the edge point is a boundary pixel point of the pixel point p; if not, the pixel point p has no corresponding boundary pixel point. Thus, boundary pixel point sets corresponding to the edge P1P2 and the edge P3P4 are obtained;
similarly, the boundary pixel point sets corresponding to the edge P1P4 and the edge P2P3 can also be obtained.
S5, performing linear fitting on the boundary pixel points to obtain four boundary lines, calculating four intersection points of the four boundary lines, projecting the four intersection points to the upper surface of the package, and calculating the distance between the adjacent projected intersection points to obtain the length and width of the package;
specifically, linear fitting is respectively carried out on the boundary pixel point sets corresponding to each edge, and linear equations of four boundary lines are obtained;
the intersection point of two adjacent straight lines is used as the vertex of the rectangle wrapping the upper surface, so that four vertexes are obtained;
projecting the four vertices onto the upper surface of the wrap, transforming by the following formula:
Figure BDA0003474589180000061
wherein, (C, R) is two-dimensional pixel coordinate of boundary pixel, (x, y, z) is three-dimensional coordinate of boundary pixel after projection under camera coordinate system, fxAnd fyIs the effective focal length, (C)0，R0) And Z is the distance value from the upper surface of the package to the camera as the pixel datum point.
For example, the two-dimensional pixel coordinate (C, R) of one vertex is (350, 250), the distance value Z of the upper surface of the package from the camera is 600, the effective focal length fx is 556.433, and fy is 559.148; the pixel reference point (C0, R0) is (326.056, 245.587); according to the calculation of a formula,
Figure BDA0003474589180000062
obtaining the three-dimensional coordinates after the vertex projection: (25.819,4.735, 600).
Obtaining the length and width of the package according to the projected vertex distance;
s6, the difference between the depth distance value near the outer side of the minimum circumscribed rectangular area and the inner depth distance value is the height of the package;
as shown in fig. 4, firstly, according to four vertex pixel coordinates P1P2P3P4 of the minimum circumscribed rectangle corresponding to the package upper surface, points Q1 and Q2 are taken from points P1 and P2 along the direction of a vector P4P1 at a distance of (0.2 to 0.4) × L2, so as to obtain a rectangular region P1P2Q 1 corresponding to the side P1P2, and similarly, a rectangular region corresponding to the other three sides can be obtained, and a value is taken to be 0.2 to 0.4, so as to expand a point distance from the obtained rectangular region to the outside and search for a point of a package supporting surface, and for a special form of express package, a value range of 0.2 to 0.4 is obtained by the inventor through batch data collection and analysis, and meets the requirement of expanding a point distance from the obtained rectangular region to the outside.
And obtaining depth values of all plane points in the four rectangular areas, taking the 70 th-80 th percentile of the depth values, namely the depth distance value near the outer side of the minimum external rectangular area corresponding to the upper surface of the package, and obtaining the distance value of the package supporting surface from the camera because the outer side of the rectangular area is the package supporting surface.
And obtaining the depth values of all plane points in the rectangular area P1P2P3P4, and taking the median of the depth values to obtain the depth distance value inside the minimum circumscribed rectangular area, which is the distance value of the upper surface of the wrapping distance camera.
The difference between the distance value of the upper surface of the parcel from the camera and the distance value of the parcel supporting surface from the camera is the height of the parcel.
Thereby calculating the volume of the package.
Example 2
The method for measuring the rectangular parcel volume based on vertical shooting by the depth camera in the embodiment has the same specific steps as the embodiment 1,
as shown in fig. 5, wherein, fig. 5(a) is a schematic structural diagram of the single parcel in the light projection of the vision camera, and the actual dimensions are 295mm in length, 240mm in width and 125mm in height.
Fig. 5(b) is a graph of the point cloud and the boundary effect of a single parcel measured in this embodiment, and the calculated result of the processing after the steps of S1 to 6 of the present invention is 294mm in length, 240mm in width and 126mm in height.
Example 3
The method for measuring the rectangular parcel volume based on vertical shooting by the depth camera in the embodiment has the same specific steps as the embodiment 1,
as shown in fig. 6, fig. 6(a) is a schematic structural diagram of a package in a complex scene in a light projection of a visual camera, wherein the top layer of the complex scene is a package to be measured, the actual size of the package is 130mm long, 80mm wide and 58mm high, and the rest are set interfering objects.
Fig. 6(b) is a point cloud and boundary effect graph of the parcel in the complex scene measured in this embodiment, after the steps of S1 to 6 of the present invention, the method accurately identifies the parcel to be measured, and the processing calculation result is 132mm in length, 81mm in width, and 57mm in height.
The above embodiments are all based on Visual Studio 2019 software, utilize third-party libraries PCL and OpenCV, and adopt C + + language code to realize the whole implementation case system, and operate in a windows10 system with a CPU AMD-R7-5800H, and process regular depth images with 640x480 resolution acquired by a single depth camera, wherein the calculation time is within 8ms, and the length, width and high-precision errors are within 3 mm.
The present invention and its embodiments have been described above schematically, without limitation, and what is shown in the drawings is only one of the embodiments of the present invention, and the actual structure is not limited thereto. Therefore, if the person skilled in the art receives the teaching, without departing from the spirit of the invention, the person skilled in the art shall not inventively design the similar structural modes and embodiments to the technical solution, but shall fall within the scope of the invention.

Claims (8)

1. A rectangular parcel volume measurement method based on vertical shooting of a depth camera is characterized by comprising the following steps:
s1, acquiring a regular depth map of the package by using a depth camera for vertical shooting;
s2, selecting a threshold according to depth gradient information in the neighborhood of part of randomly selected pixels in the depth map, and respectively segmenting a plane area and an edge area;
s3, extracting the minimum circumscribed rectangle from the plane area, removing the minimum circumscribed rectangle with an undersize size, and then obtaining the minimum circumscribed rectangle area of the upper surface of the package according to the principle that the upper surface of the package is closest to the camera;
s4, searching boundary pixel points in the minimum circumscribed rectangular area;
s5, performing linear fitting on the boundary pixel points to obtain four boundary lines, calculating four intersection points of the four boundary lines, projecting the four intersection points to the upper surface of the package, and calculating the distance between the adjacent projected intersection points to obtain the length and the width of the package;
and S6, calculating the volume of the parcel by taking the difference between the depth distance value near the outer side of the minimum circumscribed rectangular area and the internal depth distance value as the height of the parcel.
2. The rectangular parcel volume measurement method based on vertical shooting by a depth camera according to claim 1, characterized in that: the threshold calculation in step S2 specifically includes:
A. randomly selecting a% pixel points of the total pixels, calculating the absolute value of the depth distance difference value of each selected pixel point and 8 pixel points which are separated from the neighborhood of the selected pixel point by b pixel distances to obtain an array, wherein the T1 th percentile of the array is a first segmentation threshold, the T2 th percentile is a second segmentation threshold, and the T1 is smaller than the T2;
B. traversing all pixel points in the depth map, calculating the absolute value of the depth distance difference value of the current pixel point and 8 pixel points which are separated from the neighborhood of the current pixel point by b pixel distances, wherein if the absolute value is smaller than a first segmentation threshold, the current pixel point is a plane point, and if the absolute value is larger than a second segmentation threshold, the current pixel point is an edge point;
C. all plane points constitute a plane area and all edge points constitute an edge area.
3. The rectangular parcel volume measurement method based on vertical shooting by a depth camera according to claim 1, characterized in that: in step S3, specifically, the method includes:
A. extracting a minimum circumscribed rectangle from the plane area, and removing the minimum circumscribed rectangle with the side length less than d pixels;
B. and calculating the median of the depth values of all pixel points in the contour corresponding to each minimum external rectangle to obtain the median array of the contours corresponding to all the minimum external rectangles, and then taking the minimum external rectangle corresponding to the minimum value of the median array as the minimum external rectangle area of the upper surface of the package according to the principle that the upper surface of the package is closest to the camera.
4. The rectangular parcel volume measurement method based on vertical shooting by a depth camera according to claim 1, characterized in that: in step S4, specifically, the method includes:
A. obtaining four vertex pixel coordinates P1P2P3P4 of a minimum circumscribed rectangle corresponding to the upper surface of the package, wherein the side lengths are respectively P1P 2-L1, and P1P 4-L2;
B. respectively taking midpoints M1 and M2 of opposite sides P1P4 and P2P3, respectively finding a first edge point in the range of e x L2 along the positive and negative directions of a vector P1P2 of each pixel point P on the line segment M1M2, wherein e is larger than 0.5; if the edge point is found, the edge point is a boundary pixel point of the pixel point p; if not, the pixel point p has no corresponding boundary pixel point; thus, boundary pixel point sets corresponding to the edge P1P2 and the edge P3P4 are obtained;
C. similarly, the boundary pixel point sets corresponding to the edge P1P4 and the edge P2P3 are obtained.
5. The rectangular parcel volume measurement method based on vertical shooting by a depth camera according to claim 1, characterized in that: in step S5, specifically, the method includes:
A. respectively performing linear fitting on the boundary pixel point set corresponding to each edge to obtain linear equations of four boundary lines;
B. the intersection point of two adjacent straight lines is used as the vertex of the rectangle wrapping the upper surface, so that four vertexes are obtained;
C. and projecting the four vertexes to the upper surface of the package, and obtaining the length and the width of the package according to the distance of the projected vertexes.
6. The rectangular parcel volume measurement method based on vertical shooting by a depth camera according to claim 1, characterized in that: in step S6, specifically, the method includes:
A. according to the pixel coordinates P1P2P3P4 of four vertexes of the minimum circumscribed rectangle corresponding to the upper surface of the package, respectively taking points Q1 and Q2 at a distance f × L2 from points P1 and P2 along the direction of a vector P4P1 to obtain a rectangular region P1P2Q2Q1 corresponding to the side P1P2, and similarly obtaining rectangular regions corresponding to the other three sides;
B. obtaining depth values of all plane points in the four rectangular areas, taking the Tth percentile of the depth values, namely the depth distance value near the outer side of the minimum external rectangular area corresponding to the upper surface of the package, and obtaining the distance value of the package support surface from the camera because the outer side of the rectangular area is the package support surface;
C. obtaining depth values of all plane points in the rectangular area P1P2P3P4, and taking the median of the depth values to obtain a depth distance value inside the minimum circumscribed rectangular area, which is the distance value of the upper surface distance camera;
D. the difference between the distance value of the upper surface of the parcel from the camera and the distance value of the parcel supporting surface from the camera is the height of the parcel.
7. The rectangular parcel volume measurement method based on vertical shooting by a depth camera according to claim 5, characterized in that: in step S5, four vertices are projected onto the package upper surface, and transformed by the following formula:
Figure FDA0003474589170000021
wherein, (C, R) is two-dimensional pixel coordinate of boundary pixel, (x, y, z) is three-dimensional coordinate of boundary pixel after projection under camera coordinate system, fxAnd fyIs the effective focal length, (C)0，R0) And Z is the distance value from the upper surface of the package to the camera as the pixel datum point.
8. The rectangular parcel volume measurement method based on vertical shooting by a depth camera according to claim 6, characterized in that: in step S6, f is 0.2-0.4; and the Tth percentile is 70-80 th percentile.
CN202210052053.2A 2022-01-18 2022-01-18 Rectangular package volume measurement method based on vertical shooting of depth camera Active CN114396875B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210052053.2A CN114396875B (en) 2022-01-18 2022-01-18 Rectangular package volume measurement method based on vertical shooting of depth camera

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210052053.2A CN114396875B (en) 2022-01-18 2022-01-18 Rectangular package volume measurement method based on vertical shooting of depth camera

Publications (2)

Publication Number Publication Date
CN114396875A true CN114396875A (en) 2022-04-26
CN114396875B CN114396875B (en) 2023-11-10

Family

ID=81229989

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210052053.2A Active CN114396875B (en) 2022-01-18 2022-01-18 Rectangular package volume measurement method based on vertical shooting of depth camera

Country Status (1)

Country Link
CN (1) CN114396875B (en)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114882404A (en) * 2022-05-06 2022-08-09 安徽工业大学 Method and system for counting number of people in and out in real time based on depth camera
CN115077379A (en) * 2022-06-08 2022-09-20 安徽工业大学 Method for measuring volume of multiple parcels based on vertical shooting of depth camera
CN115682941A (en) * 2022-12-27 2023-02-03 广东技术师范大学 Packing box geometric dimension measuring method based on structured light camera
CN117611657A (en) * 2024-01-22 2024-02-27 江西农业大学 Pleurotus eryngii volume calculation method

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105869115A (en) * 2016-03-25 2016-08-17 浙江大学 Depth image super-resolution method based on kinect2.0
CN107388960A (en) * 2016-05-16 2017-11-24 杭州海康机器人技术有限公司 A kind of method and device for determining object volume
CN108648230A (en) * 2018-05-14 2018-10-12 南京阿凡达机器人科技有限公司 A kind of package dimensions measurement method, system, storage medium and mobile terminal
CN109472822A (en) * 2018-12-29 2019-03-15 上海大学 Dimension of object measurement method based on depth image processing
CN110017773A (en) * 2019-05-09 2019-07-16 福建（泉州）哈工大工程技术研究院 A kind of package volume measuring method based on machine vision
JP3230773U (en) * 2020-10-27 2021-02-18 立普思股▲ふん▼有限公司ＬＩＰＳ Ｃｏｒｐｏｒａｔｉｏｎ Volume measurement system
CN113538557A (en) * 2020-04-15 2021-10-22 深圳市光鉴科技有限公司 Box volume measuring device based on three-dimensional vision
CN113870292A (en) * 2021-09-10 2021-12-31 北京航空航天大学 Edge detection method and device for depth image and electronic equipment

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105869115A (en) * 2016-03-25 2016-08-17 浙江大学 Depth image super-resolution method based on kinect2.0
CN107388960A (en) * 2016-05-16 2017-11-24 杭州海康机器人技术有限公司 A kind of method and device for determining object volume
CN108648230A (en) * 2018-05-14 2018-10-12 南京阿凡达机器人科技有限公司 A kind of package dimensions measurement method, system, storage medium and mobile terminal
CN109472822A (en) * 2018-12-29 2019-03-15 上海大学 Dimension of object measurement method based on depth image processing
CN110017773A (en) * 2019-05-09 2019-07-16 福建（泉州）哈工大工程技术研究院 A kind of package volume measuring method based on machine vision
CN113538557A (en) * 2020-04-15 2021-10-22 深圳市光鉴科技有限公司 Box volume measuring device based on three-dimensional vision
JP3230773U (en) * 2020-10-27 2021-02-18 立普思股▲ふん▼有限公司ＬＩＰＳ Ｃｏｒｐｏｒａｔｉｏｎ Volume measurement system
CN113870292A (en) * 2021-09-10 2021-12-31 北京航空航天大学 Edge detection method and device for depth image and electronic equipment

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114882404A (en) * 2022-05-06 2022-08-09 安徽工业大学 Method and system for counting number of people in and out in real time based on depth camera
CN115077379A (en) * 2022-06-08 2022-09-20 安徽工业大学 Method for measuring volume of multiple parcels based on vertical shooting of depth camera
CN115682941A (en) * 2022-12-27 2023-02-03 广东技术师范大学 Packing box geometric dimension measuring method based on structured light camera
CN115682941B (en) * 2022-12-27 2023-03-07 广东技术师范大学 Packing box geometric dimension measuring method based on structured light camera
CN117611657A (en) * 2024-01-22 2024-02-27 江西农业大学 Pleurotus eryngii volume calculation method
CN117611657B (en) * 2024-01-22 2024-04-23 江西农业大学 Pleurotus eryngii volume calculation method

Also Published As

Publication number Publication date
CN114396875B (en) 2023-11-10

Similar Documents

Publication Publication Date Title
CN114396875B (en) Rectangular package volume measurement method based on vertical shooting of depth camera
CN109029253B (en) Package volume measuring method and system, storage medium and mobile terminal
CN109872397B (en) Three-dimensional reconstruction method of airplane parts based on multi-view stereo vision
US10115035B2 (en) Vision system and analytical method for planar surface segmentation
CN113436260B (en) Mobile robot pose estimation method and system based on multi-sensor tight coupling
TWI729995B (en) Generating a merged, fused three-dimensional point cloud based on captured images of a scene
CN110230998B (en) Rapid and precise three-dimensional measurement method and device based on line laser and binocular camera
CN109801333B (en) Volume measurement method, device and system and computing equipment
JP5618569B2 (en) Position and orientation estimation apparatus and method
CA3114457A1 (en) Volume measurement method and system, apparatus and computer-readable storage medium
CN107767456A (en) A kind of object dimensional method for reconstructing based on RGB D cameras
CN111144213B (en) Object detection method and related equipment
CN107977996B (en) Space target positioning method based on target calibration positioning model
CN109472820B (en) Monocular RGB-D camera real-time face reconstruction method and device
KR20140027468A (en) Depth measurement quality enhancement
CN111123242B (en) Combined calibration method based on laser radar and camera and computer readable storage medium
CN111814792B (en) Feature point extraction and matching method based on RGB-D image
KR101549155B1 (en) Method of automatic extraction of building boundary from lidar data
CN112184793B (en) Depth data processing method and device and readable storage medium
CN110415363A (en) A kind of object recognition positioning method at random based on trinocular vision
CN113362385A (en) Cargo volume measuring method and device based on depth image
CN111524193B (en) Method and device for measuring two-dimensional size of object
CN116129037A (en) Visual touch sensor, three-dimensional reconstruction method, system, equipment and storage medium thereof
CN113932712B (en) Melon and fruit vegetable size measurement method based on depth camera and key points
CN113379826A (en) Method and device for measuring volume of logistics piece

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant