CN112037283A - Truck positioning and box aligning detection method based on machine vision - Google Patents
Truck positioning and box aligning detection method based on machine vision Download PDFInfo
- Publication number
- CN112037283A CN112037283A CN202010921668.5A CN202010921668A CN112037283A CN 112037283 A CN112037283 A CN 112037283A CN 202010921668 A CN202010921668 A CN 202010921668A CN 112037283 A CN112037283 A CN 112037283A
- Authority
- CN
- China
- Prior art keywords
- container
- image
- target container
- corner fitting
- truck
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 24
- 238000000034 method Methods 0.000 claims abstract description 18
- 238000000605 extraction Methods 0.000 claims abstract description 16
- 238000013459 approach Methods 0.000 claims abstract description 5
- 230000011218 segmentation Effects 0.000 claims description 17
- 238000013135 deep learning Methods 0.000 claims description 8
- 238000007689 inspection Methods 0.000 claims 1
- 238000010586 diagram Methods 0.000 description 4
- 238000009434 installation Methods 0.000 description 2
- 241001292396 Cirrhitidae Species 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 230000001788 irregular Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
- G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B66—HOISTING; LIFTING; HAULING
- B66C—CRANES; LOAD-ENGAGING ELEMENTS OR DEVICES FOR CRANES, CAPSTANS, WINCHES, OR TACKLES
- B66C13/00—Other constructional features or details
- B66C13/16—Applications of indicating, registering, or weighing devices
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B66—HOISTING; LIFTING; HAULING
- B66C—CRANES; LOAD-ENGAGING ELEMENTS OR DEVICES FOR CRANES, CAPSTANS, WINCHES, OR TACKLES
- B66C13/00—Other constructional features or details
- B66C13/18—Control systems or devices
- B66C13/46—Position indicators for suspended loads or for crane elements
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/11—Region-based segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/60—Analysis of geometric attributes
- G06T7/66—Analysis of geometric attributes of image moments or centre of gravity
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10004—Still image; Photographic image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30108—Industrial image inspection
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Mechanical Engineering (AREA)
- Geometry (AREA)
- Automation & Control Theory (AREA)
- Length Measuring Devices By Optical Means (AREA)
Abstract
The invention discloses a method for positioning a container truck and detecting a container based on machine vision, which comprises the following steps: acquiring a first image, wherein the first image at least comprises a vehicle plate lock button image corresponding to a corner fitting of a target container, extracting the characteristics of the first image, calculating the pose of a container truck, and controlling the corresponding distance of movement of a trolley, a cart and a lifting appliance to carry out primary positioning; and when the target container approaches the target position of the container truck, acquiring a second image, wherein the second image at least comprises a corner fitting image of the target container and a corresponding lock button image, performing feature extraction on the second image, and calculating the relative pose of the target container and the container truck, so as to control the corresponding distance of movement of the trolley, the cart and the hanger to perform dynamic container loading. According to the container truck positioning and container aligning detection method based on machine vision, the corner piece image of the target container and the corresponding vehicle plate lock button image are obtained, and the positions of the container truck and the target container are respectively obtained, so that the container truck trailer loading precision is higher.
Description
Technical Field
The invention relates to the field of crane loading and unloading, in particular to a truck positioning and box aligning detection method based on machine vision.
Background
At present, the container loading and unloading of the container to and from the container at the port mostly adopts a manual operation mode, and a tire crane driver operates the tire crane to load and unload the container to and from the container below the tire crane (a framework type container truck trailer and a flat plate type container truck trailer). The manual operation of a tire crane driver has the defects of low operation efficiency and unstable operation quality, and the condition that the container is pounded into the card collecting guide plate violently often occurs, so that the container and the card collecting guide plate are damaged to a certain extent. Some automatic modification schemes in the prior art, such as scanning the profile of the container truck by using a laser radar, and performing loading and unloading operations of the container truck after determining the position of the container truck. However, for the trucks with different models and the trailers with different models, the template library needs to be established for comparison, and deviation is easy to occur, so that automatic loading and unloading are unsuccessful.
Therefore, it is necessary to provide a method for positioning a container truck and detecting a container, which can realize the automatic positioning and loading/unloading of the container truck.
Disclosure of Invention
The invention aims to solve the technical problem of providing a container truck positioning and container aligning detection method based on machine vision, which is characterized in that the coordinates of the central points of corner pieces of a target container and vehicle plate lock buttons corresponding to the corner pieces of the target container are calculated by acquiring the corner piece images of the target container and the vehicle plate lock button images corresponding to the corner pieces of the target container so as to respectively acquire the positions of a container truck and the target container truck, so that the accuracy of loading the container truck to a container truck trailer is higher.
The invention adopts the technical scheme to solve the technical problems and provides a method for positioning a container truck and detecting a container based on machine vision, which comprises the following steps:
acquiring a first image, wherein the first image at least comprises a vehicle plate lock button image corresponding to a corner fitting of a target container, extracting the characteristics of the first image, calculating the center point coordinate of a vehicle plate lock button corresponding to the corner fitting of the target container, and calculating the position and posture of the container truck, so as to control a trolley, a cart and a hanger to move corresponding distances for primary positioning;
when the target container is close to a target position of the container truck, acquiring a second image, wherein the second image at least comprises a corner fitting image of the target container and a vehicle plate lock button image corresponding to the corner fitting of the target container, performing feature extraction on the second image, respectively calculating the center point coordinate of the corner fitting of the target container and the center point coordinate of the vehicle plate lock button corresponding to the corner fitting of the target container, and calculating the relative position and attitude of the target container and the container truck, thereby controlling the corresponding distance of movement of a trolley, a cart and a hanger to perform dynamic container loading.
Preferably, the feature extraction of the first image comprises point cloud clustering segmentation of a vehicle plate lock button image corresponding to a corner fitting of the target container;
and the second image is subjected to feature extraction, namely point cloud clustering segmentation is carried out on the corner fitting image of the target container and the vehicle plate lock button image corresponding to the corner fitting of the target container.
Preferably, the method further comprises the following steps:
performing feature extraction on the first image comprises performing deep learning target detection or instance segmentation on a vehicle board lock button image corresponding to a corner fitting of the target container, extracting an ROI (region of interest) of the vehicle board lock button image corresponding to the corner fitting of the target container, and calculating a 3D (three-dimensional) coordinate corresponding to the ROI center point of the vehicle board lock button image;
and the second image is subjected to feature extraction, namely, deep learning target detection or example segmentation is carried out on the corner fitting image of the target container and the vehicle plate lock button image corresponding to the corner fitting of the target container, the corner fitting image of the target container and the ROI of the vehicle plate lock button image corresponding to the corner fitting of the target container are extracted, and the 3D coordinate corresponding to the ROI center point of the corner fitting image of the target container and the 3D coordinate corresponding to the ROI center point of the vehicle plate lock button image corresponding to the corner fitting of the target container are respectively calculated.
Preferably, the second image includes two corner fitting images of the long side of the target container and two corresponding car board lock button images of the corner fittings of the long side of the target container.
Preferably, the method further comprises the following steps:
after extracting the ROI of the vehicle plate lock button image corresponding to the corner fitting of the target container, calculating the edge contour of the corner fitting of the target container;
and after extracting the corner fitting image of the target container and the ROI of the vehicle plate lock button image corresponding to the corner fitting of the target container, calculating the edge contour of the corner fitting of the target container and the edge contour of the vehicle plate lock button corresponding to the corner fitting of the target container.
Preferably, calculating the pose of the hub comprises calculating the pose of the hub (Xt, Yt, Zt, θ) according to the following formula, based on the width H of the bed of the hub trailer:
Xt＝(Pl1.x+Pl2.x)/2+H/2*cosθ
Yt＝(Pl1.y+Pl2.y)/2+H/2*sinθ
Zt＝(Pl1.z+Pl2.z)/2
theta is a heading angle of the container truck, the heading angle is an included angle between a straight line connected with centers of the truck plate lock buttons corresponding to the two corner pieces on the long side of the target container and a longitudinal coordinate of a cart coordinate system, (Xt, Yt, Zt) is a coordinate of a central point of the truck plate of the container truck in the cart coordinate system, Pl1 is a coordinate of a central point of one truck plate lock button corresponding to the two corner pieces on the long side of the target container in the cart coordinate system, and Pl2 is a coordinate of a central point of the other truck plate lock button corresponding to the two corner pieces on the long side of the target container in the cart coordinate system.
Preferably, the first image and the second image are acquired by an image acquisition device which is installed at a leg portion of the truck on a side close to the collection truck bed, and the image acquisition device faces the yard.
Preferably, the installation height of the image acquisition device is flush with the vehicle plate of the truck trailer, and the image acquisition device obtains the height of the truck lock button according to the acquired first image or second image so as to automatically adjust the height of the image acquisition device.
Preferably, the image acquisition device comprises a laser radar, a binocular camera, a depth camera and an RGBD camera.
Preferably, the dimensions of the target container include a 20-inch container and a 40-inch container, and when the target container is a 20-inch container, the corner fitting images of the 20-inch container are acquired by two of the image acquiring devices, and when the target container is a 40-inch container, the corner fitting images of the 40-inch container are acquired by the other two of the image acquiring devices.
Compared with the prior art, the invention has the following beneficial effects: according to the container truck positioning and aligning detection method based on machine vision, the central point coordinates of corner fittings of a target container and vehicle plate lock buttons corresponding to the corner fittings of the target container are calculated by acquiring the corner fitting images of the target container and the vehicle plate lock button images corresponding to the corner fittings of the target container, and the relative positions of the corner fittings and the corresponding lock buttons are obtained in real time, so that a trolley, a cart and a hanger are controlled to move corresponding distances to perform dynamic aligning;
further, point cloud clustering segmentation or deep learning target detection or instance segmentation is carried out on the corner fitting image of the target container and the vehicle plate lock button image corresponding to the corner fitting of the target container, the ROI of the vehicle plate lock button image corresponding to the corner fitting of the target container is extracted, the 3D coordinate corresponding to the ROI center point of the vehicle plate lock button image is calculated, the center or boundary coordinate of the vehicle plate lock button image is estimated according to the point cloud in the ROI, and the detection accuracy and robustness can be improved.
Drawings
FIG. 1 is a flow chart of a method for machine vision based hub positioning and bin alignment detection in an embodiment of the present invention;
FIG. 2 is a schematic structural diagram of a device for locating a container truck and detecting a container according to an embodiment of the present invention;
fig. 3 is a schematic structural diagram of a truck trailer of the truck positioning and box aligning detection device based on machine vision in the embodiment of the present invention.
Detailed Description
The invention is further described below with reference to the figures and examples.
In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present invention. However, it will be apparent to one of ordinary skill in the art that the present invention may be practiced without these specific details. Accordingly, the particular details set forth are merely exemplary, and the particular details may be varied from the spirit and scope of the present invention and still be considered within the spirit and scope of the present invention.
Fig. 1 is a flowchart of a method for truck positioning and box-to-box detection based on machine vision in an embodiment of the present invention, fig. 2 is a schematic structural diagram of a device for truck positioning and box-to-box detection based on machine vision in an embodiment of the present invention, and fig. 3 is a schematic structural diagram of a truck trailer of the device for truck positioning and box-to-box detection based on machine vision in an embodiment of the present invention. Referring now to fig. 1, the present invention provides a method for container truck positioning and container alignment detection based on machine vision, comprising the steps of:
step 101: acquiring a first image, wherein the first image at least comprises a vehicle plate lock button image corresponding to a corner fitting of a target container;
step 102: extracting the characteristics of the first image, calculating the coordinates of the central point of a vehicle plate lock button corresponding to the corner fitting of the target container, and calculating the position and posture of the container truck, so as to control the corresponding distance of movement of a trolley, a cart and a hanger to carry out primary positioning;
step 103: when the target container approaches a container truck target position, acquiring a second image, wherein the second image at least comprises a corner fitting image of the target container and a vehicle plate lock button image corresponding to a corner fitting of the target container;
step 104: and performing feature extraction on the second image, respectively calculating the center point coordinate of the corner fitting of the target container and the center point coordinate of the vehicle plate lock button corresponding to the corner fitting of the target container, and calculating the relative pose of the target container and the container truck, so as to control the corresponding moving distance of a trolley, a cart and a hanger to perform dynamic container loading.
In a specific implementation, as shown in fig. 2, the first image and the second image are acquired by image acquisition devices C1, C2, C3 and C4, the image acquisition devices C1, C2, C3 and C4 are installed at the leg of a cart near one side of a truck-collecting lane, and the image acquisition devices C1, C2, C3 and C4 face a yard. The installation height of the image acquisition devices C1, C2, C3 and C4 is flush with the plate of the truck trailer, and the height of the truck lock button is obtained by the image acquisition devices C1, C2, C3 and C4 according to the acquired first image or second image so as to automatically adjust the height of the image acquisition devices. The image acquisition devices C1, C2, C3, C4 include a laser radar, a binocular camera, a depth camera, and an RGBD camera. In a specific implementation, the number of the image acquisition devices can be more than 4, so as to acquire more accurate image information.
The sizes of the target containers include a 20-inch container and a 40-inch container, and when the target container is a 20-inch container, the corner fitting images of the 20-inch container are acquired by two of the image acquiring devices, and when the target container is a 40-inch container, the corner fitting images of the 40-inch container are acquired by the other two of the image acquiring devices. In a specific implementation, as shown in fig. 2 and 3, cameras C1 and C2 are used to detect the container truck attitude and the container 40 feet are aligned, cameras C3 and C4 are used to detect the container 20 feet are aligned, or cameras C3 and C4 are used to detect the container truck attitude and the container 40 feet are aligned, and cameras C1 and C2 are used to detect the container 20 feet are aligned, so that the cameras can acquire the complete corner fitting image of the target container and the vehicle panel lock button image corresponding to the corner fitting of the target container. When the target container is a 20-inch container, the locking buttons of the corner fittings of the target container are the locking buttons 32, 33, 36, 37 or 31, 301, 35, 391 or 302, 34, 392, 38, and when the target container is a 40-inch container, the locking buttons of the corner fittings of the target container are the locking buttons 31, 34, 35, 38.
Preferably, the second image includes two corner fitting images of the long side of the target container and two corresponding car board lock button images of the corner fittings of the long side of the target container. When two corner fittings on one long side of the target container and two corresponding vehicle plate locking buttons are successfully aligned, two corner fittings on the other long side of the target container can be successfully aligned with two corresponding vehicle plate locking buttons due to the rigid body characteristic of the container.
In a specific implementation, as shown in fig. 3, when the target container is a 20-inch container, the locking buttons for the two corner pieces on the long side of the target container are the locking buttons 32, 33 or 36, 37, the locking buttons 31, 301 or 35, 391, the locking buttons 302, 34 or 392, 38, when two corner pieces on one long side of the 20-inch container and the corresponding locking buttons 32, 33 or 36, 37 on the board are successful, the locking buttons 31, 301 or 35, 391 are also possible, the locking buttons 302, 34 or 392, 38 are also possible, and two corner pieces on the other long side of the 20-inch container and the corresponding locking buttons 36, 37 or 32, 33 on the other two boards are successful, the locking buttons 35, 391 or 31, 301 are also possible, the locking buttons 392, 38 or 302, 34. When the target container is a 40-inch container, the locking buttons of the locking buttons 31, 34 or 35, 38 are corresponding to two corner pieces on the long side of the target container, and when two corner pieces on one long side of the 40-inch container and the corresponding locking buttons 31, 34 or 35, 38 are successfully aligned with each other, two corner pieces on the other long side of the 40-inch container and the corresponding other two locking buttons 35, 38 or 31, 34 are successfully aligned with each other.
In specific implementation, the pose of the container truck is calculated according to the acquired first image, and the trolley, the cart and the lifting appliance are controlled to move corresponding distances, so that the target container approaches to the target position of the container truck. In an initial state, because the image acquisition device is installed at the leg of the cart close to one side of the truck collection lane, if the target container is far away from the cart floor, the first image only can include the cart floor lock button image corresponding to the corner fitting of the target container, and the corner fitting image of the target container cannot be acquired. If the target container is closer to the vehicle plate, the first image comprises an image of the corner fitting of the target container and an image of the vehicle plate lock button corresponding to the corner fitting of the target container. And when the target container approaches the target position of the container truck, acquiring a second image, wherein the second image comprises an angle piece image of the target container and a vehicle plate lock button image corresponding to the angle piece of the target container, calculating the relative position of the target container and the container truck according to the acquired second image, and controlling the trolley, the cart and the hanger to move corresponding distances for dynamic container loading.
Feature extraction of the first image may include point cloud cluster segmentation of a vehicle floor lock button image corresponding to a corner fitting of the target container; and the second image is subjected to feature extraction, namely point cloud clustering segmentation is carried out on the corner fitting image of the target container and the vehicle plate lock button image corresponding to the corner fitting of the target container.
Or performing feature extraction on the first image, including performing deep learning target detection or instance segmentation on a vehicle board lock button image corresponding to a corner fitting of the target container, extracting a Region of Interest (ROI) of the vehicle board lock button image corresponding to the corner fitting of the target container, and calculating a 3D coordinate corresponding to a center point of the ROI of the vehicle board lock button image; and the second image is subjected to feature extraction, namely, deep learning target detection or example segmentation is carried out on the corner fitting image of the target container and the vehicle plate lock button image corresponding to the corner fitting of the target container, the corner fitting image of the target container and the ROI of the vehicle plate lock button image corresponding to the corner fitting of the target container are extracted, and the 3D coordinate corresponding to the ROI center point of the corner fitting image of the target container and the 3D coordinate corresponding to the ROI center point of the vehicle plate lock button image corresponding to the corner fitting of the target container are respectively calculated.
In machine vision and image processing, a region to be processed, called a region of interest ROI, is delineated from a processed image in the form of a box, a circle, an ellipse, an irregular polygon, or the like. Various operators (operators) and functions are commonly used in machine vision software such as Halcon, OpenCV, Matlab and the like to obtain a region of interest (ROI), and the image is processed in the next step. The corner fittings of the container, also called box corners and hanging corners, are mainly used at each corner of the container, and the number of the corner fittings is usually 4, and the corner fittings are respectively arranged at 4 corners of the container. The corner fittings play a key role in the operations of hoisting, carrying, fixing, stacking and fastening of the container, and also play a role in protecting the whole container body as the outermost edge of the container body.
In specific implementation, after extracting the ROI of a vehicle plate lock button image corresponding to the corner fitting of the target container, calculating the edge contour of the corner fitting of the target container; and after extracting the corner fitting image of the target container and the ROI of the vehicle plate lock button image corresponding to the corner fitting of the target container, calculating the edge contour of the corner fitting of the target container and the edge contour of the vehicle plate lock button corresponding to the corner fitting of the target container.
Referring now to fig. 3, in the case that the target container shown in fig. 3 is a 40-foot container, cameras C1 and C2 are used for acquiring the first image and the second image, or cameras C1, C2, C3 and C4 are used for acquiring the first image and the second image at the same time, so as to acquire the corresponding image data more accurately, perform point cloud clustering segmentation on the acquired first image and second image, extract the point clouds of the images of the lock buttons 35 and 38, and calculate the centers Pl1 and Pl2 thereof.
In a specific implementation, according to the width H of the board of the truck trailer, the pose (Xt, Yt, Zt, θ) of the truck can be calculated according to the following formula:
Xt＝(Pl1.x+Pl2.x)/2+H/2*cosθ
Yt＝(Pl1.y+Pl2.y)/2+H/2*sinθ
Zt＝(Pl1.z+Pl2.z)/2
wherein θ is a heading angle of the container truck, the heading angle is an included angle between a straight line connecting centers of truck plate lock buttons corresponding to two corner fittings on a long side of the target container and a longitudinal coordinate of a cart coordinate system, that is, an included angle between a straight line connecting centers Pl1, Pl2 of lock buttons 35, 38 and a longitudinal coordinate of the cart coordinate system, (Xt, Yt, Zt) is a coordinate of a center point of the container truck plate in the cart coordinate system, Pl1 is a coordinate of the center point of the container truck plate lock button 35 in the cart coordinate system, and Pl2 is a coordinate of the center point of the container truck plate lock button 38 in the cart coordinate system.
According to the calculated central points of the container locking buttons 31, 34, 35 and 38 and the central points of the corner fittings corresponding to the 40-inch containers at the positions Pln and Pcn (n is 1,2,3 and 4) of the corresponding camera coordinate system, the relative position Pn of each corner fitting of the 40-inch container and the corresponding locking button can be obtained in real time (x, y and z) at Pln-Pcn, so that the corresponding distance of movement of the trolley, the trolley and the sling is controlled to carry out dynamic container alignment.
When the target container is a 20-size container, the position and orientation calculation method of the 20-size target container is consistent with that of the 40-size container, and the description is omitted here.
In summary, according to the method for positioning a container truck and detecting a container pair based on machine vision provided by this embodiment, coordinates of center points of corner fittings of a target container and vehicle floor lock buttons corresponding to the corner fittings of the target container are calculated by obtaining the images of the corner fittings of the target container and the images of the vehicle floor lock buttons corresponding to the corner fittings of the target container, and relative positions of the corner fittings and the corresponding lock buttons are obtained in real time, so that a trolley, a cart, and a spreader are controlled to move by corresponding distances to perform dynamic container pair;
further, point cloud clustering segmentation or deep learning target detection or instance segmentation is carried out on the corner fitting image of the target container and the vehicle plate lock button image corresponding to the corner fitting of the target container, the ROI of the vehicle plate lock button image corresponding to the corner fitting of the target container is extracted, the 3D coordinate corresponding to the ROI center point of the vehicle plate lock button image is calculated, the center or boundary coordinate of the vehicle plate lock button image is estimated according to the point cloud in the ROI, and the detection accuracy and robustness can be improved.
Although the present invention has been described with respect to the preferred embodiments, it will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention as defined by the appended claims.
Claims (10)
1. A method for positioning a container truck and detecting a container based on machine vision is characterized by comprising the following steps:
acquiring a first image, wherein the first image at least comprises a vehicle plate lock button image corresponding to a corner fitting of a target container;
extracting the characteristics of the first image, calculating the coordinates of the central point of a vehicle plate lock button corresponding to the corner fitting of the target container, and calculating the position and posture of the container truck, so as to control the corresponding distance of movement of a trolley, a cart and a hanger to carry out primary positioning;
when the target container approaches a container truck target position, acquiring a second image, wherein the second image at least comprises a corner fitting image of the target container and a vehicle plate lock button image corresponding to a corner fitting of the target container;
and performing feature extraction on the second image, respectively calculating the center point coordinate of the corner fitting of the target container and the center point coordinate of the vehicle plate lock button corresponding to the corner fitting of the target container, and calculating the relative pose of the target container and the container truck, so as to control the corresponding moving distance of a trolley, a cart and a hanger to perform dynamic container loading.
2. The machine-vision-based method of hub positioning and bin checking according to claim 1,
performing feature extraction on the first image comprises performing point cloud clustering segmentation on a vehicle plate lock button image corresponding to a corner fitting of the target container;
and the second image is subjected to feature extraction, namely point cloud clustering segmentation is carried out on the corner fitting image of the target container and the vehicle plate lock button image corresponding to the corner fitting of the target container.
3. The machine-vision-based method for locating a pallet and detecting a bin as claimed in claim 1, further comprising:
performing feature extraction on the first image comprises performing deep learning target detection or instance segmentation on a vehicle board lock button image corresponding to a corner fitting of the target container, extracting an ROI (region of interest) of the vehicle board lock button image corresponding to the corner fitting of the target container, and calculating a 3D (three-dimensional) coordinate corresponding to the ROI center point of the vehicle board lock button image;
and the second image is subjected to feature extraction, namely, deep learning target detection or example segmentation is carried out on the corner fitting image of the target container and the vehicle plate lock button image corresponding to the corner fitting of the target container, the corner fitting image of the target container and the ROI of the vehicle plate lock button image corresponding to the corner fitting of the target container are extracted, and the 3D coordinate corresponding to the ROI center point of the corner fitting image of the target container and the 3D coordinate corresponding to the ROI center point of the vehicle plate lock button image corresponding to the corner fitting of the target container are respectively calculated.
4. The machine-vision-based method for locating a pallet and detecting a container as claimed in claim 1, wherein said second image comprises two images of corner fittings of a long side of said target container and two images of a locking button of a deck corresponding to two corner fittings of a long side of said target container.
5. The machine-vision-based method for locating a pallet and detecting a bin as claimed in claim 3, further comprising:
after extracting the ROI of the vehicle plate lock button image corresponding to the corner fitting of the target container, calculating the edge contour of the corner fitting of the target container;
and after extracting the corner fitting image of the target container and the ROI of the vehicle plate lock button image corresponding to the corner fitting of the target container, calculating the edge contour of the corner fitting of the target container and the edge contour of the vehicle plate lock button corresponding to the corner fitting of the target container.
6. The machine-vision-based method of hub positioning and bin pair detection according to claim 1, wherein calculating the pose of the hub comprises calculating the pose of the hub (Xt, Yt, Zt, θ) according to the following formula, based on the width H of the bed of the hub trailer:
Xt＝(Pl1.x+Pl2.x)/2+H/2*cosθ
Yt＝(Pl1.y+Pl2.y)/2+H/2*sinθ
Zt＝(Pl1.z+Pl2.z)/2
theta is a heading angle of the container truck, the heading angle is an included angle between a straight line connected with centers of the truck plate lock buttons corresponding to the two corner pieces on the long side of the target container and a longitudinal coordinate of a cart coordinate system, (Xt, Yt, Zt) is a coordinate of a central point of the truck plate of the container truck in the cart coordinate system, Pl1 is a coordinate of a central point of one truck plate lock button corresponding to the two corner pieces on the long side of the target container in the cart coordinate system, and Pl2 is a coordinate of a central point of the other truck plate lock button corresponding to the two corner pieces on the long side of the target container in the cart coordinate system.
7. The machine-vision-based method of truck positioning and bin alignment according to claim 1, wherein the first and second images are acquired by an image acquisition device mounted on a leg of a truck near one side of the truck bed, the image acquisition device facing the yard.
8. The machine-vision-based truck positioning and bin alignment detection method according to claim 7, wherein the image capturing device is mounted at a height flush with a floor of the truck trailer, and the image capturing device obtains a height of a truck lock button according to the first image or the second image so as to automatically adjust the height of the image capturing device.
9. The machine-vision-based method for locating a pallet and detecting a box in a container as claimed in claim 8, wherein said image capturing device comprises a laser radar, a binocular camera, a depth camera and an RGBD camera.
10. The machine-vision-based pallet location and pair-bin inspection method of claim 7, wherein said dimensions of said target container include a 20-inch container and a 40-inch container, wherein when said target container is a 20-inch container, corner fitting images of said 20-inch container are acquired by two of said image acquisition devices, and when said target container is a 40-inch container, corner fitting images of said 40-inch container are acquired by the other two of said image acquisition devices.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010921668.5A CN112037283B (en) | 2020-09-04 | 2020-09-04 | Machine vision-based integrated card positioning and box alignment detection method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010921668.5A CN112037283B (en) | 2020-09-04 | 2020-09-04 | Machine vision-based integrated card positioning and box alignment detection method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112037283A true CN112037283A (en) | 2020-12-04 |
| CN112037283B CN112037283B (en) | 2024-04-30 |
Family
ID=73590553
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010921668.5A Active CN112037283B (en) | 2020-09-04 | 2020-09-04 | Machine vision-based integrated card positioning and box alignment detection method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112037283B (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113140007A (en) * | 2021-05-17 | 2021-07-20 | 上海驭矩信息科技有限公司 | Dense point cloud based collection card positioning method and device |
| CN113460888A (en) * | 2021-05-24 | 2021-10-01 | 武汉港迪智能技术有限公司 | Automatic box grabbing method for gantry crane lifting appliance |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101251381A (en) * | 2007-12-29 | 2008-08-27 | 武汉理工大学 | Dual container positioning system based on machine vision |
| KR20110069205A (en) * | 2009-12-17 | 2011-06-23 | 한국과학기술원 | Apparatus for estimating position and distance of container in container landing system and method thereof |
| CN108263950A (en) * | 2018-02-05 | 2018-07-10 | 上海振华重工(集团)股份有限公司 | Harbour gantry crane suspender based on machine vision it is automatic case system and method |
| CN110171779A (en) * | 2019-06-26 | 2019-08-27 | 中国铁道科学研究院集团有限公司运输及经济研究所 | Front handling mobile crane lifts by crane safely control system and control method |
| CN110902570A (en) * | 2019-11-25 | 2020-03-24 | 上海驭矩信息科技有限公司 | Dynamic measurement method and system for container loading and unloading operation |
| CN111137279A (en) * | 2020-01-02 | 2020-05-12 | 广州赛特智能科技有限公司 | Port unmanned truck collection station parking method and system |
| WO2020098933A1 (en) * | 2018-11-14 | 2020-05-22 | Abb Schweiz Ag | System and method to load a container on a landing target |
-
2020
- 2020-09-04 CN CN202010921668.5A patent/CN112037283B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101251381A (en) * | 2007-12-29 | 2008-08-27 | 武汉理工大学 | Dual container positioning system based on machine vision |
| KR20110069205A (en) * | 2009-12-17 | 2011-06-23 | 한국과학기술원 | Apparatus for estimating position and distance of container in container landing system and method thereof |
| CN108263950A (en) * | 2018-02-05 | 2018-07-10 | 上海振华重工(集团)股份有限公司 | Harbour gantry crane suspender based on machine vision it is automatic case system and method |
| WO2020098933A1 (en) * | 2018-11-14 | 2020-05-22 | Abb Schweiz Ag | System and method to load a container on a landing target |
| CN110171779A (en) * | 2019-06-26 | 2019-08-27 | 中国铁道科学研究院集团有限公司运输及经济研究所 | Front handling mobile crane lifts by crane safely control system and control method |
| CN110902570A (en) * | 2019-11-25 | 2020-03-24 | 上海驭矩信息科技有限公司 | Dynamic measurement method and system for container loading and unloading operation |
| CN111137279A (en) * | 2020-01-02 | 2020-05-12 | 广州赛特智能科技有限公司 | Port unmanned truck collection station parking method and system |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113140007A (en) * | 2021-05-17 | 2021-07-20 | 上海驭矩信息科技有限公司 | Dense point cloud based collection card positioning method and device |
| CN113140007B (en) * | 2021-05-17 | 2023-12-19 | 上海驭矩信息科技有限公司 | Concentrated point cloud-based set card positioning method and device |
| CN113460888A (en) * | 2021-05-24 | 2021-10-01 | 武汉港迪智能技术有限公司 | Automatic box grabbing method for gantry crane lifting appliance |
| CN113460888B (en) * | 2021-05-24 | 2023-11-24 | 武汉港迪智能技术有限公司 | Automatic box grabbing method for gantry crane lifting appliance |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112037283B (en) | 2024-04-30 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN105431370B (en) | For method and system container automatically unloaded using container crane in unloading target | |
| US11262189B2 (en) | Monitoring container transfer device on lowering container onto transport platform or lifting away from transport platform | |
| CN112027918B (en) | Detection method for preventing lifting of container truck based on machine vision | |
| US11667503B2 (en) | Vehicle-mounted device, cargo handling machine, control circuit, control method, and program thereof | |
| CN111114524A (en) | Hasp maneuvering | |
| CN112037283A (en) | Truck positioning and box aligning detection method based on machine vision | |
| US20220189055A1 (en) | Item detection device, item detection method, and industrial vehicle | |
| CN112010177B (en) | Automatic container landing method for ground containers in storage yard | |
| CN107067439B (en) | Container truck positioning and guiding method based on vehicle head detection | |
| WO2022121460A1 (en) | Agv intelligent forklift, and method and apparatus for detecting platform state of floor stack inventory areas | |
| JP2023040090A (en) | Method for generating landing solution for container on landing surface and system background | |
| CN111498525A (en) | Automatic loading positioning system and method | |
| CN114890280A (en) | Detection alignment method and device for lifting appliance | |
| CN109384150A (en) | The container tapered end framing of automated handling slings method with anti- | |
| CN207276067U (en) | The container tapered end framing of automated handling slings system with anti- | |
| KR20110069205A (en) | Apparatus for estimating position and distance of container in container landing system and method thereof | |
| JP7339532B2 (en) | loading and unloading system | |
| Li-Sheng et al. | Preceding vehicle detection based on multi-characteristics fusion | |
| CN111932576B (en) | Object boundary measuring method and device based on depth camera | |
| CN213325730U (en) | Automatic loading positioning system | |
| Park et al. | Container chassis alignment and measurement based on vision for loading and unloading containers automatically | |
| TWI663558B (en) | Rolled steel storage system and method of recognizing geometric center of rolled steel | |
| CN111854678A (en) | Pose measurement method based on semantic segmentation and Kalman filtering under monocular vision | |
| JP2020190814A (en) | Trajectory generation device | |
| CN113636298B (en) | Bag stacking state detection system and method of bagged material car loader |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |