WO2024120172A1 - 目标物定位方法、装置、设备和存储介质 - Google Patents

目标物定位方法、装置、设备和存储介质 Download PDF

Info

Publication number
WO2024120172A1
WO2024120172A1 PCT/CN2023/132785 CN2023132785W WO2024120172A1 WO 2024120172 A1 WO2024120172 A1 WO 2024120172A1 CN 2023132785 W CN2023132785 W CN 2023132785W WO 2024120172 A1 WO2024120172 A1 WO 2024120172A1
Authority
WO
WIPO (PCT)
Prior art keywords
shelf
target
position information
identified
grid
Prior art date
Application number
PCT/CN2023/132785
Other languages
English (en)
French (fr)
Inventor
郝越凡
Original Assignee
北京极智嘉科技股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from CN202211578417.7A external-priority patent/CN118172526A/zh
Priority claimed from CN202310889463.7A external-priority patent/CN116902467A/zh
Application filed by 北京极智嘉科技股份有限公司 filed Critical 北京极智嘉科技股份有限公司
Publication of WO2024120172A1 publication Critical patent/WO2024120172A1/zh

Links

Classifications

    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B65—CONVEYING; PACKING; STORING; HANDLING THIN OR FILAMENTARY MATERIAL
    • B65G—TRANSPORT OR STORAGE DEVICES, e.g. CONVEYORS FOR LOADING OR TIPPING, SHOP CONVEYOR SYSTEMS OR PNEUMATIC TUBE CONVEYORS
    • B65G1/00—Storing articles, individually or in orderly arrangement, in warehouses or magazines
    • B65G1/02—Storage devices
    • B65G1/04—Storage devices mechanical
    • B65G1/137—Storage devices mechanical with arrangements or automatic control means for selecting which articles are to be removed

Definitions

  • the present disclosure relates to the field of robotics technology, and in particular to a method, device, equipment and storage medium for locating a target object.
  • a transfer robot In a warehouse environment, when moving boxes on a shelf in the warehouse, a transfer robot usually first moves the shelf with multiple boxes to a preset fixed position of a workstation, and then a mechanical arm in a box-picking device on the workstation picks up the corresponding box, thereby moving the box to the target position.
  • a mechanical arm in a box-picking device on the workstation picks up the corresponding box, thereby moving the box to the target position.
  • the actual position where the transfer robot moves the shelf will have a certain deviation from the preset fixed position, and the mechanical arm in the box-picking device will be unable to pick up the corresponding box. Therefore, it is necessary to accurately locate the current position of the box.
  • Embodiments of the present disclosure provide a method, apparatus, device, and storage medium for locating a target object.
  • a method for locating a target object is provided, which is applied to a terminal device, wherein the terminal device is respectively connected to a control device and a visual scanning device on a box-removing device, and the control device is connected to a mechanical arm on the box-removing device.
  • the method comprises: after a transfer robot transports a shelf to a workstation, acquiring image information of a position to be identified on the shelf collected by the visual scanning device; wherein the position to be identified comprises an identification to be identified and/or a target grid, wherein the identification to be identified comprises at least one of a visual identification code set on the shelf, a top angle of the shelf, and an intersection position of grids in the shelf, and the number of visual identification codes set on the shelf is less than the number of grids in the shelf; the target grid comprises a grid for placing a target object to be transported; based on the position to be identified
  • the method comprises the steps of: obtaining image information of a position to be identified, calculating first position information of the position to be identified in the first coordinate system; wherein the first coordinate system is a coordinate system constructed based on a visual scanning device; obtaining second position information of the position to be identified in the second coordinate system; wherein the second coordinate system is a coordinate system constructed based on a shelf; calculating a conversion relationship between the
  • the number of the above-mentioned markers to be identified is greater than or equal to four, and the positions of the markers to be identified are not on a straight line.
  • the mark to be identified includes the four top corners of the front area of the shelf, or the four top corners of the top area of the shelf.
  • the mark to be identified includes the intersection position of the grid openings in the front area of the shelf, or the intersection position of the grid openings in the top area of the shelf.
  • the mark to be identified includes the top corner of the shelf front area and the intersection of the grid in the shelf front area; or, the mark to be identified includes the top corner of the shelf top area and the intersection of the grid in the shelf top area.
  • the above-mentioned identification to be identified includes visual identification codes, the number of visual identification codes is greater than four, and each visual identification code is respectively set at the four top corners of the front area of the shelf and at least one preset target position of the top area; or, each visual identification code is respectively set at the four top corners of the top area of the shelf and at least one preset target position of the front area; wherein the preset target positions include the top corner positions and the center positions.
  • the identification to be identified includes a visual identification code, and the number of the visual identification codes is four, and each visual identification code is respectively set at the four top corners of the front area or the top area of the shelf.
  • the above-mentioned identification mark to be identified includes a visual identification code, the number of visual identification codes is three, and each visual identification code is respectively set at any three top corners of the front area or the top area of the shelf; or, the number of visual identification codes is two, and each visual identification code is respectively set at two top corners located at the diagonal line of the front area or the top area of the shelf; or, the number of visual identification code is one, and the visual identification code is set at the center position of the front area or the top area of the shelf.
  • the above-mentioned position to be identified is a target grid; the above-mentioned calculation of the first position information of the position to be identified in the first coordinate system based on the image information of the position to be identified includes: calculating the first position information of a preset target point on the target grid in the first coordinate system based on the image information of the target grid.
  • the preset target points are four vertices of the target grid.
  • the above-mentioned acquisition of image information of the position to be identified on the shelf collected by the visual scanning device includes: acquiring two-dimensional image information of the target grid on the shelf collected by the visual scanning device; the above-mentioned calculation of the first position information of the preset target point on the target grid in the first coordinate system based on the image information of the target grid includes: based on the two-dimensional image information of the target grid, screening out the preset target points on the target grid; and using the pixel position information of the preset target point in the two-dimensional image information as the first position information.
  • the above-mentioned two-dimensional image information based on the target grid is used to screen out preset target points on the target grid, including: using a preset algorithm to perform image processing on the two-dimensional image information of the target grid, and extracting multiple straight lines in the two-dimensional image information; calculating the intersection between every two straight lines as candidate target points; based on preset size information and shape information of the target grid, screening out preset target points from the candidate target points.
  • the above-mentioned preset algorithm is used to perform image processing on the two-dimensional image information of the target grid to extract multiple straight lines in the two-dimensional image information, including: performing gradient operation processing on the two-dimensional image information of the target grid to generate a gradient map of the target grid; performing Hough transform processing on the gradient map to extract multiple straight lines in the gradient map.
  • the preset target point is the center point corresponding to the four sides of the target grid respectively.
  • the above-mentioned acquisition of image information of the position to be identified on the shelf collected by the visual scanning device includes: acquiring three-dimensional image information of the target grid on the shelf collected by the visual scanning device; the above-mentioned calculation of the first position information of the preset target point on the target grid in the first coordinate system based on the image information of the target grid includes: based on the three-dimensional image information of the target grid, screening out the preset target point on the target grid; and using the position information of the preset target point relative to the visual scanning device as the first position information.
  • the above-mentioned three-dimensional image information based on the target grid screens out preset target points on the target grid, including: using a preset algorithm to perform image processing on the three-dimensional image information of the target grid, and extracting four straight lines that constitute the target grid from the three-dimensional image information; for each straight line, respectively obtaining pixel position information corresponding to a preset number of pixel points located around each straight line; and determining the preset target point based on each pixel position information.
  • the above-mentioned calculation of the transformation relationship between the first coordinate system and the second coordinate system based on the first position information and the second position information includes: using a preset algorithm to calculate the first position information and the second position information to generate rotation parameters and translation parameters; based on the rotation parameters and translation parameters, determining the transformation relationship between the first coordinate system and the second coordinate system.
  • the above-mentioned determination of the position of the object to be transported relative to the box loading device based on the third position information and the conversion relationship is as follows:
  • the fourth position information of the target object to be transported includes: determining the intermediate position information of the target object to be transported relative to the visual scanning device based on the third position information and the conversion relationship; determining the fourth position information of the target object to be transported relative to the box picking device based on the intermediate position information and the preset position relationship between the visual scanning device and the box picking device.
  • the obtaining of third position information of at least one target object to be transported on the shelf in the second coordinate system includes: obtaining third position information of a slot on the shelf for placing the target object to be transported in the second coordinate system.
  • the above-mentioned acquisition of the second position information of the mark to be identified in the second coordinate system includes: constructing the second coordinate system with a preset point on the shelf as the origin; and calculating the second position information of the mark to be identified in the second coordinate system based on the preset size information of the grid on the shelf for placing the target object to be transported.
  • a target positioning device which includes: a first acquisition module, configured to acquire image information of a position to be identified collected by a visual scanning device after a transfer robot transports a shelf to a workstation; wherein the position to be identified includes a mark to be identified and/or a target grid, and the mark to be identified includes at least one of a visual identification code set on the shelf, a top corner of the shelf, and an intersection of grids in the shelf, and the number of visual identification codes set on the shelf is less than the number of grids in the shelf; the target grid includes a grid for placing a target object to be transported; a first calculation module, configured to calculate first position information of the position to be identified in a first coordinate system based on the image information of the position to be identified; wherein the first calculation module is configured to calculate first position information of the position to be identified in a first coordinate system based on the image information of the position to be identified;
  • a coordinate system is a coordinate system constructed based on a visual scanning device
  • an electronic device comprising: a processor and a memory, the memory being used to store computer executable instructions; the processor being used to read instructions from the memory and execute the instructions to implement the aforementioned first aspect and the method of any implementation manner of the first aspect.
  • a computer-readable storage medium in which computer instructions are stored.
  • the computer instructions are used to enable the computer to execute the method in the aforementioned first aspect and any implementation manner of the first aspect.
  • a computer program product which includes a computer program stored on a computer-readable storage medium, and the computer program includes program instructions.
  • the program instructions When the program instructions are executed by a computer, the computer executes the method in any implementation manner of the aforementioned first aspect.
  • the target object positioning method, device, equipment and storage medium provided by the embodiments of the present disclosure can determine the position information of the target object to be transported relative to the box retrieval device by identifying the position to be identified on the shelf (such as the visual identification code, the intersection position of the top corner of the shelf and the grid in the shelf, and the target grid on the shelf). Therefore, there is no need to install a visual identification code on the shelf, or only a small number of visual identification codes need to be installed on the shelf to determine the position of the target object to be transported relative to the box retrieval device.
  • the manufacturing cost and installation cost of the visual identification code can be reduced; in addition, since the image information of the identification to be identified only needs to be collected once, and the position information of all the targets to be transported on the shelf can be determined at one time through the calculated conversion relationship, the efficiency of positioning the targets to be transported is improved.
  • FIG1 is a schematic diagram of a scene of a target positioning method provided according to some embodiments of the present disclosure
  • FIG2 is a flow chart of a method for locating a target object according to some embodiments of the present disclosure
  • FIG3A is a schematic diagram of a shelf front area including a mark to be identified according to some embodiments of the present disclosure
  • FIG3B is a schematic diagram of another example of a shelf front area including a mark to be identified according to some embodiments of the present disclosure
  • FIG3C is a schematic diagram of setting a visual identification code on the front area of a shelf according to some embodiments of the present disclosure
  • FIG4A is a schematic diagram of setting a visual identification code on a top area of a shelf according to some embodiments of the present disclosure
  • FIG4B is a schematic diagram of four vertices of a target grid provided according to some embodiments of the present disclosure.
  • FIG4C is a flowchart of calculating first position information according to some embodiments of the present disclosure.
  • FIG5 is a flow chart of screening preset target points according to some embodiments of the present disclosure.
  • FIG6 is a flow chart of extracting multiple straight lines according to some embodiments of the present disclosure.
  • FIG7 is a schematic diagram of four sides of a target grid provided according to some embodiments of the present disclosure.
  • FIG8 is a flowchart of another method for calculating first position information according to some embodiments of the present disclosure.
  • FIG9 is a flowchart of another method for screening preset target points according to some embodiments of the present disclosure.
  • FIG10 is a flowchart of calculating second position information according to some embodiments of the present disclosure.
  • FIG11 is a flow chart of a calculation conversion relationship provided according to some embodiments of the present disclosure.
  • FIG12 is a flow chart of determining fourth position information according to some embodiments of the present disclosure.
  • FIG13 is a schematic diagram of the structure of a target positioning device provided according to some embodiments of the present disclosure.
  • FIG. 14 is a schematic diagram of the structure of an electronic device provided according to some embodiments of the present disclosure.
  • a visual identification code is usually set at each grid where the cargo box is placed on the shelf.
  • the visual acquisition device in the box-picking device is used to obtain the image information of the visual identification code at the grid where the object to be transported is placed, and then the position information of the object to be transported relative to the visual scanning device is calculated based on the image information.
  • the visual scanning device Since the visual scanning device is installed on the box-picking device and the relative position relationship between the two is known, the accurate position information of the object to be transported relative to the box-picking device can be calculated, and then the robot arm can be controlled to pick up the object to be transported based on the position information.
  • the cost of installing the visual identification code will increase; in addition, since the image information of the visual identification code at each grid needs to be collected before all the objects to be transported on the shelf can be accurately located, the efficiency of positioning the cargo box is reduced.
  • the disclosed embodiments propose a method, device, equipment and storage medium for locating a target object.
  • the position information of the target object to be transported relative to the box retrieval device can be determined. Therefore, there is no need to install a visual identification code on the shelf, or only a small number of visual identification codes need to be installed on the shelf to determine the position of the target object to be transported relative to the box retrieval device. Since there is no need to install a visual identification code at each grid opening on the shelf, the manufacturing cost and installation cost of the visual identification code can be reduced.
  • the image information of the identification to be identified only needs to be collected once, and the conversion relationship obtained by calculation can be used to determine the position information of all the targets to be transported (such as cargo boxes) on the shelf at one time, The information is used to improve the efficiency of positioning the objects to be transported.
  • FIG. 1 is a scene schematic diagram of a target positioning method provided according to some embodiments of the present disclosure.
  • the terminal device 102 is respectively connected to the control device 104 and the visual scanning device 1061 on the box-picking device 106, and the control device 104 is connected to the mechanical arm 1062 on the box-picking device 106.
  • An upper-layer business system is deployed in the terminal device 102 , and the terminal device 102 is configured to execute the target positioning method provided in the following embodiment of the present disclosure.
  • the visual scanning device 1061 may be a camera, which may be a 2D camera or a 3D camera.
  • the visual scanning device 1061 is configured to scan and identify the position to be identified on the shelf (such as a visual identification code, a shelf top corner, a shelf grid intersection position, or a target grid), and collect image information of the position to be identified.
  • the control device 104 may include a programmable logic controller (PLC).
  • PLC programmable logic controller
  • the control device 104 is configured to receive the position information of the target object to be transported relative to the box-picking device sent by the terminal device 102, and control the robot arm 1062 to pick up the target object to be transported on the shelf based on the position information.
  • the robot arm 1062 may be a suction cup type robot arm or other types of robot arms, which is not limited in the embodiments of the present disclosure.
  • FIG2 is a flow chart of a target positioning method provided according to some embodiments of the present disclosure, the method comprising the following steps:
  • Step 202 After the transfer robot moves the shelf to the workstation, image information of the position to be identified on the shelf collected by the visual scanning device is obtained.
  • the position to be identified includes a mark to be identified and/or a target grid
  • the mark to be identified includes at least one of a visual identification code set on the shelf, a top corner of the shelf and an intersection position of a grid in the shelf
  • the number of visual identification codes set on the shelf is less than the number of grids in the shelf.
  • each shelf has multiple slots, and each slot is used to place cargo boxes, original boxes, material boxes, containers, goods, etc.
  • the target grid includes a grid for placing the target object to be transported.
  • a number of visual identification codes less than the number of grid openings in the shelf can be set on the shelf, and the visual identification codes can be used as identification marks to be identified on the shelf.
  • the visual identification code and the top corner of the shelf can be used as identification marks to be identified on the shelf.
  • the visual identification code and the intersection of the grid openings in the shelf can be used as identification marks to be identified on the shelf.
  • the visual identification code, the top corner of the shelf, and the intersection of the grid openings in the shelf can be used together as identification marks to be identified on the shelf.
  • the number of identification marks to be identified may be greater than or equal to four, and the positions of the identification marks to be identified are not in a straight line.
  • the more identification marks to be identified the more accurate the position of the object to be transported is determined.
  • the embodiment of the present disclosure does not limit the number of identification marks to be identified.
  • the mark to be identified may include the top corners of the shelf.
  • the number of the top corners of the shelf is greater than or equal to four.
  • the mark to be identified includes the four top corners of the front area of the shelf, or the four top corners of the top area of the shelf.
  • the mark to be identified may also include the four top corners of the side area of the shelf.
  • the mark to be identified includes, in addition to the top corners of the shelf, the intersection position of the grid in the shelf and/or the visual identification code
  • the number of the top corners of the shelf may be less than four.
  • the embodiment of the present disclosure does not limit the position and number of the top corners of the shelf included in the mark to be identified.
  • the position of the top corners of the shelf included in the mark to be identified is related to the installation position of the visual scanning device.
  • Figure 3A is a schematic diagram of a shelf front area including a mark to be identified provided by some embodiments of the present disclosure.
  • the gray dots in Figure 3A represent the four vertex corners of the shelf front area, and the mark to be identified on the shelf may be the four vertex corners of the shelf front area.
  • the identification to be identified includes the intersection positions of the grids in the shelf.
  • the number of intersection positions of the shelf grids is greater than or equal to four.
  • the identification to be identified includes at least four intersection positions of the grids located in the front area of the shelf, or at least four intersection positions of the grids located in the top area of the shelf.
  • the identification to be identified may also include at least four intersection positions of the grids located in the side area of the shelf.
  • the number of intersection positions of the shelf grids may be less than four.
  • the embodiments of the present disclosure do not limit the position and number of the intersection positions of the grids included in the identification to be identified, and the intersection positions of the grids included in the identification to be identified are related to the installation position of the visual scanning device.
  • Figure 3B is a schematic diagram of another shelf front area including a mark to be identified provided by some embodiments of the present disclosure.
  • the shelf includes compartments A to L, and the intersection of each compartment is a mark to be identified.
  • the gray dots in Figure 3B all represent marks to be identified.
  • the mark to be identified may include both the top corner of the shelf and the intersection of the grid in the shelf.
  • the mark to be identified includes the top corner of the front area of the shelf and the intersection of the grid in the front area of the shelf.
  • the mark to be identified includes the top corner of the top area of the shelf and the intersection of the grid in the top area of the shelf.
  • the identification to be identified may also be a visual identification code.
  • the visual identification code is an identification code used for positioning, and usually an Apriltag code may be used.
  • the Apriltag code is a visual positioning identifier, which is similar to a QR code or a barcode.
  • an ArUco code may also be used, and the disclosed embodiment is not limited to this. The principle of using the Apriltag code and the ArUco code for positioning is a prior art and will not be repeated here.
  • FIG. 3C is a schematic diagram of setting a visual identification code on the front area of a shelf provided by the embodiment of the present disclosure. As shown in Figure 3C, the shelf includes grids A to L, and the gray dot under each grid represents the visual identification code corresponding to the grid, that is, a visual identification code needs to be set at each grid. While the number of visual identification codes set on the shelf in the embodiment of the present disclosure is less than the number of grids in the shelf, the embodiment of the present disclosure does not limit the position of the visual identification code set on the shelf.
  • the number of visual identification codes can be greater than four and less than the number of grids in the shelf, and each visual identification code is respectively set at the four top corners of the front area of the shelf and at least one preset target position of the top area, or each visual identification code is respectively set at the four top corners of the top area of the shelf and at least one preset target position of the front area.
  • the preset target position includes the top corner position and the center position, and the example in which each visual identification code is respectively set at the four top corners of the front area of the shelf and the four top corners of the top area is used for explanation.
  • the two determined location information can then be compared. For example, a difference comparison can be made. If the difference is less than the preset threshold, it means that the determined location information of the shelf is accurate; if the difference is greater than the preset threshold, it means that the determined location information of the shelf is inaccurate. At this time, the installation position of the visual identification code can be rechecked to see if there is any deviation. This can not only reduce the manufacturing cost and installation cost of the visual identification code, but also improve the accuracy of cargo box positioning.
  • the number of visual identification codes may be four, and each visual identification code may be set at the four corners of the front area or the top area of the shelf. Please continue to refer to Figure 3C, it can be seen that the dots at the four corners of the shelf represent a visual identification code respectively, and the four visual identification codes can be identified as TAG1, TAG2, TAG3, and TAG4 respectively.
  • FIG4A is a schematic diagram of setting a visual identification code on the top area of a shelf according to some embodiments of the present disclosure.
  • the four visual identification codes on the shelf are represented by small rectangular boxes located at the four top corners.
  • the disclosed embodiment does not limit the position and number of the visual identification codes set on the shelf.
  • the setting position of the visual identification code is related to the installation position of the visual scanning device, and the number of visual identification codes set can be less than the number of grids in the shelf.
  • the number of visual identification codes can be three, and each visual identification code is set at any three top corners of the front area or the top area of the shelf. Since the surface where the entire shelf is located can still be determined by the three visual identification codes, the location information of the shelf can be determined.
  • the number of visual identification codes may be two, and each visual identification code is respectively set at two top corners located at the diagonal line of the front area or the top area of the shelf.
  • the number of visual identification codes may also be one, and the visual identification code is set at the center of the front area or the top area of the shelf. Combining two visual identification codes or one visual identification code and shelf size information can determine the location information of the shelf, and further reduce the manufacturing cost and installation cost of the visual identification code.
  • the target slot can be any slot on the shelf, or a slot on the shelf for placing the target object to be transported.
  • the target object to be transported can be a cargo box, or an original box, a material box, a container, the cargo itself, or other target objects on the shelf, which is not limited in the embodiments of the present disclosure.
  • any grid on the shelf can be used as a position to be identified to determine the position of the target object to be transported relative to the box-taking device; the grid used to place the target object to be transported can also be used as a position to be identified to determine the position of the target object to be transported relative to the box-taking device, and the embodiments of the present disclosure are not limited to this. Since the box-taking device moves to the grid where the target object to be transported is placed when taking the target object to be transported on the shelf, the image information of the grid where the target object to be transported is placed (such as the target grid) can be collected by a visual scanning device to determine the position of the target object to be transported relative to the box-taking device.
  • the grid used to place the target object to be transported can be used as a position to be identified to determine the position of the target object to be transported relative to the box retrieval device.
  • the visual scanning device can be a camera, which can be a 2D camera or a 3D camera.
  • the visual scanning device is used to scan and identify the identification mark and/or target grid on the shelf, and collect image information of the position to be identified.
  • a control instruction can be sent to the visual scanning device to control the visual scanning device to collect image information of the position to be identified.
  • the collected image information of the position to be identified can also include image information of objects around the position to be identified.
  • the implementation method of obtaining the position information of the target object to be transported relative to the box-removing device is similar to the implementation method of obtaining the position information of the target object to be transported relative to the box-removing device when the identification to be identified is the top corner of the shelf and/or the intersection position of the grid in the shelf.
  • the following embodiments respectively use the position to be identified including the visual identification code and the target grid as examples to illustrate the implementation method of obtaining the position information of the target object to be transported relative to the box-removing device.
  • Step 204 Calculate first position information of the position to be identified in a first coordinate system based on the image information of the position to be identified.
  • the first coordinate system is a coordinate system constructed based on a visual scanning device.
  • the method of obtaining the position information of the visual identification code in the first coordinate system is similar to that of obtaining the position information of the visual identification code in the first coordinate system, which will not be repeated here.
  • the above step 204 includes: calculating first position information of a preset target point on the target grid in a first coordinate system based on the image information of the target grid.
  • the preset target points may be four vertices of the target grid.
  • FIG4B is a schematic diagram of four vertices of a target grid provided according to some embodiments of the present disclosure. As shown in FIG4B , taking the target grid as grid E as an example, the four vertices of the target grid E are E1, E2, E3, and E4, respectively. E1, E2, E3, and E4 may be used as preset target points. If the visual scanning device is a 2D camera, two-dimensional image information including four fixed points of the target grid on the shelf collected by the 2D camera may be obtained.
  • FIG4C is a flowchart of calculating first position information according to some embodiments of the present disclosure. As shown in FIG4C , the method includes the following steps:
  • Step 402 based on the two-dimensional image information of the target grid, select the preset target points on the target grid.
  • the two-dimensional image information of the target grid may include image information of other objects in addition to the grid image information, so the four vertices of the target grid need to be screened out according to the two-dimensional image information.
  • FIG5 is a flow chart of screening preset target points according to some embodiments of the present disclosure. As shown in FIG5 , the method includes the following steps:
  • Step 502 Use a preset algorithm to perform image processing on the two-dimensional image information of the target grid, and extract multiple straight lines from the two-dimensional image information.
  • the preset algorithm may include gradient operation and Hough transform.
  • the gradient operation is used to detect the edge information of the image and obtain the gradient map of the image.
  • the Hough transform is used to extract multiple straight lines from the generated gradient map.
  • other algorithms such as the least squares method, the ray method, etc., may also be used, and the embodiments of the present disclosure are not limited to this.
  • FIG6 is a flow chart of extracting multiple straight lines according to some embodiments of the present disclosure. As shown in FIG6 , the method includes the following steps:
  • Step 602 Perform gradient calculation on the two-dimensional image information of the target grid to generate a gradient map of the target grid.
  • Step 604 Perform Hough transform processing on the gradient image to extract multiple straight lines in the gradient image.
  • the image gradient represents the speed of change of the grayscale value of the image.
  • the grayscale values on both sides of the edge of the image differ greatly, and the gradient value is large. Therefore, performing a gradient operation on an image can highlight the information of the image edge. Since the pixel value is a discrete value, its gradient is actually a differential operation, which is essentially a spatial filter. By performing a gradient operation on the two-dimensional image information of the target grid, the edge information of the two-dimensional image can be highlighted, and the gradient map of the target grid can be generated.
  • Hough transform is a feature extraction operation process that is widely used in image analysis, computer vision and digital image processing. Hough transform is used to identify features in objects, such as lines.
  • the algorithm process is roughly as follows: you can first give an object and the type of shape to be identified. The algorithm will perform voting in the parameter space to determine the shape of the object, which is determined by the local maximum in the cumulative space.
  • Hough transform processing By performing Hough transform processing on the gradient map, multiple straight lines in the gradient map are extracted.
  • the accuracy and efficiency of extracting straight lines using Hough transform processing are both high, thereby improving the accuracy and efficiency of screening preset target points.
  • Step 504 Calculate the intersection point between every two straight lines as a candidate target point.
  • Step 506 based on the preset size information and shape information of the target grid, select a preset target point from the candidate target points.
  • intersection point between the two straight lines can be calculated. After the intersection point between each two straight lines is calculated, all the calculated intersection points can be used as candidate target points.
  • the lines connecting the candidate target points will form multiple closed areas. Since the target grid is usually rectangular and its size information is also is known, so the area of the closed area corresponding to the target grid is also determined. Then, the closed area corresponding to the target grid can be determined first through the preset size information and shape information of the target grid, that is, the rectangular area corresponding to the target grid, and the four vertices constituting the rectangular area are used as the preset target points, that is, the preset target points are selected from the candidate target points.
  • Step 404 Use pixel position information of the preset target point in the two-dimensional image information as first position information.
  • the preset target point may be the center point corresponding to the four sides of the target grid.
  • FIG7 is a schematic diagram of the four sides of a target grid provided according to some embodiments of the present disclosure. As shown in FIG7, taking the target grid as grid E as an example, the target grid E is a rectangle surrounded by four straight lines, that is, the target grid E includes four sides. If the visual scanning device is a 3D camera, the three-dimensional image information of the target grid on the shelf collected by the visual scanning device can be obtained.
  • FIG8 is a flowchart of another method for calculating first position information according to some embodiments of the present disclosure. As shown in FIG8 , the method includes the following steps:
  • Step 802 based on the three-dimensional image information of the target grid, select the preset target points on the target grid.
  • Step 804 Use the position information of the preset target point relative to the visual scanning device as the first position information.
  • the three-dimensional image information of the target grid may include image information of other objects except the grid, so it is necessary to filter out the center points corresponding to the four sides of the target grid respectively according to the three-dimensional image information.
  • FIG9 is a flowchart of another method for screening preset target points according to some embodiments of the present disclosure. As shown in FIG9 , the method includes the following steps:
  • Step 902 Use a preset algorithm to perform image processing on the three-dimensional image information of the target grid, and extract four straight lines constituting the target grid from the three-dimensional image information.
  • the preset algorithm may use gradient operation and Hough transform
  • the method for extracting four straight lines constituting the target grid from the three-dimensional image information may refer to the above-mentioned process of extracting multiple straight lines from the two-dimensional image information of the target grid, which will not be repeated here.
  • Step 904 for each straight line, respectively obtain pixel position information corresponding to a preset number of pixel points located around each straight line.
  • Step 906 Determine a preset target point based on the position information of each pixel.
  • the pixel position information corresponding to a preset number of pixel points located around each straight line can be obtained respectively.
  • the pixel position information of five pixel points around each straight line can be obtained. Therefore, the five pixel position information can be averaged and the average value can be used as the pixel position information of a center point corresponding to the straight line.
  • the center points corresponding to the four sides of the target grid can be calculated, that is, four center points can be obtained. Then, the position information of the preset target point relative to the visual scanning device can be directly used as the above-mentioned first position information.
  • Step 206 Obtain second position information of the position to be identified in a second coordinate system.
  • the second coordinate system is a coordinate system constructed based on the shelf.
  • the position to be identified includes a visual identification code
  • the coordinate system XYZ is the second coordinate system.
  • FIG10 is a flow chart of calculating second position information provided by an embodiment of the present disclosure. As shown in FIG10 , the method includes the following steps:
  • Step 1002 construct a second coordinate system with a preset point on the shelf as the origin.
  • Step 1004 Calculate the position to be identified based on the preset size information of the grid for placing the target object to be transported on the shelf Second position information in a second coordinate system.
  • the preset point can be any predetermined position in the shelf.
  • the preset point can be the vertex at the upper left corner of the four corners of the shelf, so that the vertex can be used as the origin of the second coordinate system, and the X-axis is established in the horizontal direction, the Y-axis is established in the vertical direction, and the Z-axis is perpendicular to the X-axis and the Y-axis respectively, thereby constructing the second coordinate system.
  • N in the preset size information of each grid opening, N can be used to represent the size of the grid opening in the horizontal direction, M represents the size of the grid opening in the vertical direction, and the size in the Z-axis direction can be uniformly set to 0.
  • the image information of the mark to be identified collected by the above-mentioned visual scanning device may include all grids in the front area of the shelf.
  • the visual scanning device may be a fisheye camera. After the transfer robot transports the shelf to the workstation, the fisheye camera collects image information including all grids in the front area of the shelf, and the second position information of the intersection position of each grid in the mark to be identified in the second coordinate system can be determined through the image information.
  • the visual scanning device collects image information of the upper grids of the shelf (for example, the upper half of the grids, the lower half of the grids, the left half of the grids or the right half of the grids in the front area of the shelf; for another example, the upper half of the grids, the lower half of the grids, the left half of the grids or the right half of the grids in the top area of the shelf), and the second position information of the intersection position of each grid in the mark to be identified in the second coordinate system can be determined through the image information.
  • the visual scanning device can also collect multiple images including the identification mark to be identified during the process of the transfer robot moving the shelf to the workstation. The second position information of the intersection position of each grid in the identification mark to be identified in the second coordinate system can be determined through the multiple images.
  • the second coordinate system can also be a coordinate system constructed based on the target grid on the shelf.
  • the coordinate system XY in Figure 4B is the second coordinate system.
  • the origin of the second coordinate system can be any predetermined position in the target grid.
  • the second coordinate system can be constructed with the vertex E1 of the target grid E as the origin, and the X axis is established in the horizontal direction, the Y axis is established in the vertical direction, and the size in the Z direction can be set to 0, so the Z axis is not marked in Figure 4B.
  • Step 208 Calculate a conversion relationship between the first coordinate system and the second coordinate system based on the first position information and the second position information.
  • FIG11 is a flow chart of a method for calculating a conversion relationship according to some embodiments of the present disclosure. As shown in FIG11 , the method includes the following steps:
  • Step 1102 Calculate the first position information and the second position information using a preset algorithm to generate rotation parameters and translation parameters.
  • Step 1104 Determine a conversion relationship between the first coordinate system and the second coordinate system based on the rotation parameter and the translation parameter.
  • the preset algorithm may be a PnP (Perspective-n-Point) algorithm, which is a method for solving the correspondence between 3D and 2D points, and describes how to estimate the camera's posture when n 3D space points and their positions are known.
  • PnP Perspective-n-Point
  • the rotation parameter R and the translation parameter t can be calculated using the PnP algorithm.
  • the solution process is a prior art and will not be described in detail here.
  • the conversion relationship between the first coordinate system and the second coordinate system can be determined based on the rotation parameter R and the translation parameter t.
  • PTAG Camera PTAG Shelf R+t (1)
  • PTAG Camera represents the first position information of the position to be identified
  • PTAG Shelf represents the second position information of the position to be identified
  • R represents the rotation parameter
  • t represents the translation parameter.
  • Step 210 obtain the third position information of at least one target object to be transported on the shelf in the second coordinate system, determine the fourth position information of the target object to be transported relative to the box picking device based on the third position information and the conversion relationship, and send the fourth position information to the control device, triggering the control device to control the robotic arm to pick up the target object to be transported based on the fourth position information.
  • the method for determining the third position information of the object to be transported in the second coordinate system can refer to the process of determining the second position information described above. Taking the object to be transported placed at the opening E in FIG. 3C as an example, the third position information of the object to be transported can be expressed as
  • the above-mentioned obtaining the third position information of at least one target object to be transported on the shelf in the second coordinate system includes: obtaining the third position information of the target grid on the shelf in the second coordinate system.
  • the third position information of the target grid in the second coordinate system may be the same as the second position information of a preset target point on the target grid, or may be different from the second position information of the preset target point on the target grid.
  • the third position information of the target slot E may be the position information of E0 in the second coordinate system, expressed as It may also be the position information of E1, E2, E3 or E4 in the second coordinate system.
  • the third position information of the target object to be transported in the second coordinate system can be obtained through step 210.
  • the third position information of the target object to be transported in the second coordinate system can be the third position information of any position on the grid where the target object to be transported is placed in the second coordinate system.
  • FIG. 12 is a flow chart of determining fourth position information provided by some examples of the present disclosure. As shown in FIG. 12 , the method includes the following steps:
  • Step 1202 Based on the third position information and the conversion relationship, determine the intermediate position information of the target object to be transported relative to the visual scanning device.
  • the position information of the target object to be transported relative to the visual scanning device i.e., the intermediate position information
  • the position information of the target object to be transported relative to the visual scanning device i.e., the intermediate position information
  • PE Camera represents the position information of the object to be transported placed at the opening E in the first coordinate system (eg, the middle position information)
  • PE Shelf represents the position information of the object to be transported placed at the opening E in the second coordinate system (eg, the third position information).
  • the preset position relationship between the visual scanning device and the box picking device is also known. Therefore, based on the preset position relationship and the intermediate position information calculated by formula (2), the intermediate position information can be converted into the fourth position information of the target object to be transported relative to the box picking device.
  • the terminal device can send the fourth position information to the control device, and the control device can control the robot arm to pick up the target object to be transported based on the fourth position information.
  • the fourth position information of each object to be transported relative to the box-taking device can be calculated by the above formula (2), so that all the fourth position information is sent to the receiving device. To the control device, so that multiple objects to be transported can be picked up at one time.
  • the method of returning the target object to be transported is similar to the above method, except that when returning the target object to be transported, step 210 obtains the third position information of the cargo position to be returned corresponding to the target object to be transported on the shelf in the second coordinate system, and then determines the fourth position information of the cargo position to be returned relative to the box-taking device based on the third position information and the conversion relationship, and sends the fourth position information to the control device, triggering the control device to control the mechanical arm to return the target object to be transported based on the fourth position information. Moreover, when returning the transported target object using this method, multiple transported targets can be returned at one time.
  • FIG. 13 is a structural example diagram of a target positioning device provided by some embodiments of the present disclosure.
  • the target positioning device 1300 includes: a first acquisition module 1301, a first calculation module 1302, a second acquisition module 1303, a second calculation module 1304 and a determination module 1305.
  • the device may also include other more or fewer units/modules, such as a storage unit, a sending unit, etc.
  • the first acquisition module 1301 is configured to acquire image information of a position to be identified on the shelf collected by the visual scanning device after the transfer robot transports the shelf to the workstation; the position to be identified includes a mark to be identified and/or a target grid, and the mark to be identified includes at least one of a visual identification code set on the shelf, a top corner of the shelf, and an intersection of grids in the shelf, and the number of visual identification codes set on the shelf is less than the number of grids in the shelf.
  • the target grid includes a grid for placing a target object to be transported.
  • the first calculation module 1302 is configured to calculate first position information of the position to be identified in a first coordinate system based on the image information of the position to be identified; wherein the first coordinate system is a coordinate system constructed based on a visual scanning device.
  • the second acquisition module 1303 is configured to acquire second position information of the position to be identified in a second coordinate system; wherein the second coordinate system is a coordinate system constructed based on the shelf.
  • the second calculation module 1304 is configured to calculate a conversion relationship between the first coordinate system and the second coordinate system based on the first position information and the second position information.
  • the determination module 1305 is configured to obtain the third position information of at least one target object to be transported on the shelf in the second coordinate system, determine the fourth position information of the target object to be transported relative to the box picking device based on the third position information and the conversion relationship, and send the fourth position information to the control device, triggering the control device to control the robotic arm to pick up the target object to be transported based on the fourth position information.
  • the mark to be identified includes the four top corners of the front area of the shelf, or the four top corners of the top area of the shelf.
  • the mark to be identified includes the intersection position of the grid openings in the front area of the shelf, or the intersection position of the grid openings in the top area of the shelf.
  • the mark to be identified includes the top corner of the shelf front area and the intersection of the grid in the shelf front area; or, the mark to be identified includes the top corner of the shelf top area and the intersection of the grid in the shelf top area.
  • the number of the markers to be identified is greater than or equal to four, and the positions of the markers to be identified are not on a straight line.
  • the number of visual identification codes is greater than four, and each visual identification code is respectively set at the four top corners of the front area of the shelf and at least one preset target position of the top area, or each visual identification code is respectively set at the four top corners of the top area of the shelf and at least one preset target position of the front area; wherein the preset target positions include top corner positions and center positions.
  • the number of visual identification codes is four, and each visual identification code is respectively set in the front area of the shelf or At the four corners of the top area.
  • the number of visual identification codes is three, and each visual identification code is respectively set at any three top corners of the front area or the top area of the shelf; or, the number of visual identification codes is two, and each visual identification code is respectively set at two top corners located at the diagonal line of the front area or the top area of the shelf; or, the number of visual identification codes is one, and the visual identification code is set at the center position of the front area or the top area of the shelf.
  • the position to be identified is a target grid; the first calculation module 1302 is configured to calculate first position information of a preset target point on the target grid in a first coordinate system based on image information of the target grid.
  • the preset target points are the four vertices of the target grid
  • the first acquisition module 1301 is configured to acquire the two-dimensional image information of the target grid on the shelf collected by the visual scanning device
  • the first calculation module 1302 is configured to filter out the preset target points on the target grid based on the two-dimensional image information of the target grid; and use the pixel position information of the preset target points in the two-dimensional image information as the first position information.
  • the first computing module 1302 is configured to perform image processing on the two-dimensional image information of the target grid using a preset algorithm to extract multiple straight lines from the two-dimensional image information; calculate the intersection between every two straight lines as candidate target points; and filter out preset target points from the candidate target points based on preset size information and shape information of the target grid.
  • the first calculation module 1302 is configured to perform gradient operation processing on the two-dimensional image information of the target grid to generate a gradient map of the target grid; perform Hough transform processing on the gradient map to extract multiple straight lines in the gradient map.
  • the preset target points are the center points corresponding to the four sides of the target grid respectively.
  • the first calculation module 1302 is configured to obtain the three-dimensional image information of the target grid on the shelf collected by the visual scanning device; the first calculation module 1302 is configured to filter out the preset target points on the target grid based on the three-dimensional image information of the target grid; and use the position information of the preset target points relative to the visual scanning device as the first position information.
  • the first computing module 1302 is configured to perform image processing on the three-dimensional image information of the target grid using a preset algorithm, extract four straight lines constituting the target grid from the three-dimensional image information; for each straight line, obtain pixel position information corresponding to a preset number of pixel points located around each straight line; and determine a preset target point based on each pixel position information.
  • the second calculation module 1304 is configured to calculate the first position information and the second position information using a preset algorithm to generate rotation parameters and translation parameters; based on the rotation parameters and translation parameters, determine the conversion relationship between the first coordinate system and the second coordinate system.
  • the above-mentioned determination module 1305 is configured to determine the intermediate position information of the target object to be transported relative to the visual scanning device based on the third position information and the conversion relationship; and determine the fourth position information of the target object to be transported relative to the box picking device based on the intermediate position information and the preset position relationship between the visual scanning device and the box picking device.
  • the second acquisition module 1303 is configured to acquire third position information of a slot on the shelf for placing the target object to be transported in the second coordinate system.
  • the second acquisition module 1303 is configured to construct a second coordinate system with a preset target point on the shelf as the origin; based on preset size information of the grid on the shelf for placing the target object to be transported, calculate the second position information of the visual identification code in the second coordinate system.
  • Some embodiments of the present disclosure further provide an electronic device, which may be the server in the aforementioned embodiment, and is configured to implement all or part of the steps of the aforementioned target positioning method.
  • FIG14 is a schematic diagram of the structure of an electronic device provided in some embodiments of the present disclosure.
  • the electronic device includes: at least one processor, a memory and at least one interface.
  • it may also include a communication bus for connecting the above-mentioned components.
  • At least one processor can be a CPU or a processing chip, which is configured to read and execute the computer program stored in the memory.
  • the memory may be a non-transitory memory, which may include a volatile memory, such as a high-speed random access memory (RAM), or a non-volatile memory, such as at least one disk storage.
  • a volatile memory such as a high-speed random access memory (RAM)
  • a non-volatile memory such as at least one disk storage.
  • At least one interface includes an input/output interface and a communication interface
  • the communication interface can be a wired or wireless interface to achieve communication between the electronic device and other devices.
  • the input/output interface can be used to connect peripherals, such as a display screen, a keyboard, etc.
  • Some embodiments of the present disclosure provide a computer device, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the following steps are implemented:
  • image information of the identification to be identified on the shelf is acquired by the visual scanning device; the identification to be identified includes at least one of the visual identification code set on the shelf, the top corner of the shelf and the intersection position of the grid openings in the shelf, and the number of visual identification codes set on the shelf is less than the number of grid openings in the shelf.
  • first position information of the mark to be identified in a first coordinate system is calculated; wherein the first coordinate system is a coordinate system constructed based on a visual scanning device.
  • the second position information of the mark to be identified in the second coordinate system is obtained; wherein the second coordinate system is a coordinate system constructed based on the shelf.
  • Obtain third position information of at least one target object to be transported on the shelf in the second coordinate system determine fourth position information of the target object to be transported relative to the box picking device based on the third position information and the conversion relationship, and send the fourth position information to the control device, triggering the control device to control the robotic arm to pick up the target object to be transported based on the fourth position information.
  • the mark to be identified includes the four top corners of the front area of the shelf, or the four top corners of the top area of the shelf.
  • the mark to be identified includes the intersection position of the grid openings in the front area of the shelf, or the intersection position of the grid openings in the top area of the shelf.
  • the mark to be identified includes the top corner of the shelf front area and the intersection of the grid of the shelf front area; or, the mark to be identified includes the top corner of the shelf top area and the intersection of the grid of the shelf top area.
  • the number of the markers to be identified is greater than or equal to four, and the positions of the markers to be identified are not on a straight line.
  • the number of visual identification codes may be greater than four, and each visual identification code is respectively set at the four top corners of the front area of the shelf and at least one preset target position of the top area, or each visual identification code is respectively set at the four top corners of the top area of the shelf and at least one preset target position of the front area; wherein the preset target positions include the top corner positions and the center positions.
  • the number of visual identification codes may be four, and each visual identification code is respectively set at the four top corners of the front area or the top area of the shelf.
  • the number of visual identification codes can be three, and each visual identification code is respectively set at any three top corners of the front area or the top area of the shelf; or, the number of visual identification codes can be two, and each visual identification code is respectively set at two top corners located at the diagonal line of the front area or the top area of the shelf; or, the number of visual identification codes is one, and the visual identification code is set at the center position of the front area or the top area of the shelf.
  • the processor when the processor executes the computer program, it also implements the following steps: using a preset algorithm to calculate the first position information and the second position information to generate rotation parameters and translation parameters; based on the rotation parameters and translation parameters, determining the conversion relationship between the first coordinate system and the second coordinate system.
  • the processor when the processor executes the computer program, it also implements the following steps: based on the third position information and the conversion relationship, determining the intermediate position information of the target object to be transported relative to the visual scanning device; based on the intermediate position information and the preset position relationship between the visual scanning device and the box picking device, determining the fourth position information of the target object to be transported relative to the box picking device.
  • the processor when the processor executes the computer program, it also implements the following steps: constructing a second coordinate system with a preset target point on the shelf as the origin; and calculating the second position information of the mark to be identified in the second coordinate system based on the preset size information of the grid on the shelf for placing the target object to be transported.
  • the memory stores computer-readable program instructions.
  • the processor reads and executes the program instructions in the memory, a target positioning method in the aforementioned embodiment can be implemented.
  • some embodiments of the present disclosure further provide a computer program product for storing computer-readable program instructions, which, when executed by a processor, can implement a target positioning method in the aforementioned embodiments.
  • the logic and/or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, which can be embodied in any computer-readable medium for use by an instruction execution system, apparatus or device (such as a computer-based system, a system including a processor or other system that can fetch instructions from an instruction execution system, apparatus or device and execute instructions), or used in combination with these instruction execution systems, apparatuses or devices.
  • a "computer-readable medium” can be any apparatus that can contain, store, communicate, propagate, or transport the program for use by an instruction execution system, apparatus, or device or in conjunction with such instruction execution system, apparatus, or device.
  • Computer-readable media include the following: an electrical connection having one or more wires (electronic device), a portable computer disk case (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disk read-only memory (CDROM).
  • an electrical connection having one or more wires electronic device
  • a portable computer disk case magnetic device
  • RAM random access memory
  • ROM read-only memory
  • EPROM or flash memory erasable and programmable read-only memory
  • CDROM portable compact disk read-only memory
  • the computer readable medium may even be a paper or other suitable medium on which the program can be printed, because the program can be obtained electronically, for example, by optically scanning the paper or other medium, and then editing, interpreting or processing in other suitable ways as necessary, and then stored in the computer memory. It should be understood that various parts of the present disclosure can be implemented in hardware, software, firmware or a combination thereof.
  • a plurality of steps or methods may be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system.
  • a suitable instruction execution system For example, if implemented by hardware, as in another embodiment, it may be implemented by any one of the following technologies known in the art or a combination thereof: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

Landscapes

  • Engineering & Computer Science (AREA)
  • Mechanical Engineering (AREA)

Abstract

一种目标物定位方法、装置、设备和存储介质，通过获取货架上待识别位置的图像信息，并基于该图像信息计算待识别位置的第一位置信息，再通过获取待识别位置的第二位置信息，基于这两种位置信息计算得到通用的位置坐标转换关系，最终基于该转换关系能够确定出所有待搬运目标物相对于取箱装置的第四位置信息，进而基于该第四位置信息拿取到该待搬运目标物。

Description

目标物定位方法、装置、设备和存储介质
本申请要求于2022年12月14日提交的申请号为202211609084.X的中国专利申请的优先权，以及于2023年7月19日提交的申请号为202310889463.7的中国专利申请的优先权，以及于2022年12月9日提交的申请号为202211578417.7的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本公开涉及机器人技术领域，尤其涉及一种目标物定位方法、装置、设备和存储介质。
背景技术
在仓库环境中，对仓库中货架上的货箱进行搬运时，通常是先由移载机器人将放置多个货箱的货架搬运至工作站的预设固定位置处，再由工作站上取箱装置中的机械臂拿取相应的货箱，从而将该货箱搬运至目标位置。然而，由于移载机器人自身的运动控制***异常、引导移载机器人进行移动的地面标识码安装误差等原因，会导致移载机器人将货架搬运放置的实际位置与预设固定位置具有一定偏差，进而使得取箱装置中的机械臂无法拿取到相应的货箱。因而，就需要对货箱的当前位置进行准确定位。
发明内容
本公开实施例提供了一种目标物定位方法、装置、设备和存储介质。
根据本公开一些实施例，提供一种目标物定位方法，应用于终端设备，终端设备分别与控制装置和取箱装置上的视觉扫描设备相连接，控制装置与取箱装置上的机械臂相连接，该方法包括：在移载机器人将货架搬运到工作站后，获取视觉扫描设备采集的货架上待识别位置的图像信息；其中，所述待识别位置包括待识别标识和/或目标格口，待识别标识包括设置在货架上的视觉标识码、货架的顶角和货架中格口的交叉位置中的至少一项，且货架上设置的视觉标识码的数量小于货架中格口的数量；目标格口包括用于放置待搬运目标物的格口；基于待识别位置的图像信息，计算待识别位置在第一坐标系下的第一位置信息；其中，第一坐标系为基于视觉扫描设备所构建的坐标系；获取待识别位置在第二坐标系下的第二位置信息；其中，第二坐标系为基于货架所构建的坐标系；基于第一位置信息及第二位置信息，计算第一坐标系与第二坐标系之间的转换关系；获取货架上至少一个待搬运目标物在第二坐标系下的第三位置信息，基于第三位置信息及转换关系，确定待搬运目标物相对于取箱装置的第四位置信息，并将第四位置信息发送至控制装置，触发控制装置控制机械臂基于第四位置信息拿取待搬运目标物。
在一些实施例中，上述待识别标识的数量大于或等于四个，且各待识别标识的位置不在一条直线上。
在一些实施例中，上述待识别标识包括所述货架正面区域的四个顶角，或者，货架顶部区域的四个顶角。
在一些实施例中，上述待识别标识包括货架正面区域的格口的交叉位置，或者，货架顶部区域的格口的交叉位置。
在一些实施例中，上述待识别标识包括货架正面区域的顶角和货架正面区域的格口的交叉位置；或者，待识别标识包括货架顶部区域的顶角和货架顶部区域格口的交叉位置。
在一些实施例中，上述待识别标识包括视觉标识码，视觉标识码的数量大于四个，各视觉标识码分别设置于货架正面区域的四个顶角处及顶部区域的至少一个预设目标位置处；或，各视觉标识码分别设置于货架顶部区域的四个顶角处及正面区域的至少一个预设目标位置处；其中，预设目标位置包括顶角位置和中心位置。
在一些实施例中，上述待识别标识包括视觉标识码，视觉标识码的数量为四个，各视觉标识码分别设置于货架正面区域或顶部区域的四个顶角处。
在一些实施例中，上述待识别标识包括视觉标识码，视觉标识码的数量为三个，各视觉标识码分别设置于货架正面区域或顶部区域的任意三个顶角处；或者，视觉标识码的数量为两个，各视觉标识码分别设置于货架正面区域或顶部区域的位于对角线处的两个顶角处；或者，视觉标识码的数量为一个，视觉标识码设置于货架正面区域或顶部区域的中心位置处。
在一些实施例中，上述待识别位置为目标格口；上述基于待识别位置的图像信息，计算待识别位置在第一坐标系下的第一位置信息，包括：基于目标格口的图像信息，计算目标格口上预设目标点在第一坐标系下的第一位置信息。
在一些实施例中，预设目标点为目标格口的四个顶点，上述获取视觉扫描设备采集的货架上待识别位置的图像信息，包括：获取视觉扫描设备采集的货架上目标格口的二维图像信息；上述基于目标格口的图像信息，计算目标格口上预设目标点在第一坐标系下的第一位置信息，包括：基于目标格口的二维图像信息，筛选出目标格口上的预设目标点；将预设目标点在二维图像信息中的像素位置信息作为第一位置信息。
在一些实施例中，上述基于目标格口的二维图像信息，筛选出目标格口上的预设目标点，包括：采用预设算法对目标格口的二维图像信息进行图像处理，提取出二维图像信息中的多条直线；计算每两条直线之间的交点，作为候选目标点；基于目标格口的预设尺寸信息及形状信息，从候选目标点中筛选出预设目标点。
在一些实施例中，上述采用预设算法对目标格口的二维图像信息进行图像处理，提取出二维图像信息中的多条直线，包括：对目标格口的二维图像信息进行梯度运算处理，生成目标格口的梯度图；对梯度图进行霍夫变换处理，提取出梯度图中的多条直线。
在一些实施例中，预设目标点为目标格口的四条边分别对应的中心点，上述获取视觉扫描设备采集的货架上待识别位置的图像信息，包括：获取视觉扫描设备采集的货架上目标格口的三维图像信息；上述基于目标格口的图像信息，计算目标格口上预设目标点在第一坐标系下的第一位置信息，包括：基于目标格口的三维图像信息，筛选出目标格口上的预设目标点；将预设目标点相对于视觉扫描设备的位置信息作为第一位置信息。
在一些实施例中，上述基于目标格口的三维图像信息，筛选出目标格口上的预设目标点，包括：采用预设算法对目标格口的三维图像信息进行图像处理，从三维图像信息中提取出组成目标格口的四条直线；针对各条直线，分别获取位于各条直线周围的预设数量个像素点对应的像素位置信息；基于各个像素位置信息，确定预设目标点。
在一些实施例中，上述基于第一位置信息及第二位置信息，计算第一坐标系与第二坐标系之间的转换关系，包括：采用预设算法对第一位置信息及第二位置信息进行计算，生成旋转参数和平移参数；基于旋转参数和平移参数，确定第一坐标系与第二坐标系之间的转换关系。
在一些实施例中，上述基于第三位置信息及转换关系，确定待搬运目标物相对于取箱装 置的第四位置信息，包括：基于第三位置信息及转换关系，确定待搬运目标物相对于视觉扫描设备的中间位置信息；基于中间位置信息、及视觉扫描设备与取箱装置之间的预设位置关系，确定待搬运目标物相对于取箱装置的第四位置信息。
在一些实施例中，上述获取货架上至少一个待搬运目标物在第二坐标系下的第三位置信息，包括：获取货架上用于放置待搬运目标物的格口在第二坐标系下的第三位置信息。
在一些实施例中，上述获取待识别标识在第二坐标系下的第二位置信息，包括：以货架上的预设点为原点构建第二坐标系；基于货架上用于放置待搬运目标物的格口的预设尺寸信息，计算待识别标识在第二坐标系下的第二位置信息。
根据本公开一些实施例，提供了一种目标物定位装置，该装置包括：第一获取模块，被配置为在移载机器人将货架搬运到工作站后，获取视觉扫描设备采集的待识别位置的图像信息；其中，待识别位置包括待识别标识和/或目标格口，待识别标识包括设置在货架上的视觉标识码、货架的顶角和货架中格口的交叉位置中的至少一项，且货架上设置的视觉标识码的数量小于货架中格口的数量；目标格口包括用于放置待搬运目标物的格口；第一计算模块，被配置为基于待识别位置的图像信息，计算待识别位置在第一坐标系下的第一位置信息；其中，第一坐标系为基于视觉扫描设备所构建的坐标系；第二获取模块，被配置为获取待识别位置在第二坐标系下的第二位置信息；其中，第二坐标系为基于货架所构建的坐标系；第二计算模块，被配置为基于第一位置信息及第二位置信息，计算第一坐标系与第二坐标系之间的转换关系；确定模块，被配置为获取货架上至少一个待搬运目标物在第二坐标系下的第三位置信息，基于第三位置信息及转换关系，确定待搬运目标物相对于取箱装置的第四位置信息，并将第四位置信息发送至控制装置，触发控制装置控制机械臂基于第四位置信息拿取待搬运目标物。
根据本公开一些实施例，提供了一种电子设备(计算机设备)，包括：处理器和存储器，存储器，用于存储计算机可执行指令；处理器，用于从存储器中读取指令，并执行指令以实现前述第一方面以及第一方面任一实现方式的方法。
根据本公开一些实施例，提供了一种计算机可读存储介质，该计算机可读存储介质中存储计算机指令，该计算机指令用于使该计算机执行前述第一方面以及第一方面的任一实现方式中的方法。
根据本公开一些实施例，提供了一种计算机程序产品，该计算机程序产品包括存储在计算机可读存储介质上的计算程序，该计算机程序包括程序指令，当该程序指令被计算机执行时，使该计算机执行前述第一方面的任一实现方式中的方法。
本公开实施例提供的目标物定位方法、装置、设备和存储介质，通过识别货架上的待识别位置(如，视觉标识码、货架的顶角和货架中格口的交叉位置、货架上的目标格口)，就能够确定出待搬运目标物相对于取箱装置的位置信息。因此不需要在货架上安装视觉标识码，或者只需在货架上安装较少数量的视觉标识码，就能确定出待搬运目标物相对于取箱装置的位置。由于本公开不需要在货架上的每个格口处都安装一个视觉标识码，因此能够降低视觉标识码的制造成本和安装成本；另外，由于只需要采集一次待识别标识的图像信息，并通过计算得到的转换关系可以一次性确定出货架上所有待搬运目标物的位置信息，提高了对待搬运目标物进行定位的效率。
附图说明
为了更清楚地说明本公开实施例的技术方案，下面将对实施例中所需要使用的附图作简 单地介绍，显而易见地，下面描述中的附图仅仅是本公开的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其它的附图。
图1为根据本公开一些实施例提供的一种目标物定位方法的场景示意图；
图2为根据本公开一些实施例提供的一种目标物定位方法的流程图；
图3A为根据本公开一些实施例提供的一种货架正面区域包括待识别标识的示意图；
图3B为根据本公开一些实施例提供的又一种货架正面区域包括待识别标识的示意图；
图3C为根据本公开一些实施例提供的一种在货架正面区域设置视觉标识码的示意图；
图4A为根据本公开一些实施例提供的一种在货架顶部区域设置视觉标识码的示意图；
图4B为根据本公开一些实施例提供的一种目标格口四个顶点的示意图；
图4C为根据本公开一些实施例提供的一种计算第一位置信息的流程图；
图5为根据本公开一些实施例提供的一种筛选预设目标点的流程图；
图6为根据本公开一些实施例提供的一种提取多条直线的流程图；
图7为根据本公开一些实施例提供的一种目标格口四条边的示意图；
图8为根据本公开一些实施例提供的另一种计算第一位置信息的流程图；
图9为根据本公开一些实施例提供的另一种筛选预设目标点的流程图；
图10为根据本公开一些实施例提供的一种计算第二位置信息的流程图；
图11为根据本公开一些实施例提供的一种计算转换关系的流程图；
图12为根据本公开一些实施例提供的一种确定第四位置信息的流程图；
图13为根据本公开一些实施例提供的一种目标物定位装置的结构示意图；
图14为根据本公开一些实施例提供的一种电子设备的结构示意图。
具体实施方式
为了使本技术领域的人员更好地理解本公开实施例中的技术方案，并使本公开实施例的上述目的、特征和优点能够更加明显易懂，下面结合附图对本公开实施例中的技术方案作进一步详细的说明。
通常，在货架上搬运货箱时，为了对货箱的当前位置进行准确定位，通常会在货架上放置货箱的每个格口处设置一个视觉标识码，通过预先设定好的位置信息先对待搬运目标物进行粗略定位后，再采用取箱装置中的视觉采集设备获取放置该待搬运目标物的格口处视觉标识码的图像信息，进而基于该图像信息计算得到待搬运目标物相对于视觉扫描设备的位置信息。由于视觉扫描设备安装在取箱装置上且两者的相对位置关系已知，从而可以计算得到待搬运目标物相对于取箱装置的准确位置信息，进而，可以基于该位置信息控制机械臂拿取该待搬运目标物。然而，由于需要在每个格口处均设置一个视觉标识码，会增加安装视觉标识码的成本；另外，由于需要采集每个格口处视觉标识码的图像信息后，才能对该货架上所有待搬运目标物进行准确定位，降低了对货箱进行定位的效率。
为此，本公开实施例提出一种目标物定位方法、装置、设备和存储介质，通过识别包括视觉标识码、货架的顶角和货架中格口的交叉位置、货架上的目标格口中的至少一项的待识别位置，就能够确定出待搬运目标物相对于取箱装置的位置信息，因此不需要在货架上安装视觉标识码，或者只需在货架上安装较少数量的视觉标识码，就能确定出待搬运目标物相对于取箱装置的位置。由于不需要在货架上的每个格口处都安装一个视觉标识码，因此能够降低视觉标识码的制造成本和安装成本；另外，由于只需要采集一次待识别标识的图像信息，并通过计算得到的转换关系可以一次性确定出货架上所有待搬运目标物(如货箱)的位置信 息，提高了对待搬运目标物进行定位的效率。
本公开实施例提供的目标物定位方法应用于终端设备，图1为根据本公开一些实施例提供的一种目标物定位方法的场景示意图，如图1所示，终端设备102分别与控制装置104和取箱装置106上的视觉扫描设备1061相连接，控制装置104与取箱装置106上的机械臂1062相连接。
终端设备102中部署了上层业务***，终端设备102被配置为执行本公开下述实施例提供的目标物定位方法。
视觉扫描设备1061可以为相机，该相机可以为2D相机，也可以为3D相机。视觉扫描设备1061被配置为对货架上的待识别位置(如视觉标识码、货架顶角、货架格***叉位置或目标格口)进行扫描识别，采集到待识别位置的图像信息。
控制装置104可以包括可编程逻辑控制器(Programmable Logic Controller，PLC)。控制装置104被配置为接收终端设备102发送的待搬运目标物相对于取箱装置的位置信息，并基于该位置信息控制机械臂1062在货架上拿取待搬运目标物。机械臂1062可以为吸盘式的机械臂，也可以为其他类型的机械臂，本公开实施例对此不作限定。
以下结合附图对本公开实施例提供的技术方案进行详细说明。图2为根据本公开一些实施例提供的一种目标物定位方法的流程图，该方法包括以下步骤：
步骤202、在移载机器人将货架搬运到工作站后，获取视觉扫描设备采集的货架上待识别位置的图像信息。
其中，待识别位置包括待识别标识和/或目标格口，待识别标识包括设置在货架上的视觉标识码、货架的顶角和货架中格口的交叉位置中的至少一项，且当待识别标识包括设置在货架上的视觉标识码时，该货架上设置的视觉标识码的数量小于货架中格口的数量。
仓库***中通常会放置多个货架，每个货架上设有多个格口，每个格口用于放置货箱、原箱、料箱、容器、货物等。
在一些实施例中，无需在货架上设置视觉标识码，可以将货架的顶角和/或货架中格口的交叉位置作为待识别位置，也可以将目标格口作为待识别位置。该目标格口包括用于放置待搬运目标物的格口。
在一些实施例中，可以在货架上设置数量小于货架中格口数量的视觉标识码，将视觉标识码作为货架上的待识别标识。在另一些实施例中，可以将视觉标识码和货架顶角作为货架上的待识别标识。或者，也可以将视觉标识码和货架中格***叉位置作为货架上的待识别标识。或者，还可以将视觉标识码、货架顶角和货架中格***叉位置共同作为货架上的待识别标识。
示例性地，待识别标识的数量可以大于或等于四个，且各待识别标识的位置不在一条直线上。待识别标识的数量越多，确定的待搬运目标物的位置越准确，本公开实施例对于待识别标识的数量不作限定。
示例性地，在待识别位置为待识别标识的情况下，待识别标识可以包括货架顶角。当待识别标识仅包括货架顶角时，该货架顶角的数量大于或等于四个。例如，待识别标识包括货架正面区域的四个顶角，或者，货架顶部区域的四个顶角。再例如，待识别标识还可以包括货架侧面区域的四个顶角。当待识别标识除包括货架顶角外，还包括货架中格***叉位置和/或视觉标识码时，该货架顶角的数量可以小于四个。本公开实施例对于待识别标识包括的货架顶角的位置和数量不作限定，待识别标识包括的货架顶角的位置与视觉扫描设备的安装位置有关。
例如，图3A为本公开一些实施例提供的一种货架正面区域包括待识别标识的示意图，如图3A所示，图3A中的灰色圆点表示货架正面区域的四个顶角，货架上的待识别标识可以为该货架正面区域的四个顶角。
示例性地，待识别标识包括货架中格口的交叉位置。当待识别标识仅包括货架中格口的交叉位置时，该货架格***叉位置的数量大于或等于四个。例如，待识别标识包括至少四个位于货架正面区域的格口的交叉位置，或者，至少四个位于货架顶部区域的格口的交叉位置。再例如，待识别标识还可以包括至少四个位于货架侧面区域的格口的交叉位置。当待识别标识除包括格口的交叉位置外，还包括货架顶角和/或视觉标识码时，该货架格口的交叉位置的数量可以小于四个。本公开实施例对于待识别标识包括的格口的交叉位置的位置和数量不作限定，待识别标识包括的格口的交叉位置与视觉扫描设备的安装位置有关。
例如，图3B为本公开一些实施例提供的又一种货架正面区域包括待识别标识的示意图，如图3B所示，货架包括格口A至格口L，每个格口的交叉点为一个待识别标识，图3B中的灰色圆点均表示待识别标识。
示例性地，待识别标识可以既包括货架顶角，又包括货架中格口的交叉位置。例如，待识别标识包括货架正面区域的顶角和货架正面区域的格口的交叉位置。再例如，待识别标识包括货架顶部区域的顶角和货架顶部区域的格口的交叉位置。
示例性地，待识别标识也可以为视觉标识码。该视觉标识码是用于进行定位的标识码，通常可以采用Apriltag码，Apriltag码作为一种视觉定位标志符，其类似于二维码或者条形码。当然也可以采用ArUco码，本公开实施例对此不作限定。采用Apriltag码和ArUco码进行定位的原理为现有技术，在此不再赘述。
在待识别标识包括视觉标识码时，本公开实施例中货架上所安装的视觉标识码的数量小于货架中格口的数量，因此不需要在货架的每个格口处都安装一个视觉标识码。图3C为本公开实施例提供的一种在货架正面区域设置视觉标识码的示意图，如图3C所示，货架包括格口A至格口L，每个格口下面的灰色圆点表示与该格口对应的视觉标识码，即需要在每个格口处都会设置一个视觉标识码。而本公开实施例中货架上设置的视觉标识码的数量小于货架中格口的数量，本公开实施例对于货架上设置的视觉标识码的位置不作限定。
在一些实施例中，视觉标识码的数量可以大于四个且小于货架中格口的数量，各视觉标识码分别设置于货架正面区域的四个顶角处及顶部区域的至少一个预设目标位置处，或，各视觉标识码分别设置于货架顶部区域的四个顶角处及正面区域的至少一个预设目标位置处。
其中，预设目标位置包括顶角位置和中心位置，以各视觉标识码分别设置于货架正面区域的四个顶角处及顶部区域的四个顶角处为例进行说明。
由于既可以通过正面区域四个顶角处的视觉标识码确定出货架的位置信息，也可以通过顶部区域四个顶角处的视觉标识码确定出货架的位置信息，接着可以将确定出来的两个位置信息进行比较，例如，可以作差比较，若差值小于预设阈值，则说明所确定出来的货架的位置信息是准确的；若差值大于预设阈值，则说明所确定出来的货架的位置信息不准确，此时可以重新检查视觉标识码的安装位置等是否出现偏差，从而可以在降低视觉标识码的制造成本和安装成本外，还可以提高对货箱定位的准确性。
在一些实施例中，视觉标识码的数量可以为四个，各视觉标识码可以分别设置于货架正面区域或顶部区域的四个顶角处。请继续参考图3C，可以看出，在该货架的四个顶角处的圆点分别表示一个视觉标识码，四个视觉标识码可以分别标识为TAG1、TAG2、TAG3、TAG4。
图4A为本公开一些实施例提供的一种在货架顶部区域设置视觉标识码的示意图，如图 4A所示，货架上的四个视觉标识码采用位于四个顶角处的小矩形框表示。通过采用四个视觉标识码就可以实现对货架上所有待搬运目标物的定位，大大降低了视觉标识码的制造成本和安装成本，并且也降低了后期对视觉标识码的维护成本。本公开实施例对于货架上设置的视觉标识码的位置和数量不作限定，该视觉标识码的设置位置与视觉扫描设备的安装位置有关，视觉标识码的设置数量小于货架中格口的数量即可。
在另一些实施例中，视觉标识码的数量可以为三个，各视觉标识码分别设置于货架正面区域或顶部区域的任意三个顶角处，由于通过三个视觉标识码依然可以确定出整个货架所在的面，从而就可以确定出货架的位置信息。
在另一些实施例中，视觉标识码的数量可以为两个，各视觉标识码分别设置于货架正面区域或顶部区域的位于对角线处的两个顶角处。视觉标识码的数量也可以为一个，该视觉标识码设置于货架正面区域或顶部区域的中心位置处。结合两个视觉标识码或一个视觉标识码，以及货架尺寸信息，能够确定出货架的位置信息，另外，还进一步降低了视觉标识码的制造成本和安装成本。
在待识别位置为目标格口的情况下，该目标格口可以为货架上的任一格口，也可以为货架上用于放置待搬运目标物的格口。待搬运目标物可以是货箱，也可以是原箱、料箱、容器、货物本身等目标物，当然也可以是货架中的其他目标物，本公开实施例对此不作限定。
例如，可以将货架上的任一格口作为待识别位置，以确定待搬运目标物相对于取箱装置的位置；也可以将用于放置待搬运目标物的格口作为待识别位置，以确定待搬运目标物相对于取箱装置的位置，本公开实施例对此不作限定。由于取箱装置在货架上拿取待搬运目标物时，会移动至放置待搬运目标物的格口，因此可以通过视觉扫描设备采集放置待搬运目标物的格口(如目标格口)的图像信息，以确定待搬运目标物相对于取箱装置的位置。
在一些示例中，由于放置目标物的货架可能发生形变，或因制作工艺导致货架上格口的尺寸与预设尺寸存在偏差，为了进一步提升待搬运目标物的定位准确度，可以将用于放置待搬运目标物的格口作为待识别位置以确定待搬运目标物相对于取箱装置的位置。
示例性地，视觉扫描设备可以为相机，该相机可以为2D相机，也可以为3D相机，视觉扫描设备用于对货架上的待识别标识和/或目标格口进行扫描识别，采集到待识别位置的图像信息。在移载机器人将货架搬运到工作站后，可以通过向视觉扫描设备发送控制指令，以控制视觉扫描设备采集到待识别位置的图像信息。需要说明的是，采集到的待识别位置的图像信息中还可以包含该待识别位置周围物体的图像信息。
需要说明的是，待识别标识为视觉标识码时，获取待搬运目标物相对于取箱装置的位置信息的实现方式，与待识别标识为货架的顶角和/或货架中格口的交叉位置时，获取待搬运目标物相对于取箱装置的位置信息的实现方式类似。下述实施例分别以待识别位置包括视觉标识码和目标格口为例对获取待搬运目标物相对于取箱装置的位置信息的实现方式进行示例性说明。
步骤204、基于待识别位置的图像信息，计算待识别位置在第一坐标系下的第一位置信息。
其中，第一坐标系为基于视觉扫描设备所构建的坐标系。
示例性地，以待识别位置包括视觉标识码为例，请继续参考图3C，四个视觉标识码在第一坐标系下的位置信息可以分别表示为：PTAG1Camera＝(X1,Y1,Z1)、PTAG2Camera＝(X2,Y2,Z2)、PTAG3Camera＝(X3,Y3,Z3)、PTAG4Camera＝(X4,Y4,Z4)。由于待识别位置包括货架的顶角和/或货架中格口的交叉位置时，获取货架的顶角和/或货架中格口的交叉位置在第一坐标系下的第一位 置信息的方式与获取视觉标识码在第一坐标系下的位置信息类似，在此不再赘述。
示例性地，以待识别位置包括目标格口为例，上述步骤204包括：基于目标格口的图像信息，计算目标格口上预设目标点在第一坐标系下的第一位置信息。
在一些实施例中，预设目标点可以为目标格口的四个顶点。图4B为根据本公开一些实施例提供的一种目标格口四个顶点的示意图，如图4B所示，以目标格口为格口E为例，目标格口E的四个顶点分别为E1、E2、E3、E4，可以将E1、E2、E3、E4作为预设目标点。若视觉扫描设备为2D相机，可以获取由2D相机采集的包括货架上目标格口的四个定点的二维图像信息。
图4C为根据本公开一些实施例提供的一种计算第一位置信息的流程图，如图4C所示，该方法包括以下步骤：
步骤402、基于目标格口的二维图像信息，筛选出目标格口上的预设目标点。
其中，目标格口的二维图像信息中除包括格口的图像信息外，还可能包括其他物体的图像信息，因此需要根据该二维图像信息筛选出目标格口的四个顶点。
图5为根据本公开一些实施例提供的一种筛选预设目标点的流程图，如图5所示，该方法包括以下步骤：
步骤502、采用预设算法对目标格口的二维图像信息进行图像处理，提取出二维图像信息中的多条直线。
其中，预设算法可以包括梯度运算和霍夫变换。梯度运算用于检测图像的边缘信息，得到图像的梯度图。霍夫变换用于从生成的梯度图中提取出多条直线。当然从梯度图中提取直线时，也可以采用其他算法，例如最小二乘法、射线法等算法，本公开实施例对此不作限定。
图6为根据本公开一些实施例提供的一种提取多条直线的流程图，如图6所示，该方法包括以下步骤：
步骤602、对目标格口的二维图像信息进行梯度运算处理，生成目标格口的梯度图。
步骤604、对梯度图进行霍夫变换处理，提取出梯度图中的多条直线。
其中，图像梯度代表图像灰度值变化的速度，对于一幅图像而言，图像的边缘部分两侧灰度值相差较大，梯度值大，因此对一幅图像进行梯度运算就可以突出图像边缘的信息。由于像素值为离散值，所以其梯度其实就是差分运算，其本质上也就是一种空间滤波。通过对目标格口的二维图像信息进行梯度运算处理，从而可以突出该二维图像边缘的信息，即可生成目标格口的梯度图。
霍夫变换是一种特征提取的运算过程，被广泛应用在图像分析、计算机视觉及数位影像处理。霍夫变换是用来辨别找出物件中的特征，例如线条。算法流程大致为：可以先给定一个物件、要辨别的形状的种类，该算法会在参数空间中执行投票来决定物体的形状，而这是由累加空间里的局部最大值来决定。通过对梯度图进行霍夫变换处理，提取出梯度图中的多条直线。并且，采用霍夫变换处理提取直线的准确性和效率均较高，从而提高了筛选预设目标点的准确性和效率。
步骤504、计算每两条直线之间的交点，作为候选目标点。
步骤506、基于目标格口的预设尺寸信息及形状信息，从候选目标点中筛选出预设目标点。
其中，在提取处的多条直线中，若两条直线相交，那么就可以计算得到这两条直线之间的交点。在计算出每两条直线之间的交点后，可以将计算得到的所有交点作为候选目标点。
候选目标点的连线会形成多个封闭区域，由于目标格口通常为矩形，并且其尺寸信息也 是已知的，从而目标格口对应的封闭区域的面积也是确定的。进而就可以通过目标格口的预设尺寸信息及形状信息，先确定出目标格口对应的封闭区域，即为目标格口对应的矩形区域，从而将组成该矩形区域的四个顶点作为预设目标点，即实现了从候选目标点中筛选出预设目标点。
步骤404、将预设目标点在二维图像信息中的像素位置信息作为第一位置信息。
例如，如图4B所示，目标格口的四个顶点E1、E2、E3、E4在二维图像信息中的像素位置信息可以分别表示为：PE1Pixel＝(u1,v1)、PE2Pixel＝(u2,v2)、PE3Pixel＝(u3,v3)、PE4Pixel＝(u4,v4)，即得到预设目标点的第一位置信息。
在另一些实施例中，预设目标点可以为目标格口的四条边分别对应的中心点。图7为根据本公开一些实施例提供的一种目标格口四条边的示意图，如图7所示，以目标格口为格口E为例，目标格口E为四条直线围成的矩形，即目标格口E包含四条边。若视觉扫描设备为3D相机，可以获取视觉扫描设备采集的货架上目标格口的三维图像信息。
图8为根据本公开一些实施例提供的另一种计算第一位置信息的流程图，如图8所示，该方法包括以下步骤：
步骤802、基于目标格口的三维图像信息，筛选出目标格口上的预设目标点。
步骤804、将预设目标点相对于视觉扫描设备的位置信息作为第一位置信息。
其中，目标格口的三维图像信息中可以包含除了格口以外其他物体的图像信息，从而需要根据该三维图像信息筛选出目标格口的四条边分别对应的中心点。
图9为根据本公开一些实施例提供的另一种筛选预设目标点的流程图，如图9所示，该方法包括以下步骤：
步骤902、采用预设算法对目标格口的三维图像信息进行图像处理，从三维图像信息中提取出组成目标格口的四条直线。
示例性地，预设算法可以采用梯度运算和霍夫变换，从三维图像信息中提取组成目标格口的四条直线的实现方式可以参考上述从目标格口的二维图像信息中提取多条直线的过程，在此不再赘述。
步骤904、针对各条直线，分别获取位于各条直线周围的预设数量个像素点对应的像素位置信息。
步骤906、基于各个像素位置信息，确定预设目标点。
其中，针对各条直线，可以分别获取位于各条直线周围的预设数量个像素点对应的像素位置信息。例如，可以获取各条直线周围的五个像素点的像素位置信息。从而可以将这五个像素位置信息求平均后，将该平均值作为与该条直线对应的一个中心点的像素位置信息，基于此，就可以求出目标格口的四条边分别对应的中心点，即得到四个中心点。进而可以直接将预设目标点相对于视觉扫描设备的位置信息作为上述第一位置信息。
步骤206、获取待识别位置在第二坐标系下的第二位置信息。
其中，第二坐标系为基于货架所构建的坐标系。
示例性地，以待识别位置包括视觉标识码为例，请继续参考图3C，图3C中的坐标系XYZ即为第二坐标系。
图10为本公开实施例提供的一种计算第二位置信息的流程图，如图10所示，该方法包括以下步骤：
步骤1002、以货架上的预设点为原点构建第二坐标系。
步骤1004、基于货架上用于放置待搬运目标物的格口的预设尺寸信息，计算待识别位置 在第二坐标系下的第二位置信息。
其中，预设点可以为货架中任意一预先确定的位置，如图3C所示，预设点可以为货架四个顶角中处于左上角位置的顶点，从而可以将该顶点作为第二坐标系的原点，并且以水平方向建立X轴，垂直方向建立Y轴，Z轴分别与X轴和Y轴垂直，从而构建出第二坐标系。
其中，在图3C中，每个格口的预设尺寸信息中可以用N来表示格口在水平方向上的尺寸，M表示格口的在垂直方向上的尺寸，在Z轴方向上的尺寸可以统一设置为0。以待识别标识为设置在货架正面区域的四个顶角处的视觉标识码为例，四个视觉标识码在第二坐标系下的第二位置信息可以分别表示为：PTAG1Shelf＝(0,0,0)、PTAG2Shelf＝(3N,0,0)、PTAG3Shelf＝(0,4M,0)、PTAG4Shelf＝(3N,4M,0)。
在一些实施例中，以待识别标识包括货架中格口的交叉位置，该格***叉位置为货架正面区域的格***叉位置为例，为了确定各待识别标识在第二坐标系下的第二位置信息，上述视觉扫描设备采集的待识别标识的图像信息可以包括货架正面区域的所有格口。例如，视觉扫描设备可以为鱼眼相机，在移载机器人将货架搬运到工作站后，鱼眼相机采集包括货架正面区域的所有格口的图像信息，通过该图像信息可以确定待识别标识中各格***叉位置在第二坐标系下的第二位置信息。再例如，在移载机器人将货架搬运到工作站后，视觉扫描设备采集货架上部分格口(比如，货架正面区域的上半部分格口、下半部分格口、左半部分格口或右半部分格口；再比如，货架顶部区域的上半部分格口、下半部分格口、左半部分格口或右半部分格口)的图像信息，通过该图像信息可以确定待识别标识中各格***叉位置在第二坐标系下的第二位置信息。又例如，视觉扫描设备也可以在移载机器人将货架搬运到工作站的过程中，采集包括待识别标识的多张图像，通过该多张图像可以确定待识别标识中各格***叉位置在第二坐标系下的第二位置信息。
示例性地，以待识别位置包括目标格口为例，第二坐标系也可以为基于货架上的目标格口所构建的坐标系。请继续参考图4B，图4B中的坐标系XY即为第二坐标系。其中，在构建第二坐标系时，该第二坐标系的原点可以为目标格口中任意一预先确定的位置。如图4B所示，可以以目标格口E的顶点E1为原点构建第二坐标系，并且以水平方向建立X轴，垂直方向建立Y轴，在Z方向上的尺寸可以设置为0，因此未在图4B中标注Z轴。
在图4B中，每个格口的预设尺寸信息中可以用N来表示格口在水平方向上的尺寸，M表示格口的在垂直方向上的尺寸。从而目标格口E的四个顶点在第二坐标系下的第二位置信息可以分别表示为：PE1Cell＝(0,0,0)、PE2Cell＝(N,0,0)、PE3Cell＝(0,M,0)、PE4Cell＝(N,M,0)。
步骤208、基于第一位置信息及第二位置信息，计算第一坐标系与第二坐标系之间的转换关系。
图11为根据本公开一些实施例提供的一种计算转换关系的流程图，如图11所示，该方法包括以下步骤：
步骤1102、采用预设算法对第一位置信息及第二位置信息进行计算，生成旋转参数和平移参数。
步骤1104、基于旋转参数和平移参数，确定第一坐标系与第二坐标系之间的转换关系。
示例性地，预设算法可以为PnP(Perspective-n-Point)算法，该算法是一种求解3D到2D点的对应方法，其描述了当知道n个3D空间点及其位置，如何估计相机的位姿。通过下述公式(1)，将四组第一位置信息和四组第二位置信息代入该公式后，采用PnP算法就可以计算得到旋转参数R和平移参数t，该求解过程为现有技术，在此不再赘述。从而就可以基于该旋转参数R和平移参数t确定出第一坐标系与第二坐标系之间的转换关系。
PTAGCamera＝PTAGShelfR+t         (1)
其中，PTAGCamera表示待识别位置的第一位置信息；PTAGShelf表示待识别位置的第二位置信息；R表示旋转参数；t表示平移参数。
步骤210、获取货架上至少一个待搬运目标物在第二坐标系下的第三位置信息，基于第三位置信息及转换关系，确定待搬运目标物相对于取箱装置的第四位置信息，并将第四位置信息发送至控制装置，触发控制装置控制机械臂基于第四位置信息拿取待搬运目标物。
其中，待搬运目标物在第二坐标系下的第三位置信息的确定方法请参考上述确定第二位置信息的过程。以图3C中格口E处放置的待搬运目标物为例，该待搬运目标物的第三位置信息可以表示为
示例性地，在待识别位置为目标格口，且目标格口为放置待搬运目标物的格口的情况下，上述获取货架上至少一个待搬运目标物在第二坐标系下的第三位置信息，包括：获取货架上目标格口在第二坐标系下的第三位置信息。在一些示例中，目标格口在第二坐标系下的第三位置信息可以与目标格口上预设目标点的第二位置信息相同，也可以与目标格口上预设目标点的第二位置信息不同。
例如，以图4B中目标格口为格口E，且目标格口E为放置待搬运目标物的格口为例，该目标格口E的第三位置信息可以为E0在第二坐标系下的位置信息，表示为也可以为E1、E2、E3或E4在第二坐标系下的位置信息。
示例性地，在待识别位置为目标格口，且目标格口与放置待搬运目标物的格口为不同格口的情况下，通过步骤210能够获取待搬运目标物在第二坐标系下的第三位置信息，该待搬运目标物在第二坐标系下的第三位置信息可以为放置该待搬运目标物的格口上任一位置在第二坐标系下的第三位置信息。
图12为本公开实施一些例提供的一种确定第四位置信息的流程图，如图12所示，该方法包括以下步骤：
步骤1202、基于第三位置信息及转换关系，确定待搬运目标物相对于视觉扫描设备的中间位置信息。
步骤1204、基于中间位置信息、及视觉扫描设备与取箱装置之间的预设位置关系，确定待搬运目标物相对于取箱装置的第四位置信息。
其中，继续以图3C中格口E处放置的待搬运目标物为例，基于第三位置信息及转换关系，通过下述公式(2)，可以计算得到该待搬运目标物相对于视觉扫描设备位置信息，即为中间位置信息。
PECamera＝PEShelfR+t          (2)
其中，PECamera表示格口E处放置的待搬运目标物在第一坐标系下的位置信息(如，中间位置信息)，PEShelf表示格口E处放置的待搬运目标物在第二坐标系下的位置信息(如，第三位置信息)。
由于视觉扫描设备是人工安装在取箱装置上的，那么视觉扫描设备与取箱装置之间的预设位置关系也是已知的，从而就可以基于预设位置关系，以及通过公式(2)计算得到的中间位置信息，将中间位置信息转换为待搬运目标物相对于取箱装置的第四位置信息。
在计算得到待搬运目标物相对于取箱装置的第四位置信息后，终端设备可以将第四位置信息发送至控制装置，控制装置基于第四位置信息可以控制机械臂拿取待搬运目标物。
另外，需要说明的是，在一次性需要拿取多个待搬运目标物时，均可以通过上述公式(2)计算得到各待搬运目标物相对于取箱装置的第四位置信息，从而将所有第四位置信息均发送 至控制装置，这样就可以实现一次性需要拿取多个待搬运目标物。
在一些实施例中，归还待搬运目标物(如货箱)的方法与上述方法类似，区别在于归还待搬运目标物时步骤210获取的是该待搬运目标物在货架上对应的待归还货位在第二坐标系下的第三位置信息，然后基于该第三位置信息及转换关系，确定待归还货位相对于取箱装置的第四位置信息，并将第四位置信息发送至控制装置，触发控制装置控制机械臂基于第四位置信息归还待搬运目标物。而且采用该方法归还搬运目标物时可以一次性归还多个搬运目标物。
下面介绍与前述方法实施例相对应的装置实施例。
本公开一些实施例还提供一种目标物定位装置，被配置为执行前述实施例中的目标物定位方法。图13为本公开一些实施例提供的一种目标物定位装置的结构示例图，如图13所示，该目标物定位装置1300包括：第一获取模块1301、第一计算模块1302、第二获取模块1303、第二计算模块1304和确定模块1305。此外，该装置还可以包括其他更多或更少的单元/模块，比如存储单元、发送单元等。
第一获取模块1301，被配置为在移载机器人将货架搬运到工作站后，获取视觉扫描设备采集的货架上待识别位置的图像信息；待识别位置包括待识别标识和/或目标格口，待识别标识包括设置在货架上的视觉标识码、货架的顶角和货架中格口的交叉位置中的至少一项，且货架上设置的视觉标识码的数量小于货架中格口的数量。目标格口包括用于放置待搬运目标物的格口。
第一计算模块1302，被配置为基于待识别位置的图像信息，计算待识别位置在第一坐标系下的第一位置信息；其中，第一坐标系为基于视觉扫描设备所构建的坐标系。
第二获取模块1303，被配置为获取待识别位置在第二坐标系下的第二位置信息；其中，第二坐标系为基于货架所构建的坐标系。
第二计算模块1304，被配置为基于第一位置信息及第二位置信息，计算第一坐标系与第二坐标系之间的转换关系。
确定模块1305，被配置为获取货架上至少一个待搬运目标物在第二坐标系下的第三位置信息，基于第三位置信息及转换关系，确定待搬运目标物相对于取箱装置的第四位置信息，并将第四位置信息发送至控制装置，触发控制装置控制机械臂基于第四位置信息拿取待搬运目标物。
在一些实施例中，待识别标识包括货架正面区域的四个顶角，或者，货架顶部区域的四个顶角。
在一些实施例中，待识别标识包括货架正面区域的格口的交叉位置，或者，货架顶部区域的格口的交叉位置。
在一些实施例中，待识别标识包括货架正面区域的顶角和货架正面区域的格口的交叉位置；或者，待识别标识包括货架顶部区域的顶角和货架顶部区域的格口的交叉位置。
在一些实施例中，待识别标识的数量大于或等于四个，且各待识别标识的位置不在一条直线上。
在一些实施例中，视觉标识码的数量大于四个，各视觉标识码分别设置于货架正面区域的四个顶角处及顶部区域的至少一个预设目标位置处，或，各视觉标识码分别设置于货架顶部区域的四个顶角处及正面区域的至少一个预设目标位置处；其中，预设目标位置包括顶角位置和中心位置。
在一些实施例中，视觉标识码的数量为四个，各视觉标识码分别设置于货架正面区域或 顶部区域的四个顶角处。
在一些实施例中，视觉标识码的数量为三个，各视觉标识码分别设置于货架正面区域或顶部区域的任意三个顶角处；或者，视觉标识码的数量为两个，各视觉标识码分别设置于货架正面区域或顶部区域的位于对角线处的两个顶角处；或者，视觉标识码的数量为一个，该视觉标识码设置于货架正面区域或顶部区域的中心位置处。
在一些实施例中，待识别位置为目标格口；第一计算模块1302，被配置为基于目标格口的图像信息，计算目标格口上预设目标点在第一坐标系下的第一位置信息。
示例性地，预设目标点为目标格口的四个顶点，第一获取模块1301被配置为获取视觉扫描设备采集的货架上目标格口的二维图像信息；第一计算模块1302，被配置为基于目标格口的二维图像信息，筛选出目标格口上的预设目标点；将预设目标点在二维图像信息中的像素位置信息作为第一位置信息。
在一些示例中，第一计算模块1302，被配置为采用预设算法对目标格口的二维图像信息进行图像处理，提取出二维图像信息中的多条直线；计算每两条直线之间的交点，作为候选目标点；基于目标格口的预设尺寸信息及形状信息，从候选目标点中筛选出预设目标点。
第一计算模块1302，被配置为对目标格口的二维图像信息进行梯度运算处理，生成目标格口的梯度图；对梯度图进行霍夫变换处理，提取出梯度图中的多条直线。
在一些实施例中，预设目标点为目标格口的四条边分别对应的中心点，第一计算模块1302，被配置为获取视觉扫描设备采集的货架上目标格口的三维图像信息；第一计算模块1302，被配置为基于目标格口的三维图像信息，筛选出目标格口上的预设目标点；将预设目标点相对于视觉扫描设备的位置信息作为第一位置信息。
示例性地，第一计算模块1302，被配置为采用预设算法对目标格口的三维图像信息进行图像处理，从三维图像信息中提取出组成目标格口的四条直线；针对各条直线，分别获取位于各条直线周围的预设数量个像素点对应的像素位置信息；基于各个像素位置信息，确定预设目标点。
示例性地，上述第二计算模块1304被配置为采用预设算法对第一位置信息及第二位置信息进行计算，生成旋转参数和平移参数；基于旋转参数和平移参数，确定第一坐标系与第二坐标系之间的转换关系。
示例性地，上述确定模块1305被配置为基于第三位置信息及转换关系，确定待搬运目标物相对于视觉扫描设备的中间位置信息；基于中间位置信息、及视觉扫描设备与取箱装置之间的预设位置关系，确定待搬运目标物相对于取箱装置的第四位置信息。
示例性地，第二获取模块1303被配置为获取货架上用于放置待搬运目标物的格口在第二坐标系下的第三位置信息。
示例性地，上述第二获取模块1303被配置为以货架上的预设目标点为原点构建第二坐标系；基于货架上用于放置待搬运目标物的格口的预设尺寸信息，计算视觉标识码在第二坐标系下的第二位置信息。
本公开一些实施例还提供了一种电子设备，该电子设备可以是前述实施例中的服务器，被配置为实现前述目标物定位方法步骤中的全部或部分。
图14为本公开一些实施例提供的一种电子设备的结构示意图，如图14所示，该电子设备包括：至少一个处理器、存储器和至少一个接口，此外，还可以包括通信总线，用于连接上述这些部件。
其中，至少一个处理器可以是CPU或处理芯片，被配置为读取并执行存储器中存储的计 算机程序指令，以使至少一个处理器能够执行前述各个实施例中的方法流程。
存储器可以为非暂态存储器(non-transitory memory)，其可以包含易失性存储器，例如高速随机存取存储器(Random Access Memory，RAM)，也可以包括非易失性存储器(non-volatile memory)，例如至少一个磁盘存储器。
至少一个接口包括输入输出接口，以及通信接口，通信接口可以是有线或者无线接口，从而实现电子设备与其他设备之间的通信连接。输入输出接口可以用于连接外设，比如显示屏、键盘等。
本公开一些实施例，提供了一种计算机设备，包括存储器和处理器，存储器中存储有计算机程序，该处理器执行计算机程序时实现以下步骤：
在移载机器人将货架搬运到工作站后，获取视觉扫描设备采集的货架上待识别标识的图像信息；待识别标识包括设置在货架上的视觉标识码、货架的顶角和货架中格口的交叉位置中的至少一项，且货架上设置的视觉标识码的数量小于货架中格口的数量。
基于待识别标识的图像信息，计算待识别标识在第一坐标系下的第一位置信息；其中，第一坐标系为基于视觉扫描设备所构建的坐标系。
获取待识别标识在第二坐标系下的第二位置信息；其中，第二坐标系为基于货架所构建的坐标系。
基于第一位置信息及第二位置信息，计算第一坐标系与第二坐标系之间的转换关系。
获取货架上至少一个待搬运目标物在第二坐标系下的第三位置信息，基于第三位置信息及转换关系，确定待搬运目标物相对于取箱装置的第四位置信息，并将第四位置信息发送至控制装置，触发控制装置控制机械臂基于第四位置信息拿取待搬运目标物。
在一些实施例中，待识别标识包括货架正面区域的四个顶角，或者，货架顶部区域的四个顶角。
在一些实施例中，待识别标识包括货架正面区域格口的交叉位置，或者，货架顶部区域格口的交叉位置。
在一些实施例中，待识别标识包括货架正面区域的顶角和货架正面区域格口的交叉位置；或者，待识别标识包括货架顶部区域的顶角和货架顶部区域格口的交叉位置。
在一些实施例中，待识别标识的数量大于或等于四个，且各待识别标识的位置不在一条直线上。
在一些实施例中，视觉标识码的数量可以大于四个，各视觉标识码分别设置于货架正面区域的四个顶角处及顶部区域的至少一个预设目标位置处，或，各视觉标识码分别设置于货架顶部区域的四个顶角处及正面区域的至少一个预设目标位置处；其中，预设目标位置包括顶角位置和中心位置。
在一些实施例中，视觉标识码的数量可以为四个，各视觉标识码分别设置于货架正面区域或顶部区域的四个顶角处。
在一些实施例中，视觉标识码的数量可以为三个，各视觉标识码分别设置于货架正面区域或顶部区域的任意三个顶角处；或者，视觉标识码的数量可以为两个，各视觉标识码分别设置于货架正面区域或顶部区域的位于对角线处的两个顶角处；或者，视觉标识码的数量为一个，该视觉标识码设置于货架正面区域或顶部区域的中心位置处。
在一些实施例中，处理器执行计算机程序时还实现以下步骤：采用预设算法对第一位置信息及第二位置信息进行计算，生成旋转参数和平移参数；基于旋转参数和平移参数，确定第一坐标系与第二坐标系之间的转换关系。
在一些实施例中，处理器执行计算机程序时还实现以下步骤：基于第三位置信息及转换关系，确定待搬运目标物相对于视觉扫描设备的中间位置信息；基于中间位置信息、及视觉扫描设备与取箱装置之间的预设位置关系，确定待搬运目标物相对于取箱装置的第四位置信息。
在一些实施例中，处理器执行计算机程序时还实现以下步骤：以货架上的预设目标点为原点构建第二坐标系；基于货架上用于放置待搬运目标物的格口的预设尺寸信息，计算待识别标识在第二坐标系下的第二位置信息。
本公开一些实施例提供的计算机设备，其实现原理和技术效果与上述方法实施例类似，在此不再赘述。
在一些实施方式中，存储器存储了计算机可读程序指令，当处理器读取并执行该存储器中的程序指令时，可实现前述实施例中的一种目标物定位方法。
此外，本公开一些实施例还提供了一种计算机程序产品，用于存储计算机可读程序指令，该指令被处理器执行时，可实现前述实施例中的一种目标物定位方法。
本说明书中的各个实施例均采用相关的方式描述，各个实施例之间相同相似的部分互相参见即可，每个实施例重点说明的都是与其他实施例的不同之处。尤其，对于装置实施例而言，由于其基本相似于方法实施例，所以描述的比较简单，相关之处参见方法实施例的部分说明即可。
在流程图中表示或在此以其他方式描述的逻辑和/或步骤，例如，可以被认为是用于实现逻辑功能的可执行指令的定序列表，可以具体实现在任何计算机可读介质中，以供指令执行***、装置或设备(如基于计算机的***、包括处理器的***或其他可以从指令执行***、装置或设备取指令并执行指令的***)使用，或结合这些指令执行***、装置或设备而使用。
就本说明书而言，″计算机可读介质″可以是任何可以包含、存储、通信、传播或传输程序以供指令执行***、装置或设备或结合这些指令执行***、装置或设备而使用的装置。
计算机可读介质的更具体的示例(非穷尽性列表)包括以下：具有一个或多个布线的电连接部(电子装置)，便携式计算机盘盒(磁装置)，随机存取存储器(RAM)，只读存储器(ROM)，可擦除可编辑只读存储器(EPROM或闪速存储器)，光纤装置，以及便携式光盘只读存储器(CDROM)。
另外，计算机可读介质甚至可以是可在其上打印程序的纸或其他合适的介质，因为可以例如通过对纸或其他介质进行光学扫描，接着进行编辑、解译或必要时以其他合适方式进行处理来以电子方式获得程序，然后将其存储在计算机存储器中。应当理解，本公开的各部分可以用硬件、软件、固件或它们的组合来实现。
在上述实施方式中，多个步骤或方法可以用存储在存储器中且由合适的指令执行***执行的软件或固件来实现。例如，如果用硬件来实现，和在另一实施方式中一样，可用本领域公知的下列技术中的任一项或他们的组合来实现：具有用于对数据信号实现逻辑功能的逻辑门电路的离散逻辑电路，具有合适的组合逻辑门电路的专用集成电路，可编程门阵列(PGA)，现场可编程门阵列(FPGA)等。
以上的本公开实施方式并不构成对本公开保护范围的限定。

Claims (21)

  1. 一种目标物定位方法，应用于终端设备，所述终端设备分别与控制装置和取箱装置上的视觉扫描设备相连接，所述控制装置与所述取箱装置上的机械臂相连接，包括：
    在移载机器人将货架搬运到工作站后，获取所述视觉扫描设备采集的所述货架上待识别位置的图像信息；其中，所述待识别位置包括待识别标识和/或目标格口，所述待识别标识包括设置在所述货架上的视觉标识码、所述货架的顶角和所述货架中格口的交叉位置中的至少一项，且所述货架上设置的视觉标识码的数量小于所述货架中格口的数量；所述目标格口包括用于放置待搬运目标物的格口；
    基于所述待识别位置的图像信息，计算所述待识别位置在第一坐标系下的第一位置信息；其中，所述第一坐标系为基于所述视觉扫描设备所构建的坐标系；
    获取所述待识别位置在第二坐标系下的第二位置信息；其中，所述第二坐标系为基于所述货架所构建的坐标系；
    基于所述第一位置信息及所述第二位置信息，计算所述第一坐标系与所述第二坐标系之间的转换关系；
    获取所述货架上至少一个待搬运目标物在所述第二坐标系下的第三位置信息，基于所述第三位置信息及所述转换关系，确定所述待搬运目标物相对于所述取箱装置的第四位置信息，并将所述第四位置信息发送至所述控制装置，触发所述控制装置控制所述机械臂基于所述第四位置信息拿取所述待搬运目标物。
  2. 根据权利要求1所述的目标物定位方法，其中，所述待识别标识的数量大于或等于四个，且各所述待识别标识的位置不在一条直线上。
  3. 根据权利要求1所述的目标物定位方法，其中，所述待识别标识包括所述货架正面区域的四个顶角，或者，所述货架顶部区域的四个顶角。
  4. 根据权利要求1所述的目标物定位方法，其中，所述待识别标识包括所述货架正面区域的格口的交叉位置，或者，所述货架顶部区域的格口的交叉位置。
  5. 根据权利要求1所述的目标物定位方法，其中，所述待识别标识包括所述货架正面区域的顶角和所述货架正面区域的格口的交叉位置；或者，所述待识别标识包括所述货架顶部区域的顶角和所述货架顶部区域的格口的交叉位置。
  6. 根据权利要求1所述的目标物定位方法，其中，所述待识别标识包括设置在所述货架上的视觉标识码，所述视觉标识码的数量大于四个，各所述视觉标识码分别设置于所述货架正面区域的四个顶角处及顶部区域的至少一个预设目标位置处；或，各所述视觉标识码分别设置于所述货架顶部区域的四个顶角处及所述正面区域的至少一个预设目标位置处；其中，所述预设目标位置包括顶角位置和中心位置。
  7. 根据权利要求1所述的目标物定位方法，其中，所述待识别标识包括设置在所述货架上的视觉标识码，所述视觉标识码的数量为四个，各所述视觉标识码分别设置于所述货架正面区域或顶部区域的四个顶角处。
  8. 根据权利要求1所述的目标物定位方法，其中，所述待识别标识包括设置在所述货架上的视觉标识码，所述视觉标识码的数量为三个，各所述视觉标识码分别设置于所述货架正面区域或顶部区域的任意三个顶角处；或者，
    所述视觉标识码的数量为两个，各所述视觉标识码分别设置于所述货架正面区域或顶部区域的位于对角线处的两个顶角处；或者，
    所述视觉标识码的数量为一个，所述视觉标识码设置于所述货架正面区域或顶部区域的中心位置处。
  9. 根据权利要求1所述的目标物定位方法，其中，所述待识别位置为所述目标格口；所述基于所述待识别位置的图像信息，计算所述待识别位置在第一坐标系下的第一位置信息，包括：
    基于所述目标格口的图像信息，计算所述目标格口上预设目标点在第一坐标系下的第一位置信息。
  10. 根据权利要求9所述的目标物定位方法，其中，所述预设目标点为所述目标格口的四个顶点，所述获取所述视觉扫描设备采集的所述货架上待识别位置的图像信息，包括：
    获取所述视觉扫描设备采集的所述货架上所述目标格口的二维图像信息；
    所述基于所述目标格口的图像信息，计算所述目标格口上预设目标点在第一坐标系下的第一位置信息，包括：
    基于所述目标格口的二维图像信息，筛选出所述目标格口上的所述预设目标点；
    将所述预设目标点在所述二维图像信息中的像素位置信息作为所述第一位置信息。
  11. 根据权利要求10所述的目标物定位方法，其中，所述基于所述目标格口的二维图像信息，筛选出所述目标格口上的所述预设目标点，包括：
    采用预设算法对所述目标格口的二维图像信息进行图像处理，提取出所述二维图像信息中的多条直线；
    计算每两条所述直线之间的交点，作为候选目标点；
    基于所述目标格口的预设尺寸信息及形状信息，从所述候选目标点中筛选出所述预设目标点。
  12. 根据权利要求11所述的目标物定位方法，其中，所述采用预设算法对所述目标格口的二维图像信息进行图像处理，提取出所述二维图像信息中的多条直线，包括：
    对所述目标格口的二维图像信息进行梯度运算处理，生成所述目标格口的梯度图；
    对所述梯度图进行霍夫变换处理，提取出所述梯度图中的多条直线。
  13. 根据权利要求9所述的目标物定位方法，其中，所述预设目标点为所述目标格口的四条边分别对应的中心点，所述获取所述视觉扫描设备采集的所述货架上待识别位置的图像信息，包括：
    获取所述视觉扫描设备采集的所述货架上所述目标格口的三维图像信息；
    所述基于所述目标格口的图像信息，计算所述目标格口上预设目标点在第一坐标系下的第一位置信息，包括：
    基于所述目标格口的三维图像信息，筛选出所述目标格口上的所述预设目标点；
    将所述预设目标点相对于视觉扫描设备的位置信息作为所述第一位置信息。
  14. 根据权利要求13所述的目标物定位方法，其中，所述基于所述目标格口的三维图像信息，筛选出所述目标格口上的所述预设目标点，包括：
    采用预设算法对所述目标格口的三维图像信息进行图像处理，从所述三维图像信息中提取出组成所述目标格口的四条直线；
    针对各条所述直线，分别获取位于各条所述直线周围的预设数量个像素点对应的像素位置信息；
    基于各个所述像素位置信息，确定所述预设目标点。
  15. 根据权利要求1-14中任一项所述的目标物定位方法，其中，所述基于所述第一位置信 息及所述第二位置信息，计算所述第一坐标系与所述第二坐标系之间的转换关系，包括：
    采用预设算法对所述第一位置信息及所述第二位置信息进行计算，生成旋转参数和平移参数；
    基于所述旋转参数和所述平移参数，确定所述第一坐标系与所述第二坐标系之间的转换关系。
  16. 根据权利要求1-14中任一项所述的目标物定位方法，其中，所述基于所述第三位置信息及所述转换关系，确定所述待搬运目标物相对于所述取箱装置的第四位置信息，包括：
    基于所述第三位置信息及所述转换关系，确定所述待搬运目标物相对于所述视觉扫描设备的中间位置信息；
    基于所述中间位置信息、及所述视觉扫描设备与所述取箱装置之间的预设位置关系，确定各所述待搬运目标物相对于所述取箱装置的所述第四位置信息。
  17. 根据权利要求16所述的目标物定位方法，其中，所述获取所述货架上至少一个待搬运目标物在所述第二坐标系下的第三位置信息，包括：
    获取所述货架上用于放置所述待搬运目标物的格口在所述第二坐标系下的第三位置信息。
  18. 根据权利要求1-14中任一项所述的目标物定位方法，其中，所述获取所述待识别位置在第二坐标系下的第二位置信息，包括：
    以所述货架上的预设点为原点构建所述第二坐标系；
    基于所述货架上用于放置所述待搬运目标物的格口的预设尺寸信息，计算所述待识别位置在所述第二坐标系下的所述第二位置信息。
  19. 一种目标物定位装置，包括：
    第一获取模块，被配置为在移载机器人将货架搬运到工作站后，获取视觉扫描设备采集的所述货架上待识别位置的图像信息；其中，所述待识别位置包括待识别标识和/或目标格口，所述待识别标识包括设置在所述货架上的视觉标识码、所述货架的顶角和所述货架中格口的交叉位置中的至少一项，且所述货架上设置的视觉标识码的数量小于所述货架中格口的数量，所述目标格口包括用于放置待搬运目标物的格口；
    第一计算模块，被配置为基于所述待识别位置的图像信息，计算所述待识别位置在第一坐标系下的第一位置信息；其中，所述第一坐标系为基于所述视觉扫描设备所构建的坐标系；
    第二获取模块，被配置为获取所述待识别位置在第二坐标系下的第二位置信息；其中，所述第二坐标系为基于所述货架所构建的坐标系；
    第二计算模块，被配置为基于所述第一位置信息及所述第二位置信息，计算所述第一坐标系与所述第二坐标系之间的转换关系；
    确定模块，被配置为获取所述货架上至少一个待搬运目标物在所述第二坐标系下的第三位置信息，基于所述第三位置信息及所述转换关系，确定所述待搬运目标物相对于取箱装置的第四位置信息，并将所述第四位置信息发送至控制装置，触发所述控制装置控制机械臂基于所述第四位置信息拿取所述待搬运目标物。
  20. 一种电子设备，包括：处理器和存储器；
    所述存储器，用于存储计算机可执行指令；
    所述处理器，用于从所述存储器中读取所述指令，并执行所述指令以实现如权利要求1至18中任一所述的方法。
  21. 一种计算机可读存储介质，所述存储介质存储有计算机程序指令，当计算机读取所述指令时，执行如权利要求1至18中任一所述的方法。
PCT/CN2023/132785 2022-12-09 2023-11-21 目标物定位方法、装置、设备和存储介质 WO2024120172A1 (zh)

Applications Claiming Priority (6)

Application Number Priority Date Filing Date Title
CN202211578417.7 2022-12-09
CN202211578417.7A CN118172526A (zh) 2022-12-09 2022-12-09 一种目标定位方法、装置、设备和存储介质
CN202211609084 2022-12-14
CN202211609084.X 2022-12-14
CN202310889463.7A CN116902467A (zh) 2022-12-14 2023-07-19 一种货箱定位方法、装置、设备和存储介质
CN202310889463.7 2023-07-19

Publications (1)

Publication Number Publication Date
WO2024120172A1 true WO2024120172A1 (zh) 2024-06-13

Family

ID=91378562

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2023/132785 WO2024120172A1 (zh) 2022-12-09 2023-11-21 目标物定位方法、装置、设备和存储介质

Country Status (1)

Country Link
WO (1) WO2024120172A1 (zh)

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106556341A (zh) * 2016-10-08 2017-04-05 浙江国自机器人技术有限公司 一种基于特征信息图形的货架位姿偏差检测方法和***
US20200042932A1 (en) * 2017-04-12 2020-02-06 Hitachi Transport System, Ltd. Article management support apparatus, article management support system, and article management support method
CN111178317A (zh) * 2020-01-06 2020-05-19 广东博智林机器人有限公司 检测定位方法、***、装置、电子设备及存储介质
WO2021249572A1 (zh) * 2020-06-12 2021-12-16 深圳市海柔创新科技有限公司 取货控制的方法、***、搬运机器人及存储介质
CN113936010A (zh) * 2021-10-15 2022-01-14 北京极智嘉科技股份有限公司 货架定位方法、装置、货架搬运设备及存储介质
CN114065881A (zh) * 2020-07-29 2022-02-18 阿里巴巴集团控股有限公司 关系建立、对象识别及提示方法、装置、设备和存储介质
CN114820781A (zh) * 2022-03-31 2022-07-29 深圳市如本科技有限公司 基于机器视觉的智能搬运方法、装置、***及存储介质
CN116902467A (zh) * 2022-12-14 2023-10-20 北京极智嘉科技股份有限公司 一种货箱定位方法、装置、设备和存储介质

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106556341A (zh) * 2016-10-08 2017-04-05 浙江国自机器人技术有限公司 一种基于特征信息图形的货架位姿偏差检测方法和***
US20200042932A1 (en) * 2017-04-12 2020-02-06 Hitachi Transport System, Ltd. Article management support apparatus, article management support system, and article management support method
CN111178317A (zh) * 2020-01-06 2020-05-19 广东博智林机器人有限公司 检测定位方法、***、装置、电子设备及存储介质
WO2021249572A1 (zh) * 2020-06-12 2021-12-16 深圳市海柔创新科技有限公司 取货控制的方法、***、搬运机器人及存储介质
CN114065881A (zh) * 2020-07-29 2022-02-18 阿里巴巴集团控股有限公司 关系建立、对象识别及提示方法、装置、设备和存储介质
CN113936010A (zh) * 2021-10-15 2022-01-14 北京极智嘉科技股份有限公司 货架定位方法、装置、货架搬运设备及存储介质
CN114820781A (zh) * 2022-03-31 2022-07-29 深圳市如本科技有限公司 基于机器视觉的智能搬运方法、装置、***及存储介质
CN116902467A (zh) * 2022-12-14 2023-10-20 北京极智嘉科技股份有限公司 一种货箱定位方法、装置、设备和存储介质

Similar Documents

Publication Publication Date Title
KR102460038B1 (ko) 창고 보관 맵을 신속하게 결정하는 방법, 기기, 저장 매체 및 로봇
JP5469216B2 (ja) バラ積みされた物品をロボットで取出す装置
CN111260289A (zh) 一种基于视觉导航的微型无人机仓库盘点***及方法
EP3961149A1 (en) Device, method, and program for detecting position and orientation of object
CN111881892B (zh) 有序点云5d纹理网格数据结构生成方法、装置、设备及介质
KR20160003776A (ko) 자세 추정 방법 및 로봇
CN111199562A (zh) 用于在机器人拣选时快速物体检测的***和方法
CN109213202B (zh) 基于光学伺服的货物摆放方法、装置、设备和存储介质
JP5780083B2 (ja) 検品装置、検品システム、検品方法及びプログラム
CN112002015B (zh) 无序点云生成网格数据结构的方法、装置、设备及介质
AU2021288667A1 (en) Control method and apparatus for warehouse robot, and robot and warehouse system
CN113269801A (zh) 用于处理候选边缘的方法和计算***
CN116902467A (zh) 一种货箱定位方法、装置、设备和存储介质
CN114757878A (zh) 焊接示教方法、装置、终端设备及计算机可读存储介质
WO2024120172A1 (zh) 目标物定位方法、装置、设备和存储介质
CN112197708B (zh) 测量方法及装置、电子设备及存储介质
WO2024012463A1 (zh) 一种定位方法及装置
CN112102391A (zh) 测量方法及装置、电子设备及存储介质
CN116309882A (zh) 一种面向无人叉车应用的托盘检测和定位方法及***
CN111860035A (zh) 图书封面检测方法、装置及存储介质和电子设备
CN113759346B (zh) 一种激光雷达标定方法、装置、电子设备及存储介质
US11823414B2 (en) Information processing device, information processing method, and information processing non-transitory computer readable medium
CN112288038B (zh) 基于图像分类的物体识别或物体注册的方法及计算***
CN112734783B (zh) 用于处理候选边缘的方法和计算***
CN114155291A (zh) 箱体的位姿识别方法及装置、终端、存储介质