CN112884848B - Intelligent crown block control system, method and terminal based on binocular vision - Google Patents
Intelligent crown block control system, method and terminal based on binocular vision Download PDFInfo
- Publication number
- CN112884848B CN112884848B CN202110243051.7A CN202110243051A CN112884848B CN 112884848 B CN112884848 B CN 112884848B CN 202110243051 A CN202110243051 A CN 202110243051A CN 112884848 B CN112884848 B CN 112884848B
- Authority
- CN
- China
- Prior art keywords
- max
- crown block
- image processing
- target
- overhead traveling
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 22
- 238000012545 processing Methods 0.000 claims abstract description 70
- 238000004422 calculation algorithm Methods 0.000 claims abstract description 57
- 238000004891 communication Methods 0.000 claims abstract description 13
- 239000000284 extract Substances 0.000 claims abstract description 8
- 238000005516 engineering process Methods 0.000 claims description 19
- 230000002093 peripheral effect Effects 0.000 claims description 8
- 238000001914 filtration Methods 0.000 claims description 5
- 238000007781 pre-processing Methods 0.000 claims description 5
- 238000004590 computer program Methods 0.000 claims description 4
- 238000004364 calculation method Methods 0.000 claims description 3
- 238000011161 development Methods 0.000 claims description 3
- 238000013519 translation Methods 0.000 claims description 2
- 229910000831 Steel Inorganic materials 0.000 description 7
- 239000010959 steel Substances 0.000 description 7
- 238000010586 diagram Methods 0.000 description 5
- 230000000694 effects Effects 0.000 description 5
- 238000000605 extraction Methods 0.000 description 3
- 238000005259 measurement Methods 0.000 description 3
- 230000007547 defect Effects 0.000 description 2
- 238000009434 installation Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 229910000838 Al alloy Inorganic materials 0.000 description 1
- 230000006978 adaptation Effects 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 238000002156 mixing Methods 0.000 description 1
- 230000008092 positive effect Effects 0.000 description 1
- 230000002035 prolonged effect Effects 0.000 description 1
- 238000012800 visualization Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
- G06T7/85—Stereo camera calibration
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/70—Denoising; Smoothing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/13—Edge detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/181—Segmentation; Edge detection involving edge growing; involving edge linking
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10004—Still image; Photographic image
- G06T2207/10012—Stereo images
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30244—Camera pose
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Multimedia (AREA)
- Length Measuring Devices By Optical Means (AREA)
- Control And Safety Of Cranes (AREA)
Abstract
The invention discloses an intelligent crown block control system, method and terminal based on binocular vision, and relates to the technical field of intelligent crown block control. The binocular camera is arranged on a support of the crown block structure, continuously shoots a target warehousing area, generates a photo stream with a unit of two photos, and simultaneously transmits the photo stream to the computer; the image processing module is in two-way communication with the binocular camera, receives the photo stream transmitted by the binocular camera in real time through the computer, inputs the photo stream into the image processing algorithm program, extracts the three-dimensional information of the target warehousing object in the photo through the algorithm program, and transmits the three-dimensional information to the crown block power driving module. Compared with the traditional method of acquiring position information by using naked eye observation and a laser range finder, the method introduces a binocular vision camera and an image processing algorithm to acquire the position information of the target warehousing object, and transmits the position information to the crown block system to realize the loading and unloading of the target warehousing object by the intelligent crown block.
Description
Technical Field
The invention belongs to the technical field of intelligent crown block control, and particularly relates to an intelligent crown block control system, method and information data processing terminal based on a binocular vision three-dimensional positioning technology.
Background
At present, a laser scanner is generally adopted for realizing intelligent control and unloading of industrial practical application of an overhead travelling crane control system, the laser scanner is used for assisting the overhead travelling crane in three-dimensional positioning of a target warehousing object, and the laser scanner is higher in price at present, so that a larger burden in cost is brought to the overhead travelling crane for realizing intelligent loading and unloading work; in addition, the current laser scanner needs to move in the horizontal direction when scanning long objects, so that the positioning time of target warehousing objects is prolonged, and the loading and unloading efficiency of the overhead travelling crane is reduced in a phase-changing manner. In addition, the laser scanner and the rotary table are high in precision when the laser device is adopted, so that the laser measuring system is high in cost and long in debugging period, and the operation efficiency of a factory is lowered in a phase-changing manner. Therefore, it is important to introduce a cheap and convenient-to-debug mechanical vision technology into the intelligent loading and unloading of the overhead travelling crane.
Currently, a mechanical vision system is also introduced into an intelligent overhead travelling crane system to assist the overhead travelling crane in loading and unloading target objects, but the application of the mechanical vision system still remains in a more basic level. For example, CN111243010A discloses an intelligent crown block image recognition steel coil vehicle loading system and method, which adopts the combined application of a single camera and a laser range finder, and is used for measuring three-dimensional information of a steel coil loaded on a vehicle. The disadvantages are that: the system only uses mechanical vision to measure the width information of the steel coil, and can acquire the three-dimensional information of the steel coil only by combining the laser range finders, has strict requirements on the parking position of a loading vehicle in a warehouse, is narrow in application aspect, and is only applied to acquiring the position information of the steel coil.
Through the above analysis, the problems and defects of the prior art are as follows: the overhead traveling crane control system realizes that intelligent control unloads and generally adopts laser scanner, realizes the location through laser scanner, nevertheless becomes laser measurement system with high costs, and the debugging cycle length is longer to the time that the part has the warehouse entry object laser scanner location of special length to be spent, is unfavorable for overhead traveling crane control system intelligent loading and unloading work's universal realization.
The significance for solving the problems and the defects is as follows: the intelligent crown block control system based on the binocular vision technology can relatively accurately acquire the three-dimensional information of the target cargo by introducing the binocular vision technology capable of well acquiring the depth information when the cargo is put in storage, so that the crown block can accurately grab the target object. Meanwhile, the binocular vision system is low in cost, convenient to debug and stable in positioning time, and can be suitable for three-dimensional information detection of various target objects. The intelligent loading and unloading work of the crown block control system can be widely realized.
Disclosure of Invention
In order to solve the problems in the prior art, the invention provides an intelligent crown block control system based on a binocular vision three-dimensional positioning technology.
The invention is realized in this way, an intelligent crown block control system based on binocular vision three-dimensional positioning technology is provided with:
a binocular camera mounted on the support of the crown block structure for continuously shooting the target storage region to generate a photo stream with two photos, and simultaneously transmitting the photo stream to a computer
The image processing module is in two-way communication with the binocular camera, receives the photo stream transmitted by the binocular camera in real time through the computer, inputs the photo stream into an image processing algorithm program, extracts the three-dimensional information of the target warehousing object in the photo through the algorithm program and transmits the three-dimensional information to the crown block power driving module;
the crane mechanical module consists of a transverse moving mechanical trolley and a longitudinal moving mechanical trolley, the two mechanical trolleys are respectively provided with four motors for providing power, in addition, a lifting motor for realizing loading and unloading work is arranged on the longitudinal moving trolley, the nine motors are connected with a driver in the crane power driving module, and the driver provides a driving signal;
and the overhead traveling crane power driving module is in one-way communication with the image processing module to control the mechanical structure of the overhead traveling crane, and the PLC controls the driver to drive the overhead traveling crane mechanical module to be right above the target warehousing object according to the received three-dimensional information.
In one embodiment, the binocular camera is composed of a pair of monocular cameras which know internal parameters of the cameras (parameters related to the characteristics of the cameras themselves, such as focal length, pixel size and the like) and external parameters obtained by stereo calibration (translation position and rotation direction between the two cameras), are installed to ensure that baselines of the group of binocular cameras are on the same horizontal line as much as possible, and the taken photos are transmitted to the image processing module.
In one embodiment, the image processing module comprises a computer carrying an image processing algorithm and a communication line;
the computer is responsible for receiving a group of pictures transmitted by the binocular camera, operating an image processing algorithm at the same time, and transmitting a coordinate information result to the crown block power driving module after the algorithm is operated; the image processing algorithm is written by C + + and simultaneously utilizes an open source OPENCV development library for improving the running efficiency of the algorithm and reducing the writing and debugging cost of the algorithm.
In one embodiment, the overhead traveling crane mechanical module comprises an overhead traveling crane mechanical structure and a motor installed on the mechanical structure, a control signal of the motor is provided by an overhead traveling crane driving module, the control signal provided by the overhead traveling crane driving module consists of a forward and reverse rotation signal and a rotating speed signal, and the motor realizes transverse and longitudinal movement of the overhead traveling crane mechanical structure and loading and unloading of target warehousing objects according to the received control signal.
In one embodiment, the crown block power driving module consists of a PLC, a frequency converter and an encoder arranged on a motor;
the PLC receives the central point coordinate information transmitted by the image processing module, analyzes and calculates the distance that the crown block needs to travel in the horizontal and vertical directions from the current coordinate position to the central point coordinate position of the target warehousing object, and then transmits the running time and frequency parameters to the frequency converter;
the frequency converter controls the motor to drive the crown block mechanical module to a specified place given by the PLC;
and the encoder feeds back the actual travelling distance of the motor to the PLC, the PLC recalculates the error distance between the encoder and the target coordinate according to the fed-back information, and transmits the parameters to the frequency converter again until the crown block moves to within the acceptable error range.
In one embodiment, the image processing algorithm processes the target-binned object as follows:
step one, setting a target warehousing area as an interest area, and then judging whether warehousing objects appear in the interest area or not; if yes, entering the next step and stopping transmitting the photo stream; if not, repeating the first step;
preprocessing two pictures of the target warehousing objects, extracting peripheral outlines, and filtering out image noise and other useless image parts;
And thirdly, operating an algorithm for extracting mutually matched feature points on the two images on the extracted peripheral contour, and then calculating the coordinate information of the target feature points by using the mathematical principle of binocular vision according to the extracted feature points.
In one embodiment, in step two, the image processing algorithm extracts a plurality of groups of random feature points in two photographs of the target contour, puts coordinate information of the feature points into a set, compares the coordinate information one by one to obtain a maximum value and a minimum value of coordinates on a horizontal coordinate X axis, a vertical coordinate Y axis and a depth coordinate Z axis respectively, and sets the maximum value and the minimum value of coordinates as three groups of coordinates X axis, Y axis and Z axis respectivelymax，Xmin；Ymax，Ymin；Zmax，Zmin；
Then, the image processing algorithm is operated once again to obtain a new group of data, and the new group of data is set as Xmax′、Xmin′、Ymax′、Ymin′、Zmax′、Zmin′；
Mixing Xmax' and Xmax、XminAnd Xmin' by contrast, if Xmax' greater than XmaxThen take Xmax', discard Xmax(ii) a If Xmin' less than XminThen take Xmin', discard Xmin；
Will Ymax' and Ymax、YminAnd Ymin' sequential comparison, if Ymax' greater than YmaxThen take Ymax', discard Ymax(ii) a If Y ismin' less than YminThen take Ymin', discard Ymin；
Will Zmax' and Zmax、ZminAnd Zmin' inBy way of comparison, if Zmax' greater than ZmaxThen take Zmax', discard Zmax(ii) a If Z is min' less than ZminThen take Zmin', discard Zmin。
In one embodiment, new data X will be obtainedmax″，Xmin″；Ymax″，Ymin″；Zmax″，ZminSubstituting the central point coordinate X into the formula for calculating the central point of the target warehousing object to obtain the central point coordinate X calculated by the algorithmcen，Ycen，Zcen；
The central point calculation formula is:
after the coordinates of the central point of the object in the target storage are obtained, three groups of data of an X axis, a Y axis and a Z axis and the coordinates of the central point are displayed on a computer terminal, and meanwhile, the coordinate information of the central point is transmitted to a crown block power driving module.
Another object of the present invention is to provide a method for implementing the system for controlling an intelligent overhead traveling crane based on the binocular vision three-dimensional positioning technology, wherein the method for controlling an intelligent overhead traveling crane based on the binocular vision three-dimensional positioning technology comprises the following steps:
step one, a computer controls a binocular camera to continuously shoot a warehousing area to form a photo stream, and the photo stream is transmitted to the computer;
step two, the computer inputs the transmitted photo stream into an image processing algorithm, and the algorithm judges the warehousing area in the photo stream to judge whether warehousing objects appear or not; if no warehousing object appears, returning to the judging step; if the warehousing object appears, stopping transmitting the photo stream, and simultaneously calculating the coordinate information of the central point of the target warehousing object according to the two photos; then transmitting the coordinate information to an overhead traveling crane system;
Thirdly, analyzing and calculating the distance of the central point coordinate position of the object to be warehoused, which is moved from the current coordinate position of the crown block, in the horizontal and vertical directions by a PLC in the crown block system according to the central point coordinate information, and then transmitting the running time and frequency parameters to a frequency converter;
and fourthly, the frequency converter controls the motor to drive the crane mechanical module to a specified place given by the PLC, meanwhile, an encoder on the motor feeds back the actual travelling distance of the motor to the PLC, the PLC calculates the error distance between the crane mechanical module and the target coordinate according to the fed-back information, and transmits the parameters to the frequency converter again until the crane moves within the acceptable error range.
Another object of the present invention is to provide an information data processing terminal including a memory and a processor, the memory storing a computer program, the computer program, when executed by the processor, causing the processor to execute the intelligent crown block control method based on the binocular vision three-dimensional positioning technology.
By combining all the technical schemes, the invention has the advantages and positive effects that:
firstly, different from the traditional method of acquiring the position information by using a laser range finder, the invention introduces a binocular vision camera and an image processing algorithm to acquire the position information of the target warehousing object, and transmits the position information to an overhead traveling crane system to realize the loading and unloading of the target warehousing object by an intelligent overhead traveling crane.
Secondly, the intelligent overhead traveling crane acquires the three-dimensional information of the target warehousing object through a binocular vision system, so that the loading and unloading efficiency of the intelligent overhead traveling crane is greatly improved, and the application cost of a three-dimensional positioning system is also reduced; the problem of higher application cost caused by the adoption of a laser range finder is avoided; the image processing algorithm fully utilizes the computing advantages of a computer system to carry out complex online computation, and realizes the extraction and transmission of the real-time three-dimensional information of the target warehousing object.
Thirdly, the invention can realize more reliable three-dimensional information extraction aiming at target warehousing objects with different shapes, and reduces the influence factors caused by the shapes of the warehousing objects to the minimum, thereby expanding the general range of three-dimensional positioning of the intelligent overhead travelling crane to the maximum. As shown in fig. 7 to 9, feature points can be extracted from a plurality of shapes of the warehousing object, thereby realizing three-dimensional positioning.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and together with the description, serve to explain the principles of the disclosure.
Fig. 1 is a flowchart of an image processing algorithm provided in an embodiment of the present invention.
FIG. 2 is a mathematical schematic of binocular vision provided by an embodiment of the present invention;
in the figure, the similar principle of the triangle is shown as follows: z/f-x/xl z/f-x-b-xr z/f-y/yl-y/yr.
The simplified P point coordinate is: z is f × b/(xl-xr)
x＝xl×z/f
y＝yl×z/f
Where xl, yl, xr, yr are known as the pixel coordinates of the point P of the left camera and the right camera, respectively.
Fig. 3 is a block diagram of the overall structure of the system according to the embodiment of the present invention.
Fig. 4 is a schematic view of a mechanical installation of the system provided by the embodiment of the present invention.
Fig. 5 is a flowchart of the operation of the system according to the embodiment of the present invention.
Fig. 6 is a diagram of the effect of matched feature points obtained by using a binocular camera according to an embodiment of the present invention.
Fig. 7 is an effect diagram of feature points extracted from an object put in storage according to an embodiment of the present invention.
Fig. 8 is an effect diagram of feature points extracted from another warehousing object provided by the embodiment of the invention.
Fig. 9 is an effect diagram of feature points extracted from another warehousing object according to the embodiment of the present invention.
Detailed Description
In order to make the aforementioned objects, features and advantages of the present invention comprehensible, embodiments accompanied with figures are described in detail below. In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present invention. This invention may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein, but rather should be construed as broadly as the present invention is capable of modification in various respects, all without departing from the spirit and scope of the present invention.
It will be understood that when an element is referred to as being "secured to" another element, it can be directly on the other element or intervening elements may also be present. When an element is referred to as being "connected" to another element, it can be directly connected to the other element or intervening elements may also be present. As used herein, the terms "vertical," "horizontal," "left," "right," and the like are for purposes of illustration only and are not intended to represent the only embodiments.
Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. The terminology used in the description of the invention herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. As used herein, the term "and/or" includes any and all combinations of one or more of the associated listed items.
The invention provides an intelligent crown block control system based on a binocular vision three-dimensional positioning technology. The system shoots the outline picture of the object in storage when goods are in storage, accurately acquires the three-dimensional information of the target object in storage through an image processing algorithm, then quickly transmits the three-dimensional position information to the intelligent overhead traveling crane system, and drives the overhead traveling crane to accurately grab the target object in storage and convey the target object to a storage area.
In order to realize the purpose, the invention adopts the following technical scheme:
the basic idea of the invention is to obtain the three-dimensional position information of a target object by using a binocular vision camera to obtain the contour of the target object and combining an image processing algorithm, and transmit the position information to an overhead traveling crane system to assist the overhead traveling crane in grabbing the target object.
The invention aims to provide an intelligent crown block control system based on a binocular vision three-dimensional positioning technology, which is used for acquiring three-dimensional information of a target warehousing object in real time through the binocular vision positioning technology in an actual factory and transmitting the three-dimensional information to a crown block system. The intelligent crown block can automatically grab the target object, the cost of the crown block system for realizing intelligent loading and unloading is reduced, and the crown block can be used for three-dimensionally positioning objects with various shapes.
The specific technical scheme is as follows:
the invention relates to an intelligent crown block control system based on a binocular vision three-dimensional positioning technology, which comprises a binocular camera, an image processing module, a crown block mechanical module and a crown block power driving module, wherein the image processing module comprises a computer carrying an image processing algorithm and a communication circuit, the crown block mechanical module comprises a crown block mechanical structure and a motor arranged on the mechanical structure, the crown block power driving module comprises a driver used for driving the motor of the crown block mechanical structure and a PLC used for controlling the driver and being responsible for communication, the binocular camera is connected with the image processing module to realize two-way communication, the image processing module is in one-way communication with the crown block power driving module, and the crown block power driving module controls the crown block mechanical structure.
The binocular camera continuously shoots a target warehousing area, generates a photo stream with the unit of two photos, and simultaneously transmits the photo stream to a computer.
The computer in the image processing module receives the photo stream transmitted by the binocular camera in real time and inputs the photo stream into the image processing algorithm program, the computer extracts the three-dimensional information of the target warehousing object in the photo through the algorithm program and transmits the three-dimensional information to the crown block power driving module, and the PLC in the driving module controls the driver to drive the crown block mechanical module to be right above the target warehousing object according to the received three-dimensional information.
The image processing algorithm comprises the following steps of processing the target warehousing object:
1) firstly, a target warehousing area is set as an interest area, and then whether warehousing objects appear in the interest area is judged. If yes, entering the next step and stopping transmitting the photo stream; if not, repeating the first step;
2) preprocessing two pictures of the target warehousing object, extracting a peripheral outline, and filtering out image noise and other useless image parts;
3) on the extracted peripheral contour, an algorithm for extracting mutually matched feature points (points with violent change of image gray values or points with larger curvature on the image edges) on the two images is operated, and then the coordinate information of the target feature points is calculated by using the mathematical principle of binocular vision according to the extracted feature points;
The binocular camera consists of a pair of monocular cameras with known internal parameters (parameters related to the characteristics of the camera itself, such as the focal length, pixel size and the like of the camera) and external parameters (parameters in a real world coordinate system, such as the position, rotation direction and the like of the camera), is debugged and installed to ensure that baselines of the group of binocular cameras are on the same horizontal line (the error of binocular vision positioning can be reduced by using a high-pixel camera, providing a telephoto lens for the camera or increasing the baseline between the two cameras), and transmits the shot photo stream to the image processing module;
the image processing module comprises a computer and an image processing algorithm, the computer is responsible for receiving a group of pictures transmitted by the binocular camera, simultaneously operates the image processing algorithm, and transmits a coordinate information result to the crown block power driving module after the algorithm operation is completed. The image processing algorithm is written by C + + and an open-source OPENCV development library is applied at the same time, so that the running efficiency of the algorithm is improved, and the writing and debugging cost of the algorithm is reduced.
In order to facilitate debugging of engineering personnel, the image processing algorithm performs visualization processing when preprocessing pictures, filtering the images, extracting matching feature points and calculating coordinate information of the feature points, namely, the pictures processed by the algorithm in each step are stored in a computer end in sequence, the step names are signed, and the processed images are output. Therefore, the purpose of quickly analyzing and debugging the image processing algorithm can be achieved, and the time cost problem of installation and debugging of the system is reduced.
The flow of the image processing algorithm (see fig. 1) is: firstly, judging whether a target warehousing object exists in a warehousing area or not, then preprocessing photos of two target warehousing objects, extracting a peripheral contour, filtering image noise and other useless image parts, operating an algorithm for extracting mutually matched feature points (points with violent change of image gray values or points with larger curvature on image edges) on two images on the extracted peripheral contour, and then calculating the coordinate information of the target feature points by using the mathematical principle of binocular vision (see figure 2) according to the extracted feature points;
the image processing algorithm extracts a plurality of groups of random characteristic points in two pictures of the target contour, puts the coordinate information of the characteristic points into a set, compares the coordinate information one by one to respectively obtain the maximum value and the minimum value of coordinates on an X axis (horizontal direction coordinate), a Y axis (vertical direction coordinate) and a Z axis (depth coordinate), and respectively sets the maximum value and the minimum value of the coordinates as three groups of coordinates (X coordinate)max，Xmin)、(Ymax，Ymin)、(Zmax，Zmin)。
Then, the image processing algorithm is operated once again to obtain a new group of data, and the new group of data is set as Xmax′、Xmin′、Ynax′、Ymin′、Zmax′、Zmin'. X is to bemax' and Xmax、XminAnd X min' and so on. If Xmax' greater than XmaxThen take Xmax', discard Xmax(ii) a If Xmin' less than XminThen take Xmin', discard Xmin. The same is carried out on two groups of data on Y-axis and Z-axis coordinatesThe operation of (2). By the one-step operation, accidental conditions influencing data are eliminated, and the robust performance of the image algorithm is improved.
After the above operation, a completely new set of data (X) is obtainedmax″，Xmin″)，(Ymax″，Ymin″)，(Zmax″，ZminAnd the data are substituted into a formula for calculating the center point of the target warehousing object to obtain the center point coordinate (X) calculated by the algorithmcen，Ycen，Zcen)。
The central point calculation formula is:
after the coordinates of the central point of the object in the target storage are obtained, three groups of data of an X axis, a Y axis and a Z axis and the coordinates of the central point are displayed on a computer terminal, so that the debugging of an image processing algorithm by engineering personnel is facilitated, and meanwhile, the coordinate information of the central point is transmitted to a crown block power driving module.
The crown block power driving module is composed of a PLC, a frequency converter and an encoder arranged on a motor, wherein the PLC is used for receiving the central point coordinate information transmitted by the image processing module, analyzing and calculating the distance of the crown block which needs to travel in the horizontal and vertical directions from the current coordinate position to the central point coordinate position of the target warehousing object, then transmitting the parameters (running time and frequency) to the frequency converter, and the frequency converter controls the motor to drive the crown block mechanical module to the designated place given by the PLC. Meanwhile, the encoder on the motor feeds back the actual traveling distance of the motor to the PLC, the PLC calculates the error distance between the motor and the target coordinate according to the fed-back information, and transmits the parameters to the frequency converter again until the overhead traveling crane moves within the acceptable error range.
The overhead traveling crane mechanical module comprises an overhead traveling crane mechanical structure and a motor part used for driving the overhead traveling crane to perform horizontal and vertical movement, and a control signal of the motor is provided by the overhead traveling crane driving module.
The basic idea of the invention is to obtain a positioning mode more suitable for an intelligent crown block system to target warehousing objects with undefined shapes by utilizing a binocular vision system capable of obtaining more accurate depth information and combining an image processing algorithm.
Example (b):
the control system comprises a binocular camera, an image processing module, a crown block mechanical module and a crown block power driving module, wherein the binocular camera and the image processing module are called a binocular vision system, the crown block mechanical module and the crown block power driving module are called a crown block system, and the two systems are connected with each other to realize one-way communication of the binocular vision system to the intelligent crown block system. The mechanical mounting structure of the whole system is (see fig. 4): the binocular camera is installed on a support of the overhead travelling crane structure (the height of the support needs to ensure that the binocular camera can shoot a stereoscopic image of a target warehousing object), the two cameras keep a certain distance (the measurement error is smaller when the distance between the general binocular cameras is 0.8-2.2 times of the working distance, so that the proper distance can be selected according to the actual environment and the approximate size of the target warehousing object to ensure the measurement accuracy), and meanwhile, the base lines of the two cameras are ensured to be on the same horizontal line.
The binocular camera consists of two CCD cameras with the models of large constant MER-125-30 UM/UC-L. Is connected with a computer with an intel core i7 processor through a USB interface.
In the crown block mechanical module, a mechanical trolley frame body which moves transversely and longitudinally is made of aluminum alloy, 8 (4) AC motors with rated power of 250W are respectively used for providing power, and a lifting motor with rated power of 500W is used for loading and unloading.
In the crown block power driving module, the model selected by the PLC is Siemens S7-1200, and the PLC is responsible for communication control with a computer and a driver. The drivers are 3 drivers with rated driving power of 1.2kw and model of standing grain HCFA-X3D, wherein two drivers are responsible for driving 8 250W alternating current motors, and the other driver is responsible for driving a 500W lifting motor.
After a group of pictures of the target warehousing object are obtained, the pictures are sent to a computer to run an image processing algorithm to obtain the coordinates of the geometric center point of the warehousing object. The computer transmits the obtained coordinates to the overhead traveling crane system, and the overhead traveling crane system drives the overhead traveling crane to move to the target coordinate location according to the received coordinate information, so that the intelligent loading and unloading process of the warehousing object can be realized.
As shown in fig. 5, the system work flow is:
the computer controls the binocular camera to continuously shoot the warehousing area to form a photo stream (with the unit of two photos), and meanwhile the photo stream is transmitted to the computer.
The computer inputs the transmitted photo stream into an image processing algorithm, and the algorithm judges the warehousing area in the photo stream to judge whether warehousing objects appear. And if the warehousing object does not appear, returning to the judging step. And if the warehousing object appears, stopping transmitting the photo stream, and simultaneously calculating the coordinate information of the central point of the target warehousing object according to the two photos. And then transmits the coordinate information to the overhead traveling crane system.
And the PLC in the crown block system analyzes and calculates the travel distance of the crown block from the current coordinate position to the central point coordinate position of the target warehousing object in the horizontal and vertical directions according to the central point coordinate information, and then transmits the running time and frequency parameters to the frequency converter.
And the frequency converter controls the motor to drive the crown block mechanical module to a specified place given by the PLC. Meanwhile, the encoder on the motor feeds back the actual traveling distance of the motor to the PLC, the PLC calculates the error distance between the motor and the target coordinate according to the fed-back information, and transmits the parameters to the frequency converter again until the overhead traveling crane moves within the acceptable error range.
The invention can realize the extraction of the three-dimensional coordinate information of the target object needing to be loaded and unloaded by the overhead traveling crane system, and simultaneously calculate the geometric center coordinate of the target object by utilizing the extracted coordinate information, and the overhead traveling crane system realizes the intelligent loading and unloading process of the target object needing to be loaded and unloaded according to the center coordinate, thereby improving the operation efficiency of the overhead traveling crane system.
The invention realizes the positioning of geometric central points of warehoused objects with different shapes and enables the overhead traveling crane system to realize the intelligent loading and unloading process of target objects by combining a high-performance and easily-debugged image processing algorithm and a binocular vision system which can better extract depth information and has good economy with the overhead traveling crane system.
The invention positions the steel coil-like (i.e. hollow cylinder) warehousing object under the conditions of the embodiment. The effect of the matched feature points obtained after shooting by the binocular camera and processing by the image processing algorithm is shown in fig. 6.
By solving the three-dimensional position information of the matched characteristic points, the coordinates of the center point of the warehousing object of the steel coil type can be obtained. Through 7 times of experiments, the experimental results are shown in the following table:
experimental results (error unit: mm; time unit: second)
Other embodiments of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the disclosure disclosed herein. This application is intended to cover any variations, uses, or adaptations of the disclosure following, in general, the principles of the disclosure and including such departures from the present disclosure as come within known or customary practice within the art to which the disclosure pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the disclosure being indicated by the following claims.
It will be understood that the present disclosure is not limited to the precise arrangements described above and shown in the drawings and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure should be limited only by the attached claims.
Claims (6)
1. The utility model provides an intelligence overhead traveling crane control system of three-dimensional location technique based on binocular vision which characterized in that, intelligence overhead traveling crane control system of three-dimensional location technique based on binocular vision is provided with:
the binocular camera is arranged on a support of the crown block structure, continuously shoots a target warehousing area, generates a photo stream with a unit of two photos, and simultaneously transmits the photo stream to the computer;
The image processing module is in two-way communication with the binocular camera, receives the photo stream transmitted by the binocular camera in real time through the computer, inputs the photo stream into an image processing algorithm program, extracts the three-dimensional information of the target warehousing object in the photo through the algorithm program and transmits the three-dimensional information to the crown block power driving module;
the crane mechanical module consists of a transverse moving mechanical trolley and a longitudinal moving mechanical trolley, the two mechanical trolleys are respectively provided with four motors for providing power, the longitudinal moving trolley is provided with a lifting motor for realizing loading and unloading work, the nine motors are all connected with a driver in the crane power driving module, and the driver provides a driving signal;
the overhead traveling crane power driving module is in one-way communication with the image processing module to control the mechanical structure of the overhead traveling crane, and the PLC controls the driver to drive the overhead traveling crane mechanical module to be right above the target warehousing object according to the received three-dimensional information;
in the image processing module, the image processing algorithm processes the target warehousing object as follows:
step one, setting a target warehousing area as an interest area, and then judging whether warehousing objects appear in the interest area or not; if yes, entering the next step and stopping transmitting the photo stream; if not, repeating the first step;
Preprocessing two pictures of the target warehousing objects, extracting peripheral outlines, and filtering out image noise and other useless image parts;
thirdly, on the extracted peripheral contour, operating an algorithm for extracting mutually matched feature points on the two images, and then calculating coordinate information of the target feature points by using a binocular vision mathematical principle according to the extracted feature points;
in step three, the image processing algorithm extracts a plurality of groups of random characteristic points in the pictures of the two target contours, puts the coordinate information of the characteristic points into a set, and then carries out image processing on the characteristic pointsThe coordinate information is compared one by one to respectively obtain the maximum value and the minimum value of the coordinate on the X axis of the horizontal coordinate, the Y axis of the vertical coordinate and the Z axis of the depth coordinate, and the maximum value and the minimum value are respectively set as three groups of coordinates Xmax，Xmin；Ymax，Ymin；Zmax，Zmin；
Then, the image processing algorithm is operated again to obtain a new group of data, and the new group of data is set as Xmax’、Xmin′、Ymax′、Ymin′、Zmax′、Zmin′；
X is to bemax' and Xmax、XminAnd Xmin' sequential comparison, if Xmax' greater than XmaxThen take Xmax', discard Xmax(ii) a If Xmin' less than XminThen take Xmin', discard Xmin；
Will Ymax' and Ymax、YminAnd Ymin' sequential comparison, if Ymax' greater than YmaxThen take Ymax', discard Ymax(ii) a If Y is min' less than YminThen take Ymin', discard Ymin；
Will Zmax' and Zmax、ZminAnd Zmin' sequential comparison, if Zmax' greater than ZmaxThen take Zmax', discard Zmax(ii) a If Z ismin' less than ZminThen take Zmin', discard Zmin；
After the above operation, a completely new set of data (X) is obtainedmax″，Xmin″)，(Ymax″，Ymin″)，(Zmax″，ZminAnd the data are substituted into a formula for calculating the center point of the target warehousing object to obtain the center point coordinate (X) calculated by the algorithmcen，Ycen，Zcen)；
The central point calculation formula is:
after central point coordinates of an object in a target storage are obtained, three groups of data of an X axis, a Y axis and a Z axis and the central point coordinates are displayed on a computer terminal, and meanwhile, central point coordinate information is transmitted to a crown block power driving module;
the crown block power driving module consists of a PLC, a frequency converter and an encoder arranged on a motor;
the PLC receives the central point coordinate information transmitted by the image processing module, analyzes and calculates the distance that the crown block needs to travel in the horizontal and vertical directions from the current coordinate position to the central point coordinate position of the target warehousing object, and then transmits the running time and frequency parameters to the frequency converter;
the frequency converter controls the motor to drive the crown block mechanical module to a specified place given by the PLC;
and the encoder feeds back the actual travelling distance of the motor to the PLC, the PLC recalculates the error distance between the encoder and the target coordinate according to the fed-back information, and transmits the parameters to the frequency converter again until the crown block moves to within the acceptable error range.
2. The intelligent overhead traveling crane control system based on binocular vision three-dimensional positioning technology as claimed in claim 1, wherein the binocular camera is composed of a pair of monocular cameras which know the intrinsic parameters of the camera's own characteristics by a Zhang friend camera calibration method and the extrinsic parameters of the translation position and the rotation direction between the two cameras obtained by stereo calibration, the baselines of the binocular cameras are on the same horizontal line, and the stream of the taken photos is transmitted to the image processing module.
3. The intelligent overhead traveling crane control system based on binocular vision three-dimensional positioning technology as claimed in claim 1, wherein the image processing module comprises a computer carrying an image processing algorithm and a communication line;
the computer is responsible for receiving a group of pictures transmitted by the binocular camera, operating an image processing algorithm at the same time, and transmitting a coordinate information result to the crown block power driving module after the algorithm is operated; the image processing algorithm is written by C + + and simultaneously utilizes an open source OPENCV development library for improving the running efficiency of the algorithm and reducing the writing and debugging cost of the algorithm.
4. The intelligent overhead traveling crane control system based on binocular vision three-dimensional positioning technology of claim 1, wherein the overhead traveling crane mechanical module comprises an overhead traveling crane mechanical structure and a motor installed on the mechanical structure, and a control signal of the motor is provided by an overhead traveling crane driving module;
The control signal provided by the crown block driving module consists of a positive and negative rotation signal and a rotating speed signal, and the motor realizes the transverse and longitudinal movement of the crown block mechanical structure and the loading and unloading work of the target warehousing object according to the received control signal.
5. A method for implementing the intelligent overhead traveling crane control system based on binocular vision three-dimensional positioning technology according to any one of claims 1 to 4, wherein the intelligent overhead traveling crane control method based on binocular vision three-dimensional positioning technology comprises the following steps:
step one, a computer controls a binocular camera to continuously shoot an entering area to form a photo stream, and the photo stream is transmitted to the computer;
step two, the computer inputs the transmitted photo stream into an image processing algorithm, and the algorithm judges whether a warehousing area in the photo stream appears or not; if no warehousing object appears, returning to the judging step; if the warehousing object appears, stopping transmitting the photo stream, and simultaneously calculating the coordinate information of the central point of the target warehousing object according to the two photos; then transmitting the coordinate information to an overhead travelling crane system;
thirdly, analyzing and calculating the distance of the central point coordinate position of the object to be warehoused, which is moved from the current coordinate position of the crown block, in the horizontal and vertical directions by a PLC in the crown block system according to the central point coordinate information, and then transmitting the running time and frequency parameters to a frequency converter;
And step four, the frequency converter controls the motor to drive the mechanical module of the overhead travelling crane to a specified place given by the PLC, meanwhile, an encoder on the motor feeds the actual travelling distance of the motor back to the PLC, the PLC calculates the error distance between the motor and the target coordinate according to the fed-back information, and transmits the parameters to the frequency converter again until the overhead travelling crane moves within an acceptable error range.
6. An information data processing terminal, characterized in that the information data processing terminal comprises a memory and a processor, the memory stores a computer program, when the computer program is executed by the processor, the processor is caused to execute the intelligent crown block control method based on binocular vision three-dimensional positioning technology according to claim 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110243051.7A CN112884848B (en) | 2021-03-05 | 2021-03-05 | Intelligent crown block control system, method and terminal based on binocular vision |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110243051.7A CN112884848B (en) | 2021-03-05 | 2021-03-05 | Intelligent crown block control system, method and terminal based on binocular vision |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112884848A CN112884848A (en) | 2021-06-01 |
| CN112884848B true CN112884848B (en) | 2022-06-28 |
Family
ID=76055530
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110243051.7A Active CN112884848B (en) | 2021-03-05 | 2021-03-05 | Intelligent crown block control system, method and terminal based on binocular vision |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112884848B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114332345B (en) * | 2021-09-23 | 2023-06-20 | 北京科技大学 | Binocular vision-based metallurgical reservoir area local three-dimensional reconstruction method and system |
| CN113968538B (en) * | 2021-09-30 | 2024-04-19 | 国网江苏省电力有限公司电力科学研究院 | Traveling crane hoisting method, traveling crane hoisting system, computer equipment and storage medium |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN202897859U (en) * | 2012-11-23 | 2013-04-24 | 河北联合大学 | Vision-based obstacle avoidance pre-warning device of crown block robot |
| EP2950521A2 (en) * | 2014-05-29 | 2015-12-02 | LG Electronics Inc. | Camera capable of reducing motion blur in a low luminance environment and vehicle including the same |
| CN110539109A (en) * | 2019-08-28 | 2019-12-06 | 广东工业大学 | Robot automatic welding system and method based on single and binocular vision |
| CN112083681A (en) * | 2019-06-14 | 2020-12-15 | 湖南釜晟智能科技有限责任公司 | Overhead traveling crane integrated control system |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110009690A (en) * | 2019-03-23 | 2019-07-12 | 西安电子科技大学 | Binocular stereo vision image measuring method based on polar curve correction |
| CN110969652B (en) * | 2019-11-20 | 2023-10-03 | 上海交通大学 | Shooting method and system based on mechanical arm monocular camera serving as binocular stereoscopic vision |
| CN112051853B (en) * | 2020-09-18 | 2023-04-07 | 哈尔滨理工大学 | Intelligent obstacle avoidance system and method based on machine vision |
-
2021
- 2021-03-05 CN CN202110243051.7A patent/CN112884848B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN202897859U (en) * | 2012-11-23 | 2013-04-24 | 河北联合大学 | Vision-based obstacle avoidance pre-warning device of crown block robot |
| EP2950521A2 (en) * | 2014-05-29 | 2015-12-02 | LG Electronics Inc. | Camera capable of reducing motion blur in a low luminance environment and vehicle including the same |
| CN112083681A (en) * | 2019-06-14 | 2020-12-15 | 湖南釜晟智能科技有限责任公司 | Overhead traveling crane integrated control system |
| CN110539109A (en) * | 2019-08-28 | 2019-12-06 | 广东工业大学 | Robot automatic welding system and method based on single and binocular vision |
Non-Patent Citations (2)
| Title |
|---|
| 天车机器人设计;王福斌 等;《机床与液压》;20151115;第1-4页 * |
| 王福斌 等.天车机器人设计.《机床与液压》.2015, * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112884848A (en) | 2021-06-01 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110389348B (en) | Positioning and navigation method and device based on laser radar and binocular camera | |
| CN210046133U (en) | Welding seam visual tracking system based on laser structured light | |
| CN112884848B (en) | Intelligent crown block control system, method and terminal based on binocular vision | |
| CN112010024B (en) | Automatic container grabbing method and system based on laser and vision fusion detection | |
| CN106127145B (en) | Pupil diameter and tracking | |
| CN112643207B (en) | Laser automatic derusting system and method based on computer vision | |
| CN111445380B (en) | Method and device for realizing real-time binocular stereo matching based on ZYNQ | |
| CN115482195A (en) | Train part deformation detection method based on three-dimensional point cloud | |
| Han et al. | Grasping control method of manipulator based on binocular vision combining target detection and trajectory planning | |
| CN114770461A (en) | Monocular vision-based mobile robot and automatic grabbing method thereof | |
| Park et al. | Global map generation using LiDAR and stereo camera for initial positioning of mobile robot | |
| CN112379605B (en) | Bridge crane semi-physical simulation control experiment system and method based on visual servo | |
| CN113701750A (en) | Fusion positioning system of underground multi-sensor | |
| CN108983791A (en) | A kind of intelligence vehicle control | |
| CN111975776A (en) | Robot movement tracking system and method based on deep learning and Kalman filtering | |
| CN108564626B (en) | Method and apparatus for determining relative pose angle between cameras mounted to an acquisition entity | |
| Chen et al. | A method for mobile robot obstacle avoidance based on stereo vision | |
| WO2021017083A1 (en) | Industrial robot modeling and positioning system, and robot | |
| CN111028231B (en) | Workpiece position acquisition system based on ARM and FPGA | |
| Yang et al. | A general line tracking algorithm based on computer vision | |
| Song et al. | Localization Method Based on Image Processing for Autonomous Driving of Mobile Robot in the Linear Infrastructure | |
| CN117805235A (en) | Casting detection method, device, system and storage medium | |
| CN111452036B (en) | Workpiece grabbing method based on line laser binocular stereoscopic vision | |
| CN117260734A (en) | Robot autonomous guiding method and system based on automatic dimming laser vision | |
| Xu et al. | Truck Size Measurement System Based on Computer Vision |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |