CN116148259B - Vehicle defect positioning system, method, device and storage medium - Google Patents
Vehicle defect positioning system, method, device and storage medium Download PDFInfo
- Publication number
- CN116148259B CN116148259B CN202211694317.0A CN202211694317A CN116148259B CN 116148259 B CN116148259 B CN 116148259B CN 202211694317 A CN202211694317 A CN 202211694317A CN 116148259 B CN116148259 B CN 116148259B
- Authority
- CN
- China
- Prior art keywords
- defect
- coordinates
- spraying
- coordinate
- vehicle
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 230000007547 defect Effects 0.000 title claims abstract description 256
- 238000000034 method Methods 0.000 title claims abstract description 29
- 238000001514 detection method Methods 0.000 claims abstract description 25
- 238000005507 spraying Methods 0.000 claims description 74
- 239000011159 matrix material Substances 0.000 claims description 48
- 238000012360 testing method Methods 0.000 claims description 26
- 238000012545 processing Methods 0.000 claims description 24
- 238000006243 chemical reaction Methods 0.000 claims description 21
- 230000009466 transformation Effects 0.000 claims description 12
- 230000004807 localization Effects 0.000 claims description 11
- 238000013519 translation Methods 0.000 claims description 9
- 230000009471 action Effects 0.000 claims description 5
- 239000007921 spray Substances 0.000 claims description 5
- 230000003068 static effect Effects 0.000 claims description 3
- 239000003973 paint Substances 0.000 description 5
- 238000005516 engineering process Methods 0.000 description 3
- 238000010422 painting Methods 0.000 description 3
- 230000009286 beneficial effect Effects 0.000 description 2
- 238000004891 communication Methods 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 238000007689 inspection Methods 0.000 description 2
- 238000004458 analytical method Methods 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 239000011248 coating agent Substances 0.000 description 1
- 238000000576 coating method Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 239000003550 marker Substances 0.000 description 1
- 238000010295 mobile communication Methods 0.000 description 1
- 238000003062 neural network model Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000008569 process Effects 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 230000007723 transport mechanism Effects 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/84—Systems specially adapted for particular applications
- G01N21/88—Investigating the presence of flaws or contamination
- G01N21/8851—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B05—SPRAYING OR ATOMISING IN GENERAL; APPLYING FLUENT MATERIALS TO SURFACES, IN GENERAL
- B05B—SPRAYING APPARATUS; ATOMISING APPARATUS; NOZZLES
- B05B13/00—Machines or plants for applying liquids or other fluent materials to surfaces of objects or other work by spraying, not covered by groups B05B1/00 - B05B11/00
- B05B13/02—Means for supporting work; Arrangement or mounting of spray heads; Adaptation or arrangement of means for feeding work
- B05B13/04—Means for supporting work; Arrangement or mounting of spray heads; Adaptation or arrangement of means for feeding work the spray heads being moved during spraying operation
- B05B13/0431—Means for supporting work; Arrangement or mounting of spray heads; Adaptation or arrangement of means for feeding work the spray heads being moved during spraying operation with spray heads moved by robots or articulated arms, e.g. for applying liquid or other fluent material to 3D-surfaces
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0004—Industrial image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
- G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/84—Systems specially adapted for particular applications
- G01N21/88—Investigating the presence of flaws or contamination
- G01N21/8851—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges
- G01N2021/8854—Grading and classifying of flaws
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/84—Systems specially adapted for particular applications
- G01N21/88—Investigating the presence of flaws or contamination
- G01N21/8851—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges
- G01N2021/8854—Grading and classifying of flaws
- G01N2021/888—Marking defects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30248—Vehicle exterior or interior
Landscapes
- Engineering & Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Biochemistry (AREA)
- Chemical & Material Sciences (AREA)
- Analytical Chemistry (AREA)
- Life Sciences & Earth Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Health & Medical Sciences (AREA)
- Immunology (AREA)
- Pathology (AREA)
- Robotics (AREA)
- Quality & Reliability (AREA)
- Signal Processing (AREA)
- Length Measuring Devices By Optical Means (AREA)
- Investigating Materials By The Use Of Optical Means Adapted For Particular Applications (AREA)
Abstract
The invention discloses a vehicle defect positioning system, a vehicle defect positioning method, a vehicle defect positioning device and a storage medium, and relates to the technical field of vehicle detection. According to the vehicle defect detection method, the image acquisition device is used for shooting a vehicle body image, the vehicle body image is input into the defect detection model, the defect position pixel coordinates under the pixel coordinate system are obtained, the defect position pixel coordinates are converted into the defect position space coordinates under the machine coordinate system, and then the position calibration device is controlled to mark the vehicle body according to the defect position space coordinates, so that automatic positioning of the defect position on the vehicle body is achieved, and the vehicle defect detection efficiency is improved.
Description
Technical Field
The present invention relates to the field of vehicle detection technologies, and in particular, to a system, a method, an apparatus, and a storage medium for locating a vehicle defect.
Background
The automobile coating is an important link in the automobile production process, the quality of the paint surface determines the product quality, and even tiny surface defects can be degraded instantaneously, so the paint surface quality detection is an important inspection item before the whole automobile leaves the factory. In the past, vehicle body defect detection after spraying mainly uses a marker pen to mark defect positions through manual visual, or uses a camera to shoot and identify defects and displays approximate defect positions on the surface of a vehicle in a picture form on a screen, and when an on-site operator performs secondary returning, the on-site operator cannot timely position specific defect positions of the vehicle, so that the defect positioning is long in time consumption and low in efficiency.
Disclosure of Invention
The present invention aims to solve at least one of the technical problems existing in the prior art. Therefore, the invention provides a vehicle defect positioning system, a method, a device and a storage medium, which can automatically position the defect position on the vehicle body and improve the vehicle defect detection efficiency.
In one aspect, an embodiment of the invention provides a vehicle defect positioning system, which comprises an image acquisition device, a data processing device and a position calibration device;
the image acquisition device is used for shooting an image of the vehicle body;
the data processing device is used for inputting the vehicle body image into a defect detection model to obtain a defect position pixel coordinate under a pixel coordinate system, and converting the defect position pixel coordinate into a defect position space coordinate under a machine coordinate system;
the position calibration device is used for marking on the vehicle body according to the space coordinates of the defect position.
According to some embodiments of the invention, the image acquisition device comprises a plurality of cameras, the position calibration device comprises a plurality of spraying manipulators, and the data processing device stores a two-dimensional coordinate conversion matrix corresponding to the cameras and the spraying manipulators one by one.
According to some embodiments of the invention, the data processing apparatus determines the defect location spatial coordinates by:
acquiring a first number and a camera attitude parameter of a camera shooting the car body image;
determining a second number of the spraying manipulator to perform spraying action;
inquiring a two-dimensional coordinate transformation matrix according to the first number and the second number;
determining a defect position plane coordinate according to the two-dimensional coordinate conversion matrix and the defect position pixel coordinate;
and determining the space coordinates of the defect position according to the camera attitude parameters and the plane coordinates of the defect position.
According to some embodiments of the invention, the camera pose parameter includes a shooting distance and a shooting angle between the camera and a plane of the vehicle body, and the determining the spatial coordinates of the defect position according to the camera pose parameter and the coordinates of the plane of the defect position includes the following steps:
determining a defect position height coordinate according to the shooting distance and the shooting angle;
and determining the space coordinates of the defect position according to the height coordinates of the defect position and the plane coordinates of the defect position.
According to some embodiments of the invention, the two-dimensional coordinate transformation matrix is obtained by:
selecting a group of cameras and spraying manipulators;
collecting a test image through the camera, and calibrating pixel coordinates of nine test points on the test image;
controlling the spraying manipulator to correspondingly move to the positions of the test points, and acquiring the space positions of nine test points;
determining a rotation quantity matrix and a translation quantity matrix according to the pixel coordinates and the space coordinates of the test points;
and determining a two-dimensional coordinate conversion matrix of each group of cameras and the spraying manipulator according to the rotation amount matrix and the translation amount matrix.
According to some embodiments of the invention, the data processing apparatus is further configured to:
inputting the vehicle body image into the defect detection model to obtain a defect contour coordinate set;
selecting two defect contour coordinates with the largest coordinate distance in the defect contour coordinate set;
and determining the pixel coordinates of the defect position according to the two selected defect contour coordinates.
According to some embodiments of the invention, the data processing apparatus is further configured to:
determining a defect category according to the defect contour coordinate set;
when the defect type is a block defect, determining the radius of an circumscribed circle and the center coordinates of the circumscribed circle of the block defect according to the defect contour coordinate set, and controlling the position calibration device to perform fixed-point spraying according to the radius of the circumscribed circle and the center coordinates of the circumscribed circle;
and when the defect type is a strip defect, determining a spraying path according to the defect contour coordinate set, and performing mobile spraying according to the spraying path.
On the other hand, the embodiment of the invention also provides a vehicle defect positioning method, which is applied to the data processing device of the vehicle defect positioning system according to the previous embodiment, and comprises the following steps:
shooting a car body image through an image acquisition device;
inputting the vehicle body image into a defect detection model to obtain a defect position pixel coordinate under a pixel coordinate system;
converting the pixel coordinates of the defect position into space coordinates of the defect position under a machine coordinate system;
and controlling the position calibration device to mark the vehicle body according to the space coordinates of the defect position.
In another aspect, an embodiment of the present invention further provides a vehicle defect positioning device, including:
at least one processor;
at least one memory for storing at least one program;
the at least one program, when executed by the at least one processor, causes the at least one processor to implement the vehicle defect localization method as previously described.
In another aspect, embodiments of the present invention also provide a computer-readable storage medium storing computer-executable instructions for causing a computer to perform a vehicle defect localization method as described above.
The technical scheme of the invention has at least one of the following advantages or beneficial effects: the method comprises the steps of shooting a vehicle body image through an image acquisition device, inputting the vehicle body image into a defect detection model to obtain defect position pixel coordinates under a pixel coordinate system, converting the defect position pixel coordinates into defect position space coordinates under a machine coordinate system, and controlling a position calibration device to mark the vehicle body according to the defect position space coordinates, so that automatic positioning of the defect position on the vehicle body is realized, and vehicle defect detection efficiency is improved.
Drawings
FIG. 1 is a flow chart of a method for locating a vehicle defect provided by an embodiment of the invention;
FIG. 2 is a schematic diagram of a defect positioning apparatus for a vehicle according to an embodiment of the present invention;
fig. 3 is a schematic view of a camera gesture provided in an embodiment of the present invention.
Detailed Description
Embodiments of the present invention are described in detail below, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals refer to the same or similar elements or elements having the same or similar functions throughout. The embodiments described below by referring to the drawings are illustrative only and are not to be construed as limiting the invention.
In the description of the present invention, it should be understood that the direction or positional relationship indicated with respect to the description of the orientation, such as up, down, left, right, etc., is based on the direction or positional relationship shown in the drawings, is merely for convenience of describing the present invention and simplifying the description, and does not indicate or imply that the apparatus or element to be referred to must have a specific orientation, be constructed and operated in a specific orientation, and thus should not be construed as limiting the present invention.
In the description of the present invention, the description of first, second, etc. is for the purpose of distinguishing between technical features only, and should not be construed as indicating or implying a relative importance or implicitly indicating the number of technical features indicated or implicitly indicating the precedence of the technical features indicated.
The embodiment of the invention provides a vehicle defect positioning system which comprises an image acquisition device, a data processing device and a position calibration device.
The image acquisition device is used for shooting an image of the vehicle body;
the data processing device is used for inputting the vehicle body image into the defect detection model to obtain a defect position pixel coordinate under a pixel coordinate system, and converting the defect position pixel coordinate into a defect position space coordinate under a machine coordinate system;
the position calibration device is used for marking the vehicle body according to the space coordinates of the defect position.
In this embodiment, the image acquisition device includes a plurality of cameras, and the position calibration device includes a plurality of spraying manipulators, for example, the image acquisition device can two cameras, and two cameras set up respectively in the left and right sides of station, are used for gathering the picture of automobile body left and right sides respectively. The position calibration device comprises two spraying manipulators which are respectively arranged at the left side and the right side of the station and are respectively used for marking the defect positions of the vehicle bodies at the left side and the right side. The spraying manipulator is provided with a round spray head with a multi-through hole design, water-soluble paint is loaded in the spray head, and the spraying manipulator can adjust the spraying range of the paint by adjusting the opening and closing of the through holes on the spray head. The marked points can be cleaned after the manual inspection is finished by adopting the water-soluble paint.
In this embodiment, the defect detection model is a model constructed based on a deep neural network, and a large number of picture samples are input into the deep neural network model to train the model to learn various defect information in the vehicle body image. In the application process, the acquired car body picture is input into a trained defect detection model, the defect detection model carries out recognition analysis on pixels in the car body picture, so that a defect area is segmented, and a defect contour coordinate set is further output.
In this embodiment, after the defect detection model identifies the defect in the image, the pixel coordinates of the defect position in the pixel coordinate system are obtained, and at this time, the pixel coordinates of the defect position in the pixel coordinate system need to be converted into the spatial coordinates of the defect position in the machine coordinate system, and the spraying manipulator can be operated to the corresponding defect position based on the spatial coordinates of the defect position to perform spraying. The defect position pixel coordinates and the defect position spatial coordinates can be converted by a two-dimensional coordinate conversion matrix. The data processing device stores a two-dimensional coordinate conversion matrix corresponding to the cameras and the spraying manipulator one by one.
In this embodiment, the two-dimensional coordinate transformation matrix between the camera and the spraying robot is determined by 9-point calibration. Specifically, a group of cameras and spraying manipulators are selected, test images are collected through the selected cameras in a fixed posture, pixel coordinates of nine test points are calibrated on the test images, then the spraying manipulators are controlled to correspondingly move to the positions of the test points, space positions of the nine test points are obtained, after the pixel coordinates of the test points under a pixel coordinate system and the space coordinates of the test points under a machine coordinate system are obtained, a rotation amount matrix and a translation amount matrix between the pixel coordinates and the space coordinates representing the same position can be determined according to the pixel coordinates and the space coordinates of the test points, and then a two-dimensional coordinate conversion matrix of each group of cameras and the spraying manipulators can be determined according to the rotation amount matrix and the translation amount matrix. The two-dimensional coordinate transformation matrix is expressed as follows:
wherein R is a rotation matrix, and t is a translation matrix.
In some embodiments, the data processing apparatus determines the defect location spatial coordinates by:
acquiring a first number and a camera attitude parameter of a camera for shooting a car body image;
determining a second number of the spraying manipulator to perform spraying action;
inquiring the two-dimensional coordinate transformation matrix according to the first number and the second number;
determining the plane coordinates of the defect position according to the two-dimensional coordinate conversion matrix and the pixel coordinates of the defect position;
and determining the space coordinates of the defect position according to the camera attitude parameters and the plane coordinates of the defect position.
Illustratively, assume that the first left camera is selected to capture an image of the vehicle body, the camera being numbered "xz1", the second left painting robot is selected to mark the corresponding actual location of the defect in the image of the vehicle body, and the painting robot is numbered "pz2". The data processing device stores a two-dimensional coordinate conversion matrix of various combination relations between the camera and the spraying manipulator. Firstly, inquiring a corresponding two-dimensional coordinate conversion matrix according to the numbers 'xz 1' and 'pz 2', inputting a vehicle body image into a defect detection model to obtain a defect position pixel coordinate, multiplying the abscissa of the defect position pixel coordinate by the two-dimensional coordinate conversion matrix to obtain the abscissa of the defect position space coordinate, and multiplying the ordinate of the defect position pixel coordinate by the two-dimensional coordinate conversion matrix to obtain the ordinate of the defect position space coordinate. The abscissa x and ordinate y of the spatial coordinates of the defect position form the plane coordinates of the defect position.
Further, referring to fig. 3, the camera pose parameters include a photographing distance and a photographing angle of the camera from the plane of the vehicle body, and a defect position height coordinate z may be determined according to the photographing distance and the photographing angle, and the defect position height coordinate z is shown in the following formula:
z＝cosθ*D；
where θ represents a shooting angle of the selected camera, and D represents a shooting distance of the selected camera.
Based on the defect-position height coordinates z and the defect-position plane coordinates (x, y), the defect-position space coordinates (x, y, z) are determined.
In other embodiments, assuming that the first left camera is selected to capture an image of the vehicle body, the vehicle body may need to be marked during actual operation by the second left painting robot. The data processing device only has a two-dimensional coordinate conversion matrix of the left first camera shooting the car body image and the left first spraying manipulator, and does not have a two-dimensional coordinate conversion matrix of the left first camera shooting the car body image and the left second spraying manipulator, at this time, coordinate conversion between pixel coordinates and robot coordinates can be carried out according to the two-dimensional coordinate conversion matrix of the left first camera shooting the car body image and the left first spraying manipulator, and then the coordinates of the left first spraying manipulator are converted into the coordinates of the left second spraying manipulator by utilizing a common coordinate system between robots, so that the position of the defect can be positioned by the left second spraying manipulator, and spraying operation can be carried out.
Different robot coordinate systems can construct a common coordinate system by using the same base coordinate, and the coordinate systems of a plurality of robots are established by calibrating the same reference system. Specifically, the tail ends of any two robots are respectively contacted with the same position of the target object, the position coordinates of the same position under different robot coordinate systems are respectively obtained, and the coordinate system conversion relation between any two robots is obtained according to the position coordinates obtained by any two robots.
According to some embodiments of the present invention, the data processing apparatus is further configured to input the vehicle body image into a defect detection model to obtain a defect contour coordinate set, where the contour coordinate set includes a plurality of defect contour coordinates, and all the defect contour coordinates form a defect contour. And selecting two defect contour coordinates with the largest coordinate distance in the defect contour coordinate set, and calculating the midpoint coordinates of the two selected defect contour coordinates, wherein the midpoint coordinates are the pixel coordinates of the defect position.
According to some embodiments of the present invention, various types of defects exist in a vehicle body, and the defects are mainly classified into block defects and bar defects, wherein the block defects include pits, stains and the like, and the bar defects include scratches. In the embodiment, different spraying modes are adopted aiming at different types of defects so as to improve the spraying efficiency and effect. And for the block defects, determining the radius of the circumscribed circle and the center coordinates of the circumscribed circle of the block defects according to the defect contour coordinate set, wherein the center coordinates of the circumscribed circle are the pixel coordinates of the defect positions, and converting to obtain the space coordinates of the defect positions. The position calibration device performs static fixed-point spraying according to the obtained space coordinates of the defect position, and the spraying range is adjusted to be the same as the radius of the circumscribed circle of the actual defect by opening or closing the through hole of the spray head according to the radius of the circumscribed circle of the block defect.
And extracting a plurality of characteristic coordinate points from the defect contour coordinate set for the strip defects, converting the plurality of characteristic coordinate points into robot coordinates to obtain a spraying path of the spraying manipulator, and moving and spraying the spraying manipulator according to the spraying path.
The embodiment of the invention also provides a vehicle defect positioning method, which is applied to the data processing device of the vehicle defect positioning system according to the previous embodiment, referring to fig. 1, and the vehicle defect positioning method according to the embodiment of the invention includes, but is not limited to, the following steps:
step S110, shooting a car body image through an image acquisition device;
step S120, inputting a vehicle body image into a defect detection model to obtain a defect position pixel coordinate under a pixel coordinate system;
step S130, converting the pixel coordinates of the defect position into the space coordinates of the defect position under a machine coordinate system;
and step S140, controlling the position calibration device to mark the vehicle body according to the space coordinates of the defect position.
According to some embodiments of the present invention, in step S130, the step of converting the defect-location pixel coordinates into defect-location spatial coordinates in the machine coordinate system includes the steps of:
step S210, a first number and camera attitude parameters of a camera for shooting a car body image are acquired;
step S220, determining a second number of the spraying manipulator to perform spraying action;
step S230, inquiring a two-dimensional coordinate transformation matrix according to the first number and the second number;
step S240, determining the plane coordinates of the defect position according to the two-dimensional coordinate transformation matrix and the pixel coordinates of the defect position;
step S250, determining the space coordinates of the defect position according to the camera attitude parameters and the plane coordinates of the defect position.
According to some embodiments of the present invention, the camera pose parameter includes a shooting distance and a shooting angle between the camera and the plane of the vehicle body, and step S250, determining the spatial coordinates of the defect position according to the camera pose parameter and the coordinates of the plane of the defect position, includes the following steps:
step S310, determining the height coordinates of the defect position according to the shooting distance and the shooting angle;
step S320, determining the space coordinates of the defect position according to the height coordinates of the defect position and the plane coordinates of the defect position.
According to some embodiments of the invention, the two-dimensional coordinate transformation matrix is obtained by:
step S410, selecting a group of cameras and spraying manipulators;
step S420, collecting a test image through a camera, and calibrating pixel coordinates of nine test points on the test image;
step S430, controlling the spraying manipulator to correspondingly move to the positions of the test points, and acquiring the spatial positions of nine test points;
step S440, determining a rotation amount matrix and a translation amount matrix according to the pixel coordinates and the space coordinates of the test points;
step S450, determining a two-dimensional coordinate transformation matrix of each group of cameras and the spraying manipulator according to the rotation amount matrix and the translation amount matrix.
According to some embodiments of the present invention, in step S120, the step of inputting the vehicle body image into the defect detection model to obtain the pixel coordinates of the defect location in the pixel coordinate system further includes, but is not limited to, the following steps:
step S510, inputting the vehicle body image into the defect detection model to obtain a defect contour coordinate set;
step S520, selecting two defect contour coordinates with the largest coordinate distance in the defect contour coordinate set;
in step S530, the pixel coordinates of the defect position are determined according to the two selected defect contour coordinates.
According to some embodiments of the present invention, the method for locating a vehicle defect according to the embodiment of the present invention further includes the steps of:
step S610, determining defect category according to the defect contour coordinate set;
step S620, when the defect type is a block defect, determining the radius of the circumscribed circle and the center coordinates of the circumscribed circle of the block defect according to the defect contour coordinate set, and controlling the position calibration device to perform fixed-point spraying according to the radius of the circumscribed circle and the center coordinates of the circumscribed circle;
and step S630, when the defect type is a strip defect, determining a spraying path according to the defect contour coordinate set, and performing mobile spraying according to the spraying path.
It can be understood that the foregoing embodiments of the defect positioning system of the vehicle are applicable to the embodiment of the defect positioning method of the vehicle, and the specific functions of the embodiment of the defect positioning method of the vehicle are the same as those of the embodiment of the defect positioning system of the vehicle, and the beneficial effects achieved by the embodiment of the defect positioning method of the vehicle are the same as those achieved by the embodiment of the defect positioning method of the vehicle.
Referring to fig. 2, fig. 2 is a schematic view of a defect positioning apparatus for a vehicle according to an embodiment of the present invention. The defect positioning device for a vehicle according to an embodiment of the present invention includes one or more control processors and a memory, and fig. 2 illustrates one control processor and one memory as an example.
The control processor and the memory may be connected by a bus or otherwise, for example in fig. 2.
The memory, as a non-transitory computer readable storage medium, may be used to store non-transitory software programs as well as non-transitory computer executable programs. In addition, the memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one magnetic disk storage device, flash memory device, or other non-transitory solid state storage device. In some embodiments, the memory optionally includes memory remotely located relative to the control processor, the remote memory being connectable to the vehicle defect localization apparatus via a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
It will be appreciated by those skilled in the art that the device configuration shown in FIG. 2 is not limiting of the vehicle defect locating device and may include more or fewer components than shown, or certain components may be combined, or a different arrangement of components.
The non-transitory software program and instructions required to implement the vehicle defect localization method applied to the vehicle defect localization apparatus in the above-described embodiments are stored in the memory, and when executed by the control processor, the vehicle defect localization method applied to the vehicle defect localization apparatus in the above-described embodiments is executed.
In addition, an embodiment of the present invention further provides a computer readable storage medium storing computer executable instructions that are executed by one or more control processors to cause the one or more control processors to perform the method for locating a vehicle defect in the method embodiment.
Those of ordinary skill in the art will appreciate that all or some of the steps, systems, and methods disclosed above may be implemented as software, firmware, hardware, and suitable combinations thereof. Some or all of the physical components may be implemented as software executed by a processor, such as a central processing unit, digital signal processor, or microprocessor, or as hardware, or as an integrated circuit, such as an application specific integrated circuit. Such software may be distributed on computer readable media, which may include computer storage media (or non-transitory media) and communication media (or transitory media). The term computer storage media includes both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data, as known to those skilled in the art. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital Versatile Disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information and which can be accessed by a computer. Furthermore, as is well known to those of ordinary skill in the art, communication media typically embodies computer readable instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transport mechanism and includes any information delivery media.
The embodiments of the present invention have been described in detail with reference to the accompanying drawings, but the present invention is not limited to the above embodiments, and various changes can be made within the knowledge of one of ordinary skill in the art without departing from the spirit of the present invention.
Claims (6)
1. The defect positioning system of the vehicle is characterized by comprising an image acquisition device, a data processing device and a position calibration device;
the image acquisition device is used for shooting an image of the vehicle body;
the data processing device is used for inputting the vehicle body image into a defect detection model to obtain a defect contour coordinate set; obtaining a defect position pixel coordinate and a defect category under a pixel coordinate system according to the defect contour coordinate set, and converting the defect position pixel coordinate into a defect position space coordinate under a machine coordinate system;
the position calibration device is used for marking on a vehicle body according to the space coordinates of the defect position, wherein the data processing device is also used for determining the radius of an circumscribed circle and the coordinates of the center of the circumscribed circle of the block defect according to the outline coordinates set of the defect when the defect type is the block defect, controlling the position calibration device to perform fixed-point spraying according to the radius of the circumscribed circle and the coordinates of the center of the circumscribed circle, enabling the position calibration device to perform static fixed-point spraying according to the space coordinates of the obtained defect position, and adjusting the spraying range to be the same as the radius of the circumscribed circle of the actual defect by opening or closing a spray nozzle through hole according to the radius of the circumscribed circle of the block defect; when the defect type is a strip defect, determining a spraying path according to the defect contour coordinate set, and performing mobile spraying according to the spraying path;
the image acquisition device comprises a plurality of cameras, the position calibration device comprises a plurality of spraying manipulators, and the data processing device stores two-dimensional coordinate conversion matrixes corresponding to the cameras and the spraying manipulators one by one;
the data processing apparatus determines the defect location spatial coordinates by:
acquiring a first number and a camera attitude parameter of a camera shooting the car body image;
determining a second number of the spraying manipulator to perform spraying action;
inquiring a two-dimensional coordinate transformation matrix according to the first number and the second number;
determining a defect position plane coordinate according to the two-dimensional coordinate conversion matrix and the defect position pixel coordinate;
determining a defect position space coordinate according to the camera attitude parameter and the defect position plane coordinate;
the camera attitude parameters comprise shooting distance and shooting angle of a camera and a plane of a car body, and a defect position height coordinate z is determined according to the shooting distance and the shooting angle and is shown in the following formula:
z＝cosθ*D；
wherein θ represents a photographing angle of the selected camera, and D represents a photographing distance of the selected camera;
and determining the space coordinates of the defect position according to the height coordinates of the defect position and the plane coordinates of the defect position.
2. The vehicle defect localization system of claim 1, wherein the two-dimensional coordinate transformation matrix is obtained by:
selecting a group of cameras and spraying manipulators;
collecting a test image through the camera, and calibrating pixel coordinates of nine test points on the test image;
controlling the spraying manipulator to correspondingly move to the positions of the test points, and acquiring the space positions of nine test points;
determining a rotation quantity matrix and a translation quantity matrix according to the pixel coordinates and the space coordinates of the test points;
and determining a two-dimensional coordinate conversion matrix of each group of cameras and the spraying manipulator according to the rotation amount matrix and the translation amount matrix.
3. The vehicle defect localization system of claim 1, wherein the data processing device is further configured to:
selecting two defect contour coordinates with the largest coordinate distance in the defect contour coordinate set;
and determining the pixel coordinates of the defect position according to the two selected defect contour coordinates.
4. A vehicle defect positioning method is characterized in that the vehicle defect positioning method is applied to a data processing device of the vehicle defect positioning system according to claim 1, wherein the data processing device stores a two-dimensional coordinate conversion matrix corresponding to cameras and spraying manipulators one by one; the vehicle defect positioning method comprises the following steps:
shooting a car body image through an image acquisition device;
inputting the vehicle body image into a defect detection model to obtain a defect contour coordinate set; obtaining a defect position pixel coordinate and a defect category under a pixel coordinate system according to the defect contour coordinate set;
acquiring a first number and a camera attitude parameter of a camera shooting the car body image;
determining a second number of the spraying manipulator to perform spraying action;
inquiring a two-dimensional coordinate transformation matrix according to the first number and the second number;
determining a defect position plane coordinate according to the two-dimensional coordinate conversion matrix and the defect position pixel coordinate;
determining a defect position space coordinate according to the camera attitude parameter and the defect position plane coordinate;
controlling the position calibration device to mark the vehicle body according to the space coordinates of the defect position, wherein when the defect type is a block defect, determining the radius of an circumscribed circle and the coordinates of the center of the circumscribed circle of the block defect according to the defect contour coordinate set, controlling the position calibration device to perform fixed-point spraying according to the radius of the circumscribed circle and the coordinates of the center of the circumscribed circle, enabling the position calibration device to perform static fixed-point spraying according to the space coordinates of the obtained defect position, and opening or closing a nozzle through hole according to the radius of the circumscribed circle of the block defect so as to adjust the spraying range to be the same as the radius of the circumscribed circle of the actual defect; when the defect type is a strip defect, determining a spraying path according to the defect contour coordinate set, and performing mobile spraying according to the spraying path;
the camera attitude parameters comprise shooting distance and shooting angle of a camera and a plane of a car body, and a defect position height coordinate z is determined according to the shooting distance and the shooting angle and is shown in the following formula:
z＝cosθ*D；
wherein θ represents a photographing angle of the selected camera, and D represents a photographing distance of the selected camera;
and determining the space coordinates of the defect position according to the height coordinates of the defect position and the plane coordinates of the defect position.
5. A vehicle defect positioning apparatus, characterized by comprising:
at least one processor;
at least one memory for storing at least one program;
the at least one program, when executed by the at least one processor, causes the at least one processor to implement the vehicle defect localization method of claim 4.
6. A computer-readable storage medium in which a processor-executable program is stored, characterized in that the processor-executable program is for implementing the vehicle defect localization method as claimed in claim 4 when being executed by the processor.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211694317.0A CN116148259B (en) | 2022-12-28 | 2022-12-28 | Vehicle defect positioning system, method, device and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211694317.0A CN116148259B (en) | 2022-12-28 | 2022-12-28 | Vehicle defect positioning system, method, device and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN116148259A CN116148259A (en) | 2023-05-23 |
| CN116148259B true CN116148259B (en) | 2024-03-22 |
Family
ID=86355466
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211694317.0A Active CN116148259B (en) | 2022-12-28 | 2022-12-28 | Vehicle defect positioning system, method, device and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116148259B (en) |
Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107588732A (en) * | 2016-07-07 | 2018-01-16 | 苏州华兴致远电子科技有限公司 | Rail side Train Parts height measurement method and system |
| FR3069691A1 (en) * | 2017-07-25 | 2019-02-01 | Safran | METHOD FOR DETERMINING THE POSITION AND ORIENTATION OF A MATRIX OPTICAL SENSOR IN RELATION TO A CARRIER ROBOT |
| CN110609037A (en) * | 2019-07-12 | 2019-12-24 | 北京旷视科技有限公司 | Product defect detection system and method |
| CN110634140A (en) * | 2019-09-30 | 2019-12-31 | 南京工业大学 | Large-diameter tubular object positioning and inner wall defect detection method based on machine vision |
| CN112013830A (en) * | 2020-08-20 | 2020-12-01 | 中国电建集团贵州电力设计研究院有限公司 | Accurate positioning method for unmanned aerial vehicle inspection image detection defects of power transmission line |
| CN113021328A (en) * | 2019-12-09 | 2021-06-25 | 广东博智林机器人有限公司 | Hand-eye calibration method, device, equipment and medium |
| CN113927165A (en) * | 2021-10-20 | 2022-01-14 | 中北大学 | Rapid positioning and repairing method and system for robot wire filling laser cladding defects |
| CN114290338A (en) * | 2022-03-03 | 2022-04-08 | 龙旗电子(惠州)有限公司 | Two-dimensional hand-eye calibration method, device, storage medium, and program product |
| CN114720475A (en) * | 2022-03-24 | 2022-07-08 | 湖南视比特机器人有限公司 | Intelligent detection and polishing system and method for automobile body paint surface defects |
| CN114882108A (en) * | 2022-04-13 | 2022-08-09 | 广东工业大学 | Method for estimating grabbing pose of automobile engine cover under two-dimensional image |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111815719B (en) * | 2020-07-20 | 2023-12-22 | 阿波罗智能技术(北京)有限公司 | External parameter calibration method, device and equipment of image acquisition equipment and storage medium |
-
2022
- 2022-12-28 CN CN202211694317.0A patent/CN116148259B/en active Active
Patent Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107588732A (en) * | 2016-07-07 | 2018-01-16 | 苏州华兴致远电子科技有限公司 | Rail side Train Parts height measurement method and system |
| FR3069691A1 (en) * | 2017-07-25 | 2019-02-01 | Safran | METHOD FOR DETERMINING THE POSITION AND ORIENTATION OF A MATRIX OPTICAL SENSOR IN RELATION TO A CARRIER ROBOT |
| CN110609037A (en) * | 2019-07-12 | 2019-12-24 | 北京旷视科技有限公司 | Product defect detection system and method |
| CN110634140A (en) * | 2019-09-30 | 2019-12-31 | 南京工业大学 | Large-diameter tubular object positioning and inner wall defect detection method based on machine vision |
| CN113021328A (en) * | 2019-12-09 | 2021-06-25 | 广东博智林机器人有限公司 | Hand-eye calibration method, device, equipment and medium |
| CN112013830A (en) * | 2020-08-20 | 2020-12-01 | 中国电建集团贵州电力设计研究院有限公司 | Accurate positioning method for unmanned aerial vehicle inspection image detection defects of power transmission line |
| CN113927165A (en) * | 2021-10-20 | 2022-01-14 | 中北大学 | Rapid positioning and repairing method and system for robot wire filling laser cladding defects |
| CN114290338A (en) * | 2022-03-03 | 2022-04-08 | 龙旗电子(惠州)有限公司 | Two-dimensional hand-eye calibration method, device, storage medium, and program product |
| CN114720475A (en) * | 2022-03-24 | 2022-07-08 | 湖南视比特机器人有限公司 | Intelligent detection and polishing system and method for automobile body paint surface defects |
| CN114882108A (en) * | 2022-04-13 | 2022-08-09 | 广东工业大学 | Method for estimating grabbing pose of automobile engine cover under two-dimensional image |
Non-Patent Citations (1)
| Title |
|---|
| 蔡伟维.宝来轿车故障检修图解.四川科学技术出版社,2005,336. * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN116148259A (en) | 2023-05-23 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106780623B (en) | Rapid calibration method for robot vision system | |
| CN108399639A (en) | Fast automatic crawl based on deep learning and arrangement method | |
| CN114549519B (en) | Visual detection method and system for automobile spraying production line and readable storage medium | |
| CN104942496B (en) | Car body-in-white welding spot positioning method and device based on robot visual servo | |
| CN101358836A (en) | Method for recognising weld spot central position based on computer vision | |
| CN108748149B (en) | Non-calibration mechanical arm grabbing method based on deep learning in complex environment | |
| US20220230348A1 (en) | Method and apparatus for determining a three-dimensional position and pose of a fiducial marker | |
| CN110463376B (en) | Machine plugging method and machine plugging equipment | |
| CN114720475A (en) | Intelligent detection and polishing system and method for automobile body paint surface defects | |
| CN114714355B (en) | Embedded vision tracking control system of autonomous mobile welding robot | |
| CN113276106A (en) | Climbing robot space positioning method and space positioning system | |
| CN113021328A (en) | Hand-eye calibration method, device, equipment and medium | |
| CN111767780A (en) | AI and vision combined intelligent hub positioning method and system | |
| CN114299039B (en) | Robot and collision detection device and method thereof | |
| CN113280209B (en) | System for detecting pipeline excess, use method of system and detection method | |
| CN113245094B (en) | Robot spraying system and method for automobile brake drum | |
| CN113601501B (en) | Flexible operation method and device for robot and robot | |
| CN116148259B (en) | Vehicle defect positioning system, method, device and storage medium | |
| CN113378701A (en) | Ground multi-AGV state monitoring method based on unmanned aerial vehicle | |
| CN112819894A (en) | Collecting card guiding and positioning system and method, computer storage medium and gantry crane | |
| CN116276938B (en) | Mechanical arm positioning error compensation method and device based on multi-zero visual guidance | |
| Gao et al. | An automatic assembling system for sealing rings based on machine vision | |
| CN112584041B (en) | Image identification dynamic deviation rectifying method | |
| US12002196B2 (en) | Electronic device and method for marking defects of products | |
| CN109186554B (en) | Method for automatically positioning coordinates of scene in real-time video fixed track inspection |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |