CN111414912B - Method, device and equipment for identifying characteristic points of butt-joint type welding seam and storage medium - Google Patents
Method, device and equipment for identifying characteristic points of butt-joint type welding seam and storage medium Download PDFInfo
- Publication number
- CN111414912B CN111414912B CN202010247486.4A CN202010247486A CN111414912B CN 111414912 B CN111414912 B CN 111414912B CN 202010247486 A CN202010247486 A CN 202010247486A CN 111414912 B CN111414912 B CN 111414912B
- Authority
- CN
- China
- Prior art keywords
- region
- interest
- line segment
- binarization
- weld
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/25—Determination of region of interest [ROI] or a volume of interest [VOI]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0004—Industrial image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/11—Region-based segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/136—Segmentation; Edge detection involving thresholding
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/28—Quantising the image, e.g. histogram thresholding for discrimination between background and foreground patterns
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10004—Still image; Photographic image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30108—Industrial image inspection
- G06T2207/30152—Solder
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Multimedia (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Quality & Reliability (AREA)
- Image Analysis (AREA)
Abstract
The embodiment of the invention discloses a method, a device, equipment and a storage medium for identifying characteristic points of butt-joint type welding seams. The method comprises the following steps: acquiring a frame of weld image, and acquiring a first region of interest and a second region of interest in the weld image, wherein the region ranges of the first region of interest and the second region of interest are determined according to SURF characteristic points in the first frame of weld image, the first region of interest indicates a workpiece surface stripe region, and the second region of interest indicates a weld bottom stripe region; respectively carrying out binarization segmentation processing on the first region of interest and the second region of interest to obtain a first binarization region of interest and a second binarization region of interest; and determining two weld characteristic points in the first binarization interested area, and determining two weld characteristic points in the second binarization interested area. By the technical scheme, various interferences existing in field application can be reduced or eliminated, and the stability of extraction of the weld characteristic points is improved.
Description
Technical Field
The embodiment of the invention relates to the technical field of image recognition, in particular to a method, a device, equipment and a storage medium for recognizing characteristic points of butt-joint type welding seams.
Background
With the rapid development of industrial automation, welding automation is also widely accepted and applied by industrial fields, especially in the welding of large steel structure equipment meeting high difficulty and high requirements.
The welding automation is realized, the automatic tracking of the welding seam is a key, and the most key technical problem for realizing the welding seam tracking is to realize the automatic identification of the welding seam, namely the extraction of the characteristic point of the welding seam. In field application, various interferences such as arc light, splashing, metal reflection and the like exist, extraction of characteristic points of a welding seam is seriously influenced, so that the welding tracking effect is poor, and how to reduce or eliminate the interferences is always a problem to be solved in the field of welding seam tracking.
Disclosure of Invention
The embodiment of the invention provides a method, a device, equipment and a storage medium for identifying characteristic points of butt-joint type welding seams, which are used for reducing or eliminating various interferences existing in field application and improving the stability of extraction of the characteristic points of the welding seams.
In a first aspect, an embodiment of the present invention provides a method for identifying a characteristic point of a butt-joint weld, including:
acquiring a frame of welding seam image, and acquiring a first interested area and a second interested area in the welding seam image; wherein the region ranges of the first region of interest and the second region of interest are determined according to SURF (speed Up Robust feature) feature points in a first frame of weld image, the first region of interest indicates a workpiece surface stripe region, and the second region of interest indicates a weld bottom stripe region;
respectively carrying out binarization segmentation processing on the first region of interest and the second region of interest to obtain a first binarization region of interest and a second binarization region of interest;
and determining two weld characteristic points in the first binarization interested area, and determining two weld characteristic points in the second binarization interested area.
In a second aspect, an embodiment of the present invention further provides an apparatus for identifying a characteristic point of a butt-joint weld, including:
the interesting area acquisition module is used for acquiring a frame of welding seam image and acquiring a first interesting area and a second interesting area in the welding seam image; wherein the area ranges of the first region of interest and the second region of interest are determined according to SURF feature points in a first frame of weld image, the first region of interest indicates a workpiece surface stripe area, and the second region of interest indicates a weld bottom stripe area;
the binarization segmentation processing module is used for respectively carrying out binarization segmentation processing on the first region of interest and the second region of interest to obtain a first binarization region of interest and a second binarization region of interest;
and the weld characteristic point identification module is used for determining two weld characteristic points in the first binarization interested area and determining two weld characteristic points in the second binarization interested area.
In a third aspect, an embodiment of the present invention further provides a computer device, including a memory, a processor, and a computer program stored on the memory and executable on the processor, where the processor executes the computer program to implement the method for identifying the characteristic points of the butt-joint weld according to any embodiment of the present invention.
In a fourth aspect, the embodiments of the present invention further provide a computer-readable storage medium, on which a computer program is stored, where the computer program, when executed by a processor, implements the method for identifying characteristic points of a butt-joint weld according to any embodiment of the present invention.
According to the identification method, the device, the equipment and the storage medium of the butt-joint type weld characteristic point provided by the embodiment of the invention, the first region of interest and the second region of interest determined according to the SURF characteristic point in the first frame of weld image are firstly obtained in the obtained weld image, then the first region of interest and the second region of interest are respectively subjected to binary segmentation processing, and finally the weld characteristic point is respectively identified in the first region of interest and the second region of interest after the binary segmentation processing, so that various interferences existing in field application, such as various interferences of arc light, splashing, metal reflection and the like, can be reduced or eliminated, and the stability of extracting the weld characteristic point is improved.
Drawings
Fig. 1 is a flowchart of a method for identifying characteristic points of a butt-joint weld according to a first embodiment of the present invention;
FIG. 2 is an example of SURF feature points detected in the first embodiment of the present invention;
FIG. 3 is a flowchart of a method for identifying characteristic points of a butt-joint weld according to a second embodiment of the present invention;
FIG. 4 is an example of a first region of interest and a second region of interest in a second embodiment of the present invention;
fig. 5 is a schematic structural diagram of an identification device for characteristic points of a butt-joint type weld joint in a third embodiment of the present invention;
fig. 6 is a schematic structural diagram of a computer device in the fourth embodiment of the present invention.
Detailed Description
The present invention will be described in further detail with reference to the accompanying drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the invention and are not limiting of the invention. It should be further noted that, for the convenience of description, only some of the structures related to the present invention are shown in the drawings, not all of the structures.
Before discussing exemplary embodiments in more detail, it should be noted that some exemplary embodiments are described as processes or methods depicted as flowcharts. Although a flowchart may describe the operations (or steps) as a sequential process, many of the operations can be performed in parallel, concurrently or simultaneously. In addition, the order of the operations may be re-arranged. The process may be terminated when its operations are completed, but may have additional steps not included in the figure. The processes may correspond to methods, functions, procedures, subroutines, and the like.
Example one
Fig. 1 is a flowchart of a method for identifying a characteristic point of a butt-joint weld according to an embodiment of the present invention, which may be implemented by an apparatus for identifying a characteristic point of a butt-joint weld according to an embodiment of the present invention, where the apparatus may be implemented in software and/or hardware, and may be generally integrated into a weld image acquisition device (including a processor), where the apparatus may be applied to a welding field application where various interferences, such as arc light, spatter, and metal reflection, may occur, and thus an error may occur in the identification of a characteristic point of a butt-joint (e.g., a V-shaped opening) weld.
As shown in fig. 1, the method of this embodiment specifically includes:
s110, acquiring a frame of welding seam image, and acquiring a first interested area and a second interested area in the welding seam image; wherein the area ranges of the first region of interest and the second region of interest are determined according to SURF feature points in a first frame of weld image, the first region of interest indicates a workpiece surface stripe area, and the second region of interest indicates a weld bottom stripe area.
The welding seam image refers to an image of a welding seam acquired by welding seam image acquisition equipment, and the welding seam image comprises laser stripes. The weld seam referred to in the present embodiment generally refers to a butt weld seam, and the butt weld seam refers to a weld seam welded between a bevel face of a weldment or between a bevel face of a weldment and an end (surface) face of another weldment, for example, a weld seam with a V-shaped groove.
Region of Interest (ROI), refers to the image area near the weld. The purpose of selecting the region of interest is to select only the image region near the weld for processing, so that the algorithm processing speed can be improved, and a lot of interference signals such as arc light, splash, metal reflection and the like can be reduced.
Specifically, the region of interest may be divided according to the distribution characteristics of the laser stripes in the butt-joint type weld image, that is, the region of interest is divided into a workpiece surface stripe region and a weld bottom stripe region.
The first region of interest and the second region of interest in each frame of weld image are determined according to SURF feature points detected in the first frame of weld image, that is, after the first frame of weld image is obtained, SURF feature points in the weld image are detected, and the region of interest is determined according to SURF feature point distribution. Generally, SURF feature points included in the workpiece surface stripe region are more than SURF feature points included in the weld bottom stripe region, and thus one region of interest including more SURF feature points may be used as a first region of interest, that is, a workpiece surface stripe region, and one region of interest including less SURF feature points may be used as a second region of interest, that is, a weld bottom stripe region.
The SURF is an accelerated version of SIFT (Scale-invariant feature transform), which is a description used in the field of image processing, has Scale invariance, and can detect key points in an image. SURF can be used for object localization and recognition, face recognition, 3D reconstruction, object tracking, and extracting points of interest, etc., and is commonly used for object recognition and image matching.
SURF feature points refer to points in the image that have contrast with the surroundings, such as corner points, boundary points, dark points in the bright place, bright points in the dark place, and so on. In the present embodiment, the SURF feature points specifically refer to bright points (with a gray value close to 255) in a dark place (with a gray value close to zero) in the weld image. Fig. 2 shows an example of detected SURF feature points.
Further, taking as an example that each of the regions of interest is a horizontal region or a quasi-horizontal region (i.e. a region close to horizontal), when the first region of interest and the second region of interest are divided according to SURF feature points, the division may be performed according to ordinate of the SURF feature points, and ordinate values of the SURF feature points included in each region of interest are close (or within a certain numerical range), for example, ordinate values of SURF feature points detected on the surface of the workpiece are close, and ordinate values of SURF feature points detected on the bottom of the weld are close. It is noted that the ordinate is for the laser stripe level or approximately level in the weld image, and when the laser stripe is vertical or approximately vertical in the weld image, the first region of interest and the second region of interest may be divided according to the abscissa value of the SURF feature point.
After the area ranges of the first region of interest and the second region of interest are determined according to the SURF characteristic points in the first frame of weld image, the area ranges of the two regions of interest are stored for each frame of subsequently acquired weld image.
Furthermore, after a frame of weld image is obtained and before a first region of interest and a second region of interest are obtained in the weld image, median filtering processing can be performed on the weld image, so that pulse noise in the weld image can be effectively filtered.
And S120, performing binarization segmentation processing on the first region of interest and the second region of interest respectively to obtain a first binarization region of interest and a second binarization region of interest.
The image binarization is a process of setting the gray value of a pixel point on an image to be 0 or 255, namely, the whole image presents an obvious black and white effect.
In an example, the first region of interest and the second region of interest may be subjected to binarization segmentation processing, so as to obtain a first binarization region of interest and a second binarization region of interest, specifically: respectively calculating a first binarization segmentation threshold value and a second binarization segmentation threshold value of the first region of interest and the second region of interest; carrying out binarization segmentation processing on the first region of interest according to the first binarization segmentation threshold value to obtain a first binarization region of interest; and performing binarization segmentation processing on the second region of interest according to the second binarization segmentation threshold value to obtain a second binarization region of interest.
Explaining by taking an interested area as an example, firstly, a binary segmentation threshold value of the interested area is calculated, and the steps are as follows:
A1) calculating an average gray value T of the region of interest, and taking the T as a segmentation threshold;
B1) dividing the region of interest into two portions according to a division threshold T, namely a portion R1 having a gray value greater than T and a portion R2 having a gray value less than T;
C1) calculating average gray values m1 and m2 of R1 and R2, respectively;
D1) setting the segmentation threshold T to (m1+ m 2)/2;
E1) and repeating the steps B1) to D1) until the difference value between the T values of the continuous iteration is less than a certain set value, wherein the T value is the binarization segmentation threshold value of the region of interest.
Then, performing binarization segmentation processing on the region of interest according to a binarization segmentation threshold value T obtained by calculation of the region of interest, obtaining pixel points with gray values lower than the binarization segmentation threshold value T, setting the gray values of the pixel points to be 0, obtaining pixel points with gray values higher than the binarization segmentation threshold value T, and setting the gray values of the pixel points to be 255, so as to obtain a binarization region of interest.
Furthermore, the binarization interested area can be subjected to corrosion first and then expansion treatment, so that the effect of repairing the structural light fracture is achieved.
S130, determining two weld joint feature points in the first binarization interested area, and determining two weld joint feature points in the second binarization interested area.
After the first binarization interested area and the second binarization interested area are obtained, the weld characteristic points are respectively identified in the two binarization interested areas. Specifically, different technical means can be adopted to identify two weld characteristic points included in the first binarization interested area and the second binarization interested area according to the distribution of the high-brightness pixel points in the first binarization interested area and the second binarization interested area.
According to the technical scheme provided by the embodiment of the invention, the first region of interest and the second region of interest determined according to the SURF characteristic points in the first frame of weld image are firstly obtained in the obtained weld image, then the first region of interest and the second region of interest are respectively subjected to binarization segmentation processing, and finally the weld characteristic points are respectively identified in the first region of interest and the second region of interest after the binarization segmentation processing, so that various interferences existing in field application, such as arc light, splash, metal reflection and other interferences, can be reduced or eliminated, and the stability of weld characteristic point extraction is improved.
Example two
Fig. 3 is a flowchart of a method for identifying characteristic points of a butt-joint weld according to a second embodiment of the present invention. On the basis of the above technical solution, the embodiment embodies a dividing manner of the first region of interest and the second region of interest, and includes: after a first frame of welding seam image is obtained, SURF characteristic points in the first frame of welding seam image are detected; dividing the SURF characteristic points into first type characteristic points and second type characteristic points according to the longitudinal coordinate values of the SURF characteristic points, wherein the first type characteristic points indicate characteristic points in a stripe region on the surface of the workpiece, and the second type characteristic points indicate characteristic points in a stripe region at the bottom of the welding seam; determining a first region of interest according to the distribution of the first type of feature points; and determining a second region of interest according to the distribution of the second type of feature points.
As shown in fig. 3, the method of this embodiment specifically includes:
s210, obtaining a frame of welding seam image, and carrying out median filtering on the welding seam image.
And S220, judging whether the welding seam image is a first frame welding seam image, if so, executing S230, and if not, executing S270.
And S230, detecting SURF characteristic points in the first frame of welding seam image.
S240, dividing the SURF characteristic points into first-class characteristic points and second-class characteristic points according to the longitudinal coordinate values of the SURF characteristic points, wherein the first-class characteristic points indicate the characteristic points in the stripe region on the surface of the workpiece, and the second-class characteristic points indicate the characteristic points in the stripe region at the bottom of the welding seam.
In general, the number of feature points of the first type is greater than the number of feature points of the second type.
Taking SURF feature points shown in fig. 2 as an example, the ordinate values of the SURF feature points are obtained (the origin of the coordinate system may be arbitrarily selected), and assuming that the number of SURF feature points is N, the ordinate values { y ] of the SURF feature points are obtained according to the SURF feature points1,y2,…,yNDivide the SURF feature points into two classes (assume C)1Class and C2Class), the specific method is as follows:
A2) initialization: setting two classes { C1,C2Taking two SURF feature points from all SURF feature points at will, and taking longitudinal coordinate values of the two SURF feature points as C1Class and C2Mean value u of class1And u2；
B2) Separately calculate lambdak＝|yi-uk1, N, k is 1,2, and dividing all SURF feature points into classes C1 and C2 according to the calculation result;
wherein, for the ith SURF feature point, the ordinate value is yiIf λ1＜λ2Distance C of ith SURF feature point1Class is closer, mark the ith SURF feature point as belonging to C1Class, if λ1＞λ2Distance C of ith SURF feature point2Near class is toThe ith SURF feature point label belongs to C2Class;
C2) calculating C1Class and C2Current mean of class
And
k＝1,2，
is CkThe number of SURF feature points included in the class;
D2) judgment of
And { u1,u2Whether the differences are consistent or not can be completely consistent, if so, stopping iteration and exiting to obtain SURF feature points in the C1 class and SURF feature points in the C2 class, and if not, then judging whether the differences are small or not
k is 1,2, return execution B2).
Assuming that SURF feature points in the C1 class are more than SURF feature points in the C2 class, SURF feature points in the C1 class are feature points of the first class, and SURF feature points in the C2 class are feature points of the second class.
And S250, determining a first region of interest according to the distribution of the first type of characteristic points.
As a specific implementation manner of this embodiment, a first region of interest may be determined according to the distribution of the first-class feature points, specifically:
selecting a first class of target feature points with the gray value smaller than a set gray threshold value from the first class of feature points; fitting a first target straight line according to the coordinate values of the first type of target feature points; and taking the first target straight line as a center, and taking a first preset pixel range as the first region of interest.
Obtaining the first kind of characteristic points and deleting the gray in the first kind of characteristic pointsFeature points with the value smaller than a set gray threshold value are obtained, a set Z of the remaining feature points in the first class of feature points is obtained, and a straight line y k is fitted according to SURF feature points in the set Z0x+b0. In one example, the line y-k0x+b0The fitting method of (1) is as follows:
A3) initializing k and b (k is 0, b is 0), setting a distance threshold value sigma, initializing a number threshold value p (p is 0), and setting a total iteration number t;
B3) randomly selecting two feature points in the set Z, wherein the coordinates are x1, y1 and x2, y 2;
C3) determining fitting straight line parameters k and b according to { x1, y1} and { x2, y2}, wherein:
k＝(y1-y2)/(x1-x2),b＝y1-k*x1；
D3) calculating the distance from each feature point in the set Z to the straight line y which is kx + b and recording as dm;
E3) counting the number s of all feature points with dm smaller than a distance threshold value sigma;
F3) if s is larger than p, p is s, and { k0, b0} - { k, b }, continuing to execute G3), otherwise, executing G3);
G3) and (4) determining whether the iteration number i is i +1, if i is less than t, returning to execute B3), and otherwise, exiting the loop.
After determining and fitting the straight line y-k0x+b0Then, the first predetermined pixel range is determined as the first region of interest by taking the straight line as a central boundary, for example, the first predetermined pixel range may be defined by taking the straight line y ═ k0x+b0For the boundary, the area ranges of the first predetermined number of pixels (e.g. 25 pixels) are counted up and down, respectively, as shown in area 1 in fig. 4.
S260, determining a second region of interest according to the distribution of the second type of feature points, and executing S280.
As a specific implementation manner of this embodiment, a second region of interest may be determined according to the distribution of the second type of feature points, specifically: acquiring the mean value of the vertical coordinates of the second type of feature points, and determining a second target straight line according to the mean value of the vertical coordinates; and taking the second target straight line as a center, and taking a second preset pixel range as the second region of interest.
The mean value of the ordinate of the second class of feature points is the mean value u of the above-mentioned C2 class2According to said mean value u of ordinate2Determining a second target straight line, i.e. y ═ u2. The second target straight line is used as a center, and a second preset pixel range is determined as a second region of interest, for example, the second preset pixel range may be defined by a straight line y-u2For the boundary, the area ranges of the second predetermined number of pixels (e.g. 20 pixels) are counted up and down, respectively, as shown in area 2 in fig. 4.
Further, the ranges of the first interested region and the second interested region determined according to the first frame of weld image are saved, so that the subsequently acquired weld image can determine the corresponding first interested region and the second interested region.
S270, acquiring a first region of interest and a second region of interest in the weld image, and executing S280.
For the weld image of the non-first frame, when the first interested region and the second interested region are determined, the range of the first interested region and the second interested region determined according to the weld image of the first frame is used as the basis.
S280, performing binarization segmentation processing on the first region of interest and the second region of interest respectively to obtain a first binarization region of interest and a second binarization region of interest.
And S290, determining two weld joint characteristic points in the first binarization interested area.
As an optional implementation manner of this embodiment, two weld feature points may be determined in the first binarized region of interest, specifically: calculating a straight line segment in the first binarization interested area, and dividing the straight line segment into a left line segment set and a right line segment set; respectively sequencing the left line segment set and the right line segment set to determine a first line segment and a second line segment; the first line segment is the longest single-pixel line segment in the left line segment set, and the second line segment is the longest single-pixel line segment in the right line segment set; and taking the right end point of the first line segment and the left end point of the second line segment as weld joint feature points.
The first binarization interested area is a binarization bevel area, the laser stripe in the area is composed of a left part and a right part, and a weld characteristic point can be determined in each part.
Typically, huffman's variation can be used to compute all the straight line segments in the first binarized region of interest and divide these into left and right sets of line segments. And then sorting the line segments in each line segment set according to length, and finding the longest single-pixel line segment (namely, a first line segment) in the left line segment set and the longest single-pixel line segment (namely, a second line segment) in the right line segment set. And aiming at the longest single pixel line segment in the left line segment set, the right end point of the single pixel line segment is the weld joint characteristic point, and aiming at the longest single pixel line segment in the right line segment set, the left end point of the single pixel line segment is the weld joint characteristic point.
S2100, determining two weld joint feature points in the second binarization interested region.
The second binarization interested area is a binarization welding seam bottom area, the laser stripe in the area is composed of a part, and a welding seam characteristic point can be respectively determined at two ends of the part.
As an optional implementation manner of this embodiment, two weld feature points may be determined in the second binarization region of interest, specifically: calculating a single-pixel central line segment of the laser stripe in the second binarization interested area; and taking two end points of the single-pixel central line segment as the weld characteristic points.
Typically, a Hessian matrix method can be used to calculate a single-pixel central line segment of the laser stripe, and two left and right end points of the line segment are found, where the two end points are weld characteristic points in the second binarization region of interest.
For those parts of the present embodiment that are not explained in detail, please refer to the previous embodiments, and further description is omitted here.
In the technical scheme, the region of interest is divided into the first region of interest and the second region of interest according to the distribution of the laser stripes in the weld image, different binarization segmentation threshold values are adopted for different regions of interest to carry out image segmentation, and different modes are adopted for different binarization regions of interest to carry out weld characteristic point identification, so that various interferences existing in field application, such as arc light, splashing, metal reflection and other interferences, are effectively reduced or eliminated, and the stability of weld characteristic point extraction is further improved.
EXAMPLE III
Fig. 5 is a schematic structural diagram of an apparatus for identifying characteristic points of a butt-joint type weld according to a third embodiment of the present invention, which is applicable to a situation where there are various interferences such as arc light, spatter, and metal reflection in a welding field application, so that an error may occur in the identification of characteristic points of the weld, and the apparatus may be implemented in software and/or hardware, and may be generally integrated in a weld image acquisition device (including a processor).
As shown in fig. 5, the identification device for characteristic points of butt-joint type welding seams specifically includes: a region of interest acquisition module 310, a binarization segmentation processing module 320 and a weld feature point identification module 330. Wherein the content of the first and second substances,
a region-of-interest obtaining module 310, configured to obtain a frame of weld image, and obtain a first region of interest and a second region of interest in the weld image; wherein the area ranges of the first region of interest and the second region of interest are determined according to SURF feature points in a first frame of weld image, the first region of interest indicates a workpiece surface stripe area, and the second region of interest indicates a weld bottom stripe area;
a binarization segmentation processing module 320, configured to perform binarization segmentation processing on the first region of interest and the second region of interest respectively to obtain a first binarization region of interest and a second binarization region of interest;
and the weld feature point identification module 330 is configured to determine two weld feature points in the first binarized region of interest, and determine two weld feature points in the second binarized region of interest.
According to the technical scheme provided by the embodiment of the invention, the first region of interest and the second region of interest determined according to the SURF characteristic points in the first frame of weld image are firstly obtained in the obtained weld image, then the first region of interest and the second region of interest are respectively subjected to binarization segmentation processing, and finally the weld characteristic points are respectively identified in the first region of interest and the second region of interest after the binarization segmentation processing, so that various interferences existing in field application, such as arc light, splash, metal reflection and other interferences, can be reduced or eliminated, and the stability of weld characteristic point extraction is improved.
Further, the above apparatus further comprises: a feature point detection module, a feature point classification module, a first region of interest determination module and a second region of interest determination module, wherein,
the characteristic point detection module is used for detecting SURF characteristic points in a first frame of welding seam image after the first frame of welding seam image is obtained;
the characteristic point classification module is used for dividing the SURF characteristic points into first-class characteristic points and second-class characteristic points according to the longitudinal coordinate values of the SURF characteristic points, wherein the first-class characteristic points indicate the characteristic points in the stripe region on the surface of the workpiece, and the second-class characteristic points indicate the characteristic points in the stripe region at the bottom of the welding seam;
the first interesting area determining module is used for determining a first interesting area according to the distribution of the first type of characteristic points;
and the second region of interest determining module is used for determining a second region of interest according to the distribution of the second type of feature points.
In an example, the first region of interest determination module is specifically configured to: selecting a first class of target feature points with the gray value smaller than a set gray threshold value from the first class of feature points; fitting a first target straight line according to the coordinate values of the first type of target feature points; and taking the first target straight line as a center, and taking a first preset pixel range as the first region of interest.
In an example, the second region of interest determination module is specifically configured to: acquiring the mean value of the vertical coordinates of the second type of feature points, and determining a second target straight line according to the mean value of the vertical coordinates; and taking the second target straight line as a center, and taking a second preset pixel range as the second region of interest.
Further, the binarization segmentation processing module 320 is specifically configured to: respectively calculating a first binarization segmentation threshold value and a second binarization segmentation threshold value of the first region of interest and the second region of interest; carrying out binarization segmentation processing on the first region of interest according to the first binarization segmentation threshold value to obtain a first binarization region of interest; and performing binarization segmentation processing on the second region of interest according to the second binarization segmentation threshold value to obtain a second binarization region of interest.
On the basis of the above technical solution, when the weld feature point identification module 330 determines two weld feature points in the first binarized region of interest, it is specifically configured to: calculating a straight line segment in the first binarization interested area, and dividing the straight line segment into a left line segment set and a right line segment set; respectively sequencing the left line segment set and the right line segment set to determine a first line segment and a second line segment; the first line segment is the longest single-pixel line segment in the left line segment set, and the second line segment is the longest single-pixel line segment in the right line segment set; and taking the right end point of the first line segment and the left end point of the second line segment as weld joint feature points.
On the basis of the above technical solution, when the weld feature point identification module 330 determines two weld feature points in the second binarization region of interest, it is specifically configured to: calculating a single-pixel central line segment of the laser stripe in the second binarization interested area; and taking two end points of the single-pixel central line segment as the weld characteristic points.
The identification device for the characteristic points of the butt-joint type welding seam can execute the identification method for the characteristic points of the butt-joint type welding seam provided by any embodiment of the invention, and has the corresponding functional modules and beneficial effects of executing the identification method for the characteristic points of the butt-joint type welding seam.
Example four
Fig. 6 is a schematic structural diagram of a computer device according to a fourth embodiment of the present invention, and as shown in fig. 6, the computer device includes:
one or more processors 410, one processor 410 being exemplified in FIG. 6;
a memory 420;
the apparatus may further include: an input device 430 and an output device 440.
The processor 410, the memory 420, the input device 430 and the output device 440 of the apparatus may be connected by a bus or other means, for example, in fig. 6.
The memory 420 serves as a non-transitory computer-readable storage medium, and may be used to store software programs, computer-executable programs, and modules, such as program instructions/modules corresponding to a method for identifying characteristic points of a butt-joint weld according to an embodiment of the present invention (for example, the region-of-interest obtaining module 310, the binarization segmentation processing module 320, and the weld characteristic point identification module 330 shown in fig. 5). The processor 410 executes various functional applications and data processing of the computer device by executing the software programs, instructions and modules stored in the memory 420, namely, implementing the identification method of the characteristic points of the butt-joint type weld seam of the above-mentioned method embodiment.
The memory 420 may include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required for at least one function; the storage data area may store data created according to use of the computer device, and the like. Further, the memory 420 may include high speed random access memory, and may also include non-transitory memory, such as at least one magnetic disk storage device, flash memory device, or other non-transitory solid state storage device. In some embodiments, memory 420 may optionally include memory located remotely from processor 410, which may be connected to the terminal device via a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
The input device 430 may be used to receive input numeric or character information and generate key signal inputs related to user settings and function control of the computer apparatus. The output device 440 may include a display device such as a display screen.
EXAMPLE five
An embodiment of the present invention further provides a storage medium containing computer-executable instructions, which when executed by a computer processor, are configured to perform a method for identifying characteristic points of a butt-type weld, the method including:
acquiring a frame of welding seam image, and acquiring a first interested area and a second interested area in the welding seam image; wherein the area ranges of the first region of interest and the second region of interest are determined according to SURF feature points in a first frame of weld image, the first region of interest indicates a workpiece surface stripe area, and the second region of interest indicates a weld bottom stripe area;
respectively carrying out binarization segmentation processing on the first region of interest and the second region of interest to obtain a first binarization region of interest and a second binarization region of interest;
and determining two weld characteristic points in the first binarization interested area, and determining two weld characteristic points in the second binarization interested area.
Optionally, the computer-executable instructions, when executed by a computer processor, may be further used to implement a technical solution of a method for identifying characteristic points of a butt-joint weld provided in any embodiment of the present invention.
From the above description of the embodiments, it is obvious for those skilled in the art that the present invention can be implemented by software and necessary general hardware, and certainly, can also be implemented by hardware, but the former is a better embodiment in many cases. Based on such understanding, the technical solutions of the present invention may be embodied in the form of a software product, which can be stored in a computer-readable storage medium, such as a floppy disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a FLASH Memory (FLASH), a hard disk or an optical disk of a computer, and includes several instructions for enabling a computer device (which may be a personal computer, a server, or a network device) to execute the methods according to the embodiments of the present invention.
It should be noted that, in the embodiment of the apparatus for identifying a butt-joint weld characteristic point, each included unit and module is only divided according to functional logic, but is not limited to the above division as long as the corresponding function can be realized; in addition, specific names of the functional units are only for convenience of distinguishing from each other, and are not used for limiting the protection scope of the present invention.
It is to be noted that the foregoing is only illustrative of the preferred embodiments of the present invention and the technical principles employed. It will be understood by those skilled in the art that the present invention is not limited to the particular embodiments described herein, but is capable of various obvious changes, rearrangements and substitutions as will now become apparent to those skilled in the art without departing from the scope of the invention. Therefore, although the present invention has been described in greater detail by the above embodiments, the present invention is not limited to the above embodiments, and may include other equivalent embodiments without departing from the spirit of the present invention, and the scope of the present invention is determined by the scope of the appended claims.
Claims (7)
1. A method for identifying characteristic points of butt welding seams is characterized by comprising the following steps:
acquiring a frame of welding seam image, and acquiring a first interested area and a second interested area in the welding seam image; wherein the area ranges of the first region of interest and the second region of interest are determined according to SURF feature points in a first frame of weld image, the first region of interest indicates a workpiece surface stripe area, and the second region of interest indicates a weld bottom stripe area;
respectively carrying out binarization segmentation processing on the first region of interest and the second region of interest to obtain a first binarization region of interest and a second binarization region of interest;
determining two weld joint feature points in the first binarization interested area, and determining two weld joint feature points in the second binarization interested area;
the SURF characteristic points refer to bright points in the dark positions in the weld images;
after a first frame of welding seam image is obtained, SURF characteristic points in the first frame of welding seam image are detected;
dividing the SURF characteristic points into first type characteristic points and second type characteristic points according to the longitudinal coordinate values of the SURF characteristic points, wherein the first type characteristic points indicate characteristic points in a stripe region on the surface of the workpiece, and the second type characteristic points indicate characteristic points in a stripe region at the bottom of the welding seam;
determining a first region of interest according to the distribution of the first type of feature points;
determining a second region of interest according to the distribution of the second type of feature points;
determining two weld feature points in the first binarized region of interest, including:
calculating a straight line segment in the first binarization interested area, and dividing the straight line segment into a left line segment set and a right line segment set;
respectively sequencing the left line segment set and the right line segment set to determine a first line segment and a second line segment; the first line segment is the longest single-pixel line segment in the left line segment set, and the second line segment is the longest single-pixel line segment in the right line segment set;
taking the right end point of the first line segment and the left end point of the second line segment as weld joint feature points;
determining two weld feature points in the second binarization interested region, including:
calculating a single-pixel central line segment of the laser stripe in the second binarization interested area;
and taking two end points of the single-pixel central line segment as the weld characteristic points.
2. The method according to claim 1, wherein determining a first region of interest from the distribution of the first class of feature points comprises:
selecting a first class of target feature points with the gray value smaller than a set gray threshold value from the first class of feature points;
fitting a first target straight line according to the coordinate values of the first type of target feature points;
and taking the first target straight line as a center, and taking a first preset pixel range as the first region of interest.
3. The method of claim 1, wherein determining a second region of interest from the distribution of the second type of feature points comprises:
acquiring the mean value of the vertical coordinates of the second type of feature points, and determining a second target straight line according to the mean value of the vertical coordinates;
and taking the second target straight line as a center, and taking a second preset pixel range as the second region of interest.
4. The method according to claim 1, wherein performing binarization segmentation processing on the first region of interest and the second region of interest respectively to obtain a first binarization region of interest and a second binarization region of interest comprises:
respectively calculating a first binarization segmentation threshold value and a second binarization segmentation threshold value of the first region of interest and the second region of interest;
carrying out binarization segmentation processing on the first region of interest according to the first binarization segmentation threshold value to obtain a first binarization region of interest;
and performing binarization segmentation processing on the second region of interest according to the second binarization segmentation threshold value to obtain a second binarization region of interest.
5. An identification device of butt-joint type welding seam characteristic points is characterized by comprising:
the interesting area acquisition module is used for acquiring a frame of welding seam image and acquiring a first interesting area and a second interesting area in the welding seam image; wherein the area ranges of the first region of interest and the second region of interest are determined according to SURF feature points in a first frame of weld image, the first region of interest indicates a workpiece surface stripe area, and the second region of interest indicates a weld bottom stripe area;
the binarization segmentation processing module is used for respectively carrying out binarization segmentation processing on the first region of interest and the second region of interest to obtain a first binarization region of interest and a second binarization region of interest;
the weld joint feature point identification module is used for determining two weld joint feature points in the first binarization interested area and determining two weld joint feature points in the second binarization interested area;
the SURF characteristic points refer to bright points in the dark positions in the weld images;
the characteristic point detection module is used for detecting SURF characteristic points in a first frame of welding seam image after the first frame of welding seam image is obtained;
the characteristic point classification module is used for dividing the SURF characteristic points into first-class characteristic points and second-class characteristic points according to the longitudinal coordinate values of the SURF characteristic points, wherein the first-class characteristic points indicate the characteristic points in the stripe region on the surface of the workpiece, and the second-class characteristic points indicate the characteristic points in the stripe region at the bottom of the welding seam;
the first interesting area determining module is used for determining a first interesting area according to the distribution of the first type of characteristic points;
the second region of interest determining module is used for determining a second region of interest according to the distribution of the second type of feature points;
the weld feature point identification module, when determining two weld feature points in the first binarized region of interest, is specifically configured to: calculating a straight line segment in the first binarization interested area, and dividing the straight line segment into a left line segment set and a right line segment set; respectively sequencing the left line segment set and the right line segment set to determine a first line segment and a second line segment; the first line segment is the longest single-pixel line segment in the left line segment set, and the second line segment is the longest single-pixel line segment in the right line segment set; taking the right end point of the first line segment and the left end point of the second line segment as weld joint feature points;
the weld feature point identification module is specifically configured to, when determining two weld feature points in the second binarization interested region: calculating a single-pixel central line segment of the laser stripe in the second binarization interested area; and taking two end points of the single-pixel central line segment as the weld characteristic points.
6. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor implements the method according to any of claims 1-4 when executing the program.
7. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the method according to any one of claims 1-4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010247486.4A CN111414912B (en) | 2020-03-31 | 2020-03-31 | Method, device and equipment for identifying characteristic points of butt-joint type welding seam and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010247486.4A CN111414912B (en) | 2020-03-31 | 2020-03-31 | Method, device and equipment for identifying characteristic points of butt-joint type welding seam and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111414912A CN111414912A (en) | 2020-07-14 |
| CN111414912B true CN111414912B (en) | 2021-10-12 |
Family
ID=71491722
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010247486.4A Active CN111414912B (en) | 2020-03-31 | 2020-03-31 | Method, device and equipment for identifying characteristic points of butt-joint type welding seam and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111414912B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112200000B (en) * | 2020-09-11 | 2022-10-11 | 上海交通大学 | Welding stability recognition model training method and welding stability recognition method |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP3353638B2 (en) * | 1997-03-28 | 2002-12-03 | 日本鋼管株式会社 | Fillet welding method for corners of structures |
| CN103500321B (en) * | 2013-07-03 | 2016-12-07 | 无锡信捷电气股份有限公司 | Vision guide welding robot weld seam method for quickly identifying based on double dynamic windows |
| CN108280829A (en) * | 2018-02-12 | 2018-07-13 | 广东省海洋工程装备技术研究所 | Welding seam method, computer installation and computer readable storage medium |
| CN110814465B (en) * | 2019-11-28 | 2021-09-03 | 华东交通大学理工学院 | Universal method for automatically extracting welding seam contour |
-
2020
- 2020-03-31 CN CN202010247486.4A patent/CN111414912B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN111414912A (en) | 2020-07-14 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108629343B (en) | License plate positioning method and system based on edge detection and improved Harris corner detection | |
| CN109242853B (en) | PCB defect intelligent detection method based on image processing | |
| CN115908269B (en) | Visual defect detection method, visual defect detection device, storage medium and computer equipment | |
| CN111127402A (en) | Visual detection method for welding quality of robot | |
| US20070110309A1 (en) | Shadow detection in images | |
| Wakaf et al. | Defect detection based on extreme edge of defective region histogram | |
| CN111402323B (en) | Butt weld joint identification method, device, equipment and storage medium | |
| CN111582000B (en) | Bar code positioning method, bar code positioning device and related equipment | |
| CN111882568B (en) | Fire image edge extraction processing method, terminal and system | |
| CN112419298B (en) | Bolt node plate rust detection method, device, equipment and storage medium | |
| KR20130072073A (en) | Apparatus and method for extracting edge in image | |
| CN115760820A (en) | Plastic part defect image identification method and application | |
| CN111127498A (en) | Canny edge detection method based on edge self-growth | |
| CN112528868A (en) | Illegal line pressing judgment method based on improved Canny edge detection algorithm | |
| CN110060239B (en) | Defect detection method for bottle opening of bottle | |
| CN111275659A (en) | Weld image processing method and device, terminal device and storage medium | |
| CN111414912B (en) | Method, device and equipment for identifying characteristic points of butt-joint type welding seam and storage medium | |
| CN117351011B (en) | Screen defect detection method, apparatus, and readable storage medium | |
| CN113971681A (en) | Edge detection method for belt conveyor in complex environment | |
| CN114913112A (en) | Method, device and equipment for detecting double edges of wafer | |
| CN117132797A (en) | POLMARK positioning detection method and system for LCD display screen | |
| WO2024016686A1 (en) | Corner detection method and apparatus | |
| Luo et al. | Adaptive canny and semantic segmentation networks based on feature fusion for road crack detection | |
| CN115984211A (en) | Visual positioning method, robot and storage medium | |
| CN113538500B (en) | Image segmentation method and device, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |