CN111145154A - Machine vision-based serial steel wire anti-loosening structure detection method - Google Patents
Machine vision-based serial steel wire anti-loosening structure detection method Download PDFInfo
- Publication number
- CN111145154A CN111145154A CN201911361531.2A CN201911361531A CN111145154A CN 111145154 A CN111145154 A CN 111145154A CN 201911361531 A CN201911361531 A CN 201911361531A CN 111145154 A CN111145154 A CN 111145154A
- Authority
- CN
- China
- Prior art keywords
- image
- theta
- max
- line segments
- steel wire
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0004—Industrial image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/20—Administration of product repair or maintenance
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10004—Still image; Photographic image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30108—Industrial image inspection
- G06T2207/30164—Workpiece; Machine component
Landscapes
- Engineering & Computer Science (AREA)
- Business, Economics & Management (AREA)
- Human Resources & Organizations (AREA)
- Quality & Reliability (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Economics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Entrepreneurship & Innovation (AREA)
- Marketing (AREA)
- Operations Research (AREA)
- Strategic Management (AREA)
- Tourism & Hospitality (AREA)
- General Business, Economics & Management (AREA)
- Image Analysis (AREA)
Abstract
The invention provides a machine vision-based method for detecting a serial steel wire anti-loose structure of an aircraft engine. The invention improves the detection efficiency of the anti-loose structure of the series steel wire on the engine, replaces manpower with a machine, and improves the intelligent level of the detection of the anti-loose structure of the series steel wire in the assembly of the engine.
Description
Technical Field
The invention relates to the field of assembly and maintenance of aero-engines, in particular to a detection method of an aero-engine series steel wire anti-loose structure based on machine vision.
Background
In order to ensure that the connection structure of the aero-engine is reliable and safe, a plurality of anti-loosening structures with series steel wires are required to be installed on the aero-engine to ensure the tightness of threads, so the steel wires are also called as fuses, and the structure is shown as a schematic diagram 1. During the flying process of the airplane, the threaded fasteners such as bolts tend to loosen due to the influence of factors such as vibration of an engine and an airplane body, temperature change and the like. Therefore, in addition to the tightening torque requirements for some fasteners on an aircraft, it is necessary to add tandem wires to prevent loosening.
At present, the installation process of the steel wire anti-loose structure of the aircraft engine requires four checks, namely checking whether threaded fasteners are neglected for installation, checking whether steel wires among the fasteners are wound, checking whether the winding direction of series steel wires is consistent with the screwing direction of threads, and checking whether the deflection of the steel wires exceeds the standard (is loosened). In the traditional method, technicians carry out manual detection, the risks of neglected loading and wrong loading exist, and the manual detection efficiency is relatively low especially for large aircraft engines.
The machine vision technology is applied to various fields, the core of the machine vision technology is image preprocessing and target feature extraction, and the methods are divided into two categories, namely machine learning and traditional algorithms. The machine learning method has strong adaptability and recognition accuracy under the support of sufficient effective image data, and has the prominent defects that the interpretability is poor, and a large amount of effective image data is difficult to obtain. In addition, the traditional method has definite calculation logic and better detection precision under definite characteristics and scenes, and has the defects of complex algorithm and poor multi-scene adaptability. In combination with the background, the traditional method has high accuracy for a specific scene due to strong interpretability, does not need a large amount of data support, and has applicability in bolt and series steel wire detection.
Disclosure of Invention
The invention provides a steel wire anti-loose structure detection technology based on machine vision, aiming at the defects of the existing steel wire anti-loose structure detection method, through an image preprocessing method, the interference of scratches, light spots, shadows and edge profiles is eliminated, a fastening bolt and a steel wire are identified, whether the steel wire is neglected to be installed or not is checked, the consistency of the winding direction and the screwing direction of a series steel wire is judged, and whether the steel wire is tensioned or not is judged. The invention improves the detection efficiency of the anti-loose structure of the series steel wire on the engine, replaces manpower with a machine, and improves the intelligent level of the detection of the anti-loose structure of the series steel wire in the assembly of the engine.
The technical scheme of the invention is as follows:
the serial steel wire anti-loosening structure detection method based on machine vision is characterized by comprising the following steps: the method comprises the following steps:
step 1: and (3) checking neglected installation of the fastening bolt:
acquiring an anti-loosening structure image of the series steel wire, converting the image into a gray image, and performing reverse binarization processing on the gray image according to a set threshold value to obtain a binary image; setting an inner core area to perform morphological closed operation on the binary image, then performing corrosion and expansion processing to eliminate scratch influence, performing Hough circle detection on the bolt, and storing pixel coordinate data of the circle center into a dynamic array FasteningCenters; comparing the number of circles obtained by Hough circle detection with the number of fastening bolts required in the series steel wire anti-loosening structure corresponding to the image, and judging whether the bolt fasteners are neglected to be installed in the series steel wire anti-loosening structure corresponding to the image; after determining that the circle obtained by Hough circle detection corresponds to the fastening bolt required in the serial steel wire anti-loosening structure corresponding to the image one by one, entering step 2;
step 2: and (3) steel wire neglected loading inspection:
introducing the gray level image obtained in the step 1, performing Gaussian filtering processing, then performing reverse binary and morphological closed operation processing to obtain a binary image with connected high-value points, and then performing corrosion processing, wherein the number of corrosion processing iterations is less than that in the step 1;
calling the FasteningCenters dynamic arrays in the step 1, and respectively extracting the most significant values U of the circle center coordinates on the U axis and the V axis of the image coordinate system by comparing the pixel coordinate values of the circle center point of the Hough circlemin,umax,vmin,vmaxCombining the two maximum values to obtain A (u)min,vmin)、B(umin,vmax)、C(umax,vmax) And D (u)max,vmin) Four points, connecting the four ABCD points in sequence by line segments to form a rectangular bounding box, then expanding each boundary of the bounding box to completely surround the bolt, wherein the vertex coordinate is A' (u)min-l,vmin-l),B′(umin-l,vmax+l),C′(umax+l,vmax+l),D′(umax+l,vmin-l);
Taking rectangular outlines which are sequentially connected with four vertexes A 'B' C 'D' as boundaries, introducing the outlines into a mask function, covering areas outside the rectangles A 'B' C 'D', using Canny operators to extract edges of the rectangular inner outlines, then executing Hough line detection, and storing detected line segment endpoint coordinates into LinesPoints dynamic arrays;
traversing all line segment data in the LinesPoints dynamic array, and deleting redundant line segments according to the following two rules:
rule 1: calculating the inclination angles corresponding to all the line segment slopes by using the pixel coordinates of the line segment end points, and when the difference between the inclination angles of the two line segments is in the rangeConsidering the two line segments as the line segments with similar inclination angles, and entering the rule 2 to continue judging;
the way in which the tilt angle is calculated from the slope is:
the slope k is in the range of (-infinity, + ∞) and the inverse tangent of the slope k is calculated in the range ofThe range of the tilt angle is [0,. pi.), so when arctank > 0, θ is arctank, when arctank < 0, takeMaking the inclination angle correspond to theta; for the limit case, whenWhen the range of the inclination angle close to theta is defined asWhen in useThen, the tilt angle (theta, pi) ∪ [0, pi-theta) is defined to be close to theta;
rule 2: calculating the distance between the end points on the left side of the line segments with the similar inclination angles determined in the rule 1, setting a threshold value M, and if the distance exceeds the threshold value, determining that the two steel wires are two steel wires, but the pixel inclination angles are similar; if the distance is less than M, determining that redundant line segments exist and only keeping one of the redundant line segments;
comparing the number of the remaining line segments after the redundant line segments are deleted with the number of steel wires to be installed required in the serial steel wire anti-loosening structure corresponding to the image, and judging whether the steel wires are neglected to be installed or not; and storing the coordinates of the end points of the rest line segments into a ValidLinesP dynamic array; after determining that the residual line segments correspond to the steel wires to be additionally installed required in the serial steel wire anti-loosening structure corresponding to the image one by one, entering step 3;
and step 3: judging the consistency of the winding direction of the series steel wires and the screwing direction of the bolts:
traversing pixel coordinates of center points in fasteningcenter, connecting different center points pairwise by using nested circulation, then performing redundancy removing operation on the obtained line segments by using a rule 1 and a rule 2 to obtain the connected line segments with the same number as the number of steel wires to be added required in the anti-loosening structure of the series steel wires corresponding to the image, calculating the inclination angles of the connected line segments and storing the inclination angles into a Bolt _ theta dynamic array; simultaneously calculating the inclination angle of the line segment in the ValidLinesP and storing the inclination angle into a Fuse _ theta array;
for a certain tilt angle theta in Bolt _ thetaBFinding the tilt angle theta close to the Fuse _ theta arrayFThen, the following rules are adopted for judgment:
rule 4: read out θ separatelyF,θBCorresponding line segment endpoint coordinates F1(uF1,vF1),F2(uF2,vF2) And B1(uB1,vB1),B2(uB2,vB2) Let endpoint F1F2And B1B2Respectively substituting the linear equations to obtain a two-point linear equation about the variables (u, v):
solving a system of equations of a first order of two for (u, v), in the interval of the variables u (u)F1,uF2) The internal equation set has a solution;
if a set of thetaF,θBIf the rule 3 and the rule 4 are met simultaneously, judging that the connecting direction of the series steel wires is consistent with the bolt tightening direction, otherwise, judging that the connecting direction of the series steel wires is inconsistent;
and 4, step 4: judging whether the deflection of the series steel wires exceeds the standard or not:
for a certain tilt angle theta in Bolt _ thetaBObtaining the coordinates P of two end points of the corresponding connected line segment1(uP1,vP1) And P2(uP2,vP2) And rotating the binary image in the step 1 by-thetaBTo makeKeeping the horizontal in the image coordinate system and obtaining the coordinate P after rotation1′(u′P1,v′P),P′2(u′P2,v′P);
Get umin=min{u′P1,u′P2},umax=max{u′P1,u′P2And taking the vertex coordinate of the rectangular mask as M1(umin+W,v′P-H),M2(umin+W,v′P+H),M3(umax-W,v′P+H),M4(umax-W,v′P-H) performing rectangular edge mask processing on the rotated binary image, wherein W and H are set offset amounts;
traversing all pixel points in the binary image mask, and carrying out comparison on all high-value pixel points Qi(ui,vi) And then a least square regression line fitting is carried out,the straight line equation au + bv + c is 0, and the distances of all the high-value pixel points relative to the regression line are calculated:
to diTaking the mean value to obtainAnd will beComparing with the set deflection standard exceeding threshold S ifJudging that the tandem wire is not tensioned, ifThe tandem wire is determined to be in a tensioned state.
Further preferred scheme, said a series connection steel wire anti loosening structure detection method based on machine vision, its characterized in that: in step 2, the number of iterations of the etching treatment is smaller than that in step 1
Advantageous effects
The invention provides a machine vision-based method for detecting a serial steel wire anti-loose structure of an aircraft engine, which improves the detection efficiency and the intelligent level through an image processing method, can effectively prevent the serial steel wire from being neglected and wrongly installed, and ensures the safety of the aircraft engine.
Additional aspects and advantages of the invention will be set forth in part in the description which follows and, in part, will be obvious from the description, or may be learned by practice of the invention.
Drawings
The above and/or additional aspects and advantages of the present invention will become apparent and readily appreciated from the following description of the embodiments, taken in conjunction with the accompanying drawings of which:
FIG. 1: a schematic diagram of a serial steel wire anti-loose structure;
FIG. 2: step 1, a bolt fastener neglected loading inspection flow chart;
FIG. 3: step 2, a serial steel wire neglected loading inspection flow chart;
FIG. 4: step 3, a series steel wire misloading inspection flow chart;
FIG. 5: the relation between the bolt center connecting line and the steel wire is shown schematically;
FIG. 6: step 4, a flow chart of detecting the deflection of the series steel wires;
FIG. 7: step 4, setting a schematic diagram of parameters of a mask boundary W and a mask boundary H;
FIG. 8: a schematic diagram of an original captured image;
FIG. 9: a bolt fastener center detection schematic diagram;
FIG. 10: the binary image schematic diagram is obtained after edge mask processing;
FIG. 11: generating a redundant line segment schematic diagram by Hough line detection;
FIG. 12: simplifying the schematic diagram of the treated Hough line;
FIG. 13: a schematic diagram of a winding direction misloading inspection result;
FIG. 14: schematic diagram of identification results of forward and reverse installation of the series steel wires;
FIG. 15: the characteristic schematic diagram of local gray scale amplification of the steel wire after the mask is covered;
FIG. 16: a binary image high-value point least square fitting straight line schematic diagram;
FIG. 17: schematic diagram for judging the deflection of the series steel wires.
Detailed Description
The invention provides a steel wire anti-loosening structure detection method based on machine vision, aiming at the defects of the existing steel wire anti-loosening structure detection method, which comprises the following steps:
step 1: and (3) checking neglected installation of the fastening bolt, wherein the flow is shown in FIG. 2:
image data of the serial steel wire anti-loosening structure acquired by the industrial camera are numbered in sequence, and the numbering process is automatically generated by a program according to the shooting sequence and stored in a storage device;
the collected image is imported into an image processing program by a storage device, and is converted into a gray image, and the gray image is subjected to reverse binarization processing according to a set threshold value;
setting an inner core area to perform morphological closed operation on the binary image, then performing corrosion and expansion treatment to eliminate scratch influence, performing Hough circle detection on the bolt, storing pixel coordinate data of a circle center into a dynamic array named as FasteningCenters, and performing circular ring marking through a circle center pixel point;
and comparing the number obtained by detecting the Hough circle with the number corresponding to the numbering process requirement, and judging whether the bolt fastener is neglected to be installed at the corresponding position of the image by the program.
Step 2: and (3) checking the neglected loading of the steel wire, wherein the flow is shown in figure 3:
and (3) introducing the gray level image obtained in the step 1, and performing Gaussian filtering processing, wherein the resolution of the camera image for experiment is 6240 × 4160, and the resolution is higher, so that the filtering kernel Size adopted here is (5,5), and if the filtering kernel is too small, the filtering effect is not obvious, and certain influence is caused on detection.
Then, reverse binary and morphological closed operation processing is carried out to obtain a binary image with high-value point communication, then corrosion processing is carried out, and since the pixel characteristics of the steel wire are slender and easy to corrode compared with the bolt, in order to keep the serial steel wire outline of the collected image, the iteration times are fewer than that in the step 1And (4) etching treatment.
Calling the FasteningCenters dynamic arrays in the step 1, and respectively extracting the most significant values U of the circle center coordinates on the U axis and the V axis of the image coordinate system by comparing the pixel coordinate values of the circle center point of the Hough circlemin,umax,vmin,vmaxUnder an image coordinate system, an upper left corner point of an image is a coordinate origin, a horizontal axis is a U axis, a vertical axis is a V axis, and coordinates on the axes are positive integers and represent pixel positions.
Combining the two maximum values to obtain A (u)min,vmin)、B(umin,vmax)、C(umax,vmax) AndD(umax,vmin) Four points, connecting the four ABCD points in sequence by line segments to form a rectangular bounding box, then expanding each boundary of the bounding box to completely surround the bolt, wherein the vertex coordinate is A' (u)min-l,vmin-l),B′(umin-l,vmax+l),C′(umax+l,vmax+l),D′(umax+l,vmin-l)。
Taking the rectangular contour with the four vertexes A 'B' C 'D' connected in sequence as a boundary, introducing the contour into a mask function, wherein the mask function is as follows: traversing all pixel point coordinates of the image, and when the pixel points are positioned inside or on a rectangular boundary, keeping pixel point color information, or else, removing the pixel point color information; covering the area outside the rectangle A 'B' C 'D' by a mask function, extracting the edge of the inner contour of the rectangle by using a Canny operator, then executing Hough line detection, and storing the detected line segment endpoint coordinates into a LinesPoints dynamic array.
Because the Hough line detection is used for generating the redundant line segment, all line segment data in the LinesPoints dynamic array are traversed, and the redundant line segment is deleted according to the following two rules:
rule 1: and eliminating principles with similar inclination angles. Calculating the inclination angles corresponding to all the line segment slopes by using the pixel coordinates of the line segment end points, and when the difference between the inclination angles of the two line segments is in the rangeConsidering the two line segments as the line segments with similar inclination angles, and entering the rule 2 to continue judging;
the way in which the tilt angle is calculated from the slope is:
the slope k is in the range of (-infinity, + ∞) and the inverse tangent of the slope k is calculated in the range ofThe range of the tilt angle is [0,. pi.), so when arctank > 0, θ is arctank, when arctank < 0, takeMake the slopeThe angle corresponds to theta; for the limit case, whenWhen the range of the inclination angle close to theta is defined asWhen in useThen, the tilt angle (theta, pi) ∪ [0, pi-theta) is defined to be close to theta;
rule 2: and (5) an endpoint-close rejection principle. Calculating the distance between the end points on the left side of the line segments with the similar inclination angles determined in the rule 1, setting a threshold value M, and if the distance exceeds the threshold value, determining that the two steel wires are two steel wires, but the pixel inclination angles are similar; if the distance is less than M, determining that redundant line segments exist and only keeping one of the redundant line segments;
traversing all line segment data in the LinesPoints dynamic array, comparing the number of the remaining line segments of which the redundant line segments are deleted by the two judgment rules with the number of steel wires to be additionally arranged required in the serial steel wire anti-loosening structure corresponding to the image, and judging whether the steel wires are neglected to be arranged; and storing the coordinates of the end points of the rest line segments into a ValidLinesP dynamic array; and (3) determining that the residual line segments correspond to the additionally-installed steel wires required in the series steel wire anti-loosening structure corresponding to the image one by one, and then entering the step (3).
And step 3: judging the consistency of the winding direction of the series steel wires and the screwing direction of the bolts:
traversing pixel coordinates of center points in fasteningcenter, connecting different center points pairwise by using nested circulation, then performing redundancy removing operation on the obtained line segments by using a rule 1 and a rule 2 to obtain the connected line segments with the same number as the number of steel wires to be added required in the anti-loosening structure of the series steel wires corresponding to the image, calculating the inclination angles of the connected line segments and storing the inclination angles into a Bolt _ theta dynamic array; simultaneously calculating the inclination angle of the line segment in the ValidLinesP and storing the inclination angle into a Fuse _ theta array;
for a certain tilt angle theta in Bolt _ thetaBFinding the tilt angle theta close to the Fuse _ theta arrayFThen using the following ruleAnd (4) line judgment:
rule 4: read out θ separatelyF,θBCorresponding line segment endpoint coordinates F1(uF1,vF1),F2(uF2,vF2) And B1(uB1,vB1),B2(uB2,vB2) Let endpoint F1F2And B1B2Respectively substituting the linear equations to obtain a two-point linear equation about the variables (u, v):
solving a system of equations of a first order of two for (u, v), in the interval of the variables u (u)F1,uF2) The internal equation set has a solution;
if a set of thetaF,θBIf the rule 3 and the rule 4 are met simultaneously, the connection direction of the series steel wires is judged to be consistent with the bolt tightening direction, and the program marks a plus sign at the center of the bolt connecting line; otherwise, the connection direction is wrong, and the program marks a negative sign at the center of the bolt connecting line.
And 4, step 4: judging whether the deflection of the series steel wires exceeds the standard or not:
for a certain tilt angle theta in Bolt _ thetaBObtaining the coordinates P of two end points of the corresponding connected line segment1(uP1,vP1) And P2(uP2,vP2) And rotating the binary image in the step 1 by-thetaBTo makeKeeping the horizontal in the image coordinate system and obtaining the coordinate P after rotation1′(u′P1,v′P),P2′(u′P2,v′P)。
Get umin=min{u′P1,u′P2},umax=max{u′P1,u′P2And taking the vertex coordinate of the rectangular mask as M1(umin+W,v′P-H),M2(umin+W,v′P+H),M3(umax-W,v′P+H),M4(umax-W,v′P-H) performing rectangular edge mask processing on the rotated binary image, wherein W and H are set offset amounts.
Traversing all pixel points in the binary image mask, and carrying out comparison on all high-value pixel points Qi(ui,vi) And performing least square regression line fitting to obtain a line equation au + bv + c which is 0, and calculating the distance between all high-value pixel points and the regression line:
to diTaking the mean value to obtainAnd will beComparing with the set deflection standard exceeding threshold S ifJudging that the tandem wire is not tensioned, ifThe tandem wire is determined to be in a tensioned state.
Embodiments of the present invention will be described in detail below with reference to fig. 6-16, which are exemplary and intended to be illustrative of the invention and should not be construed as limiting the invention.
This embodiment is described by taking a group of bolt fasteners wound around tandem steel wires as an example, and as shown in fig. 8, the results of the first 3 steps are demonstrated, wherein there are 5 bolt fasteners, and the 5 tandem steel wires are connected two by two to form a pentagon, and obvious scratches and partial shadows can be observed in the pentagon, and alternatively, steps 3 and 4 are additionally described by using fig. 13 and 16, respectively.
Step 1, detecting missing bolt installation:
in the image acquisition stage, the resolution of an image acquired by an industrial camera is 6240 multiplied by 4160, and the diameter of a bolt pixel is kept between 100 and 500 pixels. According to the flow shown in fig. 2, the filtering is converted into a single-channel gray image, the threshold value threshold of inverse binarization is 80, the kernel of closed operation is a pixel matrix of 50 × 50, and the corrosion degree just completely corrodes the steel wire contour and the scratch for high-value point corrosion and expansion respectively is 30.
The result of the Hough circle detection is shown in FIG. 8, and the center of the circle is offset from the center of the bolt but within the allowable range. And if the number of Hough circles detected by the corresponding numbered position images does not accord with the known number, sending a prompt.
Step 2, detecting missing of the series steel wire:
according to the pixel coordinates of the 5 central points detected in the step 1, the maximum value and the minimum value on each axis are searched for the coordinate components on the U axis and the V axis in a traversing manner, l is set to be 200, four vertexes of the mask are obtained, so that the edge interference is removed, and then the binary image is copied to the internal area of the mask, as shown in fig. 9, the mask can cover the edge interference caused by illumination and shooting position conditions.
The masked binary image was subjected to morphological close operation with a kernel of 40 × 40, and slightly eroded with an erosion iteration number iterative of 10.
Edge detection is carried out by using Canny operator, parameters are set to be threshold 1 ═ 150 and threshold2 ═ 500, then Hough line detection is carried out, and accumulator distance resolution rho ═ 0.5 and accumulator angle resolution are setThe accumulator threshold is 100, the minimum line length is 1000, and the connection point is maximumThe gap is maxLineGap of 200, and the detection result is as shown in fig. 10, and a large number of redundant line segments are detected in the detection result, and 227 lines are detected.
The redundant line segments are judged by using the rules 1 and 2, the threshold value M in the rule 2 is set to be 500, and the simplified Hough line segments correspond to the steel wires one by one as shown in fig. 11.
And (4) checking the detection result of the series steel wires and the position due number, and outputting the information without missing installation.
Step 3, checking the winding direction of the series steel wires:
and importing the 5 Hough circle center points identified in the step 1 and the end point data of the Hough line screened in the step 2. The 5 circle centers are connected pairwise to generate 10 connecting lines, and redundancy exists.
Calculating the inclination angles of circle center connecting lines and Hough lines by using a rule 1, and then screening 5 circle center connecting lines associated with the Hough lines according to the rule 1 and the rule 2, wherein each circle center connecting line and the corresponding Hough line are compiled into a group.
According to the illustration in fig. 5, whether the position relationship between each group of Hough lines and the circle center connecting line is correct is judged according to the rules 3 and 4, if the judgment is correct, the program marks a positive sign at the center of the circle center connecting line, and as a result, the connection relationship between 5 groups is correct as illustrated in fig. 12. In addition, the control group shown in FIG. 13 was connected in reverse on the left and in forward on the right.
And outputting information corresponding to the positive and negative connection of the picture according to the measured result, and sending out reminding information if the picture is wrongly installed.
Step 4, detecting the deflection of the series steel wire:
and (4) importing circle center connecting lines related to the serial anti-loosening steel wires in the step (3) and an inclination angle theta corresponding to the circle center connecting lines. To obtain the circle center of P'1(u′P1,v′P) And P'2(u′P2,v′P) For example, let W be 200 pixels and H be 100 pixels, and obtain pixel coordinate values of four vertices, and mask-process the pixel coordinate values to obtain the local enlarged feature shown in fig. 14. Wherein, the left image mask vertex pixel coordinate is: (1923,2203), (1923,2403), (4077,2403), (4077,2203); the coordinates of the vertices of the mask pixels of the right image are: (2213,2061),(2213,2261),(4001,2261),(4001,2061).
For the image binary processing in the mask, traverse all high-value points in the search area by row and column, and then fit a straight line by the least square method, as shown in fig. 15.
Calculating the average deviation of all high-value points relative to the straight line by using a point-to-straight line distance formula, and setting a threshold value S to be 20, wherein the average deviation on the left side is 6.157068, the deviation is within the threshold value, and the deviation is in a tensioning state; the right mean deviation is 46.205240, exceeding the threshold, and is relaxed.
The program marks the image according to the judgment as shown in fig. 16, and sends out a prompt according to the relaxation information, and the content contains the value of which the average deviation exceeds the standard.
Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention, and that variations, modifications, substitutions and alterations can be made in the above embodiments by those of ordinary skill in the art without departing from the principle and spirit of the present invention.
Claims (2)
1. A serial steel wire anti-loosening structure detection method based on machine vision is characterized in that: the method comprises the following steps:
step 1: and (3) checking neglected installation of the fastening bolt:
acquiring an anti-loosening structure image of the series steel wire, converting the image into a gray image, and performing reverse binarization processing on the gray image according to a set threshold value to obtain a binary image; setting an inner core area to perform morphological closed operation on the binary image, then performing corrosion and expansion processing to eliminate scratch influence, performing Hough circle detection on the bolt, and storing pixel coordinate data of the circle center into a dynamic array FasteningCenters; comparing the number of circles obtained by Hough circle detection with the number of fastening bolts required in the series steel wire anti-loosening structure corresponding to the image, and judging whether the bolt fasteners are neglected to be installed in the series steel wire anti-loosening structure corresponding to the image; after determining that the circle obtained by Hough circle detection corresponds to the fastening bolt required in the serial steel wire anti-loosening structure corresponding to the image one by one, entering step 2;
step 2: and (3) steel wire neglected loading inspection:
introducing the gray level image obtained in the step 1, performing Gaussian filtering processing, then performing reverse binary and morphological closed operation processing to obtain a binary image with connected high-value points, and then performing corrosion processing, wherein the number of corrosion processing iterations is less than that in the step 1;
calling the FasteningCenters dynamic arrays in the step 1, and respectively extracting the most significant values U of the circle center coordinates on the U axis and the V axis of the image coordinate system by comparing the pixel coordinate values of the circle center point of the Hough circlemin,umax,vmin,vmaxCombining the two maximum values to obtain A (u)min,vmin)、B(umin,vmax)、C(umax,vmax) And D (u)max,vmin) Four points, connecting the four ABCD points in sequence by line segments to form a rectangular bounding box, then expanding each boundary of the bounding box to completely surround the bolt, wherein the vertex coordinate is A' (u)min-l,vmin-l),B′(umin-l,vmax+l),C′(umax+l,vmax+l),D′(umax+l,vmin-l);
Taking rectangular outlines which are sequentially connected with four vertexes A 'B' C 'D' as boundaries, introducing the outlines into a mask function, covering areas outside the rectangles A 'B' C 'D', using Canny operators to extract edges of the rectangular inner outlines, then executing Hough line detection, and storing detected line segment endpoint coordinates into LinesPoints dynamic arrays;
traversing all line segment data in the LinesPoints dynamic array, and deleting redundant line segments according to the following two rules:
rule 1: calculating the inclination angles corresponding to all the line segment slopes by using the pixel coordinates of the line segment end points, and when the difference between the inclination angles of the two line segments is in the rangeConsidering the two line segments as the line segments with similar inclination angles, and entering the rule 2 to continue judging;
the way in which the tilt angle is calculated from the slope is:
the slope k ranges from (— infinity, + ∞) and is invertedTangent calculation in the range ofThe range of the tilt angle is [0,. pi.), so when arctank > 0, θ is arctank, when arctank < 0, takeMaking the inclination angle correspond to theta; for the limit case, whenWhen the range of the inclination angle close to theta is defined asWhen in useThen, the tilt angle (theta, pi) ∪ [0, pi-theta) is defined to be close to theta;
rule 2: calculating the distance between the end points on the left side of the line segments with the similar inclination angles determined in the rule 1, setting a threshold value M, and if the distance exceeds the threshold value, determining that the two steel wires are two steel wires, but the pixel inclination angles are similar; if the distance is less than M, determining that redundant line segments exist and only keeping one of the redundant line segments;
comparing the number of the remaining line segments after the redundant line segments are deleted with the number of steel wires to be installed required in the serial steel wire anti-loosening structure corresponding to the image, and judging whether the steel wires are neglected to be installed or not; and storing the coordinates of the end points of the rest line segments into a ValidLinesP dynamic array; after determining that the residual line segments correspond to the steel wires to be additionally installed required in the serial steel wire anti-loosening structure corresponding to the image one by one, entering step 3;
and step 3: judging the consistency of the winding direction of the series steel wires and the screwing direction of the bolts:
traversing pixel coordinates of center points in fasteningcenter, connecting different center points pairwise by using nested circulation, then performing redundancy removing operation on the obtained line segments by using a rule 1 and a rule 2 to obtain the connected line segments with the same number as the number of steel wires to be added required in the anti-loosening structure of the series steel wires corresponding to the image, calculating the inclination angles of the connected line segments and storing the inclination angles into a Bolt _ theta dynamic array; simultaneously calculating the inclination angle of the line segment in the ValidLinesP and storing the inclination angle into a Fuse _ theta array;
for a certain tilt angle theta in Bolt _ thetaBFinding the tilt angle theta close to the Fuse _ theta arrayFThen, the following rules are adopted for judgment:
rule 4: read out θ separatelyF,θBCorresponding line segment endpoint coordinates F1(uF1,vF1),F2(uF2,vF2) And B1(uB1,vB1),B2(uB2,vB2) Let endpoint F1F2And B1B2Respectively substituting the linear equations to obtain a two-point linear equation about the variables (u, v):
solving a system of equations of a first order of two for (u, v), in the interval of the variables u (u)F1,uF2) The internal equation set has a solution;
if a set of thetaF,θBIf the rule 3 and the rule 4 are met simultaneously, judging that the connecting direction of the series steel wires is consistent with the bolt tightening direction, otherwise, judging that the connecting direction of the series steel wires is inconsistent;
and 4, step 4: judging whether the deflection of the series steel wires exceeds the standard or not:
for a certain tilt angle theta in Bolt _ thetaBObtaining the coordinates P of two end points of the corresponding connected line segment1(uP1,vP1) And P2(uP2,vP2) And combining the binary image in the step 1Rotation-thetaBTo makeKeeping the image horizontal in the image coordinate system, and obtaining the coordinate after rotation as P'1(u′P1,v′P),P′2(u′P2,v′P);
Get umin=min{u′P1,u′P2},umax=max{u′P1,u′P2And taking the vertex coordinate of the rectangular mask as M1(umin+W,v′P-H),M2(umin+W,v′P+H),M3(umax-W,v′P+H),M4(umax-W,v′P-H) performing rectangular edge mask processing on the rotated binary image, wherein W and H are set offset amounts;
traversing all pixel points in the binary image mask, and carrying out comparison on all high-value pixel points Qi(ui,vi) And performing least square regression line fitting to obtain a line equation au + bv + c which is 0, and calculating the distance between all high-value pixel points and the regression line:
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201911361531.2A CN111145154B (en) | 2019-12-25 | 2019-12-25 | Machine vision-based serial steel wire anti-loosening structure detection method |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201911361531.2A CN111145154B (en) | 2019-12-25 | 2019-12-25 | Machine vision-based serial steel wire anti-loosening structure detection method |
Publications (2)
Publication Number | Publication Date |
---|---|
CN111145154A true CN111145154A (en) | 2020-05-12 |
CN111145154B CN111145154B (en) | 2022-04-01 |
Family
ID=70520363
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN201911361531.2A Active CN111145154B (en) | 2019-12-25 | 2019-12-25 | Machine vision-based serial steel wire anti-loosening structure detection method |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN111145154B (en) |
Cited By (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN113469966A (en) * | 2021-06-25 | 2021-10-01 | 西南交通大学 | Train bolt looseness detection method based on anti-loosening line identification |
CN113635286A (en) * | 2021-08-20 | 2021-11-12 | 菲烁易维(重庆)科技有限公司 | Device and method for controlling bolt tightening based on machine vision technology |
Citations (8)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20160231253A1 (en) * | 2011-05-17 | 2016-08-11 | Gii Acquisition, Llc Dba General Inspection, Llc | Method and system for optically inspecting parts |
CN106340019A (en) * | 2016-08-31 | 2017-01-18 | 西南交通大学 | Method for detecting adverse state of high-speed rail overhead line system inclined cable fixing hook |
CN106372667A (en) * | 2016-08-31 | 2017-02-01 | 西南交通大学 | Method for detecting adverse state of inclined sleeve part screws of high-speed train overhead line system |
CN106570857A (en) * | 2016-09-11 | 2017-04-19 | 西南交通大学 | High-speed railway overhead contact system lateral conductor fixation hook nut falling bad state detection method based on HOG features |
CN109141232A (en) * | 2018-08-07 | 2019-01-04 | 常州好迪机械有限公司 | A kind of circle plate casting online test method based on machine vision |
CN109389566A (en) * | 2018-10-19 | 2019-02-26 | 辽宁奇辉电子***工程有限公司 | Subway height adjusting valve fastening nut defective mode detection method based on boundary characteristic |
CN110097536A (en) * | 2019-04-10 | 2019-08-06 | 东南大学 | Hexagon bolt looseness detection method based on deep learning and Hough transformation |
CN110276747A (en) * | 2019-06-06 | 2019-09-24 | 中南大学 | A kind of insulator breakdown detection and failure ranking method based on image analysis |
-
2019
- 2019-12-25 CN CN201911361531.2A patent/CN111145154B/en active Active
Patent Citations (8)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20160231253A1 (en) * | 2011-05-17 | 2016-08-11 | Gii Acquisition, Llc Dba General Inspection, Llc | Method and system for optically inspecting parts |
CN106340019A (en) * | 2016-08-31 | 2017-01-18 | 西南交通大学 | Method for detecting adverse state of high-speed rail overhead line system inclined cable fixing hook |
CN106372667A (en) * | 2016-08-31 | 2017-02-01 | 西南交通大学 | Method for detecting adverse state of inclined sleeve part screws of high-speed train overhead line system |
CN106570857A (en) * | 2016-09-11 | 2017-04-19 | 西南交通大学 | High-speed railway overhead contact system lateral conductor fixation hook nut falling bad state detection method based on HOG features |
CN109141232A (en) * | 2018-08-07 | 2019-01-04 | 常州好迪机械有限公司 | A kind of circle plate casting online test method based on machine vision |
CN109389566A (en) * | 2018-10-19 | 2019-02-26 | 辽宁奇辉电子***工程有限公司 | Subway height adjusting valve fastening nut defective mode detection method based on boundary characteristic |
CN110097536A (en) * | 2019-04-10 | 2019-08-06 | 东南大学 | Hexagon bolt looseness detection method based on deep learning and Hough transformation |
CN110276747A (en) * | 2019-06-06 | 2019-09-24 | 中南大学 | A kind of insulator breakdown detection and failure ranking method based on image analysis |
Non-Patent Citations (5)
Title |
---|
JAE-HYUNG PARK 等: "Novel bolt-loosening detection technique using image processing for bolt joints in steel bridges", 《ADVANCES IN STRUCTURAL ENGINEERING AND MECHANICS》 * |
JAE-HYUNG PARK 等: "Vision-based technique for bolt-loosening detection in wind turbine tower", 《WIND AND STRUCTURES》 * |
YOUNG-JIN CHA 等: "Vision-based detection of loosened bolts using the Hough transform and support vector machines", 《AUTOMATION IN CONSTRUCTION》 * |
袁云峰: "在线螺栓拧紧设备的检测和评定方法", 《柴油机设计与制造》 * |
黄潜 等: "基于三维扫描的机车走行部螺栓识别与定位", 《光电工程》 * |
Cited By (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN113469966A (en) * | 2021-06-25 | 2021-10-01 | 西南交通大学 | Train bolt looseness detection method based on anti-loosening line identification |
CN113469966B (en) * | 2021-06-25 | 2023-04-18 | 西南交通大学 | Train bolt looseness detection method based on anti-loosening line identification |
CN113635286A (en) * | 2021-08-20 | 2021-11-12 | 菲烁易维(重庆)科技有限公司 | Device and method for controlling bolt tightening based on machine vision technology |
Also Published As
Publication number | Publication date |
---|---|
CN111145154B (en) | 2022-04-01 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN106934803B (en) | method and device for detecting surface defects of electronic device | |
CN108369650B (en) | Method for identifying possible characteristic points of calibration pattern | |
CN107228860B (en) | Gear defect detection method based on image rotation period characteristics | |
US11354814B2 (en) | Vision-based fastener loosening detection | |
CN111145154B (en) | Machine vision-based serial steel wire anti-loosening structure detection method | |
CN110033431B (en) | Non-contact detection device and detection method for detecting corrosion area on surface of steel bridge | |
TWI521448B (en) | Vehicle identification system and method | |
CN112750107A (en) | Spherical workpiece identification method and system based on contour edge number fitting and convex hull processing | |
CN111814852B (en) | Image detection method, image detection device, electronic equipment and computer readable storage medium | |
CN108827979A (en) | A kind of module group lens appearance detecting method | |
CN113516619B (en) | Product surface flaw identification method based on image processing technology | |
CN115861351A (en) | Edge detection method, defect detection method and detection device | |
CN112288682A (en) | Electric power equipment defect positioning method based on image registration | |
CN114024503A (en) | Solar cell color separation and defect detection system and method thereof | |
CN116883446A (en) | Real-time monitoring system for grinding degree of vehicle-mounted camera lens | |
CN112288680B (en) | Automatic defect area extraction method and system for automobile hub X-ray image | |
CN111833350A (en) | Machine vision detection method and system | |
CN117333696A (en) | Bolt detection method and device based on machine vision | |
CN116664512A (en) | VEH connector assembly safety monitoring alarm system based on visual identification | |
CN114266748B (en) | Method and device for judging surface integrity of process board in field of rail traffic overhaul | |
CN113902667B (en) | Thread turning identification method and system for machine vision | |
CN115880481A (en) | Curve positioning algorithm and system based on edge profile | |
US11508052B2 (en) | Systems and methods for quantifying light flares in images | |
CN114359414A (en) | Lens smudge identification method and device, computer equipment and storage medium | |
CN114399479A (en) | Detection method and device for bolt cotter pin of railway vehicle and computer equipment |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
GR01 | Patent grant | ||
GR01 | Patent grant |