CN108176608B

CN108176608B - Nut defect inspection method and device based on machine vision

Info

Publication number: CN108176608B
Application number: CN201711423743.XA
Authority: CN
Inventors: 陈智能; 徐毅
Original assignee: Institute of Automation of Chinese Academy of Science
Current assignee: Institute of Automation of Chinese Academy of Science
Priority date: 2017-12-25
Filing date: 2017-12-25
Publication date: 2019-08-23
Anticipated expiration: 2037-12-25
Also published as: CN108176608A

Abstract

The present invention relates to part defect detection fields, and in particular to a kind of nut defect inspection method and device based on machine vision, in order to improve the efficiency of nut defects detection.The present invention obtains overhead view image, side elevation image and the oblique-view image of nut first, and carries out binary conversion treatment to this 3 kinds of images；Then the characteristic parameter of nut to be detected is extracted on the image after binary conversion treatment, it is compared with prior preset characteristic parameter, so that it is determined that nut to be detected with the presence or absence of internal diameter excessive or too small, upper groove cutting excessively, do not open slot, upper slot open it is anti-, outer groove discrepancy of quantity, turned upside down, do not open internal screw thread, and the problems such as whether being overlapped with other nuts, and then classify to nut to be detected, to distinguish waste product, substandard products, abnormal article and qualified product.Compared with traditional artificial detection, the present invention can efficiently detect all nuts one by one, greatly improve production efficiency, and significantly improve the qualification rate of factory nut.

Description

Nut defect detection method and device based on machine vision

Technical Field

The invention relates to the field of part defect detection, in particular to a nut defect detection method and device based on machine vision.

Background

Nuts are basic parts for tightly connecting mechanical equipment, and are widely used in various fields such as automobile manufacturing, building, machinery, rail transit, public facilities, casting industry and the like. Due to the difference of materials, specifications, technical requirements and the like, nut products on the market are various in types. The nuts produced by the automatic assembly line have defective products in a certain proportion due to the influences of complicated and variable production flows, different equipment operation and maintenance conditions and improper manual setting and operation. Typical defects are:

1) the inner diameter of the nut is too large or too small, so that the nut and the bolt or the screw cannot be screwed together or the biting force cannot reach the corresponding standard after the nut and the bolt or the screw are screwed together. Since defects are difficult to solve by simple rework, such nuts are generally treated as waste;

2) the process defects, such as the lack of internal threads, the lack of grooves and the like, are mainly caused by the omission of specific processing steps in production and processing. After the defects are clear, the missed processing steps can be supplemented to ensure that the nuts become qualified products, so the nuts are generally treated as defective products;

3) the upper groove is opened reversely, the nut is not placed correctly on the production line, for example, the bottom of the nut faces upwards, so that the grooving process which is supposed to be carried out on one side of the nut in the subsequent processing is wrongly carried out on the other side, and the whole nut is also waste.

Although nut manufacturing enterprises have controlled the proportion of the defective nuts at a lower value through means of process and flow optimization, fine management and the like, nut quality detection is still an essential link from the viewpoint of ensuring product quality. The quality detection of the nuts aims to find and remove the defective nuts through the quality detection step, so that the defective nuts do not flow into the terminal market. The traditional nut quality inspection is mainly finished manually, and workers determine whether the nut has defects or not through the modes of measuring the inner diameter by a professional measuring scale, observing by naked eyes and the like. However, the production of the nut has the characteristics of low single-product value and large quantity base number, and the comprehensive quality inspection of the nut is unrealistic to realize manually. At present, nut production enterprises generally adopt a quality inspection form of manual sampling inspection, and the quality inspection form can find mass continuous nut quality problems, and plays no obvious role in accidental quality problems, while the latter is not rare in practical production. The nut industry urgently needs a method and a sorting device which can carry out comprehensive and accurate quality detection on the nut.

Disclosure of Invention

In order to solve the problems in the prior art, the invention provides a nut defect detection method and device based on machine vision, which greatly improve the efficiency of detecting the nut defects and ensure the detection accuracy.

In one aspect of the invention, a nut defect detection method based on machine vision is provided, which comprises the following steps:

respectively acquiring a top view image, a side view image and an oblique view image of the nut to be detected;

carrying out binarization processing on the overlook image; extracting the characteristic parameters of the binarized overhead view image, comparing the extracted characteristic parameters of the overhead view image with preset overhead view image characteristic parameters, and judging whether the nut to be detected has defects or not;

carrying out binarization processing on the side view image; extracting the characteristic parameters of the side-looking image after binarization, comparing the extracted characteristic parameters of the side-looking image with preset characteristic parameters of the side-looking image, and judging whether the nut to be detected has defects or not;

carrying out binarization processing on the squint image; extracting characteristic parameters of the binarized squint image, comparing the extracted characteristic parameters of the squint image with preset squint image characteristic parameters, and judging whether the nut to be detected has defects or not;

the top view image, the side view image, and the oblique view image are images taken from right above, the side surface, and obliquely above the nut, respectively, when the nut is horizontally placed.

Preferably, "binarizing the overhead image" includes:

converting the overlook image into a gray image, and eliminating noise points after graying through Gaussian filtering;

converting the noise-reduced overlook image into a binary image through self-adaptive binarization operation;

and performing closed operation on the binary image to eliminate noise in the image.

Preferably, "binarizing the side view image" includes:

graying and histogram equalization are carried out on the side-looking image, and the contrast of the side-looking image is enhanced; eliminating noise points after graying through Gaussian filtering;

the image is binarized through an OTSU self-adaptive threshold method (an OTSU algorithm is a high-efficiency algorithm for binarizing the image proposed by OTSU of Japanese scholars in 1979), and the foreground is a low-gray-value area in the image;

Preferably, "binarizing the oblique-view image" includes:

graying and histogram equalization are carried out on the squint image, and the contrast of the squint image is enhanced; eliminating noise points after graying through Gaussian filtering;

binarizing the image by an OTSU self-adaptive threshold method, wherein the foreground is a high gray value area in the image;

Preferably, the preset top-view image characteristic parameters include: the method comprises the following steps of (1) presetting a maximum radius value of an inner circle of a nut and a preset minimum radius value of the inner circle of the nut;

the method comprises the following steps of extracting characteristic parameters of the binarized overhead view image, comparing the extracted characteristic parameters of the overhead view image with preset overhead view image characteristic parameters, and judging whether a nut to be detected has defects or not, wherein the method comprises the following steps:

extracting the radius value of the inner circle of the nut in the image according to the binarized overlook image;

and comparing the extracted radius value of the inner circle of the nut in the image with a preset maximum radius value of the inner circle of the nut and a preset minimum radius value of the inner circle of the nut respectively, and judging whether the nut to be detected has the defect of overlarge or undersize inner diameter.

Preferably, "extracting a radius value of an inner circle of the nut in the image according to the binarized overhead view image" includes:

finding a preset number of points on the inner circle of the nut according to the binarized overlook image;

based on the property that the perpendicular bisector of any two points on the circle passes through the circle center, the intersection point of the two perpendicular bisectors is obtained and is used as a candidate point of the circle center; repeatedly executing to obtain the circle center candidate points in a preset number;

discarding the outlier circle center candidate points, and solving the position average value of the remaining circle center candidate points as the detected circle center coordinates;

and respectively calculating the distance between the detected circle center coordinate and each point on the found nut inner circle according to the detected circle center coordinate and the found points on the nut inner circle with the preset number, and solving the average value of the distances to serve as the extracted radius value of the nut inner circle.

Preferably, the preset side view image characteristic parameters include: the image width ratio comprises a preset image left edge area, a preset image right edge area, a preset left and right boundary distance and an image width ratio;

"extract behind the binarization the characteristic parameter of side looking the image to the characteristic parameter that will extract side looking the image compares with preset side looking the image characteristic parameter, judges whether there is the defect in the nut that waits to detect", include:

determining the position of the nut in the image according to the side view image after binarization;

judging whether the nut to be detected is in lap joint with other nuts or not;

wherein, whether the judgement waits to detect the nut and has the condition with other nut overlap joints specifically does:

respectively extracting the positions of a left boundary and a right boundary in an image, and respectively judging whether the left boundary is positioned in a preset image left edge area or not and whether the right boundary is positioned in a preset image right edge area or not according to a preset image left edge area and a preset image right edge area;

if the left boundary is located in the preset image left edge area or the right boundary is located in the preset image right edge area, calculating the ratio of the distance between the left boundary and the right boundary to the image width, and judging whether the ratio is greater than the ratio of the distance between the left boundary and the right boundary to the image width; if so, the nut to be detected is considered to have the condition of being lapped with other nuts.

Preferably, the "determining the position of the nut in the image according to the side view image after binarization" includes:

counting the number of foreground pixel points of each line in the side-looking image after binarization to obtain a line projection vector;

counting the number of foreground pixel points of each column in the side-looking image after binarization to obtain a column projection vector;

and respectively searching the pixel value jump position in the row projection vector and the pixel value jump position in the column projection vector, and further determining the specific position of the nut to be detected in the image.

Preferably, the preset strabismus image characteristic parameters include: a preset white pixel accounts for a third threshold;

"extract behind the binarization the characteristic parameter of the oblique view image to the characteristic parameter of the oblique view image who will extract compares with preset oblique view image characteristic parameter, judges to detect whether there is a defect in the nut", include:

calculating the proportion of white pixels in an image area where the internal threads are located according to the binarized squint image;

and if the proportion of the white pixels in the image area where the internal thread is located is smaller than the preset third threshold value of the proportion of the white pixels, determining that the internal thread of the nut to be detected is not opened.

Preferably, "calculating the proportion of white pixels in the image region where the internal thread is located according to the binarized squint image" includes:

determining the position of the nut in the image according to the binarized squint image;

intercepting an image area where the internal thread is located according to the position of the nut in the image and the relative position of the preset internal thread in the nut;

performing closed operation on the intercepted image area where the internal thread is located, and eliminating noise points;

and respectively calculating the total number of pixels and the number of white pixels in the image area where the internal thread is located, and further calculating the ratio of the number of the white pixels to the total number of the pixels to obtain the proportion of the white pixels in the image area where the internal thread is located.

Preferably, "determining the position of the nut in the image according to the binarized squint image" includes: counting the number of foreground pixel points of each line in the oblique-view image after binarization to obtain a line projection vector; counting the number of foreground pixel points of each column in the oblique-view image after binarization to obtain a column projection vector;

Preferably, if the inner diameter of the bottom end of the qualified nut is greater than the inner diameter of the top end, and the top end is provided with a plurality of upper grooves, the preset top view image characteristic parameters further include: a preset radius difference threshold value and a preset radius increment l₁、l₂And l is₂<l₁；

After "judge that there is too big or too little defect of internal diameter in the nut to be detected", still include: if the nut to be detected has the defect of overlarge inner diameter, further judging whether the nut to be detected is inverted up and down; the upper part and the lower part are inverted, namely the top end of the nut to be detected is downward, the bottom end of the nut to be detected is upward, and the nut to be detected is horizontally placed;

wherein, whether the nut to be detected is inverted up and down is judged, and the method specifically comprises the following steps:

calculating the difference value between the extracted radius value r of the inner circle of the nut and the preset radius value of the inner circle of the nut;

if the calculated difference value is larger than the preset radius difference threshold value, taking a radius value on the binary overlook image as [ r, r + l ]₁]An annular region within the range such that the annular region includes the entire nutThe pixel point of (2);

and analyzing the connected domains of the annular area, wherein if at least one foreground connected domain exists, the foreground connected domain simultaneously meets the following two conditions:

pixel points in the foreground connected domain appear in four quadrants of the nut;

the distances between all foreground pixel points in the foreground connected domain and the circle center are larger than r + l₂；

The nut to be detected is considered to be inverted up and down, otherwise, the inner diameter is overlarge;

the four quadrants of the nut refer to four image areas, namely, an upper left image area, an upper right image area, a lower left image area and a lower right image area, which are obtained by horizontally and vertically splitting an image from the circle center.

Preferably, if the circumference of qualified nut is opened with the outer groove, then preset side view image characteristic parameter still includes: the preset third picture cutting height, the preset picture cutting width, the preset number of white pixel points and the preset number of outer grooves;

after "judge whether there is the condition of overlapping with other nuts in the nut that waits to detect", still include:

if the nut to be detected does not overlap with other nuts, judging whether the nut to be detected has the defect that the number of the outer grooves is not in accordance;

wherein, whether the judgement waits to detect the nut and has the defect that the outer groove quantity is inconsistent specifically is:

extracting the positions of the left boundary, the right boundary and the lower boundary in the image according to the side view image after binarization;

according to the positions of the extracted left and right boundaries and the lower boundary in the image, on the side-looking image before binarization, taking the lower boundary as a base and the preset third sectional image height as a height, respectively intercepting an area with a preset sectional image width from the left boundary to the right and intercepting an area with a preset sectional image width from the right boundary to the left to obtain an image of the left boundary area and an image of the right boundary area;

respectively carrying out binarization and closing operations on the two intercepted images;

respectively calculating the number of white pixel points in each white pixel communication area in the two images; respectively judging whether the number of white pixel points in each white pixel connected region is greater than the preset number of white pixel points, if so, considering the white pixel connected region as an outer groove;

and if the number of the outer grooves detected on at least one image of the two images is not equal to the preset number of the outer grooves, determining that the nut to be detected has the defect of non-conformity of the number of the outer grooves.

Preferably, if the top end of the qualified nut is provided with a plurality of upper grooves, the preset side view image characteristic parameters further include: the method comprises the steps of presetting a first cut-off height, a preset white pixel ratio first threshold value and a preset white pixel ratio second threshold value;

after "judge that it is inconsistent with the defect of outer groove quantity to wait to detect the nut", still include:

if the nut to be detected does not have the defect that the number of the outer grooves is not accordant, judging whether the nut to be detected has the defect of excessive cutting of the upper grooves;

wherein,

whether the nut to be detected has the defect of excessive cutting in the upper groove or not is judged, and the method specifically comprises the following steps:

extracting the positions of the left boundary, the right boundary and the upper boundary of the nut in the image according to the side view image after binarization;

taking the extracted left and right boundaries of the nut as two side edges, taking the upper boundary as a top edge, and intercepting a preset first sectional height from top to bottom to obtain an intercepted image area;

in the intercepted image area, respectively calculating the number of white pixel points in each white pixel communication area; respectively judging whether the number of white pixel points in each white pixel connected region is greater than the preset number of white pixel points, if so, considering the white pixel connected region as an upper groove; further calculating the number N of upper slots in the intercepted image area;

in the intercepted image area, respectively calculating the total number of all pixels in the area and the total number of white pixels, and further calculating the ratio of the total number of the white pixels to the total number of all pixels to be used as the proportion P of the white pixels in the area;

if the calculated ratio P between the upper groove number N and the white pixel satisfies the following condition:

determining that the nut to be detected has the defect of excessive cutting on the upper groove;

wherein, P₁、P₂Respectively, the preset white pixel accounts for a first threshold value, the preset white pixel accounts for a second threshold value, and P₁>P₂。

Preferably, if a plurality of upper grooves are formed in the qualified nut, the preset top view image characteristic parameters include: the preset maximum radius value of the inner circle of the nut, the preset minimum radius value of the inner circle of the nut and a preset radius increment l₃、l₄And l is₄<l₃(ii) a The preset side-view image characteristic parameters comprise: a preset second screenshot height and a preset white pixel proportion fourth threshold;

performing binarization processing on the side view image; extract behind the binarization the side view image's characteristic parameter to will extract the side view image's characteristic parameter compares with preset side view image characteristic parameter, judges to wait to detect whether there is defect in the nut "after, still include:

judging whether the nut to be detected has the defect of no upper groove or reverse upper groove opening according to the binarized overlook image;

according to the side view image after binarization, further distinguishing whether the nut to be detected has the defect that no upper groove is formed or the upper groove is reversely formed;

the method comprises the following steps of firstly, obtaining an overlook image, judging whether a nut to be detected has the defect that a groove is not opened or is reversely opened according to the overlook image after binarization, and specifically comprises the following steps:

extracting the radius value r of the inner circle of the nut in the image according to the binarized overlook image;

judging whether the extracted radius value of the inner circle of the nut is larger than the preset minimum radius value of the inner circle of the nut and smaller than the preset maximum radius value of the inner circle of the nut; if yes, taking a radius value on the binarized overlook image as [ r, r + l ]₃]An annular region within the range, the annular region comprising pixel points of the entire nut;

the distances between all foreground pixel points in the foreground connected domain and the circle center are larger than r + l₄；

Judging that the nut to be detected has the defect of no upper groove or reverse upper groove opening;

the four quadrants of the nut refer to four image areas of an upper left image area, an upper right image area, a lower left image area and a lower right image area which are obtained by horizontally and vertically splitting an image from the circle center;

"according to after the binarization side view image, further distinguish and wait to examine the nut and exist not to open the groove or go up the groove and open the defect of turning over", specifically do:

according to the side view image after binarization, positions of a left boundary, a right boundary and a lower boundary of the nut in the image are respectively extracted;

in the side view image after binarization, taking the left and right boundaries of the extracted nut as two side edges, and taking the lower boundary as a bottom edge, and intercepting a preset second screenshot height from bottom to top to obtain an intercepted image area;

calculating the total number of pixels and the number of white pixels in the intercepted image area, and further calculating the ratio of the number of the white pixels to the total number of the pixels to obtain the occupation ratio of the white pixels in the intercepted image area;

and when the occupation ratio of the obtained white pixels is larger than the preset white pixel occupation ratio fourth threshold, considering that the upper groove of the nut to be detected is reversely opened, and otherwise, considering that the upper groove of the nut to be detected is not opened.

In another aspect of the present invention, a nut defect detecting apparatus based on machine vision is provided, including: the device comprises a transmission device, a visual acquisition unit, an image detection unit and a nut classification unit;

the conveying equipment is used for placing and conveying the nuts to be detected; each nut to be detected is independently and horizontally placed on the conveying equipment; the visual acquisition unit comprises a photoelectric switch and three cameras which are respectively arranged right above, on the side surface and obliquely above the conveying equipment;

the photoelectric switch is used for triggering the three cameras to shoot simultaneously when the nut to be detected enters a target area; the three cameras are respectively used for shooting a top view image, a side view image and an oblique view image of the nut to be detected; the image detection unit is used for receiving the top view image, the side view image and the oblique view image which are shot by the three cameras and carrying out defect detection on the nut to be detected based on the nut defect detection method based on the machine vision; and the nut classification unit is used for classifying the nuts to be detected according to the defect detection result of the image detection unit.

Preferably, the nut sorting unit sorts the nuts into: defective, waste, abnormal, and accepted products;

the substandard product includes: the nut is not provided with the upper groove and the internal thread; the waste product, comprising: the inner diameter is too large or too small, the upper groove is reversely opened, and the upper groove cuts excessively; the abnormal article comprises: the nut is inverted up and down, the number of the outer grooves is not consistent, and the overlapping problem exists; the qualified product is the nut with the detection result meeting the requirement.

Preferably, the nut defect detecting apparatus further includes: a white light source; and the white light source is used for illuminating the nut to be detected.

Preferably, the nut sorting unit includes: a PLC (Programmable Logic Controller), a motor, three mechanical pushers and three nut accommodating devices;

the PLC is used for receiving the detection result sent by the image detection unit and controlling the motor to act according to the detection result; the motor drives the three mechanical push handles according to the instructions of the PLC; the three nut accommodating devices are respectively used for accommodating waste products, defective products and abnormal products; the three mechanical pushing hands correspond to the three nut accommodating devices one by one, and waste products, defective products or abnormal products are pushed into the corresponding nut accommodating devices under the driving of the motor.

The invention has the beneficial effects that:

the method comprises the steps of shooting images of the nut to be detected from three different angles, carrying out binarization processing, extracting characteristic parameters of the nut to be detected from the images after binarization processing, and comparing the characteristic parameters with preset characteristic parameters, so that the nuts with various defects are screened out. Compared with the traditional manual detection, the method can rapidly and efficiently detect all nuts one by one, has good classification effect, can greatly improve the production efficiency, and simultaneously obviously improves the qualification rate of the outgoing nuts.

Drawings

FIG. 1 is top, oblique, and side view images of a nut, and an example image of a typical defect and anomaly, in an embodiment of the present invention;

FIG. 2 is a schematic diagram of an embodiment of a machine vision based nut defect detection method of the present invention;

FIG. 3 is a schematic diagram of the machine vision-based nut defect inspection apparatus of the present invention;

fig. 4 is a schematic diagram of relative positions of a camera and a nut to be detected during photographing in an embodiment of the nut defect detection apparatus based on machine vision.

Detailed Description

Preferred embodiments of the present invention are described below with reference to the accompanying drawings. It should be understood by those skilled in the art that these embodiments are only for explaining the technical principle of the present invention, and are not intended to limit the scope of the present invention.

In addition to the general defects of overlarge or undersize nut inner diameter, process loss, reverse groove opening and the like mentioned in the background technology, the condition that a small amount of nuts of other types and the nuts of types to be detected are mixed together and conveyed to a quality inspection link also exists. In addition, for the automatic nut quality inspection device constructed based on the machine vision technology, the situations of wrong placement of nuts and nut overlapping also exist, the former means that the nuts enter quality inspection equipment on a production line in a mode that the bottoms of the nuts face upwards, and the latter means that a plurality of nuts are overlapped together and are not distinguished, and both the situations are caused by the fact that nut scattering and sorting operations in the previous links of the production line are unsuccessful. Because the three conditions (mixed with other types, wrong placement and overlapping) do not correspond to the quality problem, the three conditions are collectively called as abnormal products and are sorted without distinction, and the abnormal products can be scattered and sorted again and then conveyed to a quality inspection device for detection.

FIG. 1 is top, oblique, and side view images of a nut, and an example image of a typical flaw and a faulty part, in an embodiment of the invention; wherein (a) an overhead image; (b) an oblique view image; (c) a side view image; (d) side view images after being inverted up and down; (e) an overhead view image after being inverted up and down; (f) an overhead view image of the ungrooved; (g) side view images of the ungrooved; (h) side view images of the upper groove with an inverse shape; (i) squint images without internal threads; (j) side view images with different outer groove numbers; (k) a side view image of the upper groove cut over; (l) Side view images of the problem of bridging.

It is worth pointing out that although nuts of different specifications and oriented to different application scenarios may have their own respective defect types and determination criteria, they mostly follow principles similar to the present invention in image acquisition: the information of the nut surface is collected as completely as possible. In addition, different types of nuts have partial intersection on a defect set, and even if different defects exist, the detection algorithm has a certain commonality. Therefore, the invention has stronger reference significance even for other nuts.

FIG. 2 is a schematic diagram of the nut defect detection method based on machine vision according to the present invention. As shown in fig. 2, we examine the defects of the over-size or over-size of the inner diameter, and upside down, by analyzing the top view image of the nut; the problems of excessive cutting of the upper groove, inconsistent quantity of the outer grooves and overlapping are detected by analyzing a side view image of the nut; detecting the defect of the unopened internal thread by analyzing the squint image; the two conditions are specifically distinguished by analyzing the side view image after the nut is not grooved or is reversely grooved through analyzing the top view image.

The nut defect detection method of the embodiment comprises the following steps:

step S1, respectively acquiring a top view image, a side view image and an oblique view image of the nut to be detected;

step S2, performing binarization processing on the overhead view image; extracting characteristic parameters of the binarized overlook image, comparing the extracted characteristic parameters of the overlook image with preset characteristic parameters of the overlook image, and judging whether the nut to be detected has defects or not;

step S3, performing binarization processing on the lateral view image; extracting characteristic parameters of the side-looking image after binarization, comparing the extracted characteristic parameters of the side-looking image with preset characteristic parameters of the side-looking image, and judging whether the nut to be detected has defects or not;

step S4, performing binarization processing on the squint image; extracting characteristic parameters of the binarized squint image, comparing the extracted characteristic parameters of the squint image with preset characteristic parameters of the squint image, and judging whether the nut to be detected has defects or not;

the top view image, the side view image and the oblique view image referred to herein refer to images taken from directly above, horizontally laterally side and obliquely above the nut when the nut is horizontally placed, and the resolution in this embodiment may preferably be 2048 × 2048, 1280 × 850 and 800 × 850, respectively.

In this embodiment, the preset top view image characteristic parameters include: the preset maximum radius value of the inner circle of the nut and the preset minimum radius value of the inner circle of the nut.

Correspondingly, step S2 specifically includes:

in steps S211 to S213, binarization processing is performed on the overhead view image.

Step S211, converting the overlook image into a gray image, and eliminating noise points after graying through Gaussian filtering;

step S212, converting the noise-reduced overlook image into a binary image through self-adaptive binarization operation; in the generated binary image, foreground pixel points mainly correspond to edges in the image;

step S213 is to perform a close operation on the binary image to eliminate noise such as small black holes in the image, thereby obtaining a binarized overhead image.

In step S221-224, the radius value of the inner circle of the nut in the image is extracted from the binarized overhead view image.

Step S221, finding a preset number of points on the inner circle of the nut according to the binarized overlook image;

step S222, knowing the coordinates of two points on the circle based on the property that the perpendicular bisector of any two points on the circle passes through the center of the circle, the slope k and the y-axis intercept b of the perpendicular bisector of the two points can be obtained, as shown in formulas (1) and (2):

k＝(y1-y2)÷(x1-x2) (1)

b＝(y2×x1-y1×x2)÷(x1-x2) (2)

the intersection point of two perpendicular bisectors can be obtained from the slopes k and b, and is used as a candidate point of the circle center, as shown in formulas (3) and (4):

y＝k1×x+b1 (3)

y＝k2×x+b2 (4)

repeatedly executing formulas (1) - (4) according to the found points with the preset number on the inner circle of the nut to obtain the circle center candidate points with the preset number;

step S223, discarding the circle center candidate points of the outliers, and solving the position average value of the remaining circle center candidate points as the detected circle center coordinates;

step S224, respectively calculating the distance between the detected circle center coordinate and each point on the found nut inner circle according to the detected circle center coordinate and the found preset number of points on the nut inner circle, and calculating the average value of the distances to be used as the extracted radius value of the nut inner circle.

In step S230, the extracted radius value of the inner circle of the nut in the image is respectively compared with a preset maximum radius value of the inner circle of the nut and a preset minimum radius value of the inner circle of the nut, and whether the nut to be detected has a defect of an excessively large or excessively small inner diameter is determined.

For example, for a standard 26.5 ID nut, the preset maximum radius value of the inner circle of the nut is 13.386, and the preset minimum radius value of the inner circle of the nut is 13.106; if the radius value of the inner circle of the nut in the current image is larger than the preset maximum radius value of the inner circle of the nut, the defect that the inner diameter is too large is considered to exist, and if the radius value of the inner circle of the nut in the current image is smaller than the preset minimum radius value of the inner circle of the nut, the defect that the inner diameter is too small is considered to exist.

In this embodiment, the preset side view image characteristic parameters include: a preset left image edge region, a preset right image edge region, and a preset ratio of left and right boundary distance to image width (preferably 0.75 in this embodiment).

Correspondingly, step S3 specifically includes:

in steps S311 to 313, the side view image is subjected to binarization processing.

Step S311, carrying out graying and histogram equalization on the side view image, and enhancing the contrast of the side view image; eliminating noise points after graying through Gaussian filtering;

step S312, binaryzation is carried out on the image through an OTSU self-adaptive threshold method; because the nut area is darker than the background, a binary image is generated, and foreground pixel points mainly correspond to a low-gray-value area in the image;

and step 313, performing closed operation on the binary image, eliminating noise in the image, and obtaining a side view image after binarization processing.

In steps S321-S323, the position of the nut in the image is determined according to the side view image after binarization.

Step S321, counting the number of foreground pixel points in each line in the side-looking image after binarization to obtain a line projection vector for recording the number of foreground pixel points in each line, wherein the dimension of the line projection vector is equal to the height of the image;

step S322, counting the number of foreground pixel points in each row in the side-looking image after binarization to obtain a row projection vector for recording the numerical value of the foreground pixel in each row, wherein the dimension of the row projection vector is equal to the width of the image;

in step S323, since the foreground is mainly the nut region, the specific position of the nut to be detected in the image can be determined by respectively searching the pixel value jump position in the row projection vector and the pixel value jump position in the column projection vector.

In steps S331 to S332, it is determined whether the nut to be detected is overlapped with other nuts.

Step S331, respectively extracting positions of a left boundary and a right boundary in an image, and respectively judging whether the left boundary is located in a preset image left edge area or not and whether the right boundary is located in a preset image right edge area or not according to the preset image left edge area and the preset image right edge area;

step S332, if the left boundary is located in the preset image left edge area or the right boundary is located in the preset image right edge area, calculating a ratio of a distance between the left boundary and the right boundary to an image width, and judging whether the ratio is greater than a ratio of a preset left boundary distance to a preset right boundary distance to the image width; if so, the nut to be detected is considered to have the condition of being lapped with other nuts.

In this embodiment, the preset strabismus image characteristic parameters include: the preset white pixel accounts for the third threshold (preferably 0.1 in the present embodiment).

Correspondingly, step S4 specifically includes:

in steps S411 to S413, the oblique-view image is subjected to binarization processing.

Step S411, carrying out graying and histogram equalization on the squint image, and enhancing the contrast of the squint image; eliminating noise points after graying through Gaussian filtering;

step S412, carrying out binarization on the image by an OTSU self-adaptive threshold method; the foreground which we want to obtain is an internal thread in the squint image, and in the binary image generation, foreground pixel points mainly correspond to a high-gray-value area in the image;

in step S413, a closed operation is performed on the binary image to remove noise in the image, thereby obtaining a binary processed squint image.

In steps S421-S423, the position of the nut in the image is determined according to the binarized squint image. The specific process can be seen in steps S321-S323, and the position of the nut in the image is determined according to the side view image after binarization.

In steps S431 to S433, the proportion of white pixels in the image region where the internal thread is located is calculated from the binarized squint image.

Step S431, intercepting an image area where the internal thread is located according to the position of the nut in the image and the relative position of the preset internal thread in the nut;

step S432, performing closed operation on the intercepted image area where the internal thread is located, and eliminating noise points;

step S433, calculating the total number of pixels and the number of white pixels in the image area where the internal thread is located, and further calculating the ratio between the number of white pixels and the total number of pixels, to obtain the proportion of white pixels in the image area where the internal thread is located.

In step S440, the white pixel proportion calculated in step S433 is compared with a preset white pixel proportion third threshold, and if the proportion of the white pixel in the image region where the internal thread is located is smaller than the preset white pixel proportion third threshold, it is determined that the nut to be detected is not internally threaded.

In this embodiment, if the internal diameter of qualified nut bottom is greater than the internal diameter on top, and the top is opened has a plurality of upper grooves, then predetermined overlook image characteristic parameter, still include: a preset radius difference threshold value and a preset radius increment l₁、l₂And l is₂<l₁；

Accordingly, after step S230, step S240 is further included:

in step S240, if the nut to be detected has a defect of an excessively large inner diameter, further determining whether the nut to be detected is upside down; the upper part and the lower part are inverted, and the top end of the nut to be detected is downward, the bottom end of the nut to be detected is upward and the nut to be detected is horizontally placed.

step S241, calculating the difference value between the extracted radius value r of the inner circle of the nut and a preset radius value of the inner circle of the nut;

step S242, if the calculated difference is greater than the preset radius difference threshold, taking a radius value of [ r, r + l ] on the binarized overhead view image₁]An annular region within the range, such that the annular region includes pixel points of the entire nut;

step S243, analyzing the connected domain of the annular area to obtain one or more foreground connected domains; if at least one foreground connected domain exists, the foreground connected domain simultaneously meets the following two conditions: (1) pixel points in the foreground connected domain appear in four quadrants of the nut; (2) the distances between all foreground pixel points in the foreground connected domain and the circle center are larger than r + l₂(ii) a And considering that the nut to be detected is inverted up and down, otherwise, the inner diameter is overlarge. The four quadrants of the nut refer to four image areas of the upper left, the upper right, the lower left and the lower right obtained by horizontally and vertically splitting the image from the circle center.

In this embodiment, if the circumference of qualified nut is opened there is the outer groove, then preset side view image characteristic parameter still includes: a preset third screenshot height (9/10 for the qualified nut height in this embodiment), a preset screenshot width (1/10 for the qualified nut width in this embodiment), a preset number of white pixels (preferably 50 in this embodiment), and a preset number of outer slots (5 in this embodiment).

Accordingly, after step S332, step S340 is further included:

in step S340, if the nut to be detected does not overlap with other nuts, determining whether the nut to be detected has a defect that the number of outer grooves is not in conformity;

wherein, judge and wait to detect whether there is the inconsistent defect of outer groove quantity in the nut, specifically do:

step S341, extracting the positions of the left and right boundaries and the lower boundary in the image according to the binarized side view image;

step 342, according to the positions of the extracted left and right boundaries and the lower boundary in the image, on the side view image before binarization, taking the lower boundary as a base and the preset third image height as a height, respectively intercepting an area with a preset image capture width from the left boundary to the right and an area with a preset image capture width from the right boundary to the left to obtain an image of the left boundary area and an image of the right boundary area;

step S343, carry on binarization and close operation to two pictures intercepted separately;

step S344, calculating the number of white pixel points in each white pixel connected region in the two images respectively; respectively judging whether the number of white pixel points in each white pixel connected region is greater than the preset number of white pixel points, if so, considering the white pixel connected region as an outer groove;

step S345, if the number of the outer slots detected on at least one of the two images is not equal to the preset number of the outer slots, determining that the nut to be detected has a defect that the number of the outer slots is not in conformity.

In this embodiment, if the top of qualified nut is opened there are a plurality of upper grooves, then preset side view image characteristic parameter still includes: a preset first cut height (1/4 for the qualified nut height in this embodiment), a preset number of white pixels (200 for this embodiment), and a preset white pixel ratio of the first threshold P₁(0.3 in the present embodiment), and the predetermined white pixel ratio is the second threshold value P₂(0.2 in this example).

Accordingly, after step S345, step S350 is further included:

in step S350, if the nut to be detected does not have the defect that the number of the outer grooves is not in conformity with the number of the outer grooves, whether the nut to be detected has the defect that the cutting of the upper groove is excessive is judged;

wherein, step S351, judge whether the nut that waits to detect exists the excessive defect of last groove cutting, specifically do:

step S352, extracting the positions of the left and right boundaries and the upper boundary of the nut in the image according to the side view image after binarization;

step S353, taking the extracted left and right boundaries of the nut as two side edges, taking the upper boundary as a top edge, and intercepting a preset first sectional image height from top to bottom to obtain an intercepted image area;

step S354, respectively calculating the number of white pixel points in each white pixel communication area in the intercepted image area; respectively judging whether the number of white pixel points in each white pixel connected region is greater than the preset number of white pixel points, if so, considering the white pixel connected region as an upper groove; further calculating the number N of upper slots in the intercepted image area;

step S355, in the intercepted image area, respectively calculating the total number of all pixels and the total number of white pixels in the area, and further calculating the ratio of the total number of the white pixels to the total number of all pixels as the proportion P of the white pixels in the area;

in step S356, if the calculated ratio P between the upper bin number N and the white pixel satisfies the formula (5):

In the present invention, the defect detection can also be performed by combining images in two or three different directions, for example: if a plurality of upper grooves are formed in the qualified nut, the defects that the upper grooves are not formed or the upper grooves are reversely formed need to be detected by combining the top view image and the side view image.

In this embodiment, if a plurality of upper grooves are formed on the qualified nut, the preset overlook image characteristic parameters further include: the preset maximum radius value of the inner circle of the nut, the preset minimum radius value of the inner circle of the nut and a preset radius increment l₃、l₄And l is₄<l₃(ii) a The preset side-looking image characteristic parameters further comprise: a preset second screenshot height (1/6 for the qualified nut height in this embodiment), and a preset white pixel fraction of a fourth threshold (preferably 0.05 in this embodiment);

accordingly, after step S356, step S360 is further included:

in step 360, judging whether the nut to be detected has the defect of no upper groove or reverse upper groove opening according to the binarized overlook image; and further distinguishing whether the nut to be detected has the defect that the upper groove is not opened or is reversely opened according to the side view image after binarization.

The method comprises the following steps:

step S361, extracting the radius value r of the inner circle of the nut in the image according to the binarized overlook image;

step S362, judging whether the extracted radius value of the inner circle of the nut is larger than a preset minimum radius value of the inner circle of the nut and smaller than the preset maximum radius value of the inner circle of the nut; if yes, taking a radius value on the binarized overlook image as [ r, r + l ]₃]An annular region of the region within which the magnetic field is generated,the annular region comprises pixel points of the whole nut;

step S363, performing connected domain analysis on the annular region to obtain one or more foreground connected domains; if at least one foreground connected domain exists, the foreground connected domain simultaneously meets the following two conditions: (1) pixel points in the foreground connected domain appear in four quadrants of the nut; (2) the distances between all foreground pixel points in the foreground connected domain and the circle center are larger than r + l₄Judging that the nut to be detected has the defect of no upper groove or reverse upper groove opening; the four quadrants of the nut refer to four image areas of an upper left image area, an upper right image area, a lower left image area and a lower right image area which are obtained by horizontally and vertically splitting an image from the circle center;

step S364, respectively extracting the positions of the left boundary, the right boundary and the lower boundary of the nut in the image according to the side view image after binarization;

step S365, in the side view image after binarization, taking the left and right boundaries of the extracted nut as two side edges, and taking the lower boundary as a bottom edge, and intercepting a preset second screenshot height from bottom to top to obtain an intercepted image area;

step 366, calculating the total number of pixels in the intercepted image area and the number of white pixels, and further calculating the ratio of the number of the white pixels to the total number of the pixels to obtain the ratio of the white pixels in the intercepted image area;

step S367, when the occupation ratio of the obtained white pixels is greater than a preset white pixel occupation ratio fourth threshold, determining that the groove on the nut to be detected is reversed, otherwise, determining that the groove on the nut to be detected is not opened.

Fig. 3 is a schematic diagram of a structure of an embodiment of the nut defect detecting device based on machine vision, and fig. 4 is a schematic diagram of a relative position between a camera and a nut to be detected during photographing in the embodiment of the nut defect detecting device based on machine vision. The defect detection device of the embodiment divides the nut to be detected into: waste products, defective products, abnormal products and qualified products. Wherein the waste product comprises: the inner diameter is too large or too small, the upper groove is reversely opened, and the upper groove is excessively cut; the inferior goods include: no groove is formed and no internal thread is formed; the abnormal product comprises: the number of the outer grooves is not consistent, and the outer grooves are inverted and lapped up and down.

As shown in fig. 3, the nut defect detecting apparatus 10 of the present embodiment includes: the device comprises a transmission device 110, a vision acquisition unit 120 (comprising a photoelectric switch 121, and three cameras 122 and 124 respectively arranged right above, at the side and at the oblique upper side of the transmission device 110), an image detection unit 130, a nut classification unit 140 (a PLC controller 141, a motor 142, three mechanical pushers 143 and 145, and three nut accommodating devices 146 and 148). As shown in fig. 4, when photographing, the three cameras 122, 123, and 124 are located right above, at the side, and obliquely above the nut 20 to be inspected, respectively.

Wherein, the conveying device 110 is used for placing and conveying the nut 20 to be detected; each nut 20 to be detected is independently and horizontally placed on the conveying device 120; the photoelectric switch 121 is used for triggering the three cameras to shoot simultaneously when the nut 20 to be detected enters the target area; the three cameras are respectively used for shooting a top view image, a side view image and an oblique view image of the nut 20 to be detected; the image detection unit 130 is configured to receive the top view image, the side view image and the oblique view image captured by the three cameras, and perform defect detection on the nut 20 to be detected based on the above-mentioned nut defect detection method based on machine vision; the nut classification unit 140 is configured to classify nuts to be detected according to the defect detection result of the image detection unit 130; the PLC 141 is used for receiving the detection result sent by the image detection unit 130 and controlling the motor 142 to act according to the detection result; the motor 142 drives any one of the three mechanical pushing hands 143 and 145 at a time according to the instruction of the PLC 141; three nut accommodating devices 146 and 148 for accommodating waste products, defective products and abnormal products respectively; the three mechanical pushing hands 143 and 145 correspond to the three nut accommodating devices 146 and 148 one by one respectively, the motors 142 drive the waste products, defective products or abnormal products which are detected at present to be pushed into the corresponding nut accommodating devices, and the normal nuts are continuously left on the conveying equipment to be conveyed to the next link.

In this embodiment, the nut sorting unit 140 sorts nuts into: defective, waste, abnormal, and accepted products;

wherein, the substandard product includes: the nut is not provided with the upper groove and the internal thread; the waste products include: the inner diameter is too large or too small, the upper groove is reversely opened, and the upper groove cuts excessively; the abnormal product comprises: the nut is inverted up and down, the number of the outer grooves is not consistent, and the overlapping problem exists; the qualified product is a nut (i.e., a nut without the above problems or defects) whose detection result meets the requirements.

The nut defect detecting apparatus 10 of the present embodiment further includes: a white light source 150 (not shown in FIG. 3); and a white light source 150 for illuminating the nut to be inspected so that a clear image can be photographed.

Those of skill in the art will appreciate that the method steps of the examples described in connection with the embodiments disclosed herein may be embodied in electronic hardware, computer software, or combinations of both, and that the components and steps of the examples have been described above generally in terms of their functionality in order to clearly illustrate the interchangeability of electronic hardware and software. Whether such functionality is implemented as electronic hardware or software depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.

So far, the technical solutions of the present invention have been described in connection with the preferred embodiments shown in the drawings, but it is easily understood by those skilled in the art that the scope of the present invention is obviously not limited to these specific embodiments. Equivalent changes or substitutions of related technical features can be made by those skilled in the art without departing from the principle of the invention, and the technical scheme after the changes or substitutions can fall into the protection scope of the invention.

Claims

1. A nut defect detection method based on machine vision is characterized by comprising the following steps:

the top view image, the side view image and the oblique view image are images photographed from right above, a side surface and an oblique upper side of the nut, respectively, when the nut is horizontally placed;

wherein,

the preset overlook image characteristic parameters comprise: the method comprises the following steps of (1) presetting a maximum radius value of an inner circle of a nut and a preset minimum radius value of the inner circle of the nut;

comparing the extracted radius value of the inner circle of the nut in the image with a preset maximum radius value of the inner circle of the nut and a preset minimum radius value of the inner circle of the nut respectively, and judging whether the nut to be detected has the defect of overlarge or undersize inner diameter;

"extracting the radius value of the inner circle of the nut in the image according to the binarized overlook image" includes:

2. The nut defect detection method according to claim 1, wherein the "binarizing the overhead view image" includes:

3. The nut defect detection method according to claim 1, wherein the "binarizing processing the side view image" includes:

binarizing the image by an OTSU self-adaptive threshold method, wherein the foreground is a low gray value area in the image;

4. The nut defect detection method according to claim 1, wherein the "binarizing processing the oblique view image" includes:

5. The nut defect detecting method according to claim 3,

the preset side-view image characteristic parameters comprise: the image width ratio comprises a preset image left edge area, a preset image right edge area, a preset left and right boundary distance and an image width ratio;

judging whether the nut to be detected is in lap joint with other nuts or not;

wherein,

the judgment of whether the nut to be detected is lapped with other nuts specifically comprises the following steps:

6. The nut defect detection method according to claim 5, wherein determining the position of the nut in the image according to the side view image after binarization comprises:

7. The nut defect detecting method according to claim 4,

the preset strabismus image characteristic parameters comprise: a preset white pixel accounts for a third threshold;

8. The nut defect detection method according to claim 7, wherein calculating the proportion of white pixels in the image area where the internal thread is located according to the binarized squint image comprises:

9. The nut defect detection method according to claim 8, wherein determining the position of the nut in the image according to the binarized squint image comprises:

counting the number of foreground pixel points of each line in the oblique-view image after binarization to obtain a line projection vector;

counting the number of foreground pixel points of each column in the oblique-view image after binarization to obtain a column projection vector;

10. The nut defect detecting method according to claim 1,

if the internal diameter of qualified nut bottom is greater than the internal diameter on top, and the top is opened has a plurality of upper grooves, then predetermined overlook image characteristic parameter still includes: a preset radius difference threshold value and a preset radius increment l₁、l₂And l is₂<l₁；

After "judge that there is too big or too little defect of internal diameter in the nut to be detected", still include:

if the nut to be detected has the defect of overlarge inner diameter, further judging whether the nut to be detected is inverted up and down; the upper part and the lower part are inverted, namely the top end of the nut to be detected is downward, the bottom end of the nut to be detected is upward, and the nut to be detected is horizontally placed;

wherein,

the judgment of whether the nut to be detected is inverted up and down specifically comprises the following steps:

if the calculated difference value is larger than the preset radius difference threshold value, taking a radius value on the binary overlook image as [ r, r + l ]₁]An annular region within the range, the annular region comprising pixel points of the entire nut;

11. The nut defect detection method according to claim 5, wherein if an outer groove is formed in the circumferential direction of the qualified nut, the preset side view image characteristic parameters further comprise: the preset third picture cutting height, the preset picture cutting width, the preset number of white pixel points and the preset number of outer grooves;

wherein,

the judgment of whether the nut to be detected has the defect that the number of the outer grooves is inconsistent is specifically as follows:

12. The nut defect detecting method according to claim 11,

if the top of qualified nut is opened there are a plurality of upper grooves, then preset side view image characteristic parameter still includes: the method comprises the steps of presetting a first cut-off height, a preset white pixel ratio first threshold value and a preset white pixel ratio second threshold value;

wherein,

13. The nut defect detecting method according to claim 1,

if the qualified nut is provided with a plurality of upper grooves, the preset overlook image characteristic parameters comprise: the preset maximum radius value of the inner circle of the nut, the preset minimum radius value of the inner circle of the nut and a preset radius increment l₃、l₄And l is₄<l₃(ii) a The preset side-view image characteristic parameters comprise: a preset second screenshot height and a preset white pixel proportion fourth threshold;

wherein,

"according to the binary overlook image, judge whether there is not the nut to be detected that has not opened the groove or opened the defect that the groove is opened reversely", specifically:

14. A nut defect detection device based on machine vision, characterized by comprising: the device comprises a transmission device, a visual acquisition unit, an image detection unit and a nut classification unit;

wherein,

the conveying equipment is used for placing and conveying the nut to be detected; each nut to be detected is independently and horizontally placed on the conveying equipment;

the visual acquisition unit comprises a photoelectric switch and three cameras which are respectively arranged right above, on the side surface and obliquely above the conveying equipment;

the photoelectric switch is used for triggering the three cameras to shoot simultaneously when the nut to be detected enters a target area;

the three cameras are respectively used for shooting a top view image, a side view image and an oblique view image of the nut to be detected;

the image detection unit is used for receiving the top view image, the side view image and the oblique view image which are shot by the three cameras and carrying out defect detection on the nut to be detected based on the nut defect detection method based on the machine vision of any one of claims 1 to 13;

and the nut classification unit is used for classifying the nuts to be detected according to the defect detection result of the image detection unit.

15. The nut defect detecting device according to claim 14, wherein the nut sorting unit sorts the nuts into: defective, waste, abnormal, and accepted products;

the substandard product includes: the nut is not provided with the upper groove and the internal thread;

the waste product, comprising: the inner diameter is too large or too small, the upper groove is reversely opened, and the upper groove cuts excessively;

the abnormal article comprises: the nut is inverted up and down, the number of the outer grooves is not consistent, and the overlapping problem exists;

the qualified product is the nut with the detection result meeting the requirement.

16. The nut defect detecting device according to claim 14, further comprising: a white light source;

and the white light source is used for illuminating the nut to be detected.

17. The nut defect detecting device according to claim 15, wherein the nut sorting unit includes: the device comprises a PLC, a motor, three mechanical push handles and three nut accommodating devices;

the PLC is used for receiving the detection result sent by the image detection unit and controlling the motor to act according to the detection result;

the motor drives the three mechanical push handles according to the instructions of the PLC;

the three nut accommodating devices are respectively used for accommodating waste products, defective products and abnormal products;

the three mechanical pushing hands correspond to the three nut accommodating devices one by one, and waste products, defective products or abnormal products are pushed into the corresponding nut accommodating devices under the driving of the motor.