CN117351314A

CN117351314A - Glass bottle defect identification method and system

Info

Publication number: CN117351314A
Application number: CN202311379015.9A
Authority: CN
Inventors: 陈国荣; 杨俊良; 袁钰; 杜重灵; 段泓俊; 魏金圣
Original assignee: Chongqing University of Science and Technology
Current assignee: Chongqing University of Science and Technology
Priority date: 2023-10-23
Filing date: 2023-10-23
Publication date: 2024-01-05

Abstract

The invention relates to the technical field of image processing, and particularly discloses a glass bottle defect identification method and system, wherein a bottle mouth, a bottle bottom and a bottle body of a glass bottle are shot and imaged to obtain corresponding three views, and pretreatment is carried out to weaken and inhibit noise of the acquired three views to obtain a pretreated image; the single-channel preprocessed images are further fused, the images in three directions are subjected to feature extraction, and the images are fused into three-channel feature images according to different weights to detect defects, so that the defect detection accuracy can be improved, and meanwhile, the detection task can be completed at a high speed; then, the computing capacity of the convolutional neural network is utilized to collect and process image features of different degrees of images, then, the convolution and pooling of different degrees are carried out to carry out feature fusion, then, convolution up-sampling and the like are carried out to obtain a target feature image, the image features can be fully extracted, and the defect recognition accuracy is improved.

Description

Glass bottle defect identification method and system

Technical Field

The invention relates to the technical field of image processing, in particular to a glass bottle defect identification method and system.

Background

Quality inspection is a critical but also challenging task in the glass bottle manufacturing process. Aiming at the current market situation, domestic glass bottles have high market share, so the detection requirement on the glass bottles is huge. However, currently, in terms of glass bottle quality detection, the manual method is still dominant. The image of the glass bottle is shot through multiple angles by using hardware such as an industrial camera and a proper light source, so that all parts of the bottle body are ensured to be captured under the sufficient illumination condition. These images are then analyzed and screened using image processing algorithms to achieve machine vision inspection. The method is widely applied in the global scope, and compared with the traditional manual or semi-manual detection method, the machine vision detection method has the advantages of higher efficiency, stronger stability, higher detection speed and the like.

It is worth mentioning that the machine vision detection completely replaces manual operation, and completely avoids direct contact between operators and glass bottles and related equipment. This advantage has led to a wide acceptance in the inspection industry for machine vision inspection, and in particular, in glass bottle inspection, as a promising technique. However, the existing glass bottle defect recognition technology adopts a complex recognition network, such as a glass bottle opening defect detection method, device, equipment and storage medium disclosed by application number 202211234402.9, which adopts a multi-scale multi-attention convolutional neural network MSMA-CNN module to detect defects, and the network is complex and the recognition accuracy is still not high enough.

Disclosure of Invention

The invention provides a glass bottle defect identification method and a system, which solve the technical problems that: how to realize high-precision defect identification of glass bottles by a simpler neural network.

In order to solve the technical problems, the invention provides a glass bottle defect identification method, which comprises the following steps:

s1, shooting aiming at a bottle opening, a bottle bottom and a bottle body of a glass bottle to obtain a corresponding bottle opening view, a bottle bottom view and a bottle body view;

s2, preprocessing the bottleneck view, the bottle bottom view and the bottle body view to obtain a preprocessed bottleneck view, a preprocessed bottle bottom view and a preprocessed bottle body view;

s3, fusing the pretreated bottle mouth view, the pretreated bottle bottom view and the pretreated bottle body view to obtain a fused image;

s4, inputting the fusion image into a trained defect recognition network to perform defect position positioning, and outputting a defect recognition result image.

Further, in the step S4, the defect recognition network performs defect location positioning, and outputs a defect recognition result image, and specifically includes the steps of:

s41, extracting features of the fusion image to generate a target feature image;

s42, performing edge detection on the characteristic image to obtain an edge detection image;

s43, performing defect position positioning on the edge detection image to obtain a defect identification result image.

Further, the step S41 specifically includes the steps of:

s411, inputting the fusion image into a convolutional neural network to generate a first feature map;

s412, inputting the first feature map into a first convolution layer, a second convolution layer, a third convolution layer, a fourth convolution layer and a maximum pooling layer respectively to obtain a second feature map, a third feature map, a fourth feature map, a fifth feature map and a sixth feature map, and fusing and splicing the second feature map, the third feature map, the fourth feature map, the fifth feature map and the sixth feature map to obtain a seventh feature map; performing convolution processing with the convolution kernel of 1×1 on the first feature map to obtain an eighth feature map;

s413, performing convolution processing with the convolution kernel of 1 multiplied by 1 on the seventh feature map, and then performing up-sampling and fusion and splicing with the eighth feature map to obtain a ninth feature map;

and S414, performing convolution processing of the ninth feature map with a convolution kernel of 3 multiplied by 3, and then performing up-sampling to obtain a target feature image.

Further, in the step S412:

the first convolution layer executes cavity convolution processing with a convolution kernel of 1 multiplied by 1 and an expansion coefficient of r=1 on the input first feature map to obtain the second feature map;

the second convolution layer executes a cavity convolution process with a convolution kernel of 3×3 and an expansion coefficient of r=6 on the input first feature map to obtain the third feature map;

the third convolution layer performs a hole convolution process with a convolution kernel of 3×3 and an expansion coefficient of r=12 on the input first feature map to obtain the fourth feature map;

and the fourth convolution layer performs a hole convolution process with a convolution kernel of 3×3 and an expansion coefficient of r=18 on the input first feature map, so as to obtain the fifth feature map.

Further, the step S42 includes the steps of:

s421, calculating the amplitude and the direction of the gradient of each point of the target characteristic image by using the finite difference of the first-order partial derivatives;

s422, detecting and scanning the target feature image, judging whether the amplitude of the point on the edge is the point with the largest amplitude in the same direction, if so, reserving, and if not, deleting;

s423, judging the point with the pixel value larger than the threshold value T2 in the reserved points as a boundary point, discarding the point smaller than the threshold value T1, wherein T1 is smaller than T2, judging whether the point is connected with a real boundary point or not for the point between the threshold values T1 and T2, judging that the point is the boundary point if the point is connected with the real boundary point, and discarding the point if the point is not connected with the real boundary point;

s424, carrying out Canny edge detection on the target feature image subjected to the image enhancement processing in the steps S421 to S423 to obtain an edge detection image, wherein the edge detection image comprises an outer contour and an inner contour.

Further, in the step S421, the amplitude of the target feature image is calculatedAnd direction->The formula of (2) is as follows:

respectively performing functions after the target characteristic image and the two first-order difference convolution templates are operated;

the step S43 specifically includes the steps of:

s431, reserving pixels of a detection area between the outer contour and the inner contour in the edge detection image, and removing the pixels in the outer contour and the inner contour;

s432, performing threshold segmentation on the detection region to enable pixel values at the defect to be 255 and the rest pixel values to be 0, so as to obtain a communication region of the defect;

s433, judging whether the connected areas are the specified defects according to the sizes of the connected areas, if so, reserving the connected areas, judging that the glass bottle is a defective glass bottle, and if not, setting the pixel value of the pixel point of the connected areas to 0.

Further, in the training process, the loss function of the defect recognition network is designed as follows:

Loss＝0.4*Loss _obj +0.3*Loss _rect +0.3*Loss _cls

wherein, loss _obj Representing confidence Loss, loss _rect Representing rectangular box Loss, loss _cls Representing the classification loss, calculated by the following formula:

wherein p is _o Representing target confidence scores in a prediction box, p _iou IOU value, w, representing prediction frame and target frame corresponding thereto _obj Representation calculationWeight of time positive sample, +.>Represents p _o And p _iou Two kinds of cross entropy loss in the middle; IOU represents the overlapping proportion of the predicted frame and the actual frame, b represents the center point of the predicted frame, b ^gt Represents the center point of the actual frame, c represents the diagonal length of the smallest rectangle containing the predicted frame and the actual frame, w represents the width of the predicted frame, w ^gt Representing the width of the actual frame, C _w A width of a minimum rectangle representing a prediction frame and an actual frame, h represents a height of the prediction frame, h ^gt Representing the height of the actual frame, C _h The width of the smallest rectangle representing the predicted and actual frames, ρ () represents the euclidean distance; c _p Representing prediction categories，c _gt Representing the true category, w _cls Representing the calculation->Weight of time positive sample, +.>Representation c _p And c _gt Two classes in between cross entropy loss.

Further, the step S2 performs pretreatment, specifically includes the steps of:

s21, performing Gaussian filtering on the bottleneck view, the bottle bottom view and the bottle body view to obtain respective first intermediate images;

s22, performing median filtering on the first intermediate images to obtain respective second intermediate images;

s23, carrying out gray value transformation on the second intermediate images to obtain respective third intermediate images;

s24, sharpening the third intermediate image to obtain respective fourth intermediate images;

s25, performing frequency domain filtering on the fourth intermediate image to obtain the pretreated bottle mouth view, the pretreated bottle bottom view and the pretreated bottle body view;

in the step S23, the formula for performing the gray value conversion is:

wherein g2 (x, y) is a second intermediate image, the gray value range of the original pixel point is [ a, b ], the gray value range which is wanted to be reached by image enhancement is [ c, d ], and the output third intermediate image is g3 (x, y);

in the step S24, the gradient value is used as a sharpening output, where a modulus point of the gradient vector at a midpoint (x, y) of the third intermediate image g3 (x, y), that is, a fourth intermediate image g4 (x, y), is expressed as:

g4(x,y)＝|Δ _x g3(x,y)|+|Δ _y g3(x,y)|

Δ _x g3(x,y)、Δ _y g3 (x, y) represent the first order difference in the x direction and the first order difference in the y direction at the midpoint (x, y) of the third intermediate image g3 (x, y), respectively, expressed as:

further, the step S3 specifically includes the steps of:

s31, carrying out a shared convolution operation on the pretreated bottleneck view, the pretreated bottle bottom view and the pretreated bottle body view of a single channel, wherein the convolution kernel size is 3 multiplied by 3, then activating the pretreated bottle bottom view, the pretreated bottle bottom view and the pretreated bottle body view through a ReLU function, and extracting features in a filling mode of same to obtain a three-view feature map;

s32, splicing the three-view feature images by adopting feature weights of 0.5 of the bottle body, 0.3 of the bottle mouth and 0.2 of the bottle bottom to form fusion features;

s33, the fusion characteristic is subjected to convolution operation, the convolution kernel size is 1 multiplied by 1, the channel number of the output image is adjusted to be 3, and then the fusion image of 3 channels, the size of which is consistent with that of the input image in the step S31, is finally output through a linear activation function.

The invention also provides a glass bottle defect identification system, which is characterized in that: the system comprises an image acquisition module, an image preprocessing module and a defect identification module; the image acquisition module, the image preprocessing module, the image fusion module and the defect identification module are respectively used for executing the steps S1, S2, S3 and S4 in the method;

the image acquisition module comprises a mechanical arm, a monocular camera and two reflecting mirrors, wherein the monocular camera is arranged right in front of a glass bottle on the detection station, and the two reflecting mirrors are symmetrically arranged at the left rear and the right rear of the glass bottle; the mechanical arm is used for:

the method comprises the steps that a glass bottle is vertically placed on a detection station, and at the moment, a monocular camera shoots a bottle body of the glass bottle to obtain a bottle body image;

the middle part of the bottle body of the glass bottle which is vertically placed is clamped to rotate, so that the bottle mouth of the glass bottle faces the monocular camera, and at the moment, the camera shoots the bottle mouth of the glass bottle to obtain a bottle mouth image;

the middle part of the bottle body of the glass bottle which is vertically placed is clamped to rotate, so that the bottle bottom of the glass bottle is opposite to the monocular camera, and at the moment, the camera shoots the bottle bottom of the glass bottle to obtain a bottle bottom image;

and clamping the middle part of the bottle body of the glass bottle to separate the glass bottle from the detection station.

According to the glass bottle defect identification method and system, the bottle mouth, the bottle bottom and the bottle body of the glass bottle are shot and imaged to obtain the corresponding three views, and pretreatment is carried out to weaken and inhibit noise of the collected three views to obtain a pretreated image; the single-channel preprocessed images are further fused, the images in three directions are subjected to feature extraction, and the images are fused into three-channel feature images according to different weights to detect defects, so that the defect detection accuracy can be improved, and meanwhile, the detection task can be completed at a high speed; then, the computing capacity of the convolutional neural network is utilized to collect and process image features of different degrees of images, then, the convolution and pooling of different degrees are carried out to carry out feature fusion, then, convolution up-sampling and the like are carried out to obtain a target feature image, the image features can be fully extracted, and the defect recognition accuracy is improved; and finally, carrying out edge detection and defect position positioning on the target characteristic image so as to identify whether the acquired image has defects or not and position the defects, thereby having higher identification precision. The invention adopts the convolutional neural network and a plurality of convolutional layers, up-sampling, edge detection, defect detection and the like to construct the defect recognition network, and the image is preprocessed, so that the network has simple structure, high recognition precision and high recognition speed.

Drawings

FIG. 1 is a schematic diagram of a method for identifying defects of a glass bottle according to an embodiment of the present invention;

FIG. 2 is a schematic diagram of a three-view collection vial provided by an embodiment of the present invention;

FIG. 3 is a diagram of the relative positional relationship of the glass bottle, the monocular camera and the reflector provided by the embodiment of the invention;

FIG. 4 is a schematic view of a light source applied to a bottle mouth part according to an embodiment of the present invention;

fig. 5 is a schematic diagram of three views of a collected glass bottle provided by an embodiment of the present invention.

Detailed Description

The following examples are given for the purpose of illustration only and are not to be construed as limiting the invention, including the drawings for reference and description only, and are not to be construed as limiting the scope of the invention as many variations thereof are possible without departing from the spirit and scope of the invention.

The embodiment of the invention provides a glass bottle defect identification method, which comprises the following steps as shown in a flow chart of fig. 1:

s2, preprocessing the bottle mouth view, the bottle bottom view and the bottle body view to obtain a preprocessed bottle mouth view, a preprocessed bottle bottom view and a preprocessed bottle body view;

(1) For step S1

As shown in fig. 2, in this example, a monocular camera is used to collect bottle mouth data first, then a bottle body is rotated to collect bottle bottom data, and then the bottle body is rotated to collect bottle body data.

In order to successfully collect three views of a glass bottle, particularly a bottle body, the invention also provides that two reflectors are positioned at two sides of the glass bottle, and the position relationship among a monocular camera (camera), the glass bottle and the reflectors is shown in figure 3. Fig. 3 (a), (b) and (c) are top, front and side views (left or right) of a monocular camera (camera), carafe, mirror, respectively, with the carafe in the inspection station.

In addition, in the case of the image acquisition, a light source is added to the bottle mouth part, the bottle mouth part becomes bright due to illumination, and the defect part becomes dark due to the light propagation principle. The light source is an annular shadowless incandescent lamp with adjustable voltage of 0V-20V as a lighting source, the intensity of the light source can be controlled by adjusting the voltage, objects with different positions and different materials can be conveniently irradiated, and a specific construction mode is shown in figure 4.

Finally, three views of the bottle body at the bottom of the bottle mouth and the bottle body are respectively shown in (a), (b) and (c) of fig. 5.

(2) For step S2

The step further preprocesses the three views, and specifically comprises the steps of:

s21, performing Gaussian filtering on the bottle mouth view, the bottle bottom view and the bottle body view to obtain respective first intermediate images;

s25, performing frequency domain filtering on the fourth intermediate image to obtain a pretreated bottle mouth view, a pretreated bottle bottom view and a pretreated bottle body view.

In step S21, the fused image is gaussian filtered as follows:

wherein g1 (x, y) represents the output first intermediate image, and (x, y) represents the input image coordinates, a smoothing window with the size of z x z is set, a convolution kernel delta conforming to Gaussian distribution is preset for the window, each pixel in the image is traversed by the convolution kernel, the weighted average value calculated by convolution replaces the central pixel of the convolution kernel, gaussian smoothing filtering is completed, and the purpose of reducing image noise is achieved.

In step S22, the first intermediate image is median filtered, with the following formula:

g2(x,y)＝median(g1(x-k,y-l),(k,l∈W))

g2 (x, y) represents the second intermediate image obtained after median filtering, W is a two-dimensional template, typically 5×5 or 3×3 in size, k, l is the abscissa of the pixel points in the two-dimensional template, and mean () represents the median filtering function. The median filtering process is to sort the image pixels according to the pixel values by using W to form a monotonically ascending or descending two-dimensional data sequence until the sliding window traverses the image to finish smoothing.

In step S23, the gray value conversion is performed on the second intermediate image, and the formula is as follows:

wherein g2 (x, y) is a second intermediate image, the gray value range of the original pixel point is [ a, b ], the gray value range which is wanted to be reached by the image enhancement is [ c, d ], and the output third intermediate image is g3 (x, y). The formula is a conversion relation of gray values when the conversion function is a linear function, and gray level change can make gray level characteristics of the image more obvious.

In step S24, sharpening is performed by an image sharpening algorithm to emphasize boundary pixels of an object in an image and details to be emphasized, and to weaken the phenomenon of image blurring. The sharpening effect is achieved by a filtering technology in a frequency domain, and differential processing is required to be carried out on pixel values in an image in a space domain, wherein the formula is as follows:

in the third aspect, the differential equation is replaced by the differential equationFirst order difference in x direction and first order difference delta in y direction at midpoint (x, y) of intermediate image g3 (x, y) _x g3 (x, y) and delta _y g3 (x, y) can be expressed as:

the modulo point of the gradient vector at point (x, y), i.e. the fourth intermediate image g4 (x, y), can be expressed as:

g4(x,y)＝|Δ _x g3(x,y)|+|Δ _y g3(x,y)|

the gradient value is used as sharpening output, and the output image only displays the edge contour with relatively strong gray value change.

In step S25, the image is subjected to frequency domain filtering, and the discrete function of pixel values in the two-dimensional space of the image can be analyzed in the frequency domain through fourier transformation, and the frequency spectrum is processed and analyzed to change the frequency characteristics of the image, specifically as follows:

the triangular form fourier series of the function f (T) with period T is expanded as:

wherein,represents angular frequency, a _k And b _k Is a Fourier coefficient, x is an independent variable, n is an integer greater than zero, a ₀ Is a direct current component.

After the image preprocessing, the noise of the fused image can be weakened and suppressed, the image is enhanced, the boundary pixels of the target in the image and the details which want to be highlighted are highlighted, the phenomenon of image blurring is weakened, and the convolutional neural network can conveniently extract the required characteristics rapidly.

(3) For step S3

The step S3 specifically comprises the steps of:

s31, carrying out a shared convolution operation on a bottle opening view after pretreatment, a bottle bottom view after pretreatment and a bottle body view after pretreatment of 256×256×1, wherein the convolution kernel size is 3×3, activating the bottle opening view after pretreatment by a ReLU function, and extracting features in a same filling mode to obtain a three-view feature map; 256×256 represents the length and width of an image, 1 represents the number of image channels;

s33, the fusion characteristic is subjected to a convolution operation, the convolution kernel size is 1 multiplied by 1, the channel number of the output image is adjusted to be 3, and then a linear activation function is performed to finally output a fusion image (256 multiplied by 3) of 3 channels, which is consistent with the input image size in the step S31.

(4) For step S4

Referring to fig. 1, step S4 specifically includes the steps of:

s41, extracting features of the processed image to generate a target feature image;

The step S41 specifically includes the steps of:

s411, inputting the processed image into a convolutional neural network to generate a first feature map;

s412, inputting the first feature map into the first convolution layer, the second convolution layer, the third convolution layer, the fourth convolution layer and the maximum pooling layer respectively to obtain a second feature map, a third feature map, a fourth feature map, a fifth feature map and a sixth feature map, and fusing and splicing the second feature map, the third feature map, the fourth feature map, the fifth feature map and the sixth feature map to obtain a seventh feature map; performing convolution processing with the convolution kernel of 1×1 on the first feature map to obtain an eighth feature map;

s413, performing convolution processing of the seventh feature map with a convolution kernel of 1 multiplied by 1, and then performing up-sampling and fusion and splicing with the eighth feature map to obtain a ninth feature map;

s414, performing convolution processing of the ninth feature map with a convolution kernel of 3 multiplied by 3, and then performing up-sampling to obtain a target feature image.

In step S412:

the first convolution layer executes cavity convolution processing with a convolution kernel of 1 multiplied by 1 and an expansion coefficient of r=1 on the input first feature map to obtain a second feature map;

the second convolution layer executes cavity convolution processing with a convolution kernel of 3×3 and an expansion coefficient of r=6 on the input first feature map to obtain a third feature map;

the third convolution layer executes cavity convolution processing with a convolution kernel of 3×3 and an expansion coefficient of r=12 on the input first feature map to obtain a fourth feature map;

the fourth convolution layer performs a hole convolution process with a convolution kernel of 3×3 and a coefficient of expansion of r=18 on the input first feature map, and obtains a fifth feature map.

Step S42 includes the steps of:

s422, detecting and scanning the target characteristic image, judging whether the amplitude of the point on the edge is the point with the largest amplitude in the same direction, if so, reserving, and if not, deleting;

In step S421, the amplitude of the target feature image is calculatedAnd direction->The formula of (2) is as follows:

the function is respectively a function after the target characteristic image and the two first-order difference convolution templates are operated.

The step S43 specifically includes the steps of:

s432, performing threshold segmentation on the detection area to enable the pixel value of the defect to be 255 and the other pixel values to be 0, so as to obtain a communication area of the defect;

In step S432, since a light source is added to the bottle mouth portion at the time of image acquisition, the bottle mouth portion becomes bright due to illumination, and the defective portion becomes dark due to the principle of light propagation. Therefore, the detection area image is subjected to threshold segmentation, and the difference of the pixel values of the defect positions is highlighted, so that the pixel value of the defect positions is 255. According to the principle of the connected region, the connected region of the defective portion of the glass bottle is obtained by detecting and extending the pixel points of the surrounding 3×3 region with the point of the image pixel value of 255 as the center.

In step S433, the area coefficient of the defect is set to be a (a > 0), a is the pixel value of the connected region with the gray value of 255 after the threshold segmentation, a threshold B (B > 0) is set, and when a > B, the bottle is judged to be a defective bottle, otherwise, the bottle is judged to be a non-defective bottle.

In addition, it should be noted that the defect recognition network needs to be trained, verified and tested in advance, and a related data set needs to be constructed.

Firstly, a large number of fusion images are acquired according to steps S1 and S2 to obtain a data set I _NHB (RGB three channel color image), dataset I _NHB Expressed as: i _NHB ＝{I ₁ ,I ₂ ,…,I _n-1 ,I _n Then for data set I _NHB Cutting and defect marking to obtain a corresponding label set I _Label (RGB Single channel binary image), label collection I _label Expressed as: i _label ＝{L ₁ ,L ₂ ,…,L _n-1 ,L _n }，I _n Representing data set I _NHB An nth fused image; l (L) _n Is I _n The corresponding label.

Then, for data set I _NHB Image and label collection I in (1) _Label Performing 90 DEG, 180 DEG and 270 DEG rotation to obtain an enhanced data set I _{NHB_En} And target tag collection I _{Label_En} 。

Then, the data set I will be enhanced _{NHB_En} And target tag collection I _{Label_En} Randomly split according to a ratio of 7:3 and according to the enhanced data set I _{NHB_En} For target label collection I _{Label_En} Performing uniform processing to form a target training sample data setTarget test data set->And target verification sample dataset +.>Are used for training, verification and testing.

In this embodiment, the target training sample datasetTarget test data set->And target verification sample dataset +.>The image sizes in (2) are 256×256×3, wherein 256 represents the width and height of the image, and 3 represents the three-channel image. Target tag collection I _{Label_En} The label size is 256×256×1, where 256 denotes the width and height of the image and 1 denotes that the image is a single-channel image.

During training, the loss function is designed as:

Loss＝0.4*Loss _obj +0.3*Loss _rect +0.3*Loss _cls

wherein p is _o Representing target confidence scores in a prediction box, p _iou IOU value, w, representing prediction frame and target frame corresponding thereto _obj Representation calculationWeight of time positive sample, +.>Represents p _o And p _iou Two kinds of cross entropy loss in the middle; IOU represents the overlapping proportion of the predicted frame and the actual frame, b represents the center point of the predicted frame, b ^gt Represents the center point of the actual frame, c represents the diagonal length of the smallest rectangle containing the predicted frame and the actual frame, w represents the width of the predicted frame, w ^gt Representing the width of the actual frame, C _w A width of a minimum rectangle representing a prediction frame and an actual frame, h represents a height of the prediction frame, h ^gt Representing the height of the actual frame, C _h The width of the smallest rectangle representing the predicted and actual frames, ρ () represents the euclidean distance; c _p Representing prediction category, c _gt Representing the true category, w _cls Representing the calculation->Weight of time positive sample, +.>Representation c _p And c _gt Two classes in between cross entropy loss.

The defect recognition network training process specifically comprises the following steps:

step 1: training a target training sample datasetAnd target tag collection I _Label _ _En Simultaneously inputting a model for training;

step 2: calculating a model loss function value and recording current model parameters;

step 3: judging whether the current model is better than the stored optimal model, if so, storing the current model as the optimal model, otherwise, not storing;

step 4: repeatedly executing epoch times in the step 1, the step 2 and the step 3;

step 5: loading the optimal model and collecting the target test data setAnd target verification sample data setInputting a model for testing; the step 5 specifically comprises the steps of:

step 51, loading an optimal model;

step 52, target test datasetInputting a model and storing a result;

step 53, validating the target sample datasetInputting a model and storing a result;

step 54, combining the results of step 52 and step 53 with the tag collection I _Label _ _En Performing comparison, classification and comparison;

and step 55, calculating the classification accuracy.

In step 3, the model performance is compared with the model loss value, and the lower the loss value is, the better the model performance is, and the worse the anti-regularization is.

As shown in the following table 1, the defect identification network constructed by the invention can accurately identify the defective glass bottle, and the average identification accuracy (Acc) is as high as 96.67%. As shown in the following table 2, compared with other defect recognition methods, the average recognition accuracy of the defect recognition network constructed by the method is highest, which is as high as 95.80%, and is higher than that of other algorithms by more than 2%, because the method can simultaneously extract the characteristics of the pictures in three directions, and the images are fused into the characteristic images of three channels according to different weights to detect the defects, the defect detection accuracy can be improved, and meanwhile, the detection task can be completed at a higher speed. And the running speed of the method is still faster. As shown in table 2 below, the recognition accuracy of the present invention was 90.88% without performing the image preprocessing, and the recognition accuracy of the present invention was improved to 95.80% after performing the image preprocessing, and it was found that the image preprocessing contributed to the improvement of the recognition accuracy.

TABLE 1

Sample type	Quantity of	T (positive sample)	F (negative sample)	Acc/％	Error/％
						Intact bottle	200	195	5	97.50	2.50
Defect bottle	100	95	5	95.00	5.0
						Total number of	300	290	10	96.67	3.33

TABLE 2

In summary, according to the method and the system for identifying defects of the glass bottle provided by the embodiment of the invention, through shooting and imaging the bottle mouth, the bottle bottom and the bottle body of the glass bottle, the corresponding three views are obtained for preprocessing, so that noise of the acquired three views is weakened and suppressed, and a preprocessed image is obtained; the single-channel preprocessed images are further fused, the images in three directions are subjected to feature extraction, and the images are fused into three-channel feature images according to different weights to detect defects, so that the defect detection accuracy can be improved, and meanwhile, the detection task can be completed at a high speed; then, the computing capacity of the convolutional neural network is utilized to collect and process image features of different degrees of images, then, the convolution and pooling of different degrees are carried out to carry out feature fusion, then, convolution up-sampling and the like are carried out to obtain a target feature image, the image features can be fully extracted, and the defect recognition accuracy is improved; and finally, carrying out edge detection and defect position positioning on the target characteristic image so as to identify whether the acquired image has defects or not and position the defects, thereby having higher identification precision. The invention adopts the convolutional neural network and a plurality of convolutional layers, up-sampling, edge detection, defect detection and the like to construct the defect recognition network, and the image is preprocessed, so that the network has simple structure, high recognition precision and high recognition speed.

Example 2

The embodiment provides a glass bottle defect identification system, which comprises an image acquisition module, an image preprocessing module and a defect identification module; the image acquisition module, the image preprocessing module, the image fusion module and the defect recognition module are respectively used for executing steps S1, S2, S3 and S4 in the method of the embodiment 1.

Specifically, as shown in fig. 3, the image acquisition module comprises a mechanical arm, a monocular camera and two reflecting mirrors, wherein the monocular camera is arranged right in front of the glass bottle on the detection station, and the two reflecting mirrors are symmetrically arranged at the left rear and the right rear of the glass bottle; the mechanical arm is used for:

the method comprises the steps that a glass bottle is vertically placed on a detection station, and a monocular camera shoots a bottle body of the glass bottle at the moment to obtain a bottle body image;

the middle part of the bottle body of the vertically placed glass bottle is clamped to rotate, so that the bottle mouth of the glass bottle faces the monocular camera, and the camera shoots the bottle mouth of the glass bottle at the moment to obtain a bottle mouth image;

the middle part of the bottle body of the glass bottle which is vertically placed is clamped to rotate, so that the bottle bottom of the glass bottle faces to the monocular camera, and the camera shoots the bottle bottom of the glass bottle at the moment to obtain a bottle bottom image;

the middle part of the bottle body of the glass bottle is clamped, and the glass bottle is taken away from the detection station.

The specific shooting sequence can be set by itself, and is not limited to the above-described process. The work specifically completed by the image acquisition module, the image preprocessing module, the image fusion module and the defect recognition module is already described in detail in embodiment 1, and will not be repeated here.

The above examples are preferred embodiments of the present invention, but the embodiments of the present invention are not limited to the above examples, and any other changes, modifications, substitutions, combinations, and simplifications that do not depart from the spirit and principle of the present invention should be made in the equivalent manner, and the embodiments are included in the protection scope of the present invention.

Claims

1. The glass bottle defect identification method is characterized by comprising the following steps:

2. The method according to claim 1, wherein in the step S4, the defect recognition network performs defect location and outputs a defect recognition result image, and the method specifically comprises the steps of:

3. The method for identifying defects of glass bottles according to claim 2, wherein said step S41 comprises the steps of:

4. A method of identifying defects in glass bottles as claimed in claim 3 wherein in said step S412:

5. The method for identifying defects of glass bottles as claimed in claim 4 wherein said step S42 comprises the steps of:

6. Root of Chinese characterThe method according to claim 5, wherein in the step S421, the amplitude of the target feature image is calculatedAnd direction->The formula of (2) is as follows:

the step S43 specifically includes the steps of:

7. The method of claim 6, wherein during training, the loss function of the defect recognition network is designed to:

Loss＝0.4*Loss _obj +0.3*Loss _rect +0.3*Loss _cls

wherein p is _o Representing target confidence scores in a prediction box, p _iou IOU value wo representing prediction frame and target frame corresponding to the prediction frame _bj Representation calculationWeight of time positive sample, +.>Represents po and p _i o _u Two kinds of cross entropy loss in the middle; IOU represents the overlapping proportion of the predicted frame and the actual frame, b represents the center point of the predicted frame, b ^gt Represents the center point of the actual frame, c represents the diagonal length of the smallest rectangle containing the predicted frame and the actual frame, w represents the width of the predicted frame, w ^gt Representing the width of the actual frame, C _w A width of a minimum rectangle representing a prediction frame and an actual frame, h represents a height of the prediction frame, h ^gt Representing the height of the actual frame, C _h The width of the smallest rectangle representing the predicted and actual frames, ρ () represents the euclidean distance; c _p Representing prediction category, c _gt Representing realityCategory, w _cls Representing the calculation->Weight of time positive sample, +.>Representation c _p And c _gt Two classes in between cross entropy loss.

8. The method for identifying defects of glass bottles according to any one of claims 1 to 7, wherein said step S2 comprises the steps of:

in the step S23, the formula for performing the gray value conversion is:

g4(x,y)＝|Δ _x g3(x,y)|+|Δ _y g3(x,y)|

9. the method for identifying defects of glass bottles according to claim 8, wherein said step S3 comprises the steps of:

10. A glass bottle defect identification system, characterized in that: the system comprises an image acquisition module, an image preprocessing module and a defect identification module; the image acquisition module, the image preprocessing module, the image fusion module and the defect identification module are respectively used for executing the steps S1, S2, S3 and S4 of any one of claims 1 to 9;