CN117934980A - Glass container defect detection method and system based on attention supervision adjustment - Google Patents
Glass container defect detection method and system based on attention supervision adjustment Download PDFInfo
- Publication number
- CN117934980A CN117934980A CN202410338088.1A CN202410338088A CN117934980A CN 117934980 A CN117934980 A CN 117934980A CN 202410338088 A CN202410338088 A CN 202410338088A CN 117934980 A CN117934980 A CN 117934980A
- Authority
- CN
- China
- Prior art keywords
- container
- feature
- glass container
- defect
- bottle
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 230000007547 defect Effects 0.000 title claims abstract description 226
- 239000011521 glass Substances 0.000 title claims abstract description 153
- 238000001514 detection method Methods 0.000 title claims abstract description 135
- 238000000034 method Methods 0.000 claims abstract description 37
- 238000012549 training Methods 0.000 claims abstract description 19
- 238000012544 monitoring process Methods 0.000 claims abstract description 14
- 238000012795 verification Methods 0.000 claims abstract description 12
- 238000012360 testing method Methods 0.000 claims abstract description 10
- 238000002372 labelling Methods 0.000 claims abstract description 6
- 238000000605 extraction Methods 0.000 claims description 62
- 238000010586 diagram Methods 0.000 claims description 26
- 230000004927 fusion Effects 0.000 claims description 20
- 230000006870 function Effects 0.000 claims description 16
- 230000004913 activation Effects 0.000 claims description 11
- 238000010606 normalization Methods 0.000 claims description 9
- 238000001914 filtration Methods 0.000 claims description 6
- 238000011176 pooling Methods 0.000 claims description 6
- 239000004575 stone Substances 0.000 claims description 6
- 238000011045 prefiltration Methods 0.000 claims description 5
- 238000005457 optimization Methods 0.000 claims description 4
- 238000012935 Averaging Methods 0.000 claims description 3
- 238000005516 engineering process Methods 0.000 description 3
- 238000007689 inspection Methods 0.000 description 3
- 238000011161 development Methods 0.000 description 2
- 230000018109 developmental process Effects 0.000 description 2
- 239000007788 liquid Substances 0.000 description 2
- 238000004519 manufacturing process Methods 0.000 description 2
- 238000003860 storage Methods 0.000 description 2
- 235000013405 beer Nutrition 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 235000013361 beverage Nutrition 0.000 description 1
- 230000002950 deficient Effects 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 230000011218 segmentation Effects 0.000 description 1
- 238000010200 validation analysis Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
- Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
- Y02P90/30—Computing systems specially adapted for manufacturing
Landscapes
- Investigating Materials By The Use Of Optical Means Adapted For Particular Applications (AREA)
- Image Processing (AREA)
Abstract
The invention relates to the technical field of industrial detection, and discloses a glass container defect detection method and system based on attention monitoring adjustment, wherein the method comprises the following steps: (1) Acquiring images of a bottle body, a bottle neck, a bottle mouth and a bottle bottom of a glass container to obtain a glass container image data set B; (2) Labeling the glass container image data set B to obtain a real box of defects in the image data set B, and dividing the real box into a training set, a verification set and a test set; (3) building a glass container defect detection model; (4) Inputting the glass container image into a trained glass container defect detection model, and outputting a defect detection result. The system comprises a glass container image acquisition module and a glass container defect detection module. The invention improves the accuracy of glass container defect detection and has higher robustness.
Description
Technical Field
The invention relates to a method and a system for detecting defects of a glass container, belonging to the technical field of industrial detection.
Background
Before filling liquids such as beverages and beer, the glass containers need to be ensured to have no potential safety hazard, otherwise, when filling liquids, defective glass containers easily damage a filling production line, thereby causing economic loss. Therefore, an omnidirectional 360-degree defect detection of the glass container is required.
In the aspect of glass container defect detection, the machine vision defect detection technology has replaced traditional manual visual detection, so that the defect detection efficiency of the glass container is greatly improved. With the increase of the demand of automatic and intelligent detection, the development of intelligent defect detection is gradually limited by a machine vision technology which is excessively dependent on manual design, and the development of the robustness of glass container defect detection is also limited.
In recent years, an unsupervised, small sample-based defect detection method is increasingly used for detecting defects of industrial products such as glass containers. The defect detection accuracy of such methods is practically unsatisfactory for industry.
In addition, the segmentation-based method is also applied to the detection of defects in industrial products such as glass containers. But in practice the defect detection speed of such methods is not satisfactory for the industry.
Therefore, there is a need in the market for a defect detection method with high accuracy and high speed, which is also a practical requirement in the field of glass container defect detection.
Disclosure of Invention
In order to solve the problems of the existing glass container defect detection technology, the invention provides a glass container defect detection method based on attention supervision adjustment, which automatically supervises and adjusts the extracted glass container defect characteristics through an attention mechanism, thereby realizing an accurate and rapid glass container defect detection function. A system for implementing the method is also provided.
In order to achieve the above purpose, the present invention adopts the following technical scheme.
In a first aspect, a proposed method for detecting defects of a glass container based on attention-monitoring adjustment comprises the steps of:
(1) Acquiring images of a bottle body, a bottle neck, a bottle mouth and a bottle bottom of a glass container to obtain a glass container image data set B;
The glass container image dataset ，/>For the body image of the first glass container,/>J image for the ith glass container,/>,/>When j is 1, the bottle body is represented by j, when j is 2, the bottle neck is represented by j, when j is 3, the bottle bottom is represented by j, and when j is 4, the bottle mouth is represented by j.
(2) And labeling the glass container image data set B to obtain a real box of the defects in the image data set, and dividing the real box into a training set, a verification set and a test set.
(3) Building a glass container defect detection model;
The glass container defect detection model comprises a container defect feature extraction group, an attention supervision and adjustment module, a container feature conflict pre-filtering module, a container feature fusion module and a container defect detection unit; the container defect feature extraction group comprises five defect feature extraction groups which are sequentially connected, namely a first defect feature extraction group, a second defect feature extraction group, a third defect feature extraction group, a fourth defect feature extraction group and a fifth defect feature extraction group, wherein the first to fifth defect feature extraction groups are sequentially connected to respectively obtain container features I to five (the output of each of the container features I to five is the output of each of the defect feature extraction groups), and the attention monitoring and adjusting module monitors and adjusts each of the defect feature extraction groups; the third, fourth and fifth defect feature extraction groups input container features three to five into the container feature conflict prefiltering module at the same time, and output prefiltered container prefiltering features I; the container characteristic IV, the container characteristic V and the container prefiltering characteristic I are input to a container characteristic fusion module at the same time, and a container fusion characteristic I and a container fusion characteristic II are obtained; finally, the container fusion characteristics I and II are simultaneously input into a container defect detection unit, and a glass container defect detection result is output.
(4) Inputting the glass container image into a trained glass container defect detection model, and outputting a defect detection result.
Further, in the step (1):
the glass container is subjected to omnibearing image acquisition of a bottle body, a bottle neck, a bottle bottom and a bottle mouth through an industrial camera, a lens, a light source and an image acquisition card, the shot images have different resolutions according to different shooting positions, the bottle body position can be 444 multiplied by 840, and the bottle neck position can be 800 multiplied by 1100. The industrial camera adopts a CCD industrial camera, and the CCD industrial camera adopts an area array CCD industrial camera; in the aspect of the light source, a strip light source is mainly adopted at the bottle body and the bottleneck part, and an annular light source is mainly adopted at the bottle bottom and the bottleneck part.
In the step (2):
① The training set, the verification set and the test set are obtained by dividing the glass container image data set B according to the proportion of 6:2:2.
② The marking of the glass container image data set B is to mark images of a bottle body, a bottle neck, a bottle bottom and a bottle mouth part of a glass container by adopting a rectangular frame, and assign corresponding label serial numbers to each defect to form a glass container defect detection data set;
Specifically, the image of the body, neck, bottom and neck portion of the glass container is marked with a rectangular frame using Labelimg software, and the label number of the double-mouth defect of the neck detection portion is 0, the label number of the break defect of the neck detection portion is 1, the label number of the mouth face bubble defect of the neck detection portion is 2, the label number of the mouth defect of the neck detection portion is 3, the label number of the pit defect of the neck detection portion is 4, the label number of the scissors defect of the neck detection portion is 5, the label number of the bottle bottom bubble defect of the bottle bottom detection portion is 6, the label number of the bottle bottom stone defect of the bottle bottom detection portion is 7, the label number of the bottle bottom crack defect of the bottle bottom detection portion is 8, the label number of the bottle bottom oil stain defect of the bottle bottom detection portion is 9, the label number of the bubble defect of the bottle body and the bottle bottom detection portion is 10, the stone defect of the bottle body and the neck detection portion is 11, the label number of the bottle body and the bottle neck detection portion is 12, and the label of the bottle neck detection portion is 13.
In the step (3):
the glass container defect detection model is obtained by using a training set and a verification set, and comprises the following specific processes:
j-image of ith glass container in training set Inputting the feature images into the first to fifth container defect feature extraction groups, and outputting to obtain feature atlas/>; Map/>、/>、/>Inputting the characteristic conflict pre-filtering module into a container, and outputting to obtain a characteristic diagram/>; The feature map/>、/>And/>Input to a container feature fusion module, and output to obtain feature atlas/>; Feature atlas/>Inputting the glass container defects into a container defect detection unit, and outputting to obtain final characteristics f, wherein the final characteristics f comprise the category, the confidence coefficient and the coordinate information of a prediction frame of the glass container defects; calculating the total loss/>, using the final feature fAnd carrying out gradient optimization on the glass container defect detection model through an optimizer. The total loss/>Is confidence loss/>Classification loss/>Loss of frame regression/>Is added up; confidence loss/>And classification loss/>Calculating by adopting binary cross entropy Loss functions respectively, and further balancing the influence of the lost samples by adopting Focal Loss; frame regression loss/>Loss with container defect IoU; the container defect IoU loss is defined as follows:
。
wherein IoU is the intersection ratio of the predicted block and the real block, Center point loss and aspect ratio loss in CIoU employed.
Wherein f is defined as follows:
，
wherein b is defined as follows:
，
Wherein C represents the minimum closed region covering the real region and the predicted region, tanh is a tanh function, and w, h, wgt and hgt represent the height and width of the predicted box, and the height and width of the real box, respectively.
The defect feature extraction group I consists of a convolution group and a feature extraction group, wherein the convolution group consists of a convolution layer with the convolution kernel size of 3 multiplied by 3, a batch normalization layer and a ReLU activation function; the feature extraction group consists of a convolution group and a convolution attention group, wherein the convolution group consists of a convolution layer with the convolution kernel size of 1 multiplied by 1, a batch normalization layer and a ReLU activation function, and the convolution attention group consists of a convolution layer with the convolution kernel size of 3 multiplied by 3, a batch normalization layer, a channel attention layer and a ReLU activation function.
The attention monitoring and adjusting module is used for monitoring and adjusting the defect characteristics of the glass container by controlling the global defect characteristics of the glass container image; the global defect feature is controlled by serially connecting global average pooling and one-dimensional convolution. The specific process for controlling the global defect characteristics is as follows:
j-image of ith glass container in training set Input to the first supervision adjustment module, image/>Input to the global averaging pooling layer, output feature map/>Feature map/>Input to one-dimensional convolution, output feature map/>Feature mapRespectively through two MLP layers, outputting a characteristic diagram/>And/>Feature map/>After being activated by Sigmoid, the feature map is outputImage/>Respectively and with the characteristic diagram/>And feature map/>Multiplication, output feature map/>And feature map/>; Feature map/>Through a convolution layer with the convolution kernel size of 1 multiplied by 1, a characteristic diagram/> isoutput. Feature map/>And feature map/>After addition, the feature map is output. Completing the attention monitoring and adjusting process; feature map/>Outputting a feature map/>, after passing through the convolution group and the feature extraction group of the first container defect feature extraction group; Identical,/>Inputting the extracted defect characteristics into a second supervision and adjustment module and the first container defect characteristic extraction group, and outputting a characteristic diagram/>, after the operation；/>，/>And/>The same procedure was used.
The third, fourth and fifth defect feature extraction groups input container features three to five to the container feature conflict prefiltering module at the same time, and the process of outputting prefiltered container prefiltering feature one is as follows:
In container feature three Introducing a coordinate attention mechanism to help container feature three to better focus on space information and outputting a feature map/>Five feature maps/>, of deep container featuresAnd container feature four feature map/>Performing addition operation and outputting a characteristic diagram/>; At the same time, the three feature maps/>, of the container features of the shallow layerAnd container feature four feature map/>Performing addition operation and outputting a characteristic diagram/>By this process, conflicts are pre-filtered, and then the feature maps/>, of both phases are usedAnd/>Spliced together and output a feature map/>; Further optimizing the integrated feature map/>, using a lightweight residual extraction structureOutput of feature map/>; Map of features to be obtained/>And feature map/>Further fusing to obtain prefilter feature I/>。
In a second aspect, a proposed attention-based adjustment glass container defect detection system performs the attention-based adjustment glass container defect detection method proposed in the first aspect, comprising:
the glass container image acquisition module is used for acquiring images of a bottle body, a bottle neck, a bottle mouth and a bottle bottom of the glass container;
and the glass container defect detection module inputs the glass container image into a trained glass container defect detection model and outputs a defect detection result. The glass container defect detection model has been described above and will not be described in detail herein.
In a third aspect, an electronic device is provided that includes a memory and a processor and computer instructions stored on the memory and running on the processor that, when executed by the processor, perform the steps recited in the method for detecting glass container defects based on attention-deficit adjustment.
In a fourth aspect, a computer readable storage medium is provided for storing computer instructions that, when executed by a processor, perform the steps recited in a method for detecting glass container defects based on attention monitor adjustment.
Compared with the prior art, the invention has the beneficial effects that:
1. according to the invention, a plurality of container features are simultaneously fed into the container feature conflict prefiltering module, so that container defect features from a plurality of paths and a plurality of scales are obtained, and finally, a prefiltering container feature is output, the feature retains defect feature information of more details, the accuracy of glass container defect detection is improved, and the glass container defect detection method has higher robustness.
2. The invention is based on the attention monitor adjustment module, and reduces the information loss in the down sampling operation by monitoring and adjusting the container defect characteristic extraction group, thereby further improving the defect detection performance of the glass container.
Drawings
FIG. 1 is a flow chart of a glass container defect detection method based on attention monitor adjustment of the present invention.
FIG. 2 is a block diagram of a glass container defect detection model in accordance with the present invention.
Fig. 3 is a block diagram of an attention monitor adjustment module in the present invention.
FIG. 4 is a block diagram of a container feature conflict pre-filter module in accordance with the present invention.
Detailed Description
In order to improve the accuracy of glass container defect detection, the method for detecting glass container defects based on attention monitoring adjustment disclosed by the invention, as shown in fig. 1, specifically comprises the following steps.
S1: collecting images of a bottle body, a bottle neck, a bottle bottom and a bottle mouth of the glass container to obtain a glass container image data set B,，/>For the body image of the first glass container,/>J image for the ith glass container,/>,/>When j is 1, the bottle body is represented by j, when j is 2, the bottle neck is represented by j, when j is 3, the bottle bottom is represented by j, and when j is 4, the bottle mouth is represented by j.
S2: and labeling the glass container image data set B to obtain a real box of the defects in the image data set, and dividing the real box into a training set, a verification set and a test set.
S3: and (5) constructing a glass container defect detection model.
S4: and obtaining a deployable glass container defect detection model by using the training set and the verification set. J-image of ith glass container in training setInput to the first to fifth container defect feature extraction groups, output to obtain feature atlasMap/>、/>、/>Inputting the characteristic conflict pre-filtering module into a container, and outputting to obtain a characteristic diagram/>The feature map/>、/>And/>Input to a container feature fusion module, and output to obtain a feature atlasFeature atlas/>And inputting the information into a container defect detection unit, and outputting to obtain a final feature f, wherein the final feature f comprises the category, the confidence coefficient and the coordinate information of a prediction frame of the glass container defect. Calculating total loss using final feature fAnd carrying out gradient optimization on the glass container defect detection model through an optimizer.
S5: and deploying the glass container defect detection model, and testing the defect detection system by using a test set and an actual production line.
The defect detection system comprises a glass container image acquisition module and a glass container defect detection module, and is used for calling a glass container defect detection method based on attention supervision adjustment so as to detect defects.
The above steps are described in detail below.
Step S1, acquiring images of a bottle body, a bottle neck, a bottle bottom and a bottle mouth of a glass container, and obtaining a glass container image data set B.
S1-1: the glass container passes through quality inspection equipment, an industrial camera, a lens, a light source and an image acquisition card on the quality inspection equipment perform all-round image acquisition on the bottle body, the bottle neck, the bottle bottom and the bottle mouth of the glass container, the shot images have different resolutions according to different shooting positions, the bottle body position can be 444 multiplied by 840, and the bottle neck position can be 800 multiplied by 1100.
S1-2: wherein, the industrial camera adopts CCD industrial camera, and the CCD industrial camera adopts area array CCD industrial camera; in the aspect of light sources, strip light sources are mainly used at bottle bodies and bottle neck positions, and annular light sources are mainly used at bottle bottoms and bottle neck positions.
And step S2, marking the glass container image data set B to obtain a real box of the defects in the image data set, and dividing the real box into a training set, a verification set and a test set.
S2-1: the glass container inspection dataset was divided into a training set, a validation set, and a test set in a 6:2:2 ratio.
S2-2: and marking the images of the bottle body, the bottle neck, the bottle bottom and the bottle mouth part of the glass container by adopting a rectangular frame, and endowing each defect with a corresponding label serial number to form a glass container defect detection data set.
S2-3: specifically, the image of the body, neck, bottom and mouth of the glass container is marked with a rectangular frame using Labelimg software, and the label number of the double-mouth defect of the mouth detection part is 0, the label number of the break defect of the mouth detection part is 1, the label number of the mouth surface bubble defect of the mouth detection part is 2, the label number of the mouth defect of the mouth detection part is 3, the label number of the pit defect of the mouth detection part is 4, the label number of the scissors defect of the mouth detection part is 5, the label number of the bottle bottom bubble defect of the bottle bottom detection part is 6, the label number of the bottle bottom stone defect of the bottle bottom detection part is 7, the label number of the bottle bottom crack defect of the bottle bottom detection part is 8, the label number of the bottle bottom oil stain defect of the bottle bottom detection part is 9, the label number of the bubble defect of the bottle body and the bottle bottom detection part is 10, the label number of the stone defect of the bottle body and the bottle bottom detection part is 11, the label number of the oil stain defect of the bottle body and the bottle bottom detection part is 12, and the label number of the bottle neck label of the bottle neck detection part is 13.
Step S3, a glass container defect detection model is built.
S3-1: as shown in fig. 2, the glass container defect detection model includes a plurality of container defect feature extraction groups, an attention monitor adjustment module, a container feature conflict pre-filtering module, a container feature fusion module, and a container defect detection unit, which are sequentially connected.
S3-2: specifically, the first to fifth container defect feature extraction groups are sequentially connected to obtain container features one to five respectively, and the attention monitoring and adjusting module monitors and adjusts each container defect feature extraction group; the third, fourth and fifth container defect feature extraction groups input container features three to five into the container feature conflict prefiltering module at the same time, and output prefiltered container prefiltering features I; the container features IV, V and the container prefiltering feature I are input to a container feature fusion module at the same time, so that a container fusion feature I and a container fusion feature II are obtained; finally, the container fusion characteristics I and II are simultaneously input into a container defect detection unit, and a glass container defect detection result is output.
S3-3: the container defect feature extraction groups comprise five defect feature extraction groups which are sequentially connected, namely a defect feature extraction group I, a defect feature extraction group II, a defect feature extraction group III, a defect feature extraction group IV and a defect feature extraction group V, and the attention monitoring and adjusting module monitors and adjusts each container defect feature extraction group. Specifically, the defect feature extraction group I is composed of a convolution group and a feature extraction group. The convolution group consists of a convolution layer with a convolution kernel size of 3×3, a batch normalization layer, and a ReLU activation function. The feature extraction group consists of a convolution group and a convolution attention group, wherein the convolution group consists of a convolution layer with the convolution kernel size of 1 multiplied by 1, a batch normalization layer and a ReLU activation function, and the convolution attention group consists of a convolution layer with the convolution kernel size of 3 multiplied by 3, a batch normalization layer, a channel attention layer and a ReLU activation function.
S3-4: the attention monitor adjustment module is shown in fig. 3, and the monitor adjustment mechanism controls the global defect feature of the glass container image to realize monitor adjustment of the defect feature of the glass container.
S3-5: the global control of the attention monitor adjustment module is realized by carrying out serial connection on global average pooling and one-dimensional convolution. J-image of ith glass container in training setInput to a first supervisory adjustment module, imageInput to the global averaging pooling layer, output feature map/>Feature map/>Input to one-dimensional convolution, output feature map/>Feature map/>Respectively through two MLP layers, outputting a characteristic diagram/>And/>Feature map/>After Sigmoid activation, the feature map/>, is outputImage/>Respectively and with the characteristic diagram/>And feature map/>Multiplication, output feature map/>And feature map/>Feature map/>Through a convolution layer with the convolution kernel size of 1 multiplied by 1, a characteristic diagram/> isoutputFeature map/>And feature map/>After addition, the feature map/>Completing the attention monitoring and adjusting process, and the feature map/>Outputting a feature map/>, after passing through the convolution group and the feature extraction group of the first container defect feature extraction groupIdentical,/>Inputting the extracted defect characteristics into a second supervision and adjustment module and the first container defect characteristic extraction group, and outputting a characteristic diagram/>, after the operation，/>，/>And/>The same procedure was used.
S3-6: the container feature conflict prefilter module, as shown in fig. 4, has more channels for the deep container feature five and higher resolution for the shallow container feature three. First, in container feature threeIntroducing a coordinate attention mechanism to help container feature three to better focus on space information and outputting a feature map/>Features of each layer have different semantic depths, and direct fusion by using methods such as splicing and the like can cause the problem of feature dislocation, so that five feature graphs/>, namely deep container features, are obtainedAnd container feature four feature map/>Performing addition operation and outputting a characteristic diagram/>At the same time, the three feature maps/>, of the container features of the shallow layerAnd container feature four feature map/>Performing addition operation and outputting a characteristic diagram/>By this process, conflicts are pre-filtered, and then the feature maps/>, of both phases are usedAnd/>Spliced together and output a feature map/>Finally, the integrated feature map/>, is further optimized using a lightweight residual extraction structureOutput of feature map/>Finally, the obtained characteristic diagram/>And feature map/>Further fusing to obtain prefilter feature I/>。
S3-7: map the characteristic map、/>And/>Input to a container feature fusion module, and output to obtain a feature atlasFeature atlas/>And inputting the information into a container defect detection unit, and outputting to obtain a final feature f, wherein the final feature f comprises the category, the confidence coefficient and the coordinate information of a prediction frame of the glass container defect.
And step S4, obtaining a deployable glass container defect detection model by utilizing the training set and the verification set.
S4-1: loss of glass container defect detection modelIs confidence loss/>Classification loss/>Loss of frame regression/>Is added to the sum of (3).
S4-2: confidence lossAnd classification loss/>Calculating by adopting binary cross entropy Loss functions respectively, and further balancing the influence of the lost samples by adopting Focal Loss; frame regression loss/>The container defect IoU loss is used.
S4-3: wherein, the container defect IoU loss is defined as follows:
。
wherein IoU is the intersection ratio of the predicted block and the real block, Center point loss and aspect ratio loss in CIoU employed.
Wherein f is defined as follows:
。
wherein b is defined as follows:
。
Wherein C represents the minimum closed region covering the real region and the predicted region, tanh is a tanh function, and w, h, wgt and hgt represent the height and width of the predicted box, and the height and width of the real box, respectively.
S4-4: and in the training and verification process, adopting an SGD optimizer to perform gradient optimization on the glass container defect detection model.
The glass container defect detection system for realizing the glass container defect detection method based on the attention monitoring adjustment comprises the following components:
1. The glass container image acquisition module is used for acquiring images of a bottle body, a bottle neck, a bottle mouth and a bottle bottom of the glass container;
2. And the glass container defect detection module inputs the glass container image into a trained glass container defect detection model and outputs a defect detection result. The glass container defect detection model has been described above and will not be described in detail herein.
The invention also provides an electronic device comprising a memory, a processor and computer instructions stored on the memory and running on the processor, wherein the computer instructions, when executed by the processor, complete the steps of the glass container defect detection method based on the attention monitor adjustment.
The invention also proposes a computer readable storage medium storing computer instructions which, when executed by a processor, perform the steps described for a glass container defect detection method based on attention supervised adjustment.
Claims (10)
1. The glass container defect detection method based on attention monitoring adjustment is characterized by comprising the following steps of:
(1) Acquiring images of a bottle body, a bottle neck, a bottle mouth and a bottle bottom of a glass container to obtain a glass container image data set B;
The glass container image dataset ，/>For the body image of the first glass container,/>J image for the ith glass container,/>,/>When j is 1, the bottle body is represented, when j is 2, the bottle neck is represented, when j is 3, the bottle bottom is represented, and when j is 4, the bottle mouth is represented;
(2) Labeling the glass container image data set B to obtain a real box of defects in the image data set B, and dividing the real box into a training set, a verification set and a test set;
(3) Building a glass container defect detection model;
The glass container defect detection model comprises a container defect feature extraction group, an attention supervision and adjustment module, a container feature conflict pre-filtering module, a container feature fusion module and a container defect detection unit; the container defect feature extraction group comprises five defect feature extraction groups which are sequentially connected, namely a defect feature extraction group I, a defect feature extraction group II, a defect feature extraction group III, a defect feature extraction group IV and a defect feature extraction group V, wherein the first to fifth defect feature extraction groups are sequentially connected to respectively obtain container features I to five, and the attention supervision and adjustment module carries out supervision and adjustment on each defect feature extraction group; the third, fourth and fifth defect feature extraction groups input container features three to five into the container feature conflict prefiltering module at the same time, and output prefiltered container prefiltering features I; the container characteristic IV, the container characteristic V and the container prefiltering characteristic I are input to a container characteristic fusion module at the same time, and a container fusion characteristic I and a container fusion characteristic II are obtained; finally, the container fusion characteristics I and II are simultaneously input into a container defect detection unit, and a glass container defect detection result is output;
(4) Inputting the glass container image into a trained glass container defect detection model, and outputting a defect detection result.
2. The method for detecting defects of glass containers based on attention monitor adjustment according to claim 1, wherein the training set, the verification set and the test set in the step (2) are obtained by dividing the glass container image data set B in a ratio of 6:2:2.
3. The method for detecting defects of glass containers based on attention monitor adjustment according to claim 1, wherein the labeling of the glass container image dataset B in the step (2) is to label the images of the body, the bottleneck, the bottle bottom and the bottleneck parts of the glass container by using a rectangular frame, and assign a corresponding label serial number to each defect, so as to form a glass container defect detection dataset.
4. The method for detecting defects of a glass container based on attention monitor adjustment according to claim 3, wherein the specific process of labeling the images of the body, the bottleneck, the bottle bottom and the bottleneck part of the glass container by adopting the rectangular frame is as follows:
The method comprises the steps of marking images of a bottle body, a bottle neck, a bottle bottom and a bottle mouth part of a glass container by using Labelimg software, marking the images by using a rectangular frame, enabling a label number of double-mouth defects of the bottle mouth detection part to be 0, enabling a label number of broken-mouth defects of the bottle mouth detection part to be 1, enabling a label number of bubble defects of the mouth surface of the bottle mouth detection part to be 2, enabling a label number of insufficient defects of the mouth detection part to be 3, enabling a label number of pit defects of the bottle mouth detection part to be 4, enabling a label number of scissor marks of the bottle mouth detection part to be 5, enabling a label number of bubble defects of the bottle bottom to be 6, enabling a label number of bottle bottom stones of the bottle bottom detection part to be 7, enabling a label number of bottle bottom cracks of the bottle bottom detection part to be 8, enabling a label number of bubble defects of the bottle bottom detection part to be 9, enabling a label number of oil stain defects of the bottle body and the bottle bottom detection part to be 10, enabling a label number of stone defects of the bottle body and the bottle bottom detection part to be 11, enabling a label of oil stain defects of the bottle body and the bottle bottom detection part to be 12, and enabling a label of a bottle neck label to be 13 to be a bottle bottom detection part to be a bottle neck label.
5. The method for detecting defects of glass containers based on attention monitor adjustment according to claim 1, wherein the glass container defect detection model in the step (3) is obtained by using a training set and a verification set, and the specific process is as follows:
j-image of ith glass container in training set Inputting the feature images into the first to fifth container defect feature extraction groups, and outputting to obtain feature atlas/>; Map/>、/>、/>Inputting the characteristic conflict pre-filtering module into a container, and outputting to obtain a characteristic diagram/>; The feature map/>、/>And/>Input to a container feature fusion module, and output to obtain feature atlas/>; Feature atlas/>Inputting the glass container defects into a container defect detection unit, and outputting to obtain final characteristics f, wherein the final characteristics f comprise the category, the confidence coefficient and the coordinate information of a prediction frame of the glass container defects; calculating the total loss/>, using the final feature fAnd carrying out gradient optimization on the glass container defect detection model through an optimizer.
6. The method for detecting defects of glass containers based on attention monitor adjustment according to claim 5, wherein the total lossIs confidence loss/>Classification loss/>Loss of frame regression/>Is added up; confidence loss/>And classification loss/>Calculating by adopting binary cross entropy Loss functions respectively, and further balancing the influence of the lost samples by adopting Focal Loss; frame regression loss/>Loss with container defect IoU; the container defect IoU loss is defined as follows:
wherein IoU is the intersection ratio of the predicted and real boxes,/> Center point loss and aspect ratio loss in CIoU employed;
Wherein f is defined as follows:
，
wherein b is defined as follows:
，
Wherein C represents the minimum closed region covering the real region and the predicted region, tanh is a tanh function, and w, h, wgt and hgt represent the height and width of the predicted box, and the height and width of the real box, respectively.
7. The method for detecting defects of glass containers based on attention monitor adjustment according to claim 1, wherein the first defect feature extraction group consists of a convolution group and a feature extraction group, the convolution group consists of a convolution layer with a convolution kernel size of 3×3, a batch normalization layer and a ReLU activation function; the feature extraction group consists of a convolution group and a convolution attention group, wherein the convolution group consists of a convolution layer with the convolution kernel size of 1 multiplied by 1, a batch normalization layer and a ReLU activation function, and the convolution attention group consists of a convolution layer with the convolution kernel size of 3 multiplied by 3, a batch normalization layer, a channel attention layer and a ReLU activation function.
8. The method for detecting defects of a glass container based on attention monitor adjustment according to claim 1, wherein the attention monitor adjustment module controls the global defect characteristics of the image of the glass container to realize the monitor adjustment of the defect characteristics of the glass container; the global defect feature is controlled by carrying out serial connection through global average pooling and one-dimensional convolution; the specific process for controlling the global defect characteristics is as follows:
j-image of ith glass container in training set Input to the first supervision adjustment module, image/>Input to the global averaging pooling layer, output feature map/>Feature map/>Input to one-dimensional convolution, output feature map/>Feature map/>Respectively through two MLP layers, outputting a characteristic diagram/>And/>Feature map/>After Sigmoid activation, the feature map/>, is outputImage/>Respectively and with the characteristic diagram/>And feature map/>Multiplication, output feature map/>And feature map/>Feature map/>Through a convolution layer with the convolution kernel size of 1 multiplied by 1, a characteristic diagram/> isoutputFeature map/>And feature map/>After addition, the feature map/>Completing the attention monitoring and adjusting process; feature map/>Outputting a feature map/>, after passing through the convolution group and the feature extraction group of the first container defect feature extraction group; Identical,/>Inputting the extracted defect characteristics into a second supervision and adjustment module and the first container defect characteristic extraction group, and outputting a characteristic diagram/>, after the operation；/>，/>And/>The same procedure was used.
9. The method for detecting defects of glass containers based on attention monitor adjustment according to claim 1, wherein the third, fourth and fifth defect feature extraction groups input container features three to five simultaneously to the container feature conflict prefiltering module, and a process of outputting prefiltered container prefiltering feature one is as follows:
In container feature three Introducing a coordinate attention mechanism to help container feature three to better focus on space information and outputting a feature map/>Five feature maps/>, of deep container featuresAnd container feature four feature map/>Performing addition operation and outputting a characteristic diagram/>; At the same time, the three feature maps/>, of the container features of the shallow layerAnd container feature four feature map/>Performing addition operation and outputting a characteristic diagram/>By this process, conflicts are pre-filtered, and then the feature maps/>, of both phases are usedAnd/>Spliced together and output a feature map/>; Further optimizing the integrated feature map/>, using a lightweight residual extraction structureOutputting a characteristic diagram; Map of features to be obtained/>And feature map/>Further fusing to obtain prefilter feature I/>。
10. A glass container defect detection system based on attention monitor adjustment for performing the glass container defect detection method based on attention monitor adjustment according to any one of claims 1 to 9, comprising:
the glass container image acquisition module is used for acquiring images of a bottle body, a bottle neck, a bottle mouth and a bottle bottom of the glass container;
And the glass container defect detection module inputs the glass container image into a trained glass container defect detection model and outputs a defect detection result.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410338088.1A CN117934980B (en) | 2024-03-25 | 2024-03-25 | Glass container defect detection method and system based on attention supervision adjustment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410338088.1A CN117934980B (en) | 2024-03-25 | 2024-03-25 | Glass container defect detection method and system based on attention supervision adjustment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN117934980A true CN117934980A (en) | 2024-04-26 |
| CN117934980B CN117934980B (en) | 2024-05-31 |
Family
ID=90768682
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202410338088.1A Active CN117934980B (en) | 2024-03-25 | 2024-03-25 | Glass container defect detection method and system based on attention supervision adjustment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117934980B (en) |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114092389A (en) * | 2021-09-13 | 2022-02-25 | 浙江大学 | Glass panel surface defect detection method based on small sample learning |
| CN115311504A (en) * | 2022-10-10 | 2022-11-08 | 之江实验室 | Weak supervision positioning method and device based on attention repositioning |
| CN115496752A (en) * | 2022-11-16 | 2022-12-20 | 齐鲁工业大学 | Steel surface defect detection method based on one-stage target detection algorithm |
| CN115880298A (en) * | 2023-03-02 | 2023-03-31 | 湖南大学 | Glass surface defect detection method and system based on unsupervised pre-training |
| WO2023070911A1 (en) * | 2021-10-27 | 2023-05-04 | 西安工程大学 | Self-attention-based method for detecting defective area of color-textured fabric |
| CN116721071A (en) * | 2023-06-05 | 2023-09-08 | 南京邮电大学 | Industrial product surface defect detection method and device based on weak supervision |
| CN116994047A (en) * | 2023-08-01 | 2023-11-03 | 北京工商大学 | Small sample image defect target detection method based on self-supervision pre-training |
-
2024
- 2024-03-25 CN CN202410338088.1A patent/CN117934980B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114092389A (en) * | 2021-09-13 | 2022-02-25 | 浙江大学 | Glass panel surface defect detection method based on small sample learning |
| WO2023070911A1 (en) * | 2021-10-27 | 2023-05-04 | 西安工程大学 | Self-attention-based method for detecting defective area of color-textured fabric |
| CN115311504A (en) * | 2022-10-10 | 2022-11-08 | 之江实验室 | Weak supervision positioning method and device based on attention repositioning |
| CN115496752A (en) * | 2022-11-16 | 2022-12-20 | 齐鲁工业大学 | Steel surface defect detection method based on one-stage target detection algorithm |
| CN115880298A (en) * | 2023-03-02 | 2023-03-31 | 湖南大学 | Glass surface defect detection method and system based on unsupervised pre-training |
| CN116721071A (en) * | 2023-06-05 | 2023-09-08 | 南京邮电大学 | Industrial product surface defect detection method and device based on weak supervision |
| CN116994047A (en) * | 2023-08-01 | 2023-11-03 | 北京工商大学 | Small sample image defect target detection method based on self-supervision pre-training |
Non-Patent Citations (2)
| Title |
|---|
| LIU, ZF等: "A context-aware progressive attention aggregation network for fabric defect detection", 《ARXIV》, 30 June 2023 (2023-06-30) * |
| 谢源;苗玉彬;许凤麟;张铭;: "基于半监督深度卷积生成对抗网络的注塑瓶表面缺陷检测模型", 计算机科学, no. 07, 31 December 2020 (2020-12-31) * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN117934980B (en) | 2024-05-31 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111598861B (en) | Improved Faster R-CNN model-based non-uniform texture small defect detection method | |
| CN113643268B (en) | Industrial product defect quality inspection method and device based on deep learning and storage medium | |
| CN112037219A (en) | Metal surface defect detection method based on two-stage convolution neural network | |
| CN113920107A (en) | Insulator damage detection method based on improved yolov5 algorithm | |
| CN114399672A (en) | Railway wagon brake shoe fault detection method based on deep learning | |
| CN113838044A (en) | Panel defect detection method and system based on machine vision | |
| CN115049619B (en) | Efficient flaw detection method for complex scene | |
| CN114120317B (en) | Optical element surface damage identification method based on deep learning and image processing | |
| CN110929795A (en) | Method for quickly identifying and positioning welding spot of high-speed wire welding machine | |
| CN115546565A (en) | YOLOCBF-based power plant key area pipeline oil leakage detection method | |
| CN113222982A (en) | Wafer surface defect detection method and system based on improved YOLO network | |
| CN115035082B (en) | Method for detecting defects of transparent parts of aircraft based on YOLOv4 improved algorithm | |
| CN114742818A (en) | Visual defect detection system based on industrial area array camera and detection method thereof | |
| CN113240647A (en) | Mobile phone shell rear cover defect detection method and system based on deep learning | |
| CN112819748A (en) | Training method and device for strip steel surface defect recognition model | |
| CN117593304A (en) | Semi-supervised industrial product surface defect detection method based on cross local global features | |
| CN117541568A (en) | Deep learning-based automobile brake disc surface defect detection method | |
| CN114494250A (en) | Floor defect detection method based on neural network model | |
| CN114663382A (en) | Surface defect detection method for electronic component based on YOLOv5 convolutional neural network | |
| CN113962980A (en) | Glass container flaw detection method and system based on improved YOLOV5X | |
| CN117269193B (en) | Intelligent detection method for apparent mass of synthetic leather | |
| CN117934980B (en) | Glass container defect detection method and system based on attention supervision adjustment | |
| CN116402753A (en) | Improved YOLOv 5-based steel defect detection method | |
| CN115082650A (en) | Implementation method of automatic pipeline defect labeling tool based on convolutional neural network | |
| CN115222705A (en) | Chemical fiber spinning cake appearance defect detection method based on improved YOLOV5 |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |