CN112132796A - Visual detection method and system for improving detection precision by means of feedback data autonomous learning - Google Patents
Visual detection method and system for improving detection precision by means of feedback data autonomous learning Download PDFInfo
- Publication number
- CN112132796A CN112132796A CN202010966215.4A CN202010966215A CN112132796A CN 112132796 A CN112132796 A CN 112132796A CN 202010966215 A CN202010966215 A CN 202010966215A CN 112132796 A CN112132796 A CN 112132796A
- Authority
- CN
- China
- Prior art keywords
- detection
- module
- feedback information
- data set
- result
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0004—Industrial image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
Abstract
The invention discloses a visual detection method for improving detection precision by feedback data autonomous learning, which comprises the following steps: A. the server receives feedback information uploaded by the detection terminal; B. classifying the feedback information; C. adding problem image data into a training data set according to classification, optimizing detection algorithm parameters by applying the training data set with the added data, issuing the optimized algorithm parameters to a detection terminal, and updating detection algorithm software to improve detection accuracy. Correspondingly, the invention also provides a visual detection system for improving the detection precision by the aid of the feedback data for autonomous learning. The method has the characteristics that the algorithm detection capability is improved through the feedback data in the self-learning practical application, the method is better suitable for diversified application scenes, the workload of manually developing the customized algorithm is reduced, and meanwhile, the traceability is better.
Description
Technical Field
The invention relates to the technical field of visual detection, in particular to a visual detection method and a visual detection system for improving detection precision by means of feedback data autonomous learning.
Background
In the pharmaceutical industry, a light inspection machine is a device for detecting glass medicine bottles filled with medicines. The lamp inspection machine can be divided into a manual lamp inspection machine, a semi-automatic lamp inspection machine and a full-automatic lamp inspection machine according to functions, wherein the semi-automatic lamp inspection machine and the full-automatic lamp inspection machine are still widely used in pharmaceutical industries at home and abroad at present. The basic principle of the semi-automatic lamp inspection machine is that in the automatic transmission process of the medicine bottles to be inspected, the medicine bottles are inspected manually, unqualified products are removed, and the inspection mode can be direct visual judgment or indirect visual judgment on the medicine bottle sequence images shot by the camera device through an electronic display screen. If the computer image algorithm analysis software replaces manual work to detect and analyze the medicine bottle sequence images shot by the camera device and sends a command to the electromechanical control device according to the detection result to remove unqualified bottles, the automatic lamp inspection machine is used.
Theoretically, the full-automatic lamp inspection machine has great advantages compared with a semi-automatic lamp inspection machine: such as improving efficiency, reducing human instability factors, etc. However, in the practical application process, the concept of the full-automatic lamp inspection machine is proposed for many years, and a plurality of domestic and foreign products are available, but the popularization or the wide substitution of the semi-automatic lamp inspection machine cannot be realized. The reason is mainly as follows: firstly, the maturity and the accuracy of an image analysis algorithm do not meet the requirements of actual lamp inspection work, and secondly, the black box working mode of the full-automatic lamp inspection machine causes false inspection and missing inspection and is difficult to trace.
Disclosure of Invention
The invention aims to provide a visual detection method and a visual detection system for improving detection precision by feedback data autonomous learning, which have high detection accuracy, can better adapt to diversified application scenes and have better traceability.
In order to achieve the purpose, the invention adopts the following technical scheme:
a visual detection method for improving detection precision by feedback data autonomous learning is applied to a server, and comprises the following steps:
A. receiving feedback information uploaded by a detection terminal, wherein the feedback information is generated by the detection terminal from an image with a detection result inconsistent with a rechecking result, the detection terminal is provided with a deep learning algorithm module, the detection result is obtained by detecting the image of the product by the deep learning algorithm module, and the rechecking result is obtained by rechecking the detection result;
B. the feedback information comprises problem image data and label information corresponding to the problem image data, and the feedback information is classified;
C. and adding the problem image data into a training data set according to the classification, optimizing the detection algorithm parameters by using the training data set with the added data, and issuing the optimized algorithm parameters to the detection terminal.
Further, the label information further comprises a detection terminal number, product specification, detection time and a rechecking result.
Further, in the step B, a plurality of defect types and a plurality of reasons are pre-stored in the server, and the reason is a possible reason that the detection result is inconsistent with the rechecking result;
judging product defects in the problem image data, and matching defect types and reasons;
the problem image data is then classified according to the label information, the defect category, and the reason.
Further, the training data set comprises a qualified product data set and an unqualified product data set, and the unqualified product data set is divided into a plurality of subdata sets according to the types of product defects;
classifying the problem image data with the review result of being qualified and not matched with the defect category into a qualified product data set, and classifying the problem image data with the defect category into a corresponding subdata set;
the image data and the label information of the problem corresponding to the unknown reason are stored in a separate directory.
Further, firstly, the feedback information with the same equipment number and the same product specification is classified into one type according to the label information, and then the problem image data is classified according to defect types and reasons of each type.
A visual detection method for improving detection precision by feedback data autonomous learning is applied to a detection terminal, the detection terminal is provided with a deep learning algorithm module, and the method comprises the following steps:
a. acquiring an image of a product on a transmission line, and detecting the image by a deep learning algorithm module to obtain a detection result;
b. receiving a rechecking result, generating feedback information when the detection result is inconsistent with the rechecking result, and uploading the feedback information to a server, wherein the feedback information comprises problem image data and label information corresponding to the problem image data;
c. receiving algorithm parameters sent by a server, and updating corresponding parameters in the deep learning algorithm module; the algorithm parameters are algorithm parameters obtained by classifying the feedback information by the server, adding the problem image data into a training data set according to the classification and optimizing the detection algorithm parameters by applying the training data set;
d. and (4) continuously detecting the image by the trained deep learning algorithm module, and repeating the steps b and c.
Further, a Yolov5 model is carried on a deep learning algorithm module, and a Yolov5 model is trained by a universal data set before the step a is carried out;
in step c, the training data set sent by the server is input to the Yolov5 model, and the Yolov5 model is trained.
A visual detection system for improving detection precision by autonomous learning of feedback data comprises a server and a detection terminal, wherein the detection terminal is provided with a deep learning algorithm module;
the detection terminal is used for acquiring images of products on a transmission line, and the deep learning algorithm module is used for detecting the images to obtain a detection result; receiving a rechecking result, generating feedback information when the detection result is inconsistent with the rechecking result, and uploading the feedback information to the server;
the server is used for receiving feedback information uploaded by the detection terminal, wherein the feedback information comprises problem image data and label information corresponding to the problem image data, and classifying the feedback information; and adding the problem image data into a training data set according to the classification, optimizing the algorithm parameters of the detection algorithm by using the training data set of the newly added problem data, and issuing the optimized algorithm parameters to the detection terminal.
Further, the server comprises a first receiving module, a first storage module, a judging module, a classifying module, a data set module and a sending module;
the first receiving module is used for receiving feedback information uploaded by the detection terminal;
the first storage module is used for storing a plurality of defect types and a plurality of reasons, wherein the reasons refer to possible reasons for inconsistency between the detection result and the rechecking result;
the judging module is used for judging the product defects and reasons in the problem image data and matching the defect types and reasons;
the data set module is used for storing a qualified product data set and an unqualified product data set, and the unqualified product data set is divided into a plurality of subdata sets according to the product defect types;
the classification module classifies the problem image data according to defect categories and reasons, classifies the problem image data which is qualified in the review result and is not matched with the defect categories into a qualified product data set, classifies the problem image data matched with the defect categories into a corresponding subdata set, and stores the problem image data and label information corresponding to the unknown reasons into an independent directory;
and the issuing module is used for issuing the optimized algorithm parameters to the detection terminal.
Furthermore, the detection terminal also comprises an image acquisition module, a rechecking judgment module, a second storage module, an uploading module and a second receiving module;
the image acquisition module is used for acquiring an image of a product on a transmission line;
the rechecking judgment module is used for receiving a rechecking result and generating feedback information when the detection result is inconsistent with the rechecking result;
the second storage module is used for storing the feedback information;
the uploading module is used for uploading feedback information periodically;
and the second receiving module is used for receiving the algorithm parameters sent by the server.
The invention has the beneficial effects that:
the visual detection method adjusts and updates the training data set according to the feedback information so as to optimize algorithm parameters in the training data set, retrains the deep learning algorithm module by adopting the optimized algorithm parameters, improves the detection accuracy of the deep learning algorithm module, and the deep learning algorithm module has high detection accuracy after the training is carried out for multiple times, so that the visual detection method can have higher and higher detection accuracy by applying accumulation after being applied for a period of time in actual production; in addition, the method can also automatically adapt to the adjustment of the detection requirements of users, such as adding new product specifications and redefining the detection standard; meanwhile, the feedback information contains label information, so that the traceability of data is realized, and the method can be accepted by customers and widely applied. The visual detection method and the visual detection system improve the algorithm detection capability by feedback data, can better adapt to diversified application scenes, and reduce the workload of artificial customized algorithm development.
Drawings
FIG. 1 is a flowchart of a visual inspection method for improving inspection accuracy by autonomous learning with feedback data according to an embodiment of the present invention;
FIG. 2 is a flowchart of a visual inspection method for improving inspection accuracy by autonomous learning with feedback data according to an embodiment of the present invention;
FIG. 3 is a frame diagram of a deep learning network carried by a deep learning algorithm module;
FIG. 4 is a block diagram of the framework of the BottleneckCSP module and the SPP module;
FIG. 5 is a diagram of a vision inspection system with feedback data for autonomous learning to improve inspection accuracy, according to an embodiment of the present invention;
the system comprises a server 1, a first receiving module 11, a first storage module 12, a judging module 13, a classifying module 14, a data set module 15 and a sending module 16, wherein the server is connected with the first receiving module 11; the system comprises a detection terminal 2, an image acquisition module 21, a review judgment module 22, a second storage module 23, an uploading module 24, a deep learning algorithm module 25 and a second receiving module 26.
Detailed Description
The technical solution of the present invention is further described below with reference to the accompanying drawings and the detailed description.
In recent years, a new-generation artificial intelligence technology represented by deep learning is rapidly developed and matured, and compared with the traditional machine learning technology, the image target detection performance level is remarkably improved. However, the detection performance of algorithms such as deep learning and the like depends on an accurate and comprehensive training data set. In the practical application scenario of the light inspection machine, the product specifications and inspection requirements of different users are different, and the representative samples provided by the light inspection machine do not necessarily cover all the typical cases in the practical reference. Therefore, the requirements of each user and the conditions of the medicine bottles need to be trained and adjusted respectively in practical application on the basis of a unified algorithm model. The visual detection method is carried out based on a deep learning algorithm. The details are as follows.
The invention provides a visual detection method for improving detection precision by autonomously learning feedback data, which is applied to a server and comprises the following steps:
A. receiving feedback information uploaded by a detection terminal, wherein the feedback information is generated by the detection terminal from an image with a detection result inconsistent with a rechecking result, the detection terminal is provided with a deep learning algorithm module, the detection result is obtained by detecting the image of the product by the deep learning algorithm module, and the rechecking result is obtained by rechecking the detection result;
B. classifying the feedback information;
C. the feedback information comprises problem image data, the problem image data is added into a training data set according to classification, the training data set of the newly added problem data is applied to optimize the parameters of the detection algorithm, and the optimized algorithm parameters are issued to the detection terminal. The problem image data refers to an image with inconsistent detection result and rechecking result, namely a problem image.
And performing rechecking operation in the detection process, namely judging whether the detection result of the detection terminal is correct or not, namely judging whether the detection result of the image is correct or not by the deep learning algorithm module. Specifically, the detection terminal is provided with a display screen, a detection result is displayed on the display screen, whether the detection result is correct or not is judged manually, the rechecking result is input into the detection terminal in a man-machine interaction mode, and when the detection result is inconsistent with the rechecking result, the detection terminal automatically generates feedback information. The case where the detection result and the rechecking result are inconsistent is: the detection result is a qualified product, and the rechecking result is an unqualified product; the detection result is non-qualified, and the rechecking result is qualified. The server and the detection terminal establish remote connection.
The visual detection method of the invention uses the training data set added with feedback information adjustment for training and optimizing the parameters of the deep learning algorithm module, adopts the deep learning algorithm module after parameter optimization for detection, improves the detection accuracy, and has higher detection accuracy after the deep learning algorithm module is trained for multiple times, so that the visual detection method of the invention can have higher and higher detection accuracy by applying accumulation after being applied for a period of time in the actual production; in addition, the method can also automatically adapt to the adjustment of the detection requirements of users, such as adding new product specifications and redefining the detection standard; meanwhile, the feedback information contains label information, so that the traceability of data is realized, and the method can be accepted by customers and widely applied. The visual detection method provided by the invention improves the algorithm detection capability by using the feedback data, can better adapt to diversified application scenes, and reduces the workload of artificial customized algorithm development.
The visual inspection method of the invention can be applied to the appearance inspection of various products, such as the appearance inspection of bottle bodies and the appearance inspection of box bodies, and is also applied to the appearance inspection of transparent packages, such as transparent glass medicine bottles. When the product is a transparent glass vial, the shape of the contents, such as the state of caking, can also be detected.
Further, the feedback information further includes label information corresponding to the problem image data, and the label information includes a detection terminal number, a product specification, detection time, and a rechecking result.
It should be noted that, in the visual inspection method of the present invention, there may be more than one inspection terminal, and multiple inspection terminals may correspond to different products, or inspect and inspect products of different specifications at different times.
The label and the product specification of the detection terminal are added into the label information, so that the feedback information can be accurately classified, and the detection traceability can be realized.
Further, in the step B, a plurality of defect types and a plurality of reasons are pre-stored in the server, and the reason is a possible reason that the detection result is inconsistent with the rechecking result;
judging product defects in the problem image data, and matching defect types and reasons;
the problem image data is then classified according to the label information, the defect category, and the reason.
Specifically, for glass medicine bottles, the defect types include no cover, body defect, bottom falling and bottle cover defect; possible causes of inconsistency include failure, standard changes, new specifications, and lack of detail.
Wherein, the failure reason refers to: the specifications and defects of the inspected product are included in the prior requirement definition and training data set, and the algorithm can be inspected but cannot be inspected successfully, and is defined as failure;
the reasons for the standard variation mean: the standard definition of the defect of a manufacturer is changed, so that the algorithm detection based on the original standard training is inconsistent with the rechecking detection result based on the new standard;
the reasons for the new specifications are: the product specification corresponding to the data is not in the detection requirement and training data set defined previously, so that the algorithm cannot carry out accurate detection and is inconsistent with the rechecking result;
the unknown reason is determined to be an unknown reason when the problem image data cannot be matched with any other reason.
The classification rules are shown in the following table:
| device numbering | 0001/0002/0003/…/9999 |
| Specification of medicine bottle | 2ml/7ml/…/50ml |
| Review the results | Pass/fail |
| Defective classification of defective products | Uncovered/defective/fallen/… bottle body |
| The reason of the error judgment of the algorithm at this time | Fail/standard change/new specification/unknown |
Further, the training data set comprises a qualified product data set and an unqualified product data set, and the unqualified product data set is divided into a plurality of subdata sets according to the types of product defects;
classifying the problem image data with the review result of being qualified and not matched with the defect category into a qualified product data set, and classifying the problem image data with the defect category into a corresponding subdata set;
the image data and the label information of the problem corresponding to the unknown reason are stored in a separate directory.
The method divides the training data set into a plurality of subsets, and after a period of time, when one or more subsets are updated, the method only needs to issue the parameters corresponding to the updated subsets to the detection terminal, thereby reducing the amount of information transmission and reducing the data processing amount of the detection terminal.
The feedback information corresponding to the unknown reasons is added into the independent catalog so as to be convenient for analyzing and processing the information of the unknown reasons, preferably, a prompt is set in the independent catalog, and when the quantity of the feedback information in the independent catalog reaches a certain value, the prompt is given to prompt a worker to process the information of the catalog. Specifically, the staff classifies the information in the independent directory and adds the classified problem image data into the training data set, so that the training data set is further accurately adjusted, and the detection accuracy of the detection terminal is improved.
Further, firstly, the feedback information with the same equipment number and the same product specification is classified into one type according to the label information, and then the problem image data is classified according to defect types and reasons of each type.
When the detection terminal detects products with various specifications within a period of time, the plurality of detection terminals upload feedback information within a period of time, and at the moment, the feedback information needs to be classified according to the same equipment number and the product specifications. Meanwhile, training data sets corresponding to products of different specifications are set in the server, so that the same detection terminal can detect products of various specifications. Preferably, the server automatically issues the optimized algorithm parameters to the detection terminal according to the feedback information uploaded by the detection terminal.
Correspondingly, the invention also provides a visual detection method for improving detection precision by the aid of feedback data autonomous learning, which is applied to a detection terminal, wherein the detection terminal is provided with a deep learning algorithm module, and the method comprises the following steps:
a. acquiring an image of a product on a transmission line, and detecting the image by a deep learning algorithm module to obtain a detection result;
b. receiving a rechecking result, generating feedback information when the detection result is inconsistent with the rechecking result, and uploading the feedback information to a server;
c. receiving algorithm parameters sent by a server, training a deep learning algorithm module by using the training algorithm parameters, wherein feedback information comprises problem image data, the server classifies the feedback information, adds the problem image data into a training data set according to the classification, and optimizes the detection algorithm parameters by using the training data set after the new data is added, and the algorithm parameters sent by the server are optimized algorithm parameters;
d. and (4) continuously detecting the image by the trained deep learning algorithm module, and repeating the steps b and c.
The detection terminal is provided with a camera and is used for acquiring images of products on the transmission line. The detection terminal uploads feedback information to the server periodically and automatically receives the optimized algorithm parameters sent by the server.
The detection terminal adopts the depth algorithm detection module after algorithm parameter optimization updating, and detection accuracy is improved. In the visual detection method, the detection terminal repeatedly performs detection, uploads feedback information and updates detection algorithm parameters. After the training is carried out for multiple times, the deep learning algorithm module has high detection accuracy, and after the detection terminal is applied for a period of time in actual production, the detection terminal can have higher and higher detection accuracy through application accumulation; in addition, the method can automatically adapt to the adjustment of the detection requirements of users, such as adding new product specifications and redefining detection standards; meanwhile, the feedback information contains the label information, so that the traceability of data is realized, and the method can be accepted by customers and widely applied.
Further, a Yolov5 model is carried on a deep learning algorithm module, and a Yolov5 model is trained by a universal data set before the step a is carried out;
in step c, the training data set sent by the server is input to the Yolov5 model, and the Yolov5 model is trained.
The visual detection method adopts a Yolov5 model to carry out image detection. The Yolov5 Model includes a Model backbone (backbone), a Model Neck (Model neutral), and a Model Head (Model Head: detection output). The Yolo series of loss calculations are based on the object score, class probability score, and bounding box regression score. Yolov5 uses GIOU Loss as the Loss of bounding box, and uses binary cross entropy and logs Loss functions to calculate the Loss of class probability and target score. The fl _ gamma parameter may also be used to activate the Focal loss calculation loss function.
The model skeleton is mainly used for extracting important features from an input image. A CSP network (Cross stage Partial Networks) of YOLOV5 was used as a model skeleton. CSPNet solves the problem of repeated gradient information of network optimization existing in other large convolutional neural network frameworks, and integrates the gradient change into a characteristic diagram from beginning to end, so that the parameter number and FLOPS (floating point operation times per second) of the model are reduced, the inference speed and accuracy are ensured, and the size of the model is reduced.
The model neck is mainly used to generate the feature pyramid. The feature pyramid is beneficial to improving the generalization of the model on object scaling and helping the model to identify the same object with different sizes and proportions. In the invention, the characteristics are aggregated by taking PANET as the sock.
The model header is used primarily to perform the final decision part. It applies an anchor box on the features and generates the final output vector with class probabilities, object scores and bounding boxes. The head parts yolov3 and yolov4 are all the same and are also three output heads.
It should be noted that Yolov3, Yolov4, and Yolov5 are all existing depth learning image detection algorithms.
In the Yolov5 model of the present invention, the Leaky ReLU and sigmoid activation functions are employed, the Leaky ReLU activation function being used for the intermediate/hidden layer and the sigmoid activation function being used for the final detection layer. And the sigmoid activation function is adopted to further detect the detection result and distinguish qualified products from unqualified products.
Specifically, the model backbone is composed of a BottleneckCSP module and an SPP module, and the number of the BottleneckCSP modules is increased or decreased according to the task complexity. The BottleneckCSP module is divided into a Bottleneck and a CSP. Bottlenneck is a classical residual structure, which is a 1 × 1 convolutional layer (conv + batch _ norm + leakage relu), then a 3 × 3 convolutional layer, and finally added to the initial input by the residual structure. CSP: the original input is divided into two branches, convolution operation is respectively carried out to reduce the number of channels by half, then a branch I carries out Bottleneck x N operation, and subsequently a concat branch I and a branch II are carried out, so that the input and the output of the Bottleneck CSP are the same in size. The change of the gradient is integrated into the characteristic diagram from beginning to end, so that the parameter quantity and the FLOPS value of the model are reduced, the reasoning speed and the accuracy are ensured, and the size of the model is reduced.
An SPP module: SPP outputs the input after passing through a convolutional layer of 1x1, then performs down sampling through three parallel Maxpools, adds the result and the initial characteristic of the result and outputs the result, and finally restores the result to the same size as the input by using a convolution kernel of 512. The method has the effect of spatial pyramid pooling, so that feature maps of any size can be converted into feature vectors of fixed sizes, and target feature maps of different sizes are ensured to be consistent as much as possible, so that the deep learning network can be suitable for targets of different scales.
PANET: path Aggregation Network (PANET), based on Mask R-CNN and FPN frameworks, while enhancing information dissemination, has the ability to accurately retain spatial information, which helps properly position pixels to form a Mask.
According to the method, a two-classification function is added on the basis of a Yolov5 model, specifically, a feature graph extracted by a backhaul module and an spp module of the Yolov5 model is used as input, two classifications are performed by a sigmoid activation function through a plurality of convolutional neural networks and a full connection layer, a current detection object is output by the networks to be qualified or unqualified, a simple target detection task can be effectively avoided, and a defect which is to be unidentifiable due to missed detection and cannot be identified is output as one classification.
Correspondingly, the invention also provides a visual detection system with learning capability, which comprises a server 1 and a detection terminal 2, wherein the detection terminal 2 is provided with a deep learning algorithm module 25;
the detection terminal 2 is used for acquiring images of products on a transmission line, and the deep learning algorithm module 25 detects the images to obtain a detection result; receiving a rechecking result, generating feedback information when the detection result is inconsistent with the rechecking result, and uploading the feedback information to the server 1;
the server 1 is used for receiving the feedback information uploaded by the detection terminal 2 and classifying the feedback information; the feedback information comprises problem image data, the problem image data is added into a training data set according to classification, the training data set of the newly added problem data is applied to optimize the detection algorithm parameters, and the optimized algorithm parameters are issued to the detection terminal.
By adopting the visual detection system, the deep learning algorithm module can be continuously trained, and the detection accuracy is continuously improved; in addition, the method can automatically adapt to the adjustment of the detection requirements of users, such as adding new product specifications and redefining detection standards; furthermore, label information is added into the feedback information, so that the detection can be traced, and the method can be accepted by customers and is widely applied.
Further, the server 1 includes a first receiving module 11, a first storage module 12, a judging module 13, a classifying module 14, a data set module 15 and a distributing module 16;
the first receiving module 11 is configured to receive feedback information uploaded by the detection terminal;
the first storage module 12 is configured to store a plurality of defect categories and a plurality of reasons, where a reason is a possible reason that a detection result is inconsistent with a rechecking result;
the judging module 13 is used for judging product defects and reasons in the problem image data and matching defect types and reasons;
the data set module 14 is used for storing a qualified product data set and an unqualified product data set, wherein the unqualified product data set is divided into a plurality of subdata sets according to the types of product defects;
the classification module 15 classifies the problem image data according to the defect category and the reason, classifies the problem image data which is qualified in the review result and is not matched with the defect category into a qualified product data set, classifies the problem image data matched with the defect category into a corresponding subdata set, and stores the problem image data corresponding to the unknown reason and the label information into an independent directory;
and the issuing module 16 is configured to issue the optimized algorithm parameter to the detection terminal.
Further, the detection terminal 2 further includes an image acquisition module 21, a review judgment module 22, a second storage module 23, an upload module 24, and a second receiving module 26;
the image acquisition module 21 is used for acquiring an image of a product on a transmission line;
the rechecking judgment module 22 is configured to receive a rechecking result, and generate feedback information when the detection result is inconsistent with the rechecking result;
a second storage module 23, configured to store the feedback information;
the uploading module 24 is used for uploading feedback information periodically;
and a second receiving module 26, configured to receive the algorithm parameter sent by the server 1.
After the image obtaining module 21 obtains the image, the detection terminal generates tag information at the same time, and sends the image and the corresponding tag information to the deep learning algorithm module 25 at the same time. The detecting terminal 2 stores the feedback information in the second storage module, and sets a fixed uploading time in the uploading module 24, for example, the feedback information is uploaded once every week.
Specifically, the image obtaining module 21 includes a camera, and the camera is used for obtaining an image of a product on the transmission line. Preferably, set up a plurality of cameras, realize the acquirement to product multi-angle image to realize detecting the product full angle. The detection terminal 2 is further provided with a display screen for displaying the detection result so as to facilitate manual review to judge whether the detection structure is correct or not, and the review result is input into the detection terminal through the man-machine interaction module.
The technical principle of the present invention is described above in connection with specific embodiments. The description is made for the purpose of illustrating the principles of the invention and should not be construed in any way as limiting the scope of the invention. Based on the explanations herein, those skilled in the art will be able to conceive of other embodiments of the present invention without inventive effort, which would fall within the scope of the present invention.
Claims (10)
1. A visual detection method for improving detection precision by feedback data autonomous learning is applied to a server and is characterized by comprising the following steps:
A. receiving feedback information uploaded by a detection terminal, wherein the feedback information is generated by the detection terminal from an image with a detection result inconsistent with a rechecking result, the detection terminal is provided with a deep learning algorithm module, the detection result is obtained by detecting the image of a product by the deep learning algorithm module, and the rechecking result is obtained by rechecking the detection result;
B. classifying the feedback information;
C. the feedback information comprises problem image data, the problem image data is added into a training data set according to classification, the training data set with the added data is used for optimizing the detection algorithm parameters, and the optimized algorithm parameters are sent to the detection terminal.
2. The visual inspection method of claim 1, wherein the feedback information further comprises label information corresponding to the problem image data, and the label information comprises a detection terminal number, a product specification, a detection time, and a review result.
3. The visual inspection method for improving inspection accuracy through autonomous learning of feedback data according to claim 2, wherein in the step B, a plurality of defect categories and a plurality of reasons are pre-stored in the server, and the reasons are possible reasons for inconsistency between the inspection result and the review result;
judging product defects in the problem image data, and matching defect types and reasons;
the problem image data is then classified according to the label information, the defect category, and the reason.
4. The visual inspection method of claim 3, wherein the training data set comprises a qualified product data set and an unqualified product data set, and the unqualified product data set is divided into a plurality of subdata sets according to the product defect type;
classifying the problem image data with a rechecking result being qualified and without a corresponding defect category into a qualified product data set, and classifying the problem image data with the defect category into a corresponding subdata set;
the image data and the label information of the problem corresponding to the unknown reason are stored in a separate directory.
5. The visual inspection method of claim 3, wherein the feedback information with the same equipment number and the same product specification is classified into one category according to the label information, and then the problem image data is classified into the defect category and the reason for each category.
6. A visual detection method for improving detection precision by feedback data autonomous learning is applied to a detection terminal and is characterized in that the detection terminal is provided with a deep learning algorithm module, and the method comprises the following steps:
a. acquiring an image of a product on a transmission line, and detecting the image by a deep learning algorithm module to obtain a detection result;
b. receiving a rechecking result, generating feedback information when the detection result is inconsistent with the rechecking result, and uploading the feedback information to a server;
c. receiving algorithm parameters sent by a server, and updating corresponding parameters in the deep learning algorithm module; the feedback information comprises problem image data, the server classifies the feedback information and adds the problem image data into a training data set according to the classification, the server optimizes the deep learning algorithm module parameters by applying the training data set added with the problem data, and the algorithm parameters sent by the server are optimized algorithm parameters;
d. and (4) continuously detecting the image by the trained deep learning algorithm module, and repeating the steps b and c.
7. The visual inspection method for improving inspection accuracy through autonomous learning of feedback data according to claim 6, wherein the deep learning algorithm module is equipped with a Yolov5 model, and the Yolov5 model is trained with a common data set before step a;
in step c, the training data set sent by the server is input to the Yolov5 model, and the Yolov5 model is trained.
8. A visual detection system for improving detection precision by autonomous learning of feedback data is characterized by comprising a server and a detection terminal, wherein the detection terminal is provided with a deep learning algorithm module;
the detection terminal is used for acquiring images of products on a transmission line, and the deep learning algorithm module is used for detecting the images to obtain a detection result; receiving a rechecking result, generating feedback information when the detection result is inconsistent with the rechecking result, and uploading the feedback information to the server;
the server is used for receiving the feedback information uploaded by the detection terminal and classifying the feedback information; the feedback information comprises problem image data, the problem image data is added into a training data set according to classification, the server optimizes algorithm parameters by using the training data set added with the problem data, and the optimized algorithm parameters are issued to the detection terminal.
9. The vision inspection system of claim 8, wherein the server comprises a first receiving module, a first storage module, a judgment module, a classification module, a data set module, and a distribution module;
the first receiving module is used for receiving feedback information uploaded by the detection terminal;
the first storage module is used for storing a plurality of defect types and a plurality of reasons, wherein the reasons are possible reasons for inconsistency between the detection result and the rechecking result;
the judging module is used for judging the product defects and reasons in the problem image data and matching the defect types and reasons;
the data set module is used for storing a qualified product data set and an unqualified product data set, and the unqualified product data set is divided into a plurality of subdata sets according to the types of product defects;
the classification module classifies the problem image data according to defect types and reasons, classifies the problem image data with a rechecking result of being qualified and no corresponding defect type into a qualified product data set, classifies the problem image data with the defect type into a corresponding subdata set, and stores the problem image data and label information corresponding to the unknown reasons into an independent directory;
and the issuing module is used for issuing the optimized algorithm parameters to the detection terminal.
10. The vision inspection system for improving inspection accuracy through autonomous learning of feedback data according to claim 8, wherein the inspection terminal further comprises an image acquisition module, a review judgment module, a second storage module, an upload module and a second receiving module;
the image acquisition module is used for acquiring an image of a product on a transmission line;
the rechecking judgment module is used for receiving a rechecking result and generating feedback information when the detection result is inconsistent with the rechecking result;
the second storage module is used for storing the feedback information;
the uploading module is used for uploading the feedback information periodically;
and the second receiving module is used for receiving the algorithm parameters sent by the server.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010966215.4A CN112132796A (en) | 2020-09-15 | 2020-09-15 | Visual detection method and system for improving detection precision by means of feedback data autonomous learning |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010966215.4A CN112132796A (en) | 2020-09-15 | 2020-09-15 | Visual detection method and system for improving detection precision by means of feedback data autonomous learning |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112132796A true CN112132796A (en) | 2020-12-25 |
Family
ID=73846995
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010966215.4A Pending CN112132796A (en) | 2020-09-15 | 2020-09-15 | Visual detection method and system for improving detection precision by means of feedback data autonomous learning |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112132796A (en) |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112749681A (en) * | 2021-01-25 | 2021-05-04 | 长威信息科技发展股份有限公司 | Violation detection method based on edge equipment and deep learning |
| CN112767349A (en) * | 2021-01-18 | 2021-05-07 | 桂林优利特医疗电子有限公司 | Reticulocyte recognition method and system |
| CN112766344A (en) * | 2021-01-12 | 2021-05-07 | 南京信息工程大学 | Improved contraband detection method based on YOLOv5 optimizer |
| CN112801959A (en) * | 2021-01-18 | 2021-05-14 | 华南理工大学 | Auxiliary assembly system based on visual feature recognition |
| CN113033376A (en) * | 2021-03-22 | 2021-06-25 | 陕西科技大学 | Captive goat counting method based on deep learning |
| CN113256568A (en) * | 2021-05-09 | 2021-08-13 | 长沙长泰智能装备有限公司 | Machine vision plate counting general system and method based on deep learning |
| CN116893175A (en) * | 2023-07-04 | 2023-10-17 | 千顺智能(珠海)有限公司 | Multi-station multi-defect unified connector measurement system based on deep learning |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103559520A (en) * | 2013-11-06 | 2014-02-05 | 广东兰贝斯信息科技有限公司 | Application method of label system of fresh aquatic food |
| CN108154504A (en) * | 2017-12-25 | 2018-06-12 | 浙江工业大学 | A kind of detection method of the Surface Defects in Steel Plate based on convolutional neural networks |
| US20180341248A1 (en) * | 2017-05-24 | 2018-11-29 | Relativity Space, Inc. | Real-time adaptive control of additive manufacturing processes using machine learning |
| CN109376247A (en) * | 2018-11-13 | 2019-02-22 | 扬州大学 | A kind of software defect automatic classification method based on correlation rule |
| CN109389136A (en) * | 2017-08-08 | 2019-02-26 | 上海为森车载传感技术有限公司 | Classifier training method |
| CN109919925A (en) * | 2019-03-04 | 2019-06-21 | 联觉(深圳)科技有限公司 | Printed circuit board intelligent detecting method, system, electronic device and storage medium |
| US20200090320A1 (en) * | 2018-08-31 | 2020-03-19 | Alibaba Group Holding Limited | System and method for training a damage identification model |
| CN110930350A (en) * | 2018-09-19 | 2020-03-27 | 鸿富锦精密电子(天津)有限公司 | Machine learning method and automatic optical detection equipment applying same |
-
2020
- 2020-09-15 CN CN202010966215.4A patent/CN112132796A/en active Pending
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103559520A (en) * | 2013-11-06 | 2014-02-05 | 广东兰贝斯信息科技有限公司 | Application method of label system of fresh aquatic food |
| US20180341248A1 (en) * | 2017-05-24 | 2018-11-29 | Relativity Space, Inc. | Real-time adaptive control of additive manufacturing processes using machine learning |
| CN109389136A (en) * | 2017-08-08 | 2019-02-26 | 上海为森车载传感技术有限公司 | Classifier training method |
| CN108154504A (en) * | 2017-12-25 | 2018-06-12 | 浙江工业大学 | A kind of detection method of the Surface Defects in Steel Plate based on convolutional neural networks |
| US20200090320A1 (en) * | 2018-08-31 | 2020-03-19 | Alibaba Group Holding Limited | System and method for training a damage identification model |
| CN110930350A (en) * | 2018-09-19 | 2020-03-27 | 鸿富锦精密电子(天津)有限公司 | Machine learning method and automatic optical detection equipment applying same |
| CN109376247A (en) * | 2018-11-13 | 2019-02-22 | 扬州大学 | A kind of software defect automatic classification method based on correlation rule |
| CN109919925A (en) * | 2019-03-04 | 2019-06-21 | 联觉(深圳)科技有限公司 | Printed circuit board intelligent detecting method, system, electronic device and storage medium |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112766344A (en) * | 2021-01-12 | 2021-05-07 | 南京信息工程大学 | Improved contraband detection method based on YOLOv5 optimizer |
| CN112767349A (en) * | 2021-01-18 | 2021-05-07 | 桂林优利特医疗电子有限公司 | Reticulocyte recognition method and system |
| CN112801959A (en) * | 2021-01-18 | 2021-05-14 | 华南理工大学 | Auxiliary assembly system based on visual feature recognition |
| CN112801959B (en) * | 2021-01-18 | 2023-08-22 | 华南理工大学 | Auxiliary assembly system based on visual feature recognition |
| CN112767349B (en) * | 2021-01-18 | 2024-05-03 | 桂林优利特医疗电子有限公司 | Reticulocyte identification method and system |
| CN112749681A (en) * | 2021-01-25 | 2021-05-04 | 长威信息科技发展股份有限公司 | Violation detection method based on edge equipment and deep learning |
| CN113033376A (en) * | 2021-03-22 | 2021-06-25 | 陕西科技大学 | Captive goat counting method based on deep learning |
| CN113256568A (en) * | 2021-05-09 | 2021-08-13 | 长沙长泰智能装备有限公司 | Machine vision plate counting general system and method based on deep learning |
| CN116893175A (en) * | 2023-07-04 | 2023-10-17 | 千顺智能(珠海)有限公司 | Multi-station multi-defect unified connector measurement system based on deep learning |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112132796A (en) | Visual detection method and system for improving detection precision by means of feedback data autonomous learning | |
| CN113192040B (en) | Fabric flaw detection method based on YOLO v4 improved algorithm | |
| CN108686978B (en) | ARM-based fruit category and color sorting method and system | |
| CN110991435A (en) | Express waybill key information positioning method and device based on deep learning | |
| CN107966454A (en) | A kind of end plug defect detecting device and detection method based on FPGA | |
| CN109064459A (en) | A kind of Fabric Defect detection method based on deep learning | |
| CN111401418A (en) | Employee dressing specification detection method based on improved Faster r-cnn | |
| CN111444939A (en) | Small-scale equipment component detection method based on weak supervision cooperative learning in open scene of power field | |
| US20220076404A1 (en) | Defect management apparatus, method and non-transitory computer readable medium | |
| CN112017172A (en) | System and method for detecting defects of deep learning product based on raspberry group | |
| CN106645180A (en) | Method for checking defects of substrate glass, field terminal and server | |
| CN113902710A (en) | Method and system for detecting surface defects of industrial parts based on anomaly detection algorithm | |
| CN114549507A (en) | Method for detecting fabric defects by improving Scaled-YOLOv4 | |
| CN113962980A (en) | Glass container flaw detection method and system based on improved YOLOV5X | |
| CN112819780A (en) | Method and system for detecting surface defects of silk ingots and silk ingot grading system | |
| CN112183374A (en) | Automatic express sorting device and method based on raspberry group and deep learning | |
| Anush et al. | Bottle line detection using digital image processing with machine learning | |
| CN113610831B (en) | Wood defect detection method based on computer image technology and transfer learning | |
| CN113838015B (en) | Electrical product appearance defect detection method based on network cooperation | |
| CN115937492A (en) | Transformer equipment infrared image identification method based on feature identification | |
| CN111160330B (en) | Training method for improving image recognition accuracy with assistance of electronic tag recognition | |
| Ye et al. | A Deep Learning-based Generic Solder Defect Detection System | |
| Liu et al. | Research on image recognition of supermarket commodity based on convolutional neural network | |
| Dighvijay et al. | A Faster R-CNN implementation of presence inspection for parts on industrial produce | |
| CN111474181A (en) | Panel defect detecting and classifying system in box forming process stage |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |