CN114266751A - AI technology-based product packaging bag coding defect detection method and system - Google Patents
AI technology-based product packaging bag coding defect detection method and system Download PDFInfo
- Publication number
- CN114266751A CN114266751A CN202111589616.3A CN202111589616A CN114266751A CN 114266751 A CN114266751 A CN 114266751A CN 202111589616 A CN202111589616 A CN 202111589616A CN 114266751 A CN114266751 A CN 114266751A
- Authority
- CN
- China
- Prior art keywords
- image
- packaging bag
- product packaging
- character
- recognition
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000004806 packaging method and process Methods 0.000 title claims abstract description 79
- 230000007547 defect Effects 0.000 title claims abstract description 68
- 238000001514 detection method Methods 0.000 title claims abstract description 57
- 238000005516 engineering process Methods 0.000 title claims abstract description 24
- 238000005520 cutting process Methods 0.000 claims abstract description 15
- 102100032202 Cornulin Human genes 0.000 claims abstract description 11
- 101000920981 Homo sapiens Cornulin Proteins 0.000 claims abstract description 11
- 238000007781 pre-processing Methods 0.000 claims abstract description 11
- 238000013136 deep learning model Methods 0.000 claims abstract description 6
- 238000000034 method Methods 0.000 claims description 43
- 238000012545 processing Methods 0.000 claims description 11
- 238000013528 artificial neural network Methods 0.000 claims description 9
- 238000013527 convolutional neural network Methods 0.000 claims description 7
- 230000006870 function Effects 0.000 claims description 7
- 230000002159 abnormal effect Effects 0.000 claims description 6
- 230000003044 adaptive effect Effects 0.000 claims description 4
- 239000000284 extract Substances 0.000 claims description 4
- 230000007797 corrosion Effects 0.000 claims description 3
- 238000005260 corrosion Methods 0.000 claims description 3
- 230000000877 morphologic effect Effects 0.000 claims description 3
- 230000009467 reduction Effects 0.000 claims description 3
- 230000002441 reversible effect Effects 0.000 claims description 2
- 238000005070 sampling Methods 0.000 claims description 2
- 238000004519 manufacturing process Methods 0.000 description 26
- 238000012015 optical character recognition Methods 0.000 description 19
- 238000003860 storage Methods 0.000 description 6
- 238000007689 inspection Methods 0.000 description 5
- 238000009432 framing Methods 0.000 description 4
- 238000012360 testing method Methods 0.000 description 4
- 238000010586 diagram Methods 0.000 description 3
- 238000005286 illumination Methods 0.000 description 3
- 239000011159 matrix material Substances 0.000 description 3
- 238000012790 confirmation Methods 0.000 description 2
- 238000013135 deep learning Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000013473 artificial intelligence Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000012937 correction Methods 0.000 description 1
- 230000002950 deficient Effects 0.000 description 1
- 230000003111 delayed effect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000008030 elimination Effects 0.000 description 1
- 238000003379 elimination reaction Methods 0.000 description 1
- 238000003780 insertion Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 230000008569 process Effects 0.000 description 1
- 230000000306 recurrent effect Effects 0.000 description 1
- 230000000717 retained effect Effects 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 230000001960 triggered effect Effects 0.000 description 1
Images
Landscapes
- Image Analysis (AREA)
- Image Processing (AREA)
Abstract
The invention relates to a product packaging bag coding defect detection method based on AI technology, which comprises the following steps: step S1, obtaining the product packaging bag image and cutting; step S2, according to the cut product packaging bag image, triggering image recognition based on soft triggering or hard triggering; step S3, preprocessing the cut product packaging bag image; step S4, forming a text box by adopting a DB algorithm based on the preprocessed image; step S5, according to the obtained text box, character recognition is carried out based on the deep learning model CRNN to obtain the content result of character prediction and the corresponding probability value; and step S6, acquiring the detection result of the defect target based on the content result of the character prediction and the corresponding probability value. The invention realizes the product package identification and defect detection of various simple or complex backgrounds of various types of packages such as plastic packages, paper boxes and the like.
Description
Technical Field
The invention relates to the field of intelligent identification, in particular to a product packaging bag coding defect detection method and system based on AI technology.
Background
With the development of artificial intelligence technology, the recognition of product packaging bag coding characters and the defect detection by using an OCR technology and a machine vision method become practical.
The existing methods for recognizing the code characters of the product packaging bags mainly comprise the traditional OCR technology and the OCR technology based on the deep neural network. The traditional OCR technology forms a mature technical flow system, which mainly comprises image preprocessing, denoising, graying, binaryzation and tilt correction; text positioning, namely positioning a character area in the image for subsequent identification; character recognition, extracting features of single characters, classifying by a classifier, and performing post-optimization processing. However, although the conventional OCR method has high recognition efficiency, the generalization capability and robustness are poor, and the method is only suitable for images with simple backgrounds. The deep learning-based OCR technology integrates functions of image preprocessing, text positioning, character recognition and the like into a whole by utilizing a deep neural network, and can realize end-to-end pixel-level character marking and keyword character detection.
How to break through the defects of poor generalization capability and robustness of the traditional image identification method and higher dependence of deep learning on a large amount of labeled data and hardware conditions is also a key problem to be solved.
Disclosure of Invention
In view of the above, the present invention provides a product packaging bag coding defect detection method and system based on AI technology, which realizes product packaging identification and defect detection for various types of packaging such as plastic packaging, paper boxes and the like with various simple or complex backgrounds.
In order to achieve the purpose, the invention adopts the following technical scheme:
a product packaging bag coding defect detection method based on AI technology comprises the following steps:
step S1, obtaining the product packaging bag image and cutting;
step S2, according to the cut product packaging bag image, triggering image recognition based on soft triggering or hard triggering;
step S3, preprocessing the cut product packaging bag image;
step S4, forming a text box by adopting a DB algorithm based on the preprocessed image;
step S5, according to the obtained text box, character recognition is carried out based on the deep learning model CRNN to obtain the content result of character prediction and the corresponding probability value;
and step S6, acquiring the detection result of the defect target based on the content result of the character prediction and the corresponding probability value.
Further, the step S1 is specifically: the method comprises the steps that an industrial camera is used for collecting images of product packaging bags on a high-speed conveying belt, the images of the product packaging bags shot by the camera are cut, and after a cutting area is selected by shooting a first image, images in the cutting area are automatically reserved in the follow-up real-time collection.
Furthermore, the soft trigger adopts an interval time method, a similarity comparison method and an edge strength method to position the image needing to be detected and identified.
Further, the hard triggering automatically triggers subsequent character positioning identification and defect detection through a serial port output signal hard triggering method of a coder connected with the printing and packaging equipment and the PLC.
Further, the pretreatment specifically comprises: converting the read RGB image into a gray image, performing rapid noise reduction on the gray image, performing reverse color processing on the binary image after adaptive threshold algorithm processing, and finally performing morphological processing such as corrosion, expansion and the like.
Further, the DB algorithm specifically includes: the backbone network selects ResNet50_ vd, converts the output of the characteristic pyramid into the same size in an up-sampling mode, and generates characteristic and characteristic layers in a cascade mode; then, predicting a probability map and a text probability map through the feature layer for calculating the probability that the pixel belongs to the text to form a text probability map, and then forming a dynamic threshold map according to the dynamic threshold of each pixel; and generating a DB binary image through the text probability image and the dynamic threshold value image, and generating an expansion label according to the DB binary image to form a text box.
Further, the deep learning model CRNN specifically includes: a CRNN algorithm is adopted, a CNN + RNN network structure is adopted in a characteristic learning stage, and a CTC algorithm is selected during alignment; and the backbone network selects ResNet34_ vd, CNN extracts image pixel characteristics, RNN extracts image time sequence characteristics, CTC induces connection characteristics among characters, and the final output value is a text recognition prediction result and a corresponding prediction probability value.
Further, the step S6 is specifically:
(1) directly comparing the length of a character string between a standard character string input by a user and a predicted result character string aiming at the conditions of multi-printing and missing printing, wherein the condition that the standard character string length is smaller than the predicted result character string length is regarded as the missing printing, and otherwise, the condition that the standard character string length is larger than the predicted result character string length is regarded as the multi-printing;
(2) directly comparing whether a standard character string input by a user is the same as a character string of a prediction result or not according to the misprinting condition, and if a plurality of characters are different, determining that the misprinting condition is misprinted;
(3) aiming at the character defect condition, in order to reasonably judge whether defect characters appear or not, a statistical 3 sigma principle is adopted to judge whether the probability of the predicted characters is abnormal or not according to the probability, and the probability is smaller than
The character of (2) is regarded as an abnormal character;
(4) if there are no 3 kinds of errors, the defect detection result of the input picture is considered to be completely correct.
A product packaging bag coding defect detection system based on AI technology comprises an image automatic acquisition unit, a trigger recognition unit, an OCR character positioning recognition defect detection unit and a result output unit;
the automatic image acquisition unit automatically acquires the product packaging bag images on the high-speed conveyor belt in real time through the industrial camera, the packaging bag images can be automatically cut by utilizing the self-identification function of the system, and the cut images are transmitted to the trigger identification module;
the trigger recognition module positions the collected image by a soft triggering method of an interval time method, a similarity comparison method and an edge strength method to be detected and recognized, or automatically triggers a subsequent OCR character positioning recognition and defect detection module by a serial port output signal hard triggering method of a coder connected with printing and packaging equipment and a PLC;
the OCR character positioning and recognizing defect detecting unit carries out preprocessing, text positioning and OCR character recognition on a packaging bag image to be recognized by utilizing a deep neural network, and outputs a character recognition result and a confidence coefficient to a defect detecting module so as to support two modes of dynamic recognition and offline recognition; then, defect detection judges whether wrong printing, missing printing, multiple printing and fuzzy conditions exist according to an OCR result, and supports two modes of dynamic detection and off-line detection;
and the result output module outputs error category information to the detected error or defect frame or returns an Ng signal through a 232 serial port, and simultaneously, the normal frame image and the defect frame image are respectively and circularly stored.
Compared with the prior art, the invention has the following beneficial effects:
1. the invention realizes the product package identification and defect detection of various simple or complex backgrounds of various types of packages such as plastic packages, paper boxes and the like;
2. the system has high accuracy in identifying characters of the production date of the product packaging bag, high defect detection accuracy, adaptability to images with complex backgrounds and low contrast, strong scene adaptability, and high speed of completing one-time acquisition, identification and defect detection by integrating the functions of image real-time acquisition, real-time character positioning, real-time OCR (optical character recognition), real-time error and defect detection.
Drawings
FIG. 1 is a schematic diagram of the system architecture of the present invention;
FIG. 2 is a schematic diagram of a DB algorithm network in accordance with an embodiment of the present invention;
FIG. 3 is a schematic diagram of a CRNN algorithm network in accordance with an embodiment of the present invention;
FIG. 4 is an operation interface of the module for identifying the production date of the packaging bag on line in real time according to an embodiment of the present invention;
FIG. 5 is a parameter setting interface in accordance with an embodiment of the present invention;
FIG. 6 shows an embodiment of the present invention in which the parameter "Interval time" is a drop-down menu;
FIG. 7 shows a parameter "serial port setup" in an embodiment of the invention;
FIG. 8 is an off-line package production date identification interface in accordance with an embodiment of the present invention.
Detailed Description
The invention is further explained below with reference to the drawings and the embodiments.
Referring to fig. 1, the invention provides a product packaging bag coding defect detection system based on AI technology, which comprises an image automatic acquisition unit, a trigger recognition unit, an OCR character positioning recognition defect detection unit and a result output unit;
the automatic image acquisition unit automatically acquires the product packaging bag images on the high-speed conveyor belt in real time through an industrial camera, the packaging bag images can be automatically cut by utilizing the self-identification function of the system, and the cut images are transmitted to the trigger identification module;
the triggering recognition module positions the collected image by a soft triggering method of an interval time method, a similarity comparison method and an edge strength method to be detected and recognized, or automatically triggers a subsequent OCR character positioning recognition and defect detection module by a serial port output signal hard triggering method of a coder connected with printing and packaging equipment and a PLC;
the OCR character positioning and recognizing defect detecting unit carries out preprocessing, text positioning and OCR character recognition on a packaging bag image to be recognized by utilizing a deep neural network, and outputs a character recognition result and confidence to a defect detecting module so as to support two modes of dynamic recognition and offline recognition; then, defect detection judges whether wrong printing, missing printing, multiple printing and fuzzy conditions exist according to an OCR result, and supports two modes of dynamic detection and off-line detection;
and the result output module outputs error category information to the detected error or defect frame or returns an Ng signal through a 232 serial port, and simultaneously, the normal frame image and the defect frame image are respectively and circularly stored.
In this embodiment, the image acquisition module is composed of three parts, namely acquisition control, camera setting and image cropping. The hardware for image acquisition control consists of an industrial camera, an optical lens and a light source.
Preferably, in the embodiment, a direct-insert type integrated LED annular light source and a patch type integrated annular industrial light source are selected. The direct-insertion type integrated LED annular light source has the advantages of high illumination brightness and good directivity, and the patch type integrated annular industrial light source has the advantages of uniform illumination and surface reflection elimination.
Preferably, in this embodiment, the image of the product packaging bag captured by the camera is automatically cropped, and after the cropping area is selected by framing the first image, the image in the cropping area can be automatically retained by subsequent real-time acquisition. The cutting area can be an area where the character to be identified and detected is located and an acceptable printing area, the size of the picture is greatly reduced through cutting, the identification speed can be greatly improved, and meanwhile the efficiency of detecting whether the character is printed in a wrong area or not can be achieved.
In this embodiment, the trigger recognition module adopts two trigger recognition methods, namely soft trigger and hard trigger
The soft triggering method is designed for adapting to various complex working conditions based on the machine vision technology, is suitable for static and dynamic recognition modes, and does not need an external sensor. The soft triggering realizes three main methods of interval time shooting, an edge intensity algorithm and a similarity neural network comparison algorithm.
When the product packaging bags are conveyed on a uniform-speed conveyor belt, an interval time shooting method is used for judging whether to trigger character recognition. When the product packaging bags are transported on the uniform-speed conveyor belt, the camera can continuously shoot a plurality of pictures, wherein a plurality of pictures are irrelevant to the identification purpose, the speed of the uniform-speed conveyor belt is fixed, and the placing positions of the product packaging bags and the camera are fixed distances, so that the time of shooting the products by the camera each time is also fixed, and the pictures to be identified can be positioned by using the interval time method.
When the product packaging bags are conveyed on a uniform or non-uniform conveyor belt, an edge strength method is provided for judging whether to trigger character recognition. When a group of pictures are continuously shot by the camera, the difference value between the picture to be detected and the previous m pictures and the next m pictures is calculated, an edge strength curve can be drawn, and the picture corresponding to the highest point of the curve is the picture of the product packaging bag to be identified.
When the product packaging bags are conveyed on the non-uniform speed conveyor belt, an image similarity comparison algorithm can be used for judging whether to trigger character recognition. For the same product packaging bag, in an actual situation, the first picture is a standard and defect-free picture which can be determined and needs to be identified, similarity comparison is carried out on the continuously shot picture and the first picture by utilizing a neural network algorithm, and the picture which needs to be identified later is the picture which needs to be identified later when the similarity with the first picture is greater than a certain threshold value.
In addition, in the dynamic recognition mode, a hard triggering method can be adopted to judge whether the character recognition is triggered. Wherein the hard trigger receives signals sent by a sensor or an encoder connected to the packaging machine through an RS232 serial port to trigger recognition and detection, and the rejection signals can be transmitted to a lower computer or a PLC through a 232 serial port line.
The USB interface of the computer is externally connected with a USB-to-RS 232 serial port line, and the rejection signal can be transmitted to a lower computer or a PLC through the 232 serial port line.
In this embodiment, the OCR character positioning and recognizing defect detecting portion is composed of four portions, namely, image preprocessing, character positioning detection, character recognition and defect detection, and specifically includes the following steps:
image preprocessing: firstly, converting a read RGB image into a gray image, performing rapid noise reduction processing on the gray image, performing inverse color processing on a binary image after adaptive threshold algorithm processing, and finally performing morphological processing such as corrosion and expansion, wherein a convolution kernel selects 5 x 5, and an adaptive threshold algorithm selects a Gaussian function.
Character positioning detection: a DB algorithm (differential Binarization) that can quickly give a better framing effect is used. As shown in fig. 2, the backbone network selects ResNet50_ vd, converts the output of the feature pyramid into the same size by an upsampling method, and generates a feature layer and a feature layer in a cascade manner; and then, a text probability map is formed by calculating the probability that the pixel belongs to the text through the feature layer prediction probability map and the text probability map, and then a dynamic threshold map is formed according to the dynamic threshold of each pixel, wherein the minimum probability threshold is set to be 0.7 in the character detection stage of the system, namely, the fixed frame result with the algorithm output value less than 0.7 is considered to be an interference result possibly and is considered to be an invalid fixed frame. And finally, generating a DB binary image through the text probability image and the dynamic threshold value image, and generating an expansion label according to the DB binary image to form a text box. Moreover, a large number of tests show that the framing result of the network is directly adopted, so that the framing result of the edge part is possibly conservative, the subsequent character recognition algorithm is influenced, and the most edge character is not easy to be correctly recognized.
Character recognition: the method adopts an end-to-end deep learning model CRNN to carry out character recognition, does not need to segment character strings, but converts character recognition into a sequence learning problem, although the input images have different scales and different text lengths, the text recognition can carry out character cutting and recognition on the whole text image after passing through a CNN and an RNN.
As shown in fig. 3, the present invention adopts a CRNN algorithm (Convolutional Recurrent Neural Network), and the feature learning stage adopts a Network structure of CNN + RNN, and selects a CTC algorithm during alignment, which is mainly used for identifying a text sequence of an indefinite length end to end. ResNet34_ vd is selected as a backbone network, the input value of the network is a fixed frame region obtained by a 2.1-node character detection algorithm, image pixel characteristics are extracted by a CNN, image time sequence characteristics are extracted by an RNN, and CTC induces connection characteristics among characters, the final output value is a prediction result of text recognition and a corresponding prediction probability value, wherein a minimum probability threshold value of 0.7 is set in the character recognition stage of the system, namely the recognition result with the algorithm output value less than 0.7 is considered to be possibly an interference factor and is considered to be an invalid result.
And (3) defect detection: the CRNN algorithm in the character recognition stage can obtain the content result predicted by the character and the corresponding probability value. To achieve the detection of defect targets, the system specifies the following defect detection criteria:
1. directly comparing the length of a character string between a standard character string input by a user and a predicted result character string aiming at the conditions of multi-printing and missing printing, wherein the condition that the standard character string length is smaller than the predicted result character string length is regarded as the missing printing, and otherwise, the condition that the standard character string length is larger than the predicted result character string length is regarded as the multi-printing;
2. directly comparing whether a standard character string input by a user is the same as a character string of a prediction result or not according to the misprinting condition, and if a plurality of characters are different, determining that the misprinting condition is misprinted;
3. aiming at the character defect condition, in order to reasonably judge whether defect characters appear or not, a statistical 3 sigma principle is adopted to judge whether the probability of the predicted characters is abnormal or not according to the probability, and the probability is smaller than
The character of (2) is regarded as an abnormal character;
4. if there are no 3 kinds of errors, the defect detection result of the input picture is considered to be completely correct.
Example 1:
in this embodiment, the system is divided into two modules, namely, an online real-time identification module for the production date of the packaging bag and an offline identification module for the production date of the packaging bag.
(1) On-line real-time identification packaging bag production date module
The operation interface of the module for identifying the production date of the packaging bag in real time on line is shown in figure 4.
The online real-time identification interface parameter information is shown in table 3.
TABLE 3 Online real-time identification of interface parameter information
| Principal parameters | Description of the invention |
| Degree of exposure | Has been set to 1.4271ms |
| Reference characters | Inputting correct date of manufacture characters of product packaging bags |
| Resolution ratio | Has been set at 640 x 480, ranging from 64 x 48 to 1280 x 960, and has moved to the right by 0.1 times |
| Storage path | The storage positions of the pictures and the recognition results have default paths and can be reset |
| Interval of time | How long a picture is taken at intervals, default setting is 5 milliseconds/frame |
| Label inspection | Inputting reference characters, and judging whether character defects exist |
| Is provided with | Setting parameter information: m value of screenshot interval and edge intensity method |
| Manual screen shot | Take the whole picture |
| Automatic cutting | Cutting out character area of picture production date |
| Opening recognition, starting screenshot | Recognizing the cut picture |
| Serial port arrangement | After the equipment is connected with the serial port, the corresponding setting can be carried out |
| Delayed frame number setting | Interval several-packet identification |
| Setting standard pictures | Setting a first correct picture when comparing the similarity |
| Identification lattice | When the character is a dot matrix character, it is turned on |
In table 3, when the parameter "storage path" runs the file for the first time, two subfolders "right" and "error" are automatically created under the storage path, the "right" folder stores the correct picture, the "error" folder stores the defective picture, if the label check is performed, the correct picture is stored in the two folders, and if the label check is performed, the default path is "C:/picture".
The parameter "tag check" is a drop down menu, also containing a "no tag check" mode. In the label inspection mode, correct production date needs to be input under reference characters, so that whether the production date characters on the product packaging bags have defects or not is judged. In the mode of 'no-label inspection', the production date does not need to be input under 'reference characters', and the picture is directly identified.
The parameter "set" can set the screenshot interval time multiple (base number 1 ms) and set the interface as shown in fig. 5 using the value of parameter M in the edge intensity method.
The parameter "Interval" is a drop-down menu, with multiple modes selectable, as shown in FIG. 6.
The "time-between" mode indicates how long a picture is taken (only for the constant speed conveyor case).
The similarity comparison mode is that when the product packaging bags are conveyed on the conveyor belt, the neural network can be used for measuring the similarity degree of two pictures, the picture with the highest similarity degree with the first product packaging bag picture is the product packaging bag picture needing to be identified, and the condition is suitable for the conveyor belt with non-uniform speed.
The 'edge strength' mode is that when a camera continuously shoots a plurality of pictures, which picture needs to be judged for identification, so that the difference value between the current picture to be detected and the previous m pictures and the next m pictures can be drawn to draw an edge strength curve, and the highest point of the curve is the picture needing to be identified. The three modes are soft trigger modes.
The serial port control mode is a hard trigger mode, in which a control function can be realized through a serial port, namely, an upper computer sends a command with a specific protocol format to a controller through the serial port to further control peripheral equipment or devices.
The parameter "serial port setting" can be set after the device is connected with a serial port, and the serial port information setting interface is shown in fig. 7, wherein the parameters "serial port number", "baud rate", "delay time (base number millisecond)", and "password" keep default value setting when the user does not input.
The operation process of the online real-time identification packaging bag production date interface comprises the following steps:
before starting the application software of 'test 9_24. exe', the USB socket of the computer needs to be connected with the camera, and the light source is turned on. And then starting the 'test 9_24. exe' application program software, displaying an online real-time packaging bag production date identification interface shown in fig. 4, displaying a picture of a product packaging bag captured by a camera in real time on the interface, and adjusting the camera and the light source until the picture is clear and the illumination is uniform, and the production date characters on the product packaging bag are in the horizontal position of the picture, wherein the identification effect is the best.
The mode of 'interval time', 'similarity comparison' and the like can be selected by pulling down the menu bar at 'interval time'.
And then clicking the manual screenshot, clicking the automatic cutting, selecting a picture in the popped folder dialog box, cutting a production date character area on the picture, popping the cut picture after cutting, and then closing the two picture dialog boxes.
Then, if the "tag check" mode is selected, the production date is input in the "reference character", the "confirm" is clicked after the confirmation is correct, the "browse" is clicked in the "storage path", and the storage position of the picture and the identification result is selected.
Clicking 'start identification' and then clicking 'start screenshot', the system automatically identifies the production date of the packaging bag, displays the identification result on the interface, simultaneously stores the identification result in a corresponding folder path, and clicks 'stop screenshot' to stop identification. If the code character needing to be identified is a dot matrix character, clicking the identification dot matrix after clicking the start identification, and then clicking the start screenshot.
If the mode of 'no-label inspection' is selected, the date does not need to be input in the 'reference characters', the 'opening identification' is directly clicked, then the 'starting screenshot' is clicked, the system automatically identifies the production date of the packaging bag, the identification result is displayed on the interface, meanwhile, the identification result is stored in the corresponding folder path, and the 'stopping screenshot' is clicked, so that the identification is stopped.
(2) Off-line identification packaging bag production date module
The offline identification package production date interface is shown in fig. 8. The interface parameter "label check" is a pull-down menu, and comprises a "label check" mode and a "no label check" mode, and the "label check" mode can check whether the character has defects. The "no tag test" directly recognizes the character. The "picture path" selects the position where the picture to be identified is located. The "reference text" sets the correct date of production.
The operation flow of the offline identification packaging bag production date interface is as follows:
clicking the upper left corner to identify, displaying the off-line identification interface shown in fig. 8, clicking the picture path column to browse, and selecting a certain picture or a certain picture folder.
Then, in the column of 'identification method', if the 'label check' mode is selected, the production date is input in the 'reference character', the 'determination' is clicked after the confirmation is correct, if a certain picture is selected, the 'start identification' is clicked, the system identifies the production date of the packaging bag, and the identification result is displayed on the interface. If a certain picture folder is selected, "automatic identification" is clicked.
If the mode of 'no-label inspection' is selected, a certain picture or a certain picture folder can be selected without inputting the date in the 'reference characters', if a certain picture is selected, the 'start identification' is clicked, the system identifies the production date of the packaging bag, and the identification result is displayed on the interface. If a certain picture folder is selected, "automatic identification" is clicked.
The above description is only a preferred embodiment of the present invention, and all equivalent changes and modifications made in accordance with the claims of the present invention should be covered by the present invention.
Claims (9)
1. A product packaging bag coding defect detection method based on AI technology is characterized by comprising the following steps:
step S1, obtaining the product packaging bag image and cutting;
step S2, according to the cut product packaging bag image, triggering image recognition based on soft triggering or hard triggering;
step S3, preprocessing the cut product packaging bag image;
step S4, forming a text box by adopting a DB algorithm based on the preprocessed image;
step S5, according to the obtained text box, character recognition is carried out based on the deep learning model CRNN to obtain the content result of character prediction and the corresponding probability value;
and step S6, acquiring the detection result of the defect target based on the content result of the character prediction and the corresponding probability value.
2. The AI-technology-based product packaging bag coding defect detection method according to claim 1, wherein the step S1 specifically comprises: the method comprises the steps that an industrial camera is used for collecting images of product packaging bags on a high-speed conveying belt, the images of the product packaging bags shot by the camera are cut, and after a cutting area is selected by shooting a first image, images in the cutting area are automatically reserved in the follow-up real-time collection.
3. The AI-technology-based product packaging bag coding defect detection method of claim 1, wherein the soft trigger employs an interval time method, a similarity comparison method and an edge strength method to locate the identification image to be detected.
4. The AI-technology-based product packaging bag coding defect detection method of claim 1, wherein the hard triggering automatically triggers subsequent character location recognition and defect detection by connecting an encoder on the printing and packaging equipment and a serial port output signal hard triggering method of the PLC.
5. The AI-technology-based product packaging bag coding defect detection method of claim 1, wherein the pre-processing specifically comprises: converting the read RGB image into a gray image, performing rapid noise reduction on the gray image, performing reverse color processing on the binary image after adaptive threshold algorithm processing, and finally performing morphological processing such as corrosion, expansion and the like.
6. The AI-technology-based product packaging bag coding defect detection method of claim 1, wherein the DB algorithm specifically comprises: the backbone network selects ResNet50_ vd, converts the output of the characteristic pyramid into the same size in an up-sampling mode, and generates characteristic and characteristic layers in a cascade mode; then, predicting a probability map and a text probability map through the feature layer for calculating the probability that the pixel belongs to the text to form a text probability map, and then forming a dynamic threshold map according to the dynamic threshold of each pixel; and generating a DB binary image through the text probability image and the dynamic threshold value image, and generating an expansion label according to the DB binary image to form a text box.
7. The AI-technology-based product packaging bag coding defect detection method of claim 1, wherein the deep learning model CRNN specifically is: a CRNN algorithm is adopted, a CNN + RNN network structure is adopted in a characteristic learning stage, and a CTC algorithm is selected during alignment; and the backbone network selects ResNet34_ vd, CNN extracts image pixel characteristics, RNN extracts image time sequence characteristics, CTC induces connection characteristics among characters, and the final output value is a text recognition prediction result and a corresponding prediction probability value.
8. The AI-technology-based product packaging bag coding defect detection method according to claim 1, wherein the step S6 specifically comprises:
(1) directly comparing the length of a character string between a standard character string input by a user and a predicted result character string aiming at the conditions of multi-printing and missing printing, wherein the condition that the standard character string length is smaller than the predicted result character string length is regarded as the missing printing, and otherwise, the condition that the standard character string length is larger than the predicted result character string length is regarded as the multi-printing;
(2) directly comparing whether a standard character string input by a user is the same as a character string of a prediction result or not according to the misprinting condition, and if a plurality of characters are different, determining that the misprinting condition is misprinted;
(3) aiming at the character defect condition, in order to reasonably judge whether defect characters appear or not, a statistical 3 sigma principle is adopted to judge whether the probability of the predicted characters is abnormal or not according to the probability, and the probability is smaller than
The character of (2) is regarded as an abnormal character;
(4) if there are no 3 kinds of errors, the defect detection result of the input picture is considered to be completely correct.
9. A product packaging bag coding defect detection system based on AI technology is characterized by comprising an image automatic acquisition unit, a trigger recognition unit, an OCR character positioning recognition defect detection unit and a result output unit;
the automatic image acquisition unit automatically acquires the product packaging bag images on the high-speed conveyor belt in real time through the industrial camera, the packaging bag images can be automatically cut by utilizing the self-identification function of the system, and the cut images are transmitted to the trigger identification module;
the trigger recognition module positions the collected image by a soft triggering method of an interval time method, a similarity comparison method and an edge strength method to be detected and recognized, or automatically triggers a subsequent OCR character positioning recognition and defect detection module by a serial port output signal hard triggering method of a coder connected with printing and packaging equipment and a PLC;
the OCR character positioning and recognizing defect detecting unit carries out preprocessing, text positioning and OCR character recognition on a packaging bag image to be recognized by utilizing a deep neural network, and outputs a character recognition result and a confidence coefficient to a defect detecting module so as to support two modes of dynamic recognition and offline recognition; then, defect detection judges whether wrong printing, missing printing, multiple printing and fuzzy conditions exist according to an OCR result, and supports two modes of dynamic detection and off-line detection;
and the result output module outputs error category information to the detected error or defect frame or returns an Ng signal through a 232 serial port, and simultaneously, the normal frame image and the defect frame image are respectively and circularly stored.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111589616.3A CN114266751A (en) | 2021-12-23 | 2021-12-23 | AI technology-based product packaging bag coding defect detection method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111589616.3A CN114266751A (en) | 2021-12-23 | 2021-12-23 | AI technology-based product packaging bag coding defect detection method and system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114266751A true CN114266751A (en) | 2022-04-01 |
Family
ID=80829204
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111589616.3A Pending CN114266751A (en) | 2021-12-23 | 2021-12-23 | AI technology-based product packaging bag coding defect detection method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114266751A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114689512A (en) * | 2022-04-15 | 2022-07-01 | 中科芯集成电路有限公司 | Mask detection method integrating machine vision and deep learning |
| CN116433623A (en) * | 2023-03-31 | 2023-07-14 | 杭州数创自动化控制技术有限公司 | Defect position marking and identifying method, system, equipment and medium |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112184648A (en) * | 2020-09-22 | 2021-01-05 | 苏州中科全象智能科技有限公司 | Piston surface defect detection method and system based on deep learning |
| CN112949455A (en) * | 2021-02-26 | 2021-06-11 | 武汉天喻信息产业股份有限公司 | Value-added tax invoice identification system and method |
| CN113242223A (en) * | 2021-04-30 | 2021-08-10 | 刘厚泽 | Website detection method and device |
| CN113505789A (en) * | 2021-07-09 | 2021-10-15 | 南京荣新智能科技有限公司 | Electrical equipment nameplate text recognition method based on CRNN |
| CN113537227A (en) * | 2021-06-28 | 2021-10-22 | 杭州电子科技大学 | Structured text recognition method and system |
| CN113591866A (en) * | 2021-07-29 | 2021-11-02 | 云南大学 | Special job certificate detection method and system based on DB and CRNN |
-
2021
- 2021-12-23 CN CN202111589616.3A patent/CN114266751A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112184648A (en) * | 2020-09-22 | 2021-01-05 | 苏州中科全象智能科技有限公司 | Piston surface defect detection method and system based on deep learning |
| CN112949455A (en) * | 2021-02-26 | 2021-06-11 | 武汉天喻信息产业股份有限公司 | Value-added tax invoice identification system and method |
| CN113242223A (en) * | 2021-04-30 | 2021-08-10 | 刘厚泽 | Website detection method and device |
| CN113537227A (en) * | 2021-06-28 | 2021-10-22 | 杭州电子科技大学 | Structured text recognition method and system |
| CN113505789A (en) * | 2021-07-09 | 2021-10-15 | 南京荣新智能科技有限公司 | Electrical equipment nameplate text recognition method based on CRNN |
| CN113591866A (en) * | 2021-07-29 | 2021-11-02 | 云南大学 | Special job certificate detection method and system based on DB and CRNN |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114689512A (en) * | 2022-04-15 | 2022-07-01 | 中科芯集成电路有限公司 | Mask detection method integrating machine vision and deep learning |
| CN116433623A (en) * | 2023-03-31 | 2023-07-14 | 杭州数创自动化控制技术有限公司 | Defect position marking and identifying method, system, equipment and medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106875373B (en) | Mobile phone screen MURA defect detection method based on convolutional neural network pruning algorithm | |
| US11830174B2 (en) | Defect inspecting device, defect inspecting method, and storage medium | |
| CN109550712A (en) | A kind of chemical fiber wire tailfiber open defect detection system and method | |
| CN114266751A (en) | AI technology-based product packaging bag coding defect detection method and system | |
| CN107966454A (en) | A kind of end plug defect detecting device and detection method based on FPGA | |
| CN113221889B (en) | Chip character anti-interference recognition method and device | |
| CN110245663A (en) | One kind knowing method for distinguishing for coil of strip information | |
| CN112085024A (en) | Tank surface character recognition method | |
| CN114155527A (en) | Scene text recognition method and device | |
| CN111861990B (en) | Method, system and storage medium for detecting bad appearance of product | |
| CN113111903A (en) | Intelligent production line monitoring system and monitoring method | |
| CN114445707A (en) | Intelligent visual fine detection method for defects of bottled water labels | |
| CN116228741A (en) | PCBA (printed circuit board assembly) component defect detection method and device | |
| CN111144151A (en) | High-speed dynamic bar code real-time detection method based on image recognition | |
| CN114972246A (en) | Die-cutting product surface defect detection method based on deep learning | |
| CN116758545A (en) | Paper medicine packaging steel seal character recognition method based on deep learning | |
| CN112884741B (en) | Printing apparent defect detection method based on image similarity comparison | |
| CN110570398A (en) | Cable joint welding spot qualification detection method based on deep learning technology | |
| CN111062379A (en) | Identification error-proofing recognition method, device, storage medium and system | |
| CN117214178A (en) | Intelligent identification method for appearance defects of package on packaging production line | |
| CN117152731A (en) | Method and system for identifying key information of label of warehouse-in material box of cigarette factory | |
| CN116994049A (en) | Full-automatic flat knitting machine and method thereof | |
| CN114581911B (en) | Steel coil label identification method and system | |
| CN112001396B (en) | Bearing surface deformation and character mixed defect image detection method | |
| CN112508935A (en) | Product packaging detection method and system based on deep learning and product packaging sorting system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |