CN116883697A - Device code detection method, device, system and readable storage medium - Google Patents
Device code detection method, device, system and readable storage medium Download PDFInfo
- Publication number
- CN116883697A CN116883697A CN202310897864.7A CN202310897864A CN116883697A CN 116883697 A CN116883697 A CN 116883697A CN 202310897864 A CN202310897864 A CN 202310897864A CN 116883697 A CN116883697 A CN 116883697A
- Authority
- CN
- China
- Prior art keywords
- code
- equipment
- information
- image
- device code
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 72
- 238000003860 storage Methods 0.000 title claims abstract description 23
- 238000000034 method Methods 0.000 claims abstract description 28
- 238000007781 pre-processing Methods 0.000 claims description 20
- 238000013528 artificial neural network Methods 0.000 claims description 12
- 239000011521 glass Substances 0.000 claims description 7
- 230000033001 locomotion Effects 0.000 claims description 6
- 238000005520 cutting process Methods 0.000 claims description 5
- 238000011835 investigation Methods 0.000 abstract description 6
- 238000004519 manufacturing process Methods 0.000 description 21
- 230000007547 defect Effects 0.000 description 20
- 238000010586 diagram Methods 0.000 description 9
- 230000002085 persistent effect Effects 0.000 description 6
- 238000012549 training Methods 0.000 description 5
- 238000012795 verification Methods 0.000 description 5
- 230000000007 visual effect Effects 0.000 description 5
- 238000004422 calculation algorithm Methods 0.000 description 4
- 230000003287 optical effect Effects 0.000 description 4
- 230000008569 process Effects 0.000 description 4
- 238000004458 analytical method Methods 0.000 description 3
- 238000004891 communication Methods 0.000 description 3
- 238000012015 optical character recognition Methods 0.000 description 3
- 238000012545 processing Methods 0.000 description 3
- 230000002829 reductive effect Effects 0.000 description 3
- 239000004065 semiconductor Substances 0.000 description 3
- 230000000295 complement effect Effects 0.000 description 2
- 238000012937 correction Methods 0.000 description 2
- 238000013135 deep learning Methods 0.000 description 2
- 238000013136 deep learning model Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 229910044991 metal oxide Inorganic materials 0.000 description 2
- 150000004706 metal oxides Chemical class 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000007639 printing Methods 0.000 description 2
- 230000003068 static effect Effects 0.000 description 2
- 230000001960 triggered effect Effects 0.000 description 2
- 238000013459 approach Methods 0.000 description 1
- 238000003491 array Methods 0.000 description 1
- 230000008901 benefit Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000004590 computer program Methods 0.000 description 1
- 238000013527 convolutional neural network Methods 0.000 description 1
- 230000007797 corrosion Effects 0.000 description 1
- 238000005260 corrosion Methods 0.000 description 1
- 230000003247 decreasing effect Effects 0.000 description 1
- 230000002950 deficient Effects 0.000 description 1
- 239000002355 dual-layer Substances 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000009776 industrial production Methods 0.000 description 1
- 230000000670 limiting effect Effects 0.000 description 1
- 230000000873 masking effect Effects 0.000 description 1
- 239000011159 matrix material Substances 0.000 description 1
- 230000036961 partial effect Effects 0.000 description 1
- 230000002441 reversible effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06K—GRAPHICAL DATA READING; PRESENTATION OF DATA; RECORD CARRIERS; HANDLING RECORD CARRIERS
- G06K17/00—Methods or arrangements for effecting co-operative working between equipments covered by two or more of main groups G06K1/00 - G06K15/00, e.g. automatic card files incorporating conveying and reading operations
- G06K17/0022—Methods or arrangements for effecting co-operative working between equipments covered by two or more of main groups G06K1/00 - G06K15/00, e.g. automatic card files incorporating conveying and reading operations arrangements or provisious for transferring data to distant stations, e.g. from a sensing device
- G06K17/0025—Methods or arrangements for effecting co-operative working between equipments covered by two or more of main groups G06K1/00 - G06K15/00, e.g. automatic card files incorporating conveying and reading operations arrangements or provisious for transferring data to distant stations, e.g. from a sensing device the arrangement consisting of a wireless interrogation device in combination with a device for optically marking the record carrier
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06K—GRAPHICAL DATA READING; PRESENTATION OF DATA; RECORD CARRIERS; HANDLING RECORD CARRIERS
- G06K7/00—Methods or arrangements for sensing record carriers, e.g. for reading patterns
- G06K7/10—Methods or arrangements for sensing record carriers, e.g. for reading patterns by electromagnetic radiation, e.g. optical sensing; by corpuscular radiation
- G06K7/14—Methods or arrangements for sensing record carriers, e.g. for reading patterns by electromagnetic radiation, e.g. optical sensing; by corpuscular radiation using light without selection of wavelength, e.g. sensing reflected white light
- G06K7/1404—Methods for optical code recognition
- G06K7/1408—Methods for optical code recognition the method being specifically adapted for the type of code
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/24—Aligning, centring, orientation detection or correction of the image
- G06V10/242—Aligning, centring, orientation detection or correction of the image by image rotation, e.g. by 90 degrees
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/30—Noise filtering
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
- G06V30/10—Character recognition
- G06V30/19—Recognition using electronic means
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Computing Systems (AREA)
- Software Systems (AREA)
- Medical Informatics (AREA)
- Databases & Information Systems (AREA)
- Computer Networks & Wireless Communication (AREA)
- General Engineering & Computer Science (AREA)
- Electromagnetism (AREA)
- Toxicology (AREA)
- Image Analysis (AREA)
Abstract
The present disclosure provides a device code detection method, apparatus, device, system, and readable storage medium, the device code detection method including: acquiring at least two target images, wherein the at least two target images are obtained by respectively shooting at least two equipment code areas on a target product through at least two cameras; respectively identifying the equipment code areas in the at least two target images to obtain equipment information in each equipment code area; matching the device information in the at least two device code areas; and when the equipment information in the at least two equipment code areas is not matched, determining the target product as a problem product. The method and the device can reduce the investigation difficulty and improve the detection efficiency.
Description
Technical Field
The present disclosure relates to the field of industrial automation, and in particular, to a device code detection method, a device code detection apparatus, a device code detection device, a device code detection system, and a computer readable storage medium.
Background
Automated equipment on industrial production lines often needs to scan bar codes attached to processed products to bind current process and product serial numbers, and then apply this information to various production systems, databases, and the like. The quality of the bar code itself, the accuracy of the bar code scanning is therefore of paramount importance.
The code scanning gun is a device widely applied in the field of industrial automatic production. The code scanning gun is a device which generally emits light beams through a laser diode, captures light reflected by a bar code, converts the light into an electric signal through a photoelectric converter and then converts the electric signal into bar code information through decoding software. After the bar code information is obtained by the bar code scanning gun, the bar code information is sent to the automation equipment, so that the automation equipment knows the current specific product number.
The code scanning gun has the advantage of low cost, but has two major problems. Firstly, in the production process, the bar code pasting range is larger, the effective visual field of the bar code scanning gun is smaller, and the condition that the bar code exceeds the visual field of the bar code scanning gun to cause the bar code scanning failure often occurs. Secondly, when the code scanning problem occurs, for example, the barcode identification is unsuccessful or the barcode identification is successful but the barcode identification information has a problem, the problem cannot be traced.
Disclosure of Invention
The disclosure provides a device code detection method, a detection device, a system and a computer readable storage medium, which can reduce the investigation difficulty and improve the detection efficiency.
In a first aspect, the present disclosure provides a device code detection method, including:
Acquiring at least two target images, wherein the at least two target images are obtained by respectively shooting at least two equipment code areas on a target product through at least two cameras;
respectively identifying the equipment code areas in the at least two target images to obtain equipment information in each equipment code area;
matching the device information in the at least two device code areas;
and when the equipment information in the at least two equipment code areas is not matched, determining the target product as a problem product.
Optionally, the device code area includes a device code and text;
the method further comprises the steps of:
identifying the equipment codes and the characters in each equipment code area to obtain equipment code information and character information;
matching the equipment code information and the text information of each equipment code area;
and when the equipment code information and the text information in one equipment code area are not matched, determining the target product as a problem product.
Optionally, the method further comprises:
when the equipment code information and the text information in each equipment code area are matched, the equipment code information and the text information are confirmed
And when the device information in the at least two device code areas is matched, determining the target product as a normal product.
Optionally, the identifying the device code regions in the at least two target images respectively, to obtain device information in each device code region, includes:
cutting each target image to obtain a device code image and a text image in the target image;
and respectively identifying the equipment code image and the text image corresponding to each target image to obtain the equipment code information and the text information.
Optionally, the method further comprises:
performing first image preprocessing on the equipment code image;
and performing second image preprocessing on the text image.
Optionally, the first image preprocessing includes at least one of:
adjusting at least one of brightness, contrast and sharpness of the device code image;
performing a shrinking operation on the device code image;
denoising the equipment code image;
and rotating the equipment code image by a preset angle.
Optionally, the identifying the device code regions in the at least two target images respectively, to obtain device information in each device code region, further includes:
When the device code information cannot be identified from the device code image, performing a first operation, the first operation including: rotating the equipment code image by the preset angle, and identifying the rotated equipment code image again;
and when the equipment code information cannot be identified from the equipment code image after the rotation, repeatedly executing the first operation until the equipment code information can be identified from the equipment code image after the last rotation.
Optionally, the method further comprises:
and determining the position of the equipment code region from the at least two target images according to the first deep neural network.
Optionally, the target product is a photovoltaic module, and the at least two device code areas include at least two of three device code areas respectively located on a glass surface of the photovoltaic module, a frame of the photovoltaic module, and a back plate of the photovoltaic module.
Optionally, the camera is a fly-by camera or a line scan camera, and the camera is in motion when shooting at least two device code areas located on a target product.
In a second aspect, the present disclosure provides a device code detection apparatus, including:
the acquisition module is used for acquiring at least two target images, wherein the at least two target images are obtained by respectively shooting at least two equipment code areas on a target product through at least two cameras;
the identification module is used for respectively identifying the equipment code areas in the at least two target images to obtain equipment information in each equipment code area;
the matching module is used for matching the equipment information in the at least two equipment code areas;
and the determining module is used for determining the target product as a problem product when the equipment information in the at least two equipment code areas are unmatched.
In a third aspect, the present disclosure provides an apparatus code detection apparatus, comprising: a memory and a processor, the memory having executable code stored thereon, which when processed by the processor, causes the processor to perform any of the device code detection methods described herein.
In a fourth aspect, the present disclosure provides a device code detection system, comprising:
at least two cameras for shooting at least two equipment code areas on the target product to obtain at least two target images;
The apparatus code detection apparatus of the third aspect, configured to determine whether the target product is a problem product based on the at least two target images.
In a fifth aspect, the present disclosure provides a computer-readable storage medium having executable code stored thereon, which when executed by a processor, causes the processor to perform any one of the device code detection methods.
In the embodiment of the disclosure, at least two target images are obtained by shooting at least two equipment code areas on a target product by using a camera, and each target image is used for identifying the equipment code area in a subsequent image, so that the situation that the equipment code cannot be identified due to the fact that the equipment code exceeds the field of view of the camera due to the fact that the field of view of the camera is large; in addition, the image is adopted to identify the equipment information in the equipment code area, so that if the target product is determined to be the problem product, the problem of the equipment code area can be traced through the image conveniently, the follow-up targeted correction according to the problem obtained by image investigation is facilitated, and the problem investigation difficulty is reduced.
Drawings
Fig. 1 is a flow diagram of one embodiment of a device code detection method of the present disclosure.
Fig. 2 is a top view of one embodiment of a usage scenario of a device code detection method in the present disclosure.
Fig. 3 is a flow diagram of one embodiment of a device code detection method in the present disclosure.
Fig. 4 is a block diagram of an embodiment of a device code detection apparatus of the present disclosure.
Fig. 5 is a block diagram of an embodiment of a device code detection device of the present disclosure.
Fig. 6 is a block diagram of one embodiment of a device code detection system of the present disclosure.
Detailed Description
Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While embodiments of the present disclosure are illustrated in the drawings, it should be understood that the present disclosure may be embodied in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.
The terminology used in the present disclosure is for the purpose of describing particular embodiments only and is not intended to be limiting of the disclosure. As used in this disclosure and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should also be understood that the term "and/or" as used herein refers to and encompasses any or all possible combinations of one or more of the associated listed items.
It should be understood that although the terms "first," "second," "third," etc. may be used in this disclosure to describe various information, these information should not be limited to these terms. These terms are only used to distinguish one type of information from another. For example, first information may also be referred to as second information, and similarly, second information may also be referred to as first information, without departing from the scope of the present disclosure. Thus, a feature defining "a first" or "a second" may explicitly or implicitly include one or more such feature. In the description of the present disclosure, the meaning of "a plurality" is two or more, unless explicitly defined otherwise.
As shown in fig. 1, fig. 1 is a flow chart illustrating an embodiment of a device code detection method of the present disclosure. The device code detection method 100 includes:
step S101, at least two target images are acquired, wherein the at least two target images are obtained by respectively shooting at least two equipment code areas on a target product through at least two cameras.
In a production line, equipment codes are often pasted or printed on a product in different working procedures of the product, and are used for tracing and tracking management of each working procedure of the product. In the disclosed embodiments, the target product may be any target on a production line. For example, the target product is a photovoltaic module on a production line. On a production line of photovoltaic modules, an equipment code for identifying a product serial number or a product serial number of each photovoltaic module is typically adhered or printed on each photovoltaic module. According to different procedures, each photovoltaic module can be provided with two or more equipment codes, and the two or more equipment codes are respectively pasted or printed in two or more equipment code areas. Optionally, each of at least two device code regions on the target product includes a device code and/or text.
In order to identify the equipment codes in the equipment code areas, cameras are adopted to respectively shoot at least two equipment code areas on a target product in the method and the device code areas, so that at least two target images are obtained.
Step S102, respectively identifying the equipment code areas in the at least two target images, and obtaining the equipment information in each equipment code area.
Alternatively, the device code in the device code area may be a bar code, a two-dimensional code, other high-dimensional optical code (e.g., two-dimensional code matrix, stack code, etc.) on the target product, or other type of code capable of identifying the device information of the product, without limitation. After the device code area is acquired, the device code can be identified according to the type of the device code (such as a bar code or a two-dimensional code) so as to acquire corresponding device information.
The device information may be an ID of the target product (e.g., a product serial number), or category information of the target product (e.g., a product serial number), or a process serial number of one of the processes of the target product on the production line, etc., which is not limited herein.
And step S103, matching the device information in the at least two device code areas.
And step S104, when the equipment information in the at least two equipment code areas is unmatched, determining the target product as a problem product.
There are various ways of matching device information in each device code area. For example, in some examples, the device information in each device code region is directly compared for agreement, if so, then the device information is matched, and if there is an inconsistency, then there is a mismatch in the device information in each device code region. For another example, the device information in each device code area may confirm the correspondence through a preset rule or a table in a preset database. When the device information in each device code area is matched, specifically checking whether the device information in each device code area has a corresponding relation through the preset rule or confirming whether the device information in each device code area has a corresponding relation by inquiring a table in a preset database. Matching is performed if the corresponding relation exists, and not matching is performed if the corresponding relation does not exist.
In the embodiment of the disclosure, at least two target images are obtained by shooting at least two equipment code areas on a target product by using a camera, and each target image is used for identifying the equipment code area in a subsequent image, so that the situation that the equipment code cannot be identified due to the fact that the equipment code exceeds the field of view of the camera due to the fact that the field of view of the camera is large; in addition, the image is adopted to identify the equipment information in the equipment code area, so that if the target product is determined to be the problem product, the problem of the equipment code area can be traced through the image conveniently, the follow-up targeted correction according to the problem obtained by image investigation is facilitated, and the problem investigation difficulty is reduced.
Alternatively, the camera may be a fly-by camera or a line scan camera, which can capture images continuously during movement of the target product, without requiring the target product to be stationary to form a complete image. Alternatively, the camera may be another camera, such as an area array camera, without limitation.
The aerial camera is different from the traditional digital camera or the traditional industrial camera, and can take a picture in the moving process of the object, so that an image of the object at the moment of taking the picture is obtained. The fly-swatter camera is generally a camera based on a CCD (Charge Coupled Device ) or CMOS (Complementary Metal Oxide Semiconductor, complementary metal oxide semiconductor) chip, implementing the fly-swatter function by a short exposure mode. Since the shooting object is shooting in motion, the aerial camera generally performs compensation calculation by an algorithm because there is a small time difference between the shooting moment and the acquired image.
The line scanning camera is a camera adopting a line array image sensor, and is mainly a CCD. In particular, the line scan camera generally needs to keep moving relative to the subject, and when the subject moves in the field of view of the line scan camera, the line image sensor continuously acquires a series of pixel lines, and then reconstructs the plurality of pixel lines into a two-dimensional image. A line scan camera may also be used in embodiments of the present disclosure to capture a target product.
In the prior art, the code scanning gun has high code scanning recognition rate aiming at static bar codes, but has lower recognition rate on bar codes in motion; because the continuous speed of production line is increased, use on the production assembly line to sweep the yard rifle and require to be stopped when the product of being detected reaches and sweep the yard point position, can reduce detection efficiency, consequently sweep yard recognition rate and detection efficiency incompatibility. The adoption of the fly-swatter camera or the line scanning camera can avoid the problems of the code scanning gun, and the code scanning recognition rate and the detection efficiency are considered.
Optionally, at least two cameras may be fixed on a structural member on the production line, so that a moving route of the target product on the production line is located in a visual field range of the camera, and when the target product passes through the visual field range of the camera, the camera is triggered to shoot the target product. As shown in fig. 2, fig. 2 is a schematic structural diagram of one embodiment of a usage scenario of the device code detection method in the present disclosure. In this example, the target product is specifically a photovoltaic module, and the photovoltaic module has three device code areas thereon, which are respectively located on a glass surface, a frame and a back plate of the photovoltaic module. The photovoltaic module moves along the production line. At several locations in the production line, brackets 20 are provided through which the line flows. The support 20 is provided with a moving rail 221, and the light source 21 fixed on the moving rail 221 can slide on the moving rail 221, so that the position can be conveniently adjusted. The stand is also provided with fixing locations for fixing the three cameras 22, 23 and 24, respectively. The three cameras face the glass face, the frame and the backboard of the photovoltaic module on the assembly line respectively. Each camera triggers shooting when a corresponding equipment code area on the photovoltaic module enters the visual field.
Optionally, three crossbars parallel to the moving rail 221 are provided on the stand 20, and three cameras 22, 23 and 24 are respectively fixed to the three crossbars 222, 223 and 224 and respectively movable on the three crossbars to facilitate adjustment of the respective positions. The three cameras 22, 23 and 24 trigger shooting as the photovoltaic module passes through the cavity portion between the rail 221 and rails 223, 224 with the pipeline.
In some application scenes, because the light source conditions are complex and have large changes, for example, sunlight energy of a skylight is irradiated in some workshops, so that barcode imaging reflection on the glass surface is serious. Optionally, a light shielding plate can be additionally arranged at the top of the bracket to shield the interference light source, so that the stability of equipment code identification is ensured.
Optionally, the device code area includes a device code and text. The device information is determined based on the device code information and the text information. In some examples, the device code information and the text information are the same, and the device code information and the text information are backed up mutually, so that when one of the device code information and the text information cannot be identified, the device information can be identified through the other device, that is, the device information can be the device code information or the text information. In some examples, the device code information and the text information are different, but there is a correspondence; the device information may be device code information, text information, or information determined from the device code information and the text information.
Optionally, before the device information in the at least two device code areas is matched in step S103, the device code and the text in each device code area may be further identified, so as to obtain device code information and text information; comparing the equipment code information and the text information of each equipment code area; and when the equipment code information and the text information in one equipment code area are not matched, determining the target product as a problem product. In the example where the device code information and the text information are the same information, the two information are different, and the two information are determined to be not matched. In the example where the device code information and the text information are different information but have a correspondence relationship, the device code information and the text information are determined to be not matched if the correspondence relationship does not exist.
Optionally, when the device code information and the text information in each device code area are matched, determining the device information of the device code area according to the device code information and the text information; and carrying out matching verification on the device information in each device code area. And when the device information in the at least two device code areas is matched, determining the target product as a normal product. And when the equipment code information and the text information of one equipment code area are not matched or the equipment information in the at least two equipment code areas are not matched with each other, determining the target product as a problem product.
The device code detection method in the embodiment of the disclosure can identify two information contents of the device code and the text on the device code area, and perform matching verification on the two contents, and if the two contents are not matched, the bad device code can be controlled by the card. The device code and the characters are not matched due to bad factors such as printing ink leakage, barcode fold bubbles, printing inadequacy and the like, and compared with the prior art that the character recognition cannot be carried out by a code scanning gun and the matching verification of the device code and the characters cannot be carried out, the device code detection method in the embodiment of the disclosure can improve the detection accuracy.
Optionally, when the device code areas in the at least two target images are respectively identified in step S102, specifically, clipping each target image to obtain a device code image and a text image in the target image; and respectively identifying the equipment code image and the text image corresponding to each target image to obtain the equipment code information and the text information. Optionally, before the device code image and the text image corresponding to each target image are respectively identified, first image preprocessing may be performed on the device code image; and performing second image preprocessing on the text image to improve the recognition rate of the equipment code image and the text image.
There are various methods for performing the first image preprocessing on the device code image. In one example, the device code image may be image binarized to form a pure black and white image. In one example, the high frequency portion of the device code image may be superimposed with different coefficients to improve image sharpness. In one example, the device code image may be subjected to a corrosion operation and then an expansion operation to achieve the effect of removing noise in the blank area. In one example, a zoom-in or zoom-out operation may be performed on the device code image. When the device code is a bar code, since the bar code is formed by ordering a plurality of black bars and blanks with different widths according to a certain coding rule, the excessive space between two adjacent black bars in the bar code can be caused by the excessive device code in the device code image, and the adjacent two black bars are not regarded as parts of the same bar code during recognition, so that the bar code recognition fails; therefore, the device code image can be reduced before being identified, so that the identification success probability of the device code is improved.
There are various methods for performing the second image preprocessing on the text image. In one example, brightness adjustment may be performed on a text image to increase or decrease brightness values of pixel values in the text image. For example, the average gray value of the text image may be measured, and then the brightness value of the text image may be increased or decreased according to whether the average gray value is higher than a preset gray value, so that the gray value of the text image approaches or changes to the preset gray value. In one example, contrast adjustment may be performed on the text image to enhance detail contrast of bright and dark fine lines of the text image. In one example, a sharpening algorithm (e.g., un harp masking) may be employed to sharpen the literal image; for example, the sharpness of the target image is enhanced by extracting the high-frequency component of the target image and then superimposing the high-frequency component with the original target image. For another example, the sharpening operation may be performed after the gaussian blur is performed on the text image, so as to solve the problems of the blur and noise of the text image. In one example, the text image may be cropped to reduce other objects in the text image than text, thereby reducing interference with recognizing text.
Because the camera is fixed on the production line and usually has a certain distance from the product when shooting the product, the size of the equipment code is generally not very large, the equipment code in the shot target image generally occupies only a small area, in order to improve the identification efficiency of the equipment code, optionally, before the equipment code areas in the at least two target images are respectively identified in step S102, the equipment code areas are captured from the target image, and then the captured equipment code areas are cut to obtain the equipment code image and the text image in the target image.
When capturing the device code area in the target image, a preset area in the target image may alternatively be determined as the device code area. Because the placement position of the target product on the production line is relatively fixed, the production line speed is balanced, the equipment codes are generally stuck or printed in a plurality of fixed areas of the target product, the position of the camera on the production line is also fixed in advance, and the shooting time is also preset and fixed, so that the equipment codes generally appear in a plurality of fixed areas in the target image shot by the camera. The fixed area may be determined in advance as a preset area, and the preset area in the target image may be determined as a device code area.
Or, alternatively, the device code region may be determined from an identification lookup within the image. Alternatively, the device code region may be determined from the target image based on a first deep neural network. The first deep neural network can be a target detection algorithm based on deep learning, and is obtained through training after annotation learning of a large number of equipment code pictures. The first deep neural network may be utilized to automatically capture the location of the device code region.
Alternatively, device code capture efficiency may also be provided in combination of both. For example, firstly, confirming whether a device code area exists in a preset area in a target image, and if so, determining the preset area as the device code area; and if the equipment code region does not exist, determining equipment code regions from the target image in other regions according to a first deep learning algorithm.
After each target image is cut to obtain a device code image and a text image in the target image, various methods for identifying the device code image and the text image are available. The identification device code image can be identified by adopting a corresponding method according to the type of the device code. The character image may be recognized by detecting and recognizing characters in the image by using an OCR (Optical Character Recognition ) method, or by using other character recognition methods, which are not limited herein.
Optionally, when the device code image is identified, a defect of the device code is also identified, so as to obtain a device code defect identification result. Alternatively, the defect of the device code is identified only if the device code is identified. In the case where the device code cannot be identified, it may be determined that the device code does not appear in the image, and thus defects of the device code are not identified. Or optionally, in the case that the equipment code cannot be identified, the defect of the equipment code is still identified, so as to obtain an equipment code defect identification result, so as to determine the reason that the equipment code cannot be identified and then perform corresponding processing on the equipment code defect. Alternatively, the identification device code defect may be processed simultaneously with the identification device code to obtain the device information. In some cases, even if the equipment code has a defect, the equipment information contained in the equipment code can be normally identified, and meanwhile, the defect of the equipment code can be identified, so that the subsequent problem caused by the defect can be avoided.
There are various methods for identifying defects of the device code, for example, image identification can be performed according to the characteristics of the device code. Alternatively, the device code defect recognition result may be obtained according to the second deep neural network. There are a variety of methods for training the second deep neural network. For example, by collecting pictures of a large number of defective bar codes and manually calibrating, data obtained by calibration is divided into a training set and a verification set, and the training set is given to a deep learning model for training. The deep learning model can be composed of a convolutional neural network and a fully-connected neural network, and a binary cross entropy loss function is added, so that a second deep neural network with optimized neural network weight can be successfully trained.
When the second deep neural network is utilized to identify the equipment code defects, the prediction probabilities that the equipment code defects in the equipment code region are various different preset defects can be output only by inputting the image corresponding to the equipment code region into the network. The probability threshold can be set, and only when the predicted probability of the equipment code region corresponding to a certain preset defect exceeds the probability threshold, the equipment codes in the equipment code region are considered to have the preset defect. The preset defects can be various, for example, at least one or more of equipment code breakage, equipment code skew, equipment code reverse, equipment code partial falling, equipment code complete falling and equipment code mismatch.
Alternatively, in a case where the acquired device code identification result includes a device code skew, or in a case where the device code cannot be identified, a first operation may be performed after performing a rotation operation on an image corresponding to a device code area, where the first operation includes: and rotating the equipment code image by the preset angle, and identifying the rotated equipment code image again.
And when the equipment code information cannot be identified from the equipment code image after the rotation, repeatedly executing the first operation until the equipment code information can be identified from the equipment code image after the last rotation. When the device code area is rotated, the device code area can be rotated at a fixed rotation angle or at a dynamically adjusted rotation angle. For example, the device code may be rotated to a horizontal angle at one time by recognizing the inclination angle of the device code in the device code area, and then the rotated image of the device code may be recognized again.
In one example, the rotation may be in the same direction at a rotation angle of 3 degrees each time. Alternatively, the rotation angle may be limited to a preset angle (e.g., 15 degrees), that is, the rotation is stopped when the device code information is not recognized after a preset number of consecutive rotations (e.g., 10 times).
In one example, after each rotation, the device code image is further re-subjected to a first image pre-processing and then identified.
A specific example of the device code detection method in the present disclosure is described below by way of example with reference to fig. 3. As shown in fig. 3, fig. 3 is a flow chart illustrating an embodiment of a device code detection method of the present disclosure. As described above, on the production line of the photovoltaic module, as the target product, there are generally two or more device codes on each photovoltaic module, and the two or more device code areas are respectively adhered or printed on each photovoltaic module. In this embodiment, taking three bar codes on each photovoltaic module as an example, the three bar codes are respectively adhered to three bar code areas of the glass surface, the frame and the back plate of the photovoltaic module. Step S301: triggering photographing. When the target product on the production line reaches the corresponding detection position, three cameras are triggered to respectively shoot three bar code areas on the target product, so that three target images are obtained.
Step S302: capturing and cutting. And capturing the position of a bar code area in each target image, and cutting the bar code area to obtain a bar code image and a text image.
Step S303: the first image is preprocessed. The first image preprocessing may be explained with reference to the above, and will not be described herein.
Step S304: and (5) bar code analysis. And carrying out bar code analysis on the bar code image subjected to the first image pretreatment to obtain bar code information.
Step S305: and (5) preprocessing a second image. The second image preprocessing may be explained with reference to the above, and will not be described here again.
Step S306: and (5) text analysis. And performing second image preprocessing on the character image, and performing character recognition by adopting an OCR method to obtain character information.
Step S307: the bar code matches the text. The PLC (Programmable logic Controller, programmable controller) compares the bar code information and the text information of each target image. If the bar code information and the text information of one target image are not matched, the PLC outputs an NG signal. If each target image accords with the matching of the bar code information and the text information, the PLC performs mutual matching verification on the equipment information of the three target images.
Step S308: matching of three barcodes. If the device information of the two target images appears out of the three target images to be not matched with each other, the PLC outputs an NG signal. If the device information in the three target images all match, the PLC outputs an OK signal.
Upon receipt of the NG signal, the MES (Manufacturing Execution System ) determines the target product as the problem product. Upon receiving the OK signal, the MES determines the target product as a normal product. The MES can also send the shot target image or the captured equipment code region information to other equipment or systems, so that the problem of equipment code region can be traced through the image conveniently. Specifically, the MES may use multiple communication protocols such as HTTP, modBus TCP, etc.
The disclosure also provides a device code detection apparatus. As shown in fig. 4, fig. 4 is a block diagram of an embodiment of the device code detection apparatus of the present disclosure. The device code detection apparatus 400 includes:
the acquiring module 401 is configured to acquire at least two target images, where the at least two target images are obtained by respectively shooting at least two device code areas located on a target product with at least two cameras;
An identification module 402, configured to identify the device code areas in the at least two target images, respectively, to obtain device information in each of the device code areas;
a matching module 403, configured to match device information in the at least two device code areas;
a determining module 404, configured to determine the target product as a problem product when there is a mismatch in the device information in the at least two device code areas.
Optionally, the device code area includes a device code and text;
the identification module 402 is further configured to identify the device code and the text in each device code area, so as to obtain device code information and text information;
the matching module 403 is further configured to match device code information and text information of each device code area;
the determining module 404 is further configured to determine the target product as a problem product when there is a mismatch between the device code information and the text information in one of the device code regions.
Optionally, the identifying module 402 is further configured to determine, when the device code information and the text information in each device code area match, the device information of the device code area according to the device code information and the text information;
The determining module 404 is further configured to determine the target product as a normal product when the device information in the at least two device code areas matches.
Optionally, the identifying the device code regions in the at least two target images respectively, to obtain device information in each device code region, includes:
cutting each target image to obtain a device code image and a text image in the target image;
and respectively identifying the equipment code image and the text image corresponding to each target image to obtain the equipment code information and the text information.
Optionally, the device code detection apparatus 400 further includes:
the first image preprocessing module is used for carrying out first image preprocessing on the equipment code image;
and the second image preprocessing module is used for carrying out second image preprocessing on the text image.
Optionally, the first image preprocessing includes at least one of:
adjusting at least one of brightness, contrast and sharpness of the device code image;
performing a shrinking operation on the device code image;
denoising the equipment code image;
and rotating the equipment code image by a preset angle.
Optionally, the identifying the device code regions in the at least two target images respectively, to obtain device information in each device code region, further includes:
when the device code information cannot be identified from the device code image, performing a first operation, the first operation including: rotating the equipment code image by the preset angle, and identifying the rotated equipment code image again;
and when the equipment code information cannot be identified from the equipment code image after the rotation, repeatedly executing the first operation until the equipment code information can be identified from the equipment code image after the last rotation.
Optionally, the device code detection apparatus 400 further includes:
and the capturing module is used for determining the position of the equipment code region from the at least two target images according to the first deep neural network.
Optionally, the target product is a photovoltaic module, and the at least two device code areas include at least two of three device code areas respectively located on a glass surface of the photovoltaic module, a frame of the photovoltaic module, and a back plate of the photovoltaic module.
Optionally, the camera is a fly-by camera or a line scan camera, and the camera is in motion when shooting at least two device code areas located on a target product.
The disclosure also provides an equipment code detection device. As shown in fig. 5, fig. 5 is a block diagram of an embodiment of a device code detection device of the present disclosure. The device code detection device 500 includes: a memory 501 and a processor 502. The processor 502 may be a central processing unit (Central Processing Unit, CPU), but may also be other general purpose processors, digital signal processors (Digital Signal Processor, DSP), application specific integrated circuits (Application Specific Integrated Circuit, ASIC), field programmable gate arrays (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or the like. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
Memory 501 may include various types of storage units, such as system memory, read Only Memory (ROM), and persistent storage. Where the ROM may store static data or instructions that are required by the processor 502 or other modules of the computer. The persistent storage may be a readable and writable storage. The persistent storage may be a non-volatile memory device that does not lose stored instructions and data even after the computer is powered down. In some embodiments, the persistent storage device employs a mass storage device (e.g., magnetic or optical disk, flash memory) as the persistent storage device. In other embodiments, the persistent storage may be a removable storage device (e.g., diskette, optical drive). The system memory may be a read-write memory device or a volatile read-write memory device, such as dynamic random access memory. The system memory may store instructions and data that are required by some or all of the processors at runtime. Furthermore, memory 501 may include any combination of computer-readable storage media, including various types of semiconductor memory chips (e.g., DRAM, SRAM, SDRAM, flash memory, programmable read-only memory), magnetic disks, and/or optical disks may also be employed. In some embodiments, memory 501 may include readable and/or writable removable storage devices such as Compact Discs (CDs), digital versatile discs (e.g., DVD-ROM, dual layer DVD-ROM), blu-ray discs read only, super-density discs, flash memory cards (e.g., SD cards, min SD cards, micro-SD cards, etc.), magnetic floppy disks, and the like. The computer readable storage medium does not contain a carrier wave or an instantaneous electronic signal transmitted by wireless or wired transmission.
The memory 501 has stored thereon executable code that, when processed by the processor 502, may cause the processor 502 to perform some or all of the methods described above.
The disclosure also provides a device code detection system. The equipment code detection system comprises a fly-shooting camera or a line scanning camera and is used for shooting equipment codes positioned on target products to obtain target images; and the equipment code detection equipment is used for acquiring equipment information and/or equipment code defect identification results according to the target image. The device code detection device may be a device code detection device 500 as shown in fig. 5.
In one example, as shown in fig. 6, fig. 6 is a block diagram of the structure of one embodiment of the device code detection system of the present disclosure, the device code detection system 600 including at least two cameras 61 and a device code detection device 62. The device code detection device 62 includes an acquisition module 621, an identification module 622, a matching module 623, a determination module 624, and a communication module 625.
The acquiring module 621 is configured to acquire at least two target images obtained by capturing target products with at least two cameras 61, and send the at least two target images to the identifying module 622. The identifying module 622 is configured to identify the device code regions in the at least two target images, respectively, to obtain device information in each of the device code regions. The matching module 623 is configured to match device information in the at least two device code areas. The determining module 624 is configured to determine the target product as a problem product when there is a mismatch in the device information in the at least two device code regions. The determination module may send at least two target images corresponding to the target product determined to be the problem product to other devices or systems via the communication module 625. The operations performed by the modules in the device code detection device 62 may be explained above, and will not be described herein.
The present disclosure may also be embodied as a computer-readable storage medium (or non-transitory machine-readable storage medium or machine-readable storage medium) having stored thereon executable code (or a computer program or computer instruction code) which, when executed by a processor of an electronic device (or server, etc.), causes the processor to perform some or all of the steps of the above-described methods according to the present disclosure.
The foregoing description of the embodiments of the present disclosure has been presented for purposes of illustration and description, and is not intended to be exhaustive or limited to the embodiments disclosed. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the various embodiments described. The terminology used herein was chosen in order to best explain the principles of the embodiments, the practical application, or the improvement of technology in the marketplace, or to enable others of ordinary skill in the art to understand the embodiments disclosed herein. Various elements of the embodiments or examples may be omitted or replaced with equivalent elements thereof. Furthermore, the steps may be performed in a different order than described in the present disclosure. Further, various elements of the embodiments or examples may be combined in various ways. It is important that as technology evolves, many of the elements described herein may be replaced by equivalent elements that appear after the disclosure. Equivalent structures or equivalent flow modifications of the claims, the description and the drawings of the present disclosure, or direct or indirect application in other related technical fields are included in the scope of the claims of this patent.
Claims (14)
1. A device code detection method, comprising:
acquiring at least two target images, wherein the at least two target images are obtained by respectively shooting at least two equipment code areas on a target product through at least two cameras;
respectively identifying the equipment code areas in the at least two target images to obtain equipment information in each equipment code area;
matching the device information in the at least two device code areas;
and when the equipment information in the at least two equipment code areas is not matched, determining the target product as a problem product.
2. The device code detection method of claim 1, wherein the device code area includes a device code and text;
the method further comprises the steps of:
identifying the equipment codes and the characters in each equipment code area to obtain equipment code information and character information;
matching the equipment code information and the text information of each equipment code area;
and when the equipment code information and the text information in one equipment code area are not matched, determining the target product as a problem product.
3. The device code detection method of claim 2, wherein the method further comprises:
When the equipment code information and the text information in each equipment code area are matched, determining the equipment information of the equipment code area according to the equipment code information and the text information;
and when the device information in the at least two device code areas is matched, determining the target product as a normal product.
4. The device code detection method according to claim 2, wherein the identifying device code regions in the at least two target images, respectively, to obtain device information in each of the device code regions, includes:
cutting each target image to obtain a device code image and a text image in the target image;
and respectively identifying the equipment code image and the text image corresponding to each target image to obtain the equipment code information and the text information.
5. The device code detection method of claim 4, wherein the method further comprises:
performing first image preprocessing on the equipment code image;
and performing second image preprocessing on the text image.
6. The device code detection method of claim 5, wherein the first image preprocessing includes at least one of:
Adjusting at least one of brightness, contrast and sharpness of the device code image;
performing a shrinking operation on the device code image;
denoising the equipment code image;
and rotating the equipment code image by a preset angle.
7. The device code detection method according to claim 6, wherein the identifying device code regions in the at least two target images, respectively, results in device information in each of the device code regions, further comprising:
when the device code information cannot be identified from the device code image, performing a first operation, the first operation including: rotating the equipment code image by the preset angle, and identifying the rotated equipment code image again;
and when the equipment code information cannot be identified from the equipment code image after the rotation, repeatedly executing the first operation until the equipment code information can be identified from the equipment code image after the last rotation.
8. The device code detection method of claim 1, wherein the method further comprises:
and determining the position of the equipment code region from the at least two target images according to the first deep neural network.
9. The device code detection method of claim 1, wherein the target product is a photovoltaic module, and the at least two device code regions include at least two of three device code regions respectively located on a glass face of the photovoltaic module, a frame of the photovoltaic module, and a back plate of the photovoltaic module.
10. The device code detection method of claim 1, wherein the camera is a fly-by camera or a line scan camera, and the camera is in motion when capturing at least two device code areas located on a target product.
11. A device code detection apparatus, comprising:
the acquisition module is used for acquiring at least two target images, wherein the at least two target images are obtained by respectively shooting at least two equipment code areas on a target product through at least two cameras;
the identification module is used for respectively identifying the equipment code areas in the at least two target images to obtain equipment information in each equipment code area;
the matching module is used for matching the equipment information in the at least two equipment code areas;
and the determining module is used for determining the target product as a problem product when the equipment information in the at least two equipment code areas are unmatched.
12. A device code detection device, comprising: a memory and a processor, the memory having executable code stored thereon, which when processed by the processor, causes the processor to perform the device code detection method of any of claims 1 to 10.
13. A device code detection system, comprising:
at least two cameras for shooting at least two equipment code areas on the target product to obtain at least two target images;
the device code detection apparatus of claim 12, for determining whether the target product is a problem product based on the at least two target images.
14. A computer readable storage medium having stored thereon executable code which when executed by a processor causes the processor to perform the device code detection method of any of claims 1 to 10.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310897864.7A CN116883697A (en) | 2023-07-21 | 2023-07-21 | Device code detection method, device, system and readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310897864.7A CN116883697A (en) | 2023-07-21 | 2023-07-21 | Device code detection method, device, system and readable storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116883697A true CN116883697A (en) | 2023-10-13 |
Family
ID=88258421
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310897864.7A Pending CN116883697A (en) | 2023-07-21 | 2023-07-21 | Device code detection method, device, system and readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116883697A (en) |
-
2023
- 2023-07-21 CN CN202310897864.7A patent/CN116883697A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN101464418B (en) | Flaw detection method and apparatus | |
| CN110595999B (en) | Image acquisition system | |
| US20110121077A1 (en) | Increasing imaging quality of a bar code reader | |
| US11941859B2 (en) | Camera and method for processing image data | |
| CN113591967A (en) | Image processing method, device and equipment and computer storage medium | |
| US11265455B1 (en) | Method of differentiating between focus drift and a change in distance to target for variable focus lens | |
| US6941026B1 (en) | Method and apparatus using intensity gradients for visual identification of 2D matrix symbols | |
| US10990778B2 (en) | Apparatus and method for recognizing barcode based on image detection | |
| US11188727B1 (en) | Efficient parallel barcode subpixel alignment | |
| CN115953373B (en) | Glass defect detection method, device, electronic equipment and storage medium | |
| CN109918957A (en) | A kind of terminal, the system and method for the remote two-dimensional code scanning of double vision | |
| CN107742119B (en) | Object contour extraction and matching device and method based on back-image imaging | |
| CN115752969A (en) | Method, system and equipment for detecting sealing performance of aluminum foil seal | |
| Yang et al. | Accurate localization of four extreme corners for barcode images captured by mobile phones | |
| CN109359497A (en) | A kind of positioning and recognition methods in VAT invoice two-dimensional image code region | |
| CN116883697A (en) | Device code detection method, device, system and readable storage medium | |
| CN116167394A (en) | Bar code recognition method and system | |
| US20230007185A1 (en) | Method of detecting and correcting focus drift of variable focus lens for fixed focus applications | |
| CN116883756A (en) | Device code detection method, device, system and readable storage medium | |
| CN109738459A (en) | Surface cleanness on-line monitoring system | |
| CN111815560B (en) | Photovoltaic power station fault detection method and device, portable detection equipment and storage medium | |
| CN116046791B (en) | Method and device for detecting defect of adhesive dispensing | |
| CN114004799A (en) | Tire bar code identification method, system and equipment based on machine vision | |
| KR102203441B1 (en) | Apparatus and Method for Classifying Electronic Components | |
| CN114444531A (en) | Method and system for collecting data for training Machine Learning (ML) models |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |