CN116468726A - Online foreign matter line detection method and system - Google Patents

Online foreign matter line detection method and system Download PDF

Info

Publication number
CN116468726A
CN116468726A CN202310694231.6A CN202310694231A CN116468726A CN 116468726 A CN116468726 A CN 116468726A CN 202310694231 A CN202310694231 A CN 202310694231A CN 116468726 A CN116468726 A CN 116468726A
Authority
CN
China
Prior art keywords
gray
image
value
detection
pixel
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202310694231.6A
Other languages
Chinese (zh)
Other versions
CN116468726B (en
Inventor
洪亚德
何荣茂
翁锦祥
丁凯明
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Xiamen G First Oeic Co ltd
Original Assignee
Xiamen G First Oeic Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Xiamen G First Oeic Co ltd filed Critical Xiamen G First Oeic Co ltd
Priority to CN202310694231.6A priority Critical patent/CN116468726B/en
Publication of CN116468726A publication Critical patent/CN116468726A/en
Application granted granted Critical
Publication of CN116468726B publication Critical patent/CN116468726B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0004—Industrial image inspection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T1/00—General purpose image data processing
    • G06T1/0014—Image feed-back for automatic industrial control, e.g. robot with camera
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/10—Image enhancement or restoration using non-spatial domain filtering
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/20—Image enhancement or restoration using local operators
    • G06T5/30—Erosion or dilatation, e.g. thinning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/11—Region-based segmentation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/60—Analysis of geometric attributes
    • G06T7/62—Analysis of geometric attributes of area, perimeter, diameter or volume
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/90—Determination of colour characteristics
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/97—Determining parameters from multiple pictures
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20048—Transform domain processing
    • G06T2207/20056—Discrete and fast Fourier transform, [DFT, FFT]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30108—Industrial image inspection
    • G06T2207/30121—CRT, LCD or plasma display

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Quality & Reliability (AREA)
  • Geometry (AREA)
  • Robotics (AREA)
  • Image Processing (AREA)

Abstract

The invention discloses a method and a system for detecting an online foreign matter line, wherein, firstly, a glass substrate photo containing a bonding area is collected in a detection machine; carrying out Gaussian blur, logarithmic subtraction and gray mapping on the glass substrate photo, detecting the image to be preprocessed by adopting an image processing algorithm, and identifying a defect area in the second gray image; combining the defect areas, and re-dividing the defect areas into areas with specified sizes by utilizing a sliding window to generate area pictures with the same size; inputting the regional picture into a deep learning detection module, identifying the regional picture by utilizing a VGG model, and selecting whether to be manually re-judged according to an identification result; and the VGG model sends the final result back to the detection machine. The detection method provided by the invention combines image processing and deep learning, and meets the requirements of industrial production on detection precision under the condition of meeting the requirements of on-line detection speed.

Description

Online foreign matter line detection method and system
Technical Field
The invention belongs to the technical field of computer vision, and particularly relates to an online foreign matter line detection method and system.
Background
With the development of display technology, flat panel display devices such as a liquid crystal display (LiquidCrystalDisplay, LCD) have been widely used in various consumer electronic products such as mobile phones, televisions, tablet computers, notebook computers, security monitoring devices, and vehicle-mounted display screens due to their advantages of high image quality, power saving, thin body, and wide application range. In the production process, an IC integrated circuit chip and an FPC flexible circuit board are generally bonded on a glass substrate; bonding, also known as Bonding, i.e., pressing, requires Bonding certain electronic components to the glass. In the bonding process, if any foreign matter exists between the glass and the component, whether the connection is electrically conducted or not directly influences the display effect of the display panel, for example, a line which is transverse or vertical can be caused in imaging, so that the connection test of the glass substrate and the IC device is a necessary detection link. However, the existing particle indentation detection machine only calculates the number and the offset of conductive particles, and cannot detect metal foreign matters, glass foreign matters, scratches, fragments, electric corrosion and the like, and if the defects are light, poor quality occurs in a small range, and if the defects are heavy, a line of products is scrapped, so that huge economic loss is caused.
For defects such as metal foreign matters, glass foreign matters, scratches, fragments, electric corrosion and the like, the traditional defect detection method is an artificial visual detection method for appearance detection, and a large number of workers still do the work in a plurality of industries such as mobile phones, flat panel displays and the like at present. The artificial vision detection method needs to be carried out under the microscope or strong light illumination condition, has the defects of great damage to eyes of detection personnel, strong subjectivity, limited human eye space and time resolution, large detection uncertainty, easy ambiguity generation, low efficiency and the like, and is difficult to meet the detection requirements of modern industry on high speed and high resolution.
In order to solve the problems of strong subjectivity, limited human eye space and time resolution, large detection uncertainty, easy ambiguity generation, low efficiency and the like existing in the bonding detection of the manual visual detection method, the Chinese patent application with the publication number of CN114822338A discloses a detection method of a display panel, which comprises the following steps: providing a display panel to be detected, wherein the display panel comprises a first substrate and a second substrate which are oppositely arranged; a detection circuit is arranged on the first substrate, and the detection circuit and the bonding circuit on the first substrate are positioned on two opposite side surfaces of the first substrate; pressing the detection circuit and the bonding circuit for detection; and cutting off the detection circuit. The invention adopts the method of power-on detection to detect the display panel, and usually, a circuit board is required to be physically connected with test equipment so as to transmit current or measurement signals, which may require an additional connector or a test fixture and increase the complexity and time cost of the test; the power-on detection method is mainly used for detecting circuit connectivity and electrical parameters such as resistance, voltage and the like on a circuit board, and other types of defects cannot be effectively detected.
Therefore, the Chinese patent application with publication number of CN116091503A discloses a method for judging the defect of the foreign matters of the panel, which comprises the following steps: acquiring an image to be identified of a target panel; inputting the image to be identified into a target detection model to detect target defects, and obtaining a first image; inputting the first image into a classification model for classification to obtain a classification result; carrying out gray processing on the first image with the OK classification result to obtain a target line area; and obtaining a defect judging result of the target panel based on the line width value of the target line area. The invention adopts an image analysis method to detect the foreign matter defects of the target panel, but has complex model design, so that the detection efficiency and the detection precision can not meet the requirement of on-line detection.
Disclosure of Invention
The invention provides an online foreign matter line detection method and system, and aims to solve the problems of low speed and low precision of the existing detection method.
In order to solve the technical problems, the invention provides an online foreign matter line detection method, which comprises the following steps:
s1: and collecting a photo of the glass substrate containing the bonding area at a detection machine.
S2: and preprocessing the glass substrate photo, wherein the preprocessing mode comprises a Gaussian blur operation and a gray mapping operation which are sequentially arranged. The Gaussian blur operation is used for blurring tiny noise points and interference in the glass substrate photo and generating a first gray level image. The gray mapping operation maps the gray value of each pixel of the first gray image to a set numerical interval, generates a new gray value, and combines the new gray value into a second gray image.
S3: and detecting the second gray level image by adopting an image processing algorithm, and identifying a defect area in the second gray level image. The image processing algorithm specifically comprises the following steps:
s3-1: and binarizing the input second gray level image to obtain a binary image.
S3-2: and executing a segmentation operation on the binary image by adopting a connectivity analysis method to obtain a detection area image.
S3-3: and carrying out morphological corrosion treatment on each detection area image to obtain a corrosion image.
S3-4: the spatial domain of each erosion image is transformed into a frequency domain signal by a discrete fourier transform.
S3-5: and constructing a band-pass filter in the frequency domain, wherein the band-pass filter only allows signals in a specified range to pass through and is used for filtering high-frequency components in the frequency domain signals to obtain low-frequency signals.
S3-6: and restoring the low-frequency signal into a spatial domain to obtain a filtered image.
S3-7: and subtracting the pixel gray values of the second gray image and the filtered image to obtain a gray difference value of each pixel, and generating a difference map.
S3-8: and constructing a minimum circumscribed rectangle in a region with a difference value not being zero on the difference map, calculating the area of the circumscribed rectangle, and judging the region as a defect region if the area is larger than a set threshold value.
S4: and merging the defect areas, traversing and calculating a union set of the two defect areas, if the union set is larger than a set value, not processing, otherwise merging. And then the sliding window is utilized to be divided into areas with specified sizes again, and the area pictures with the same size are generated.
S5: and inputting the region picture into a deep learning detection module, identifying the region picture by utilizing a VGG model, taking the identification result which is the same as the defective region as a final result, submitting the identification result which is different from the defective region to a manual re-judgment, and taking the identification result after the manual re-judgment as the final result.
S6: and the VGG model sends the final result back to the detection machine.
Preferably, the gaussian blur firstly constructs a gaussian kernel, convolves the constructed gaussian kernel with a glass substrate photo, performs weighted average on neighborhood pixels around each pixel and the gaussian kernel to obtain a blurred pixel gray value, and the convolves the pixel gray value by using the following formula:
in the method, in the process of the invention,for the first gray-scale image coordinate pixel +.>Gray value at>、/>To traverse the coordinates of the center point of the Gaussian kernel matrix for the glass substrate photograph, < + >>Is the variance of the constructed gaussian kernel.
Preferably, the preprocessing mode further includes a log subtraction operation set between the gaussian blur operation and the gray mapping operation, where the log subtraction operation generates an intermediate gray image according to a difference between a natural log of gray values of each pixel in the first gray image and a natural log of gray values of corresponding pixels in the glass substrate photo, and the intermediate gray image is used as an input of the gray mapping operation, and the calculation method includes:
in the method, in the process of the invention,for intermediate gray-scale image coordinates->Gray value of pixel at +.>Photo coordinates for glass substrate->Gray value of pixel at +.>For the first gray-scale image coordinates +.>A gray value of the pixel.
Preferably, the gray mapping has a value range of 0-255, and the specific method of gray mapping is as follows:
in the method, in the process of the invention,for the second gray level image coordinates +.>Gray value at>For inputting the image of gray mapping operation in coordinates +.>Gray value at>Maximum gray value of image for inputting gray mapping operation,/->The minimum gray value of the image for the input gray mapping operation.
Preferably, the binarization processing adopts a local binarization method, the local binarization method firstly calculates an average gray value of the second gray image, then randomly selects one gray value to divide a gray histogram of the second gray image into a foreground color and a background color, respectively calculates the average gray value of the foreground color, the average gray value of the background color, the proportion of the number of foreground pixels to the total number of pixels and the proportion of the number of background pixels to the total number of pixels, and finally calculates the variance according to the following formula:
in the method, in the process of the invention,for variance->For the proportion of foreground pixels to total pixels, +.>Is the average gray value of the foreground color, +.>Is the average gray value of the second gray image, for example>For the proportion of the number of background-color pixels to the total number of pixels, < >>Is the average gray value of the background color.
Selecting different arbitrary gray values to calculate varianceSo that the variance->The maximum gray value is the optimal gray threshold value of the local binarization method, the gray of the pixel point with the gray value larger than the optimal gray threshold value is set to 255, and the gray of the pixel point with the gray value not larger than the optimal gray threshold value is set to 0.
Preferably, the morphological etching treatment adopts an omnidirectional etching mode, the omnidirectional etching traverses the detection area, the gray values of the target pixel and the adjacent upper, lower, left and right pixels are considered, whether the target pixel is consistent with the etched structural elements or not is confirmed, if so, the pixel point is reserved, and otherwise, the pixel point is deleted.
Preferably, the signal range allowed to pass by the band-pass filter is set to be 3-10 of the sigma value of the Gaussian filter function.
Preferably, the threshold in step S3-8 is set to 16 pixels.
Preferably, the recognition result after the artificial re-judgment is also used as a training set of the VGG model for training the VGG model.
Correspondingly, the invention also provides an online foreign matter line detection system, which comprises a detection machine, an image preprocessing module and an image processing module deep learning and re-judging server, wherein the detection machine is provided with an optical imaging module and a material loading and transmitting module, and the detection system is configured into the online foreign matter line detection method.
The optical imaging module is used for collecting a glass substrate photo containing a bonding area on the detection machine.
The material loading and conveying module is used for continuously conveying materials for on-line detection.
The image preprocessing module is used for carrying out Gaussian blur and gray mapping processing on the image acquired by the optical imaging module, and the processed image is used as a second gray image to be input into the image processing module.
The image processing module detects the second gray level image, identifies a defective area in the second gray level image, and merges the defective areas to form an area image according to the requirement of the deep learning detection module on the size of the input image.
And the deep learning re-judging server receives the region picture and the detection result of the image processing module, adopts the VGG module to identify the region picture, and trains according to the detection result of the image processing module. And sending the identification result after the identification of the regional picture back to the detection machine.
Compared with the prior art, the invention has the following technical effects:
1. the online foreign matter line detection method combines image processing and deep learning, detects the bonding image by using an image processing algorithm after preprocessing the bonding image, and then rechecks and trains by adopting a deep learning detection module; the detection method can detect defects such as micro cracks, foreign matters and the like on the bonding line, is flexible to deploy, and meets the requirements of industrial production on detection precision under the condition of meeting the requirements of on-line detection speed.
2. The online foreign matter line detection method provided by the invention can highlight the difference between the glass substrate photo and the photo after Gaussian blur by generating the first gray image through the logarithmic subtraction method, particularly in the edge and detail area, the edge and detail generally have larger gray change, so that the characteristics can be enhanced by taking the logarithm after the gray difference is calculated, the characteristics are more obvious in the generated first gray image, and the detection precision can be effectively improved.
3. The online foreign matter line detection method provided by the invention has the advantages that the gray level of the intermediate gray level image is mapped in the image preprocessing process, the gray level of each pixel of the intermediate gray level image is readjusted, the brightness difference in the image is enhanced, the contrast of the image is adjusted, the details are more obvious, the interested line foreign matter target is highlighted, and the false detection omission rate in the detection process can be effectively reduced.
4. The online foreign matter line detection method provided by the invention uses local binarization for the picture, avoids the characteristic that global binarization is easily affected by shadow, uses the local binarization, and by selecting the optimal threshold value, the foreground color and the background color are more reasonably distinguished in the binarization process, and finally the obtained binary image is more accurate, thereby effectively improving the detection precision.
5. The online foreign matter line detection method provided by the invention carries out morphological corrosion treatment on each detection area, eliminates boundaries, tiny objects and single isolated pixels in the image, can eliminate interference points with small areas in the detection area, and can effectively improve the image processing speed.
6. The online foreign matter line detection method provided by the invention converts the spatial domain of the picture into the frequency domain through the discrete Fourier transform, filters the high-frequency signal in the frequency domain, can effectively reduce noise in the image, makes the image clearer, has more obvious details, and simultaneously reduces redundant information in the image, thereby realizing image compression and finally improving the detection precision and efficiency.
Drawings
FIG. 1 is a schematic flow chart of an online foreign matter line detection method according to the present invention;
fig. 2 is a process flow diagram of the image processing algorithm of the present invention.
The reference numerals are: 1. a second gray scale image; 2. a binary image; 3. detecting an area image; 4. corroding the image; 5. a spectrogram; 6. filtering the image; 7. a difference map; 8. and (5) detecting a result diagram.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the technical solutions of the present invention will be clearly and completely described below with reference to the specific embodiments of the present application and with reference to the accompanying drawings.
Example 1
As shown in fig. 1, the present invention provides an online foreign matter line detection method, which includes the following steps:
s1: and collecting a photo of the glass substrate containing the bonding area at a detection machine.
S2: and preprocessing the glass substrate photo, wherein the preprocessing mode comprises a Gaussian blur operation and a gray mapping operation which are sequentially arranged. The Gaussian blur operation is used for blurring tiny noise points and interference in the glass substrate photo and generating a first gray level image. The gray mapping operation maps the gray value of each pixel of the first gray image to a set numerical interval, generates a new gray value, and combines the new gray value into a second gray image.
The specific steps of preprocessing the image are as follows:
s2-1: and carrying out Gaussian blur processing on the input glass substrate photo.
In the embodiment, a two-bit Gaussian filter is constructed with the size of 3 multiplied by 3 and the standard deviation of 1.5, convolution operation is carried out on the constructed Gaussian kernel and an input image, and weighted average is carried out on neighborhood pixels around each pixel and the Gaussian kernel, so that a blurred pixel value is obtained; and selecting an expansion, filling or clipping processing mode for pixels at the boundary to keep the sizes of the output image and the input image consistent, and obtaining the image after Gaussian blur processing. The convolution operation uses the following formula:
in the method, in the process of the invention,image coordinate pixels after Gaussian blur>Gray value at>、/>To traverse the coordinates of the center point of the Gaussian kernel matrix for the glass substrate photograph, < + >>For the variance of the constructed Gaussian kernel, generated +.>A first gray scale image is generated in accordance with the corresponding pixel location combinations.
S2-2: in this embodiment, a log subtraction operation may be further added between the gaussian blur and the gray mapping operation, where the log subtraction operation obtains the intermediate gray image according to a difference between a natural log of gray values of each pixel in the first gray image and a natural log of gray values of corresponding pixels in the glass substrate photo.
The logarithm subtraction method comprises the following steps:
in the method, in the process of the invention,for intermediate gray-scale image coordinates->Gray value of pixel at +.>Photo coordinates for glass substrate->Gray value of pixel at +.>For the first gray-scale image coordinates +.>A gray value of the pixel.
S2-3: and performing gray mapping on the first gray level image or the intermediate gray level image, mapping the gray level value of the first gray level image or the intermediate gray level image to a set numerical value interval, generating a new gray level value, and combining the new gray level value into a second gray level image 1.
The value range of the gray mapping is 0-255, and the specific method of the gray mapping is as follows:
in the method, in the process of the invention,for the second gray level image 1 coordinates +.>Gray values at, in case of using a logarithmic subtraction operation, < >>For intermediate gray-scale image coordinates->Gray value at>Is the maximum gray value of the intermediate gray image,a minimum gray value for the intermediate gray image; in case no log subtraction operation is used, < > is used>For the first gray-scale image coordinates +.>Gray value at>Is the maximum gray value of the first gray image, is->Is the minimum gray value of the first gray image. .
S3: and detecting the second gray level image 1 by adopting an image processing algorithm, and identifying a defect area in the second gray level image 1. The image processing algorithm is shown in fig. 2, and specifically comprises the following steps:
s3-1: binarizing the inputted second gray level image 1 to obtain a binary image 2, wherein the binary image 2 removes the gray level of the second gray level image 1, and the gray level value of each pixel is adjusted to 0 or 255 according to a threshold value.
The binarization processing adopts a local binarization method, the local binarization method firstly calculates the average gray value of the second gray image 1, then randomly selects one gray value to divide the gray histogram of the second gray image 1 into two parts of foreground color and background color, respectively calculates the average gray value of the foreground color, the average gray value of the background color, the proportion of the number of foreground pixels to the total number of pixels and the proportion of the number of background pixels to the total number of pixels, and finally calculates the variance according to the following formula:
in the method, in the process of the invention,for variance->For the proportion of foreground pixels to total pixels, +.>Is the average gray value of the foreground color, +.>Is the average gray value of the second gray image 1, for example>For the proportion of the number of background-color pixels to the total number of pixels, < >>Is the average gray value of the background color.
Selecting different arbitrary gray values to calculate varianceSo that the variance->The maximum gray value is the optimal gray threshold value of the local binarization method, the gray of the pixel point with the gray value larger than the optimal gray threshold value is set to 255, and the gray of the pixel point with the gray value not larger than the optimal gray threshold value is set to 0.
S3-2: performing a segmentation operation on the binary image 2 to obtain a detection area image, wherein the segmentation operation adopts a connectivity analysis method, and the connectivity analysis method segments according to the shape and the area of the connected area in the binary image 2 to obtain the detection area, and the specific method is as follows:
the binary pattern 2 to be subjected to connectivity analysis is used as an input, the pixels in the binary pattern 2 are traversed by using a scanning line algorithm, connected regions are marked, the shape and area characteristics of each marked connected region are calculated, the connected region is divided according to the characteristics of the connected region and a set area threshold value, and the divided result is output as a detection region image 3.
S3-3: and carrying out morphological corrosion treatment on each detection area image 3 to obtain a corrosion image 4.
The morphological corrosion treatment adopts an omnidirectional corrosion mode, the omnidirectional corrosion traverses each pixel of the detection area image 3, the gray values of a target pixel and four adjacent pixels are considered, whether the target pixel is consistent with the corroded structural element or not is confirmed, if so, the pixel point is reserved, and otherwise, the pixel point is deleted.
Constructing square structural elements with the size of 3 multiplied by 3 pixels, moving the structural elements on the detection area image 3 according to different directions, comparing the structural elements with pixels of the image, and at each position, if the structural elements completely cover corresponding pixel areas in the image, retaining the pixels, otherwise corroding the pixels; applying three etching operations to the detection zone image 3, each etching further shrinking the boundary or eliminating smaller objects; and taking the detection area image after multiple corrosions as a final output result to obtain a corrosion image 4.
S3-4: the spatial domain of each erosion image 4 is converted to a frequency domain signal by discrete fourier transform (DFT, discrete Fourier Transform), resulting in a spectrogram 2, 5. Discrete Fourier transform to generate a complex array frequency domain imageWherein->And->Is the frequency coordinate, +.>Frequency domain information of the corrosion image is included.
S3-5: at the position ofA band-pass filter is built in, and the band-pass filter only allows signals in a specified range to pass through and is used for filtering high-frequency components in the frequency domain signals to obtain low-frequency signals.
Constructing a zero matrix of the same size as the input imageThe matrix will be used to represent the frequency domain response of the filter;
for each frequency coordinateCalculate its distance Euclidean distance to the frequency center +.>；
The filter response for each frequency coordinate is calculated according to the formula of the gaussian filter function:
wherein,,is the standard deviation of the Gaussian filter function, +.>Is the frequency coordinate +.>Distance Euclidean distance to frequency center;
image of frequency domainAnd filter response->Element-wise multiplication to obtain a filtered frequency domain image +.>。
In the present embodiment, the signal range allowed to pass by the constructed band-pass filter is set to be 3-10 sigma of the Gaussian filter function, so that the frequency domain image can be effectively removedInterference of medium-high frequency signals.
S3-6: frequency domain image that will preserve low frequency signalsAn inverse fourier transform (IDFT, inverseDiscrete fourier transform) is performed to reduce to the spatial domain, resulting in a filtered image 6.
S3-7: and subtracting the pixel-by-pixel gray level value of the second gray level image 1 and the filtered image 6 to obtain a gray level difference value of each pixel, and generating a difference value 7 in fig. 2. Traversing the second gray levelEach pixel of image 1 and filtered image 6, for each pixel location in the imageThe difference between the gray values of the second gray image 1 and the filtered image 6 at this position is calculated, and the difference value can be calculated using the following formula:
wherein,,representing the second gray level image 1 at position +.>Gray value at>Representing the filtered image 6 at position +.>Gray values at that point.
Finally, the differential value of each pixel is calculatedSaving the two images into a new image to obtain a differential graph, wherein each pixel represents the gray value difference of the two images at the corresponding positions.
S3-8: and constructing a minimum circumscribed rectangle in a region with a difference value not being zero in the difference diagram 7 in the figure 2, calculating the area of the circumscribed rectangle, and judging the region as a defect region if the area is larger than a set threshold value. The threshold area may be set according to a picture acquired by the optical imaging device, in this embodiment, the threshold is set to 16 pixels, and if the area of the circumscribed rectangle exceeds the area occupied by 16 pixels, the area is marked as a defect area, so as to obtain the detection result 8 in fig. 2.
S4: and merging the defect areas, traversing and calculating a union set of the two defect areas, if the union set is larger than a set value, not processing, otherwise merging. And then the sliding window is utilized to be divided into areas with specified sizes again, and the area pictures with the same size are generated.
S5: and inputting the region picture into a deep learning detection module, identifying the region picture by utilizing a VGG model, taking the identification result which is the same as the defective region as a final result, submitting the identification result which is different from the defective region to a manual re-judgment, and taking the identification result after the manual re-judgment as the final result. The recognition result after the artificial re-judgment is also used as a training set of the VGG model for training the VGG model.
S6: and the VGG model sends the final result back to the detection machine.
Example two
The invention provides an online foreign matter line detection system, which comprises a detection machine, an image preprocessing module and an image processing module deep learning and re-judging server, wherein an optical imaging module and a material loading and transmitting module are arranged on the detection machine, and the detection system is configured to execute the online foreign matter line detection method in the embodiment one.
The optical imaging module is used for collecting a glass substrate photo containing a bonding area on the detection machine.
The material loading and conveying module is used for continuously conveying materials for on-line detection.
The image preprocessing module is used for performing Gaussian blur and gray mapping processing on the image acquired by the optical imaging module, and inputting the processed image serving as a second gray level image 1 into the image processing module. The image preprocessing module can also add a log subtraction operation between the Gaussian blur operation and the gray mapping operation, wherein the log subtraction operation generates an intermediate gray level image according to the difference value between the natural logarithm of the gray level value of each pixel in the Gaussian blurred image and the natural logarithm of the gray level value of the corresponding pixel in the image acquired by the optical imaging module, and the intermediate gray level image is used as the input of the gray mapping operation.
The image processing module detects the second gray level image 1, identifies a defective area in the second gray level image 1, and combines the defective areas to form an area picture according to the requirement of the deep learning detection module on the size of the input picture.
And the deep learning re-judging server receives the region picture and the detection result of the image processing module, adopts the VGG module to identify the region picture, and trains according to the detection result of the image processing module. And sending the identification result after the identification of the regional picture back to the detection machine.
The foregoing is merely a preferred embodiment of the present invention, and it should be noted that modifications and improvements could be made by those skilled in the art without departing from the inventive concept, which falls within the scope of the present invention.

Claims (10)

1. An on-line foreign matter line detection method is characterized by comprising the following steps:
s1: collecting a glass substrate photo containing a bonding area on a detection machine;
s2: preprocessing the glass substrate photo, wherein the preprocessing mode comprises a Gaussian blur operation and a gray mapping operation which are sequentially arranged; the Gaussian blur operation is used for blurring tiny noise points and interference in the glass substrate photo and generating a first gray level image; the gray mapping operation maps the gray value of each pixel of the first gray image to a set numerical value interval, generates a new gray value, and combines the new gray value into a second gray image;
s3: detecting the second gray level image by adopting an image processing algorithm, and identifying a defect area in the second gray level image; the image processing algorithm specifically comprises the following steps:
s3-1: binarizing the input second gray level image to obtain a binary image;
s3-2: performing segmentation operation on the binary image by adopting a connectivity analysis method to obtain a detection area image;
s3-3: carrying out morphological corrosion treatment on each detection area image to obtain a corrosion image;
s3-4: converting the spatial domain of each erosion image to a frequency domain signal by discrete fourier transform;
s3-5: constructing a band-pass filter in a frequency domain, wherein the band-pass filter only allows signals in a specified range to pass through and is used for filtering high-frequency components in the frequency domain signals to obtain low-frequency signals;
s3-6: restoring the low-frequency signal into a spatial domain to obtain a filtered image;
s3-7: subtracting the pixel-by-pixel gray values of the second gray image and the filtered image to obtain a gray difference value of each pixel, and generating a difference image;
s3-8: constructing a minimum circumscribed rectangle in a region with a difference value not being zero on the difference graph, calculating the area of the circumscribed rectangle, and judging the region as a defect region if the area is larger than a set threshold value;
s4: combining the defect areas, traversing and calculating a union set of the two defect areas, if the union set is larger than a set value, not processing, otherwise, combining; then, dividing the region into regions with specified sizes again by utilizing a sliding window, and generating region pictures with the same size;
s5: inputting the region picture into a deep learning detection module, identifying the region picture by utilizing a VGG model, taking the identification result which is the same as the defect region as a final result, submitting the identification result which is different from the defect region to a manual re-judgment, and taking the identification result after the manual re-judgment as the final result;
s6: and the VGG model sends the final result back to the detection machine.
2. The online foreign object line detection method according to claim 1, wherein the gaussian blur firstly constructs a gaussian kernel, convolves the constructed gaussian kernel with a glass substrate photo, and performs weighted average on neighborhood pixels around each pixel and the gaussian kernel to obtain a blurred pixel gray value, and the convolution operation uses the following formula:
in the method, in the process of the invention,for the first gray-scale image coordinate pixel +.>Gray value at>、/>To traverse the coordinates of the center point of the Gaussian kernel matrix for the glass substrate photograph, < + >>Is the variance of the constructed gaussian kernel.
3. The method according to claim 1, wherein the preprocessing mode further includes a log subtraction operation between a gaussian blur operation and a gray-scale mapping operation, the log subtraction operation generates an intermediate gray-scale image according to a difference between a natural logarithm of a gray-scale value of each pixel in the first gray-scale image and a natural logarithm of a gray-scale value of a corresponding pixel in the glass substrate photo, and the method includes the steps of:
in the method, in the process of the invention,for intermediate gray-scale image coordinates->Gray value of pixel at +.>Photo coordinates for glass substrateGray value of pixel at +.>For the first gray-scale image coordinates +.>A gray value of the pixel.
4. The online foreign object line detection method according to claim 1 or 3, wherein the gray scale map has a value range of 0 to 255, and the gray scale map is specifically implemented as follows:
in the method, in the process of the invention,for the second gray level image coordinates +.>Gray value at>For inputting the image of gray mapping operation in coordinates +.>Gray value at>Maximum gray value of image for inputting gray mapping operation,/->The minimum gray value of the image for the input gray mapping operation.
5. The method for detecting an online foreign object line according to claim 1, wherein the binarizing process adopts a local binarizing method, the local binarizing method firstly calculates an average gray value of the second gray image, then arbitrarily selects one gray value to divide a gray histogram of the second gray image into two parts of a foreground color and a background color, respectively calculates the average gray value of the foreground color, the average gray value of the background color, a proportion of the number of pixels of the foreground color to the total number of pixels, and a proportion of the number of pixels of the background color to the total number of pixels, and finally calculates the variance according to the following formula:
in the method, in the process of the invention,for variance->For the proportion of foreground pixels to total pixels, +.>As the average gray value of the foreground color,is the average gray value of the second gray image, for example>For the proportion of the number of background-color pixels to the total number of pixels, < >>An average gray value for the background color;
selecting different arbitrary gray values to calculate varianceSo that the variance->The maximum gray value is the optimal gray threshold value of the local binarization method, the gray of the pixel point with the gray value larger than the optimal gray threshold value is set to 255, and the gray of the pixel point with the gray value not larger than the optimal gray threshold value is set to 0.
6. The method for detecting an online foreign object line according to claim 1, wherein the morphological etching process uses an omnidirectional etching method, the omnidirectional etching traverses a detection area, and the gray values of a target pixel and four pixels adjacent to each other are considered to confirm whether the target pixel is consistent with an etched structural element, if so, the pixel point is reserved, otherwise, the pixel point is deleted.
7. The method according to claim 1, wherein the signal range allowed to pass through the band-pass filter is set to be 3-10 as a gaussian filter function sigma value.
8. The on-line foreign matter line detection method according to claim 1, wherein the threshold value in step S3-8 is set to 16 pixels.
9. The method for detecting an online foreign object line according to claim 1, wherein the recognition result after the artificial re-judgment is further used as a training set of a VGG model for training the VGG model.
10. An online foreign matter line detection system, characterized by comprising a detection machine, an image preprocessing module and an image processing module deep learning and re-judging server, wherein an optical imaging module and a material loading and transmitting module are arranged on the detection machine, and the detection system is configured to execute the online foreign matter line detection method according to any one of claims 1-9;
the optical imaging module is used for collecting a glass substrate photo containing a bonding area on the detection machine;
the material loading and conveying module is used for continuously conveying materials for online detection;
the image preprocessing module is used for carrying out Gaussian blur and gray mapping processing on the image acquired by the optical imaging module, and inputting the processed image as a second gray image into the image processing module;
the image processing module detects the second gray level image, identifies a defective area in the second gray level image, and combines the defective areas to form an area image according to the requirement of the deep learning detection module on the size of the input image;
the deep learning re-judging server receives the regional picture and the detection result of the image processing module, adopts the VGG module to identify the regional picture, and trains according to the detection result of the image processing module; and sending the identification result after the identification of the regional picture back to the detection machine.
CN202310694231.6A 2023-06-13 2023-06-13 Online foreign matter line detection method and system Active CN116468726B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202310694231.6A CN116468726B (en) 2023-06-13 2023-06-13 Online foreign matter line detection method and system

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202310694231.6A CN116468726B (en) 2023-06-13 2023-06-13 Online foreign matter line detection method and system

Publications (2)

Publication Number Publication Date
CN116468726A true CN116468726A (en) 2023-07-21
CN116468726B CN116468726B (en) 2023-10-03

Family

ID=87181063

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202310694231.6A Active CN116468726B (en) 2023-06-13 2023-06-13 Online foreign matter line detection method and system

Country Status (1)

Country Link
CN (1) CN116468726B (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117058143A (en) * 2023-10-12 2023-11-14 深圳市合成快捷电子科技有限公司 Intelligent detection method and system for pins of circuit board
CN117557786A (en) * 2024-01-11 2024-02-13 深圳思谋信息科技有限公司 Material quality detection method, device, computer equipment and storage medium

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107451999A (en) * 2017-08-16 2017-12-08 中惠创智无线供电技术有限公司 foreign matter detecting method and device based on image recognition
CN107678192A (en) * 2017-07-16 2018-02-09 中科院成都信息技术股份有限公司 A kind of Mura defects detection method and system based on machine vision
CN109146860A (en) * 2018-08-03 2019-01-04 北京工商大学 Full-automatic mechanical equipment installs leakage detection method and device
CN110276759A (en) * 2019-06-28 2019-09-24 东北大学 A kind of bad line defect diagnostic method of Mobile phone screen based on machine vision
WO2021146935A1 (en) * 2020-01-21 2021-07-29 京东方科技集团股份有限公司 Image defect determining method and apparatus, and electronic device and storage medium
CN113570605A (en) * 2021-09-28 2021-10-29 深圳市绘晶科技有限公司 Defect detection method and system based on liquid crystal display panel
CN115690101A (en) * 2022-12-29 2023-02-03 摩尔线程智能科技(北京)有限责任公司 Defect detection method, defect detection apparatus, electronic device, storage medium, and program product

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107678192A (en) * 2017-07-16 2018-02-09 中科院成都信息技术股份有限公司 A kind of Mura defects detection method and system based on machine vision
CN107451999A (en) * 2017-08-16 2017-12-08 中惠创智无线供电技术有限公司 foreign matter detecting method and device based on image recognition
CN109146860A (en) * 2018-08-03 2019-01-04 北京工商大学 Full-automatic mechanical equipment installs leakage detection method and device
CN110276759A (en) * 2019-06-28 2019-09-24 东北大学 A kind of bad line defect diagnostic method of Mobile phone screen based on machine vision
WO2021146935A1 (en) * 2020-01-21 2021-07-29 京东方科技集团股份有限公司 Image defect determining method and apparatus, and electronic device and storage medium
CN113570605A (en) * 2021-09-28 2021-10-29 深圳市绘晶科技有限公司 Defect detection method and system based on liquid crystal display panel
CN115690101A (en) * 2022-12-29 2023-02-03 摩尔线程智能科技(北京)有限责任公司 Defect detection method, defect detection apparatus, electronic device, storage medium, and program product

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117058143A (en) * 2023-10-12 2023-11-14 深圳市合成快捷电子科技有限公司 Intelligent detection method and system for pins of circuit board
CN117058143B (en) * 2023-10-12 2024-01-26 深圳市合成快捷电子科技有限公司 Intelligent detection method and system for pins of circuit board
CN117557786A (en) * 2024-01-11 2024-02-13 深圳思谋信息科技有限公司 Material quality detection method, device, computer equipment and storage medium
CN117557786B (en) * 2024-01-11 2024-05-10 深圳思谋信息科技有限公司 Material quality detection method, device, computer equipment and storage medium

Also Published As

Publication number Publication date
CN116468726B (en) 2023-10-03

Similar Documents

Publication Publication Date Title
CN116468726B (en) Online foreign matter line detection method and system
CN115908269B (en) Visual defect detection method, visual defect detection device, storage medium and computer equipment
CN107808161B (en) Underwater target identification method based on optical vision
US6463175B1 (en) Structure-guided image processing and image feature enhancement
CN115205223B (en) Visual inspection method and device for transparent object, computer equipment and medium
CN112215810B (en) Fatigue test crack monitoring method and device
Sridevi et al. A survey on monochrome image segmentation methods
CN113781424B (en) Surface defect detection method, device and equipment
CN109584215A (en) A kind of online vision detection system of circuit board
CN109899281A (en) A kind of fracturing unit Weak fault diagnostic method and device based on infrared thermal imagery
CN114820594A (en) Method for detecting edge sealing defects of plate based on image, related equipment and storage medium
Lin et al. Surface defect detection of machined parts based on machining texture direction
CN112308872A (en) Image edge detection method based on multi-scale Gabor first-order derivative
CN112927223A (en) Glass curtain wall detection method based on infrared thermal imager
Zhu et al. A fast PCB hole detection method based on geometric features
Ashoori-Lalimi et al. An efficient method for vehicle license plate detection in complex scenes
CN113705564B (en) Pointer type instrument identification reading method
CN107818583A (en) Cross searching detection method and device
Ahmad et al. A geometric-based method for recognizing overlapping polygonal-shaped and semi-transparent particles in gray tone images
CN116402771A (en) Defect detection method and device and model training method and device
CN113487569B (en) Complex background image defect detection method and system based on combination of frequency domain and space domain
CN112116579B (en) Defect detection method and device for transparent medicine bottle
CN112329572B (en) Rapid static living body detection method and device based on frame and flash point
CN114937003A (en) Multi-type defect detection system and method for glass panel
Meng et al. Defect detection of glassivation passivation parts wafer surface with random texture and different brightness

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant