CN116188475B - Intelligent control method, system and medium for automatic optical detection of appearance defects - Google Patents

Intelligent control method, system and medium for automatic optical detection of appearance defects Download PDF

Info

Publication number
CN116188475B
CN116188475B CN202310494369.1A CN202310494369A CN116188475B CN 116188475 B CN116188475 B CN 116188475B CN 202310494369 A CN202310494369 A CN 202310494369A CN 116188475 B CN116188475 B CN 116188475B
Authority
CN
China
Prior art keywords
defect
information
defects
detected
features
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202310494369.1A
Other languages
Chinese (zh)
Other versions
CN116188475A (en
Inventor
何生茂
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Dezhong Shenzhen Laser Intelligent Technology Co ltd
Original Assignee
Dezhong Shenzhen Laser Intelligent Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Dezhong Shenzhen Laser Intelligent Technology Co ltd filed Critical Dezhong Shenzhen Laser Intelligent Technology Co ltd
Priority to CN202310494369.1A priority Critical patent/CN116188475B/en
Publication of CN116188475A publication Critical patent/CN116188475A/en
Application granted granted Critical
Publication of CN116188475B publication Critical patent/CN116188475B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/84—Systems specially adapted for particular applications
    • G01N21/88—Investigating the presence of flaws or contamination
    • G01N21/8851—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0004—Industrial image inspection
    • G06T7/001—Industrial image inspection using an image reference approach
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/30—Determination of transform parameters for the alignment of images, i.e. image registration
    • G06T7/33—Determination of transform parameters for the alignment of images, i.e. image registration using feature-based methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/25—Determination of region of interest [ROI] or a volume of interest [VOI]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/54—Extraction of image or video features relating to texture
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/56—Extraction of image or video features relating to colour
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
    • G06V10/75—Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries
    • G06V10/757—Matching configurations of points or features
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
    • G06V10/761—Proximity, similarity or dissimilarity measures
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/762—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using clustering, e.g. of similar faces in social networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/80—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level
    • G06V10/806—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level of extracted features
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/84—Systems specially adapted for particular applications
    • G01N21/88—Investigating the presence of flaws or contamination
    • G01N21/8851—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges
    • G01N2021/8883—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges involving the calculation of gauges, generating models
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/84—Systems specially adapted for particular applications
    • G01N21/88—Investigating the presence of flaws or contamination
    • G01N21/8851—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges
    • G01N2021/8887—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges based on image processing techniques
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10004—Still image; Photographic image
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30108—Industrial image inspection

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Multimedia (AREA)
  • Health & Medical Sciences (AREA)
  • Evolutionary Computation (AREA)
  • General Health & Medical Sciences (AREA)
  • Software Systems (AREA)
  • Medical Informatics (AREA)
  • Databases & Information Systems (AREA)
  • Computing Systems (AREA)
  • Artificial Intelligence (AREA)
  • Chemical & Material Sciences (AREA)
  • Quality & Reliability (AREA)
  • Pathology (AREA)
  • Immunology (AREA)
  • Biochemistry (AREA)
  • Analytical Chemistry (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Signal Processing (AREA)
  • Image Analysis (AREA)
  • Investigating Materials By The Use Of Optical Means Adapted For Particular Applications (AREA)

Abstract

The invention discloses an intelligent control method, a system and a medium for automatic optical detection of appearance defects, which comprise the following steps: acquiring image information of an object to be detected, preprocessing the image information, and generating an interested region of the object to be detected; searching a corresponding image template according to the characteristic information of the object to be detected, registering with the region of interest of the object to be detected, and carrying out partial image segmentation according to the registration result to obtain a defect region; constructing a defect identification model, extracting multi-dimensional characteristics of a defect area, and carrying out characteristic fusion on the multi-dimensional characteristics to obtain multi-dimensional fusion characteristics; performing defect identification by utilizing multi-dimensional fusion characteristics, and outputting the types of defects and the technological processes corresponding to the defects; and generating production early warning corresponding to the process flow. The invention improves the detection precision and detection efficiency of the appearance defects by learning the multidimensional characteristics of the defects, and is beneficial to improving the yield of products.

Description

Intelligent control method, system and medium for automatic optical detection of appearance defects
Technical Field
The invention relates to the technical field of appearance defect detection, in particular to an intelligent control method, system and medium for automatic optical detection of appearance defects.
Background
Appearance defect detection refers to quality detection of appearance of an article, and comprises detection of defects, flaws, scratches, pits, cracks and other problems. Appearance defects have great influence on the quality and the attractiveness of the product, so that quality problems and customer complaints caused by the product defects can be effectively avoided by carrying out appearance defect detection. Appearance defect detection is typically achieved by computer vision techniques, using image processing algorithms and machine learning algorithms. For example, segmentation and marking of defects may be performed using image processing techniques, while automatic classification and identification may be performed using machine learning algorithms. Appearance defect detection is widely applied to industrial production, such as the fields of electronic products, automobile parts, foods, medicines and the like.
Aiming at the problems of low detection efficiency, poor consistency, high labor cost and the like of the artificial appearance defects in the current product production, the development of the industry is greatly limited, and the detection equipment adopted by a few manufacturers is difficult to meet the requirements of actual production due to low detection efficiency. Aiming at the automatic optical detection of the appearance defects of the object, the defects in the image are detected and classified based on a machine vision image processing algorithm and a machine learning algorithm, and the appearance defects of the object are identified. The robustness of the feature information extracted by the traditional image processing method is poor, and the detection effect is affected, so that how to identify the defects with high precision through deep learning in the optical detection of the appearance defects is an urgent problem which cannot be solved in real-time detection.
Disclosure of Invention
In order to solve the technical problems, the invention provides an intelligent control method, an intelligent control system and an intelligent control medium for automatic optical detection of appearance defects.
The first aspect of the present invention provides an intelligent control method for automatic optical detection of appearance defects, comprising:
acquiring image information of an object to be detected, preprocessing the image information of the object to be detected, and generating an interested region of the object to be detected;
searching a corresponding image template according to the characteristic information of the object to be detected, registering the image template with an interested region of the object to be detected, and carrying out partial image segmentation according to registration results to obtain a defect region;
constructing a defect identification model, extracting multi-dimensional characteristics of a defect area, and carrying out characteristic fusion on the multi-dimensional characteristics to obtain multi-dimensional fusion characteristics;
training the defect identification model through a defect sample with label information, carrying out defect identification by utilizing the multi-dimensional fusion characteristics, and outputting the types of defects and the technological processes corresponding to the defects;
and analyzing the abnormal information of the production process flow according to the types of the defects and the process flow corresponding to the defects, and generating related early warning information.
In the scheme, a corresponding image template is searched according to characteristic information of an object to be detected, the image template is registered with an interested region of the object to be detected, partial image segmentation is carried out according to a registration result, and a defect region is obtained, specifically:
Extracting processing information of an object to be detected, and searching a multi-angle image template of the object to be detected in a related database according to the processing information;
acquiring shape features in images of different angles from a multi-angle graph template of an object to be detected, simplifying the repeated shape features, determining feature points according to the shape features, and matching the feature points according to self-adaptive traversal;
and generating a matching point pair through a matching result, determining whether matching is successful or not according to proportion information of the matching point pair to all the characteristic points, comparing pixel points after the matching is successful, acquiring deviation pixel points, performing image segmentation according to a preset image block size, and acquiring a defect area.
In the scheme, a defect identification model is constructed, multi-dimensional characteristics of a defect area are extracted, and the multi-dimensional characteristics are subjected to characteristic fusion to obtain multi-dimensional fusion characteristics, specifically:
extracting position features, colors and texture features of the defect region on an object to be detected, and performing similarity calculation within a preset range according to the position features, the colors and the shape features to obtain a similar region with the same size as the defect region;
calculating information entropy of the defect area and the similar area, and taking the information entropy deviation of the defect area and the similar area as the characteristic of the defect area;
Inputting the defect area into a YOLOv5 network, extracting features with different scales by using a feature pyramid, performing feature fusion to generate multi-scale fusion features, and performing multi-core learning on the multi-scale fusion features of the defect area, the color, the texture features and the defect area information entropy;
and determining kernel functions of various features on defect identification by using a voting method, determining the weights of the kernel functions according to the defect identification rate of the single feature, performing kernel fusion on the kernel functions by using linear combination, and outputting multi-dimensional fusion features of the defect region.
In this scheme, through the defect sample that has label information to carry out training to defect identification model, utilize the multidimension degree fusion characteristic to carry out defect identification, the kind of output defect and the technological process that the defect corresponds specifically do:
obtaining a defect sample according to historical defect detection data of an object to be detected, retrieving a historical defect detection result to obtain a defect category of the defect sample, and setting a category label through the defect category to obtain an initial defect sample set;
extracting a production process flow of an object to be detected, dividing the production process flow according to corresponding processing equipment information, constructing a sub-flow sequence, and acquiring configuration abnormality information and historical fault information of the processing equipment;
Matching the time stamp of the configuration abnormal information and the time stamp of the historical fault information with the time stamp of the defect sample, and setting a fault label of the defect sample according to the configuration abnormal information and the historical fault information of the processing equipment if the time deviation is smaller than a preset time threshold;
performing cluster analysis on the defect sample, establishing a corresponding relation between a clustering result and each sub-process in the sub-process sequence, and obtaining an updated defect sample set;
constructing a defect identification model based on deep learning, dividing the defect identification model into a defect classification branch and a defect tracing branch, taking YOLOv5 as a backbone network in the defect classification branch, and carrying out multidimensional feature fusion by a multi-core learning method;
training defect classification branches by using the initial defect sample set, and identifying and classifying defects through training the defect classification branches after reaching standards to obtain defect classification results;
and constructing defect tracing branches through SVM classifiers, constructing an equivalent number of SVM classifiers according to the number of subsequences, training by using the updated defect sample set, inputting defect classification results into the defect tracing branches, and acquiring a sub-flow which leads to defect generation.
In the scheme, the defect sample is subjected to cluster analysis, a corresponding relation between a clustering result and each sub-flow in the sub-flow sequence is established, and an updated defect sample set is obtained, specifically:
Obtaining a fault label of a defect sample, wherein the defect sample has at least one fault label, and the equipment of each sub-process and corresponding marked abnormal information are used as main component directions;
projecting in the main component direction according to configuration abnormality information and historical fault information in the fault label, obtaining characteristic scattered point distribution, obtaining a marked defect sample in each sub-process according to the characteristic scattered point distribution, and taking the marked defect sample as an initial clustering center;
obtaining Euclidean distance from each defect sample in an initial defect sample set to an initial clustering center, distributing the defect sample to the initial clustering center closest to the initial clustering center, and generating a clustering result in each initial clustering center;
continuously updating the clustering center and the clustering result through iterative computation, averaging the defect samples in the clustering result to obtain a new clustering center, and comparing the distance between the defect sample and the belonging clustering center with the distance between the defect sample and the clustering center in the previous time in each iteration;
if the distance between the defect sample and the other cluster centers is smaller than or equal to the distance, the defect sample is reserved in the cluster center, otherwise, the distance between the defect sample and the other cluster centers is calculated, and the defect sample belongs to the cluster center closest to the cluster center;
When the iteration number threshold is met, the clustering result of the last iteration is selected as a final clustering result, and a corresponding relation between the clustering result and each sub-process in the sub-process sequence is established according to the class cluster in the final clustering result, so that an updated defect sample set is obtained.
In the scheme, the abnormal information of the production process flow is analyzed according to the types of the defects and the process flow corresponding to the defects, and relevant early warning information is generated, specifically:
obtaining the type information of the defects and the sub-processes corresponding to the defects, carrying out independent marking on the sub-processes, retrieving standard processing parameters of an object to be detected, and extracting standard processing parameters of the marked sub-processes from the standard processing parameters;
analyzing the parameter deviation of the operation parameters of the processing equipment in the sub-process and the standard processing parameters of the marking sub-process, and generating processing early warning information when the parameter deviation is larger than a preset deviation threshold value, and displaying the early warning information corresponding to the defects in a preset mode;
and generating parameter compensation through parameter deviation, correcting the processing equipment, and optimizing standard processing parameters according to the parameter compensation and defect conditions of the process flow.
The second aspect of the present invention also provides an intelligent control system for automatic optical detection of appearance defects, the system comprising: the intelligent control method for the automatic optical detection of the appearance defects comprises a memory and a processor, wherein the memory comprises an intelligent control method program for the automatic optical detection of the appearance defects, and the intelligent control method program for the automatic optical detection of the appearance defects realizes the following steps when being executed by the processor:
Acquiring image information of an object to be detected, preprocessing the image information of the object to be detected, and generating an interested region of the object to be detected;
searching a corresponding image template according to the characteristic information of the object to be detected, registering the image template with an interested region of the object to be detected, and carrying out partial image segmentation according to registration results to obtain a defect region;
constructing a defect identification model, extracting multi-dimensional characteristics of a defect area, and carrying out characteristic fusion on the multi-dimensional characteristics to obtain multi-dimensional fusion characteristics;
training the defect identification model through a defect sample with label information, carrying out defect identification by utilizing the multi-dimensional fusion characteristics, and outputting the types of defects and the technological processes corresponding to the defects;
and analyzing the abnormal information of the production process flow according to the types of the defects and the process flow corresponding to the defects, and generating related early warning information.
The third aspect of the present invention also provides a computer-readable storage medium, in which a smart control method program for automatic optical detection of an appearance defect is included, which when executed by a processor, implements the steps of a smart control method for automatic optical detection of an appearance defect as described in any one of the above.
The invention discloses an intelligent control method, a system and a medium for automatic optical detection of appearance defects, which comprise the following steps: acquiring image information of an object to be detected, preprocessing the image information, and generating an interested region of the object to be detected; searching a corresponding image template according to the characteristic information of the object to be detected, registering with the region of interest of the object to be detected, and carrying out partial image segmentation according to the registration result to obtain a defect region; extracting multi-dimensional characteristics of the defect area, and carrying out characteristic fusion on the multi-dimensional characteristics to obtain multi-dimensional fusion characteristics; constructing a defect identification model, carrying out defect identification by utilizing multi-dimensional fusion characteristics, and outputting the types of defects and the corresponding technological processes of the defects; and generating production early warning corresponding to the process flow. The invention improves the detection precision and detection efficiency of the appearance defects by learning the multidimensional characteristics of the defects, and is beneficial to improving the yield of products.
Drawings
FIG. 1 is a flow chart of an intelligent control method for automatic optical detection of appearance defects according to the present invention;
FIG. 2 is a flow chart of a method of acquiring multi-dimensional characteristics of a defect region in accordance with the present invention;
FIG. 3 illustrates a flow chart of a method of defect identification using multi-dimensional fusion features in accordance with the present invention;
FIG. 4 shows a block diagram of an intelligent control system for automatic optical detection of defects in appearance in accordance with the present invention.
Detailed Description
In order that the above-recited objects, features and advantages of the present invention will be more clearly understood, a more particular description of the invention will be rendered by reference to the appended drawings and appended detailed description. It should be noted that, in the case of no conflict, the embodiments of the present application and the features in the embodiments may be combined with each other.
In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present invention, however, the present invention may be practiced in other ways than those described herein, and therefore the scope of the present invention is not limited to the specific embodiments disclosed below.
FIG. 1 is a flow chart of an intelligent control method for automatic optical detection of appearance defects according to the present invention.
As shown in fig. 1, a first aspect of the present invention provides an intelligent control method for automatic optical detection of appearance defects, which includes:
s102, obtaining image information of an object to be detected, preprocessing the image information of the object to be detected, and generating an interested region of the object to be detected;
s104, searching a corresponding image template according to the characteristic information of the object to be detected, registering the image template with the region of interest of the object to be detected, and carrying out partial image segmentation according to the registration result to obtain a defect region;
S106, constructing a defect identification model, extracting multi-dimensional characteristics of a defect area, and carrying out characteristic fusion on the multi-dimensional characteristics to obtain multi-dimensional fusion characteristics;
s108, training the defect identification model through a defect sample with label information, carrying out defect identification by utilizing the multi-dimensional fusion characteristics, and outputting the types of defects and the process flows corresponding to the defects;
s110, analyzing abnormal information of the production process flow according to the types of the defects and the process flow corresponding to the defects, and generating related early warning information.
It should be noted that, determining a lighting scheme according to the shape and size information of the object to be measured, lighting through a preset light source, and in the field of machine vision, the types of the commonly used light sources include: the method comprises the steps of debugging and determining shooting parameters of a camera array by using a halogen light source, a fluorescent light source, an infrared light source, an LED light source and the like, obtaining image information of an object to be detected by using the camera array, and carrying out pretreatment such as filtering, denoising, ashing, background removal and the like on the image information to obtain image information only comprising the object to be detected as an interested area;
extracting processing information of an object to be detected, including processing materials, processing patterns and the like, and searching a multi-angle image template of the object to be detected in a related database according to the processing information; acquiring shape features in images of different angles from a multi-angle graph template of an object to be detected, simplifying repeated shape features, determining feature points according to the shape features, selecting a region of interest, and matching the feature points according to self-adaptive traversal; and generating a matching point pair through a matching result, determining whether matching is successful or not according to proportion information of the matching point pair to all the characteristic points, comparing pixel points after the matching is successful, acquiring deviation pixel points, performing image segmentation according to a preset image block size, and acquiring a defect area.
FIG. 2 is a flow chart of a method of the present invention for obtaining multi-dimensional characteristics of a defect area.
According to the embodiment of the invention, a defect identification model is constructed, the multidimensional features of a defect area are extracted, and feature fusion is carried out on the multidimensional features to obtain multidimensional fusion features, specifically:
s202, extracting position features, colors and texture features of a defect region on an object to be detected, and carrying out similarity calculation within a preset range according to the position features, the colors and the shape features to obtain a similar region with the same size as the defect region;
s204, calculating information entropy of the defect area and the similar area, and taking the information entropy deviation of the defect area and the similar area as the characteristic of the defect area;
s206, inputting the defect area into a YOLOv5 network, extracting features with different scales by using a feature pyramid, performing feature fusion to generate multi-scale fusion features, and performing multi-core learning on the multi-scale fusion features of the defect area, the color, the texture features and the defect area information entropy;
and S208, determining kernel functions of various features on defect recognition by using a voting method, determining the weights of the kernel functions according to the defect recognition rate of the single feature, performing kernel fusion on the kernel functions by using linear combination, and outputting multi-dimensional fusion features of the defect region.
It should be noted that, the richness of the features is represented by the information entropy of each region of the image, whether defects exist or not can be represented according to the difference of the information entropy, the image information of the defect region is input into a YOLOv5 network, the features are extracted by convolution calculation, the extracted features are input into a feature pyramid structure in the YOLOv5 network, multi-scale semantic features are obtained from top to bottom in the feature map, the features of different scales are subjected to feature fusion to generate multi-scale fusion features, and the multi-scale fusion features, the information entropy deviation features and the color texture features are respectively used as single features for defect identification. Selecting an optimal kernel function for a single feature by utilizing a voting method in multi-kernel learning; the method comprises the steps of obtaining a related image data set through a big data means, carrying out gray level conversion and scale normalization on an image, respectively extracting single features in the image data set, carrying out defect recognition based on the single features according to a preset model, marking according to defect recognition results, determining the recognition result of the single features, determining the recognition rate of the single features, obtaining the weight value of the single features by the ratio of the recognition rate of the single features to the sum of the recognition rates of all the features, training the weight value serving as the weight of a kernel function, carrying out kernel fusion on the kernel function in a linear combination mode, and outputting multi-dimensional fusion features of a defect region.
FIG. 3 illustrates a flow chart of a method of defect identification using multi-dimensional fusion features in accordance with the present invention.
According to the embodiment of the invention, the defect identification model is trained through the defect sample with the label information, the multi-dimensional fusion characteristic is utilized for defect identification, and the types of defects and the process flows corresponding to the defects are output, specifically:
s302, obtaining a defect sample according to historical defect detection data of an object to be detected, retrieving a historical defect detection result to obtain a defect type of the defect sample, and setting a type label through the defect type to obtain an initial defect sample set;
s304, extracting a production process flow of an object to be detected, dividing the production process flow according to corresponding processing equipment information, constructing a sub-flow sequence, and acquiring configuration abnormal information and historical fault information of the processing equipment;
s306, matching the time stamp of the configuration abnormal information and the time stamp of the historical fault information with the time stamp of the defect sample, and setting a fault label of the defect sample according to the configuration abnormal information and the historical fault information of the processing equipment if the time deviation is smaller than a preset time threshold;
s308, carrying out cluster analysis on the defect sample, establishing a corresponding relation between a clustering result and each sub-process in the sub-process sequence, and obtaining an updated defect sample set;
S310, constructing a defect recognition model based on deep learning, dividing the defect recognition model into a defect classification branch and a defect tracing branch, taking YOLOv5 as a backbone network in the defect classification branch, and carrying out multidimensional feature fusion by a multi-core learning method;
s312, training defect classification branches by using the initial defect sample set, and identifying and classifying defects through training the defect classification branches after reaching standards to obtain defect classification results;
s314, constructing defect tracing branches through SVM classifiers, constructing an equivalent number of SVM classifiers according to the number of subsequences, training by using the updated defect sample set, inputting defect classification results into the defect tracing branches, and obtaining a sub-flow which leads to defect generation.
It should be noted that, the defect recognition model is divided into a defect classification branch and a defect tracing branch, the target detection loss of defect recognition classification is divided into a classification loss and a regression loss by using YOLOv5 as a main network in the defect classification branch, and the aspect ratio of the anchor frame is considered by using CIoU loss as a loss, so that the regression speed and the accuracy of the anchor frame are effectively improved.
The fault label of a defect sample is obtained, wherein the defect sample has at least one fault label, and equipment of each sub-flow and corresponding marked abnormal information are taken as main component directions; projecting in the main component direction according to configuration abnormality information and historical fault information in the fault label, obtaining characteristic scattered point distribution, obtaining a marked defect sample in each sub-process according to the characteristic scattered point distribution, and taking the marked defect sample as an initial clustering center; obtaining Euclidean distance from each defect sample in an initial defect sample set to an initial clustering center, distributing the defect sample to the initial clustering center closest to the initial clustering center, and generating a clustering result in each initial clustering center; continuously updating the clustering center and the clustering result through iterative computation, averaging the defect samples in the clustering result to obtain a new clustering center, and comparing the distance between the defect sample and the belonging clustering center with the distance between the defect sample and the clustering center in the previous time in each iteration; if the distance between the defect sample and the other cluster centers is smaller than or equal to the distance, the defect sample is reserved in the cluster center, otherwise, the distance between the defect sample and the other cluster centers is calculated, and the defect sample belongs to the cluster center closest to the cluster center; when the iteration number threshold is met, the clustering result of the last iteration is selected as a final clustering result, and a corresponding relation between the clustering result and each sub-process in the sub-process sequence is established according to the class cluster in the final clustering result, so that an updated defect sample set is obtained.
It should be noted that, the kind information of the defect and the sub-process corresponding to the defect are obtained, the sub-process is marked separately, the standard processing parameters of the object to be measured are searched, and the standard processing parameters of the marked sub-process are extracted from the standard processing parameters; analyzing the parameter deviation of the operation parameters of the processing equipment in the sub-process and the standard processing parameters of the marking sub-process, and generating processing early warning information when the parameter deviation is larger than a preset deviation threshold value, and displaying the early warning information corresponding to the defects in a preset mode; and generating parameter compensation through parameter deviation, correcting the processing equipment, and optimizing standard processing parameters according to the parameter compensation and defect conditions of the process flow.
According to the embodiment of the invention, the defective objects are classified secondarily according to the defect types, specifically:
obtaining the defect type and defect size information of the object with the defect, and setting a defect label of the object with the defect;
dividing the defects into repairable defects and unrepairable defects according to defect types, and setting defect size thresholds of the defect types in the repairable defects;
searching in the defect labels according to the repairable defects, temporarily suspending the defect labels which accord with the defect types corresponding to the repairable defects, extracting the defect size information in the suspended defect labels, and comparing the defect size threshold values;
If the defect label is smaller than or equal to the defect label, performing defect repair on the object with the defect, otherwise, classifying the defect label as an unrepairable defect;
in the repairing process, a similar area with the same size as the defect area is obtained, and the characteristic information of the similar area is used as a repairing reference of the defect area.
FIG. 4 shows a block diagram of an intelligent control system for automatic optical detection of defects in appearance in accordance with the present invention.
The second aspect of the present invention also provides an intelligent control system 4 for automatic optical detection of appearance defects, the system comprising: a memory 41, and a processor 42, wherein the memory includes a smart control method program for automatic optical detection of appearance defects, and the smart control method program for automatic optical detection of appearance defects realizes the following steps when executed by the processor:
acquiring image information of an object to be detected, preprocessing the image information of the object to be detected, and generating an interested region of the object to be detected;
searching a corresponding image template according to the characteristic information of the object to be detected, registering the image template with an interested region of the object to be detected, and carrying out partial image segmentation according to registration results to obtain a defect region;
constructing a defect identification model, extracting multi-dimensional characteristics of a defect area, and carrying out characteristic fusion on the multi-dimensional characteristics to obtain multi-dimensional fusion characteristics;
Training the defect identification model through a defect sample with label information, carrying out defect identification by utilizing the multi-dimensional fusion characteristics, and outputting the types of defects and the technological processes corresponding to the defects;
and analyzing the abnormal information of the production process flow according to the types of the defects and the process flow corresponding to the defects, and generating related early warning information.
It should be noted that, determining a lighting scheme according to the shape and size information of the object to be measured, lighting through a preset light source, and in the field of machine vision, the types of the commonly used light sources include: the method comprises the steps of debugging and determining shooting parameters of a camera array by using a halogen light source, a fluorescent light source, an infrared light source, an LED light source and the like, obtaining image information of an object to be detected by using the camera array, and carrying out pretreatment such as filtering, denoising, ashing, background removal and the like on the image information to obtain image information only comprising the object to be detected as an interested area;
extracting processing information of an object to be detected, including processing materials, processing patterns and the like, and searching a multi-angle image template of the object to be detected in a related database according to the processing information; acquiring shape features in images of different angles from a multi-angle graph template of an object to be detected, simplifying repeated shape features, determining feature points according to the shape features, selecting a region of interest, and matching the feature points according to self-adaptive traversal; and generating a matching point pair through a matching result, determining whether matching is successful or not according to proportion information of the matching point pair to all the characteristic points, comparing pixel points after the matching is successful, acquiring deviation pixel points, performing image segmentation according to a preset image block size, and acquiring a defect area.
According to the embodiment of the invention, a defect identification model is constructed, the multidimensional features of a defect area are extracted, and feature fusion is carried out on the multidimensional features to obtain multidimensional fusion features, specifically:
extracting position features, colors and texture features of the defect region on an object to be detected, and performing similarity calculation within a preset range according to the position features, the colors and the shape features to obtain a similar region with the same size as the defect region;
calculating information entropy of the defect area and the similar area, and taking the information entropy deviation of the defect area and the similar area as the characteristic of the defect area;
inputting the defect area into a YOLOv5 network, extracting features with different scales by using a feature pyramid, performing feature fusion to generate multi-scale fusion features, and performing multi-core learning on the multi-scale fusion features of the defect area, the color, the texture features and the defect area information entropy;
and determining kernel functions of various features on defect identification by using a voting method, determining the weights of the kernel functions according to the defect identification rate of the single feature, performing kernel fusion on the kernel functions by using linear combination, and outputting multi-dimensional fusion features of the defect region.
It should be noted that, the richness of the features is represented by the information entropy of each region of the image, whether defects exist or not can be represented according to the difference of the information entropy, the image information of the defect region is input into a YOLOv5 network, the features are extracted by convolution calculation, the extracted features are input into a feature pyramid structure in the YOLOv5 network, multi-scale semantic features are obtained from top to bottom in the feature map, the features of different scales are subjected to feature fusion to generate multi-scale fusion features, and the multi-scale fusion features, the information entropy deviation features and the color texture features are respectively used as single features for defect identification. Selecting an optimal kernel function for a single feature by utilizing a voting method in multi-kernel learning; the method comprises the steps of obtaining a related image data set through a big data means, carrying out gray level conversion and scale normalization on an image, respectively extracting single features in the image data set, carrying out defect recognition based on the single features according to a preset model, marking according to defect recognition results, determining the recognition result of the single features, determining the recognition rate of the single features, obtaining the weight value of the single features by the ratio of the recognition rate of the single features to the sum of the recognition rates of all the features, training the weight value serving as the weight of a kernel function, carrying out kernel fusion on the kernel function in a linear combination mode, and outputting multi-dimensional fusion features of a defect region.
According to the embodiment of the invention, the defect identification model is trained through the defect sample with the label information, the multi-dimensional fusion characteristic is utilized for defect identification, and the types of defects and the process flows corresponding to the defects are output, specifically:
obtaining a defect sample according to historical defect detection data of an object to be detected, retrieving a historical defect detection result to obtain a defect category of the defect sample, and setting a category label through the defect category to obtain an initial defect sample set;
extracting a production process flow of an object to be detected, dividing the production process flow according to corresponding processing equipment information, constructing a sub-flow sequence, and acquiring configuration abnormality information and historical fault information of the processing equipment;
matching the time stamp of the configuration abnormal information and the time stamp of the historical fault information with the time stamp of the defect sample, and setting a fault label of the defect sample according to the configuration abnormal information and the historical fault information of the processing equipment if the time deviation is smaller than a preset time threshold;
performing cluster analysis on the defect sample, establishing a corresponding relation between a clustering result and each sub-process in the sub-process sequence, and obtaining an updated defect sample set;
constructing a defect identification model based on deep learning, dividing the defect identification model into a defect classification branch and a defect tracing branch, taking YOLOv5 as a backbone network in the defect classification branch, and carrying out multidimensional feature fusion by a multi-core learning method;
Training defect classification branches by using the initial defect sample set, and identifying and classifying defects through training the defect classification branches after reaching standards to obtain defect classification results;
and constructing defect tracing branches through SVM classifiers, constructing an equivalent number of SVM classifiers according to the number of subsequences, training by using the updated defect sample set, inputting defect classification results into the defect tracing branches, and acquiring a sub-flow which leads to defect generation.
It should be noted that, the defect recognition model is divided into a defect classification branch and a defect tracing branch, the target detection loss of defect recognition classification is divided into a classification loss and a regression loss by using YOLOv5 as a main network in the defect classification branch, and the aspect ratio of the anchor frame is considered by using CIoU loss as a loss, so that the regression speed and the accuracy of the anchor frame are effectively improved.
The fault label of a defect sample is obtained, wherein the defect sample has at least one fault label, and equipment of each sub-flow and corresponding marked abnormal information are taken as main component directions; projecting in the main component direction according to configuration abnormality information and historical fault information in the fault label, obtaining characteristic scattered point distribution, obtaining a marked defect sample in each sub-process according to the characteristic scattered point distribution, and taking the marked defect sample as an initial clustering center; obtaining Euclidean distance from each defect sample in an initial defect sample set to an initial clustering center, distributing the defect sample to the initial clustering center closest to the initial clustering center, and generating a clustering result in each initial clustering center; continuously updating the clustering center and the clustering result through iterative computation, averaging the defect samples in the clustering result to obtain a new clustering center, and comparing the distance between the defect sample and the belonging clustering center with the distance between the defect sample and the clustering center in the previous time in each iteration; if the distance between the defect sample and the other cluster centers is smaller than or equal to the distance, the defect sample is reserved in the cluster center, otherwise, the distance between the defect sample and the other cluster centers is calculated, and the defect sample belongs to the cluster center closest to the cluster center; when the iteration number threshold is met, the clustering result of the last iteration is selected as a final clustering result, and a corresponding relation between the clustering result and each sub-process in the sub-process sequence is established according to the class cluster in the final clustering result, so that an updated defect sample set is obtained.
It should be noted that, the kind information of the defect and the sub-process corresponding to the defect are obtained, the sub-process is marked separately, the standard processing parameters of the object to be measured are searched, and the standard processing parameters of the marked sub-process are extracted from the standard processing parameters; analyzing the parameter deviation of the operation parameters of the processing equipment in the sub-process and the standard processing parameters of the marking sub-process, and generating processing early warning information when the parameter deviation is larger than a preset deviation threshold value, and displaying the early warning information corresponding to the defects in a preset mode; and generating parameter compensation through parameter deviation, correcting the processing equipment, and optimizing standard processing parameters according to the parameter compensation and defect conditions of the process flow.
The third aspect of the present invention also provides a computer-readable storage medium, in which a smart control method program for automatic optical detection of an appearance defect is included, which when executed by a processor, implements the steps of a smart control method for automatic optical detection of an appearance defect as described in any one of the above.
In the several embodiments provided in this application, it should be understood that the disclosed apparatus and method may be implemented in other ways. The above described device embodiments are only illustrative, e.g. the division of the units is only one logical function division, and there may be other divisions in practice, such as: multiple units or components may be combined or may be integrated into another system, or some features may be omitted, or not performed. In addition, the various components shown or discussed may be coupled or directly coupled or communicatively coupled to each other via some interface, whether indirectly coupled or communicatively coupled to devices or units, whether electrically, mechanically, or otherwise.
The units described above as separate components may or may not be physically separate, and components shown as units may or may not be physical units; can be located in one place or distributed to a plurality of network units; some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
In addition, each functional unit in each embodiment of the present invention may be integrated in one processing unit, or each unit may be separately used as one unit, or two or more units may be integrated in one unit; the integrated units may be implemented in hardware or in hardware plus software functional units.
Those of ordinary skill in the art will appreciate that: all or part of the steps for implementing the above method embodiments may be implemented by hardware related to program instructions, and the foregoing program may be stored in a computer readable storage medium, where the program, when executed, performs steps including the above method embodiments; and the aforementioned storage medium includes: a mobile storage device, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), a magnetic disk or an optical disk, or the like, which can store program codes.
Alternatively, the above-described integrated units of the present invention may be stored in a computer-readable storage medium if implemented in the form of software functional modules and sold or used as separate products. Based on such understanding, the technical solutions of the embodiments of the present invention may be embodied in essence or a part contributing to the prior art in the form of a software product stored in a storage medium, including several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the methods described in the embodiments of the present invention. And the aforementioned storage medium includes: a removable storage device, ROM, RAM, magnetic or optical disk, or other medium capable of storing program code.
The foregoing is merely illustrative of the present invention, and the present invention is not limited thereto, and any person skilled in the art will readily recognize that variations or substitutions are within the scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.

Claims (8)

1. An intelligent control method for automatic optical detection of appearance defects is characterized by comprising the following steps:
acquiring image information of an object to be detected, preprocessing the image information of the object to be detected, and generating an interested region of the object to be detected;
searching a corresponding image template according to the characteristic information of the object to be detected, registering the image template with an interested region of the object to be detected, and carrying out partial image segmentation according to registration results to obtain a defect region;
constructing a defect identification model, extracting multi-dimensional characteristics of a defect area, and carrying out characteristic fusion on the multi-dimensional characteristics to obtain multi-dimensional fusion characteristics;
training the defect identification model through a defect sample with label information, carrying out defect identification by utilizing the multi-dimensional fusion characteristics, and outputting the types of defects and the technological processes corresponding to the defects;
analyzing abnormal information of the production process flow according to the types of the defects and the process flow corresponding to the defects, and generating related early warning information;
constructing a defect identification model, extracting multi-dimensional characteristics of a defect area, and carrying out characteristic fusion on the multi-dimensional characteristics to obtain multi-dimensional fusion characteristics, wherein the multi-dimensional fusion characteristics are specifically as follows:
extracting position features, colors and texture features of the defect region on an object to be detected, and performing similarity calculation within a preset range according to the position features, the colors and the shape features to obtain a similar region with the same size as the defect region;
Calculating information entropy of the defect area and the similar area, and taking the information entropy deviation of the defect area and the similar area as the characteristic of the defect area;
inputting the defect area into a YOLOv5 network, extracting features with different scales by using a feature pyramid, performing feature fusion to generate multi-scale fusion features, and performing multi-core learning on the multi-scale fusion features of the defect area, the color, the texture features and the defect area information entropy;
and determining kernel functions of various features on defect identification by using a voting method, determining weights of the kernel functions according to defect identification rates of single features, performing kernel fusion on the kernel functions by using linear combination, and outputting multi-dimensional fusion features of defect areas.
2. The intelligent control method for automatic optical detection of appearance defects according to claim 1, wherein the method is characterized in that the corresponding image template is retrieved according to the characteristic information of the object to be detected, the image template is registered with the region of interest of the object to be detected, and the local image segmentation is performed according to the registration result to obtain the defect region, specifically:
extracting processing information of an object to be detected, and searching a multi-angle image template of the object to be detected in a related database according to the processing information;
Acquiring shape features in images of different angles from a multi-angle graph template of an object to be detected, simplifying the repeated shape features, determining feature points according to the shape features, and matching the feature points according to self-adaptive traversal;
and generating a matching point pair through a matching result, determining whether matching is successful or not according to proportion information of the matching point pair to all the characteristic points, comparing pixel points after the matching is successful, acquiring deviation pixel points, performing image segmentation according to a preset image block size, and acquiring a defect area.
3. The intelligent control method for automatic optical detection of appearance defects according to claim 1, wherein the defect identification model is trained by using defect samples with label information, the defect identification is performed by using the multi-dimensional fusion characteristics, and the types of defects and the technological processes corresponding to the defects are specifically as follows:
obtaining a defect sample according to historical defect detection data of an object to be detected, retrieving a historical defect detection result to obtain a defect category of the defect sample, and setting a category label through the defect category to obtain an initial defect sample set;
extracting a production process flow of an object to be detected, dividing the production process flow according to corresponding processing equipment information, constructing a sub-flow sequence, and acquiring configuration abnormality information and historical fault information of the processing equipment;
Matching the time stamp of the configuration abnormal information and the time stamp of the historical fault information with the time stamp of the defect sample, and setting a fault label of the defect sample according to the configuration abnormal information and the historical fault information of the processing equipment if the time deviation is smaller than a preset time threshold;
performing cluster analysis on the defect sample, establishing a corresponding relation between a clustering result and each sub-process in the sub-process sequence, and obtaining an updated defect sample set;
constructing a defect identification model based on deep learning, dividing the defect identification model into a defect classification branch and a defect tracing branch, taking YOLOv5 as a backbone network in the defect classification branch, and carrying out multidimensional feature fusion by a multi-core learning method;
training defect classification branches by using the initial defect sample set, and identifying and classifying defects through training the defect classification branches after reaching standards to obtain defect classification results;
and constructing defect tracing branches through SVM classifiers, constructing an equivalent number of SVM classifiers according to the number of subsequences, training by using the updated defect sample set, inputting defect classification results into the defect tracing branches, and acquiring a sub-flow which leads to defect generation.
4. The intelligent control method for automatic optical detection of appearance defects according to claim 3, wherein the method comprises the steps of performing cluster analysis on the defect samples, establishing a corresponding relation between a cluster result and each sub-process in the sub-process sequence, and obtaining an updated defect sample set, wherein the method comprises the following specific steps:
Obtaining a fault label of a defect sample, wherein the defect sample has at least one fault label, and the equipment of each sub-process and corresponding marked abnormal information are used as main component directions;
projecting in the main component direction according to configuration abnormality information and historical fault information in the fault label, obtaining characteristic scattered point distribution, obtaining a marked defect sample in each sub-process according to the characteristic scattered point distribution, and taking the marked defect sample as an initial clustering center;
obtaining Euclidean distance from each defect sample in an initial defect sample set to an initial clustering center, distributing the defect sample to the initial clustering center closest to the initial clustering center, and generating a clustering result in each initial clustering center;
continuously updating the clustering center and the clustering result through iterative computation, averaging the defect samples in the clustering result to obtain a new clustering center, and comparing the distance between the defect sample and the belonging clustering center with the distance between the defect sample and the clustering center in the previous time in each iteration;
if the distance between the defect sample and the other cluster centers is smaller than or equal to the distance, the defect sample is reserved in the cluster center, otherwise, the distance between the defect sample and the other cluster centers is calculated, and the defect sample belongs to the cluster center closest to the cluster center;
When the iteration number threshold is met, the clustering result of the last iteration is selected as a final clustering result, and a corresponding relation between the clustering result and each sub-process in the sub-process sequence is established according to the class cluster in the final clustering result, so that an updated defect sample set is obtained.
5. The intelligent control method for automatic optical detection of appearance defects according to claim 1, wherein the method is characterized in that the abnormal information of the production process flow is analyzed according to the types of the defects and the process flow corresponding to the defects, and related early warning information is generated, specifically:
obtaining the type information of the defects and the sub-processes corresponding to the defects, carrying out independent marking on the sub-processes, retrieving standard processing parameters of an object to be detected, and extracting standard processing parameters of the marked sub-processes from the standard processing parameters;
analyzing the parameter deviation of the operation parameters of the processing equipment in the sub-process and the standard processing parameters of the marking sub-process, and generating processing early warning information when the parameter deviation is larger than a preset deviation threshold value, and displaying the early warning information corresponding to the defects in a preset mode;
and generating parameter compensation through parameter deviation, correcting the processing equipment, and optimizing standard processing parameters according to the parameter compensation and defect conditions of the process flow.
6. An intelligent control system for automatic optical detection of appearance defects, the system comprising: the intelligent control method for the automatic optical detection of the appearance defects comprises a memory and a processor, wherein the memory comprises an intelligent control method program for the automatic optical detection of the appearance defects, and the intelligent control method program for the automatic optical detection of the appearance defects realizes the following steps when being executed by the processor:
acquiring image information of an object to be detected, preprocessing the image information of the object to be detected, and generating an interested region of the object to be detected;
searching a corresponding image template according to the characteristic information of the object to be detected, registering the image template with an interested region of the object to be detected, and carrying out partial image segmentation according to registration results to obtain a defect region;
constructing a defect identification model, extracting multi-dimensional characteristics of a defect area, and carrying out characteristic fusion on the multi-dimensional characteristics to obtain multi-dimensional fusion characteristics;
training the defect identification model through a defect sample with label information, carrying out defect identification by utilizing the multi-dimensional fusion characteristics, and outputting the types of defects and the technological processes corresponding to the defects;
analyzing abnormal information of the production process flow according to the types of the defects and the process flow corresponding to the defects, and generating related early warning information;
Constructing a defect identification model, extracting multi-dimensional characteristics of a defect area, and carrying out characteristic fusion on the multi-dimensional characteristics to obtain multi-dimensional fusion characteristics, wherein the multi-dimensional fusion characteristics are specifically as follows:
extracting position features, colors and texture features of the defect region on an object to be detected, and performing similarity calculation within a preset range according to the position features, the colors and the shape features to obtain a similar region with the same size as the defect region;
calculating information entropy of the defect area and the similar area, and taking the information entropy deviation of the defect area and the similar area as the characteristic of the defect area;
inputting the defect area into a YOLOv5 network, extracting features with different scales by using a feature pyramid, performing feature fusion to generate multi-scale fusion features, and performing multi-core learning on the multi-scale fusion features of the defect area, the color, the texture features and the defect area information entropy;
and determining kernel functions of various features on defect identification by using a voting method, determining weights of the kernel functions according to defect identification rates of single features, performing kernel fusion on the kernel functions by using linear combination, and outputting multi-dimensional fusion features of defect areas.
7. The intelligent control system for automatic optical inspection of appearance defects according to claim 6, wherein the defect recognition model is trained by using defect samples with label information, and the multi-dimensional fusion features are used for defect recognition, and the types of defects and the technological processes corresponding to the defects are specifically as follows:
Obtaining a defect sample according to historical defect detection data of an object to be detected, retrieving a historical defect detection result to obtain a defect category of the defect sample, and setting a category label through the defect category to obtain an initial defect sample set;
extracting a production process flow of an object to be detected, dividing the production process flow according to corresponding processing equipment information, constructing a sub-flow sequence, and acquiring configuration abnormality information and historical fault information of the processing equipment;
matching the time stamp of the configuration abnormal information and the time stamp of the historical fault information with the time stamp of the defect sample, and setting a fault label of the defect sample according to the configuration abnormal information and the historical fault information of the processing equipment if the time deviation is smaller than a preset time threshold;
performing cluster analysis on the defect sample, establishing a corresponding relation between a clustering result and each sub-process in the sub-process sequence, and obtaining an updated defect sample set;
constructing a defect identification model based on deep learning, dividing the defect identification model into a defect classification branch and a defect tracing branch, taking YOLOv5 as a backbone network in the defect classification branch, and carrying out multidimensional feature fusion by a multi-core learning method;
training defect classification branches by using the initial defect sample set, and identifying and classifying defects through training the defect classification branches after reaching standards to obtain defect classification results;
And constructing defect tracing branches through SVM classifiers, constructing an equivalent number of SVM classifiers according to the number of subsequences, training by using the updated defect sample set, inputting defect classification results into the defect tracing branches, and acquiring a sub-flow which leads to defect generation.
8. A computer-readable storage medium, characterized by: the computer readable storage medium comprises a smart control method program for automatic optical detection of appearance defects, which when executed by a processor, implements the smart control method steps for automatic optical detection of appearance defects according to any one of claims 1 to 5.
CN202310494369.1A 2023-05-05 2023-05-05 Intelligent control method, system and medium for automatic optical detection of appearance defects Active CN116188475B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202310494369.1A CN116188475B (en) 2023-05-05 2023-05-05 Intelligent control method, system and medium for automatic optical detection of appearance defects

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202310494369.1A CN116188475B (en) 2023-05-05 2023-05-05 Intelligent control method, system and medium for automatic optical detection of appearance defects

Publications (2)

Publication Number Publication Date
CN116188475A CN116188475A (en) 2023-05-30
CN116188475B true CN116188475B (en) 2023-07-25

Family

ID=86442706

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202310494369.1A Active CN116188475B (en) 2023-05-05 2023-05-05 Intelligent control method, system and medium for automatic optical detection of appearance defects

Country Status (1)

Country Link
CN (1) CN116188475B (en)

Families Citing this family (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116501001B (en) * 2023-06-27 2023-09-05 江苏宝孚克新能源科技有限公司 Flexible aluminum alloy cable production process optimization control method and system
CN116645362B (en) * 2023-06-29 2024-01-23 日照鲁光电子科技有限公司 Intelligent quality detection method and system for silicon carbide wafer
CN116523909B (en) * 2023-06-30 2023-09-29 广州市斯睿特智能科技有限公司 Visual detection method and system for appearance of automobile body
CN116934837B (en) * 2023-09-13 2023-11-24 上海超群检测科技股份有限公司 Object detection processing method, device, system, electronic equipment and storage medium
CN116953196B (en) * 2023-09-20 2023-12-05 上海飞挺管业制造有限公司 Defect detection and safety state assessment method for steel tee joint
CN117654907A (en) * 2023-11-29 2024-03-08 嘉兴嘉视自动化科技有限公司 Automatic eliminating method and system for strip detector

Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2022053001A1 (en) * 2020-09-10 2022-03-17 上海航天精密机械研究所 Weld seam internal defect intelligent detection device and method, and medium

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103868935A (en) * 2014-02-14 2014-06-18 中国科学院合肥物质科学研究院 Cigarette appearance quality detection method based on computer vision
CN108982508B (en) * 2018-05-23 2020-11-24 江苏农林职业技术学院 Plastic package IC chip defect detection method based on feature template matching and deep learning
CN115791822A (en) * 2022-11-08 2023-03-14 磐柔(厦门)工业智能有限公司 Visual detection algorithm and detection system for wafer surface defects
CN115880248B (en) * 2022-12-13 2024-02-09 哈尔滨耐是智能科技有限公司 Surface scratch defect identification method and visual detection equipment

Patent Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2022053001A1 (en) * 2020-09-10 2022-03-17 上海航天精密机械研究所 Weld seam internal defect intelligent detection device and method, and medium

Also Published As

Publication number Publication date
CN116188475A (en) 2023-05-30

Similar Documents

Publication Publication Date Title
CN116188475B (en) Intelligent control method, system and medium for automatic optical detection of appearance defects
CN111179251B (en) Defect detection system and method based on twin neural network and by utilizing template comparison
CN108765412B (en) Strip steel surface defect classification method
CN110148130B (en) Method and device for detecting part defects
CN108734184B (en) Method and device for analyzing sensitive image
CN108764361B (en) Working condition identification method of indicator diagram of beam-pumping unit based on integrated learning
TW202013248A (en) Method and apparatus for vehicle damage identification
CN111242899B (en) Image-based flaw detection method and computer-readable storage medium
Liu et al. Unsupervised segmentation and elm for fabric defect image classification
CN111914902B (en) Traditional Chinese medicine identification and surface defect detection method based on deep neural network
CN108460344A (en) Dynamic area intelligent identifying system in screen and intelligent identification Method
CN109886932A (en) Gear ring of wheel speed sensor detection method of surface flaw based on SVM
CN111161237A (en) Fruit and vegetable surface quality detection method, storage medium and sorting device thereof
CN115661160B (en) Panel defect detection method, system, device and medium
CN112907561A (en) Notebook appearance flaw detection method based on deep learning
CN113591948A (en) Defect pattern recognition method and device, electronic equipment and storage medium
CN115147363A (en) Image defect detection and classification method and system based on deep learning algorithm
CN114066848A (en) FPCA appearance defect visual inspection system
CN117392042A (en) Defect detection method, defect detection apparatus, and storage medium
Zhang et al. Fabric defect detection based on visual saliency map and SVM
CN112836724A (en) Object defect recognition model training method and device, electronic equipment and storage medium
CN110458231B (en) Ceramic product detection method, device and equipment
Jahangir Alam et al. Analysis of a printed complex image quality checking method of fabric cloth for development of an automated quality checking system
CN116664540A (en) Rubber sealing ring surface defect detection method based on Gaussian line detection
CN113610831B (en) Wood defect detection method based on computer image technology and transfer learning

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant
PE01 Entry into force of the registration of the contract for pledge of patent right

Denomination of invention: A Smart Control Method, System, and Medium for Automatic Optical Detection of Appearance Defects

Effective date of registration: 20231226

Granted publication date: 20230725

Pledgee: Bank of Communications Limited Shenzhen Branch

Pledgor: Dezhong (Shenzhen) laser Intelligent Technology Co.,Ltd.

Registration number: Y2023980074404

PE01 Entry into force of the registration of the contract for pledge of patent right