CN115272249B

CN115272249B - Defect detection method, device, computer equipment and storage medium

Info

Publication number: CN115272249B
Application number: CN202210915080.8A
Authority: CN
Inventors: 徐尚; 聂强; 林愉欢; 刘永; 汪铖杰; 吴永坚
Original assignee: Tencent Technology Shenzhen Co Ltd
Current assignee: Tencent Technology Shenzhen Co Ltd
Priority date: 2022-08-01
Filing date: 2022-08-01
Publication date: 2024-07-09
Anticipated expiration: 2042-08-01
Also published as: CN115272249A

Abstract

The embodiment of the application discloses a defect detection method, a defect detection device, computer equipment and a storage medium; the embodiment of the application can be applied to various scenes such as industrial detection, artificial intelligence and the like. The embodiment of the application can acquire at least one component image of the electronic component and shooting angle information corresponding to the component image; detecting a defect area of the component image to obtain a defect area in the component image; carrying out defect information identification on the defect area to obtain defect information corresponding to the defect area; combining shooting angle information corresponding to the component images, and carrying out information filtering processing on defect information corresponding to the component images of the electronic components in a plurality of different dimensions to obtain filtered defect information; and judging the electronic component according to the filtered defect information to obtain a defect detection result of the electronic component. By the embodiment of the application, the over-killing rate and the omission rate of defect detection can be effectively reduced, and the quality of defect detection is improved.

Description

Defect detection method, device, computer equipment and storage medium

Technical Field

The present application relates to the field of computer technologies, and in particular, to a defect detection method, a defect detection device, a computer device, and a storage medium.

Background

The industrial defect detection refers to quality detection of industrial components in the production and manufacturing process, and when the components are produced, the quality of the components is improved by the industrial defect detection. For example, a user identification (Subscriber Identity Module, SIM) card is used as a relatively universal part in a mobile phone, various defects are easy to generate in large-scale generation, and the defective part is picked out by a large amount of input quality inspection personnel in a traditional mode and a human eye observation mode is utilized.

Disclosure of Invention

The embodiment of the application provides a defect detection method, a device, computer equipment and a storage medium, which can reduce the omission ratio and the overstock ratio of a defective industrial component, thereby improving the quality of defect detection of the industrial component.

The embodiment of the application provides a defect detection method, which comprises the following steps:

Acquiring at least one component image of an electronic component and shooting angle information corresponding to the component image;

detecting a defect area of the component image to obtain at least one defect area in the component image;

Carrying out defect information identification on the defect area to obtain defect information corresponding to the defect area in the component image;

Combining shooting angle information corresponding to the component images, and carrying out information filtering processing on defect information corresponding to at least one component image of the electronic component in a plurality of different dimensions to obtain filtered defect information of the electronic component;

And judging the electronic component according to the filtered defect information to obtain a defect detection result of the electronic component.

Correspondingly, the embodiment of the application also provides a defect detection device, which comprises:

The electronic device comprises an acquisition unit, a display unit and a display unit, wherein the acquisition unit is used for acquiring at least one component image of an electronic component and shooting angle information corresponding to the component image;

A defect area detection unit, configured to perform defect area detection on the component image, so as to obtain at least one defect area in the component image;

the identification unit is used for carrying out defect information identification on the defect area to obtain defect information corresponding to the defect area in the component image;

The information filtering unit is used for carrying out information filtering processing on defect information corresponding to at least one component image of the electronic component in a plurality of different dimensions in combination with shooting angle information corresponding to the component image to obtain filtered defect information of the electronic component;

And the judging unit is used for judging the electronic component according to the filtered defect information to obtain a defect detection result of the electronic component.

In an embodiment, the information filtering unit may include:

The first information filtering subunit is used for carrying out information filtering on the defect information of the component image in the angle dimension by combining shooting angle information corresponding to the component image to obtain first filtered defect information;

The second information filtering subunit is used for carrying out information filtering on the first filtered defect information in the confidence degree dimension to obtain second filtered defect information;

the third information filtering subunit is used for carrying out information filtering on the second filtered defect information in the grade dimension to obtain third filtered defect information;

And the fourth information filtering subunit is used for carrying out information filtering on the third filtered defect information in the area dimension to obtain the filtered defect information of the electronic component.

In an embodiment, the first information filtering subunit may include:

The information identification module is used for identifying shooting angle information of the component image to obtain defect type range information corresponding to the component image under the shooting angle;

the first filtering module is used for filtering defect types which do not accord with the defect type range information in the defect information of the component image to obtain the first filtered defect information.

In an embodiment, the second information filtering subunit may include:

The first threshold determining unit is used for determining a confidence threshold corresponding to the defect type in the defect information after the first filtering;

the first comparison module is used for comparing the confidence coefficient parameter corresponding to the defect category of the first filtered defect information with a confidence coefficient threshold value to obtain a comparison result;

And the second filtering module is used for filtering the defect types which do not accord with the preset confidence threshold in the first filtered defect information according to the comparison result to obtain the second filtered defect information.

In an embodiment, the third information filtering subunit may include:

A second threshold determining unit, configured to determine a defect level threshold corresponding to a defect type in the second filtered defect information;

the second comparison module is used for comparing the defect grade corresponding to the defect type in the second filtered defect information with the defect grade threshold value to obtain a comparison result;

And the third filtering module is used for filtering the defect types of which the defect grade in the second filtered defect information does not accord with the defect grade threshold according to the comparison result to obtain the third filtered defect information.

In an embodiment, the fourth information filtering subunit may include:

a third threshold determining unit, configured to determine a defect area threshold corresponding to a defect type in the third filtered defect information;

the third comparison module is used for comparing the defect area information corresponding to the defect type in the third filtered defect information with the defect area threshold value to obtain a comparison result;

and the fourth filtering module is used for filtering the defect types of which the defect area information in the third filtered defect information does not accord with the defect area threshold value according to the comparison result to obtain the filtered defect information.

In an embodiment, the defect area detecting unit may include:

the high-resolution sampling subunit is used for carrying out high-resolution sampling on the component image to obtain high-resolution sampling information corresponding to the component image;

The multi-scale feature extraction subunit is used for carrying out multi-scale feature extraction on the high-resolution sampling information to obtain multi-scale features of the component image;

And the defect area detection subunit is used for detecting the defect areas of the multi-scale features of the component image to obtain at least one defect area in the component image.

In an embodiment, the identifying unit may include:

a type identification subunit, configured to perform type identification on the defect area, so that a defect type corresponding to the defect area is obtained;

a defect level identification subunit, configured to identify a defect level of the defect area based on the defect type, so as to obtain a defect level corresponding to the defect type of the defect area;

and the integration subunit is used for integrating the defect area, the defect type corresponding to the defect area and the defect grade to obtain defect information corresponding to the defect area in the component image.

In an embodiment, the type identification subunit may include:

The screening module is used for screening out the screened defect areas from the defect areas according to a preset screening proportion;

the downsampling module is used for downsampling the screened defect area to obtain a downsampled area;

And the type identification module is used for carrying out type identification on the downsampling area to obtain a defect type corresponding to the downsampling area.

In an embodiment, the defect detecting apparatus may further include:

The information acquisition module is used for acquiring a defect detection model to be trained and a component image sample carrying marking information;

the defect detection module is used for predicting defects of the component image samples by utilizing the defect detection model to be trained to obtain predicted defect information corresponding to the component image samples;

The cleaning module is used for cleaning the predicted defect information corresponding to the component image sample according to the labeling information of the component image sample to obtain the cleaned predicted defect information;

and the adjusting module is used for adjusting the defect detection model to be trained by utilizing the post-cleaning predicted defect information to obtain the defect detection model.

Embodiments of the present application also provide a computer program product or computer program comprising computer instructions stored in a computer readable storage medium. The computer instructions are read from a computer-readable storage medium by a processor of a computer device, and executed by the processor, cause the computer device to perform the methods provided in the various alternatives of the above aspect.

Correspondingly, the embodiment of the application also provides a storage medium, wherein the storage medium stores instructions, and the instructions realize the defect detection method provided by any one of the embodiments of the application when being executed by a processor.

The embodiment of the application can acquire at least one component image of the electronic component and shooting angle information corresponding to the component image; detecting a defect area of the component image to obtain at least one defect area in the component image; carrying out defect information identification on the defect area to obtain defect information corresponding to the defect area in the component image; combining shooting angle information corresponding to the component images, and carrying out information filtering processing on defect information corresponding to at least one component image of the electronic component in a plurality of different dimensions to obtain filtered defect information of the electronic component; and judging the electronic component according to the filtered defect information to obtain a defect detection result of the electronic component. By the embodiment of the application, the over-killing rate and the omission rate of defect detection can be effectively reduced, and the quality of defect detection is improved.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are needed in the description of the embodiments will be briefly described below, it being obvious that the drawings in the following description are only some embodiments of the present application, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.

FIG. 1 is a schematic view of a defect detection method according to an embodiment of the present application;

FIG. 2 is a schematic flow chart of a defect detection method according to an embodiment of the present application;

FIG. 3 is a schematic diagram of another embodiment of a defect detection method according to the present application;

FIG. 4 is a schematic diagram of another embodiment of a defect detection method according to the present application;

FIG. 5 is a schematic flow chart of a defect detection method according to an embodiment of the present application;

FIG. 6 is a schematic flow chart of a defect detection method according to an embodiment of the present application;

FIG. 7 is a schematic diagram of a defect detecting apparatus according to an embodiment of the present application;

Fig. 8 is a schematic structural diagram of a computer device according to an embodiment of the present application.

Detailed Description

The following description of the embodiments of the present application will be made clearly and fully with reference to the accompanying drawings, in which embodiments of the application are shown, however, in which embodiments are shown, by way of illustration, only, and not in any way all embodiments of the application. All other embodiments, which can be made by those skilled in the art based on the embodiments of the application without making any inventive effort, are intended to fall within the scope of the application.

The embodiment of the application provides a defect detection method which can be executed by a defect detection device, and the defect detection device can be integrated in computer equipment. The computer device may include at least one of a terminal, a server, and the like. That is, the defect detection method provided by the embodiment of the present application may be executed by a terminal, a server, or both a terminal and a server capable of communicating with each other.

The terminals may include, but are not limited to, smart phones, tablet computers, notebook computers, personal computers (Personal Computer, PCs), smart appliances, wearable electronics, VR/AR devices, vehicle terminals, smart voice interaction devices, and the like.

The server may be an interworking server or a background server among a plurality of heterogeneous systems, may be an independent physical server, may be a server cluster or a distributed system formed by a plurality of physical servers, and may be a cloud server for providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, basic cloud computing services such as big data and an artificial intelligence platform, and the like.

It should be noted that the embodiments of the present application may be applied to various scenarios, including, but not limited to, cloud technology, artificial intelligence, industrial detection, and the like.

In an embodiment, as shown in fig. 1, the defect detecting device may be integrated on a computer device such as a terminal or a server, so as to implement the defect detecting method according to the embodiment of the present application. Specifically, the computer device may acquire at least one component image of the electronic component and shooting angle information corresponding to the component image; detecting a defect area of the component image to obtain at least one defect area in the component image; carrying out defect information identification on the defect area to obtain defect information corresponding to the defect area in the component image; combining shooting angle information corresponding to the component images, and carrying out information filtering processing on defect information corresponding to at least one component image of the electronic component in a plurality of different dimensions to obtain filtered defect information of the electronic component; and judging the electronic component according to the filtered defect information to obtain a defect detection result of the electronic component.

The following detailed description is given, respectively, of the embodiments, and the description sequence of the following embodiments is not to be taken as a limitation of the preferred sequence of the embodiments.

The embodiment of the application will be described from the perspective of a defect detection device, which may be integrated in a computer device, which may be a server or a terminal.

As shown in fig. 2, a defect detection method is provided, and the specific process includes:

101. and acquiring at least one component image of the electronic component and shooting angle information corresponding to the component image.

The components may include electronic components and parts of machines and instruments. For example, the electronic components may include subscriber identity (Subscriber Identity Module, SIM) cards, resistors, capacitors, inductors, potentiometers, valves, relays, integrated circuits, types of circuits, and the like.

The component image may include an image in which an appearance, an internal structure, and the like of the electronic component are recorded.

In an embodiment, in order to improve accuracy of defect detection, the electronic component may be photographed from a plurality of different angles, so as to obtain at least one component image. For example, the electronic component may be photographed from the front, the back, and the side, respectively, to obtain a plurality of component images.

For example, as shown in fig. 3, when the electronic component is a SIM card, the SIM card may be photographed from the front, back, and side, respectively, to obtain a plurality of component images.

In one embodiment, the photographing angle information may be used to describe from which angle the electronic component is photographed. For example, the photographing angle information may be used to illustrate that the SIM card is photographed from the front. For another example, the photographing angle information may be used to illustrate that the SIM card is photographed from the side, and so on.

In an embodiment, the embodiment of the application designs shooting the electronic component from a plurality of angles, wherein each angle is called a specific point position, and finally, whether the electronic component is the electronic component with the defect or the electronic component without the defect is confirmed through the joint detection of the defects on different point position pictures.

In one embodiment, a defective electronic component may refer to an electronic component that is not satisfactory in quality inspection.

For example, defects of the electronic component may include nine types, namely, chipping, scratch, twinkling, bright line, white, electroplating, abnormal color, glue overflow, and grading, and the like.

Wherein Bai Meng, the starting stage belongs to the integral type defect of the electronic component, the electroplating, the heterochromatic and the glue overflow belong to the area type defect, the bright line belongs to the linear type defect, the rest is broken, scratched and the pock belongs to the point type defect.

102. And detecting the defect area of the component image to obtain at least one defect area in the component image.

In an embodiment, after at least one component image of the electronic component is obtained, the at least one component image may be subjected to region detection to obtain at least one defect region of the component image.

For example, 9 component images are obtained, and region detection can be performed on the 9 component images to obtain a defect region corresponding to each component image.

In an embodiment, in order to accurately detect a defect of an electronic component, when the defect detection device detects a defect of an image of the component, the defect detection device detects areas where the component may have defects, thereby avoiding missing defects.

Accordingly, the defective region may include a region detected from the component image that may include a defect. For example, the defect region may or may not include a defect of the component image. For example, 20 defect areas are detected from the component image, wherein 3 of the 20 defect areas are defects including the component image, and 17 areas not including the component image.

In an embodiment, when there are multiple defects in the electronic component, since the areas of the defects may be different, the size of the defect area corresponding to each defect may also be different. For example, among defect areas corresponding to the component image, some defect areas are 20-dimension by 20-dimension, and some defect areas are 14-dimension by 14-dimension. In addition, in the process of detecting the defect area, a plurality of defect areas may all contain the same defect in the electronic component, and the coverage range of the defect areas is different. In this case, the defect areas may be different in size.

In one embodiment, there are various methods for detecting a defective area of the component image, so as to obtain at least one defective area in the component image.

In an embodiment, the defect detection model may be used to detect a defect region in the component image, so as to obtain at least one defect region in the component image.

Wherein the defect detection model is an artificial intelligence model.

Wherein artificial intelligence is the intelligence of simulating, extending and expanding a person using a digital computer or a machine controlled by a digital computer, sensing the environment, obtaining knowledge, and using knowledge to obtain optimal results. In other words, artificial intelligence is an integrated technology of computer science that attempts to understand the essence of intelligence and to produce a new intelligent machine that can react in a similar way to human intelligence. Artificial intelligence, i.e. research on design principles and implementation methods of various intelligent machines, enables the machines to have functions of sensing, reasoning and decision.

The artificial intelligence technology is a comprehensive subject, and relates to the technology with wide fields, namely the technology with a hardware level and the technology with a software level. Artificial intelligence infrastructure technologies generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technologies, operation/interaction systems, mechatronics, and the like. The artificial intelligence software technology mainly comprises a computer vision technology, a voice processing technology, a natural language processing technology, machine learning/deep learning and other directions.

The machine learning (MACHINE LEARNING, ML) is a multi-domain interdisciplinary, and relates to a plurality of disciplines such as probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory and the like. It is specially studied how a computer simulates or implements learning behavior of a human to acquire new knowledge or skills, and reorganizes existing knowledge structures to continuously improve own performance. Machine learning is the core of artificial intelligence, a fundamental approach to letting computers have intelligence, which is applied throughout various areas of artificial intelligence. Machine learning and deep learning typically include techniques such as artificial neural networks, belief networks, reinforcement learning, transfer learning, induction learning, teaching learning, and the like.

For example, the defect detection model may be at least one of a convolutional neural network (Convolutional Neural Networks, CNN), a deconvolution neural network (De-Convolutional Networks, DN), a deep neural network (Deep Neural Networks, DNN), a deep convolutional inverse graph network (Deep Convolutional INVERSE GRAPHICS Networks, DCIGN), a Region-based convolutional network (Region-based Convolutional Networks, RCNN), a Region-based fast convolutional network (fast Region-based Convolutional Networks, FASTER RCNN), a bi-directional codec (Bidirectional Encoder Representations from Transformers, BERT) model, a feature pyramid network (Feature Pyramid Networks, FPN), and a high resolution network (HR-Net), among others.

In an embodiment, the defect detection model may include a feature extractor, a defect region detector, a defect detection head, and a class classification detection head.

The feature extractor and the defect area detector can be used for detecting the defect area of the component image to obtain at least one defect area in the component image.

The defect detection head and the grade classification head can be used for identifying defect information of the defect area to obtain defect information corresponding to the defect area in the component image.

In one embodiment, the feature extractor, defect region detector, defect detection head, and class classification detection head may all be artificial intelligence models.

For example, a backbone network and a multi-scale feature extraction network may be included in the feature extractor. Wherein the backbone network may be an HR-NET network, etc. The multi-scale feature extraction network may be a FPN network, or the like.

For example, the defective area detector may be a convolutional network. For example, the candidate detection box may be a CNN network or a DNN network, or the like.

For example, the defect detection head may be a convolutional network. For example, the defect detection head may be a CNN network or a DNN network, or the like.

For example, the rank classification detection head may be a classifier. For example, the rank classification detection head may be a multi-classifier or a classifier, or the like.

In an embodiment, the feature extractor and the defect region detector in the defect detection model may be used to detect a defect region in the component image, so as to obtain at least one defect region in the component image.

In an embodiment, in order to reduce the omission ratio of defect detection, when the defect area detection is performed on the component image, the defect area is detected as much as possible, so as to avoid missing the defect in the electronic component. In order to detect the defect areas as much as possible, the defect areas of the component image may be sampled by a high resolution processing method, so as to obtain at least one defect area of the component image.

Specifically, the step of detecting a defect area of the component image to obtain at least one defect area in the component image may include:

High-resolution sampling is carried out on the component image, and high-resolution sampling information corresponding to the component image is obtained;

Extracting multi-scale features of the high-resolution sampling information to obtain multi-scale features of the component image;

And detecting the defect areas of the multi-scale features of the component image to obtain at least one defect area in the component image.

In one embodiment, the conventional defect area detection method uses a small to large resolution for sampling, or upsampling and then upsampling for recovery, which may result in information loss. In order to detect the defect areas as much as possible, so as to reduce the omission ratio of defect detection, the application can ensure high resolution when sampling the component images, maintain a multi-resolution to sample the component images in parallel, and exchange information with different resolutions in parallel, so that the features are richer in semantics and more accurate in space.

Specifically, the step of performing high resolution sampling on the component image to obtain high resolution sampling information corresponding to the component image may include:

Carrying out multi-resolution parallel convolution on the component image to obtain a plurality of convolution information corresponding to the component image;

performing multi-resolution fusion on a plurality of convolution information of the component images to obtain fusion convolution information of the component images;

and carrying out downsampling treatment on the fusion convolution information to obtain high-resolution sampling information.

In an embodiment, the component image may be subjected to multi-resolution parallel convolution to obtain a plurality of convolution information corresponding to the component image. For example, the component image may be convolved in parallel using a plurality of different high-dimensional convolution checks to obtain a plurality of convolution information for the component image. When the multi-resolution parallel convolution is carried out on the component images, the convolution information of the component images can be checked by utilizing the high-dimensional convolution to continue sampling so as to form a plurality of sampling branches.

In an embodiment, multiple convolution information of the component images may be fused in multiple resolutions to obtain fused convolution information of the component images. For example, a plurality of sampling branches are formed in the process of multi-resolution parallel convolution, and convolution information formed in each sampling branch can be subjected to cross fusion to obtain fusion convolution information of the component images. There are various ways of cross-fusing. For example, cross-fusion may be achieved by weighting and then adding. For another example, the addition may be performed directly to achieve cross-fusion.

In one embodiment, the fused convolution information may be downsampled to obtain high resolution sampling information. The information compression can be performed on the rich information formed in the above steps by downsampling the fusion convolution information, so as to obtain high-resolution sampling information of the component image. Because the high-resolution sampling information is obtained by changing resolution sampling, the high-resolution sampling information contains rich information of component images, which is beneficial to detection of defect areas.

In an embodiment, in order to detect as many defective areas as possible, multi-scale feature extraction may be performed on the high-resolution sampling information to obtain multi-scale features of the component image, so that more information in the component image may be further mined.

The multi-scale feature extraction of the high-resolution sampling information may refer to feature extraction of the draft resolution sampling information through different scales, so as to obtain multi-scale features of the component image. The multi-scale features of the component image may describe features of the component image from a plurality of different scales. For example, there are features for describing texture features of an image, there are features for describing semantic features of an image, and so on.

Specifically, the step of performing multi-scale feature extraction on the high-resolution sampling information to obtain multi-scale features of the component image may include:

performing information expansion on the high-resolution sampling information to obtain a plurality of expanded sampling information;

The multi-head attention mechanism is utilized to respectively conduct feature extraction on the expanded sampling information, and feature information corresponding to each piece of expanded sampling information is obtained;

and fusing the characteristic information corresponding to each piece of expanded sampling information to obtain the multi-scale characteristic information of the component image.

In an embodiment, the high resolution sampling information may be information-extended to obtain a plurality of extended sampling information. For example, the high-resolution sampling information may be a 125-dimensional matrix, and then the high-resolution sampling information may be expanded into a plurality of matrices with different dimensions, to obtain expanded sampling information. For example, the high resolution sampling information may be expanded into a 120-dimensional by 120-dimensional matrix, a 84-dimensional by 84-dimensional matrix, a 64-dimensional by 64-dimensional matrix, and so on.

And then, the feature extraction can be respectively carried out on the expanded sampling information by utilizing a multi-head attention mechanism, so as to obtain the feature information corresponding to each piece of expanded sampling information. Wherein, multi-head-attention is to use multiple queries to calculate multiple information choices from the input information in parallel. Each focusing on a different part of the input information.

For example, the high resolution sampling information may be expanded into 4 expanded sampling information. Then, the 4 pieces of expanded sampling information can be respectively extracted by utilizing a multi-head attention mechanism generated based on different scales, so that the characteristic information corresponding to each piece of expanded sampling information is obtained.

And then, the characteristic information corresponding to each piece of expanded sampling information can be fused to obtain the multi-scale characteristic information of the component image. For example, each piece of expanded sampling information may be spliced to obtain multi-scale feature information of the component image.

Then, defect area detection can be carried out on the multi-scale characteristics of the component image, and at least one defect area in the component image is obtained. The multi-scale features can be used for explaining texture features, semantic features, light features and the like of the component image, so that defect areas in the component image can be detected as much as possible through the multi-scale features of the component image.

In one embodiment, a defective area in each component image of the electronic component may be detected, via step 102. For example, 9 component images of the electronic component are taken. The defective areas in the 9 component images can be detected, via step 102.

103. And carrying out defect information identification on the defect area to obtain defect information corresponding to the defect area in the component image.

In an embodiment, information identification may be performed on a defect area in each component image of the electronic component, so as to obtain defect information corresponding to the defect area in the component image.

The defect information may include information describing defects in the multi-component image, among others. For example, the defect information may include defect type, defect level, and defect area.

The defect type may indicate whether the defect of the electronic component is a broken spot or scratch, and the like.

The defect level may refer to a severity level corresponding to a defect type. For example, the defect level may include 4 levels, where the first level is the least severe and the fourth level may be the most severe.

Wherein the defect area may refer to the size of the defect. For example, the defective area may be an area of a defective area.

From the defect information it is possible to know which defects are included in the defect area, the severity of the defect and the size of the defect.

In one embodiment, there are various ways to identify defect information of the defect area, so as to obtain defect information corresponding to the defect area in the component image.

In an embodiment, defect information identification may be performed on the defect area by using the defect detection model, so as to obtain defect information corresponding to the defect area in the component image. For example, the defect information of the defect region may be identified by using the defect detection head and the grade classification detection head in the defect detection model, so as to obtain defect information corresponding to the defect region in the component image. For example, assuming that the defect detection head is a CNN network, the defect area may be identified by using the CNN network, so as to obtain a defect type corresponding to the defect area. In addition, the grade classification head can be used for identifying the defect area to obtain the defect grade corresponding to the defect type in the defect area. Then, the defect grade and the defect type can be integrated to obtain defect information corresponding to the defect area.

In one embodiment, before the defect detection model is used to detect the defect area of the component image, the defect detection model to be trained is required to be trained, so as to obtain the defect detection model conforming to the performance.

Specifically, before the step of performing defect area detection on the component image by using the defect detection model to obtain at least one defect area in the component image, the method provided by the embodiment of the application may further include:

Obtaining a defect detection model to be trained and a component image sample carrying labeling information;

performing defect prediction on the component image sample by using a defect detection model to be trained to obtain predicted defect information corresponding to the component image sample;

Cleaning the predicted defect information corresponding to the component image sample according to the labeling information of the component image sample to obtain the cleaned predicted defect information;

and adjusting the defect detection model to be trained by using the predicted defect information after cleaning to obtain the defect detection model.

The defect detection model to be trained can comprise a model which is not trained or still needs to be continuously trained when the requirement is not met.

The component image sample may include training data used in training the defect detection model to be trained. The component image sample can be images obtained by shooting a plurality of electronic components through different angles.

The labeling information can be used for explaining defects of the electronic components in the component image sample. For example, the labeling effect of the component image may be as shown in fig. 4, where BD-QX-S4 may refer to labeling information of the component image. BD may refer to a defect of an electronic component in the component image, which is a burn-in point. The labeling of other defects of the electronic component can be expressed as: the method comprises the steps of breaking points (BD), scratching (GS), pits (MD), bright Lines (LX), bai Meng (BM), electroplating (DD), different colors (YS), glue overflow (YJ) and grading (QJ). QX may refer to imaging of the component image being sharp. In addition to QX, the callout may include MH (blur) and KBQ (obscure). S4 may indicate that the defect level of the broken point is severe. In addition to S4, the defect levels may also include S1-S3, where S1 may refer to the lowest defect level, i.e., the defect is not severe.

In an embodiment, defect prediction may be performed on the component image sample by using a defect detection model to be trained, so as to obtain predicted defect information corresponding to the component image sample. The process of performing defect prediction on the component image sample by using the defect detection model to be trained may refer to step 102 and step 103, and will not be repeated here. The initial predictions can be obtained from the model over a large amount of data, which would require cleaning due to insufficient model capacity. The erroneous label is corrected and the missing label is replenished. The model may then be retrained on the cleaned data. By continuously repeating, the detection capability of the model can be continuously improved, so that the defect detection model with the performance meeting the requirements is obtained.

In an embodiment, the embodiment of the application can detect the defect information from the component image in a hierarchical detection mode. The method of the hierarchical detection may be a detection method from coarse to fine. For example, a defective region in the component image is first roughly located, and at least one defective region in the component image is obtained. Then, further, a defect type of the electronic component is identified from the defect area. On the premise of identifying the defect type, the defect grade corresponding to the defect type of the electronic component can be further identified. And then, integrating the defect area obtained by identification, the defect type corresponding to the defect area and the defect grade to obtain the defect information of the electronic component.

Specifically, the step of identifying defect information of the defect area to obtain defect information corresponding to the defect area in the component image may include:

performing type identification on the defect area to obtain a defect type corresponding to the defect area;

performing defect grade identification on the defect area based on the defect type to obtain a defect grade corresponding to the defect type of the defect area;

And integrating the defect area, the defect type corresponding to the defect area and the defect grade to obtain defect information corresponding to the defect area in the component image.

In an embodiment, to reduce the omission ratio, the area where the component may have defects is detected, so as to avoid missing the defects. Thus, a large number of areas that do not actually include a defect, or areas that include only a small portion of the defect, may be detected from among the defective areas detected by step 102. In order to reduce the over-killing rate, the defect areas need to be screened, so that the number of defect areas which do not comprise defects is balanced, and the defect detection accuracy is improved. For example, in general, a plurality of defective areas may be detected through step 102. The defect areas include a large number of defect areas containing no defects and a small number of defect areas containing defects. For example, 10 defective areas are detected, 8 of which are defective areas not containing defects, and 2 of which are defective areas containing defects. In order to balance the number of defect areas not containing defects and improve the accuracy of defect detection, screening areas can be screened.

Specifically, the step of performing type identification on the defect area to obtain a defect type corresponding to the defect area may include:

screening out the screened defect areas according to a preset screening proportion;

performing downsampling treatment on the screened defect area to obtain a downsampled area;

and performing type recognition on the downsampling area to obtain the defect type corresponding to the downsampling area.

In an embodiment, the defect area after screening may be screened out from the defect areas according to a preset screening ratio. The defect area after screening may refer to an area where defect information identification is required. Wherein, since the defect area includes the defect area including the defect and the defect area not including the defect, the preset screening ratio may be used to describe the ratio between the defect area including the defect and the defect area not including the defect in the defect area after the screening. For example, assuming that the preset screening ratio is 1:3, the ratio between the defect area including the defect and the defect area including no defect in the screened defect area is 1:3.

In an embodiment, when there are multiple defects in the electronic component, since the areas of the defects may be different, the size of the defect area corresponding to each defect may also be different. For example, among defect areas corresponding to the component image, some defect areas are 20-dimension by 20-dimension, and some defect areas are 14-dimension by 14-dimension. In order to facilitate the screening of defect types contained in the defect areas, the screened defect areas can be subjected to downsampling treatment, so that the defect areas are downsampled into downsampled areas with the same dimension, and the defect types are conveniently identified.

Then, the type identification can be performed on the downsampled region, and the defect type corresponding to the downsampled region can be obtained. For example, the type of the downsampled region can be identified by using an artificial intelligence technology such as CNN, etc., so as to obtain the defect type corresponding to the downsampled region.

In an embodiment, after obtaining the defect type corresponding to the defect area, the defect level identification may be performed on the defect area based on the defect type to obtain the defect level corresponding to the defect type of the defect area.

For example, defect level identification may be performed on the downsampled region based on the defect type to obtain a defect level corresponding to the defect type of the defect region. For example, the down-sampling area may be identified by using a classifier to obtain a defect level corresponding to the defect type of the defect area.

In an embodiment, the defect area, and the defect type and the defect level corresponding to the defect area may be integrated to obtain defect information corresponding to the defect area in the component image. For example, the area of the defect area, the defect type corresponding to the defect area and the defect grade can be integrated according to a preset format to obtain defect information corresponding to the defect area in the component image. For example, the preset format may be [ defect area identification, defect area, defect type, defect level ], and then the area of the defect area, the defect type corresponding to the defect area, and the defect level may be integrated according to the preset format. For example, the integrated defect information may be [001,12×12, bd, s4], and so on.

The defect region may be a screened defect region after screening. For example, in step 103, the defect information of all the defect areas obtained in step 102 is not identified, but only the selected defect areas are identified, and the defect information corresponding to the selected defect areas is obtained.

In an embodiment, the defect information may be a multidimensional vector, and the defect information includes defect information corresponding to each of the screened defect areas. For example, there are 7 post-screening defect areas, and the defect information may include information about the 7 post-screening defect areas.

104. And carrying out information filtering processing on defect information corresponding to at least one component image of the electronic component in a plurality of different dimensions by combining shooting angle information corresponding to the component image to obtain filtered defect information of the electronic component.

In an embodiment, since one electronic component is photographed from multiple angles, it is finally necessary to comprehensively determine the result of combining multiple component images, to determine whether the electronic component is a defective component or a component without a defect. All defects in the electronic component, whether slight defects or uncertain defects, can be detected by steps 102 and 103, with the goal of detecting as many defects as possible, thus ensuring a very low omission ratio. The omission ratio may refer to a percentage of undetected defective electronic components in the number of detected electronic components after the automatic defect detection.

While ensuring a very low omission ratio, it is also necessary to ensure a very low overstock ratio, so that the entire defect detection process is high-quality. The over-killing rate refers to the percentage of the number of defect-free electronic components in the detected defective electronic components to the total number of detected electronic components after automatic defect detection.

In one embodiment, the fault detection may be accompanied by the detection of defect information of the component image. For example, a place of an electronic component is not a defect, but is detected as a defect. For another example, defects exist somewhere in the electronic component, but the severity of the defects is low and can be ignored, and at this time the defects can be ignored, etc. Therefore, after the defect information of the component images is obtained, the defect information corresponding to at least one component image of the electronic component can be subjected to information filtering processing in a plurality of different dimensions by combining the shooting angle information corresponding to the component images. By carrying out information filtering processing on the defect information in a plurality of different dimensions, the condition of false detection can be reduced, and thus the over-killing rate is reduced.

In an embodiment, information filtering defect information from a plurality of different dimensions may refer to filtering defect information from a plurality of different aspects. For example, the plurality of dimensions may include an angular dimension, a confidence dimension, a rank dimension, and an area dimension, among others. The defect information can be measured by different methods and different measurement indexes in different dimensions, so that whether the defect information needs to be filtered is judged.

In an embodiment, when the multiple dimensions may include an angle dimension, a confidence dimension, a level dimension, and an area dimension, the step of combining shooting angle information corresponding to component images, performing information filtering processing on defect information corresponding to at least one component image of an electronic component in multiple different dimensions to obtain filtered defect information of the electronic component may include:

combining shooting angle information corresponding to the component image, and carrying out information filtering on defect information of the component image in an angle dimension to obtain first filtered defect information;

Performing information filtering on the first filtered defect information in the confidence degree dimension to obtain second filtered defect information;

Performing information filtering on the second filtered defect information in the grade dimension to obtain third filtered defect information;

And carrying out information filtering on the third filtered defect information in the area dimension to obtain the filtered defect information of the electronic component.

In one embodiment, due to the specificity of angle shooting, certain defects are theoretically not present in a component image taken at a particular angle for that component image. For example, assuming that the electronic component is a SIM card, for a front-side captured component image, some defects are unlikely to appear on the front-side captured component image. Therefore, the defect information of the component image can be subjected to information filtering in the angle dimension by combining shooting angle information corresponding to the component image, so that the first filtered defect information is obtained.

Specifically, the step of "combining shooting angle information corresponding to a component image, performing information filtering on defect information of the component image in an angle dimension to obtain first filtered defect information" may include:

identifying shooting angle information of the component image to obtain defect type range information corresponding to the component image under the shooting angle;

And filtering defect types which do not accord with the defect type range information in the defect information of the component image to obtain first filtered defect information.

In an embodiment, the shooting angle information of the component image may be identified, so as to obtain defect type range information corresponding to the component image under the shooting angle. The defect type range information may indicate a defect type that may occur in the component image under the photographing angle information. For example, when the photographing angle information of the component image is the front, the defect type range information may include a broken dot, a scratch, a pock, and a bright line. That is, for the front-side photographed component image, the defect types that may occur in the component image theoretically include only chipping, scratching, twinkling, and bright lines.

In an embodiment, the defect type that does not conform to the defect type range information in the defect information of the component image may be filtered to obtain the first filtered defect information. For example, the defect type and the defect type range information corresponding to each defect area in the defect information may be matched. When the defect type of the defect area is not the defect type in the defect type range information, the defect area and its corresponding information may be filtered.

In an embodiment, in order to improve accuracy of defect detection, information filtering may be performed on the first filtered defect information in the confidence dimension to obtain second filtered defect information. The performing information filtering on the first filtered defect information in the confidence dimension may refer to performing information filtering on the first filtered defect information according to the confidence of the defect type, so as to obtain second filtered defect information.

The confidence level may be reliability, and refers to the reliability of the predicted object. For example, the confidence of a defect type may refer to how high the confidence of the predicted defect type is. For example, if the confidence of the defect type of the defect area is 99%, it can be said that the confidence of the defect type is high, and it can be said that a defect is indeed present in the defect area. For example, if the defect type of the defect area is 40%, it may be said that the reliability of the defect type is low, and it may be considered that the defect type is not defective or that other types of defects exist, and at this time, the defect type of the defect area may be filtered out, so as to avoid affecting the defect detection result of the electronic component.

Specifically, the step of performing information filtering on the first filtered defect information in the confidence dimension to obtain second filtered defect information may include:

determining a confidence threshold corresponding to the defect type in the first filtered defect information;

comparing the confidence coefficient parameter corresponding to the defect category of the first filtered defect information with a confidence coefficient threshold value to obtain a comparison result;

And filtering the defect types which do not accord with the preset confidence threshold in the first filtered defect information according to the comparison result to obtain the second filtered defect information.

The confidence parameter may be used to describe a confidence level of the defect type correspondence of the defect area. Confidence parameters may be generated by step 103. In step 103, when the defect area is identified, a confidence coefficient parameter for each defect type in the defect area is generated, and then the defect type with the highest confidence coefficient parameter is selected as the defect type corresponding to the defect area.

In one embodiment, because the electronic component may include multiple defect types, the confidence threshold for each defect type may be different. Accordingly, a confidence threshold corresponding to the defect type in the first filtered defect information may be determined. For example, for a defect type dip, its corresponding confidence threshold may be 97%. For another example, for defect type pits, the corresponding confidence threshold may be 98%, and so on.

For example, the confidence parameter corresponding to the defect class of each defect region in the first filtered defect information may be compared to a confidence threshold. If the confidence value corresponding to the defect type of the defect area is larger than the confidence threshold value, the information of the defect area is not filtered. And if the confidence coefficient parameter corresponding to the defect type of the defect area is smaller than the confidence coefficient threshold value, filtering the defect area to obtain second filtered defect information.

In one embodiment, to measure the severity of the defect, the embodiment of the present application also sets a defect level for each defect type. For example, the defect levels may include defect level 1 through defect level 4, where defect level 1 may indicate that the severity of the defect is low and defect level 4 may indicate that the severity of the defect is high. Wherein certain defects may be ignored when their defect level is low. Therefore, the second filtered defect information can be subjected to information filtering in the grade dimension, so that third filtered defect information is obtained. The filtering of the second filtered defect information in the level dimension may refer to filtering the defect level of the defect type in the second filtered defect information to obtain third filtered defect information.

Specifically, the step of performing information filtering on the second filtered defect information in the level dimension to obtain third filtered defect information may include:

determining a defect grade threshold corresponding to the defect type in the second filtered defect information;

Comparing the defect grade corresponding to the defect type in the second filtered defect information with the defect grade threshold value to obtain a comparison result;

and filtering the defect types of which the defect levels do not accord with the defect level threshold in the second filtered defect information according to the comparison result to obtain the third filtered defect information.

For example, the defect level corresponding to the defect type of each region type in the second filtered defect information may be compared with the defect threshold. If the defect level of the defective area is smaller than the defect level threshold, it may be indicated that the defect of the defective area is not serious and may be ignored, so that the information corresponding to the defective area may be filtered. When the defect level of the defective area is greater than or equal to the defect level threshold, it may be indicated that the defect severity of the defective area is high and not negligible.

In one embodiment, when the defect area of some defects on the electronic component is not large, the defects may be ignored. For example, if the defect area is smaller than a certain threshold value, the defect area is not large enough, the defect is not obvious, the defect is negligible, and therefore filtering can be performed.

Therefore, the information filtering can be performed on the third filtered defect information in the area dimension to obtain the filtered defect information of the electronic component. Specifically, the step of performing information filtering on the third filtered defect information in the area dimension to obtain filtered defect information of the electronic component may include:

Determining a defect area threshold corresponding to the defect type in the third filtered defect information;

Comparing the defect area information corresponding to the defect type in the third filtered defect information with a defect area threshold value to obtain a comparison result;

And filtering the defect types of which the defect area information does not accord with the defect area threshold value in the third filtered defect information according to the comparison result to obtain filtered defect information.

The defect area information may be used for the area size corresponding to the defect area. Defect area information of the defect area may be generated by step 102. In step 102, when detecting a defective area of the component image, defective area information corresponding to the defective area is generated.

For example, defect area information corresponding to the defect type may be compared with a defect area threshold. When the defective area information is smaller than the defective area threshold value, the defective area may be filtered out. And when the area information of the defect area is larger than or equal to the area threshold value, the defect area and the corresponding information thereof are reserved.

In an embodiment, the filtered defect information corresponding to each component image of the electronic component may be obtained by performing information filtering on each component image of the electronic component. And then, judging the electronic components according to the filtered defect information corresponding to each component image of the electronic components to obtain a defect detection result of the electronic components.

In one embodiment, the number of incorrect defects is reduced greatly by layer-by-layer filtration, so that the over-killing rate is reduced, and the workload of manually performing re-judgment from defects detected by a machine is reduced.

105. And judging the electronic component according to the filtered defect information to obtain a defect detection result of the electronic component.

In an embodiment, the electronic component may be distinguished by combining the filtered defect information of each component image, so as to obtain a defect detection result of the electronic component. For example, the defect area, the defect type corresponding to the defect area, and the grade corresponding to the defect type remaining after filtering of each component image may be integrated to obtain a defect detection result of whether the electronic component has a defect. For example, if some electronic component images may detect the same defective area, the repeated defective areas may be filtered, leaving only one. If the electronic component has defects, the defect area is positioned, the defect type corresponding to the defect area and the defect grade corresponding to the defect type are what.

The embodiment of the application provides a defect detection method, which comprises the following steps: acquiring at least one component image of an electronic component and shooting angle information corresponding to the component image; detecting a defect area of the component image to obtain at least one defect area in the component image; carrying out defect information identification on the defect area to obtain defect information corresponding to the defect area in the component image; combining shooting angle information corresponding to the component images, and carrying out information filtering processing on defect information corresponding to at least one component image of the electronic component in a plurality of different dimensions to obtain filtered defect information of the electronic component; and judging the electronic component according to the filtered defect information to obtain a defect detection result of the electronic component. In the embodiment of the application, firstly, a region with a defect in a component image is roughly positioned, and at least one defect region in the component image is obtained. Then, further, a defect type of the electronic component is identified from the defect area. On the premise of identifying the defect type, the defect grade corresponding to the defect type of the electronic component can be further identified. By the method, defects in the component image can be detected as much as possible, so that the omission rate of defect detection is reduced. In addition, after detecting the defect information corresponding to the defect area in the component image, the defect information is subjected to information filtering processing in a plurality of different dimensions. Through layer-by-layer filtration, the number of incorrect defects is greatly reduced, so that the over-killing rate is reduced, and the workload of manually performing repeated judgment on the defects detected by a machine is reduced. Therefore, through the embodiment of the application, the over-killing rate and the omission rate of defect detection can be effectively reduced, and the quality of defect detection is improved.

According to the method described in the above embodiments, examples are described in further detail below.

The method of the embodiment of the application will be described by taking the example of integrating the defect detection method on computer equipment.

In one embodiment, as shown in fig. 5, a defect detection method specifically includes the following steps:

201. the computer equipment acquires at least one component image of the electronic component and shooting angle information corresponding to the component image.

It is assumed that the electronic component is a SIM card holder. The SIM card holder is used as a relatively universal part in the mobile phone, various defects are easy to generate in large-scale generation, and the traditional mode utilizes a human eye observation mode to pick out the defective part by inputting a large amount of quality inspection staff, so that the method not only consumes a large amount of manpower, but also has larger influence on the level of the quality inspection staff. By the embodiment of the application, the defect detection of the SIM card holder can be automatically realized.

For example, the SIM card may be photographed from the front, back, and side to obtain a plurality of component images.

202. And the computer equipment detects the defect area of the component image to obtain at least one defect area in the component image.

For example, as shown in fig. 6, a feature extractor and a defect region detector, which may utilize a defect detection model, may be used to detect a defect region in a component image, resulting in at least one defect region in the component image.

For example, a backbone network in a feature extractor and a multi-scale feature extractor may be input to a component image, and multi-scale features of the component image may be obtained, some of which describe texture features of the image and some of which describe semantic features of the image. The candidate detector may then be utilized to detect the defective area through the multi-scale features of the component image.

203. And the computer equipment performs defect information identification on the defect area to obtain defect information corresponding to the defect area in the component image.

For example, as shown in fig. 6, the computer device may use the defect detection head in the defect recognition model to perform type recognition on the defect area, so as to obtain the defect type corresponding to the defect area. Then, the defect grade identification can be performed on the defect area by using the grade classification head, so as to obtain the defect grade corresponding to the defect type of the defect area.

204. And the computer equipment performs information filtering processing on defect information corresponding to at least one component image of the electronic component in a plurality of different dimensions by combining shooting angle information corresponding to the component image, so as to obtain filtered defect information of the electronic component.

For example, as shown in fig. 6, the information filtering processing may be performed on the defect information corresponding to the at least one component image of the SIM card in a plurality of different dimensions from the angle dimension, the confidence dimension, the level dimension and the area dimension, so as to obtain the filtered defect information of the SIM card.

Wherein, for the angle dimension, some defects are unlikely to appear on the component images shot at certain angles. Therefore, the shooting angle corresponding to the component picture can be obtained by judging the ID of the component picture, and defects in the shooting angle can filter defect information.

Wherein, for the confidence dimension, each defect can be set with a different confidence threshold, and defects below the confidence threshold represent lower confidence, possibly only textures that are long like the defect, and can be filtered.

Wherein for the level dimension, a defect level threshold may be set for each defect type. For example, belonging to a defect level threshold of th_h, below which the defect level is low, not a serious defect, may be filtered.

For the area dimension, for some area defects, such as different colors, if the defect area is smaller than a certain threshold, the defect area is not large enough and is not obvious, and filtering can be performed.

Through layer-by-layer filtration, the number of incorrect defects is greatly reduced, so that the over-killing rate is reduced, and the workload of manually performing repeated judgment on the defects detected by a machine is reduced.

205. And the computer equipment judges the electronic components according to the filtered defect information to obtain a defect detection result of the electronic components.

For example, the computer device may determine the SIM card according to the filtered defect information, to obtain a defect detection result of the SIM card. The computer device may then output the defect detection result of the SIM card. For example, the computer device may output whether the SIM card is defective. If there is a defect, where the defect area is, the type of defect corresponding to the defect area, and what the defect level corresponding to the type of defect is.

In the embodiment of the application, the computer equipment can acquire at least one component image of the electronic component and shooting angle information corresponding to the component image; the computer equipment can detect the defect area of the component image to obtain at least one defect area in the component image; the computer equipment performs defect information identification on the defect area to obtain defect information corresponding to the defect area in the component image; the computer equipment combines shooting angle information corresponding to the component images to carry out information filtering processing on defect information corresponding to at least one component image of the electronic component in a plurality of different dimensions, so as to obtain filtered defect information of the electronic component; and the computer equipment judges the electronic components according to the filtered defect information to obtain a defect detection result of the electronic components. In the embodiment of the application, firstly, a region with a defect in a component image is roughly positioned, and at least one defect region in the component image is obtained. By the embodiment of the application, the over-killing rate and the omission rate of defect detection can be effectively reduced, and the quality of defect detection is improved.

In order to better implement the defect detection method provided by the embodiment of the present application, in an embodiment, a defect detection device is also provided, and the defect detection device may be integrated in a computer device. Wherein the meaning of the nouns is the same as in the defect detection method described above, specific implementation details can be referred to the description in the method embodiments.

In one embodiment, a defect detection apparatus is provided, which may be integrated in a computer device, as shown in fig. 7, and includes: an acquisition unit 301, a defective area detection unit 302, an identification unit 303, an information filtering unit 304, and a discrimination unit 305 are specifically as follows:

An acquiring unit 301, configured to acquire at least one component image of an electronic component and shooting angle information corresponding to the component image;

A defect area detection unit 302, configured to perform defect area detection on the component image, so as to obtain at least one defect area in the component image;

an identifying unit 303, configured to identify defect information of the defect area, so as to obtain defect information corresponding to the defect area in the component image;

The information filtering unit 304 is configured to combine the shooting angle information corresponding to the component images, perform information filtering processing on defect information corresponding to at least one component image of the electronic component in a plurality of different dimensions, and obtain filtered defect information of the electronic component;

And the judging unit 305 is configured to judge the electronic component according to the filtered defect information, so as to obtain a defect detection result of the electronic component.

In an embodiment, the information filtering unit 304 may include:

In an embodiment, the first information filtering subunit may include:

In an embodiment, the second information filtering subunit may include:

In an embodiment, the third information filtering subunit may include:

In an embodiment, the fourth information filtering subunit may include:

In an embodiment, the defect area detecting unit 302 may include:

In an embodiment, the identifying unit 303 may include:

In an embodiment, the type identification subunit may include:

In an embodiment, the defect detecting apparatus may further include:

In the implementation, each unit may be implemented as an independent entity, or may be implemented as the same entity or several entities in any combination, and the implementation of each unit may be referred to the foregoing method embodiment, which is not described herein again.

The defect detection device can improve the accuracy of defect detection on the components.

The embodiment of the application also provides a computer device, which can comprise a terminal or a server, for example, the computer device can be used as a defect detection terminal, and the terminal can be a mobile phone, a tablet computer and the like; for another example, the computer device may be a server, such as a defect detection server, or the like. As shown in fig. 8, a schematic structural diagram of a terminal according to an embodiment of the present application is shown, specifically:

The computer device may include one or more processors 401 of a processing core, memory 402 of one or more computer readable storage media, a power supply 403, and an input unit 404, among other components. Those skilled in the art will appreciate that the computer device structure shown in FIG. 8 is not limiting of the computer device and may include more or fewer components than shown, or may be combined with certain components, or a different arrangement of components. Wherein:

The processor 401 is a control center of the computer device, connects various parts of the entire computer device using various interfaces and lines, performs various functions of the computer device and processes data by running or executing software programs and/or modules stored in the memory 402, and calling data stored in the memory 402. Optionally, processor 401 may include one or more processing cores; preferably, the processor 401 may integrate an application processor and a modem processor, wherein the application processor mainly processes an operating system, a user page, an application program, etc., and the modem processor mainly processes wireless communication. It will be appreciated that the modem processor described above may not be integrated into the processor 401.

The memory 402 may be used to store software programs and modules, and the processor 401 executes various functional applications and data processing by executing the software programs and modules stored in the memory 402. The memory 402 may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program (such as a sound playing function, an image playing function, etc.) required for at least one function, and the like; the storage data area may store data created according to the use of the computer device, etc. In addition, memory 402 may include high-speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, the memory 402 may also include a memory controller to provide the processor 401 with access to the memory 402.

The computer device further comprises a power supply 403 for supplying power to the various components, preferably the power supply 403 may be logically connected to the processor 401 by a power management system, so that functions of charge, discharge, and power consumption management may be performed by the power management system. The power supply 403 may also include one or more of any of a direct current or alternating current power supply, a recharging system, a power failure detection circuit, a power converter or inverter, a power status indicator, and the like.

The computer device may also include an input unit 404, which input unit 404 may be used to receive input numeric or character information and to generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function control.

Although not shown, the computer device may further include a display unit or the like, which is not described herein. In particular, in this embodiment, the processor 401 in the computer device loads executable files corresponding to the processes of one or more application programs into the memory 402 according to the following instructions, and the processor 401 executes the application programs stored in the memory 402, so as to implement various functions as follows:

The specific implementation of each operation above may be referred to the previous embodiments, and will not be described herein.

According to one aspect of the present application, there is provided a computer program product or computer program comprising computer instructions stored in a computer readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions to cause the computer device to perform the methods provided in the various alternative implementations of the above embodiments.

It will be appreciated by those of ordinary skill in the art that all or part of the steps of the various methods of the above embodiments may be performed by a computer program, or by computer program control related hardware, which may be stored in a computer readable storage medium and loaded and executed by a processor.

To this end, an embodiment of the present application further provides a storage medium in which a computer program is stored, the computer program being capable of being loaded by a processor to perform the steps of any of the defect detection methods provided by the embodiments of the present application. For example, the computer program may perform the steps of:

The steps in any defect detection method provided by the embodiment of the present application can be executed by the computer program stored in the storage medium, so that the beneficial effects of any defect detection method provided by the embodiment of the present application can be achieved, and detailed descriptions of the previous embodiments are omitted.

The foregoing has described in detail the methods, apparatuses, computer devices and storage medium for detecting defects according to the embodiments of the present application, and specific examples have been provided herein to illustrate the principles and embodiments of the present application, and the above description of the embodiments is only for aiding in the understanding of the methods and core ideas of the present application; meanwhile, as those skilled in the art will have variations in the specific embodiments and application scope in light of the ideas of the present application, the present description should not be construed as limiting the present application.

Claims

1. A defect detection method, comprising:

Identifying shooting angle information of the component image to obtain defect type range information corresponding to the component image under the shooting angle; filtering defect types which do not accord with the defect type range information in the defect information of the component image to obtain first filtered defect information; performing information filtering on the first filtered defect information in the confidence degree dimension to obtain second filtered defect information; performing information filtering on the second filtered defect information in the grade dimension to obtain third filtered defect information; performing information filtering on the third filtered defect information in the area dimension to obtain filtered defect information of the electronic component;

2. The method of claim 1, wherein the performing information filtering on the first filtered defect information in the confidence dimension to obtain second filtered defect information comprises:

comparing the confidence coefficient parameter corresponding to the defect type of the first filtered defect information with the confidence coefficient threshold value to obtain a comparison result;

and filtering the defect types which do not accord with the confidence coefficient threshold value in the first filtered defect information according to the comparison result to obtain the second filtered defect information.

3. The method of claim 1, wherein the performing information filtering on the second filtered defect information in the level dimension to obtain third filtered defect information includes:

4. The method of claim 1, wherein the performing information filtering on the third filtered defect information in the area dimension to obtain filtered defect information of the electronic component comprises:

Comparing the defect area information corresponding to the defect type in the third filtered defect information with the defect area threshold value to obtain a comparison result;

And filtering the defect types of which the defect area information does not accord with the defect area threshold value in the third filtered defect information according to the comparison result to obtain the filtered defect information.

5. The method of claim 1, wherein the performing defect region detection on the component image to obtain at least one defect region in the component image comprises:

Performing high-resolution sampling on the component image to obtain high-resolution sampling information corresponding to the component image;

performing multi-scale feature extraction on the high-resolution sampling information to obtain multi-scale features of the component image;

And detecting the defect area of the multi-scale feature of the component image to obtain at least one defect area in the component image.

6. The method of claim 1, wherein the identifying the defect information of the defect area to obtain defect information corresponding to the defect area in the component image includes:

performing type recognition on the defect area to obtain a defect type corresponding to the defect area;

7. The method of claim 6, wherein the performing type recognition on the defect area to obtain a defect type corresponding to the defect area comprises:

screening out the screened defect areas from the defect areas according to a preset screening proportion;

And performing type recognition on the downsampling area to obtain a defect type corresponding to the downsampling area.

8. The method of claim 1, wherein the performing defect region detection on the component image to obtain at least one defect region in the component image comprises:

Performing defect region detection on the component image by using a defect detection model to obtain at least one defect region in the component image;

The step of identifying the defect information of the defect area to obtain the defect information corresponding to the defect area in the component image comprises the following steps:

And carrying out defect information identification on the defect area by using the defect detection model to obtain defect information corresponding to the defect area in the component image.

9. The method of claim 8, wherein before performing defect region detection on the component image using a defect detection model to obtain at least one defect region in the component image, the method further comprises:

Performing defect prediction on the component image sample by using the defect detection model to be trained to obtain predicted defect information corresponding to the component image sample;

Cleaning the predicted defect information corresponding to the component image sample according to the labeling information of the component image sample to obtain cleaned predicted defect information;

and adjusting the defect detection model to be trained by using the post-cleaning predicted defect information to obtain the defect detection model.

10. A defect detection apparatus, comprising:

The information filtering unit is used for identifying shooting angle information of the component image to obtain defect type range information corresponding to the component image under the shooting angle; filtering defect types which do not accord with the defect type range information in the defect information of the component image to obtain first filtered defect information; performing information filtering on the first filtered defect information in the confidence degree dimension to obtain second filtered defect information; performing information filtering on the second filtered defect information in the grade dimension to obtain third filtered defect information; performing information filtering on the third filtered defect information in the area dimension to obtain filtered defect information of the electronic component;

11. A computer device comprising a memory and a processor; the memory stores an application program, and the processor is configured to execute the application program in the memory to perform the operations in the defect detection method according to any one of claims 1 to 9.

12. A computer readable storage medium storing a plurality of instructions adapted to be loaded by a processor to perform the steps in the defect detection method of any of claims 1 to 9.

13. A computer program product comprising a computer program or instructions which, when executed by a processor, carries out the steps of the defect detection method of any of claims 1 to 9.