CN115272249A

CN115272249A - Defect detection method and device, computer equipment and storage medium

Info

Publication number: CN115272249A
Application number: CN202210915080.8A
Authority: CN
Inventors: 徐尚; 聂强; 林愉欢; 刘永; 汪铖杰; 吴永坚
Original assignee: Tencent Technology Shenzhen Co Ltd
Current assignee: Tencent Technology Shenzhen Co Ltd
Priority date: 2022-08-01
Filing date: 2022-08-01
Publication date: 2022-11-01
Anticipated expiration: 2042-08-01
Also published as: CN115272249B

Abstract

The embodiment of the application discloses a defect detection method, a defect detection device, computer equipment and a storage medium; the embodiment of the application can be applied to various scenes such as industrial detection, artificial intelligence and the like. The method and the device for acquiring the electronic component image can acquire at least one component image of the electronic component and shooting angle information corresponding to the component image; detecting a defect area of the component image to obtain a defect area in the component image; identifying defect information of the defect area to obtain defect information corresponding to the defect area; performing information filtering processing on defect information corresponding to the component image of the electronic component in a plurality of different dimensions by combining shooting angle information corresponding to the component image to obtain filtered defect information; and judging the electronic component according to the filtered defect information to obtain a defect detection result of the electronic component. Through the embodiment of the application, the over-killing rate and the missing rate of the defect detection can be effectively reduced, and the quality of the defect detection is improved.

Description

Defect detection method and device, computer equipment and storage medium

Technical Field

The present application relates to the field of computer technologies, and in particular, to a defect detection method and apparatus, a computer device, and a storage medium.

Background

The industrial defect detection refers to quality detection of industrial components in the production and manufacturing process, and the components generally need to pass through the industrial defect detection after being produced, so that the quality of the components is improved. For example, a Subscriber Identity Module (SIM) card is a relatively general component in a mobile phone, and various defects are easily generated in large-scale generation, and a conventional method involves a large amount of input quality testing personnel, and selects out defective components by means of human eye observation, so that the method not only consumes a large amount of manpower, but also is greatly influenced by the level of the quality testing personnel, and the quality of defect detection is reduced.

Disclosure of Invention

The embodiment of the application provides a defect detection method, a defect detection device, computer equipment and a storage medium, which can reduce the omission factor and the over-killing factor of defective industrial components, thereby improving the quality of defect detection of the industrial components.

The embodiment of the application provides a defect detection method, which comprises the following steps:

acquiring at least one component image of an electronic component and shooting angle information corresponding to the component image;

detecting a defect area of the component image to obtain at least one defect area in the component image;

identifying defect information of the defect area to obtain defect information corresponding to the defect area in the component image;

performing information filtering processing on defect information corresponding to at least one component image of the electronic component in a plurality of different dimensions by combining shooting angle information corresponding to the component image to obtain filtered defect information of the electronic component;

and judging the electronic component according to the filtered defect information to obtain a defect detection result of the electronic component.

Correspondingly, the embodiment of the present application further provides a defect detecting apparatus, including:

the device comprises an acquisition unit, a processing unit and a display unit, wherein the acquisition unit is used for acquiring at least one component image of an electronic component and shooting angle information corresponding to the component image;

the defect area detection unit is used for detecting the defect area of the component image to obtain at least one defect area in the component image;

the identification unit is used for identifying the defect information of the defect area to obtain the defect information corresponding to the defect area in the component image;

the information filtering unit is used for carrying out information filtering processing on defect information corresponding to at least one component image of the electronic component in a plurality of different dimensions by combining shooting angle information corresponding to the component image to obtain filtered defect information of the electronic component;

and the judging unit is used for judging the electronic component according to the filtered defect information to obtain a defect detection result of the electronic component.

In an embodiment, the information filtering unit may include:

the first information filtering subunit is configured to perform information filtering on the defect information of the component image in an angle dimension in combination with shooting angle information corresponding to the component image, so as to obtain first filtered defect information;

the second information filtering subunit is configured to perform information filtering on the first filtered defect information in the confidence degree dimension to obtain second filtered defect information;

a third information filtering subunit, configured to perform information filtering on the second filtered defect information in a level dimension to obtain third filtered defect information;

and the fourth information filtering subunit is configured to perform information filtering on the third filtered defect information in an area dimension to obtain filtered defect information of the electronic component.

In an embodiment, the first information filtering subunit may include:

the information identification module is used for identifying the shooting angle information of the component image to obtain the corresponding defect type range information of the component image under the shooting angle;

and the first filtering module is used for filtering the defect types which do not accord with the defect type range information in the defect information of the component image to obtain the first filtered defect information.

In an embodiment, the second information filtering subunit may include:

the first threshold determining unit is used for determining a confidence threshold corresponding to the defect type in the first filtered defect information;

the first comparison module is used for comparing the confidence coefficient parameter corresponding to the defect type of the first filtered defect information with a confidence coefficient threshold value to obtain a comparison result;

and the second filtering module is used for filtering the defect types which do not meet the preset confidence level threshold in the first filtered defect information according to the comparison result to obtain second filtered defect information.

In an embodiment, the third information filtering subunit may include:

a second threshold determining unit, configured to determine a defect level threshold corresponding to a defect type in the second filtered defect information;

the second comparison module is used for comparing the defect grade corresponding to the defect type in the second filtered defect information with the defect grade threshold value to obtain a comparison result;

and the third filtering module is used for filtering the defect types with the defect grades not meeting the defect grade threshold in the second filtered defect information according to the comparison result to obtain third filtered defect information.

In an embodiment, the fourth information filtering subunit may include:

a third threshold determining unit, configured to determine a defect area threshold corresponding to a defect type in the third filtered defect information;

the third comparison module is used for comparing the defect area information corresponding to the defect type in the third filtered defect information with the defect area threshold value to obtain a comparison result;

and the fourth filtering module is used for filtering the defect type of the defect region area information which does not accord with the defect area threshold value in the third filtered defect information according to the comparison result to obtain the filtered defect information.

In an embodiment, the defective region detecting unit may include:

the high-resolution sampling subunit is used for carrying out high-resolution sampling on the component image to obtain high-resolution sampling information corresponding to the component image;

the multi-scale feature extraction subunit is used for performing multi-scale feature extraction on the high-resolution sampling information to obtain multi-scale features of the component image;

and the defect area detection subunit is used for detecting the defect area of the multi-scale features of the component image to obtain at least one defect area in the component image.

In an embodiment, the identification unit may include:

the type identification subunit is used for carrying out type identification on the defect area to obtain a defect type corresponding to the defect area;

the defect grade identification subunit is used for identifying the defect grade of the defect area based on the defect type to obtain the defect grade corresponding to the defect type of the defect area;

and the integration subunit is used for integrating the defect area, the defect type corresponding to the defect area and the defect grade to obtain the defect information corresponding to the defect area in the component image.

In an embodiment, the type identification subunit may include:

the screening module is used for screening the screened defect areas in the defect areas according to a preset screening proportion;

the down-sampling module is used for carrying out down-sampling treatment on the screened defect area to obtain a down-sampled area;

and the type identification module is used for identifying the type of the down-sampling area to obtain the defect type corresponding to the down-sampling area.

In an embodiment, the defect detecting apparatus may further include:

the information acquisition module is used for acquiring a defect detection model to be trained and a component image sample carrying labeling information;

the defect detection module is used for predicting the defects of the component image samples by using the defect detection model to be trained to obtain predicted defect information corresponding to the component image samples;

the cleaning module is used for cleaning the predicted defect information corresponding to the component image sample according to the labeling information of the component image sample to obtain the cleaned predicted defect information;

and the adjusting module is used for adjusting the defect detection model to be trained by utilizing the cleaned predicted defect information to obtain the defect detection model.

Embodiments of the present application also provide a computer program product or computer program comprising computer instructions stored in a computer-readable storage medium. The computer instructions are read by a processor of a computer device from a computer-readable storage medium, and the computer instructions are executed by the processor to cause the computer device to perform the method provided in the various alternatives of the above aspect.

Correspondingly, an embodiment of the present application further provides a storage medium, where the storage medium stores instructions, and the instructions, when executed by a processor, implement the defect detection method provided in any embodiment of the present application.

The method and the device for acquiring the electronic component image can acquire at least one component image of the electronic component and shooting angle information corresponding to the component image; detecting a defect area of the component image to obtain at least one defect area in the component image; identifying defect information of the defect area to obtain defect information corresponding to the defect area in the component image; performing information filtering processing on defect information corresponding to at least one component image of the electronic component in multiple different dimensions by combining shooting angle information corresponding to the component image to obtain filtered defect information of the electronic component; and judging the electronic component according to the filtered defect information to obtain a defect detection result of the electronic component. Through the embodiment of the application, the over-killing rate and the missing rate of the defect detection can be effectively reduced, and the quality of the defect detection is improved.

Drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.

Fig. 1 is a schematic view of a scene of a defect detection method provided in an embodiment of the present application;

FIG. 2 is a schematic flowchart of a defect detection method according to an embodiment of the present application;

FIG. 3 is a schematic diagram of another scenario of a defect detection method according to an embodiment of the present application;

FIG. 4 is a schematic diagram of another scenario of a defect detection method provided in an embodiment of the present application;

FIG. 5 is a schematic flowchart of a defect detection method according to an embodiment of the present application;

FIG. 6 is a schematic flowchart of a defect detection method according to an embodiment of the present application;

FIG. 7 is a schematic structural diagram of a defect detection apparatus according to an embodiment of the present application;

fig. 8 is a schematic structural diagram of a computer device provided in an embodiment of the present application.

Detailed Description

The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, however, the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments of the present application. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.

The embodiment of the application provides a defect detection method, which can be executed by a defect detection device, and the defect detection device can be integrated in a computer device. Wherein the computer device may comprise at least one of a terminal and a server, etc. That is, the defect detection method provided in the embodiment of the present application may be executed by a terminal, may be executed by a server, or may be executed by both a terminal and a server that are capable of communicating with each other.

The terminal may include, but is not limited to, a smart phone, a tablet Computer, a notebook Computer, a Personal Computer (PC), a smart home appliance, a wearable electronic device, a VR/AR device, a vehicle-mounted terminal, a smart voice interaction device, and the like.

The server may be an interworking server or a background server among a plurality of heterogeneous systems, an independent physical server, a server cluster or a distributed system formed by a plurality of physical servers, a cloud server providing basic cloud computing services such as cloud service, a cloud database, cloud computing, cloud functions, cloud storage, network service, cloud communication, middleware service, domain name service, security service, big data and artificial intelligence platforms, and the like.

It should be noted that the embodiments of the present application can be applied to various scenarios, including but not limited to cloud technology, artificial intelligence, industrial detection, and the like.

In an embodiment, as shown in fig. 1, the defect detection apparatus may be integrated on a computer device such as a terminal or a server, so as to implement the defect detection method provided in the embodiment of the present application. Specifically, the computer equipment can acquire at least one component image of the electronic component and shooting angle information corresponding to the component image; detecting a defect area of the component image to obtain at least one defect area in the component image; identifying defect information of the defect area to obtain defect information corresponding to the defect area in the component image; performing information filtering processing on defect information corresponding to at least one component image of the electronic component in multiple different dimensions by combining shooting angle information corresponding to the component image to obtain filtered defect information of the electronic component; and judging the electronic component according to the filtered defect information to obtain a defect detection result of the electronic component.

The following are detailed below, and it should be noted that the order of description of the following examples is not intended to limit the preferred order of the examples.

The embodiments of the present application will be described in terms of a defect detection apparatus, which may be integrated in a computer device, where the computer device may be a server or a terminal.

As shown in fig. 2, a defect detection method is provided, and the specific flow includes:

101. and acquiring at least one component image of the electronic component and shooting angle information corresponding to the component image.

The components may include electronic components and components of machines and instruments. For example, the electronic components may include Subscriber Identity Module (SIM) cards, resistors, capacitors, inductors, potentiometers, valves, relays, integrated circuits, various types of circuits, and the like.

The component image may include an image in which the appearance or the internal structure of the electronic component is recorded.

In one embodiment, to improve the accuracy of defect detection, the electronic component may be photographed from a plurality of different angles to obtain at least one component image. For example, the electronic component may be photographed from the front, back, and side surfaces, respectively, to obtain a plurality of component images.

For example, as shown in fig. 3, when the electronic component is a SIM card, the SIM card may be photographed from the front, back, and side surfaces, respectively, to obtain a plurality of component images.

In one embodiment, the imaging angle information may be used to describe the angle from which the electronic component is imaged. For example, the shooting angle information may be used to explain that the SIM card is shot from the front. For another example, the shooting angle information may be used to explain that the SIM card is shot from the side, and so on.

In an embodiment, the electronic component is shot from a plurality of angles, each angle is called a specific point, and the electronic component is finally determined to be a defective electronic component or a non-defective electronic component through joint detection of defects on different point images.

In one embodiment, a defective electronic component may refer to an electronic component that is not satisfactory under quality control.

For example, defects of electronic components may include nine types, which are chipping, scratching, pitting, bright line, bai Meng, plating, off-color, flash, and off-grade, respectively.

Bai Meng, the initial stage belongs to the integral defect of an electronic component, the electroplating, heterochromatic and glue overflow belong to the area defect, the bright line belongs to the linear defect, the residual broken points and scratches belong to the point defect.

102. And detecting the defect area of the component image to obtain at least one defect area in the component image.

In an embodiment, after acquiring the at least one device image of the electronic device, the at least one device image may be subjected to area detection to obtain at least one defect area of the device image.

For example, 9 meta-device images are acquired, and the 9 meta-device images may be subjected to area detection to obtain a defect area corresponding to each component image.

In an embodiment, in order to accurately detect the defects of the electronic component, when the defect detection device detects the defects of the image of the component, the regions of the component, which may have the defects, are all detected, so that the situation of missing the defects is avoided.

Accordingly, the defective region may include a region that may include a defect detected from the component image. For example, the defect area may or may not include a defect of the device image. For example, 20 defective regions are detected from the component image, where 3 of the 20 defective regions are defects including the component image and 17 regions not including the component image.

In one embodiment, when a plurality of defects exist in an electronic component, the size of the defect region corresponding to each defect may be different because the area of the defect may be different. For example, the defective regions corresponding to the component images have a size of 20 dimensions by 20 dimensions, and have a size of 14 dimensions by 14 dimensions. In addition, in the process of detecting the defect area, a plurality of defect areas may include the same defect in the electronic component, and only the coverage areas of the defect areas are different. In this case, the size of the defective region may be different.

In one embodiment, there are several methods for detecting a defective area in a device image to obtain at least one defective area in the device image.

In an embodiment, the defect detection model may be used to detect a defect region of the device image, so as to obtain at least one defect region in the device image.

Wherein the defect detection model is an artificial intelligence model.

The artificial intelligence is a theory, a method, a technology and an application system which simulate, extend and expand human intelligence by using a digital computer or a machine controlled by the digital computer, sense the environment, acquire knowledge and obtain the best result by using the knowledge. In other words, artificial intelligence is a comprehensive technique of computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can react in a manner similar to human intelligence. Artificial intelligence is the research of the design principle and the realization method of various intelligent machines, so that the machines have the functions of perception, reasoning and decision making.

The artificial intelligence technology is a comprehensive subject and relates to the field of extensive technology, namely the technology of a hardware level and the technology of a software level. The artificial intelligence base technologies generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technologies, operation/interaction systems, mechatronics, and the like. The artificial intelligence software technology mainly comprises a computer vision technology, a voice processing technology, a natural language processing technology, machine learning/deep learning and the like.

Machine Learning (ML) is a multi-domain cross subject, and relates to multiple subjects such as probability theory, statistics, approximation theory, convex analysis and algorithm complexity theory. The special research on how a computer simulates or realizes the learning behavior of human beings so as to acquire new knowledge or skills and reorganize the existing knowledge structure to continuously improve the performance of the computer. Machine learning is the core of artificial intelligence, is the fundamental approach for computers to have intelligence, and is applied to all fields of artificial intelligence. Machine learning and deep learning generally include techniques such as artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and teaching learning.

For example, the defect detection model may be at least one of a Convolutional Neural Network (CNN), a deconvolution Neural network (De-Convolutional network, DN), a Deep Neural Network (DNN), a Deep Convolutional Inverse network (DCIGN), a Region-based Convolutional network (RCNN), a Region-based fast Convolutional network (fast forward network, RCNN), a Bidirectional Encoder/decoder (BERT) model, a Feature Pyramid network (Feature Pyramid network, FPN), and a high-resolution network (HR-network), among others.

In one embodiment, the defect detection model may include a feature extractor, a defect area detector, a defect detection head, and a class classification detection head.

The feature extractor and the defect area detector may be configured to perform defect area detection on the component image to obtain at least one defect area in the component image.

The defect detection head and the grade classification head can be used for identifying defect information of the defect area to obtain defect information corresponding to the defect area in the component image.

In one embodiment, the feature extractor, the defect region detector, the defect detection head, and the level classification detection head may all be artificial intelligence models.

For example, a backbone network and a multi-scale feature extraction network may be included in the feature extractor. Wherein the backbone network may be an HR-NET network, etc. The multi-scale feature extraction network may be an FPN network, or the like.

For example, the defect region detector may be a convolutional network. For example, the candidate detection box may be a CNN network or a DNN network, and so on.

For example, the defect detection head may be a convolutional network. For example, the defect detection head may be a CNN network or a DNN network, or the like.

For example, the level classification detection head may be a classifier. For example, the level classification detection head may be a multi-classifier or a bi-classifier, and so on.

In an embodiment, the feature extractor and the defect region detector in the defect detection model may be used to perform defect region detection on the component image, so as to obtain at least one defect region in the component image.

In one embodiment, in order to reduce the missing rate of defect detection, when detecting the defect region of the component image, the defect region is detected as much as possible, so that the defects in the electronic component are avoided being missed. In order to detect the defect regions as much as possible, the defect region of the component image may be sampled by high resolution processing to obtain at least one defect region of the component image.

Specifically, the step of "detecting a defect area in the image of the component to obtain at least one defect area in the image of the component" may include:

carrying out high-resolution sampling on the component image to obtain high-resolution sampling information corresponding to the component image;

carrying out multi-scale feature extraction on the high-resolution sampling information to obtain multi-scale features of the component image;

and detecting the defect area of the multi-scale features of the component image to obtain at least one defect area in the component image.

In one embodiment, the conventional defect region detection method uses sampling from small to large resolution, or up-sampling and up-sampling for recovery, which may result in the loss of information. In order to detect the defect area as much as possible, and therefore the missing rate of defect detection is reduced, when the device image is sampled, high resolution is guaranteed, one multi-resolution is maintained to sample the device image in parallel, and meanwhile, information with different resolutions is exchanged in parallel, so that the features are semantically richer and spatially more accurate.

Specifically, the step of performing high-resolution sampling on the component image to obtain high-resolution sampling information corresponding to the component image may include:

performing multi-resolution parallel convolution on the component images to obtain a plurality of convolution information corresponding to the component images;

performing multi-resolution fusion on a plurality of convolution information of the component images to obtain fusion convolution information of the component images;

and performing down-sampling processing on the fusion convolution information to obtain high-resolution sampling information.

In an embodiment, the component images may be subjected to multi-resolution parallel convolution to obtain a plurality of convolution information corresponding to the component images. For example, the device image may be subjected to parallel convolution using a plurality of different high-dimensional convolution kernels to obtain a plurality of convolution information of the device image. When multi-resolution parallel convolution is carried out on the component images, convolution information of the component images can be continuously sampled by using a high-dimensional convolution core so as to form a plurality of sampling branches.

In an embodiment, multi-resolution fusion can be performed on a plurality of convolution information of the component image to obtain fusion convolution information of the component image. For example, a plurality of sampling branches are formed in the multi-resolution parallel convolution process, and the convolution information formed in each sampling branch can be cross-fused to obtain the fused convolution information of the component image. The cross-fusion can be performed in various ways. For example, cross-fusion may be achieved by weighting and then adding. For another example, the cross-fusion can be achieved by directly adding.

In an embodiment, the fused convolution information may be downsampled to obtain high resolution sampling information. By down-sampling the fusion convolution information, the rich information formed in the above steps can be subjected to information compression to obtain high-resolution sampling information of the component image. Because the high-resolution sampling information is obtained by changing the resolution sampling, the high-resolution sampling information contains rich information of the image of the component, which is beneficial to the detection of the defect area.

In one embodiment, in order to detect as many defect regions as possible, multi-scale feature extraction may be performed on the high-resolution sampling information to obtain multi-scale features of the device image, so that more information in the device image may be further mined.

The multi-scale feature extraction of the high-resolution sampling information may refer to feature extraction of the high-resolution sampling information through different scales to obtain multi-scale features of the component image. The multi-scale features of the component image may describe features of the component image from a plurality of different scales. For example, some features are used to describe texture features of an image, some features are used to describe semantic features of an image, and so on.

Specifically, the step of performing multi-scale feature extraction on the high-resolution sampling information to obtain multi-scale features of the component image may include:

carrying out information expansion on the high-resolution sampling information to obtain a plurality of expanded sampling information;

respectively extracting the characteristics of the expanded sampling information by using a multi-head attention mechanism to obtain the characteristic information corresponding to each expanded sampling information;

and fusing the characteristic information corresponding to each expanded sampling information to obtain the multi-scale characteristic information of the component image.

In an embodiment, the information expansion may be performed on the high resolution sampling information to obtain a plurality of expanded sampling information. For example, the high resolution sampling information may be a matrix of 125 dimensions x 125 dimensions, and then the high resolution sampling information may be expanded into a plurality of matrices of different dimensions, resulting in expanded sampling information. For example, the high resolution sample information may be expanded into 120-dimensional by 120-dimensional matrices, 84-dimensional by 84-dimensional matrices, and 64-dimensional by 64-dimensional matrices, and so on.

Then, feature extraction can be performed on the expanded sampling information by using a multi-head attention mechanism, so as to obtain feature information corresponding to each expanded sampling information. Among them, the Multi-head-attention mechanism (Multi-head-attention) uses multiple queries to compute and select multiple information from input information in parallel. Each focusing on a different part of the input information.

For example, the high resolution sampling information may be expanded into 4 pieces of expanded sampling information. Then, feature extraction can be performed by using the 4 pieces of extended sampling information generated based on different scales, so as to obtain feature information corresponding to each piece of extended sampling information.

Then, the feature information corresponding to each expanded sampling information can be fused to obtain the multi-scale feature information of the component image. For example, each expanded sampling information may be spliced to obtain multi-scale feature information of the component image.

Then, the multi-scale features of the component image can be detected to obtain at least one defect region in the component image. Because the multi-scale features can explain the texture features, semantic features, light features and the like of the component images, the defect regions in the component images can be detected as many as possible through the multi-scale features of the component images.

In one embodiment, a defective area in each component image of an electronic component may be detected, via step 102. For example, 9-dimensional device images of the electronic device are taken. Through step 102, defective regions in the 9 meta-device images can be detected.

103. And identifying the defect information of the defect area to obtain the defect information corresponding to the defect area in the component image.

In an embodiment, the defect area in each component image of the electronic component may be identified to obtain defect information corresponding to the defect area in the component image.

The defect information may include information describing a defect in the multi-component image. For example, the defect information may include a defect type, a defect level, and a defect area.

The defect type may indicate whether the defect of the electronic component is a crack or a scratch, and the like.

Wherein, the defect grade may refer to a severity corresponding to the defect type. For example, the defect levels may include 4 levels, where the first level is the lowest severity and the fourth level may be the highest severity.

Wherein the defect area may refer to the size of the defect. For example, the defect area may be an area of a defect region.

By means of the defect information it is possible to know which defects are included in the defective area, the severity of the defects and the size of the defects.

In an embodiment, there are multiple ways to identify the defect information of the defect area to obtain the defect information corresponding to the defect area in the device image.

In an embodiment, the defect information of the defect area may be identified by using a defect detection model, so as to obtain defect information corresponding to the defect area in the component image. For example, the defect information of the defect area may be identified by using a defect detection head and a class classification detection head in the defect detection model, so as to obtain the defect information corresponding to the defect area in the component image. For example, assuming that the defect detection head is a CNN network, the CNN network may be used to identify the defect area, so as to obtain a defect type corresponding to the defect area. In addition, the grade classification head can be used for identifying the defect area to obtain the defect grade corresponding to the defect type in the defect area. Then, the defect grade and the defect type can be integrated to obtain defect information corresponding to the defect area.

In an embodiment, before the defect detection model is used to detect the defect area of the image of the component, the defect detection model to be trained needs to be trained to obtain a defect detection model meeting the performance.

Specifically, before the step of detecting a defect region of a component image by using a defect detection model to obtain at least one defect region in the component image, the method provided in the embodiment of the present application may further include:

acquiring a defect detection model to be trained and a component image sample carrying labeling information;

performing defect prediction on the component image sample by using a defect detection model to be trained to obtain predicted defect information corresponding to the component image sample;

according to the labeling information of the component image sample, cleaning the predicted defect information corresponding to the component image sample to obtain the cleaned predicted defect information;

and adjusting the defect detection model to be trained by utilizing the cleaned predicted defect information to obtain the defect detection model.

The defect detection model to be trained may include a model which is not trained or does not meet the requirement and still needs to be trained.

The component image sample may include training data used when the defect detection model to be trained is trained. The component image sample can be an image obtained by shooting a plurality of electronic components at different angles.

The labeling information can be used for explaining the defects of the electronic components in the component image sample. For example, the labeling effect of the component image can be as shown in fig. 4, wherein BD-QX-S4 can refer to the labeling information of the component image. The BD may refer to a defect of an electronic component in a component image, which is a burn-in point. The labeling of other defects of the electronic component can be expressed as: bump (BD), scratch (GS), pock (MD), bright Line (LX), bai Meng (BM), plating (DD), heterochromatic (YS), flash (YJ), and finish (QJ). QX may refer to the imaging of the component image being sharp. In addition to QX, annotations can include MH (fuzzy) and KBQ (unclear). S4 may indicate that the defect level of the breakout point is severe. In addition to S4, the defect levels may also include S1-S3, where S1 may refer to the lowest defect level, i.e., the defect is not severe.

In an embodiment, the defect detection model to be trained can be used for predicting defects of the component image sample, so as to obtain predicted defect information corresponding to the component image sample. The step 102 and the step 103 may be referred to in the process of predicting defects of the component image sample by using the defect detection model to be trained, and the description is not repeated here. Initial predictions are made by the model over a large amount of data, and these predictions need to be cleaned due to insufficient model capability. And correcting the wrong label and supplementing the missed label. The model can then be retrained on the cleaned data. By repeating the steps continuously, the detection capability of the model can be improved continuously, so that the defect detection model with the performance meeting the requirements can be obtained.

In one embodiment, the present application may adopt a hierarchical detection manner, defect information is detected from the component image. The hierarchical detection mode may refer to a coarse-to-fine detection mode. For example, a region having a defect in the image of the component is first roughly located, i.e., at least one defective region in the image of the component is obtained. Then, further, the defect type of the electronic component is identified from the defective region. On the premise of identifying the defect type, the defect grade corresponding to the defect type of the electronic component can be further identified. And then integrating the identified defect area, the defect type corresponding to the defect area and the defect grade to obtain the defect information of the electronic component.

Specifically, the step of "identifying the defect information of the defect area to obtain the defect information corresponding to the defect area in the component image" may include:

identifying the type of the defective area to obtain the defect type corresponding to the defective area;

based on the defect type, carrying out defect grade identification on the defect area to obtain a defect grade corresponding to the defect type of the defect area;

and integrating the defect region and the defect type and the defect grade corresponding to the defect region to obtain defect information corresponding to the defect region in the component image.

In one embodiment, in order to reduce the missing rate, the regions of the component where defects may exist are all detected, so that the occurrence of missing defects is avoided. Therefore, a large number of regions that do not actually include a defect, or regions that include only a small portion of a defect, may be detected from the defective regions detected in step 102. In order to reduce the over-killing rate, the defective areas need to be screened, so that the number of the defective areas without defects is balanced, and the accuracy of defect detection is improved. For example, generally, a plurality of defective regions can be detected in step 102. These defect areas include a large number of defect areas containing no defects and a small number of defect areas containing defects. For example, 10 defective regions are detected, 8 of which are defective regions containing no defect and 2 of which are defective regions containing a defect. In order to balance the number of defect regions that do not contain defects and improve the accuracy of defect detection, the screening regions may be screened.

Specifically, the step of "identifying the type of the defective area to obtain the defect type corresponding to the defective area" may include:

screening the screened defect area from the defect areas according to a preset screening proportion;

performing down-sampling treatment on the screened defect area to obtain a down-sampled area;

and identifying the type of the down-sampling area to obtain the defect type corresponding to the down-sampling area.

In an embodiment, the screened defect regions may be screened out from the defect regions according to a preset screening ratio. The screened defect area may refer to an area that needs to be identified by defect information. The preset screening ratio may be used to indicate a ratio between a defect area including a defect and a defect area not including a defect in the screened defect area, where the defect area includes a defect and the defect area does not include a defect. For example, assuming that the preset screening ratio is 1:3, the ratio between the defect region containing the defect and the defect region not containing the defect in the screened defect region is 1:3.

In one embodiment, when a plurality of defects exist in the electronic component, the sizes of the defect regions corresponding to the defects may be different because the areas of the defects may be different. For example, some of the defect regions corresponding to the component images have a size of 20 dimensions by 20 dimensions, and some of the defect regions have a size of 14 dimensions by 14 dimensions. In order to facilitate the defect types contained in the screened defect regions, the screened defect regions can be subjected to down-sampling treatment, so that the defect regions are down-sampled into down-sampled regions with the same dimension, and the defect types can be conveniently identified.

Then, type identification can be performed on the down-sampling area to obtain a defect type corresponding to the down-sampling area. For example, the type of the downsampled region may be identified by using an artificial intelligence technology such as CNN, so as to obtain a defect type corresponding to the downsampled region.

In an embodiment, after the defect type corresponding to the defect area is obtained, the defect level of the defect area may be identified based on the defect type, so as to obtain the defect level corresponding to the defect type of the defect area.

For example, based on the defect type, the defect level of the downsampled region may be identified to obtain a defect level corresponding to the defect type of the defect region. For example, the classifier may be used to perform level identification on the down-sampled area to obtain a defect level corresponding to the defect type of the defect area.

In an embodiment, the defect area, and the defect type and the defect grade corresponding to the defect area may be integrated to obtain defect information corresponding to the defect area in the image of the component. For example, the area of the defect region, the defect type and the defect grade corresponding to the defect region may be integrated according to a preset format to obtain defect information corresponding to the defect region in the component image. For example, the predetermined format may be [ defect region identifier, defect region area, defect type, defect grade ], and then the area of the defect region, the defect type corresponding to the defect region, and the defect grade may be integrated according to the predetermined format. For example, the integrated defect information may be [001,12 × 12, bd, s4], and so on.

The defect area may be a screened defect area after screening. For example, in step 103, the defect information of all the defect areas obtained in step 102 is not recognized, but only the defect areas after screening are recognized, and the defect information corresponding to the defect areas after screening is obtained.

In an embodiment, the defect information may be a multi-dimensional vector, and the defect information includes defect information corresponding to each filtered defect area. For example, there are 7 filtered defect regions, and the defect information may include information related to the 7 filtered defect regions.

104. And performing information filtering processing on the defect information corresponding to at least one component image of the electronic component in a plurality of different dimensions by combining the shooting angle information corresponding to the component image to obtain the filtered defect information of the electronic component.

In an embodiment, since one electronic component is photographed from multiple angles, it is finally necessary to perform comprehensive judgment by combining the results of multiple component images, and determine whether the electronic component is a defective component or a non-defective component. Through the

steps

102 and 103, all defects in the electronic component can be detected, and whether the defects are slight defects or uncertain defects, the aim is to detect the defects as many as possible, so that the low omission ratio is ensured. The missing inspection rate may refer to the percentage of the undetected defective electronic components in the number of detected electronic components after passing the automatic defect detection.

And the low omission factor is ensured, and simultaneously, the low overdischarge rate is also required to be ensured, so that the whole process of defect detection is high in quality. Wherein, the over-killing rate can refer to the over-killing rate of the electronic components with defects detected after automatic defect detection, it is the number of defect-free electronic components that is a percentage of the total number of tested electronic components.

In one embodiment, the defect information of the component image may be detected with a false detection. For example, a certain part of the electronic component is not a defect, but is detected as a defect. For example, although a defect exists in a certain part of an electronic component, the severity of the defect is low, and the defect can be ignored and omitted. Therefore, after the defect information of the component image is obtained, the information filtering processing can be performed on the defect information corresponding to at least one component image of the electronic component in a plurality of different dimensions by combining the shooting angle information corresponding to the component image. By filtering the defect information in a plurality of different dimensions, the condition of error detection can be reduced, thereby reducing the over-killing rate.

In an embodiment, information filtering defect information from multiple different dimensions may refer to filtering defect information from multiple different aspects. For example, the plurality of dimensions may include an angle dimension, a confidence dimension, a level dimension, and an area dimension, among others. Different dimensions can adopt different methods and different measurement indexes to measure the defect information, so that whether the defect information needs to be filtered or not is judged.

In an embodiment, when the multiple dimensions may include an angle dimension, a confidence dimension, a level dimension, and an area dimension, the step of "combining shooting angle information corresponding to the component image, performing information filtering processing on defect information corresponding to at least one component image of the electronic component in multiple different dimensions to obtain filtered defect information of the electronic component" may include:

performing information filtering on the defect information of the component image in an angle dimension by combining shooting angle information corresponding to the component image to obtain first filtered defect information;

performing information filtering on the first filtered defect information in a confidence degree dimension to obtain second filtered defect information;

performing information filtering on the second filtered defect information in a grade dimension to obtain third filtered defect information;

and performing information filtering on the third filtered defect information in the area dimension to obtain the filtered defect information of the electronic component.

In one embodiment, due to the particularity of the angle capture, certain defects may not theoretically appear in the device image at a particular angle for the device image captured at that angle. For example, assuming that the electronic component is a SIM card, there are some drawbacks that the component image photographed on the front side cannot be made to appear on the component image photographed on the front side. Therefore, the shooting angle information corresponding to the component image can be combined, and the defect information of the component image is subjected to information filtering in the angle dimension to obtain the first filtered defect information.

Specifically, the step "combine the shooting angle information that the components and parts image corresponds, carry out information filtering at the angle dimension to the defect information of components and parts image, obtain first defect information after filtering", can include:

identifying the shooting angle information of the component image to obtain the corresponding defect type range information of the component image under the shooting angle;

and filtering the defect types which do not accord with the defect type range information in the defect information of the component image to obtain first filtered defect information.

In an embodiment, the shooting angle information of the component image may be identified to obtain the defect type range information corresponding to the component image under the shooting angle. The defect type range information may indicate a defect type that may occur in the image of the component under the shooting angle information. For example, when the shooting angle information of the component image is the front surface, the defect type range information may include chipping, scratching, pocking, and bright lines. That is, for the front-shot device image, the types of defects that may appear in the device image theoretically include only chipping, scratching, pitting, and bright lines.

In an embodiment, the defect types that do not meet the defect type range information in the defect information of the component image may be filtered to obtain first filtered defect information. For example, the defect type and the defect type range information corresponding to each defect area in the defect information may be matched. When the defect type of the defect area is not a defect type in the defect type range information, the defect area and its corresponding information may be filtered.

In an embodiment, in order to improve the accuracy of defect detection, information filtering may be performed on the first filtered defect information in a confidence dimension to obtain second filtered defect information. The information filtering of the first filtered defect information in the confidence degree dimension may refer to performing information filtering on the first filtered defect information according to the confidence degree of the defect type to obtain second filtered defect information.

Here, the confidence level may be reliability, and means reliability of the predicted object. For example, the confidence level of a defect type may refer to how high the confidence level of the predicted defect type is. For example, if the confidence of the defect type of the defect area is 99%, it can be said that the confidence of the defect type is high, and it can be considered that the defect exists in the defect area. For another example, if the defect type of the defect area is 40%, it may be said that the reliability of the defect type is low, and it may be considered that there is no defect in the defect type or there are other types of defects, and at this time, the defect type of the defect area may be filtered out to avoid affecting the defect detection result of the electronic component.

Specifically, the step of performing information filtering on the first filtered defect information in a confidence degree dimension to obtain second filtered defect information may include:

determining a confidence threshold corresponding to the defect type in the first filtered defect information;

comparing the confidence coefficient parameter corresponding to the defect type of the first filtered defect information with a confidence coefficient threshold value to obtain a comparison result;

and filtering the defect types which do not accord with the preset confidence level threshold in the first filtered defect information according to the comparison result to obtain second filtered defect information.

The confidence coefficient parameter may be used to describe a confidence coefficient corresponding to the defect type of the defect region. Confidence parameters may be generated by step 103. In step 103, when the type of the defect region is identified, a confidence parameter for each defect type in the defect region is generated, and then the defect type with the highest confidence parameter is selected as the defect type corresponding to the defect region.

In one embodiment, since the electronic component may include multiple defect types, the confidence threshold may be different for each defect type. Therefore, a confidence threshold corresponding to the defect type in the first filtered defect information can be determined. For example, for defect type breakouts, the corresponding confidence threshold may be 97%. As another example, for defect type pocks, the corresponding confidence threshold may be 98%, and so on.

For example, the confidence parameter corresponding to the defect type of each defect region in the first filtered defect information may be compared with a confidence threshold. If the confidence coefficient value corresponding to the defect type of the defect region is greater than the confidence coefficient threshold value, the information of the defect region is not filtered. And if the confidence coefficient parameter corresponding to the defect type of the defect region is smaller than the confidence coefficient threshold value, filtering the defect region to obtain second filtered defect information.

In one embodiment, to measure the severity of the defect, the embodiment of the present application further sets a defect grade for each defect type. For example, the defect levels may include a defect level 1 to a defect level 4, wherein the defect level 1 may indicate that the severity of the defect is low, and the defect level 4 may indicate that the severity of the defect is high. Wherein, when the defect grade of some defects is lower, the defects can be ignored. Therefore, the second filtered defect information may be subjected to information filtering in the level dimension to obtain third filtered defect information. The information filtering of the second filtered defect information in the level dimension may refer to filtering the defect level of the defect type in the second filtered defect information to obtain third filtered defect information.

Specifically, the step of performing information filtering on the second filtered defect information in a level dimension to obtain third filtered defect information may include:

determining a defect grade threshold corresponding to the defect type in the second filtered defect information;

comparing the defect grade corresponding to the defect type in the second filtered defect information with the defect grade threshold value to obtain a comparison result;

and filtering the defect types with the defect grades not meeting the defect grade threshold in the second filtered defect information according to the comparison result to obtain third filtered defect information.

For example, the defect level corresponding to the defect type of each area type in the second filtered defect information may be compared with a defect threshold. If the defect level of the defective area is smaller than the defect level threshold, it can be said that the defect of the defective area is not serious and can be ignored, so that the information corresponding to the defective area can be filtered. And when the defect level of the defect area is greater than or equal to the defect level threshold, it can be shown that the defect severity of the defect area is high and cannot be ignored.

In one embodiment, when the defect area of some defects on the electronic component is not large, the defects can be ignored. For example, if the defect area is smaller than a certain threshold value, the defect type is different in color, which means that the defect area is not large enough and the defect is not obvious, and the defect can be ignored, so that the filtering can be performed.

Therefore, the third filtered defect information can be subjected to information filtering in the area dimension, and the filtered defect information of the electronic component can be obtained. Specifically, the step of performing information filtering on the third filtered defect information in the area dimension to obtain the filtered defect information of the electronic component may include:

determining a defect area threshold corresponding to the defect type in the third filtered defect information;

comparing the defect area information corresponding to the defect type in the third filtered defect information with a defect area threshold to obtain a comparison result;

and filtering the defect type of which the defect area information does not accord with the defect area threshold value in the third filtered defect information according to the comparison result to obtain the filtered defect information.

The defect area information may be used for the area size corresponding to the defect area. Defect region area information for the defect region may be generated by step 102. In step 102, when detecting a defective region of the component image, defective region area information corresponding to the defective region is generated.

For example, the defect area information corresponding to the defect type may be compared with a defect area threshold. When the defect region area information is less than the defect area threshold, the defect region may be filtered out. And when the area information of the defect area is larger than or equal to the area threshold value, the defect area and the corresponding information thereof are reserved.

In an embodiment, by filtering information of each component image of the electronic component, filtered defect information corresponding to each component image of the electronic component can be obtained. Then, the electronic components can be distinguished according to the filtered defect information corresponding to each component image of the electronic components, and the defect detection result of the electronic components is obtained.

In one embodiment, the number of incorrect defects is reduced greatly by filtering layer by layer, so that the over-killing rate is reduced, and the workload of manual re-judgment from the defects detected by a machine is reduced.

105. And judging the electronic component according to the filtered defect information to obtain a defect detection result of the electronic component.

In an embodiment, the filtered defect information of each component image may be combined to discriminate the electronic component, so as to obtain a defect detection result of the electronic component. For example, the defect regions remaining after filtering each component image, the defect types corresponding to the defect regions, and the grades corresponding to the defect types may be integrated to obtain the defect detection result of whether the electronic component has a defect. For example, where there are some electronic device images that may detect the same defect region, duplicate defect regions may be filtered, leaving only one. If the electronic component has a defect, where the defect area is located, the defect type corresponding to the defect area and the defect grade corresponding to the defect type are all the same.

The embodiment of the application provides a defect detection method, which comprises the following steps: acquiring at least one component image of an electronic component and shooting angle information corresponding to the component image; detecting a defect area of the component image to obtain at least one defect area in the component image; identifying defect information of the defect area to obtain defect information corresponding to the defect area in the component image; performing information filtering processing on defect information corresponding to at least one component image of the electronic component in a plurality of different dimensions by combining shooting angle information corresponding to the component image to obtain filtered defect information of the electronic component; and judging the electronic component according to the filtered defect information to obtain a defect detection result of the electronic component. In the embodiment of the application, firstly, a region with a defect in the component image is roughly positioned, and at least one defective region in the component image is obtained. Then, further, the defect type of the electronic component is identified from the defective region. On the premise of identifying the defect type, the defect grade corresponding to the defect type of the electronic component can be further identified. By the aid of the grading detection mode, defects in the images of the components can be detected as much as possible, and accordingly the missing rate of defect detection is reduced. In addition, after detecting the defect information corresponding to the defective area in the component image, information filtering processing is also performed on the defect information in a plurality of different dimensions. Through layer-by-layer filtering, the number of incorrect defects is reduced greatly, the over-killing rate is reduced, and the workload of manual re-judgment from the defects detected by a machine is reduced. Therefore, the over-killing rate and the missing rate of the defect detection can be effectively reduced, and the quality of the defect detection is improved.

The method described in the above examples is further illustrated in detail below by way of example.

The method of the embodiment of the present application will be described by taking an example that the defect detection method is integrated on a computer device.

In an embodiment, as shown in fig. 5, a defect detection method includes the following specific processes:

201. the computer equipment acquires at least one component image of the electronic component and shooting angle information corresponding to the component image.

It is assumed that the electronic component is a SIM card holder. The SIM card support is used as a relatively universal part in a mobile phone, various defects are easily generated in large-scale generation, the traditional method is characterized in that a large amount of quality testing personnel are invested, defective parts are picked out in a mode of human eye observation, and the method not only consumes a large amount of manpower, but also is greatly influenced by the level of the quality testing personnel. Through the embodiment of the application, the defect detection of the SIM card holder can be automatically realized.

For example, the SIM card may be photographed from the front, back, and side to obtain a plurality of component images.

202. And the computer equipment detects the defect area of the component image to obtain at least one defect area in the component image.

For example, as shown in fig. 6, a feature extractor and a defect region detector that may utilize a defect detection model may be used to perform defect region detection on a component image to obtain at least one defect region in the component image.

For example, the component image may be input to a backbone network and a multi-scale feature extractor in the feature extractor, and multi-scale features of the component image may be obtained, where the multi-scale features may describe texture features of the image and may describe semantic features of the image. The candidate detectors may then be used to detect defective regions from the multi-scale features of the component image.

203. And the computer equipment identifies the defect information of the defect area to obtain the defect information corresponding to the defect area in the component image.

For example, as shown in fig. 6, the computer device may perform type identification on the defect area by using the defect detection head in the defect identification model, so as to obtain a defect type corresponding to the defect area. Then, the grade classification head can be used for identifying the defect grade of the defect area to obtain the defect grade corresponding to the defect type of the defect area.

204. And the computer equipment combines the shooting angle information corresponding to the component images to perform information filtering processing on the defect information corresponding to at least one component image of the electronic component in a plurality of different dimensions to obtain the filtered defect information of the electronic component.

For example, as shown in fig. 6, the defect information corresponding to at least one component image of the SIM card may be subjected to information filtering processing in multiple different dimensions from an angle dimension, a confidence dimension, a level dimension, and an area dimension, so as to obtain filtered defect information of the SIM card.

In the angle dimension, some defects cannot appear on the component images shot at certain angles. Therefore, the shooting angle corresponding to the component picture can be obtained by judging the ID of the component picture, and the defect information can be filtered by the defect appearing in the shooting angle.

For the confidence measure, different confidence thresholds can be set for each type of defect, and defects below the confidence thresholds represent lower confidence, and may only be textures that are grown to be similar to the defects, and may be filtered.

Wherein for the level dimension, a defect level threshold may be set for each defect type. For example, the threshold for defect level is th _ h, and a value below th _ h indicates that the defect level is low and is not a serious defect, and filtering may be performed.

For the area dimension, for some area defects, such as different colors, if the defect area is smaller than a certain threshold, it represents that the defect area is not large enough and not obvious, and filtering may be performed.

Through filtering layer by layer, the number of incorrect defects is reduced greatly finally, so that the over-killing rate is reduced, and the workload of manual re-judgment from the defects detected by a machine is reduced.

205. And the computer equipment judges the electronic components according to the filtered defect information to obtain the defect detection result of the electronic components.

For example, the computer device may distinguish the SIM card according to the filtered defect information, so as to obtain a defect detection result of the SIM card. The computer device may then output the defect detection result of the SIM card. For example, the computer device may output whether the SIM card is defective. If there is a defect, where the defect area is, what the defect type corresponding to the defect area and what the defect level corresponding to the defect type are.

In the embodiment of the application, the computer equipment can acquire at least one component image of the electronic component and shooting angle information corresponding to the component image; the computer equipment can detect the defect area of the component image to obtain at least one defect area in the component image; the computer equipment identifies the defect information of the defect area to obtain the defect information corresponding to the defect area in the component image; the computer equipment combines the shooting angle information corresponding to the component images to filter information of defect information corresponding to at least one component image of the electronic component in multiple different dimensions to obtain filtered defect information of the electronic component; and the computer equipment judges the electronic component according to the filtered defect information to obtain a defect detection result of the electronic component. In the embodiment of the application, firstly, a region with a defect in the component image is roughly positioned, and at least one defective region in the component image is obtained. Through the embodiment of the application, the over-killing rate and the missing rate of the defect detection can be effectively reduced, and the quality of the defect detection is improved.

In order to better implement the defect detection method provided by the embodiment of the present application, in an embodiment, a defect detection apparatus is further provided, and the defect detection apparatus may be integrated in a computer device. The terms are the same as those in the defect detection method, and details of implementation can be referred to the description in the method embodiment.

In an embodiment, a defect detecting apparatus is provided, which may be specifically integrated in a computer device, as shown in fig. 7, and includes: the acquiring unit 301, the defective area detecting unit 302, the identifying unit 303, the information filtering unit 304, and the judging unit 305 are as follows:

an obtaining unit 301, configured to obtain at least one component image of an electronic component and shooting angle information corresponding to the component image;

a defect area detection unit 302, configured to perform defect area detection on the component image to obtain at least one defect area in the component image;

the identifying unit 303 is configured to identify defect information of the defect area to obtain defect information corresponding to the defect area in the component image;

the information filtering unit 304 is configured to perform information filtering processing on defect information corresponding to at least one component image of the electronic component in multiple different dimensions in combination with shooting angle information corresponding to the component image, so as to obtain filtered defect information of the electronic component;

a judging unit 305, configured to judge the electronic component according to the filtered defect information, so as to obtain a defect detection result of the electronic component.

In an embodiment, the information filtering unit 304 may include:

the third information filtering subunit is configured to perform information filtering on the second filtered defect information in a level dimension to obtain third filtered defect information;

In an embodiment, the first information filtering subunit may include:

In an embodiment, the second information filtering subunit may include:

the first comparison module is used for comparing the confidence coefficient parameters corresponding to the defect types of the first filtered defect information with the confidence coefficient threshold value to obtain a comparison result;

In an embodiment, the third information filtering subunit may include:

In an embodiment, the fourth information filtering subunit may include:

In an embodiment, the defective area detecting unit 302 may include:

In an embodiment, the identifying unit 303 may include:

In an embodiment, the type identification subunit may include:

the down-sampling module is used for performing down-sampling processing on the screened defect area to obtain a down-sampled area;

In an embodiment, the defect detecting apparatus may further include:

In a specific implementation, the above units may be implemented as independent entities, or may be combined arbitrarily to be implemented as the same or several entities, and the specific implementation of the above units may refer to the foregoing method embodiments, which are not described herein again.

The defect detection device can improve the accuracy of defect detection of the components.

The embodiment of the present application further provides a computer device, where the computer device may include a terminal or a server, for example, the computer device may be used as a defect detection terminal, and the terminal may be a mobile phone, a tablet computer, or the like; for another example, the computer device may be a server, such as a defect detection server. As shown in fig. 8, it shows a schematic structural diagram of a terminal according to an embodiment of the present application, specifically:

the computer device may include components such as a processor 401 of one or more processing cores, memory 402 of one or more computer-readable storage media, a power supply 403, and an input unit 404. Those skilled in the art will appreciate that the computer device configuration illustrated in FIG. 8 does not constitute a limitation of computer devices, and may include more or fewer components than those illustrated, or some components may be combined, or a different arrangement of components. Wherein:

the processor 401 is a control center of the computer device, connects various parts of the entire computer device using various interfaces and lines, and performs various functions of the computer device and processes data by operating or executing software programs and/or modules stored in the memory 402 and calling data stored in the memory 402. Alternatively, processor 401 may include one or more processing cores; preferably, the processor 401 may integrate an application processor and a modem processor, wherein the application processor mainly handles operating systems, user pages, application programs, and the like, and the modem processor mainly handles wireless communications. It will be appreciated that the modem processor described above may not be integrated into the processor 401.

The memory 402 may be used to store software programs and modules, and the processor 401 executes various functional applications and data processing by operating the software programs and modules stored in the memory 402. The memory 402 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application program required by at least one function (such as a sound playing function, an image playing function, etc.), and the like; the storage data area may store data created according to use of the computer device, and the like. Further, the memory 402 may include high speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other volatile solid state storage device. Accordingly, the memory 402 may also include a memory controller to provide the processor 401 access to the memory 402.

The computer device further comprises a power supply 403 for supplying power to the various components, and preferably, the power supply 403 is logically connected to the processor 401 via a power management system, so that functions of managing charging, discharging, and power consumption are implemented via the power management system. The power supply 403 may also include any component of one or more dc or ac power sources, recharging systems, power failure detection circuitry, power converters or inverters, power status indicators, and the like.

The computer device may also include an input unit 404, the input unit 404 being operable to receive input numeric or character information and to generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function control.

Although not shown, the computer device may further include a display unit and the like, which are not described in detail herein. Specifically, in this embodiment, the processor 401 in the computer device loads the executable file corresponding to the process of one or more application programs into the memory 402 according to the following instructions, and the processor 401 runs the application programs stored in the memory 402, thereby implementing various functions as follows:

combining the shooting angle information corresponding to the component image, and performing information filtering processing on the defect information corresponding to at least one component image of the electronic component in multiple different dimensions to obtain filtered defect information of the electronic component;

The above operations can be implemented in the foregoing embodiments, and are not described in detail herein.

According to an aspect of the application, a computer program product or computer program is provided, comprising computer instructions, the computer instructions being stored in a computer readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions to cause the computer device to perform the method provided in the various alternative implementations of the above embodiments.

It will be understood by those skilled in the art that all or part of the steps of the methods of the above embodiments may be performed by a computer program, which may be stored in a computer-readable storage medium and loaded and executed by a processor, or by related hardware controlled by the computer program.

To this end, the present application further provides a storage medium, in which a computer program is stored, where the computer program can be loaded by a processor to execute the steps in any one of the defect detection methods provided in the present application. For example, the computer program may perform the steps of:

Since the computer program stored in the storage medium can execute the steps in any defect detection method provided in the embodiments of the present application, the beneficial effects that can be achieved by any defect detection method provided in the embodiments of the present application can be achieved, and detailed descriptions are omitted here for the foregoing embodiments.

The foregoing detailed description has provided a defect detection method, apparatus, computer device, and storage medium provided in the embodiments of the present application, and specific examples have been applied herein to illustrate the principles and implementations of the present application, and the description of the foregoing embodiments is only used to help understand the method and its core ideas of the present application; meanwhile, for those skilled in the art, according to the idea of the present application, there may be variations in the specific embodiments and the application scope, and in summary, the content of the present specification should not be construed as a limitation to the present application.

Claims

1. A method of defect detection, comprising:

2. The method of claim 1, wherein the plurality of different dimensions includes an angle dimension, a confidence dimension, a rank dimension, and an area dimension; the combination of the shooting angle information corresponding to the component image, the information filtering processing of the defect information corresponding to at least one component image of the electronic component in multiple different dimensions to obtain the filtered defect information of the electronic component includes:

performing information filtering on the first filtered defect information in a confidence dimension, obtaining second filtered defect information;

and performing information filtering on the third filtered defect information in an area dimension to obtain filtered defect information of the electronic component.

3. The method according to claim 2, wherein the combining the shooting angle information corresponding to the component image to perform information filtering on the defect information of the component image in an angle dimension to obtain first filtered defect information includes:

and filtering the defect types which do not accord with the defect type range information in the defect information of the component image to obtain the first filtered defect information.

4. The method of claim 2, wherein the information filtering the first filtered defect information in a confidence dimension to obtain second filtered defect information comprises:

and filtering the defect types which do not accord with the preset confidence level threshold value in the first filtered defect information according to the comparison result to obtain second filtered defect information.

5. The method of claim 2, wherein the information filtering the second filtered defect information in a level dimension to obtain third filtered defect information comprises:

6. The method according to claim 2, wherein the performing information filtering on the third filtered defect information in an area dimension to obtain filtered defect information of the electronic component includes:

comparing the defect area information corresponding to the defect type in the third filtered defect information with the defect area threshold to obtain a comparison result;

and filtering the defect type of the defect region area information which does not accord with the defect area threshold value in the third filtered defect information according to the comparison result to obtain the filtered defect information.

7. The method according to claim 1, wherein the performing defect region detection on the component image to obtain at least one defect region in the component image comprises:

performing multi-scale feature extraction on the high-resolution sampling information to obtain multi-scale features of the component image;

and detecting the defect region of the multi-scale features of the component image to obtain at least one defect region in the component image.

8. The method according to claim 1, wherein the identifying the defect information of the defect area to obtain the defect information corresponding to the defect area in the image of the component includes:

identifying the type of the defect area to obtain the defect type corresponding to the defect area;

based on the defect type, identifying the defect grade of the defect area to obtain the defect grade corresponding to the defect type of the defect area;

and integrating the defect region, the defect type corresponding to the defect region and the defect grade to obtain defect information corresponding to the defect region in the component image.

9. The method according to claim 8, wherein the performing type identification on the defective area to obtain a defect type corresponding to the defective area comprises:

screening the screened defect area from the defect area according to a preset screening proportion;

performing down-sampling processing on the screened defect area to obtain a down-sampled area;

and identifying the type of the down sampling area to obtain the defect type corresponding to the down sampling area.

10. The method according to claim 1, wherein the performing defect region detection on the component image to obtain at least one defect region in the component image comprises:

detecting a defect region of the component image by using a defect detection model to obtain at least one defect region in the component image;

the identifying the defect information of the defect area to obtain the defect information corresponding to the defect area in the component image includes:

and identifying the defect information of the defect area by using the defect detection model to obtain the defect information corresponding to the defect area in the component image.

11. The method of claim 10, wherein before the defect region detection is performed on the component image by using the defect detection model to obtain at least one defect region in the component image, the method further comprises:

performing defect prediction on the component image sample by using the defect detection model to be trained to obtain predicted defect information corresponding to the component image sample;

and adjusting the defect detection model to be trained by utilizing the predicted defect information after cleaning to obtain the defect detection model.

12. A defect detection apparatus, comprising:

13. A computer device comprising a memory and a processor; the memory stores an application program, and the processor is configured to execute the application program in the memory to perform the operations of the defect detection method according to any one of claims 1 to 11.

14. A computer readable storage medium storing instructions adapted to be loaded by a processor to perform the steps of the defect detection method of any one of claims 1 to 11.

15. A computer program product comprising a computer program or instructions, characterized in that the computer program or instructions, when executed by a processor, performs the steps in the defect detection method of any one of claims 1 to 11.