CN114240882A

CN114240882A - Defect detection method and device, electronic equipment and storage medium

Info

Publication number: CN114240882A
Application number: CN202111546950.0A
Authority: CN
Inventors: 沈琦; 暴天鹏; 吴立威
Original assignee: Shenzhen Sensetime Technology Co Ltd
Current assignee: Shenzhen Sensetime Technology Co Ltd
Priority date: 2021-12-16
Filing date: 2021-12-16
Publication date: 2022-03-25

Abstract

The present disclosure relates to a defect detection method and apparatus, an electronic device, and a storage medium, in which a defect detection result is obtained by acquiring an image to be detected and performing defect detection on the image to be detected through a defect detection network. The defect detection network is obtained through training of a target data set, the target data set comprises a first data set with real labels and/or a second data set with pseudo labels, the pseudo labels in the second data set are generated through a label generation model, and a confidence threshold of the label generation model is determined based on the first data set. The method performs defect detection network training together with the sample marked with the real label and the sample marked with the pseudo label by a semi-supervised method, so as to improve the accuracy of the defect detection network through a large amount of sample training, and obtain a more accurate defect detection result during defect detection.

Description

Defect detection method and device, electronic equipment and storage medium

Technical Field

The present disclosure relates to the field of computer technologies, and in particular, to a defect detection method and apparatus, an electronic device, and a storage medium.

Background

At present, deep learning is widely applied to various fields, and the efficiency of the defect detection process can be improved. However, training of the defect detection network needs to be based on a large amount of high-quality labeled data, and when the labeled data amount is small, the training result of the defect detection network is poor, so that the defect detection result obtained through the defect detection network is influenced.

Disclosure of Invention

The present disclosure provides a defect detection method and apparatus, an electronic device, and a storage medium, which aim to improve the accuracy of a defect detection result obtained through a defect detection network.

According to a first aspect of the present disclosure, there is provided a defect detection method, including:

acquiring an image to be detected;

carrying out defect detection on the image to be detected through a defect detection network to obtain a defect detection result;

the defect detection network is obtained through training of a target data set, the target data set comprises a first data set with real labels and/or a second data set with pseudo labels, the pseudo labels in the second data set are generated through a label generation model, and a confidence threshold of the label generation model is determined based on the first data set.

In a possible implementation manner, the generation process of the pseudo tag in the second data set includes:

inputting each second image data in the second data set into the tag generation model to obtain detection information corresponding to each second image data, wherein the detection information comprises a detection result and a corresponding confidence coefficient;

and screening out a detection result meeting the requirement from the detection information of the second image data as a pseudo label of the second image data.

In one possible implementation, the first data set includes at least two first image data and a real tag of each of the first image data;

determining a confidence threshold for the tag generation model based on the first dataset, including:

inputting each first image data into a label generation model obtained through training to obtain detection information of each first image data, wherein the detection information comprises a detection result and a corresponding confidence coefficient;

and determining a confidence threshold of the label generation model according to the detection information of each first image data and the real label.

In a possible implementation manner, the determining a confidence threshold of the tag generation model according to the detection information of each of the first image data and the real tag includes:

determining a candidate threshold:

screening the detection information of each first image data according to the candidate threshold value to obtain target detection information;

determining adjustment information according to the target detection information and the real label;

and determining a confidence threshold according to the adjusting information and the candidate threshold.

In a possible implementation manner, the determining adjustment information according to the target detection information and the real tag includes:

determining a recall rate and/or a detection accuracy according to the target detection information and the real label;

and determining adjustment information according to the recall rate and/or the detection accuracy.

In a possible implementation manner, a detection frame set is determined according to each piece of target detection information, and the detection frame set includes at least one detection image frame;

determining a labeling frame set according to a real label of target image data corresponding to each target detection information, wherein the labeling frame set comprises at least one labeling image frame;

determining a first matching parameter, a second matching parameter and a third matching parameter according to the detection frame set and the labeling frame set, wherein the first matching parameter represents the number of detection image frames matched with the labeling image frames, the second matching parameter represents the number of labeling image frames not matched with the detection image frames, and the third matching parameter represents the number of detection image frames not matched with the labeling image frames;

determining the recall rate according to the first matching parameter and the second matching parameter, and/or determining the detection accuracy according to the first matching parameter and the third matching parameter.

In one possible implementation, the determining a confidence threshold according to the adjustment information and the candidate threshold includes:

responding to the adjustment information meeting a preset adjustment condition, and determining the candidate threshold as a confidence threshold;

and in response to the adjusting information not meeting the preset adjusting condition, re-determining the adjusting information until the adjusting information meets the preset condition.

In a possible implementation manner, the adjustment condition is that the recall rate is greater than a first adjustment threshold, and the detection accuracy is greater than a second adjustment threshold;

the re-determining the adjustment information in response to the adjustment information not satisfying a preset adjustment condition includes:

in response to the recall rate not being greater than the first adjustment threshold, updating the tag generation model and re-determining adjustment information until the adjustment information meets a preset condition; alternatively, the first and second electrodes may be,

and responding to the detection accuracy not larger than a second adjustment threshold, adjusting the candidate threshold, and re-determining the adjustment information until the adjustment information meets a preset condition.

According to a second aspect of the present disclosure, there is provided a defect detecting apparatus including:

the image acquisition module is used for acquiring an image to be detected;

the defect detection module is used for carrying out defect detection on the image to be detected through a defect detection network to obtain a defect detection result;

determining a candidate threshold:

According to a third aspect of the present disclosure, there is provided an electronic device comprising: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to invoke the memory-stored instructions to perform the above-described method.

According to a fourth aspect of the present disclosure, there is provided a computer readable storage medium having stored thereon computer program instructions which, when executed by a processor, implement the above-described method.

The embodiment of the invention performs defect detection network training together with the sample marked with the real label and the sample marked with the pseudo label by a semi-supervised method, so as to improve the accuracy of the defect detection network through a large amount of sample training, and obtain a more accurate defect detection result during defect detection.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure. Other features and aspects of the present disclosure will become apparent from the following detailed description of exemplary embodiments, which proceeds with reference to the accompanying drawings.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the disclosure.

FIG. 1 shows a flow diagram of a defect detection method according to an embodiment of the present disclosure;

FIG. 2 illustrates a flow diagram of a process of determining a confidence threshold in accordance with an embodiment of the present disclosure;

FIG. 3 shows a schematic diagram of a process of determining a confidence threshold in accordance with an embodiment of the present disclosure;

FIG. 4 shows a schematic diagram of a process of training a defect detection model in accordance with an embodiment of the present disclosure;

FIG. 5 shows a schematic diagram of a defect detection apparatus according to an embodiment of the present disclosure;

FIG. 6 shows a schematic diagram of an electronic device in accordance with an embodiment of the disclosure;

fig. 7 shows a schematic diagram of another electronic device according to an embodiment of the disclosure.

Detailed Description

Various exemplary embodiments, features and aspects of the present disclosure will be described in detail below with reference to the accompanying drawings. In the drawings, like reference numbers can indicate functionally identical or similar elements. While the various aspects of the embodiments are presented in drawings, the drawings are not necessarily drawn to scale unless specifically indicated.

The word "exemplary" is used exclusively herein to mean "serving as an example, embodiment, or illustration. Any embodiment described herein as "exemplary" is not necessarily to be construed as preferred or advantageous over other embodiments.

The term "and/or" herein is merely an association describing an associated object, meaning that three relationships may exist, e.g., a and/or B, may mean: a exists alone, A and B exist simultaneously, and B exists alone. In addition, the term "at least one" herein means any one of a plurality or any combination of at least two of a plurality, for example, including at least one of A, B, C, and may mean including any one or more elements selected from the group consisting of A, B and C.

Furthermore, in the following detailed description, numerous specific details are set forth in order to provide a better understanding of the present disclosure. It will be understood by those skilled in the art that the present disclosure may be practiced without some of these specific details. In some instances, methods, means, elements and circuits that are well known to those skilled in the art have not been described in detail so as not to obscure the present disclosure.

FIG. 1 shows a flow diagram of a defect detection method according to an embodiment of the present disclosure. In a possible implementation manner, the defect detection method of the embodiment of the disclosure may be executed by an electronic device such as a terminal device or a server. The terminal device may be a User Equipment (UE), a mobile device, a User terminal, a cellular phone, a cordless phone, a Personal Digital Assistant (PDA), a handheld device, a computing device, a vehicle-mounted device, a wearable device, and other devices capable of deploying a deep learning model, and the terminal device implements the defect detection method according to the embodiment of the present disclosure by using a processor to call a computer readable instruction stored in a memory. Optionally, the defect detection method of the embodiment of the present disclosure may also be executed by a server, and the server may be a single server or a server cluster formed by a plurality of servers.

The defect detection method disclosed by the embodiment of the disclosure can be applied to any scene for determining and detecting any object, such as any defect detection application scene for detecting the defects of the weld joints of the battery, detecting the product parts and the like.

As shown in fig. 1, the defect detection method of the embodiment of the present disclosure may include the following steps:

and step S10, acquiring an image to be detected.

In a possible implementation manner, the image to be detected may be acquired by the electronic device through a built-in image acquisition device or a built-in image acquisition device, or may be directly received from the image to be detected sent by another electronic device. Optionally, the image to be detected is an image which needs to be subjected to defect detection, and the image includes an object to be detected for defects. For example, when the embodiment of the present disclosure is used for detecting the weld defects of the battery, the battery top cover may be included in the image to be detected. When the method and the device are used for detecting the parts of the contact net, the contact net can be included in the image to be detected.

And step S20, carrying out defect detection on the image to be detected through a defect detection network to obtain a defect detection result.

In a possible implementation mode, after the image to be detected is determined, the image to be detected is input into a defect detection network obtained through training, and a defect detection result is directly output after the defect detection is directly performed by the defect detection network. Optionally, the defect detection network is obtained by training a target data set, where the target data set includes a first data set with real tags and/or a second data set with pseudo tags, the pseudo tags in the second data set are generated by a tag generation model, and a confidence threshold of the tag generation model is determined based on the first data set.

Alternatively, the first data set and the second data set may be predetermined by an electronic device performing defect detection network training, which may be the same as or different from the electronic device performing the defect detection method. The first data set can be a test set of a label generation model, the test set comprises a plurality of labeled data, and the label generation model is obtained through training of electronic equipment based on a preset training set. The second data set may include a plurality of unlabeled data that are from the same source as the data in the first data set. For example, the first data set includes at least one first image data and a genuine label of each first image data, and the second data set includes at least one second image data. Optionally, the data sources of the first image data and the second image data are the same, and may be acquired by the same image acquisition system, for example. Further, when the data source obtains less data, the determining process of the first data set and the second data set may also be to obtain a small amount of data and perform data augmentation to obtain a large amount of data. And after labeling part of data in the mass data, determining a first data set, and determining a second data set by the rest of data without labeling.

Further, the real label of the first image data can be obtained through manual labeling, and the content of the real label can be determined according to the function of the deep learning model to be trained. For example, when the embodiment of the present disclosure is used for detecting a weld defect, at least one labeling image frame may be included in the real label for characterizing a region of the first image data where the weld is located. Optionally, other attribute information, such as a category label characterizing the defect category, may also be included in the real label.

In a possible implementation manner, the electronic device obtains the label generation model through pre-training of a pre-labeled training set, the training set for training the label generation model is the same as the data type included in the first data set, and the content included in the labeling information is also the same. For example, when the first data set includes a plurality of first image data and a true label of each first image data, the data type included in the training set is also image data, and includes label information corresponding to each image data. Meanwhile, when the real label comprises at least one marked image frame coordinate representing the area where the object is located in the first image data, the marked information of the image data in the training set also comprises at least one marked image frame coordinate. When the label is trained to generate the model, the image data in the training set is used as the model input, and the corresponding labeling information is used as the output to adjust the model parameters until the requirements are met.

Optionally, after the first data set is determined, a confidence threshold of the label generation model is determined based on the first data set, so as to label a pseudo label for each unlabeled second image data in the second data set according to the confidence threshold of the label generation model and the label generation model.

FIG. 2 illustrates a flow diagram of a process of determining a confidence threshold in accordance with an embodiment of the present disclosure. As shown in fig. 2, the process of determining the confidence threshold of the tag generation model based on the first data set according to the embodiment of the present disclosure may include steps S30 and S40.

Step S30 is to input each of the first image data into a label generation model obtained by training, and obtain detection information of each of the first image data.

In a possible implementation manner, each piece of first image data in the first data set is input into the trained label generation model, so as to obtain detection information corresponding to the input first image data. The detection information includes a detection result and a corresponding confidence level, and the confidence level is used for representing the reliability degree of the detection result. The content in the detection result is consistent with the content included in the genuine label. For example, when the real label includes an annotated image frame coordinate for characterizing the defect area and a category label for characterizing the defect category, the detection result may include a detection image frame coordinate for characterizing the area where the detected object is located and a category label for characterizing the type of the detected object. When the real label represents the defect category, the detection result only comprises the category label representing the defect category.

Step S40, determining a confidence threshold of the label generation model according to the detection information of each of the first image data and the real label.

In a possible implementation manner, after the detection information of each first image data in the first data set is obtained through the tag generation model, the confidence threshold of the tag generation model may be determined according to the detection information of each first image data and the real tag in the first data set. Optionally, the confidence threshold is used for screening a detection result obtained by performing data detection on the tag generation model.

Optionally, the process of determining the confidence threshold according to the detection information of the first image data and the real tag may be to determine a candidate threshold, and then screen the detection information of each first image data according to the candidate threshold to obtain the target detection information. And determining adjustment information according to the target detection information and the real label. And determining a confidence threshold according to the adjustment information and the candidate threshold.

In a possible implementation manner, the candidate threshold may be preset, so as to obtain whether the performance of the evaluation label generation model meets the requirement under the condition of the candidate threshold. And in the case of not meeting the requirements, readjusting the candidate threshold or updating the training label generation model until the performance of the label generation model meets the requirements.

Alternatively, the candidate threshold may be any value between 0 and 1, and may be adjusted according to a preset adjustment rule when the performance of the tag generation model does not meet the requirement. For example, by a preset fixed magnitude of 0.1, 0.2, etc.

Further, after the candidate threshold is determined, screening the detection information obtained after the first image data input label generation model is input according to the candidate threshold. The detection information comprises a detection result and a confidence coefficient, the detection information with the confidence coefficient smaller than the candidate threshold value can be deleted by screening the detection information through the candidate threshold value, the detection information with the confidence coefficient not smaller than the candidate threshold value is reserved, and the detection information is determined as target detection information.

In a possible implementation manner, the target detection information includes at least one detection image frame, which is used to characterize an area where the label generation model detects the defect in the target image data. The real label comprises at least one labeling image frame used for representing the area where the actual defect in the target image data is located. Alternatively, the recall rate and/or the detection accuracy may be determined according to the target detection information and the real tag, and then the adjustment information may be determined according to the recall rate and/or the detection accuracy. The tuning information is used to evaluate the performance of the tag generation model and may include at least one parameter. For example, only recall rate or detection accuracy, or both.

Optionally, the process of determining the adjustment information may include: determining a detection frame set according to the target detection information, wherein the detection frame set comprises at least one detection image frame, determining a labeling frame set according to a real label of target image data corresponding to each target detection information, and the labeling frame set comprises at least one labeling image frame. And determining a first matching parameter, a second matching parameter and a third matching parameter according to the detection frame set and the labeling frame set, wherein the first matching parameter represents the number of the detection image frames matched with the labeling image frames, the second matching parameter represents the number of the labeling image frames not matched with the detection image frames, and the third matching parameter represents the number of the detection image frames not matched with the labeling image frames. And determining the recall rate according to the first matching parameter and the second matching parameter, and/or determining the detection accuracy according to the first matching parameter and the third matching parameter. The detection frame set can include detection image frames in all target detection information, and the labeling frame set can include labeling image frames of all target detection information corresponding to real labels, so that the recall rate and the detection accuracy of the whole sample are calculated. Or, a corresponding detection frame set and a corresponding label frame set may be determined for each target detection information, and the recall rate and the detection accuracy are calculated respectively, and then the recall rate and the detection accuracy of the whole sample are obtained by calculating a weighted sum and the like.

Optionally, the first matching parameter, the second matching parameter, and the third matching parameter may be determined according to matching conditions of an image frame and an annotation image frame in a detection frame set and an annotation frame set corresponding to the target detection information. In a possible implementation manner, the recall rate may be a ratio of the first matching parameter to a sum of the first matching parameter and the second matching parameter, and the detection accuracy may be a ratio of the first matching parameter to a sum of the first matching parameter and the third matching parameter. That is, the recall ratio and the detection accuracy can be calculated by the formula one and the formula two.

Wherein, the formula I is used for calculating the recall rate of the label generation model. And the second formula is used for calculating the detection accuracy of the tag generation model under the condition of the current recall rate. tp is a first matching parameter, and can be obtained by determining whether each detection image frame in the detection frame set has a matched labeled image frame in the labeled frame set, and counting the number of the detection image frames having the matched labeled image frame. fn is a second matching parameter, and can be obtained by determining whether each labeled image frame in the labeled frame set has a matched detection image frame in the detection frame set, and counting the number of labeled image frames without the matched detection image frame in the labeled frame set. fp is a third matching parameter, and can be obtained by determining whether each detection image frame in the detection frame set has a matched labeled image frame in the labeled frame set, and counting the number of the detection image frames without the matched labeled image frame in the detection frame set.

The explanation will be given by taking an example in which the annotation frame set includes 10 annotation image frames, and the detection frame set includes 15 detection image frames. When 5 labeled image frames exist in the labeled frame set and are respectively matched with 5 detection image frames in the detection frame set, the first matching parameter is 5, the second matching parameter is the number of labeled image frames without matching detection image frames in the labeled frame set, namely, the value 5 of 10-5 is directly calculated as the second matching parameter. The third matching parameter is the number of the detected image frames without matching labeling image frames in the detected frame set, namely, the value 10 of 15-5 is directly calculated as the third matching parameter.

In a possible implementation manner, the process of determining whether the detection image frame and the annotation image frame are matched according to the embodiment of the present disclosure may be: and determining the intersection ratio of each detection image frame and each labeled image frame, and for each detection image frame, determining that the detection frame image frame is matched with the labeled image frame in response to the intersection ratio of the detection image frame and one labeled image frame being greater than the intersection ratio threshold. Wherein, the Intersection-over-Union (IoU) value is the overlapping rate of two image frames, i.e. the ratio of the Intersection and Union of the detection image frame and the labeling image frame. And when the intersection ratio value is larger than a preset intersection ratio threshold value, judging that the two image frames are matched.

In one possible implementation, after the adjustment information of the detection information of each first image data is output according to the candidate threshold and the current tag generation model, the confidence threshold may be determined according to the adjustment information and the candidate threshold. Alternatively, an adjustment condition may be preset, and in response to the adjustment information satisfying the preset adjustment condition, the candidate threshold is determined as the confidence threshold, and in response to the adjustment information not satisfying the preset adjustment condition, the adjustment information is re-determined until the adjustment information satisfies the preset condition. That is to say, when the current adjustment information meets the adjustment condition, the performance of the tag generation model is judged to meet the requirement, and the candidate threshold is directly determined to be the confidence threshold. When the preset adjusting condition is not met, the performance of the label generation model or the candidate threshold is judged to be not met, and the adjusting information can be re-determined in a mode of updating the label generation model or the candidate threshold.

Alternatively, the process of re-determining the adjustment information may be to update the tag generation model and re-determine the adjustment information in response to the recall rate not being greater than the first adjustment threshold until the adjustment information satisfies a preset condition. Or, in response to the detection accuracy not being greater than a second adjustment threshold, adjusting the candidate threshold, and re-determining adjustment information until the adjustment information satisfies a preset condition. Further, the above determination process may also have a corresponding order, for example, the recall rate and the magnitude of the first adjustment threshold are determined first, and the detection accuracy and the magnitude of the second adjustment threshold are determined again when the recall rate and the magnitude of the first adjustment threshold are greater than the first adjustment threshold.

Further, after the adjustment information is re-determined, whether the adjustment information meets the preset adjustment condition is judged again, and the adjustment information is re-determined again under the condition that the adjustment information does not meet the preset adjustment condition until the adjustment condition is met.

FIG. 3 shows a schematic diagram of a process of determining a confidence threshold in accordance with an embodiment of the present disclosure. As shown in fig. 3, the detection information 32 is determined by inputting each first image data 30 in the first data set into the label generation model 31. Each detection information 32 is screened according to a preset candidate threshold 33 to obtain at least one target detection information 34, and then the adjustment information 35 is determined according to each target detection information 34 and the real label corresponding to the first image data 30.

In a possible implementation manner, the adjustment information 35 includes a recall rate and a detection accuracy, and it is determined whether the recall rate satisfies a first adjustment condition 36, and if so, it is determined whether the detection accuracy satisfies a second adjustment condition 37, and if also satisfied, the current candidate threshold 33 is determined as a confidence threshold 38. When the recall ratio does not satisfy the first adjustment condition 36, the label generation model 31 is updated by retraining or updating the training set, so as to re-determine the adjustment information 35. The candidate threshold value 33 is adjusted when the detection accuracy does not satisfy the second adjustment condition 37 to re-determine the adjustment information 35. Alternatively, the first adjustment condition 36 may be that the recall rate is not less than the first adjustment threshold, and the second adjustment condition 37 may be that the detection accuracy is not less than the second adjustment threshold.

In a possible implementation manner, after the confidence threshold of the tag generation model is determined, each piece of second image data in the second data set may be input into the tag generation model, so as to obtain the detection information of each piece of second image data. Optionally, the detection information of the second image data also includes a confidence level and at least one detection image frame coordinate. Further, other information may also be included, such as a class label that ensures that the type of object is detected. The detection information of each second image data can be screened according to the confidence threshold value, and the detection information meeting the requirement is obtained and used as the pseudo label of the second image data.

Optionally, a specific manner of screening the detection information of the second image data according to the confidence threshold may be to delete the detection information including the confidence level smaller than the confidence threshold, and retain the detection information including the confidence level not smaller than the confidence threshold. And further using the screened detection information as a pseudo label of the corresponding second image data. That is, when the confidence level in the detection information obtained after the second image data is input to the tag generation model is not less than the confidence level threshold, the detection information or the detection result thereof is directly used as the pseudo tag of the second image data.

In a possible implementation manner, after determining the pseudo label of each second image data in the second data set, the second image data having the corresponding pseudo label in the second data set may be acquired, and the defect detection network may be directly trained according to each acquired second image data and the corresponding pseudo label. Optionally, the second image data and the pseudo labels, as well as the training set to which the first preliminary data and the real labels are added, may also be extracted to train the fault detection network according to the determined training set. Wherein the training set may be obtained from uniformly sampling the first image data and the second image data. That is, the data in the first data set and the data in the second data set may be uniformly sampled, a training set may be determined according to sample data obtained by the sampling, and the defect detection model may be trained according to the training set. Furthermore, the first image data in the first data set and the second image data in the second data set have the same source, so as to ensure that the sources of the labeled data in the training set obtained by uniform sampling are the same. For example, when the embodiment of the present disclosure is used to detect a defect of a contact network part of a high-speed rail line, because there is a certain difference in distribution of different high-speed rail line contact network parts, a corresponding first data set and a second data set having a pseudo tag are respectively determined for different high-speed rail lines, so as to ensure that whether the labeled data in the determined training set is from the same high-speed rail line.

Alternatively, the defect detection network may be an untrained blank deep neural network, or a deep neural network that has been trained by a training set of training label generation models.

Fig. 4 is a schematic diagram illustrating a process of training a second model according to an embodiment of the disclosure, and as shown in fig. 4, a label generation model 41 obtained by training is tested through a first data set 40, and a confidence threshold 42 of the label generation model is obtained. Inputting each second image data in the second data set 43 into the tag generation model 41, and obtaining the pseudo tag 44 of each second image data after screening through the confidence threshold 42. Further, the first data set 40 and the second data set 43 added with the pseudo label 44 are uniformly sampled to obtain a training set 45, and the defect detection network 46 is trained according to the training set 45.

According to the embodiment of the invention, the model obtained by adjusting training is used as the automatic labeling model when the defect detection network is trained, and a large amount of unmarked data is automatically labeled based on the automatic labeling model, so that the semi-supervised data automatic labeling process is realized, and the manual labeling cost is saved. Furthermore, a training set is determined through a data set obtained through automatic labeling, and defect detection network training is carried out, so that the training effect of the defect detection network is ensured. The defect detection network trained based on the method can accurately detect the image to be detected, and the accuracy of the defect detection result is improved.

It is understood that the above-mentioned method embodiments of the present disclosure can be combined with each other to form a combined embodiment without departing from the logic of the principle, which is limited by the space, and the detailed description of the present disclosure is omitted. Those skilled in the art will appreciate that in the above methods of the specific embodiments, the specific order of execution of the steps should be determined by their function and possibly their inherent logic.

In addition, the present disclosure also provides a defect detection apparatus, an electronic device, a computer-readable storage medium, and a program, which can be used to implement any defect detection method provided by the present disclosure, and the corresponding technical solutions and descriptions and corresponding descriptions in the methods section are not repeated.

Fig. 5 shows a schematic diagram of a defect detecting apparatus according to an embodiment of the present disclosure, and as shown in fig. 5, the defect detecting apparatus of an embodiment of the present disclosure may include an image acquiring module 50 and a defect detecting module 51.

An image obtaining module 50, configured to obtain an image to be detected;

a defect detection module 51, configured to perform defect detection on the image to be detected through a defect detection network to obtain a defect detection result;

inputting each first image data into a label generation model obtained through training so as to determine detection information of each first image data, wherein the detection information comprises a detection result and a corresponding confidence coefficient;

determining a candidate threshold:

In some embodiments, functions of or modules included in the apparatus provided in the embodiments of the present disclosure may be used to execute the method described in the above method embodiments, and specific implementation thereof may refer to the description of the above method embodiments, and for brevity, will not be described again here.

Embodiments of the present disclosure also provide a computer-readable storage medium having stored thereon computer program instructions, which when executed by a processor, implement the above-mentioned method. The computer readable storage medium may be a volatile or non-volatile computer readable storage medium.

An embodiment of the present disclosure further provides an electronic device, including: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to invoke the memory-stored instructions to perform the above-described method.

The disclosed embodiments also provide a computer program product comprising computer readable code or a non-transitory computer readable storage medium carrying computer readable code, which when run in a processor of an electronic device, the processor in the electronic device performs the above method.

The electronic device may be provided as a terminal, server, or other form of device.

Fig. 6 shows a schematic diagram of an electronic device 800 according to an embodiment of the disclosure. For example, the electronic device 800 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, or the like terminal.

Referring to fig. 6, electronic device 800 may include one or more of the following components: a processing component 802, a memory 804, a power component 806, a multimedia component 808, an audio component 810, an input/output (I/O) interface 812, a sensor component 814, and a communication component 816.

The processing component 802 generally controls overall operation of the electronic device 800, such as operations associated with display, telephone calls, data communications, camera operations, and recording operations. The processing components 802 may include one or more processors 820 to execute instructions to perform all or a portion of the steps of the methods described above. Further, the processing component 802 can include one or more modules that facilitate interaction between the processing component 802 and other components. For example, the processing component 802 can include a multimedia module to facilitate interaction between the multimedia component 808 and the processing component 802.

The memory 804 is configured to store various types of data to support operations at the electronic device 800. Examples of such data include instructions for any application or method operating on the electronic device 800, contact data, phonebook data, messages, pictures, videos, and so forth. The memory 804 may be implemented by any type or combination of volatile or non-volatile memory devices such as Static Random Access Memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic or optical disks.

The power supply component 806 provides power to the various components of the electronic device 800. The power components 806 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the electronic device 800.

The multimedia component 808 includes a screen that provides an output interface between the electronic device 800 and a user. In some embodiments, the screen may include a Liquid Crystal Display (LCD) and a Touch Panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive an input signal from a user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundary of a touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 808 includes a front facing camera and/or a rear facing camera. The front camera and/or the rear camera may receive external multimedia data when the electronic device 800 is in an operation mode, such as a shooting mode or a video mode. Each front camera and rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.

The audio component 810 is configured to output and/or input audio signals. For example, the audio component 810 includes a Microphone (MIC) configured to receive external audio signals when the electronic device 800 is in an operational mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signals may further be stored in the memory 804 or transmitted via the communication component 816. In some embodiments, audio component 810 also includes a speaker for outputting audio signals.

The I/O interface 812 provides an interface between the processing component 802 and peripheral interface modules, which may be keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to: a home button, a volume button, a start button, and a lock button.

The sensor assembly 814 includes one or more sensors for providing various aspects of state assessment for the electronic device 800. For example, the sensor assembly 814 may detect an open/closed state of the electronic device 800, the relative positioning of components, such as a display and keypad of the electronic device 800, the sensor assembly 814 may also detect a change in the position of the electronic device 800 or a component of the electronic device 800, the presence or absence of user contact with the electronic device 800, orientation or acceleration/deceleration of the electronic device 800, and a change in the temperature of the electronic device 800. Sensor assembly 814 may include a proximity sensor configured to detect the presence of a nearby object without any physical contact. The sensor assembly 814 may also include a light sensor, such as a Complementary Metal Oxide Semiconductor (CMOS) or Charge Coupled Device (CCD) image sensor, for use in imaging applications. In some embodiments, the sensor assembly 814 may also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

The communication component 816 is configured to facilitate wired or wireless communication between the electronic device 800 and other devices. The electronic device 800 may access a wireless network based on a communication standard, such as a wireless network (WiFi), a second generation mobile communication technology (2G) or a third generation mobile communication technology (3G), or a combination thereof. In an exemplary embodiment, the communication component 816 receives a broadcast signal or broadcast related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 816 further includes a Near Field Communication (NFC) module to facilitate short-range communications. For example, the NFC module may be implemented based on Radio Frequency Identification (RFID) technology, infrared data association (IrDA) technology, Ultra Wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

In an exemplary embodiment, the electronic device 800 may be implemented by one or more Application Specific Integrated Circuits (ASICs), Digital Signal Processors (DSPs), Digital Signal Processing Devices (DSPDs), Programmable Logic Devices (PLDs), Field Programmable Gate Arrays (FPGAs), controllers, micro-controllers, microprocessors or other electronic components for performing the above-described methods.

In an exemplary embodiment, a non-transitory computer-readable storage medium, such as the memory 804, is also provided that includes computer program instructions executable by the processor 820 of the electronic device 800 to perform the above-described methods.

Fig. 7 shows a schematic diagram of another electronic device 1900 according to an embodiment of the disclosure. For example, the electronic device 1900 may be provided as a server. Referring to fig. 7, electronic device 1900 includes a processing component 1922 further including one or more processors and memory resources, represented by memory 1932, for storing instructions, e.g., applications, executable by processing component 1922. The application programs stored in memory 1932 may include one or more modules that each correspond to a set of instructions. Further, the processing component 1922 is configured to execute instructions to perform the above-described method.

The electronic device 1900 may also include a power component 1926 configured to perform power management of the electronic device 1900, a wired or wireless network interface 1950 configured to connect the electronic device 1900 to a network, and an input/output (I/O) interface 1958. The electronic device 1900 may operate based on an operating system, such as the Microsoft Server operating system (Windows Server), stored in the memory 1932^TM) Apple Inc. of the present application based on the graphic user interface operating System (Mac OS X)^TM) Multi-user, multi-process computer operating system (Unix)^TM) Free and open native code Unix-like operating System (Linux)^TM) Open native code Unix-like operating System (FreeBSD)^TM) Or the like.

In an exemplary embodiment, a non-transitory computer readable storage medium, such as the memory 1932, is also provided that includes computer program instructions executable by the processing component 1922 of the electronic device 1900 to perform the above-described methods.

The present disclosure may be systems, methods, and/or computer program products. The computer program product may include a computer-readable storage medium having computer-readable program instructions embodied thereon for causing a processor to implement various aspects of the present disclosure.

The computer readable storage medium may be a tangible device that can hold and store the instructions for use by the instruction execution device. The computer readable storage medium may be, for example, but is not limited to, an electronic memory device, a magnetic memory device, an optical memory device, an electromagnetic memory device, a semiconductor memory device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of the computer readable storage medium would include the following: a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a Static Random Access Memory (SRAM), a portable compact disc read-only memory (CD-ROM), a Digital Versatile Disc (DVD), a memory stick, a floppy disk, a mechanical coding device, such as punch cards or in-groove projection structures having instructions stored thereon, and any suitable combination of the foregoing. Computer-readable storage media as used herein is not to be construed as transitory signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide or other transmission medium (e.g., optical pulses through a fiber optic cable), or electrical signals transmitted through electrical wires.

The computer-readable program instructions described herein may be downloaded from a computer-readable storage medium to a respective computing/processing device, or to an external computer or external storage device via a network, such as the internet, a local area network, a wide area network, and/or a wireless network. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers and/or edge servers. The network adapter card or network interface in each computing/processing device receives computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in a computer-readable storage medium in the respective computing/processing device.

The computer program instructions for carrying out operations of the present disclosure may be assembler instructions, Instruction Set Architecture (ISA) instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source or object code written in any combination of one or more programming languages, including an object oriented programming language such as Smalltalk, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The computer-readable program instructions may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider). In some embodiments, the electronic circuitry that can execute the computer-readable program instructions implements aspects of the present disclosure by utilizing the state information of the computer-readable program instructions to personalize the electronic circuitry, such as a programmable logic circuit, a Field Programmable Gate Array (FPGA), or a Programmable Logic Array (PLA).

Various aspects of the present disclosure are described herein with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the disclosure. It will be understood that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer-readable program instructions.

These computer-readable program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks. These computer-readable program instructions may also be stored in a computer-readable storage medium that can direct a computer, programmable data processing apparatus, and/or other devices to function in a particular manner, such that the computer-readable medium storing the instructions comprises an article of manufacture including instructions which implement the function/act specified in the flowchart and/or block diagram block or blocks.

The computer readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other devices to cause a series of operational steps to be performed on the computer, other programmable apparatus or other devices to produce a computer implemented process such that the instructions which execute on the computer, other programmable apparatus or other devices implement the functions/acts specified in the flowchart and/or block diagram block or blocks.

The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of instructions, which comprises one or more executable instructions for implementing the specified logical function(s). In some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

The computer program product may be embodied in hardware, software or a combination thereof. In an alternative embodiment, the computer program product is embodied in a computer storage medium, and in another alternative embodiment, the computer program product is embodied in a Software product, such as a Software Development Kit (SDK), or the like.

Having described embodiments of the present disclosure, the foregoing description is intended to be exemplary, not exhaustive, and not limited to the disclosed embodiments. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen in order to best explain the principles of the embodiments, the practical application, or improvements made to the technology in the marketplace, or to enable others of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A method of defect detection, the method comprising:

acquiring an image to be detected;

2. The method of claim 1, wherein the generating of the pseudo tag in the second data set comprises:

3. The method according to claim 1 or 2, wherein the first data set comprises at least two first image data and a real tag for each of the first image data;

4. The method of claim 3, wherein determining the confidence threshold for the tag generation model based on the detection information for each of the first image data and the real tag comprises:

determining a candidate threshold:

5. The method of claim 4, wherein determining adjustment information based on the object detection information and the authentic tag comprises:

6. The method of claim 5, wherein determining recall and/or detection accuracy from the target detection information and the authentic tag comprises:

determining a detection frame set according to each target detection information, wherein the detection frame set comprises at least one detection image frame;

7. The method according to any of claims 4-6, wherein the determining a confidence threshold from the adjustment information and the candidate threshold comprises:

8. The method of claim 7, wherein the adjustment condition is that the recall rate is greater than a first adjustment threshold and the detection accuracy is greater than a second adjustment threshold;

9. A defect detection apparatus, the apparatus comprising:

the image acquisition module is used for acquiring an image to be detected;

10. An electronic device, comprising:

a processor;

a memory for storing processor-executable instructions;

wherein the processor is configured to invoke the memory-stored instructions to perform the method of any of claims 1 to 8.

11. A computer readable storage medium having computer program instructions stored thereon, which when executed by a processor implement the method of any one of claims 1 to 8.