CN113674203A

CN113674203A - Defect detection model training method and device and defect detection method and device

Info

Publication number: CN113674203A
Application number: CN202110796039.9A
Authority: CN
Inventors: 张文超; 张一凡; 冯扬扬; 刘杰
Original assignee: Goertek Inc
Current assignee: Goertek Inc
Priority date: 2021-07-14
Filing date: 2021-07-14
Publication date: 2021-11-19
Also published as: WO2023284712A1

Abstract

The application discloses a defect detection model training method and device and a defect detection method and device. The defect detection model training method comprises the following steps: acquiring a product image, wherein the product image comprises an annular area to be subjected to key detection; extracting the annular area in the product image, and dividing the area form of the annular area into a corner form and a side line form; performing region segmentation and splicing on the annular region according to the corner form and the side line form to obtain a plurality of sub-images; and training a defect detection model corresponding to each subgraph by using each subgraph. The defect detection of the present application includes: obtaining a product image to be detected, extracting an annular region in the product image, and performing region segmentation and splicing on the annular region according to the corner morphology and the side line morphology of the annular region to obtain a plurality of sub-images; and inputting each subgraph into a corresponding defect detection model to obtain a defect detection result corresponding to each subgraph.

Description

Defect detection model training method and device and defect detection method and device

Technical Field

The present application relates to the field of image processing technologies, and in particular, to a defect detection model training method and apparatus, and a defect detection method and apparatus.

Background

In the imaging link of products manufactured in industry, the produced products have various defects of different forms due to the change of environmental factors such as the precision of mechanical positioning, the unstable process of a production line, polishing and the like. In the traditional precision manufacturing industry, the defects of the produced products are generally classified and detected by manual work. In recent years, traditional manufacturing has gradually shifted to smart manufacturing. The target detection based on deep learning has the advantages of low detection cost, stable detection result, less manual requirements, easiness in maintenance and the like, and gradually becomes an important ring for the transition from traditional manufacturing to automation.

In the related art, when the defect detection is performed on the product image based on the defect detection model, the problem that the precision of the defect detection result of the key area is poor due to the fact that the detection standards of the whole product in the product image are inconsistent exists.

Disclosure of Invention

The present application aims to solve at least one of the above technical drawbacks, and particularly proposes a technical solution for improving the defect detection precision by obtaining an annular region that needs to be focused on from a product image to perform defect detection.

The embodiment of the application adopts the following technical scheme:

in one aspect of the embodiments of the present application, a method for training a defect detection model is provided, including: acquiring a product image, wherein the product image comprises an annular area to be subjected to key detection;

extracting the annular area in the product image, and dividing the area form of the annular area into a corner form and a side line form;

performing region segmentation and splicing on the annular region according to the corner form and the side line form to obtain a plurality of sub-images;

and training a defect detection model corresponding to each subgraph by using each subgraph.

In another aspect of the embodiments of the present application, a defect detection method is further provided, including: obtaining a product image to be detected, extracting an annular region in the product image, and performing region segmentation and splicing on the annular region according to the corner morphology and the side line morphology of the annular region to obtain a plurality of sub-images;

inputting each subgraph into a corresponding defect detection model to obtain a defect detection result corresponding to each subgraph; the defect detection model is obtained by training the defect detection model training method.

In another aspect of the embodiment of the present application, a defect detection model training apparatus is further provided, which is used for implementing the defect detection model training method.

In another aspect of the embodiments of the present application, a defect detection apparatus is further provided, which is used to implement the defect detection method.

The embodiment of the application adopts at least one technical scheme which can achieve the following beneficial effects:

the defect detection standard of the whole product surface is inconsistent, and the key region to be detected is a scene of the annular region, the annular region focused on in the product image is extracted, because the defect detection standard of the annular region is consistent, as long as the defect type of the product image is relatively comprehensive, a defect detection model suitable for the annular region can be trained quickly by means of a small amount of training set data, and the annular region is divided and spliced, so that the interference of a large redundant region in the middle of the annular region can be removed, and the model training speed and the model detection efficiency are improved.

Drawings

The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this application, illustrate embodiment(s) of the application and together with the description serve to explain the application and not to limit the application. In the drawings:

FIG. 1 is a flowchart of a defect detection model training method shown in an embodiment of the present application;

fig. 2 is a schematic diagram of a network structure of the Unet shown in the embodiment of the present application;

FIG. 3 is a mask diagram shown in an embodiment of the present application;

FIG. 4 is a schematic view of the annular region segmentation shown in the embodiments of the present application;

FIG. 5 is a first sub-diagram schematic diagram shown in the embodiment of the present application;

fig. 6 is a second sub-diagram schematic diagram before the splicing process shown in the embodiment of the present application;

fig. 7 is a schematic diagram of a second sub-diagram after splicing shown in the embodiment of the present application;

FIG. 8 is a schematic view of a defect detection process shown in an embodiment of the present application;

FIG. 9 is a flowchart of a defect detection method shown in an embodiment of the present application;

FIG. 10 is a block diagram of a defect detection model training apparatus shown in an embodiment of the present application;

fig. 11 is a block diagram of a defect detection apparatus shown in an embodiment of the present application;

fig. 12 is a schematic view of an electronic device shown in an embodiment of the present application.

Detailed Description

In order to make the objects, technical solutions and advantages of the present application more apparent, the technical solutions of the present application will be described in detail and completely with reference to the following specific embodiments of the present application and the accompanying drawings. It should be apparent that the described embodiments are only some of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.

In the process of researching and practicing product defect detection, the inventor of the present application finds that a partial region of a product has a large influence on performance and efficiency and has a low tolerance to a defect, for example, a magnetic gap region of a magnetic circuit product has a low tolerance to a defect compared with other regions, and if a global product image including the partial region and other regions of the product is used as an image to be detected for defect detection, the defect detection accuracy of the partial region is affected.

In view of the above problems, the inventors of the present application thought: in an application scene that the defect detection standards on the whole product surface are inconsistent and the key area to be detected is an annular area, the annular area is divided and spliced to obtain a spliced image which can cover the original annular area with the minimum area, and the defect detection precision and efficiency are improved by utilizing the characteristics that defects in the spliced image are more concentrated and the defect deformation is smaller.

The technical solutions provided by the embodiments of the present application are described in detail below with reference to the accompanying drawings.

An embodiment of the present application provides a defect detection model training method, and fig. 1 is a flowchart of a defect detection model training method shown in the embodiment of the present application, and as shown in fig. 1, the method at least includes the following steps S110 to S140:

step S110, acquiring a product image, wherein the product image comprises an annular area to be subjected to emphasis detection.

When the defect detection model is trained, a certain number of product images are required to be obtained as original training samples. The specific method for acquiring the product image can be flexibly set by a person skilled in the art according to actual needs, and is not specifically limited herein.

Step S120, extracting an annular region in the product image, and dividing the region shape of the annular region into a corner shape and an edge shape.

After the original product image is obtained, the product image needs to be subjected to area detection to determine whether an annular area with lower tolerance to defects than other areas exists in the product image.

For the condition that the annular region exists, the region form of the annular region is further divided into a corner form and a side line form, so that the annular region can be subjected to image segmentation and splicing according to the corner form and the side line form, the purpose of removing the interference of a large redundant region in the middle of the annular region is achieved, and the redundant region is prevented from wasting computing power and reasoning time during defect detection model learning.

And S130, performing region segmentation and splicing on the annular region according to the corner form and the side line form to obtain a plurality of subgraphs.

In order to remove the redundant area of the larger middle piece of the annular area, the embodiment performs area segmentation on the annular area based on the corner shape and the edge shape, and segments the corner area conforming to the corner shape and the edge area conforming to the edge shape from the annular area, and the corner area and the edge area segmented based on the method can completely cover the original annular area.

And step S140, training a defect detection model corresponding to each subgraph by using each subgraph.

After subgraphs with more concentrated defects and consistent detection standards are obtained, the subgraphs can be input into a corresponding defect detection model for training, and the training can be stopped after the detection effect reaches the expectation.

It should be noted that, in this embodiment, in order to improve the accuracy of the defect detection model, a corresponding defect detection model is constructed for each sub-graph, for example, when two sub-graphs are obtained by splicing, two defect detection models are constructed, and the defect detection models are trained by using the sub-graphs of the corresponding types, respectively.

In the embodiment shown in fig. 1, the annular region of key interest in the product image is extracted, and as the defect detection standards in the annular region are consistent, as long as the defect types in the product image are relatively comprehensive, a defect detection model suitable for the annular region can be quickly trained by means of a small amount of training set data, and by segmenting and splicing the annular region, the interference of a large redundant region in the middle of the annular region can be removed, and the model training speed and the model detection efficiency are improved.

In one embodiment of the present application, extracting an annular region in the product image comprises: processing the product image by using the trained image segmentation network, generating a mask image corresponding to the annular region by using the image segmentation network, and extracting the annular region from the product image according to the mask image.

In the embodiment, when the annular region is extracted, the annular region to be subjected to highlight detection can be extracted from the product image by using an image segmentation algorithm, a shallow Unet network structure is preferably used, and the Unet network can quickly complete the fitting task of the image designated region on small sample data of dozens of images.

When the annular region is extracted by using the Unet network, the Unet network needs to be trained by using sample data. The sample data here refers to: an original sample image and its corresponding mask image. The annular region to be subjected to highlight detection in the original sample image can be marked by using a marking tool labelme to generate a json marking file, a mask image with the same size as the original sample image is generated according to the json marking file, the gray value of the annular region to be subjected to highlight detection in the mask image is 1, and the gray values of other positions are 0. Thus, each sample data in the sample set has a corresponding mask image.

When the Unet network is trained, an original sample picture and a corresponding mask image are simultaneously input into the Unet network, as shown in FIG. 2, the first half part of the Unet network performs feature extraction on the original sample picture through a convolution kernel of 3x3, the number of the first two convolution kernels is 64, every two subsequent convolution operations are increased, the number of the convolution kernels is changed to 2 times of the original number, every two convolution kernels are matched with a maximum pooling operation of 2x2, and the length and the width of a feature map are reduced by half after every maximum pooling operation; the number of the convolution kernels in the second half part is reduced from 1024, the number of the convolution kernels is reduced by half after each convolution operation, and the length and the width of the feature map are doubled after each up-sampling matched with the up-sampling.

In this embodiment, a new feature fusion mode of concatenation (copy and crop) is added to the second half of the Unet network, and the feature maps acquired in the first half are subjected to channel-level concatenation, that is, the size and the number of channels of the feature maps obtained by concatenation are the sum of the two feature maps to be concatenated.

And then, comparing the mask image generated at the rear half part of the Unet network with the mask image corresponding to the original sample picture, calculating a loss function, and continuously correcting the model parameters of the Unet network.

The loss function adopts a Dice-coeffient loss function, the Dice-coeffient loss function has a better effect on the class imbalance problem compared with the Unet original loss function, and the similarity between the mask image generated by the Unet network and the original mask image can be measured quickly.

After the Unet network is trained, the acquired product image can be input into the Unet network, and the mask image corresponding to the annular region is obtained through the output of the Unet network.

In this way, the annular region of the pixel position in the product image can be marked according to the pixel position of the annular region in the adjusted mask image, so that the annular region shown in fig. 3 can be obtained, and at this time, the annular region can be extracted from the product image by using an image cropping technology.

As can be seen from fig. 3, the extracted annular region includes a large middle redundant region, which not only wastes computation power and inference time during learning of the defect detection model, but also is not beneficial to defect detection of the annular region. To solve the problem, in the embodiment of the present application, the annular region is segmented according to the corner shape and the edge shape of the annular region, so as to obtain four corner regions and four edge regions, the four corner regions are spliced into a first sub-graph as shown in fig. 5, and the four edge regions are spliced into a second sub-graph as shown in fig. 7.

When the annular area is segmented, four corner areas which accord with corner forms are extracted according to a first size proportion, two first type edge areas which accord with edge forms are extracted according to a second size proportion, and two second type edge areas which accord with edge forms are extracted according to a third size proportion, wherein the first size proportion, the second size proportion and the third size proportion indicate the proportion of the extracted area according to the first size proportion, the second size proportion or the third size proportion and the length and the width of the annular area in the product image. Taking the example of extracting the corner region, the length of the corner region is proportional to the length of the annular region, and the width of the corner region is proportional to the width of the annular region, and these two proportional values are the first size proportion.

In practical application, the annular area is further subjected to segmentation processing according to the size information of the product defects, so that the extracted corner area and the edge area have an overlapping area, and the overlapping area can be set according to the size of the minimum product defect on the annular area.

Taking the splitting scenario shown in fig. 4 as an example, the length and width of the corner region are exemplarily set to 1/5 and 1/4 of the length and width of the original annular region, and the size of the overlap region is set to the size of the smallest product defect on the annular region, so that four corner regions that mark the annular region in the clockwise direction with the corner region and the edge region shown in fig. 4 are sequentially

regions

1, 2, 3, and 4; according to the direction of the long edge of the annular area, marking two first-class edge areas of the annular area as 5 and 6 in sequence; according to the direction of the wide side of the annular area, the two second type edge areas of the marked annular area are 7 and 8 in sequence.

Since the four corner regions of the ring region are the main reason for constituting the large redundant region in fig. 3, the area of the redundant region can be effectively reduced by splicing the four corner regions together. During splicing, the four corner regions are spliced into the annular region according to the original positions of the four corner regions in the annular region, and referring to fig. 4, the four

corner regions

1, 2, 3 and 4 are spliced clockwise into a first sub-graph which is approximately square.

Because the segmented side line regions comprise the first side line region and the second side line region, the image forms of the two first side line regions are completely the same, and the image forms of the two second side line regions are also completely the same, before the side line regions are spliced, the two first side line regions can be spliced to obtain the first side line splicing region, and the two second side line regions can be spliced to obtain the second side line splicing region. As shown in fig. 5 and fig. 6, the length of the first edge-like region is greater than that of the second edge-like rectangle, and the directions of the long sides of the first edge-like region and the second edge-like rectangle are different.

On the basis, the lengths and the directions of two sideline splicing areas obtained by splicing the sideline areas need to be subjected to consistency adjustment, on one hand, the directions of the lengths of the first sideline splicing area and the second sideline splicing area are processed into the same direction, for example, the first sideline splicing area is rotated by a preset angle to enable the first sideline splicing area and the second sideline splicing area to be in the same direction; on the other hand, the length of the first edge splicing region is adjusted and aligned with the length of the second edge splicing region, for example, the first edge splicing region is shortened, stretched or spliced, so that the first edge splicing region is aligned with the second edge splicing region in length.

In an embodiment, after the directions of the lengths of the first edge splicing area and the second edge splicing area are processed to be the same direction, the length of the first edge splicing area is set to be the same as the length of the second edge splicing area, and the first edge splicing area and the second edge splicing area which have the same length and the same length direction are subjected to area splicing to obtain a second sub-image.

Referring to fig. 7, in the present embodiment, after the

border line regions

7 and 8 are spliced, they are rotated 90 degrees counterclockwise and then are disposed at the lower sides of the

border line regions

5 and 6, because the length of the

border line regions

5 and 6 is greater than that of the

border line regions

7 and 8, the height of the

border line regions

5 and 6 is not changed, and the length resize is (adjusted) to the length of the

border line regions

7 and 8; and then spliced with the

border regions

7, 8 to form the final second sub-image 2. In the process of splicing the second sub-image, the lengths of the splicing areas of the two sidelines are adjusted to be flush, so that the defect generates small deformation.

Based on the second sub-graph stitching method for shortening the length of the first edge line region, if the first edge line stitching region has a defect, the width of the defect is not changed, the length of the defect is shortened, and the defect is slightly deformed. Referring to fig. 6 and 7, the defect, indicated by the circle in the border area 6, is unchanged in width but shortened in length, whereas the different defects, indicated by the diamond in the border area 8, are not deformed.

Of course, in practical applications, the height of the

edge line regions

7 and 8 may be unchanged, and the length resize may be equal to the length of the

edge line regions

5 and 6, or a blank region with the same width may be spliced at one end of the

edge line regions

7 and 8, and the alignment in the length direction of the two edge line splicing regions may be achieved through the blank region. Generally, when sub-images are spliced, the last spliced sub-layer is close to a square as much as possible, so that the deformation amount of defects is smaller in the model learning process.

It should be noted that, in the above embodiment, the four edge regions are spliced into one sub-graph, in practical application, the two first edge regions may also be spliced into one sub-graph, and the two second edge regions may also be spliced into another sub-graph. A specific stitching scheme may refer to the edge region aspect ratio, for example, when the edge region aspect ratio solves 2: 1, the first-class edge regions can be spliced into a subgraph.

The first sub-graph and the second sub-graph obtained by the embodiment have at least the following advantages:

(1) the first sub-graph and the second sub-graph can cover the annular area to be subjected to key point detection by the minimum splicing area, and simultaneously avoid the interference of the black redundant area of the middle large piece;

(2) the same defect types in the two subgraphs are more concentrated, for example, the same defect types (such as impurities, wool fibers and other defects) in the first subgraph are more concentrated, and several specific defect types (such as Huasi crush, edge magnetic damage and other defects) in the second subgraph are also more concentrated, so that the concentrated training and learning of a defect detection model are facilitated;

(3) compared with the size of an original product image 2448x2048, the sub-image size is smaller, about only 400x400 pixels are obtained, and time is saved during training and prediction;

(4) the length-width ratio of the two obtained subgraphs is close to 1, so that the deformation of the defect during training and defect detection is minimum, the influence on the defect form is minimum, and the defect detection precision can be improved.

After a first sub-graph corresponding to the corner shape of the annular region and a second sub-graph corresponding to the edge shape of the annular region are obtained, the first sub-graph can be used for training a first defect detection model, and the second sub-graph can be used for training a second defect detection model, wherein the defect detection model can be a yolov4 network structure, and can be a network structure in other forms, and the defect detection model can be flexibly selected in practical application, and the embodiment of the application is not particularly limited.

According to the embodiment of the application, the original product image is subjected to annular region recognition, extraction, segmentation, splicing and other processing, and a training set is constructed based on the subgraph obtained by the processing, so that a defect detection model with high precision and high efficiency can be trained quickly under the condition that the defect types in the training set are comprehensive.

As shown in fig. 8, a defect detection flow schematic of the embodiment of the present application is provided. Firstly, training a Unet network, extracting an annular region to be subjected to key point detection by using the trained Unet network, zeroing pixels of other regions except the annular region in a product image, segmenting four corner regions from the annular region and splicing into a first subgraph, segmenting four edge regions from the annular region and splicing into a second subgraph, training a first defect detection model by using the first subgraph, and training a second defect detection model by using the second subgraph, thereby completing model training.

After the model training is completed, the trained defect detection model can be used for detecting the defects of the product image to be detected.

An embodiment of the present application provides a defect detection method, fig. 9 is a flowchart of the defect detection method shown in the embodiment of the present application, and as shown in fig. 9, the method at least includes the following steps S910 to S920:

step S910, a product image to be detected is obtained, an annular region in the product image is extracted, and the annular region is subjected to region segmentation and splicing according to the corner morphology and the side line morphology of the annular region to obtain a plurality of sub-images.

After the to-be-detected product image is obtained, the to-be-detected product image can be processed according to the processing process of the training data in the model training process, and a plurality of sub-images are obtained.

Step S920, inputting each subgraph into a corresponding defect detection model to obtain a defect detection result corresponding to each subgraph; wherein the defect detection model is trained based on the defect detection model training method described above.

In an embodiment of the present application, inputting each sub-graph into a corresponding defect detection model to obtain a defect detection result corresponding to each sub-graph, includes: and marking the defect detection results in the multiple sub-images in the product image to be detected according to the pixel positions of the sub-images in the product image to be detected.

In the production line application, if only an NG (Not Good) product and NG categories need to be detected without locating specific positions of defects in the product, in this case, to reduce time consumption, detection results of two subgraphs can be directly fed back without merging subgraph detection results into an original product image. If the defect position needs to be emphasized in theoretical research or application, the defect detection results of the two sub-images need to be fed back to the original product image according to the cut coordinates and labeled on the original product image so as to determine the defect position.

An embodiment of the present application provides a defect detection model training apparatus, fig. 10 is a block diagram of a structure of the defect detection model training apparatus shown in the embodiment of the present application, and as shown in fig. 10, an apparatus 1000 in the embodiment includes:

an image obtaining unit 1010, configured to obtain a product image, where the product image includes an annular region to be subjected to highlight detection;

a region processing unit 1020, configured to extract the annular region in the product image, and divide a region shape of the annular region into a corner shape and an edge shape; performing region segmentation and splicing on the annular region according to the corner form and the side line form to obtain a plurality of sub-images;

and a model training unit 1030, configured to train, with each sub-graph, a defect detection model corresponding to the sub-graph.

In some embodiments, the zone processing unit 1020 includes: a splitting module and a splicing module;

the segmentation module is used for segmenting the annular area according to the corner forms and the side line forms to obtain four corner areas and four side line areas;

and the splicing module is used for splicing the four corner areas into a first sub-image and splicing the four edge areas into a second sub-image.

In some embodiments, the segmentation module is further configured to extract four corner regions conforming to the corner shape according to a first size scale, extract two first edge-like regions conforming to the edge shape according to a second size scale, and extract two second edge-like regions conforming to the edge shape according to a third size scale.

In some embodiments, the dividing module is further configured to perform a dividing process on the annular region according to the size information of the product defect, so that the extracted corner region and the edge region have an overlapping region.

In some embodiments, the splicing module is configured to splice the four corner regions into the annular region according to the original orientations of the four corner regions in the annular region; splicing the two first type side line areas to obtain a first side line splicing area, and splicing the two second type side line areas to obtain a second side line splicing area; processing the direction of the length of the first sideline splicing area and the second sideline splicing area into the same direction, and setting the length of the first sideline splicing area to be the same as the length of the second sideline splicing area; and carrying out region splicing on the first side line splicing region and the second side line splicing region which have the same length and the same length direction to obtain a second sub-image.

In some embodiments, the zone processing unit 1020 further comprises: the extraction module is used for processing the product image by utilizing the trained image segmentation network and generating a mask image corresponding to the annular region through the image segmentation network; and extracting an annular region from the product image according to the mask image.

In some embodiments, the extraction module is further configured to adjust an image size of the mask image to obtain an adjusted mask image, where the size of the adjusted mask image is the same as the size of the product image; and marking the annular region of the pixel position in the product image according to the pixel position of the annular region in the adjusted mask image.

It can be understood that the defect detection model training apparatus can implement the steps of the defect detection model training method provided in the foregoing embodiment, and the explanations related to the defect detection model training method are applicable to the defect detection model training apparatus, and are not repeated herein.

An embodiment of the present application provides a defect detection apparatus, fig. 11 is a block diagram of a structure of the defect detection apparatus shown in the embodiment of the present application, and as shown in fig. 11, an apparatus 1100 in the embodiment includes:

the image processing unit 1110 is configured to acquire a product image to be detected, extract an annular region in the product image, and perform region segmentation and splicing on the annular region according to a corner form and a side line form of the annular region to obtain a plurality of sub-images;

a defect detection unit 1120, configured to input each sub-image into a corresponding defect detection model, so as to obtain a defect detection result corresponding to each sub-image;

wherein the defect detection model is trained based on the defect detection model training method described above.

Fig. 12 is a schematic structural diagram of an electronic device according to an embodiment of the present application. Referring to fig. 12, at a hardware level, the electronic device includes a processor and a memory, and optionally further includes an internal bus and a network interface. The Memory may include a Memory, such as a Random-Access Memory (RAM), and may further include a non-volatile Memory, such as at least 1 disk Memory. Of course, the electronic device may also include hardware required for other services.

The processor, the network interface, and the memory may be connected to each other via an internal bus, which may be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus, an EISA (Extended Industry Standard Architecture) bus, or the like. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one double-headed arrow is shown in FIG. 12, but that does not indicate only one bus or one type of bus.

And the memory is used for storing programs. In particular, the program may include program code comprising computer operating instructions. The memory may include both memory and non-volatile storage and provides instructions and data to the processor.

The processor reads a corresponding computer program from the nonvolatile memory into the memory and then runs the computer program to form a defect detection model training device or a defect detection device on a logic level. The processor is used for executing the program stored in the memory and is specifically used for executing the following operations:

acquiring a product image, wherein the product image comprises an annular area to be subjected to key detection;

Or, to perform the following operations:

obtaining a product image to be detected, extracting an annular region in the product image, and performing region segmentation and splicing on the annular region according to the corner morphology and the side line morphology of the annular region to obtain a plurality of sub-images;

inputting each subgraph into a corresponding defect detection model to obtain a defect detection result corresponding to each subgraph; wherein the defect detection model is trained based on the defect detection model training method described above.

The method performed by the defect detection model training apparatus disclosed in the embodiment of fig. 1 of the present application may be applied to a processor, or the method performed by the defect detection apparatus disclosed in the embodiment of fig. 9 of the present application may be applied to a processor, or implemented by a processor. The processor may be an integrated circuit chip having signal processing capabilities. In implementation, the steps of the above method may be performed by integrated logic circuits of hardware in a processor or instructions in the form of software. The Processor may be a general-purpose Processor, including a Central Processing Unit (CPU), a Network Processor (NP), and the like; but also Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field Programmable Gate Arrays (FPGAs) or other Programmable logic devices, discrete Gate or transistor logic devices, discrete hardware components. The various methods, steps, and logic blocks disclosed in the embodiments of the present application may be implemented or performed. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like. The steps of the method disclosed in connection with the embodiments of the present application may be directly implemented by a hardware decoding processor, or implemented by a combination of hardware and software modules in the decoding processor. The software module may be located in ram, flash memory, rom, prom, or eprom, registers, etc. storage media as is well known in the art. The storage medium is located in a memory, and a processor reads information in the memory and completes the steps of the method in combination with hardware of the processor.

The electronic device may also perform the method performed by the defect detection model training apparatus of fig. 1, and implement the functions of the defect detection model training apparatus in the embodiment shown in fig. 1; alternatively, the electronic device may further execute the method executed by the defect detection apparatus in fig. 9, and implement the function of the defect detection apparatus in the embodiment shown in fig. 1, which is not described herein again in this embodiment of the present application.

The present application also provides a computer-readable storage medium storing one or more programs, where the one or more programs include instructions, which when executed by an electronic device including a plurality of application programs, enable the electronic device to perform the method performed by the defect detection model training apparatus in the embodiment shown in fig. 1, or perform the method performed by the defect detection apparatus in the embodiment shown in fig. 9.

As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.

The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

In a typical configuration, a computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.

The memory may include forms of volatile memory in a computer readable medium, Random Access Memory (RAM) and/or non-volatile memory, such as Read Only Memory (ROM) or flash memory (flash RAM). Memory is an example of a computer-readable medium.

Computer-readable media, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), Read Only Memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), Digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information that can be accessed by a computing device. As defined herein, a computer readable medium does not include a transitory computer readable medium such as a modulated data signal and a carrier wave.

It should also be noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.

As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.

The above description is only an example of the present application and is not intended to limit the present application. Various modifications and changes may occur to those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims

1. A defect detection model training method is characterized by comprising the following steps:

2. The method of claim 1, wherein performing region segmentation and stitching on the annular region to obtain a plurality of subgraphs comprises:

according to the corner forms and the side line forms, the annular area is subjected to segmentation processing to obtain four corner areas and four side line areas;

the four corner regions are spliced into a first sub-graph, and the four edge regions are spliced into a second sub-graph.

3. The method of claim 2, wherein segmenting the annular region into four corner regions and four edge regions according to the corner shape and the rectangular shape comprises:

four corner regions conforming to the corner morphology are extracted according to a first size ratio,

extracting two first edge line areas according with the edge line shape according to the second size proportion,

and extracting two second type edge regions which accord with the edge shape according to the third size proportion.

4. The method of claim 3, wherein when the annular region is subjected to the segmentation process based on the corner shape and the rectangular shape, further comprising:

and performing segmentation processing on the annular area according to the size information of the product defect, so that the extracted corner area and the edge area have an overlapping area.

5. The method of claim 2, wherein stitching four corner regions into a first subgraph comprises:

and splicing the four corner regions into the annular region according to the original orientation of the four corner regions in the annular region.

6. The method of claim 3, wherein the first type of edge region has a length greater than the second type of edge rectangle, and wherein stitching the four edge regions into a second subgraph comprises:

splicing the two first type side line areas to obtain a first side line splicing area, and splicing the two second type side line areas to obtain a second side line splicing area;

processing the direction of the length of the first sideline splicing area and the second sideline splicing area into the same direction, and setting the length of the first sideline splicing area to be the same as the length of the second sideline splicing area;

and carrying out region splicing on the first side line splicing region and the second side line splicing region which have the same length and the same length direction to obtain a second sub-image.

7. The method of claim 1, wherein extracting an annular region in the product image comprises:

processing the product image by using the trained image segmentation network, and generating a mask image corresponding to the annular region through the image segmentation network;

and extracting an annular region from the product image according to the mask image.

8. The method of claim 7, wherein extracting an annular region from the product image based on the mask image comprises:

adjusting the image size of the mask image to obtain an adjusted mask image, wherein the size of the adjusted mask image is the same as that of the product image;

and marking the annular region of the pixel position in the product image according to the pixel position of the annular region in the adjusted mask image.

9. A method of defect detection, comprising:

inputting each subgraph into a corresponding defect detection model to obtain a defect detection result corresponding to each subgraph;

the defect detection model is obtained by training based on the defect detection model training method of any one of claims 1-8.

10. A defect detection model training device, characterized in that the device is used for realizing the defect detection model training method of any one of claims 1 to 8.

11. A defect detection apparatus, characterized in that said apparatus is adapted to implement the defect detection method of claim 9.