CN113066069A - Adjusting method and device, adjusting equipment and storage medium - Google Patents
Adjusting method and device, adjusting equipment and storage medium Download PDFInfo
- Publication number
- CN113066069A CN113066069A CN202110351892.XA CN202110351892A CN113066069A CN 113066069 A CN113066069 A CN 113066069A CN 202110351892 A CN202110351892 A CN 202110351892A CN 113066069 A CN113066069 A CN 113066069A
- Authority
- CN
- China
- Prior art keywords
- coefficient
- adjustment
- detection model
- image
- adjusting
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0004—Industrial image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10004—Still image; Photographic image
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Artificial Intelligence (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Life Sciences & Earth Sciences (AREA)
- General Engineering & Computer Science (AREA)
- Quality & Reliability (AREA)
- Image Analysis (AREA)
Abstract
An adjustment method, an adjustment apparatus, an adjustment device, and a non-volatile computer-readable storage medium. The adjusting method comprises the steps of inputting an image to be detected to a detection model so as to output the confidence coefficient of the image to be detected; determining an adjusting coefficient according to the confidence coefficient, wherein the adjusting coefficient is in negative correlation with the confidence coefficient; calculating a loss value according to the adjusting coefficient and a preset loss function; and adjusting the detection model according to the loss value so that the detection model converges. The confidence coefficient of the image to be detected is output by inputting the image to be detected to the detection model, so that the adjustment coefficient is determined according to the confidence coefficient, the adjustment coefficient is larger when the confidence coefficient is lower (namely the image to be detected is more difficult), the influence on the detection model is larger when the detection model is adjusted based on the loss value calculated by the adjustment coefficient, the training effect of the detection model on a difficult sample is improved, the accuracy of the detection model can be ensured even if the number of the difficult sample is small, and missing detection or over-detection is prevented.
Description
Technical Field
The present application relates to the field of detection technologies, and in particular, to an adjustment method, an adjustment apparatus, an adjustment device, and a non-volatile computer-readable storage medium.
Background
At present, when a wafer is detected, images of the wafer are generally collected to serve as samples, but the probability of the defects of different types of the wafer is different, so that the difficulty of the samples is different, the probability of the defects is higher, the more the samples are, the simpler the samples are, and the more the samples are, and vice versa, when a detection model is trained, the less the number of the difficult samples is, so that the training effect of the detection model on the difficult samples is poor, the accuracy of the detection model when the defects of the wafer are detected is low, and the missed detection or the over-detection is easy to occur.
Disclosure of Invention
An adjustment method, an adjustment apparatus, an adjustment device, and a non-volatile computer-readable storage medium are provided.
The adjusting method comprises the steps of inputting an image to be detected to a detection model so as to output the confidence coefficient of the image to be detected; determining an adjusting coefficient according to the confidence coefficient, wherein the adjusting coefficient is in negative correlation with the confidence coefficient; calculating a loss value according to the adjusting coefficient and a preset loss function; and adjusting the detection model according to the loss value so that the detection model converges.
The adjusting device of the embodiment of the application comprises a first input and output module, a determining module, a calculating module and an adjusting module. The first input and output module is used for inputting an image to be detected to a detection model so as to output the confidence coefficient of the image to be detected; the determining module is used for determining an adjusting coefficient according to the confidence coefficient, and the adjusting coefficient is inversely related to the confidence coefficient; the calculation module is used for calculating a loss value according to the adjustment coefficient and a preset loss function; and the adjusting module is used for adjusting the detection model according to the loss value so as to make the detection model converge.
The adjusting device of the embodiment of the application comprises a processor. The processor is configured to: inputting an image to be detected to a detection model so as to output the confidence coefficient of the image to be detected; determining an adjusting coefficient according to the confidence coefficient, wherein the adjusting coefficient is in negative correlation with the confidence coefficient; calculating a loss value according to the adjusting coefficient and a preset loss function; and adjusting the detection model according to the loss value so that the detection model converges.
A non-transitory computer-readable storage medium containing a computer program of embodiments of the application, which, when executed by one or more processors, causes the processors to perform the adjustment method. The adjusting method comprises the steps of inputting an image to be detected to a detection model so as to output the confidence coefficient of the image to be detected; determining an adjusting coefficient according to the confidence coefficient, wherein the adjusting coefficient is in negative correlation with the confidence coefficient; calculating a loss value according to the adjusting coefficient and a preset loss function; and adjusting the detection model according to the loss value so that the detection model converges.
According to the adjusting method, the adjusting device, the adjusting equipment and the nonvolatile computer readable storage medium, the image to be detected is input into the detection model to output the confidence coefficient of the image to be detected, so that the adjusting coefficient is determined according to the confidence coefficient, the lower the confidence coefficient is, the lower the detection accuracy of the image to be detected by the detection model is, and the harder the image to be detected is, therefore, the lower the confidence coefficient is (namely, the harder the image to be detected is), the larger the adjusting coefficient is, the larger the influence on the detection model is when the detection model is adjusted based on the loss value calculated by the adjusting coefficient is, the training effect of the detection model on a difficult sample is improved, and even if the number of the difficult samples is small, the accuracy of the detection model can be ensured, and missing detection or over-detection is prevented.
Additional aspects and advantages of the present application will be set forth in part in the description which follows and, in part, will be obvious from the description, or may be learned by practice of the present application.
Drawings
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings needed to be used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative efforts.
FIG. 1 is a schematic flow chart of a tuning method according to some embodiments of the present application;
FIG. 2 is a block schematic diagram of an adjustment apparatus according to certain embodiments of the present application;
FIG. 3 is a schematic plan view of an adjustment apparatus according to certain embodiments of the present application;
FIG. 4 is a schematic flow chart of an adjustment method according to some embodiments of the present application;
FIG. 5 is a schematic diagram of the conditioning method of certain embodiments of the present application;
FIG. 6 is a schematic flow chart of an adjustment method according to some embodiments of the present application;
FIG. 7 is a schematic flow chart of an adjustment method according to some embodiments of the present application;
FIG. 8 is a schematic flow chart of an adjustment method according to some embodiments of the present application;
FIGS. 9-13 are schematic illustrations of an adjustment method according to certain embodiments of the present application; and
FIG. 14 is a schematic diagram of a connection between a processor and a computer-readable storage medium according to some embodiments of the present application.
Detailed Description
Embodiments of the present application will be further described below with reference to the accompanying drawings. The same or similar reference numbers in the drawings identify the same or similar elements or elements having the same or similar functionality throughout. In addition, the embodiments of the present application described below in conjunction with the accompanying drawings are exemplary and are only for the purpose of explaining the embodiments of the present application, and are not to be construed as limiting the present application.
Referring to fig. 1 to 3, an adjustment method according to an embodiment of the present disclosure includes the following steps:
011: inputting an image to be detected to a detection model so as to output the confidence coefficient of the image to be detected;
012: determining an adjusting coefficient according to the confidence coefficient, wherein the adjusting coefficient is negatively correlated with the confidence coefficient;
013: calculating a loss value according to the adjustment coefficient and a preset loss function; and
014: the detection model is adjusted according to the loss value so that the detection model converges.
The adjusting apparatus 10 of the embodiment of the present application includes a first input/output module 11, a determining module 12, a calculating module 13, and an adjusting module 14. The first input/output module 11 is configured to input an image to be detected to the detection model, so as to output a confidence level of the image to be detected; the determining module 12 is configured to determine an adjustment coefficient according to the confidence level, where the adjustment coefficient and the confidence level are negatively correlated; the calculating module 13 is configured to calculate a loss value according to the adjustment coefficient and a preset loss function; the adjusting module 14 is configured to adjust the detection model according to the loss value, so that the detection model converges. That is, step 011 can be implemented by the first input-output module 11, step 012 can be performed by the determination module 12, step 013 can be performed by the calculation module 13, and step 014 can be performed by the adjustment module 14.
The adjustment device 100 of the embodiment of the present application includes a processor 20. The processor 20 is configured to input the image to be detected to the detection model, so as to output a confidence level of the image to be detected; determining an adjusting coefficient according to the confidence coefficient, wherein the adjusting coefficient is negatively correlated with the confidence coefficient; calculating a loss value according to the adjustment coefficient and a preset loss function; and adjusting the detection model according to the loss value so that the detection model converges. That is, step 011, step 012, step 013, and step 014 may be performed by processor 20.
Specifically, the to-be-measured image may be obtained by shooting the to-be-measured workpiece 200 through the adjusting apparatus 100, and the to-be-measured workpiece 200 may be a wafer. The adjustment device 100 may be a measuring machine. It is understood that the specific form of the adjustment apparatus 100 is not limited to a measuring machine, but may be any apparatus capable of detecting the workpiece 200 to be detected, and the workpiece 200 to be detected is not limited to a wafer.
The adjustment device 100 comprises a processor 20, a motion platform 30 and a sensor 40. Both the processor 20 and the sensor 40 may be located on the motion platform 30. The motion platform 30 can be used to carry the workpiece 200 to be measured, and the motion platform 30 moves to drive the sensor 40 to move, so that the sensor 40 collects information of the workpiece 200 to be measured, thereby generating an image to be measured.
For example, the motion platform 30 includes an XY motion platform 31 and a Z motion platform 32, and the sensor 40 is disposed on the motion platform 30, specifically: the sensor 40 is arranged on the Z-motion platform 32, wherein the XY-motion platform 31 is used for controlling the workpiece 200 to be measured to move along the horizontal plane, so as to change the relative positions of the workpiece 200 to be measured and the sensor 40 in the horizontal plane, and the Z-motion platform 32 is used for controlling the sensor 40 to move along the direction vertical to the horizontal plane, so that the three-dimensional position (i.e. the relative position in the horizontal plane and the relative position in the direction vertical to the horizontal plane) of the sensor 40 relative to the workpiece 200 to be measured is realized through the cooperation of the XY-motion platform 31 and the.
It is to be understood that the motion stage 30 is not limited to the above-described structure, and only needs to be able to change the three-dimensional position of the sensor 40 relative to the workpiece 200 to be measured.
The sensor 40 may be one or more and the plurality of sensors 40 may be different types of sensors 40, e.g., the sensors 40 may include visible light cameras, depth cameras, etc. In the present embodiment, the sensor 40 is a visible light camera.
In the case of an image to be measured, the workpiece 200 to be measured may be placed on the motion platform 30, and the processor 20 controls the motion platform 30 to move, so that the sensor 40 captures an original image of the workpiece as the image to be measured.
When the sensor 40 shoots each time, the field range only covers a part of the area of the workpiece 200 to be measured, and different areas of the workpiece 200 to be measured are shot by moving the workpiece 200 to be measured, so that a plurality of original images can be obtained, and the plurality of original images can be used as the images to be measured. It can be understood that the layout of the circuits of the wafer is generally regular, and the pattern area of the entire wafer can be composed of the minimum repeating units, and the sensor 40 captures one minimum repeating unit at a time to obtain one image to be measured. The defects of the wafer generally include foreign objects, residual glue, oxidation, bubbles, wrinkles, cracks, and the like, and for some defects such as foreign objects, wrinkles, bubbles, and the like, the difference from the minimum repeating unit of the wafer is large and is easy to detect, the corresponding image to be measured is a simple sample, and for defects that may not be different from the minimum repeating unit of the wafer, such as small cracks, oxidation, and the like, the corresponding image to be measured is a difficult sample.
When the workpiece 200 to be detected for shooting the original image is selected, the selected workpieces 200 to be detected can all be the same type of workpiece, so that the detection model obtained after subsequent adjustment is specially used for detecting the workpiece of the type, and the detection accuracy is improved. Of course, the selected workpiece 200 to be detected may also include different types of workpieces, so that the detection model obtained after adjustment can simultaneously realize defects of multiple types of workpieces, and the application is wide.
In order to improve the adjustment effect, when a wafer is selected, a wafer pattern or a plurality of wafers with different wafer background patterns can be selected, so that a plurality of images to be detected with different image backgrounds can be obtained, the diversity of the images to be detected can be improved, the adjustment effect can be improved, meanwhile, the influence of the image background on the adjusted detection model can be reduced, and the defect detection of the detection model can be accurately carried out even under different image backgrounds.
After the processor 20 acquires the image to be detected, the image to be detected is input to the detection model for detection, and the detection model outputs the type, the position and the confidence coefficient of the defect of the image to be detected. If the confidence is 90%, the accuracy of the defect type is up to 90%. The higher the confidence coefficient is, the more accurate the detection model detects the defects (such as bubbles) is (i.e. the better the training effect is), which indicates that the bubble defects are trained by a large number of images to be detected with bubble defects, and the simpler the images to be detected are; conversely, the lower the confidence, the less accurate the detection model is in detecting such defects (e.g., oxidation) (i.e., the less effective the training), indicating that the oxidation defect is trained only by a smaller number of images under test having oxidation defects, which is the more difficult the image under test is.
Therefore, according to the confidence level of the image to be detected, the difficulty level of the image to be detected can be determined, so that when the confidence level is lower, a larger adjustment coefficient is determined, and then the processor 20 calculates a loss value according to the adjustment coefficient and a preset loss function to adjust the detection model, so that the detection model converges. For example, when the confidence is 0.5, that is, the difficulty level of the sample is moderate, the preset loss function can accurately calculate the loss value at this time, and the loss value does not need to be increased any more, so as to improve the training effect of the difficult sample, so that when the confidence is 0.5, the adjustment coefficient can be determined to be 1, that is, the loss value is not adjusted, and when the confidence is less than 0.5, that the sample is difficult at this time, that is, the adjustment coefficient needs to be increased to increase the loss value, so as to improve the training effect of the difficult sample.
The detection model convergence refers to that the detection accuracy of the detection model reaches a preset accuracy (e.g., 90%, 95%, 98%, etc.), for example, a plurality of images to be detected are input to determine the ratio of the number of accurately detected images to all the images to be detected, so as to determine the preset accuracy.
Because the confidence coefficient is lower (the sample is harder), the adjusting coefficient is larger, so that the loss value is larger, the harder the sample is, the loss value is larger, the adjusting degree of the detection model is larger, the training effect of the difficult sample is improved, the detection precision of the detection model is improved, and missing detection and over-detection are prevented.
The detection model may be a second order detection algorithm (e.g., Faster R-CNN and its variants), a first order detection algorithm (e.g., Yolov3 and its variants), an anchor-free detection algorithm (e.g., CenterNet and its variants), etc., without limitation.
Finally, the processor 20 detects the image of the workpiece 200 after the sensor 40 captures the image of the workpiece 200 according to the converged detection model, so as to identify the defect in the image of the workpiece 200.
According to the adjusting method, the adjusting device 10 and the adjusting equipment 100, the image to be detected is input into the detection model to output the confidence of the image to be detected, so that the adjusting coefficient is determined according to the confidence, the lower the confidence is, the lower the detection accuracy of the image to be detected by the detection model is, and the harder the image to be detected is, therefore, the lower the confidence is (namely, the harder the image to be detected is), the larger the adjusting coefficient is, so that the influence on the detection model is larger when the detection model is adjusted based on the loss value calculated by the adjusting coefficient, the training effect of the detection model on a difficult sample is improved, and even if the number of the difficult samples is small, the accuracy of the detection model can be ensured, so that missing detection or over-detection is prevented.
Referring to fig. 2, 3 and 4, in some embodiments, step 012 includes:
0121: and determining an adjusting coefficient according to the confidence coefficient and a preset adjusting threshold value.
In some embodiments, the determining module 12 is further configured to determine an adjustment coefficient according to the confidence level and a preset adjustment threshold. That is, step 0121 may be performed by the determining module 12.
In some embodiments, the processor 20 is further configured to determine an adjustment factor based on the confidence level and a preset adjustment threshold. That is, step 0121 may be performed by processor 20.
Specifically, in order to prevent the adjustment coefficient from being too large, when determining the adjustment coefficient, the adjustment coefficient is first calculated according to the confidence coefficient and a preset adjustment function, and then when the adjustment coefficient is larger than a preset adjustment threshold, the adjustment threshold is used as the adjustment coefficient to perform subsequent calculation of the loss value. For example, the adjustment function may be f (p)t)＝tan(π/2-ptPi/2), wherein ptAs confidence, f (p)t) For adjusting the coefficient, the function curve of the adjustment function is shown in fig. 5, and it can be seen that the adjustment coefficient is 1 when the confidence is 0.5, the adjustment coefficient is less than 1 when the confidence is greater than 0.5, and the adjustment coefficient is greater than 1 when the confidence is less than 0.5. When the confidence level is too low, the adjustment coefficient calculated according to the adjustment function is close to infinity, which is obviously unreasonable, so that after the adjustment coefficient is calculated according to the adjustment function and the confidence level, the adjustment coefficient is compared with a preset adjustment threshold, and when the adjustment coefficient is greater than the preset adjustment threshold, the adjustment threshold is used as the adjustment coefficient to calculate the subsequent loss value. The preset adjusting threshold is an empirical value, the value is different in different detection fields, and the adjusting threshold is determined according to a detection model in the wafer detection field.
Referring to fig. 2, fig. 3 and fig. 6, in some embodiments, the adjusting method further includes:
015: obtaining a plurality of detection models corresponding to a plurality of different adjustment thresholds;
016: inputting a preset verification set to a plurality of detection models to output a plurality of recognition rates;
01212: and determining an adjustment coefficient according to the adjustment threshold with the highest confidence coefficient and recognition rate.
In some embodiments, the adjusting apparatus 10 further includes an obtaining module 15 and a second input-output module 16. The obtaining module 15 is configured to obtain a plurality of detection models corresponding to a plurality of different adjustment thresholds. The second input/output module 16 is configured to input a preset verification set to the plurality of detection models to output a plurality of recognition rates. The determining module 12 is further configured to determine an adjustment coefficient according to the adjustment threshold with the highest confidence and recognition rate. That is, step 015 may be performed by the obtaining module 15, step 016 may be performed by the second input and output module 16, and step 01212 may be performed by the determining module 12.
In some embodiments, processor 20 is further configured to obtain a plurality of detection models corresponding to a plurality of different adjustment thresholds; inputting a preset verification set to a plurality of detection models to output a plurality of recognition rates; and determining an adjustment coefficient according to the adjustment threshold with the highest confidence coefficient and recognition rate. That is, step 015, step 016 and step 01212 may be performed by the processor 20.
Specifically, since the preset adjustment threshold is an empirical value, generally, only the range of one adjustment threshold can be determined, then a plurality of adjustment thresholds are taken within the range, then the adjustment coefficient can be calculated according to each adjustment threshold, then the detection model is trained based on the loss value determined by the adjustment coefficient, when the adjustment threshold is determined, the detection model does not need to be trained until convergence, and only one round of training is performed according to various images to be measured, so that the calculation amount for determining the adjustment threshold is reduced.
Each adjustment threshold corresponds to one detection model, then a preset verification set is input into the detection models corresponding to the multiple adjustment thresholds, so that the recognition rate of each detection model can be obtained, the adjustment threshold corresponding to the detection model with the highest recognition rate is the adjustment threshold with the better training effect, when the adjustment coefficient is calculated, the adjustment coefficient is determined according to the adjustment threshold with the highest recognition rate and the confidence coefficient, and the detection effect of the detection model can be improved. The preset verification set comprises a plurality of images of which the types and the positions of the defects are accurately detected.
For example, the range of the adjustment threshold is 5 to 25, 5 adjustment thresholds are 5, 10, 15, 20, and 25, then training is performed according to the 5 adjustment thresholds, for example, training is performed according to different adjustment thresholds, the same number (for example, 100) of images to be detected are input under each adjustment threshold, and training is performed on the detection models, so as to obtain 5 trained detection models, then a preset verification set is input to determine the recognition rate of each detection model, thereby determining the detection model with the highest recognition rate, and further determining the final adjustment threshold. And then adjusting the adjustment coefficient according to the final adjustment threshold value to realize the training of the detection model until the detection model converges, wherein the training of the detection model is to converge, and generally requires multiple rounds of training, for example, if 100 images to be tested are in one round, 3 rounds, 4 rounds, 5 rounds or even more rounds of training may be required to converge, and the number of the required images to be tested is more.
Referring to fig. 2, 3 and 7, in some embodiments, step 014 includes:
0141: inputting a preset verification set to the detection model to output the recognition rate;
0142: when the recognition rate is greater than a preset threshold value, determining that the detection model converges; and
0143: and when the recognition rate is smaller than the preset threshold value, inputting the image to be detected which is different from the image to be detected which is adjusted before again to adjust the detection model until the detection model converges.
In some embodiments, the adjusting module 14 is further configured to input a preset validation set to the detection model to output the recognition rate; when the recognition rate is greater than a preset threshold value, determining that the detection model converges; and when the recognition rate is smaller than the preset threshold value, inputting the image to be detected which is different from the image to be detected which is adjusted before again to adjust the detection model until the detection model converges. That is, step 0141, step 0142 and step 0143 may be performed by the adjustment module 14.
In some embodiments, the processor 20 is further configured to input a preset validation set to the detection model to output the recognition rate; when the recognition rate is greater than a preset threshold value, determining that the detection model converges; and when the recognition rate is smaller than the preset threshold value, inputting the image to be detected which is different from the image to be detected which is adjusted before again to adjust the detection model until the detection model converges. That is, step 0141, step 0142 and step 0143 may be performed by processor 20.
Specifically, after a round of training (for example, one round of adjustment is performed after 100 images to be detected), a preset verification set is input to the detection model, then the detection model outputs a detection result of each image in the verification set, and then the detection result is compared with defect information of the corresponding image, so that the number of the images which are not accurately detected in the verification set can be determined, and the identification rate can be determined according to the ratio of the number to the total number of the images in the verification set.
When the recognition rate reaches (i.e., is greater than or equal to) a preset threshold (e.g., 90%, 95%, etc.), it can be determined that the detection model has converged, and training is not required to be continued. And when the recognition rate does not reach (i.e. is less than) the preset threshold, determining that the detection model does not converge, at this time, performing the next round of training, inputting an image to be detected different from the image to be detected in the previous round of training, training the detection model again, inputting the verification set again after the training is finished to obtain the recognition rate, and thus judging whether the detection model converges, and repeating the steps until the detection model converges.
Referring to fig. 2, fig. 3 and fig. 8, in some embodiments, before inputting the image to be detected into the detection model, the adjusting method further includes:
017: and carrying out amplification processing on the plurality of images to be detected, wherein the amplification processing comprises at least one of mirroring, translation, rotation, shearing and deformation.
In certain embodiments, the conditioning apparatus 10 further comprises an amplification module 17. The amplification module 17 is configured to perform amplification processing on the multiple images to be detected, where the amplification processing includes at least one of mirroring, translation, rotation, shearing, and deformation. That is, step 017 may be performed by the amplification module 17.
In some embodiments, processor 20 is further configured to perform an amplification process on the plurality of images under test, the amplification process including at least one of mirroring, translation, rotation, shearing, and deformation. That is, step 017 may be performed by the processor 20.
Specifically, when the number of samples of the image to be measured is small, the processor 20 may perform the amplification process on the image to be measured in order to further increase the number and diversity of the image to be measured.
Referring to fig. 9, for example, the processor 20 performs a mirror image process on each image P1 to obtain a mirror image P2 of each image P1, and uses the mirror image as a new image P1. The mirror image P2 after the mirror image processing is mirror-symmetric to the image to be measured P1, and the symmetry axis may be any, for example, the mirror image processing is performed with any side of the image to be measured P1 as the symmetry axis (in fig. 9, the mirror image processing is performed with the rightmost side of the image to be measured P1 as the symmetry axis), or the mirror image processing is performed with the diagonal line of the image to be measured P1 or the connecting line of the midpoints of any two sides as the symmetry axis, so as to obtain a plurality of new images to be measured through the mirror image processing.
Referring to fig. 10, for another example, the processor 20 performs a translation process on each image P1 to obtain a translated image P3 of each image P1, and uses the translated image as a new image P1. Specifically, a predetermined image area (i.e., an area occupied by the image to be measured P1) is determined by using the image to be measured P1, then the image to be measured P1 is translated, such as left translation, right translation, left-up translation, and the like (in fig. 10, right translation), then the image of the predetermined image area (i.e., the translated image P3) is used as a new image to be measured P1, and the position of the defect after translation in the image is changed, so that a plurality of new images to be measured P1 are obtained.
Referring to fig. 11, for another example, the processor 20 performs a rotation process on each image P1 to obtain a rotated image P4 of each image P1, and uses the rotated image as a new image P1. Specifically, a predetermined image area is determined by using the image P1 to be measured, then the image P1 to be measured is rotated, for example, clockwise or counterclockwise by 10 degrees, 30 degrees, 60 degrees, 90 degrees, 140 degrees and the like (fig. 11 is rotated counterclockwise by 30 degrees), then the image of the predetermined image area (and the rotated image P4) is used as a new image P1 to be measured, and the positions of the rotated defects in the image are changed, so that a plurality of new images P1 to be measured are obtained.
Referring to fig. 12, for another example, the processor 20 performs a cropping process on each image P1 to obtain a cropped image P5 of each image to be tested, and uses the cropped image as a new image P1. Specifically, a predetermined image area is determined by using an image P1 to be measured, then the image P1 to be measured is cut, for example, 1/4, 1/3, 1/2 of the image P1 to be measured (fig. 12 is 1/2 for cutting the image to be measured), and then an image of the predetermined image area (i.e., the cut image P5) is used as a new image P1 to be measured, so as to obtain a plurality of new images P1 to be measured.
Referring to fig. 13, for example, the processor 20 performs a warping process on each of the images P1 to obtain a warped image P6 of each of the images P1, and uses the warped image as a new image P1. Specifically, a predetermined image area is determined by using an image to be measured P1, then the image to be measured P1 is deformed, for example, the image to be measured is compressed along the transverse direction, so that the original rectangular image to be measured P1 becomes a rectangle with a notch, then the image of the predetermined image area (i.e., the deformed image P6) is used as a new image to be measured P1, and the positions and shapes of the deformed defects in the image are changed, so that a plurality of new images to be measured P1 are obtained.
Of course, the processor 20 may also perform the translation processing and the rotation processing on the image to be detected at the same time; or simultaneously carrying out translation processing, rotation processing and mirror image processing; or simultaneously carrying out translation processing, rotation processing, mirror image processing and shearing processing; alternatively, the translation processing, the rotation processing, and the mirror processing are performed simultaneously, and the translation processing, the rotation processing, and the mirror processing are performed a plurality of times respectively at different distances, different angles, and different symmetry axes, which are not listed here.
By carrying out amplification processing on the images to be detected, a large number of images to be detected can be obtained without obtaining more images to be detected, the diversity of the images to be detected is better, and the adjustment effect on the detection model can be improved.
Referring to fig. 14, one or more non-transitory computer-readable storage media 300 containing a computer program 302 according to an embodiment of the present disclosure, when the computer program 302 is executed by one or more processors 20, enable the processor 20 to perform the calibration method according to any of the embodiments described above.
For example, referring to fig. 1-3, the computer program 302, when executed by the one or more processors 20, causes the processors 20 to perform the steps of:
011: inputting an image to be detected to a detection model so as to output the confidence coefficient of the image to be detected;
012: determining an adjusting coefficient according to the confidence coefficient, wherein the adjusting coefficient is negatively correlated with the confidence coefficient;
013: calculating a loss value according to the adjustment coefficient and a preset loss function; and
014: the detection model is adjusted according to the loss value so that the detection model converges.
As another example, referring to fig. 2, 3 and 4 in conjunction, when the computer program 302 is executed by the one or more processors 20, the processors 20 may further perform the steps of:
0121: and determining an adjusting coefficient according to the confidence coefficient and a preset adjusting threshold value.
In the description herein, references to the description of the terms "one embodiment," "some embodiments," "an illustrative embodiment," "an example," "a specific example" or "some examples" or the like mean that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the present application. In this specification, schematic representations of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. Furthermore, the various embodiments or examples and features of the various embodiments or examples described in this specification can be combined and combined by those skilled in the art without contradiction.
Any process or method descriptions in flow charts or otherwise described herein may be understood as representing modules, segments, or portions of code which include one or more executable instructions for implementing specific logical functions or steps of the process, and the scope of the preferred embodiments of the present application includes other implementations in which functions may be executed out of order from that shown or discussed, including substantially concurrently or in reverse order, depending on the functionality involved, as would be understood by those reasonably skilled in the art of the embodiments of the present application.
Although embodiments of the present application have been shown and described above, it is to be understood that the above embodiments are exemplary and not to be construed as limiting the present application, and that changes, modifications, substitutions and alterations can be made to the above embodiments by those of ordinary skill in the art within the scope of the present application.
Claims (10)
1. An adjustment method, comprising:
inputting an image to be detected to a detection model so as to output the confidence coefficient of the image to be detected;
determining an adjusting coefficient according to the confidence coefficient, wherein the adjusting coefficient is in negative correlation with the confidence coefficient;
calculating a loss value according to the adjusting coefficient and a preset loss function; and
adjusting the detection model according to the loss value so that the detection model converges.
2. The adjustment method according to claim 1, wherein the adjustment coefficient is 1 when the confidence is 0.5.
3. The adjustment method according to claim 1, wherein said determining an adjustment coefficient according to said confidence level comprises:
and determining the adjustment coefficient according to the confidence coefficient and a preset adjustment threshold value.
4. The adjustment method according to claim 3, wherein the determining the adjustment coefficient according to the confidence and a preset adjustment threshold comprises:
when the adjustment coefficient is larger than the adjustment threshold, determining the adjustment threshold as the adjustment coefficient.
5. The adjustment method according to claim 3, further comprising:
obtaining a plurality of detection models corresponding to a plurality of different adjustment thresholds;
inputting a preset verification set to a plurality of detection models to output a plurality of recognition rates;
the determining the adjustment coefficient according to the confidence and a preset adjustment threshold includes:
and determining the adjusting coefficient according to the confidence coefficient and the adjusting threshold with the highest recognition rate.
6. The adjusting method according to claim 1, wherein the adjusting the detection model according to the loss value so that the detection model converges comprises:
inputting a preset verification set to the detection model to output a recognition rate;
when the recognition rate is greater than a preset threshold value, determining that the detection model converges; and
and when the recognition rate is smaller than the preset threshold value, inputting the image to be detected which is different from the image to be detected which is adjusted before to adjust the detection model again until the detection model converges.
7. The adjusting method according to claim 1, further comprising, before inputting the image to be detected into the detection model:
and carrying out amplification processing on the images to be detected, wherein the amplification processing comprises at least one of mirroring, translation, rotation, shearing and deformation.
8. An adjustment device, comprising:
the first input and output module is used for inputting an image to be detected to the detection model so as to output the confidence coefficient of the image to be detected;
the determining module is used for determining an adjusting coefficient according to the confidence coefficient, and the adjusting coefficient is inversely related to the confidence coefficient;
the calculation module is used for calculating a loss value according to the adjustment coefficient and a preset loss function; and
and the adjusting module is used for adjusting the detection model according to the loss value so as to make the detection model converge.
9. The adjusting equipment is characterized by comprising a processor, a detection module and a display module, wherein the processor is used for inputting an image to be detected to the detection module so as to output the confidence coefficient of the image to be detected; determining an adjusting coefficient according to the confidence coefficient, wherein the adjusting coefficient is in negative correlation with the confidence coefficient; calculating a loss value according to the adjusting coefficient and a preset loss function; and adjusting the detection model according to the loss value so that the detection model converges.
10. A non-transitory computer-readable storage medium containing a computer program which, when executed by one or more processors, causes the processors to perform the adjustment method of any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110351892.XA CN113066069A (en) | 2021-03-31 | 2021-03-31 | Adjusting method and device, adjusting equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110351892.XA CN113066069A (en) | 2021-03-31 | 2021-03-31 | Adjusting method and device, adjusting equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113066069A true CN113066069A (en) | 2021-07-02 |
Family
ID=76564997
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110351892.XA Pending CN113066069A (en) | 2021-03-31 | 2021-03-31 | Adjusting method and device, adjusting equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113066069A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113556600A (en) * | 2021-07-13 | 2021-10-26 | 广州虎牙科技有限公司 | Drive control method and device based on time sequence information, electronic equipment and readable storage medium |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20180174046A1 (en) * | 2016-12-15 | 2018-06-21 | Beijing Kuangshi Technology Co., Ltd. | Target detection method and device, neural network training method and device |
| CN109902722A (en) * | 2019-01-28 | 2019-06-18 | 北京奇艺世纪科技有限公司 | Classifier, neural network model training method, data processing equipment and medium |
| CN111523621A (en) * | 2020-07-03 | 2020-08-11 | 腾讯科技（深圳）有限公司 | Image recognition method and device, computer equipment and storage medium |
| CN112163637A (en) * | 2020-10-19 | 2021-01-01 | 平安国际智慧城市科技股份有限公司 | Image classification model training method and device based on unbalanced data |
| CN112488104A (en) * | 2020-11-30 | 2021-03-12 | 华为技术有限公司 | Depth and confidence estimation system |
-
2021
- 2021-03-31 CN CN202110351892.XA patent/CN113066069A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20180174046A1 (en) * | 2016-12-15 | 2018-06-21 | Beijing Kuangshi Technology Co., Ltd. | Target detection method and device, neural network training method and device |
| CN109902722A (en) * | 2019-01-28 | 2019-06-18 | 北京奇艺世纪科技有限公司 | Classifier, neural network model training method, data processing equipment and medium |
| CN111523621A (en) * | 2020-07-03 | 2020-08-11 | 腾讯科技（深圳）有限公司 | Image recognition method and device, computer equipment and storage medium |
| CN112163637A (en) * | 2020-10-19 | 2021-01-01 | 平安国际智慧城市科技股份有限公司 | Image classification model training method and device based on unbalanced data |
| CN112488104A (en) * | 2020-11-30 | 2021-03-12 | 华为技术有限公司 | Depth and confidence estimation system |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113556600A (en) * | 2021-07-13 | 2021-10-26 | 广州虎牙科技有限公司 | Drive control method and device based on time sequence information, electronic equipment and readable storage medium |
| CN113556600B (en) * | 2021-07-13 | 2023-08-18 | 广州虎牙科技有限公司 | Drive control method and device based on time sequence information, electronic equipment and readable storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110889823B (en) | SiC defect detection method and system | |
| CN112161619B (en) | Pose detection method, three-dimensional scanning path planning method and detection system | |
| CN112884743B (en) | Detection method and device, detection equipment and storage medium | |
| JP2017515097A (en) | Automatic in-line inspection and measurement using projected images | |
| CN110672007A (en) | Workpiece surface quality detection method and system based on machine vision | |
| WO2019216362A1 (en) | Inspection device and inspection method | |
| CN116433780B (en) | Automatic calibration method for laser structured light based on machine vision | |
| US6519358B1 (en) | Parallax calculating apparatus, distance calculating apparatus, methods of the same, and information providing media | |
| CN113983929B (en) | Method for measuring and evaluating position degree and profile degree of runner blade profile holes of runner blade profile plate | |
| CN113066069A (en) | Adjusting method and device, adjusting equipment and storage medium | |
| US20170125271A1 (en) | Position detection apparatus, position detection method, information processing program, and storage medium | |
| CN116543247A (en) | Data set manufacturing method and verification system based on photometric stereo surface reconstruction | |
| CN112595262B (en) | Binocular structured light-based high-light-reflection surface workpiece depth image acquisition method | |
| CN116468961A (en) | Image classification method and intelligent flaw detection system for injection molding product | |
| CN116051498A (en) | Wafer surface defect detection method and detection model | |
| US8547439B2 (en) | Testing an optical characteristic of a camera component | |
| CN112884744A (en) | Detection method and device, detection equipment and storage medium | |
| JP2005283267A (en) | Through hole measuring device, method, and program for through hole measurement | |
| JP2004095657A (en) | Semiconductor inspecting apparatus | |
| CN115103124A (en) | Active alignment method for camera module | |
| CN112926437B (en) | Detection method and device, detection equipment and storage medium | |
| CN112926438B (en) | Detection method and device, detection equipment and storage medium | |
| CN113192021A (en) | Detection method and device, detection equipment and storage medium | |
| JP3864491B2 (en) | Disparity calculating device, distance calculating device, and methods thereof | |
| CN112884691A (en) | Data enhancement and device, data enhancement equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |