CN113658036B - Data augmentation method, device, computer and medium based on countermeasure generation network - Google Patents
Data augmentation method, device, computer and medium based on countermeasure generation network Download PDFInfo
- Publication number
- CN113658036B CN113658036B CN202110970515.4A CN202110970515A CN113658036B CN 113658036 B CN113658036 B CN 113658036B CN 202110970515 A CN202110970515 A CN 202110970515A CN 113658036 B CN113658036 B CN 113658036B
- Authority
- CN
- China
- Prior art keywords
- picture
- pseudo
- network
- target domain
- domain
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000013434 data augmentation Methods 0.000 title claims abstract description 63
- 238000000034 method Methods 0.000 title claims abstract description 50
- 208000035753 Periorbital contusion Diseases 0.000 claims description 53
- 235000002673 Dioscorea communis Nutrition 0.000 claims description 47
- 241000544230 Dioscorea communis Species 0.000 claims description 47
- 230000006870 function Effects 0.000 claims description 27
- 238000004590 computer program Methods 0.000 claims description 11
- 238000004364 calculation method Methods 0.000 claims description 5
- 238000012549 training Methods 0.000 claims description 5
- 238000012546 transfer Methods 0.000 claims description 4
- 230000008485 antagonism Effects 0.000 claims description 2
- 230000003416 augmentation Effects 0.000 claims description 2
- 230000003190 augmentative effect Effects 0.000 description 4
- 238000002595 magnetic resonance imaging Methods 0.000 description 3
- 230000003321 amplification Effects 0.000 description 2
- 238000002591 computed tomography Methods 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 238000003199 nucleic acid amplification method Methods 0.000 description 2
- 230000001360 synchronised effect Effects 0.000 description 2
- 210000001015 abdomen Anatomy 0.000 description 1
- 210000004556 brain Anatomy 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 230000009977 dual effect Effects 0.000 description 1
- 210000002216 heart Anatomy 0.000 description 1
- 238000003709 image segmentation Methods 0.000 description 1
- 210000003127 knee Anatomy 0.000 description 1
- 230000003902 lesion Effects 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000004088 simulation Methods 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 210000002784 stomach Anatomy 0.000 description 1
- 238000002604 ultrasonography Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/04—Context-preserving transformations, e.g. by using an importance map
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Biophysics (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Biomedical Technology (AREA)
- Health & Medical Sciences (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Life Sciences & Earth Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The application relates to the field of data augmentation of medical images, and discloses a method, a device, computer equipment and a storage medium for data augmentation based on an countermeasure generation network, wherein the method comprises the following steps: acquiring a first picture in a target domain and a second picture in a source domain; inputting the first picture into a first generation network to obtain a pseudo second picture; inputting the pseudo second picture into a second generation network to obtain a pseudo first picture; calculating a consistency constraint value of the first picture and the pseudo first picture; inputting the second picture and the pseudo second picture into a discrimination network to obtain a classification result of the discrimination network on the second picture and the pseudo second picture; calculating a loss value of the classification result according to a loss function; and if the sum of the consistency constraint value and the loss value meets a preset value, adding the pseudo first picture into a target domain to increase the picture data in the target domain. The application can improve the generation efficiency of the data samples.
Description
Technical Field
The present application relates to the field of data augmentation of medical images, and in particular, to a method, apparatus, computer device, and storage medium for data augmentation based on an countermeasure generation network.
Background
With the development of internet technology, a large amount of sample data is required during machine learning and training, but in a special scene, certain types of sample data cannot be easily obtained, especially, the probability of occurrence of medical image data in the medical field in reality is low, so that the specific types of medical image data cannot be easily obtained, for example, the actual sample data of the sample data of heavy dark circles is less, and the sample data acquisition efficiency is low and the cost is high.
Disclosure of Invention
The application mainly aims to provide a method, a device, computer equipment and a storage medium for data augmentation based on an countermeasure generation network, and aims to solve the problem of low accuracy of data augmentation under the scene of small sample data quantity at present.
In order to achieve the above object, the present application proposes a method for data augmentation based on an countermeasure generation network, comprising:
Acquiring a first picture in a target domain and a second picture in a source domain;
Inputting the first picture into a first generation network to obtain a pseudo second picture; inputting the pseudo second picture into a second generation network to obtain a pseudo first picture; the first generation network and the second generation network are mutually opposite networks;
calculating a consistency constraint value of the first picture and the pseudo first picture;
Inputting the second picture and the pseudo second picture into a discrimination network to obtain a classification result of the discrimination network on the second picture and the pseudo second picture;
Calculating a loss value of the classification result according to a loss function;
And if the sum of the consistency constraint value and the loss value meets a preset value, adding the pseudo first picture into a target domain to increase the picture data in the target domain.
Further, after the first picture in the target domain and the second picture in the source domain are acquired, the method further includes:
Inputting the second picture into a second generation network to obtain a fake first picture; inputting the fake first picture into a first generation network to obtain a fake second picture;
calculating candidate consistency constraint values of the second picture and the fake second picture;
Inputting the second picture and the fake second picture into a second discrimination network to obtain candidate classification results of the second discrimination network on the second picture and the fake second picture;
calculating a candidate loss value of the candidate classification result according to a loss function;
And if the sum of the candidate consistency constraint value and the candidate loss value meets a preset value, adding the false second picture into the source domain so as to increase the picture data in the source domain.
Further, the source domain and the target domain are mutually mapped data sets.
Further, if the sum of the consistency constraint value and the loss value meets a preset value, before adding the pseudo first picture to the target domain, the method includes:
Acquiring an execution stage of current data augmentation;
and matching a preset value of the sum of the consistency constraint value and the loss value according to the execution stage.
Further, before the obtaining the first picture in the target domain and the second picture in the source domain, the method further includes:
selecting one of a plurality of to-be-selected domains as a target domain, and acquiring a picture grade of the target domain;
matching candidate picture grades adjacent to the picture grade of the target domain according to a preset grade rule;
And determining a source domain corresponding to the target domain according to the candidate picture level.
Further, the selecting one from the plurality of candidate domains as the target domain includes:
acquiring the number of pictures of each domain to be selected;
sorting the domains to be selected according to the number of pictures;
and selecting at least one corresponding domain to be selected from the sorted domains to be selected according to the number of pictures.
Further, the target field comprises a picture of a heavy black eye picture level and a light black eye picture level, and the source field comprises a picture of a light black eye picture level and a no black eye picture level; the matching the candidate picture level adjacent to the picture level of the target domain according to the preset level rule includes:
if the candidate picture level is the heavy black eye picture level, determining that the picture level of the source domain corresponding to the target domain is the light black eye picture level;
And if the candidate picture grade is the light black eye picture grade, determining that the picture grade of the source domain corresponding to the target domain is the black eye-free picture grade.
The application also provides a device for data augmentation based on the countermeasure generation network, comprising:
the data acquisition module is used for acquiring a first picture in a target domain and a second picture in a source domain;
the generation network module is used for inputting the first picture into a first generation network to obtain a pseudo second picture; inputting the pseudo second picture into a second generation network to obtain a pseudo first picture; the first generation network and the second generation network are mutually opposite networks;
the constraint calculating module is used for calculating a consistency constraint value of the first picture and the pseudo first picture;
The result classification module is used for inputting the second picture and the pseudo second picture into a discrimination network to obtain classification results of the discrimination network on the second picture and the pseudo second picture;
The loss calculation module is used for calculating a loss value of the classification result according to a loss function;
and the data adding module is used for adding the pseudo first picture into the target domain to increase the picture data in the target domain if the sum of the consistency constraint value and the loss value meets the preset value.
The application also provides a computer device comprising a memory storing a computer program and a processor implementing the steps of the method of data augmentation based on an countermeasure generation network of any of the above, when the computer program is executed.
The application also provides a computer readable storage medium having stored thereon a computer program which when executed by a processor performs the steps of the method of data augmentation based on an countermeasure generation network of any of the above.
The embodiment of the application provides a data augmentation method based on an countermeasure generation network, which comprises the steps of firstly acquiring a first picture in a target domain and a second picture in a source domain, wherein the target domain and the source domain are a group of data pairs, and sample data are required to be extracted from the target domain and the source domain at the same time, namely, one first picture is selected from the target domain and one second picture is selected from the source domain as the data pairs, and the first picture is input into the first generation network to obtain a pseudo second picture; inputting the pseudo second picture into a second generation network to obtain a pseudo first picture; the first generation network and the second generation network are opposite networks, a consistency constraint value of the first picture and the pseudo first picture is calculated, the consistency constraint value is the image matching degree of the first picture and the pseudo first picture, the second picture and the pseudo second picture are input into a discrimination network, a classification result of the discrimination network on the second picture and the pseudo second picture is obtained, then a loss value of the classification result is calculated according to a loss function, the loss value represents a difference value between the classification result and a real result of the second picture and the pseudo second picture, if the sum of the consistency constraint value and the loss value meets a preset value, the first picture is represented to obtain the pseudo first picture meeting the preset requirement after passing through the first generation network and the second generation network, and then the pseudo first picture is added into a target domain, so that the picture data in the target domain is increased, and the widening efficiency and the accuracy of the picture in the target domain are improved.
Drawings
FIG. 1 is a flow chart of a method for data augmentation based on an countermeasure generation network according to an embodiment of the present application;
FIG. 2 is a schematic diagram illustrating an exemplary architecture of an apparatus for data augmentation based on an countermeasure generation network according to the present application;
FIG. 3 is a block diagram schematically illustrating the structure of a computer device according to an embodiment of the present application.
The achievement of the objects, functional features and advantages of the present application will be further described with reference to the accompanying drawings, in conjunction with the embodiments.
Detailed Description
The present application will be described in further detail with reference to the drawings and examples, in order to make the objects, technical solutions and advantages of the present application more apparent. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the application.
Referring to fig. 1, an embodiment of the present application provides a method for data augmentation based on an countermeasure generation network, including steps S10-S60, and the steps of the method for data augmentation based on an countermeasure generation network are explained in detail as follows.
S10, acquiring a first picture in a target domain and a second picture in a source domain.
The embodiment is applied to a data augmentation scene of medical images, wherein in the medical application scene, a sample image is a medical image, and the type of an object contained in the sample image is a focus, namely a part on an organism, where a lesion occurs. Medical images refer to images of internal tissues taken in a non-invasive manner for medical or medical research, e.g., stomach, abdomen, heart, knee, brain, such as CT (Computed Tomography ), MRI (Magnetic Resonance Imaging, magnetic resonance imaging), US (ultra sonic, ultrasound), X-ray images, electroencephalograms, and images generated by medical instruments by optical photography lamps. The method for realizing the data augmentation based on the countermeasure generation network by developing a data augmentation model based on the countermeasure generation network comprises the steps of obtaining a first picture in a target domain and a second picture in a source domain, wherein the target domain and the source domain are a group of data pairs, the picture in the target domain is defined as the first picture, the picture in the source domain is defined as the second picture, then selecting one first picture from the target domain and one second picture from the source domain as the data pairs respectively, and inputting the data augmentation model based on the countermeasure generation network.
S20, inputting the first picture into a first generation network to obtain a pseudo second picture; inputting the pseudo second picture into a second generation network to obtain a pseudo first picture; the first generation network and the second generation network are mutually opposite networks.
In this embodiment, the countermeasure generation network includes a first generation network and a second generation network, after a first picture in a target domain and a second picture in a source domain are acquired, the first picture is input into the first generation network to obtain a pseudo second picture, and the first generation network can select elements existing in the first picture as random noise to be added into the first picture, so as to generate pseudo data corresponding to the first picture, and define the pseudo second picture; and inputting the pseudo second picture into a second generation network, wherein the second generation network and the first generation network are mutually opposite networks, the second generation network can select elements existing in the first picture as random noise to be added into the pseudo second picture, so that pseudo data corresponding to the pseudo second picture is generated, and the picture generated by the second generation network is defined as the pseudo first picture because the first generation network and the second generation network are mutually opposite networks.
S30, calculating a consistency constraint value of the first picture and the pseudo first picture.
In this embodiment, after obtaining a pseudo first picture, a consistency constraint value of the first picture and the pseudo first picture is calculated, where the consistency constraint value is an image matching degree of the first picture and the pseudo first picture, the first picture and the pseudo first picture are respectively segmented into a plurality of corresponding area images through image segmentation, then the matching degree of the area images is calculated, and then the matching degree of all the area images is counted to obtain an image matching degree of the first picture and the pseudo first picture, so as to obtain the consistency constraint value of the first picture and the pseudo first picture.
S40, inputting the second picture and the pseudo second picture into a discrimination network to obtain a classification result of the discrimination network on the second picture and the pseudo second picture.
In this embodiment, after obtaining a pseudo second picture, the second picture and the pseudo second picture are input to a discrimination network, the discrimination network determines that the second picture and the pseudo second picture are real data in a source domain or false data generated by a first generation network, so as to obtain a classification result of the discrimination network on the second picture and the pseudo second picture, in one embodiment, the second picture and the pseudo second picture are input to the discrimination network, so as to obtain a classification result a1 of the discrimination network on the second picture and a classification result a2 of the discrimination network on the pseudo second picture, and then a classification result a of the discrimination network on the second picture and the pseudo second picture is generated based on the classification result a1 and the classification result a2, wherein the classification result a includes the classification result a1 and the classification result a2, such as the classification result a (a 1, a 2).
S50, calculating the loss value of the classification result according to the loss function.
In this embodiment, after the classification results of the second picture and the pseudo second picture by the discrimination network are obtained, a loss value of the classification result is calculated according to a loss function, wherein the loss function is composed of a true value and a predicted value, that is, the true value and the predicted value corresponding to the classification results of the second picture and the pseudo second picture are calculated, then the true value and the predicted value are added to obtain the loss value of the classification result, in one embodiment, the classification result a1 of the second picture is calculated according to the discrimination network by the loss function to obtain the loss value S1, and the classification result a2 of the pseudo second picture is calculated according to the discrimination network to obtain the loss value S2, and then the loss value S of the classification result a is generated according to the loss value S1 and the loss value S2, wherein the loss value represents the difference value between the classification results of the second picture and the pseudo second picture and the true result.
And S60, if the sum of the consistency constraint value and the loss value meets a preset value, adding the pseudo first picture into a target domain so as to increase the picture data in the target domain.
In this embodiment, after calculating the loss value of the loss function of the classification result, if the sum of the consistency constraint value and the loss value meets a preset value, characterizing that the first picture passes through a first generation network and a second generation network, then a pseudo first picture meeting a preset requirement can be obtained, and characterizing that the first picture passes through the first generation network, then a pseudo second picture meeting the preset requirement can be obtained, that is, the countermeasure generation network model can operate correctly, then the pseudo first picture is added into a target domain, so that picture data in the target domain is increased, and the amplification efficiency and accuracy of the picture data in the target domain are improved.
The embodiment provides a method for data augmentation based on an countermeasure generation network, which comprises the steps of firstly acquiring a first picture in a target domain and a second picture in a source domain, wherein the target domain and the source domain are a group of data pairs, and sample data are required to be extracted from the target domain and the source domain at the same time, namely, one first picture is selected from the target domain and one second picture is selected from the source domain as the data pairs, and the first picture is input into the first generation network to obtain a pseudo second picture; inputting the pseudo second picture into a second generation network to obtain a pseudo first picture; the first generation network and the second generation network are opposite networks, a consistency constraint value of the first picture and the pseudo first picture is calculated, the consistency constraint value is the image matching degree of the first picture and the pseudo first picture, the second picture and the pseudo second picture are input into a discrimination network, a classification result of the discrimination network on the second picture and the pseudo second picture is obtained, then a loss value of the classification result is calculated according to a loss function, the loss value represents a difference value between the classification result and a real result of the second picture and the pseudo second picture, if the sum of the consistency constraint value and the loss value meets a preset value, the first picture is represented to obtain the pseudo first picture meeting the preset requirement after passing through the first generation network and the second generation network, and then the pseudo first picture is added into a target domain, so that the picture data in the target domain is increased, and the widening efficiency and the accuracy of the picture in the target domain are improved.
In one embodiment, after the obtaining the first picture in the target domain and the second picture in the source domain, the method further includes:
Inputting the second picture into a second generation network to obtain a fake first picture; inputting the fake first picture into a first generation network to obtain a fake second picture;
calculating candidate consistency constraint values of the second picture and the fake second picture;
Inputting the second picture and the fake second picture into a second discrimination network to obtain candidate classification results of the second discrimination network on the second picture and the fake second picture;
calculating a candidate loss value of the candidate classification result according to a loss function;
And if the sum of the candidate consistency constraint value and the candidate loss value meets a preset value, adding the false second picture into the source domain so as to increase the picture data in the source domain.
In this embodiment, not only the image data augmentation may be performed on the target domain, but also the image data augmentation may be performed on the source domain, and specifically, the second picture is input to the second generation network, so as to obtain a false first picture; inputting the fake first picture into a first generation network to obtain a fake second picture, wherein the first generation network and the second generation network are mutually opposite networks; then calculating candidate consistency constraint values of the second picture and the fake second picture; inputting the second picture and the fake second picture into a second discrimination network to obtain candidate classification results of the second discrimination network on the second picture and the fake second picture; calculating a candidate loss value of the candidate classification result according to a loss function; and if the sum of the candidate consistency constraint value and the candidate loss value meets a preset value, adding the false second picture into the source domain to increase the picture data in the source domain, thereby improving the data augmentation efficiency of the target domain and the source domain.
In one embodiment, the source domain and the target domain are mutually mapped data sets.
In this embodiment, the source domain and the target domain are mutually mapped data sets, and any image existing in the source domain has mutually mapped images in the target domain, where the mutually mapped images are images with different designated ranges and the same other ranges.
In one embodiment, if the sum of the consistency constraint value and the loss value meets a preset value, before adding the pseudo first picture to the target domain, the method includes:
Acquiring an execution stage of current data augmentation;
and matching a preset value of the sum of the consistency constraint value and the loss value according to the execution stage.
In this embodiment, before determining whether the sum of the consistency constraint value and the loss value meets a preset value, and adding the pseudo first picture to the target domain, the current execution stage of data augmentation is obtained, and then the preset value of the sum of the consistency constraint value and the loss value is matched according to the execution stage, for example, in the initial stage of data augmentation, the preset value of the sum of the consistency constraint value and the loss value with smaller precision is set, so that data is rapidly augmented, in the middle stage of data augmentation, the preset value of the sum of the consistency constraint value and the loss value with higher precision is set, and the accuracy of the augmented data is improved.
In one embodiment, before the obtaining the first picture in the target domain and the second picture in the source domain, the method further includes:
selecting one of a plurality of to-be-selected domains as a target domain, and acquiring a picture grade of the target domain;
matching candidate picture grades adjacent to the picture grade of the target domain according to a preset grade rule;
And determining a source domain corresponding to the target domain according to the candidate picture level.
In this embodiment, before acquiring a first picture in a target domain and a second picture in a source domain, when data augmentation is required for the pictures in different domains, firstly selecting one of a plurality of to-be-selected domains as the target domain, then acquiring the picture levels of the target domain, wherein the picture levels in each domain are the same level, the picture levels of the two pictures as data pairs need to be adjacent picture levels, namely, matching candidate picture levels adjacent to the picture levels of the target domain according to a preset level rule, determining the source domain corresponding to the target domain according to the candidate picture levels, thereby determining two domains of the adjacent picture levels, one of the two domains being the target domain, one of the two domains being the source domain, and then selecting one of the first pictures and one of the second pictures from the source domain as a data pair, for example, the picture levels of the to-be-selected domains include C1, C2 and C3, and if an image is selected from the target domain C1, the image must be selected from the source domain C2; if an image is selected from the target field C2, the image must be selected from the source field C3, so that an excessive deviation in data augmentation is avoided, and the accuracy of the data augmentation is improved.
In one embodiment, the selecting one from the plurality of candidate domains as the target domain includes:
acquiring the number of pictures of each domain to be selected;
sorting the domains to be selected according to the number of pictures;
and selecting at least one corresponding domain to be selected from the sorted domains to be selected according to the number of pictures.
In this embodiment, in a process of selecting one of a plurality of candidate domains as a target domain, the number of pictures of each candidate domain is obtained, then the candidate domains are sorted according to the number of pictures, then at least the corresponding candidate domain is selected in turn from the sorted candidate domains according to the number of pictures as the target domain, and the candidate domain with the largest number of pictures is subjected to data augmentation, so that after the number of pictures of the candidate domain meets the requirement, the next sorted candidate domain is subjected to data augmentation until the number of pictures of all the candidate domains meets the requirement, the number of pictures of all the candidate domains is augmented, the phenomenon that the data augmentation cannot be accurately completed due to insufficient data of the adjacent candidate domains in the candidate domains is avoided, and the data augmentation efficiency and accuracy are improved.
In one embodiment, the target field comprises a picture of a heavy black eye picture level and a light black eye picture level, and the source field comprises a picture of a light black eye picture level and a no black eye picture level; the matching the candidate picture level adjacent to the picture level of the target domain according to the preset level rule includes:
if the candidate picture level is the heavy black eye picture level, determining that the picture level of the source domain corresponding to the target domain is the light black eye picture level;
And if the candidate picture grade is the light black eye picture grade, determining that the picture grade of the source domain corresponding to the target domain is the black eye-free picture grade.
In this embodiment, in a process of matching candidate picture levels adjacent to the picture level of the target domain according to a preset level rule, setting three picture levels including a heavy black eye picture level, a light black eye picture level and a black eye-free picture level, wherein the target domain includes pictures of the heavy black eye picture level and the light black eye picture level, and the source domain includes pictures of the light black eye picture level and the black eye-free picture level; if the candidate picture grade is the heavy black eye picture grade, determining that the picture grade of the source domain corresponding to the target domain is the light black eye picture grade, and if the candidate picture grade is the light black eye picture grade, determining that the picture grade of the source domain corresponding to the target domain is the black eye free picture grade, the distortion caused by overlarge grade change of the picture data of the augmented data can be avoided, and therefore the accuracy of data augmentation is improved.
In the embodiment of the present application, a data augmentation method based on an countermeasure generation network is implemented by using a trained data augmentation model based on the countermeasure generation network, where a model for training to perform data augmentation on black eye images is described as an example, the training process based on the data augmentation model of the countermeasure generation network specifically includes classifying sample data into three types, a heavy black eye sample C1, a light black eye sample C2 and a normal sample C3, then taking the C1 and C2 samples as a target field (X), and taking the C2 and C3 samples as a source field (Y), constructing a CycleGAN network (generating the countermeasure network), where the CycleGAN network includes two generation networks and two discrimination networks, then selecting sample data to be input to the CycleGAN network, the sample data to be input each time is a set of images, and each set of images is selected according to the following rule, if the target field selects an image in C1, the source field must be selected from the images in C2; if the target field selects the image in C2, the source field must be selected from the image in C3 because in the real data distribution, the amount of C1 data is smaller, if C1 and C2 are the target fields, C3 is the source field. The model will tend to convert the C3 sample to the C2 sample, resulting in simulation data for a mostly light black eye sample, and different numbers of heavy and light samples can be generated by the selection rules of the sample data described above.
After the sample data is selected, respectively inputting the selected pictures to generate corresponding positions in an countermeasure network, wherein each time a group of images is selected as x, y, x represents a target domain image, y represents a source domain image, and a generating network G is defined to transfer the images from the target domain to the source domain, a generating network F represents transfer of the images from the source to the target domain, the selected x is input to the G network and then to the F network, F (x) is output, the selected y is input to the F network and then to the G network, G (y) is output, F (x) and x are calculated to form a consistency constraint (L cycle1), G (F (y)) and y are calculated to form a consistency constraint (L cycle2), and G (x) and y are input to a distinguishing network D Y to be classified by 0-1, and calculation is performedSimilarly, x and F (y) are input into a discrimination network D X to calculateAnd then subjecting the L cycle1、Lcycle2 to/(Back propagation is performed based on L cycle1、Lcycle2,Parameters in the corresponding network are updated continually such that L cycle1、Lcycle2,/>The values of (2) satisfy the preset values, and the parameter updating of the generation network and the two discrimination networks is completed, thereby completing the training of the data augmentation model based on the countermeasure generation network.
Referring to fig. 2, the present application also provides an apparatus for data augmentation based on an countermeasure generation network, comprising:
a data acquisition module 10, configured to acquire a first picture in a target domain and a second picture in a source domain;
The generating network module 20 is configured to input the first picture to a first generating network to obtain a pseudo second picture; inputting the pseudo second picture into a second generation network to obtain a pseudo first picture;
A constraint calculating module 30, configured to calculate a consistency constraint value of the first picture and the pseudo first picture;
The result classification module 40 is configured to input the second picture and the pseudo second picture to a discrimination network, and obtain a classification result of the discrimination network on the second picture and the pseudo second picture;
A loss calculation module 50 for calculating a loss value of the classification result according to the function;
The data adding module 60 is configured to add the pseudo first picture to the target domain to increase the picture data in the target domain if it is determined that the sum of the consistency constraint value and the loss value meets a preset value.
As described above, it will be understood that each component of the apparatus for data augmentation based on an countermeasure network set forth in the present application may implement the functions of any one of the methods for data augmentation based on an countermeasure network set forth above.
In one embodiment, the apparatus further comprises an augmentation module to perform:
Inputting the second picture into a second generation network to obtain a fake first picture; inputting the fake first picture into a first generation network to obtain a fake second picture;
calculating candidate consistency constraint values of the second picture and the fake second picture;
Inputting the second picture and the fake second picture into a second discrimination network to obtain candidate classification results of the second discrimination network on the second picture and the fake second picture;
calculating a candidate loss value of the candidate classification result according to a loss function;
And if the sum of the candidate consistency constraint value and the candidate loss value meets a preset value, adding the false second picture into the source domain so as to increase the picture data in the source domain.
In one embodiment, the source domain and the target domain are mutually mapped data sets.
In one embodiment, the apparatus further comprises a stage module for performing:
Acquiring an execution stage of current data augmentation;
and matching a preset value of the sum of the consistency constraint value and the loss value according to the execution stage.
In one embodiment, the apparatus further comprises a ranking module for performing:
selecting one of a plurality of to-be-selected domains as a target domain, and acquiring a picture grade of the target domain;
matching candidate picture grades adjacent to the picture grade of the target domain according to a preset grade rule;
And determining a source domain corresponding to the target domain according to the candidate picture level.
In an embodiment, the ranking module further comprises a selection unit for performing:
acquiring the number of pictures of each domain to be selected;
sorting the domains to be selected according to the number of pictures;
and selecting at least one corresponding domain to be selected from the sorted domains to be selected according to the number of pictures.
In one embodiment, the target field comprises a picture of a heavy black eye picture level and a light black eye picture level, and the source field comprises a picture of a light black eye picture level and a no black eye picture level; the ranking module further comprises a determining unit for performing:
if the candidate picture grade is judged to be the heavy black eye picture grade, determining that the picture grade of the source domain corresponding to the target domain is a light black eye picture grade;
And if the candidate picture grade is judged to be the light black eye picture grade, determining that the picture grade of the source domain corresponding to the target domain is the black eye-free picture grade.
Referring to fig. 3, in an embodiment of the present application, there is further provided a computer device, which may be a mobile terminal, and an internal structure thereof may be as shown in fig. 3. The computer device includes a processor, a memory, a network interface, and a display device and an input device connected by a system bus. The network interface of the computer device is used for communicating with an external terminal through network connection. The input means of the computer device is for receiving input from a user. The computer is designed to provide computing and control capabilities. The memory of the computer device includes a storage medium. The storage medium stores an operating system, computer programs, and a database. The database of the computer device is used for storing data. The computer program is executed by a processor to implement a method of data augmentation based on an antagonism generation network.
The processor performs the method of data augmentation based on the countermeasure generation network, comprising: acquiring a first picture in a target domain and a second picture in a source domain; inputting the first picture into a first generation network to obtain a pseudo second picture; inputting the pseudo second picture into a second generation network to obtain a pseudo first picture; the first generation network and the second generation network are mutually opposite networks; calculating a consistency constraint value of the first picture and the pseudo first picture; inputting the second picture and the pseudo second picture into a discrimination network to obtain a classification result of the discrimination network on the second picture and the pseudo second picture; calculating a loss value of the loss function of the classification result according to the loss function; and if the sum of the consistency constraint value and the loss value meets a preset value, adding the pseudo first picture into a target domain to increase the picture data in the target domain.
The computer equipment provides a data augmentation method based on an countermeasure generation network, first a first picture in a target domain and a second picture in a source domain are acquired, the target domain and the source domain are a group of data pairs, sample data need to be extracted from the target domain and the source domain at the same time, namely, one first picture is selected from the target domain and one second picture is selected from the source domain as the data pairs, and the first picture is input into the first generation network to obtain a pseudo second picture; inputting the pseudo second picture into a second generation network to obtain a pseudo first picture; the first generation network and the second generation network are opposite networks, a consistency constraint value of the first picture and the pseudo first picture is calculated, the consistency constraint value is the image matching degree of the first picture and the pseudo first picture, the second picture and the pseudo second picture are input into a discrimination network, a classification result of the discrimination network on the second picture and the pseudo second picture is obtained, then a loss value of the classification result is calculated according to a loss function, the loss value represents a difference value between the classification result and a real result of the second picture and the pseudo second picture, if the sum of the consistency constraint value and the loss value meets a preset value, the first picture is represented to obtain the pseudo first picture meeting the preset requirement after passing through the first generation network and the second generation network, and then the pseudo first picture is added into a target domain, so that the picture data in the target domain is increased, and the widening efficiency and the accuracy of the picture in the target domain are improved.
An embodiment of the present application also provides a computer-readable storage medium having stored thereon a computer program which, when executed by the processor, implements a method of data augmentation based on an countermeasure generation network, comprising the steps of: acquiring a first picture in a target domain and a second picture in a source domain; inputting the first picture into a first generation network to obtain a pseudo second picture; inputting the pseudo second picture into a second generation network to obtain a pseudo first picture; the first generation network and the second generation network are mutually opposite networks; calculating a consistency constraint value of the first picture and the pseudo first picture; inputting the second picture and the pseudo second picture into a discrimination network to obtain a classification result of the discrimination network on the second picture and the pseudo second picture; calculating a loss value of the classification result according to a loss function; and if the sum of the consistency constraint value and the loss value meets a preset value, adding the pseudo first picture into a target domain to increase the picture data in the target domain.
The computer readable storage medium provides a method for data augmentation based on an countermeasure generation network, which comprises the steps of firstly acquiring a first picture in a target domain and a second picture in a source domain, wherein the target domain and the source domain are a group of data pairs, extracting sample data from the target domain and the source domain at the same time, namely respectively selecting a first picture from the target domain and a second picture from the source domain as the data pairs, and inputting the first picture into the first generation network to obtain a pseudo second picture; inputting the pseudo second picture into a second generation network to obtain a pseudo first picture; the first generation network and the second generation network are mutually opposite networks, a consistency constraint value of the first picture and the pseudo first picture is calculated, the consistency constraint value is the image matching degree of the first picture and the pseudo first picture, the second picture and the pseudo second picture are input into a discrimination network, the classification result of the discrimination network on the second picture and the pseudo second picture is obtained, then a loss value of a loss function of the classification result is calculated according to a loss function, the loss value represents the difference value between the classification result and a real result of the second picture and the pseudo second picture, if the sum of the consistency constraint value and the loss value meets a preset value, the first picture is represented to obtain the pseudo first picture meeting the preset requirement after passing through the first generation network and the second generation network, and then the pseudo first picture is added into a target domain, so that the picture data in the target domain is increased, and the amplification efficiency and the accuracy of the picture data in the target domain are improved.
Those skilled in the art will appreciate that implementing all or part of the above described methods may be accomplished by way of a computer program stored on a non-transitory computer readable storage medium, which when executed, may comprise the steps of the embodiments of the methods described above.
Any reference to memory, storage, database, or other medium provided by the present application and used in embodiments may include non-volatile and/or volatile memory.
The nonvolatile memory can include Read Only Memory (ROM), programmable ROM (PROM), electrically Programmable ROM (EPROM), electrically Erasable Programmable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms such as Static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual speed data rate SDRAM (SSRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (SYNCHLINK) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), among others.
It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, apparatus, article, or method that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, apparatus, article, or method. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, apparatus, article, or method that comprises the element.
The foregoing is merely a preferred embodiment of the present application and is not intended to limit the scope of the application.
All equivalent structures or equivalent flow changes made by the specification and the attached drawings of the application or directly or indirectly applied to other related technical fields are included in the protection scope of the application.
Claims (8)
1. A method of data augmentation based on an countermeasure generation network, comprising:
Acquiring a first picture in a target domain and a second picture in a source domain;
Inputting the first picture into a first generation network to obtain a pseudo second picture; inputting the pseudo second picture into a second generation network to obtain a pseudo first picture; the first generation network and the second generation network are mutually opposite networks;
calculating a consistency constraint value of the first picture and the pseudo first picture;
Inputting the second picture and the pseudo second picture into a discrimination network to obtain a classification result of the discrimination network on the second picture and the pseudo second picture;
Calculating a loss value of the classification result according to a loss function;
if the sum of the consistency constraint value and the loss value meets a preset value, adding the pseudo first picture into a target domain to increase picture data in the target domain;
after the first picture in the target domain and the second picture in the source domain are acquired, the method further comprises:
Inputting the second picture into a second generation network to obtain a fake first picture; inputting the fake first picture into a first generation network to obtain a fake second picture;
calculating candidate consistency constraint values of the second picture and the fake second picture;
Inputting the second picture and the fake second picture into a second discrimination network to obtain candidate classification results of the second discrimination network on the second picture and the fake second picture;
calculating a candidate loss value of the candidate classification result according to a loss function;
if the sum of the candidate consistency constraint value and the candidate loss value meets a preset value, adding the false second picture into a source domain to increase picture data in the source domain;
the calculating the loss value of the classification result according to the loss function comprises the following steps:
Inputting the second picture and the pseudo second picture into a discrimination network to obtain a classification result a1 of the discrimination network on the second picture and a classification result a2 of the discrimination network on the pseudo second picture, and generating a classification result A of the discrimination network on the second picture and the pseudo second picture based on the classification result a1 and the classification result a2, wherein the classification result A comprises the classification result a1 and the classification result a2;
Calculating a loss value S1 according to a loss function on a classification result a1 of the second picture by the discrimination network, calculating a loss value S2 according to a classification result a2 of the pseudo second picture by the discrimination network, and generating a loss value S of a classification result A according to the loss value S1 and the loss value S2, wherein the loss value represents the difference value between the classification results of the second picture and the pseudo second picture and the real result;
before the first picture in the target domain and the second picture in the source domain are acquired, the method further comprises:
selecting one of a plurality of to-be-selected domains as a target domain, and acquiring a picture grade of the target domain;
matching candidate picture grades adjacent to the picture grade of the target domain according to a preset grade rule;
determining a source domain corresponding to the target domain according to the candidate picture level;
The target domain comprises a picture of a heavy black eye picture level and a light black eye picture level, the source domain comprises a picture of a light black eye picture level and a non-black eye picture level, and the data augmentation method based on the countermeasure generation network comprises a data augmentation model; the matching the candidate picture level adjacent to the picture level of the target domain according to the preset level rule includes:
taking a heavy black eye sample C1, a light black eye sample C2 and a normal sample C3 as sample data;
Taking C1 and C2 samples as a target domain X, taking C2 and C3 samples as a source domain Y, and constructing CycleGAN a network, wherein the CycleGAN network comprises two generation networks and two discrimination networks;
Selecting sample data and inputting the sample data into CycleGAN networks, wherein the sample data is a group of images, and the selection rule of each group of images is that if a target domain selects an image in C1, a source domain selects an image in C2; if the target domain selects the image in C2, the source domain selects from the image in C3;
After sample data are selected, respectively inputting the selected pictures to corresponding positions in a generated countermeasure network, wherein each time a group of images is selected as x and y, wherein x represents a target domain image, y represents a source domain image, a generated network G is defined to transfer the images from the target domain to the source domain, and a generated network F represents the transfer of the images from the source to the target domain;
Inputting the selected x into a G network, inputting the selected x into an F network, outputting F (G (x)), inputting the selected y into the F network, inputting the selected y into the G network, outputting G (F (y)), calculating a consistency constraint L cycle1 by the F (G (x)) and the x, and calculating a consistency constraint L cycle2 by the G (F (y)) and the y;
g (x) and y are input into a discrimination network D Y to be classified into 0-1 and calculated Similarly, x and F (y) are input into a discrimination network D X, and calculation/>
The L cycle1、Lcycle2,Back propagation is performed based on L cycle1、Lcycle2,Updating parameters in the corresponding network to make L cycle1、Lcycle2,/>The value of (2) satisfies a preset value, and based on the parameter update of the generating network and the two discriminating networks, the training of the data augmentation model is obtained.
2. The method of data augmentation based on an countermeasure generation network of claim 1, wherein the source domain and target domain are mutually mapped data sets.
3. The method of data augmentation based on an countermeasure generation network of claim 1, wherein the if the sum of the consistency constraint value and the loss value satisfies a preset value, before adding the pseudo first picture to a target domain, comprises:
Acquiring an execution stage of current data augmentation;
and matching a preset value of the sum of the consistency constraint value and the loss value according to the execution stage.
4. The method for data augmentation based on an countermeasure generation network of claim 1, wherein selecting one of a plurality of candidate fields as a target field comprises:
acquiring the number of pictures of each domain to be selected;
sorting the domains to be selected according to the number of pictures;
and selecting at least one corresponding domain to be selected from the sorted domains to be selected according to the number of pictures.
5. The method of claim 1, wherein the target field comprises a picture of a heavy black eye picture level and a light black eye picture level, and the source field comprises a picture of a light black eye picture level and a no black eye picture level; the matching the candidate picture level adjacent to the picture level of the target domain according to the preset level rule includes:
if the candidate picture level is the heavy black eye picture level, determining that the picture level of the source domain corresponding to the target domain is the light black eye picture level;
And if the candidate picture grade is the light black eye picture grade, determining that the picture grade of the source domain corresponding to the target domain is the black eye-free picture grade.
6. An apparatus for data augmentation based on an antagonism generation network for implementing the method of any one of claims 1-5, comprising:
the data acquisition module is used for acquiring a first picture in a target domain and a second picture in a source domain;
The generation network module is used for inputting the first picture into a first generation network to obtain a pseudo second picture; inputting the pseudo second picture into a second generation network to obtain a pseudo first picture;
the constraint calculating module is used for calculating a consistency constraint value of the first picture and the pseudo first picture;
The result classification module is used for inputting the second picture and the pseudo second picture into a discrimination network to obtain classification results of the discrimination network on the second picture and the pseudo second picture;
The loss calculation module is used for calculating a loss value of the classification result according to a loss function;
The data adding module is used for adding the pseudo first picture to the target domain to increase the picture data in the target domain if the sum of the consistency constraint value and the loss value meets a preset value;
the augmentation module is used for inputting the second picture into a second generation network to obtain a fake first picture; inputting the fake first picture into a first generation network to obtain a fake second picture;
calculating candidate consistency constraint values of the second picture and the fake second picture;
Inputting the second picture and the fake second picture into a second discrimination network to obtain candidate classification results of the second discrimination network on the second picture and the fake second picture;
calculating a candidate loss value of the candidate classification result according to a loss function;
And if the sum of the candidate consistency constraint value and the candidate loss value meets a preset value, adding the false second picture into the source domain so as to increase the picture data in the source domain.
7. A computer device comprising a memory and a processor, the memory storing a computer program, characterized in that the processor, when executing the computer program, implements the steps of the method of any one of claims 1 to 5 for data augmentation based on an countermeasure generation network.
8. A computer readable storage medium having stored thereon a computer program, characterized in that the computer program when executed by a processor implements the steps of the method of data augmentation based on an countermeasure generation network of any one of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110970515.4A CN113658036B (en) | 2021-08-23 | 2021-08-23 | Data augmentation method, device, computer and medium based on countermeasure generation network |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110970515.4A CN113658036B (en) | 2021-08-23 | 2021-08-23 | Data augmentation method, device, computer and medium based on countermeasure generation network |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113658036A CN113658036A (en) | 2021-11-16 |
| CN113658036B true CN113658036B (en) | 2024-05-31 |
Family
ID=78492554
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110970515.4A Active CN113658036B (en) | 2021-08-23 | 2021-08-23 | Data augmentation method, device, computer and medium based on countermeasure generation network |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113658036B (en) |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110222728A (en) * | 2019-05-15 | 2019-09-10 | 图灵深视(南京)科技有限公司 | The training method of article discrimination model, system and article discrimination method, equipment |
| CN110796199A (en) * | 2019-10-30 | 2020-02-14 | 腾讯科技（深圳）有限公司 | Image processing method and device and electronic medical equipment |
| CN111382596A (en) * | 2018-12-27 | 2020-07-07 | 鸿富锦精密工业(武汉)有限公司 | Face recognition method and device and computer storage medium |
| CN111539897A (en) * | 2020-05-09 | 2020-08-14 | 北京百度网讯科技有限公司 | Method and apparatus for generating image conversion model |
| CN111539467A (en) * | 2020-04-17 | 2020-08-14 | 北京工业大学 | GAN network architecture and method for data augmentation of medical image data set based on generation of countermeasure network |
| CN112529154A (en) * | 2020-12-07 | 2021-03-19 | 北京百度网讯科技有限公司 | Image generation model training method and device and image generation method and device |
| WO2021120752A1 (en) * | 2020-07-28 | 2021-06-24 | 平安科技（深圳）有限公司 | Region-based self-adaptive model training method and device, image detection method and device, and apparatus and medium |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP7046786B2 (en) * | 2018-12-11 | 2022-04-04 | 株式会社日立製作所 | Machine learning systems, domain converters, and machine learning methods |
-
2021
- 2021-08-23 CN CN202110970515.4A patent/CN113658036B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111382596A (en) * | 2018-12-27 | 2020-07-07 | 鸿富锦精密工业(武汉)有限公司 | Face recognition method and device and computer storage medium |
| CN110222728A (en) * | 2019-05-15 | 2019-09-10 | 图灵深视(南京)科技有限公司 | The training method of article discrimination model, system and article discrimination method, equipment |
| CN110796199A (en) * | 2019-10-30 | 2020-02-14 | 腾讯科技（深圳）有限公司 | Image processing method and device and electronic medical equipment |
| CN111539467A (en) * | 2020-04-17 | 2020-08-14 | 北京工业大学 | GAN network architecture and method for data augmentation of medical image data set based on generation of countermeasure network |
| CN111539897A (en) * | 2020-05-09 | 2020-08-14 | 北京百度网讯科技有限公司 | Method and apparatus for generating image conversion model |
| WO2021120752A1 (en) * | 2020-07-28 | 2021-06-24 | 平安科技（深圳）有限公司 | Region-based self-adaptive model training method and device, image detection method and device, and apparatus and medium |
| CN112529154A (en) * | 2020-12-07 | 2021-03-19 | 北京百度网讯科技有限公司 | Image generation model training method and device and image generation method and device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113658036A (en) | 2021-11-16 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11887311B2 (en) | Method and apparatus for segmenting a medical image, and storage medium | |
| CN105917353B (en) | Feature extraction and matching for biological identification and template renewal | |
| CN112102237A (en) | Brain tumor recognition model training method and device based on semi-supervised learning | |
| CN111862044B (en) | Ultrasonic image processing method, ultrasonic image processing device, computer equipment and storage medium | |
| Chaki et al. | A deep learning based four-fold approach to classify brain MRI: BTSCNet | |
| CN113688912B (en) | Method, device, equipment and medium for generating countermeasure sample based on artificial intelligence | |
| CN110210543B (en) | Image classification system, method, apparatus and storage medium | |
| US11263494B2 (en) | Classification device, classification method, program, and information recording medium | |
| CN113240606B (en) | Traditional Chinese medicine inspection image color correction method and system | |
| CN112836653A (en) | Face privacy method, device and apparatus and computer storage medium | |
| CN111402217A (en) | Image grading method, device, equipment and storage medium | |
| Albuquerque et al. | Object detection for automatic cancer cell counting in zebrafish xenografts | |
| CN112101456A (en) | Attention feature map acquisition method and device and target detection method and device | |
| CN115631397A (en) | Target detection method and device based on bimodal image | |
| CN113486925B (en) | Model training method, fundus image generation method, model evaluation method and device | |
| CN111598144B (en) | Training method and device for image recognition model | |
| CN112101438B (en) | Left-right eye classification method, device, server and storage medium | |
| CN113433130A (en) | Method and device for generating confocal imaging by wide-field imaging | |
| CN113658036B (en) | Data augmentation method, device, computer and medium based on countermeasure generation network | |
| CN112926508A (en) | Training method and device of living body detection model | |
| Yang et al. | Finding badly drawn bunnies | |
| Wu et al. | Multifocus image fusion using random forest and hidden Markov model | |
| He et al. | Discovering intrinsic properties of human observers’ visual search and mathematical observers’ scanning | |
| CN114372970A (en) | Operation reference information generation method and device | |
| Syamala et al. | Brain MRI image bias correction using generative adversarial network |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |