CN112614053B - Method and system for generating multiple images based on single image of antagonistic neural network - Google Patents
Method and system for generating multiple images based on single image of antagonistic neural network Download PDFInfo
- Publication number
- CN112614053B CN112614053B CN202011568784.XA CN202011568784A CN112614053B CN 112614053 B CN112614053 B CN 112614053B CN 202011568784 A CN202011568784 A CN 202011568784A CN 112614053 B CN112614053 B CN 112614053B
- Authority
- CN
- China
- Prior art keywords
- image
- discriminator
- module
- generated
- generator
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 42
- 238000013528 artificial neural network Methods 0.000 title claims abstract description 30
- 230000003042 antagnostic effect Effects 0.000 title claims abstract description 22
- 230000003321 amplification Effects 0.000 claims description 21
- 238000003199 nucleic acid amplification method Methods 0.000 claims description 21
- 230000003190 augmentative effect Effects 0.000 claims description 16
- 238000013527 convolutional neural network Methods 0.000 claims description 14
- 230000004927 fusion Effects 0.000 claims description 12
- 239000011159 matrix material Substances 0.000 claims description 9
- 230000004913 activation Effects 0.000 claims description 7
- 238000012937 correction Methods 0.000 claims description 7
- 238000010606 normalization Methods 0.000 claims description 7
- 238000012545 processing Methods 0.000 abstract description 8
- 230000008859 change Effects 0.000 abstract description 4
- 230000007547 defect Effects 0.000 abstract description 4
- 238000007796 conventional method Methods 0.000 abstract 1
- 238000010586 diagram Methods 0.000 description 4
- 238000012549 training Methods 0.000 description 4
- 230000008569 process Effects 0.000 description 3
- 238000001514 detection method Methods 0.000 description 2
- 230000000877 morphologic effect Effects 0.000 description 2
- 238000012360 testing method Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 238000003062 neural network model Methods 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
- G06T3/4046—Scaling of whole images or parts thereof, e.g. expanding or contracting using neural networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
- G06T3/4038—Image mosaicing, e.g. composing plane images from plane sub-images
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Computation (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a method and a system for generating a plurality of images based on a single image of an antagonistic neural network, which relate to the field of image processing and aim to overcome the defects of the conventional method and the conventional system for amplifying the imagesThe system can not change the self contour characteristics of the image, and has the problems of high labor and time cost and the need of a large number of original images, wherein the method specifically comprises the following steps: step one, obtaining a single sample image; step two, establishing an image pyramid; step three, obtaining a pre-generated image, and sending the pre-generated image to a pre-discriminator for discrimination; fourthly, generating an image and an image Z in the front0Fused feed to generator G0In (2), a generated image R is obtained0And judging; and obtaining a fused image; step five, sending the fused image into a generator GNIn (2), a generated image R is obtainedNAnd judging; and will generate an image RNZooming by a zooming coefficient r to obtain an Nth fused image; let N add 1 and perform step five again until N ═ T-1.
Description
Technical Field
The invention relates to the field of image processing, in particular to a method and a system for amplifying an image based on an antagonistic neural network.
Background
The existing method for amplifying the image comprises the following steps: the following problems mainly exist in image amplification based on traditional image processing, image processing amplification based on application software and image amplification based on the existing anti-neural network:
1. image amplification based on image processing is the mainstream image amplification method at present, and includes flipping, rotating, scaling, random cropping or zero padding, color dithering, noise adding and the like.
2. The processing and amplification of the image through software is a stable image amplification mode at present. Taking the Photoshop software as an example, the Photoshop software can carry out corresponding transformation according to the morphological characteristics of the image to achieve the effect of image amplification, but in practical application, the Photoshop software has the following two defects, one is that the labor cost and the time cost are too high; the form of the conversion is changed according to the change of the thought of the software user, and the reliability is not high.
3. Image amplification based on the existing anti-neural network is an emerging image amplification mode at present, but a large number of original images are required in the process of training a generator and an arbiter.
Disclosure of Invention
The invention aims to overcome the problems that the existing method and system for amplifying the image cannot change the self contour characteristics of the image, the labor and time cost is high, and a large number of original images are needed, and provides a method and system for generating a plurality of images based on a single image of an anti-neural network.
The invention relates to a method for generating a plurality of images based on a single image of an antagonistic neural network, wherein the antagonistic neural network comprises a pre-generator, a pre-discriminator and a generator G0～GT-1And a discriminator D0～DT-1The method comprises the following steps:
step one, obtaining a single sample image;
establishing an image pyramid, wherein the layer number of the image pyramid is T, and a fixed scaling coefficient r is arranged between two adjacent layers of images of the image pyramid;
and taking the sample image as the image of the T-th layer; and the images from the first layer to the T layer of the image pyramid are set as { Z }0,…,ZT-1}；
Step three, sending the first Gaussian noise image into a pre-generator to obtain a pre-generated image, and sending the pre-generated image into a pre-discriminator to be discriminated; first Gaussian noise image and image Z0Equal in size;
if the pre-discriminator judges that the pre-generated image is true, the pre-generated image is used as an amplification image of the sample image;
fourthly, generating an image and an image Z in the front0Fused feed to generator G0In (2), a generated image R is obtained0And will generate an image R0And image Z0Are all sent into a discriminator D0Carrying out judgment;
if discriminator D0Discriminating the generated image R0If true, image R will be generated0An augmented image as a sample image;
and will generate an image R0Scaling by a scaling factor R to obtain a scaled image R0Fusing the image with the Gaussian noise image with the corresponding size to obtain a 0 th fused image;
step five, sending the N-1 th fusion image into a generator GNIn (2), a generated image R is obtainedNAnd will generate an image RNAnd image ZNAre all sent into a discriminator DNCarrying out judgment; wherein Z isN、GN、DNSubscript N in the formula (I) is a positive integer, and the initial value of N is 1;
if discriminator DNDiscriminating the generated image RNIf true, image R will be generatedNAn augmented image as a sample image;
and will generate an image RNScaling by a scaling factor R to obtain a scaled image RNFusing the image with the Gaussian noise image with the corresponding size to obtain an Nth fused image;
let N add 1 and perform step five again until N ═ T-1, obtain all the augmented images of the sample image.
Further, in the first step, the size of the sample image is set to be M multiplied by N multiplied by 3, and M is less than or equal to N;
in the second step, the image size of the first layer of the image pyramid is
And the number of layers T of the image pyramid is obtained by the following formula:
further, in step three, the pre-generator comprises 3 sequentially connected full convolution networks;
the full convolution network comprises a convolution module with a convolution kernel of 3x3, a batch normalization BatchNorm module and a leakage correction linear unit LR activation function module which are sequentially connected;
the structure of the pre-arbiter is a Convolutional Neural Network (CNN) classifier structure.
Further, in step four and step five, the generator G0～GT-1And discriminator D0～DT-1Each comprising 5 sequentially connected full convolutional networks;
and the mean value of the output matrix of each discriminator is used as the two-classification output of the corresponding discriminator.
Further, the specific steps of acquiring a single sample image with a real fault in the step one are as follows:
the method comprises the steps of scanning a moving object to be detected line by line to obtain a plurality of high-definition linear array images, and splicing the plurality of high-definition linear array images into a complete image of the object to be detected;
and secondly, intercepting the part to be detected of the object to be detected according to the parameters of the object to be detected to obtain a subgraph, and selecting an image which needs to be amplified and has a real fault from the intercepted subgraph as a sample image.
The invention relates to a system for generating a plurality of images based on a single image of an antagonistic neural network, wherein the antagonistic neural network comprises a pre-generator, a pre-discriminator and a generator G0～GT-1And a discriminator D0～DT-1The system comprises:
the sample image acquisition module is used for acquiring a single sample image;
the image pyramid module is connected with the sample image acquisition module and used for establishing an image pyramid, the layer number of the image pyramid is T, and a fixed scaling coefficient r is arranged between two adjacent layers of images of the image pyramid;
and taking the sample image as the image of the T-th layer; and the images from the first layer to the T layer of the image pyramid are set as { Z }0,…,ZT-1}；
The pre-generation judging module is connected with the image pyramid module and used for sending the first Gaussian noise image into the pre-generator to obtain a pre-generated image and sending the pre-generated image into the pre-discriminator to be judged; first Gaussian noise image and image Z0Equal in size;
if the pre-discriminator judges that the pre-generated image is true, the pre-generated image is used as an amplification image of the sample image;
a first generation judging module connected with the pre-generation judging module and used for leading the pre-generated image to be connected with the image Z0Fused feed to generator G0In (2), a generated image R is obtained0And sent to the second studentA composition judging module and generates an image R0And image Z0Are all sent into a discriminator D0Carrying out judgment;
if discriminator D0Discriminating the generated image R0If true, image R will be generated0An augmented image as a sample image;
and will generate an image R0Scaling by a scaling factor R to obtain a scaled image R0Fusing the image with the Gaussian noise image with the corresponding size to obtain a 0 th fused image;
the second generation discrimination module is connected with the first generation discrimination module and is used for receiving the (N-1) th fusion image and the N and sending the (N-1) th fusion image to the generator GNIn (2), a generated image R is obtainedNAnd will generate an image RNAnd image ZNAre all sent into a discriminator DNCarrying out judgment; wherein Z isN、GN、DNSubscript N in (1) is a positive integer;
if discriminator DNDiscriminating the generated image RNIf true, image R will be generatedNAn augmented image as a sample image;
and will generate an image RNScaling by a scaling factor R to obtain a scaled image RNFusing the image with the Gaussian noise image with the corresponding size to obtain an Nth fused image;
a calling module connected with the first generation discrimination module and the second generation discrimination module, and a discriminator D0After the discrimination is completed, let N equal to 1 and send it to the second generation discrimination module, discriminator DNAnd adding 1 to N and sending the N to a second generation judging module every time the judgment is finished, and obtaining all the amplification images of the sample image until N is T-1.
Furthermore, the size of the sample image acquired by the sample image acquisition module is M multiplied by N multiplied by 3, and M is less than or equal to N;
the image size of the first layer of the image pyramid in the image pyramid module is
And the number of layers T of the image pyramid is obtained by the following formula:
where r is the scaling factor.
Further, the pre-generation judging module comprises a pre-generator G and a pre-discriminator D;
the pre-generator G comprises 3 sequentially connected full convolutional networks;
the full convolution network comprises a convolution module with a convolution kernel of 3x3, a batch normalization BatchNorm module and a leakage correction linear unit LR activation function module which are sequentially connected;
the structure of the pre-arbiter D is a convolutional neural network CNN classifier structure.
Further, the first generation discrimination module comprises a generator G0And a discriminator D0The second generation discrimination module comprises a generator G1～GT-1And a discriminator D1～DT-1；
Generator G0～GT-1And discriminator D0～DT-1Each comprising 5 sequentially connected full convolutional networks;
and the mean value of the output matrix of each discriminator is used as the two-classification output of the corresponding discriminator.
Further, the sample image acquisition module includes:
the image scanning and splicing module is used for scanning a moving object to be detected line by line to obtain a plurality of high-definition linear array images, and splicing the plurality of high-definition linear array images into a complete image of the object to be detected;
and the sub-image interception selection module is connected with the image scanning and splicing module and is used for intercepting the part to be detected of the object to be detected according to the parameters of the object to be detected to obtain a sub-image and selecting an image which needs to be amplified and has a real fault from the intercepted sub-image as a sample image.
The invention has the beneficial effects that:
and performing countermeasure training on the image layer by adopting an image pyramid mode. The small-size image generates a main body outline, and the large-size image generates main body details, so that the generated image has most of the main body information of the original image, and meanwhile, part of detail information is reserved. And is
Compared with the image amplification based on the existing image processing, the image amplified by the method and the system has certain morphological characteristic change.
Compared with the method for processing and amplifying the image based on the application software, the method and the system can save a large amount of labor cost and time cost and have higher reliability.
Compared with the image amplification mode based on the existing antagonistic neural network, the method and the system only need a single sample image.
Drawings
FIG. 1 is a flow chart of a method of generating multiple images based on a single image of an anti-neural network according to a first embodiment;
FIG. 2 is a schematic diagram of the method and system for generating multiple images based on a single image of an anti-neural network of the present invention;
FIG. 3 is a schematic structural diagram of an image pyramid in the method and system for generating multiple images based on a single image of an anti-neural network according to the present invention;
FIG. 4 is a schematic diagram of a sample image in the method and system for generating multiple images based on a single image of an anti-neural network according to the present invention;
FIG. 5 is a schematic diagram of the augmented image in the method and system for generating multiple images based on a single image of the anti-neural network according to the present invention.
Detailed Description
It should be noted that, in a non-conflicting manner, features included in each or each of the implementations disclosed in the present application may be combined with each other.
First embodiment, the method for generating a plurality of images based on a single image of an antagonistic neural network according to the present embodiment, the antagonistic neural network includes a pre-generator, a pre-discriminator, and a generator G0～GT-1Sum discriminatorD0～DT-1As shown in fig. 1, the method specifically includes:
step one, obtaining a single sample image;
establishing an image pyramid, wherein the layer number of the image pyramid is T, and a fixed scaling coefficient r is arranged between two adjacent layers of images of the image pyramid;
and taking the sample image as the image of the T-th layer; and the images from the first layer to the T layer of the image pyramid are set as { Z }0,…,ZT-1}；
Step three, sending the first Gaussian noise image into a pre-generator to obtain a pre-generated image, and sending the pre-generated image into a pre-discriminator to be discriminated; first Gaussian noise image and image Z0Equal in size;
if the pre-discriminator judges that the pre-generated image is true, the pre-generated image is used as an amplification image of the sample image;
fourthly, generating an image and an image Z in the front0Fused feed to generator G0In (2), a generated image R is obtained0And will generate an image R0And image Z0Are all sent into a discriminator D0Carrying out judgment;
if discriminator D0Discriminating the generated image R0If true, image R will be generated0An augmented image as a sample image;
and will generate an image R0Scaling by a scaling factor R to obtain a scaled image R0Fusing the image with the Gaussian noise image with the corresponding size to obtain a 0 th fused image;
step five, sending the N-1 th fusion image into a generator GNIn (2), a generated image R is obtainedNAnd will generate an image RNAnd image ZNAre all sent into a discriminator DNCarrying out judgment; wherein Z isN、GN、DNSubscript N in the formula (I) is a positive integer, and the initial value of N is 1;
if discriminator DNDiscriminating the generated image RNIf true, image R will be generatedNAn augmented image as a sample image;
and will generate an image RNScaling by a scaling factor R to obtain a scaled image RNFusing the image with the Gaussian noise image with the corresponding size to obtain an Nth fused image;
let N add 1 and perform step five again until N ═ T-1, obtain all the augmented images of the sample image.
Specifically, in order to enable the generated amplified image to have detail characteristics of original sample data, the model is designed by adopting the idea that a main body outline is generated through a small-size image and main body details are generated through a large-size image.
Let the image pyramid be { Z } in order from the top (first layer) to the bottom (T-th layer)0，Z1，...，ZT-1The network model principle of the method includes the following specific steps, as shown in fig. 2:
1. will have a size of
First gaussian noise image x0Sending into a pre-generator G to obtain a pre-generated image
Then will be
Sending the data to a pre-discriminator D for discrimination;
2. will be provided with
And image pyramid top layer image Z0After the fusion (picture superposition mode can be adopted), the fusion data is sent to a first generator G0In (2), obtaining a first generated image R0Then Z is0And R0Into a first Markov decision device D0Carrying out judgment;
3. r is to be0Multiplied by a fixed scaling factor r and corresponding size of Gaussian noise image x1Is sent into the secondGenerator G1In (2), obtaining a second generation image R1Then Z is1And R1Into a second Markov decision device D1Carrying out judgment;
4. r is to be1Multiplied by a fixed scaling factor r and corresponding size of Gaussian noise image x2Is sent to a third generator G2In (3), a third generated image R is obtained2Then Z is2And R2Into a third Markov decision device D2Carrying out judgment;
5. and so on until R is gotT-2Multiplied by a fixed scaling factor r and corresponding size of Gaussian noise image xT-1Is sent to the Tth generator GT-1In (2), a generated image R is obtainedT-1Then Z isT-1And RT-1Into the T Markov decision device DT-1Is determined.
Further, in the first step, the size of the sample image is set to be M multiplied by N multiplied by 3, and M is less than or equal to N;
in the second step, the image size of the first layer of the image pyramid is
And the number of layers T of the image pyramid is obtained by the following formula:
in particular, the mechanical structure of the exterior of the motor vehicle is complex in composition, and some parts play a crucial role in the driving process of the motor vehicle, and the integrity of these parts becomes important in order to ensure the safe and fast form of the motor vehicle. Because real faults have unknown, heterogeneous, rare and diverse properties, data samples of the real faults are few, and further high-precision fault detection cannot be realized. Therefore, firstly, the real fault image of the motor train needs to be generated. According to the method for generating the motor car fault images, a plurality of antagonistic neural network models are used for training single real fault images with different scales, and then a plurality of fault images are generated. The basic idea is as follows: and sending the selected image with the real fault into the model of the method for training, and then generating the selected sample. The method for generating a plurality of fault images based on the single railway motor train fault image of the antagonistic neural network comprises the following steps:
the method comprises the steps of establishing an image pyramid, wherein a railway motor car fault sample is intercepted according to the position of a motor car fault or a part, so that the size of the intercepted image is not a fixed value, and in order to achieve the accuracy of sample generation, the method does not directly adopt the traditional scaling treatment, but establishes the image pyramid through a scaling mode according to the size of the image. Setting the size of the image as M multiplied by N multiplied by 3(M is less than or equal to N) (wherein 3 is a channel value), in order to ensure that the pyramid top-level image contains the main body outline information of the original image, manually setting the size of the top-level image as the prior knowledge
Then, assuming that the pyramid layer number is T and a fixed scaling coefficient r exists between the upper layer and the lower layer, the image pyramid structure is as shown in fig. 3.
The number of layers T of the image pyramid is calculated according to the following formula, and the size and the number of layers T of each layer of image in the image pyramid are positive integers in the calculation process. Thus the image pyramid layer number is only related to the short side length of the image.
Further, in step three, the pre-generator comprises 3 sequentially connected full convolution networks;
the full convolution network comprises a convolution module with a convolution kernel of 3x3, a batch normalization BatchNorm module and a leakage correction linear unit LR activation function module which are sequentially connected;
the structure of the pre-arbiter is a Convolutional Neural Network (CNN) classifier structure.
Further, in step four and step five, the generator G0～GT-1And discriminator D0～DT-1Each comprising 5 sequentially connected full convolutional networks;
and the mean value of the output matrix of each discriminator is used as the two-classification output of the corresponding discriminator.
Specifically, in fig. 2W denotes image fusion, S denotes image multiplication by a fixed scaling factor r, and the pre-generator G is a full convolution network with 3 activation functions formatted as convolution kernel (3x3) + batch normalization (BatchNorm) + leakage correction linear unit (LR, LeakyReLu), which is chosen because images of arbitrary size and aspect ratio can be generated.
The pre-arbiter D adopts the traditional CNN classifier structure. Subsequent generator Gi(i-0, 1.., T-1) is a full convolution network with 5 formats of (3x3) + BatchNorm + leak relu. Subsequent discriminator Di(i 0, 1.., T-1) is a markov discriminator, which is structured with a generator GiThe same is true. Different from the conventional CNN classifier structure, the output of the Markov classifier is a matrix, each output in the matrix represents a receptive field in the original image, and finally the average value of the output matrix is taken as the output of the second classification. Comparing discriminator D with discriminator DiAnd the detail characteristics of the sample can be better distinguished.
Further, the specific steps of acquiring a single sample image with a real fault in the step one are as follows:
the method comprises the steps of scanning a moving object to be detected line by line to obtain a plurality of high-definition linear array images, and splicing the plurality of high-definition linear array images into a complete image of the object to be detected;
and secondly, intercepting the part to be detected of the object to be detected according to the parameters of the object to be detected to obtain a subgraph, and selecting an image which needs to be amplified and has a real fault from the intercepted subgraph as a sample image.
Specifically, the specific steps of obtaining a single sample image with real faults in the step one are as follows
The method comprises the following steps of:
high-definition linear array imaging devices are arranged on two sides and the bottom of a rail of the motor car, the train starts an image acquisition device through a trigger sensor, the slowly moving motor car is scanned line by line, a plurality of high-definition linear array images with the size of 1440 x 1440 are obtained, and then the high-definition linear array images are spliced into a complete train image according to the axle distance information and the priori knowledge of the motor car.
The first step is to intercept the image of the subgraph to be amplified:
and intercepting different modules or parts of the motor train according to the train wheelbase, the train type and the prior knowledge, and manually selecting an image which needs to be amplified and has a real fault in an intercepted subgraph.
Model testing
The real fault image of a single railway motor car is tested by using the model of the method, the image of a sample with the real fault is shown in figure 4, and the amplified image obtained by the test result is shown in figure 5.
It can be seen from fig. 5 that the generated augmented image has a higher similarity with the original image main body information, and the detail information features thereof have a certain difference, which can effectively improve the defects in the prior art. Therefore, the method can well solve the problems that the number of data samples of real faults is small, and high-precision fault detection cannot be realized subsequently and the defects in the prior art.
In the second embodiment, the system of the present embodiment generates a plurality of images based on a single image of an antagonistic neural network, and the antagonistic neural network includes a pre-generator, a pre-discriminator, and a generator G0～GT-1And a discriminator D0～DT-1The system comprises:
the sample image acquisition module 1 is used for acquiring a single sample image;
the image pyramid module 2 is connected with the sample image acquisition module 1 and used for establishing an image pyramid, the layer number of the image pyramid is T, and a fixed scaling coefficient r is arranged between two adjacent layers of images of the image pyramid;
and taking the sample image as the image of the T-th layer; and isLet the images of the first layer to the T-th layer of the image pyramid be { Z }0,…,ZT-1}；
The pre-generation judging module 3 is connected with the image pyramid module 2 and used for sending the first Gaussian noise image into the pre-generator to obtain a pre-generated image and sending the pre-generated image into the pre-discriminator to be judged; first Gaussian noise image and image Z0Equal in size;
if the pre-discriminator judges that the pre-generated image is true, the pre-generated image is used as an amplification image of the sample image;
a first generation discrimination module 4 connected to the pre-generation discrimination module 3 for generating the pre-generated image and the image Z0Fused feed to generator G0In (2), a generated image R is obtained0And sends the image R to a second generation discrimination module 50And image Z0Are all sent into a discriminator D0Carrying out judgment;
if discriminator D0Discriminating the generated image R0If true, image R will be generated0An augmented image as a sample image;
and will generate an image R0Scaling by a scaling factor R to obtain a scaled image R0Fusing the image with the Gaussian noise image with the corresponding size to obtain a 0 th fused image;
a second generation discrimination module 5 connected with the first generation discrimination module 4 for receiving the N-1 th fusion image and N and sending the N-1 th fusion image to the generator GNIn (2), a generated image R is obtainedNAnd will generate an image RNAnd image ZNAre all sent into a discriminator DNCarrying out judgment; wherein Z isN、GN、DNSubscript N in (1) is a positive integer;
if discriminator DNDiscriminating the generated image RNIf true, image R will be generatedNAn augmented image as a sample image;
and will generate an image RNScaling by a scaling factor R to obtain a scaled image RNTo corresponding sizeThe Gaussian noise images are fused to obtain an Nth fused image;
a calling module 6 connected with the first generation discrimination module 4 and the second generation discrimination module 5, and a discriminator D0After the discrimination is completed, let N equal to 1 and send it to the second generation discrimination module 5, the discriminator DNAnd adding 1 to N and sending the N to the second generation judging module 5 every time the judgment is finished, and obtaining all the amplification images of the sample image until N is T-1.
Furthermore, the size of the sample image acquired by the sample image acquisition module is M multiplied by N multiplied by 3, and M is less than or equal to N;
the image size of the first layer of the image pyramid in the image pyramid module 2 is
And the number of layers T of the image pyramid is obtained by the following formula:
further, the pre-generation discrimination module 3 comprises a pre-generator G and a pre-discriminator D;
the pre-generator G comprises 3 sequentially connected full convolutional networks;
the full convolution network comprises a convolution module with a convolution kernel of 3x3, a batch normalization BatchNorm module and a leakage correction linear unit LR activation function module which are sequentially connected;
the structure of the pre-arbiter D is a convolutional neural network CNN classifier structure.
Further, the first generation discrimination module 4 includes a generator G0And a discriminator D0The second generation discrimination module 5 comprises a generator G1～GT-1And a discriminator D1～DT-1；
Generator G0～GT-1And discriminator D0～DT-1Each comprising 5 sequentially connected full convolutional networks;
and the mean value of the output matrix of each discriminator is used as the two-classification output of the corresponding discriminator.
Further, the specimen image acquisition module 1 includes:
the image scanning and splicing module 1-1 is used for scanning a moving object to be detected line by line to obtain a plurality of high-definition linear array images, and splicing the plurality of high-definition linear array images into a complete image of the object to be detected;
and the subgraph interception selection module 1-2 is connected with the image scanning splicing module 1-1 and is used for intercepting the part to be detected of the object to be detected according to the parameters of the object to be detected to obtain a subgraph and selecting an image which needs to be amplified and has a real fault from the intercepted subgraphs as a sample image.
Claims (8)
1. Method for generating a plurality of images based on a single image of an antagonistic neural network, characterized in that said antagonistic neural network comprises a pre-generator, a pre-discriminator, a generator G0～GT-1And a discriminator D0～DT-1The method comprises the following steps:
step one, obtaining a single sample image;
establishing an image pyramid, wherein the layer number of the image pyramid is T, and a fixed scaling coefficient r is arranged between two adjacent layers of images of the image pyramid;
and taking the sample image as an image of a T-th layer; and the images from the first layer to the T layer of the image pyramid are set as { Z }0,…,ZT-1}；
Step three, sending the first Gaussian noise image into a pre-generator to obtain a pre-generated image, and sending the pre-generated image into a pre-discriminator to be discriminated; the first Gaussian noise image and image Z0Equal in size;
if the pre-discriminator judges that the pre-generated image is true, the pre-generated image is used as an amplification image of the sample image;
fourthly, generating an image and an image Z in the front0Fused feed to generator G0In (2), a generated image R is obtained0And will generate an imageR0And image Z0Are all sent into a discriminator D0Carrying out judgment;
if discriminator D0Discriminating the generated image R0If true, image R will be generated0An augmented image as a sample image;
and will generate an image R0Scaling by a scaling factor R to obtain a scaled image R0Fusing the image with the Gaussian noise image with the corresponding size to obtain a 0 th fused image;
step five, sending the N-1 th fusion image into a generator GNIn (2), a generated image R is obtainedNAnd will generate an image RNAnd image ZNAre all sent into a discriminator DNCarrying out judgment; wherein Z isN、GN、DNSubscript N in the formula (I) is a positive integer, and the initial value of N is 1;
if discriminator DNDiscriminating the generated image RNIf true, image R will be generatedNAn augmented image as a sample image;
and will generate an image RNScaling by a scaling factor R to obtain a scaled image RNFusing the image with the Gaussian noise image with the corresponding size to obtain an Nth fused image;
step six, adding 1 to N and executing the step five again until N is T-1, and obtaining all amplification images of the sample image;
in the first step, the size of a sample image is set to be M multiplied by N multiplied by 3, and M is less than or equal to N; 3 is a channel value;
in the second step, the image size of the first layer of the image pyramid is set as
And the number of layers T of the image pyramid is obtained by the following formula:
2. the method of claim 1, wherein the pre-generator comprises 3 sequentially connected full convolution networks in step three;
the full convolution network comprises a convolution module with a convolution kernel of 3x3, a batch normalization BatchNorm module and a leakage correction linear unit LR activation function module which are sequentially connected;
the structure of the pre-arbiter is a Convolutional Neural Network (CNN) classifier structure.
3. The method for generating multiple images based on a single image of an antagonistic neural network as claimed in claim 2, characterized in that in the fourth and fifth step, the generator G0～GT-1And discriminator D0～DT-1Each comprising 5 sequentially connected full convolutional networks;
and the mean value of the output matrix of each discriminator is used as the two-classification output of the corresponding discriminator.
4. The method for generating multiple images based on a single image of a neural network as claimed in claim 3, wherein the specific steps of obtaining a single sample image with real fault in the first step are as follows:
the method comprises the steps of scanning a moving object to be detected line by line to obtain a plurality of high-definition linear array images, and splicing the plurality of high-definition linear array images into a complete image of the object to be detected;
and step two, intercepting the part to be detected of the object to be detected according to the parameters of the object to be detected to obtain a subgraph, and selecting an image which needs to be amplified and has a real fault from the intercepted subgraph as a sample image.
5. System for generating a plurality of images based on a single image of an antagonistic neural network, characterized in that said antagonistic neural network comprises a pre-generator, a pre-discriminator, a generator G0～GT-1And a discriminator D0～DT-1The system comprises:
the sample image acquisition module is used for acquiring a single sample image;
the image pyramid module is connected with the sample image acquisition module and used for establishing an image pyramid, the layer number of the image pyramid is T, and a fixed scaling coefficient r is arranged between two adjacent layers of images of the image pyramid;
and taking the sample image as an image of a T-th layer; and the images from the first layer to the T layer of the image pyramid are set as { Z }0,…,ZT-1}；
The pre-generation judging module is connected with the image pyramid module and used for sending the first Gaussian noise image into the pre-generator to obtain a pre-generated image and sending the pre-generated image into the pre-discriminator to be judged; the first Gaussian noise image and image Z0Equal in size;
if the pre-discriminator judges that the pre-generated image is true, the pre-generated image is used as an amplification image of the sample image;
a first generation judging module connected with the pre-generation judging module and used for leading the pre-generated image to be connected with the image Z0Fused feed to generator G0In (2), a generated image R is obtained0And sending the image R to a second generation judging module0And image Z0Are all sent into a discriminator D0Carrying out judgment;
if discriminator D0Discriminating the generated image R0If true, image R will be generated0An augmented image as a sample image;
and will generate an image R0Scaling by a scaling factor R to obtain a scaled image R0Fusing the image with the Gaussian noise image with the corresponding size to obtain a 0 th fused image;
the second generation discrimination module is connected with the first generation discrimination module and is used for receiving the (N-1) th fusion image and the N and sending the (N-1) th fusion image to the generator GNIn (2), a generated image R is obtainedNAnd will generate an image RNAnd image ZNAre all sent into a discriminator DNCarrying out judgment; wherein Z isN、GN、DNSubscript N in (1) is a positive integer;
if discriminator DNDiscriminating the generated image RNIf true, image R will be generatedNAn augmented image as a sample image;
and will generate an image RNScaling by a scaling factor R to obtain a scaled image RNFusing the image with the Gaussian noise image with the corresponding size to obtain an Nth fused image;
a calling module connected with the first generation discrimination module and the second generation discrimination module, and a discriminator D0After the discrimination is completed, let N equal to 1 and send it to the second generation discrimination module, discriminator DNAdding 1 to N and sending the N to a second generation judging module when the judgment is finished once, and obtaining all the amplified images of the sample image until N is T-1;
the size of the sample image acquired by the sample image acquisition module is MxNx3, and M is less than or equal to N; 3 is a channel value;
the image size of the first layer of the image pyramid in the image pyramid module is
And the number of layers T of the image pyramid is obtained by the following formula:
6. the system for generating a plurality of images based on a single image of an antagonistic neural network according to claim 5, wherein the pre-generation discrimination module comprises a pre-generator G and a pre-discriminator D;
the pre-generator G comprises 3 sequentially connected full convolutional networks;
the full convolution network comprises a convolution module with a convolution kernel of 3x3, a batch normalization BatchNorm module and a leakage correction linear unit LR activation function module which are sequentially connected;
the structure of the pre-arbiter D is a convolutional neural network CNN classifier structure.
7. The system of claim 6, wherein the first generation discrimination module comprises a generator G0And a discriminator D0The second generation discrimination module comprises a generator G1～GT-1And a discriminator D1～DT-1；
Generator G0～GT-1And discriminator D0～DT-1Each comprising 5 sequentially connected full convolutional networks;
and the mean value of the output matrix of each discriminator is used as the two-classification output of the corresponding discriminator.
8. The system for generating multiple images based on a single image of an antagonistic neural network of claim 7, wherein the sample image acquisition module comprises:
the image scanning and splicing module is used for scanning a moving object to be detected line by line to obtain a plurality of high-definition linear array images, and splicing the plurality of high-definition linear array images into a complete image of the object to be detected;
and the sub-image interception selection module is connected with the image scanning and splicing module and is used for intercepting the part to be detected of the object to be detected according to the parameters of the object to be detected to obtain a sub-image and selecting an image which needs to be amplified and has a real fault from the intercepted sub-image as a sample image.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011568784.XA CN112614053B (en) | 2020-12-25 | 2020-12-25 | Method and system for generating multiple images based on single image of antagonistic neural network |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011568784.XA CN112614053B (en) | 2020-12-25 | 2020-12-25 | Method and system for generating multiple images based on single image of antagonistic neural network |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112614053A CN112614053A (en) | 2021-04-06 |
| CN112614053B true CN112614053B (en) | 2021-08-24 |
Family
ID=75248620
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011568784.XA Active CN112614053B (en) | 2020-12-25 | 2020-12-25 | Method and system for generating multiple images based on single image of antagonistic neural network |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112614053B (en) |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111145116A (en) * | 2019-12-23 | 2020-05-12 | 哈尔滨工程大学 | Sea surface rainy day image sample augmentation method based on generation of countermeasure network |
Family Cites Families (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104408478B (en) * | 2014-11-14 | 2017-07-25 | 西安电子科技大学 | A kind of hyperspectral image classification method based on the sparse differentiation feature learning of layering |
| CN105741255B (en) * | 2014-12-08 | 2018-11-16 | 展讯通信（上海）有限公司 | Image interfusion method and device |
| US11598880B2 (en) * | 2018-10-04 | 2023-03-07 | The Boeing Company | Detecting fault states of an aircraft |
| CN110310344A (en) * | 2019-05-21 | 2019-10-08 | 北京大学第三医院（北京大学第三临床医学院） | A kind of image generating method and system generating confrontation network based on Virtual Conditional |
| CN111160100A (en) * | 2019-11-29 | 2020-05-15 | 南京航空航天大学 | Lightweight depth model aerial photography vehicle detection method based on sample generation |
| CN112070692B (en) * | 2020-08-26 | 2023-10-20 | 西安理工大学 | Single backlight image enhancement method based on virtual exposure |
| CN112052893A (en) * | 2020-09-01 | 2020-12-08 | 西安电子科技大学 | Semi-supervised image classification method based on generation countermeasure network |
-
2020
- 2020-12-25 CN CN202011568784.XA patent/CN112614053B/en active Active
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111145116A (en) * | 2019-12-23 | 2020-05-12 | 哈尔滨工程大学 | Sea surface rainy day image sample augmentation method based on generation of countermeasure network |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112614053A (en) | 2021-04-06 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111524135B (en) | Method and system for detecting defects of tiny hardware fittings of power transmission line based on image enhancement | |
| CN114445706A (en) | Power transmission line target detection and identification method based on feature fusion | |
| CN110378398B (en) | Deep learning network improvement method based on multi-scale feature map jump fusion | |
| CN113920107A (en) | Insulator damage detection method based on improved yolov5 algorithm | |
| CN114494829B (en) | SAR image multi-scale ship target detection method | |
| CN114782759B (en) | Method for detecting densely-occluded fish based on YOLOv5 network | |
| CN116503318A (en) | Aerial insulator multi-defect detection method, system and equipment integrating CAT-BiFPN and attention mechanism | |
| CN115187921A (en) | Power transmission channel smoke detection method based on improved YOLOv3 | |
| CN114565959A (en) | Target detection method and device based on YOLO-SD-Tiny | |
| CN116524312A (en) | Infrared small target detection method based on attention fusion characteristic pyramid network | |
| CN112699952B (en) | Train fault image amplification method and system based on deep learning | |
| CN112233088A (en) | Brake hose loss detection method based on improved Faster-rcnn | |
| CN111160100A (en) | Lightweight depth model aerial photography vehicle detection method based on sample generation | |
| CN114972332A (en) | Bamboo laminated wood crack detection method based on image super-resolution reconstruction network | |
| CN112614053B (en) | Method and system for generating multiple images based on single image of antagonistic neural network | |
| CN116403115B (en) | Large-format remote sensing image target detection method | |
| CN114549414A (en) | Abnormal change detection method and system for track data | |
| CN111881914A (en) | License plate character segmentation method and system based on self-learning threshold | |
| CN116778091A (en) | Deep learning multi-view three-dimensional reconstruction algorithm based on path aggregation | |
| CN111612803A (en) | Vehicle image semantic segmentation method based on image definition | |
| CN116883981A (en) | License plate positioning and identifying method, system, computer equipment and storage medium | |
| CN115984949A (en) | Low-quality face image recognition method and device with attention mechanism | |
| CN115578256A (en) | Unmanned aerial vehicle aerial insulator infrared video panorama splicing method and system | |
| CN110189370B (en) | Monocular image depth estimation method based on full convolution dense connection neural network | |
| CN111127327B (en) | Picture inclination detection method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |