CN112164126A - Method and device for generating composite picture, electronic equipment and storage medium - Google Patents
Method and device for generating composite picture, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN112164126A CN112164126A CN202010964974.7A CN202010964974A CN112164126A CN 112164126 A CN112164126 A CN 112164126A CN 202010964974 A CN202010964974 A CN 202010964974A CN 112164126 A CN112164126 A CN 112164126A
- Authority
- CN
- China
- Prior art keywords
- picture
- pictures
- generating
- defect
- module
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 55
- 238000003860 storage Methods 0.000 title claims abstract description 11
- 239000002131 composite material Substances 0.000 title claims description 22
- 230000007547 defect Effects 0.000 claims abstract description 44
- 230000002950 deficient Effects 0.000 claims abstract description 18
- 230000011218 segmentation Effects 0.000 claims abstract description 14
- 238000012549 training Methods 0.000 claims abstract description 13
- 238000012545 processing Methods 0.000 claims abstract description 6
- 230000015654 memory Effects 0.000 claims description 15
- 230000015572 biosynthetic process Effects 0.000 claims description 5
- 238000005457 optimization Methods 0.000 claims description 5
- 238000003786 synthesis reaction Methods 0.000 claims description 5
- 230000002194 synthesizing effect Effects 0.000 claims description 4
- 238000003709 image segmentation Methods 0.000 claims description 3
- 238000001514 detection method Methods 0.000 abstract description 7
- 238000010586 diagram Methods 0.000 description 9
- 230000000694 effects Effects 0.000 description 7
- 239000011159 matrix material Substances 0.000 description 6
- 238000005520 cutting process Methods 0.000 description 4
- 230000009286 beneficial effect Effects 0.000 description 3
- 238000004891 communication Methods 0.000 description 2
- 238000013434 data augmentation Methods 0.000 description 2
- 238000009776 industrial production Methods 0.000 description 2
- 230000000007 visual effect Effects 0.000 description 2
- 238000004458 analytical method Methods 0.000 description 1
- 230000003416 augmentation Effects 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 238000012937 correction Methods 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 238000009826 distribution Methods 0.000 description 1
- 238000003702 image correction Methods 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 230000007787 long-term memory Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012163 sequencing technique Methods 0.000 description 1
- 230000006403 short-term memory Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000012795 verification Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T11/00—2D [Two Dimensional] image generation
- G06T11/60—Editing figures and text; Combining figures or text
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/044—Recurrent networks, e.g. Hopfield networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/11—Region-based segmentation
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Biomedical Technology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Life Sciences & Earth Sciences (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Image Processing (AREA)
Abstract
The invention relates to the technical field of industrial defect detection, in particular to a method and a device for generating a synthetic picture, electronic equipment and a storage medium, wherein the generation method comprises the following steps: the method comprises the steps of carrying out segmentation processing on an input picture to obtain segmented pictures, wherein adjacent segmented pictures have partial overlapping areas; numbering the segmented pictures, wherein the numbering and the segmented pictures form a numbering picture information pair; adding random noise as generator input, and generating a countermeasure network by utilizing the segmentation picture training to finally generate a plurality of defect pictures; predicting the number corresponding to the defective picture according to the number picture information pair; and carrying out picture splicing on the corresponding defective picture according to the predicted serial number index to obtain a synthesized picture. The generation method provided by the invention can generate a vivid synthetic picture, effectively broadens the picture style, and solves the problems of limited broadening style and macroscopic generated picture boundary in the existing broadening method.
Description
Technical Field
The invention relates to the technical field of industrial defect detection, in particular to a method and a device for generating a synthetic picture, electronic equipment and a storage medium.
Background
In industrial production, each factory usually sets up a camera to monitor the quality of industrial products, but in industrial defect detection, few defect sample pictures are obtained, the defects are small, the shot pictures are large, and the data set is difficult to be enlarged by directly using the conventional deep learning method. The traditional data augmentation method adopts rotation, cutting and combination modes, the rotation and combination modes are limited, and the cut picture is only a cup of salary. Currently, a large number of realistic pictures can be generated by the generation countermeasure network, and the boundaries of the original pictures can be extended by using the generation countermeasure network to obtain high resolution pictures.
In practice, the inventors found that the above prior art has the following disadvantages:
the traditional data augmentation method has limited picture patterns and poor augmentation effect on high-resolution pictures; the generation countermeasure network generates a high resolution picture whose picture center is visible to the naked eye at the boundary of the extension boundary.
Disclosure of Invention
In order to solve the above technical problems, an object of the present invention is to provide a method, an apparatus, an electronic device, and a storage medium for generating a composite picture, wherein the technical solution is as follows:
in a first aspect, an embodiment of the present invention provides a method for generating a composite picture, where the method includes the following steps:
the method comprises the steps of carrying out segmentation processing on an input picture to obtain segmented pictures, wherein adjacent segmented pictures have partial overlapping areas;
numbering the segmented pictures, wherein the numbering and the segmented pictures form a numbering picture information pair;
adding random noise as generator input, and generating a countermeasure network by utilizing the segmentation picture training to finally generate a plurality of defect pictures;
predicting the number corresponding to the defective picture according to the number picture information pair;
and carrying out picture splicing on the corresponding defective picture according to the predicted serial number index to obtain a synthesized picture.
Further, the method for generating the confrontation network to finally generate a plurality of defect pictures by using the segmentation picture training with the addition of random noise as generator input comprises the following steps:
the generator is used for inputting the noise, the generator adopts a DAE structure of a depth automatic encoder, the automatic encoder adopts a U-shaped network structure and comprises an encoder and a decoder, and the encoder and the decoder are connected through a compressed vector;
the generation countermeasure network generates a plurality of the defect pictures.
Further, the method for indexing the corresponding defective picture according to the predicted number comprises the following steps:
and predicting the number corresponding to the defect picture by adopting a long-short term memory network (LSTM).
Further, the generation method further comprises:
and judging whether the defect picture is true or not through a discriminator, and further optimizing a generator network according to a judgment result.
In a second aspect, an embodiment of the present invention provides a device for generating a composite picture, where the device includes:
the image segmentation module is used for segmenting an input image to obtain segmented images, and the adjacent segmented images have partial overlapping areas;
the picture numbering module is used for numbering the segmented pictures, and the serial numbers and the segmented pictures form serial number picture information pairs;
the picture generation module is used for adding random noise as generator input, and generating a countermeasure network by utilizing the segmentation picture training to finally generate a plurality of defect pictures;
the number prediction module is used for predicting the number corresponding to the defective picture according to the number picture information pair; and
and the picture synthesis module is used for carrying out picture splicing on the corresponding defective picture according to the predicted serial number index to obtain a synthesized picture.
Further, the picture generation module further includes:
the input module is used for inputting the noise into a generator, the generator adopts a DAE network structure of a depth automatic encoder, the automatic encoder adopts a U-shaped structure and comprises an encoder and a decoder, and the encoder and the decoder are connected through a compressed vector; and
and the generating module is used for generating a plurality of defect pictures by using the generation countermeasure network.
Further, the picture synthesis module predicts the number corresponding to the defective picture by using a long-short term memory network (LSTM).
Further, the generation device further comprises a judgment optimization module, wherein the judgment optimization module is used for judging whether the defect picture is true through a discriminator and further optimizing the generator network according to a judgment result.
In a third aspect, an embodiment of the present invention provides an electronic device, including a memory and a processor, where: the memory is configured to store one or more computer instructions, wherein the one or more computer instructions, when executed by the processor, implement the generation method as any one of the above.
In a fourth aspect, an embodiment of the present invention provides a storage medium, where a computer-readable program is stored, and when the program is executed, the generation method according to any one of the above items is implemented.
The invention has the following beneficial effects:
in summary, the embodiment of the present invention provides a method for synthesizing a picture, which has the following advantages:
(1) the method has the advantages that the method can be used for manufacturing the spliced picture data set, the high-resolution picture is cut to obtain the segmented pictures, the adjacent segmented pictures have partial overlapping regions, and the partial overlapping regions exist between the adjacent segmented pictures, so that the method is beneficial to feature matching and realizes seamless splicing.
(2) The picture generation is realized by adopting the generation of the countermeasure network, and the game process of generating the countermeasure network ensures that the generated picture is more vivid;
(3) predicting the picture number, providing an index for the picture splicing process, and realizing quick splicing;
the generation method provided by the embodiment of the invention can generate a vivid synthetic picture, effectively broadens the picture style, solves the problems of limited broadening style and visible picture boundary in the existing broadening method, and is beneficial to the commodity quality analysis and defect detection in industrial production.
Drawings
Fig. 1 is a flowchart illustrating steps of a method for generating a composite picture according to an embodiment of the present invention;
fig. 2 is a schematic cut-out diagram of a method for generating a composite picture according to an embodiment of the present invention;
fig. 3 is a block diagram illustrating an overall flowchart of a method for generating a composite picture according to another embodiment of the present invention;
FIG. 4 is a schematic diagram of a simplified LSTM structure according to an embodiment of the present invention;
FIG. 5 is a diagram illustrating the effect of feature matching according to an embodiment of the present invention;
FIG. 6 is a diagram illustrating the effect of direct picture join according to an embodiment of the present invention;
FIG. 7 is a diagram illustrating the effect of processing a composite picture using a homography matrix according to an embodiment of the present invention;
fig. 8 is a block diagram illustrating a configuration of an apparatus for generating a composite picture according to an embodiment of the present invention;
fig. 9 is a block diagram of an electronic device according to an embodiment of the present invention.
Detailed Description
To further illustrate the technical means and effects of the present invention adopted to achieve the predetermined objects, the following detailed description of the method, apparatus, electronic device and storage medium for generating a composite image according to the present invention with reference to the accompanying drawings and preferred embodiments shows the following detailed descriptions. In the following description, different "one embodiment" or "another embodiment" refers to not necessarily the same embodiment. Furthermore, the particular features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.
Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. The terminology used herein in the description of the invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. As used herein, the term "and/or" includes any and all combinations of one or more of the associated listed items.
The following describes specific schemes of a method, an apparatus, an electronic device, and a storage medium for generating a composite picture according to the present invention in detail with reference to the accompanying drawings.
Please refer to fig. 1, which is a flowchart illustrating a method for generating a composite picture according to an embodiment of the present invention, in which an input picture is divided, a countermeasure network is generated to generate a plurality of defect pictures, a long-term and short-term memory network LSTM predicts corresponding numbers of the defect pictures, and the defect pictures are spliced by number indexes to obtain the composite picture. Specifically, the generation method comprises the following steps:
and S001, carrying out segmentation processing on the input picture to obtain segmented pictures, wherein adjacent segmented pictures have partial overlapping areas.
In order to subsequently carry out picture splicing according to feature point matching, when pictures are segmented, adjacent segmented pictures need to have an overlapping region, and a window is moved when cutting is carried out each time, so that the upper region, the lower region, the left region and the right region of the window are partially crossed.
When a high-resolution image is cut, a plurality of image blocks obtained in one row are recorded, the number of each row is equal, and finally the whole number is obtained.
Taking a 5 × 5 window as an example, please refer to fig. 2, where a 9 × 9 square grid shown in fig. 2 represents an image, the image is divided by a 5 × 5 window, and the size of the divided image is 5 × 5 at a time on a 9 × 9 high resolution image, and the shaded area is the overlapping area.
In step S002, the divided pictures are numbered, and the numbered and divided pictures constitute a numbered picture information pair.
And S003, adding random noise as generator input, and generating a confrontation network by utilizing the segmentation picture training to finally generate a plurality of defect pictures.
And step S004, predicting the number corresponding to the defect picture according to the number picture information pair.
And step S005, carrying out picture splicing on the corresponding defect picture according to the predicted serial number index to obtain a synthesized picture.
In summary, the present invention provides a method for generating a composite picture, in which an input picture is divided to obtain adjacent divided pictures with partially overlapped regions; numbering the divided pictures, wherein the numbering and the divided pictures form a numbering picture information pair; adding random noise as generator input, generating a countermeasure network by utilizing the training of the segmentation pictures, and finally generating a plurality of defect pictures; predicting the number corresponding to the defect picture according to the number picture information pair; and indexing the corresponding defective pictures according to the predicted numbers to carry out picture splicing to obtain a synthesized picture. The invention adopts a cutting method that adjacent divided pictures have overlapping areas to divide the pictures, numbers the corresponding divided pictures, and finally splices the defective pictures according to the predicted numbers to realize the generation of the synthesized pictures.
Preferably, referring to fig. 3, step S003 is implemented by generating a countermeasure network, adding random noise as input of the generator, and generating the countermeasure network by using the segmented picture training to finally generate a plurality of defect pictures. The generator adopts a DAE network structure of a depth automatic encoder, the automatic encoder adopts a U-shaped structure and comprises an encoder and a decoder, the encoder and the decoder are connected through a compressed vector, the encoder compresses input into a vector x, the decoder decompresses the vector x, and the vector x is a pivot connected between the encoder and the decoder. The input of the encoder and the output of the decoder are set to have the same dimensionality, the pictures are spliced to obtain corresponding high-resolution pictures, zero-sum game is carried out with a discriminator network, and the value of a generator is minimized in the countermeasure process to achieve the optimum. The specific network structure is as follows:
an encoder: input → conv () -batchnorm () -relu () -conv () -sigmoid () → x
A decoder: x → deconv () -batcnorm-relu () -deconv () -sigmoid () → output.
Wherein, the sizes of the convolution kernels in conv () and deconv () are both 3 × 3, and the step size is 2.
Preferably, referring to fig. 3 again, step S004 uses the long-short term memory network LSTM to predict the corresponding number of the defect picture. After the pictures are divided, the divided pictures are numbered in sequence to form numbered picture information pairs. The role of the LSTM network is: the numbers are used as training data to predict the numbers of the defect pictures generated by a series of generators. Referring to fig. 4, since the deviation of the memory information becomes more and more serious as the network grows deeper, in order to avoid this situation, the embodiment of the present invention sets the hidden layer of the long-short term memory network LSTM to 3 layers, as shown in fig. 4.
Preferably, referring to fig. 3 again, the above-mentioned generating method further includes determining whether the defect picture is true by the discriminator, where the discriminator is substantially a binary network, and the closer the value is to 1, the better the effect is, the greater the similarity between the picture generated by the generator and the true picture is, in fig. 3, R indicates that the discriminator discriminates that the generated picture is false, and a indicates that the image is true. The discriminators and generators pass through the zero-sum game, maximizing the value of the discriminators. The specific network structure is as follows: input → conv () -lreluu () -conv () -batchnorm () -lrelur () -conv () -sigmoid () → output. Where the convolution kernel size from the first conv () is 3 × 3 with a step size of 1, the remaining conv () convolution kernel size is 3 × 3 with a step size of 2, and the lrelu () parameter is 0.2.
Preferably, the method for performing picture splicing according to the predicted number index corresponding defect picture in step S005 is to search for feature points by using a Scale Invariant Feature Transform (SIFT) algorithm, then perform feature point matching, finally perform splicing, and splice by row and then column. The method comprises the following specific steps:
the SIFT feature detection comprises three steps of scale space extreme value detection, key point positioning and direction determination. The extreme value detection in the scale space is to search the image positions in all scales to find potential interest points which are invariable in scale and rotation. The key point positioning is that the position of the key point is determined according to the stability degree of the key point on each candidate interest point position. The determined direction is the direction of the key point after the positioning is confirmed, and for 8 directions of the key point: up, down, left, right, left-up, left-down, right-up, and right-down, the direction with the largest vector value among the 8 vector values is selected as the direction of the keypoint. After the positions and the directions of the key points are determined, the key points are used as feature points, and then feature point matching is performed, and fig. 5 shows the feature matching result. Since there is a partially overlapping region in the neighboring region between adjacent divided pictures, the RANSAC (Random Sample Consensus) process is performed after feature point matching is performed on the overlapping region.
According to the principle of photographic imaging, due to the problem of shooting angle, a complete picture is shot into two pictures with repeated areas, and the process of cutting the pictures into parts with overlapped areas is simulated. The homography matrix can be used for converting the visual angles, and images shot at different angles are converted to the same visual angle for image correction so as to realize image splicing. The picture correction formula is as follows: x1 = Hx2, where H is a homography, x1 denotes a first matrix of image proxels, x2 denotes a second matrix of image proxels, and H transforms x1 and x2 to the same viewing angle, enabling image stitching.
And indexing according to the numbers predicted in the step S004, sequencing the numbers obtained by prediction according to the numerical value, and splicing the defect pictures corresponding to the adjacent numerical values. According to the number of the low-resolution pictures of each line specified in the step S001, all the lines are divided to be spliced according to the lines, then the results of line splicing are utilized to carry out column splicing, and finally, the complete high-resolution synthetic picture is obtained through combination.
As a preferred solution of the embodiment of the present invention, since the high resolution picture obtained at this time is somewhat blurred, the composite picture is deblurred by using poisson distribution, and a clear composite picture is finally obtained.
As a preferred embodiment of the present invention, the generating method further includes a verification step, specifically, when the picture splicing is not performed yet, the generated picture is compared with the baseline, and the generated picture can be falsified. In addition, the comparison is carried out after the Maximum Mean Difference (MMD) is calculated, wherein the generated picture has a smaller MMD value and a best effect compared with baseline. After the pictures are spliced, a questionnaire survey mode is adopted, so that the respondents can select between the spliced pictures processed by the homography matrix and the original pictures, more than 60% of people feel tangled and can not select the spliced pictures, and the obtained high-resolution pictures are good. The splicing result after the homography matrix conversion is used is shown in fig. 7, and compared with the direct picture connection in fig. 6, the seamless splicing of the pictures is realized by removing the overlapping area.
Referring to fig. 8, a block diagram of a generating apparatus for synthesizing pictures according to an embodiment of the present invention is shown, where the generating apparatus includes a picture dividing module 801, a picture numbering module 802, a picture generating module 803, a numbering prediction module 804, and a picture synthesizing module 805. Specifically, the image segmentation module 801 is configured to segment an input image to obtain segmented images, where adjacent segmented images have a partial overlapping area; the picture numbering module 802 is configured to number the divided pictures, and the number and the divided pictures form a numbered picture information pair; the picture generation module 803 is configured to add random noise as an input of the generator, generate a countermeasure network by using the segmented picture training, and finally generate a plurality of defect pictures; the number prediction module 804 is used for predicting a number corresponding to the defect picture according to the number picture information pair; the picture synthesis module 805 is configured to perform picture splicing on the defect pictures corresponding to the predicted serial number indexes to obtain a synthesized picture.
Preferably, the picture generating module 803 further includes an input module and a generating module, the input module is configured to input the segmented picture and the noise into the generator, the generator adopts a DAE network structure of a depth automatic encoder, the automatic encoder adopts a U-shaped structure, and includes an encoder and a decoder, and the encoder and the decoder are connected by a compressed vector; the generation module is used for generating a plurality of defect pictures by utilizing the generation countermeasure network.
Preferably, the picture synthesis module 805 predicts the number corresponding to the defect picture by using the long-short term memory network LSTM.
Preferably, the generating device further comprises a judgment optimization module, and the judgment optimization module is used for judging whether the defect picture is true through the discriminator and further optimizing the generator network according to the judgment result.
Referring to fig. 9, which illustrates an electronic device provided by an embodiment of the present invention, including a memory 901 and a processor 902, it is understood by those skilled in the art that the structure of the terminal device illustrated in fig. 9 does not constitute a limitation of the terminal device, and may include more or less components than those illustrated, or may combine some components, or may be arranged in different components. Wherein: the memory 901 is used to store one or more computer instructions, wherein the one or more computer instructions are executed by the processor 902 to implement: carrying out segmentation processing on an input picture to obtain a partial overlapping area of adjacent segmented pictures; numbering the segmented pictures, wherein the numbering and the segmented pictures form a numbering picture information pair; the random noise is added as the input of a generator, and a countermeasure network is generated by utilizing the segmentation picture training to finally generate a plurality of defect pictures; predicting the number corresponding to the defective picture according to the number picture information pair; and carrying out picture splicing on the corresponding defective picture according to the predicted serial number index to obtain a synthesized picture.
In other embodiments, the electronic device further comprises a communication interface 903 coupled to the memory 901 and the processor 902 via a bus or other means for enabling the subject to communicate with other devices or communication networks.
An embodiment of the present invention provides a storage medium, where a computer-readable program is stored, and when the program is executed, the method for generating a composite picture provided in any one of the above embodiments is performed.
Finally, it should be noted that: the above examples are only intended to illustrate the technical solution of the present invention, but not to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.
Claims (10)
1. A method for generating a composite picture, the method comprising the steps of:
the method comprises the steps of carrying out segmentation processing on an input picture to obtain segmented pictures, wherein adjacent segmented pictures have partial overlapping areas;
numbering the segmented pictures, wherein the numbering and the segmented pictures form a numbering picture information pair;
adding random noise as generator input, and generating a countermeasure network by utilizing the segmentation picture training to finally generate a plurality of defect pictures;
predicting the number corresponding to the defective picture according to the number picture information pair;
and carrying out picture splicing on the corresponding defective picture according to the predicted serial number index to obtain a synthesized picture.
2. The method for generating a composite picture according to claim 1, wherein the method for generating a confrontation network to finally generate a plurality of defect pictures by using the segmented picture training with random noise added as a generator input comprises the following steps:
inputting the noise into a generator, wherein the generator adopts a DAE network structure of a depth automatic encoder, the automatic encoder adopts a U-shaped structure and comprises an encoder and a decoder, and the encoder and the decoder are connected through a compressed vector;
the generation countermeasure network generates a plurality of the defect pictures.
3. A method as claimed in claim 1 or 2, wherein the method for indexing the corresponding defective picture according to the predicted number comprises the following steps:
and predicting the number corresponding to the defect picture by adopting a long-short term memory network (LSTM).
4. The method according to claim 2, further comprising:
and judging whether the defect picture is true or not through a discriminator, and further optimizing a generator network according to a judgment result.
5. A composite picture generation apparatus, comprising:
the image segmentation module is used for segmenting an input image to obtain segmented images, and the adjacent segmented images have partial overlapping areas;
the picture numbering module is used for numbering the segmented pictures, and the serial numbers and the segmented pictures form serial number picture information pairs;
the picture generation module is used for adding random noise as generator input, and generating a countermeasure network by utilizing the segmentation picture training to finally generate a plurality of defect pictures;
the number prediction module is used for predicting the number corresponding to the defective picture according to the number picture information pair; and
and the picture synthesis module is used for carrying out picture splicing on the corresponding defective picture according to the predicted serial number index to obtain a synthesized picture.
6. The apparatus for generating a composite picture according to claim 5, wherein the picture generation module further comprises:
the input module is used for inputting the noise into a generator, the generator adopts a DAE network structure of a depth automatic encoder, the automatic encoder adopts a U-shaped structure and comprises an encoder and a decoder, and the encoder and the decoder are connected through a compressed vector; and
and the generating module is used for generating a plurality of defect pictures by using the generation countermeasure network.
7. The apparatus as claimed in claim 5 or 6, wherein the picture synthesizing module uses a long-short term memory network (LSTM) to predict the number corresponding to the defective picture.
8. The apparatus of claim 6, further comprising a judgment optimization module, for judging whether the defect picture is true by the discriminator, and further optimizing the generator network according to the judgment result.
9. An electronic device comprising a memory and a processor, wherein:
the memory is configured to store one or more computer instructions, wherein the one or more computer instructions, when executed by the processor, implement the generation method of any one of claims 1 to 4.
10. A storage medium, characterized in that the storage medium stores a computer-readable program which, when executed, implements the generation method according to any one of claims 1 to 4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010964974.7A CN112164126A (en) | 2020-09-15 | 2020-09-15 | Method and device for generating composite picture, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010964974.7A CN112164126A (en) | 2020-09-15 | 2020-09-15 | Method and device for generating composite picture, electronic equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112164126A true CN112164126A (en) | 2021-01-01 |
Family
ID=73859123
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010964974.7A Pending CN112164126A (en) | 2020-09-15 | 2020-09-15 | Method and device for generating composite picture, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112164126A (en) |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR20190084146A (en) * | 2017-12-18 | 2019-07-16 | 서울대학교산학협력단 | Image decoding apparatus based on airborn and method of decoding image using the same |
| CN111524142A (en) * | 2020-03-10 | 2020-08-11 | 浙江工业大学 | Automatic segmentation method for cerebrovascular image |
| CN111652233A (en) * | 2020-06-03 | 2020-09-11 | 哈尔滨工业大学（威海） | Text verification code automatic identification method for complex background |
-
2020
- 2020-09-15 CN CN202010964974.7A patent/CN112164126A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR20190084146A (en) * | 2017-12-18 | 2019-07-16 | 서울대학교산학협력단 | Image decoding apparatus based on airborn and method of decoding image using the same |
| CN111524142A (en) * | 2020-03-10 | 2020-08-11 | 浙江工业大学 | Automatic segmentation method for cerebrovascular image |
| CN111652233A (en) * | 2020-06-03 | 2020-09-11 | 哈尔滨工业大学（威海） | Text verification code automatic identification method for complex background |
Non-Patent Citations (4)
| Title |
|---|
| SIDDHARTH PANDEY 等: "An image augmentation approach using two-stage generative adversarial network for nuclei image segmentation", 《BIOMEDICAL SIGNAL PROCESSING AND CONTROL》 * |
| 杨晓 等: "应用于压力容器红外无损检测的SIFT图像拼接方法", 《电子测量与仪器学报》 * |
| 陈文兵 等: "基于条件生成式对抗网络的数据增强方法", 《计算机应用》 * |
| 黄文锋 等: "基于计算机视觉的飞机燃油非接触式测量***", 《图学学报》 * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108229277B (en) | Gesture recognition method, gesture control method, multilayer neural network training method, device and electronic equipment | |
| CN110136066B (en) | Video-oriented super-resolution method, device, equipment and storage medium | |
| Wu et al. | Revisiting light field rendering with deep anti-aliasing neural network | |
| CN110246163B (en) | Image processing method, image processing device, image processing apparatus, and computer storage medium | |
| US8665341B2 (en) | Methods and apparatus for rendering output images with simulated artistic effects from focused plenoptic camera data | |
| CN111583097A (en) | Image processing method, image processing device, electronic equipment and computer readable storage medium | |
| US9076221B2 (en) | Removing an object from an image | |
| CN110381268B (en) | Method, device, storage medium and electronic equipment for generating video | |
| CN112184585B (en) | Image completion method and system based on semantic edge fusion | |
| US10726612B2 (en) | Method and apparatus for reconstructing three-dimensional model of object | |
| CN110782412B (en) | Image processing method and device, processor, electronic device and storage medium | |
| CN108242063B (en) | Light field image depth estimation method based on GPU acceleration | |
| Chauhan et al. | Deep learning-based single-image super-resolution: A comprehensive review | |
| CN112243518A (en) | Method and device for acquiring depth map and computer storage medium | |
| Rodriguez-Pardo et al. | Seamlessgan: Self-supervised synthesis of tileable texture maps | |
| CN111260655A (en) | Image generation method and device based on deep neural network model | |
| CN113658091A (en) | Image evaluation method, storage medium and terminal equipment | |
| CN106845555A (en) | Image matching method and image matching apparatus based on Bayer format | |
| Akhyar et al. | A beneficial dual transformation approach for deep learning networks used in steel surface defect detection | |
| CN113298931A (en) | Reconstruction method and device of object model, terminal equipment and storage medium | |
| EP2966613A1 (en) | Method and apparatus for generating a super-resolved image from an input image | |
| Ullah et al. | Perceptual quality assessment of panoramic stitched contents for immersive applications: a prospective survey | |
| CN112164126A (en) | Method and device for generating composite picture, electronic equipment and storage medium | |
| CN115619678A (en) | Image deformation correction method and device, computer equipment and storage medium | |
| CN115423697A (en) | Image restoration method, terminal and computer storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |