CN113505772A - License plate image generation method and system based on generation countermeasure network - Google Patents
License plate image generation method and system based on generation countermeasure network Download PDFInfo
- Publication number
- CN113505772A CN113505772A CN202110700097.7A CN202110700097A CN113505772A CN 113505772 A CN113505772 A CN 113505772A CN 202110700097 A CN202110700097 A CN 202110700097A CN 113505772 A CN113505772 A CN 113505772A
- Authority
- CN
- China
- Prior art keywords
- image
- license plate
- text
- training
- background
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 70
- 238000012549 training Methods 0.000 claims abstract description 196
- 238000012360 testing method Methods 0.000 claims abstract description 33
- 230000006870 function Effects 0.000 claims description 48
- 238000006243 chemical reaction Methods 0.000 claims description 29
- 230000004927 fusion Effects 0.000 claims description 26
- 238000004590 computer program Methods 0.000 claims description 10
- 230000008439 repair process Effects 0.000 claims description 9
- 238000003860 storage Methods 0.000 claims description 8
- 230000002194 synthesizing effect Effects 0.000 claims description 8
- 238000012545 processing Methods 0.000 claims description 7
- 238000000605 extraction Methods 0.000 claims description 6
- 238000004422 calculation algorithm Methods 0.000 claims description 5
- 238000004364 calculation method Methods 0.000 claims description 4
- 239000010410 layer Substances 0.000 description 59
- 238000010586 diagram Methods 0.000 description 15
- 230000008569 process Effects 0.000 description 5
- 238000012795 verification Methods 0.000 description 5
- 238000004891 communication Methods 0.000 description 4
- 230000004913 activation Effects 0.000 description 3
- 238000013459 approach Methods 0.000 description 3
- 230000015572 biosynthetic process Effects 0.000 description 3
- 238000013434 data augmentation Methods 0.000 description 3
- 238000001514 detection method Methods 0.000 description 3
- 230000000694 effects Effects 0.000 description 3
- 238000003786 synthesis reaction Methods 0.000 description 3
- 238000013528 artificial neural network Methods 0.000 description 2
- 230000003190 augmentative effect Effects 0.000 description 2
- 239000003086 colorant Substances 0.000 description 2
- 238000013135 deep learning Methods 0.000 description 2
- 238000012544 monitoring process Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 238000005070 sampling Methods 0.000 description 2
- 238000012216 screening Methods 0.000 description 2
- 230000009466 transformation Effects 0.000 description 2
- ORILYTVJVMAKLC-UHFFFAOYSA-N Adamantane Natural products C1C(C2)CC3CC1CC2C3 ORILYTVJVMAKLC-UHFFFAOYSA-N 0.000 description 1
- 230000008602 contraction Effects 0.000 description 1
- 230000007797 corrosion Effects 0.000 description 1
- 238000005260 corrosion Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000012217 deletion Methods 0.000 description 1
- 230000037430 deletion Effects 0.000 description 1
- 238000009795 derivation Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000005286 illumination Methods 0.000 description 1
- 230000002452 interceptive effect Effects 0.000 description 1
- 238000002372 labelling Methods 0.000 description 1
- 230000014759 maintenance of location Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000010606 normalization Methods 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 238000003672 processing method Methods 0.000 description 1
- 230000000717 retained effect Effects 0.000 description 1
- 230000011218 segmentation Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 239000002344 surface layer Substances 0.000 description 1
- 238000013519 translation Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/22—Matching criteria, e.g. proximity measures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/25—Fusion techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- General Engineering & Computer Science (AREA)
- Life Sciences & Earth Sciences (AREA)
- General Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Bioinformatics & Computational Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Evolutionary Biology (AREA)
- General Health & Medical Sciences (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Molecular Biology (AREA)
- Computational Linguistics (AREA)
- Biophysics (AREA)
- Biomedical Technology (AREA)
- Health & Medical Sciences (AREA)
- Image Processing (AREA)
Abstract
The invention provides a license plate image generation method and a license plate image generation system based on a generation countermeasure network, wherein the method comprises the following steps: acquiring a target text image and a test license plate original image; inputting the original test license plate image and the target text image into a text replacement model to obtain a license plate prediction image output by the text replacement model; the text replacement model is obtained by training a pseudo license plate image formed by a background training image obtained by erasing a text based on the training license plate original image and the training license plate original image; the text replacement model is used for replacing the text content of the pre-acquired target text image with the original image of the tested license plate to obtain a license plate prediction image. The invention realizes the generation of the license plate prediction image and ensures that the characteristics of the license plate background, the character style and the like of the original license plate image to be tested are reserved.
Description
Technical Field
The invention relates to the technical field of computer vision, in particular to a license plate image generation method and a license plate image generation system based on a generation countermeasure network.
Background
With the increasing number of motor vehicles in China, various traffic management problems emerge continuously. In order to meet the application requirements of automatic management of machines such as traffic road monitoring, automatic management of parking lots, vehicle tracking, automatic toll management of highways and the like, automatic detection and identification of license plates are required. Under the promotion of the wave of deep learning, a license plate detection and recognition system based on deep learning becomes a hot spot problem, however, training of a deep neural network needs a large amount of data, the labor cost of the acquisition and labeling work is high, the quality of a large amount of data is poor, the data quantity of the training data is easy to be insufficient or the image quality is poor and does not accord with the application standard, and the like, and the system is a great limiting factor of the license plate detection and recognition system.
Because the license plate is a real scene image with characters and has a fixed format, the following method is adopted to realize the text replacement of the license plate at present so as to expand the training data volume of the deep neural network, and the method comprises the following steps: a network framework approach to image-to-image conversion, a Full Convolution Regression Network (FCRN) approach, and a style retention network (SRNet) approach. The network framework method for converting the image into the image can color the edge image, generate a corresponding real scene image from the segmentation image, and if the mask of the license plate is given as input, the network can output the license plate image with the same text content as that on the mask of the license plate; the FCNR method synthesizes data in two styles but with different texts, and the main idea of the technology is to randomly select fonts, colors and deformation parameters to generate a style text and then render the style text on a background image; the SRNet method has good performance in the aspect of scene text editing, and text content can be changed while original image background textures are kept.
However, the image quality of the license plate image generated by the network framework method of image-to-image conversion is poor, and the method needs matched data as a training set, however, many images in the real world do not have corresponding matched images; the text image synthesized by the FCNR method has a large difference with the image of the real scene; the SRNet method can only be used for training using data synthesized from a natural scene image data set (SynthText), and is mostly used for training in english, and in addition, it is inferior in the aspect of font style and the like in original images, and has poor generalization ability in license plate image generation tasks in countries other than english.
Disclosure of Invention
The invention provides a license plate image generation method and system based on a generation countermeasure network, which are used for overcoming the defects that the quality of a generated license plate image is poor and a target text is uncontrollable in the prior art and realizing the generation of the license plate image with clear image quality and vivid style.
The invention provides a license plate image generation method based on a generation countermeasure network, which comprises the following steps: acquiring a target text image and a test license plate original image; inputting the original test license plate image and the target text image into a text replacement model to obtain a license plate prediction image output by the text replacement model; the text replacement model is obtained by training a pseudo license plate image formed by a background training image obtained by erasing a text based on the training license plate original image and the training license plate original image; the text replacement model is used for replacing the text content of the pre-acquired target text image with the original image of the tested license plate to obtain a license plate prediction image.
According to the license plate image generation method based on the generation countermeasure network provided by the invention, the text replacement model comprises the following steps: the text conversion layer is used for converting the foreground style of the target text image into the foreground style of the original image of the tested license plate based on the original image of the tested license plate to obtain a target text foreground image; the background repairing layer is used for carrying out text erasing and texture filling on the original image of the tested license plate to form a license plate background image; and the text fusion layer performs text fusion on the target text foreground image and the license plate background image to obtain a license plate prediction image.
According to the license plate image generation method based on the generation countermeasure network provided by the invention, the training of the text replacement model comprises the following steps: erasing a text of an original image of the training license plate to obtain a background image; synthesizing based on a foreground text image obtained in advance and the background image to obtain a pseudo license plate image; inputting the target text image, a pre-acquired license plate background image, a pre-acquired text skeleton image, the foreground text image and the pseudo license plate image into the text replacement model to obtain a license plate training image output by the text replacement model; and constructing a loss function based on the foreground image, the background image, the license plate training image, the text skeleton image, the target text image and the pseudo license plate image, and converging based on the loss function to finish training.
According to the license plate image generation method based on the generation countermeasure network provided by the invention, the target text image, the pre-acquired license plate background image, the pre-acquired text skeleton image, the foreground text image and the pseudo license plate image are input into the text replacement model to obtain the license plate training image output by the text replacement model, and the license plate training image generation method comprises the following steps: inputting the target text image, the pre-acquired license plate background image, the pre-acquired text skeleton image, the foreground text image and the pseudo license plate image into the text conversion layer to obtain a foreground training image output by the text conversion layer; inputting the original image of the training license plate to the background repairing layer to obtain a background training image output by the background repairing layer; and inputting the foreground training image and the background training image into the text fusion layer to obtain a license plate training image output by the text fusion layer.
According to the license plate image generation method based on the generation countermeasure network provided by the invention, the loss function is constructed based on the foreground text image, the background image, the license plate training image, the text skeleton image, the target text image and the pseudo license plate image, and the method comprises the following steps: constructing a text conversion loss function based on the fake license plate image, the target text image, the text skeleton image, the foreground text image and the foreground training image; constructing a background repair loss function based on the fake license plate, the background image and the background training image; constructing a text fusion loss function based on the fake license plate image and the license plate training image; and obtaining a loss function based on the text conversion loss function, the background repair loss function and the text fusion loss function.
According to the license plate image generation method based on the generation countermeasure network provided by the invention, the text erasing is carried out on the original image of the training license plate to obtain the background image, and the method comprises the following steps: performing convolution operation on the training license plate original image and the corresponding pre-acquired background original image to obtain a feature map; performing deconvolution operation based on the feature map to generate a background map; and performing layer-by-layer transverse connection based on the convolved intermediate layer characteristics so as to enable the generated background image to keep the texture characteristics of the original image of the training license plate.
According to the license plate image generation method based on the generation countermeasure network provided by the invention, before the synthesis of the standard font image based on the pre-manufacture, the pre-acquired foreground text image and the background image to obtain the pseudo license plate image, the method comprises the following steps: making a standard font graph with the same text content of the original image of the training license plate; performing difference calculation based on the original image of the training license plate and the background image to obtain a difference image; extracting the difference image by using a skeleton extraction algorithm to obtain the text skeleton image; binarizing the difference image to obtain a text mask image with a black background and a white text; and performing intersection image processing by using the text mask image and the original image of the training license plate to obtain the foreground text image.
The invention also provides a license plate image generation system based on the generation countermeasure network, which comprises the following steps: the acquisition module acquires a target text image and an original image of the tested license plate; the text replacement module is used for obtaining a license plate prediction image output by the text replacement module based on the test license plate original image and the target text image; the text replacement module is used for performing text erasing on the training license plate original image to obtain a background training image and a pseudo license plate image formed by the background training image and training the training license plate original image; the text replacement module is used for replacing the text content of the pre-acquired target text image with the original image of the tested license plate to obtain a license plate prediction image.
The invention also provides an electronic device, which comprises a memory, a processor and a computer program stored on the memory and capable of running on the processor, wherein the processor executes the program to realize the steps of the license plate image generation method based on the generation countermeasure network.
The present invention also provides a non-transitory computer readable storage medium having stored thereon a computer program which, when being executed by a processor, implements the steps of the license plate image generation method based on generation of a countermeasure network as described in any of the above.
According to the license plate image generation method and system based on the generation countermeasure network, the text replacement model is used for carrying out text replacement on the original image of the tested license plate based on the pre-acquired target text image so as to obtain the text content with the target text image and the license plate test image of the original image background of the tested license plate, so that a new license plate is generated, the generated new license plate is ensured to keep the characteristics of the original license plate background, the character style and the like, and the license plate serial number of the original image of the license plate is replaced so as to effectively expand the data set.
Drawings
In order to more clearly illustrate the technical solutions of the present invention or the prior art, the drawings needed for the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and those skilled in the art can also obtain other drawings according to the drawings without creative efforts.
FIG. 1 is a schematic flow chart of a license plate image generation method based on a generation countermeasure network provided by the invention;
FIG. 2 is a schematic diagram of a training process of a text replacement model provided by the present invention;
FIG. 3 is an example of a license plate image generated by the method for generating a license plate image based on a countermeasure network according to the present invention;
FIG. 4 is an example of a training set of text replacement models provided by the present invention;
FIG. 5 is a schematic input and output diagram of a license plate image generation method based on a generation countermeasure network according to the present invention;
FIG. 6 is a schematic diagram of a license plate image generation result based on a generation countermeasure network provided by the present invention;
FIG. 7 is a schematic structural diagram of a license plate image generation system based on a generation countermeasure network provided by the invention;
fig. 8 is a schematic structural diagram of an electronic device provided in the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings, and it is obvious that the described embodiments are some, but not all embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Fig. 1 shows a flow diagram of a license plate image generation method based on a generation countermeasure network of the present invention, and the method includes:
s01, acquiring a target text image and a test license plate original image;
s02, inputting the original test license plate image and the target text image into a text replacement model to obtain a license plate prediction image output by the text replacement model; the text replacement model is obtained by training a pseudo license plate image formed by a background training image obtained by erasing a text based on a training license plate original image and the training license plate original image; the text replacement model is used for replacing the text content of the pre-acquired target text image with the original image of the tested license plate to obtain a license plate prediction image.
It should be noted that S0N in this specification does not represent the order of the license plate image generation method based on the generation countermeasure network, and the license plate image generation method based on the generation countermeasure network of the present invention is specifically described below with reference to fig. 2 to 6.
And step S01, acquiring a target text image and a test license plate original image.
In this embodiment, the target text image is formed as required, and the target text image is produced by using fixed attributes such as fonts, colors, backgrounds and the like on the premise of giving the target text; the text content of the target text image is different from the text content and style in the original test license plate image, so that other style characteristics of the target text image except the text content can be conveniently rendered into the text style, such as the style characteristics of character shape, color and the like, in the original test license plate image. In addition, the original image of the test license plate can be obtained based on equipment with a camera shooting function, such as a camera, a mobile terminal or a vehicle data recorder.
Step S02, inputting the original test license plate image and the target text image into a text replacement model to obtain a license plate prediction image output by the text replacement model; the text replacement model is obtained by training a pseudo license plate image formed by a background training image obtained by erasing a text based on a training license plate original image and the training license plate original image; the text replacement model is used for replacing a pre-acquired target text image with a test license plate original image to obtain a license plate prediction image.
In this embodiment, the text replacement model includes: the text conversion layer is used for converting the foreground style of the target text image into the foreground style of the original image of the tested license plate based on the original image of the tested license plate to obtain a target text foreground image; the background repairing layer is used for carrying out text erasing and texture filling on the original image of the tested license plate to form a license plate background image; and the text fusion layer performs text fusion on the target text foreground image and the license plate background image to obtain a license plate prediction image. Through the three modules, the background with the original license plate image is generated, the license plate number is replaced by the specified target text, and the font and the background style of the target text are kept good.
The text erasing is performed on the original test license plate image through the background repairing layer so as to prevent the target text foreground image from interfering with the generated license plate background image, and then the texture filling is performed on the original test license plate image subjected to the text erasing so as to enable the generated license plate background image to be more vivid.
In the embodiment, the text conversion layer comprises three downsampling convolution layers and four residual error structures, the downsampling convolution layers and the residual error structures are fused through channel-level connection, and a target text foreground image with the text style of the original license plate image to be tested is obtained through three layers of upsampling, a convolution and polynomial product, batch normalization and a linear rectification function (conv-BN-ReLU) block; an Encoder (Encoder) in the background repairing layer is three downsampling convolutional layers and four residual modules, and a Decoder (Decoder) is three upsampling convolutional layers to generate a license plate background image with the same size as the original image of the tested license plate; the text fusion layer comprises an FCN structure of an Encoder-Decoder, however, in the up-sampling stage of the Decoder, the decoding feature diagram of the background restoration layer needs to be connected with the corresponding feature diagram with the same resolution, so that the restoration of the background details is facilitated, and the license plate prediction diagram synthesized by the target text foreground diagram and the license plate background diagram is more realistic.
In an optional embodiment, the license plate image generation method based on the generation countermeasure network may further include the steps of:
referring to fig. 2, before inputting the original image of the tested license plate and the pre-acquired target text image into the text replacement model, training the text replacement model specifically includes:
s11, erasing the text of the original image of the training license plate to obtain a background image;
s12, synthesizing based on the foreground text image and the background image which are obtained in advance to obtain a pseudo license plate image;
s13, inputting the target text image, the pre-acquired license plate background image, the pre-acquired text skeleton image, the foreground text image and the pseudo license plate image into a text replacement model to obtain a license plate training image output by the text replacement model;
and S14, constructing a loss function based on the foreground image, the background image, the license plate training image, the text skeleton image, the target text image and the pseudo license plate image, and converging based on the loss function to finish training.
Firstly, erasing a text of an original image of a training license plate to obtain a background image.
In this embodiment, the text content of the original image of the training license plate may be erased by using the generation confrontation network model. Generating a countermeasure network (GAN) includes image translation, image text deletion, image text editing, and the like. Editing text in a real image is replacing the text in the source image with another text while preserving its style attributes such as texture, font, brightness, etc. Because the license plate is a real scene image with characters and has a fixed format, the method for generating the license plate image data by using the generation countermeasure network is an effective method for expanding the data volume of the training set and improving the image quality.
Specifically, a countermeasure network model is generated, including: and the generator is used for erasing the text based on the original image of the training license plate to obtain a background image. In this embodiment, training with a generator to obtain a background image, and erasing a text based on a training license plate original image to obtain the background image includes: the method comprises the following steps: performing convolution operation on the basis of the training license plate original image and the corresponding pre-acquired background original image to obtain a feature map; performing deconvolution operation based on the feature map to generate a background map; and performing layer-by-layer transverse connection on the intermediate layer characteristics based on convolution so as to enable the generated background image to keep the texture characteristics of the original image of the training license plate.
It should be noted that the background original image can be obtained in advance based on manual erasing; convolution may employ a lightweight residual network (Resnet 8), based on the last convolutional layer of Resnet18, applied to the predictive text or non-text score map by the 1 × 1 convolutional layer converted by the last fully-connected layer; the deconvolution module consists of five deconvolution layers, the kernel size of each layer is set to 4, the stride step size is set to 2, and the fill size is set to 1.
In addition, because the features of the lower level contain stronger semantics, the features of the higher level contain weaker semantics, and the features of the higher level also contain detailed information such as pixel-level color, texture, and position information of an object, the features of the lower level and the higher level are integrated when the features are fused by transversely connecting the features through the transverse connecting module, so that the texture features are well retained, and the texture features are represented as surface layer information of the license plate image. The transverse connection module comprises a transform block and an upsampling block, wherein: the transformation block takes the middle layer of the convolution module as input, reduces the element size of input features by adopting the 1 × 1 convolution of a contraction layer, stacks two 3 × 3 convolution layers with the same size to execute nonlinear transformation so as to replace large kernel convolution to realize a large receiving field, improves the calculation efficiency, and amplifies a feature map channel by utilizing an expansion layer in a 1 × 1 convolution mode; an up-sampling block, which amplifies the characteristic diagram output by the convolution module by using a deconvolution module; the feature map of the upsampled block is element-wise summed with the corresponding feature map in the transformed block.
In an optional embodiment, the license plate image generation method based on the generation countermeasure network may further include the steps of:
before text erasure is carried out on the original image of the training license plate by utilizing the generated countermeasure network model, the generation countermeasure network model is required to be trained, and the training generates the countermeasure network model, which comprises the following steps: training the generator based on the original image of the training license plate and the corresponding original image of the background, and outputting a background training prediction image; inputting the background training prediction image and the background original image into a discriminator to judge the authenticity of the background image based on the background original image; and performing alternate iterative training based on the generator and the discriminator, and judging whether to stop training based on the background training prediction graph and the discrimination result.
Firstly, a training set for generating an confrontation network model is constructed based on training license plate original images and corresponding background original images, and the method specifically comprises the following steps: acquiring a real license plate as an original image of a training license plate; and manually erasing the original image of the training license plate by utilizing manpower in advance to obtain the corresponding original image of the license plate. It should be noted that the number of training license plate artwork which is manually erased in advance can be selected to be a smaller number, for example, 100, and the specific number can be determined according to the actual design requirement. The real license plate can be selected from the license plate which takes the first character as a Chinese character, the subsequent character string consists of 24 English letters and 10 Arabic numerals, and the license plate has various license plate styles due to different illumination conditions, shooting angles, distance degrees and other factors, and has specificity, and refer to fig. 3. The acquisition of the real license plate comprises the following steps: acquiring road data in provinces and cities by using a vehicle data recorder in a vehicle; and/or shooting vehicle data in multi-province city by using a camera. It should be noted that the camera may employ a mobile device, such as a cell phone, a video camera, etc.
Training the countermeasure network model, and generating the countermeasure network model as follows:
wherein D represents a discriminator; g represents a generator;
expressing the identification capability of the maximum discriminator D on the original image of the training license plate and the original image of the background;
expressed as the discriminative power of the minimum arbiter D on the background training prediction graph generated by the generator G;
the average value of x and z is expressed, x obeys z, x is expressed as the original image of the training license plate, and z is expressed as the background training prediction image corresponding to x; g (x, z), which is the output of the generator G with x and z as input, i.e. the background training prediction graph; d (x, G (x, z)) is a discrimination probability indicating whether the original training license plate image output by the discriminator D matches the background training prediction image.
In the actual training process, the generator G may be fixed to train the discriminator D, and in the training process of the discriminator D, in order to make the discrimination capability of the discriminator stronger, that is, the discrimination probability for the original image of the training license plate tends to 1, and the discrimination probability for the background training prediction image generated by the generator tends to 0, so as to obtain the maximum value of the discriminator D
Then, fixing the discriminator D to train the generator G, in order to ensure the generated background training prediction graph is true, making the discrimination probability of the background training prediction graph generated by the generator tend to 1, and at this moment, the discriminator D is a constant to obtain the minimum value of the generator G
And then performing alternate iterative training based on the generator and the arbiter, so that the generator and the arbiter mutually promote, and judging whether to stop training based on the background training prediction graph and the discrimination result. Judging whether to stop training based on the background training prediction graph and the judgment result, wherein the judging step comprises the following steps: and (4) constraining the countermeasure network model based on the background training prediction graph and the generator loss function and the discriminator loss function until the generator loss function and the discriminator loss function are converged, and finishing training when the generated training license plate background enables the discriminator to be difficult to distinguish true from false.
A generator-based derivation of a loss function Lr, expressed as:
Lr＝LC+LT
wherein L isCExpressed as a content loss function, expressed as:
wherein, IoutExpressed as a feature map extracted from the deconvolution path, i.e. the original output; i iscompOutput expressed as text erasure area, IcompAnd IoutThe non-text area forms the real background of the license plate; an is the activation map of the nth selected layer.
LTExpressed as a texture loss function, expressed as:
wherein (H)nWn)CnIs the shape of a high-level activation map, An is the activation map of the nth selected layer, which is used for making up the difference region between the text region and the non-text texture appearance region so as to facilitate the realization of the methodFunctionality where the network can capture global styles to more reasonably replace text regions, IoutExpressed as raw output, IcompRepresented as the output of a text erasure area.
Deriving a loss function L based on a discriminatorDExpressed as:
wherein, sum (M)i) Expressed as a scale factor, selecting appropriate proportions to adjust different numbers of text erased areas, nxn expressed as erased text patches, LiRepresented as a label, P, corresponding to the text patchiDenoted as the arbiter prediction probability.
When a countermeasure network model is generated and trained, optimizing by adopting an Adam optimization strategy, specifically, setting the batch processing size to be 50 during training; the initial learning rate was set to 0.0005; the weight attenuation parameter is set to be 2 x 10-4; the number of iterations was set to 1000 rounds.
And secondly, synthesizing based on a foreground text image and a background image which are acquired in advance to obtain a fake license plate image. It should be noted that, a foreground text image, a background image and a pre-made standard font image corresponding to a plurality of original images for training license plates are freely combined to obtain a plurality of pseudo license plate images with different combinations, so that the pseudo license plate images are conveniently used for inputting a text replacement model in the following process, and a text skeleton image and a foreground text image are used as supervision information to train the text replacement model.
In the embodiment, after the generation of the confrontation network model is finished, a real license plate set is used for testing to obtain a large number of background images, and the background images with better erasing effect are obtained by screening the background images; and obtaining multiple groups of supervision label images in a ratio of 1: 15 based on the background image by data augmentation processing, for example, screening 2005 background images with better erasing effect, and obtaining 30075 groups of augmented data after data augmentation, wherein the data augmentation adopts free combination of a foreground text image and the background image, the augmented data can be the label images, and each group of label images comprises 7 label images. Referring to fig. 4, the first line is a pseudo license plate image, a text skeleton image, a difference image and a foreground text image from left to right in sequence; the second row sequentially comprises a license plate background, a license plate original image and a text mask image from left to right. By the method for acquiring the monitoring image of the test label, the original image of each license plate is processed to obtain a background image and a foreground text image corresponding to each real license plate, so that the images of the first background prediction image and the foreground text image of the real license plate can be conveniently and freely combined in the follow-up process to obtain the spliced pseudo license plate image.
In an optional embodiment, before synthesizing based on a foreground text image and a background image obtained in advance, obtaining a pseudo-license plate image, the method includes: performing difference value calculation based on the original image of the training license plate and the background image to obtain a difference value image; extracting the difference image by using a skeleton extraction algorithm to obtain a text skeleton image; binarizing the difference image to obtain a text mask image with a black background and a white text; and performing intersection image processing by using the text mask image and the original image of the training license plate to obtain a foreground text image.
It should be noted that the background of the difference map is substantially black, the pixel is 0, and the outline of the pixel with color is the position of the text; the skeleton extraction algorithm mainly utilizes image corrosion and expansion in a classical image processing method to extract stroke information of a text, and sets the number of corresponding pixel points according to the width of a stroke, for example, 3 pixel points; the pixel value of the foreground text image is the same as the pixel value of the text in the license plate original image.
And then, inputting the target text image, the pre-acquired license plate background image, the pre-acquired text skeleton image, the foreground text image and the pseudo license plate image into a text replacement model to obtain a license plate training image output by the text replacement model.
In this embodiment, inputting a target text image, a pre-acquired license plate background image, a pre-acquired text skeleton image, a foreground text image, and a pseudo license plate image into a text replacement model to obtain a license plate training image output by the text replacement model, includes: inputting a target text image, a license plate background image, a text skeleton image, a foreground text image and a pseudo license plate image which are acquired in advance into a text conversion layer to obtain a foreground training image output by the text conversion layer; inputting the original image of the training license plate to a background repairing layer to obtain a background training image output by the background repairing layer; and inputting the foreground training image and the background training image into the text fusion layer to obtain the license plate training image output by the text fusion layer.
And finally, constructing a loss function based on the foreground text image, the background image, the license plate training image, the text skeleton image, the target text image and the pseudo license plate image, and converging based on the loss function to finish training.
Firstly, a target text image and a pseudo license plate image with standard font texts are used for training a text conversion layer, and a foreground training image is output. Specifically, the foreground text image and the text skeleton image are used for performing text style conversion on the target text image, that is, the target text image is subjected to style conversion, such as text color, font, shape and other information conversion, so that the generated foreground training image contains specified text content and specified text image style characteristics. In this embodiment, the specified text image style feature may be a foreground text feature of the original license plate training image.
And constructing a text conversion loss function based on the fake license plate image, the target text image, the text skeleton image, the foreground training image and the foreground text image, converging the text conversion loss function to finish training, and obtaining a text conversion loss function LTExpressed as:
wherein G isTA generator representing a text conversion layer; i istRepresented as a foreground text map; i issA fake license plate map representing the input text conversion layer; t istRepresented as a foreground text map; gT(It，Is) Representing a foreground training graph generated for a generator; i Tt-GT(It，Is)||1Expressed as using L1 norm for TtAnd GT(It，Is) Absolute error ofCarrying out constraint; t isskRepresented as a text skeleton diagram for constraining OskTraining; o isskA text skeleton graph represented as an output;
is shown as a pair TskAnd OskMaking pixel-level summation and cross-over ratio to facilitate the discrimination of TskAnd OskThe similarity of (2);
is denoted by TskAnd OskA difference of (a); alpha is a coefficient. Minimizing L during trainingTThe function is such that the foreground prediction map generated by the generator is as identical as possible to the foreground text map.
Training the background repairing layer by using the fake license plate, and outputting a background training picture; and constructing a background restoration loss function based on the fake license plate, the background training image and the background image, and converging according to the background restoration loss function to finish training. Background repair loss function LBExpressed as:
wherein D isBA discriminator expressed as a background restoration layer; gBA generator represented as a background repair layer; i issDenoted as a fake license plate; t isbExpressed as a background map;
is shown as relating to Tb、IsThe mean value of (a); dB(Tb，Is) The judgment probability is expressed as whether the background image output by the discriminator is matched with the original image of the training license plate;
is shown as relating to IsThe mean value of (a); gB(Is) Representing a background training picture generated by a discriminator based on a fake license plate; dB(GB(Is)，Is) Expressing the discrimination probability of whether the background training image output by the discriminator is matched with the fake license plate or not; beta is expressed as a hyperparameter; beta | | | Tb-GB(Is)||1Expressed as the norm L1, i.e., the absolute error between the background map generated by the minimization generator and the background training map.
Training the text fusion layer by using the foreground training image and the background training image, and outputting a license plate training image; constructing a text fusion loss function based on the license plate training image and the fake license plate image, converging according to the text fusion loss function to finish training, and fusing a text loss function LFExpressed as:
wherein G isFAnd DFGenerators and discriminators, T, respectively, for text fusion layersfAnd OfThe license plate training images are respectively a fake license plate image at the output end and a license plate training image predicted by the generator;
is shown as relating to Tf、ItThe mean value of (a); df(Tf，It) Expressing the discrimination probability of whether the fake license plate image output by the discriminator is matched with the foreground text image or not;
is shown as relating to ItThe mean value of (a); dF(Of，It) And expressing the discrimination probability of whether the license plate training image predicted by the generator output by the discriminator is matched with the foreground text image.
It should be noted that, through the constraint of the loss functions of the text conversion layer, the background repair layer and the text fusion layer, iterative training is performed on the text conversion layer, the background repair layer and the text fusion layer for many times, and if the text conversion loss function, the background repair loss function and the text fusion loss function are converged, so that the output license plate training image and the pseudo license plate have almost the same effect, the training is finished.
When the text replacement model is trained, the batch processing size is set to be 8; the initial learning rate was set to 0.0001; the maximum number of iterations is set to 50000; the momentum parameter is set to 0.9.
After training is completed, the results are verified in order to ensure the validity of the experimental results. In order to generate the license plate image, the license plate training image at the second stage can be directly tested, referring to the attached figure 5, under the condition that an input verification license plate original image and a target text image with a standard font are given, a new license plate image with a verification license plate original image background and a target text with a standard font text image can be output, the conditions of the background except the text content of the verification license plate original image, the font shape, the color, the brightness of the image and the like of the text can be kept, the harmony and the sense of reality of the background and the foreground characters of the real license plate image are kept, and the generated license plate verification image is ensured to keep the background and the text style of the verification license plate original image. More examples of generating license plates can be seen in fig. 6, where each row represents a plurality of new license plates with different provinces and serial numbers generated from one original license plate image, and the style characteristics of the background and foreground characters of the license plate can be maintained, and each character structure stroke of the generated license plate is correct and has harmony with the background. The result chart proves that the method and the device can not only solve the problem of license plate generation under the conditions of clear image quality and normal and good positions, but also well maintain the background and foreground styles of the original license plate image for the shadow caused by dark overall brightness and light irradiation, or the slightly deviated license plate position and the license plate image under a certain exposure condition, ensure that only the license plate text is changed, and ensure that the generated license plate training test image is similar to the real license plate image and has stronger sense of reality.
In summary, the invention performs text replacement on the original test license plate image based on the pre-obtained target text image through the text replacement model to obtain the text content with the target text image and the license plate test image with the original test license plate image background, so as to generate a new license plate, ensure that the generated new license plate maintains the characteristics of the original license plate background, the character style and the like, and replace the serial number of the license plate of the original license plate image to effectively expand the data set.
The license plate image generation system based on the generation countermeasure network provided by the invention is described below, and the license plate image generation system based on the generation countermeasure network described below and the license plate image generation method based on the generation countermeasure network described above can be referred to each other correspondingly.
Referring to fig. 7, fig. 7 shows a license plate image generation system based on a generation countermeasure network, the system comprising:
the acquisition module 71 is used for acquiring a target text image and a test license plate original image;
the text replacement module 72 is used for obtaining a license plate prediction image output by the text replacement module 72 based on the input original image of the tested license plate and a pre-acquired target text image; the text replacement module 72 is configured to perform text erasure on the training license plate original image to obtain a background training image, and form a pseudo license plate image and train the training license plate original image; the text replacement module 72 is configured to replace the text content of the pre-acquired target text image with the original test license plate image to obtain a license plate prediction image.
Specifically, the obtaining module 71 includes a drawing unit and an image capturing unit, where the drawing unit is configured to draw according to a given target text content and a standard font to obtain a target text image; and the camera shooting unit is used for shooting the license plate image as an original image of the tested license plate so as to be conveniently input into the text replacement module 72 subsequently.
The text replacement module 72 comprises a text conversion unit, and converts the foreground style of the target text image into the foreground style of the original image of the tested license plate based on the original image of the tested license plate to obtain a target text foreground image; the background repairing unit is used for carrying out text erasing and texture filling on the original image of the tested license plate to form a license plate background image; and the text fusion unit is used for performing text fusion on the target text foreground image and the license plate background image to obtain a license plate prediction image. Through the three units, the background with the license plate original image is generated, the license plate number is replaced by the specified target text, and the font and the background style of the target text are kept good.
In an optional embodiment, in order to train the text replacement module 72, the system further includes a text erasing module and a license plate synthesizing module, wherein the text erasing module is configured to erase a text of an original image of the trained license plate to obtain a background image; and the license plate synthesis module is used for synthesizing a foreground text image and the background image based on the pre-acquired foreground text image to obtain a pseudo license plate image.
In this embodiment, the text erasing module includes a generator, and erases a text based on the original image of the training license plate to obtain a background image. Still further, the generator comprises: the convolution unit is used for performing convolution operation on the basis of the training license plate original image and the corresponding pre-acquired background original image to obtain a characteristic diagram; the deconvolution unit is used for carrying out deconvolution operation based on the characteristic graph to generate a background graph; and the transverse connection unit is used for carrying out layer-by-layer transverse connection on the basis of the intermediate layer characteristics of the convolution unit so as to retain the texture characteristics. Note that the background artwork may be obtained based on manual erasure.
A license plate synthesis module comprising: the calculating unit is used for calculating a difference value based on the original image of the training license plate and the background image to obtain a difference value image; the extraction unit is used for extracting the difference image by utilizing a skeleton extraction algorithm so as to obtain a text skeleton image; the binarization unit is used for binarizing the difference image to obtain a text mask image with a black background and a white text; and the intersection unit is used for performing intersection image processing by using the text mask image and the original image of the tested license plate so as to obtain a foreground text image.
Fig. 8 illustrates a physical structure diagram of an electronic device, and as shown in fig. 8, the electronic device may include: a processor (processor)810, a communication Interface 820, a memory 830 and a communication bus 840, wherein the processor 810, the communication Interface 820 and the memory 830 communicate with each other via the communication bus 840. The processor 810 may invoke logic instructions in the memory 830 to perform a license plate image generation method based on generating a countermeasure network, the method comprising: acquiring a target text image and a test license plate original image; inputting the original test license plate image and the target text image into a text replacement model to obtain a license plate prediction image output by the text replacement model; the text replacement model is obtained by training a pseudo license plate image formed by a background training image obtained by erasing a text based on a training license plate original image and the training license plate original image; the text replacement model is used for replacing the text content of the pre-acquired target text image with the original image of the tested license plate to obtain a license plate prediction image.
In addition, the logic instructions in the memory 830 may be implemented in software functional units and stored in a computer readable storage medium when the logic instructions are sold or used as independent products. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.
In another aspect, the present invention also provides a computer program product comprising a computer program stored on a non-transitory computer-readable storage medium, the computer program comprising program instructions, which when executed by a computer, enable the computer to execute the license plate image generation method based on generation of a countermeasure network provided by the above methods, the method comprising: acquiring a target text image and a test license plate original image; inputting the original test license plate image and the target text image into a text replacement model to obtain a license plate prediction image output by the text replacement model; the text replacement model is obtained by training a pseudo license plate image formed by a background training image obtained by erasing a text based on a training license plate original image and the training license plate original image; the text replacement model is used for replacing the text content of the pre-acquired target text image with the original image of the tested license plate to obtain a license plate prediction image.
In yet another aspect, the present invention further provides a non-transitory computer-readable storage medium, on which a computer program is stored, the computer program being implemented by a processor to execute the license plate image generation method based on generation of a countermeasure network provided in the foregoing, the method including: acquiring a target text image and a test license plate original image; inputting the original test license plate image and the target text image into a text replacement model to obtain a license plate prediction image output by the text replacement model; the text replacement model is obtained by training a pseudo license plate image formed by a background training image obtained by erasing a text based on a training license plate original image and the training license plate original image; the text replacement model is used for replacing the text content of the pre-acquired target text image with the original image of the tested license plate to obtain a license plate prediction image.
The above-described embodiments of the apparatus are merely illustrative, and the units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment. One of ordinary skill in the art can understand and implement it without inventive effort.
Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware. With this understanding in mind, the above-described technical solutions may be embodied in the form of a software product, which can be stored in a computer-readable storage medium such as ROM/RAM, magnetic disk, optical disk, etc., and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the methods described in the embodiments or some parts of the embodiments.
Finally, it should be noted that: the above examples are only intended to illustrate the technical solution of the present invention, but not to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.
Claims (10)
1. A license plate image generation method based on a generation countermeasure network is characterized by comprising the following steps:
acquiring a target text image and a test license plate original image;
inputting the original test license plate image and the target text image into a text replacement model to obtain a license plate prediction image output by the text replacement model;
the text replacement model is obtained by training a pseudo license plate image formed by a background training image obtained by erasing a text based on the training license plate original image and the training license plate original image;
the text replacement model is used for replacing the text content of the pre-acquired target text image with the original image of the tested license plate to obtain a license plate prediction image.
2. The license plate image generation method based on the generation countermeasure network of claim 1, wherein the text replacement model comprises:
the text conversion layer is used for converting the foreground style of the target text image into the foreground style of the original image of the tested license plate based on the original image of the tested license plate to obtain a target text foreground image;
the background repairing layer is used for carrying out text erasing and texture filling on the original image of the tested license plate to form a license plate background image;
and the text fusion layer performs text fusion on the target text foreground image and the license plate background image to obtain a license plate prediction image.
3. The method for generating the license plate image based on the generation countermeasure network of claim 2, wherein training the text replacement model comprises:
erasing a text of an original image of the training license plate to obtain a background image;
synthesizing based on a foreground text image obtained in advance and the background image to obtain a pseudo license plate image;
inputting the target text image, a pre-acquired license plate background image, a pre-acquired text skeleton image, the foreground text image and the pseudo license plate image into the text replacement model to obtain a license plate training image output by the text replacement model;
and constructing a loss function based on the foreground text image, the background image, the license plate training image, the text skeleton image, the target text image and the pseudo license plate image, and converging based on the loss function to finish training.
4. The method for generating a license plate image based on a generation countermeasure network of claim 3, wherein the inputting the target text image, the pre-acquired license plate background image, the pre-acquired text skeleton image, the foreground text image and the pseudo license plate image into the text replacement model to obtain the license plate training image output by the text replacement model comprises:
inputting the target text image, the pre-acquired license plate background image, the pre-acquired text skeleton image, the foreground text image and the pseudo license plate image into the text conversion layer to obtain a foreground training image output by the text conversion layer;
inputting the original image of the training license plate to the background repairing layer to obtain a background training image output by the background repairing layer;
and inputting the foreground training image and the background training image into the text fusion layer to obtain a license plate training image output by the text fusion layer.
5. The method for generating a license plate image based on a generation countermeasure network of claim 4, wherein the constructing a loss function based on the foreground text image, the background image, the license plate training image, the text skeleton image, the target text image and the pseudo license plate image comprises:
constructing a text conversion loss function based on the fake license plate image, the target text image, the text skeleton image, the foreground text image and the foreground training image;
constructing a background repair loss function based on the fake license plate, the background image and the background training image;
constructing a text fusion loss function based on the fake license plate image and the license plate training image;
and obtaining a loss function based on the text conversion loss function, the background repair loss function and the text fusion loss function.
6. The method for generating the license plate image based on the generation countermeasure network of claim 3, wherein the text erasing of the original training license plate image to obtain the background image comprises:
performing convolution operation on the training license plate original image and the corresponding pre-acquired background original image to obtain a feature map;
performing deconvolution operation based on the feature map to generate a background map;
and performing layer-by-layer transverse connection based on the convolved intermediate layer characteristics so as to enable the generated background image to keep the texture characteristics of the original image of the training license plate.
7. The method for generating the license plate image based on the generation countermeasure network of claim 3, wherein before the step of synthesizing the pre-made standard font image, the pre-obtained foreground text image and the background image to obtain the pseudo license plate image, the method comprises:
making a standard font graph with the same text content of the original image of the training license plate;
performing difference calculation based on the original image of the training license plate and the background image to obtain a difference image;
extracting the difference image by using a skeleton extraction algorithm to obtain the text skeleton image;
binarizing the difference image to obtain a text mask image with a black background and a white text;
and performing intersection image processing by using the text mask image and the original image of the training license plate to obtain the foreground text image.
8. A license plate image generation system based on a generation countermeasure network is characterized by comprising:
the acquisition module acquires a target text image and an original image of the tested license plate;
the text replacement module is used for obtaining a license plate prediction image output by the text replacement module based on the test license plate original image and the target text image;
the text replacement module is used for performing text erasing on the training license plate original image to obtain a background training image and a pseudo license plate image formed by the background training image and training the training license plate original image;
the text replacement module is used for replacing the text content of the pre-acquired target text image with the original image of the tested license plate to obtain a license plate prediction image.
9. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the steps of the license plate image generation method based on generation of countermeasure network according to any one of claims 1 to 7 when executing the program.
10. A non-transitory computer readable storage medium, on which a computer program is stored, wherein the computer program, when being executed by a processor, implements the steps of the license plate image generation method based on generation of countermeasure network according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110700097.7A CN113505772B (en) | 2021-06-23 | 2021-06-23 | License plate image generation method and system based on generation countermeasure network |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110700097.7A CN113505772B (en) | 2021-06-23 | 2021-06-23 | License plate image generation method and system based on generation countermeasure network |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113505772A true CN113505772A (en) | 2021-10-15 |
| CN113505772B CN113505772B (en) | 2024-05-10 |
Family
ID=78010540
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110700097.7A Active CN113505772B (en) | 2021-06-23 | 2021-06-23 | License plate image generation method and system based on generation countermeasure network |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113505772B (en) |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114529897A (en) * | 2022-04-24 | 2022-05-24 | 浙江大华技术股份有限公司 | License plate generation method, electronic device and storage medium |
| CN115310405A (en) * | 2022-07-21 | 2022-11-08 | 北京汉仪创新科技股份有限公司 | Font replacement method, system, device and medium based on countermeasure generation network |
| CN115810185A (en) * | 2022-12-21 | 2023-03-17 | 南通大学 | Lightweight license plate identification method based on generation countermeasure network data enhancement |
| CN116091363A (en) * | 2023-04-03 | 2023-05-09 | 南京信息工程大学 | Handwriting Chinese character image restoration method and system |
| CN116167910A (en) * | 2023-01-03 | 2023-05-26 | 深圳前海环融联易信息科技服务有限公司 | Text editing method, text editing device, computer equipment and computer readable storage medium |
| CN115310405B (en) * | 2022-07-21 | 2024-07-26 | 北京汉仪创新科技股份有限公司 | Font replacement method, system, device and medium based on countermeasure generation network |
Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9367763B1 (en) * | 2015-01-12 | 2016-06-14 | Xerox Corporation | Privacy-preserving text to image matching |
| CN109190620A (en) * | 2018-09-03 | 2019-01-11 | 苏州科达科技股份有限公司 | License plate sample generating method, system, equipment and storage medium |
| US20190130278A1 (en) * | 2017-10-26 | 2019-05-02 | Nvidia Corporation | Progressive modification of generative adversarial neural networks |
| US20190251401A1 (en) * | 2018-02-15 | 2019-08-15 | Adobe Inc. | Image composites using a generative adversarial neural network |
| CN110458918A (en) * | 2019-08-16 | 2019-11-15 | 北京百度网讯科技有限公司 | Method and apparatus for output information |
| CN110516577A (en) * | 2019-08-20 | 2019-11-29 | Oppo广东移动通信有限公司 | Image processing method, device, electronic equipment and storage medium |
| CN110868598A (en) * | 2019-10-17 | 2020-03-06 | 上海交通大学 | Video content replacement method and system based on countermeasure generation network |
| CN111460837A (en) * | 2020-03-31 | 2020-07-28 | 广州大学 | Character-level confrontation sample generation method and device for neural machine translation |
| CN112580623A (en) * | 2020-12-25 | 2021-03-30 | 北京百度网讯科技有限公司 | Image generation method, model training method, related device and electronic equipment |
| CN112734881A (en) * | 2020-12-01 | 2021-04-30 | 北京交通大学 | Text synthesis image method and system based on significance scene graph analysis |
-
2021
- 2021-06-23 CN CN202110700097.7A patent/CN113505772B/en active Active
Patent Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9367763B1 (en) * | 2015-01-12 | 2016-06-14 | Xerox Corporation | Privacy-preserving text to image matching |
| US20190130278A1 (en) * | 2017-10-26 | 2019-05-02 | Nvidia Corporation | Progressive modification of generative adversarial neural networks |
| US20190251401A1 (en) * | 2018-02-15 | 2019-08-15 | Adobe Inc. | Image composites using a generative adversarial neural network |
| CN109190620A (en) * | 2018-09-03 | 2019-01-11 | 苏州科达科技股份有限公司 | License plate sample generating method, system, equipment and storage medium |
| CN110458918A (en) * | 2019-08-16 | 2019-11-15 | 北京百度网讯科技有限公司 | Method and apparatus for output information |
| CN110516577A (en) * | 2019-08-20 | 2019-11-29 | Oppo广东移动通信有限公司 | Image processing method, device, electronic equipment and storage medium |
| CN110868598A (en) * | 2019-10-17 | 2020-03-06 | 上海交通大学 | Video content replacement method and system based on countermeasure generation network |
| CN111460837A (en) * | 2020-03-31 | 2020-07-28 | 广州大学 | Character-level confrontation sample generation method and device for neural machine translation |
| CN112734881A (en) * | 2020-12-01 | 2021-04-30 | 北京交通大学 | Text synthesis image method and system based on significance scene graph analysis |
| CN112580623A (en) * | 2020-12-25 | 2021-03-30 | 北京百度网讯科技有限公司 | Image generation method, model training method, related device and electronic equipment |
Non-Patent Citations (3)
| Title |
|---|
| BOXI YU 等: "Mask-guided GAN for robust text editing in the scene", 《ELSEVIER》 * |
| MING SUN 等: "Image Generation Framework for Unbalanced License Plate Data Set", 《2019 INTERNATIONAL CONFERENCE ON DATA MINING WORKSHOPS (ICDMW)》 * |
| 伍刻 等: "基于归因注意力的图像生成模型研究", 《武汉理工大学学报》 * |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114529897A (en) * | 2022-04-24 | 2022-05-24 | 浙江大华技术股份有限公司 | License plate generation method, electronic device and storage medium |
| CN115310405A (en) * | 2022-07-21 | 2022-11-08 | 北京汉仪创新科技股份有限公司 | Font replacement method, system, device and medium based on countermeasure generation network |
| CN115310405B (en) * | 2022-07-21 | 2024-07-26 | 北京汉仪创新科技股份有限公司 | Font replacement method, system, device and medium based on countermeasure generation network |
| CN115810185A (en) * | 2022-12-21 | 2023-03-17 | 南通大学 | Lightweight license plate identification method based on generation countermeasure network data enhancement |
| CN116167910A (en) * | 2023-01-03 | 2023-05-26 | 深圳前海环融联易信息科技服务有限公司 | Text editing method, text editing device, computer equipment and computer readable storage medium |
| CN116167910B (en) * | 2023-01-03 | 2024-04-26 | 深圳前海环融联易信息科技服务有限公司 | Text editing method, text editing device, computer equipment and computer readable storage medium |
| CN116091363A (en) * | 2023-04-03 | 2023-05-09 | 南京信息工程大学 | Handwriting Chinese character image restoration method and system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113505772B (en) | 2024-05-10 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113505772B (en) | License plate image generation method and system based on generation countermeasure network | |
| CN111723585B (en) | Style-controllable image text real-time translation and conversion method | |
| CN111292264B (en) | Image high dynamic range reconstruction method based on deep learning | |
| US11854244B2 (en) | Labeling techniques for a modified panoptic labeling neural network | |
| CN114219943B (en) | CT image organ at risk segmentation system based on deep learning | |
| CN109872278B (en) | Image cloud layer removing method based on U-shaped network and generation countermeasure network | |
| CN112184585B (en) | Image completion method and system based on semantic edge fusion | |
| CN110766020A (en) | System and method for detecting and identifying multi-language natural scene text | |
| CN113255659B (en) | License plate correction detection and identification method based on MSAFF-yolk 3 | |
| CN117409192B (en) | Data enhancement-based infrared small target detection method and device | |
| CN115131797A (en) | Scene text detection method based on feature enhancement pyramid network | |
| CN117197763A (en) | Road crack detection method and system based on cross attention guide feature alignment network | |
| CN116645592A (en) | Crack detection method based on image processing and storage medium | |
| CN113553797A (en) | End-to-end consistent-style character and image generation method | |
| CN116402067B (en) | Cross-language self-supervision generation method for multi-language character style retention | |
| Li et al. | SPN2D-GAN: semantic prior based night-to-day image-to-image translation | |
| CN116863476A (en) | Image generation method and system for removing seal noise based on mask guidance | |
| CN116798041A (en) | Image recognition method and device and electronic equipment | |
| CN116778497A (en) | Method and device for identifying hand well number, computer equipment and storage medium | |
| CN117671509B (en) | Remote sensing target detection method and device, electronic equipment and storage medium | |
| Vansteenkiste et al. | Taming adversarial domain transfer with structural constraints for image enhancement | |
| CN117422787B (en) | Remote sensing image map conversion method integrating discriminant and generative model | |
| CN117593223A (en) | Bill repairing method and device, electronic equipment and storage medium | |
| Liu et al. | Classification guided thick fog removal network for drone imaging: ClassifyCycle | |
| CN117746130A (en) | Weak supervision deep learning classification method based on remote sensing image punctiform semantic tags |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB03 | Change of inventor or designer information | ||
| CB03 | Change of inventor or designer information |
Inventor after: Liu Yan Inventor after: Sun Yifan Inventor after: Zhang Xiaobo Inventor after: Yin Xucheng Inventor after: Yang Chun Inventor before: Liu Yan Inventor before: Sun Yifan Inventor before: Zhang Xiaobo Inventor before: Yin Xucheng Inventor before: Yang Chun |
|
| GR01 | Patent grant | ||
| GR01 | Patent grant |