CN111899191A - Text image restoration method and device and storage medium - Google Patents
Text image restoration method and device and storage medium Download PDFInfo
- Publication number
- CN111899191A CN111899191A CN202010703681.3A CN202010703681A CN111899191A CN 111899191 A CN111899191 A CN 111899191A CN 202010703681 A CN202010703681 A CN 202010703681A CN 111899191 A CN111899191 A CN 111899191A
- Authority
- CN
- China
- Prior art keywords
- image
- set image
- network
- original
- training
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 43
- 238000012549 training Methods 0.000 claims abstract description 124
- 230000007547 defect Effects 0.000 claims abstract description 75
- 230000008439 repair process Effects 0.000 claims abstract description 68
- 238000012360 testing method Methods 0.000 claims abstract description 54
- 238000012545 processing Methods 0.000 claims abstract description 31
- 238000012217 deletion Methods 0.000 claims abstract description 13
- 230000037430 deletion Effects 0.000 claims abstract description 13
- 230000007246 mechanism Effects 0.000 claims description 19
- 230000002950 deficient Effects 0.000 claims description 13
- 238000005070 sampling Methods 0.000 claims description 12
- 230000008569 process Effects 0.000 claims description 11
- 238000004590 computer program Methods 0.000 claims description 6
- 230000004048 modification Effects 0.000 claims description 6
- 238000012986 modification Methods 0.000 claims description 6
- 238000005457 optimization Methods 0.000 claims description 5
- 230000006735 deficit Effects 0.000 claims 1
- 230000007812 deficiency Effects 0.000 abstract description 5
- 230000000694 effects Effects 0.000 description 12
- 238000013527 convolutional neural network Methods 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 230000003042 antagnostic effect Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000015572 biosynthetic process Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000004321 preservation Methods 0.000 description 1
- 238000009877 rendering Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10004—Still image; Photographic image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30176—Document
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Computation (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Health & Medical Sciences (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Biophysics (AREA)
- Biomedical Technology (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Evolutionary Biology (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Computational Linguistics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Quality & Reliability (AREA)
- Image Processing (AREA)
Abstract
The invention provides a text image restoration method, a text image restoration device and a storage medium, wherein the method comprises the following steps: obtaining a plurality of original medical text images from image acquisition equipment, and randomly classifying the original medical text images to obtain an original training set image and an original test set image; respectively carrying out image deletion processing on the original training set image and the original test set image to obtain a defect training set image and a defect test set image; constructing a training model, and training the training model according to the defect training set image to obtain a final repair model; and repairing the defect test set image according to the final repair model to obtain a repaired image result. The method overcomes the defects of large workload, low efficiency and insufficient accuracy of the traditional manual text image restoration, can restore the condition that the text in the image has structural deficiency, can restore the incomplete text to have certain readability, and has a very deep significance for reading the incomplete text.
Description
Technical Field
The invention mainly relates to the technical field of image restoration, in particular to a text image restoration method, a text image restoration device and a storage medium.
Background
Image restoration has originated in european art restoration, a process by which an artistic craftsman recovers damaged artwork. Because the artwork is easily influenced by factors such as human, environment, material and the like in the preservation process, the artwork is difficult to avoid being damaged by some factors, such as cracks, loss, mildew, falling, scratches and the like. In the revival period of European literature, the art works in the middle century are repaired in order to recover the damaged or lost parts of the art works and maintain the overall effect of the works. Image restoration is performed by professional artists, repairing the damaged work of art according to the nature and degree of the work of art, in order to hopefully restore the original appearance of the work of art. However, the method is often in great risk because the artwork is extremely precious and cannot be compensated once an accident occurs.
Image inpainting is an important task in computer vision, and has numerous applications in image-based rendering and computational photography, and the main difficulty of image inpainting is how to synthesize visually realistic and semantically reasonable pixels in missing parts of an image, and the sense of incongruity cannot be stored between these pixels and existing pixels.
Disclosure of Invention
The invention provides a text image restoration method, a text image restoration device and a storage medium, aiming at the defects of the prior art.
The technical scheme for solving the technical problems is as follows: a text image restoration method comprises the following steps:
obtaining a plurality of original medical text images from image acquisition equipment, and randomly classifying the original medical text images to obtain an original training set image and an original test set image;
respectively carrying out image deletion processing on the original training set image and the original test set image to obtain a defect training set image and a defect test set image;
constructing a training model, and training the training model according to the defect training set image to obtain a final repair model;
and repairing the defect test set image according to the final repair model to obtain a repaired image result.
Another technical solution of the present invention for solving the above technical problems is as follows: a text image restoration apparatus comprising:
the random classification processing module is used for obtaining a plurality of original medical text images from image acquisition equipment and randomly classifying the original medical text images to obtain an original training set image and an original test set image;
the defect image processing module is used for respectively carrying out image deletion processing on the original training set image and the original test set image to obtain a defect training set image and a defect test set image;
the training model processing module is used for constructing a training model and training the training model according to the defect training set image to obtain a final repairing model;
and the repair image result obtaining module is used for repairing the defect test set image according to the final repair model to obtain a repair image result.
The invention has the beneficial effects that: the method comprises the steps of obtaining an original training set image and an original test set image by randomly classifying a plurality of original medical text images, obtaining a defect training set image and a defect test set image by respectively carrying out image deletion processing on the original training set image and the original test set image, constructing a training model, obtaining a final repairing model by training the training model according to the defect training set image, further improving the image repairing effect, obtaining a repairing image result according to the final repairing model by carrying out repairing processing on the defect test set image, overcoming the defects of large workload, low efficiency and insufficient accuracy of the traditional manual repairing text image, repairing the condition that the text in the image has structure deletion, repairing the defective text to a certain readability, and reading the defective text to a great significance.
Drawings
Fig. 1 is a schematic flowchart of a text image restoration method according to an embodiment of the present invention;
fig. 2 is a block diagram of a text image restoration apparatus according to an embodiment of the present invention.
Detailed Description
The principles and features of this invention are described below in conjunction with the following drawings, which are set forth by way of illustration only and are not intended to limit the scope of the invention.
Fig. 1 is a schematic flow chart of a text image restoration method according to an embodiment of the present invention.
As shown in fig. 1, a text image restoration method includes the following steps:
obtaining a plurality of original medical text images from image acquisition equipment, and randomly classifying the original medical text images to obtain an original training set image and an original test set image;
respectively carrying out image deletion processing on the original training set image and the original test set image to obtain a defect training set image and a defect test set image;
constructing a training model, and training the training model according to the defect training set image to obtain a final repair model;
and repairing the defect test set image according to the final repair model to obtain a repaired image result.
Specifically, the original medical text images are Chinese character images captured from the medical text images, the content of the images is mostly 1 to 4 characters, 56867 images exist in total, wherein the images corresponding to the same Chinese character are not unique, and the situation that the same Chinese character corresponds to a plurality of images can occur; because the use frequency of each Chinese character in the medical text is unequal, the more frequently the Chinese characters are used, the number of the obtained images is correspondingly increased, and each type of image cannot reach the same scale; putting all the images into the same folder, randomly selecting the images from the folder, dividing the images into two parts, wherein one part is used as an original training image, the other part is used as an original test image, the number of the original test images is 20% of the number of all the images, and the randomness is enhanced after the images are disordered.
In the embodiment, the original training set image and the original test set image are obtained by randomly classifying a plurality of original medical text images, the defect training set image and the defect test set image are obtained by respectively carrying out image deletion processing on the original training set image and the original test set image, the training model is constructed, the final repairing model is obtained by training the training model according to the defect training set image, the image repairing effect is further improved, the repairing image result is obtained by carrying out repairing processing on the defect test set image according to the final repairing model, the defects of large workload, low efficiency and insufficient accuracy of the traditional manual repairing text image are overcome, the condition that the text in the image has the structure deletion can be repaired, the defective text can be repaired to have certain readability, and the reading of the defective text is of great significance.
Optionally, as an embodiment of the present invention, the process of performing image deletion processing on the original training set image and the original test set image respectively to obtain a defect training set image and a defect test set image includes:
adding a preset binary patch at a random position on each image in the original training set image to obtain a defective training set image;
and adding a preset binary patch at a random position on each image in the original test set image to obtain a defect test set image.
In the above embodiment, a preset binary patch is added to a random position on each image in the original training set image to obtain a defective training set image; the method has the advantages that the preset binary patches are added at random positions of the images in the original test set image to obtain the defective test set image, objectivity of data can be guaranteed, human factors are reduced, accuracy of a subsequent repair model is effectively improved, the defects of large workload, low efficiency and insufficient accuracy of the traditional manual text image repair are overcome, the condition that the text in the image has structural deficiency can be repaired, the defective text can be repaired to have certain readability, and the method has a deep significance in reading the defective text.
Optionally, as an embodiment of the present invention, the constructing a training model, and training the training model according to the defect training set image to obtain a final repair model includes:
constructing a generated countermeasure network, and training the generated countermeasure network by using the defect training set image to obtain a basic restoration model;
and optimizing the basic restoration model by using an attention mechanism optimization algorithm to obtain a final restoration model.
Specifically, an attention mechanism is added in the network, the Chinese characters with missing structures in the image are repaired, and the Chinese characters are formed by strokes, so that the attention mechanism is more hopefully introduced in the repairing process, the network pays attention to the missing information in a certain stroke and the residual information of the strokes originally connected with the network, and the repairing effect of the repairing model is improved; the generation countermeasure network model is divided into a repair network and a discrimination network, and the two networks are defined as a convolutional neural network; corresponds to two parts: a generator (G) and a Discriminator (D), the generator receiving as input a random noise (an initial distribution), generating an image from the noise, and taking it as input to the Discriminator; the nature of the discriminator is a two-classifier which is used to discriminate whether the input data is from real data or the generation of G-network, and the final result returned is the probability that the image is from training data. If the image is from training data, D should return a value close to 1, otherwise a value close to 0 should be returned to indicate that the image is false, from G-network generation. Therefore, the basic restoration model obtained by the text image restoration method based on the attention mechanism and the generation countermeasure network can effectively improve the accuracy and effect of image restoration, and the final restoration model obtained by adding the attention mechanism to the basic restoration model for optimization can further improve the image restoration effect and can restore the incomplete text to have certain readability.
In the above embodiment, the basic repairing model is obtained by training the generated countermeasure network with the defect training set image; the final repairing model is obtained by optimizing the basic repairing model by using an attention mechanism optimization algorithm, the accuracy and the effect of image repairing can be effectively improved, the image repairing effect is further improved, the incomplete text can be repaired to have certain readability, the defects of large workload, low efficiency and insufficient accuracy of the traditional manual repairing of the text image are overcome, the condition that the text in the image has structural deficiency can be repaired, the incomplete text can be repaired to have certain readability, and the method has a deep significance in reading the incomplete text.
Optionally, as an embodiment of the present invention, the training the generated countermeasure network by using the defect training set image to obtain a basic repair model includes:
s1: inputting the defect training set image into the generated countermeasure network for training according to preset iterative training times to obtain a first repair model, wherein the first repair model comprises a first repair network and a first discrimination network;
s2: repairing the defect training set image through the first repairing network to obtain a first intermediate repairing image;
s3: performing discrimination processing on the first intermediate restored picture through the first discrimination network to obtain a first predicted value;
s4: judging whether the first predicted value is larger than or equal to a preset judgment value, if so, taking the first repairing model as the basic repairing model; otherwise, go to step S5;
s5: inputting the defect training set image into the generated countermeasure network for training according to preset iterative training times to obtain a second repair model, wherein the second repair model comprises a second repair network and a second judgment network;
s6: repairing the defect training set image through the second repairing network to obtain a second intermediate repairing image;
s7: judging the second intermediate repair picture through the second judgment network to obtain a second predicted value;
s8: judging whether the second predicted value is larger than or equal to a preset judgment value, and if so, taking the second repairing model as the basic repairing model; if so, the process returns to step S1.
It should be understood that the formation of the antagonistic network architecture is divided into a repair network and a discrimination network in the network structure, and both networks are defined as convolutional neural networks.
In the above embodiment, the first repairing model and the second repairing model are respectively obtained according to the defect training set image and the generated confrontation network model, and the basic repairing model is obtained according to whether the judgment result meets the preset judgment value, so that the expected repairing model is obtained while the higher repairing efficiency is ensured, and the stability and the reliability of the text image repairing are realized.
Optionally, as an embodiment of the present invention, the optimizing the basic repair model by using an attention mechanism optimization algorithm to obtain a final repair model includes:
the basic restoration model comprises an original first-stage restoration network, an original second-stage restoration network and a judgment network, wherein the original first-stage restoration network comprises a 3 x 3 convolution layer, a down-sampling layer, a cavity convolution layer and an up-sampling layer;
adding a preset 1 × 1 convolutional layer to the front of the 3 × 3 convolutional layer to obtain a first-stage repair network;
performing network branch modification processing on the original second-stage repair network by using an attention mechanism algorithm to obtain a second-stage repair network;
and combining the first-stage repair network, the second-stage repair network and the judgment network according to the sequence to obtain a final repair model.
It should be appreciated that the original first-stage repair network alters the convolutional layer structure before the hole convolutional layer, adding several 1 x 1 convolutional layers, increasing the non-linearity and reducing the training time.
Specifically, the original first-stage restoration network includes a 3 × 3 convolution layer, a down-sampling layer, a hole convolution layer, and an up-sampling layer, and the first-stage restoration network obtained by adding the 1 × 1 convolution layer includes the 1 × 1 convolution layer, the 3 × 3 convolution layer, the down-sampling layer, the hole convolution layer for expanding the receptive field, and the up-sampling layer for restoring and outputting the picture.
In the above embodiment, a first-stage repair network is obtained before a preset 1 × 1 convolutional layer is added to the 3 × 3 convolutional layer; modifying the network branches of the original second-stage repair network by using an attention mechanism algorithm to obtain a second-stage repair network; the final repairing model is obtained by combining the first-stage repairing network, the second-stage repairing network and the judging network according to the sequence, the nonlinearity is increased, the training time is reduced, the accuracy and the effect of image repairing can be effectively improved, the image repairing effect is further improved, the incomplete text can be repaired to have certain readability, the defects of large workload, low efficiency and insufficient accuracy of the traditional manual text image repairing are overcome, the condition that the text in the image has structural deficiency can be repaired, the incomplete text can be repaired to have certain readability, and the reading of the incomplete text is of great significance.
Optionally, as an embodiment of the present invention, the process of performing network branch modification processing on the original second-stage repair network by using an attention mechanism algorithm to obtain a second-stage repair network includes:
the original second-stage repair network comprises two convolution branches;
performing convolution branch modification on any volume branch in the original second-stage repair network by using an attention mechanism algorithm to obtain an attention branch;
and combining the attention branch and the other convolution branch to obtain a second-stage repair network.
Specifically, an attention mechanism is introduced to change a branch, wherein one convolution branch acts on a network structure before the cavity convolution, a combination structure of convolution and a down-sampling layer is adopted, finally, a cavity convolution layer is added, the attention layer replaces the cavity convolution in the other branch, two convolution layers are added, output results of the two branches are combined, operation such as convolution up-sampling layer is carried out to reduce the picture to the input size, and a final restoration result graph is obtained.
In the above embodiment, an attention mechanism algorithm is used to modify the convolution branch of any convolution branch in the original second-stage repair network to obtain an attention branch; the attention branch and the other convolution branch are combined to obtain a second-stage repairing network, so that the accuracy and the effect of image repairing can be effectively improved, the image repairing effect is further improved, the incomplete text can be repaired to have certain readability, the defects of large workload, low efficiency and insufficient accuracy of the traditional manual repairing of the text image are overcome, the condition that the text in the image has structural deficiency can be repaired, the incomplete text can be repaired to have certain readability, and the method has a deep significance in reading the incomplete text.
Optionally, as another embodiment of the present invention, a batch of training samples are randomly selected from the training set, the sample image is randomly selected, and a binary patch is added at a random position to obtain a damaged image, an image with a size of 128 × 128 and a maximum patch size of 64 × 64 is set in the network, and the batch size is set to 16; the repairing image takes a damaged image and a binary patch as input data together, the input data is put into a repairing model for repairing, the model comprises a generator and a discriminator, two repairing stages are arranged in the generator, the input data enters a first-stage repairing network, the input data is input into the network, required characteristics are obtained through the steps of convolution, deconvolution and the like, a primary repaired image is obtained according to the characteristics and is input into a second-stage repairing network to complete subsequent repairing tasks, in order to improve the repairing efficiency, an attention mechanism is applied, a convolution branch and an attention branch are combined, one branch acts on a network structure before the cavity convolution, a combination structure of the convolution and a down sampling layer is adopted, finally a cavity convolution layer is added, the convolution cavity is replaced by the attention layer in the other branch, two convolution layers are added, after the output results of the two branches are combined, operations such as convolution and sampling layer uploading are carried out to reduce the image to the input size, and a final restoration result graph is obtained; at this time, the output result is the repairing result of the whole network, and a clear repairing image can be obtained.
Fig. 2 is a block diagram of a text image restoration apparatus according to an embodiment of the present invention.
Alternatively, as another embodiment of the present invention, as shown in fig. 2, a text image restoration apparatus includes:
the random classification processing module is used for obtaining a plurality of original medical text images from image acquisition equipment and randomly classifying the original medical text images to obtain an original training set image and an original test set image;
the defect image processing module is used for respectively carrying out image deletion processing on the original training set image and the original test set image to obtain a defect training set image and a defect test set image;
the training model processing module is used for constructing a training model and training the training model according to the defect training set image to obtain a final repairing model;
and the repair image result obtaining module is used for repairing the defect test set image according to the final repair model to obtain a repair image result.
Optionally, as an embodiment of the present invention, the defect image processing module is specifically configured to:
and respectively adding random positions of the original training set image and the original test set image according to a preset binary patch to obtain a defect training set image and a defect test set image, wherein the defect training set image and the defect test set image both carry the preset binary patch.
Alternatively, another embodiment of the present invention provides a text image restoration apparatus, including a memory, a processor, and a computer program stored in the memory and executable on the processor, and when the processor executes the computer program, the text image restoration method as described above is implemented. The device may be a computer or the like.
Alternatively, another embodiment of the present invention provides a computer-readable storage medium storing a computer program which, when executed by a processor, implements the text image inpainting method as described above.
It is clear to those skilled in the art that, for convenience and brevity of description, the specific working processes of the above-described apparatuses and units may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.
In the several embodiments provided in the present application, it should be understood that the disclosed apparatus and method may be implemented in other ways. For example, the above-described apparatus embodiments are merely illustrative, and for example, a division of a unit is merely a logical division, and an actual implementation may have another division, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed.
Units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment of the present invention.
In addition, functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.
The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a computer readable storage medium. It will be understood that the technical solution of the present invention essentially contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product stored in a storage medium and including instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.
While the invention has been described with reference to specific embodiments, it will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention as defined by the appended claims. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.
Claims (10)
1. A text image restoration method is characterized by comprising the following steps:
obtaining a plurality of original medical text images from image acquisition equipment, and randomly classifying the original medical text images to obtain an original training set image and an original test set image;
respectively carrying out image deletion processing on the original training set image and the original test set image to obtain a defect training set image and a defect test set image;
constructing a training model, and training the training model according to the defect training set image to obtain a final repair model;
and repairing the defect test set image according to the final repair model to obtain a repaired image result.
2. The method according to claim 1, wherein the process of performing image deletion processing on the original training set image and the original test set image respectively to obtain a defective training set image and a defective test set image comprises:
adding a preset binary patch at a random position on each image in the original training set image to obtain a defective training set image;
and adding a preset binary patch at a random position on each image in the original test set image to obtain a defect test set image.
3. The method according to claim 1, wherein the process of constructing a training model and training the training model according to the defect training set image to obtain a final repair model comprises:
constructing a generated countermeasure network, and training the generated countermeasure network by using the defect training set image to obtain a basic restoration model;
and optimizing the basic restoration model by using an attention mechanism optimization algorithm to obtain a final restoration model.
4. The method according to claim 3, wherein the training the generated countermeasure network with the images of the impairment training set to obtain a basic repair model comprises:
s1: inputting the defect training set image into the generated countermeasure network for training according to preset iterative training times to obtain a first repair model, wherein the first repair model comprises a first repair network and a first discrimination network;
s2: repairing the defect training set image through the first repairing network to obtain a first intermediate repairing image;
s3: performing discrimination processing on the first intermediate restored picture through the first discrimination network to obtain a first predicted value;
s4: judging whether the first predicted value is larger than or equal to a preset judgment value, if so, taking the first repairing model as the basic repairing model; otherwise, go to step S5;
s5: inputting the defect training set image into the generated countermeasure network for training according to preset iterative training times to obtain a second repair model, wherein the second repair model comprises a second repair network and a second judgment network;
s6: repairing the defect training set image through the second repairing network to obtain a second intermediate repairing image;
s7: judging the second intermediate repair picture through the second judgment network to obtain a second predicted value;
s8: judging whether the second predicted value is larger than or equal to a preset judgment value, and if so, taking the second repairing model as the basic repairing model; if so, the process returns to step S1.
5. The method according to claim 3, wherein the optimizing the basic repairing model by using the attention mechanism optimizing algorithm to obtain the final repairing model comprises:
the basic restoration model comprises an original first-stage restoration network, an original second-stage restoration network and a judgment network, wherein the original first-stage restoration network comprises a 3 x 3 convolution layer, a down-sampling layer, a cavity convolution layer and an up-sampling layer;
adding a preset 1 × 1 convolutional layer to the front of the 3 × 3 convolutional layer to obtain a first-stage repair network;
performing network branch modification processing on the original second-stage repair network by using an attention mechanism algorithm to obtain a second-stage repair network;
and combining the first-stage repair network, the second-stage repair network and the judgment network according to the sequence to obtain a final repair model.
6. The method according to claim 5, wherein the performing network branch modification processing on the original second-stage repair network by using the attention mechanism algorithm to obtain a second-stage repair network comprises:
the original second-stage repair network comprises two convolution branches;
performing convolution branch modification on any volume branch in the original second-stage repair network by using an attention mechanism algorithm to obtain an attention branch;
and combining the attention branch and the other convolution branch to obtain a second-stage repair network.
7. A text image restoration apparatus, comprising:
the random classification processing module is used for obtaining a plurality of original medical text images from image acquisition equipment and randomly classifying the original medical text images to obtain an original training set image and an original test set image;
the defect image processing module is used for respectively carrying out image deletion processing on the original training set image and the original test set image to obtain a defect training set image and a defect test set image;
the training model processing module is used for constructing a training model and training the training model according to the defect training set image to obtain a final repairing model;
and the repair image result obtaining module is used for repairing the defect test set image according to the final repair model to obtain a repair image result.
8. The text image restoration method according to claim 7, wherein the defective image processing module is specifically configured to:
and respectively adding random positions of the original training set image and the original test set image according to a preset binary patch to obtain a defect training set image and a defect test set image, wherein the defect training set image and the defect test set image both carry the preset binary patch.
9. A text image restoration apparatus comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that when the computer program is executed by the processor, the text image restoration method according to any one of claims 1 to 6 is implemented.
10. A computer-readable storage medium, in which a computer program is stored, which, when being executed by a processor, carries out a text image inpainting method according to any one of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010703681.3A CN111899191B (en) | 2020-07-21 | 2020-07-21 | Text image restoration method, device and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010703681.3A CN111899191B (en) | 2020-07-21 | 2020-07-21 | Text image restoration method, device and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111899191A true CN111899191A (en) | 2020-11-06 |
| CN111899191B CN111899191B (en) | 2024-01-26 |
Family
ID=73189674
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010703681.3A Active CN111899191B (en) | 2020-07-21 | 2020-07-21 | Text image restoration method, device and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111899191B (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112560732A (en) * | 2020-12-22 | 2021-03-26 | 电子科技大学中山学院 | Multi-scale feature extraction network and feature extraction method thereof |
| CN114742170A (en) * | 2022-04-22 | 2022-07-12 | 马上消费金融股份有限公司 | Countermeasure sample generation method, model training method, image recognition method and device |
| CN115797216A (en) * | 2022-12-14 | 2023-03-14 | 齐鲁工业大学 | Inscription character restoration model and restoration method based on self-coding network |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102283742A (en) * | 2011-08-15 | 2011-12-21 | 重庆大学 | Light stimulation based retinal prosthesis repairing device and repairing method |
| CN108830827A (en) * | 2017-05-02 | 2018-11-16 | 通用电气公司 | Neural metwork training image generation system |
| CN109003253A (en) * | 2017-05-24 | 2018-12-14 | 通用电气公司 | Neural network point cloud generates system |
| CN109377448A (en) * | 2018-05-20 | 2019-02-22 | 北京工业大学 | A kind of facial image restorative procedure based on generation confrontation network |
| CN109559287A (en) * | 2018-11-20 | 2019-04-02 | 北京工业大学 | A kind of semantic image restorative procedure generating confrontation network based on DenseNet |
| CN111161158A (en) * | 2019-12-03 | 2020-05-15 | 河海大学 | Image restoration method based on newly-formed network structure |
-
2020
- 2020-07-21 CN CN202010703681.3A patent/CN111899191B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102283742A (en) * | 2011-08-15 | 2011-12-21 | 重庆大学 | Light stimulation based retinal prosthesis repairing device and repairing method |
| CN108830827A (en) * | 2017-05-02 | 2018-11-16 | 通用电气公司 | Neural metwork training image generation system |
| CN109003253A (en) * | 2017-05-24 | 2018-12-14 | 通用电气公司 | Neural network point cloud generates system |
| CN109377448A (en) * | 2018-05-20 | 2019-02-22 | 北京工业大学 | A kind of facial image restorative procedure based on generation confrontation network |
| CN109559287A (en) * | 2018-11-20 | 2019-04-02 | 北京工业大学 | A kind of semantic image restorative procedure generating confrontation network based on DenseNet |
| CN111161158A (en) * | 2019-12-03 | 2020-05-15 | 河海大学 | Image restoration method based on newly-formed network structure |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112560732A (en) * | 2020-12-22 | 2021-03-26 | 电子科技大学中山学院 | Multi-scale feature extraction network and feature extraction method thereof |
| CN112560732B (en) * | 2020-12-22 | 2023-07-04 | 电子科技大学中山学院 | Feature extraction method of multi-scale feature extraction network |
| CN114742170A (en) * | 2022-04-22 | 2022-07-12 | 马上消费金融股份有限公司 | Countermeasure sample generation method, model training method, image recognition method and device |
| CN114742170B (en) * | 2022-04-22 | 2023-07-25 | 马上消费金融股份有限公司 | Countermeasure sample generation method, model training method, image recognition method and device |
| CN115797216A (en) * | 2022-12-14 | 2023-03-14 | 齐鲁工业大学 | Inscription character restoration model and restoration method based on self-coding network |
| CN115797216B (en) * | 2022-12-14 | 2024-05-24 | 齐鲁工业大学 | Self-coding network-based steganography character restoration model and restoration method |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111899191B (en) | 2024-01-26 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111899191A (en) | Text image restoration method and device and storage medium | |
| JP2022548712A (en) | Image Haze Removal Method by Adversarial Generation Network Fusing Feature Pyramids | |
| JP2020508531A (en) | Image quality evaluation method and image quality evaluation system | |
| CN113642659B (en) | Training sample set generation method and device, electronic equipment and storage medium | |
| CN111640130A (en) | Table reduction method and device | |
| CN109859204B (en) | Convolutional neural network model inspection method and device | |
| CN110675339A (en) | Image restoration method and system based on edge restoration and content restoration | |
| CN113283446A (en) | Method and device for identifying target object in image, electronic equipment and storage medium | |
| CN111144335A (en) | Method and device for building deep learning model | |
| CN111461211B (en) | Feature extraction method for lightweight target detection and corresponding detection method | |
| CN112836756A (en) | Image recognition model training method and system and computer equipment | |
| CN107203764A (en) | Long microblog picture recognition methods and device | |
| CN114758145A (en) | Image desensitization method and device, electronic equipment and storage medium | |
| CN114339049A (en) | Video processing method and device, computer equipment and storage medium | |
| CN115630660B (en) | Barcode positioning method and device based on convolutional neural network | |
| CN112801911A (en) | Method and device for removing Chinese character noise in natural image and storage medium | |
| CN108230251A (en) | Combined type image recovery method and device | |
| CN113034432B (en) | Product defect detection method, system, device and storage medium | |
| CN115439850B (en) | Method, device, equipment and storage medium for identifying image-text characters based on examination sheets | |
| CN116543333A (en) | Target recognition method, training method, device, equipment and medium of power system | |
| CN113610832B (en) | Logo defect detection method, device, equipment and storage medium | |
| CN111754518B (en) | Image set expansion method and device and electronic equipment | |
| CN115631192A (en) | Control method, device, equipment and medium for valve pressure tester | |
| CN104516899A (en) | Updating method and device for word stock | |
| Kartik et al. | Decoding of graphically encoded numerical digits using deep learning and edge detection techniques |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |