CN111899191B - Text image restoration method, device and storage medium - Google Patents
Text image restoration method, device and storage medium Download PDFInfo
- Publication number
- CN111899191B CN111899191B CN202010703681.3A CN202010703681A CN111899191B CN 111899191 B CN111899191 B CN 111899191B CN 202010703681 A CN202010703681 A CN 202010703681A CN 111899191 B CN111899191 B CN 111899191B
- Authority
- CN
- China
- Prior art keywords
- repair
- network
- image
- set image
- training
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 41
- 230000008439 repair process Effects 0.000 claims abstract description 176
- 238000012549 training Methods 0.000 claims abstract description 139
- 230000007547 defect Effects 0.000 claims abstract description 97
- 238000012360 testing method Methods 0.000 claims abstract description 54
- 238000012545 processing Methods 0.000 claims abstract description 35
- 238000012217 deletion Methods 0.000 claims abstract description 12
- 230000037430 deletion Effects 0.000 claims abstract description 12
- 230000007246 mechanism Effects 0.000 claims description 22
- 238000012986 modification Methods 0.000 claims description 9
- 230000004048 modification Effects 0.000 claims description 9
- 238000005457 optimization Methods 0.000 claims description 7
- 238000004590 computer program Methods 0.000 claims description 6
- 230000002950 deficient Effects 0.000 claims description 5
- 230000007812 deficiency Effects 0.000 abstract description 2
- 230000000694 effects Effects 0.000 description 12
- 238000010586 diagram Methods 0.000 description 4
- 238000013527 convolutional neural network Methods 0.000 description 2
- 238000006467 substitution reaction Methods 0.000 description 2
- 238000013459 approach Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000004321 preservation Methods 0.000 description 1
- 238000009877 rendering Methods 0.000 description 1
- 230000007847 structural defect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10004—Still image; Photographic image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30176—Document
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Computation (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Health & Medical Sciences (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Biophysics (AREA)
- Biomedical Technology (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Evolutionary Biology (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Computational Linguistics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Quality & Reliability (AREA)
- Image Processing (AREA)
Abstract
The invention provides a text image restoration method, a device and a storage medium, wherein the method comprises the following steps: obtaining a plurality of original medical text images from an image acquisition device, and randomly classifying the plurality of original medical text images to obtain an original training set image and an original testing set image; respectively carrying out image deletion processing on the original training set image and the original test set image to obtain a defect training set image and a defect test set image; constructing a training model, and training the training model according to the defect training set image to obtain a final repair model; and repairing the defect test set image according to the final repairing model to obtain a repairing image result. The invention overcomes the defects of large workload, low efficiency and insufficient accuracy of the traditional manual repair of the text image, can repair the condition of structural deficiency of the text in the image, can repair the incomplete text to a certain reading property, and has a great significance for reading the incomplete text.
Description
Technical Field
The invention mainly relates to the technical field of image restoration, in particular to a text image restoration method, a text image restoration device and a storage medium.
Background
Image restoration originates from european art restoration, a process in which an artist restores a broken artwork. Because artwork is easily affected by factors such as manpower, environment, materials and the like in the preservation process, damage such as cracks, missing, mildewing, falling, scratches and the like can be hardly avoided. In the European literature resumption period, in order to recover damaged or lost parts of the artwork, the overall effect of the artwork is maintained, and the repair of the artwork in the middle century is started. Image restoration is performed by professional artists, and is performed according to the damaged quality and degree of the artwork, so that the original appearance of the artwork is expected to be restored. However, this approach is often risky, as artwork is extremely precious and will not be able to be remedied once an accident occurs.
Image restoration is an important task in computer vision, and has many applications in image-based rendering and computational photography, and the main difficulty in image restoration is how to synthesize visually realistic and semantically reasonable pixels in the missing part of an image, and there is no sense of violation between these pixels and existing pixels.
Disclosure of Invention
The invention aims to solve the technical problem of providing a text image restoration method, a text image restoration device and a storage medium aiming at the defects of the prior art.
The technical scheme for solving the technical problems is as follows: a text image restoration method comprises the following steps:
obtaining a plurality of original medical text images from an image acquisition device, and randomly classifying the plurality of original medical text images to obtain an original training set image and an original testing set image;
respectively carrying out image deletion processing on the original training set image and the original test set image to obtain a defect training set image and a defect test set image;
constructing a training model, and training the training model according to the defect training set image to obtain a final repair model;
and repairing the defect test set image according to the final repairing model to obtain a repairing image result.
The other technical scheme for solving the technical problems is as follows: a text image restoration device comprising:
the random classification processing module is used for obtaining a plurality of original medical text images from the image acquisition equipment, and carrying out random classification on the plurality of original medical text images to obtain an original training set image and an original testing set image;
the defect image processing module is used for respectively carrying out image deletion processing on the original training set image and the original test set image to obtain a defect training set image and a defect test set image;
the training model processing module is used for constructing a training model, and training the training model according to the defect training set image to obtain a final repair model;
and the repair image result obtaining module is used for carrying out repair processing on the defect test set image according to the final repair model to obtain a repair image result.
The beneficial effects of the invention are as follows: the original training set images and the original test set images are obtained by randomly classifying a plurality of original medical text images, the defect training set images and the defect test set images are obtained by carrying out image deletion processing on the original training set images and the original test set images respectively, a training model is built, a final repair model is obtained by training the training model according to the defect training set images, the effect of image repair is further improved, the repair image result is obtained by carrying out repair processing on the defect test set images according to the final repair model, the defects of large workload, low efficiency and insufficient accuracy of the traditional manual repair text images are overcome, the situation that the structure of the text in the images is lost can be repaired, the defect text can be repaired to a certain reading property, and the defect text reading is of great significance.
Drawings
Fig. 1 is a schematic flow chart of a text image restoration method according to an embodiment of the present invention;
fig. 2 is a block diagram of a text image restoration device according to an embodiment of the present invention.
Detailed Description
The principles and features of the present invention are described below with reference to the drawings, the examples are illustrated for the purpose of illustrating the invention and are not to be construed as limiting the scope of the invention.
Fig. 1 is a flow chart of a text image restoration method according to an embodiment of the present invention.
As shown in fig. 1, a text image restoration method includes the following steps:
obtaining a plurality of original medical text images from an image acquisition device, and randomly classifying the plurality of original medical text images to obtain an original training set image and an original testing set image;
respectively carrying out image deletion processing on the original training set image and the original test set image to obtain a defect training set image and a defect test set image;
constructing a training model, and training the training model according to the defect training set image to obtain a final repair model;
and repairing the defect test set image according to the final repairing model to obtain a repairing image result.
Specifically, the plurality of original medical text images are Chinese character images cut from the medical text images, the content of the images is mostly 1 to 4 characters, a total of 56867 images exist, wherein the images corresponding to the same Chinese character are not unique, and the situation that the same Chinese character corresponds to a plurality of images can occur; because the use frequency of each Chinese character in the medical text is not equal, the number of the obtained images is correspondingly increased along with the more frequent use of the Chinese characters, so that each type of images cannot reach the same scale; all images are put into the same folder, the images are randomly selected from the folder and divided into two parts, one part is used as an original training image, the other part is used as an original test image, the original test image is 20% of the number of all images, and the randomness of the images is enhanced after the images are disordered.
In the embodiment, the original training set image and the original test set image are obtained by randomly classifying the plurality of original medical text images, the defect training set image and the defect test set image are obtained by respectively carrying out image deletion processing on the original training set image and the original test set image, a training model is constructed, a final repair model is obtained by training the training model according to the defect training set image, the effect of image repair is further improved, the result of repairing the defect test set image is obtained by carrying out repair processing on the final repair model, the defects of large workload, low efficiency and insufficient accuracy of the traditional manual repair text image are overcome, the situation that the text in the image has structural deletion can be repaired, the defect text can be repaired to have certain reading performance, and the defect text reading is of great significance.
Optionally, as an embodiment of the present invention, the process of performing image deletion processing on the original training set image and the original test set image to obtain a defect training set image and a defect test set image includes:
adding preset binary patches at random positions on each image in the original training set image to obtain a defect training set image;
and adding preset binary patches at random positions on each image in the original test set image to obtain a defect test set image.
In the above embodiment, adding a preset binary patch at a random position on each image in the original training set image to obtain a defect training set image; the defect test set image is obtained by adding preset binary patches at random positions on each image in the original test set image, the objectivity of data can be ensured, the human factors are reduced, the accuracy of a subsequent repair model is effectively improved, the defects of large workload, low efficiency and insufficient accuracy of the traditional manual repair text image are overcome, the condition that the text in the image has structural defects can be repaired, the defective text can be repaired to have certain readability, and the defect text reading method has very great significance.
Optionally, as an embodiment of the present invention, the process of constructing a training model and training the training model according to the defect training set image to obtain a final repair model includes:
constructing a generated countermeasure network, and training the generated countermeasure network by utilizing the defect training set image to obtain a basic repair model;
and optimizing the basic repair model by using an attention mechanism optimization algorithm to obtain a final repair model.
Specifically, attention mechanisms are added in the network to repair Chinese characters with missing structures in the images, and the Chinese characters are formed by one stroke, so that attention mechanisms are more hopefully introduced in the repair process, the network is enabled to pay attention to the missing information in one stroke and the residual information of the stroke originally connected with the missing information, and the repair effect of the repair model is improved; the generating countermeasure network model is divided into a repair network and a discrimination network, and the two networks are all defined as convolutional neural networks; corresponding to two parts: a generator (G) and a Discriminator (D), the generator receiving as input a random noise (an initial distribution), generating an image from the noise and taking it as input to the Discriminator; the nature of the arbiter is a classifier that is used to determine whether the input data is from real data or the generation of the G-network, and the final returned result is the probability that the image is from training data. If the image is from training data, D should return a value near 1, otherwise a value near 0 should be returned to indicate that the image is false, as generated from the G network. Therefore, the basic repair model obtained by the text image repair method based on the attention mechanism and the generated countermeasure network can effectively improve the accuracy and effect of image repair, and the obtained final repair model can further improve the effect of image repair by adding the attention mechanism to the basic repair model to optimize, so that the incomplete text can be repaired to have certain reading performance.
In the above embodiment, the training of the generated countermeasure network by using the defect training set image is performed to obtain a basic repair model; the final repair model is obtained by optimizing the basic repair model through an attention mechanism optimization algorithm, so that the accuracy and the effect of image repair can be effectively improved, the effect of image repair can be further improved, the defect that the traditional manual repair text image has a large workload, low efficiency and insufficient accuracy is overcome, the situation that the text in the image has structural deficiency can be repaired, the defect text can be repaired to have a certain readability, and the method has a very great significance for reading the defect text.
Optionally, as an embodiment of the present invention, the training the generating an countermeasure network using the defect training set image, the process of obtaining a basic repair model includes:
s1: inputting the defect training set image into the generating countermeasure network for training according to preset iterative training times to obtain a first repair model, wherein the first repair model comprises a first repair network and a first discrimination network;
s2: repairing the defect training set image through the first repairing network to obtain a first intermediate repairing picture;
s3: judging the first intermediate restoration picture through the first judging network to obtain a first predicted value;
s4: judging whether the first predicted value is larger than or equal to a preset judging value, if so, taking the first repair model as the basic repair model; otherwise, executing the step S5;
s5: inputting the defect training set image into the generating countermeasure network for training according to preset iterative training times to obtain a second repair model, wherein the second repair model comprises a second repair network and a second discrimination network;
s6: repairing the defect training set image through the second repairing network to obtain a second intermediate repairing picture;
s7: judging the second intermediate restoration picture through the second judging network to obtain a second predicted value;
s8: judging whether the second predicted value is larger than or equal to a preset judging value, and if so, taking the second repair model as the basic repair model; if so, returning to the step S1.
It should be understood that the architecture of the generated countermeasure network is divided into a repair network and a discrimination network, and the two networks are all defined as convolutional neural networks.
In the above embodiment, the first repair model and the second repair model are obtained according to the defect training set image and the generated countermeasure network model, and the basic repair model is obtained according to whether the preset judgment value is satisfied or not, so that the expected repair model can be obtained while higher repair efficiency is ensured, and the stability and reliability of text image repair are realized.
Optionally, as an embodiment of the present invention, the optimizing the basic repair model by using an attention mechanism optimization algorithm, and the process of obtaining a final repair model includes:
the basic repair model comprises an original first-stage repair network, an original second-stage repair network and a discrimination network, wherein the original first-stage repair network comprises a 3 multiplied by 3 convolution layer, a downsampling layer, a cavity convolution layer and an upsampling layer;
before adding a preset 1×1 convolution layer to the 3×3 convolution layer, obtaining a first-stage repair network;
performing network branch modification processing on the original second-stage repair network by using an attention mechanism algorithm to obtain a second-stage repair network;
and combining the first-stage repair network, the second-stage repair network and the discrimination network according to the sequence to obtain a final repair model.
It should be appreciated that the original first-stage repair network alters the convolutional layer structure before the hole convolutional layer, adding several 1 x 1 convolutional layers, increasing nonlinearity and reducing training time.
Specifically, the original first-stage repairing network comprises a 3×3 convolution layer, a downsampling layer, a hole convolution layer and an upsampling layer, and the first-stage repairing network obtained by adding the 1×1 convolution layer comprises the 1×1 convolution layer, the 3×3 convolution layer, the downsampling layer, the hole convolution layer for expanding the receptive field and the upsampling layer for restoring and outputting the picture.
In the above embodiment, the first-stage repair network is obtained before adding the preset 1×1 convolutional layer to the 3×3 convolutional layer; modifying the network branch of the original second-stage repair network by using an attention mechanism algorithm to obtain a second-stage repair network; the final repair model is obtained by combining the first-stage repair network, the second-stage repair network and the discrimination network according to the sequence, so that nonlinearity is increased, training time is reduced, accuracy and effect of image repair can be effectively improved, the effect of image repair is further improved, defective texts can be repaired to have certain readability, the defects of large workload, low efficiency and insufficient accuracy of traditional manual text image repair are overcome, the situation that the structure of the texts in the images is missing can be repaired, defective texts can be repaired to have certain readability, and the method has great significance for defective text reading.
Optionally, as an embodiment of the present invention, the process of performing network branch modification processing on the original second-stage repair network by using an attention mechanism algorithm to obtain a second-stage repair network includes:
the original second-stage repair network comprises two convolution branches;
performing convolution branch modification on any convolution branch in the original second-stage repair network by using an attention mechanism algorithm to obtain an attention branch;
and combining the attention branch with another convolution branch to obtain the second-stage repair network.
Specifically, a branch is changed by introducing an attention mechanism, one convolution branch acts on a network structure before hole convolution, a combination structure of convolution and a downsampling layer is adopted, a hole convolution layer is added finally, the attention layer is used for replacing the hole convolution in the other branch, the two convolution layers are added, after output results of the two branches are combined, operations such as convolution upsampling layer and the like are carried out to restore a picture to an input size, and a final restoration result diagram is obtained.
In the above embodiment, the attention mechanism algorithm is utilized to modify the convolution branch of any convolution branch in the original second-stage repair network to obtain an attention branch; the second-stage repair network is obtained by combining the attention branch and the other convolution branch, so that the accuracy and the effect of image repair can be effectively improved, the effect of image repair can be further improved, the defect that the traditional manual repair text image has large workload, low efficiency and insufficient accuracy is overcome, the situation that the structure of the text in the image is missing can be repaired, the defect text can be repaired to have certain readability, and the method has very deep significance for reading the defect text.
Optionally, as another embodiment of the present invention, a batch of training samples is randomly selected from the training set, a binary patch is added to a random position selected from the sample image to obtain a damaged image, 128×128 images with patch sizes of maximum size of 64×64 are set in the network, and the batch size is set to 16; the repair image takes the damaged image and the binary patch together as input data, the input data is put into a repair model for repair, the model comprises a generator and a discriminator, two repair stages are arranged in the generator, the input data enters a first stage repair network, required characteristics are obtained through steps of convolution, deconvolution and the like after the input data is input into the network, a preliminary repair image is obtained according to the characteristics, the preliminary repair image is input into a second stage repair network to complete a subsequent repair task, in order to improve repair efficiency, an attention mechanism is applied, convolution branches and attention branches are combined, one branch acts in a network structure before cavity convolution, a combination structure of convolution and a downsampling layer is adopted, finally a cavity convolution layer is added, the attention layer replaces cavity convolution in the other branch, after the output results of the two branches are combined, operations such as convolution upsampling layer and the like are carried out again to restore the image to the input size, and a final repair result diagram is obtained; at this time, the output result is the repair result of the whole network, and a clear repair image can be obtained.
Fig. 2 is a block diagram of a text image restoration device according to an embodiment of the present invention.
Alternatively, as another embodiment of the present invention, as shown in fig. 2, a text image restoration apparatus includes:
the random classification processing module is used for obtaining a plurality of original medical text images from the image acquisition equipment, and carrying out random classification on the plurality of original medical text images to obtain an original training set image and an original testing set image;
the defect image processing module is used for respectively carrying out image deletion processing on the original training set image and the original test set image to obtain a defect training set image and a defect test set image;
the training model processing module is used for constructing a training model, and training the training model according to the defect training set image to obtain a final repair model;
and the repair image result obtaining module is used for carrying out repair processing on the defect test set image according to the final repair model to obtain a repair image result.
Optionally, as an embodiment of the present invention, the defect image processing module is specifically configured to:
and respectively carrying out random position addition on the original training set image and the original test set image according to a preset binary patch to obtain a defect training set image and a defect test set image, wherein the defect training set image and the defect test set image both carry the preset binary patch.
Alternatively, another embodiment of the present invention provides a text image restoration apparatus including a memory, a processor, and a computer program stored in the memory and executable on the processor, which when executed by the processor, implements the text image restoration method as described above. The device may be a computer or the like.
Alternatively, another embodiment of the present invention provides a computer-readable storage medium storing a computer program which, when executed by a processor, implements the text image restoration method as described above.
It will be clear to those skilled in the art that, for convenience and brevity of description, specific working procedures of the apparatus and units described above may refer to corresponding procedures in the foregoing method embodiments, which are not described herein again.
In the several embodiments provided in this application, it should be understood that the disclosed apparatus and method may be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative, e.g., the division of elements is merely a logical functional division, and there may be additional divisions of actual implementation, e.g., multiple elements or components may be combined or integrated into another system, or some features may be omitted, or not performed.
The units described as separate units may or may not be physically separate, and units shown as units may or may not be physical units, may be located in one place, or may be distributed over a plurality of network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the embodiment of the present invention.
In addition, each functional unit in the embodiments of the present invention may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit. The integrated units may be implemented in hardware or in software functional units.
The integrated units, if implemented in the form of software functional units and sold or used as stand-alone products, may be stored in a computer readable storage medium. For such understanding, the technical solution of the present invention is essentially or part of what contributes to the prior art, or all or part of the technical solution may be embodied in the form of a software product stored in a storage medium, comprising several instructions for causing a computer device (which may be a personal computer, a server, a network device, etc.) to perform all or part of the steps of the method of the various embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), a magnetic disk, or an optical disk, or other various media capable of storing program codes.
The present invention is not limited to the above embodiments, and various equivalent modifications and substitutions can be easily made by those skilled in the art within the technical scope of the present invention, and these modifications and substitutions are intended to be included in the scope of the present invention. Therefore, the protection scope of the invention is subject to the protection scope of the claims.
Claims (7)
1. The text image restoration method is characterized by comprising the following steps of:
obtaining a plurality of original medical text images from an image acquisition device, and randomly classifying the plurality of original medical text images to obtain an original training set image and an original testing set image;
respectively carrying out image deletion processing on the original training set image and the original test set image to obtain a defect training set image and a defect test set image;
constructing a training model, and training the training model according to the defect training set image to obtain a final repair model;
repairing the defect test set image according to the final repairing model to obtain a repairing image result;
the process of constructing a training model and training the training model according to the defect training set image to obtain a final repair model comprises the following steps:
constructing a generated countermeasure network, and training the generated countermeasure network by utilizing the defect training set image to obtain a basic repair model;
optimizing the basic repair model by using an attention mechanism optimization algorithm to obtain a final repair model;
the process of training the generated countermeasure network by using the defect training set image to obtain a basic repair model comprises the following steps:
s1: inputting the defect training set image into the generating countermeasure network for training according to preset iterative training times to obtain a first repair model, wherein the first repair model comprises a first repair network and a first discrimination network;
s2: repairing the defect training set image through the first repairing network to obtain a first intermediate repairing picture;
s3: judging the first intermediate restoration picture through the first judging network to obtain a first predicted value;
s4: judging whether the first predicted value is larger than or equal to a preset judging value, if so, taking the first repair model as the basic repair model; otherwise, executing the step S5;
s5: inputting the defect training set image into the generating countermeasure network for training according to preset iterative training times to obtain a second repair model, wherein the second repair model comprises a second repair network and a second discrimination network;
s6: repairing the defect training set image through the second repairing network to obtain a second intermediate repairing picture;
s7: judging the second intermediate restoration picture through the second judging network to obtain a second predicted value;
s8: judging whether the second predicted value is larger than or equal to a preset judging value, and if so, taking the second repair model as the basic repair model; if the number is smaller than the preset number, returning to the step S1;
the process for optimizing the basic repair model by using the attention mechanism optimization algorithm to obtain a final repair model comprises the following steps:
the basic repair model comprises an original first-stage repair network, an original second-stage repair network and a discrimination network, wherein the original first-stage repair network comprises a 3 multiplied by 3 convolution layer, a downsampling layer, a cavity convolution layer and an upsampling layer;
before adding a preset 1×1 convolution layer to the 3×3 convolution layer, obtaining a first-stage repair network;
performing network branch modification processing on the original second-stage repair network by using an attention mechanism algorithm to obtain a second-stage repair network;
and combining the first-stage repair network, the second-stage repair network and the discrimination network according to the sequence to obtain a final repair model.
2. The text image restoration method according to claim 1, wherein the process of performing image deletion processing on the original training set image and the original test set image to obtain a defect training set image and a defect test set image includes:
adding preset binary patches at random positions on each image in the original training set image to obtain a defect training set image;
and adding preset binary patches at random positions on each image in the original test set image to obtain a defect test set image.
3. The text image restoration method according to claim 1, wherein the process of performing network branch modification processing on the original second-stage restoration network by using an attention mechanism algorithm to obtain a second-stage restoration network includes:
the original second-stage repair network comprises two convolution branches;
performing convolution branch modification on any convolution branch in the original second-stage repair network by using an attention mechanism algorithm to obtain an attention branch;
and combining the attention branch with another convolution branch to obtain the second-stage repair network.
4. A text image restoration device, comprising:
the random classification processing module is used for obtaining a plurality of original medical text images from the image acquisition equipment, and carrying out random classification on the plurality of original medical text images to obtain an original training set image and an original testing set image;
the defect image processing module is used for respectively carrying out image deletion processing on the original training set image and the original test set image to obtain a defect training set image and a defect test set image;
the training model processing module is used for constructing a training model, and training the training model according to the defect training set image to obtain a final repair model;
the repair image result obtaining module is used for carrying out repair treatment on the defect test set image according to the final repair model to obtain a repair image result;
the training model processing module is specifically used for:
constructing a generated countermeasure network, and training the generated countermeasure network by utilizing the defect training set image to obtain a basic repair model;
optimizing the basic repair model by using an attention mechanism optimization algorithm to obtain a final repair model;
in the training model processing module, the process of training the generated countermeasure network by using the defect training set image to obtain a basic repair model comprises the following steps:
s1: inputting the defect training set image into the generating countermeasure network for training according to preset iterative training times to obtain a first repair model, wherein the first repair model comprises a first repair network and a first discrimination network;
s2: repairing the defect training set image through the first repairing network to obtain a first intermediate repairing picture;
s3: judging the first intermediate restoration picture through the first judging network to obtain a first predicted value;
s4: judging whether the first predicted value is larger than or equal to a preset judging value, if so, taking the first repair model as the basic repair model; otherwise, executing the step S5;
s5: inputting the defect training set image into the generating countermeasure network for training according to preset iterative training times to obtain a second repair model, wherein the second repair model comprises a second repair network and a second discrimination network;
s6: repairing the defect training set image through the second repairing network to obtain a second intermediate repairing picture;
s7: judging the second intermediate restoration picture through the second judging network to obtain a second predicted value;
s8: judging whether the second predicted value is larger than or equal to a preset judging value, and if so, taking the second repair model as the basic repair model; if the number is smaller than the preset number, returning to the step S1;
in the training model processing module, the process of optimizing the basic repair model by using an attention mechanism optimization algorithm to obtain a final repair model comprises the following steps:
the basic repair model comprises an original first-stage repair network, an original second-stage repair network and a discrimination network, wherein the original first-stage repair network comprises a 3 multiplied by 3 convolution layer, a downsampling layer, a cavity convolution layer and an upsampling layer;
before adding a preset 1×1 convolution layer to the 3×3 convolution layer, obtaining a first-stage repair network;
performing network branch modification processing on the original second-stage repair network by using an attention mechanism algorithm to obtain a second-stage repair network;
and combining the first-stage repair network, the second-stage repair network and the discrimination network according to the sequence to obtain a final repair model.
5. The text image restoration device as defined in claim 4, wherein said defective image processing module is specifically configured to:
and respectively carrying out random position addition on the original training set image and the original test set image according to a preset binary patch to obtain a defect training set image and a defect test set image, wherein the defect training set image and the defect test set image both carry the preset binary patch.
6. A text image restoration device comprising a memory, a processor and a computer program stored in said memory and executable on said processor, characterized in that the text image restoration method according to any one of claims 1 to 3 is implemented when said computer program is executed by said processor.
7. A computer readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the text image restoration method according to any one of claims 1 to 3.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010703681.3A CN111899191B (en) | 2020-07-21 | 2020-07-21 | Text image restoration method, device and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010703681.3A CN111899191B (en) | 2020-07-21 | 2020-07-21 | Text image restoration method, device and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111899191A CN111899191A (en) | 2020-11-06 |
| CN111899191B true CN111899191B (en) | 2024-01-26 |
Family
ID=73189674
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010703681.3A Active CN111899191B (en) | 2020-07-21 | 2020-07-21 | Text image restoration method, device and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111899191B (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112560732B (en) * | 2020-12-22 | 2023-07-04 | 电子科技大学中山学院 | Feature extraction method of multi-scale feature extraction network |
| CN114742170B (en) * | 2022-04-22 | 2023-07-25 | 马上消费金融股份有限公司 | Countermeasure sample generation method, model training method, image recognition method and device |
| CN115797216B (en) * | 2022-12-14 | 2024-05-24 | 齐鲁工业大学 | Self-coding network-based steganography character restoration model and restoration method |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102283742A (en) * | 2011-08-15 | 2011-12-21 | 重庆大学 | Light stimulation based retinal prosthesis repairing device and repairing method |
| CN108830827A (en) * | 2017-05-02 | 2018-11-16 | 通用电气公司 | Neural metwork training image generation system |
| CN109003253A (en) * | 2017-05-24 | 2018-12-14 | 通用电气公司 | Neural network point cloud generates system |
| CN109377448A (en) * | 2018-05-20 | 2019-02-22 | 北京工业大学 | A kind of facial image restorative procedure based on generation confrontation network |
| CN109559287A (en) * | 2018-11-20 | 2019-04-02 | 北京工业大学 | A kind of semantic image restorative procedure generating confrontation network based on DenseNet |
| CN111161158A (en) * | 2019-12-03 | 2020-05-15 | 河海大学 | Image restoration method based on newly-formed network structure |
-
2020
- 2020-07-21 CN CN202010703681.3A patent/CN111899191B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102283742A (en) * | 2011-08-15 | 2011-12-21 | 重庆大学 | Light stimulation based retinal prosthesis repairing device and repairing method |
| CN108830827A (en) * | 2017-05-02 | 2018-11-16 | 通用电气公司 | Neural metwork training image generation system |
| CN109003253A (en) * | 2017-05-24 | 2018-12-14 | 通用电气公司 | Neural network point cloud generates system |
| CN109377448A (en) * | 2018-05-20 | 2019-02-22 | 北京工业大学 | A kind of facial image restorative procedure based on generation confrontation network |
| CN109559287A (en) * | 2018-11-20 | 2019-04-02 | 北京工业大学 | A kind of semantic image restorative procedure generating confrontation network based on DenseNet |
| CN111161158A (en) * | 2019-12-03 | 2020-05-15 | 河海大学 | Image restoration method based on newly-formed network structure |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111899191A (en) | 2020-11-06 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111899191B (en) | Text image restoration method, device and storage medium | |
| CN108520503A (en) | A method of based on self-encoding encoder and generating confrontation network restoration face Incomplete image | |
| Lubenko et al. | Going from small to large data in steganalysis | |
| CN112088378A (en) | Image hidden information detector | |
| TW201841504A (en) | Method and device for constructing the sensing of video compression | |
| CN113183759A (en) | Method and device for displaying characters of instrument panel | |
| CN111461211A (en) | Feature extraction method for lightweight target detection and corresponding detection method | |
| CN113468905B (en) | Graphic code identification method, graphic code identification device, computer equipment and storage medium | |
| CN112200789B (en) | Image recognition method and device, electronic equipment and storage medium | |
| CN113902671A (en) | Image steganography method and system based on random texture | |
| CN111191584B (en) | Face recognition method and device | |
| CN116049644A (en) | Feature screening and clustering and binning method and device, electronic equipment and storage medium | |
| Wang et al. | Mixed distortion image enhancement method based on joint of deep residuals learning and reinforcement learning | |
| Ding et al. | [Retracted] Digital Image Restoration Based on Multicontour Batch Scanning | |
| CN104516899A (en) | Updating method and device for word stock | |
| CN113379637A (en) | Image restoration method, system, medium, and device based on progressive learning strategy | |
| CN112102205A (en) | Image deblurring method and device, electronic equipment and storage medium | |
| CN112435196B (en) | Character repairing method and system based on deep learning | |
| Grommelt et al. | Fake or JPEG? Revealing Common Biases in Generated Image Detection Datasets | |
| CN115115537B (en) | Image restoration method based on mask training | |
| Thomas et al. | Analysis of image inpainting and object removal methodologies | |
| CN113610704B (en) | Image generation method, device, equipment and readable storage medium | |
| CN110716739B (en) | Code change information statistical method, system and readable storage medium | |
| CN117113010B (en) | Power transmission channel safety monitoring method and system based on convolutional network lightweight | |
| CN118097089A (en) | Night warehousing robot target detection method and system based on integral network |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |