CN109064408A - A kind of method and device of multi-scale image super-resolution rebuilding - Google Patents
A kind of method and device of multi-scale image super-resolution rebuilding Download PDFInfo
- Publication number
- CN109064408A CN109064408A CN201811133627.9A CN201811133627A CN109064408A CN 109064408 A CN109064408 A CN 109064408A CN 201811133627 A CN201811133627 A CN 201811133627A CN 109064408 A CN109064408 A CN 109064408A
- Authority
- CN
- China
- Prior art keywords
- resolution
- image
- scale
- super
- residual error
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 50
- 238000013528 artificial neural network Methods 0.000 claims description 10
- 230000006870 function Effects 0.000 claims description 10
- 238000005070 sampling Methods 0.000 claims description 7
- 238000004590 computer program Methods 0.000 claims description 6
- 230000004913 activation Effects 0.000 claims description 4
- 230000006835 compression Effects 0.000 claims description 4
- 238000007906 compression Methods 0.000 claims description 4
- 230000001052 transient effect Effects 0.000 claims description 3
- 238000003475 lamination Methods 0.000 claims description 2
- 210000004218 nerve net Anatomy 0.000 claims 1
- 239000010410 layer Substances 0.000 description 36
- 238000010586 diagram Methods 0.000 description 13
- 238000013135 deep learning Methods 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 2
- 230000003321 amplification Effects 0.000 description 1
- 230000003416 augmentation Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000003199 nucleic acid amplification method Methods 0.000 description 1
- 238000000926 separation method Methods 0.000 description 1
- 239000002356 single layer Substances 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
- G06T3/4053—Scaling of whole images or parts thereof, e.g. expanding or contracting based on super-resolution, i.e. the output image resolution being higher than the sensor resolution
Landscapes
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Image Processing (AREA)
Abstract
The embodiment of the invention provides a kind of multi-scale image super resolution ratio reconstruction method and devices, utilize the multi-scale image Super-resolution reconstruction established model based on residual error structure and reconstruction structure, it is multiple dimensioned second resolution image by first resolution image procossing, second resolution is greater than first resolution.First resolution image is thus capable of sufficiently recovering high frequency detail using more further features by study residual error；First resolution image rebuilds structure by entering after multilayer residual error structure, rebuilds structure and becomes large-sized in the case where keeping the image resolution ratio passed through constant, to export the second resolution image of different scale.For the second resolution image of different scale, it only need to residual error structure Jing Guo the corresponding number of plies, residual error structure without the second resolution image for different scale all by the fixed number of plies, has saved the processing time, the efficiency of image super-resolution is improved under the premise of guaranteeing image definition.
Description
Technical field
The present embodiments relate to field of image processing more particularly to a kind of multi-scale image super resolution ratio reconstruction method and
Device.
Background technique
The super-resolution rebuilding of traditional low-resolution image mainly uses interpolation method, such as nearest first method, bilinear interpolation
Method and three times interpolation method etc..These methods are calculate with points several around it approaching for the value of each pixel on image
It obtains, and these images obtained using the method for interpolation method are excessively smooth, are lost many high frequency details, therefore cause final
Obtained high-definition picture cannot retain the details of original image well.
Currently, there are also the methods based on deep learning for the super-resolution rebuilding of low-resolution image, it is main using big
The high score rate image of amount establishes learning database and generates learning model, introduces during restoring to low point of annual rate image by learning
The priori knowledge that model obtains is practised, to obtain the high frequency detail of image.But the current this method based on deep learning is used only
The superficial feature of low-resolution image, for low resolution further feature with less.
But the super resolution ratio reconstruction method of the current low-resolution image based on deep learning is generally only capable of obtaining list
The high-definition picture of scale, if desired obtains multiple dimensioned high-definition picture, then needs to establish the super of multiple and different scales
Resolution model obtains multiple dimensioned high-definition picture using the super-resolution model of different scale, and process is cumbersome；Or it utilizes
Single model obtains multiple dimensioned high-definition picture, cannot obtain multiple dimensioned high-definition picture simultaneously, and efficiency is lower.
Summary of the invention
In order to solve that multiple dimensioned height cannot be obtained simultaneously for the super resolution ratio reconstruction method of low-resolution image at present
The problem of image in different resolution, overcomes the above problem the embodiment of the invention provides one kind or at least is partially solved the above problem
Multi-scale image super resolution ratio reconstruction method.
According to a first aspect of the embodiments of the present invention, a kind of multi-scale image super resolution ratio reconstruction method, the party are provided
Method includes: to obtain first resolution image；First resolution image is input to trained multi-scale image Super-resolution reconstruction
Established model exports multiple dimensioned second resolution image；Multi-scale image Super-resolution reconstruction established model is based on second resolution
Image pattern and corresponding first resolution image pattern obtain after being trained, and second resolution is greater than first and differentiates
Rate；Multi-scale image Super-resolution reconstruction established model is the multilayer neural network based on residual error structure and reconstruction structure.
According to a second aspect of the embodiments of the present invention, a kind of multi-scale image super-resolution rebuilding device, the dress are provided
Setting includes: acquisition module, for obtaining first resolution image；Processing module, for first resolution image to be input to instruction
The multi-scale image Super-resolution reconstruction established model perfected, exports multiple dimensioned second resolution image；Multi-scale image super-resolution
Rate reconstruction model is to obtain after being trained based on second resolution image pattern and corresponding first resolution image pattern
It arrives, second resolution is greater than first resolution；Multi-scale image Super-resolution reconstruction established model is based on residual error structure and to rebuild knot
The multilayer neural network of structure.
According to a third aspect of the embodiments of the present invention, provide a kind of electronic equipment, the electronic equipment include: memory,
Processor and storage are on a memory and the computer program that can run on a processor, processor caller are instructed and can be held
Multi-scale image super-resolution provided by any possible implementation in the various possible implementations of row first aspect
Rate method for reconstructing.
According to a fourth aspect of the embodiments of the present invention, a kind of non-transient computer readable storage medium is provided, is deposited thereon
Contain computer program, the computer program make computer execute first aspect various possible implementations in it is any can
Multi-scale image super resolution ratio reconstruction method provided by the implementation of energy.
The embodiment of the invention provides a kind of multi-scale image super resolution ratio reconstruction method and devices, using based on residual error knot
Structure and the multi-scale image Super-resolution reconstruction established model for rebuilding structure, it is defeated by multilayer residual error structure and corresponding different scale images
First resolution image procossing is the second resolution image of different scale by the network structure of reconstruction structure out.First point
Resolution image passes through after multilayer residual error structure, and while size constancy, resolution ratio increases, and each layer of residual error structure is defeated
Enter it is related with outputting and inputting for upper one layer of residual error structure, make first resolution image by study residual error, use first point
The further feature of resolution image is thus capable of sufficiently recovering the high frequency detail of first resolution image, preferably rebuilds to image；First
Image in different resolution rebuilds structure by entering after multilayer residual error structure, rebuilds structure and is keeping the image resolution ratio passed through constant
In the case where become large-sized, to export the second resolution image of different scale.For the second resolution figure of different scale
Picture, only need to residual error structure Jing Guo the corresponding number of plies, the second resolution image without being directed to different scale all passes through fixing layer
Several residual error structures, has saved the processing time, and the efficiency of image super-resolution is improved under the premise of guaranteeing image definition,
It can obtain the second resolution image of different scale simultaneously, it is easy to operate, use scope is wide.
Detailed description of the invention
Fig. 1 is the process according to a kind of multi-scale image super resolution ratio reconstruction method of a preferred embodiment of the invention
Schematic diagram；
Fig. 2 is the structure according to a kind of multi-scale image super-resolution rebuilding device of a preferred embodiment of the invention
Schematic diagram；
Fig. 3 is the structural schematic diagram according to a kind of electronic equipment of a preferred embodiment of the invention；
Fig. 4 is the structure according to a kind of multi-scale image Super-resolution reconstruction established model of a preferred embodiment of the invention
Schematic diagram；
Fig. 5 is the structural schematic diagram according to the residual error structure of a preferred embodiment of the invention；
Fig. 6 is the structural schematic diagram according to the reconstruction structure of a preferred embodiment of the invention.
Specific embodiment
With reference to the accompanying drawings and examples, specific embodiments of the present invention will be described in further detail.Implement below
Example is not intended to limit the scope of the invention for illustrating the present invention.
Currently, the super resolution ratio reconstruction method of the low-resolution image based on deep learning is generally only capable of obtaining single scale
High-definition picture, if desired obtains multiple dimensioned high-definition picture, then needs to establish the super-resolution of multiple and different scales
Model obtains multiple dimensioned high-definition picture using the super-resolution model of different scale, cannot obtain simultaneously multiple dimensioned
High-definition picture, or multiple dimensioned high-definition picture is obtained using single model, multiple dimensioned high score cannot be obtained simultaneously
Resolution image, efficiency is lower, process is cumbersome.
Fig. 1 is the process according to a kind of multi-scale image super resolution ratio reconstruction method of a preferred embodiment of the invention
Schematic diagram, Fig. 4 are the structure according to a kind of multi-scale image Super-resolution reconstruction established model of a preferred embodiment of the invention
Schematic diagram, as shown in Figure 1 and Figure 4, the embodiment of the invention provides a kind of multi-scale image super resolution ratio reconstruction methods, comprising:
S101, first resolution image is obtained.
Specifically, in step s101, it is necessary first to obtain first resolution image to be reconstructed, first resolution image
Refer to that needing to input multi-scale image Super-resolution reconstruction established model is handled, therefore desirable for the second resolution image pair obtained
The original image answered.
S102, first resolution image are input to trained multi-scale image Super-resolution reconstruction established model, export more rulers
The second resolution image of degree, second resolution are greater than first resolution；Multi-scale image Super-resolution reconstruction established model be based on
Second resolution image pattern and corresponding first resolution image pattern obtain after being trained；Multi-scale image oversubscription
Resolution reconstruction model is the multilayer neural network based on residual error structure and reconstruction structure.
Specifically, in step s 102, the training data of multi-scale image Super-resolution reconstruction established model be based on in advance really
The corresponding first resolution image pattern of fixed second resolution image pattern and predetermined second resolution image sample
This, every group of training data is based on an at least second resolution image pattern and at least one and second resolution image pattern phase
Corresponding first resolution image pattern.
Further, first resolution image pattern is passed through into multilayer residual error network, improves first resolution image pattern
Resolution ratio, make first resolution image pattern in the case where size constancy clarity improve, then undergo reconstruction structure, expand
Big picture size exports the second resolution image pattern of single size.For various sizes of second resolution image pattern,
It needs to utilize reconstruction structure corresponding with the size.
Can be by the image procossing of low resolution by the multi-scale image Super-resolution reconstruction established model obtained after training
Therefore first resolution image is input to trained multi-scale image super-resolution by the high-resolution image of different scale
Rate reconstruction model can export multiple dimensioned second resolution image.
It should be noted that the number of plies of multilayer residual error structure is 2 in Fig. 4, but actually the number of plies is not limited to, can basis
The actual conditions of image procossing determine.
Multi-scale image super resolution ratio reconstruction method provided in an embodiment of the present invention can obtain the second of different scale simultaneously
Image in different resolution, and be directed to the second resolution image of different scale, only need to residual error structure Jing Guo the corresponding number of plies, without needle
The residual error structure for all passing through the fixed number of plies to the second resolution image of different scale, has saved the processing time, is guaranteeing image
The efficiency of image super-resolution is improved under the premise of clarity, can obtain the second resolution image of different scale simultaneously, is grasped
Make convenient and efficient, improves the super-resolution rebuilding efficiency of multi-scale image.
Further, the input of every layer of reconstruction structure is the output of upper one layer of residual error structure, the output of every layer of reconstruction structure
For the second resolution image of single size.
Specifically, the second resolution image of single size need to only be obtained by a corresponding reconstruction structure, i.e., single
The second resolution image of size is the output that the multilayer residual error structure that sequence carries out and single layer rebuild structure.
Based on the above embodiment, the acquisition of the training data of multi-scale image Super-resolution reconstruction established model includes following step
It is rapid:
S1, down-sampling is carried out to preset second resolution image pattern by the down-sampling function in OpenCV,
Noise is added, and carries out JPEG compression, obtains first resolution image pattern corresponding with second resolution image pattern；
S2, the region that the second Pixel Dimensions are cut randomly in second resolution image pattern, and cut first resolution
The region of the first Pixel Dimensions corresponding with the region of the second Pixel Dimensions in image pattern；The ruler of second resolution image pattern
The ratio of very little and first resolution image pattern size is identical as the ratio of the second Pixel Dimensions and the first Pixel Dimensions；By
The region of one Pixel Dimensions and the region of the second Pixel Dimensions are as training data.
For example, second resolution image pattern is contracted to 1/2 size using the down-sampling function in OpenCV, then to
The second resolution image pattern of 1/2 size is added noise and carries out JPEG compression, obtains and second resolution image pattern pair
The first resolution image pattern answered；The region of the Pixel Dimensions of 96*96 in second resolution image pattern is cut randomly, then
Cut 48* corresponding with the region of Pixel Dimensions of 96*96 in second resolution image pattern in first resolution image pattern
The region of 48 Pixel Dimensions, by the region of the Pixel Dimensions of 48*48 and second resolution in multiple groups first resolution image pattern
Training data of the region of 96*96 Pixel Dimensions as multi-scale image Super-resolution reconstruction established model in image pattern, training are more
Scale image super-resolution rebuilding model, to be reconstructed first can be differentiated by having multi-scale image Super-resolution reconstruction established model
The function that 2 times of rate image augmentation.Wherein, if correspondence refers to second resolution image pattern and first resolution image pattern size
Unanimously, then the region that the region cut randomly in the second resolution image pattern and first resolution image pattern are cut is in image
Same place region.
In another example second resolution image pattern is contracted to 1/3 size using the down-sampling function in OpenCV, then
Noise is added to the second resolution image pattern of 1/3 size and carries out JPEG compression, obtains and second resolution image pattern
Corresponding first resolution image pattern；The region of the Pixel Dimensions of 144*144 in second resolution image pattern is cut randomly,
Then it cuts corresponding with the region of Pixel Dimensions of 144*144 in second resolution image pattern in first resolution image pattern
48*48 Pixel Dimensions region, by the region of the Pixel Dimensions of 48*48 in multiple groups first resolution image pattern and second point
Training data of the region of 144*144 Pixel Dimensions as multi-scale image Super-resolution reconstruction established model in resolution image pattern,
Training multi-scale image Super-resolution reconstruction established model, having multi-scale image Super-resolution reconstruction established model can be by be reconstructed the
One image in different resolution expands 3 times of function.
And so on, the region of the first Pixel Dimensions in the second resolution image pattern of multiple is corresponded to by multiple groups difference
Region with the second Pixel Dimensions in corresponding first resolution image pattern is as training data, to multi-scale image super-resolution
Rate reconstruction model is trained, enable multi-scale image Super-resolution reconstruction established model by first resolution image procossing for difference
The full resolution pricture of scale.
In the embodiment of the present invention, the training data of multi-scale image Super-resolution reconstruction established model is preset second point
The region of the second Pixel Dimensions in resolution image pattern, and by second resolution image pattern down-sampling and carry out data increasing
The region of the first Pixel Dimensions in first resolution image pattern operated by force.In order to enable the region of the first Pixel Dimensions exists
Locating region in first resolution image pattern, with the regions of the second Pixel Dimensions in second resolution image pattern locating area
Domain is corresponding, by the ratio of the first Pixel Dimensions and the first Pixel Dimensions and the size of second resolution image pattern and first point
The ratio of the size of resolution image pattern is set as identical.Due to the region of the second Pixel Dimensions in second resolution image pattern,
The region of the first Pixel Dimensions is any selection in first resolution image pattern, so that by multiple groups training data
First resolution image reconstruction to be reconstructed can be the second of different scale by training, multi-scale image Super-resolution reconstruction established model
Image in different resolution.
Fig. 4 is the structure according to a kind of multi-scale image Super-resolution reconstruction established model of a preferred embodiment of the invention
Schematic diagram, Fig. 5 are the structural schematic diagram according to the residual error structure of a preferred embodiment of the invention, as shown in Figure 4 and Figure 5,
The sum of the output of upper one layer of residual error structure and the input of upper one layer of residual error structure are the input of next layer of residual error structure.Wherein, residual
Poor structure includes the first convolutional layer, nonlinear activation function layer and the second convolutional layer that sequence carries out.
Specifically, in each layer of residual error structure include the first convolutional layer, nonlinear activation function layer and that sequence carries out
Two convolutional layers, the sum of the input of the first convolutional layer of upper one layer of residual error structure and the output of the second convolutional layer is residual as next layer
The input of poor structure.First resolution image passes through after residual error structure, and while size constancy, resolution ratio increases, and every
The input of one layer of residual error structure is related with outputting and inputting for upper one layer of residual error structure, and first resolution image is residual by learning
Difference can preferably rebuild image.
Fig. 6 is according to the structural schematic diagram of the reconstruction structure of a preferred embodiment of the invention, as shown in fig. 6, rebuilding
Structure includes third convolutional layer, super-pixel layer and the Volume Four lamination that sequence carries out, and super-pixel layer is for expanding third convolutional layer
Output object size.
Specifically, super-pixel layer is the layer to image amplification, the first convolutional layer, nonlinear activation function in residual error structure
Layer and the second convolutional layer, and the third convolutional layer rebuild in structure are not changed the size of image, therefore are passing through
It crosses before super-pixel layer, the size of image and input image size at the beginning are consistent, only when by super-pixel layer
Picture size just carries out the expansion of different scale, this sampled images still maintains original size before entering super-pixel layer, calculates
It measures not too large, reduces the calculation amount of entire neural network, realization is rapidly rebuild.
Further, the corresponding residual error structure for rebuilding structure process of second resolution image for the second scale is exported
The number of plies of the number of plies residual error structure for rebuilding structure process corresponding more than the second resolution image that output is the first scale, second
Scale is greater than the first scale.It is directed to the second resolution image of different scale in this way, only first resolution image need to be passed through phase
The residual error structure for answering the number of plies all makes first resolution image by solid without being directed to the second resolution image of different scale
The residual error structure of given layer number can be only achieved the final effect handled low-resolution image as high-definition picture, save
Image processing time improves the efficiency of image super-resolution under the premise of guaranteeing image definition.
Fig. 2 is the structure according to a kind of multi-scale image super-resolution rebuilding device of a preferred embodiment of the invention
Schematic diagram, as shown in Fig. 2, the multi-scale image super-resolution rebuilding device includes obtaining module 201 and processing module 202,
In:
Module 201 is obtained for obtaining first resolution image；Processing module 202 is for inputting first resolution image
To trained multi-scale image Super-resolution reconstruction established model, multiple dimensioned second resolution image is exported；Multi-scale image is super
Resolution reconstruction model is to be trained based on second resolution image pattern and corresponding first resolution image pattern
After obtain, second resolution be greater than first resolution；Multi-scale image Super-resolution reconstruction established model is based on residual error structure and again
Build the multilayer neural network of structure.
Specifically, it obtains module 201 and obtains first resolution image to be reconstructed first, first resolution image, which refers to, to be needed
It inputs multi-scale image Super-resolution reconstruction established model to be handled, therefore desirable for the corresponding original of second resolution image obtained
Beginning image.Then the first resolution image to be reconstructed that processing module 202 will acquire the acquisition of module 201 is input to multiple dimensioned figure
As Super-resolution reconstruction established model exports multiple dimensioned second resolution figure through multi-scale image super-resolution rebuilding model treatment
Picture；Multi-scale image Super-resolution reconstruction established model is based on second resolution image pattern and corresponding first resolution figure
Decent be trained after obtain, second resolution be greater than first resolution；Multi-scale image Super-resolution reconstruction established model is base
In the multilayer neural network of residual error structure and reconstruction structure.
A kind of multi-scale image super-resolution rebuilding device provided in an embodiment of the present invention, by obtaining first resolution figure
Picture；First resolution image is input to trained multi-scale image Super-resolution reconstruction established model, exports multiple dimensioned second
Image in different resolution；Multi-scale image Super-resolution reconstruction established model is based on second resolution image pattern and corresponding first
Image in different resolution sample obtains after being trained, and second resolution is greater than first resolution；Multi-scale image super-resolution rebuilding
Model is the multilayer neural network based on residual error structure and reconstruction structure.
Based on the above embodiment, the embodiment of the invention provides a kind of electronic equipment, for completing above method embodiment
In multi-scale image super resolution ratio reconstruction method.Fig. 3 is a kind of electronic equipment according to a preferred embodiment of the invention
Structural schematic diagram, as shown in figure 3, the electronic equipment includes processor 301, memory 302 and bus 303.Wherein, processor
301 and memory 302 mutual communication completed by bus 303.Processor 301, which can call, to be stored on memory 302
And the computer program that can be run on processor 301, the method to execute the various embodiments described above offer, for example, obtain
Take first resolution image；First resolution image is input to trained multi-scale image Super-resolution reconstruction established model, it is defeated
Multiple dimensioned second resolution image out, multi-scale image Super-resolution reconstruction established model be based on second resolution image pattern and
Corresponding first resolution image pattern obtains after being trained, and second resolution is greater than first resolution；Multiple dimensioned figure
As Super-resolution reconstruction established model is the multilayer neural network based on residual error structure and reconstruction structure.
In addition, the logical order in above-mentioned memory 302 can be realized by way of SFU software functional unit and conduct
Independent product when selling or using, can store in a computer readable storage medium.Based on this understanding, originally
The technical solution of the inventive embodiments substantially part of the part that contributes to existing technology or the technical solution in other words
It can be embodied in the form of software products, which is stored in a storage medium, including several fingers
It enables and using so that a computer equipment (can be personal computer, server or the network equipment etc.) executes the present invention respectively
The all or part of the steps of a embodiment method.And storage medium above-mentioned includes: USB flash disk, mobile hard disk, read-only memory
(ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), magnetic or disk
Etc. the various media that can store program code.
The embodiment of the invention also provides a kind of non-transient computer readable storage mediums, are stored thereon with computer journey
Sequence, the computer program make computer execute multi-scale image super resolution ratio reconstruction method provided by corresponding embodiment, such as
It include: to obtain first resolution image；First resolution image is input to trained multi-scale image super-resolution rebuilding
Model, exports multiple dimensioned second resolution image, and multi-scale image Super-resolution reconstruction established model is based on second resolution figure
Decent and corresponding first resolution image pattern obtain after being trained, and second resolution is greater than first resolution；
Multi-scale image Super-resolution reconstruction established model is the multilayer neural network based on residual error structure and reconstruction structure.
Those of ordinary skill in the art will appreciate that: realize that all or part of the steps of above method embodiment can pass through
The relevant hardware of program instruction is completed, and program above-mentioned can be stored in a computer readable storage medium, the program
When being executed, step including the steps of the foregoing method embodiments is executed；And storage medium above-mentioned includes: ROM, RAM, magnetic disk or light
The various media that can store program code such as disk.
The embodiments such as device and equipment described above are only schematical, wherein single as illustrated by the separation member
Member may or may not be physically separated, and component shown as a unit may or may not be physics
Unit, it can it is in one place, or may be distributed over multiple network units.It can select according to the actual needs
Some or all of the modules therein achieves the purpose of the solution of this embodiment.Those of ordinary skill in the art are not paying creation
In the case where the labour of property, it can understand and implement.
Through the above description of the embodiments, those skilled in the art can be understood that each embodiment can
It realizes by means of software and necessary general hardware platform, naturally it is also possible to pass through hardware.Based on this understanding, on
Stating technical solution, substantially the part that contributes to existing technology can be embodied in the form of software products in other words, should
Computer software product may be stored in a computer readable storage medium, such as ROM/RAM, magnetic disk, CD, including several fingers
It enables and using so that a computer equipment (can be personal computer, server or the network equipment etc.) executes each implementation
The method of certain parts of example or embodiment.
Finally, it should be noted that the above embodiments are merely illustrative of the technical solutions of the present invention, rather than its limitations；Although
Present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that: it still may be used
To modify the technical solutions described in the foregoing embodiments or equivalent replacement of some of the technical features；
And these are modified or replaceed, technical solution of various embodiments of the present invention that it does not separate the essence of the corresponding technical solution spirit and
Range.
Claims (10)
1. a kind of multi-scale image super resolution ratio reconstruction method characterized by comprising
Obtain first resolution image；
The first resolution image is input to trained multi-scale image Super-resolution reconstruction established model, is exported multiple dimensioned
Second resolution image, the multi-scale image Super-resolution reconstruction established model are based on second resolution image pattern and right with it
The first resolution image pattern answered obtains after being trained, and the second resolution is greater than the first resolution；It is described more
Scale image super-resolution rebuilding model is the multilayer neural network based on residual error structure and reconstruction structure.
2. a kind of multi-scale image super resolution ratio reconstruction method according to claim 1, which is characterized in that every layer described heavy
The input for building structure is the output of upper one layer of residual error structure, and every layer of output for rebuilding structure is the second resolution of single size
Rate image.
3. a kind of multi-scale image super resolution ratio reconstruction method according to claim 1, which is characterized in that described multiple dimensioned
The acquisition of the training data of image super-resolution rebuilding model the following steps are included:
Down-sampling is carried out to the second resolution image pattern by the down-sampling function in OpenCV, noise is added, goes forward side by side
Row JPEG compression obtains first resolution image pattern corresponding with the second resolution image pattern；
The second resolution image pattern is cut randomly into the region of the second Pixel Dimensions, and cuts the first resolution figure
The region of the first Pixel Dimensions corresponding with the region of second Pixel Dimensions in decent；The second resolution image sample
The ratio and second Pixel Dimensions and first pixel of size originally and the size of the first resolution image pattern
The ratio of size is identical；
Using the region of first Pixel Dimensions and the region of second Pixel Dimensions as the training data.
4. a kind of multi-scale image super resolution ratio reconstruction method according to claim 1, which is characterized in that upper one layer of residual error
The sum of the output of structure and the input of upper one layer of residual error structure are the input of next layer of residual error structure.
5. a kind of multi-scale image super resolution ratio reconstruction method according to claim 1, which is characterized in that the residual error knot
Structure includes the first convolutional layer, nonlinear activation function layer and the second convolutional layer that sequence carries out.
6. a kind of multi-scale image super resolution ratio reconstruction method according to claim 1, which is characterized in that the reconstruction knot
Structure includes third convolutional layer, super-pixel layer and the Volume Four lamination that sequence carries out, and the super-pixel layer is for expanding the third
The size of the output object of convolutional layer.
7. a kind of multi-scale image super resolution ratio reconstruction method according to claim 1, which is characterized in that exporting is second
The corresponding number of plies for rebuilding the residual error structure that structure is passed through of the second resolution image of scale is more than that output is the first scale
The corresponding number of plies for rebuilding the residual error structure that structure is passed through of two image in different resolution, second scale are greater than first scale.
8. a kind of multi-scale image super-resolution rebuilding device characterized by comprising
Module is obtained, for obtaining first resolution image；
Processing module is modeled for the first resolution image to be input to trained multi-scale image Super-resolution reconstruction
Type exports multiple dimensioned second resolution image；The multi-scale image Super-resolution reconstruction established model is based on second resolution
Image pattern and corresponding first resolution image pattern obtain after being trained, and the second resolution is greater than described the
One resolution ratio；The multi-scale image Super-resolution reconstruction established model is the multilayer nerve net based on residual error structure and reconstruction structure
Network.
9. a kind of electronic equipment, comprising: memory, processor and storage are on a memory and the calculating that can run on a processor
Machine program, which is characterized in that the processor is realized multiple dimensioned as described in any one of claim 1 to 7 when executing described program
Image super-resolution rebuilding method.
10. a kind of non-transient computer readable storage medium, is stored thereon with computer program, which is characterized in that the calculating
The multi-scale image super resolution ratio reconstruction method as described in any one of claim 1 to 7 is realized when machine program is executed by processor.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811133627.9A CN109064408A (en) | 2018-09-27 | 2018-09-27 | A kind of method and device of multi-scale image super-resolution rebuilding |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811133627.9A CN109064408A (en) | 2018-09-27 | 2018-09-27 | A kind of method and device of multi-scale image super-resolution rebuilding |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN109064408A true CN109064408A (en) | 2018-12-21 |
Family
ID=64766223
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811133627.9A Pending CN109064408A (en) | 2018-09-27 | 2018-09-27 | A kind of method and device of multi-scale image super-resolution rebuilding |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109064408A (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111476719A (en) * | 2020-05-06 | 2020-07-31 | Oppo广东移动通信有限公司 | Image processing method, image processing device, computer equipment and storage medium |
| CN111754438A (en) * | 2020-06-24 | 2020-10-09 | 安徽理工大学 | Underwater image restoration model based on multi-branch gating fusion and restoration method thereof |
| CN113298705A (en) * | 2020-02-24 | 2021-08-24 | 华为技术有限公司 | Image super-resolution processing method and device |
| CN114066722A (en) * | 2021-11-03 | 2022-02-18 | 北京字节跳动网络技术有限公司 | Method and device for acquiring image and electronic equipment |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2015180053A1 (en) * | 2014-05-28 | 2015-12-03 | 北京大学深圳研究生院 | Method and apparatus for rapidly reconstructing super-resolution image |
| CN107358575A (en) * | 2017-06-08 | 2017-11-17 | 清华大学 | A kind of single image super resolution ratio reconstruction method based on depth residual error network |
| CN107578377A (en) * | 2017-08-31 | 2018-01-12 | 北京飞搜科技有限公司 | A kind of super-resolution image reconstruction method and system based on deep learning |
| CN108537731A (en) * | 2017-12-29 | 2018-09-14 | 西安电子科技大学 | Image super-resolution rebuilding method based on compression multi-scale feature fusion network |
-
2018
- 2018-09-27 CN CN201811133627.9A patent/CN109064408A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2015180053A1 (en) * | 2014-05-28 | 2015-12-03 | 北京大学深圳研究生院 | Method and apparatus for rapidly reconstructing super-resolution image |
| CN107358575A (en) * | 2017-06-08 | 2017-11-17 | 清华大学 | A kind of single image super resolution ratio reconstruction method based on depth residual error network |
| CN107578377A (en) * | 2017-08-31 | 2018-01-12 | 北京飞搜科技有限公司 | A kind of super-resolution image reconstruction method and system based on deep learning |
| CN108537731A (en) * | 2017-12-29 | 2018-09-14 | 西安电子科技大学 | Image super-resolution rebuilding method based on compression multi-scale feature fusion network |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113298705A (en) * | 2020-02-24 | 2021-08-24 | 华为技术有限公司 | Image super-resolution processing method and device |
| CN111476719A (en) * | 2020-05-06 | 2020-07-31 | Oppo广东移动通信有限公司 | Image processing method, image processing device, computer equipment and storage medium |
| CN111476719B (en) * | 2020-05-06 | 2023-05-12 | Oppo广东移动通信有限公司 | Image processing method, device, computer equipment and storage medium |
| CN111754438A (en) * | 2020-06-24 | 2020-10-09 | 安徽理工大学 | Underwater image restoration model based on multi-branch gating fusion and restoration method thereof |
| CN114066722A (en) * | 2021-11-03 | 2022-02-18 | 北京字节跳动网络技术有限公司 | Method and device for acquiring image and electronic equipment |
| CN114066722B (en) * | 2021-11-03 | 2024-03-19 | 抖音视界有限公司 | Method and device for acquiring image and electronic equipment |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109064408A (en) | A kind of method and device of multi-scale image super-resolution rebuilding | |
| Hui et al. | Lightweight image super-resolution with information multi-distillation network | |
| Liu et al. | Residual feature aggregation network for image super-resolution | |
| US11645529B2 (en) | Sparsifying neural network models | |
| Wu et al. | Fast end-to-end trainable guided filter | |
| Lai et al. | Deep laplacian pyramid networks for fast and accurate super-resolution | |
| Muqeet et al. | Multi-attention based ultra lightweight image super-resolution | |
| Ahn et al. | Image super-resolution via progressive cascading residual network | |
| DE102017115519A1 (en) | Superpixel method for folding neural networks | |
| Zhou et al. | Efficient image super-resolution using vast-receptive-field attention | |
| CN105488776B (en) | Super-resolution image reconstruction method and device | |
| CN111429347A (en) | Image super-resolution reconstruction method and device and computer-readable storage medium | |
| CN111105352A (en) | Super-resolution image reconstruction method, system, computer device and storage medium | |
| CN106845529A (en) | Image feature recognition methods based on many visual field convolutional neural networks | |
| CN111951167B (en) | Super-resolution image reconstruction method, super-resolution image reconstruction device, computer equipment and storage medium | |
| Li et al. | Deep interleaved network for image super-resolution with asymmetric co-attention | |
| CN110533594A (en) | Model training method, image rebuilding method, storage medium and relevant device | |
| CN108665509A (en) | A kind of ultra-resolution ratio reconstructing method, device, equipment and readable storage medium storing program for executing | |
| CN112215755A (en) | Image super-resolution reconstruction method based on back projection attention network | |
| Wang et al. | Single image super-resolution with attention-based densely connected module | |
| Yang et al. | Lightweight group convolutional network for single image super-resolution | |
| CN106651772A (en) | Super-resolution reconstruction method of satellite cloud picture | |
| DE112019003587T5 (en) | Learning device, operating program of learning device, and operating method of learning device | |
| CN107481192A (en) | Image processing method, device, storage medium, computer program and electronic equipment | |
| CN115358932A (en) | Multi-scale feature fusion face super-resolution reconstruction method and system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20181221 |
|
| RJ01 | Rejection of invention patent application after publication |