CN107798667A - Face Enhancement Method based on residual error study - Google Patents
Face Enhancement Method based on residual error study Download PDFInfo
- Publication number
- CN107798667A CN107798667A CN201711184197.9A CN201711184197A CN107798667A CN 107798667 A CN107798667 A CN 107798667A CN 201711184197 A CN201711184197 A CN 201711184197A CN 107798667 A CN107798667 A CN 107798667A
- Authority
- CN
- China
- Prior art keywords
- face
- low
- image set
- residual error
- quality
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 24
- 230000002708 enhancing effect Effects 0.000 claims abstract description 22
- 238000013528 artificial neural network Methods 0.000 claims abstract description 11
- 230000001815 facial effect Effects 0.000 claims abstract description 10
- 238000010276 construction Methods 0.000 claims description 4
- 235000013399 edible fruits Nutrition 0.000 claims description 2
- 238000011084 recovery Methods 0.000 abstract description 2
- 238000005516 engineering process Methods 0.000 description 4
- 238000011840 criminal investigation Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000005070 sampling Methods 0.000 description 2
- 238000013135 deep learning Methods 0.000 description 1
- 238000005286 illumination Methods 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 230000007774 longterm Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 230000007935 neutral effect Effects 0.000 description 1
- 238000010606 normalization Methods 0.000 description 1
- 238000005728 strengthening Methods 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/73—Deblurring; Sharpening
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30196—Human being; Person
- G06T2207/30201—Face
Landscapes
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Image Processing (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a kind of face Enhancement Method based on residual error study, comprise the following steps：S10, prepare the face image set with noise and corresponding original face image set, respectively as the low-quality sample and exemplar of training set；S20, residual error learning neural network is constructed, come fitting parameter, generation face enhancing model using training set；S30, strengthens the residual image of the pending face of model calculating by face, and face enhancing is carried out according to the residual image of face.Face Enhancement Method provided by the invention based on residual error study, the noise in facial image, increase facial detail are handled by learning residual error, or even some unknown noises can be tackled, its anti-noise ability is good, detail recovery ability is strong.
Description
Technical field
The present invention relates to image enhaucament and the technical field of deep learning, more particularly to a kind of face based on residual error study
Enhancement Method.
Background technology
With the construction and development of various regions smart city, the technology such as video security protection and video criminal investigation is more and more closed
Note.Present video imaging technique, it is subjected to the influence of such as low illumination external environment condition so that what is gathered arrives video image matter
Amount is not high.The especially fuzzy, facial image of low-quality, the difficulty of recognition of face is substantially increased, also cause security protection and criminal investigation etc.
Work is more difficult.
Existing face Enhancement Method is mostly realized based on key point or image local.The identification essence of the former key point
True degree direct influence is to enhancing effect.And unfortunately image gets over low-quality, key point is more difficult to identify.The latter is then based on local
Property, so needing each structure of face, such as eye, mouth, nose, more accurately align.Regrettably, the face in video
Can not possibly all face camera lenses, and manually alignment will devote a tremendous amount of time.These all cause the application scenarios of this method by
Limit, it is as a result undesirable.
The content of the invention
For the problems of present technology, the present invention proposes a kind of face Enhancement Method based on residual error study, should
Method is perfectly aligned without face, and without face face camera lens, even without key point is calculated, it can learn low-quality people automatically
Noise and details in face, face enhancing effect is good, and application scenarios are more extensive.
To achieve these goals, face Enhancement Method proposed by the present invention, comprises the following steps：
S10, prepare the face image set with noise and corresponding original face image set, respectively as training set
Low-quality sample and exemplar；
S20, residual error learning neural network is constructed, carry out fitting parameter using the training set in the step S10, generate face
Strengthen model：F (x)-y=r；Wherein, f (x) is exemplar, and y is low-quality sample, and r is residual image；
S30, strengthen the residual image of the pending face of model calculating by face, pedestrian is entered according to the residual image of face
Face strengthens.
Preferably, in the step S10, the face image set with noise and corresponding original face image set are prepared
Including：
S11, original face image set is carried out plus made an uproar plus be fuzzy, obtains the face image set of low-quality, the i.e. people with noise
Face image collection, and using the face image set with noise as low-quality sample；
S12, using original face image set as exemplar, by the low-quality sample in the step S11 and the low-quality sample
Exemplar corresponding to this is separated into some image blocks, the training set of generating structure by corresponding relation.
Preferably, in the step S10, the face image set with noise and corresponding original image set bag are prepared
Include：
S11 ', the face image set of low-quality, the i.e. face image set with noise, and the band is made an uproar are gathered under low-quality environment
The face image set of sound is as low-quality sample；
S12 ', using the facial image of original face image set as exemplar, by the low-quality sample in step S11 ' and
Exemplar is separated into some image blocks, the training set of generating structure by corresponding relation corresponding to the low-quality sample.
Preferably, in the step S11 ', low-quality environment and the application of face enhancing of the face image set of low-quality are gathered
Environment is identical.
Preferably, the residual error learning neural network constructed in the step S20 includes the first layer, some being sequentially connected
Intermediate layer and residual error layer, its construction process are as follows：
S21, convolution is carried out to low-quality sample using the convolution kernel of 64 3 × 3 in first layer, and use amendment linear unit
ReLU carries out non-linearization to the result of convolution；
S22, the input sample using the nonlinearized result obtained in step S21 as first intermediate layer, first
Individual intermediate layer carries out convolution to the input sample using the convolution kernel of 64 3 × 3 × 64, and convolution results are carried out with batch specification
Change, non-linearization is carried out to convolution results using linear unit R eLU is corrected；It regard the result of first intermediate layer output as the
The input sample in two intermediate layers, first intermediate layer is exported using the convolution kernel of 64 3 × 3 × 64 second centre
Convolution results carry out convolution, convolution results are carried out with batch standardization, and convolution results are carried out using linear unit R eLU is corrected
Non-linearization, circulated with this, until last intermediate layer；
S23, the input sample using the nonlinearized result finally obtained in step S22 as residual error layer, in residual error layer
Convolution is carried out to the input sample using 3 × 3 × 64 convolution kernel；
S24, using training set corresponding with low-quality sample, the parameter of regression criterion learning neural network, generation face increases
Strong model.
Compared to present face Enhancement Method, the present invention has advantages below：
1st, application scenarios are more extensive, it is not necessary to propose various harsh limitation standards to facial image；
2nd, can not only be to antinoise, moreover it is possible to strengthen facial detail to the high treating effect of face enhancing；
3rd, during long-term use, it can constantly lift anti-noise and strengthen the ability of details.
Brief description of the drawings
In order to illustrate more clearly about the embodiment of the present invention or technical scheme of the prior art, below will be to embodiment or existing
There is the required accompanying drawing used in technology description to be briefly described, it should be apparent that, drawings in the following description are only this
Some embodiments of invention, for those of ordinary skill in the art, on the premise of not paying creative work, can be with
Structure according to these accompanying drawings obtains other accompanying drawings.
Fig. 1 is the flow chart of face Enhancement Method one embodiment of the present invention based on residual error study；
The object of the invention is realized, functional characteristics and advantage will be described further referring to the drawings in conjunction with the embodiments.
Embodiment
The present invention proposes a kind of face Enhancement Method based on residual error study.
Reference picture 1, Fig. 1 are the flow chart of face Enhancement Method one embodiment of the present invention based on residual error study.
As shown in figure 1, in embodiments of the present invention, the face Enhancement Method comprises the following steps：
S10, prepare the face image set with noise and corresponding original face image set, respectively as training set
Low-quality sample and exemplar.Wherein, exemplar is the high-quality sample corresponding to low-quality sample.
S20, residual error learning neural network is constructed, carry out fitting parameter using the training set in the step S10, generate face
Strengthen model：F (x)-y=r；Wherein, f (x) is exemplar, and y is low-quality sample, and r is residual image.
S30, strengthen the residual image of the pending face of model calculating by face, pedestrian is entered according to the residual image of face
Face strengthens.
Specifically, in the present embodiment, in step S10, the face image set with noise and corresponding original is prepared
The process of face image set is as follows：
S11, original face image set is carried out plus made an uproar plus be fuzzy, obtains the face image set of low-quality, the i.e. people with noise
Face image collection, and using the face image set with noise as low-quality sample.In addition, original face image set can also be carried out down
Sampling, the mode for reusing bicubic interpolation up-sampling obtain the face image set of low-quality.
S12, using original face image set as exemplar.By the low-quality sample in step S11 and the low-quality sample pair
The exemplar answered is separated into the image block of some suitable sizes, the training set of generating structure by corresponding relation.It should illustrate
, the size of image block selects by data characteristic.In the present embodiment, the size of image block have selected 40 × 40.
In order to cause the enhancing effect of face enhancing model, in another embodiment of the invention, in step S10
The process for preparing the face image set with noise and corresponding original face image set is：
S11 ', the face image set of low-quality, the i.e. face image set with noise, and the band is made an uproar are gathered under low-quality environment
The face image set of sound is as low-quality sample；Wherein, the low-quality environment of the face image set of low-quality and answering for face enhancing are gathered
Should be identical with environment.
S12 ', using the facial image of original face image set as exemplar, by the low-quality sample in step S11 ' and
Exemplar corresponding to the low-quality sample is separated into the image block of some suitable sizes, the training of generating structure by corresponding relation
Collection.In the present embodiment, the size of low-quality sample and exemplar have selected 40 × 40.
By gathering corresponding low-quality face sample under strengthening the low-quality environment to be applied of model in face, face may be such that
Strengthen the unknown noise under model learning to the low-quality environment.This causes face enhancing model both to increase details, can be with
The noise of some unknown species is handled, so as to improve the fault-tolerance of face enhancing and lifting enhancing effect.
Specifically, the residual error learning neural network constructed in step S20 include be sequentially connected first layer, some centres
Layer and residual error layer, its construction process are as follows：
S21, input sample of the image block as first layer of the low-quality sample of some 40 × 40 sizes will be divided into,
First layer carries out convolution to these image blocks using the convolution kernel of 64 3 × 3, and using the linear unit R eLU of amendment to convolution
As a result non-linearization is carried out.
S22, the input sample using the nonlinearized result obtained in step S21 as intermediate layer, used in intermediate layer
The convolution kernel of 64 3 × 3 × 64 carries out convolution to the input sample, and convolution results are carried out batch with standardization, finally using repairing
Linear positive unit R eLU carries out non-linearization to convolution results.Input sample using nonlinearized result as next intermediate layer
This, carries out second of convolution operation, to convolution knot to the nonlinearized result of second of convolution of Last intermediate layer output
Fruit carries out batch standardization, and carries out non-linearization to convolution results using linear unit R eLU is corrected, and is circulated with this, until convolution
Untill intermediate layer to the end.In the present embodiment, totally 46 layers of intermediate layer.It should be noted that the number of plies in intermediate layer is basis
Specific data and scene feature determine.
In step S22, by standardizing to convolution results each time, i.e., the normalization in small lot can be with
Accelerate the training effectiveness to face enhancing model, the enhancing effect of lifting face enhancing model.
S23, the input sample using the nonlinearized result finally obtained in step S22 as residual error layer, in residual error layer
Convolution is carried out to the input sample using 3 × 3 × 64 convolution kernel；
S24, using training set corresponding with low-quality sample, the parameter of regression criterion learning neural network, generation face increases
Strong model.
Technical scheme is by building residual error learning neural network, by by with the face image set of noise and its
Corresponding original face image set is separated into the image block of some suitable sizes, and carrys out regression criterion using these image blocks and learn
The parameter of neutral net, regenerate and train face to strengthen model.The residual error of pending facial image is calculated, learns low-quality sample
Noise and lose details, so as to carry out face enhancing.
Compared to traditional face enhancing model, the face enhancing model in the present invention handles face figure by learning residual error
Noise, increase facial detail as in so that face enhancing model copes with a variety of noises, even unknown noise,
Therefore the present invention has stronger anti-noise ability and detail recovery ability.Also, the present invention can also continue in use
Renewal learning storehouse, lifting face enhancing model performance, improves the effect that face strengthens.
The preferred embodiments of the present invention are the foregoing is only, are not intended to limit the scope of the invention, it is every at this
Under the inventive concept of invention, the equivalent structure transformation made using description of the invention and accompanying drawing content, or directly/use indirectly
It is included in other related technical areas in the scope of patent protection of the present invention.
Claims (5)
1. a kind of face Enhancement Method based on residual error study, it is characterised in that comprise the following steps：
S10, prepare the face image set with noise and corresponding original face image set, respectively as the low-quality of training set
Sample and exemplar；
S20, residual error learning neural network is constructed, come fitting parameter, generation face enhancing using the training set in the step S10
Model：F (x)-y=r；Wherein, f (x) is exemplar, and y is low-quality sample, and r is residual image；
S30, strengthens the residual image of the pending face of model calculating by face, and face increasing is carried out according to the residual image of face
By force.
2. the face Enhancement Method as claimed in claim 1 based on residual error study, it is characterised in that accurate in the step S10
The standby face image set with noise and corresponding original face image set include：
S11, original face image set is carried out plus made an uproar plus be fuzzy, obtains the face image set of low-quality, i.e., the face figure with noise
Image set, and using the face image set with noise as low-quality sample；
S12, using original face image set as exemplar, by the low-quality sample in the step S11 and the low-quality sample pair
The exemplar answered is separated into some image blocks, the training set of generating structure by corresponding relation.
3. the face Enhancement Method as claimed in claim 1 based on residual error study, it is characterised in that accurate in the step S10
The standby face image set with noise and corresponding original image set include：
S11 ', the face image set of low-quality, the i.e. face image set with noise, and by this with noise are gathered under low-quality environment
Face image set is as low-quality sample；
S12 ' is low with this by the low-quality sample in step S11 ' using the facial image of original face image set as exemplar
Exemplar corresponding to matter sample is separated into some image blocks, the training set of generating structure by corresponding relation.
4. the face Enhancement Method as claimed in claim 3 based on residual error study, it is characterised in that in the step S11 ',
The low-quality environment for gathering the face image set of low-quality is identical with the application environment that face strengthens.
5. the face Enhancement Method based on residual error study as described in Claims 1 to 4 any one, it is characterised in that described
The residual error learning neural network constructed in step S20 includes first layer, some intermediate layers and the residual error layer being sequentially connected,
Its construction process is as follows：
S21, convolution is carried out to low-quality sample using the convolution kernel of 64 3 × 3 in first layer, and use the linear unit R eLU of amendment
Non-linearization is carried out to the result of convolution；
S22, the input sample using the nonlinearized result obtained in step S21 as first intermediate layer, in first
Interbed carries out convolution to the input sample using the convolution kernel of 64 3 × 3 × 64, and convolution results are carried out with batch standardization, makes
Non-linearization is carried out to convolution results with linear unit R eLU is corrected；Using the result of first intermediate layer output as in second
The input sample of interbed, in the convolution knot that second centre is exported using the convolution kernel of 64 3 × 3 × 64 to first intermediate layer
Fruit carries out convolution, and convolution results are carried out with batch standardization, and it is non-linear to use the linear unit R eLU of amendment to carry out convolution results
Change, circulated with this, until last intermediate layer；
S23, the input sample using the nonlinearized result finally obtained in step S22 as residual error layer, 3 are used in residual error layer
× 3 × 64 convolution kernel carries out convolution to the input sample；
S24, use training set corresponding with low-quality sample, the parameter of regression criterion learning neural network, generation face enhancing mould
Type.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711184197.9A CN107798667A (en) | 2017-11-23 | 2017-11-23 | Face Enhancement Method based on residual error study |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711184197.9A CN107798667A (en) | 2017-11-23 | 2017-11-23 | Face Enhancement Method based on residual error study |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN107798667A true CN107798667A (en) | 2018-03-13 |
Family
ID=61536457
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201711184197.9A Pending CN107798667A (en) | 2017-11-23 | 2017-11-23 | Face Enhancement Method based on residual error study |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107798667A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109785252A (en) * | 2018-12-25 | 2019-05-21 | 山西大学 | Based on multiple dimensioned residual error dense network nighttime image enhancing method |
| CN112991223A (en) * | 2021-04-06 | 2021-06-18 | 深圳棱镜空间智能科技有限公司 | Image enhancement method, device, equipment and medium based on reversible neural network |
| CN113610042A (en) * | 2021-08-18 | 2021-11-05 | 睿云联(厦门)网络通讯技术有限公司 | Face recognition living body detection method based on pre-training picture residual error |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105760859A (en) * | 2016-03-22 | 2016-07-13 | 中国科学院自动化研究所 | Method and device for identifying reticulate pattern face image based on multi-task convolutional neural network |
| CN106204467A (en) * | 2016-06-27 | 2016-12-07 | 深圳市未来媒体技术研究院 | A kind of image de-noising method based on cascade residual error neutral net |
| CN106875361A (en) * | 2017-02-17 | 2017-06-20 | 深圳市唯特视科技有限公司 | A kind of method that poisson noise is removed based on depth convolutional neural networks |
| CN107248144A (en) * | 2017-04-27 | 2017-10-13 | 东南大学 | A kind of image de-noising method based on compression-type convolutional neural networks |
| CN107256541A (en) * | 2017-06-15 | 2017-10-17 | 北京航空航天大学 | A kind of multi-spectral remote sensing image defogging method based on convolutional neural networks |
-
2017
- 2017-11-23 CN CN201711184197.9A patent/CN107798667A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105760859A (en) * | 2016-03-22 | 2016-07-13 | 中国科学院自动化研究所 | Method and device for identifying reticulate pattern face image based on multi-task convolutional neural network |
| CN106204467A (en) * | 2016-06-27 | 2016-12-07 | 深圳市未来媒体技术研究院 | A kind of image de-noising method based on cascade residual error neutral net |
| CN106875361A (en) * | 2017-02-17 | 2017-06-20 | 深圳市唯特视科技有限公司 | A kind of method that poisson noise is removed based on depth convolutional neural networks |
| CN107248144A (en) * | 2017-04-27 | 2017-10-13 | 东南大学 | A kind of image de-noising method based on compression-type convolutional neural networks |
| CN107256541A (en) * | 2017-06-15 | 2017-10-17 | 北京航空航天大学 | A kind of multi-spectral remote sensing image defogging method based on convolutional neural networks |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109785252A (en) * | 2018-12-25 | 2019-05-21 | 山西大学 | Based on multiple dimensioned residual error dense network nighttime image enhancing method |
| CN109785252B (en) * | 2018-12-25 | 2023-03-24 | 山西大学 | Night image enhancement method based on multi-scale residual error dense network |
| CN112991223A (en) * | 2021-04-06 | 2021-06-18 | 深圳棱镜空间智能科技有限公司 | Image enhancement method, device, equipment and medium based on reversible neural network |
| CN113610042A (en) * | 2021-08-18 | 2021-11-05 | 睿云联(厦门)网络通讯技术有限公司 | Face recognition living body detection method based on pre-training picture residual error |
| CN113610042B (en) * | 2021-08-18 | 2023-05-23 | 睿云联(厦门)网络通讯技术有限公司 | Face recognition living body detection method based on pre-training picture residual error |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| WO2023092813A1 (en) | Swin-transformer image denoising method and system based on channel attention | |
| CN109035163B (en) | Self-adaptive image denoising method based on deep learning | |
| CN108932697B (en) | Distortion removing method and device for distorted image and electronic equipment | |
| CN109871845B (en) | Certificate image extraction method and terminal equipment | |
| CN107577985A (en) | The implementation method of the face head portrait cartooning of confrontation network is generated based on circulation | |
| CN107798667A (en) | Face Enhancement Method based on residual error study | |
| CN102567955B (en) | Method and system for inpainting images | |
| CN111145123B (en) | Image denoising method based on U-Net fusion retention details | |
| CN109120937A (en) | A kind of method for video coding, coding/decoding method, device and electronic equipment | |
| CN109151475A (en) | A kind of method for video coding, coding/decoding method, device and electronic equipment | |
| CN107729885A (en) | A kind of face Enhancement Method based on the study of multiple residual error | |
| CN111915513A (en) | Image denoising method based on improved adaptive neural network | |
| CN107133590A (en) | A kind of identification system based on facial image | |
| CN110033416A (en) | A kind of car networking image recovery method of the more granularities of combination | |
| CN112766413A (en) | Bird classification method and system based on weighted fusion model | |
| CN116342953A (en) | Dual-mode target detection model and method based on residual shrinkage attention network | |
| CN107330387A (en) | Pedestrian detection method based on view data | |
| CN113066027A (en) | Screen shot image moire removing method facing Raw domain | |
| CN116523875A (en) | Insulator defect detection method based on FPGA pretreatment and improved YOLOv5 | |
| CN115588237A (en) | Three-dimensional hand posture estimation method based on monocular RGB image | |
| CN116416156A (en) | Swin transducer-based medical image denoising method | |
| CN109801224A (en) | A kind of image processing method, device, server and storage medium | |
| CN112084936B (en) | Face image preprocessing method, device, equipment and storage medium | |
| CN116109538A (en) | Image fusion method based on simple gate unit feature extraction | |
| CN110889811A (en) | Photo repair system construction method, photo repair method and system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20180313 |
|
| RJ01 | Rejection of invention patent application after publication |