CN103065292B - Face super resolution rebuilding method based on principal component sparse expression - Google Patents

Abstract

A face super resolution rebuilding method based on principal component sparse expression comprises the following steps: enabling an input low resolution facial image, an input low resolution facial sample image and an input high resolution facial sample image to be respectively divided into image blocks which are mutually overlapped, conducting principal component decomposition for each position image block of the images, obtaining a principal component expression base, conducting sparse restraining projection for each image block of the input low resolution facial image according to the corresponding principal component expression base of an image block in a sample database, converting an obtained principal component sparse expression coefficient into a sample expression space, replacing each position block of the low resolution facial image by the corresponding position block of the high resolution facial image, combining and joining the image blocks with high resolution together, and obtaining an output high resolution image. According to the face super resolution rebuilding method based on the principal component sparse expression, the principal component sparse expression of the position blocks is provided, inner information and noise information of the input image blocks are distinguished, expression accuracy of the image blocks under noise environment is improved, and impersonal image quality of the high resolution rebuilding image is improved.

Description

A kind of face super-resolution reconstruction method based on main composition sparse expression

Technical field

The present invention relates to face image super-resolution field, be specifically related to a kind of face super-resolution reconstruction method to noise robustness based on main composition sparse expression.

Background technology

In recent years, video monitoring system is widely applied in city safety work.But, under a lot of application scenarios, because camera is away from paid close attention to target range, makes the imaging pixel of target in monitor video usually less, lack enough detailed information, identification demand cannot be met.Particularly in criminal investigation application, interested target face imaging resolution is too low, cannot meet the demand of human eye identification, causes difficulty for effective locking evidence.Therefore, carry out super-resolution enhancing for the low-resolution face image in low-quality monitor video, and then obtain more local detail information so that identify, become problem demanding prompt solution in criminal investigation business.For the increase resolution problem of face low-resolution image, the low-resolution face image of input is reconstructed high-definition picture by the prior imformation of image pattern by human face super-resolution technology, and is widely used.

Human face super-resolution technology has become a hot research problem, and a large amount of Super-Resolution for Face Images based on Sample Storehouse study emerges in large numbers in recent years.Super-Resolution for Face Images based on study mainly make use of the sample pair of high-low resolution image, and study obtains the relation between high-low resolution image, is derived produce corresponding high-definition picture by the low-resolution image of input.

2000, Simon and Kanade has delivered the Super-Resolution for Face Images proposed first based on study, at document 1(S.Baker and T.Kanade.Hallucinating faces.In FG, Grenoble, France, Mar.2000,83-88.) in also referred to as face illusion (face hallucination) method.Calendar year 2001, the people such as Liu are at document 2(C.Liu, H.Y.Shum, andC.S.Zhang.A two-step approach to hallucinating faces:global parametric model and localnonparametric model.In CVPR, pp.192 – 198,2001.) the middle two-step approach proposing two steps synthesis facial images of parameter global face and nonparametric local face algorithm.Based on study Super-Resolution for Face Images with its excellent algorithm performance with rebuild effect progressively obtain pays close attention to widely with study.

Based on the Super-Resolution for Face Images learnt according to the processing mode of facial image being divided into overall face algorithm and local face algorithm, whole secondary face processes as a vector by overall situation face algorithm, the facial image rebuild of the method on the whole with input human face similarity and there is certain robustness to noise, but there is aliasing effect in the marginal portion rebuilding image.Local face method is that whole secondary facial image is divided into block, carries out, then piece together whole sub-picture to the process of reconstruction exporting high-definition picture according to piecemeal, and the facial image subjective quality that this method is rebuild is better, but to noise-sensitive.The people such as Chang in 2004 are at document 3(H.Chang, D.Y.Yeung, and Y.M.Xiong.Super-resolution through neighborembedding.In CVPR, pp.275 – 282,2004.) in, the piecemeal of hypothesis high-low resolution facial image has Geometrical consistency, and utilize the expression coefficient of input low-resolution image block to remain to high resolution space synthesis high-definition picture, achieve good subjective and objective reconstruction quality.Neighbour's block number that carrying out selected by the method is expressed is fixing, therefore may there is the problem of over-fitting and constraint deficiency when expressing input picture block.For this problem, the people such as Ma in 2010 are at document 4(X.Ma, J.P Zhang, and C.Qi.Hallucinating face by position-patch.Pattern Recognition, 43 (6): 3178 – 3194, 2010.) a kind of position-based block constraint Super-Resolution for Face Images is proposed in, propose the prior-constrained algorithm of facial image block of locations, improve the arest neighbors system of selection of image block, have selected all block of locations carry out expressing and synthesize, but this algorithm does not consider the impact that noise is expressed for image block, therefore reconstruction quality is in a noisy environment not good.Local face algorithm based on piecemeal directly utilizes pixel domain feature to carry out expressing and synthesizes, under noise conditions, existing piecemeal local face algorithm cannot internal characteristics in differentiate between images block and noise contribution, noise is also expressed, make the high-definition picture synthesized also contains noise information, reduce the synthesis quality of such algorithm.

Summary of the invention

The object of the invention is to provide a kind of face image super-resolution reconstruction method based on main composition sparse expression, solve the existing similar problem based on noise and internal characteristics cannot be distinguished in piecemeal expression algorithm, utilize main composition sparse expression to express according to the internal characteristics of the adaptively selected image of picture material, improve the quality of the high-resolution human face image of synthesis.

For achieving the above object, the technical solution used in the present invention is a kind of face super-resolution reconstruction method based on main composition sparse expression, comprises the steps:

Step 1, image block, comprises the overlapping region size of dividing block size and piecemeal according to the image preset, carries out piecemeal, obtain corresponding image block to the low-resolution face image inputted, low resolution face sample image and high-resolution human face sample image;

Step 2, carries out main composition decomposition based on the image block on all each positions of low resolution face sample image, and the main composition obtaining image block expresses base;

Step 3, expresses base with the main composition of step 2 gained low resolution face sample image epigraph block, asks for the main composition sparse expression coefficient of the low-resolution face image respective image block of input, is then converted into speciality uniform space and expresses coefficient;

Step 4, replaces with the image block of all low resolution face sample images the image block of high-resolution human face sample image corresponding to position, expresses coefficient weighting synthesis high-resolution human face image block with step 3 gained speciality uniform space;

Step 5, synthesizes gained high-resolution human face image block and carries out split according to image block position, obtain a high-resolution human face image by step 4.

And step 2 implementation is as follows,

Get the image block of certain position (i, j) of M low resolution face sample image, the image block of each d × d pixel is launched into a column vector, M image block Column vector groups composograph block matrix

Main composition is expressed base and is obtained by following formula,

$E_l(i,j)=Y_L^M(i,j)V_l(i,j){\mathrm{\Λ}}_l{(i,j)}^{-\frac{1}{2}}$

Wherein, V _l(i, j) and Λ _l(i, j) is matrix respectively covariance matrix eigenvectors matrix and eigenvalue matrix, E _l(i, j) is the main composition expression base of the image block on position (i, j).

And step 3 implementation is as follows,

For the image block on certain position (i, j) of the low-resolution face image of input, obtain the sparse expression coefficient of a main composition by following formula,

$\widetilde{\mathrm{\α}}(i,j)=\underset{\min }{\arg }({\left|\right|X_L(i,j)-E_l(i,j)\mathrm{\α}(i,j)\left|\right|}_2^2+\mathrm{\λ}{\left|\mathrm{\α}(i,j)\right|}_1)$

Wherein, E _l(i, j) be each low resolution face sample image position (i, the main composition of the image block j) expresses base, λ be image reconstruction error and express coefficient openness between controlling elements, α (i, j) be required input low-resolution face image position (i, j) on the main composition sparse expression coefficient of image block what represent is two normal forms, || ₁what represent is a normal form;

Be converted into speciality uniform space by following formula and express coefficient,

$c(i,j)=V_l(i,j){\mathrm{\Λ}}_l{(i,j)}^{-\frac{1}{2}}\mathrm{\α}(i,j)$

Wherein, the speciality uniform space of the image block on the low-resolution face image position (i, j) that namely inputs of c (i, j) expresses coefficient.

And step 4 implementation is as follows,

Get the image block of the position (i, j) of M high-resolution human face sample image, the image block of each Td × Td pixel is launched into a column vector, M image block Column vector groups composograph block matrix t is the enlargement factor of high-resolution human face sample image relative to corresponding low resolution face sample image;

$X_H(i,j)=Y_H^M(i,j)c(i,j)+M(i,j)$

Wherein, X _h(i, j) be for the corresponding high-resolution human face image block of image block synthesis gained on the low-resolution face image position (i, j) of input, M (i, j) be image block average on M high-resolution human face sample image position (i, j).

The invention provides the face super-resolution reconstruction method to noise robustness, by the main composition sparse expression of block of locations, the internal characteristics in input picture block and noise are distinguished, inhibit the impact of input picture noise, algorithm is compared with the localized mass method (document 4) of sparse expression with fixing neighbour's block number (document 3) herein, to the noise of input picture, there is better expression mechanism, under noise conditions, the higher-quality high-resolution human face image of final acquisition.

Accompanying drawing explanation

Fig. 1 is the process flow diagram of the embodiment of the present invention;

Fig. 2 is that the block of the facial image of the embodiment of the present invention divides schematic diagram.

Embodiment

Technical solution of the present invention can adopt software engineering to realize automatic flow and run.Below in conjunction with drawings and Examples, technical solution of the present invention is further described.See Fig. 1, embodiment of the present invention concrete steps are:

Step 1, image block, first sets point block size of image, and the overlapping region size of piecemeal, then carries out piecemeal to the low-resolution face image inputted, low resolution face sample image, high-resolution human face sample image.

The low-resolution face image of input and facial image to be reconstructed.For providing training sample, generally provide multiple high resolving power sample images and low resolution sample image, and high resolving power sample facial image and low resolution sample facial image are one_to_one corresponding.The size of high-definition picture is the integral multiple of low-resolution image size, gets the enlargement factor of image.In embodiment, high-resolution human face sample image size is 120 × 100 pixels, and corresponding low resolution face sample image size is 30 × 25 pixels, and enlargement factor T is here 4.Low resolution face sample image is that corresponding high-resolution human face sample image is doubly got by Bicubic down-sampling four.All high-resolution human face sample images form high resolving power training set, and all low resolution face sample images form low resolution training set.

Illustrate with embodiment, if the low-resolution face image X of input _l, high resolving power training set with low resolution training set wherein M is the right number of high-low resolution face sample image, and k is the right sequence number of high-low resolution face sample image.

Shown in Figure 2, a point block operations is carried out for low-resolution image and high-definition picture, suppose that the size of low-resolution image is p × q pixel, its point of block size is d × d pixel, so corresponding high-definition picture is 4p × 4q pixel, and its point of block size is 4d × 4d pixel, supposes that the overlaid pixel number of low-resolution image block is e, then corresponding high-definition picture overlaid pixel is 4e, guarantees the one-to-one relationship of high-low resolution image block.With the overlap of the square of d pixel and e pixel, piecemeal is carried out to low-resolution image like this, line number and the columns of piecemeal can be obtained:

$m=\mathrm{ceil}\left(\frac{q-e}{d-e}\right)---\left(1\right)$

$n=\mathrm{ceil}\left(\frac{p-e}{d-e}\right)$

Ceil (.) represents the smallest positive integral returning and be greater than or equal to and specify expression formula.Like this for the image of p × q pixel size, can be divided into the image block that m × n size is d × d pixel, left-to-right from image of embodiment, order layout block of locations top to bottm, as can be seen from formula (1), the image block number of high-low resolution is equal.As in Fig. 2,4 image block positions of image upper left are (1,1), (1,2), (2,1), (2,2).

Can by low-resolution face image X _lthe set of partitioned image block gained is { X _l(i, j) | 1≤i≤m, 1≤j≤n}, by high resolving power training set with low resolution training set correspondingly the set of partitioned image block gained is respectively with m represents the number of face sample image in high-resolution and low-resolution training set, and (i, j) represents line number and the row number of the image block divided, m and n represents the image block number that each row and every a line mark off respectively.

Embodiment is used represent the position (i comprising M low resolution face sample image, j) image block matrix, this matrix gets the position (i of M low resolution face sample image, j) image block, the image block of each d × d pixel is launched into a column vector, M image block Column vector groups composograph block matrix its size is d ²× M, 0 < i≤m, 0 < j≤n.

Step 2, based on the image block on each positions of low resolution face sample image all in low resolution training set, (the image block matrix on position (i, j) is ), carry out main composition decomposition, the main composition obtaining image block expresses base.

In embodiment, main composition is expressed base and is obtained by following formula:

$E_l(i,j)=Y_L^M(i,j)V_l(i,j){\mathrm{\&Lambda;}}_l{(i,j)}^{-\frac{1}{2}}---\left(2\right)$

Wherein, V _l(i, j) and Λ _l(i, j) is matrix respectively covariance matrix eigenvectors matrix and eigenvalue matrix, E _lthe main composition expression dictionary of image block, E _l(i, j) is the main composition expression base of the image block on position (i, j).

Step 3, base is expressed with the main composition of step 2 gained low resolution face sample image epigraph block, ask for the main composition sparse expression coefficient of the low-resolution face image respective image block of input, realize the internal characteristics of low-resolution face image to input and being automatically separated of noise.Namely for the image block on each position of the low-resolution face image of input, calculate the sparse expression coefficient at the main composition base of the block of locations of correspondence, and this sparse expression coefficients conversion is obtained with sample image block as expressing the expression coefficient of base to sample space.

In embodiment, like this for the image block X on each position of the low-resolution face image of input _l(i, j), can obtain the expression coefficient that a main composition is sparse:

$\widetilde{\mathrm{\&alpha;}}(i,j)=\underset{\min }{\arg }({\left|\right|X_L(i,j)-E_l(i,j)\mathrm{\&alpha;}(i,j)\left|\right|}_2^2+\mathrm{\&lambda;}{\left|\mathrm{\&alpha;}(i,j)\right|}_1)---\left(3\right)$

Wherein, X _limage block on the low-resolution face image position (i, j) that (i, j) is input, E _l(i, j) be each low resolution face sample image position (i, the main composition of the image block j) expresses base, λ be image reconstruction error and express coefficient openness between controlling elements, its value expresses openness balanced with between reconstruction error item of coefficient for obtaining, generally experimentally determine best value, α (i, j) be the low-resolution face image position (i of required input, the main composition sparse expression coefficient of the image block j) what represent is two normal forms, || ₁what represent is a normal form, and solving of this objective function can utilize existing mathematical tool to carry out.

After obtaining main composition sparse expression, main composition due to high-low resolution expresses the Geometrical consistency that coefficient does not have stream shape, embodiment is expressed in more consistent feature space by expressing coefficients conversion to the high-low resolution taking image block as expression base, expression coefficient c (i, j) after conversion is:

$c(i,j)=V_l(i,j){\mathrm{\&Lambda;}}_l{(i,j)}^{-\frac{1}{2}}\mathrm{\&alpha;}(i,j)---\left(4\right)$

Image block on the low-resolution face image position (i, j) of input can be expressed as:

$X_L(i,j)=Y_L^M(i,j)c(i,j)+m(i,j)---\left(5\right)$

The speciality uniform space of the image block on the low-resolution face image position (i, j) that c (i, j) namely inputs expresses coefficient.C can be established to be the column vector comprising M element during concrete enforcement, use c _krepresent the composite coefficient of corresponding each sample, 1≤k≤M, as c in Fig. 1 ₁, c ₂, c ₃, c ₄c _m; M (i, j) is the graph block matrix of low resolution face sample image position (i, j) average according to every row, the m (i, j) obtained is d ²the column vector of × 1.

Step 4, replaces with the image block of all low resolution face sample images the image block of high-resolution human face sample image corresponding to position, expresses coefficient weighting synthesis high-resolution human face image block with step 3 gained speciality uniform space.Be exactly in fact that the low resolution sample block matrix in formula (5) and sample average matrix are replaced the high resolving power sample block matrix and the sample average matrix that become correspondence position, express weight coefficient weighting synthesis high-resolution human face image block X with step 3 gained _h(i, j).

Embodiment uses the expression weight coefficient obtained in step 3 to represent that the expression formula of high-resolution human face image block is:

$X_H(i,j)=Y_H^M(i,j)c(i,j)+M(i,j)---\left(6\right)$

Wherein, X _h(i, j) is for the corresponding high-resolution human face image block of image block synthesis gained on the low-resolution face image position (i, j) of input, be the image block matrix of the position (i, j) of M high-resolution human face sample image, M (i, j) is the image block average on M high-resolution human face sample image position (i, j).Embodiment according to structure consistent mode, gets the image block of the position (i, j) of M high-resolution human face sample image, and the image block of each 4d × 4d pixel is launched into a column vector, M image block Column vector groups composograph block matrix its size is (4d) ²× M, 0 < i≤m, 0 < j≤n.Equally, M (i, j) is the graph block matrix of high-resolution human face sample image position (i, j) average according to every row, the M (i, j) obtained is (4d) ²the column vector of × 1.

Step 5, synthesizes gained high-resolution human face image block and carries out split according to image block position, obtain a high-resolution human face image by step 4.

In embodiment, in the split process of image block, result in repeatedly adding up of partial pixel, a counter can be set and calculate accumulative frequency, split image is average to accumulative frequency, finally obtain high resolution output image.

The present invention is different from the local face method of document 3 and 4, propose based on image block main composition sparse expression algorithm, enhance the robustness to input noise, inhibit input noise in the expression in Sample Storehouse space, under noise conditions, appoint the high-resolution human face image that so can synthesize better quality.

Below provide Experimental comparison that the validity of this method is described.

Have employed FEI face database (document 5:Z.Wang, A.Bovik, H.Sheikh, and E.Simoncelli, " Imagequality assessment:From error visibility to structural similarity, " IEEE Trans.Image Process., vol.13, no.4, pp.600 – 612,2004.).200 different face (100 male sex, 100 women), smile expression each one of facial image in everyone is just poker-faced facial image and front, all image size unifications are 120 × 100, therefrom choose 360 to train, remaining 40 is image to be tested.Often open the high-resolution image of the training smoothing average filter of 4 × 4 (use), and 4 times of down-samplings obtain the image of the low resolution of 30 × 25.

The size dividing facial image block is respectively: high-resolution human face image is divided into the image block of 24 × 24, and overlapping is 12 pixels; Low-resolution face image is divided into the image block of 6 × 6, and overlapping is 3 pixels.Namely for high-resolution image, 4p=120,4q=100,4d=12,4e=4; For the image of low resolution, p=30, q=25, d=6, e=3.

In order to test herein, algorithm is for the robustness of noise, and test and add Gaussian noise to input picture, the variance of noise is σ=0002, and neighbour's block number K of document 3 neighborhood embedding grammar gets 100.The block of locations that 360 whole samples got by document 4 is expressed.Parameter lambda value in the inventive method is 0.04.

Experiment adopts objective quality typical peak signal to noise ratio (S/N ratio) (PSNR, unit is dB) to carry out measure algorithm reconstruction quality.Adding intensity at input picture is the Gaussian noise of 0.002, and contrast the average of the test facial image of 40 whole secondary inputs respectively, the PSNR value that the inventive method and document 3, document 4 method obtain is followed successively by 25.72,24.62,20.15.The inventive method promotes 1.1db than as documents 3 algorithm PSNR, and documents 4 algorithm PSNR improves 5.57db.

Table 1

Image objective quality index	Document 3 algorithm	Document 4 algorithm	Algorithm of the present invention
				PSNR(DB)	24.62	20.15	25.72

Specific embodiment described herein is only to the explanation for example of the present invention's spirit.Those skilled in the art can make various amendment or supplement or adopt similar mode to substitute to described specific embodiment, but can't depart from spirit of the present invention or surmount the scope that appended claims defines.

Claims (2)

1., based on a face super-resolution reconstruction method for main composition sparse expression, it is characterized in that, comprise the steps:

Step 1, image block, comprises the overlapping region size of dividing block size and piecemeal according to the image preset, carries out piecemeal, obtain corresponding image block to the low-resolution face image inputted, low resolution face sample image and high-resolution human face sample image;

Step 2, carries out main composition decomposition based on the image block on all each positions of low resolution face sample image, and the main composition obtaining image block expresses base;

Step 3, expresses base with the main composition of step 2 gained low resolution face sample image epigraph block, asks for the main composition sparse expression coefficient of the low-resolution face image respective image block of input, is then converted into speciality uniform space and expresses coefficient;

Step 4, replaces with the image block of all low resolution face sample images the image block of high-resolution human face sample image corresponding to position, expresses coefficient weighting synthesis high-resolution human face image block with step 3 gained speciality uniform space;

Step 5, synthesizes gained high-resolution human face image block and carries out split according to image block position, obtain a high-resolution human face image by step 4;

Wherein, step 2 implementation is as follows,

Get the image block of certain position (i, j) of M low resolution face sample image, the image block of each d × d pixel is launched into a column vector, M image block Column vector groups composograph block matrix

Main composition is expressed base and is obtained by following formula,

Wherein, V _l(i, j) and Λ _l(i, j) is matrix respectively covariance matrix eigenvectors matrix and eigenvalue matrix, E _l(i, j) is the main composition expression base of the image block on position (i, j);

Wherein, step 3 implementation is as follows,

For the low-resolution face image X of input _lcertain position (i, j) on image block X _l(i, j), obtains the sparse expression coefficient of a main composition by following formula

Wherein, E _l(i, j) be each low resolution face sample image position (i, the main composition of the image block j) expresses base, λ be image reconstruction error and express coefficient openness between controlling elements, α (i, j) be required input low-resolution face image position (i, j) on the main composition sparse expression coefficient of image block what represent is two normal forms, || ₁what represent is a normal form;

Be converted into speciality uniform space by following formula and express coefficient,

Wherein, the speciality uniform space of the image block on the low-resolution face image position (i, j) that namely inputs of c (i, j) expresses coefficient.

2., according to claim 1 based on the face super-resolution reconstruction method of main composition sparse expression, it is characterized in that: step 4 implementation is as follows,

Get the image block of the position (i, j) of M high-resolution human face sample image, the image block of each Td × Td pixel is launched into a column vector, M image block Column vector groups composograph block matrix t is the enlargement factor of high-resolution human face sample image relative to corresponding low resolution face sample image;

Wherein, X _h(i, j) be for the corresponding high-resolution human face image block of image block synthesis gained on the low-resolution face image position (i, j) of input, M (i, j) be image block average on M high-resolution human face sample image position (i, j).