CN106023098A - Image repairing method based on tensor structure multi-dictionary learning and sparse coding - Google Patents

Abstract

The invention discloses an image repairing method based on tensor structure multi-dictionary learning and sparse coding, which comprises the main steps as follows: classifying tensor samples according to the direction and neighboring structure information of tensors; constructing a tensor dictionary for each category; classifying the tensors of an image to be repaired in the same way; using the tensor dictionaries corresponding to the category tags of the tensors to repair the image; and calculating the weighted sum of the reconstruction results under all types of dictionaries to get a final reconstruction result of the image to be repaired. According to the invention, tensors are classified effectively and certainly according to the direction and neighboring structure information of the tensors, tensors with different details can be distinguished, the weighted sum of the reconstruction results under all types of dictionaries is calculated for reconstruction of tensors to be repaired, the defect that a single dictionary has limited expression ability is overcome, clearer edge details can be restored for repair of natural images, and the repair quality is improved. The image repairing method is used to repair damaged images.

Description

Image mending method based on the many dictionary learnings of tensor structure Yu sparse coding

Technical field

The invention belongs to technical field of image processing, further relate to the image mending in compressed sensing technical field, A kind of image mending method based on the many dictionary learnings of tensor structure Yu sparse coding.The present invention can be used for natural figure As repairing, make impaired image restoration, remove or replace original target in picture, and reach what human eye was difficult to aware Effect.

Background technology

At present in image processing field, the method repairing natural image is generally divided into three classes: based on partial differential The Nonlinear Diffusion method of equation, textures synthesis method and method based on rarefaction representation.Nonlinear Diffusion based on partial differential equation Method, utilizes the useful information provided around area to be repaired, by the cavity of defect during gradually diffusion edge repairs image.Stricture of vagina Reason synthetic method be the part that image is not lost information as the training set of derivation missing image message part, seek in training set The sample image mated most is looked for fill in the blanks district.Method based on rarefaction representation can effectively utilize the openness of image, and Incomplete image and complete image have the character of identical sparse coding coefficient under super complete dictionary so that Incomplete image is able to extensive Multiple.

The square Baolong of Shandong University is at its academic dissertation " image repair algorithm research based on textures synthesis " ([classification number] A kind of image mending method based on textures synthesis is proposed in TP391.41,2013).The method is by calculating object block and source region Territory block respective pixel color difference quadratic sum determines best matching blocks, utilizes and has lost the texture information around frame, Fill up Incomplete image.The weak point that the method exists is, owing to not accounting for the order of texture breeding, is difficult to preferably Repair the structural information of texture, repair image discontinuous, easily produce blocking effect.

Patent " a kind of Fingerprint diretion meter based on the Partial Differential Equation Model improved that Nanjing Information engineering Univ applies at it Calculation method " (application number: 2015101027345 applyings date: 2015-03-09 publication number: CN104680148A) discloses one Fingerprint image orientation computational methods based on partial differential equation.The method mainly includes two steps: first, uses basic ladder The inceptive direction field of degree Algorithm for Solving fingerprint；Second, use the Partial Differential Equation Model improved to smooth inceptive direction field.The party The weak point that method exists is can not to keep the texture information of image in image information partial differential diffusion process, be only used for Zonule image repair, when target area to be repaired is bigger, repairs result and can produce fuzzy, cause bigger distortion.

Nonlinear Diffusion methods based on partial differential equation can not keep the texture information of image in mending course, repairs out The image border come is discontinuous；Textures synthesis method does not accounts for the structural information of image block, is difficult to preferably repair texture Structural information, easily produces blocking effect；Image mending method based on rarefaction representation, needs to carry out image block at vectorization Reason, destroys the structural information of image block, and during repairing, repairs, do not have in units of single image block Effectively utilizing texture and the spatial structural form of image, the image outline after repairing is less continuous, can produce certain distortion.

Summary of the invention

Present invention aims to the deficiency of above-mentioned prior art, propose one and can complete image quickly and accurately The image mending method based on the many dictionary learnings of tensor structure Yu sparse coding repaired.

The present invention is a kind of image mending method based on the many dictionary learnings of tensor structure Yu sparse coding, including following step Rapid:

Step 1 is by the image construction tensor sample in training sample database and classifies:

1a) image in training sample database is divided into the image block that size is m × n, by non-smooth image block and its sky Between 8 neighbour's image blocks constitute tensor samplest _k∈R^m×n×9, randomly select 100 000 tensor samples as training sample setWherein R^m×n×9Represent that each tensor size is m × n × 9.

1b) the tensor sample that training sample is concentrated is divided into structure class and non-structural class, according to directivity and Near-neighbor Structure Information, further segments structure class.

Step 2 carries out tensor dictionary learning to the tensor sample of each class:

Respectively the tensor sample of each class is learnt with tensor dictionary learning algorithm, obtain L and cross complete tensor word Allusion quotationWhereinRepresent that the i-th class tensor sample is at mould 1, mould 2, the dictionary on mould 3 respectively.

Tensor to be repaired is classified by step 3:

Image to be repaired is divided into the overlapping image block that size is m × n, by each image block to be repaired and its space 8 neighbour's image blocks constitute tensor to be repairedy _k∈R^m×n×9, according to the classification in step 1, obtain the classification of tensor to be repaired Label l.

Step 4 judges the classification of tensor to be repaired, repairs with the tensor dictionary that class label is corresponding therewith:

4a) by tensor to be repairedy _kRemove defect part and obtain tensorWith tensor sparse coding to be repaired Mend tensor and carry out sparse coding under the tensor dictionary that class label therewith is corresponding, obtain code coefficientc _k,

$\underset{{\underset{\‾}{c}}_k}{\min }\left|\right|{\underset{\‾}{\tilde{y}}}_k-{\underset{\‾}{y}}_k\×{\mathrm{\Φ}_1}_1\×{\mathrm{\Φ}_2}_2\×{\mathrm{\Φ}_3}_3|{|}_2^2+||{\underset{\‾}{\tilde{y}}}_k-{\underset{\‾}{c}}_k\×\tilde{D}_1^l{}_1\×\tilde{D}_2^l{}_2\×\tilde{D}_3^l{}_3|{|}_2^2+\mathrm{\λ}\left|\right|{\underset{\‾}{c}}_k|{|}_1$

WhereinRepresent that removing defect part obtains tensorSize be m₀×n₀× 9, l represent tensor to be repaired Class label, Φ_jRepresent the tensor to be repaired perception matrix on j mould,Represent l class tensor j mould dictionaryIn perception Matrix Φ_jUnder dictionary, i.e. c _kFor sparse under l class tensor structure dictionary of tensor to be repaired Code coefficient, | | | |₁Representing 1 norm, λ controls the degree of rarefication of sparse coding coefficient；

4b) it is calculated the tensor after repairing

Tensor after repairing is rebuild by step 5:

5a) calculate the tensor after repairingReconstructed results under multiclass tensor structure dictionary

${\underset{\‾}{\hat{y}}}_k^i-{\underset{\‾}{c}}_k^i\×D_1^i{}_1\×D_2^i{}_2\×D_3^i{}_3$

Wherein, sparse coding coefficientPass throughSolve and obtain,For repairing Tensor after benefitSparse coding coefficient under the i-th class tensor structure dictionary,For the tensor after repairingAt the i-th class tensor Reconstructed results under structure dictionary；

The tensor after 5b) calculating is repaired reconstruction weights under multiclass tensor structure dictionary:

$w_i=\frac{1}{L-1}(1-\frac{e_i}{\underset{i=1}{\overset{L}{\Σ}}{e}_i})$

Wherein, w_iRepresenting the weights under the i-th class tensor structure dictionary of the tensor after repairing, L represents in structure dictionary and divides The number of dictionary, Σ represents sum operation, e_iRepresent that the tensor after repairing misses in the reconstruction of the i-th class tensor dictionary sparse coding Difference, i.e.

5c) to the tensor reconstructed results under multiclass tensor structure dictionary after repairingIt is weighted summation, obtains The reconstructed results of whole tensor to be repairedx _k,

${\underset{\‾}{x}}_k=\underset{i=1}{\overset{L}{\Σ}}{w}_i{\underset{\‾}{\hat{y}}}_k^i$

Tensor pie graph picture after rebuilding, completes image mending.

First image block and its space 8 neighbour's image block are constituted tensor by the present invention, it is to avoid image block is carried out vector Change processes, and maintains the structural information in image block, and then directivity and Near-neighbor Structure information according to tensor are to tensor sample Classifying, set up sparse super complete tensor dictionary respectively, classification carries out image mending.The present invention can complete quickly and accurately Image mending, relatively accurately recovers the partial structurtes information of image, it is thus achieved that image mending result clearly.

The present invention compared with prior art has the advantage that

First, due to the method that present invention employs image tensor block sort, then tensor to be repaired is repaired, overcome Reconstruction independent to image block in prior art, it is impossible to the shortcoming using image block local neighbor structural information so that this Bright to be provided with the repairing time short, the advantage that accuracy is high；

Second, owing to the present invention make use of the directivity in image tensor block and neighbour during image tensor block sort Structural information, overcomes in existing Unsupervised clustering technology and clusters inaccurate shortcoming so that the present invention is provided with repairing details Information advantage clearly；

Accompanying drawing explanation

Fig. 1 is the flow chart of the present invention；

Fig. 2 is image tensor classification schematic diagram；

Fig. 3 is Lena image, and wherein 3 (a) is original Lena image, and 3 (b) is impaired Lena image；

Fig. 4 is that Lena image mending result and additive method are repaired the comparison diagram of result by the present invention, and wherein 4 (a) is The result figure that Criminisi method is repaired, 4 (b) is the result figure that sparse representation method is repaired, and 4 (c) is the knot that the present invention repairs Fruit figure；

Fig. 5 is Peppers image, and wherein 5 (a) is original Peppers image, and 5 (b) is impaired Peppers image；

Fig. 6 is that Peppers image mending result and additive method are repaired the comparison diagram of result by the present invention, and wherein 6 (a) is The result figure that Criminisi method is repaired, 6 (b) is the result figure that sparse representation method is repaired, and 6 (c) is the knot that the present invention repairs Fruit figure.

Detailed description of the invention

Describe the present invention below in conjunction with the accompanying drawings.

Embodiment 1

Traditional image mending method does not has effectively use the Near-neighbor Structure information in image block to be repaired and texture Information, causes the image repairing out fuzzyyer, has bigger distortion.To this end, the present invention proposes a kind of based on tensor The many dictionary learnings of structure and the image mending method of sparse coding, see Fig. 1, comprise the following steps:

Step 1 is by the image construction tensor sample in training sample database and classifies:

1a) image in training sample database is divided into the image block that size is m × n, by non-smooth image block and its sky Between 8 neighbour's image blocks constitute tensor samplest _k∈R^m×n×9, randomly select 100 000 tensor samples as training sample setWherein R^m×n×9Representing that each tensor size is m × n × 9, k represents that tensor sample is at training sample set In index.In the present embodiment, m=9, n=9, when the affected area in image is bigger when, the numerical value of m and n is permissible Suitably increase.

1b) the tensor sample that training sample is concentrated is divided into structure class and non-structural class, according to directivity, structure class is divided For structure subclass, further according to Near-neighbor Structure information, with K-means algorithm, structure subclass is segmented further, by each structure Subclass is further divided into K class.

Step 2 carries out tensor dictionary learning to the tensor sample of each class, and including non-structural class and structure apoplexy due to endogenous wind, each is thin The subclass divided:

Respectively the tensor sample of each class is learnt with tensor dictionary learning algorithm, obtain L and cross complete tensor word Allusion quotationWhereinRepresent that the i-th class tensor sample is at mould 1, mould 2, the dictionary on mould 3 respectively.

Tensor to be repaired is classified by step 3:

Image to be repaired is divided into the overlapping image block that size is m × n, by each image block to be repaired and space 8 Neighbour's image block constitutes tensor to be repairedy _k∈R^m×n×9, according to the classification in step 1, obtain the classification of tensor to be repaired Label l.

Step 4 judges the classification of tensor to be repaired, repairs with the tensor dictionary that class label is corresponding therewith:

First determining whether whether tensor to be repaired belongs to structure class, if being not belonging to structure class, carrying out with non-structural category dictionary Repairing, if belonging to structure class, then repairing with structure apoplexy due to endogenous wind tensor dictionary, in a word with the tensor that class label is corresponding therewith Dictionary is repaired.

4a) by tensor to be repairedy _kRemove defect part and obtain tensorTensor dictionary finds and treats with this Repair the corresponding tensor dictionary of tensor class label l, with tensor sparse coding, tensor to be repaired is entered under this tensor dictionary Row sparse coding, obtains code coefficientc _k,

$\underset{{\underset{\‾}{c}}_k}{\min }\left|\right|{\underset{\‾}{\tilde{y}}}_k-{\underset{\‾}{y}}_k\×{\mathrm{\Φ}_1}_1\×{\mathrm{\Φ}_2}_2\×{\mathrm{\Φ}_3}_3|{|}_2^2+||{\underset{\‾}{\tilde{y}}}_k-{\underset{\‾}{c}}_k\×\tilde{D}_1^l{}_1\×\tilde{D}_2^l{}_2\×\tilde{D}_3^l{}_3|{|}_2^2+\mathrm{\λ}\left|\right|{\underset{\‾}{c}}_k|{|}_1$

WhereinRepresent that removing defect part obtains tensorSize be m₀×n₀× 9, l represent tensor to be repaired Class label, Φ_jRepresent the tensor to be repaired perception matrix on j mould,Represent l class tensor j mould dictionaryIn perception Matrix Φ_jUnder dictionary, i.e. c _kFor sparse under l class tensor structure dictionary of tensor to be repaired Code coefficient, | | | |₁Representing 1 norm, λ controls the degree of rarefication of sparse coding coefficient.The value of λ is between (0,1).In this example In, λ value is 0.5, and in actual applications, the value of λ can suitably adjust as required in interval.

4b) it is calculated the tensor after repairing

Step 5 carries out image reconstruction to the tensor after repairing:

5a) calculate the tensor after repairingReconstructed results under multiclass tensor structure dictionary

${\underset{\‾}{\hat{y}}}_k^i-{\underset{\‾}{c}}_k^i\×D_1^i{}_1\×D_2^i{}_2\×D_3^i{}_3$

Wherein, sparse coding coefficientPass throughSolve and obtain,For repairing Tensor after benefitSparse coding coefficient under the i-th class tensor structure dictionary,For the tensor after repairingOpen in the i-th class Reconstructed results under amount structure dictionary.

In the present invention, multiclass tensor structure dictionary is the tensor dictionary of all categories learning in step 2 to obtain.Pass through Tensor after repairing is rebuild in multiclass tensor dictionary, and single tensor dictionary can be overcome to represent some structural information Inaccurate shortcoming, improves repairing quality further.

The tensor after 5b) calculating is repaired reconstruction weights under multiclass tensor structure dictionary:

$w_i=\frac{1}{L-1}(1-\frac{e_i}{\underset{i=1}{\overset{L}{\Σ}}{e}_i})$

Wherein, w_iRepresenting the weights under the i-th class tensor structure dictionary of the tensor after repairing, L represents in structure dictionary and divides The number of dictionary, Σ represents sum operation, e_iRepresent that the tensor after repairing misses in the reconstruction of the i-th class tensor dictionary sparse coding Difference, i.e.

5c) to the tensor reconstructed results under multiclass tensor structure dictionary after repairingIt is weighted summation, obtains The reconstructed results of whole tensor to be repairedx _k,

${\underset{\‾}{x}}_k=\underset{i=1}{\overset{L}{\Σ}}{w}_i{\underset{\‾}{\hat{y}}}_k^i$

Tensor pie graph picture after rebuilding, completes image mending.

Method based on rarefaction representation needs image block is carried out vectorization process, destroys the structural information of image block, And during repairing, repair in units of single image block, do not account for the Near-neighbor Structure between image block Information.Image block and its space 8 neighbour's image block are constituted tensor, according to directivity and the Near-neighbor Structure information of tensor by the present invention Tensor sample is classified, then at each apoplexy due to endogenous wind fabric tensor dictionary, then tensor to be repaired is carried out under specific dictionary Repair, rebuild in multiclass tensor dictionary by the tensor after repairing, can overcome single tensor dictionary that some is tied The inaccurate shortcoming of structure information representation, improves repairing quality to greatest extent so that the repairing time of the present invention is short, the standard of repairing Really property is higher.

Embodiment 2

Image mending method based on the many dictionary learnings of tensor structure and sparse coding is with embodiment 1, described in step 1b The tensor sample that training sample is concentrated is divided into structure class and the concrete steps of non-structural class, sees Fig. 2, as follows:

1b.1) construct the gradient tensor of each tensor sample

1b.2) to gradient tensorCarry out HOSVD decomposition, as follows

Wherein A, B, C represent gradient tensor respectivelyAt mould 1, mould 2, the dictionary that the left singular vector of mould 3 matrix is constituted, It it is gradient tensorAt dictionary A, the core tensor on B, C, P, Q, R represent core tensor respectivelyOne-dimensional, two and three dimensions Length on direction, g_pqrRepresent core tensor(p, q, r) value of position, ×_iRepresent the multiplication on the i mould of tensor, ο table Show the apposition of vector；

1b.3) according to the following formula, gradient tensor is calculatedAt direction b₁,b₂On close degree v₁,v₂

$v_i=\underset{p=1}{\overset{P}{\Σ}}\underset{r=1}{\overset{R}{\Σ}}\left|g_{pir}\right|,i=1,2$

Wherein b₁,b₂Represent gradient tensor respectivelyIn mould 2 dictionary B first and second column vector；

1b.4) according to the following formula, gradient tensor is calculatedDirection value:

By direction valueTensor sample less than TH is as non-structural class, by direction valueTensor sample more than TH is made For structure class, whereinRepresent gradient tensorDirection value, v₁And v₂It is respectively gradient tensor at b₁,b₂On phase short range Degree.In the present invention, TH is to weigh structure class and the threshold value in non-structural, selected according to operation, generally at 0.6-in categorizing process Choosing between 0.9, in the present embodiment, TH value is 0.7, and when image block is bigger when, the numerical value of TH can suitably subtract Little.

1b.5) according to the following formula, the angle of computation structure class tensor sample:

The angle [alpha] of structure class tensor sample is divided into a class every 10 ° in scope-90 °～90 °, is averagely divided into 18 Class, obtains sorted tensor sample, and wherein, α represents the angle of structure class tensor sample, and π represents pi, b₁And b (1)₁ (2) vector b is represented respectively₁The first two value, arctan represent antitrigonometric function arc tangent operation.In actual applications, may be used With as required, adjust the interval of angle, thus change the classification number of classification.When the structural information of image is more complicated, need The when that the classification number of classification being more, angle interval can appropriateness reduce, thus the sample in each classification is more like.

1b.6) with gradient tensor that structure class tensor sample is correspondingFirst atom c of mould 3 dictionary C₁As spy Levy, the tensor sample in same structure subclass is carried out K-means cluster, by the tensor sample of each structon apoplexy due to endogenous wind again It is further divided into K class, completes the segmentation to each structure class tensor sample.In the present invention, K represents each structon The classification number that the tensor sample of apoplexy due to endogenous wind is classified further, usual value is the positive number more than 5.In this example, the value of K is 8.

The present invention make use of the directivity of tensor that tensor sample is divided into structure class and non-in image tensor categorizing process Structure class, further according to Near-neighbor Structure information, is further divided into K class by each structure subclass.This sorting technique has used tensor Directivity and Near-neighbor Structure information, overcome and cluster inaccurate shortcoming in existing unsupervised clustering so that each apoplexy due to endogenous wind Tensor sample there is higher structural similarity, thus improve the present invention and repair the ability of detailed information.

Embodiment 3

Image mending method based on the many dictionary learnings of tensor structure and sparse coding is with embodiment 1-2, step 1b.1 institute The gradient tensor constructing each tensor sample statedSpecifically comprise the following steps that

1b.1.1) calculate each image block I in tensor sample_jWith this tensor center of a sample image block I_iSimilarity:

$d_j=\exp (-\frac{\left|\right|I_i-I_j|{|}_2^2}{h^2})$

Wherein, d_jRepresent the image block I in tensor sample_jWith its center image block I_iSimilarity, h for control image block Between the smoothing parameter of similarity, the span of h is 1 to the positive number between hundreds of, and in this example, the value of h is 20.

1b.1.2) according to the following formula, each image block I in tensor sample is calculated_jIn each pixel in the horizontal direction and hang down Nogata gradient upwards:

$\▿f_j(x_k,y_k)=\[I_j(x_k,y_k)-I_j(x_{k-1},y_k),I_j(x_k,y_k)-I_j(x_k,y_{k-1})\]$

Wherein,Represent the image block I of tensor sample_jThe ladder that middle kth point is horizontally and vertically gone up Degree, (x_k,y_k) represent the coordinate at kth pixel, by image block I_jIn the gradient of all pixels according to following form structure Become gradient matrix G_j,

$G_j=\left[\begin{array}{c}\▿f_j(1,1)\\ \▿f_j(1,2)\\ \·\\ \·\\ \·\\ \▿f_j(m,n)\end{array}\right];$

1b.1.3) by each image block I in tensor sample_jGradient matrix G_jIt is multiplied by this image block and this tensor respectively Similarity d of center of a sample's image block_j, by the gradient matrix i.e. G after weighting_j×d_jIt is overlapped into gradient tensorGradient tensor

The present invention, during structure gradient tensor, has simultaneously taken account of gradient information and the image of image block in tensor Similarity between block so that this gradient tensor has texture and the ability of structural information of more preferably reflection tensor, for next step Tensor classification provide more preferable foundation.

Embodiment 4

Image mending method based on the many dictionary learnings of tensor structure and sparse coding is with embodiment 1-3, referring to the drawings 1, The present invention is implemented step be further described.

Step 1, forms tensor sample by the image in training sample database and classifies:

1a) image in training sample database is divided into the image block that size is m × n, non-smooth image block is non-with this Space 8 neighbour's image block of smooth image block constitutes tensor samplet _k∈R^m×n×9, randomly select 100 000 tensor samples As training sample setWherein R^m×n×9Representing that each tensor size is m × n × 9, k represents tensor sample This index concentrated at training sample.In the present embodiment, m=8, n=8, when the affected area in image is bigger when, The numerical value of m and n can suitably increase.

1b) the tensor sample that training sample is concentrated is divided into structure class and non-structural class, according to directivity, structure class is divided For structure subclass, further according to Near-neighbor Structure information, with K-means algorithm, structure subclass is segmented further, by each structure Subclass is further divided into K class.

1b.1) construct the gradient tensor of each tensor sample

1b.1.1) calculate each image block I in tensor sample_jWith this tensor center of a sample image block I_iSimilarity:

$d_j=\exp (-\frac{\left|\right|I_i-I_j|{|}_2^2}{h^2})$

Wherein, d_jRepresent the image block I in tensor sample_jWith its center image block I_iSimilarity, h for control image block Between the smoothing parameter of similarity.

1b.1.2) according to the following formula, each image block I in tensor sample is calculated_jIn each pixel in the horizontal direction and hang down Nogata gradient upwards:

$\▿f_j(x_k,y_k)=\[I_j(x_k,y_k)-I_j(x_{k-1},y_k),I_j(x_k,y_k)-I_j(x_k,y_{k-1})\]$

Wherein,Represent the image block I of tensor sample_jThe ladder that middle kth point is horizontally and vertically gone up Degree, (x_k,y_k) represent the coordinate at kth pixel, by image block I_jIn the gradient of all pixels according to following form structure Become gradient matrix G_j,

$G_j=\left[\begin{array}{c}\▿f_j(1,1)\\ \▿f_j(1,2)\\ \·\\ \·\\ \·\\ \▿f_j(m,n)\end{array}\right];$

1b.1.3) by each image block I in tensor sample_jGradient matrix G_jIt is multiplied by this image block and this tensor respectively Similarity d of center of a sample's image block_j, by the gradient matrix i.e. G after weighting_j×d_jIt is overlapped into gradient tensorGradient tensor

1b.2) to gradient tensorCarry out HOSVD decomposition, carry out according to the following formula:

Wherein A, B, C represent gradient tensor respectivelyAt mould 1, mould 2, the dictionary that the left singular vector of mould 3 matrix is constituted, It it is gradient tensorAt dictionary A, the core tensor on B, C, P, Q, R represent core tensor respectivelyOne-dimensional, two and three dimensions Length on direction, g_pqrRepresent core tensor(p, q, r) value of position, ×_iRepresent the multiplication on the i mould of tensor, ο table Show the apposition of vector.

1b.3) according to the following formula, gradient tensor is calculatedAt direction b₁,b₂On close degree v₁,v₂

$v_i=\underset{p=1}{\overset{P}{\Σ}}\underset{r=1}{\overset{R}{\Σ}}\left|g_{pir}\right|,i=1,2$

Wherein b₁,b₂Represent gradient tensor respectivelyIn mould 2 dictionary B first and second column vector.

1b.4) according to the following formula, gradient tensor is calculatedDirection value:

By direction valueTensor sample less than TH is as non-structural class, by direction valueTensor sample more than TH is made For structure class, whereinRepresent gradient tensorDirection value, v₁And v₂It is respectively gradient tensor at b₁,b₂On phase short range Degree.In the present embodiment, TH value 0.7.

1b.5) according to the following formula, the angle of computation structure class tensor sample:

The angle [alpha] of structure class tensor sample is divided into a class every 10 ° in scope-90 °～90 °, is averagely divided into 18 Class, obtains sorted tensor sample, and wherein, α represents the angle of structure class tensor sample, and π represents pi, b₁And b (1)₁ (2) vector b is represented respectively₁The first two value, arctan represent antitrigonometric function arc tangent operation.In actual applications, root According to needing to adjust the interval of angle, change the classification number of classification.

1b.6) with gradient tensor that structure class tensor sample is correspondingFirst atom c of mould 3 dictionary C₁As spy Levy, the tensor sample in same structure subclass is carried out K-means cluster, by the tensor sample of each structon apoplexy due to endogenous wind again It is further divided into K class, completes the segmentation to each structure class tensor sample.

Step 2, tensor dictionary learning:

Respectively the tensor sample of each class is learnt with tensor dictionary learning algorithm, obtain L and cross complete tensor word Allusion quotationWhereinRepresent that the i-th class tensor sample is at mould 1, mould 2, the dictionary on mould 3 respectively.

All of sample is not learnt by the present invention with single dictionary, but by all of tensor sample according to direction Property and Near-neighbor Structure information are first classified, and then learn tensor dictionary in each classification respectively, this approach reduces dictionary The complexity practised, improves the accuracy of dictionary, thus has reached reasonable reconstructed results.

Step 3, classifies to tensor to be repaired:

Image to be repaired is divided into the overlapping image block that size is m × n, each image block to be repaired is empty with it Between 8 neighbour's image blocks constitute tensors to be repairedy _k∈R^m×n×9, according to the classification in step 1, tensor to be repaired is divided into Structure class and non-structural class, further according to directivity and Near-neighbor Structure information, segment further structure class, obtain its class label l.Here image to be repaired is divided into the image block of overlap, causes because of piecemeal reconstruction image ratio more serious to reduce Blocking effect, promotes the reconstruction effect of image.

Step 4, repairs the tensor to be repaired tensor dictionary that class label is corresponding therewith:

4a) by tensor to be repairedy _kRemove defect part and obtain tensorWith tensor sparse coding to be repaired Tensor carries out sparse coding under the tensor dictionary of corresponding classification therewith and obtains code coefficientc _k, namely minimum to encoding error Change,

$\underset{{\underset{\‾}{c}}_k}{\min }\left|\right|{\underset{\‾}{\tilde{y}}}_k-{\underset{\‾}{y}}_k\×{\mathrm{\Φ}_1}_1\×{\mathrm{\Φ}_2}_2\×{\mathrm{\Φ}_3}_3|{|}_2^2+||{\underset{\‾}{\tilde{y}}}_k-{\underset{\‾}{c}}_k\×\tilde{D}_1^l{}_1\×\tilde{D}_2^l{}_2\×\tilde{D}_3^l{}_3|{|}_2^2+\mathrm{\λ}\left|\right|{\underset{\‾}{c}}_k|{|}_1$

WhereinRepresent that removing defect part obtains tensorSize be m₀×n₀× 9, l represent tensor to be repaired Class label, Φ_iRepresent the tensor to be repaired perception matrix on i mould,Represent l class tensor i mould dictionaryIn perception Matrix Φ_iUnder dictionary, i.e.||·||₁Representing 1 norm, λ controls the degree of rarefication of sparse coding coefficient.In this example In, the value of λ is 0.2.

4b) it is calculated the tensor after repairing

Step 5, image reconstruction:

5a) calculate the tensor after repairingReconstructed results under multiclass tensor structure dictionary

${\underset{\‾}{\hat{y}}}_k^i-{\underset{\‾}{c}}_k^i\×D_1^i{}_1\×D_2^i{}_2\×D_3^i{}_3$

Wherein, sparse coding coefficientPass throughSolve and obtain,For repairing Tensor after benefitSparse coding coefficient under the i-th class tensor structure dictionary,For the tensor after repairingOpen in the i-th class Reconstructed results under amount structure dictionary；

The tensor after 5b) calculating is repaired reconstruction weights under multiclass tensor structure dictionary:

$w_i=\frac{1}{L-1}(1-\frac{e_i}{\underset{i=1}{\overset{L}{\Σ}}{e}_i})$

Wherein, w_iRepresenting the weights under the i-th class tensor structure dictionary of the tensor after repairing, L represents in structure dictionary and divides The number of dictionary, Σ represents sum operation, e_iRepresent that the tensor after repairing misses in the reconstruction of the i-th class tensor dictionary sparse coding Difference, i.e.

5c) to the tensor reconstructed results under multiclass tensor structure dictionary after repairingIt is weighted summation, obtains The reconstructed results of whole tensor to be repairedx _k,

${\underset{\‾}{x}}_k=\underset{i=1}{\overset{L}{\Σ}}{w}_i{\underset{\‾}{\hat{y}}}_k^i$

Tensor pie graph picture after rebuilding, completes image mending.

The effect of the present invention can be further illustrated by following emulation experiment:

Embodiment 5

Image mending method based on the many dictionary learnings of tensor structure and sparse coding with embodiment 1-4,

1. emulation experiment condition and method:

Hardware test platform is: processor is Intel Core2CPU, and dominant frequency is 2.33GHz, internal memory 2GB,

Software platform is: Windows XP operating system and Matlab R2012a；

Experimental technique: be respectively the present invention, Criminisi method and image mending method based on rarefaction representation.

2. emulation experiment content:

Under these experimental conditions, following experiment is carried out.

Emulation one, utilizes the present invention and existing two kinds of methods, carries out repairing treatment to containing damaged image 3 (b) to be repaired, its White text part in middle Fig. 3 (b) is affected area, repairs result as shown in Figure 4.Wherein Fig. 4 (a) is Criminisi side The result figure that method is repaired, Fig. 4 (b) is the result figure that method based on rarefaction representation is repaired, and Fig. 4 (c) is the knot that the present invention repairs Fruit figure.

Result Fig. 4 (a) above-mentioned three kinds of methods repaired respectively, Fig. 4 (b) and Fig. 4 (c) is carried out with non-breakage image 3 (a) Relatively, from the point of view of visual effect, after the repairing of Criminisi algorithm, face and the label edge of image occur in that the obvious flaw Defect, details cannot preferably be processed by method based on rarefaction representation, and after repairing, the edge of image creates certain mould Stick with paste.As shown in Fig. 4 (c), the image repaired through the present invention is visible, maintains preferable structural information, the result figure tool after repairing Having more details, outline portion becomes apparent from nature.

If visual experience is the most variant to the resolution of image mending result, numerical indication more can accurately reflect image Quality reconstruction.

Calculate above-mentioned three kinds of methods Y-PSNR PSNR and knot to repairing result containing impaired Fig. 3 (b) to be repaired respectively Structure similarity SSIM, the effect of its index expression distinct methods repairing image, numerical value is the biggest, illustrates that repair efficiency is the best.Its knot Fruit is as shown in table 1.

Table 1: the present invention and the contrast of existing methods experiment result

The present invention repairs out the Y-PSNR PSNR and structural similarity SSIM of image than existing as can be seen from Table 1 Some both of which improve a lot.

Above test result indicate that, the present invention, no matter in objective indicator or visual effect, shows preferably Performance, keeping while effect structure, improve the repairing result of image.

Embodiment 6

Image mending method based on the many dictionary learnings of tensor structure and sparse coding is with embodiment 1-5, emulation experiment bar Part and method are with embodiment 5:

Emulation two, utilizes the present invention and existing two kinds of methods, carries out repairing treatment to containing damaged image 5 (b) to be repaired, its White ' # ' part in middle Fig. 5 (b) is affected area, repairs result as shown in Figure 6.Wherein Fig. 6 (a) is Criminisi method The result figure repaired, Fig. 6 (b) is the result figure that method based on rarefaction representation is repaired, and Fig. 6 (c) is the result that the present invention repairs Figure.

Result Fig. 6 (a) above-mentioned three kinds of methods repaired respectively, Fig. 6 (b) and Fig. 6 (c) is carried out with non-breakage image 5 (a) Relatively, from the point of view of visual effect, Criminisi algorithm repair after image, the most in the picture between the pars intermedia of Pepper Position leaves horizontal and vertical vestige significantly, creates more significantly blocking effect.After method based on rarefaction representation is repaired Image former affected area on long capsicum have vertical blurred trace, dark Pepper has obvious landscape blur Vestige.As shown in Fig. 6 (c), the image repaired through the present invention is visible, maintains preferable overall structure information, repairs result clear Clear nature, is closer to artwork.

The image reconstruction effect of the present invention and prior art is contrasted further from numerical indication.

Calculate above-mentioned three kinds of methods Y-PSNR PSNR and knot to repairing result containing impaired Fig. 5 (b) to be repaired respectively Structure similarity SSIM, the effect of its index expression distinct methods repairing image, numerical value is the biggest, illustrates that repair efficiency is the best.Its knot Fruit is as shown in table 2.

Table 2: the present invention and the contrast of existing methods experiment result

The present invention repairs out the Y-PSNR PSNR and structural similarity SSIM of image than existing as can be seen from Table 2 Some both of which improve a lot.

Above test result indicate that, the present invention, no matter in objective indicator or subjective effect, shows preferably Performance, keeping while effect structure, improve the repairing result of image.

In brief, the image mending method based on the many dictionary learnings of tensor structure Yu sparse coding of the present invention, it is main Wanting step to include: tensor sample is classified by directivity and Near-neighbor Structure information according to tensor, each class constructs respectively opens Amount dictionary, adopts image tensor to be repaired and classifies in the same way, enters it with the tensor dictionary that class label is corresponding therewith Row is repaired, and the reconstructed results under every category dictionary is weighted summation, obtains the reconstructed results of final image to be repaired.This The bright directivity according to tensor and Near-neighbor Structure information carry out classification effective, deterministic to tensor, can well distinguish tool There is the tensor of different details, when tensor to be repaired is rebuild, utilize reconstruction error that the reconstructed results under every category dictionary is carried out Weighted sum, overcomes the shortcoming that single dictionary ability to express is limited, the repairing to natural image, can recover relatively sharp limit Edge details, further increases repairing quality.Application is image mending.

Claims (3)

1. an image mending method based on the many dictionary learnings of tensor structure Yu sparse coding, it is characterised in that include with Lower step:

Step 1 is by the image construction tensor sample in training sample database and classifies:

1a) image in training sample database is divided into the image block that size is m × n, by near to non-smooth image block and its space 8 Adjacent image block constitutes tensor samplet _k∈R^m×n×9, randomly select 100 000 tensor samples as training sample set Wherein R^m×n×9Represent that each tensor size is m × n × 9；

1b) the tensor sample that training sample is concentrated is divided into structure class and non-structural class, according to directivity and Near-neighbor Structure information, Structure class is further segmented；

Step 2 carries out tensor dictionary learning to the tensor sample of each class:

Respectively the tensor sample of each class is learnt with tensor dictionary learning algorithm, obtain L and cross complete tensor dictionaryWhereinRepresent that the i-th class tensor sample is at mould 1, mould 2, the dictionary on mould 3 respectively；

Tensor to be repaired is classified by step 3:

Image to be repaired is divided into the overlapping image block that size is m × n, by near to each image block to be repaired and its space 8 Adjacent image block constitutes tensor to be repairedy _k∈R^m×n×9, according to the classification in step 1, obtain the classification mark of tensor to be repaired Sign l；

Step 4 judges the classification of tensor to be repaired, repairs with the tensor dictionary that class label is corresponding therewith:

4a) by tensor to be repairedy _kRemove defect part and obtain tensorWith tensor sparse coding to tensor to be repaired Under the tensor dictionary that class label therewith is corresponding, carry out sparse coding, obtain code coefficientc _k,

$\underset{{\underset{\‾}{c}}_k}{min}\left|\right|{\underset{\‾}{\tilde{y}}}_k-{\underset{\‾}{y}}_k\×{\mathrm{\Φ}_1}_1\×{\mathrm{\Φ}_2}_2\×{\mathrm{\Φ}_3}_3|{|}_2^2+||{\underset{\‾}{\tilde{y}}}_k-{\underset{\‾}{c}}_k\×\tilde{D}_1^l{}_1\×\tilde{D}_2^l{}_2\×\tilde{D}_3^l{}_3|{|}_2^2+\mathrm{\λ}\left|\right|{\underset{\‾}{c}}_k|{|}_1$

WhereinRepresent that removing defect part obtains tensorSize be m₀×n₀× 9, l represent the class of tensor to be repaired Distinguishing label, Φ_jRepresent the tensor to be repaired perception matrix on j mould,Represent l class tensor j mould dictionaryAt perception matrix Φ_jUnder dictionary, i.e. c _kFor the tensor to be repaired sparse coding under l class tensor structure dictionary Coefficient, | | | |₁Representing 1 norm, λ controls the degree of rarefication of sparse coding coefficient；

4b) it is calculated the tensor after repairing

Tensor after repairing is rebuild by step 5:

5a) calculate the tensor after repairingReconstructed results under multiclass tensor structure dictionary

${\underset{\‾}{\hat{y}}}_k^i={\underset{\‾}{c}}_k^i\×D_1^i{}_1\×D_2^i{}_2\×D_3^i{}_3$

Wherein, sparse coding coefficientPass throughSolve and obtain,After repairing TensorSparse coding coefficient under the i-th class tensor structure dictionary,For the tensor after repairingIn the i-th class tensor structure Reconstructed results under dictionary；

The tensor after 5b) calculating is repaired reconstruction weights under multiclass tensor structure dictionary:

$w_i=\frac{1}{L-1}(1-\frac{e_i}{\underset{i=1}{\overset{L}{\Σ}}{e}_i})$

Wherein, w_iRepresenting the weights under the i-th class tensor structure dictionary of the tensor after repairing, L represents point dictionary in structure dictionary Number, Σ represents sum operation, e_iRepresent the reconstruction error at the i-th class tensor dictionary sparse coding of the tensor after repairing, i.e.

5c) to the tensor reconstructed results under multiclass tensor structure dictionary after repairingIt is weighted summation, obtains final The reconstructed results of tensor to be repairedx _k,

${\underset{\‾}{x}}_k=\underset{i=1}{\overset{L}{\Σ}}{w}_i{\underset{\‾}{\hat{y}}}_k^i$

Tensor pie graph picture after rebuilding, completes image mending.

Image mending method based on the many dictionary learnings of tensor structure Yu sparse coding the most according to claim 1, step The tensor sample by training sample concentration described in 1b is divided into structure class and non-structural class, it is characterised in that include following step Rapid:

1b.1) construct the gradient tensor of each tensor sample

1b.2) to gradient tensorCarry out HOSVD decomposition, carry out according to the following formula:

Wherein A, B, C represent gradient tensor respectivelyAt mould 1, mould 2, the dictionary that the left singular vector of mould 3 matrix is constituted,It it is ladder Degree tensorAt dictionary A, the core tensor on B, C, P, Q, R represent core tensor respectivelyOne-dimensional, two and three dimensions direction On length, g_pqrRepresent core tensor(p, q, r) value of position, ×_iRepresent tensor i mould on multiplication, ο represent to The apposition of amount；

1b.3) according to the following formula, gradient tensor is calculatedAt direction b₁,b₂On close degree v₁,v₂:

$v_i=\underset{p=1}{\overset{P}{\Σ}}\underset{r=1}{\overset{R}{\Σ}}\left|g_{pir}\right|,i=1,2$

Wherein b₁,b₂Represent gradient tensor respectivelyIn mould 2 dictionary B first and second column vector；

1b.4) according to the following formula, gradient tensor is calculatedDirection value:

By direction valueTensor sample less than TH is as non-structural class, by direction valueTensor sample more than TH is as knot Structure class, whereinRepresent gradient tensorDirection value, v₁And v₂It is respectively gradient tensor at b₁,b₂On close degree；

1b.5) according to the following formula, the angle of computation structure class tensor sample:

The angle [alpha] of structure class tensor sample is divided into a class every 10 ° in scope-90 °～90 °, is averagely divided into 18 classes, To sorted tensor sample, wherein, α represents the angle of structure class tensor sample, and π represents pi, b₁And b (1)₁(2) respectively Represent vector b₁The first two value, arctan represent antitrigonometric function arc tangent operation；

1b.6) with gradient tensor that structure class tensor sample is correspondingFirst atom c of mould 3 dictionary C₁As feature, to Tensor sample in same structure subclass carries out K-means cluster, by the tensor sample of each structon apoplexy due to endogenous wind further It is divided into K class, completes the segmentation to each structure class tensor sample.

Tensor sample by training sample concentration the most according to claim 2 is divided into structure class and non-structural class, step The gradient tensor constructing each tensor sample described in 1b.1It is characterized in that, include following steps:

1b.1.1) calculate each image block I in tensor sample_jWith this tensor center of a sample image block I_iSimilarity:

$d_j=\exp (-\frac{\left|\right|I_i-I_j|{|}_2^2}{h^2})$

Wherein, d_jRepresent the image block I in tensor sample_jWith its center image block I_iSimilarity, h is for controlling between image block The smoothing parameter of similarity；

1b.1.2) according to the following formula, each image block I in tensor sample is calculated_jIn each pixel both horizontally and vertically On gradient:

$\▿f_j(x_k,y_k)=\[I_j(x_k,y_k)-I_j(x_{k-1},y_k),I_j(x_k,y_k)-I_j(x_k,y_{k-1})\]$

Wherein,Represent the image block I of tensor sample_jThe gradient that middle kth point is horizontally and vertically gone up, (x_k,y_k) represent the coordinate at kth pixel, by image block I_jIn the gradient of all pixels constitute according to following form Gradient matrix G_j,

$G_j=\left[\begin{array}{c}\▿f_j(1,1)\\ \▿f_j(1,2)\\ \begin{array}{c}.\\ .\\ .\end{array}\\ \▿f_j(m,n)\end{array}\right];$

1b.1.3) by each image block I in tensor sample_jGradient matrix G_jIt is multiplied by this image block and this tensor sample respectively Similarity d of center image block_j, by the gradient matrix i.e. G after weighting_j×d_jIt is overlapped into gradient tensorGradient tensor