CN112241938B - Image restoration method based on smooth Tak decomposition and high-order tensor Hakking - Google Patents

Abstract

Hanke based on smooth Take decomposition and high-order tensorA method of image restoration comprising the steps of: 1) Inputting an image to be repairedDetermining an area to be repaired of the image; 2) Constructing a high-order tensor hanke and discrete total variation model; 3) Constructing an image restoration model of smooth Take decomposition and high-order tensor Hakking by combining the step 2), restoring the color image, and finally reconstructing and outputting a high-quality visual data imageThe invention has the advantages that: the efficiency of image processing and the accuracy of image restoration are both considered.

Description

Image restoration method based on smooth Tak decomposition and high-order tensor Hakking

Technical Field

The invention relates to the field of image processing, in particular to an image restoration method.

Background

With the rapid development of modern network technology, computer communication and sampling technology, data to be analyzed mostly have very complex structures. The image data acquisition process is generally affected by various external factors, such as damage to hardware equipment, illumination, electromagnetic interference, and the like, which result in poor visual quality. In this case, the related image data may not be directly re-acquired due to the device or time limitation. Therefore, repairing various kinds of blurred, low-resolution, partial pixel missing and other images existing at present to obtain high-quality visual data is a research content with practical application value.

Image restoration is a typical image processing uncertainty problem, which can be expressed as a missing value estimation problem. The core problem of the missing value estimation is how to establish the relation between the known element and the unknown element, and the problem of image restoration can be effectively solved by adding other priori information, such as local smoothing priori, non-local self-similar priori, sparse priori, low-rank priori, sparse gradient priori and the like. In recent years, many scholars have proposed different image restoration algorithms, which are mainly classified into three categories: 1) Image restoration based on a variational differential equation; 2) Image restoration based on texture synthesis; 3) A mixing method. Bertalmia et al propose an image restoration method based on differential equations for the first time, which restores an image by diffusing information of an unbroken area into the interior of the area to be restored by diffusing the boundary of the area to be restored in different directions. This method has a good repair effect only for the damage of a small area in the image. Chan et al propose a Total Variation (TV) algorithm, which has the greatest advantage of effectively overcoming the problem of smoothing the image edges while suppressing noise by linear filtering, but the TV algorithm has the greatest disadvantage of not meeting the "discontinuous" principle in human vision. The Curvature-driven Diffusion (CDD) algorithm is an improved algorithm to the TV algorithm, which aims to solve the problem of visual discontinuities in the TV algorithm. Criminisi et al propose a sample block-based image restoration algorithm that uses boundary information of the region to be restored to calculate the priority of the block to be restored, and then searches for a sample block with the greatest similarity to the block to be restored in the unbroken region of the image to perform filling restoration. The algorithm has a good repairing effect on a large-area damaged area, but the repairing time is too long, so that the algorithm efficiency is reduced.

With the latest development of deep neural network architecture, the deep learning method has important significance in computer vision tasks such as object detection, image classification, image noise reduction and the like. However, the deep learning-based method requires a large number of marked samples, which are difficult to obtain and consume a large amount of computation, so that research and application of the conventional method are still necessary, and there is a great room for improvement.

Disclosure of Invention

The invention aims to overcome the problems in the prior art and provides an image restoration method based on smooth Take decomposition and high-order tensor Hakking. ,

in order to solve the problem of visual processing of image data distortion, the invention combines Hank knotsThe structuring technique is extended to high-order tensor visual data, and the intrinsic properties of the image are fully considered, and discrete total variation is introduced (Discrete Total Variation, TV) ^d ) The regularization term factors integrate it into a unified objective function.

The technical scheme adopted by the invention for solving the technical problems comprises the following steps:

an image restoration method based on smooth Tak decomposition and high-order tensor Hakking comprises the following steps:

step 1) inputting an image to be repairedDetermining an area to be repaired of an image and performing blocking operation on the area, wherein pixels in the image are divided into known points and unknown points, the known points are points, the pixels in the image are not 0, the unknown points are points, the pixels in the image are 0, and all the unknown points in the image form a set omega;

step 2) constructing a high-order tensor hanke and discrete total variation model;

step 3) combining the image restoration model constructed in the step 2) to restore the color image, and finally reconstructing and outputting a high-quality visual data image

The beneficial effects of the invention are mainly shown in the following steps: the Hank structuring technique is extended for application in the tensor field. Considering that the low rank and smoothness of the matrix also exist in tensors, firstly embedding data into a high-dimensional tensor, and structuring tensor Hanke through multidimensional linear replication and multidimensional folding linear operation; and secondly, taking data smoothness into consideration, introducing a discrete total variation factor for model optimization, and finally, better finding the optimal rank through a low-rank increment algorithm, wherein the algorithm has good convergence and restores the natural image more accurately.

The invention has the advantages that: the efficiency of image processing and the accuracy of image restoration are both considered.

Drawings

FIG. 1 is a schematic diagram of an area to be repaired;

FIG. 2 is a natural image with a pixel loss rate of 90%;

FIG. 3 is a natural image after restoration using the present invention;

fig. 4 is a flow chart of the method of the present invention.

Detailed Description

The technical scheme of the invention is further described below with reference to the accompanying drawings.

An image restoration method based on smooth Tak decomposition and high-order tensor Hakking comprises the following steps:

step 1) inputting an image to be repairedDetermining an area to be repaired of an image and performing blocking operation on the area, wherein pixels in the image are divided into known points and unknown points, the known points are points, the pixels in the image are not 0, the unknown points are points, the pixels in the image are 0, and all the unknown points in the image form a set omega;

step 2) constructing a high-order tensor hanke and discrete total variation model;

step 3) combining the image restoration model constructed in the step 2) to restore the color image, and finally reconstructing and outputting a high-quality visual data image

The processing procedure of the step 2) is as follows:

(2-1) the higher order hank structured image restoration model is defined as follows:

in the method, in the process of the invention,representing an input image to be repaired->Representing post-repair images,/->Representing the Frobenius norm; />Wherein 1 represents an observable pixel and 0 represents a missing pixel;is defined as

Wherein fold _(I,τ) :Through fold _(I,τ) The input N-order tensor can be constructed as a 2N-order tensor, which can be regarded as a multi-dimensional linear copy and multi-dimensional folding operation; wherein the method comprises the steps ofIs a duplication matrix, i.e. a matrix comprising a plurality of identity matrices, in the specific form:

(2-2) discrete total variation model definition is as follows:

x represents a two-dimensional image, v represents a gradient;l. represents bilinear interpolation operations, particularly on a grid(n ₁ ，n ₂ ) Interpolation on; carrying out ^* Representing an accompanying arithmetic operation; definition of discrete operator D is (Dx) ₁ [n ₁ ,n ₂ ]＝x[n ₁ +1,n ₂ ]-x[n ₁ ,n ₂ ]，(Dx) ₂ [n ₁ ,n ₂ ]＝x[n ₁ ,n ₂ +1]-x[n ₁ ,n ₂ ]；

Let l of v _1,1,2 The norms represent three vector componentsL of v _1,1,2 Norms sum, i.e.)>The discrete total variation model can therefore be redefined as:

low rank complementary information and potentially smoothing characteristics are fully utilized.

The processing procedure of the step 3) is as follows:

(3-1) constructing an image restoration model based on smooth Tak decomposition and high-order tensor Hakking, which is defined as follows

Where lambda represents the balance parameter,representing the decomposition factor U ⁽ⁿ⁾ Is the first of (2)(J, R) items, and +.>

(3-2) the solution of equation (6) depends on the variablesThe least squares method (Alternating Least Square, ALS) optimization may be used, algorithm 1 describes the main process of ALS,

algorithm 1:

input: data to be repairedNuclear tensor dimension (R) ₁ ,…,R _2N )

And (3) outputting: decomposition factorNuclear tensor->

3.2.1 initializing the factorization factor U ^(2N) And a nuclear tensor

3.2.2 when n=1, …,2N

3.2.3

3.2.4U ⁽ⁿ⁾ And algorithm 2.

3.2.5

(3-3) Algorithm 1 describes a conventional ALS-based Tak factorization, whose computation and storage bottleneck is the update factor matrix U ⁽ⁿ⁾ To solve the sub-problem U ⁽ⁿ⁾ We will make it moreThe new technology is as follows

Because of the complexity of equation (7), the above equation is solved using an alternating near-end gradient algorithm (Alternating Proximal Gradient Method, APG) algorithm,

let G (v) = lambda v _1,1,2 ，C＝-L ^* Thus can obtain

Thus, it can be converted into a dual problem:

final sub-problem U ⁽ⁿ⁾ The solving process of (2) is as described in algorithm 2;

algorithm 2:

3.3.1τ,μ＞0；θ∈[0,1]；k＝0

3.3.2 initialization U ⁽⁰⁾ ,v ⁽⁰⁾ ,

3.3.3

3.3.4

3.3.5

3.3.6k＝k+1

Wherein prox is prox _σ And prox _τ Is mapped to by the near end of (a)

(3-4) in addition, the tower-based method can obtain a satisfactory effect by minimizing the rank of the tower, but it is difficult to set an appropriate rank (R ₁ ,…,R _2N ). In our approach, we minimize the following objective function, and this procedure is to obtain an approximate minimum of sufficiently low rank

Wherein ε represents the error threshold; order theR _n′ The optimal rank for the constraint is indicated,

E(1)≥E(2)≥…≥E(R _n′ -1)≥ε≥E(R _n′ ) (13)

finally, a low rank delta optimization is used to refine the image restoration model based on smooth Tak decomposition and high order tensor Hakking, as described in algorithm 3

Algorithm 3:

input: data to be repairedΩ，ε

And (3) outputting: decomposition factorNuclear tensor->

3.4.1 initialization

3.4.2

3.4.3n′←n，R _n′ ←R _n

3.4.4 up to convergence condition:

(3-5) Tacko factor reconstruction output

(3-6) similarly, hanke structured tensorThe inverse hank structure of (c) can be expressed as:

wherein the method comprises the steps of

(3-7) finally, outputting high-quality image visual dataAnd (5) finishing image restoration.

The embodiments described in the present specification are merely examples of implementation forms of the inventive concept, and the scope of protection of the present invention should not be construed as being limited to the specific forms set forth in the examples, and the scope of protection of the present invention and equivalent technical means that can be conceived by those skilled in the art based on the inventive concept.

Claims (1)

1. An image restoration method based on smooth Tack decomposition and high-order tensor Hakking comprises the following steps:

step 1), inputting an image to be repairedDetermining an area to be repaired of an image and performing blocking operation on the area, wherein pixels in the image are divided into known points and unknown points, the known points are points, the pixels in the image are not 0, and the unknown points are points, the pixels in the image are 0; all unknown points in the image form a set omega;

step 2), constructing a high-order tensor hanke and discrete total variation model; the method specifically comprises the following steps:

(2-1) the higher order hank structured image restoration model is defined as follows:

in the method, in the process of the invention,representing an input image to be repaired->Representing post-repair images,/->Representing the Frobenius norm; />Wherein 1 represents an observable pixel and 0 represents a missing pixel; is defined as

Wherein fold _(I,τ) :Through fold _(I,τ) The input N-order tensor can be constructed as a 2N-order tensor, which can be regarded as a multi-dimensional linear copy and multi-dimensional folding operation; wherein->Is a duplication matrix, i.e. a matrix comprising a plurality of identity matrices, in the specific form:

(2-2) discrete total variation model definition is as follows:

x represents a two-dimensional image, v represents a gradient;L _. representing bilinear interpolation operations, particularly in a grid(n ₁ ,n ₂ ) Interpolation on; carrying out ^* Representing an accompanying arithmetic operation; definition of discrete operator D is (Dx) ₁ [n ₁ ,n ₂ ]＝x[n ₁ +1,n ₂ ]-x[n ₁ ,n ₂ ]，(Dx) ₂ [n ₁ ,n ₂ ]＝x[n ₁ ,n ₂ +1]-x[n ₁ ,n ₂ ]；

Let l of v _1,1,2 The norms represent three vector componentsL of v _1,1,2 Norms sum, i.e.)>The discrete total variation model can therefore be redefined as:

fully utilizing low-rank complementary information and potential smoothness characteristics;

step 3), restoring the color image by combining the image restoration model constructed in the step 2), and finally reconstructing and outputting a high-quality visual data imageThe method specifically comprises the following steps:

(3-1) constructing an image restoration model based on smooth Tak decomposition and high-order tensor Hakking, which is defined as follows

Where lambda represents the balance parameter,representing the decomposition factor U ⁽ⁿ⁾ (J, R) th item of (A), and +.>

(3-2) the solution of equation (6) depends on the variablesCan be optimized by using a least square method Alternating Least Square, and is simpleReferring to ALS, algorithm 1 describes the main process of ALS,

algorithm 1:

input: data to be repairedNuclear tensor dimension (R) ₁ ,…,R _2N )

And (3) outputting: decomposition factorNuclear tensor->

3.2.1 initializing the factorization factor U ^(2N) And a nuclear tensor

3.2.2 when n=1, …,2N

3.2.3

3.2.4 U ⁽ⁿ⁾ And algorithm 2.

3.2.5

(3-3) Algorithm 1 describes a conventional ALS-based Tak factorization, whose computation and storage bottleneck is the update factor matrix U ⁽ⁿ⁾ To solve the sub-problem U ⁽ⁿ⁾ We update it as follows

Because of the complexity of equation (7), the alternating near-end gradient algorithm Alternating Proximal Gradient Method, abbreviated APG algorithm, is used to solve the above equation,

let G (v) = lambda v _1,1,2 ，C＝-L ^* Thus can obtain

Thus, it can be converted into a dual problem:

final sub-problem U ⁽ⁿ⁾ The solving process of (2) is as described in algorithm 2;

algorithm 2:

3.3.1τ,μ>0；θ∈[0,1]；k＝0

3.3.2 initialization U ⁽⁰⁾ ,v ⁽⁰⁾ ,

3.3.3

3.3.4

3.3.5

3.3.6k＝k+1

Wherein prox is prox _σ And prox _τ Is mapped to by the near end of (a)

(3-4) additionally, the Take-based method may be obtained by rank minimization of the TakeSatisfactory results, but it is difficult to set an appropriate rank (R ₁ ,…,R _2N ) The method comprises the steps of carrying out a first treatment on the surface of the In our approach, we minimize the following objective function, and this procedure is to obtain an approximate minimum of sufficiently low rank

Wherein ε represents the error threshold; order theR _n′ The optimal rank for the constraint is indicated,

E(1)≥E(2)≥…≥E(R _n′ -1)≥ε≥E(R _n′ ) (13)

finally, a low rank delta optimization is used to refine the image restoration model based on smooth Tak decomposition and high order tensor Hakking, as described in algorithm 3

Algorithm 3:

input: data to be repairedΩ，ε

And (3) outputting: decomposition factorNuclear tensor->

3.4.1 initialization

3.4.2An algorithm 1.

3.4.3 n′←n，R _n′ ←R _n

3.4.4 up to convergence condition:

(3-5) Tacko factor reconstruction output

(3-6) similarly, hanke structured tensorThe inverse hank structure of (c) can be expressed as:

wherein the method comprises the steps of

(3-7) finally, outputting high-quality image visual dataAnd (5) finishing image restoration.