CN103247042B - A kind of image interfusion method based on similar piece - Google Patents

Abstract

Based on the image interfusion method of similar piece, relate to Digital Image Processing.Excellent effect is provided, is easy to a kind of image interfusion method based on similar piece operated.1) build the self-similar structure that multiple source images is shared: in several source images, carry out similar Block-matching, obtain the similar block structure shared, in described shared similar piece, namely contain the self-similarity of image; 2) weighting: to share similar piece in carry out image characteristics extraction after, determine the pixel value finally chosen according to the method for weighting.Owing to make use of the self-similarity of source images self, the feature for source images has certain robustness, and in practical application, the selection range of parameter is larger.By weighting, make fusion results more level and smooth.

Description

A kind of image interfusion method based on similar piece

Technical field

The present invention relates to Digital Image Processing, especially relate to multi-focused a kind of image interfusion method based on similar piece.

Background technology

Multi-focused Digital Image Processing is widely used in the fields such as satellite remote sensing, military affairs, medical science.Common multi-focus image fusing method is probably divided into two classes: Space domain and transform domain method.The advantage of Space domain is that the pixel of fused images all comes from source images, source image information is retained complete, but shortcoming often there will be block artifact; The method of transform domain can effectively suppress block artifact, but due to transform domain operation non-linear, make fusion results have distortion compared with source images.

In Space domain, there are (the Evaluationoffocusmeasuresinmulti-focusimagefusion such as W.Huang and Z.Jing, PatternRecognitionLetters, vol.28, pp:493-500,2007) common method and evaluation index have been made compared conclusion.For the block artifact that fusion results occurs, these methods often take braking measure, such as, carry out unification filtering process to image, region growth etc.

In transform domain method, (the Acategorizationofmultiscale-decomposition-basedimagefusi onschemeswithaperformancestudyforadigitalcameraapplicati on such as ZhangZhong, ProceedingsoftheIEEE, vol.87, pp.1315-1326,1999) conclusion has been made to common transform domain method, however these methods often to there is susceptibility high, the bad weakness chosen of parameter.

In addition, (the Imagefusionalgorithmbasedonspatialfrequency-motivatedpul secoupledneuralnetworksinnonsubsampledcontourlettransfor mdomain such as domestic QuXiaobo, ActaAutomaticaSinica, vol.34, pp.1508-1514,2008) also successful fusion by transform domain image combines with neural network, achieves good effect.

2006, the people such as K.Dabov (Imagedenoisingwithblock-matchingand3Dfiltering, ProceedingsoftheIEEE, vol.6064, pp.354-365,2006) method proposing similar Block-matching solves image denoising problem.This process employs the self-similarity of image, achieve successfully in denoising.

Summary of the invention

The object of the present invention is to provide excellent effect, be easy to a kind of image interfusion method based on similar piece operated.

The present invention includes following steps:

1) build the self-similar structure that multiple source images is shared: in several source images, carry out similar Block-matching, obtain the similar block structure shared, in described shared similar piece, namely contain the self-similarity of image;

2) weighting: to share similar piece in carry out image characteristics extraction after, determine the pixel value finally chosen according to the method for weighting.

In step 2) in, the method for described weighting can be: set the image definition index of extraction as C, use C ₁and C ₂represent the sharpness of source images 1 and source images 2, be expressed as a counter large with source images etc. with M, for a jth pixel, use with represent the counter of source images 1 and source images 2, given similar piece of shared D,

Work as C ₁>=C ₂time, have

$M_1^j=M_1^j+1,j\∈D;$

Work as C ₁< C ₂time, have

$M_2^j=M_2^j+1,j\&Element;D;$

After the feature of entire image is all extracted, use with represent source images 1 and source images 2 value in a jth pixel respectively, then a jth pixel F of fused images F ^jobtain in the following manner:

$F^j=\frac{M_1^j{S}_1^j}{M_1^j+M_2^j}+\frac{M_2^j{S}_2^j}{M_1^j+M_2^j}.$

Concrete grammar is as follows:

1) self-similar structure that multiple source images is shared is built: set entire image being regarded as some regions, described some regions can be overlapped, each region is regarded as the set of some image blocks, can overlap each other between described some image blocks, in each area, first choose a central block, by carrying out similarity mode between other image block and central blocks:

η＝||P _q-P _r||

Wherein, η represents similarity degree, P _qand P _rrepresent the matrix of the pixel composition of the image block centered by pixel q and pixel r respectively, || A|| is matrix norm, and it is defined as follows:

$\left|\right|A\left|\right|=\sqrt{\underset{i=1}{\overset{I}{\&Sigma;}}\underset{j=1}{\overset{J}{\&Sigma;}}|a_{ij}{|}^2}$

Wherein, a _ijrepresent the value of each pixel in image block A;

By the similarity degree numerical value obtained, find the less image block of difference each other and form analog structure in a width source images, use represent the similarity degree between image block centered by pixel i and central block, if the similarity numerical relation searched out is as follows:

${\mathrm{\&eta;}}_{q_1}\&le;{\mathrm{\&eta;}}_{q_2}\&le;...\&le;{\mathrm{\&eta;}}_{q_k}\&le;{\mathrm{\&eta;}}_{q_{k+1}}\&le;...\&le;{\mathrm{\&eta;}}_{q_{{(n-m+1)}^2}}$

Wherein, (n-m+1) ²represent in the region that size is n, total total (n-m+1) ²individual size is the image block of m.Then in different source images, obtain the analog structure shared; Make L ⁱrepresent the analog structure in the i-th width source images, with represent a jth image block similar to central block in the i-th sub-picture, suppose

$\begin{array}{cc}{L}^1=\{q_1^1,q_2^1,...,q_k^1\},& L^2=\{q_1^2,q_2^2,...,q_k^2\}\end{array}$

Then have

$L_g^{12}=L^1\&cap;L^2$

Wherein, L ¹represent the analog structure in source images 1, L ²represent the analog structure in source images 2, represent the analog structure shared, shared analog structure can embody the self-similarity of source images;

2) weighting: under spatial domain, often there is block artifact in the result of fusion, in a yardstick of transform domain, also there is similar problem; Under normal circumstances, the method for weighting can make fusion results more level and smooth, and the Weighted Rule that the present invention adopts is as follows: after carrying out image characteristics extraction to the analog structure shared, if the characteristics of image (being generally articulation index herein) extracted is C, use C ₁and C ₂represent the sharpness of source images 1 and source images 2, for a jth pixel, use with represent the counter of source images 1 and source images 2; Given similar piece of shared D, then work as C ₁>=C ₂time, have

$M_1^j=M_1^j+1,j\&Element;D;$

Work as C ₁< C ₂time, have

$M_2^j=M_2^j+1,j\&Element;D;$

After the feature of entire image is all extracted, use with represent source images 1 and source images 2 value in a jth pixel respectively, then a jth pixel F of fused images F ^jobtain in the following manner:

$F^j=\frac{M_1^j}{M_1^j+M_2^j}\&CenterDot;S_1^j+\frac{M_2^j}{M_1^j+M_2^j}\&CenterDot;S_2^j$

Wherein F represents fused images, and S represents source images.

From above formula, the pixel value of fused images belongs in the closed interval that two width source image pixels values are starting point formation.

Application under transform domain is as follows:

To several source images carry out certain conversion after, obtain the coefficient of source images under this transform method on different scale, in each yardstick, above-mentioned steps 1 carried out to coefficient) and 2) operation, be the application of the present invention on transform domain.

The invention has the beneficial effects as follows: owing to present invention utilizes the self-similarity of source images self, make the present invention have certain robustness for the feature of source images, in practical application, the selection range of parameter is larger.By above-mentioned Weighted Rule, make fusion results more level and smooth.

Accompanying drawing explanation

Fig. 1 is source images 1.

Fig. 2 is source images 2.

Fig. 3 is similar Block-matching and forms the exemplary plot of the analog structure shared.

Fig. 4 is the weight of the source images 1 obtained by Weighted Rule.

Fig. 5 is the fusion results obtained under spatial domain.

Embodiment

The embodiment of the present invention uses 256 grades of gray-scale maps to carry out mixing operation.The gray level image that the embodiment of the present invention uses is two 512 × 512 sizes, and the picture of tif form, two width source images as illustrated in fig. 1 and 2.

Because the present invention has stronger robustness to parameter, in implementation process, choose canonical parameter operation.Parameter choose is as table 1:

Table 1: parameter value in embodiment

Specific implementation process is as follows:

The first step: build the self-similar structure that multiple source images is shared

First source images is carried out border extension, what adopt in the present embodiment is take image boundary as the symmetric extension method of axis of symmetry, and the amplitude of expansion is 20 pixels.

After border extension, source images image becomes 552 × 552 sizes.Some sizes are divided into by source images to be the region (in the present embodiment n=16) of n × n, and be divided into some sizes to be the image block (in the present embodiment m=8) of m × m in each region, any two regions or the distance of any two image blocks is 1, so altogether comprise (552-16+1) in each width source images ²individual region, and comprise (16-8+1) in each region ²individual image block.

As Fig. 3, the f of outermost layer frame ¹represent source images 1, f ¹in R ¹represent the some regions in source images 1, at this region R ¹in, first have chosen a central block 31, central block 31 is marked as strip shade (as shown in Figure 3).F ²represent source images 2, R ²represent in source images 2 with R ¹the region of same position, R ²in also have a central block.

In a region, make between other image block and central blocks and carry out similarity mode:

η＝||P _q-P _r||

Wherein, η represents similarity degree, P _qand P _rrepresent the matrix of the pixel composition of the image block centered by pixel q and pixel r respectively, || A|| is matrix norm, and it is defined as follows:

$\left|\right|A\left|\right|=\sqrt{\underset{i=1}{\overset{I}{\&Sigma;}}\underset{j=1}{\overset{J}{\&Sigma;}}|a_{ij}{|}^2}$

Wherein, a _ijthe value of each element in representing matrix A;

By the similarity degree numerical value obtained, find the less image block of difference each other and form analog structure in a width source images, use η _qirepresent the similarity degree between image block centered by pixel i and central block, if the similarity numerical relation searched out is as follows:

${\mathrm{\&eta;}}_{q_1}\&le;{\mathrm{\&eta;}}_{q_2}\&le;...\&le;{\mathrm{\&eta;}}_{q_k}\&le;{\mathrm{\&eta;}}_{q_{k+1}}\&le;...\&le;{\mathrm{\&eta;}}_{q_{{(n-m+1)}^2}}$

Wherein, (n-m+1) ²represent in the region that size is n, total total (n-m+1) ²individual size is the image block of m.After searching out this magnitude relationship, choose k the most similar to central block in all image blocks (k=16 in the present embodiment, but directly perceived for simplicity, what represent in Fig. 3 is the situation of k=5) image block, as shown in Figure 3, suppose

Then in different source images, obtain the analog structure 33 shared; Make L ⁱrepresent the analog structure in the i-th width source images, with represent a jth image block similar to central block in the i-th sub-picture, suppose that 5 space rectangles around central block are 5 the image blocks the most similar selected, then claim them to be similar piece 32.Get their intersection respectively in two images, then the intersection obtained in two width images got friendship:

$\begin{array}{cc}{L}^1=\{q_1^1,q_2^1,...,q_k^1\},& L^2=\{q_1^2,q_2^2,...,q_k^2\}\end{array}$

$L_g^{12}=L^1\&cap;L^2$

Wherein, L ¹represent the intersection of in source images 1 similar piece, claim it to be analog structure in source images 1, L ²represent the analog structure in source images 2, represent the analog structure shared, as shown in Figure 3, the netted shadow representation of analog structure 33 shared.The analog structure 33 shared can embody the self-similarity of source images;

The index representing sharpness is extracted in the share similar structure of two width figure.What the present embodiment adopted is Laplce's energy of improvement and the index of (Sum-Modified-Laplacian, hereinafter referred to as SML), and SML is defined as follows:

${\mathrm{SML}}_{(x_0,y_0)}=\underset{x=x_0-N}{\overset{y=x_0+N}{\mathrm{\&Sigma;}}}\underset{x=y_0-N}{\overset{y=y_0+N}{\mathrm{\&Sigma;}}}{\&dtri;}_{ML}^{2}f(x_0,y_0)$

Wherein represent the Laplace operator improved, be defined as:

$\begin{array}{c}{\&dtri;}_{ML}^{2}f(x_0,y_0)=|2f(x_0,y_0)-f(x_0-step,y_0)-f(x_0+step,y_0)|\\ +|2f(x_0,y_0)-f(x_0,y_0-step)-f(x_0,y_0+step)|\end{array}$

Wherein f (x ₀, y ₀) expression position is (x ₀, y ₀) pixel value, step represents the step-length of each computing movement, is taken as 1 herein.

Second step: weighting

After the analog structure shared being calculated with SML, if the characteristics of image (being SML value) extracted is C, use C herein ₁and C ₂represent the sharpness of source images 1 and source images 2, for a jth pixel, use with represent the counter of source images 1 and source images 2.The given similar piece of D shared, then work as C ₁>=C ₂time, have

$M_1^j=M_1^j+1,j\&Element;D.$

Work as C ₁< C ₂time, have

$M_2^j=M_2^j+1,j\&Element;D.$

After the feature of entire image is all extracted, with be exactly the weights of two width source images on pixel j.Wherein the first width source images weight matrix M a little ₁represent, be shown as image namely as shown in Figure 4.With with represent source images 1 and source images 2 value in a jth pixel respectively, then a jth pixel of fused images F obtains in the following manner:

$F^j=\frac{M_1^j}{M_1^j+M_2^j}\&CenterDot;S_1^j+\frac{M_2^j}{M_1^j+M_2^j}\&CenterDot;S_2^j$

Wherein F represents fused images, and S represents source images.Fusion results as shown in Figure 5.

Claims (2)

1., based on the image interfusion method of similar piece, it is characterized in that comprising the following steps:

1) build the self-similar structure that multiple source images is shared: in several source images, carry out similar Block-matching, obtain the similar block structure shared, in described shared similar piece, namely contain the self-similarity of image; The concrete grammar of the self-similar structure that the multiple source images of described structure is shared is as follows: set entire image being regarded as some regions, described some regions can be overlapped, each region is regarded as the set of some image blocks, can overlap each other between described some image blocks, in each area, first choose a central block, by carrying out similarity mode between other image block and central blocks:

η＝||P _q-P _r||

Wherein, η represents similarity degree, P _qand P _rrepresent the matrix of the pixel composition of the image block centered by pixel q and pixel r respectively, || A|| is matrix norm, and it is defined as follows:

$\left|\right|A\left|\right|=\sqrt{\underset{i=1}{\overset{I}{\&Sigma;}}\underset{j=1}{\overset{J}{\&Sigma;}}|a_{ij}{|}^2}$

Wherein, a _ijrepresent the value of each pixel in image block A;

By the similarity degree numerical value obtained, find the less image block of difference each other and form analog structure in a width source images, use represent the similarity degree between image block centered by pixel i and central block, if the similarity numerical relation searched out is as follows:

${\mathrm{\&eta;}}_{q_1}\&le;{\mathrm{\&eta;}}_{q_2}\&le;...\&le;{\mathrm{\&eta;}}_{q_k}\&le;{\mathrm{\&eta;}}_{q_{k+1}}\&le;...\&le;{\mathrm{\&eta;}}_{q_{{(n-m+1)}^2}}$

Wherein, (n-m+1) ²represent in the region that size is n, total total (n-m+1) ²individual size is the image block of m, in different source images, then obtain the analog structure shared; Make L ⁱrepresent the analog structure in the i-th width source images, with represent a jth image block similar to central block in the i-th sub-picture, suppose

$\begin{array}{cc}{L}^1=\{q_1^1,q_2^1,...,q_k^1\},& L^2=\{q_1^2,q_2^2,...,q_k^2\}\end{array}$

Then have

$L_g^{12}=L^1\&cap;L^2$

Wherein, L ¹represent the analog structure in source images 1, L ²represent the analog structure in source images 2, represent the analog structure shared, shared analog structure can embody the self-similarity of source images;

2) weighting: to share similar piece in carry out image characteristics extraction after, determine the pixel value finally chosen according to the method for weighting.

2. a kind of image interfusion method based on similar piece as claimed in claim 1, is characterized in that in step 2) in, the method for described weighting is: set the image definition index of extraction as C, use C ₁and C ₂represent the sharpness of source images 1 and source images 2, be expressed as a counter large with source images etc. with M, for a jth pixel, use with represent the counter of source images 1 and source images 2, given similar piece of shared D, works as C ₁>=C ₂time, have:

$M_1^j=M_1^j+1,j\&Element;D;$

Work as C ₁< C ₂time, have:

$M_2^j=M_2^j+1,j\&Element;D;$

After the feature of entire image is all extracted, use with represent source images 1 and source images 2 value in a jth pixel respectively, then a jth pixel F of fused images F ^jobtain in the following manner:

$F^j=\frac{M_1^j{S}_1^j}{M_1^j+M_2^j}+\frac{M_2^j{S}_2^j}{M_1^j+M_2^j}.$