CN103247042A - Image fusion method based on similar blocks - Google Patents

Abstract

The invention discloses an image fusion method based on similar blocks, which relates to digital image processing. The invention provides an image fusion method based on similar blocks, which has a good effect and is easy to operate. The method comprises the following steps of (1) constructing a self-similar structure shared by a plurality of source images: performing similar block matching in the plurality of source images to obtain a shared similar block structure, wherein the shared similar blocks contain self-similarity of images; and (2) weighting: performing image feature extraction on the shared similar blocks, and determining a last selected pixel value according to a weighting method. According to the method, the self-similarity of the source images is utilized, certain robustness is realized for the characteristics of the source images, and the parameter selection range in practical application is large. Through weighting, fusion results are smoother.

Description

A kind of image interfusion method based on similar

Technical field

The present invention relates to Digital Image Processing, especially relate to multi-focused a kind of image interfusion method based on similar.

Background technology

Multi-focused Digital Image Processing is widely used in fields such as satellite remote sensing, military affairs, medical science.Common multi-focus image fusing method probably is divided into two classes: spatial domain method and transform domain method.The advantage of spatial domain method is that the pixel of fused images all comes from source images, makes source image information keep completely, but shortcoming is block pseudo-shadow to occur through regular meeting; The method of transform domain can effectively suppress block pseudo-shadow, but because transform domain operation non-linear makes fusion results compare distortion with source images.

In the spatial domain method, (Evaluation of focus measures in multi-focus image fusion such as W.Huang and Z.Jing are arranged, Pattern Recognition Letters, vol.28, pp:493-500,2007) common method and evaluation index have been made relatively conclusion.At the block pseudo-shadow that fusion results occurs, these methods tend to take braking measure, for example image are carried out unification filtering and handle region growing etc.

In the transform domain method, (A categorization of multiscale-decomposition-based image fusion schemes with a performance study for a digital camera application such as Zhang Zhong, Proceedings of the IEEE, vol.87, pp.1315-1326,1999) common transform domain method has been made conclusion, yet these methods often exist the susceptibility height, the bad weakness of choosing of parameter.

In addition, domestic (Image fusion algorithm based on spatial frequency-motivated pulse coupled neural networks in nonsubsampled contourlet transform domain such as Qu Xiaobo, Acta Automatica Sinica, vol.34, pp.1508-1514,2008) also success transform domain image is merged combines with neural network, obtained good effect.

2006, people such as K.Dabov (Image denoising with block-matching and3D filtering, Proceedings of the IEEE, vol.6064, pp.354-365,2006) method that has proposed similar coupling solves the image denoising problem.This method has been utilized the self-similarity of image, has obtained success aspect denoising.

Summary of the invention

The object of the present invention is to provide effect good, a kind of image interfusion method based on similar of easy operating.

The present invention includes following steps:

1) make up the self-similar structure that a plurality of source images are shared: carry out similar coupling in several source images, the similar block structure that obtains sharing has namely comprised the self-similarity of image in described shared similar;

2) weighting: after carrying out image characteristics extraction in share similar, determine the pixel value of choosing at last according to the method for weighting.

In step 2) in, the method for described weighting can be: the image definition index of establishing extraction is C, uses C ₁And C ₂Represent the sharpness of source images 1 and source images 2, be expressed as a counter big with source images etc. with M, for j pixel, use With

The counter of expression source images 1 and source images 2, given similar shared D,

Work as C ₁〉=C ₂The time, have

$M_1^j=M_1^j+1,j\∈D;$

Work as C ₁＜C ₂The time, have

$M_2^j=M_2^j+1,j\∈D;$

After the feature of entire image all is extracted, use With

Represent source images 1 and source images 2 values j pixel respectively, then j the pixel F of fused images F ^jObtain in the following manner:

$F^j=\frac{M_1^j{S}_1^j}{M_1^j+M_2^j}+\frac{M_2^j{S}_2^j}{M_1^j+M_2^j}.$

Concrete grammar is as follows:

1) makes up the self-similar structure that a plurality of source images are shared: entire image is regarded as the set in some zones, described some zones can be overlapped, each zone is regarded as the set of some image blocks, can overlap each other between described some image blocks, in each zone, choose a central block earlier, by between other image blocks and central block, carrying out the similarity coupling:

η＝||P _q-P _r||

Wherein, η represents similarity degree, P _qAnd P _rSubmeter is represented the matrix that the pixel of the image block centered by pixel q and pixel p is formed, || A|| is matrix norm, and it is defined as follows:

$\left|\right|A\left|\right|=\sqrt{\underset{i=1}{\overset{I}{\mathrm{\Σ}}}\underset{j=1}{\overset{J}{\mathrm{\Σ}}}{\left|a_{\mathrm{ij}}\right|}^2}$

Wherein, a _IjThe value of each pixel in the presentation video piece A;

By the similarity degree numerical value that obtains, seek the less image block of difference each other and form analog structure in the width of cloth source images, use

Expression centered by pixel i image block and the similarity degree between the central block, it is as follows to establish the similarity numerical relation that searches out:

${\mathrm{\&eta;}}_{q_1}\&le;{\mathrm{\&eta;}}_{{\mathrm{<figure-callout\; id="2"\; label="q"\; filenames="HDA00003241958800012.png,HDA00003241958800022.png"\; state="\{\{state\}\}">q</figure-callout>}}_2}\&le;\&CenterDot;\&CenterDot;\&CenterDot;\&le;{\mathrm{\&eta;}}_{q_k}\&le;{\mathrm{\&eta;}}_{q_{k+1}}\&le;\&CenterDot;\&CenterDot;\&CenterDot;\&le;{\mathrm{\&eta;}}_{q_{{(n-m+1)}^2}}$

Wherein, (n-m+1) ²Be illustrated in size in the zone of n, total total (n-m+1) ²Individual size is the image block of m.The analog structure that in different source images, obtains sharing then; Make L ⁱRepresent the analog structure in the i width of cloth source images, with Represent j similar to central block in i sub-picture image block, suppose

$L^1=\{q_1^1,{\mathrm{<figure-callout\; id="2"\; label="q"\; filenames="HDA00003241958800012.png,HDA00003241958800022.png"\; state="\{\{state\}\}">q</figure-callout>}}_2^1,\&CenterDot;\&CenterDot;\&CenterDot;,q_k^1\},$ ${\mathrm{<figure-callout\; id="2"\; label="L"\; filenames="HDA00003241958800012.png,HDA00003241958800022.png"\; state="\{\{state\}\}">L</figure-callout>}}^2=\{q_1^2,{\mathrm{<figure-callout\; id="2"\; label="q"\; filenames="HDA00003241958800012.png,HDA00003241958800022.png"\; state="\{\{state\}\}">q</figure-callout>}}_2^2,\&CenterDot;\&CenterDot;\&CenterDot;,q_k^2\}$

Then have

${\mathrm{<figure-callout\; id="12"\; label="L\; g"\; filenames="HDA00003241958800022.png"\; state="\{\{state\}\}">L</figure-callout>}}_g^{}=L^1\&cap;{\mathrm{<figure-callout\; id="2"\; label="L"\; filenames="HDA00003241958800012.png,HDA00003241958800022.png"\; state="\{\{state\}\}">L</figure-callout>}}^2$

Wherein, L ¹Analog structure in the expression source images 1, L ²Analog structure in the expression source images 2,

The analog structure that expression is shared, shared analog structure can embody the self-similarity of source images;

2) weighting: under spatial domain, often there is block pseudo-shadow in the result of fusion, in a yardstick of transform domain, also has similar problem; Generally, the method for weighting can make fusion results more level and smooth, and the weighting rule that the present invention adopts is as follows: after the analog structure of sharing was carried out image characteristics extraction, the characteristics of image (being generally articulation index herein) of establishing extraction was C, uses C ₁And C ₂Represent the sharpness of source images 1 and source images 2, for j pixel, use

With

The counter of expression source images 1 and source images 2; Given similar shared D then works as C ₁〉=C ₂The time, have

$M_1^j=M_1^j+1,j\&Element;D;$

Work as C ₁＜C ₂The time, have

$M_2^j=M_2^j+1,j\&Element;D;$

After the feature of entire image all is extracted, use

With

Represent source images 1 and source images 2 values j pixel respectively, then j the pixel F of fused images F ^jObtain in the following manner:

$F^j=\frac{M_1^j}{M_1^j+M_2^j}\&CenterDot;S_1^j+\frac{M_2^j}{M_1^j+M_2^j}\&CenterDot;S_2^j$

Wherein F represents fused images, and S represents source images.

By following formula as can be known, the pixel value of fused images belongs in the closed interval that two width of cloth source image pixels values are the starting point formation.

Application under transform domain is as follows:

After several source images are carried out certain conversion, obtain the coefficient of source images on different scale under this transform method, in each yardstick, coefficient carried out above-mentioned steps 1) and 2) operation, be the application of the present invention on transform domain.

The invention has the beneficial effects as follows: because the present invention has utilized the self-similarity of source images self, make the present invention have certain robustness for the characteristics of source images, in the practical application parameter to choose scope bigger.By above-mentioned weighting rule, make fusion results more level and smooth.

Description of drawings

Fig. 1 is source images 1.

Fig. 2 is source images 2.

Fig. 3 is similar coupling and the exemplary plot that constitutes the analog structure of sharing.

Fig. 4 is the weight by the source images that obtains 1 of weighting rule.

Fig. 5 is the fusion results that obtains under spatial domain.

Embodiment

The embodiment of the invention uses 256 grades of gray-scale maps to carry out mixing operation.The gray level image that the embodiment of the invention is used is two 512 * 512 sizes, the picture of tif form, and two width of cloth source images are as illustrated in fig. 1 and 2.

Because the present invention has stronger robustness to parameter, in implementation process, choose the canonical parameter operation.Parameter is chosen as table 1:

Table 1: parameter value among the embodiment

Specific implementation process is as follows:

The first step: make up the self-similar structure that a plurality of source images are shared

At first source images is carried out border extension, what adopt in the present embodiment is to be the symmetrical extended method of axis of symmetry with the image boundary, and the amplitude of expansion is 20 pixels.

After the border extension, the source images image becomes 552 * 552 sizes.Source images is divided into the zone that some sizes are m * m (n=16 in the present embodiment), and each zone is divided into the image block that some sizes are m * m (m=8 in the present embodiment), any two zones or the distance of any two image blocks is 1 are so comprise (552-16+1) altogether in each width of cloth source images ²Individual zone, and comprise (16-8+1) in each zone ²Individual image block.

As Fig. 3, the f of outermost layer frame ¹Represent source images 1, f ¹In R ¹The some zones in the source images 1 have been represented, at this regional R ¹In, at first chosen a central block 31, central block 31 is marked as strip shade (as shown in Figure 3).f ² Represent source images 2, R ²Represent in the source images 2 and R ¹The zone of same position, R ²In a central block is also arranged.

In a zone, make and carry out the similarity coupling between other image blocks and the central block:

η＝||P _q-P _r||

Wherein, η represents similarity degree, P _qAnd P _rSubmeter is represented the matrix that the pixel of the image block centered by pixel q and pixel r is formed, || A|| is matrix norm, and it is defined as follows:

$\left|\right|A\left|\right|=\sqrt{\underset{i=1}{\overset{I}{\mathrm{\&Sigma;}}}\underset{j=1}{\overset{J}{\mathrm{\&Sigma;}}}{\left|a_{\mathrm{ij}}\right|}^2}$

Wherein, a _IjThe value of each element in the representing matrix A;

By the similarity degree numerical value that obtains, seek the less image block of difference each other and form analog structure in the width of cloth source images, use η _QiExpression centered by pixel i image block and the similarity degree between the central block, it is as follows to establish the similarity numerical relation that searches out:

${\mathrm{\&eta;}}_{q_1}\&le;{\mathrm{\&eta;}}_{{\mathrm{<figure-callout\; id="2"\; label="q"\; filenames="HDA00003241958800012.png,HDA00003241958800022.png"\; state="\{\{state\}\}">q</figure-callout>}}_2}\&le;\&CenterDot;\&CenterDot;\&CenterDot;\&le;{\mathrm{\&eta;}}_{q_k}\&le;{\mathrm{\&eta;}}_{q_{k+1}}\&le;\&CenterDot;\&CenterDot;\&CenterDot;\&le;{\mathrm{\&eta;}}_{q_{{(n-m+1)}^2}}$

Wherein, (n-m+1) ²Be illustrated in size in the zone of n, total total (n-m+1) ²Individual size is the image block of m.Search out after this magnitude relationship, choose k the most similar to central block in all images piece (k=16 in the present embodiment, but directly perceived for simplicity, what represent among Fig. 3 is the situation of k=5) image block, as shown in Figure 3, suppose

The analog structure 33 that in different source images, obtains sharing then; Make L ⁱRepresent the analog structure in the i width of cloth source images, with

Represent j similar to central block in i sub-picture image block, suppose that 5 space rectangles around the central block are 5 image blocks the most similar that select, claim that then they are similar 32.In two width of cloth images, get their intersection respectively, again the intersection that obtains in two width of cloth images got friendship:

$L^1=\{q_1^1,{\mathrm{<figure-callout\; id="2"\; label="q"\; filenames="HDA00003241958800012.png,HDA00003241958800022.png"\; state="\{\{state\}\}">q</figure-callout>}}_2^1,\&CenterDot;\&CenterDot;\&CenterDot;{,q}_k^1\},$ ${\mathrm{<figure-callout\; id="2"\; label="L"\; filenames="HDA00003241958800012.png,HDA00003241958800022.png"\; state="\{\{state\}\}">L</figure-callout>}}^2=\{q_1^2,{\mathrm{<figure-callout\; id="2"\; label="q"\; filenames="HDA00003241958800012.png,HDA00003241958800022.png"\; state="\{\{state\}\}">q</figure-callout>}}_2^2,\&CenterDot;\&CenterDot;\&CenterDot;{,q}_k^2\}$

${\mathrm{<figure-callout\; id="12"\; label="L\; g"\; filenames="HDA00003241958800022.png"\; state="\{\{state\}\}">L</figure-callout>}}_g^{}=L^1\&cap;{\mathrm{<figure-callout\; id="2"\; label="L"\; filenames="HDA00003241958800012.png,HDA00003241958800022.png"\; state="\{\{state\}\}">L</figure-callout>}}^2$

Wherein, L ¹Similar intersection in the expression source images 1 claims that it is analog structure in the source images 1, L ²Analog structure in the expression source images 2,

The analog structure that expression is shared, as shown in Figure 3, the netted shadow representation of sharing of analog structure 33.The analog structure of sharing 33 can embody the self-similarity of source images;

In the share similar structure of two width of cloth figure, extract the index of expression sharpness.What present embodiment adopted is the index of improved Laplce's energy and (Sum-Modified-Laplacian is hereinafter to be referred as SML), and SML is defined as follows:

${\mathrm{SML}}_{(x_0,y_0)}=\underset{x=x_0-N}{\overset{y=x_0+N}{\mathrm{\&Sigma;}}}\underset{x=y_0-N}{\overset{y=y_0+N}{\mathrm{\&Sigma;}}}{\&dtri;}_{\mathrm{ML}}^{2}f(x_0,y_0)$

Wherein

Represent improved Laplace operator, be defined as:

${\&dtri;}_{\mathrm{ML}}^{2}f(x_0,y_0)=|2f(x_0,y_0)-f(x_0-\mathrm{step},y_0)-f(x_0+\mathrm{step},y_0)|$

$+|2f(x_0,y_0)-f(x_0,y_0-\mathrm{step})-f(x_0,y_0+\mathrm{step})|$

F (x wherein ₀, y ₀) the expression position is (x ₀, y ₀) pixel value, step represents the step-length that each computing is moved, and is taken as 1 herein.

Second step: weighting

After with SML the analog structure of sharing being calculated, the characteristics of image (being the SML value herein) of establishing extraction is C, uses C ₁And C ₂Represent the sharpness of source images 1 and source images 2, for j pixel, use M ₁With

The counter of expression source images 1 and source images 2.The given similar D that shares then works as C ₁〉=C ₂The time, have

$M_1^j=M_1^j+1,j\&Element;D.$

Work as C ₁＜C ₂The time, have

$M_2^j=M_2^j+1,j\&Element;D.$

After the feature of entire image all is extracted,

With

Be exactly the weights of two width of cloth source images on pixel j.First width of cloth source images weight matrix M of having a few wherein ₁Expression is shown as image namely as shown in Figure 4.With

With

Represent source images 1 and source images 2 values j pixel respectively, then j the pixel of fused images F obtains in the following manner:

$F^j=\frac{M_1^j}{M_1^j+M_2^j}\&CenterDot;S_1^j+\frac{M_2^j}{M_1^j+M_2^j}\&CenterDot;S_2^j$

Wherein F represents fused images, and S represents source images.Fusion results as shown in Figure 5.

Claims (4)

1. one kind based on similar image interfusion method, it is characterized in that may further comprise the steps:

1) make up the self-similar structure that a plurality of source images are shared: carry out similar coupling in several source images, the similar block structure that obtains sharing has namely comprised the self-similarity of image in described shared similar;

2) weighting: after carrying out image characteristics extraction in share similar, determine the pixel value of choosing at last according to the method for weighting.

2. a kind of image interfusion method based on similar according to claim 1 is characterized in that in step 2) in, the method for described weighting is: the image definition index of establishing extraction is C, uses C ₁And C ₂Represent the sharpness of source images 1 and source images 2, be expressed as a counter big with source images etc. with M, for j pixel, use

With The counter of expression source images 1 and source images 2, given similar shared D works as C ₁〉=C ₂The time, have:

$M_1^j=M_1^j+1,j\&Element;D;$

Work as C ₁＜C ₂The time, have:

$M_2^j=M_2^j+1,j\&Element;D;$

After the feature of entire image all is extracted, use

With

Represent source images 1 and source images 2 values j pixel respectively, then j the pixel F of fused images F ^jObtain in the following manner:

$F^j=\frac{M_1^j{S}_1^j}{M_1^j+M_2^j}+\frac{M_2^j{S}_2^j}{M_1^j+M_2^j}.$

3. a kind of image interfusion method based on similar according to claim 1, it is characterized in that in step 1), the concrete grammar of the self-similar structure that a plurality of source images of described structure are shared is as follows: entire image is regarded as the set in some zones, described some zones can be overlapped, each zone is regarded as the set of some image blocks, can overlap each other between described some image blocks, in each zone, choose a central block earlier, by between other image blocks and central block, carrying out the similarity coupling:

η＝||P _q-P _r||

Wherein, η represents similarity degree, P _qAnd P _rSubmeter is represented the matrix that the pixel of the image block centered by pixel q and pixel p is formed, || A|| is matrix norm, and it is defined as follows:

$\left|\right|A\left|\right|=\sqrt{\underset{i=1}{\overset{I}{\mathrm{\&Sigma;}}}\underset{j=1}{\overset{J}{\mathrm{\&Sigma;}}}{\left|a_{\mathrm{ij}}\right|}^2}$

Wherein, a _IjThe value of each pixel in the presentation video piece A;

By the similarity degree numerical value that obtains, seek the less image block of difference each other and form analog structure in the width of cloth source images, use

Expression centered by pixel i image block and the similarity degree between the central block, it is as follows to establish the similarity numerical relation that searches out:

${\mathrm{\&eta;}}_{q_1}\&le;{\mathrm{\&eta;}}_{q_2}\&le;\&CenterDot;\&CenterDot;\&CenterDot;\&le;{\mathrm{\&eta;}}_{q_k}\&le;{\mathrm{\&eta;}}_{q_{k+1}}\&le;\&CenterDot;\&CenterDot;\&CenterDot;\&le;{\mathrm{\&eta;}}_{q_{{(n-m+1)}^2}}$

Wherein, (n-m+1) ²Be illustrated in size in the zone of n, total total (n-m+1) ²Individual size is the image block of m.The analog structure that in different source images, obtains sharing then; Make L ⁱRepresent the analog structure in the i width of cloth source images, with

Represent j similar to central block in i sub-picture image block, suppose

$L^1=\{q_1^1,q_2^1,\&CenterDot;\&CenterDot;\&CenterDot;,q_k^1\},$ $L^2=\{q_1^2,q_2^2,\&CenterDot;\&CenterDot;\&CenterDot;,q_k^2\}$

Then have

$L_g^{12}=L^1\&cap;L^2$

Wherein, L ¹Analog structure in the expression source images 1, L ²Analog structure in the expression source images 2,

The analog structure that expression is shared, shared analog structure can embody the self-similarity of source images.

4. a kind of image interfusion method based on similar according to claim 1 is characterized in that in step 2) in, the concrete grammar of described weighting is: after the analog structure of sharing was carried out image characteristics extraction, the characteristics of image of establishing extraction was C, uses C ₁And C ₂Represent the sharpness of source images 1 and source images 2, for j pixel, use

With

The counter of expression source images 1 and source images 2; Given similar shared D then works as C ₁〉=C ₂The time, have:

$M_1^j=M_1^j+1,j\&Element;D;$

Work as C ₁＜C ₂The time, have:

$M_2^j=M_2^j+1,j\&Element;D;$

After the feature of entire image all is extracted, use

With

Represent source images 1 and source images 2 values j pixel respectively, then j the pixel F of fused images F ^jObtain in the following manner:

$F^j=\frac{M_1^j}{M_1^j+M_2^j}\&CenterDot;S_1^j+\frac{M_2^j}{M_1^j+M_2^j}\&CenterDot;S_2^j$

Wherein F represents fused images, and S represents source images.

Images