CN113947554A

CN113947554A - Multi-focus image fusion method based on NSST and significant information extraction

Info

Publication number: CN113947554A
Application number: CN202010693743.7A
Authority: CN
Inventors: 何小海; 吴剑; 吴晓红; 王正勇; 卿粼波; 吴小强
Original assignee: Sichuan University
Current assignee: Sichuan University
Priority date: 2020-07-17
Filing date: 2020-07-17
Publication date: 2022-01-18
Anticipated expiration: 2040-07-17
Also published as: CN113947554B

Abstract

The invention provides a multi-focus image fusion method based on NSST and significant information extraction. Mainly relates to the problem of multi-focus image fusion in the field of image fusion. Firstly, carrying out multi-scale and multi-direction decomposition on a source image through NSST (non-subsampled contourlet transform) to obtain high-frequency and low-frequency sub-bands. Secondly, adopting improved Laplace energy of a local area and constructing low-frequency subband initial fusion weight for the low-frequency subband coefficient, and adding a non-local mean value filtering correction fusion rule for correcting the low-frequency initial fusion weight; adopting a fusion rule of combining spatial frequency and energy based on correlation coefficients for the high-frequency sub-band coefficients, and correcting by a phase consistency fusion rule to construct a high-frequency sub-band fusion weight; and finally, obtaining a final fusion image through NSST inverse transformation. Because the low-frequency sub-band and the high-frequency sub-band are respectively subjected to corresponding weight correction strategies, the discrimination error rate of the focusing area is reduced. The effectiveness of the method is verified in experiments of a plurality of groups of different focusing images.

Description

Multi-focus image fusion method based on NSST and significant information extraction

Technical Field

The invention relates to a multi-focus image fusion problem in the field of image fusion, in particular to a multi-focus image fusion method based on NSST and significant information extraction.

Background

The image fusion technology belongs to a large research hotspot in the field of image processing. Information (brightness, color, space, etc.) contained in one image is limited, and therefore, it is difficult to satisfy a specific application scene only by one image with limited information. A plurality of images with different pieces of information are fused together according to a certain rule, so that an image which contains more information and can be observed more conveniently is obtained. Obviously, the goal of image fusion is to preserve as much useful information as possible while removing some redundant information. The multi-focus image fusion integrates different focus images in the same scene by a certain fusion method, so that the definition of the fused image is higher, and the contained information is richer.

The image fusion field is applied to a plurality of pixel level fusion methods, including fusion based on a spatial domain, a fusion method based on a transform domain and a fusion method based on deep learning. Among them, the transform domain-based fusion method is applied more than the other two methods. The transform domain is to transform the original data information of an image by some reversible mathematical transformation to obtain intermediate data with different feature information. And performing corresponding fusion rule processing on the intermediate data to obtain a fused image through reversible transformation. Obviously, in this type of method, the reversible transformation and fusion rules become critical. The reversible transformations that occur one after the other are pyramid transformation, wavelet transformation, non-downsampling contourlet transformation (NSCT), non-downsampling shear wave transformation (NSST), and the like. The adopted fusion rules are relatively many, and there are fusion rules based on spatial frequency, fusion rules based on energy information, fusion rules based on guided filtering, and the like. From different perspectives, scholars propose a plurality of fusion algorithms. The fused images obtained by most fusion algorithms have the phenomena of low definition, lost focusing information, fuzzy focusing edge and the like.

Disclosure of Invention

The invention provides a multi-focus image fusion method based on NSST and significant information extraction. And (3) carrying out NSST conversion on different focused images to obtain high and low frequency subband coefficients, and processing by adopting different fusion rules and a certain correction rule to finally obtain a fused image. The invention mainly achieves the aim through the following process steps:

(1) processing different focused images by using NSST transformation to obtain high and low frequency sub-band coefficients;

(2) performing primary processing on the low-frequency subband coefficient obtained in the step (1) by adopting an initial low-frequency fusion rule of improved Laplace energy and (SML) to obtain an initial low-frequency fusion weight;

(3) performing certain error correction on the result in the step (2) by using a low-frequency correction fusion rule extracted by the significance information;

(4) performing primary processing on the high-frequency sub-band coefficient obtained in the step (1) by adopting an initial high-frequency fusion rule based on a correlation coefficient to obtain an initial high-frequency fusion weight;

(5) carrying out discrimination correction on a series of high-frequency fusion weights by using Phase Consistency (PC) correction rules with different degrees;

(6) and (4) performing NSST inverse transformation on the processing results obtained in the steps (3) and (5) to obtain a fusion result.

Drawings

FIG. 1 is a multi-focus image fusion framework diagram based on NSST and salient information extraction; FIG. 2 is a block diagram of a low frequency subband modification rule;

Detailed Description

The invention introduces non-local mean value filtering (NLMF) and Guided Filtering (GF) to carry out weighting correction on low-frequency subband coefficients, combines spatial frequency and energy based on correlation coefficients to form an initial high-frequency weighting fusion rule, and simultaneously applies a phase consistency correction strategy to carry out correction judgment on initial high-frequency fusion weight.

The non-local mean filtering modification fusion rule is as follows:

after NSST transformation is carried out on different focused images, the obtained low-frequency sub-band images lose detail information, and the initial fusion weight obtained by applying the initial fusion rule based on the improved Laplace energy Sum (SML) has a judgment errorFiltering the image, and performing difference operation on the source image and the filtered image to obtain significance information D_l，

D_l＝|I_l-I_l×NMLF| (1)

(1) In the formula: i is_lAnd (L is more than 0 and less than L) is a source image, L represents the number of the source images, and x represents the filtering operation.

Then the guiding filtering is used for obtaining the detail information of the focusing area,

G_l＝guidedfilter(I_l,D_l,r,eps) (2)

finally, obtaining the low-frequency sub-band correction fusion weight by adopting a large-scale strategy

And carrying out error correction on the initial fusion weight.

The initial high frequency weighted fusion rule is as follows:

the high-frequency sub-band image comprises the detail information of the source image, and the invention combines the spatial frequency and the energy by utilizing the correlation coefficient to form an initial high-frequency fusion rule.

First, the operation of the correlation coefficient (Corr) is defined:

in the formula:

respectively representing the high frequency sub-band images and the images subjected to mean filtering of the high frequency sub-band images,

are respectively

M × N is the size of the image block taken.

Then, the spatial frequency and energy (SF _ Eng _ Corr) calculation formula based on the correlation coefficient is defined as follows:

in the formula: SF _ Corr and Eng _ Corr represent spatial frequency correlation coefficients and energy correlation coefficients, respectively, and for the focused region and the unfocused region of an image, the SF _ Corr value and the Eng _ Corr value of the focused region class tend to be larger than those of the unfocused region. Using this point, the two are combined in weight to form SF _ Eng _ Corr, and then the initial high-frequency fusion weight is obtained by using the large strategy.

The phase consistency correction strategy is as follows:

the initial high frequency fusion weights derived from the correlation coefficients ignore the correlation of the high frequency subband coefficients themselves. For this purpose, the phase consistency PC is used for correction, the procedure is as follows:

the invention adopts a new active measurement rule NAM to obtain the high-frequency correction fusion weight:

in the formula: PC, LSCM, LE represent phase consistency, local sharpness transformation and local energy respectively; alpha, beta and gamma are respectively scale factors.

For the high-frequency subband coefficient, the NAM can integrate local energy information, detail edge information and gradient information carried by the coefficient per se together through a certain proportion, so that the discrimination of a focus area is facilitated.

In order to verify the effectiveness of the multi-focus image fusion method based on NSST and significant information extraction, a series of comparison experiments are carried out. In the experiment, 3 groups of different focusing images with the sizes of 512 pixels multiplied by 512 pixels, 640 pixels multiplied by 480 pixels and 512 pixels multiplied by 512 pixels are selected to carry out a fusion experiment, and the fusion experiment is compared with the five conventional algorithms; and simultaneously, six evaluation indexes are adopted for quantitative evaluation. All the comparison pictures are registered, and the experimental results are shown in table one:

TABLE 1 average index results

Tab.1 The result of average index

It can be seen from the table that the method of the present invention improves the visual clarity of the fused image to 0.9009 and the structural similarity to 0.9945 on the premise of retaining enough fusion information. In the other indexes, the standard deviation STD is slightly reduced, and the others are improved to a certain extent. The algorithm not only effectively retains the detail information such as the contour, the texture and the like of the source image, but also has good visual effect in the focusing edge area of the image. The contrast definition of the fused image is improved to a certain extent, and the fusion effect is ideal. Therefore, the algorithm is a feasible multi-focus image fusion method.

Claims

1. A multi-focus image fusion method based on NSST and significant information extraction is characterized by comprising the following steps:

(2) performing primary processing on the low-frequency subband coefficient obtained in the step (1) by adopting an initial low-frequency fusion rule of improved Laplace energy Sum (SML) to obtain an initial low-frequency fusion weight;

(3) performing certain error correction on the result in the step (2) by using a low-frequency correction fusion rule extracted by the saliency information;

(4) performing primary processing on the high-frequency sub-band coefficient obtained in the step (1) by adopting an initial high-frequency fusion rule based on the correlation coefficient to obtain an initial high-frequency fusion weight;

2. The method according to claim 1, wherein a non-local mean filtering modified fusion rule is added in the step (3) to make the low frequency fusion weight more accurate, wherein the modified fusion rule is as follows:

filtering the source image by using non-local mean filtering, and performing difference operation on the source image and the filtered image to obtain significance information D_l：

D_l＝|I_l-I_l×NMLF| (1)

(1) In the formula: i is_l(L is more than 0 and less than L) is a source image, L represents the number of the source images, and x represents filtering operation;

and then, obtaining the detail information of the focusing area by using the guiding filtering:

G_l＝guidedfilter(I_l,D_l,r,eps) (2)

and finally, obtaining the low-frequency sub-band correction fusion weight by adopting a large strategy, and performing error correction on the initial fusion weight.

3. The method according to claim 1, wherein the spatial frequency and energy based on the correlation coefficient are applied to the initial high-frequency fusion weight in the step (4) and extracted as follows:

for the focused region and the unfocused region of the image, the spatial frequency correlation coefficient value SF _ Corr and the energy correlation coefficient value (Eng _ Corr) of the focused region tend to be larger than those of the unfocused region; by utilizing the point, the two are combined in a weighted manner to form spatial frequency and energy (SF _ Eng _ Corr) based on the correlation coefficient, and then an initial high-frequency fusion weight is obtained by utilizing a large-scale strategy;

in the formula: SF _ Corr and Eng _ Corr represent spatial frequency correlation coefficients and energy correlation coefficients, respectively.

4. The method according to claim 1, wherein in step (5), the initial high frequency fusion weight is subjected to phase consistency fusion correction, and the high frequency detail information, the local energy information and the gradient edge information are integrated together in a certain proportion, so as to facilitate the discrimination of the focus region.

5. The method according to claim 1, wherein initial fusion weights are obtained for high and low frequency subband coefficients obtained through NSST transformation according to different fusion rules, and the initial fusion weights are respectively modified by using modified fusion rules, so that the information content and the definition of the fused image are improved to a certain extent.