CN104318539A - Sonar image fusion method based on extended Piella framework - Google Patents

Abstract

The invention relates to the field of sonar image processing and specifically relates to a sonar image fusion method based on an extended Piella framework. The sonar image fusion method based on the extended Piella framework comprises the steps of (1) initializing parameters and inputting two groups of sonar images to be fused, (2) performing multiresolution conversion on the sonar images to be fused and establishing the respective multiresolution image sequences of the sonar images to be fused, and (3) performing multiresolution inverse transformation on the high frequency sub-bands and low frequency sub-bands of the images to be fused to obtain the final fusion result. The invention firstly provides a fusion method for sonar continuous frame images. Compared with an existing image fusion method, the sonar image fusion method based on the extended Piella framework is remarkably improved in both definition and detail expression of fusion.

Description

A kind of sonar image fusion method based on expansion Piella framework

Technical field

What the present invention relates to is a kind of sonar image process field.Be specifically related to a kind of sonar image fusion method based on expansion Piella framework.

Background technology

Sonar image fusion is an important branch of sonar image process field.In the application of Digital Image Processing, often needing comprehensively to analyze target object, therefore with regard to needing, several objects with target being merged.But weaken because sonar image has imaging object edge, picture contrast is low, intensity is non-homogeneous, resolution not high, very easily causes the phenomenon that fusion mass is undesirable, and therefore sonar image merges is one of difficult problem of digital image processing techniques.

G Piella. is in the multiple Image Fusion based on multiscale analysis of article " A general framework for multiresolution image fusion:from pixels to regions " analysis and comparison of 2003Information Fusion, and the general framework proposed on this basis based on multiscale analysis image interfusion method, image co-registration process based on multiscale analysis is divided into modules by this framework, and it is classified and sums up, make major part based on multiscale analysis Image Fusion can under this framework united analysis and comparing.Document " the image co-registration researchs based on the different fusion rules of wavelet transformation " have studied the Multi-Sensory Image Fusion at Pixel Level based on multi-resolution decomposition, compares according to different fusion rule and the effect merging operator structure fused images.Document " non-lower sampling based on image quality evaluation parameter shears wave zone Adaptive image fusion " proposes a kind of Multiscale image fusion methods based on image quality evaluation parameter, overcomes and to cause fusion results and the inconsistent problem of subjective assessment because of not fully taking into account human visual system.

Summary of the invention

The object of the present invention is to provide a kind of to the sonar image fusion method based on expansion Piella framework between successive frame sonar image or between the sonar image with certain correlativity and partial information loss.

The object of the present invention is achieved like this:

(1) initialization is carried out to parameter, input two groups of sonar images to be fused;

(2) multi-resolution ratio change is carried out to sonar image to be fused, sets up sonar image to be fused multi-resolution image sequence separately:

(2.1) determine and extract the activity-level measured based on image quality evaluation standard:

The activity-level measured of image quality evaluation standard:

$a_S^{\mathrm{\ϵ}.j}(x,y)={\left(G_S^{\mathrm{\ϵ},j}(x,y)\right)}^{w_G}\·{\left(C_S^{\mathrm{\ϵ},j}(x,y)\right)}^{W_C},$

${\hat{a}}_S^{\mathrm{\ϵ},j}(x,y)=\frac{a_S^{\mathrm{\ϵ},j}(x,y)}{\underset{S=A,B}{\mathrm{\Σ}}{a}_S^{\mathrm{\ϵ},j}(x,y)},$

with fen Do represents the average gradient of sonar image S to be fused on jth decomposition layer ε direction and contrast; w _gand w _crepresent the weight that corresponding image quality evaluation criterion is shared in activity-level measured respectively, represent the activity-level measured after regularization;

(2.2) the Multiresolution Decomposition coefficient of sonar image to be fused is merged:

$c_F^{\mathrm{\ϵ},j}(x,y)=w_A^{\mathrm{\ϵ},j}(x,y)\·c_A^{\mathrm{\ϵ},j}(x,y)+w_B^{\mathrm{\ϵ},j}(x,y)\·c_B^{\mathrm{\ϵ},j}(x,y),$

Wherein, represent weights, namely $w_A^{\mathrm{\ϵ},j}(x,y)={\hat{a}}_A^{\mathrm{\ϵ},j}(x,y)=1-w_B^{\mathrm{\ϵ},j}(x,y);$

(2.3) extract and merge low frequency sub-band notable feature:

$c_F^{0,N}(x,y)=w_A^{0,N-1}(x,y)\·c_A^{0,N}(x,y)+w_B^{0,N-1}(x,y)\·c_B^{0,N}(x,y),$

represent normalization weights, namely $w_A^{0,N-1}(x,y)=1-w_B^{0,N-1}(x,y),$

represent the remarkable characteristic in the N-1 layer approximation component local neighborhood of image S to be fused;

(3) high-frequency sub-band and the low frequency sub-band for the treatment of fused images carry out multiresolution inverse transformation, obtain final fusion results.

Beneficial effect of the present invention is:

The present invention first proposed the fusion method for sonar sequential frame image.Compared with image interfusion method in the past, the method all increases significantly in the sharpness merged and details expressive force.

This method and classics based on Piella framework image interfusion method compared with, in robustness, embody very large advantage, be mainly manifested in (1) comparison for Noise image co-registration result; (2) for the comparison of low-resolution image fusion results.In addition in the present invention, algorithm operation quantity is greatly reduced owing to adopting expansion Piella framework.Show that the method is accurate robust through the experiment of sonar sequential frame image.

Accompanying drawing explanation

Fig. 1 is the overall flow block diagram of this inventive method.

S1 is original image to be fused.

S2 is for carry out Multiresolution Decomposition respectively to original image to be fused.

S3, for extracting image quality evaluation standard, participates in activity-level measured and determines.

S4 is the Multiresolution Decomposition coefficient merging original image.

S5 is the high frequency to fused images, and low frequency sub-band carries out multiresolution inverse transformation.

S6 is according to fusion results, carries out visual.

The result that Fig. 2 display is merged sonar successive frame Brummer shipwreck image based on the image interfusion method expanding Piella framework.

Fig. 2 (a), (b) is Brummer shipwreck image the 73rd frame and the 63rd two field picture,

The result of Fig. 2 (c) for merging after registration.

Fig. 3 display is based on the fusion results of image interfusion method under different multiresolution analysis method of expansion Piella framework.

Fig. 3 (a), (b) is sonar successive frame seabed rust plate image the 49th frame and the 41st two field picture,

Fig. 3 (c) is the fusion results of sonar successive frame seabed rust plate image under grad pyramid multi-resolution ratio change,

Fig. 3 (d) is the fusion results of sonar successive frame seabed rust plate image under laplacian pyramid multi-resolution ratio change,

Fig. 3 (e) is the fusion results of sonar successive frame seabed rust plate image under the conversion of Mallat Wavelet Multiresolution Decomposition,

Fig. 3 (f) is the fusion results of sonar successive frame seabed rust plate image under non-downsampling Contourlet multi-resolution ratio change.

Embodiment

Below in conjunction with accompanying drawing, the present invention is described further:

The present invention relates to a kind of image interfusion method based on Piella framework in sonar image process field.Key step comprises: based on the multi-resolution decomposition of grad pyramid; Based on the fusion treatment of expansion Piella framework.First sonar image is carried out multi-resolution decomposition by grad pyramid conversion, obtain the coefficient of dissociation of each layer all directions of image to be fused.Then, by combining expansion Piella framework, fusion treatment is realized to coefficient of dissociation.Therebetween, propose a kind of expansion Piella image co-registration framework, this framework takes full advantage of the strong feature of sonar image correlativity, has abandoned match measure and decision-making module, thus effectively simplifies image co-registration process.On the basis of this extension framework, propose the multi-Resolution Image Fusion rule of a kind of combining image criteria of quality evaluation and last decomposition layer information, image quality evaluation standard is incorporated in high-frequency sub-band fusion rule by this rule on the one hand, fully taking into account fusion results mainly with human eye is the stay of two nights, and then avoids ignoring the impact of human visual system on fusion results; Be incorporated in low-frequency subband fusion rule by last decomposition layer information on the other hand, effectively overcome the existence due to multi-resolution decomposition, the information recoverable amount of current decomposition layer low frequency sub-band is significantly less than the problem of last decomposition layer.The sonar image that the present invention can realize having local message loss merges, and is particularly suitable for the fusion treatment of sonar sequential frame image.

The present invention is a kind of based on expansion Piella framework, by introducing image quality evaluation standard and last decomposition layer information realization, for the method that sonar sequential frame image merges, in sonar image fusion process, first extracted the coefficient of dissociation of image by multiresolution analysis, then build above-mentioned criteria of quality evaluation and last decomposition layer information, and then complete the height of image respectively, low-frequency subband fusion, completes fusion treatment finally by multiresolution inverse transformation.Its specific features is as follows:

(1) to the expansion of Piella framework

The feature strong according to sonar image correlativity, expands Piella framework, has abandoned match measure and decision-making module, carries out the fusion of sonar sequential frame image, and undertaken visual by fusion results in conjunction with extension framework.

(2) high-frequency sub-band is merged

Activity-level measured under extension framework introduces image quality evaluation standard (combination as average gradient and contrast), differentiates, so as to determining the proportion of image to be fused in fusion results before fusion to original image quality.

(3) low frequency sub-band is merged

From image low frequency sub-band N-1 layer to be fused, extract certain notable feature, the coefficient of dissociation of combining image low frequency sub-band n-th layer, calculate fused images low frequency sub-band coefficient of dissociation.

Below only to merge sonar successive frame Brummer shipwreck image and seabed rust plate image illustrates using method of the present invention:

Technical scheme of the present invention specifically comprises the steps:

1. pair original image carries out multi-resolution ratio change, sets up image to be fused multi-resolution image sequence separately.

2. treat fused images high, low frequency sub-band merges respectively

(1) based on determination and the extraction of the activity-level measured of image quality evaluation standard, calculate the quality assessment parameter of each pixel of original image based on window area, then multi-resolution ratio change is carried out to quality evaluating matrix, the multiresolution sequence of outcome parameter matrix is corresponding original image high-frequency sub-band quality assessment parameter, and then determines the proportion of image to be fused in fusion results;

(2) according to the corresponding relation of low frequency sub-band adjacent layer, certain notable feature in N-1 layer local neighborhood is defined, and then the weight of each pixel when determining that n-th layer approximation component merges.

3. the high-frequency sub-band of pair fused images and low frequency sub-band carry out multiresolution inverse transformation, thus obtain final fusion results.

Further:

1. pair algorithm parameter carries out initialization, inputs two groups of sonar images to be fused, as shown in Fig. 2 (a) He Fig. 2 (b), Fig. 3 (a) He Fig. 3 (b).

2. treat fused images and carry out multi-resolution ratio change, set up original image multi-resolution image sequence separately

(1) based on determination and the extraction of the activity-level measured of image quality evaluation standard

For image S to be fused, define the activity-level measured based on certain image quality evaluation standard, it is defined as:

Wherein, represent the function of image S to be fused at jth decomposition layer ε direction epigraph criteria of quality evaluation.

, have selected the combination of average gradient and contrast as activity-level measured here, in fact, according to the difference merging object and image type, also can adopt other image quality evaluation standards.

$a_S^{\mathrm{\ϵ}.j}(x,y)={\left(G_S^{\mathrm{\ϵ},j}(x,y)\right)}^{w_G}\·{\left(C_S^{\mathrm{\ϵ},j}(x,y)\right)}^{W_C}---\left(2\right)$

Wherein, with fen Do represents the average gradient of image S to be fused on jth decomposition layer ε direction and contrast; w _gand w _crepresent the weight that corresponding image quality evaluation criterion is shared in activity-level measured respectively.

Consider that the numerical value magnitude of different images criteria of quality evaluation is different, regularization is carried out to formula (2), that is:

${\hat{a}}_S^{\mathrm{\ϵ},j}(x,y)=\frac{a_S^{\mathrm{\ϵ},j}(x,y)}{\underset{S=A,B}{\mathrm{\Σ}}{a}_S^{\mathrm{\ϵ},j}(x,y)}---\left(3\right)$

Wherein, represent the activity-level measured after regularization.

(2) the Multiresolution Decomposition coefficient of image to be fused is merged

The each coefficient of dissociation treating fused images synthesizes, that is:

$c_F^{\mathrm{\ϵ},j}(x,y)=w_A^{\mathrm{\ϵ},j}(x,y)\·c_A^{\mathrm{\ϵ},j}(x,y)+w_B^{\mathrm{\ϵ},j}(x,y)\·c_B^{\mathrm{\ϵ},j}(x,y)---\left(4\right)$

Wherein, represent weights, namely $w_A^{\mathrm{\ϵ},j}(x,y)={\hat{a}}_A^{\mathrm{\ϵ},j}(x,y)=1-w_B^{\mathrm{\ϵ},j}(x,y).$

(3) low frequency sub-band notable feature is extracted and is merged

According to the corresponding relation of low frequency sub-band adjacent layer, define certain notable feature (as local variance) in N-1 layer local neighborhood, that is:

$c_F^{0,N}(x,y)=w_A^{0,N-1}(x,y)\·c_A^{0,N}(x,y)+w_B^{0,N-1}(x,y)\·c_B^{0,N}(x,y)---\left(5\right)$

Wherein, represent normalization weights, namely shown in it is defined as follows:

Wherein, represent certain the remarkable characteristic in the N-1 layer approximation component local neighborhood of image S to be fused.

3. the high-frequency sub-band of pair fused images and low frequency sub-band carry out multiresolution inverse transformation, thus obtain final fusion results.

This give the result of Brummer shipwreck and seabed rust plate successive frame sonar image fusion.

Fig. 1 is the overall flow block diagram of this inventive method.

S1 is original image to be fused.

S2 is for carry out Multiresolution Decomposition respectively to original image to be fused.

S3, for extracting image quality evaluation standard, participates in activity-level measured and determines.

S4 is the Multiresolution Decomposition coefficient merging original image.

S5 is the high frequency to fused images, and low frequency sub-band carries out multiresolution inverse transformation.

S6 is according to fusion results, carries out visual.

In Fig. 2, give image sets Fig. 2 (a) before fusion and Fig. 2 (b), and carry out the result example after image co-registration, Fig. 2 (c), from these figure, we can find out that our method has very high details expressive ability and informix ability.

In Fig. 3, give the fusion results under different multiresolution analysis method, Fig. 3 (a), Fig. 3 (b) is image sets before merging, Fig. 3 (c) is the fusion results under grad pyramid multi-resolution ratio change, Fig. 3 (d) is the fusion results under laplacian pyramid multi-resolution ratio change, Fig. 3 (e) is the fusion results under the conversion of Mallat Wavelet Multiresolution Decomposition, Fig. 3 (f) is the fusion results under non-downsampling Contourlet multi-resolution ratio change

Due in our method, take into full account human visual system, thus effectively overcome the problem of fusion results objective evaluation and subjective inconsistency, it also avoid classical Piella framed structure complicated simultaneously, thus substantially increase the precision of fusion, robustness, reduce algorithm operation quantity.

The present invention can realize between successive frame sonar image or there is certain correlativity and partial information loss sonar image between image co-registration.

In order to our fusion method of quantitative evaluation, we adopt spatial domain simple average method and this paper fusion method to sonar sequential frame image Fen Do in Fig. 2, and give the fusion results (table 1) under three kinds of popular Objective image quality evaluation indexes.This quantitative evaluation result shows, our method can obtain good syncretizing effect.

Table 1

Claims (1)

1., based on a sonar image fusion method for expansion Piella framework, it is characterized in that:

(1) initialization is carried out to parameter, input two groups of sonar images to be fused;

(2) multi-resolution ratio change is carried out to sonar image to be fused, sets up sonar image to be fused multi-resolution image sequence separately:

(2.1) determine and extract the activity-level measured based on image quality evaluation standard:

The activity-level measured of image quality evaluation standard:

$a_S^{\mathrm{\ϵ},j}(x,y)={\left(G_S^{\mathrm{\ϵ},j}(x,y)\right)}^{w_G}\·{\left(C_S^{\mathrm{\ϵ},j}(x,y)\right)}^{w_C},$

${\hat{a}}_S^{\mathrm{\ϵ},j}(x,y)=\frac{a_S^{\mathrm{\ϵ},j}(x,y)}{\underset{S=A,B}{\mathrm{\Σ}}{a}_S^{\mathrm{\ϵ},j}(x,y)},$

with fen Do represents the average gradient of sonar image S to be fused on jth decomposition layer ε direction and contrast; WG and wC represents the weight that corresponding image quality evaluation criterion is shared in activity-level measured respectively, represent the activity-level measured after regularization;

(2.2) the Multiresolution Decomposition coefficient of sonar image to be fused is merged:

$c_F^{\mathrm{\ϵ},j}(x,y)=w_A^{\mathrm{\ϵ},j}(x,y)\·C_A^{\mathrm{\ϵ},j}(x,y)+w_B^{\mathrm{\ϵ},j}(x,y)\·c_B^{\mathrm{\ϵ},j}(x,y),$

Wherein, represent weights, namely $w_A^{\mathrm{\ϵ},j}(x,y)={\hat{a}}_A^{\mathrm{\ϵ},j}(x,y)=1-w_B^{\mathrm{\ϵ},j}(x,y);$

(2.3) extract and merge low frequency sub-band notable feature:

$c_F^{0,N}(x,y)=w_A^{0,N-1}(x,y)\·c_A^{0,N}(x,y)+w_B^{0,N-1}(x,y)\·c_B^{0,N}(x,y),$

represent normalization weights, namely $w_A^{0,N-1}(x,y)=1-w_B^{0,N-1}(x,y),$

represent the remarkable characteristic in the N-1 layer approximation component local neighborhood of image S to be fused;

(3) high-frequency sub-band and the low frequency sub-band for the treatment of fused images carry out multiresolution inverse transformation, obtain final fusion results.