CN106157317A - The high-resolution remote sensing image fusion rules method guided based on dispersion tensor - Google Patents

Abstract

The present invention provides a kind of high-resolution remote sensing image fusion rules method guided based on scattered tensor, including by multispectral image and the panchromatic image piecemeal through Image registration, anisotropic model is taked to carry out the up-sampling of dispersion tensor guiding, select Remote Sensing Image Fusion, GPU parallel computation is used to accelerate, Remote Sensing Image Fusion evaluation index correlation analysis.The present invention can regulate the dispersion tensor model action intensity to image pixel by parameter in upsampling process so that edge-protected and removal noise reaches balance according to the actual requirements；Up-sampling is not affected by image quality such as the bright contrasts of raw video, makes superimposed image the most corresponding, it is to avoid in the false structure phenomenon of adjacent edges pixel.Further, propose universal model and summarize convergence strategy, by correlation analysis overall merit gained fusion evaluation quality；Give full play to existing equipment performance, utilize the calculating advantage multiple threads that GPU intensity is big.

Description

The high-resolution remote sensing image fusion rules method guided based on dispersion tensor

Technical field

The invention belongs to Remote Sensing Image Fusion technical field, particularly relate to a kind of high-resolution based on dispersion tensor guiding distant Sense image up-sampling fusion method.

Background technology

Multi-spectrum remote sensing image plays an important role, yet with radiation in fields such as agricultural, forest, mineral reserve, environment Transmitting procedure and the restriction of sensor process, its spatial resolution is relatively low, it is impossible to preferably obtain ground object detail.On the other hand, High Resolution Remote Sensing Satellites can obtain the earth observation of sub-meter grade, and its image but can only provide the information of panchromatic wave-band.Therefore, Satellite platform carries multispectral and panchromatic high sub sensor the most simultaneously, by the means of the visual fusion multispectral letter to ground Breath and detailed information carry out the most comprehensive, provide more valuable data for subsequent applications.

Visual fusion general flow includes that multispectral image up-sampling, image conversion, Gray-scale Matching, composition are replaced and image Inverse transformation.Existing research shows, the most most visual fusion algorithms are keeping fusion evaluation multispectral with original as far as possible While image spectral signature is consistent, the representability of its spatial detail has and to a certain degree declines.Reason is, most methods Need multispectral image is upsampled in pre-treatment step the resolution consistent with panchromatic image, but conventional up-sampling Method does not accounts for multispectral image and panchromatic image in concordance geometrically, causes superimposed image not have the most right Should, near atural object edge and tiny atural object, false structure often occurs.

Existing remote sensing image obtains platform towards many stars, multisensor, high spatial resolution, high spectral resolution and short time Visiting the periodic development, causing the remotely-sensed data obtained is geometric growth, and traditional CPU multiple programming cannot meet application system The efficiency that data are processed and the requirement of precision.

Therefore this area urgently can practical related art scheme occur.

Summary of the invention

The problem existed for prior art, the present invention proposes a kind of high-resolution remote sensing image guided based on scattered tensor Fusion rules method.

Technical solution of the present invention provides a kind of high-resolution remote sensing image fusion rules method guided based on scattered tensor, bag Include following steps:

Step 1, by multispectral image and panchromatic image piecemeal through Image registration；

Step 2, up-sampling, including following sub-step,

Step 2.1, selects a pair image blocks to be fused, for training parameter from step 1 acquired results；According to presetting Parameter alpha, β and γ initial value, take anisotropic model to carry out the up-sampling of dispersion tensor guiding, self-adapting detecting is to be fused Image edge also uses least square fitting；

When taking the up-sampling that anisotropic model carries out dispersion tensor guiding, below equation is utilized to describe multispectral remote sensing The point diffusion of image and upsampling process,

M (k)=X (k) D A (k)

K=1 ..., K

Wherein, variable k is the band number of multispectral image, and K is the wave band number participating in merging, and M is original multispectral Image, X is the high-resolution estimated value of multispectral image, and A is the conversion that two-dimensional points spread function is corresponding, and D is two dimension up-sampling The conversion that process is corresponding；

Use Gaussian filter to smooth original panchromatic image, be calculated gradient direction n=d/ | | d | | of each pixel And gradient intensity | | d | |, wherein d is the gradient vector calculated；When gradient is zero, it is stipulated that n=[1,0]^T, it is to avoid ladder Spend the not qualitative of direction；

The dispersion tensor t defining each pixel is as follows,

$t={\mathrm{wdd}}^T+d_{\⊥}{d}_{\⊥}^T$

Wherein, d_⊥It is defined as vertical with gradient vector d, i.e. the vector in grey scale change minimum direction, d^T、It is respectively vector d、d_⊥Turn order, edge-protected function w is

W=exp (-β | | d | |^γ)

Tensor model is used for energy function optimization, obtains

$X^{*}\left(k\right)=\underset{X\left(k\right)}{\arg }min\left(\right|\left|S\right(k)\·X(k)-M(k\left)\right|{|}^2+\mathrm{\α}\left|\right|T^{1/2}\·G\·X\left(k\right)\left|{|}^2\right)$

K=1 ..., K

Wherein, S=D A, G are the matrixing that gradient operator is corresponding, and T is the dispersion tensor t diagonal angle arrangement group of each pixel The gray scale tensor matrix become, α is the weights of anisotropy canonical constraint, and X is for treating that solution seeks image, and M is original multispectral image； X^*For gained fusion evaluation, K is the wave band number participating in merging；

Based on linear least square formula, it is calculated as follows,

X^*=(S^TS+αG^TTG)^-1S^TM

By solving double optimization problem by wave band, obtain the multispectral image of up-sampling；

Step 2.2, carries out objective indicator evaluation, if precision optimum then enters step 3, otherwise enters step 2.3；

Step 2.3, regulates parameter alpha, the current value of β and γ, returns step 2.1；

Step 3, selects Remote Sensing Image Fusion；

Step 4, uses GPU parallel computation to accelerate, with Form generation and the holding fusion results of block, including reading and process The same number of data block is to internal memory, and each process processes a pair graph block to be fused respectively；

Step 5, Remote Sensing Image Fusion evaluation index correlation analysis, it is correlated with including factor-cluster analysis objective evaluation index Property, choose independent evaluation index overall merit fusion results precision from many aspects.

And, in step 3,

If the multispectral image M up-sampling after registration is expressed as follows to the process that resolution is identical with panchromatic image,

M^L=usp (M)

Wherein, usp () represents up-sampling algorithm, M^LFor the result of up-sampling, useWithRepresent fusion respectively Multispectral image front and back, uses pan_(i,j)Represent the panchromatic image participating in merging, represent the wave band participating in merging, (i, j) table with k Show image pixel position；

The universal model proposing remote sensing image Pixel-level blending algorithm is as follows,

$X_{(k,i,j)}^H=M_{(k,i,j)}^L+F_{(k,i,j)}\·S_{(i,j)}$

Wherein, convergence strategy F_(k,i,j)Represent, panchromatic image pan_(i,j)Each pixel is (i, j) high frequency spatial of position is thin Joint information S_(i,j)Represent.

Compared to the prior art, the invention have the advantages that and beneficial effect:

(1) upsampling process can regulate dispersion tensor model pair by regulating the value of tri-independent parameters of β, γ, α The action intensity of image pixel so that edge-protected and removal noise reaches balance according to the actual requirements.Up-sampling is not by original The impact of the image quality such as the bright contrast of image, can save remote sensing image preprocessing process.The up-sampling result of gained is taken into account many Spectrum image and panchromatic image, in concordance geometrically, make superimposed image the most corresponding, it is to avoid in adjacent edges pixel False structure phenomenon.

(2) convergence strategy that universal model is summarized can reflect that spectrum is kept by each blending algorithm and spatial detail increases clearly Strong difference lays particular stress on.The evaluation index chosen by correlation analysis is separate, can melt from different dimensions overall merit gained Close the quality of image.

(3) give full play to existing equipment performance, be different from traditional method, the present embodiment by the up-sampling of multispectral image and Fusion process distributes to GPU, utilizes the calculating advantage multiple threads that GPU intensity is big, operation time can be greatly decreased.And image Piecemeal can avoid large format image cannot read in memory problem.

Accompanying drawing explanation

Fig. 1 is the flow chart of the embodiment of the present invention.

Detailed description of the invention

Below by embodiment, and combine accompanying drawing, technical scheme is described in further detail.

Seeing Fig. 1, the embodiment of the present invention provides a kind of high-resolution remote sensing image fusion rules method that scattered tensor guides, Comprise the following steps:

Step 1, by multispectral image and panchromatic image piecemeal through Image registration, the image blocks size of multispectral image It is 512 × 512 pixels, the image blocks of panchromatic image a size of 2048 × 2048 pixel.The image blocks of multispectral image and panchromatic The image blocks one_to_one corresponding of image.Input data rigid registrations in embodiment, can create storage file to be fused, will Multispectral and each wave band data of panchromatic image is stored in file by piecemeal result, it is simple to follow-up convergence strategy modularity multiplexing.

Step 2, up-sampling, including following sub-step:

Step 2.1, selects a pair image blocks to be fused, in order to training parameter from step 1 acquired results；According to presetting Parameter alpha, β and γ initial value, use dispersion tensor guide top sampling method (disffusion tensor driven, DTD), use anisotropy it is assumed that self-adapting detecting image edge to be fused use least square fitting, keep how light The edge concordance of spectrum image and panchromatic image simultaneously, improves quality and the details representability of follow-up fusion evaluation.

When being embodied as, can split the image blocks of multispectral image and the image blocks of panchromatic image of gained from step 1, Specified or randomly select a pair corresponding image blocks in position by user.Initial parameter value α=0.5 that the present embodiment is recommended, β= 50, γ=1.5.

Step 2.2, carries out objective indicator evaluation, if precision optimum then enters step 3, otherwise enters step 2.3；

The objective evaluation indexs such as SAM, RMSE, UIQI, PSNR, SD can be used according to the actual requirements when being embodied as.

Step 2.3, regulates parameter alpha, the current value of β and γ, returns step 2.1.When being embodied as, people in the art Member the regulation step-length of sets itself regulative mode, such as α, β and γ can be respectively 0.1,1,0.1, can increase according to practical situation Or reduce.

The DTD top sampling method that embodiment provides, it is achieved be described as follows:

When taking the up-sampling that anisotropic model carries out dispersion tensor guiding, below equation is utilized to describe multispectral remote sensing The point diffusion of image and upsampling process.

M (k)=X (k) D A (k)

K=1 ..., K

Wherein, variable k is the band number of multispectral image, K be participate in merge wave band number, M be registration after original Multispectral image, X is the high-resolution estimated value of multispectral image, and A is the conversion that two-dimensional points spread function is corresponding, and D is two dimension The conversion that upsampling process is corresponding.Two conversion are all linear, and this linear equation system is used for describing high-resolution scene and exists The process of multispectral image is obtained after sensor occurring some diffusion and sampling with low resolution.

A Gaussian filter is used to smooth original panchromatic image, to weaken the noise impact on gradient calculation.Calculate To gradient direction n=d/ | | d | | and gradient intensity | | d | | of each pixel, wherein d is the gradient vector calculated.For gradient It is the situation of zero (perfectly homogenous), it is stipulated that n=[1,0]^T, it is to avoid gradient direction not qualitative.

For the region of clear-cut margin, adaptive smooth can correspondingly reduce constraint strength.Anisotropy parameter tensor The key concept processed as partial differential equation remote sensing image, has the strongest suitability for this problem.Define each pixel Dispersion tensor t:

$t={\mathrm{wdd}}^T+d_{\⊥}{d}_{\⊥}^T$

Wherein, d_⊥It is defined as vertical with gradient vector d, i.e. the vector in grey scale change minimum direction, d^T It is respectively vector d d_⊥Turn order, edge-protected function w is

W=exp (-β | | d | |^γ)

When both sides of edges gray scale difference is the biggest, and the value of w is the least, i.e. smoothness constraint is the most weak.By arranging taking of parameter beta and γ Value can make bound term that the edge exceeding prescribed strength is just occurred significant response.

Tensor model is used for energy function optimization, obtains:

$X^{*}\left(k\right)=\underset{X\left(k\right)}{\arg }min\left(\right|\left|S\right(k)\·X(k)-M(k\left)\right|{|}^2+\mathrm{\α}\left|\right|T^{1/2}\·G\·X\left(k\right)\left|{|}^2\right)$

K=1 ..., K

Wherein S=D A, D, A point the most above spreads and up-sampling function, and G is the matrix that gradient operator is corresponding Conversion, T is the gray scale tensor matrix that the dispersion tensor t diagonal angle of each pixel rearranges, and α is the power of anisotropy canonical constraint Value, X is for treating that solution seeks image, and M is original multispectral image.X^*For gained fusion evaluation, K is the wave band number participating in merging.

Above formula is classical quadratic programming problem, has globally optimal solution, makes the residual error tried to achieve minimum, i.e. linear least-squares Problem, calculating process is given by:

X^*=(S^TS+αG^TTG)^-1S^TM

The gray scale tensor matrix T full rank of definition, therefore inverse matrix exists.By solving double optimization problem by wave band, Multispectral image to up-sampling.This fusion evaluation result both account for the blurring effect of a diffusion process, also maintains with high The edge concordance of resolution panchromatic wave band.The false structure phenomenon of edge can be reduced largely, it is to avoid follow-up fusion is grasped The high frequency panchromatic image information caused owing to edge is unjustified in work injects mistake phenomenon.

The disperse process of embodiment anisotropic describes multi-spectrum remote sensing image pixel and high resolution spatial panchromatic image Many-one relationship between pixel.Detect edge by calculating multi-spectrum remote sensing image pixel grey scale gradient, build dispersion tensor, For the detection of high-resolution remote sensing image edge details, for taking different up-sampling strategies to provide judgement to depend at subsequent edges According to.Tensor model is used for energy function optimization, it is thus achieved that the solution that all band residual error is minimum, improves the precision of image edge matching.

Step 3, choosing of Remote Sensing Image Fusion: analyze the mathematical model of conventional blending algorithm, and every objective Dependency between evaluation index, constitutes a set of OO Remote Sensing Image Fusion system, can be by actual demand to spectrum Keep the difference strengthened with space to lay particular stress on and select fusion method flexibly.

Use universal model to summarize convergence strategy, select spectral information holding and spatial detail to strengthen according to practical situation Difference lays particular stress on.

If through step 2 will after registration original multispectral image M up-sampling to the resolution process identical with panchromatic image, Can be expressed as:

M^L=usp (M)

Usp () represents up-sampling algorithm, M^LFor the result of up-sampling, useWithRepresent respectively before and after merging Multispectral image, uses pan_(i,j)Representing the panchromatic image participating in merging, represent the wave band participating in merging with k, (i j) represents image Location of pixels.It is proposed that the universal model of remote sensing image Pixel-level blending algorithm:

$X_{(k,i,j)}^H=M_{(k,i,j)}^L+F_{(k,i,j)}\·S_{(i,j)}$

In above formula, convergence strategy F_(k,i,j)Represent, panchromatic image pan_(i,j)Each pixel is in (i, j) high frequency spatial of position Detailed information S_(i,j)Represent.

The foundation of universal model can compare advantage and the limitation of various convergence strategy clearly, when choosing integration program Thought is clear.Final fusion results is by the convergence strategy coefficient F used_(k,i,j)With the high frequency spatial detailed information parameter extracted S_(i,j)Both together decide on.

Analyze fusion parameters F_(k,i,j)And S_(i,j)During may need use information have: participate in merge multispectral The each band class information of remote sensing image；Panchromatic image (i, j) positional information of pixel；And (i, j) the neighborhood territory pixel letter that may use Breath and statistical information based on the overall situation.

Remote Sensing Image Fusion universal model is following 2 points at the Heterosis of application aspect: one is advantageous for summarizing difference The mathematical thought of blending algorithm, it is simple to for respective advantage on-demand selection convergence strategy in follow-up fusion；Two is at algorithm Execution aspect be conducive to the integrated of module and multiplexing.

Such as following three kinds of fusion methods:

A. fusion method based on variable replacement

In fusion method based on variable replacement, S_(i,j)Can be summarized as the high frequency detail letter of the high score panchromatic wave-band extracted Breath, generally by panchromatic wave-band rectangular histogram change strengthen or after orthogonal transformation certain component of multispectral image at data space Difference is constituted, therefore S in convergence strategy_(i,j)Play Main Function, bigger to the image of follow-up fusion results；F_(k,i,j)Generally by Matrixing is constituted.

B. fusion method based on modulation intelligence

S_(i,j)After generally being filtered with it by high-resolution panchromatic wave-band image, image asks mathematic interpolation to draw, another kind of situation Time can by the synthesis wave band that panchromatic wave-band is similar to it ask difference calculate；F_(k,i,j)Can be summarized as panchromatic image to synthesize with some The ratio of wave band；The fusion results of this type of mode can be according to S_(i,j)、F_(k,i,j)Difference lay particular stress on meet obtain multispectral or high-altitude Between the different needs of resolution.

C. fusion method based on multiscale analysis

Use F_(k,i,j)Representing the difference of high score panchromatic image and its classification approximation component after down-sampled, progression is the biggest, F_(k,i,j)Relative to S_(i,j)Weights are the least, show as spatial detail and keep more preferably, but the increase of decomposed class can cause multispectral The loss of information.

Step 4, GPU parallel computation is accelerated: use GPU parallel computation to accelerate, and Form generation and holding with block merge knot Really.Whole flow process only reads the data block identical with number of processes to internal memory, each process process respectively a pair to be fused Graph block.Import and export with single compared with full width image, greatly reduce memory consumption.

Unified calculation framework CUDA (Compute Unified Device Architecture) is the exploitation of Nvidia company GPU multiple programming platform, the process thread far more than CPU can be enabled under CUDA environment simultaneously, improve greatly and calculate effect Rate.Under CUDA environment, utilize the calculating advantage multiple threads that GPU intensity is big, operation time can be greatly decreased.

Multinuclear calculates under platform, and a process calculated in core correspondence parallel environment, each process passes through process Number 0 identifies from k-1.Output fusion evaluation divides according to row major order and numbers, and line number is m, and columns is n, by 0 to m × n-1 Numbering, then during parallel computation, the process number of i-th piece of image distribution is i% (k-1)+1, the image blocks number that each process processes For (m × n)/(k-1).

Step 5, Remote Sensing Image Fusion evaluation index correlation analysis: factor-cluster analysis objective evaluation is index related, Choose independent evaluation index overall merit fusion results precision from many aspects.

Set up correlation matrix by calculating the correlation coefficient between each index, calculate each other by factor analysis Independent factor carries out cluster analysis, finally can be in the most representative independence of quality evaluation by acquired results classification Objective indicator, remote sensing image quality evaluation after follow-up fusion.Typical objectives index independent of each other is obtained according to this system, Set up based on comprehensive score objective evaluation system, for the most comprehensive qualitative overall merit fusion results quality.

5.1 calculate Pearson correlation coefficients

Use the linear correlation degree of Pearson correlation coefficients response variable conventional in multi-variate statistical analysis

${\mathrm{\ρ}}_{x,y}=\frac{\mathrm{cov}(x,y)}{\mathrm{\σ}x\mathrm{\σ}y}$

Wherein x, y are variable, cov (x, y) represents the covariance of variable x, y, and σ x σ y represents the long-pending of variable x, y standard deviation, From Cauchy-Schwarz inequality, when the linear relationship of two variablees strengthens, correlation coefficient levels off to 1 or-1.

5.2 build correlation matrix

The mathematical model of factorial analysis can be summarized as

X=AF+ ε

Wherein X is original variable vector, and A is common factor load factor matrix, by factor loading a_ijConstitute, a_ijFor i-th Variable and the correlation coefficient of jth common factor.F is because of subvector, and ε is residual vector, and each variable can be with common factor Linear function represents with residual vector sum.

5.3 factor-cluster analysis obtain evaluation index

Obtaining, by factorial analysis, the factor that independence is stronger, the index that comprises when the factor is more is to cluster the factor Analyze, obtained the typical index of each classification by classification.The present embodiment uses and averagely ties connection, obtains with Euclidean distance Cluster result independent between compact, class in class.Average knot connection is prior art, and it will not go into details for the present invention.

The separate evaluation index of above step gained is quantitatively commented for the follow-up each dimension of Remote Sensing Image Fusion result Valency.When being embodied as, those skilled in the art can use computer software mode to realize the automatic operation of above flow process.

Specific embodiment described in the present invention is only to present invention spirit explanation for example.Technology belonging to the present invention Described specific embodiment can be made various amendment or supplements or use similar mode by the technical staff in field Substitute, but without departing from the spirit of the present invention or surmount scope defined in appended claims.

Claims (2)

1. the high-resolution remote sensing image fusion rules method guided based on scattered tensor, it is characterised in that include following step Rapid:

Step 1, by multispectral image and panchromatic image piecemeal through Image registration；

Step 2, up-sampling, including following sub-step,

Step 2.1, selects a pair image blocks to be fused, for training parameter from step 1 acquired results；According to default ginseng Number α, β and γ initial value, takes anisotropic model to carry out the up-sampling of dispersion tensor guiding, self-adapting detecting image to be fused Edge also uses least square fitting；

When taking the up-sampling that anisotropic model carries out dispersion tensor guiding, below equation is utilized to describe multi-spectrum remote sensing image Some diffusion and upsampling process,

M (k)=X (k) D A (k)

K=1 ..., K

Wherein, variable k is the band number of multispectral image, and K is the wave band number participating in merging, and M is original multispectral image, X is the high-resolution estimated value of multispectral image, and A is the conversion that two-dimensional points spread function is corresponding, and D is two dimension upsampling process pair The conversion answered；

Use Gaussian filter to smooth original panchromatic image, be calculated gradient direction n=d/ | | d | | and the ladder of each pixel Degree intensity | | d | |, wherein d is the gradient vector calculated；When gradient is zero, it is stipulated that n=[1,0]^T, it is to avoid gradient side To not qualitative；

The dispersion tensor t defining each pixel is as follows,

$t={\mathrm{wdd}}^T+d_{\⊥}{d}_{\⊥}^T$

Wherein, d_⊥It is defined as vertical with gradient vector d, i.e. the vector in grey scale change minimum direction, d^T、It is respectively vector d, d_⊥ Turn order, edge-protected function w is

W=exp (-β | | d | |^γ)

Tensor model is used for energy function optimization, obtains

$\begin{array}{c}{X}^{*}\left(k\right)=\underset{X\left(k\right)}{\arg }\min \left(\right||S\left(k\right)\·X\left(k\right)-M\left(k\right)|{|}^2+\mathrm{\α}\left|\right|T^{1/2}\·G\·X\left(k\right)\left|{|}^2\right)\\ k=1,\mathrm{...},K\end{array}$

Wherein, S=D A, G are the matrixing that gradient operator is corresponding, and T is that the dispersion tensor t diagonal angle of each pixel rearranges Gray scale tensor matrix, α is the weights of anisotropy canonical constraint, and X is for treating that solution seeks image, and M is original multispectral image；X^*For Gained fusion evaluation, K is the wave band number participating in merging；

Based on linear least square formula, it is calculated as follows,

X^*=(S^TS+αG^TTG)^-1S^TM

By solving double optimization problem by wave band, obtain the multispectral image of up-sampling；

Step 2.2, carries out objective indicator evaluation, if precision optimum then enters step 3, otherwise enters step 2.3；

Step 2.3, regulates parameter alpha, the current value of β and γ, returns step 2.1；

Step 3, selects Remote Sensing Image Fusion；

Step 4, uses GPU parallel computation to accelerate, with Form generation and the holding fusion results of block, including reading and number of processes Identical data block is to internal memory, and each process processes a pair graph block to be fused respectively；

Step 5, Remote Sensing Image Fusion evaluation index correlation analysis, index related including factor-cluster analysis objective evaluation, Choose independent evaluation index overall merit fusion results precision from many aspects.

The most according to claim 1, the high-resolution remote sensing image fusion rules method guided based on scattered tensor, its feature exists In: in step 3,

If the multispectral image M up-sampling after registration is expressed as follows to the process that resolution is identical with panchromatic image,

M^L=usp (M)

Wherein, usp () represents up-sampling algorithm, M^LFor the result of up-sampling, useWithRepresent respectively before and after merging Multispectral image, use pan_(i,j)Representing the panchromatic image participating in merging, represent the wave band participating in merging with k, (i j) represents shadow As location of pixels；

The universal model proposing remote sensing image Pixel-level blending algorithm is as follows,

$X_{(k,i,j)}^H=M_{(k,i,j)}^L+F_{(k,i,j)}\·S_{(i,j)}$

Wherein, convergence strategy F_{(k, i, j)}Represent, panchromatic image pan_(i,j)Each pixel is at (i, j) the high frequency spatial details letter of position Breath S_(i,j)Represent.