CN117078538B - Correction method of remote atmospheric turbulence image based on pixel motion statistics - Google Patents
Correction method of remote atmospheric turbulence image based on pixel motion statistics Download PDFInfo
- Publication number
- CN117078538B CN117078538B CN202310893813.7A CN202310893813A CN117078538B CN 117078538 B CN117078538 B CN 117078538B CN 202310893813 A CN202310893813 A CN 202310893813A CN 117078538 B CN117078538 B CN 117078538B
- Authority
- CN
- China
- Prior art keywords
- image
- sequence
- pixel value
- images
- registration
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 64
- 238000012937 correction Methods 0.000 title claims abstract description 44
- 238000006731 degradation reaction Methods 0.000 claims abstract description 73
- 230000015556 catabolic process Effects 0.000 claims abstract description 72
- 238000011156 evaluation Methods 0.000 claims abstract description 22
- 230000009466 transformation Effects 0.000 claims abstract description 10
- 238000010276 construction Methods 0.000 claims description 11
- 238000004364 calculation method Methods 0.000 claims description 5
- 238000005457 optimization Methods 0.000 claims description 5
- 238000000354 decomposition reaction Methods 0.000 claims description 4
- 230000008569 process Effects 0.000 claims description 4
- 241000208340 Araliaceae Species 0.000 claims description 3
- 235000005035 Panax pseudoginseng ssp. pseudoginseng Nutrition 0.000 claims description 3
- 235000003140 Panax quinquefolius Nutrition 0.000 claims description 3
- 230000003190 augmentative effect Effects 0.000 claims description 3
- 235000008434 ginseng Nutrition 0.000 claims description 3
- 230000008521 reorganization Effects 0.000 claims description 3
- 230000000694 effects Effects 0.000 description 9
- 230000009286 beneficial effect Effects 0.000 description 4
- 238000003384 imaging method Methods 0.000 description 4
- 238000012935 Averaging Methods 0.000 description 3
- 238000010187 selection method Methods 0.000 description 3
- 230000008859 change Effects 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000005259 measurement Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 238000000513 principal component analysis Methods 0.000 description 2
- 238000012545 processing Methods 0.000 description 2
- 238000004458 analytical method Methods 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 239000011159 matrix material Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000000704 physical effect Effects 0.000 description 1
- 238000012847 principal component analysis method Methods 0.000 description 1
- 230000001902 propagating effect Effects 0.000 description 1
- 239000000523 sample Substances 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/30—Determination of transform parameters for the alignment of images, i.e. image registration
- G06T7/33—Determination of transform parameters for the alignment of images, i.e. image registration using feature-based methods
- G06T7/337—Determination of transform parameters for the alignment of images, i.e. image registration using feature-based methods involving reference images or patches
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02A—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
- Y02A90/00—Technologies having an indirect contribution to adaptation to climate change
- Y02A90/10—Information and communication technologies [ICT] supporting adaptation to climate change, e.g. for weather forecasting or climate simulation
Landscapes
- Engineering & Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Image Processing (AREA)
Abstract
The invention discloses a correction method of a remote atmospheric turbulence image based on pixel motion statistics. The method comprises the following steps: counting the pixel values in the remote atmosphere turbulence degradation image sequence, and constructing a reference evaluation image by taking the pixel value with the largest occurrence number of each position as the pixel value of the position; calculating the geometric score map of each degraded image, and selecting the top k degraded images with the highest scores; counting the pixel values in the k selected degraded images, and constructing a reference frame by taking the weighted average pixel value of each position as the pixel value of the position; calculating deformation fields between the k selected degraded images and the reference frame, and performing geometric transformation on the k selected degraded images by using the deformation fields to obtain a registration sequence; and correcting the registration error of the registration sequence to obtain a corrected image sequence. The definition of the reference frame is improved, and the registration error in the registration sequence is removed, so that the quality of the corrected image is improved.
Description
Technical Field
The invention belongs to the field of image processing, and particularly relates to a correction method of a remote atmospheric turbulence image based on pixel motion statistics.
Background
Atmospheric turbulence effects are widely present in high temperature and long range probe imaging, where changes in physical properties exhibited by light propagating in the atmosphere originate from the effects of the refractive index of the atmosphere, which depends on the temperature, humidity, pressure of the atmosphere and the wavelength of the light. If the atmospheric refractive index is kept constant on the propagation path of light, no disturbance is generated to the light wave. However, atmospheric turbulence causes random variations in the temperature, refractive index of the atmosphere, both in time and space. The effect of the small change of the atmospheric refractive index is equivalent to that of small lenses distributed on the light propagation path, and the small lenses enable the light beam to be focused, deflected and the like, so that the effects of light flicker, light image dithering and the like are caused. The atmospheric turbulence makes the refractive index of air change randomly along with time and space, so that the intensity, phase and directivity of light rays fluctuate in the transmission process, and finally the geometrical distortion of an imaging result is degraded. And the anisotropy, non-uniformity and randomness of atmospheric turbulence make the above-described degradation process highly ill-conditioned, highly non-linear. Thus, degradation model creation for atmospheric turbulence imaging and image restoration is a great challenge.
For correcting distortion, the current common method is to use an image registration algorithm based on optical flow or B-spline to act on the degraded image so as to align the degraded image to a reference image, wherein the clearer the reference image is, the more accurate the estimated deformation field is, and the better the registration effect is. There are two types of reference frame construction methods commonly used at present: one is to average the degraded sequence to obtain an average image as a reference image, the other is to apply a Robust Principal Component Analysis (RPCA) to the degraded sequence, a group of images are obtained by utilizing the static low-rank characteristic of the image background as the reference image, the reference images generated by the two methods are approximate and undistorted, but are relatively fuzzy, especially in the edge area, so that the estimated deformation field by utilizing the reference images is inaccurate, and finally the registration effect is reduced.
When long-distance imaging is carried out, because geometric distortion caused by atmospheric turbulence is serious, offset pixel values in degraded images cannot be completely aligned only through image registration, registration errors exist in a registration sequence, and the existing atmospheric turbulence correction method does not consider how to remove the existing registration errors, so that the image quality is further improved. In summary, the problems of the prior art include: the generated reference frame is blurred, so that estimation of a deformation field is influenced in image registration, a distortion correction result is reduced, and a registration error generated in image registration is not removed, so that the correction result is still poor.
Disclosure of Invention
Aiming at the defects of the related art, the invention aims to provide a correction method of a remote atmospheric turbulence image based on pixel motion statistics, which aims to solve the problems that a reference frame generated by the existing method is relatively blurred, the correction effect is poor in image registration, the registration error exists in the image registration process, and the correction effect is reduced.
To achieve the above object, in a first aspect, the present invention provides a method for correcting a distant atmospheric turbulence image based on pixel motion statistics, including:
s1, counting pixel values in a remote atmosphere turbulence degradation image sequence, and constructing a reference evaluation image by taking the pixel value with the largest occurrence number of each position as the pixel value of the position; comparing each degradation image in the degradation image sequence with the reference evaluation image, and calculating a geometric score map; scoring each degradation image according to the geometric score map, and selecting top k degradation images with highest scores;
s2, counting pixel values in the k selected degraded images, and constructing a reference frame by taking the weighted average pixel value of each position as the pixel value of the position; the weight of the pixel value at each position is determined according to the occurrence times of the pixel value;
s3, calculating deformation fields between the k selected degraded images and the reference frame, and performing geometric transformation on the k selected degraded images by using the deformation fields to obtain a registration sequence;
s4, correcting the registration error of the registration sequence to obtain a corrected image sequence.
Optionally, the geometric score map is: score_map t The calculation formula is as follows:
wherein { D t The sequence of images of remote atmospheric turbulence degradation is shown, H and W are shown for each image D t (x, y) represents the pixel point location coordinates in the image, D most (x, y) is the pixel value with the largest number of occurrences.
Optionally, the scoring each degraded image according to the geometric score map, and selecting the top k degraded images with the highest scores includes:
calculating to obtain each degraded image D t The geometric fraction of (2) is:
wherein H and W are the length and width of the images in the sequence of remote atmospheric turbulence degradation images, score_map, respectively t Is a geometric score graph;
and selecting top k degradation images with highest scores from the degradation image sequence according to the obtained geometric scores.
Optionally, S2 includes:
s21, inputting the selected k degradation images;
s22, recording the pixel value D appearing at each position (x, y) along the sequence time dimension k k (x, y), counting the number of times f each pixel value appears at the position, and according to the number of times f and the pixel value D k (x, y) construction of reference frame D ref The reference frame D ref The pixel value at the (x, y) position is D ref (x, y), the specific calculation formula is:
w k (x,y)＝e σf ；
wherein sigma is a harmonic parameter, w k (x, y) is the pixel value D appearing at the (x, y) position k (x, y).
Optionally, S3 includes:
s31, obtaining k selected degraded images and the reference frame D by adopting a streamer estimation algorithm ref Deformation field D between k (u,v)；
S32, the deformation field D k (u, v) acting on the selected k degraded images such that the geometry of the selected k degraded images after transformation approximates the reference frame D ref Obtaining a registration sequence { R } k }。
Optionally, S4 specifically includes:
s41, registering the sequence { R } k Forming a tensor along the time dimension kThe tensor->The method comprises the following steps:
wherein,representing a corrected image sequence>Representing registration errors;
s42, tensor alignmentConstructing a low-rank tensor restoration model by adopting a maximum posterior probability method:
in the method, in the process of the invention,for data fidelity term, H j (x) Represents regularization term ω j Representing the weighting coefficients, j e {2,3}, j=2 represents the non-local dimension, j=3 represents the time dimension, such as: h 2 (x)、H 3 (x) The non-local dimension and time dimension regularized prior term information is respectively represented; omega 2 、ω 3 Weighting coefficients respectively representing non-local dimension and time dimension prior term information;
s43, constructing a priori term H in the time dimension and the non-local dimension of the registration sequence respectively according to the non-local similarity priori and the time low-rank priori of the registration sequence j (x)：
Wherein,for the third order low rank tensor obtained after similar block reorganization, ++>Rank (& gt) represents the rank operator for the intermediate variable, (×) j ) Mode j product, lambda representing tensors i Is a super ginseng; the method comprises the steps of carrying out a first treatment on the surface of the
Converting the low rank tensor restoration model into:
s44, correcting the registration error of the registration sequence by using the converted low-rank tensor restoration model to obtain a corrected image sequence.
Optionally, S43 specifically includes:
s431, solving three variable updating types of the low-rank tension restoration model by adopting an alternate optimization strategy;
s432, fixed variableAnd->Solving for Q i j The method comprises the steps of carrying out a first treatment on the surface of the For->After the mode j is unfolded, SVD decomposition is carried out:
wherein, takeThe optimal non-local dimension subspace is: />The optimal space-time subspace is
S433, fixed variableAnd Q i j Solving->Solving for intermediate variables +.>Take out and->The two related terms, scaling the low rank constraint term to constraint with a kernel norm, can yield the following equation:
wherein,representing low-rank constraint, and solving the equation through a WNNM algorithm;
s434, fixed variable Q i j Andsolving->Solving the variable +.>Take out and->After the two terms concerned, the following equation can be obtained:
introducing auxiliary variablesLet->The constraint problem is changed into the unconstrained problem through the augmented Lagrangian algorithm, and the following formula is obtained:
where beta is the penalty parameter,is the Lagrangian multiplier;
s435, solving the minimized nonlinear energy functional through a crossover direction multiplier algorithm or a split Bregman algorithm to obtain a variable after the minimized energy functionalTo correct the image sequence.
In a second aspect, the present invention also provides a correction device for a remote atmospheric turbulence image based on pixel motion statistics, including:
the degradation image selecting module is used for counting pixel values in the remote atmospheric turbulence degradation image sequence, and constructing a reference evaluation image by taking the pixel value with the largest occurrence number of each position as the pixel value of the position; comparing each degradation image in the degradation image sequence with the reference evaluation image, and calculating a geometric score map; scoring each degradation image according to the geometric score map, and selecting top k degradation images with highest scores;
the reference frame acquisition module is used for counting the pixel values in the k selected degraded images, and constructing a reference frame by taking the weighted average pixel value of each position as the pixel value of the position; the weight of the pixel value at each position is determined according to the occurrence times of the pixel value;
the image registration module is used for calculating deformation fields between the k selected degraded images and the reference frame, and performing geometric transformation on the k selected degraded images by using the deformation fields to obtain a registration sequence;
and the registration error correction module is used for correcting the registration error of the registration sequence to obtain a corrected image sequence.
In a third aspect, the present invention also provides a computer-readable storage medium storing computer instructions for causing a processor to perform the correction method according to any one of the first aspects.
Through the technical scheme, compared with the prior art, the beneficial effects can be realized, including:
1. the invention provides a correction method of a remote atmospheric turbulence image based on pixel motion statistics, which is based on an atmospheric turbulence pixel motion statistics priori method, and provides a new selection method of a reference image, wherein an evaluation standard of geometric distortion of a degraded image is formulated, an image with lower distortion degree is selected from a degraded sequence image of atmospheric turbulence by using the evaluation standard, the selected reference image is more beneficial to constructing a clear reference frame, and the image quality of the constructed reference frame is improved, so that the quality of the corrected image is improved.
2. According to the correction method of the remote atmospheric turbulence image based on the pixel motion statistics, when the reference frame is constructed according to the reference image, the atmospheric turbulence pixel motion statistics priori method is combined, the definition of the constructed reference frame is high, so that the accuracy of estimating a deformation field in image registration is improved, and the distortion correction effect is improved; and the construction method of the reference frame can be applied to other atmospheric turbulence correction methods.
3. The invention provides a correction method of a remote atmospheric turbulence image based on pixel motion statistics, which adopts two-stage distortion correction from coarse to fine during correction, firstly uses an image registration technology to perform coarse registration on a degradation sequence, suppresses geometric distortion and obtains a group of registration sequences; then, removing registration errors in the registration sequence by adopting a low-rank tensor restoration model to obtain a group of corrected images; the correction method can effectively inhibit geometric distortion caused by atmospheric turbulence and improve the quality of corrected images.
Drawings
FIG. 1 is a flow chart of a method for correcting a remote atmospheric turbulence image based on pixel motion statistics;
FIG. 2 is a comparison of reference images selected from degraded sequence images using different selection methods;
FIG. 3 is a diagram of a reference frame constructed by the method provided by the present invention versus a reference frame constructed in the prior art;
FIG. 4 is a comparison graph of the results of image registration of a reference frame and image registration of an average image in the method provided by the invention;
fig. 5 is a schematic diagram of correcting measured degradation data by using the correction method of the remote atmospheric turbulence image based on pixel motion statistics.
Detailed Description
The present invention will be described in further detail with reference to the drawings and examples, in order to make the objects, technical solutions and advantages of the present invention more apparent. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the invention. In addition, the technical features of the embodiments of the present invention described below may be combined with each other as long as they do not collide with each other.
The description of the contents of the above embodiment will be given below in connection with a preferred embodiment.
Example 1
As shown in fig. 1, a method for correcting a remote atmospheric turbulence image based on pixel motion statistics includes:
s1, counting pixel values in a remote atmosphere turbulence degradation image sequence, and constructing a reference evaluation image by taking the pixel value with the largest occurrence number of each position as the pixel value of the position; comparing each degradation image in the degradation image sequence with the reference evaluation image, and calculating a geometric score map; scoring each degradation image according to the geometric score map, and selecting top k degradation images with highest scores;
s2, counting pixel values in the k selected degraded images, and constructing a reference frame by taking the weighted average pixel value of each position as the pixel value of the position; the weight of the pixel value at each position is determined according to the occurrence times of the pixel value;
s3, calculating deformation fields between the k selected degraded images and the reference frame, and performing geometric transformation on the k selected degraded images by using the deformation fields to obtain a registration sequence;
s4, correcting the registration error of the registration sequence to obtain a corrected image sequence.
And counting and analyzing the motion in the atmosphere turbulence degradation image sequence to obtain a motion rule of pixels obeying near zero mean Gaussian distribution. Based on pixel motion statistics priori information, selecting frames with better image quality from an input degradation sequence, exemplarily, k degradation images are selected, based on the pixel motion statistics priori information, constructing a reference frame according to k degradation images with highest scores, combining the reference frame, adopting an image registration method based on an optical flow algorithm to inhibit geometric distortion caused by atmospheric turbulence, obtaining a registration sequence, and adopting a low-rank tensor restoration model to remove registration errors in the registration sequence, so as to obtain a group of corrected images. The invention provides a new frame selection method and a reference frame construction method, a reference image with lower distortion degree is selected to construct a reference frame, and a two-stage distortion correction method from coarse to fine is utilized, so that geometric distortion caused by atmospheric turbulence is effectively restrained by removing registration errors, the definition of the reference frame is improved, and the registration errors in a registration sequence are removed, thereby improving the quality of corrected images.
Optionally, S1 specifically includes:
s11, inputting a remote atmospheric turbulence degradation image sequence { D ] t Each image D t The length and width of (a) are H and W respectively;
s12, along the degradation sequence time dimension t, recording the pixel value D appearing at each position (x, y) t (x, y), counting the number of times f each pixel value appears at the position, and counting the pixel value D with the largest number of times most (x, y) as pixel values of the position (x, y), constructing a reference evaluation image;
s13, the degraded image sequence { D } t Each degraded image D in } t Comparing the geometric score with the reference evaluation image, and calculating a geometric score as an evaluation standard of the geometric distortion of the degraded image;
the geometric score map is as follows: score_map t The calculation formula is as follows:
wherein { D t The sequence of images of remote atmospheric turbulence degradation is shown, H and W are shown for each image D t (x, y) represents the pixel point location coordinates in the image, D most (x, y) is the pixel value with the largest number of occurrences;
s14, calculating to obtain each degradation image D t The geometric fraction of (2) is:
wherein H and W are the length and width of the images in the sequence of remote atmospheric turbulence degradation images, respectively t Is a geometric score graph;
and selecting top k degradation images with highest scores from the degradation image sequence according to the obtained geometric scores.
Input an atmospheric turbulence degradation image sequence { D t The sequence of degraded images { D } includes T degraded images, each of which has a length and width set to H and W, respectively t The degraded image of 100 frames is traversed from the 1 st frame to the 100 th frame along the time dimension t of the degraded sequence, and the pixel value D appearing at each position (x, y) from the 1 st frame to the 100 th frame is counted t (x, y) and the number f. Pixel value D with the greatest number of occurrences most (x, y) extracting pixel values as positions (x, y), and finally obtaining a degraded image sequence { D } t A reference evaluation image composed of pixel values whose number of occurrences is largest for each position in the image.
Evaluating the image and each degraded image D by reference t Comparing, calculating to obtain a geometric score map, taking the geometric score as an evaluation standard for evaluating the geometric distortion degree of the degraded image, wherein the higher the geometric score is, the smaller the geometric distortion degree is, and the sequence { D ] of the degraded image is obtained according to the geometric score t Selecting the first k degradation images with higher geometric scores as the k degradation imagesChemical image sequence { D k And performing subsequent processing. The degradation image selected according to the evaluation standard has lower distortion degree, and the selected image is more beneficial to constructing a clear reference frame.
Optionally, S2 specifically includes:
s21, inputting the selected k degradation images { D ] k }；
S22, recording the pixel value D appearing at each position (x, y) along the sequence time dimension k k (x, y), counting the number of times f each pixel value appears at the position, and according to the number of times f and the pixel value D k (x, y) construction of reference frame D ref The reference frame D ref The pixel value at the (x, y) position is D ref (x, y), the specific calculation formula is:
wherein sigma is a harmonic parameter, w k (x, y) is the pixel value D appearing at the (x, y) position k (x, y).
The k selected degraded images comprise k degraded images with lower distortion degree, traversing from the 1 st frame to the k th frame along the time dimension k of the degraded sequence by the method, and counting the pixel value D appearing from the 1 st frame to the k th frame at each position (x, y) k (x, y) and the number f. According to the pixel value D k (x, y) and the number of occurrences f, calculating a pixel value D k (x, y) weight w at corresponding position k (x, y) and then according to the pixel value D k (x, y) and weight w k (x, y) calculating the reference frame D by the above formula ref Pixel value D at each position (x, y) ref (x, y) to construct reference frame D ref . Wherein the harmonic parameter sigma is related to the degradation intensity of the selected atmospheric turbulence image, the greater the turbulence degradation intensity is, the greater the pixel offset degree is, the sigma takes a smaller value, and can take a value between 0.3 and 0.5 at will, whereas the sigma takes a larger value, and can take a value between 0.8 and 1 at will. Providing a robust reference frame construction method based on the k selected degradation images, and constructing the parametersThe frame checking definition is high, and the precision in the subsequent correction is ensured. Further, the construction method of the reference frame can be applied to other atmospheric turbulence correction methods.
Optionally, S3 specifically includes:
s31, obtaining k selected degraded images and the reference frame D by adopting a streamer estimation algorithm ref Deformation field D between k (u，v)；
S32, the deformation field D k (u, v) acting on the selected k degraded images such that the geometry of the selected k degraded images after transformation approximates the reference frame D ref Obtaining a registration sequence { R } k }。
Coarse registration is carried out on the degenerated sequence by using an image registration technology, geometric distortion is restrained, and a group of registration sequences { R } k }. The deformation field can be obtained by adopting a streamer estimation algorithm to register the selected degradation images, and the deformation field can be estimated and registered by a B-spline image registration method, wherein the streamer estimation algorithm is preferably adopted in the embodiment.
Optionally, the registration error of the corrected registration sequence may be corrected by using a robust principal component analysis method, a low rank matrix decomposition method or a low rank tensor restoration model, and in this embodiment, the correction is preferably performed by using a low rank tensor restoration model.
S4 specifically comprises the following steps:
s41, registering the sequence { R } k Forming a tensor along the time dimension kThe tensor->The method comprises the following steps:
wherein,representation ofCorrection of the image sequence, +.>Representing registration errors;
s42, tensor alignmentConstructing a low-rank tensor restoration model by adopting a maximum posterior probability method:
in the method, in the process of the invention,for data fidelity term, H j (x) Represents regularization term ω j Representing the weighting coefficients, j e {2,3}, j=2 represents the non-local dimension, j=3 represents the time dimension, such as: h 2 (x)、H 3 (x) The non-local dimension and time dimension regularized prior term information is respectively represented; omega 2 、ω 3 Weighting coefficients respectively representing non-local dimension and time dimension prior term information;
s43, constructing a priori term H in the time dimension and the non-local dimension of the registration sequence respectively according to the non-local similarity priori and the time low-rank priori of the registration sequence j (x)：
Wherein,for the third order low rank tensor obtained after similar block reorganization, ++>Rank (& gt) represents the rank operator for the intermediate variable, (×) j ) Mode j product, lambda representing tensors i Is a super ginseng;
converting the low rank tensor restoration model into:
s44, correcting the registration error of the registration sequence by using the converted low-rank tensor restoration model to obtain a corrected image sequence.
Optionally, S43 specifically includes:
s431, solving three variable updating types of the low-rank tension restoration model by adopting an alternate optimization strategy;
s432, fixed variableAnd->Solving for Q i j The method comprises the steps of carrying out a first treatment on the surface of the For->After the mode j is unfolded, SVD decomposition is carried out:
wherein, takeThe optimal non-local dimension subspace is: />The optimal space-time subspace is
S433, fixed variableAnd Q i j Solving->Solving for intermediate variables +.>Take out and->The two related terms, scaling the low rank constraint term to constraint with a kernel norm, can yield the following equation:
wherein,representing low-rank constraint, and solving the equation through a WNNM algorithm;
s434, fixed variable Q i j Andsolving->Solving the variable +.>Take out and->After the two terms concerned, the following equation can be obtained:
introducing auxiliary variablesLet->The constraint problem is changed into the unconstrained problem through the augmented Lagrangian algorithm, and the following formula is obtained:
where beta is the penalty parameter,is the Lagrangian multiplier;
s435, solving the minimized nonlinear energy functional through a crossover direction multiplier algorithm or a split Bregman algorithm to obtain a variable after the minimized energy functionalTo correct the image sequence.
Three variable updating methods for solving a low-rank tensor restoration model by adopting an alternate optimization strategy specifically comprise the following steps: decomposing a low-rank tensor restoration model into three subproblems, respectively solving the three subproblems, adopting an alternate optimization strategy, carrying out iterative solution on objective functions corresponding to the three subproblems, determining variable updating, and obtaining the variable after minimizing the energy functionalThe image sequence is corrected for the resulting refinement. The three sub-problems are respectively: the first sub-problem: fixed variable->And->Solving for Q i j The method comprises the steps of carrying out a first treatment on the surface of the A second sub-problem: fixed variable->And Q i j Solving->Third sub-problem: fixed variable Q i j And->Solving for
The above-mentioned minimized nonlinear energy functional is solved in S435 by the exchange direction multiplier algorithm or the split Bregman algorithm, specifically including:
for the above formula, take outAfter the two terms related, the following equation is obtained:
solving the above equation for its closed-form solution:
wherein,representing inverse fast fourier transform, ">Representing the fast fourier transform, representing the conjugate operation,/->Representing the product operation of the elements->Representing a unit tensor.
For auxiliary variablesAfter taking its associated item, there are:
solving the above equation may be equivalent to solving the following equation:
thereby obtaining the closed-form solution of:
in each iteration, the value is updated:
β (l+1) ＝τβ (l)
obtaining punishment parameters beta and Lagrange multiplier
After rough registration is carried out on the degraded image sequence by using image registration, removing the registration error in the registration sequence by adopting a low-rank tensor restoration model to obtain a group of corrected images; the correction method adopts two-stage distortion correction from coarse to fine, effectively inhibits geometric distortion caused by atmospheric turbulence and improves the quality of corrected images.
The invention will be further described with reference to actual measurement experiments.
(1) Experimental data
In the embodiment, the actual measurement of the atmospheric turbulence degradation sequence is adopted to verify the proposed atmospheric turbulence correction method, and the size of the degradation sequence image is 512 multiplied by 512.
(2) Experimental results and analysis
Frame selection result: as shown in fig. 2, the geometric assessment algorithm provided by the embodiment of the invention performs frame selection on the degradation sequence, and averages the selected images to obtain a reference image; where (a) in fig. 2 represents a reference image obtained by randomly selecting 20 degraded images to average, (b) in fig. 2 represents a reference image obtained by averaging 20 degraded images after geometric score ranking, and (c) in fig. 2 represents a reference image obtained by averaging 20 degraded images before geometric score ranking. The reference frame obtained by utilizing the image selected by the geometric evaluation algorithm provided by the invention can be seen from the figure, the image quality is better, and the image details are richer.
Reference frame construction results: as shown in fig. 3, a reference frame construction algorithm provided by the embodiment of the present invention is compared with a reference frame construction algorithm in the prior art; here, (a) in fig. 3 represents a reference frame obtained by a Robust Principal Component Analysis (RPCA), (b) in fig. 3 represents a reference frame obtained by averaging, and (c) in fig. 3 represents a reference frame of the present invention. As can be seen from the figure, the reference frame of the present invention has no artifacts and the image is clearer than other methods.
Image registration results: as shown in fig. 4, the result of performing image registration on the reference frame and the average image provided by the embodiment of the invention is used; wherein (a) represents a degraded image, (b) represents a registered image obtained using the average image as a reference frame, and (c) represents a registered image obtained using the reference frame of the present invention. As can be seen from the figure, the geometric distortion of the registered image obtained using the reference frame of the present invention is more effectively suppressed than the registered image obtained using the average image.
Atmospheric turbulence correction results: as shown in fig. 5, the atmospheric turbulence correction method provided by the embodiment of the invention corrects the actually measured degradation data; wherein (a) - (c) in fig. 5 represent degradation images, and (e) - (f) in fig. 5 represent atmospheric turbulence correction images. It can be seen from the figure that the proposed method can effectively suppress geometrical distortions caused by atmospheric turbulence.
On the basis of the above embodiment, the embodiment of the present invention further provides a correction device for a remote atmospheric turbulence image based on pixel motion statistics, including:
the degradation image selecting module is used for counting pixel values in the remote atmospheric turbulence degradation image sequence, and constructing a reference evaluation image by taking the pixel value with the largest occurrence number of each position as the pixel value of the position; comparing each degradation image in the degradation image sequence with the reference evaluation image, and calculating a geometric score map; scoring each degradation image according to the geometric score map, and selecting top k degradation images with highest scores;
the reference frame acquisition module is used for counting the pixel values in the k selected degraded images, and constructing a reference frame by taking the weighted average pixel value of each position as the pixel value of the position; the weight of the pixel value at each position is determined according to the occurrence times of the pixel value;
the image registration module is used for calculating deformation fields between the k selected degraded images and the reference frame, and performing geometric transformation on the k selected degraded images by using the deformation fields to obtain a registration sequence;
and the registration error correction module is used for correcting the registration error of the registration sequence to obtain a corrected image sequence.
The correction device for the remote atmospheric turbulence image based on the pixel motion statistics provided by the embodiment of the invention is used for executing the correction method for the remote atmospheric turbulence image based on the pixel motion statistics provided by any embodiment of the invention, and has the corresponding functional modules and beneficial effects of the execution device.
Embodiments of the present invention also provide a computer readable storage medium storing computer instructions for causing a processor to perform the method of any of the embodiments.
It will be readily appreciated by those skilled in the art that the foregoing description is merely a preferred embodiment of the invention and is not intended to limit the invention, but any modifications, equivalents, improvements or alternatives falling within the spirit and principles of the invention are intended to be included within the scope of the invention.
Claims (8)
1. A method for correcting a distant atmospheric turbulence image based on pixel motion statistics, comprising:
s1, counting pixel values in a remote atmosphere turbulence degradation image sequence, and constructing a reference evaluation image by taking the pixel value with the largest occurrence number of each position as the pixel value of the position; comparing each degradation image in the degradation image sequence with the reference evaluation image to construct a geometric score map consisting of a score of 0 and a score of 1; scoring each degradation image according to the geometric score map, and selecting top k degradation images with highest scores;
s2, counting pixel values in the k selected degraded images, and constructing a reference frame by taking the weighted average pixel value of each position as the pixel value of the position; the weight of the pixel value at each position is determined according to the occurrence times of the pixel value;
s3, calculating deformation fields between the k selected degraded images and the reference frame, and performing geometric transformation on the k selected degraded images by using the deformation fields to obtain a registration sequence;
s4, constructing a subspace-based low-rank tensor restoration model, and removing registration errors in the registration sequence obtained in the S3 by adopting the low-rank tensor restoration model to obtain a corrected image sequence;
wherein, the geometric score graph is: score_map t The calculation formula is as follows:
wherein { D t The sequence of images of remote atmospheric turbulence degradation is shown, H and W are shown for each image D t (x, y) represents the pixel point location coordinates in the image, D most (x, y) is the pixel value with the largest number of occurrences.
2. The correction method as set forth in claim 1, wherein said scoring each degraded image according to the geometric score map and selecting the top k degraded images with the highest scores comprises:
calculating to obtain each degraded image D t The geometric fraction of (2) is:
wherein H and W are the length and width of the images in the sequence of remote atmospheric turbulence degradation images, score_map, respectively t Is a geometric score graph;
and selecting top k degradation images with highest scores from the degradation image sequence according to the obtained geometric scores.
3. The correction method as set forth in claim 1, wherein S2 includes:
s21, inputting the selected k degradation images;
s22, recording the pixel value D appearing at each position (x, y) along the sequence time dimension k k (x, y), counting the number of times f each pixel value appears at the position, and according to the number of times f and the pixel value D k (x, y) construction of reference frame D ref The method comprises the steps of carrying out a first treatment on the surface of the The reference frame D ref Pixel value D at the (x, y) position ref (x, y) is:
wherein sigma is a harmonic parameter, w k (x, y) is the pixel value D appearing at the (x, y) position k Weights of (x, y)。
4. The correction method as set forth in claim 1, wherein S3 includes:
s31, obtaining k selected degraded images and the reference frame D by adopting a streamer estimation algorithm ref Deformation field D between k (u,v)；
S32, the deformation field D k (u, v) acting on the selected k degraded images such that the geometry of the selected k degraded images after transformation approximates the reference frame D ref Obtaining a registration sequence { R } k }。
5. The correction method as set forth in claim 1, wherein S4 specifically includes:
s41, registering the sequence { R } k Forming a tensor along the time dimension kThe tensor->The method comprises the following steps:
wherein,representing a sequence of finely corrected images,/->Representing registration errors;
s42, tensor alignmentConstructing a low-rank tensor restoration model by adopting a maximum posterior probability method:
in the method, in the process of the invention,for data fidelity term, H j (x) Represents regularization term ω j Representing the weighting coefficients, j e {2,3}, j=2 represents the non-local dimension, j=3 represents the time dimension, such as: h 2 (x)、H 3 (x) The non-local dimension and time dimension regularized prior term information is respectively represented; omega 2 、ω 3 Weighting coefficients respectively representing non-local dimension and time dimension prior term information;
s43, constructing a priori term H in the time dimension and the non-local dimension of the registration sequence respectively according to the non-local similarity priori and the time low-rank priori of the registration sequence j (x)：
Wherein,for the third order low rank tensor obtained after similar block reorganization, ++>Rank (& gt) represents the rank operator for the intermediate variable, (×) j ) Mode j product, lambda representing tensors i Is a super ginseng;
converting the low rank tensor restoration model into:
s44, correcting the registration error of the registration sequence by using the converted low-rank tensor restoration model to obtain a corrected image sequence.
6. The correction method as set forth in claim 5, wherein S43 specifically includes:
s431, solving three variable updating types of the low-rank tension restoration model by adopting an alternate optimization strategy;
s432, fixed variableAnd->Solving for Q i j The method comprises the steps of carrying out a first treatment on the surface of the For->After the mode j is unfolded, SVD decomposition is carried out:
wherein, takeThe optimal non-local dimension subspace is: />The optimal space-time subspace is
S433, fixed variableAnd Q i j Solving->Solving for intermediate variables +.>Take out and->The two related terms, scaling the low rank constraint term to constraint with a kernel norm, can yield the following equation:
wherein,representing low-rank constraint, and solving the equation through a WNNM algorithm;
s434, fixed variable Q i j Andsolving->Solving the variable +.>Take out and->After the two terms concerned, the following equation can be obtained:
introducing auxiliary variablesLet->The constraint problem is changed into the unconstrained problem through the augmented Lagrangian algorithm, and the following formula is obtained:
where beta is the penalty parameter,is the Lagrangian multiplier;
s435, solving the minimized nonlinear energy functional through a crossover direction multiplier algorithm or a split Bregman algorithm to obtain a variable after the minimized energy functionalTo correct the image sequence.
7. A correction device for a remote atmospheric turbulence image based on pixel motion statistics, comprising:
the degradation image selecting module is used for counting pixel values in the remote atmospheric turbulence degradation image sequence, and constructing a reference evaluation image by taking the pixel value with the largest occurrence number of each position as the pixel value of the position; comparing each degradation image in the degradation image sequence with the reference evaluation image to construct a geometric score map consisting of a score of 0 and a score of 1; scoring each degradation image according to the geometric score map, and selecting top k degradation images with highest scores;
the reference frame acquisition module is used for counting the pixel values in the k selected degraded images, and constructing a reference frame by taking the weighted average pixel value of each position as the pixel value of the position; the weight of the pixel value at each position is determined according to the occurrence times of the pixel value;
the image registration module is used for calculating deformation fields between the k selected degraded images and the reference frame, and performing geometric transformation on the k selected degraded images by using the deformation fields to obtain a registration sequence;
the registration error correction module is used for constructing a subspace-based low-rank tensor restoration model, and removing registration errors in the registration sequence obtained by the image registration module by adopting the low-rank tensor restoration model to obtain a corrected image sequence.
8. A computer readable storage medium storing computer instructions for causing a processor to perform the correction method according to any one of claims 1-6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310893813.7A CN117078538B (en) | 2023-07-19 | 2023-07-19 | Correction method of remote atmospheric turbulence image based on pixel motion statistics |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310893813.7A CN117078538B (en) | 2023-07-19 | 2023-07-19 | Correction method of remote atmospheric turbulence image based on pixel motion statistics |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN117078538A CN117078538A (en) | 2023-11-17 |
| CN117078538B true CN117078538B (en) | 2024-02-13 |
Family
ID=88718410
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310893813.7A Active CN117078538B (en) | 2023-07-19 | 2023-07-19 | Correction method of remote atmospheric turbulence image based on pixel motion statistics |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117078538B (en) |
Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102752602A (en) * | 2012-06-19 | 2012-10-24 | 清华大学 | Video sequence loss frame recovery method and device |
| CN103310486A (en) * | 2013-06-04 | 2013-09-18 | 西北工业大学 | Reconstruction method of atmospheric turbulence degraded images |
| CN105898159A (en) * | 2016-05-31 | 2016-08-24 | 努比亚技术有限公司 | Image processing method and terminal |
| CN106920220A (en) * | 2017-02-24 | 2017-07-04 | 西北工业大学 | Based on the turbulent flow method for blindly restoring image that dark primary and alternating direction multiplier method optimize |
| CN106991659A (en) * | 2017-03-29 | 2017-07-28 | 江苏省海洋资源开发研究院(连云港) | A kind of multi-frame self-adaption optical image restoration methods for adapting to atmospheric turbulance change |
| CN110223242A (en) * | 2019-05-07 | 2019-09-10 | 北京航空航天大学 | A kind of video turbulent flow removing method based on time-space domain Residual Generation confrontation network |
| CN110796616A (en) * | 2019-10-23 | 2020-02-14 | 武汉工程大学 | Fractional order differential operator based L0Norm constraint and adaptive weighted gradient turbulence degradation image recovery method |
| CN110852963A (en) * | 2019-10-29 | 2020-02-28 | 天津大学 | Turbulence degradation image restoration method based on correlation maximum |
| CN111047544A (en) * | 2020-01-08 | 2020-04-21 | 华中科技大学 | Saturated image deblurring method based on nonlinear degradation model |
| CN114581497A (en) * | 2022-04-16 | 2022-06-03 | 合肥学院 | Improved B-spline non-rigid registration method for atmospheric turbulence image distortion correction |
| CN115170606A (en) * | 2022-06-15 | 2022-10-11 | 深圳先进技术研究院 | Motion correction method and system for whole-body dynamic PET imaging |
| CN116363014A (en) * | 2023-04-21 | 2023-06-30 | 电子科技大学长三角研究院(衢州) | Cascade self-adaptive optical image blind restoration method |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20140105515A1 (en) * | 2012-04-04 | 2014-04-17 | The Regents Of The University Of California | Stabilizing and Deblurring Atmospheric Turbulence |
| US11640655B2 (en) * | 2021-04-14 | 2023-05-02 | Raytheon Company | Systems and methods for image turbulence correction of moving targets |
| US20230046545A1 (en) * | 2021-08-06 | 2023-02-16 | Government Of The United States As Represented By The Secretary Of The Air Force | Tilt variance method of detecting and accounting for true motion of imagery subject to atmospheric turbulence |
-
2023
- 2023-07-19 CN CN202310893813.7A patent/CN117078538B/en active Active
Patent Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102752602A (en) * | 2012-06-19 | 2012-10-24 | 清华大学 | Video sequence loss frame recovery method and device |
| CN103310486A (en) * | 2013-06-04 | 2013-09-18 | 西北工业大学 | Reconstruction method of atmospheric turbulence degraded images |
| CN105898159A (en) * | 2016-05-31 | 2016-08-24 | 努比亚技术有限公司 | Image processing method and terminal |
| CN106920220A (en) * | 2017-02-24 | 2017-07-04 | 西北工业大学 | Based on the turbulent flow method for blindly restoring image that dark primary and alternating direction multiplier method optimize |
| CN106991659A (en) * | 2017-03-29 | 2017-07-28 | 江苏省海洋资源开发研究院(连云港) | A kind of multi-frame self-adaption optical image restoration methods for adapting to atmospheric turbulance change |
| CN110223242A (en) * | 2019-05-07 | 2019-09-10 | 北京航空航天大学 | A kind of video turbulent flow removing method based on time-space domain Residual Generation confrontation network |
| CN110796616A (en) * | 2019-10-23 | 2020-02-14 | 武汉工程大学 | Fractional order differential operator based L0Norm constraint and adaptive weighted gradient turbulence degradation image recovery method |
| CN110852963A (en) * | 2019-10-29 | 2020-02-28 | 天津大学 | Turbulence degradation image restoration method based on correlation maximum |
| CN111047544A (en) * | 2020-01-08 | 2020-04-21 | 华中科技大学 | Saturated image deblurring method based on nonlinear degradation model |
| CN114581497A (en) * | 2022-04-16 | 2022-06-03 | 合肥学院 | Improved B-spline non-rigid registration method for atmospheric turbulence image distortion correction |
| CN115170606A (en) * | 2022-06-15 | 2022-10-11 | 深圳先进技术研究院 | Motion correction method and system for whole-body dynamic PET imaging |
| CN116363014A (en) * | 2023-04-21 | 2023-06-30 | 电子科技大学长三角研究院(衢州) | Cascade self-adaptive optical image blind restoration method |
Non-Patent Citations (4)
| Title |
|---|
| A New Blind Video Quality Metric for Assessing Different Turbulence Mitigation Algorithms;Chiman Kwan等;《Electronics》;第10卷(第18期);第1-16页 * |
| 多帧湍流退化图像复原算法研究;斯那卓玛;《中国优秀硕士学位论文全文数据库 信息科技辑》(第2期);第I138-893页 * |
| 空气湍流影响下的视频图像恢复研究;陈华;《中国优秀硕士学位论文全文数据 信息科技辑》(第6期);第I138-584页 * |
| 自适应光学成像事后处理LoG域匹配图像质量评价;牛威等;《红外与激光工程》;第47卷(第11期);第1111005-1-9页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN117078538A (en) | 2023-11-17 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Chung et al. | Diffusion posterior sampling for general noisy inverse problems | |
| Vidal et al. | Maximum likelihood estimation of regularization parameters in high-dimensional inverse problems: An empirical bayesian approach part i: Methodology and experiments | |
| Zhong et al. | SAR image despeckling using Bayesian nonlocal means filter with sigma preselection | |
| Lanusse et al. | High resolution weak lensing mass mapping combining shear and flexion | |
| CN106127688B (en) | A kind of super-resolution image reconstruction method and its system | |
| Winkler et al. | Focus gradient correction applied to tilt series image data used in electron tomography | |
| CN101540043B (en) | Analysis iteration fast frequency spectrum extrapolation method for single image restoration | |
| CN110852973B (en) | Nonlinear restoration method and system for pulse noise blurred image | |
| Chung et al. | Direct diffusion bridge using data consistency for inverse problems | |
| CN112116541A (en) | Based on the gradient l0Fuzzy image restoration method based on norm and total variation regularization constraint | |
| CN117078538B (en) | Correction method of remote atmospheric turbulence image based on pixel motion statistics | |
| CN113379647B (en) | Multi-feature image restoration method for optimizing PSF estimation | |
| Ke et al. | Unsupervised image restoration using partially linear denoisers | |
| Lin et al. | Non-blind optical degradation correction via frequency self-adaptive and finetune tactics | |
| Robinson et al. | Bias minimizing filter design for gradient-based image registration | |
| CN106991659A (en) | A kind of multi-frame self-adaption optical image restoration methods for adapting to atmospheric turbulance change | |
| CN113487491B (en) | Image restoration method based on sparsity and non-local mean self-similarity | |
| CN112797917B (en) | High-precision digital speckle interference phase quantitative measurement method | |
| Ouyang et al. | Cryo-electron microscope image denoising based on the geodesic distance | |
| CN105913394A (en) | Degraded image sequence-based blind image restoration method | |
| CN111986136A (en) | Fuzzy image sequence fusion restoration method based on Poisson probability model | |
| Li et al. | Real-World super-resolution under the guidance of optimal transport | |
| Wang et al. | Remote sensing image on-board restoration based on adaptive wiener filter | |
| Seo et al. | Video denoising using higher order optimal space-time adaptation | |
| Chen et al. | First-order primal–dual algorithm for image restoration corrupted by mixed Poisson–Gaussian noise |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |