CN115760933A - SAR image registration method, device, equipment and medium - Google Patents

SAR image registration method, device, equipment and medium Download PDF

Info

Publication number
CN115760933A
CN115760933A CN202211170396.5A CN202211170396A CN115760933A CN 115760933 A CN115760933 A CN 115760933A CN 202211170396 A CN202211170396 A CN 202211170396A CN 115760933 A CN115760933 A CN 115760933A
Authority
CN
China
Prior art keywords
image
sar image
historical
current
registration
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202211170396.5A
Other languages
Chinese (zh)
Inventor
缪炜星
向春芹
黎晓烨
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Avic Chengdu Uav System Co ltd
Original Assignee
Avic Chengdu Uav System Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Avic Chengdu Uav System Co ltd filed Critical Avic Chengdu Uav System Co ltd
Priority to CN202211170396.5A priority Critical patent/CN115760933A/en
Publication of CN115760933A publication Critical patent/CN115760933A/en
Pending legal-status Critical Current

Links

Images

Landscapes

  • Image Processing (AREA)

Abstract

The application discloses a method, a device, equipment and a medium for SAR image registration, which relate to the technical field of image processing and comprise the following steps: acquiring a current SAR image and a corresponding historical SAR image; calculating the overlapping rate of the current SAR image and the historical SAR image according to a preset rule, determining an overlapping area when the overlapping rate is not lower than a first preset threshold, and screening a first area image and a second area image from the current SAR image and the historical SAR image based on the overlapping area; registering the first area image by using a correlation matching method based on the second area image to obtain an initial registration image, and judging whether the corresponding correlation coefficient is greater than a second preset threshold value or not; and if so, carrying out blocking processing on the initial registration image and the second region image, and registering each block in the initial registration image by using a correlation matching method based on each block of the second region image to obtain a final registration image. The registration accuracy and speed of the SAR image can be improved.

Description

SAR image registration method, device, equipment and medium
Technical Field
The invention relates to the technical field of image processing, in particular to a method, a device, equipment and a medium for SAR image registration.
Background
The unmanned aerial vehicle aerial remote sensing platform has the characteristics of long endurance time, real-time image transmission, high-risk area detection, low cost, strong maneuverability and the like, thereby being a powerful supplement for satellite remote sensing and manned aerial remote sensing. Synthetic Aperture Radar (SAR) has the characteristics of all-weather operation in all days, has good detection and resolution capabilities for artificial targets and disguised targets, provides a more effective means for maintaining national security, and is widely applied to the fields of disaster monitoring, topographic mapping, national defense and the like and plays a great role. The image registration is widely applied to the technical fields of change detection, image fusion, matching navigation and the like, and the precision and timeliness of the image registration directly influence the precision and timeliness of subsequent applications such as change detection, image fusion and the like.
For an unmanned airborne SAR system, the method has the characteristics of multiple working modes, unfixed flight line, unfixed imaging visual angle, real-time processing of images and the like, and the corresponding image registration algorithm can be rapidly and automatically matched with SAR images with different resolutions, different azimuth angles, different incidence angles and different acting distances. There are two types of current SAR image registration methods. The first type is a method based on feature matching, and specifically includes point features, line features, region features and the like; the method based on feature matching is widely applied to the field of optical image matching, for example, SIFT based on point features and an improved method thereof have mature application in the fields of optical image retrieval, biological feature recognition and the like; compared with an optical image, the SAR image is microwave imaging, speckle noise is contained in the image, and SAR images formed by different azimuth angles and incidence angles have obvious difference in the aspects of geometric features and gray features, so that the consistency of the extracted point features is poor, and mismatching is easily caused. The second type is a method based on image gray scale information matching, specifically, related matching, mutual information matching and the like exist, the image registration method based on gray scale is suitable for the condition that the size of an image is small, and only translation or small rotation exists between two images. That is, the current SAR image registration technology is often only suitable for a certain type of image, and lacks wide applicability and timeliness.
In summary, how to improve the registration accuracy and timeliness of the SAR image is a problem to be solved at present.
Disclosure of Invention
In view of this, the present invention provides a method, an apparatus, a device and a medium for registering an SAR image, which can improve the registration accuracy and timeliness of the SAR image. The specific scheme is as follows:
in a first aspect, the application discloses a method for registering an SAR image, comprising:
acquiring a current SAR image sent by an unmanned aerial vehicle, and matching a corresponding historical SAR image from a historical image library based on geographic coordinate information of the current SAR image;
calculating the overlapping rate of the current SAR image and the historical SAR image according to a preset rule, and judging whether the overlapping rate is not lower than a first preset threshold value or not;
if so, determining an overlapping area, and respectively screening a first area image and a second area image from the current SAR image and the historical SAR image based on the overlapping area;
registering the first area image based on the second area image by using a correlation matching method to obtain an initial registration image corresponding to the current SAR image, and judging whether a corresponding correlation coefficient is larger than a second preset threshold value or not;
if the difference is larger than the preset threshold value, respectively carrying out blocking processing on the initial registration image and the historical SAR image, and registering each corresponding block in the initial registration image by using a relevant matching method based on each block in the historical SAR image to obtain a final registration image.
Optionally, before calculating the overlapping rate of the current SAR image and the historical SAR image according to a preset rule, the method further includes:
respectively reading and analyzing file headers of the current SAR image and the historical SAR image to obtain a first pixel scale of the current SAR image and a second pixel scale of the historical SAR image;
and judging whether the first pixel scale is consistent with the second pixel scale, if not, determining a smaller value of the first pixel scale and the second pixel scale, and performing down-sampling processing on the SAR image corresponding to the smaller value so as to ensure that the first pixel scale is consistent with the second pixel scale.
Optionally, the calculating the overlapping rate of the current SAR image and the historical SAR image according to a preset rule includes:
acquiring a first geographical range of the current SAR image and acquiring a second geographical range of the historical SAR image;
performing image stitching processing on the current SAR image and the historical SAR image based on the first geographical range and the second geographical range to obtain a stitched image, and determining a first geographical coordinate and a second geographical coordinate of the current SAR image and the historical SAR image in the stitched image;
calculating an overlap ratio of the current SAR image and the historical SAR image based on the first geographic coordinate and the second geographic coordinate.
Optionally, the obtaining a first geographical range of the current SAR image and obtaining a second geographical range of the historical SAR image include:
respectively acquiring the longitude and latitude coordinates of the upper left corner, the pixel scale and the image width and height of the current SAR image and the historical SAR image, and determining the longitude and latitude coordinates of the lower right corner based on the longitude and latitude coordinates of the upper left corner, the pixel scale and the image width and height;
and determining a first geographical range of the current SAR image and a second geographical range of the historical SAR image based on the longitude and latitude coordinates of the upper left corner and the longitude and latitude coordinates of the lower right corner.
Optionally, the registering the first region image based on the second region image by using a correlation matching method to obtain an initial registered image corresponding to the current SAR image includes:
based on the central point of the second area image, a rectangular data block with a first preset image size is cut out from the second area image and used as a reference image, and based on the central point of the first area image, a rectangular data block with a second preset image size is cut out from the first area image and used as a template image
The template graph is roamed on the reference graph, a first correlation coefficient of each roaming position is calculated, then the maximum correlation coefficient is screened out from the first correlation coefficients of all the roaming positions, and the roaming position corresponding to the maximum correlation coefficient is determined as the best matching position;
and determining the offset relationship between the current SAR image and the historical SAR image based on the optimal matching position, and performing image transformation on the current SAR image by using the offset relationship to obtain a corresponding initial registration image.
Optionally, the registering, based on each block in the historical SAR image, each corresponding block in the initial registered image by using a correlation matching method to obtain a final registered image includes:
determining each corresponding block in the initial registration image based on each block in the historical SAR image, and calculating the coordinate of the best matching point between the two blocks;
judging whether a second correlation number corresponding to the best matching point coordinate is not lower than a third preset threshold value or not, and if not, taking the best matching point coordinate as a control point coordinate of a corresponding block; if the value is lower than the preset value, taking the central point coordinate of the corresponding block as a control point coordinate;
and obtaining a first control point set based on the control point coordinates of all the blocks, determining a first affine transformation relation between the initial registration image and the historical SAR image according to the first control point set, and then performing image transformation on the initial registration image by using the first affine transformation relation to obtain a final registration image.
Optionally, after determining whether the corresponding correlation coefficient is greater than a second preset threshold, the method further includes:
if not, registering the current SAR image by utilizing an SIFT feature matching method based on the historical SAR image to obtain a final registered image; the process of registering the first region image based on the second region image by using a SIFT feature matching method to obtain a final registered image comprises the following steps: respectively carrying out filtering enhancement processing on the first area image and the second area image to obtain corresponding gradient images, obtaining a gradient amplitude diagram of the gradient images, and constructing a Harris scale space based on the gradient amplitude diagram; carrying out extreme point detection on Harris scale spaces under different scales, calculating a gradient histogram of each extreme point, and judging whether the corresponding extreme point is a key point or not based on the gradient histogram; if the image is the key point, generating a feature descriptor based on the key point, and performing key point matching by using a nearest neighbor distance ratio method based on the feature descriptor to screen a matching point set from the historical SAR image; and constructing a second control point set based on the matching point set, determining a second affine transformation relation between the current SAR image and the historical SAR image according to the second control point set, and then performing image transformation on the current SAR image by using the second affine transformation relation to obtain a final registration image.
In a second aspect, the application discloses a SAR image registration apparatus, comprising:
the SAR image acquisition module is used for acquiring a current SAR image sent by the unmanned aerial vehicle and matching a corresponding historical SAR image from a historical image library based on the geographic coordinate information of the current SAR image;
the overlapping rate calculation module is used for calculating the overlapping rate of the current SAR image and the historical SAR image according to a preset rule and judging whether the overlapping rate is not lower than a first preset threshold value or not;
the area image screening module is used for determining an overlapping area if the current SAR image and the historical SAR image are overlapped, and respectively screening a first area image and a second area image from the current SAR image and the historical SAR image based on the overlapping area;
an initial registration module, configured to register the first region image by using a correlation matching method based on the second region image to obtain an initial registration image corresponding to the current SAR image, and determine whether a corresponding correlation coefficient is greater than a second preset threshold;
and the final registration module is used for respectively carrying out blocking processing on the initial registration image and the historical SAR image if the number of blocks in the initial registration image is larger than the number of blocks in the historical SAR image, and registering each corresponding block in the initial registration image by using a correlation matching method based on each block in the historical SAR image to obtain a final registration image.
In a third aspect, the present application discloses an electronic device, comprising:
a memory for storing a computer program;
a processor for executing the computer program to implement the steps of the SAR image registration method disclosed in the foregoing.
In a fourth aspect, the present application discloses a computer readable storage medium for storing a computer program; wherein the computer program when executed by a processor implements the steps of the SAR image registration method disclosed in the foregoing.
Therefore, the method and the device for obtaining the SAR image match comprise the steps that the current SAR image sent by the unmanned aerial vehicle is obtained, and the corresponding historical SAR image is matched from a historical image library based on the geographic coordinate information of the current SAR image; calculating the overlapping rate of the current SAR image and the historical SAR image according to a preset rule, and judging whether the overlapping rate is not lower than a first preset threshold value or not; if so, determining an overlapping area, and respectively screening a first area image and a second area image from the current SAR image and the historical SAR image based on the overlapping area; registering the first area image by using a correlation matching method based on the second area image to obtain an initial registration image corresponding to the current SAR image, and judging whether a corresponding correlation coefficient is larger than a second preset threshold value or not; if the difference is larger than the preset threshold value, respectively carrying out blocking processing on the initial registration image and the historical SAR image, and registering each corresponding block in the initial registration image by using a relevant matching method based on each block in the historical SAR image to obtain a final registration image. Therefore, after the current SAR image sent by the unmanned aerial vehicle is obtained, the corresponding historical SAR image is obtained according to the geographic coordinate information, the overlapping rate of the current SAR image and the historical SAR image is calculated, the first region image and the second region image are respectively screened out from the current SAR image and the historical SAR image based on the overlapping region when the overlapping rate is not lower than a first preset threshold value, then the first region image and the second region image are registered by using a related matching method, the initial registered image of the current SAR image relative to the historical SAR image is obtained, whether the related coefficient at the moment is larger than a second preset threshold value is judged, if so, the initial registered image and the historical SAR image are respectively subjected to blocking processing, and each block of the initial registered image and each block of the historical SAR image are registered by using the related matching method again, so that the final registered image of the initial registered image relative to the historical SAR image is obtained. Therefore, the registration problem among the SAR images is converted into the registration problem among the overlapped areas, so that invalid characteristic points can be removed, and the timeliness and the registration precision of image registration are improved; in addition, the precision of image registration is improved through two matching processes, namely a matching strategy from rough matching to fine matching.
Drawings
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the provided drawings without creative efforts.
Fig. 1 is a flowchart of an SAR image registration method disclosed in the present application;
fig. 2 is a schematic diagram illustrating a specific SAR image registration process disclosed in the present application;
fig. 3 is a flowchart of a specific SAR image registration method disclosed in the present application;
fig. 4 is a flowchart of a specific SAR image registration method disclosed in the present application;
FIG. 5 (a) is a specific simulated SAR image disclosed herein;
FIG. 5 (b) is a ROEWA filtered gradient magnitude map generated according to the present disclosure;
FIG. 5 (c) is a ROEWA filtered gradient pattern as disclosed herein;
FIG. 6 is a current SAR image of a test data set I as disclosed herein;
FIG. 7 is a historical SAR image of a test data set I as disclosed herein;
FIG. 8 is a graph illustrating the results of image registration of a test data set I as disclosed herein;
FIG. 9 is a current SAR image of a test data set II disclosed herein;
FIG. 10 is a historical SAR image of a test data set II disclosed herein;
FIG. 11 is a graph illustrating the results of image registration for a test data set II disclosed herein;
fig. 12 is a schematic structural diagram of an SAR image registration apparatus disclosed in the present application;
fig. 13 is a block diagram of an electronic device disclosed in the present application.
Detailed Description
The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The current SAR image registration methods are of two types, the first type is a method based on feature matching, the method based on feature matching is widely applied in the field of optical image matching, but compared with an optical image, the SAR image is microwave imaging, speckle noise is contained in the image, and SAR images formed by different azimuth angles and incidence angles have obvious difference in the aspects of geometric features and gray features, so that the consistency of extracted point features is poor, and mismatching is easily caused; the second type is a method based on image gray information matching, and the image registration method based on gray is suitable for the situation that the size of an image is small and only translation or small rotation exists between two images. That is, the current SAR image registration technology is often only suitable for a certain type of image, and lacks wide applicability and timeliness. Therefore, the embodiment of the application discloses a method, a device, equipment and a medium for SAR image registration, which can improve the registration accuracy and timeliness of SAR images.
Referring to fig. 1 and fig. 2, an embodiment of the present application discloses a method for registering SAR images, including:
step S11: the method comprises the steps of obtaining a current SAR image sent by an unmanned aerial vehicle, and matching a corresponding historical SAR image from a historical image library based on geographic coordinate information of the current SAR image.
In this embodiment, when the unmanned aerial vehicle executes the reconnaissance task, the captured SAR image is downloaded. It should be noted that, obtaining a current SAR Image sent by an unmanned aerial vehicle is a geo TIFF (geo tag Image File Format) Format Image which is generated after system-level geometric correction, and geo TIFF is a very common remote sensing Image data Format, which is raster data, and the data is stored in a pixel matrix form. And the image contains geographic coordinate information, and the historical image is retrieved from the historical image library based on the geographic coordinate information and the time condition of the current SAR image so as to obtain the historical SAR image in the same area as the current SAR image, wherein the historical SAR image is also a GeoTiff format image, so that the matching of the two images is completed. Only translation and a tiny rotation angle exist between the two images after system-level geometric correction, so that the image matching range can be shortened and the matching precision can be improved by means of geographic information, and meanwhile, the matching speed is improved, and the real-time registration of the images is facilitated.
Step S12: and calculating the overlapping rate of the current SAR image and the historical SAR image according to a preset rule, and judging whether the overlapping rate is not lower than a first preset threshold value or not.
In this embodiment, the overlapping rate of the current SAR image and the historical SAR image needs to be calculated according to a preset rule, and whether the overlapping rate is not lower than a first preset threshold value is judged. Wherein the first preset threshold may be set to 30%.
It should be noted that, before calculating the overlapping rate of the current SAR image and the historical SAR image according to the preset rule, the method further includes: respectively reading and analyzing file headers of the current SAR image and the historical SAR image to obtain a first pixel scale of the current SAR image and a second pixel scale of the historical SAR image; and judging whether the first pixel scale is consistent with the second pixel scale, if not, determining a smaller value of the first pixel scale and the second pixel scale, and performing down-sampling processing on the SAR image corresponding to the smaller value so as to ensure that the first pixel scale is consistent with the second pixel scale. It can be understood that before calculating the overlapping rate of the current SAR image and the historical SAR image, it is further required to determine whether the pixel scales of the current SAR image and the historical SAR image are consistent, and in this embodiment, the consistency of the scales of the two images needs to be satisfied, so as to improve the success rate of matching the subsequent images. Specifically, analyzing pixel scale parameters in file headers of the current SAR image and the historical SAR image to obtain a first pixel scale of the current SAR image and a second pixel scale of the historical SAR image, wherein the pixel scales belong to one of attributes of a Geotiff file and mean pixel sizes represented by degrees; then judging whether the first pixel proportion scale is consistent with the second pixel proportion scale, if not, the proportion scale value is required to be smallerThe SAR image is subjected to down-sampling processing, so that the pixel proportion scales of the two down-sampled images are consistent. For example, the current SAR image and the historical SAR image are respectively recorded as M 1 And M 2 Suppose M 1 The first pixel scale is s 1 ＝0.0005，M 2 The second pixel scale is s 2 If =0.0003, M is required 2 Down-sampling is carried out, and the down-sampling scale factor is F = s 1 /s 2 Specifically, the image may be down-sampled by bilinear interpolation, and after down-sampling, M is obtained 1 And M 2 The method has the advantage of scale consistency, and is beneficial to subsequent image matching.
Step S13: if so, determining an overlapping area, and respectively screening a first area image and a second area image from the current SAR image and the historical SAR image based on the overlapping area.
In a specific embodiment, if the overlapping rate is not lower than a first preset threshold, rectangular pixel coordinates of the overlapping area in the current SAR image and the historical SAR image are determined, and a first area image and a second area image are screened out from the current SAR image and the historical SAR image according to the rectangular pixel coordinates.
In another specific embodiment, if the overlapping rate is lower than the first preset threshold, a prompt message that the image overlapping rate does not meet the registration requirement is given.
Step S14: and registering the first area image based on the second area image by using a correlation matching method to obtain an initial registration image corresponding to the current SAR image, and judging whether the corresponding correlation coefficient is larger than a second preset threshold value.
In this embodiment, the first region image and the second region image are registered by using a correlation matching method, so as to obtain an initial registration image of the current SAR image relative to the historical SAR image, where the initial registration image is a rough matching position of the current SAR image and the historical SAR image. And judging whether the correlation coefficient corresponding to the coarse matching position at the moment is greater than a second preset threshold, wherein the second preset threshold can be set to be 0.5.
Step S15: if the difference is larger than the preset threshold value, respectively carrying out blocking processing on the initial registration image and the historical SAR image, and registering each corresponding block in the initial registration image by using a relevant matching method based on each block in the historical SAR image to obtain a final registration image.
In this embodiment, if the correlation coefficient is greater than the second preset threshold, fine matching based on a correlation matching method is further performed on the data of the overlap region, that is, the initial registration image and the historical SAR image, to complete image registration. The method specifically comprises the steps of respectively carrying out block processing on an initial registration image and a historical SAR image, and registering each block of the initial registration image and each block of the historical SAR image by using a relevant matching method again to obtain a final registration image of the initial registration image relative to the historical SAR image.
Therefore, the method and the device for obtaining the SAR image match acquire the current SAR image sent by the unmanned aerial vehicle, and match out the corresponding historical SAR image from the historical image library based on the geographic coordinate information of the current SAR image; calculating the overlapping rate of the current SAR image and the historical SAR image according to a preset rule, and judging whether the overlapping rate is not lower than a first preset threshold value or not; if so, determining an overlapping area, and respectively screening a first area image and a second area image from the current SAR image and the historical SAR image based on the overlapping area; registering the first area image based on the second area image by using a correlation matching method to obtain an initial registration image corresponding to the current SAR image, and judging whether a corresponding correlation coefficient is larger than a second preset threshold value or not; if the difference is larger than the preset threshold value, respectively carrying out blocking processing on the initial registration image and the historical SAR image, and registering each corresponding block in the initial registration image by using a relevant matching method based on each block in the historical SAR image to obtain a final registration image. Therefore, after the current SAR image sent by the unmanned aerial vehicle is obtained, the corresponding historical SAR image is obtained according to the geographic coordinate information, the overlapping rate of the current SAR image and the historical SAR image is calculated, the first region image and the second region image are respectively screened out from the current SAR image and the historical SAR image based on the overlapping region when the overlapping rate is not lower than a first preset threshold value, then the first region image and the second region image are registered by using a related matching method, the initial registered image of the current SAR image relative to the historical SAR image is obtained, whether the related coefficient at the moment is larger than a second preset threshold value is judged, if so, the initial registered image and the historical SAR image are respectively subjected to blocking processing, and each block of the initial registered image and each block of the historical SAR image are registered by using the related matching method again, so that the final registered image of the initial registered image relative to the historical SAR image is obtained. Therefore, the registration problem among the SAR images is converted into the registration problem among the overlapped areas, so that invalid characteristic points can be removed, and the timeliness and the registration precision of image registration are improved; in addition, the precision of image registration is improved through two matching processes, namely a matching strategy from rough matching to fine matching.
Referring to fig. 3, the embodiment of the present application discloses a specific SAR image registration method, and with respect to the previous embodiment, the present embodiment further describes and optimizes the technical solution. The method specifically comprises the following steps:
step S21: the method comprises the steps of obtaining a current SAR image sent by the unmanned aerial vehicle, and matching a corresponding historical SAR image from a historical image library based on geographic coordinate information of the current SAR image.
Step S22: and calculating the overlapping rate of the current SAR image and the historical SAR image according to a preset rule, and judging whether the overlapping rate is not lower than a first preset threshold value or not.
Step S23: if so, determining an overlapping area, and respectively screening a first area image and a second area image from the current SAR image and the historical SAR image based on the overlapping area.
Step S24: and a rectangular data block with a first preset image size is cut out from the second area image based on the center point of the second area image and is used as a reference image, and a rectangular data block with a second preset image size is cut out from the first area image based on the center point of the first area image and is used as a template image.
In this embodiment, rectangular data blocks of a first preset image size and a second preset image size are respectively cut out from the second area image and the first area image as a reference map and a template map based on the center points of the second area image and the first area image. For example, the second area image may be denoted as I in this embodiment 2 Denote the first region image as I 1 Then with I 2 Taking the rectangular data block with the size of 384 multiplied by 384 as a matched reference map; with I 1 The center point of (2) is taken as the center point of the template map, and a rectangular data block with a size of 256 × 256 is taken as a matched template map.
Step S25: and roaming the template graph on the reference graph, calculating a first correlation coefficient of each roaming position, screening out the maximum correlation coefficient from the first correlation coefficients of all the roaming positions, and determining the roaming position corresponding to the maximum correlation coefficient as the best matching position.
In this embodiment, the template map is roamed on the reference map, the first correlation coefficient of each roaming position is calculated, then the maximum correlation coefficient is selected from the first correlation coefficients of all roaming positions, and the roaming position corresponding to the maximum correlation coefficient is determined as the best matching position. That is, the template map is roamed on the reference map and the correlation coefficient is calculated, and the position with the largest correlation number is taken as the best matching position, and the calculation method of the correlation coefficient is as follows:
Figure BDA0003859442130000111
wherein C (u, v) represents a correlation coefficient, I 1 (i, j) represents the pixel grayscale value of the template image at the (i, j) point,
Figure BDA0003859442130000112
representing the mean, I, of the template image 2 (i + u, j + v) represents the pixel gray scale value of the reference image at the point (i + u, j + v), (u, v) represents the roaming position of the template image in the reference image,
Figure BDA0003859442130000113
the average value of the reference image at the displacement (u, v) is shown, and M and N respectively show the number of vertical and horizontal pixel points in the template graph.
Step S26: determining the offset relationship between the current SAR image and the historical SAR image based on the optimal matching position, performing image transformation on the current SAR image by using the offset relationship to obtain a corresponding initial registration image, and judging whether the corresponding correlation coefficient is larger than a second preset threshold value or not.
In this embodiment, the offset relationship between the current SAR image and the historical SAR image is determined based on the optimal matching position, that is, the coordinate of the optimal matching position is converted into the offset coordinate of the current SAR image in the historical SAR image, and then the current SAR image is subjected to translational transformation according to the offset coordinate to obtain a transformed initial registration image, so that the coarse registration between the initial registration image and the historical SAR image is realized. In addition, whether the correlation coefficient corresponding to the best matching position is greater than a second preset threshold value of 0.5 or not needs to be judged, and the subsequent steps are executed according to the judgment result.
Step S27: and if the difference is larger than the preset threshold value, respectively carrying out blocking processing on the initial registration image and the historical SAR image, and registering each corresponding block in the initial registration image by using a relevant matching method based on each block in the historical SAR image to obtain a final registration image.
In this embodiment, the registering, based on each block in the historical SAR image, each corresponding block in the initial registration image by using a correlation matching method to obtain a final registration image includes: determining each corresponding block in the initial registration image based on each block in the historical SAR image, and calculating the coordinate of the best matching point between the two blocks; judging whether a second correlation number corresponding to the optimal matching point coordinate is not lower than a third preset threshold value or not, and if not, taking the optimal matching point coordinate as a control point coordinate of a corresponding block; if the value is lower than the preset value, taking the central point coordinate of the corresponding block as a control point coordinate; and obtaining a first control point set based on the control point coordinates of all the blocks, determining a first affine transformation relation between the initial registration image and the historical SAR image according to the first control point set, and then performing image transformation on the initial registration image by using the first affine transformation relation to obtain a final registration image. Respectively carrying out blocking processing on the initial registration image and the historical SAR image according to the obtained initial registration image and the historical SAR image so as to realize precise registration, wherein the size of each block can be 128 x 128 specifically, calculating the best matching point coordinates of the two blocks, and then judging whether the second phase relation number corresponding to the best matching point coordinates is not lower than a third preset threshold, wherein the third preset threshold can be set to be 0.7 specifically; if the coordinate of the best matching point is not lower than 0.7, the coordinate of the control point of the corresponding block is taken as the coordinate of the control point, and if the coordinate of the center point of the block is lower than 0.7, the coordinate of the control point is taken as the coordinate of the control point; and calculating the coordinates of the control points of all the blocks, and obtaining a first control point set of the initial registration image registered to the historical SAR image, wherein the number of the control points in the point set is equal to the number of the blocks. The expression for the ith control point is as follows:
GCP i ＝{(x i1 ,y i1 )；(x i2 ,y i2 )}；
wherein (x) i1 ,y i1 ) And (x) i2 ,y i2 ) The pixel coordinates of the ith control point in the initial registered image and the historical SRA image, respectively.
And then determining a first affine transformation relation, namely an affine transformation coefficient, between the initial registration image and the historical SAR image according to the first control point set, so that the initial registration image is subjected to affine transformation by using the first affine transformation relation to obtain a transformed image, namely a final registration image, which is registered to the historical SAR image.
For more specific processing procedures of the steps S21, S22 and S23, reference may be made to corresponding contents disclosed in the foregoing embodiments, and details are not repeated herein.
It can be seen that, when the first area image and the second area image are registered by using a correlation matching method to obtain an initial registration image corresponding to the current SAR image, the method specifically includes intercepting rectangular data blocks with a first preset image size and a second preset image size from the second area image and the first area image respectively based on center points of the second area image and the first area image as a reference image and a template image, roaming the template image on the reference image, calculating a first correlation coefficient of each roaming position, then screening out a maximum correlation coefficient from the first correlation coefficients of all the roaming positions, determining a roaming position corresponding to the maximum correlation coefficient as an optimal matching position, finally determining an offset coordinate between the current SAR image and a historical SAR image based on the optimal matching position, and then performing translation transformation on the current SAR image according to the offset coordinate to obtain a transformed initial registration image, thereby completing a coarse registration process of the images. And then, when the correlation coefficient corresponding to the optimal matching position is larger than a second preset threshold value, further performing fine registration by using a correlation matching method, specifically, performing block processing on the initial registration image and the historical SAR image respectively, determining the nearest matching point coordinate of each block, further determining the control point coordinate, determining a first affine transformation relation between the initial registration image and the historical SAR image by using a first control point set formed by the control point coordinates, and finally performing affine transformation on the initial registration image by using the first affine transformation relation to obtain a transformation image registered to the historical SAR image, namely a final registration image. Therefore, the precision of image registration is improved through two times of matching by firstly carrying out coarse matching on the overlapped part of data and then carrying out block fine matching on the result of the coarse matching.
Referring to fig. 4, the embodiment of the present application discloses a specific SAR image registration method, and with respect to the previous embodiment, the present embodiment further describes and optimizes the technical solution. The method specifically comprises the following steps:
step S31: the method comprises the steps of obtaining a current SAR image sent by an unmanned aerial vehicle, and matching a corresponding historical SAR image from a historical image library based on geographic coordinate information of the current SAR image.
Step S32: and acquiring a first geographical range of the current SAR image and acquiring a second geographical range of the historical SAR image.
In this embodiment, the obtaining the first geographical range of the current SAR image and the obtaining the second geographical range of the historical SAR image include: respectively acquiring the longitude and latitude coordinates of the upper left corner, the pixel scale and the image width and height of the current SAR image and the historical SAR image, and determining the longitude and latitude coordinates of the lower right corner based on the longitude and latitude coordinates of the upper left corner, the pixel scale and the image width and height; and determining a first geographical range of the current SAR image and a second geographical range of the historical SAR image based on the longitude and latitude coordinates of the upper left corner and the longitude and latitude coordinates of the lower right corner. That is, in the present embodiment, the longitude and latitude coordinates of the lower right corner are determined by obtaining the longitude and latitude coordinates of the upper left corner, the pixel scale and the image width and height of the current SAR image and the historical SAR image, and then the first geographic range of the current SAR image and the second geographic range of the historical SAR image are determined based on the longitude and latitude coordinates of the upper left corner and the longitude and latitude coordinates of the lower right corner. For example, with the current SAR image M 1 For example, analyze M 1 Longitude and latitude coordinate of upper left corner (L) 1 ,B 1 ) Pixel scale S m1 Width and height of image (W) m1 ,H m1 ) Calculate M 1 Longitude and latitude coordinate of lower right corner (L) m1 ,B m1 )：
Figure BDA0003859442130000141
To obtain M 1 First geographical range (L) min1 ,B max1 )、(L max1 ,B min1 )：
Figure BDA0003859442130000142
In the same way, obtaining a historical SAR image M 2 Longitude and latitude coordinate of upper left corner (L) 2 ,B 2 ) And longitude and latitude coordinates of lower right corner (L) m2 ,B m2 ) And a corresponding second geographic range.
Step S33: performing image stitching processing on the current SAR image and the historical SAR image based on the first geographic range and the second geographic range to obtain a stitched image, determining a first geographic coordinate and a second geographic coordinate of the current SAR image and the historical SAR image in the stitched image, and calculating the overlapping rate of the current SAR image and the historical SAR image based on the first geographic coordinate and the second geographic coordinate.
In this embodiment, the image stitching processing is performed on the current SAR image and the historical SAR image based on the first geographic range and the second geographic range to obtain a stitched image, that is, the geographic range (L) of the stitched image is obtained min ,B max )、(L max ,B min ) The concrete mode is as follows:
1) If L is m2 ＞L max1 Then L is max ＝L m2 ；
2) If B is 2 ＞B max1 Then B is max ＝B 2 ；
3) If L is 2 ＜L min1 Then L is min ＝L 2 ；
4) If B is m2 ＜B min1 Then B is min ＝B m2 。
Then, calculating a first geographic coordinate and a second geographic coordinate of the current SAR image and the historical SAR image in the spliced image, specifically, calculating an image M 1 First pixel coordinates of an upper left corner point and a lower right corner point in a stitched image, and calculating an image M 2 Second pixel coordinates of the upper left corner point and the lower right corner point in the spliced image, and the overlapping rate of the current SAR image and the historical SAR image and the overlapping area position in M are calculated based on the first geographic coordinate and the second geographic coordinate 1 And M 2 Rectangular frame pixel coordinates in (2).
Step S34: if so, determining an overlapping area, and respectively screening a first area image and a second area image from the current SAR image and the historical SAR image based on the overlapping area.
In this embodiment, the overlap region is located at M 1 And M 2 The rectangular frame pixel coordinates in (1)And intercepting overlapped area data for matching from the front SAR image and the historical SAR image to respectively obtain a first area image and a second area image.
Step S35: and registering the first area image based on the second area image by using a correlation matching method to obtain an initial registration image corresponding to the current SAR image, and judging whether the corresponding correlation coefficient is larger than a second preset threshold value.
Step S36: and if not, registering the current SAR image by utilizing an SIFT feature matching method based on the historical SAR image to obtain a final registered image.
In this embodiment, it can be understood that, although the two images are geographically corrected, theoretically only have translation, due to the existence of a positioning error, a small-angle rotation may exist between the current SAR image and the historical SAR image, that is, at this time, the corresponding correlation coefficient is lower than the second preset threshold value by 0.5, and then, the registration processing by using the correlation matching method does not have reliability.
The process of registering the first region image based on the second region image by using a Scale Invariant Feature Transform (SIFT) Feature Transform matching algorithm to obtain a final registered image includes: respectively carrying out filtering enhancement processing on the first area image and the second area image to obtain corresponding gradient images, obtaining a gradient amplitude diagram of the gradient images, and constructing a Harris scale space based on the gradient amplitude diagram; carrying out extreme point detection on Harris scale spaces under different scales, calculating a gradient histogram of each extreme point, and judging whether the corresponding extreme point is a key point or not based on the gradient histogram; if the image is the key point, generating a feature descriptor based on the key point, and performing key point matching by using a nearest neighbor distance ratio method based on the feature descriptor to screen a matching point set from the historical SAR image; and constructing a second control point set based on the matching point set, determining a second affine transformation relation between the current SAR image and the historical SAR image according to the second control point set, and then performing image transformation on the current SAR image by using the second affine transformation relation to obtain a final registration image.
That is, the overlapped region data, that is, the first region image and the second region image, is first subjected to filter enhancement, so that the influence of speckle noise in the SAR image is reduced. Specifically, an exponential Weighted average Ratio Operator (ROEWA) may be used to perform filtering processing on the image, so as to obtain an edge-enhanced gradient image, where an expression Of the ROEWA filtering operator in the horizontal and vertical directions is as follows:
Figure BDA0003859442130000161
Figure BDA0003859442130000162
in the formula, a i Representing scale coefficient with initial value of 2, a at different scales i+1 /a i β, β is constant, value is
Figure BDA0003859442130000163
x and y denote pixel coordinates of the image, K and J denote pixel coordinate indices within the filter window range, respectively, K denotes a filter window length, J denotes a filter window width, v denotes a vertical direction, and h denotes a horizontal direction.
Then, the amplitude and direction of the gradient image are calculated, and the specific formula is as follows:
Figure BDA0003859442130000164
Figure BDA0003859442130000165
in the formula (I), the compound is shown in the specification,
Figure BDA0003859442130000166
to representThe magnitude of the gradient image is determined,
Figure BDA0003859442130000167
which represents the direction of the gradient image,
Figure BDA0003859442130000168
represents the dimension a i The gradient value of the time in the horizontal direction,
Figure BDA0003859442130000169
represents the dimension a i Vertical gradient value of time.
Referring to fig. 5, fig. 5 is a graph of a simulated SAR image and a gradient amplitude map and a directional diagram generated after the simulated SAR image is subjected to ROEWA filtering.
And then constructing a Harris scale space according to the generated gradient magnitude graph, wherein compared with a DOG (DIFFERENCE OF GAUSS, namely a DIFFERENCE OF gaussians scale space) scale space in the original SIFT algorithm, the Harris scale space can better describe linear structures in the image, such as edges, corners and the like OF the target, so that the Harris scale space has better noise resistance. The expression for the construction of the Harris scale space is as follows, where the first expression identifies the scale parameter a for the Hessian matrix formed by the gradient magnitude data i The gaussian filter of (2) performs filtering processing:
Figure BDA00038594421300001610
R(a i )＝det(M(a i ))-d·tr(M(a i )) 2
in the formula (I), the compound is shown in the specification,
Figure BDA00038594421300001611
representing a scale parameter of a i A gaussian filter in the time of day, and,
Figure BDA00038594421300001612
is a weighting coefficient, representing a convolution, G h,ai Represents the dimension a i Horizontal gradient value of time, G v,ai Represents the dimension a i A vertical direction gradient value of time; det represents the value of the matrix corresponding determinant, and tr represents the trace of the matrix, namely the sum of each element on the main diagonal of the matrix; d is an empirical constant, typically 0.04.
Further, carrying out extreme point detection on the Harris scale space image under different scales, wherein the extreme point can meet two conditions: (1) greater than a predefined threshold; (2) values greater than 8 neighborhoods. After the initial extreme point is detected, the detected extreme point can be restrained through a threshold value and a non-maximum value, unstable characteristic points are deleted by utilizing stability measurement, and an extreme point set is determined. Calculating a gradient histogram for each extreme point, and judging whether the corresponding extreme point is a key point based on the gradient histogram, specifically, the abscissa of the gradient histogram is a quantized gradient direction, and the range of the original gradient direction value range [0-180 ]]After quantization, its range is reduced to [0-HIST _ BIN]Wherein HIST _ BIN takes the value of 18; the ordinate of the gradient histogram is the sum of all gradient amplitudes corresponding to the abscissa in a circular region centered on the extreme point, i.e. traversing all gradient amplitudes G in the circular region m,ai (i, j), hist [5 ] if its corresponding gradient direction, after quantization, falls within a BIN interval, e.g. BIN =5]＝Hist[5]+G m,ai (i, j); gaussian filtering is performed on the generated Hist to eliminate the influence of abnormal values, and the coefficient of the filter is
Figure BDA0003859442130000171
And aiming at the filtered Hist, obtaining a peak value of the Hist, judging whether a local maximum exists, if so, taking the extreme point as a key point, wherein the key point can be used for image matching, and the attribute information comprises: the pixel coordinates, the scale parameters, the gradient direction and the gradient amplitude of the extreme point.
Specifically, a feature descriptor may be generated for each keypoint, where GLOH (Gradient Location-Orientation Histogram) descriptor may be specifically used, and GLOH is a log-polar descriptor, and compared with a square SIFT feature descriptor with a block region number of 4 × 4, GLOH has more robust matching performance. Assuming that the radius ratio factor of three circular regions of GLOH is γ, the radii of the three circular regions are R = {3 γ,4.11 γ,12 γ }, respectively, the descriptor is divided into 8 sector directions from the first outer circle in the inner part, and the total of 17 regions is obtained by adding the inner circle, and the gradient histogram is quantized into 8 directions in each region, so as to obtain a descriptor of a feature vector with dimensions of 17 × 8= 136.
According to the generated feature descriptors, a Nearest Neighbor Distance Ratio (NNDR) method is adopted to match key points, and the NNDR matching mainly comprises two steps:
(1) Nearest neighbor searching: finding out a matched feature point descriptor by using the minimum Euclidean distance, wherein the corresponding formula is as follows:
b min ＝arg minD(a,b)
wherein a belongs to A, B belongs to B, A and B are respectively a feature point set of the current SAR image and the historical SAR image, a and B are respectively a certain feature point in A and B, D is Euclidean distance, B is min The feature points are feature points in the historical SAR image which meet the nearest neighbor condition.
(2) Nearest neighbor distance ratio: and when the ratio of the nearest neighbor distance to the next nearest neighbor distance is smaller than a certain threshold value, the current feature point is a preliminary matching point.
And finally, eliminating wrong matching points by adopting a Random Sample Consensus (RANSAC) algorithm to obtain a matching point set, namely a second control point set which can be used for image transformation, and determining a second affine transformation relation between the current SAR image and the historical SAR image by utilizing the second control point set so as to carry out image transformation on the current SAR image by utilizing the second affine transformation relation to obtain a final registration image.
For more specific processing procedures of the foregoing steps S31 and S35, reference may be made to corresponding contents disclosed in the foregoing embodiments, and details are not repeated here.
As can be seen, in the embodiment of the present application, the specific rule adopted to calculate the overlapping rate of the current SAR image and the historical SAR image is to respectively obtain a first geographic range of the current SAR image and a second geographic range of the historical SAR image, perform image stitching processing on the current SAR image and the historical SAR image, then calculate a first geographic coordinate and a second geographic coordinate of the current SAR image and the historical SAR image in the stitched image, and calculate the overlapping rate of the current SAR image and the historical SAR image based on the first geographic coordinate and the second geographic coordinate. By converting the overall matching problem of two large-size image data into the matching problem of small-size overlapped data, the acquisition of invalid feature points in non-overlapped areas can be automatically eliminated, the timeliness of image registration is remarkably improved, and the registration accuracy is improved. In addition, although the two images are only translated theoretically after being subjected to geographic correction, due to the existence of positioning errors, small-angle rotation may exist between the current SAR image and the historical SAR image, namely the corresponding correlation coefficient is lower than a second preset threshold value of 0.5, an improved SIFT feature matching method is adopted for image matching at the moment, if the feature matching is successful, a RANSAC algorithm is used for extracting control point pairs, affine transformation correction is carried out on the current SAR image, and the current SAR image is registered to the historical SAR image; and the SAR image is enhanced before the feature points are detected, the gradient and direction features of the enhanced image are obvious, the feature point detection and matching are facilitated, the detected features have good consistency, and the image registration precision is improved. That is, the matching method based on the gray information and the matching method based on the feature detection are combined, and the success rate of image registration is improved.
The image registration accuracy and the operation efficiency after the implementation of the invention are illustrated by real-flight SAR data acquired by a certain type of unmanned aerial vehicle.
1. Test data set I
The imaging area corresponding to the data set I is an urban image of a certain grade city in western China, the geometric relation between the two images is mainly translation and scaling after system-level geometric correction, and rotation of a tiny angle exists between the images due to the influence of positioning errors.
Current SAR image M 1 As shown in fig. 6, the types of the features in the image are mainly buildings, and the related parameters are as follows: (1) image size: 1024 × 1196; (2) imaging track angle: 18.2 degrees; (3) radar range: 35km; (4) Geotiff pixel size: 0.25m.
Historical SAR image M 2 As shown in fig. 7, the relevant parameters are as follows: (1) image size: 1024 × 1075; (2) imaging track angle: 202.9 degrees; (3) radar range: 45km; (4) Geotiff pixel size: 0.5m.
Referring to fig. 8, fig. 8 is a fused image of a current SAR image and a historical SAR image after registration, as can be seen from fig. 8, the edge junction of the two images has good consistency, which illustrates that the present invention has higher registration accuracy.
2. Test data set II
The imaging area corresponding to the data set II is suburban images of a certain grade city in the western part of China, the geometric relation between the two images is mainly translation after system-level geometric correction, and rotation of a tiny angle exists between the images due to the influence of positioning errors. Compared with the test data set I, the surface feature type of the data set II is single, and the surface feature scattering characteristics in the image are different due to the fact that the imaging azimuth angle difference is large.
The current SAR image is shown in fig. 9, the type of the ground object in the image is mainly a factory building, and the related parameters are as follows: (1) image size: 1024 × 1120; (2) imaging track angle: 9.8 degrees; (3) radar range: 40km; (4) Geotiff pixel size: 0.4m.
The historical SAR image is shown in fig. 10, and the relevant parameters are as follows: (1) image size: 1024 × 1074; (2) imaging track angle: 26.5 degrees; (3) radar range: 45km; (4) Geotiff pixel size: 0.5m.
Referring to fig. 11, fig. 11 is a fused image of a current image and a historical image after registration, which shows that the edge junction of the two images has good consistency, and this illustrates that the present invention has high registration accuracy.
In addition, in the embodiment of the application, a DELL7050 computer (CPU: i 7-7770.6GHz and 8G memory) is used, and the SAR image registration process is realized by C + + programming. Registering two image data of the test data set I, reading an output registration image from a file, wherein the time is 0.45 seconds; the same treatment was carried out for test data set II, with a time of 0.41 seconds; can meet the requirement of real-time processing, and shows that the invention has high timeliness.
Referring to fig. 12, an embodiment of the present application discloses a SAR image registration apparatus, including:
the SAR image acquisition module 11 is used for acquiring a current SAR image sent by an unmanned aerial vehicle and matching a corresponding historical SAR image from a historical image library based on geographic coordinate information of the current SAR image;
an overlap ratio calculation module 12, configured to calculate an overlap ratio between the current SAR image and the historical SAR image according to a preset rule, and determine whether the overlap ratio is not lower than a first preset threshold;
the area image screening module 13 is configured to determine an overlapping area if the current SAR image is a first area image, and screen a second area image from the current SAR image and the historical SAR image respectively based on the overlapping area;
an initial registration module 14, configured to register the first region image based on the second region image by using a correlation matching method to obtain an initial registration image corresponding to the current SAR image, and determine whether a corresponding correlation coefficient is greater than a second preset threshold;
and a final registration module 15, configured to perform blocking processing on the initial registration image and the historical SAR image respectively if the initial registration image and the historical SAR image are larger than the historical SAR image, and perform registration on each corresponding block in the initial registration image by using a correlation matching method based on each block in the historical SAR image to obtain a final registration image.
Therefore, the method and the device for obtaining the SAR image match acquire the current SAR image sent by the unmanned aerial vehicle, and match out the corresponding historical SAR image from the historical image library based on the geographic coordinate information of the current SAR image; calculating the overlapping rate of the current SAR image and the historical SAR image according to a preset rule, and judging whether the overlapping rate is not lower than a first preset threshold value or not; if so, determining an overlapping area, and respectively screening a first area image and a second area image from the current SAR image and the historical SAR image based on the overlapping area; registering the first area image based on the second area image by using a correlation matching method to obtain an initial registration image corresponding to the current SAR image, and judging whether a corresponding correlation coefficient is larger than a second preset threshold value or not; if the difference is larger than the preset threshold value, respectively carrying out blocking processing on the initial registration image and the historical SAR image, and registering each corresponding block in the initial registration image by using a relevant matching method based on each block in the historical SAR image to obtain a final registration image. Therefore, after the current SAR image sent by the unmanned aerial vehicle is obtained, the corresponding historical SAR image is obtained according to the geographic coordinate information, the overlapping rate of the current SAR image and the historical SAR image is calculated, the first region image and the second region image are respectively screened out from the current SAR image and the historical SAR image based on the overlapping region when the overlapping rate is not lower than a first preset threshold value, then the first region image and the second region image are registered by using a correlation matching method, the initial registered image of the current SAR image relative to the historical SAR image is obtained, whether the correlation coefficient at the moment is larger than a second preset threshold value is judged, if the correlation coefficient is larger than the second preset threshold value, the SAR image and the historical image are further subjected to blocking processing respectively, and each block of the initial registered image and each block of the historical SAR image are registered by using the correlation matching method again, and the final registered image of the initial registered image relative to the historical SAR image is obtained. Therefore, the registration problem among the SAR images is converted into the registration problem among the overlapped areas, so that invalid characteristic points can be eliminated, and the timeliness and the registration accuracy of image registration are improved; in addition, the precision of image registration is improved through two matching processes, namely a matching strategy from rough matching to fine matching.
Fig. 13 is a schematic structural diagram of an electronic device according to an embodiment of the present application. The method specifically comprises the following steps: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input output interface 25, and a communication bus 26. Wherein the memory 22 is used for storing a computer program, which is loaded and executed by the processor 21 to implement the relevant steps in the SAR image registration method performed by an electronic device disclosed in any of the foregoing embodiments.
In this embodiment, the power supply 23 is configured to provide an operating voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and an external device, and a communication protocol followed by the communication interface is any communication protocol applicable to the technical solution of the present application, and is not specifically limited herein; the input/output interface 25 is configured to acquire external input data or output data to the outside, and a specific interface type thereof may be selected according to specific application requirements, which is not specifically limited herein.
The processor 21 may include one or more processing cores, such as a 4-core processor, an 8-core processor, and the like. The processor 21 may be implemented in at least one hardware form of a DSP (Digital Signal Processing), an FPGA (Field-Programmable Gate Array), and a PLA (Programmable Logic Array). The processor 21 may also include a main processor and a coprocessor, where the main processor is a processor for Processing data in an awake state, and is also called a Central Processing Unit (CPU); a coprocessor is a low power processor for processing data in a standby state. In some embodiments, the processor 21 may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content required to be displayed on the display screen. In some embodiments, the processor 21 may further include an AI (Artificial Intelligence) processor for processing a calculation operation related to machine learning.
In addition, the storage 22 is used as a carrier for storing resources, and may be a read-only memory, a random access memory, a magnetic disk or an optical disk, etc., the resources stored thereon include an operating system 221, a computer program 222, data 223, etc., and the storage may be a transient storage or a permanent storage.
The operating system 221 is used for managing and controlling each hardware device on the electronic device 20 and the computer program 222, so as to implement the operation and processing of the mass data 223 in the memory 22 by the processor 21, which may be Windows, unix, linux, or the like. The computer program 222 may further include a computer program that can be used to perform other specific tasks in addition to the computer program that can be used to perform the SAR image registration method disclosed in any of the foregoing embodiments and executed by the electronic device 20. The data 223 may include data received by the electronic device and transmitted from an external device, or may include data collected by the input/output interface 25 itself.
Further, the present application also discloses a computer-readable storage medium, in which a computer program is stored, and when the computer program is loaded and executed by a processor, the method steps executed in the SAR image registration process disclosed in any of the foregoing embodiments are implemented.
In the present specification, the embodiments are described in a progressive manner, and each embodiment focuses on differences from other embodiments, and the same or similar parts between the embodiments are referred to each other. The device disclosed by the embodiment corresponds to the method disclosed by the embodiment, so that the description is simple, and the relevant points can be referred to the method part for description.
Those of skill would further appreciate that the various illustrative components and method steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or combinations of both, and that the components and steps of the various examples have been described above generally in terms of their functionality in order to clearly illustrate this interchangeability of hardware and software. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the technical solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.
The methods or steps of the methods described in connection with the embodiments disclosed herein may be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. A software module may reside in Random Access Memory (RAM), memory, read Only Memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.
Finally, it should also be noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrases "comprising a," "8230," "8230," or "comprising" does not exclude the presence of additional like elements in a process, method, article, or apparatus that comprises the element.
The method, the apparatus, the device and the storage medium for registering an SAR image provided by the present invention are described in detail above, and specific examples are applied herein to explain the principle and the implementation of the present invention, and the description of the above embodiments is only used to help understanding the method and the core idea of the present invention; meanwhile, for a person skilled in the art, according to the idea of the present invention, there may be variations in the specific embodiments and the application scope, and in summary, the content of the present specification should not be construed as a limitation to the present invention.

Claims (10)

1. A SAR image registration method is characterized by comprising the following steps:
acquiring a current SAR image sent by an unmanned aerial vehicle, and matching a corresponding historical SAR image from a historical image library based on geographic coordinate information of the current SAR image;
calculating the overlapping rate of the current SAR image and the historical SAR image according to a preset rule, and judging whether the overlapping rate is not lower than a first preset threshold value or not;
if so, determining an overlapping area, and respectively screening a first area image and a second area image from the current SAR image and the historical SAR image based on the overlapping area;
registering the first area image by using a correlation matching method based on the second area image to obtain an initial registration image corresponding to the current SAR image, and judging whether a corresponding correlation coefficient is larger than a second preset threshold value or not;
if the difference is larger than the preset threshold value, respectively carrying out blocking processing on the initial registration image and the historical SAR image, and registering each corresponding block in the initial registration image by using a relevant matching method based on each block in the historical SAR image to obtain a final registration image.
2. The SAR image registration method according to claim 1, wherein before calculating the overlapping rate of the current SAR image and the historical SAR image according to a preset rule, the method further comprises:
respectively reading and analyzing file headers of the current SAR image and the historical SAR image to obtain a first pixel scale of the current SAR image and a second pixel scale of the historical SAR image;
and judging whether the first pixel proportion scale is consistent with the second pixel proportion scale, if not, determining a smaller value of the first pixel proportion scale and the second pixel proportion scale, and performing down-sampling processing on the SAR image corresponding to the smaller value so as to ensure that the first pixel proportion scale is consistent with the second pixel proportion scale.
3. The SAR image registration method according to claim 1, wherein the calculating the overlapping ratio of the current SAR image and the historical SAR image according to a preset rule comprises:
acquiring a first geographical range of the current SAR image and acquiring a second geographical range of the historical SAR image;
performing image stitching processing on the current SAR image and the historical SAR image based on the first geographical range and the second geographical range to obtain a stitched image, and determining a first geographical coordinate and a second geographical coordinate of the current SAR image and the historical SAR image in the stitched image;
calculating an overlap ratio of the current SAR image and the historical SAR image based on the first geographic coordinates and the second geographic coordinates.
4. The SAR image registration method of claim 3, wherein the obtaining a first geographic scope of the current SAR image and obtaining a second geographic scope of the historical SAR image comprises:
respectively acquiring the longitude and latitude coordinates of the upper left corner, the pixel scale and the image width and height of the current SAR image and the historical SAR image, and determining the longitude and latitude coordinates of the lower right corner based on the longitude and latitude coordinates of the upper left corner, the pixel scale and the image width and height;
and determining a first geographical range of the current SAR image and a second geographical range of the historical SAR image based on the longitude and latitude coordinates of the upper left corner and the longitude and latitude coordinates of the lower right corner.
5. The SAR image registration method according to claim 1, wherein the registering the first region image based on the second region image by using a correlation matching method to obtain an initial registration image corresponding to the current SAR image comprises:
intercepting a rectangular data block with a first preset image size from the second area image based on the central point of the second area image to be used as a reference image, intercepting a rectangular data block with a second preset image size from the first area image based on the central point of the first area image to be used as a template image, roaming the template image on the reference image, calculating a first correlation coefficient of each roaming position, screening out a maximum correlation coefficient from the first correlation coefficients of all the roaming positions, and determining the roaming position corresponding to the maximum correlation coefficient as a best matching position;
and determining the offset relationship between the current SAR image and the historical SAR image based on the optimal matching position, and performing image transformation on the current SAR image by using the offset relationship to obtain a corresponding initial registration image.
6. The SAR image registration method according to claim 5, wherein the registering each corresponding block in the initial registration image based on each block in the historical SAR image by using a correlation matching method to obtain a final registration image comprises:
determining each corresponding block in the initial registration image based on each block in the historical SAR image, and calculating the coordinate of the best matching point between the two blocks;
judging whether a second correlation number corresponding to the optimal matching point coordinate is not lower than a third preset threshold value or not, and if not, taking the optimal matching point coordinate as a control point coordinate of a corresponding block; if the coordinate is lower than the preset value, the coordinate of the central point of the corresponding block is used as the coordinate of the control point;
and obtaining a first control point set based on the control point coordinates of all the blocks, determining a first affine transformation relation between the initial registration image and the historical SAR image according to the first control point set, and then performing image transformation on the initial registration image by using the first affine transformation relation to obtain a final registration image.
7. The SAR image registration method according to claim 1, wherein after determining whether the corresponding correlation coefficient is greater than a second preset threshold, the method further comprises:
if not, registering the current SAR image by utilizing an SIFT feature matching method based on the historical SAR image to obtain a final registered image; the process of registering the first region image based on the second region image by using an SIFT feature matching method to obtain a final registered image comprises the following steps: respectively carrying out filtering enhancement processing on the first area image and the second area image to obtain corresponding gradient images, obtaining a gradient amplitude diagram of the gradient images, and constructing a Harris scale space based on the gradient amplitude diagram; carrying out extreme point detection on Harris scale spaces under different scales, calculating a gradient histogram of each extreme point, and judging whether the corresponding extreme point is a key point or not based on the gradient histogram; if the feature descriptors are the key points, generating feature descriptors based on the key points, and performing key point matching by using a nearest neighbor distance ratio method based on the feature descriptors to screen a matching point set from the historical SAR image; and constructing a second control point set based on the matching point set, determining a second affine transformation relation between the current SAR image and the historical SAR image according to the second control point set, and then performing image transformation on the current SAR image by using the second affine transformation relation to obtain a final registration image.
8. A SAR image registration apparatus, comprising:
the SAR image acquisition module is used for acquiring a current SAR image sent by the unmanned aerial vehicle and matching a corresponding historical SAR image from a historical image library based on the geographic coordinate information of the current SAR image;
the overlapping rate calculation module is used for calculating the overlapping rate of the current SAR image and the historical SAR image according to a preset rule and judging whether the overlapping rate is not lower than a first preset threshold value or not;
the area image screening module is used for determining an overlapping area if the current SAR image and the historical SAR image are overlapped, and respectively screening a first area image and a second area image from the current SAR image and the historical SAR image based on the overlapping area;
an initial registration module, configured to register the first region image by using a correlation matching method based on the second region image to obtain an initial registration image corresponding to the current SAR image, and determine whether a corresponding correlation coefficient is greater than a second preset threshold;
and the final registration module is used for respectively carrying out blocking processing on the initial registration image and the historical SAR image if the initial registration image and the historical SAR image are larger than the historical SAR image, and registering each corresponding block in the initial registration image by using a relevant matching method based on each block in the historical SAR image to obtain a final registration image.
9. An electronic device, comprising:
a memory for storing a computer program;
a processor for executing the computer program for implementing the steps of the SAR image registration method according to any of claims 1 to 7.
10. A computer-readable storage medium for storing a computer program; wherein the computer program realizes the steps of the SAR image registration method according to any one of claims 1 to 7 when executed by a processor.
CN202211170396.5A 2022-09-22 2022-09-22 SAR image registration method, device, equipment and medium Pending CN115760933A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202211170396.5A CN115760933A (en) 2022-09-22 2022-09-22 SAR image registration method, device, equipment and medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202211170396.5A CN115760933A (en) 2022-09-22 2022-09-22 SAR image registration method, device, equipment and medium

Publications (1)

Publication Number Publication Date
CN115760933A true CN115760933A (en) 2023-03-07

Family

ID=85351925

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202211170396.5A Pending CN115760933A (en) 2022-09-22 2022-09-22 SAR image registration method, device, equipment and medium

Country Status (1)

Country Link
CN (1) CN115760933A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117541469A (en) * 2024-01-10 2024-02-09 中山大学 SAR image stitching method and device based on graph theory

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117541469A (en) * 2024-01-10 2024-02-09 中山大学 SAR image stitching method and device based on graph theory
CN117541469B (en) * 2024-01-10 2024-05-10 中山大学 SAR image stitching method and device based on graph theory

Similar Documents

Publication Publication Date Title
Palenichka et al. Automatic extraction of control points for the registration of optical satellite and LiDAR images
US11042998B2 (en) Synthetic image generation from 3D-point cloud
CN107341781B (en) SAR image correction method based on improved phase consistency feature vector base map matching
CN102959946A (en) Augmenting image data based on related 3d point cloud data
WO2022067647A1 (en) Method and apparatus for determining pavement elements
CN115187798A (en) Multi-unmanned aerial vehicle high-precision matching positioning method
CN108564532B (en) Large-scale ground distance satellite-borne SAR image mosaic method
Gonçalves et al. CHAIR: Automatic image registration based on correlation and Hough transform
CN112381062A (en) Target detection method and device based on convolutional neural network
CN115760933A (en) SAR image registration method, device, equipment and medium
CN116228539A (en) Unmanned aerial vehicle remote sensing image stitching method
WO2021067252A1 (en) Image targeting via targetable 3d data
CN116907509A (en) AUV underwater auxiliary navigation method, system, equipment and medium based on image matching
CN113221883B (en) Unmanned aerial vehicle flight navigation route real-time correction method
CN116125411A (en) Method for realizing horizontal installation angle detection compensation based on freespace road end 4D millimeter wave radar
CN113516059B (en) Solid waste identification method and device, electronic device and storage medium
Zhang et al. Generation of digital surface model from high resolution satellite imagery
CN110223250B (en) SAR geometric correction method based on homography transformation
CN114879180A (en) Seamless situation perception method for real-time fusion of unmanned ship-borne multi-element multi-scale radar
CN117934518B (en) Remote sensing image segmentation method and system for ocean homeland space detailed planning
Toriya et al. Adaptive Image Scaling for Corresponding Points Matching between Images with Differing Spatial Resolutions
Nevins Georeferencing unmanned aerial systems imagery via registration with geobrowser reference imagery
CN117761695B (en) Multi-angle SAR three-dimensional imaging method based on self-adaptive partition SIFT
CN116704037B (en) Satellite lock-losing repositioning method and system based on image processing technology
Ding et al. High-Precision Geometric Positioning of Optical Satellite Images Assisted by LiDAR Data

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination