CN117109519A - Satellite linear array image stitching method and system assisted by laser altimetry data - Google Patents
Satellite linear array image stitching method and system assisted by laser altimetry data Download PDFInfo
- Publication number
- CN117109519A CN117109519A CN202311079334.8A CN202311079334A CN117109519A CN 117109519 A CN117109519 A CN 117109519A CN 202311079334 A CN202311079334 A CN 202311079334A CN 117109519 A CN117109519 A CN 117109519A
- Authority
- CN
- China
- Prior art keywords
- image
- ccd
- laser
- imaging model
- coordinates
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 36
- 238000003384 imaging method Methods 0.000 claims abstract description 132
- 238000005259 measurement Methods 0.000 claims abstract description 28
- 238000012937 correction Methods 0.000 claims description 31
- 239000011159 matrix material Substances 0.000 claims description 30
- 238000013507 mapping Methods 0.000 claims description 9
- 239000000523 sample Substances 0.000 claims description 9
- 238000010276 construction Methods 0.000 claims description 6
- 230000009466 transformation Effects 0.000 claims description 6
- 239000013598 vector Substances 0.000 claims description 3
- 230000009286 beneficial effect Effects 0.000 description 4
- 238000010586 diagram Methods 0.000 description 3
- 230000008569 process Effects 0.000 description 3
- 238000012545 processing Methods 0.000 description 2
- 230000000452 restraining effect Effects 0.000 description 2
- 230000000007 visual effect Effects 0.000 description 2
- PXFBZOLANLWPMH-UHFFFAOYSA-N 16-Epiaffinine Natural products C1C(C2=CC=CC=C2N2)=C2C(=O)CC2C(=CC)CN(C)C1C2CO PXFBZOLANLWPMH-UHFFFAOYSA-N 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 239000012634 fragment Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000001681 protective effect Effects 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 238000013519 translation Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
- G01C3/00—Measuring distances in line of sight; Optical rangefinders
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S19/00—Satellite radio beacon positioning systems; Determining position, velocity or attitude using signals transmitted by such systems
- G01S19/38—Determining a navigation solution using signals transmitted by a satellite radio beacon positioning system
- G01S19/39—Determining a navigation solution using signals transmitted by a satellite radio beacon positioning system the satellite radio beacon positioning system transmitting time-stamped messages, e.g. GPS [Global Positioning System], GLONASS [Global Orbiting Navigation Satellite System] or GALILEO
- G01S19/42—Determining position
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
- G06T3/4038—Image mosaicing, e.g. composing plane images from plane sub-images
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N1/00—Scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission; Details thereof
- H04N1/387—Composing, repositioning or otherwise geometrically modifying originals
- H04N1/3876—Recombination of partial images to recreate the original image
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/95—Computational photography systems, e.g. light-field imaging systems
- H04N23/951—Computational photography systems, e.g. light-field imaging systems by using two or more images to influence resolution, frame rate or aspect ratio
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2200/00—Indexing scheme for image data processing or generation, in general
- G06T2200/32—Indexing scheme for image data processing or generation, in general involving image mosaicing
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Radar, Positioning & Navigation (AREA)
- Remote Sensing (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Electromagnetism (AREA)
- Computer Networks & Wireless Communication (AREA)
- Computing Systems (AREA)
- Image Processing (AREA)
Abstract
The embodiment of the disclosure discloses a satellite linear array image splicing method and system assisted by laser altimetry data, and relates to the technical field of image splicing. The method comprises the following steps: constructing a strict geometric imaging model of the CCD; acquiring the point of connection of the CCD image overlapping area and the image point coordinates of laser height measurement data on the CCD image; external parameters of a strict geometric imaging model of the real CCD are calculated by combining the ranging values of the laser altimeter points, and the updated strict geometric imaging model of the real CCD is obtained; constructing a virtual CCD strict geometric imaging model according to information such as internal parameters of the real CCD strict geometric imaging model; and splicing the linear satellite images according to the virtual CCD strict geometric imaging model, the real CCD strict geometric imaging model and the elevation data. The method is simple, and the geometric positioning accuracy of the spliced image is improved by means of the ranging value of the laser altimetry data.
Description
Technical Field
The embodiment of the disclosure relates to the technical field of satellite image stitching, in particular to a satellite linear array image stitching method and system assisted by laser altimetry data.
Background
Currently, a high-resolution linear array push-broom satellite is designed by adopting a delay-integration Charge-Coupled Device (TDI-CCD) Device, so as to obtain a high-quality satellite image, and a plurality of Charge-Coupled devices (CCDs) are generally arranged in a staggered manner on a focal plane in order to increase the breadth. In actual use, images acquired by a plurality of CCDs are spliced and provided for users. At present, there are two main image splicing modes: image stitching based on image space and image stitching based on object space. Image stitching based on image side is mainly to match same name points in an image overlapping area so as to obtain translation or affine relation between images, and then stitching between the images is completed. The method is seriously dependent on the image matching precision, the splicing precision is low, meanwhile, the stability of a satellite platform is not considered, and distortion existing in the image can not be well eliminated, so that the subsequent application is influenced. Image stitching based on the object space is mainly based on an image tight imaging geometric model, and the corresponding relation construction of each CCD image is completed by constructing a unified image imaging model, so that the high-precision stitching of the images is finally realized. The method can ensure the geometric seamless of the image and eliminate the internal distortion of the image, but has no other auxiliary data, and the geometric positioning precision of the spliced image depends on satellite measurement data, so that the final geometric precision is not ideal.
Disclosure of Invention
The embodiment of the disclosure aims to provide a satellite linear array image splicing method and system assisted by laser altimetry data, so as to solve the problems in the prior art.
In order to achieve the above objective, the technical solution adopted in the embodiments of the present disclosure is as follows:
in one aspect, an embodiment of the present disclosure provides a satellite linear array image stitching method assisted by laser altimetry data, where the method includes:
and constructing a strict geometric imaging model of each slice of real CCD of the camera.
And (3) carrying out connection point matching in the overlapping area of the CCD images of each slice to obtain a certain number of connection points which are uniformly distributed.
And acquiring laser altimeter data in an imaging time range of the image to obtain image point coordinates of the laser altimeter data on the CCD image.
And according to the point coordinates and the object coordinates of the connecting point, the point coordinates, the object coordinates and the ranging value of the laser altimeter, calculating the external parameters of the strict geometric imaging model of the real CCD, and updating the strict geometric imaging model of the real CCD.
And according to the internal parameters of the real CCD strict geometric imaging model, performing straight line fitting on a camera focal plane to construct a virtual CCD, determining the internal parameters of the strict geometric imaging model of each probe element of the virtual CCD, and constructing a virtual CCD strict geometric imaging model by combining the camera posture, the orbit measurement value and the calculated external parameters of the real CCD strict geometric imaging model.
And splicing the linear satellite images according to the virtual CCD strict geometric imaging model, the real CCD strict geometric imaging model and the elevation data.
Optionally, the calculating the external parameters of the real CCD strict geometric imaging model according to the coordinates of the image point and the coordinates of the object space of the connecting point, and the coordinates of the image point, the coordinates of the object space and the ranging value of the laser altimeter, and updating the real CCD strict geometric imaging model includes:
and respectively constructing an error equation for the external parameters of the strict geometric imaging model of the real CCD, the connecting points and the object space coordinates of the laser altimeter.
And constructing a distance error equation for the laser altimetric points according to the distance measurement information of the laser altimetric points.
And (3) performing methodological on the error equation constructed in the first two steps, and solving external parameters and connection points of a strict geometric imaging model of the real CCD and the object space coordinate correction of the laser altimeter by using a least square method.
And updating external parameters and connection points of the geometric imaging model and the object space coordinates of the laser altimeter according to the correction, and repeating the steps until the iteration is finished to obtain the final real geometric imaging model of the CCD.
Optionally, the rigorous geometric imaging model of the CCD is:
wherein, (X, Y, Z) T Is the object coordinate of the image point under the ground object coordinate system, (X) S ,Y S ,Z S ) T Is the object space coordinate of the satellite body coordinate system center in the ground object space coordinate system, m is the imaging proportionality coefficient,is the transformation matrix from the satellite body coordinate system to the J2000 inertial coordinate system, < >>Is a transformation matrix from a J2000 inertial coordinate system to an object coordinate system; (psi) x ,ψ y ) Is the pointing angle of the image probe element, is the internal parameter of the image imaging strict geometric model, and represents the internal accuracy of the image, R u The matrix is formed by external parameters of an image imaging tight geometric model, and represents the external precision of the image;
rigorous geometric imaging model external parameter matrix R u Determined by the formula (2), the specific form is as follows:
wherein,omega, kappa are rotation angles around the Y, X, Z axes, respectively;
an error equation is respectively constructed for external parameters, connecting points and laser altimeter object space coordinates in a real imaging CCD rigorous geometric model, and a formula (3) is obtained:
writing formula (3) into a matrix form is:
V 1 ＝At+B 1 x-L 1 ,P 1 (4)
wherein V is 1 ＝(v x v y ) T Is the projection error of the observed value,is the external parameter correction of the image tight geometric imaging model, x= (delta X delta Y delta Z) T Is the correction of the ground object space coordinates of the laser height measurement point or connecting point, A, B 1 For the corresponding matrix of correction coefficients, L 1 And P 1 The constant and weight matrix are calculated for the initial values, respectively.
Optionally, a ranging equation of the ranging information of the laser altimeter is:
wherein (X, Y ', Z') T Is the object coordinate of the laser height measurement point under the ground object coordinate system, (X) S ,Y S ,Z S ) T The system is an object space coordinate of a satellite body coordinate system center in a ground object space coordinate system, and ρ is an accurate ranging value of a laser altimeter;
and constructing a distance error equation (6) for the laser altimetric points according to the distance measurement information of the laser altimetric points:
writing formula (6) into a matrix form is:
V 2 ＝B 2 x-L 2 ,P 2 (7)
wherein V is 2 ＝v las Is the laser altimeter distance error value, x= (Δx Δy Δz) T Is the correction of the laser height measurement point ground object space coordinate, B 2 For the corresponding matrix of correction coefficients, L 2 And P 2 The constant and weight matrix are calculated for the initial values, respectively.
Optionally, the error equation constructed in the previous two steps is normalized, and before the external parameters and the connection points of the strict geometric imaging model of the real CCD and the object space coordinate correction of the laser altimeter are solved by using a least square method, the connection points, the laser altimeter error equation (4) and the distance error equation (7) of the laser altimeter are combined to construct an integral error equation:
V＝At+Bx-L,P (8)
wherein V= (V) 1 V 2 ) T And representing the residual vectors of the observed values, wherein A and B are corresponding coefficient matrixes of the corrections, L and P are initial value calculation constants and weight matrixes respectively, and x is the coordinate correction of the object space of the connecting point and the laser altimeter.
Optionally, the acquiring laser altimeter data in the imaging time range of the image to obtain an image point coordinate of the laser altimeter data on the CCD image includes:
substituting the object space coordinates of the laser altimeter into the constructed strict geometric model of the real CCD to obtain the approximate image point coordinates of the laser altimeter on each segmented CCD image.
And performing fixed point matching on the obtained rough image point coordinates of the laser altimeter points on the CCD image and the laser altimeter point coordinates in the footprint image within the size range of the footprint to obtain the accurate image point coordinates of the laser altimeter points on the CCD image.
Optionally, the acquiring laser altimeter data in the imaging time range of the image to obtain an image point coordinate of the laser altimeter data on the CCD image further includes:
if the fixed point matching fails, the same name point matching is carried out on the footprint image and the CCD image within the footprint size range, the mapping relation between the CCD image and the footprint image is established, and the image point coordinates of the laser height measurement points on the footprint image are substituted, so that the accurate image point coordinates of the laser height measurement points on the CCD image are obtained.
Optionally, the mapping relation between the CCD image and the footprint image is:
wherein (x ', y') is the image point coordinates of the laser altimeter point on each segmented CCD image, and (x, y) is the image point coordinates of the laser point footprint image, and (a) 0 ,a 1 ,a 2 ,b 0 ,b 1 ,b 2 ) The mapping relation coefficients of the image point coordinates of each segmented CCD image and the footprint image are respectively.
Optionally, the stitching the satellite array image according to the virtual CCD strict geometric imaging model, the real CCD strict geometric imaging model and the elevation data includes:
determining the position of each pixel on the virtual CCD image on each piece of real CCD image, interpolating the gray values of pixels around the position to the virtual CCD image, repeating the interpolation until the gray values of all the virtual CCD images are obtained, and completing the splicing.
Another aspect of the disclosed embodiments provides a laser altimetry data assisted satellite linear array image stitching system, the system comprising:
the first construction module is used for constructing a strict geometric imaging model of each slice of real CCD of the camera.
And the matching module is used for carrying out connection point matching in each segmented CCD image overlapping area to obtain a certain number of connection points which are uniformly distributed.
The acquisition module is used for acquiring laser altimeter data in the imaging time range of the image and obtaining image point coordinates of the laser altimeter data on the CCD image.
And the resolving module is used for resolving the external parameters of the real CCD strict geometric imaging model according to the coordinates of the image point and the coordinates of the object space of the connecting point, the coordinates of the image point of the laser altimeter, the coordinates of the object space and the ranging value, and updating the real CCD strict geometric imaging model.
The second construction module is used for carrying out straight line fitting on a camera focal plane according to the internal parameters of the real CCD strict geometric imaging model, constructing a virtual CCD, determining the internal parameters of the strict geometric imaging model of each probe element of the virtual CCD, and constructing a virtual CCD strict geometric imaging model by combining the camera posture, the orbit measurement value and the calculated external parameters of the real CCD strict geometric imaging model.
And the splicing module is used for splicing the linear satellite images according to the virtual CCD strict geometric imaging model, the real CCD strict geometric imaging model and the elevation data.
The beneficial effects of the embodiment of the disclosure are that:
according to the satellite linear array image stitching method assisted by the laser altimetry data, the distance measurement value with higher precision of the laser altimetry data is used for restraining in the image stitching process, so that the accuracy of external parameters in an image geometric imaging model can be better improved, and the geometric positioning accuracy of stitched images is improved.
Drawings
Fig. 1 is a schematic flow chart of a satellite linear array image stitching method assisted by laser altimetry data according to an embodiment of the disclosure;
FIG. 2 is a schematic structural diagram of a satellite linear array image stitching system assisted by laser altimetry data according to an embodiment of the disclosure;
fig. 3 is a schematic diagram of a splicing result of a satellite linear array image splicing method assisted by laser altimetry data according to an embodiment of the disclosure, where (a) is a segmented CCD image and (b) is a spliced image.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present disclosure more apparent, the embodiments of the present disclosure will be further described in detail below with reference to the accompanying drawings. It should be understood that the detailed description is merely illustrative of the disclosed embodiments and is not intended to limit the disclosed embodiments.
As shown in fig. 1, an embodiment of the present disclosure provides a satellite linear array image stitching method assisted by laser altimetry data, including:
and 1, constructing a strict geometric imaging model of each slice of real CCD of the camera.
The precise geometric model of the satellite image shows the projection relationship among image points, a projection center and ground points, and the real fact is a collinearity equation. According to the data of the satellite uploading and downloading gesture, orbit and imaging time, the internal and external parameters of the geometric model provided by the on-orbit calibration are utilized, the following formula (1) is adopted to construct a precise geometric imaging model of each CCD slice of the camera, and the precise geometric model of the image can be expressed as follows by taking a high-resolution seventh satellite as an example:
wherein, (X, Y, Z) T Is the object coordinate of the image point under the ground object coordinate system, (X) S ,Y S ,Z S ) T Is the object space coordinate of the satellite body coordinate system center in the ground object space coordinate system, m is the imaging proportionality coefficient,is the transformation matrix from the satellite body coordinate system to the J2000 inertial coordinate system, < >>Is a transformation matrix from the J2000 inertial coordinate system to the object coordinate system. (psi) x ,ψ y ) Is the pointing angle of the image probe element, represents the parameters in the strict geometric model of the image imagingAnd (5) the internal precision of the image. R is R u Is a matrix formed by external parameters of an image imaging tight geometric model and represents the external precision of the image.
Rigorous geometric imaging model external parameter matrix R u Determined by the formula (2), the specific form is as follows:
wherein,ω, κ are rotational angles around the Y, X, Z axes, respectively, which need to be precisely calculated.
And step 2, carrying out connection point matching in the overlapping area of the CCD images of each slice to obtain a certain number of connection points which are uniformly distributed.
Specifically, the object-space coordinates of the connection point may be obtained from the intersection in front of the rigorous geometric imaging model established in step 1. The number of connection points is satisfactory.
And step 3, acquiring laser altimeter data in an imaging time range of the image to obtain image point coordinates of the laser altimeter data on the CCD image.
The high-resolution No. seven (GF-7) satellite is provided with a double-linear-array stereo camera and a laser altimeter, laser altimeter data can be obtained through the laser altimeter, the GF-7 laser altimeter is provided with a footprint camera for each laser, and the footprint camera is used for imaging emergent laser and ground objects, and emergent laser spots and ground object information exist in the footprint image.
Specifically, the obtaining the laser altimeter data in the imaging time range of the image to obtain the image point coordinates of the laser altimeter data on the CCD image includes:
and 31, substituting the object space coordinates of the laser altimeter into the constructed strict geometric model of the real CCD to obtain the approximate image point coordinates of the laser altimeter on each segmented CCD image.
And step 32, performing fixed-point matching on the obtained rough image point coordinates of the laser altimeter on the CCD image and the laser altimeter image point coordinates in the footprint image within the size range of the footprint to obtain the accurate image point coordinates of the laser altimeter on the CCD image.
Specifically, in step 32, the rough image point coordinates of the laser altimeter on each segmented CCD image may be used as an initial value, the laser altimeter image point coordinates of the footprint image may be used as a fixed value, and the least square matching may be performed within the footprint size range, so as to obtain the accurate image point coordinates of the laser altimeter on each segmented CCD image.
Step 33, if the fixed-point matching in step 32 fails, performing homonymous point matching on the footprint image and each segmented CCD image within the footprint size range, ensuring uniform homonymous point distribution, setting up a mapping relation between each segmented CCD image and the footprint image, and substituting the image point coordinates of the laser altimeter on the footprint image to obtain the accurate image point coordinates of the laser altimeter on each segmented CCD image.
The mapping relation between each segmented CCD image and the footprint image is as follows:
wherein (x ', y') is the image point coordinates of the laser altimeter point on each segmented CCD image, and (x, y) is the image point coordinates of the laser altimeter point on the footprint image, and (a) 0 ,a 1 ,a 2 ,b 0 ,b 1 ,b 2 ) The mapping relation coefficient of the image point coordinates of each segmented CCD image and the footprint image is respectively.
Repeating the steps 31 to 33 to obtain the accurate image point coordinates of all the laser altimeter points on the segmented CCD image.
According to the embodiment of the disclosure, the image point coordinates of the laser altimetry data on the segmented CCD image are obtained according to the footprint image of the laser altimetry data.
And 4, calculating external parameters of the real CCD strict geometric imaging model according to the point coordinates and the object coordinates of the connecting point, the point coordinates, the object coordinates and the ranging value of the laser altimeter, and updating the real CCD strict geometric imaging model.
Specifically, the satellite laser altimetry data provides accurate distance information between the satellite and the ground at the moment of laser emission, and according to a distance equation, a ranging equation of the laser altimetry data can be obtained, wherein the ranging equation is represented by the following formula (4):
wherein (X, Y ', Z') T Is the object coordinate of the laser height measurement point under the ground object coordinate system, (X) S ,Y S ,Z S ) T The method is characterized in that the satellite body coordinate system is the object space coordinate of the center of the satellite body coordinate system in the ground object space coordinate system, and ρ is the accurate ranging value of the laser altimeter.
Because the laser pulse emission time of the laser altimeter is not completely consistent with the orbit measurement system due to the limitation of the sampling frequency of the laser altimeter, the satellite coordinates of the laser pulse emission time can be obtained by interpolating orbit data from the laser emission time, and the following formula (5) is adopted:
X S ＝X 0 +a 1 t′+b 1 t′
Y S ＝Y 0 +a 2 t′+b 2 t′
Z S ＝Z 0 +a 3 t′+b 3 t′ (5)
where t' is the pulse emission time, (X) 0 ,Y 0 ,Z 0 ) T Is the initial coordinate of the center of the satellite body coordinate system in the ground object coordinate system, a i ，b i (i=1, 2, 3) is a track data polynomial coefficient.
Specifically, the calculating the external parameters of the real CCD strict geometric imaging model according to the coordinates of the image point and the coordinates of the object space of the connecting point, the coordinates of the image point of the laser altimeter, the coordinates of the object space and the ranging value, and updating the real CCD strict geometric imaging model includes:
and 41, respectively constructing an error equation for the external parameters of the strict geometric imaging model of the real CCD, the connecting point and the object space coordinates of the laser altimeter according to the image space coordinates and the object space coordinates of the connecting point and the laser altimeter.
Preferably, before the error equation is established, the formula (1) needs to be deformed to obtain the formula (1-1):
order the
Then:
expanding the above formula, the following formula is obtained:
specifically, the external parameters, the laser altimeter and the ground object coordinates of the connection points in the real imaging CCD strict geometric model are deflected by using the formula (1-3), so as to obtain an error equation (6):
writing formula (6) into a matrix form is:
V 1 ＝At+B 1 x-L 1 ,P 1 (7)
wherein V is 1 ＝(v x v y ) T Is the projection error of the observed value,is the external parameter correction of the image tight geometric imaging model, x= (delta X delta Y delta Z) T Is the laser height measurementCorrection of point or junction ground object space coordinates, A, B 1 For the corresponding matrix of correction coefficients, L 1 And P 1 The constant and weight matrix are calculated for the initial values, respectively.
And 42, constructing a distance error equation for the laser altimeter according to the distance measurement information of the laser altimeter.
Preferably, before the error equation is established, the formula (4) needs to be deformed to obtain the formula (4-1):
specifically, the ground object coordinates of the laser altimeter are deflected by using a formula (4-1) to obtain an error equation (8):
writing equation (8) into a matrix form is:
V 2 ＝B 2 x-L 2 ,P 2 (9)
wherein V is 2 ＝v las Is the laser altimeter distance error value, x= (Δx Δy Δz) T Is the correction of the laser height measurement point ground object space coordinate, B 2 For the corresponding matrix of correction coefficients, L 2 And P 2 The constant and weight matrix are calculated for the initial values, respectively.
Combining the connection point constructed in the step 41 with the laser altimeter error equation and the laser altimeter distance error equation constructed in the step 42 to construct an overall error equation:
V＝At+Bx-L,P (10)
wherein V= (V) 1 V 2 ) T And representing residual vectors of observed values, wherein A and B are corresponding coefficient matrixes of correction numbers, and L and P are initial value calculation constants and weight matrixes respectively. x is the object coordinate correction of the connecting point and the laser altimeter.
Step 43, performing methodological on the constructed integral error equation (10), and solving external parameters and connection points of a strict geometric imaging model of a real CCD by using a least square method, and measuring object space coordinate corrections of a high-point laser, wherein the correction is shown in a formula (11):
and (3) solving the formula (11) to obtain the external parameters and the connection points of the strict geometric imaging model of the real CCD and the object space coordinate correction of the laser altimeter.
And 44, updating the external parameters and the connection points of the geometric imaging model and the coordinates of the laser altimeter object according to the correction, repeating the steps until the iteration is finished, obtaining the refined value of the external parameters of the strict geometric imaging model of the real CCD, and updating the strict geometric imaging model of the real CCD to obtain the final strict geometric imaging model of the real CCD.
And 5, performing straight line fitting on a camera focal plane according to the internal parameters of the strict geometric imaging model of each real CCD, constructing a virtual CCD, determining the internal parameters of the strict geometric imaging model of each probe element of the virtual CCD, and constructing a virtual CCD strict geometric imaging model by combining the camera posture, the orbit measurement value and the external parameters of the strict geometric imaging model of the real CCD obtained after resolving.
And 6, splicing the linear satellite images according to the virtual CCD strict geometric imaging model, the real CCD strict geometric imaging model and the elevation data.
Fig. 3 is a schematic diagram of a result of splicing by the splicing method according to the embodiment of the disclosure. And according to the virtual CCD strict geometric imaging model and the strict geometric imaging model of each sliced real CCD, the disclosed elevation data is assisted to determine the position of each pixel on each sliced real CCD image on the virtual CCD image, meanwhile, the gray values of pixels around the position are interpolated into the virtual CCD image, and the interpolation is repeated until the gray values of all the virtual CCD images are obtained, so that the linear array satellite images can be spliced. The elevation data may be obtained by other means known in the art, and is not limited herein.
By adopting the technical scheme disclosed by the embodiment of the disclosure, the following beneficial effects are obtained: the embodiment of the disclosure provides a linear array image splicing method assisted by laser altimetry data, which is an image splicing method based on an object space. According to the embodiment of the disclosure, the distance measurement value with higher precision of the laser altimetry data is constrained in the image stitching process, so that the accuracy of external parameters in the image geometric imaging model can be better improved, and the geometric positioning accuracy of stitched images is improved. The embodiment of the disclosure can realize visual seamless and geometric seamless of the segmented CCD image, improves the external positioning precision of the image while eliminating the internal distortion of the image, and is more beneficial to the subsequent processing application of satellite images.
As shown in fig. 2, another aspect of the embodiments of the present disclosure provides a laser altimetry data-aided linear array image stitching system, the system comprising:
a first building block 100 is used for building a rigorous geometric imaging model of each slice of real charge coupled device CCD of the camera.
And the matching module 200 is used for matching connection points in the overlapping areas of the CCD images of the fragments to obtain a certain number of connection points which are uniformly distributed. The object-space coordinates of the connection points may be obtained from the intersection in front of the rigorous geometric imaging model built in the first build module 100. The number of connection points is sufficiently large.
The acquisition module 300 is configured to acquire laser altimetry data within an imaging time range, and obtain an image point coordinate of the laser altimetry data on a CCD image. According to the embodiment of the disclosure, the image point coordinates of the laser altimetry data on the segmented CCD image are obtained according to the footprint image of the laser altimetry data.
And the resolving module 400 is used for resolving the external parameters of the real CCD strict geometric imaging model according to the coordinates of the image point and the coordinates of the object space of the connecting point, the coordinates of the image point of the laser altimeter, the coordinates of the object space and the ranging value, and updating the real CCD strict geometric imaging model. According to the embodiment of the disclosure, the external parameters of the strict geometric imaging model of the real CCD, the object space coordinate corrections of the connecting points and the laser altimeter are obtained by constructing the related error equation of the connecting points and the laser altimeter, and the strict geometric imaging model of the real CCD is updated.
The second construction module 500 is configured to construct a virtual CCD by performing straight line fitting on a camera focal plane according to the internal parameters of the real CCD strict geometric imaging model, determining the internal parameters of the real geometric imaging model of each probe element of the virtual CCD, and constructing a virtual CCD strict geometric imaging model by combining the camera pose, the orbit measurement value and the calculated external parameters of the real CCD strict geometric imaging model.
And the stitching module 600 is configured to stitch the linear satellite images according to the virtual CCD strict geometric imaging model, the real CCD strict geometric imaging model and the elevation data. According to the embodiment of the disclosure, the position of each pixel on the virtual CCD image on each piece of real CCD image is determined, and meanwhile, the gray values of pixels around the position are interpolated into the virtual CCD image, so that the linear array satellite images are spliced.
The splicing system disclosed by the embodiment of the disclosure is simple in structure, and the embodiment of the disclosure is used for measuring the distance with higher accuracy by means of laser, so that the accuracy of external parameters in an image geometric imaging model can be better improved by restraining in the image splicing process, and the geometric positioning accuracy of spliced images is improved. The embodiment of the disclosure can realize visual seamless and geometric seamless of the segmented CCD image, improves the external positioning precision of the image while eliminating the internal distortion of the image, and is more beneficial to the subsequent processing application of satellite images.
The foregoing is merely a preferred implementation of the embodiments of the disclosure, and it should be noted that, for a person skilled in the art, several improvements and modifications may be made without departing from the principles of the embodiments of the disclosure, which should also be considered as protective scope of the embodiments of the disclosure.
Claims (10)
1. The satellite linear array image splicing method assisted by laser altimetry data is characterized by comprising the following steps of:
constructing a strict geometric imaging model of each slice of real CCD of the camera;
carrying out connection point matching in each segmented CCD image overlapping area to obtain a certain number of connection points which are uniformly distributed;
acquiring laser altimeter data in an imaging time range of an image to obtain image point coordinates of the laser altimeter data on a CCD image;
according to the point coordinates and the object coordinates of the connecting point, the point coordinates, the object coordinates and the ranging value of the laser altimeter, calculating the external parameters of the strict geometric imaging model of the real CCD, and updating the strict geometric imaging model of the real CCD;
performing straight line fitting on a camera focal plane according to internal parameters of the real CCD strict geometric imaging model, constructing a virtual CCD, determining internal parameters of the strict geometric imaging model of each probe element of the virtual CCD, and constructing a virtual CCD strict geometric imaging model by combining camera gestures, orbit measurement values and calculated external parameters of the real CCD strict geometric imaging model;
and splicing the linear satellite images according to the virtual CCD strict geometric imaging model, the real CCD strict geometric imaging model and the elevation data.
2. The method according to claim 1, wherein the calculating the external parameters of the real CCD rigorous geometric imaging model according to the point coordinates and object coordinates of the connection point, and the point coordinates, object coordinates and ranging values of the laser altimetric point, and updating the real CCD rigorous geometric imaging model comprises:
respectively constructing an error equation for the external parameters, the connecting points and the object space coordinates of the laser altimeter of the strict geometric imaging model of the real CCD;
constructing a distance error equation for the laser altimetric points according to the distance measurement information of the laser altimetric points;
the error equation constructed in the first two steps is normalized, and the external parameters and the connection points of the strict geometric imaging model of the real CCD and the coordinates correction of the laser altimeter object space are solved by using a least square method;
and updating external parameters and connection points of the geometric imaging model and the object space coordinates of the laser altimeter according to the correction, and repeating the steps until the iteration is finished to obtain the final real geometric imaging model of the CCD.
3. The method of claim 2, wherein the rigorous geometric imaging model of the CCD is:
wherein, (X, Y, Z) T Is the object coordinate of the image point under the ground object coordinate system, (X) S ,Y S ,Z S ) T Is the object space coordinate of the satellite body coordinate system center in the ground object space coordinate system, m is the imaging proportionality coefficient,is the transformation matrix from the satellite body coordinate system to the J2000 inertial coordinate system, < >>Is a transformation matrix from a J2000 inertial coordinate system to an object coordinate system; (psi) x ,ψ y ) Is the pointing angle of the image probe element, is the internal parameter of the image imaging strict geometric model, and represents the internal accuracy of the image, R u The matrix is formed by external parameters of an image imaging tight geometric model, and represents the external precision of the image;
rigorous geometric imaging model external parameter matrix R u Determined by the formula (2), the specific form is as follows:
wherein,omega, kappa are rotation angles around the Y, X, Z axes, respectively;
the error equation is respectively constructed for the external parameters of the strict geometric imaging model of the real CCD, the connecting points and the object space coordinates of the laser altimeter to obtain a formula (3):
writing formula (3) into a matrix form is:
V 1 ＝At+B 1 x-L 1 ,P 1 (4)
wherein V is 1 ＝(v x v y ) T Is the projection error of the observed value,is the external parameter correction of the image tight geometric imaging model, x= (delta X delta Y delta Z) T Is the correction of the ground object space coordinates of the laser height measurement point or connecting point, A, B 1 For the corresponding matrix of correction coefficients, L 1 And P 1 The constant and weight matrix are calculated for the initial values, respectively.
4. A method according to claim 2 or 3, wherein the ranging equation for the ranging information of the laser altimeter is:
wherein (X, Y ', Z') T Is the object coordinate of the laser height measurement point under the ground object coordinate system, (X) S ,Y S ,Z S ) T The system is an object space coordinate of a satellite body coordinate system center in a ground object space coordinate system, and ρ is an accurate ranging value of a laser altimeter;
and constructing a distance error equation (6) for the laser altimetric points according to the distance measurement information of the laser altimetric points:
writing formula (6) into a matrix form is:
V 2 ＝B 2 x-L 2 ,P 2 (7)
wherein V is 2 ＝v las Is the laser altimeter distance error value, x= (Δx Δy Δz) T Is the correction of the laser height measurement point ground object space coordinate, B 2 For the corresponding matrix of correction coefficients, L 2 And P 2 The constant and weight matrix are calculated for the initial values, respectively.
5. The method according to claim 4, wherein the error equation constructed in the previous two steps is normalized, and an integral error equation is constructed by combining the connection point and the laser altimeter error equation formula (4) and the laser altimeter error equation formula (7) before the external parameters and the connection point of the strict geometric imaging model of the real CCD and the laser altimeter object space coordinate correction are solved by using a least square method:
V＝At+Bx-L,P (8)
wherein V= (V) 1 V 2 ) T And representing the residual vectors of the observed values, wherein A and B are corresponding coefficient matrixes of the corrections, L and P are initial value calculation constants and weight matrixes respectively, and x is the coordinate correction of the object space of the connecting point and the laser altimeter.
6. A method according to any one of claims 1 to 3, wherein the obtaining laser altimetry data over the imaging time frame to obtain image point coordinates of the laser altimetry data on the CCD image comprises:
substituting the object space coordinates of the laser altimeter into the constructed strict geometric model of the real CCD to obtain the approximate image point coordinates of the laser altimeter on each segmented CCD image;
and performing fixed point matching on the obtained rough image point coordinates of the laser altimeter points on the CCD image and the laser altimeter point coordinates in the footprint image within the size range of the footprint to obtain the accurate image point coordinates of the laser altimeter points on the CCD image.
7. The method of claim 6, wherein the acquiring laser altimetry data over the imaging time frame to obtain image point coordinates of the laser altimetry data on the CCD image further comprises:
if the fixed point matching fails, the same name point matching is carried out on the footprint image and the CCD image within the footprint size range, the mapping relation between the CCD image and the footprint image is established, and the image point coordinates of the laser height measurement points on the footprint image are substituted, so that the accurate image point coordinates of the laser height measurement points on the CCD image are obtained.
8. The method of claim 7, wherein the mapping relationship between the CCD image and the footprint image is:
wherein (x ', y') is the image point coordinates of the laser altimeter point on each segmented CCD image, and (x, y) is the image point coordinates of the laser point footprint image, and (a) 0 ,a 1 ,a 2 ,b 0 ,b 1 ,b 2 ) The mapping relation coefficients of the image point coordinates of each segmented CCD image and the footprint image are respectively.
9. A method according to any one of claims 1 to 3, wherein stitching the line satellite images based on the virtual CCD rigorous geometric imaging model, the real CCD rigorous geometric imaging model, and elevation data comprises:
determining the position of each pixel on the virtual CCD image on each piece of real CCD image, interpolating the gray values of pixels around the position to the virtual CCD image, repeating the interpolation until the gray values of all the virtual CCD images are obtained, and completing the splicing.
10. A laser altimeter data aided satellite linear array image stitching system, the system comprising:
the first construction module is used for constructing a strict geometric imaging model of each slicing real charge coupled device CCD of the camera;
the matching module is used for carrying out connection point matching in each segmented CCD image overlapping area to obtain a certain number of connection points which are uniformly distributed;
the acquisition module is used for acquiring laser altimetry data in an imaging time range of the image to obtain image point coordinates of the laser altimetry data on a CCD image;
the resolving module is used for resolving external parameters of the real CCD strict geometric imaging model according to the coordinates of the image point and the coordinates of the object space of the connecting point, the coordinates of the image point of the laser altimeter, the coordinates of the object space and the ranging value, and updating the real CCD strict geometric imaging model;
the second construction module is used for carrying out straight line fitting on a camera focal plane according to the internal parameters of the real CCD strict geometric imaging model, constructing a virtual CCD, determining the internal parameters of the strict geometric imaging model of each probe element of the virtual CCD, and constructing a virtual CCD strict geometric imaging model by combining the camera posture, the orbit measurement value and the calculated external parameters of the real CCD strict geometric imaging model;
and the splicing module is used for splicing the linear satellite images according to the virtual CCD strict geometric imaging model, the real CCD strict geometric imaging model and the elevation data.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311079334.8A CN117109519B (en) | 2023-08-25 | 2023-08-25 | Satellite linear array image stitching method and system assisted by laser altimetry data |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311079334.8A CN117109519B (en) | 2023-08-25 | 2023-08-25 | Satellite linear array image stitching method and system assisted by laser altimetry data |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN117109519A true CN117109519A (en) | 2023-11-24 |
| CN117109519B CN117109519B (en) | 2024-04-09 |
Family
ID=88805102
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311079334.8A Active CN117109519B (en) | 2023-08-25 | 2023-08-25 | Satellite linear array image stitching method and system assisted by laser altimetry data |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117109519B (en) |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103914808A (en) * | 2014-03-14 | 2014-07-09 | 国家测绘地理信息局卫星测绘应用中心 | Method for splicing ZY3 satellite three-line-scanner image and multispectral image |
| CN106895851A (en) * | 2016-12-21 | 2017-06-27 | 中国资源卫星应用中心 | A kind of sensor calibration method that many CCD polyphasers of Optical remote satellite are uniformly processed |
| US20180003825A1 (en) * | 2016-06-30 | 2018-01-04 | Topcon Corporation | Laser Scanner System And Registration Method Of Point Cloud Data |
| CN111508029A (en) * | 2020-04-09 | 2020-08-07 | 武汉大学 | Satellite-borne segmented linear array CCD optical camera overall geometric calibration method and system |
| CN113538595A (en) * | 2021-07-14 | 2021-10-22 | 自然资源部国土卫星遥感应用中心 | Method for improving geometric precision of remote sensing stereo image by using laser height measurement data in auxiliary manner |
| US20230102712A1 (en) * | 2021-09-27 | 2023-03-30 | Wuhan University | Method of fully autonomous geometric calibration for linear-array remote sensing satellites |
-
2023
- 2023-08-25 CN CN202311079334.8A patent/CN117109519B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103914808A (en) * | 2014-03-14 | 2014-07-09 | 国家测绘地理信息局卫星测绘应用中心 | Method for splicing ZY3 satellite three-line-scanner image and multispectral image |
| US20180003825A1 (en) * | 2016-06-30 | 2018-01-04 | Topcon Corporation | Laser Scanner System And Registration Method Of Point Cloud Data |
| CN106895851A (en) * | 2016-12-21 | 2017-06-27 | 中国资源卫星应用中心 | A kind of sensor calibration method that many CCD polyphasers of Optical remote satellite are uniformly processed |
| CN111508029A (en) * | 2020-04-09 | 2020-08-07 | 武汉大学 | Satellite-borne segmented linear array CCD optical camera overall geometric calibration method and system |
| CN113538595A (en) * | 2021-07-14 | 2021-10-22 | 自然资源部国土卫星遥感应用中心 | Method for improving geometric precision of remote sensing stereo image by using laser height measurement data in auxiliary manner |
| US20230102712A1 (en) * | 2021-09-27 | 2023-03-30 | Wuhan University | Method of fully autonomous geometric calibration for linear-array remote sensing satellites |
Non-Patent Citations (1)
| Title |
|---|
| 唐新明 等: "高分七号卫星立体影像与激光测高数据联合区域网平差", 武汉大学学报 信息科学版, vol. 46, no. 10, 31 October 2021 (2021-10-31) * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN117109519B (en) | 2024-04-09 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108629831B (en) | Three-dimensional human body reconstruction method and system based on parameterized human body template and inertial measurement | |
| CN105091906B (en) | High-resolution optical, which pushes away, sweeps the weight imaging sensor bearing calibration of satellite stable state and system | |
| US5606627A (en) | Automated analytic stereo comparator | |
| De Franchis et al. | An automatic and modular stereo pipeline for pushbroom images | |
| Morgan | Epipolar resampling of linear array scanner scenes | |
| EP1378790B1 (en) | Method and device for correcting lens aberrations in a stereo camera system with zoom | |
| CN109727278B (en) | Automatic registration method for airborne LiDAR point cloud data and aerial image | |
| CN103697864B (en) | A kind of narrow visual field double camera image splicing method based on large virtual camera | |
| JP5134784B2 (en) | Aerial photogrammetry | |
| CN111044037B (en) | Geometric positioning method and device for optical satellite image | |
| CN111612693B (en) | Method for correcting rotary large-width optical satellite sensor | |
| CN113538595B (en) | Method for improving geometric precision of remote sensing stereo image by using laser height measurement data in auxiliary manner | |
| CN108447100B (en) | Method for calibrating eccentricity vector and visual axis eccentricity angle of airborne three-linear array CCD camera | |
| CN112258422B (en) | Automatic refinement method for rational polynomial parameters (RPC) of stereoscopic image | |
| Hattori et al. | Orientation of high-resolution satellite images based on affine projection | |
| Di et al. | High-resolution large-area digital orthophoto map generation using LROC NAC images | |
| CN113450416B (en) | TCSC method applied to three-dimensional calibration of three-dimensional camera | |
| CN113947638A (en) | Image orthorectification method for fisheye camera | |
| KR100520275B1 (en) | Method for correcting geometry of pushbroom image using solidbody rotation model | |
| CN111508028A (en) | Autonomous in-orbit geometric calibration method and system for optical stereo mapping satellite camera | |
| CN116385507A (en) | Multi-source point cloud data registration method and system based on different scales | |
| CN105571598B (en) | A kind of assay method of laser satellite altimeter footmark camera posture | |
| CN109029379B (en) | High-precision small-base-height-ratio three-dimensional mapping method | |
| Wang et al. | Geometric calibration for the aerial line scanning camera GFXJ | |
| JP4809134B2 (en) | Image processing apparatus and processing method thereof |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |