CN112419379A - Multi-channel image matching method and device of multispectral camera - Google Patents
Multi-channel image matching method and device of multispectral camera Download PDFInfo
- Publication number
- CN112419379A CN112419379A CN202011373528.5A CN202011373528A CN112419379A CN 112419379 A CN112419379 A CN 112419379A CN 202011373528 A CN202011373528 A CN 202011373528A CN 112419379 A CN112419379 A CN 112419379A
- Authority
- CN
- China
- Prior art keywords
- channel
- image
- channels
- coordinate
- parameters
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 57
- 238000006243 chemical reaction Methods 0.000 claims abstract description 83
- 238000003384 imaging method Methods 0.000 claims abstract description 36
- 239000011159 matrix material Substances 0.000 claims description 16
- 238000004590 computer program Methods 0.000 claims description 9
- 230000006870 function Effects 0.000 claims description 9
- 238000012937 correction Methods 0.000 claims description 8
- 230000009466 transformation Effects 0.000 claims description 7
- 238000000605 extraction Methods 0.000 claims description 5
- 239000004576 sand Substances 0.000 claims description 4
- 239000000126 substance Substances 0.000 claims description 3
- 238000004891 communication Methods 0.000 description 6
- 238000010586 diagram Methods 0.000 description 4
- 230000008569 process Effects 0.000 description 4
- 238000004422 calculation algorithm Methods 0.000 description 3
- 238000001514 detection method Methods 0.000 description 2
- 239000000284 extract Substances 0.000 description 2
- 238000012544 monitoring process Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 238000012545 processing Methods 0.000 description 2
- 241000282320 Panthera leo Species 0.000 description 1
- 238000010521 absorption reaction Methods 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000002310 reflectometry Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/30—Determination of transform parameters for the alignment of images, i.e. image registration
- G06T7/33—Determination of transform parameters for the alignment of images, i.e. image registration using feature-based methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
- G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10032—Satellite or aerial image; Remote sensing
- G06T2207/10036—Multispectral image; Hyperspectral image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30181—Earth observation
Landscapes
- Engineering & Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Image Processing (AREA)
Abstract
The invention provides a multi-channel image matching method and a device of a multi-spectral camera, wherein the method comprises the following steps: acquiring multi-channel images with preset groups, extracting characteristic points of each image, and matching the characteristic points of all the multi-channel images to obtain corresponding points of a reference channel and other channels; determining coordinate conversion parameters in a coordinate conversion relation according to camera imaging difference and sensor position offset by taking the image coordinates and the lens focal length of the homonymy points of other channels as coordinate parameters before conversion and taking the feature point image coordinates and the lens focal length of a reference channel as coordinate parameters after conversion; and converting other channels in all the shot multi-channel images into reference channels according to the coordinate conversion parameters, and superposing to obtain shot matching images. The method considers the two main imaging offset differences, realizes multi-channel image matching for the multispectral camera, and can solve the problems of matching failure or low precision in the agricultural field.
Description
Technical Field
The invention relates to the technical field of image processing, in particular to a multi-channel image matching method and device of a multi-spectral camera.
Background
The multispectral camera is widely applied to agriculture and forestry, homeland resources and environmental monitoring. At present, due to the technical and cost limitations, multi-spectral phases which can be carried on a consumer-grade unmanned aerial vehicle, such as a Xinjiang spirit 4 multi-spectral camera, mostly do not adopt a light splitting technology, but a plurality of cameras are arranged, and each camera independently acquires a channel image, so that multi-spectral data are formed. In the imaging mode, the sensors are not located in a uniform imaging center, so that even at the same imaging time, the observation ranges of the sensors are different, and imaging images cannot be completely overlapped. The multi-channel images which cannot be completely overlapped are directly overlapped to cause the images to have offset, so that the data cannot be applied to various target environment monitoring at a later stage. Therefore, before multi-channel image data of the multispectral camera is used, matching between images is needed, and images of all channels are matched to a consistent observation range.
Current image matching is mainly performed by geometric fine correction between images. Specifically, the homonymous points are obtained firstly, then the correction equation is fitted through the homonymous points, and the image change is carried out through the correction equation to obtain the matched image. The following is a specific process flow: feature-based image matching first extracts features from each of the images to be configured, and manual or automatic detection methods may be used. In the automatic detection method, image features can be extracted by using feature extraction methods such as SIFT, SURF, Harris corner and the like. And then, carrying out feature matching, and completing one-to-one mapping of the feature points detected in the image to be matched and the reference image, wherein the step can stably complete the matching of extracting the feature points by using a more classical RANSAC algorithm, and the result obtained in the step is the homonymy point. According to the obtained homonymous point fitting equation, quadratic equations, polynomials and the like can be selected, and parameters required by image matching conversion can be obtained. And finally, matching the images by using the fitting equation to complete the matching process between the two images.
However, in agricultural applications, the existing multispectral camera image matching only takes camera imaging differences into account, so that the matching accuracy is not high.
Disclosure of Invention
Aiming at the problems in the prior art, the invention provides a multi-channel image matching method and device of a multi-spectral camera.
The invention provides a multi-channel image matching method of a multispectral camera, which comprises the following steps: acquiring multi-channel images with preset groups, extracting characteristic points of each image, and matching the characteristic points of all the multi-channel images to obtain corresponding points of a reference channel and other channels; respectively determining coordinate conversion parameters in a coordinate conversion relation according to camera imaging difference and sensor position offset by using image coordinates and lens focal lengths of homologous points of other channels as coordinate parameters before conversion and using feature point image coordinates and lens focal lengths of a reference channel as coordinate parameters after conversion; and converting other channels in all the shot multi-channel images into reference channels according to the coordinate conversion parameters, and superposing to obtain shot matching images.
According to the multi-channel image matching method of the multispectral camera, the reference channel is a near-infrared channel, and the other channels are red-edge, blue-green and red-light channels.
According to the multi-channel image matching method of the multispectral camera, coordinate conversion parameters of imaging differences of the camera comprise a rotation matrix which is converted from other channel homonymy points to space coordinates and a rotation matrix which is converted from the space coordinates to a reference channel; and the coordinate conversion parameters of the sensor position deviation comprise the space distance between the reference channel and other channel images and a rotation matrix for converting the space coordinates into the reference channel.
According to the multi-channel image matching method of the multispectral camera, coordinate conversion parameters in a coordinate conversion relation are respectively determined according to camera imaging difference and sensor position offset, and the method comprises the following steps:
wherein x iss、ysAnd fsThe line number, the column number and the lens focal length x of the characteristic point corresponding to the reference channel on the imaget、ytAnd ftRespectively representing the uplink, the column number and the lens focal length of the image of the same name point in the image of other channels; in the coordinate transformation parameters, alpha and beta are correction coefficients Cx、CyAnd CzThe distance in space X, Y, Y between the reference image and the other channel images,
Rsthe rotation matrix is converted from the space coordinate to the reference channel, and the rotation matrix is converted from the homonymy points of other channels to the space coordinate.
The multi-channel image matching method of the multispectral camera comprises the following steps:
accordingly, the coordinate conversion parameters include
ω、κ，Cx、Cy、Cz、α、β；
Wherein the content of the first and second substances,
ωt、κtand
ωs、κsare respectively as
And RsThe flight course inclination angle, the side inclination angle and the sensor rotation angle.
According to the multi-channel image matching method of the multispectral camera, the multi-channel images with the preset group number are obtained, and the method comprises the following steps: randomly selecting an image obtained by shooting m times from n times of shooting obtained by one-time unmanned aerial vehicle operation as a multi-channel image with a preset group number m; wherein m is greater than 30% n.
According to the multi-channel image matching method of the multispectral camera, the coordinate conversion parameters in the coordinate conversion relation are determined, and the method comprises the following steps: determining the interpretation function of the measured value Y for a given parameter X:
determining a posterior distribution P (X | Y) of the parameters according to the flame function; taking the mean value of the posterior distribution as the estimated value of the parameter; wherein n represents the number of times of estimation, f (X) is the estimated value, yiIs the ith true value.
The resolution ratio of the existing multi-channel unmanned aerial vehicle is relatively low. In agricultural application, when a target crop needs to be observed in detail, the available crop features are few, enough matching points cannot be obtained between channel images to be matched, and the coordinate conversion parameters are determined based on the current least square method, so that the possibility of correction failure is high. Furthermore, for multispectral data, the reflectivity of the same feature on different channels is more different, making the acquired features unstable, which also increases the likelihood of a calibration failure.
According to the method, the posterior distribution of the parameters is determined through the interpretation function, the mean value of the posterior distribution is taken as the estimated value of the parameters, accurate coordinate conversion parameters can be obtained under the conditions that the crop characteristics are few and the obtained characteristics are unstable, and the matching precision is improved.
The invention also provides a multi-channel image matching device of the multispectral camera, which comprises the following components: the characteristic extraction module is used for acquiring multi-channel images with preset groups, extracting characteristic points of each image, and matching the characteristic points of all the multi-channel images to obtain corresponding points of the reference channel and other channels; the parameter estimation module is used for determining coordinate conversion parameters in a coordinate conversion relation by taking the image coordinates and the lens focal length of the homonymous points of other channels as coordinate parameters before conversion and taking the characteristic point image coordinates and the lens focal length of the reference channel as coordinate parameters after conversion; and the image matching module is used for converting other channels in all the shot multi-channel images into reference channels according to the coordinate conversion parameters and superposing the reference channels to obtain shot matching images.
The invention also provides an electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor executes the program to realize the steps of the multi-channel image matching method of the multispectral camera.
The invention also provides a non-transitory computer-readable storage medium having stored thereon a computer program which, when being executed by a processor, carries out the steps of the method for multi-channel image matching of a multi-spectral camera as defined in any one of the above.
According to the multi-channel image matching method and device of the multispectral camera, the coordinate conversion parameters in the coordinate conversion relation are respectively determined according to the camera imaging difference and the sensor position offset, the two main imaging offset differences are considered, the images with imaging offsets eliminated can be obtained by correcting the two main imaging offset differences, multi-channel image matching for the multispectral camera is achieved, and the problems of matching failure or low precision in the agricultural field can be solved.
Drawings
In order to more clearly illustrate the technical solutions of the present invention or the prior art, the drawings needed for the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and those skilled in the art can also obtain other drawings according to the drawings without creative efforts.
FIG. 1 is a flow chart of a multi-channel image matching method of a multi-spectral camera provided by the invention;
FIG. 2 is a schematic structural diagram of a multi-channel image matching device of a multispectral camera provided by the invention;
fig. 3 is a schematic structural diagram of an electronic device provided in the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings, and it is obvious that the described embodiments are some, but not all embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The following describes the multi-channel image matching method and device of the multispectral camera of the present invention with reference to fig. 1-3. Fig. 1 is a schematic flow chart of a multi-channel image matching method for a multispectral camera according to the present invention, and as shown in fig. 1, the multi-channel image matching method for a multispectral camera according to the present invention includes:
101. acquiring multi-channel images with preset groups, extracting the characteristic points of each image, matching the characteristic points of all the multi-channel images to obtain the homonymous points of the reference channel and other channels respectively.
When the multichannel unmanned aerial vehicle shoots an image, a plurality of channel images of the image are obtained. For the commonly used five channel images of near infrared, red edge, blue, green and red, b1, b2, b3, b4 and b5 are respectively used for representation. And taking one channel as a reference channel, and matching other channels to the reference channel. The remaining bands are matched to the b1 band, as based on b 1.
According to OpenCV, a Scale-invariant feature transform (SIFT) algorithm is used for extracting feature points of each image respectively. The b1 channel acquired by each shooting is matched with the b2, b3, b4 and b5 channel image feature points respectively by using a FLANN (fast Library for Approximate Nearest neighbors) algorithm to obtain the homonymous points of the b1 channel and b2, b3, b4 and b5 respectively. When FLANN matches, the Lowe ratio threshold is 0.5. If matching of b1 with the same name point of any other channel fails in one shooting, the data of the shooting is excluded from the feature matching operation.
102. And respectively determining coordinate conversion parameters in a coordinate conversion relation according to camera imaging difference and sensor position offset by using the image coordinates and the lens focal length of the homonymy points of other channels as coordinate parameters before conversion and using the feature point image coordinates and the lens focal length of the reference channel as coordinate parameters after conversion.
For the multispectral camera, the channels are mutually independent, the distance is short, the position is fixed, and the multispectral camera is a rigid body. Therefore, the deviation of the imaging image of each channel mainly comes from two parts, the first part is the imaging deviation amount caused by the position deviation of the sensor between the channels, and the second part is the imaging image deviation caused by the imaging difference of the respective body cameras. The two main imaging offset differences are considered and corrected to obtain an image with imaging offset removed, which is described by the following formula:
Ft＝Ffocal+FPosition；
in the formula, FtIndicating the corrected image position, FfocalAnd FPositionRespectively, the correction of the imaging shift amount due to the camera imaging difference and the sensor position shift. By taking the offset in imaging into account globally, the source of the offset error between the channels is divided into two parts, and the offset is corrected by the corresponding imaging mechanism respectively.
And determining the conversion relation of the homonymous points of other channels to the reference channel, and solving the coordinate conversion parameters in the conversion relation on the basis that the homonymous points and the characteristic points are known. The coordinate conversion parameters may be estimated using a Markov Chain Monte Carlo (MCMC) based bayesian probability programming package PyMC 3.
103. And converting other channels in all the shot multi-channel images into reference channels according to the coordinate conversion parameters, and superposing to obtain shot matching images.
Performing parameter estimationAfter counting, all other channels of the captured images are sequentially marked with point coordinates (x)s、ys) For inputting, the matched reference channel is switched to obtain the matched point coordinate (x)t、yt) Therefore, other non-near-infrared channel images are corrected by taking the near-infrared channel images as a reference to superpose the corrected images, and a matching image acquired by one-time shooting is obtained.
According to the multi-channel image matching method of the multispectral camera, the coordinate conversion parameters in the coordinate conversion relation are respectively determined according to the camera imaging difference and the sensor position offset, the two main imaging offset differences are considered, the images with imaging offsets eliminated can be obtained by correcting the imaging offset differences, multi-channel image matching for the multispectral camera is achieved, and the problems of matching failure or low precision in the agricultural field can be solved.
In one embodiment, the reference channel is a near infrared channel and the other channels are red, blue, green, and red channels, as exemplified above.
In one embodiment, the coordinate conversion parameters of the camera imaging difference comprise a rotation matrix of converting the homonymous points of other channels into space coordinates and a rotation matrix of converting the space coordinates into a reference channel; and the coordinate conversion parameters of the sensor position deviation comprise the space distance between the reference channel and other channel images and a rotation matrix for converting the space coordinates into the reference channel.
In one embodiment, the coordinate conversion parameters in the coordinate conversion relationship are determined by taking the image coordinates and the lens focal length of the homonymous points of the other channels as coordinate parameters before conversion and the feature point image coordinates and the lens focal length of the reference channel as coordinate parameters after conversion, and the coordinate conversion parameters are determined according to the following formula:
wherein x iss、ysAnd fsLines of the image respectively corresponding to the characteristic points of the reference channelColumn number and lens focal length; alpha and beta are correction coefficients respectively; cx、CyAnd CzX, Y, Y is the distance in space between the reference image and the other channel images;
Rsrespectively, a rotation matrix for converting space coordinates to reference channels, a rotation matrix for converting the same-name points of other channels to space coordinates, i.e.
RsAnd respectively the rotation matrixes of the image to be matched and the reference image.
Specifically, different coordinate systems are converted according to the rotation matrix, so that the coordinate conversion of each channel is realized, and the following formula is provided:
in one embodiment of the present invention,
accordingly, the coordinate conversion parameters include
ω、κ，Cx、Cy、Cz、α、β；
Wherein the content of the first and second substances,
ωt、κtand
ωs、κsare respectively as
And RsThe flight course inclination angle, the side inclination angle and the sensor rotation angle.
In one embodiment, the acquiring a preset number of sets of multi-channel images includes: randomly selecting an image obtained by shooting m times from n times of shooting obtained by one-time unmanned aerial vehicle operation as a multi-channel image with a preset group number m; wherein m is greater than 30% n.
Since 5 images (five channels of b1, b2, b3, b4, and b 5) are acquired at one time of shooting, the above 5m images are included. The method has the advantages that the channel images with a certain preset number are randomly selected to obtain the homonymous points in one complete image acquisition, the problem that the homonymous points of a single image are failed to be obtained can be solved, the subsequent parameter estimation precision can be improved, and errors caused by the fact that the single image is selected to be used for estimating the obtained parameters and matching the whole image are reduced.
In one embodiment, the determining the coordinate conversion parameter in the coordinate conversion relationship includes: determining the interpretation function of the measured value Y for a given parameter X:
determining a posterior distribution P (X | Y) of the parameters according to the flame function; taking the mean value of the posterior distribution as the estimated value of the parameter; wherein n represents a total of n estimates, f (X) is an estimate, yiIs the ith true value.
Assuming that the distribution of the above parameters all conform to a truncated normal distribution with variance of 1 and mean of 0, and the distribution ranges of the value ranges are as follows:
-5°≤ωt≤5°
-8°≤κt≤8°
-5°≤ωs≤5°
-8°≤κs≤8°
-20mm≤Cx≤20mm
assuming parameters to be estimated
ωt、κt、
ωs、κs、Cx、CyThe distribution of α, β follows a normal distribution with a variance of 1 and a mean of 0, and the likelihood function for the occurrence of the measured value Y for a given parameter X is shown in the above equation for each parameter X.
After the flame function P (Y | X) is obtained by the above formula, P (X | Y), namely the posterior distribution of the parameter to be estimated, can be calculated by a Bayesian formula as follows, and the average value of the posterior distribution is taken as the estimated value of the parameter, so that the calculation of the parameter to be estimated is completed.
The method fully utilizes prior knowledge, quantifies uncertainty of parameter estimation, and estimates the parameters to be estimated by using a Bayesian probability estimation model based on Markov chain Monte Carlo instead of using a least square method completely depending on data information (same name points), thereby improving accuracy of parameter estimation.
In the process of acquiring a true color image by one-time imaging shooting in flight, because a red channel and a near-infrared channel have large difference, such as strong absorption of vegetation in a red waveband and strong reflection of vegetation in the near-infrared channel, and because crops are uniformly covered and have few characteristic points, the current method is used for extracting characteristics to match the red channel and the near-infrared channel image, the processing fails, and the matched result cannot be obtained. And for this method, then can be better handle above-mentioned problem, with the true color that unmanned aerial vehicle obtained is the closest.
The following describes the multi-channel image matching device of the multispectral camera provided by the invention, and the multi-channel image matching device of the multispectral camera described below and the multi-channel image matching method of the multispectral camera described above can be referred to correspondingly.
Fig. 2 is a schematic structural diagram of a multi-channel image matching device of a multispectral camera according to an embodiment of the present invention, and as shown in fig. 2, the multi-channel image matching device of the multispectral camera includes: a feature extraction module 201, a parameter estimation module 202 and an image matching module 203. The feature extraction module 201 is configured to obtain a preset number of sets of multi-channel images, extract feature points of each image, match the feature points of all the multi-channel images, and obtain corresponding points of a reference channel and other channels; the parameter estimation module 202 is configured to determine a coordinate conversion parameter in a coordinate conversion relationship by using the image coordinates and the lens focal length of the homonymy points of the other channels as coordinate parameters before conversion, and using the feature point image coordinates and the lens focal length of the reference channel as coordinate parameters after conversion; the image matching module 203 is configured to convert other channels in all the captured multi-channel images to reference channels according to the coordinate conversion parameters, and superimpose the reference channels to obtain captured matching images.
The device embodiment provided in the embodiments of the present invention is for implementing the above method embodiments, and for details of the process and the details, reference is made to the above method embodiments, which are not described herein again.
The multi-channel image matching device of the multispectral camera provided by the embodiment of the invention respectively determines the coordinate conversion parameters in the coordinate conversion relation according to the camera imaging difference and the sensor position offset, and corrects the two main imaging offset differences to obtain the image with imaging offset eliminated, so that the multi-channel image matching for the multispectral camera is realized, and the problems of matching failure or low precision in the agricultural field can be solved.
Fig. 3 is a schematic structural diagram of an electronic device provided in the present invention, and as shown in fig. 3, the electronic device may include: a processor (processor)301, a communication Interface (communication Interface)302, a memory (memory)303 and a communication bus 304, wherein the processor 301, the communication Interface 302 and the memory 303 complete communication with each other through the communication bus 304. The processor 301 may invoke logic instructions in the memory 303 to perform a multi-channel image matching method of a multi-spectral camera, the method comprising: acquiring multi-channel images with preset groups, extracting characteristic points of each image, and matching the characteristic points of all the multi-channel images to obtain corresponding points of a reference channel and other channels; determining coordinate conversion parameters in a coordinate conversion relation by taking the image coordinates and the lens focal length of the homonymy points of other channels as coordinate parameters before conversion and taking the feature point image coordinates and the lens focal length of the reference channel as coordinate parameters after conversion; and converting other channels in all the shot multi-channel images into reference channels according to the coordinate conversion parameters, and superposing to obtain shot matching images.
In addition, the logic instructions in the memory 303 may be implemented in the form of software functional units and stored in a computer readable storage medium when the logic instructions are sold or used as independent products. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.
In another aspect, the present invention also provides a computer program product comprising a computer program stored on a non-transitory computer-readable storage medium, the computer program comprising program instructions which, when executed by a computer, enable the computer to perform the multi-channel image matching method for a multi-spectral camera provided by the above methods, the method comprising: acquiring multi-channel images with preset groups, extracting characteristic points of each image, and matching the characteristic points of all the multi-channel images to obtain corresponding points of a reference channel and other channels; determining coordinate conversion parameters in a coordinate conversion relation by taking the image coordinates and the lens focal length of the homonymy points of other channels as coordinate parameters before conversion and taking the feature point image coordinates and the lens focal length of the reference channel as coordinate parameters after conversion; and converting other channels in all the shot multi-channel images into reference channels according to the coordinate conversion parameters, and superposing to obtain shot matching images.
In yet another aspect, the present invention further provides a non-transitory computer-readable storage medium, on which a computer program is stored, the computer program being implemented by a processor to perform the multi-channel image matching method of a multispectral camera provided in the above embodiments, the method including: acquiring multi-channel images with preset groups, extracting characteristic points of each image, and matching the characteristic points of all the multi-channel images to obtain corresponding points of a reference channel and other channels; determining coordinate conversion parameters in a coordinate conversion relation by taking the image coordinates and the lens focal length of the homonymy points of other channels as coordinate parameters before conversion and taking the feature point image coordinates and the lens focal length of the reference channel as coordinate parameters after conversion; and converting other channels in all the shot multi-channel images into reference channels according to the coordinate conversion parameters, and superposing to obtain shot matching images.
The above-described embodiments of the apparatus are merely illustrative, and the units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment. One of ordinary skill in the art can understand and implement it without inventive effort.
Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware. With this understanding in mind, the above-described technical solutions may be embodied in the form of a software product, which can be stored in a computer-readable storage medium such as ROM/RAM, magnetic disk, optical disk, etc., and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the methods described in the embodiments or some parts of the embodiments.
Finally, it should be noted that: the above examples are only intended to illustrate the technical solution of the present invention, but not to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.
Claims (10)
1. A multi-channel image matching method of a multispectral camera is characterized by comprising the following steps:
acquiring multi-channel images with preset groups, extracting characteristic points of each image, and matching the characteristic points of all the multi-channel images to obtain corresponding points of a reference channel and other channels;
respectively determining coordinate conversion parameters in a coordinate conversion relation according to camera imaging difference and sensor position offset by using image coordinates and lens focal lengths of homologous points of other channels as coordinate parameters before conversion and using feature point image coordinates and lens focal lengths of a reference channel as coordinate parameters after conversion;
and converting other channels in all the shot multi-channel images into reference channels according to the coordinate conversion parameters, and superposing to obtain shot matching images.
2. The method for multi-channel image matching of a multi-spectral camera according to claim 1, wherein the reference channel is a near infrared channel and the other channels are red, blue, green and red channels.
3. The multi-channel image matching method for the multispectral camera as claimed in claim 1, wherein the coordinate transformation parameters of the camera imaging differences comprise a rotation matrix of other channels with same name points transformed into spatial coordinates and a rotation matrix of spatial coordinates transformed into reference channels;
and the coordinate conversion parameters of the sensor position deviation comprise the space distance between the reference channel and other channel images and a rotation matrix for converting the space coordinates into the reference channel.
4. The multi-channel image matching method for multispectral camera as claimed in claim 3, wherein the determining the coordinate transformation parameters in the coordinate transformation relationship according to the camera imaging difference and the sensor position offset comprises determining according to the following formula:
wherein x iss、ysAnd fsThe line number, the column number and the lens focal length x of the characteristic point corresponding to the reference channel on the imaget、ytAnd ftRespectively representing the uplink, the column number and the lens focal length of the image of the same name point in the image of other channels; in the coordinate transformation parameters, alpha and beta are correction coefficients Cx、CyAnd CzThe distance in space X, Y, Y between the reference image and the other channel images,
Rsthe rotation matrix is converted from the space coordinate to the reference channel, and the rotation matrix is converted from the homonymy points of other channels to the space coordinate.
5. The multi-channel image matching method for multispectral camera as claimed in claim 4, wherein:
accordingly, the coordinate conversion parameters include
ω、κ，Cx、Cy、Cz、α、β；
Wherein the content of the first and second substances,
ωt、κtand
ωs、κsare respectively as
And RsFlight course inclination angle, side inclination angle and sensor rotation angle。
6. The method for multi-channel image matching of multi-spectral camera according to claim 1, wherein said obtaining a predetermined number of sets of multi-channel images comprises:
randomly selecting an image obtained by shooting m times from n times of shooting obtained by one-time unmanned aerial vehicle operation as a multi-channel image with a preset group number m;
wherein m is greater than 30% n.
7. The multi-channel image matching method for multispectral camera as claimed in claim 5, wherein said determining the coordinate transformation parameters in the coordinate transformation relationship comprises:
determining the interpretation function of the measured value Y for a given parameter X:
determining a posterior distribution P (X | Y) of the parameters according to the flame function;
taking the mean value of the posterior distribution as the estimated value of the parameter;
wherein n represents the number of times of estimation, f (X) is the estimated value, yiIs the ith true value.
8. A multi-channel image matching apparatus of a multispectral camera, comprising:
the characteristic extraction module is used for acquiring multi-channel images with preset groups, extracting characteristic points of each image, and matching the characteristic points of all the multi-channel images to obtain corresponding points of the reference channel and other channels;
the parameter estimation module is used for determining coordinate conversion parameters in a coordinate conversion relation by taking the image coordinates and the lens focal length of the homonymous points of other channels as coordinate parameters before conversion and taking the characteristic point image coordinates and the lens focal length of the reference channel as coordinate parameters after conversion;
and the image matching module is used for converting other channels in all the shot multi-channel images into reference channels according to the coordinate conversion parameters and superposing the reference channels to obtain shot matching images.
9. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor when executing the program realizes the steps of the multi-channel image matching method of the multispectral camera according to any one of claims 1 to 7.
10. A non-transitory computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the method for multi-channel image matching of a multi-spectral camera according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011373528.5A CN112419379A (en) | 2020-11-30 | 2020-11-30 | Multi-channel image matching method and device of multispectral camera |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011373528.5A CN112419379A (en) | 2020-11-30 | 2020-11-30 | Multi-channel image matching method and device of multispectral camera |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112419379A true CN112419379A (en) | 2021-02-26 |
Family
ID=74828846
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011373528.5A Pending CN112419379A (en) | 2020-11-30 | 2020-11-30 | Multi-channel image matching method and device of multispectral camera |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112419379A (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113763485A (en) * | 2021-09-28 | 2021-12-07 | 北京的卢深视科技有限公司 | Temperature drift coefficient acquisition method, electronic device, storage medium, and image correction method |
| CN114648454A (en) * | 2022-03-04 | 2022-06-21 | 南京图格医疗科技有限公司 | Image correction method and system of multi-sensor camera |
| CN114894306A (en) * | 2022-07-13 | 2022-08-12 | 济南量子技术研究院 | Up-conversion array camera and imaging method thereof |
| CN114913088A (en) * | 2022-05-10 | 2022-08-16 | 宁波力显智能科技有限公司 | Multichannel alignment method and device for microscopic imaging and computer equipment |
| CN116597184A (en) * | 2023-07-11 | 2023-08-15 | 中国人民解放军63921部队 | Least square image matching method |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104217209A (en) * | 2013-06-03 | 2014-12-17 | 核工业北京地质研究院 | Method for automatically eliminating wrongly-matched registration points in remote sensing image |
| CN104570654A (en) * | 2013-10-25 | 2015-04-29 | 日本冲信息株式会社 | Image forming apparatus |
| CN106778588A (en) * | 2016-12-09 | 2017-05-31 | 国家测绘地理信息局四川测绘产品质量监督检验站 | State of flight detection method and device based on same place |
| US20190251237A1 (en) * | 2018-02-12 | 2019-08-15 | Samsung Electronics Co., Ltd. | Device and method with image matching |
-
2020
- 2020-11-30 CN CN202011373528.5A patent/CN112419379A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104217209A (en) * | 2013-06-03 | 2014-12-17 | 核工业北京地质研究院 | Method for automatically eliminating wrongly-matched registration points in remote sensing image |
| CN104570654A (en) * | 2013-10-25 | 2015-04-29 | 日本冲信息株式会社 | Image forming apparatus |
| CN106778588A (en) * | 2016-12-09 | 2017-05-31 | 国家测绘地理信息局四川测绘产品质量监督检验站 | State of flight detection method and device based on same place |
| US20190251237A1 (en) * | 2018-02-12 | 2019-08-15 | Samsung Electronics Co., Ltd. | Device and method with image matching |
Non-Patent Citations (5)
| Title |
|---|
| 刘加林;王慧琴;王可;吴萌;赵丽娟;张小红;: "基于快速稳健特征最大子矩阵的光谱图像配准方法", 激光与光电子学进展, no. 06, 29 October 2018 (2018-10-29) * |
| 李银伟;向茂生;韦立登;: "基于成像和相干信息的InSAR图像同名点提取方法", 电子与信息学报, no. 09, 15 September 2013 (2013-09-15) * |
| 程明明;王贺;安平;张洋;张兆杨;: "基于特征点匹配的多视图像校正", 液晶与显示, no. 04, 15 August 2010 (2010-08-15) * |
| 邹松;唐娉;胡昌苗;单小军;: "基于三维重建的大区域无人机影像全自动拼接方法", 计算机工程, no. 04, 15 April 2019 (2019-04-15) * |
| 韩杰;谢勇;吴国玺;喻铮铮;钱跃磊;关小果;: "顾及多相机拼接成像特征的高分一号卫星影像自适应匹配方法", 国土资源遥感, no. 04, 14 November 2017 (2017-11-14) * |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113763485A (en) * | 2021-09-28 | 2021-12-07 | 北京的卢深视科技有限公司 | Temperature drift coefficient acquisition method, electronic device, storage medium, and image correction method |
| CN114648454A (en) * | 2022-03-04 | 2022-06-21 | 南京图格医疗科技有限公司 | Image correction method and system of multi-sensor camera |
| CN114648454B (en) * | 2022-03-04 | 2024-04-02 | 南京图格医疗科技有限公司 | Image correction method and system for multi-sensor camera |
| CN114913088A (en) * | 2022-05-10 | 2022-08-16 | 宁波力显智能科技有限公司 | Multichannel alignment method and device for microscopic imaging and computer equipment |
| CN114913088B (en) * | 2022-05-10 | 2024-07-12 | 宁波力显智能科技有限公司 | Multichannel alignment method and device for microscopic imaging and computer equipment |
| CN114894306A (en) * | 2022-07-13 | 2022-08-12 | 济南量子技术研究院 | Up-conversion array camera and imaging method thereof |
| CN114894306B (en) * | 2022-07-13 | 2022-09-13 | 济南量子技术研究院 | Up-conversion array camera and imaging method thereof |
| CN116597184A (en) * | 2023-07-11 | 2023-08-15 | 中国人民解放军63921部队 | Least square image matching method |
| CN116597184B (en) * | 2023-07-11 | 2023-09-22 | 中国人民解放军63921部队 | Least square image matching method |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112419379A (en) | Multi-channel image matching method and device of multispectral camera | |
| US10417750B2 (en) | Image processing method, device and photographic apparatus | |
| US7865031B2 (en) | Method and system for automatic correction of chromatic aberration | |
| US9282326B2 (en) | Interactive camera calibration tool | |
| CN107483774B (en) | Imaging device and vehicle | |
| US20160286138A1 (en) | Apparatus and method for stitching panoramaic video | |
| EP2836984B1 (en) | Post-detection artifact reduction and removal from images | |
| CN110660105B (en) | Calibration parameter optimization method and device for panoramic looking-around system | |
| CN109523585B (en) | Multisource remote sensing image feature matching method based on direction phase consistency | |
| CN107274380B (en) | A kind of quick joining method of unmanned plane multispectral image | |
| JP2020523703A5 (en) | ||
| CN112070845A (en) | Calibration method and device of binocular camera and terminal equipment | |
| CN110986998B (en) | Satellite video camera on-orbit geometric calibration method based on rational function model | |
| WO2020114433A1 (en) | Depth perception method and apparatus, and depth perception device | |
| EP3633602A1 (en) | Image processing method, image processing apparatus, and program | |
| CN112106346A (en) | Image processing method, device, unmanned aerial vehicle, system and storage medium | |
| CN113808217A (en) | Real-time laser radar and camera calibration error self-correction method and system | |
| CN116170569A (en) | Hyperspectral image forward shooting correction method and device and storage medium | |
| CN116167921B (en) | Method and system for splicing panoramic images of flight space capsule | |
| CN111127529B (en) | Image registration method and device, storage medium and electronic device | |
| WO2023016468A1 (en) | De-pixelating method, electronic device and storage medium | |
| CN112017108B (en) | Satellite image color relative correction method based on adjustment of independent model method | |
| CN109658459A (en) | Camera calibration method, device, electronic equipment and computer readable storage medium | |
| US11642789B2 (en) | Automatic calibration method and device for robot vision system | |
| CN114782274A (en) | Unmanned aerial vehicle multispectral image radiation correction method based on illumination estimation |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |