CN111401203A - Target identification method based on multi-dimensional image fusion - Google Patents

Target identification method based on multi-dimensional image fusion Download PDF

Info

Publication number
CN111401203A
CN111401203A CN202010165922.3A CN202010165922A CN111401203A CN 111401203 A CN111401203 A CN 111401203A CN 202010165922 A CN202010165922 A CN 202010165922A CN 111401203 A CN111401203 A CN 111401203A
Authority
CN
China
Prior art keywords
image
pyramid
fusion
aplace
reference image
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202010165922.3A
Other languages
Chinese (zh)
Inventor
李良福
刘培祯
王娇颖
高强
钱钧
周国良
何曦
侯瑞
刘轩
王超
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Xian institute of Applied Optics
Original Assignee
Xian institute of Applied Optics
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Xian institute of Applied Optics filed Critical Xian institute of Applied Optics
Priority to CN202010165922.3A priority Critical patent/CN111401203A/en
Publication of CN111401203A publication Critical patent/CN111401203A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/10—Terrestrial scenes
    • G06V20/13—Satellite images
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/25—Fusion techniques
    • G06F18/251—Fusion techniques of input or preprocessed data

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • General Physics & Mathematics (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Evolutionary Biology (AREA)
  • Evolutionary Computation (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • General Engineering & Computer Science (AREA)
  • Artificial Intelligence (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Astronomy & Astrophysics (AREA)
  • Remote Sensing (AREA)
  • Multimedia (AREA)
  • Image Analysis (AREA)

Abstract

The invention belongs to the technical field of artificial intelligence and computer vision, and particularly relates to a target identification method based on multi-dimensional image fusion. According to the fusion algorithm of the Laplacian pyramid decomposition structure based on multi-resolution analysis, correlation between imaging characteristics and image features of different sensors is researched, and the fusion algorithm of the Laplacian pyramid decomposition structure based on multi-resolution analysis is realized. By using pyramidal decomposition of the image, objects of different sizes in the image can be analyzed. Meanwhile, information obtained by analyzing the lower layer with high resolution can be used for guiding the analysis of the upper layer with low resolution, so that the analysis and calculation can be greatly simplified. Because the image fusion method of the invention accords with the natural reality condition in the complex environment of the battlefield, compared with other existing image fusion methods, the invention has the characteristics of good fusion effect, rich details and the like.

Description

Target identification method based on multi-dimensional image fusion
Technical Field
The invention belongs to the technical field of artificial intelligence and computer vision, and particularly relates to a target identification method based on multi-dimensional image fusion.
Background
The image destination identification is realized by comparing the stored target information with the current image information. Description of an image is a prerequisite for object recognition, and by representing relevant features of individual objects in the image or scene, or even relationships between objects, using numbers or symbols, an abstract representation of the features of the objects and their relationships is obtained. When the image recognition technology is used for extracting the individual features in the image, a template matching model can be adopted. In some specific applications, image recognition needs to give the position of an object in addition to what the object to be recognized is. Currently, image recognition technology is widely applied in various fields, such as biomedicine, satellite remote sensing, robot vision, cargo detection, target tracking, autonomous vehicle navigation, public security, banking, transportation, military, electronic commerce, multimedia network communication, and the like. With the rapid development of artificial intelligence and computer vision technologies, target recognition based on machine vision, target recognition based on deep learning and the like appear, and the accuracy and recognition efficiency of image recognition are greatly improved.
However, there are disadvantages to the image information acquired by a single band sensor. For example, visible images are rich in detail, but cannot be imaged at night or in weak light; the infrared image can be imaged for 24 hours, but the distribution of the temperature of the object is obtained, and the observation of the details cannot be realized. By adopting an image fusion means, the multiband information of a single sensor or the information provided by different sensors can be integrated, and the redundancy and contradiction possibly existing among the information of the multiple sensors can be eliminated, so that the information transparency in the image can be enhanced, the interpretation precision, reliability and utilization rate can be improved, and clear, complete and accurate information description of the target can be formed. The efficient image fusion method can comprehensively process the information of the multi-source channel according to the needs, thereby effectively improving the utilization rate of the image information, the reliability of the system on target identification and the automation degree of the system.
In unmanned reconnaissance aircraft, vehicle-mounted panoramic situation perception, ship-borne photoelectric searching and tracking and other systems, the target identification technology based on multi-dimensional image fusion can meet multiple requirements of a military photoelectric system, the automatic and intelligent perception capability of an external scene is achieved, meanwhile, the target identification technology based on multi-dimensional image fusion has wide application in aerial survey and industrial measurement in the civil field, and therefore huge social benefits and economic benefits can be brought to the military and reconnaissance fields in China.
China journal command control and simulation 2019, Vol.28, No.1, pp.1-5 publishes a paper entitled "sea battlefield image target recognition based on deep learning", and Author Suiping et al analyze the advantages and defects of an R-CNN series model based on regional suggestion and a YO L O model based on regression in the paper, and comb up the application status of the deep learning technology in the sea battlefield image target recognition.
Disclosure of Invention
Technical problem to be solved
The technical problem to be solved by the invention is as follows: in order to meet the target identification requirement under a complex environment, how to provide a target identification method based on multi-dimensional image fusion for an unmanned system.
(II) technical scheme
In order to solve the above technical problem, the present invention provides a target identification method based on multi-dimensional image fusion, wherein the method comprises:
step 1: preprocessing the image; the method comprises the following steps:
step 11: calculating an image relative parameter transformation matrix;
when receiving an identification command sent by the unmanned reconnaissance system device, acquiring a visible light image as a reference image g through a corresponding sensorBInfrared image as candidate image gc；
Assuming that the number of matching points is N, the number of N is at least 3, and assuming that N is 3, 3 feature points, each B, are selected from the reference image1，B2And B3(ii) a In the candidate image gcIn the method, 3 corresponding matching points are selected, and are respectively C1，C2And C3；
For reference image gBAnd candidate image gCN pairs of matching points are found in the two multi-source images; reference image gBAnd candidate image gCRelative parameter transformation matrix P betweenC←BThe following least square method formula is adopted for calculation:
PC←B＝C*BT*(B*BT)-1
Figure BDA0002407456030000031
Figure BDA0002407456030000032
where C denotes odd coordinates of a matching point in the candidate image coordinate system, C is a 3 × N matrix, B denotes odd coordinates of a matching point in the reference image coordinate system, B is a 3 × N matrix, and B isTFor the transpose matrix of B, the relative parameter transform matrix PC←BIs 3 × 3 matrix;
step 12: transforming the candidate image into a reference image coordinate system;
from the reference image g, using inverse mappingBStarting from this, the reference image g is solved by a transformation functionBOn each pixel point in the candidate image gCA corresponding position on; from the reference image gBIn each point B0The odd coordinates of (g) can be calculated according to the following formulaCPoint C corresponding thereto0Odd-order coordinates of (c):
Figure BDA0002407456030000033
Figure BDA0002407456030000034
wherein, B0Has a horizontal coordinate of
Figure BDA0002407456030000035
The value range is (1,2, …, W); vertical coordinate is
Figure BDA0002407456030000036
The value range is (1,2, …, H); c0Has a horizontal coordinate of
Figure BDA0002407456030000037
The value range is (1,2, …, W); vertical coordinate is
Figure BDA0002407456030000041
The value range is (1,2, …, H);
candidate image gCMidpoint C0Giving the reference image g a pixel gray value ofBCorresponding pixel point B0Then, a transformed candidate image g is obtainedB←CAnd combine the image gB←CAs output and reference image gBCarrying out fusion;
step 2: performing image fusion on the image based on the multi-source characteristics; the method comprises the following steps:
a reference image gBAnd the transformed candidate image gB←CPerforming fusion by adoptingThe image fusion algorithm of the multi-resolution analysis pyramid decomposition structure is realized by the following steps:
step 21: carrying out Gauss tower type decomposition on the image;
for a reference image g as a source imageBIn the order of G0As the zero layer of the Gaussian pyramid, the image G of the first layer of the Gaussian pyramidlComprises the following steps:
Figure BDA0002407456030000042
in the formula: n1 is the top layer number of Gauss pyramid; clThe number of columns of the image of the l layer of the Gauss pyramid is obtained; rlW (m, n) represents a 5 × 5 window function, and has a low-pass characteristic, which is as follows:
Figure BDA0002407456030000043
step 22, establishing L aplace pyramid of the image;
Figure BDA0002407456030000044
wherein:
Figure BDA0002407456030000045
in the formula (I), the compound is shown in the specification,
Figure BDA0002407456030000046
is composed of GlInterpolated and enlarged image, size and Gl-1Are the same size, but
Figure BDA0002407456030000047
And Gl-1The values of the gray values of the new pixels interpolated between the original pixels are determined by a weighted average of the gray values of the original pixels; due to GlIs to Gl-1Obtained by low-pass filtering, i.e. GlIs fuzzificationDown-sampled Gl-1Thus, therefore, it is
Figure BDA0002407456030000051
Ratio of detail information of Gl-1Less;
thus, a decomposed image L P of L aplace pyramid layers was obtainedl：
Figure BDA0002407456030000052
Wherein N2 is layer number of L aplace pyramid top layer, L PlA layer i image representing L aplace pyramid decomposition;
step 23, reconstructing a source image by L aplace pyramid;
by transformation of the above formula
Figure BDA0002407456030000053
Step 24, fusing images based on L aplace pyramid decomposition;
and setting A and B as two source images and F as a fused image, wherein the fusion process is as follows:
241, performing L aplace pyramid decomposition on each source image to establish respective L aplace pyramids;
242, respectively fusing all decomposition layers of the image pyramid to obtain an L aplace pyramid of the fused image;
step 243, carrying out image reconstruction on the fused L aplace pyramid to obtain a final fused image;
and step 3: carrying out target identification; the method comprises the following steps:
step 31: carrying out big data annotation;
selecting a rectangular area by using a marking tool for M collected big data images, defining the label of a background area as 0, and defining the label of a target area as 1; classifying to form a training set and a verification set which have a certain scale and are used for training the deep learning model, and realizing the identification of the class 1 target; the number of M is at least 12000;
step 32: carrying out data training;
training a target classification model by using the data set marked in the previous step;
step 33: performing real-time image fusion;
collecting a visible light image and an infrared image in real time, and fusing to obtain a fused image;
step 34: carrying out target identification and positioning;
and carrying out ship target detection on the fusion image obtained in the last step by using the trained classification model, recording the size and the position of all ship targets identified by the classification model, and identifying the rectangular area on the image by using a rectangular identification frame.
(III) advantageous effects
Compared with the prior art, the invention has the following beneficial effects:
(1) the invention provides a method for fusion based on multi-dimensional image information. By researching the correlation between the imaging characteristics and the image characteristics of different sensors, the fusion algorithm of the Laplacian pyramid decomposition structure based on multi-resolution analysis is realized. With the pyramidal decomposition of the image, objects of different sizes in the image can be analyzed, e.g., high resolution layers (lower layers) can be used to analyze details and low resolution layers (top layers) can be used to analyze larger objects. Meanwhile, information obtained by analyzing the lower layer with high resolution can be used for guiding the analysis of the upper layer with low resolution, so that the analysis and calculation can be greatly simplified. The image tower decomposition provides a convenient and flexible image multi-resolution analysis method, and the image Laplacian decomposition can decompose the importance (such as edges) of the image to different tower decomposition layers according to different scales. Because the image fusion method of the invention accords with the natural reality condition in the complex environment of the battlefield, compared with other existing image fusion methods, the invention has the characteristics of good fusion effect, rich details and the like.
(2) In the invention, the fused image is subjected to target identification by adopting a convolutional neural network based on deep learning, and the method has the characteristics of accurate identification, strong scale and illumination change resistance and the like.
(3) In the present invention, feature points are used for heterogeneous image registration. And the least square method is adopted to calculate the image transformation parameters, so that the method has the advantages of high calculation precision, high speed, good fusion recognition effect and the like.
Drawings
Fig. 1(a) -1 (d) are graphs showing the results of multi-dimensional image fusion experiments. Wherein, fig. 1(a) is a multi-dimensional image fusion software start interface; FIG. 1(b) is an interface of the multi-dimensional image fusion software after loading the image video; FIG. 1(c) is a visible light and infrared image registration point selection interface; fig. 1(d) is a multi-dimensional image fusion result.
FIG. 2 is a flowchart illustrating the operation of the method for identifying an object based on multi-dimensional image fusion according to the present invention.
FIG. 3 is a schematic diagram of image fusion based on pyramid decomposition according to the present invention.
Fig. 4(a) and 4(b) are graphs of experimental results of object recognition on a ship object image video according to a preferred embodiment of the present invention.
Detailed Description
In order to make the objects, contents, and advantages of the present invention clearer, the following detailed description of the embodiments of the present invention will be made in conjunction with the accompanying drawings and examples.
In order to meet the target identification requirement in a complex environment, the invention provides a target identification method based on multi-dimensional image fusion for an unmanned system.
The method mainly aims to fuse multiband video image sequences by adopting an image fusion method and finally intelligently identify a target by adopting a deep learning method. It follows that a sequence of video images is the object of the invention that needs to be processed. The target identification technology based on multi-dimensional image fusion is used for labeling, training and learning multi-band fusion image big data and automatically identifying a plurality of targets in an image.
The image acquisition device adopts a full-Rui video visible light camera, and adopts an EXA-IR2 uncooled thermal imager for infrared. The computer hardware in the preferred embodiment adopts an I7-7700 processor, the dominant frequency is 2.80G, the size of the hard disk is 1T, and the target identification algorithm calculation for one frame of image by using the computer only needs about 0.1 second. And collecting 1 frame of visible light image and 1 frame of infrared image to perform image fusion operation. The visible light image is assumed to be a reference image, and the infrared image is assumed to be a candidate image. The basic flow of the image fusion technology is that firstly, a reference image and a candidate image are registered, so that a mathematical transformation model of the two images is established; and then, according to the established mathematical transformation model, carrying out unified coordinate transformation, namely transforming all candidate image sequences into a coordinate system of the reference image so as to form a fused image. For two images, due to the existence of scale, rotation and translation transformation, the transformation relation needs at least 3 pairs of matching points to solve.
The target identification method based on multi-dimensional image fusion provided by the preferred embodiment of the present invention completes the real-time identification of the image according to the workflow shown in fig. 2, and the identification process includes the following four major contents.
Specifically, the target identification method based on multi-dimensional image fusion provided by the invention comprises the following steps:
step 1: preprocessing the image; the method comprises the following steps:
step 11: calculating an image relative parameter transformation matrix;
when receiving an identification command sent by the unmanned reconnaissance system device, acquiring a visible light image as a reference image g through a corresponding sensorBInfrared image as candidate image gc；
Assuming that the number of matching points is N, the number of N is at least 3, and assuming that N is 3, 3 feature points, each B, are selected from the reference image1，B2And B3(ii) a In the candidate image gcIn the method, 3 corresponding matching points are selected, and are respectively C1，C2And C3；
For reference image gBAnd candidate image gCN pairs of matching points are found in the two multi-source images; if N ≧ N3, then reference image gBAnd candidate image gCRelative parameter transformation matrix P betweenC←BThe following least square method formula is adopted for calculation:
PC←B＝C*BT*(B*BT)-1
Figure BDA0002407456030000081
Figure BDA0002407456030000082
where C denotes odd coordinates of a matching point in the candidate image coordinate system, C is a 3 × N matrix, B denotes odd coordinates of a matching point in the reference image coordinate system, B is a 3 × N matrix, and B isTFor the transpose matrix of B, the relative parameter transform matrix PC←BIs 3 × 3 matrix;
step 12: transforming the candidate image into a reference image coordinate system;
due to the reference image gBAnd candidate image gCThere is a certain transformation relation between them, so it is necessary to transform them into the same coordinate system for image fusion. In the present invention, the candidate image g needs to be extractedCConversion to reference image gBIn a coordinate system.
From the reference image g, using inverse mappingBStarting from this, the reference image g is solved by a transformation functionBOn each pixel point in the candidate image gCA corresponding position on; from the reference image gBIn each point B0The odd coordinates of (g) can be calculated according to the following formulaCPoint C corresponding thereto0Odd-order coordinates of (c):
Figure BDA0002407456030000091
Figure BDA0002407456030000092
wherein, B0Has a horizontal coordinate of
Figure BDA0002407456030000093
The value range is (1,2, …, W); vertical coordinate is
Figure BDA0002407456030000094
The value range is (1,2, …, H); c0Has a horizontal coordinate of
Figure BDA0002407456030000095
The value range is (1,2, …, W); vertical coordinate is
Figure BDA0002407456030000096
The value range is (1,2, …, H);
candidate image gCMidpoint C0Pixel gray value gc(i, j) assigning a reference image gBCorresponding pixel point B0Then, a transformed candidate image g is obtainedB←CAnd combine the image gB←CAs output and reference image gBCarrying out fusion;
generally, image transformation can adopt two mapping modes: forward mapping and reverse mapping; the forward mapping is to transform the candidate image to the coordinate space where the reference image is located according to the calculated image transformation parameters; i.e. each pixel of the candidate image is scanned and the position of each pixel corresponding to the reference image is calculated in turn by means of the transformation function. When two adjacent pixel points of the candidate image are mapped to two non-adjacent pixel points of the reference image, discrete mosaic and virtual point holes occur. Therefore, a conversion idea is required, which can use a reverse idea to find the coordinates of the candidate image corresponding to each point of the reference image. The reverse mapping is from the reference image gBStarting from this, the reference image g is solved by a transformation functionBOn each pixel point in the candidate image gCTo the corresponding position on. First, a reference image g is scannedBThen according to the transformation function, calculating the candidate image gCCorresponding sampled pixel ofPoints and assigning the gray value of the point to the reference image gBThe corresponding pixel point of (2).
The reverse mapping is better than the forward mapping because each pixel of the reference image can be scanned to obtain an appropriate gray value, thereby avoiding the situation that some points of the output image in the forward mapping may not be assigned and virtual point holes and mosaics appear.
Step 2: performing image fusion on the image based on the multi-source characteristics; the method comprises the following steps:
a reference image gBAnd the transformed candidate image gB←CThe fusion is carried out, and an image fusion algorithm of a pyramid decomposition structure based on multi-resolution analysis is adopted, and the method comprises the following steps:
step 21: carrying out Gauss tower type decomposition on the image;
for a reference image g as a source imageBIn the order of G0As the zero layer (bottom layer) of the Gaussian pyramid, the image G of the first layer of the Gaussian pyramidlComprises the following steps:
Figure BDA0002407456030000101
in the formula: n1 is the top layer number of Gauss pyramid; clThe number of columns of the image of the l layer of the Gauss pyramid is obtained; rlW (m, n) represents a 5 × 5 window function (generating kernel) with a low-pass characteristic, which is as follows:
Figure BDA0002407456030000102
step 22, establishing L aplace pyramid of the image;
Figure BDA0002407456030000103
wherein:
Figure BDA0002407456030000104
in the formula (I), the compound is shown in the specification,
Figure BDA0002407456030000105
is composed of GlInterpolated and enlarged image, size and Gl-1Are the same size, but
Figure BDA0002407456030000106
And Gl-1The values of the gray values of the new pixels interpolated between the original pixels are determined by a weighted average of the gray values of the original pixels; due to GlIs to Gl-1Obtained by low-pass filtering, i.e. GlIs fuzzification, down-sampling Gl-1Thus, therefore, it is
Figure BDA0002407456030000111
Ratio of detail information of Gl-1Less;
thus, a decomposed image L P of L aplace pyramid layers was obtainedl：
Figure BDA0002407456030000112
Wherein N2 is layer number of L aplace pyramid top layer, L PlA layer i image representing L aplace pyramid decomposition;
step 23, reconstructing a source image by L aplace pyramid;
by transformation of the above formula
Figure BDA0002407456030000113
Step 24, fusing images based on L aplace pyramid decomposition;
an image fusion method based on L aplace pyramid decomposition is shown in FIG. 3, wherein A and B are two source images, F is a fused image, and the fusion process is as follows:
241, performing L aplace pyramid decomposition on each source image to establish respective L aplace pyramids;
242, respectively fusing all decomposition layers of the image pyramid to obtain an L aplace pyramid of the fused image;
step 243, carrying out image reconstruction on the fused L aplace pyramid to obtain a final fused image;
and step 3: carrying out target identification; the method comprises the following steps:
step 31: carrying out big data annotation;
selecting a rectangular area by using a marking tool for M collected big data images, defining label of a background area as 0, and defining an area label of a ship target as 1; classifying to form a training set and a verification set which have a certain scale and are used for training a deep learning model, and realizing the identification of the class 1 ship target; the number of M is at least 12000;
step 32: carrying out data training;
training a ship target classification model by using the data set marked in the previous step;
step 33: performing real-time image fusion;
collecting a visible light image and an infrared image in real time, and fusing to obtain a fused image;
step 34: carrying out target identification and positioning;
and carrying out ship target detection on the fusion image obtained in the last step by using the trained classification model, recording the size and the position of all ship targets identified by the classification model, and identifying the rectangular area on the image by using a rectangular identification frame.
Fig. 4(a) and 4(b) show experimental results of the object recognition based on multi-dimensional image fusion using the present preferred embodiment. Where there are two ship targets in fig. 4(a) and one ship target in fig. 4 (b). It can be seen that the invention has better target identification effect because of adopting the target identification method based on multi-dimensional image fusion.
The above description is only a preferred embodiment of the present invention, and it should be noted that, for those skilled in the art, several modifications and variations can be made without departing from the technical principle of the present invention, and these modifications and variations should also be regarded as the protection scope of the present invention.

Claims (1)

1. A target identification method based on multi-dimensional image fusion is characterized by comprising the following steps:
step 1: preprocessing the image; the method comprises the following steps:
step 11: calculating an image relative parameter transformation matrix;
when receiving an identification command sent by the unmanned reconnaissance system device, acquiring a visible light image as a reference image g through a corresponding sensorBInfrared image as candidate image gc；
Assuming that the number of matching points is N, the number of N is at least 3, and assuming that N is 3, 3 feature points, each B, are selected from the reference image1，B2And B3(ii) a In the candidate image gcIn the method, 3 corresponding matching points are selected, and are respectively C1，C2And C3；
For reference image gBAnd candidate image gCN pairs of matching points are found in the two multi-source images; reference image gBAnd candidate image gCRelative parameter transformation matrix P betweenC←BThe following least square method formula is adopted for calculation:
PC←B＝C*BT*(B*BT)-1
Figure FDA0002407456020000011
Figure FDA0002407456020000012
where C denotes odd coordinates of a matching point in the candidate image coordinate system, C is a 3 × N matrix, B denotes odd coordinates of a matching point in the reference image coordinate system, B is a 3 × N matrix, and B isTFor the transpose matrix of B, the relative parameter transform matrix PC←BIs 3 × 3 matrix;
step 12: transforming the candidate image into a reference image coordinate system;
from the reference image g, using inverse mappingBStarting from this, the reference image g is solved by a transformation functionBOn each pixel point in the candidate image gCA corresponding position on; from the reference image gBIn each point B0The odd coordinates of (g) can be calculated according to the following formulaCPoint C corresponding thereto0Odd-order coordinates of (c):
Figure FDA0002407456020000021
Figure FDA0002407456020000022
wherein, B0Has a horizontal coordinate of
Figure FDA0002407456020000023
The value range is (1,2, …, W); vertical coordinate is
Figure FDA0002407456020000024
The value range is (1,2, …, H); c0Has a horizontal coordinate of
Figure FDA0002407456020000025
The value range is (1,2, …, W); vertical coordinate is
Figure FDA0002407456020000026
The value range is (1,2, …, H);
candidate image gCMidpoint C0Giving the reference image g a pixel gray value ofBCorresponding pixel point B0Then, a transformed candidate image g is obtainedB←CAnd combine the image gB←CAs output and reference image gBCarrying out fusion;
step 2: performing image fusion on the image based on the multi-source characteristics; the method comprises the following steps:
a reference image gBAnd the transformed candidate image gB←CThe fusion is carried out, and an image fusion algorithm of a pyramid decomposition structure based on multi-resolution analysis is adopted, and the method comprises the following steps:
step 21: carrying out Gauss tower type decomposition on the image;
for a reference image g as a source imageBIn the order of G0As the zero layer of the Gaussian pyramid, the image G of the first layer of the Gaussian pyramidlComprises the following steps:
Figure FDA0002407456020000027
0≤l≤N1，0≤i<Cl，0≤j<Rl；
in the formula: n1 is the top layer number of Gauss pyramid; clThe number of columns of the image of the l layer of the Gauss pyramid is obtained; rlW (m, n) represents a 5 × 5 window function, and has a low-pass characteristic, which is as follows:
Figure FDA0002407456020000028
step 22, establishing L aplace pyramid of the image;
Figure FDA0002407456020000031
0≤l≤N2，0≤i<Cl，0≤j<Rl；
wherein:
Figure FDA0002407456020000032
in the formula (I), the compound is shown in the specification,
Figure FDA0002407456020000033
is composed of GlInterpolated and enlarged image, size and Gl-1Are the same size, but
Figure FDA0002407456020000034
And Gl-1The values of the gray values of the new pixels interpolated between the original pixels are determined by a weighted average of the gray values of the original pixels; due to GlIs to Gl-1Obtained by low-pass filtering, i.e. GlIs fuzzification, down-sampling Gl-1Thus, therefore, it is
Figure FDA0002407456020000035
Ratio of detail information of Gl-1Less;
thus, a decomposed image L P of L aplace pyramid layers was obtainedl：
Figure FDA0002407456020000036
Wherein N2 is layer number of L aplace pyramid top layer, L PlA layer i image representing L aplace pyramid decomposition;
step 23, reconstructing a source image by L aplace pyramid;
by transformation of the above formula
Figure FDA0002407456020000037
Step 24, fusing images based on L aplace pyramid decomposition;
and setting A and B as two source images and F as a fused image, wherein the fusion process is as follows:
241, performing L aplace pyramid decomposition on each source image to establish respective L aplace pyramids;
242, respectively fusing all decomposition layers of the image pyramid to obtain an L aplace pyramid of the fused image;
step 243, carrying out image reconstruction on the fused L aplace pyramid to obtain a final fused image;
and step 3: carrying out target identification; the method comprises the following steps:
step 31: carrying out big data annotation;
selecting a rectangular area by using a marking tool for M collected big data images, defining the label of a background area as 0, and defining the label of a target area as 1; classifying to form a training set and a verification set which have a certain scale and are used for training the deep learning model, and realizing the identification of the class 1 target; the number of M is at least 12000;
step 32: carrying out data training;
training a target classification model by using the data set marked in the previous step;
step 33: performing real-time image fusion;
collecting a visible light image and an infrared image in real time, and fusing to obtain a fused image;
step 34: carrying out target identification and positioning;
and carrying out ship target detection on the fusion image obtained in the last step by using the trained classification model, recording the size and the position of all ship targets identified by the classification model, and identifying the rectangular area on the image by using a rectangular identification frame.
CN202010165922.3A 2020-03-11 2020-03-11 Target identification method based on multi-dimensional image fusion Pending CN111401203A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202010165922.3A CN111401203A (en) 2020-03-11 2020-03-11 Target identification method based on multi-dimensional image fusion

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010165922.3A CN111401203A (en) 2020-03-11 2020-03-11 Target identification method based on multi-dimensional image fusion

Publications (1)

Publication Number Publication Date
CN111401203A true CN111401203A (en) 2020-07-10

Family

ID=71430666

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010165922.3A Pending CN111401203A (en) 2020-03-11 2020-03-11 Target identification method based on multi-dimensional image fusion

Country Status (1)

Country Link
CN (1) CN111401203A (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113283411A (en) * 2021-07-26 2021-08-20 中国人民解放军国防科技大学 Unmanned aerial vehicle target detection method, device, equipment and medium
CN114842427A (en) * 2022-03-31 2022-08-02 南京邮电大学 Intelligent traffic-oriented complex multi-target self-adaptive detection method

Citations (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102609928A (en) * 2012-01-12 2012-07-25 中国兵器工业第二0五研究所 Visual variance positioning based image mosaic method
CN103778616A (en) * 2012-10-22 2014-05-07 中国科学院研究生院 Contrast pyramid image fusion method based on area
US8885976B1 (en) * 2013-06-20 2014-11-11 Cyberlink Corp. Systems and methods for performing image fusion
CN104616273A (en) * 2015-01-26 2015-05-13 电子科技大学 Multi-exposure image fusion method based on Laplacian pyramid decomposition
CN106960428A (en) * 2016-01-12 2017-07-18 浙江大立科技股份有限公司 Visible ray and infrared double-waveband image co-registration Enhancement Method
CN107609601A (en) * 2017-09-28 2018-01-19 北京计算机技术及应用研究所 A kind of ship seakeeping method based on multilayer convolutional neural networks
CN109492700A (en) * 2018-11-21 2019-03-19 西安中科光电精密工程有限公司 A kind of Target under Complicated Background recognition methods based on multidimensional information fusion
CN109558848A (en) * 2018-11-30 2019-04-02 湖南华诺星空电子技术有限公司 A kind of unmanned plane life detection method based on Multi-source Information Fusion
CN110111581A (en) * 2019-05-21 2019-08-09 哈工大机器人（山东）智能装备研究院 Target identification method, device, computer equipment and storage medium
CN110322423A (en) * 2019-04-29 2019-10-11 天津大学 A kind of multi-modality images object detection method based on image co-registration

Patent Citations (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102609928A (en) * 2012-01-12 2012-07-25 中国兵器工业第二0五研究所 Visual variance positioning based image mosaic method
CN103778616A (en) * 2012-10-22 2014-05-07 中国科学院研究生院 Contrast pyramid image fusion method based on area
US8885976B1 (en) * 2013-06-20 2014-11-11 Cyberlink Corp. Systems and methods for performing image fusion
CN104616273A (en) * 2015-01-26 2015-05-13 电子科技大学 Multi-exposure image fusion method based on Laplacian pyramid decomposition
CN106960428A (en) * 2016-01-12 2017-07-18 浙江大立科技股份有限公司 Visible ray and infrared double-waveband image co-registration Enhancement Method
CN107609601A (en) * 2017-09-28 2018-01-19 北京计算机技术及应用研究所 A kind of ship seakeeping method based on multilayer convolutional neural networks
CN109492700A (en) * 2018-11-21 2019-03-19 西安中科光电精密工程有限公司 A kind of Target under Complicated Background recognition methods based on multidimensional information fusion
CN109558848A (en) * 2018-11-30 2019-04-02 湖南华诺星空电子技术有限公司 A kind of unmanned plane life detection method based on Multi-source Information Fusion
CN110322423A (en) * 2019-04-29 2019-10-11 天津大学 A kind of multi-modality images object detection method based on image co-registration
CN110111581A (en) * 2019-05-21 2019-08-09 哈工大机器人（山东）智能装备研究院 Target identification method, device, computer equipment and storage medium

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
李良福 等: "基于深度学习的光电***智能目标识别", 《兵工学报》, vol. 43, pages 162 - 168 *
江海军 等: "拉普拉斯金字塔融合在红外无损检测技术中的应用", 《红外技术》, vol. 41, no. 12, pages 1151 - 1155 *
韩潇 等: "基于改进拉普拉斯金字塔的图像融合方法", 《自动化与仪器仪表》, no. 5, pages 191 - 194 *

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113283411A (en) * 2021-07-26 2021-08-20 中国人民解放军国防科技大学 Unmanned aerial vehicle target detection method, device, equipment and medium
CN113283411B (en) * 2021-07-26 2022-01-28 中国人民解放军国防科技大学 Unmanned aerial vehicle target detection method, device, equipment and medium
CN114842427A (en) * 2022-03-31 2022-08-02 南京邮电大学 Intelligent traffic-oriented complex multi-target self-adaptive detection method

Similar Documents

Publication Publication Date Title
CN111862126A (en) Non-cooperative target relative pose estimation method combining deep learning and geometric algorithm
CN110728200A (en) Real-time pedestrian detection method and system based on deep learning
CN112883850B (en) Multi-view space remote sensing image matching method based on convolutional neural network
CN111582232A (en) SLAM method based on pixel-level semantic information
CN109977834B (en) Method and device for segmenting human hand and interactive object from depth image
CN110598564A (en) OpenStreetMap-based high-spatial-resolution remote sensing image transfer learning classification method
CN113160150A (en) AI (Artificial intelligence) detection method and device for invasion of foreign matters in wire network based on multi-type sample fusion and multi-complex network
CN115457396B (en) Surface target ground object detection method based on remote sensing image
CN115861619A (en) Airborne LiDAR (light detection and ranging) urban point cloud semantic segmentation method and system of recursive residual double-attention kernel point convolution network
CN111401203A (en) Target identification method based on multi-dimensional image fusion
CN114639115A (en) 3D pedestrian detection method based on fusion of human body key points and laser radar
CN116994135A (en) Ship target detection method based on vision and radar fusion
Sun et al. IRDCLNet: Instance segmentation of ship images based on interference reduction and dynamic contour learning in foggy scenes
CN117274627A (en) Multi-temporal snow remote sensing image matching method and system based on image conversion
Ji et al. Dbenet: Dual-branch ensemble network for sea-land segmentation of remote sensing images
Zhang et al. Optical and SAR image dense registration using a robust deep optical flow framework
CN114067273A (en) Night airport terminal thermal imaging remarkable human body segmentation detection method
Narayanan et al. A multi-purpose realistic haze benchmark with quantifiable haze levels and ground truth
Li et al. Progressive attention-based feature recovery with scribble supervision for saliency detection in optical remote sensing image
CN116486238B (en) Target fine granularity identification method combining point set representation and graph classification
Jiang et al. Semantic segmentation network combined with edge detection for building extraction in remote sensing images
CN109117852B (en) Unmanned aerial vehicle image adaptation area automatic extraction method and system based on sparse representation
CN113537397B (en) Target detection and image definition joint learning method based on multi-scale feature fusion
Zheng et al. Multiscale fusion network for rural newly constructed building detection in unmanned aerial vehicle imagery
Liangjun et al. MSFA-YOLO: A Multi-Scale SAR Ship Detection Algorithm Based on Fused Attention

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination