CN116343057A - Ship target detection and identification method combining SAR (synthetic aperture radar) with optical image - Google Patents
Ship target detection and identification method combining SAR (synthetic aperture radar) with optical image Download PDFInfo
- Publication number
- CN116343057A CN116343057A CN202310255329.1A CN202310255329A CN116343057A CN 116343057 A CN116343057 A CN 116343057A CN 202310255329 A CN202310255329 A CN 202310255329A CN 116343057 A CN116343057 A CN 116343057A
- Authority
- CN
- China
- Prior art keywords
- sar
- trans
- image
- evidence
- opt
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000003287 optical effect Effects 0.000 title claims abstract description 113
- 238000001514 detection method Methods 0.000 title claims abstract description 100
- 238000000034 method Methods 0.000 title claims abstract description 36
- 230000004927 fusion Effects 0.000 claims abstract description 20
- 238000013507 mapping Methods 0.000 claims abstract description 12
- 238000007500 overflow downdraw method Methods 0.000 claims abstract description 7
- 238000010586 diagram Methods 0.000 claims description 8
- 238000004364 calculation method Methods 0.000 claims description 7
- 230000001360 synchronised effect Effects 0.000 claims description 6
- 238000013145 classification model Methods 0.000 claims description 5
- 239000011159 matrix material Substances 0.000 claims description 4
- 238000012935 Averaging Methods 0.000 claims description 3
- 230000008901 benefit Effects 0.000 description 7
- 238000012549 training Methods 0.000 description 7
- 238000005516 engineering process Methods 0.000 description 6
- 238000011161 development Methods 0.000 description 5
- 238000011160 research Methods 0.000 description 5
- 238000004422 calculation algorithm Methods 0.000 description 4
- 238000012790 confirmation Methods 0.000 description 4
- 238000007667 floating Methods 0.000 description 3
- 238000003384 imaging method Methods 0.000 description 3
- 238000012545 processing Methods 0.000 description 3
- 238000012360 testing method Methods 0.000 description 2
- 230000000007 visual effect Effects 0.000 description 2
- 101100498818 Arabidopsis thaliana DDR4 gene Proteins 0.000 description 1
- BQCADISMDOOEFD-UHFFFAOYSA-N Silver Chemical compound [Ag] BQCADISMDOOEFD-UHFFFAOYSA-N 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 230000000295 complement effect Effects 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 238000005286 illumination Methods 0.000 description 1
- 230000001939 inductive effect Effects 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 238000010606 normalization Methods 0.000 description 1
- 230000010287 polarization Effects 0.000 description 1
- 229910052709 silver Inorganic materials 0.000 description 1
- 239000004332 silver Substances 0.000 description 1
- 238000004088 simulation Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/10—Terrestrial scenes
- G06V20/13—Satellite images
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/25—Determination of region of interest [ROI] or a volume of interest [VOI]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/42—Global feature extraction by analysis of the whole pattern, e.g. using frequency domain transformations or autocorrelation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V2201/00—Indexing scheme relating to image or video recognition or understanding
- G06V2201/07—Target detection
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Computing Systems (AREA)
- Software Systems (AREA)
- Medical Informatics (AREA)
- Databases & Information Systems (AREA)
- Astronomy & Astrophysics (AREA)
- Remote Sensing (AREA)
- Life Sciences & Earth Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Molecular Biology (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Image Processing (AREA)
Abstract
The invention belongs to the technical field of target detection and identification based on multi-source information fusion, and provides a ship target detection and identification method combining SAR and an optical image, which is used for improving target detection efficiency; firstly, a focus mechanism is introduced into a target detection model of an optical image, and better detection performance is obtained on the optical image; secondly, a SAR image slice is obtained by using a mapping relation between the image coordinates and the geographic coordinates, and the detection result of the optical image is secondarily confirmed, so that the interference of a pseudo target on the optical image is reduced, and the robustness of the model is improved; finally, a decision-level fusion method based on a D-S evidence theory of evidence association coefficients is introduced, probability distribution of target categories in the SAR image and the optical image is combined, collision evidence is effectively relieved, accuracy and reliability of target type identification are improved, interference capability of a model on a pseudo target is improved, and accuracy of pseudo target detection is improved to 97.5%.
Description
Technical Field
The invention belongs to the technical field of target detection and identification based on multi-source information fusion, and particularly provides an SAR and optical image ship target detection and identification method based on an improved D-S evidence theory.
Background
Remote sensing image target interpretation is one of important research directions in the fields of computer vision and image processing, and can be used for detecting specific class of object examples in digital images; because the image range that remote sensing sensor gathered is wide, therefore the target detection based on remote sensing image has wide application in military and civil fields such as intelligence reconnaissance, target monitoring, military strike, disaster relief, industrial development, etc.
At present, the mainstream target detection algorithm is based on a single data source such as an optical remote sensing image or an SAR remote sensing image, and the optical image becomes the most mainstream data source due to the advantages of rich color information, high resolution and strong semantic information. With the rapid development of target detection networks such as YOLO, the performance of a target detection method based on an optical remote sensing image is greatly improved; there are also some drawbacks and problems that are difficult to solve immediately: 1. the quality of the collected optical remote sensing data is greatly affected by the external environment, and under the conditions of bad weather, insufficient illumination or non-ideal shooting distance and angle and the like, the collected image is blurred, so that the semantic information of a key object instance in the digital image is lost; 2. because the optical remote sensing image is truly reflected to the ground color, the target detection algorithm based on the optical remote sensing image is low in robustness and reliability in terms of false target deception of the optical image, such as camouflage, stealth and the like.
Meanwhile, along with the continuous improvement of SAR imaging technology, the SAR imaging technology has the advantages of all weather, strong penetrability and rich texture information, and research on SAR remote sensing image target interpretation is also continuous and deep. The most suitable scenes of SAR images are ship target detection and building detection, but the detection and recognition performance in remote sensing target interpretation is behind the optical images due to poor readability and low resolution compared with the optical images and the interference of speckle noise and geometric distortion. To be able to better advantage of the optical sensor and the SAR sensor itself in practical tasks, researchers have migrated the idea of "multi-source information fusion" to the specific application of remote sensing target interpretation. With the rapid development and innovation of related disciplines such as sensing technology, wireless communication technology and aerospace technology in recent years, a large number of optical satellites and SAR satellites are successfully transmitted and operated worldwide, and more heterologous remote sensing data also provide a wide research space for researchers.
Currently, optical Ship data sets such as HRSC2016, xView, FGSD, DIOR, shipRSImageNet and SAR data sets such as SSDD, FUSAR-clip, AIR-SARShip, openSARShip and the like are all open sources, and possibility is provided for Ship target detection and identification of combined SAR and optical images. However, the following problems still remain in practical engineering applications: 1. because the SAR image and the optical image have different imaging modes, strict time-space synchronization is difficult to achieve, and great interference is caused to heterologous data fusion; 2. the optical image sensor and the SAR image sensor have different heights and angles for collecting data, and the SAR sensor has various polarization modes, so that the same target has various expression forms in different sensors, and the detection difficulty is increased; 3. SAR images in the current mainstream open source data set are usually acquired by satellite-borne satellites, and lack of detail information, so that the performance is poor when the SAR images are used for classifying and identifying tasks. On the basis, the remote sensing target interpretation technology based on multi-source information fusion is still a research hotspot and an important point, but a scheme for effectively combining an SAR sensor and an optical sensor is lacking in the current research, the target detection efficiency is low, and how to realize efficient fusion of optical remote sensing data and SAR remote sensing data is still an unsolved problem.
Disclosure of Invention
The invention aims to provide a ship target detection and identification method combining SAR and optical images, which is used for realizing the efficient fusion of optical remote sensing data and SAR remote sensing data, thereby improving the target detection efficiency of ship target detection and identification; firstly, a CBAM attention mechanism is introduced into a YOLO V7 target detection network model of an optical image, and better detection performance is obtained on the optical image; secondly, a SAR image slice is obtained by using a mapping relation between the image coordinates and the geographic coordinates, and a detection result of the optical image is secondarily confirmed based on the SAR image slice, so that interference of a pseudo target on the optical image is reduced, and the robustness of the model is improved; finally, a decision-level fusion method based on a D-S evidence theory of evidence association coefficients is introduced, probability distribution of target categories in the SAR image and the optical image is combined, collision evidence is effectively relieved, accuracy and reliability of target type identification are improved, interference capability of a model on a pseudo target is improved, and accuracy of pseudo target detection is improved to 97.5%.
The technical scheme adopted by the invention realizes the technical purposes as follows:
the ship target detection and identification method combining SAR and optical images is characterized by comprising the following steps of:
acquiring a space-time synchronous optical image to be detected and an SAR image to be detected;
inputting the optical image to be detected into a pre-trained target detection model to obtain coordinate parameters of a target detection frame and target class probability distribution based on the optical image;
mapping coordinate parameters of a target detection frame of the optical image into the SAR image to be detected through mapping and reflection relation of the image coordinates and the geographic coordinates to obtain a target detection frame of the SAR image, and obtaining SAR image slices based on the target detection frame;
classifying and judging SAR image slices by adopting a pre-trained SAR ship slice detection model to obtain SAR target slices;
inputting the SAR target slice into a pre-trained SAR image ship classification model to obtain target class probability distribution based on the SAR image;
and carrying out fusion recognition on the target class probability distribution based on the optical image and the target class probability distribution based on the SAR image by adopting a D-S evidence fusion method based on the evidence association coefficient to obtain the final target class probability distribution.
Further, the object detection model adopts a YOLO V7 object detection model which introduces an attention mechanism, and the model is input into an optical image, and specifically comprises: the method comprises the steps that a back box part, a back part and a head part are input into an original optical image, the back box part outputs three characteristic diagrams of a lower layer, a middle layer and a higher layer to the back part, the back part performs characteristic fusion and then outputs the characteristic fusion to the head part, and the head part outputs coordinate parameters of a target detection frame and target category probability distribution; the CBAM attention module is added before each output branch of the backup part, and the CBAM attention module is added between the RepVGG module and the convolution module of each scale branch of the head part.
Furthermore, the SAR ship slice detection model adopts a classifier based on SVM, HOG features and SIFT features of SAR image slices are extracted, feature stitching is carried out, and the result is input as a model.
Furthermore, the SAR image ship classification model adopts a ViT model, and the model is input into an SAR target slice.
Further, the specific process of the coordinate parameter mapping is as follows:
acquiring six parameters trans_opt of an optical image to be detected, and converting the coordinates of a target detection frame of the optical image from image coordinates to geographic projection coordinates:
px＝trans_opt[0]+col_opt×trans_opt[1]+row_opt×trans_opt[2]
py＝trans_opt[3]+col_opt×trans_opt[4]+row_opt×trans_opt[5]
wherein px and py are horizontal and vertical coordinates of the geographic projection coordinates, col_opt is a pixel column coordinate in the optical image, row_opt is a pixel row coordinate in the optical image, and trans_opt [0], trans_opt [1], trans_opt [2], trans_opt [3], trans_opt [4], trans_opt [5] correspond to six parameters of the optical image raster data;
acquiring six parameters trans_sar of an SAR image to be detected, and solving an equation to obtain row_sar and col_sar:
px＝trans_sar[0]+col_sar×trans_sar[1]+row_sar×trans_sar[2]
py＝trans_sar[3]+col_sar×trans_sar[4]+row_sar×trans_sar[5]
the col_sar is pixel point column coordinates in the SAR image, and row_sar is pixel point row coordinates in the SAR image; the trans_sar [0], trans_sar [1], trans_sar [2], trans_sar [3], trans_sar [4], trans_sar [5] correspond to six parameters of SAR image raster data.
Further, the D-S evidence fusion method based on the evidence association coefficient specifically comprises the following steps:
let the object class probability distribution based on the optical image and the object class probability distribution based on the SAR image be evidence m 1 And m 2 Based on m 1 And m 2 Is used for calculating evidence association coefficient r BPA ：
Wherein c (m 1 ,m 2 ) Representing evidence m 1 And m 2 Degree of association between:
wherein i, j=1, 2,3, a i And B j Evidence m respectively 1 And m 2 M of (C) 1 (A i ) Representing evidence m 1 Belongs to class A i Probability m of (2) 2 (B j ) Representing evidence m 2 Belongs to category B j Probability of (2); the |·| represents the potential of the focal element (i.e., the number of elements in the focal element);
taking the evidence association coefficient as the support degree to obtain a support degree matrix:
wherein S is 22 ＝S 11 ＝1，S 21 ＝S 12 ＝r BPA (m 1 ,m 2 )；
Calculate other evidence for any evidence m p Is a total degree of support of:
where p, q=1, 2, … n, n is the evidence number;
calculation of evidence m p Confidence level Crd (m) p )：
Wherein,,
the method of weighted averaging is used to obtain a basic probability assignment for new evidence:
wherein m represents new evidence, m (A i ) Representing new evidence m as belonging to class A i Is assigned to the base probability of (2);
and 2 times of fusion is carried out on the new evidence m by using a Dempster combination rule, and the fused target class probability distribution is obtained, so that the ship target detection and recognition is completed.
Based on the technical scheme, the invention has the beneficial effects that:
the invention provides an SAR and optical image ship target detection and identification method based on an improved D-S evidence theory, which has the following advantages:
1) According to the method, a CBAM attention mechanism is introduced into a YOLO V7 target detection network model of an optical image, and compared with a baseline model YOLO V7, better detection performance is obtained on the optical image;
2) According to the method, the slice of the target area in the SAR image is obtained by using the mapping relation between the image coordinates and the geographic coordinates, and the detection result of the optical image is secondarily confirmed by integrating the coordinate information of the optical image and the target class information in the SAR image, so that the interference of the pseudo target on the optical image is reduced, and the robustness of the model is improved;
3) According to the method, a D-S evidence theory based on the evidence association coefficient is introduced, and collision evidence is effectively relieved by combining target class probability distribution in the SAR image and target class probability distribution in the optical image, so that the accuracy and reliability of target type identification are improved, and the interference capability of the model on a pseudo target is also improved;
4) The method of the invention applies a decision-level fusion method, solves the problem that the optical image and SAR image which are strictly time-space synchronous are difficult to acquire, and reduces the dependence of the model on the requirement of the strict time-space synchronous of training data.
Drawings
FIG. 1 is a flow chart of a ship target detection and recognition method combining SAR and optical images according to the present invention.
Fig. 2 is a diagram of a spatiotemporal synchronized original optical image and SAR image in an embodiment of the present invention.
Fig. 3 is a schematic structural diagram of a YOLO V7 target detection model for directing attention to a mechanism according to an embodiment of the present invention, where (a) is a schematic structural diagram of a backup portion, and (b) is a schematic structural diagram of a head portion.
Fig. 4 is a visual result diagram of the detection and recognition of the ship target by the optical image of the YOLO V7 target detection model of the attention-drawing mechanism in the embodiment of the invention.
Fig. 5 is a visual result of ship target detection and identification combining SAR and optical images in an embodiment of the present invention.
Detailed Description
The present invention will be described in further detail with reference to the drawings and examples, which are provided to illustrate the present invention and not to limit the scope of the present invention, in order to make the objects, technical solutions and effects of the present invention more apparent.
The embodiment provides a ship target detection method combining SAR and optical images, the flow of which is shown in figure 1, and relates to an optical target detection module, a secondary confirmation module and a D-S decision module, which specifically comprises the following steps:
step 1-1, optical image data acquisition
In the embodiment, based on 4 data sets of FGSC-42, shipRSImageNet, ships in Satellite Imagery and SpaceNet-6, 2978 optical images are obtained as optical ship target detection data sets for 3 types of civil ships including Cargo ships (Cargo), tankers (tankers) and Container ships (containers), wherein 1503 Zhang Huolun images, 595 images of the Cargo ships and 878 images of the Container ships;
step 1-2 SAR image data acquisition
In the embodiment, 190 ship slices, 467 ship slices and 247 container ship slices are obtained based on ICEYE and SpaceNet-6 data sets, and 904 ship slices are taken in total; based on the clustering of the slice sizes, obtaining 4610 background slices in total according to the ratio of the number of ship slices to the number of background slices of 1:5, wherein the aspect ratio of the background slices is 1:1, 2:1 and 1:2; the ship section (positive sample) and the background section (negative sample) together form an SAR section detection data set; and taking the ship slices as SAR ship classification data sets;
an example of a spatiotemporal synchronized original optical image and SAR image is shown in fig. 2, where the left image is the optical image and the right image is the SAR image.
the YOLO V7 target detection model for the attention-directing mechanism is shown in fig. 3, and includes: the method comprises the steps that a back box part, a back part and a head part are input into an original optical image, the back box part outputs three characteristic diagrams of a lower layer, a middle layer and a higher layer to the back part, the back part performs characteristic fusion and then outputs the characteristic fusion to the head part, and the head part outputs coordinate parameters and category probability distribution of a target detection frame; the network structure of the backhaul part is shown in fig. 3 (a), and includes: the device comprises an ELAN module, an MP module and a CBAM attention module, wherein the ELAN module is responsible for feature extraction, the MP module is responsible for downsampling, and the CBAM module is responsible for distributing attention in the space and channel directions of a feature map so that a subsequent network can better extract features; as shown in fig. 3 (b), the network structure of the head part includes: the system comprises a RepVGG module, a CBAM module and a convolution module, wherein the RepVGG module is responsible for improving network performance and reasoning speed, the CBAM module is responsible for distributing attention to a feature map, and the Conv module is responsible for unifying the channel number of the output feature map;
in this embodiment, training parameters are set as follows: the batch size was set to 32, the image size was 640 x 640, and a total of 300 epochs were trained to complete the offline training of the focus-inducing YOLO V7 ship target detection model.
Step 3: extracting coordinate parameters (image coordinates) of a target detection frame of an optical image to be detected, mapping the coordinates of the target detection frame of the optical image into an SAR image to be detected through mapping and reflection relation of the image coordinates and the geographic coordinates to obtain a target detection frame of the SAR image, and obtaining SAR image slices based on the target detection frame;
step 3-1: the six parameters trans_opt of the optical image to be detected are obtained by using a gdal remote sensing data processing library of python, and the coordinates of a target detection frame in the optical image are converted from image coordinates to geographic projection coordinates:
px＝trans_opt[0]+col_opt×trans_opt[1]+row_opt×trans_opt[2]
py＝trans_opt[3]+col_opt×trans_opt[4]+row_opt×trans_opt[5]
wherein px and py are horizontal and vertical coordinates of a geographic projection coordinate, col_opt is a pixel column coordinate in the optical image, row_opt is a pixel row coordinate in the optical image, trans_opt is six parameters of raster data of the optical image, trans_opt [0] is a projection coordinate of an upper left corner abscissa of the optical image, trans_opt [1] is an optical image row rotation parameter, trans_opt [2] is an optical image pixel width, that is, resolution of an image in a horizontal space, trans_opt [3] is a projection coordinate of an upper left corner ordinate of the optical image, trans_opt [4] is an optical image column rotation parameter, trans_opt [5] is an optical image pixel width, that is, resolution of an image in a vertical space;
step 3-2: acquiring six parameters trans_sar of the SAR image by using a gdal library, and solving a binary once equation of projection coordinates calculated from the optical image to obtain row_sar and col_sar:
px＝trans_sar[0]+col_sar×trans_sar[1]+row_sar×trans_sar[2]
py＝trans_sar[3]+col_sar×trans_sar[4]+row_sar×trans_sar[5]
the col_sar is pixel point column coordinates in the SAR image, and row_sar is pixel point row coordinates in the SAR image; the trans-SAR is six parameters of SAR image raster data, the trans-SAR [0] is projection coordinate of left upper corner abscissa of SAR image, the trans-SAR [1] is SAR image line rotation parameter, the trans-SAR [2] is SAR image pixel width, that is resolution of image in horizontal space, the trans-SAR [3] is projection coordinate of left upper corner ordinate of SAR image, the trans-SAR [4] is SAR image column rotation parameter, and the trans-SAR [5] is SAR image pixel width, that is resolution of image in vertical space.
Step 4, extracting HOG features and SIFT features based on the SAR slice detection data set to train an SVM-based classifier as an SAR ship slice detection model; classifying and judging the SAR image slices obtained in the step 3 by adopting a trained detection model, inputting the step 5 for reclassifying if the SAR image slices are the ship targets, otherwise, discarding the pseudo targets;
the specific process in this embodiment is:
step 4-1: firstly, the SAR image slice is adjusted to 224 multiplied by 224;
step 4-2: using HOGDescriptor descriptors in an opencv library of python, setting winSize to 64×128, blockSize to 16×16, nbins to 9, and extracting HOG characteristics of SAR image slices;
step 4-3: extracting SIFT features of the SAR image slices by using a cv2.xfeature2d.SIFT_create method in an opencv library of python;
step 4-4: splicing two features by using a vstack method in a numpy library of python, generating an SVM classifier by using a sklearn library, and setting a kernel of the SVM classifier as an RBF function;
step 4-5: and (3) training the SVM classifier based on the SAR slice detection data set, classifying the SAR image slices obtained in the step (3) by using the trained SVM classifier, inputting the SAR image slices into the step (5) for reclassifying if the SAR image slices are the ship targets, otherwise, discarding the pseudo targets.
Step 5: training ViT model based on SAR ship classification data set to be used as SAR image ship classification model; inputting the ship target slice obtained in the step 4 into a trained ViT model to obtain target class probability distribution of the ship target slice, wherein the target class probability distribution is used as input in the step 6;
the specific process in this embodiment is:
step 5-1: because SAR images are all single-channel images, the images are read by using a PIL library of python, and are converted into a jpg format of RGB by using a overt method;
step 5-2: uniformly adjusting the images to 224 multiplied by 224, and carrying out normalization processing on the images;
step 5-3: training ViT model based on SAR ship classification data set, classifying ship target slices obtained in step 4 by using trained ViT model to obtain target class probability distribution, and inputting the target class probability distribution into D-S decision frame of step 6.
Step 6: using a D-S evidence fusion frame based on evidence association coefficients to fusion and identify the optical target class probability distribution obtained in the step 2 and the SAR target class probability distribution obtained in the step 5;
step 6-1: let the optical target class probability distribution obtained in step 2 and the SAR target class probability distribution obtained in step 5 be evidence m 1 And m 2 Based on m 1 And m 2 Is used for calculating evidence association coefficient r BPA The calculation formula is as follows:
wherein c (m 1 ,m 2 ) Representing evidence m 1 And m 2 The degree of association between the two is calculated as follows:
wherein i, j=1, 2,3, a i And B j Evidence m respectively 1 And m 2 M of (C) 1 (A i ) Representing evidence m 1 Belongs to class A i Probability m of (2) 2 (B j ) Representing evidence m 2 Belongs to category B j Probability of (2); the |·| represents the potential of the focal element (i.e., the number of elements in the focal element);
step 6-2: and (3) calculating an evidence association coefficient between every two evidences by using the method 6-1 as a support degree to obtain a support degree matrix:
wherein S is 22 ＝S 11 ＝1，S 21 ＝S 12 ＝r BPA (m 1 ,m 2 )；
Step 6-3: definition of other evidence for a certain evidence m p The total degree of support of (2) is as follows:
where p, q=1, 2, … n, n is the evidence number;
step 6-4: evidence m determination by support matrix p Confidence level Crd (m) p ) The calculation formula is as follows:
wherein,,
step 6-5: the method of weighted averaging is used to obtain a basic probability assignment for new evidence:
wherein m represents new evidence, m (A i ) Representing new evidence m as belonging to class A i Is assigned to the base probability of (2);
step 6-6: and fusing the new evidence m for 2 times by using a Dempster combination rule to obtain fused probability distribution.
Step 7: and visualizing the detection and identification results in the optical image, wherein the true target marks the true category and the corresponding probability of the true target, and the False target marks the False target.
The benefits of the invention are further illustrated in connection with simulation testing as follows:
in the embodiment, the running hardware platform is Intel Xeon Silver 4214R+NVIDIA RTX3090+16G DDR4 RAM, the software environment is CentOS7.4+CUDA11.1+PyTorrch1.10.0+Python 3.7, and the PyCharm development tool is used for carrying out algorithm development work; the data set used is an optical ship target detection data set, an SAR slice detection data set and an SAR ship classification data set of a homemade data set, wherein the test set is effective data of space Net-6 data set space time synchronization; the evaluation index is three indexes of mAP, mAR and floating point number calculation times commonly used in the field of target detection.
To demonstrate the effectiveness of the present invention, a comparison was made with a YOLO V7 baseline model based solely on optical images, the results of which are shown in table 1:
TABLE 1
| Model | mAP(IOU＝0.50) | mAR | Number of floating point number calculations |
| YOLO V7 | 0.95 | 0.925 | 16.0GFlops |
| The invention is that | 0.975 | 0.948 | 15.2GFlops |
As shown in table 1, compared with the base line network YOLO V7 model, the present invention has the advantages that the mAP is 2.5% higher than the base line model and the mAR is 2.3% higher than the base line model under the condition of iou=0.50, and meanwhile, the floating point number calculation times are reduced by 0.8 gflips, so that the reliability and accuracy of detection are improved, the time is also considered, and the complexity of the model is reduced;
furthermore, the method fully combines the advantages of the optical image and the SAR image, introduces a secondary confirmation mechanism and a D-S decision frame, improves the detection accuracy of the false target to 97.5%, effectively solves the false target deception problem of the optical image, and improves the robustness of the model. Specifically, as shown in fig. 4, the detection and recognition result of the ship target is only dependent on the optical image, as shown in fig. 5, after the secondary confirmation mechanism and the D-S evidence theory are introduced, the false target interference is reduced, and the robustness of the model is improved.
In summary, the invention aims at the problem that the optical image is easily deceptively deceived by a false target, and starts from improving the robustness of the existing remote sensing target detection algorithm, provides the ship target detection method combining SAR and the optical image, and the method is an effective and practical remote sensing ship target detection method by introducing a CBAM attention mechanism through the complementary characteristics among sensors.
While the invention has been described in terms of specific embodiments, any feature disclosed in this specification may be replaced by alternative features serving the equivalent or similar purpose, unless expressly stated otherwise; all of the features disclosed, or all of the steps in a method or process, except for mutually exclusive features and/or steps, may be combined in any manner.
Claims (6)
1. The ship target detection and identification method combining SAR and optical images is characterized by comprising the following steps of:
acquiring a space-time synchronous optical image to be detected and an SAR image to be detected;
inputting the optical image to be detected into a pre-trained target detection model to obtain coordinate parameters of a target detection frame and target class probability distribution based on the optical image;
mapping coordinate parameters of a target detection frame of the optical image into the SAR image to be detected through mapping and reflection relation of the image coordinates and the geographic coordinates to obtain a target detection frame of the SAR image, and obtaining SAR image slices based on the target detection frame;
classifying and judging SAR image slices by adopting a pre-trained SAR ship slice detection model to obtain SAR target slices;
inputting the SAR target slice into a pre-trained SAR image ship classification model to obtain target class probability distribution based on the SAR image;
and carrying out fusion recognition on the target class probability distribution based on the optical image and the target class probability distribution based on the SAR image by adopting a D-S evidence fusion method based on the evidence association coefficient to obtain the final target class probability distribution.
2. The ship target detection and recognition method combining SAR and optical image according to claim 1, wherein said target detection model adopts YOLO V7 target detection model which introduces attention mechanism, and the model input is optical image, specifically comprising: the method comprises the steps that a back box part, a back part and a head part are input into an original optical image, the back box part outputs three characteristic diagrams of a lower layer, a middle layer and a higher layer to the back part, the back part performs characteristic fusion and then outputs the characteristic fusion to the head part, and the head part outputs coordinate parameters of a target detection frame and target category probability distribution; the CBAM attention module is added before each output branch of the backup part, and the CBAM attention module is added between the RepVGG module and the convolution module of each scale branch of the head part.
3. The ship target detection and recognition method combining SAR and optical images according to claim 1, wherein the SAR ship slice detection model adopts an SVM-based classifier, the HOG characteristics and SIFT characteristics of SAR image slices are extracted, characteristic stitching is carried out, and the result is input as a model.
4. The ship target detection and recognition method combining SAR and optical images according to claim 1, wherein the SAR image ship classification model adopts a ViT model, and the model is input as SAR target slices.
5. The ship target detection and recognition method combining SAR and optical image according to claim 1, wherein the specific process of the coordinate parameter mapping is:
acquiring six parameters trans_opt of an optical image to be detected, and converting the coordinates of a target detection frame of the optical image from image coordinates to geographic projection coordinates:
px＝trans_opt[0]+col_opt×trans_opt[1]+row_opt×trans_opt[2]
py＝trans_opt[3]+col_opt×trans_opt[4]+row_opt×trans_opt[5]
wherein px and py are horizontal and vertical coordinates of the geographic projection coordinates, col_opt is a pixel column coordinate in the optical image, row_opt is a pixel row coordinate in the optical image, and trans_opt [0], trans_opt [1], trans_opt [2], trans_opt [3], trans_opt [4], trans_opt [5] correspond to six parameters of the optical image raster data;
acquiring six parameters trans_sar of an SAR image to be detected, and solving an equation to obtain row_sar and col_sar:
px＝trans_sar[0]+col_sar×trans_sar[1]+row_sar×trans_sar[2]
py＝trans_sar[3]+col_sar×trans_sar[4]+row_sar×trans_sar[5]
the col_sar is pixel point column coordinates in the SAR image, and row_sar is pixel point row coordinates in the SAR image; the trans_sar [0], trans_sar [1], trans_sar [2], trans_sar [3], trans_sar [4], trans_sar [5] correspond to six parameters of SAR image raster data.
6. The ship target detection and recognition method combining SAR and optical images according to claim 1, wherein the D-S evidence fusion method based on the evidence association coefficient is specifically as follows:
let the object class probability distribution based on the optical image and the object class probability distribution based on the SAR image be evidence m 1 And m 2 Based on m 1 And m 2 Is used for calculating evidence association coefficient r BPA ：
Wherein c (m 1 ,m 2 ) Representing evidence m 1 And m 2 Degree of association between:
wherein i, j=1, 2,3, a i And B j Evidence m respectively 1 And m 2 M of (C) 1 (A i ) Representing evidence m 1 Belongs to class A i Probability m of (2) 2 (B j ) Representing evidence m 2 Belongs to category B j Probability of (2);
taking the evidence association coefficient as the support degree to obtain a support degree matrix:
wherein S is 22 ＝S 11 ＝1，S 21 ＝S 12 ＝r BPA (m 1 ,m 2 )；
Calculate other evidence for any evidence m p Is a total degree of support of:
where p, q=1, 2, … n, n is the evidence number;
calculation of evidence m p Confidence level Crd (m) p )：
Wherein,,
the method of weighted averaging is used to obtain a basic probability assignment for new evidence:
wherein m represents new evidence, m (A i ) Representing new evidence m as belonging to class A i Is assigned to the base probability of (2);
and 2 times of fusion is carried out on the new evidence m by using a Dempster combination rule, and the fused target class probability distribution is obtained, so that the ship target detection and recognition is completed.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310255329.1A CN116343057A (en) | 2023-03-16 | 2023-03-16 | Ship target detection and identification method combining SAR (synthetic aperture radar) with optical image |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310255329.1A CN116343057A (en) | 2023-03-16 | 2023-03-16 | Ship target detection and identification method combining SAR (synthetic aperture radar) with optical image |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116343057A true CN116343057A (en) | 2023-06-27 |
Family
ID=86887099
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310255329.1A Pending CN116343057A (en) | 2023-03-16 | 2023-03-16 | Ship target detection and identification method combining SAR (synthetic aperture radar) with optical image |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116343057A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116935439A (en) * | 2023-07-18 | 2023-10-24 | 河北农业大学 | Automatic monitoring and early warning method and automatic monitoring and early warning system for delivery of pregnant sheep |
-
2023
- 2023-03-16 CN CN202310255329.1A patent/CN116343057A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116935439A (en) * | 2023-07-18 | 2023-10-24 | 河北农业大学 | Automatic monitoring and early warning method and automatic monitoring and early warning system for delivery of pregnant sheep |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN114202696B (en) | SAR target detection method and device based on context vision and storage medium | |
| CN109871902B (en) | SAR small sample identification method based on super-resolution countermeasure generation cascade network | |
| CN112288008B (en) | Mosaic multispectral image disguised target detection method based on deep learning | |
| CN111666854B (en) | High-resolution SAR image vehicle target detection method fusing statistical significance | |
| CN110675418A (en) | Target track optimization method based on DS evidence theory | |
| CN109712071B (en) | Unmanned aerial vehicle image splicing and positioning method based on track constraint | |
| CN113269040A (en) | Driving environment sensing method combining image recognition and laser radar point cloud segmentation | |
| CN112906531B (en) | Multi-source remote sensing image space-time fusion method and system based on non-supervision classification | |
| CN112348758B (en) | Optical remote sensing image data enhancement method and target identification method | |
| CN113033315A (en) | Rare earth mining high-resolution image identification and positioning method | |
| CN114399675A (en) | Target detection method and device based on machine vision and laser radar fusion | |
| CN112883850A (en) | Multi-view aerospace remote sensing image matching method based on convolutional neural network | |
| CN115100741B (en) | Point cloud pedestrian distance risk detection method, system, equipment and medium | |
| CN112561996A (en) | Target detection method in autonomous underwater robot recovery docking | |
| CN116343057A (en) | Ship target detection and identification method combining SAR (synthetic aperture radar) with optical image | |
| Shangzheng | A traffic sign image recognition and classification approach based on convolutional neural network | |
| Petković et al. | An overview on horizon detection methods in maritime video surveillance | |
| Zou et al. | Sonar Image Target Detection for Underwater Communication System Based on Deep Neural Network. | |
| CN112233079B (en) | Method and system for fusing images of multiple sensors | |
| Huang et al. | A deep learning approach to detecting ships from high-resolution aerial remote sensing images | |
| CN113537397B (en) | Target detection and image definition joint learning method based on multi-scale feature fusion | |
| CN116797894A (en) | Radar and video fusion target detection method for enhancing characteristic information | |
| CN115861669A (en) | Infrared dim target detection method based on clustering idea | |
| CN115272153A (en) | Image matching enhancement method based on feature sparse area detection | |
| CN111950549A (en) | Sea surface obstacle detection method based on fusion of sea antennas and visual saliency |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |