CN106407900B - Abnormal scene recognition method based on multi-source aerophotograph - Google Patents
Abnormal scene recognition method based on multi-source aerophotograph Download PDFInfo
- Publication number
- CN106407900B CN106407900B CN201610788622.4A CN201610788622A CN106407900B CN 106407900 B CN106407900 B CN 106407900B CN 201610788622 A CN201610788622 A CN 201610788622A CN 106407900 B CN106407900 B CN 106407900B
- Authority
- CN
- China
- Prior art keywords
- region
- source
- aerophotograph
- image
- far infrared
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/10—Terrestrial scenes
- G06V20/13—Satellite images
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
- G06F18/2411—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on the proximity to a decision surface, e.g. support vector machines
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Bioinformatics & Computational Biology (AREA)
- General Engineering & Computer Science (AREA)
- Artificial Intelligence (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Astronomy & Astrophysics (AREA)
- Remote Sensing (AREA)
- Multimedia (AREA)
- Image Analysis (AREA)
- Image Processing (AREA)
- Radiation Pyrometers (AREA)
Abstract
A kind of abnormal scene recognition method based on multi-source aerophotograph first by detecting temperature anomaly region collection and the strong region collection of dielectricity in far infrared and SAR image, and merges the two result as doubtful abnormal area collection.Three sources (visible light, far infrared, SAR) feature is extracted to doubtful abnormal area one by one, and three is merged on feature level.Finally fusion feature is input in classifier and carries out Classification and Identification.The present invention proposes the predicable using temperature anomaly or with artificial metal product as abnormal scene, and provides a set of practicable identification framework, has filled up the research blank that abnormal scene Recognition is carried out using three source images.
Description
Technical field
The present invention relates to image processing techniques, specifically combine DOG (Difference of Gaussians) spot detection
Algorithm and CFAR (Constant False-Alarm Rate) detection algorithm, for the abnormal scene recognition method of multi-source aerophotograph.
Background technique
Exception scene of the present invention is different from tradition research definition, refers in particular to uninhabited area, has abnormal human
The scene of activity trail, essential attribute show as temperature anomaly or have artificial metal's product.Mountain is often hidden deeply in these places
Desert, manpower generaI investigation expend it is huge, and have serious social harm, as illegal drug processing space, She Kong training camp and
Illegal gambling house.
For above-mentioned scene feature, researching and designing UAV flight's visible light, far infrared and SAR camera are patrolled
It looks into, and the multi-source of acquisition data of taking photo by plane is handled.Unmanned plane has noise small, not noticeable, and continuation of the journey is long, at low cost etc. excellent
Point is the tool for replacing manpower to check remote districts.In image acquired in the multi-source camera of carrying, it is seen that light
Image is reflected image, and the details and form textural characteristics of scene are able to reflect under certain illumination；Infrared image is radiation shadow
Picture has preferably target acquisition performance when illumination is poor, but reflects that the details of real scene is poor, mainly utilizes target
Thermal property；Microwave, which is not affected by weather conditions, can detect the dielectricity of target.Characteristic of the multi-source data to same target
Reaction is different, by information theory knowledge it is found that can effectively improve target identification ability in conjunction with these complementary informations.
Main contents of the present invention are the image processing techniques of multi-source aerophotograph, are specially directed to above-mentioned three source aerophotograph, are researched and developed different
The recognizer of normal scene, and verified for two big abnormal scenes (illegal drug processing space, She Kong training camp).Just at present
For, do not find while being directed to both at home and abroad the scene Recognition research of the three sources aerophotograph.
Summary of the invention
Present invention combination DOG spot detection algorithm and CFAR detection algorithm, research and development are directed to visible light, far infrared and SAR
The abnormal scene recognizer of image has filled up the research blank of this application.Technical solution of the invention is broadly divided into
Two large divisions: first part is doubtful abnormal area probe portion, and second part is abnormal scene identification division.Process of the present invention
Figure is as shown in Figure 1, two-part detailed content is as described below.
1. based at this to the hypothesis of abnormal scene: abnormal scene normally behave as temperature anomaly or have artificial metal's system
Product, first part of the present invention make full use of picture characteristics, propose respectively using the temperature in DOG detection algorithm detection far infrared image
Abnormal area is spent, using the region with strong dielectricity substance (such as metal) in CFAR algorithm detection SAR image, is finally merged
The two region is as doubtful abnormal area collection.Algorithm includes the following steps:
The first step inputs far infrared image
Second step uses DOG detection algorithm to infrared image, and the area flag that will test is RI.Temperature anomaly exists
Shown as in thermal infrared images a light color or dark color spot, set forth herein use DOG spot detection algorithm to such region
It is detected, it has calculating speed fast, the features such as strong real-time.DOG is the wavelet mother function of an empty total value, it is from one
Narrow Gauss subtracts a wide Gauss, is an approximation of mexican hat wavelet.Its defined formula is as follows under one-dimensional case:
In the case of two dimensions:
Third step inputs SAR image
4th step uses CFAR detection algorithm to SAR image, and the area flag that will test is Rs.Artificial metal product
A speck is showed in SAR image for strong reflection spot, however due to the imaging mechanism of SAR image itself, make general speck
Detection algorithm is unable to reach ideal effect, and the present invention proposes to detect such region using CFAR algorithm.CFAR is determined in background
In the case of, target detection is carried out by the false alarm rate kept constant, key is to determine adaptive threshold value i.e. decision gate
Limit.Assuming that the background of SAR image and the probability density distribution of target are respectively Pb(x) and Pt(x), T is decision threshold such as Fig. 2 institute
Show.
False alarm rate pfaWith verification and measurement ratio pdRespectively shown in following formula.
Wherein detection threshold value T is acquired by following formula.
T is acquired to be judged by following decision rule later:
5th step calculates RI, RsThe center distance of each region merges center distance less than a certain range of
Region (distance is empirical value at this, can be according to actual scene setting and image resolution ratio sets itself), obtains doubtful exception
Region collection R'.
First and second step is directed to far infrared image, third and fourth step is directed to SAR image, and the two parallel processing can contract significantly
Short algorithm calculates the time.
2. second part of the present invention is directed to doubtful abnormal area, multi-source HOG (Histogram of is extracted respectively
Oriented Gradients) feature, and PCA (PrincipalComponent Analysis) dimensionality reduction is used to multi-source feature
Afterwards, it is merged using CCA (Canonical Correlation Analysis), instruction finally is used to concatenated fusion feature
Practice SVM (the Support Vector Machine) discrimination model completed and scene classification identification is carried out to suspicious region.Algorithm packet
Include following steps:
The first step intercepts corresponding diagram in multi-source aerophotograph for each piece of doubtful abnormal area in doubtful abnormal area collection R'
As block.Therefore the corresponding one group of image of each doubtful abnormal area, is made of thermal infrared, SAR and visual picture block respectively, makees
For the input of second part.
Second step stretches image block, uniform sizes, to every group of image zooming-out HOG feature.HOG is to image geometry
And optical deformation can keep good invariance.Secondly, thick airspace is sampled, fine direction is sampled and relatively strong
Indicative of local optical normalization etc. under the conditions of, some subtle variations can be ignored without influence detection effect.Its substantially process
It can conclude are as follows:
(1) gray processing；
(2) color space standards are carried out to input picture using Gamma correction method, it is as follows that Gamma compresses formula:
I (x, y)=I (x, y)gamma (7)
(3) gradient (including size and Orientation) of each pixel of image, the gradient of each pixel (x, y) are calculated are as follows:
G
Wherein Gx(x, y), Gy(x, y), H (x, y) respectively indicate the horizontal direction gradient of pixel (x, y), vertical direction ladder
Degree and pixel value.Gradient magnitude G (x, y) and gradient direction α (x, y) at pixel (x, y) are respectively as follows:
(4) small cells is divided an image into；
(5) histogram of gradients of each cell composition is counted；
(6) block will be formed per several cell, the feature descriptor of all cell is together in series in each block
Obtain the feature descriptor of the block；
(7) feature descriptor of block all in image is together in series to obtain the HOG feature of image.
Third step is merged to multi-source aerophotograph HOG feature using after PCA dimensionality reduction using CCA.Specifically by study
Mapping matrix keeps multi-source data distribution as consistent as possible.The projection matrix of three can be obtained by optimization following formula:
Wherein ΣijFor ΦiWith ΦjCovariance square, WiFor projection matrix, wherein kth list is shown asΦ indicates to use
HOG feature after PCA dimensionality reduction.
4th step, connect multi-source mappings characteristics as fusion feature, be input in SVM classifier, to the region scene into
Row identification.SVM method is that sample space is mapped to a higher-dimension or even infinite dimensional feature by a Nonlinear Mapping p
In space (space Hilbert), so that being converted into feature space the problem of Nonlinear separability in original sample space
Linear separability the problem of, can also obtain preferable classifying quality in the less situation of number of samples.
The beneficial effects of the present invention are:
1, the present invention proposes to detect temperature anomaly region using DOG, and CFAR detects dielectricity abnormal area, and merges two then
For detection result as doubtful abnormal area collection, which achieves good detection result under the premise of false-alarm is higher.It can be simultaneously
Capable treatment mechanism more greatly accelerates the calculating speed of algorithm.This method also solve visible light when visibility is lower without
Method obtains the fatal defects of effective image, and the monitoring of round-the-clock is realized to outlying district.
2, the present invention proposes to merge visible, far infrared, the HOG feature of SAR using PCA and CCA, and PCA can be removed
Redundancy in multi-source information, and CCA then passes through and maps these information their degree of correlation is made to reach maximum.Finally based on more
Source fusion feature carries out scene Recognition using SVM, solves the excessively high disadvantage of first part's false alarm rate, realizes special scenes
Multi-source identification mission.
Detailed description of the invention
Fig. 1 is flow chart of the present invention.
Fig. 2 is the schematic diagram of judgement threshold in CFAR algorithm.
Fig. 3 (a)~(c) is successively the far infrared containing illegal drug processing space, SAR and visible images, Fig. 3 (d)
~(f) is followed successively by DOG testing result, CFAR testing result and doubtful abnormal area testing result.Fig. 3 (g) is that the present invention is right
In the identification figure of illegal drug processing space.
Fig. 4 (a)~(c) is successively the far infrared containing She Kong training camp, SAR and visible images, Fig. 4 (d)~(f)
It is followed successively by DOG testing result, CFAR testing result and doubtful abnormal area testing result.Fig. 4 (g) is the present invention for relating to probably
The identification figure of training camp.
Fig. 5 is illegal drug processing space list source/multi-source recognition result figure.
Fig. 6 is She Kong training camp Dan Yuan/multi-source recognition result figure.
Specific embodiment
Below in conjunction with drawings and examples, the present invention is described in detail: the present embodiment is being with technical solution of the present invention
Under the premise of the example implemented, demonstrate knowledge of the present invention for illegal drug processing space, She Kong training camp these two types scene
Other effect, and detailed embodiment and process are given, but protection scope of the present invention should not necessarily be limited by following embodiments.
It is please this referring initially to Fig. 1 and Fig. 3, Fig. 1 1. being using the embodiment of present invention identification illegal drug processing space first
The flow chart of invention, Fig. 3 are the input, pilot process figure and result figure of this example.As shown, present invention specific implementation is divided to two
Most of, steps are as follows for first part:
The first step inputs far infrared, SAR image, as shown in Fig. 3 (a)~(b).
Second step, while the pretreatment of pyramid noise reduction is implemented to far infrared, SAR image
Third step inverts far infrared image grayscale to obtain reversion figure, then uses DOG spot to original image and reversion figure respectively
Detection algorithm merges the two result and obtains temperature anomaly region collection, such as Fig. 3 (d).SAR image is calculated using CFAR detection simultaneously
Method obtains strong reflection region, such as Fig. 3 (e).
4th step calculates the central point distance in temperature anomaly region, strong reflection region, area of the combined distance less than 50 pixels
Domain obtains doubtful abnormal area collection, such as Fig. 3 (f).
After obtaining doubtful abnormal area collection, the second part is come into effect, it is shown that specific step is as follows:
The first step, the abnormal area coordinate obtained by first part intercept correspondence image block in multi-source aerophotograph.
Second step stretches image block, uniform sizes 32*32.For each abnormal area, extract respectively more
The HOG feature of source aerophotograph block.
Third step ties up three HOG features in each region using PCA dimensionality reduction to 64, and after use CCA fusion dimensionality reduction
HOG feature, series connection obtain fusion feature.
4th step, fusion feature is input in SVM classifier, carries out Classification and Identification to the region scene.Retain target
Scene areas removes the region of non-targeted scene, shown in obtained final recognition result such as Fig. 3 (g).
2. being please the present invention referring initially to Fig. 1 and Fig. 4, Fig. 1 followed by using the embodiment of present invention identification She Kong training camp
Flow chart, Fig. 4 be this example input, pilot process figure and result figure.As shown, present invention specific implementation is divided to two big portions
Point, steps are as follows for first part:
The first step inputs far infrared, SAR image, as shown in Fig. 4 (a)~(b).
Second step, while the pretreatment of pyramid noise reduction is implemented to far infrared, SAR image
Third step inverts far infrared image grayscale to obtain reversion figure, then uses DOG spot to original image and reversion figure respectively
Detection algorithm merges the two result and obtains temperature anomaly region collection, such as Fig. 4 (d).SAR image is calculated using CFAR detection simultaneously
Method obtains strong reflection region collection, such as Fig. 4 (e).
4th step calculates the central point distance in temperature anomaly region, strong reflection region, area of the combined distance less than 70 pixels
Domain obtains doubtful abnormal area collection, such as Fig. 4 (f).
After obtaining doubtful abnormal area collection, the second part is come into effect, it is shown that specific step is as follows:
The first step, the abnormal area coordinate obtained by first part intercept correspondence image block in multi-source aerophotograph.
Second step stretches image block, uniform sizes 32*32.For each abnormal area, extract respectively more
The HOG feature of source aerophotograph block.
Third step ties up three HOG features in each region using PCA dimensionality reduction to 64, and after use CCA fusion dimensionality reduction
HOG feature, series connection obtain fusion feature.
4th step, fusion feature is input in SVM classifier, carries out Classification and Identification to the region scene.Retain target
Scene areas removes the region of non-targeted scene, shown in obtained final recognition result such as Fig. 4 (g).
Temperature anomaly and strong dielectricity preferably define the predicable of most of abnormal scenes, wherein for verifying this
The illegal drug processing space of invention algorithm relates to probably training camping ground and is provided with both characteristics.Experiment is based on single source and multi-source number
According to discrimination of the testing algorithm to above-mentioned two classes scene.The information that test experiments result verifications multi-source data is included has
Complementarity, multi-source aerophotograph recognition methods fusion multi-source information of the invention have excellent recognition effect to this kind of scene.Specifically
Shown in test result is as follows table and Fig. 5~Fig. 6:
Table 1
Claims (3)
1. a kind of abnormal scene recognition method based on multi-source aerophotograph, which comprises the steps of:
First stage: the detection to doubtful abnormal area, specific as follows:
1. far infrared image and SAR image after input registration；
2. far infrared image grayscale is inverted to obtain reversion figure, then far infrared image and reversion figure are examined using DOG spot respectively
Method of determining and calculating merges the two result and obtains temperature anomaly region collection；CFAR detection algorithm is used to SAR image simultaneously, is obtained strong anti-
Penetrate region collection；
3. calculating the central point distance in temperature anomaly region, strong reflection region, merges the region for being less than certain distance, obtain doubtful
Abnormal area collection；
Second stage: further identifying doubtful abnormal area collection, excludes nontarget area, specific as follows:
4. intercepting correspondence image block in multi-source aerophotograph by doubtful abnormal area coordinate；
5. being stretched to image block, uniform sizes, and the HOG for extracting multi-source aerophotograph block respectively to each piece of abnormal area is special
Sign；
6. using PCA dimensionality reduction to the HOG feature of the multi-source aerophotograph block in each region, and special using the HOG after CCA fusion dimensionality reduction
Sign, series connection obtain fusion feature；
7. fusion feature is input in SVM classifier, Classification and Identification is carried out to the region scene, retains target scene region,
The region for removing non-targeted scene, obtained final recognition result.
2. the abnormal scene recognition method according to claim 1 based on multi-source aerophotograph, which is characterized in that the step is 2.
In it is parallel to far infrared image using DOG detection temperature anomaly region, detect strong reflection region using CFAR to SAR image,
And merge the testing result of the two as doubtful abnormal area collection.
3. the abnormal scene recognition method according to claim 1 based on multi-source aerophotograph, which is characterized in that the step is 6.
It is middle that HOG feature is extracted to visible images, far infrared and SAR image respectively, and CCA is used after using PCA dimensionality reduction to three
Carry out Fusion Features.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610788622.4A CN106407900B (en) | 2016-08-31 | 2016-08-31 | Abnormal scene recognition method based on multi-source aerophotograph |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610788622.4A CN106407900B (en) | 2016-08-31 | 2016-08-31 | Abnormal scene recognition method based on multi-source aerophotograph |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN106407900A CN106407900A (en) | 2017-02-15 |
| CN106407900B true CN106407900B (en) | 2019-04-19 |
Family
ID=58000598
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610788622.4A Active CN106407900B (en) | 2016-08-31 | 2016-08-31 | Abnormal scene recognition method based on multi-source aerophotograph |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106407900B (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2019061293A1 (en) * | 2017-09-29 | 2019-04-04 | 深圳市大疆创新科技有限公司 | Object detection method, object detection terminal, and computer readable medium |
| CN108256429A (en) * | 2017-12-19 | 2018-07-06 | 国网山西省电力公司阳泉供电公司 | A kind of transmission tower object detection method using high spatial resolution single polarization SAR image |
| CN110032915A (en) * | 2018-01-12 | 2019-07-19 | 杭州海康威视数字技术股份有限公司 | A kind of human face in-vivo detection method, device and electronic equipment |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103247039A (en) * | 2013-05-09 | 2013-08-14 | 河海大学常州校区 | Charged detection method of high-voltage cable based on composite vision |
| CN103487729A (en) * | 2013-09-06 | 2014-01-01 | 广东电网公司电力科学研究院 | Electrical equipment defect detection method based on fusion of ultraviolet video and infrared video |
| CN105321172A (en) * | 2015-08-31 | 2016-02-10 | 哈尔滨工业大学 | SAR, infrared and visible light image fusion method |
| CN105869408A (en) * | 2016-04-27 | 2016-08-17 | 长安大学 | Hub temperature anomaly detection and early warning method and system |
-
2016
- 2016-08-31 CN CN201610788622.4A patent/CN106407900B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103247039A (en) * | 2013-05-09 | 2013-08-14 | 河海大学常州校区 | Charged detection method of high-voltage cable based on composite vision |
| CN103487729A (en) * | 2013-09-06 | 2014-01-01 | 广东电网公司电力科学研究院 | Electrical equipment defect detection method based on fusion of ultraviolet video and infrared video |
| CN105321172A (en) * | 2015-08-31 | 2016-02-10 | 哈尔滨工业大学 | SAR, infrared and visible light image fusion method |
| CN105869408A (en) * | 2016-04-27 | 2016-08-17 | 长安大学 | Hub temperature anomaly detection and early warning method and system |
Non-Patent Citations (4)
| Title |
|---|
| Object detection capability evaluation for SAR image;Zheyuan Wang 等;《2016 IEEE International Geoscience and Remote Sensing Symposium（IGARSS）》;20160715;第1548-1551页 |
| SAR图像与其他源图像融合算法;刘振华;《现代雷达》;20070228;第29卷(第2期);第56-59页 |
| 基于NSST和稀疏表示的多源异类图像融合方法;王志社 等;《红外技术》;20150331;第37卷(第3期);第210-217页 |
| 基于特征层融合的高光谱图像异常检测算法研究;李智勇 等;《遥感学报》;20030731;第7卷(第4期);第304-308页 |
Also Published As
| Publication number | Publication date |
|---|---|
| CN106407900A (en) | 2017-02-15 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108510467B (en) | SAR image target identification method based on depth deformable convolution neural network | |
| Yang et al. | An automated cirrus cloud detection method for a ground-based cloud image | |
| CN106610969A (en) | Multimodal information-based video content auditing system and method | |
| CN109948566B (en) | Double-flow face anti-fraud detection method based on weight fusion and feature selection | |
| CN106407900B (en) | Abnormal scene recognition method based on multi-source aerophotograph | |
| CN103530648A (en) | Face recognition method based on multi-frame images | |
| CN112365467B (en) | Foggy image visibility estimation method based on single image depth estimation | |
| CN103093243B (en) | The panchromatic remote sensing image clouds of high-resolution sentences method | |
| CN112287983B (en) | Remote sensing image target extraction system and method based on deep learning | |
| CN109492534A (en) | A kind of pedestrian detection method across scene multi-pose based on Faster RCNN | |
| Tao et al. | Automatic smoky vehicle detection from traffic surveillance video based on vehicle rear detection and multi‐feature fusion | |
| CN111723656B (en) | Smog detection method and device based on YOLO v3 and self-optimization | |
| CN111539348A (en) | Face living body detection method based on polarization imaging | |
| Zhang et al. | Multi-feature embedded learning SVM for cloud detection in remote sensing images | |
| Varjo et al. | Image based visibility estimation during day and night | |
| Sun et al. | Vehicle change detection from aerial imagery using detection response maps | |
| CN109815784A (en) | A kind of intelligent method for classifying based on thermal infrared imager, system and storage medium | |
| CN111898427A (en) | Multispectral pedestrian detection method based on feature fusion deep neural network | |
| CN115797970B (en) | Dense pedestrian target detection method and system based on YOLOv5 model | |
| TWI628624B (en) | Improved thermal image feature extraction method | |
| CN109033984A (en) | A kind of night mist fast automatic detecting method | |
| Hoseop et al. | Image Database Generation and Management System for the Small-pixelized Airborne Target Recognition | |
| CN112581467B (en) | Intelligent security inspection method based on suspected dangerous goods evaluation | |
| Rumi et al. | Automated cloud classification using a ground based infra-red camera and texture analysis techniques | |
| Zhou et al. | Research on moving object detection and matching technology in multi-angle monitoring video |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |