CN117537929A - Unmanned aerial vehicle detection method, system, equipment and medium based on infrared thermal imaging - Google Patents
Unmanned aerial vehicle detection method, system, equipment and medium based on infrared thermal imaging Download PDFInfo
- Publication number
- CN117537929A CN117537929A CN202311416483.9A CN202311416483A CN117537929A CN 117537929 A CN117537929 A CN 117537929A CN 202311416483 A CN202311416483 A CN 202311416483A CN 117537929 A CN117537929 A CN 117537929A
- Authority
- CN
- China
- Prior art keywords
- pixel point
- effective pixel
- given
- unmanned aerial
- thermal imaging
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 87
- 238000001931 thermography Methods 0.000 title claims abstract description 64
- 238000000034 method Methods 0.000 claims abstract description 31
- 238000010276 construction Methods 0.000 claims abstract description 10
- 230000015654 memory Effects 0.000 claims description 26
- 230000005855 radiation Effects 0.000 claims description 19
- 238000004590 computer program Methods 0.000 claims description 14
- 230000009286 beneficial effect Effects 0.000 description 5
- 238000012216 screening Methods 0.000 description 4
- 238000010586 diagram Methods 0.000 description 3
- 230000006870 function Effects 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 238000012706 support-vector machine Methods 0.000 description 2
- 230000001419 dependent effect Effects 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 238000003062 neural network model Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01J—MEASUREMENT OF INTENSITY, VELOCITY, SPECTRAL CONTENT, POLARISATION, PHASE OR PULSE CHARACTERISTICS OF INFRARED, VISIBLE OR ULTRAVIOLET LIGHT; COLORIMETRY; RADIATION PYROMETRY
- G01J5/00—Radiation pyrometry, e.g. infrared or optical thermometry
- G01J5/48—Thermography; Techniques using wholly visual means
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01V—GEOPHYSICS; GRAVITATIONAL MEASUREMENTS; DETECTING MASSES OR OBJECTS; TAGS
- G01V9/00—Prospecting or detecting by methods not provided for in groups G01V1/00 - G01V8/00
- G01V9/005—Prospecting or detecting by methods not provided for in groups G01V1/00 - G01V8/00 by thermal methods, e.g. after generation of heat by chemical reactions
Landscapes
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Spectroscopy & Molecular Physics (AREA)
- Chemical & Material Sciences (AREA)
- Chemical Kinetics & Catalysis (AREA)
- Life Sciences & Earth Sciences (AREA)
- General Life Sciences & Earth Sciences (AREA)
- Geophysics (AREA)
- Image Processing (AREA)
Abstract
The invention discloses an unmanned aerial vehicle detection method, system, equipment and medium based on infrared thermal imaging, wherein the method comprises the following steps: traversing the thermal imaging image to be detected and synchronously marking each pixel point obtained by traversing according to a given intensity threshold condition; storing all the pixel points marked and meeting the given intensity threshold condition to a constructed effective pixel point set; splitting the non-empty effective pixel point set according to a given coordinate proximity condition to form a plurality of effective pixel point subsets; selecting all effective pixel point subsets meeting given rectangular construction standards from a plurality of effective pixel point subsets, and acquiring all maximum rectangular frames correspondingly covered by all effective pixel point subsets; and intercepting all rectangular images corresponding to all the largest rectangular frames from the thermal imaging images to be detected, and processing all the rectangular images by using a trained unmanned aerial vehicle detection model to obtain a detection result. The invention can realize the efficient and reliable unmanned aerial vehicle detection function.
Description
Technical Field
The invention relates to the technical field of unmanned aerial vehicle detection, in particular to an unmanned aerial vehicle detection method, an unmanned aerial vehicle detection system, unmanned aerial vehicle detection equipment and an unmanned aerial vehicle detection medium based on infrared thermal imaging.
Background
As unmanned aerial vehicles become more and more popular in society, especially small unmanned aerial vehicles have become consumer products for many general users, sometimes the use of unmanned aerial vehicles by users is not reasonable or even legal, which may pose a threat to public security, trade secrets or personal privacy. At present, a plurality of detection methods for unmanned aerial vehicles are proposed by students, but certain disadvantages still exist, such as: the microwaves of the radar are difficult to act on unmanned aerial vehicles with smaller volumes, and the common cameras cannot accurately detect the unmanned aerial vehicles at night, and the like. Therefore, it is necessary to propose an efficient and reliable unmanned aerial vehicle detection scheme.
Disclosure of Invention
The invention provides an unmanned aerial vehicle detection method, an unmanned aerial vehicle detection system, unmanned aerial vehicle detection equipment and an unmanned aerial vehicle detection medium based on infrared thermal imaging, which are used for solving one or more technical problems in the prior art and at least providing a beneficial selection or creation condition.
In a first aspect, there is provided an unmanned aerial vehicle detection method based on infrared thermal imaging, the method comprising:
traversing according to a given step length in a thermal imaging image to be detected, and synchronously marking each pixel point obtained by traversing according to a given intensity threshold condition;
constructing an effective pixel point set, and storing all the pixel points marked and meeting the given intensity threshold condition into the effective pixel point set;
when the effective pixel point set is a non-empty set, splitting the effective pixel point set according to a given coordinate proximity condition to form a plurality of effective pixel point subsets;
selecting all effective pixel point subsets meeting given rectangular construction standards from the plurality of effective pixel point subsets, and acquiring all maximum rectangular frames correspondingly covered by all effective pixel point subsets;
and intercepting all rectangular images corresponding to all the maximum rectangular frames from the thermal imaging images to be detected, and processing all the rectangular images by using a trained unmanned aerial vehicle detection model to obtain a detection result.
Further, the trained unmanned aerial vehicle detection model is obtained by the following steps:
acquiring a training image set, wherein each training image comprises a rectangular frame marked with a detection result of the unmanned aerial vehicle;
constructing an optimal training image set, wherein the optimal training image set comprises the training image set;
removing any training image from the training image set;
acquiring a plurality of corresponding pixel points with the infrared radiation intensity value smaller than a given infrared heat source intensity threshold value from the training image, and acquiring a corresponding maximum infrared radiation intensity value from the plurality of pixel points;
judging whether the sum value between the maximum infrared radiation intensity value and a given intensity gradient is smaller than the infrared heat source intensity threshold value or not; if yes, adding the infrared radiation intensity values corresponding to the pixel points with the intensity gradient quantity to obtain a new training image, and storing the new training image into the training image set and the optimal training image set;
judging whether the training image set is an empty set or not; if yes, training the built unmanned aerial vehicle detection model by using the optimal training image set to obtain a trained unmanned aerial vehicle detection model; if not, returning to the step of removing any training image from the training image set.
Further, the step of marking each pixel obtained by traversing according to a given intensity threshold condition includes:
when each pixel point is obtained in the traversal process of the thermal imaging image to be detected, judging whether the infrared radiation intensity value corresponding to the pixel point is greater than or equal to a given infrared heat source intensity threshold value; if yes, a positive judgment result is given to the pixel point so as to represent that the pixel point meets the given intensity threshold condition; if not, a negative judgment result is given to the pixel point so as to represent that the pixel point does not meet the given intensity threshold condition.
Further, splitting the set of valid pixels according to the given coordinate proximity condition to form a plurality of sub-sets of valid pixels includes:
any pixel point is removed from the effective pixel point set;
judging whether a plurality of pixel points with the horizontal coordinate difference value and the vertical coordinate difference value of the pixel points not exceeding the given step length exist in the effective pixel point set; if yes, the plurality of pixel points are moved out and an effective pixel point subset is formed with the pixel points; if not, the pixel points are singly formed into an effective pixel point subset;
and executing the two operations circularly until the effective pixel point set becomes an empty set.
Further, the given rectangle building criteria are: in all pixel points contained in each subset of valid pixel points, there are unequal maximum and minimum abscissa values, and unequal maximum and minimum ordinate values.
Further, the obtaining all the largest rectangular boxes correspondingly covered by the subset of all the effective pixel points includes:
for each effective pixel point subset meeting a given rectangle construction standard, acquiring a maximum abscissa value, a minimum abscissa value, a maximum ordinate value and a minimum ordinate value from all pixel points according to coordinate information of all pixel points contained in the effective pixel point subset;
determining four different vertex pixels according to the maximum abscissa value, the minimum abscissa value, the maximum ordinate value and the minimum ordinate value;
and determining the largest rectangular frame covered by the effective pixel point subset according to the four vertex pixels.
Further, when the effective pixel point set is an empty set, a detection result output recording that no unmanned aerial vehicle exists in the thermal imaging image to be detected is generated.
In a second aspect, there is provided an unmanned aerial vehicle detection system based on infrared thermal imaging, the system comprising:
the first module is used for traversing according to a given step length in the thermal imaging image to be detected and marking each pixel point obtained through traversing synchronously according to a given intensity threshold condition;
the second module is used for constructing an effective pixel point set, and storing all the pixel points marked and meeting the given intensity threshold condition into the effective pixel point set;
the third module is used for splitting the effective pixel point set into a plurality of effective pixel point subsets according to a given coordinate proximity condition when the effective pixel point set is a non-empty set;
a fourth module, configured to select, from the plurality of effective pixel point subsets, all effective pixel point subsets that meet a given rectangular construction standard, and obtain all maximum rectangular frames that are correspondingly covered by all effective pixel point subsets;
and a fifth module, configured to intercept all rectangular images corresponding to the all maximum rectangular frames in the thermal imaging image to be detected, and process all rectangular images by using a trained unmanned aerial vehicle detection model to obtain a detection result.
In a third aspect, a computer device is provided, comprising a memory storing a computer program and a processor executing the computer program to implement the method for detecting an unmanned aerial vehicle based on infrared thermal imaging according to the first aspect.
In a fourth aspect, a computer readable storage medium is provided, on which a computer program is stored which, when being executed by a processor, implements the unmanned aerial vehicle detection method based on infrared thermal imaging according to the first aspect.
The invention has at least the following beneficial effects: the obtained training image set is subjected to data enhancement in the model training stage, namely, the infrared radiation intensity value associated with the pixel points which do not meet the given intensity threshold condition in each training image is increased by a small extent, so that the number of the training images and the detection difficulty can be increased, and the detection capability and the reliability of the unmanned aerial vehicle detection model after training are effectively improved; traversing the thermal imaging image to be detected by customizing the pixel-level searching step length is beneficial to improving the execution efficiency of the whole detection method, so that the detection result of the thermal imaging image to be detected is obtained more quickly.
Drawings
The accompanying drawings are included to provide a further understanding of the invention and are incorporated in and constitute a part of this specification, illustrate and do not limit the invention.
Fig. 1 is a schematic flow chart of an unmanned aerial vehicle detection method based on infrared thermal imaging in an embodiment of the invention;
FIG. 2 is a schematic diagram of an unmanned aerial vehicle detection system based on infrared thermal imaging in an embodiment of the invention;
fig. 3 is a schematic hardware structure of a computer device in an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more apparent, the present application will be further described in detail with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the present application.
It should be noted that although a logical order is illustrated in the flowchart, in some cases, the steps illustrated or described may be performed in an order different from that in the flowchart. The terms first, second and the like in the description and in the claims and in the above-described figures, are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order, and it should be understood that the data so used may be interchanged, if appropriate, such that the embodiments of the present application described herein may be implemented in other than those illustrated or described herein.
Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of the present application only and is not intended to be limiting of the present application.
Furthermore, the described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are provided to give a thorough understanding of embodiments of the present application. One skilled in the relevant art will recognize, however, that the aspects of the application can be practiced without one or more of the specific details, or with other methods, components, devices, steps, etc. In other instances, well-known methods, devices, implementations, or operations are not shown or described in detail to avoid obscuring aspects of the application.
The flow diagrams depicted in the figures are exemplary only, and not necessarily all of the elements and operations/steps are included or performed in the order described. For example, some operations/steps may be decomposed, and some operations/steps may be combined or partially combined, so that the order of actual execution may be changed according to actual situations.
Referring to fig. 1, fig. 1 is a flow chart of an unmanned aerial vehicle detection method based on infrared thermal imaging according to an embodiment of the present invention, where the method includes the following steps:
step S110, traversing according to a given step length in a thermal imaging image to be detected, and synchronously marking each pixel point obtained by traversing according to a given intensity threshold condition;
step S120, an effective pixel point set is constructed, and all pixel points marked and meeting the given intensity threshold condition are stored into the effective pixel point set;
step S130, judging whether the effective pixel point set is an empty set or not; if yes, go to step S140; if not, executing step S150;
step S140, generating and recording that the detection result of the unmanned aerial vehicle does not exist in the thermal imaging image to be detected;
step S150, splitting the effective pixel point set according to a given coordinate proximity condition to form a plurality of effective pixel point subsets;
step S160, selecting all effective pixel point subsets meeting given rectangular construction standards from the plurality of effective pixel point subsets, and acquiring all maximum rectangular frames correspondingly covered by all effective pixel point subsets;
and S170, intercepting all rectangular images corresponding to all the maximum rectangular frames from the thermal imaging images to be detected, and processing all the rectangular images by using a trained unmanned aerial vehicle detection model to obtain a detection result.
In embodiments of the present invention, the given step size determined for assisting in traversing the image may preferably be set to 2 pixels; and determining that the infrared heat source intensity threshold for auxiliary marking pixel points, which is given in advance by the technician, can select an integer value in the range of [0,255], and the invention is preferably set to be 200.
In the embodiment of the present invention, the upper left Fang Dingdian pixel of the thermal imaging image to be measured is taken as the origin to establish a pixel coordinate system, the abscissa axis of which is along the direction of the broadside of the thermal imaging image to be measured, and the ordinate axis of which is along the direction of the high side of the thermal imaging image to be measured, and the specific implementation manner of the step S110 includes, but is not limited to, the following:
starting from the pixel point with the coordinate information of (1, 1) in the thermal imaging image to be detected, traversing the pixel point of the thermal imaging image to be detected line by line and then column by column (or line by line) according to the given step length, and judging and marking the pixel point when one pixel point is obtained, namely: judging whether the intensity value of the infrared radiation associated with the acquired pixel points is larger than or equal to the intensity threshold value of the infrared heat source; if the number is greater than or equal to the preset value, marking a forward judgment result for the obtained pixel point; if the pixel point is smaller than the preset value, marking a negative judgment result for the acquired pixel point; wherein the positive judgment result indicates that the marked pixel points meet the given intensity threshold condition, and can be preferably set to be 1; the negative determination result indicates that the marked pixel does not meet the given intensity threshold condition, and may preferably be set to 0.
When the width of the thermal imaging image to be detected is W and the height of the thermal imaging image to be detected is H, the following description is made for the traversal process of the thermal imaging image to be detected:
(1) Taking pixel point traversal of the ith row of the thermal imaging image to be detected as an example, the method specifically comprises the following steps:
a1, acquiring pixel points with coordinate information of (x, i);
a2, judging whether x+Q is less than or equal to W, wherein Q is the given step length; if the value is smaller than or equal to the value, assigning x+Q to x, and returning to execute the step A1; if the number is greater than the first threshold, finishing the traversing operation of the ith row;
the above step A1 is performed starting from x=1, and indicates that pixel traversal is regularly performed starting from the pixel with the coordinate information (1, i).
(2) Taking pixel point traversal for the j-th column of the thermal imaging image to be detected as an example, the method specifically comprises the following steps:
step B1, obtaining pixel points with coordinate information of (j, y);
step B2, judging whether y+Q is less than or equal to H; if the value is smaller than or equal to the value, assigning y+Q to y, and returning to execute the step B1; if the number is larger than the first row, finishing traversing operation of the j-th row;
the above step B1 is performed starting from y=1, and indicates that pixel traversal is regularly performed starting from the pixel whose coordinate information is (j, 1).
In the embodiment of the present invention, the specific implementation manner of the step S120 is as follows: and for all pixel points obtained by traversing in the thermal imaging image to be detected, each pixel point marked with the forward judgment result 1 is screened out from all the pixel points and is stored into the effective pixel point set.
In the embodiment of the present invention, the specific implementation process of the step S150 includes, but is not limited to, the following:
step S151, any one pixel point is moved out from the effective pixel point set, the moved pixel point is marked as a first pixel point, and the coordinate information of the first pixel point is (x r ,y r )；
Step S152, judging whether there are a plurality of pixel points meeting a given coordinate proximity condition between the effective pixel point set and the first pixel point; if yes, go to step S153; if not, executing step S154;
step S153, removing the plurality of pixel points from the effective pixel point set, forming an effective pixel point subset from the first pixel point and the plurality of pixel points, and continuing to execute step S155;
taking any one of the plurality of pixel points as an example, the coordinate information of the pixel point is (x) d ,y d ) The coincidence between the pixel point and the first pixel point in accordance with the given coordinate proximity condition should be understood as: i x d -x r Q is less than or equal to Q and y is less than or equal to y d -y r |≤Q；
Step S154, forming an effective pixel point subset by the first pixel points independently, and then continuing to execute step S155;
step S155, judging whether other pixel points exist in the effective pixel point set; if yes, returning to execute the step S151; if not, ending the splitting operation of the effective pixel point set.
In the embodiment of the present invention, assuming that the number of the plurality of effective pixel point subsets is N, N is a positive integer and N is greater than or equal to 1, the set screening process mentioned in the above step S160 includes, but is not limited to, the following:
step S161, acquiring an ith effective pixel point subset from N effective pixel point subsets, and marking all pixel points contained in the ith effective pixel point subset as all second pixel points;
step S162, acquiring coordinate information of all the second pixel points, and extracting a minimum abscissa value, a maximum abscissa value, a minimum ordinate value and a maximum ordinate value from the coordinate information;
step S163, judging whether the minimum abscissa value and the maximum abscissa value are equal; if the pixel values are equal, it is indicated that the abscissa range covered by all the second pixel points cannot be initially satisfied to form a rectangular frame, and the ith valid pixel point subset is deleted at this time, and then step S165 is executed; if not, executing step S164;
step S164, judging whether the minimum ordinate value and the maximum ordinate value are equal; if the pixel values are equal, it is indicated that the ordinate ranges covered by all the second pixel points cannot be continuously satisfied to form a rectangular frame, and the ith valid pixel point subset is deleted at this time, and then step S165 is executed; if not, it is indicated that the abscissa range and the ordinate range covered by all the second pixel points can be completely satisfied to form a rectangular frame, and at this time, the ith valid pixel point subset is reserved, and then step S165 is executed;
step S165, judging whether i+1 is less than or equal to N; if not, assigning i+1 to i, and returning to execute the step S161; if so, ending the screening of the N effective pixel point subsets.
In the above step S163, the abscissa range covered by all the second pixel points cannot be initially satisfied to form a rectangular frame, which can be understood as the following two cases: the first case is that the number of the second pixel points is 1, and the second case is that the second pixel points fall on the same vertical line, and the vertical line is parallel to the ordinate axis of the pixel coordinate system.
In the above step S164, the ordinate range covered by all the second pixel points cannot be continuously satisfied to form a rectangular frame, which can be understood as: all second pixel points fall on the same horizontal line, and the horizontal line is parallel to the abscissa axis of the pixel coordinate system.
It should be noted that, the step S161 is performed starting from i=1; the present invention allows the detection and judgment of the two ordinate values and then the detection and judgment of the two abscissa values, and it can be understood that the execution sequence of the step S163 and the step S164 is exchanged, which is not limited in the present invention.
In the embodiment of the present invention, assuming that M effective pixel point subsets are reserved after the N effective pixel point subsets are screened, M is a positive integer, M is greater than or equal to 1 and less than or equal to N, and the maximum rectangular frame acquisition process mentioned in the above step S160 includes, but is not limited to, the following:
s166, acquiring a j-th effective pixel point subset from the M effective pixel point subsets, and marking all pixel points contained in the j-th effective pixel point subset as all third pixel points;
step S167, obtaining the coordinate information of all the third pixel points, and extracting the minimum abscissa value (denoted as x min ) Maximum abscissa value (denoted as x max ) Minimum ordinate (denoted as y min ) And a maximum ordinate value (denoted as y max )；
Step S168, combining the minimum abscissa value, the maximum abscissa value, the minimum ordinate value and the maximum ordinate value extracted in the above step S167 in pairs to obtain coordinate information (x) min ,y min )、(x min ,y max )、(x max ,y min ) And (x) max ,y max ) Is included in the image data;
step S169, constructing a maximum rectangular frame with an explicit coordinate range, which is covered by the jth effective pixel point subset, based on the four vertex pixels;
step S1610, judging whether j+1 is less than or equal to M; if not, assigning j+1 to j, and returning to execute the step S166; if the number is greater than the number, ending the acquisition of M maximum rectangular frames corresponding to the M effective pixel point subsets;
note that, the above step S166 is performed starting from j=1.
In the embodiment of the present invention, it is first determined that the intensity gradient for auxiliary image data enhancement given in advance by the technician may be preferably set to 1, and the training process of the unmanned aerial vehicle detection model mentioned in the above step S170 includes, but is not limited to, the following:
step C1, acquiring a training image set acquired by a thermal imaging principle, wherein each training image comprises at least one rectangular frame, the at least one rectangular frame is marked with an unmanned aerial vehicle detection result L in advance, when l=1 is preferably set, the marked rectangular frame comprises an unmanned aerial vehicle, and when l=0 is preferably set, the marked rectangular frame does not comprise an unmanned aerial vehicle;
step C2, constructing an optimal training image set, wherein the optimal training image set already contains the training image set;
step C3, any training image is moved out from the training image set, and the number of pixel points contained in the training image is defined as K;
step C4, screening a plurality of pixel points with the infrared radiation intensity value smaller than the infrared heat source intensity threshold value from the K pixel points according to the infrared radiation intensity values associated with the K pixel points, and continuously screening the maximum infrared radiation intensity value from the infrared radiation intensity values associated with the plurality of pixel points;
step C5, obtaining a sum value between the intensity gradient and the maximum infrared radiation intensity value, and judging whether the sum value is smaller than the infrared heat source intensity threshold value or not; if the number is smaller than the preset number, executing the step C6; if the data is greater than or equal to the training image, the training image is discarded, and then the step C7 is executed;
step C6, respectively adding the infrared radiation intensity values associated with the plurality of pixel points screened in the step C4 with the intensity gradient to obtain a new training image, storing the new training image into the optimal training image set and the training image set, and continuously executing the step C7;
step C7, judging whether other training images exist in the training image set; if yes, returning to execute the step C3; if not, executing the step C8;
and step C8, training the pre-built unmanned aerial vehicle detection model through the optimal training image set so as to obtain a trained unmanned aerial vehicle detection model.
It should be noted that, the number of training images included in the optimal training image set used in the final training model must be greater than or equal to the number of training images originally included in the training image set.
It should be noted that the unmanned aerial vehicle detection model adopted in the invention is actually a two-class model, and specifically, a support vector machine (SVM, support Vector Machines) model, a naive bayes class model or other neural network model can be adopted, which is not limited in the invention.
In the embodiment of the present invention, each of the M largest rectangular frames has an explicit coordinate range, and the specific implementation process of the step S170 includes, but is not limited to, the following:
s171, M rectangular images matched with M coordinate ranges associated with the M largest rectangular frames are cut out from the thermal imaging image to be detected;
step S172, obtaining an mth rectangular image from the M rectangular images;
step S173, detecting the mth rectangular image by using a trained unmanned aerial vehicle detection model to obtain a detection result of the mth rectangular image, where the detection result is l=1 or l=0;
step 174, judging whether m+1 is less than or equal to M; if the value is less than or equal to the preset value, m+1 is assigned to m, and the step S172 is executed again; if yes, go to step S175;
step S175, integrating M detection results associated with the M rectangular images to obtain a detection result of the thermal imaging image to be detected, which is specifically expressed as follows:
(1) When the M detection results are L=0, generating detection result output representing that no unmanned aerial vehicle exists in the thermal imaging image to be detected;
(2) When a plurality of detection results are L=1 in the M detection results, counting the number of the detection results and a plurality of maximum rectangular frames associated with the detection results, taking the number of the detection results as the number of unmanned aerial vehicles obtained by detection, and taking the left upper corner vertex coordinate and the right lower corner vertex coordinate of each maximum rectangular frame as coordinate information of a corresponding unmanned aerial vehicle in the thermal imaging image to be detected for each maximum rectangular frame in the plurality of maximum rectangular frames, thereby generating and outputting detection results representing the number of unmanned aerial vehicles and the positions of the unmanned aerial vehicles in the thermal imaging image to be detected;
note that, the step S172 is performed starting from m=1.
In the embodiment of the invention, the number of training images and the detection difficulty can be increased by carrying out data enhancement on the acquired training image set in the model training stage, namely, carrying out small-amplitude increase on the infrared radiation intensity value associated with the pixel points which do not accord with the given intensity threshold condition in each training image, thereby effectively improving the detection capability and reliability of the unmanned aerial vehicle detection model after training; traversing the thermal imaging image to be detected by customizing the pixel-level searching step length is beneficial to improving the execution efficiency of the whole detection method, so that the detection result of the thermal imaging image to be detected is obtained more quickly.
Referring to fig. 2, fig. 2 is a schematic diagram illustrating a composition of an unmanned aerial vehicle detection system based on infrared thermal imaging according to an embodiment of the present invention, where the system includes:
a first module 210, configured to traverse the thermal imaging image to be detected according to a given step length, and mark each pixel obtained by the traversing according to a given intensity threshold condition synchronously;
a second module 220, configured to construct an effective pixel point set, and store all pixel points that have been assigned a label and meet the given intensity threshold condition into the effective pixel point set;
a third module 230, configured to divide the set of valid pixels into a plurality of sub-sets of valid pixels according to a given coordinate proximity condition, when the set of valid pixels is in a non-empty state;
a fourth module 240, configured to screen out all effective pixel point subsets that meet a given rectangular construction standard from the plurality of effective pixel point subsets, and obtain all maximum rectangular frames correspondingly covered by all effective pixel point subsets;
and a fifth module 250, configured to intercept all rectangular images associated with the all largest rectangular frames from the thermal imaging image to be detected, and process the all rectangular images by using a trained unmanned aerial vehicle detection model to obtain a detection result.
The content in the above method embodiment is applicable to the system embodiment, and functions implemented by the system embodiment are the same as those of the method embodiment, and beneficial effects achieved by the system embodiment are the same as those of the method embodiment, and are not repeated herein.
In addition, the embodiment of the invention further provides a computer readable storage medium, and a computer program is stored on the computer readable storage medium, and when the computer program is executed by a processor, the unmanned aerial vehicle detection method based on infrared thermal imaging in the embodiment is realized. The computer readable storage medium includes, but is not limited to, any type of disk including floppy disks, hard disks, optical disks, CD-ROMs, and magneto-optical disks, ROMs (Read-Only memories), RAMs (Random Access Memory, random access memories), EPROMs (Erasable Programmable Read-Only memories), EEPROMs (Electrically Erasable ProgrammableRead-Only memories), flash memories, magnetic cards, or optical cards. That is, a storage device includes any medium that stores or transmits information in a readable form by a device (e.g., a computer, a cell phone, etc.), which can be a read-only memory, a magnetic or optical disk, etc.
In addition, fig. 3 is a schematic hardware structure of a computer device according to an embodiment of the present invention, where the computer device includes a processor 320, a memory 330, an input unit 340, and a display unit 350. It will be appreciated by those skilled in the art that the device architecture shown in fig. 3 does not constitute a limitation of all devices, and may include more or fewer components than shown, or may combine certain components. The memory 330 may be used to store the computer program 310 and the functional modules, and the processor 320 runs the computer program 310 stored in the memory 330 to perform various functional applications and data processing of the device. The memory may be or include an internal memory or an external memory. The internal memory may include read-only memory (ROM), programmable ROM (PROM), electrically Programmable ROM (EPROM), electrically Erasable Programmable ROM (EEPROM), flash memory, or random access memory. The external memory may include a hard disk, floppy disk, USB flash disk, tape, etc. The memory 330 disclosed in embodiments of the present invention includes, but is not limited to, those types of memory described above. The memory 330 disclosed in the embodiments of the present invention is by way of example only and not by way of limitation.
The input unit 340 is used for receiving input of a signal and receiving keywords input by a user. The input unit 340 may include a touch panel and other input devices. The touch panel can collect touch operations on or near the touch panel by a user (such as operations of the user on or near the touch panel by using any suitable object or accessory such as a finger, a stylus, etc.), and drive the corresponding connection device according to a preset program; other input devices may include, but are not limited to, one or more of a physical keyboard, function keys (e.g., play control keys, switch keys, etc.), a trackball, mouse, joystick, etc. The display unit 350 may be used to display information input by a user or information provided to the user and various menus of the terminal device. The display unit 350 may take the form of a liquid crystal display, an organic light emitting diode, or the like. Processor 320 is a control center of the terminal device that uses various interfaces and lines to connect the various parts of the overall device, perform various functions and process data by running or executing software programs and/or modules stored in memory 330, and invoking data stored in memory 330.
As an embodiment, the computer device comprises a processor 320, a memory 330 and a computer program 310, wherein the computer program 310 is stored in the memory 330 and configured to be executed by the processor 320, the computer program 310 being configured to perform one of the above-described embodiments of the method for unmanned aerial vehicle detection based on infrared thermal imaging.
The terms "comprises" and "comprising," along with any variations thereof, in the description of the present application and in the above-described figures, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed or inherent to such process, method, article, or apparatus.
In this application, it should be understood that "at least one" means one or more, and "a plurality" means two or more. "and/or" for describing the association relationship of the association object, the representation may have three relationships, for example, "a and/or B" may represent: only a, only B and both a and B are present, wherein a, B may be singular or plural. The character "/" generally indicates that the context-dependent object is an "or" relationship. "at least one of" or the like means any combination of these items, including any combination of single item(s) or plural items(s). For example, at least one (one) of a, b or c may represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", wherein a, b, c may be single or plural.
Although the description of the present application has been described in considerable detail and with particularity with respect to several illustrated embodiments, it is not intended to be limited to any such detail or embodiments or any particular embodiment, but is to be construed as providing broad interpretation of such claims by reference to the appended claims, taking into account the prior art to which such claims are entitled to effectively encompass the intended scope of this application. Furthermore, the foregoing description of the embodiments contemplated by the inventors has been presented for the purpose of providing a useful description, and yet insubstantial changes to the invention that are not presently contemplated may represent equivalents of the invention.
Claims (10)
1. An unmanned aerial vehicle detection method based on infrared thermal imaging, which is characterized by comprising the following steps:
traversing according to a given step length in a thermal imaging image to be detected, and synchronously marking each pixel point obtained by traversing according to a given intensity threshold condition;
constructing an effective pixel point set, and storing all the pixel points marked and meeting the given intensity threshold condition into the effective pixel point set;
when the effective pixel point set is a non-empty set, splitting the effective pixel point set according to a given coordinate proximity condition to form a plurality of effective pixel point subsets;
selecting all effective pixel point subsets meeting given rectangular construction standards from the plurality of effective pixel point subsets, and acquiring all maximum rectangular frames correspondingly covered by all effective pixel point subsets;
and intercepting all rectangular images corresponding to all the maximum rectangular frames from the thermal imaging images to be detected, and processing all the rectangular images by using a trained unmanned aerial vehicle detection model to obtain a detection result.
2. The unmanned aerial vehicle detection method based on infrared thermal imaging according to claim 1, wherein the trained unmanned aerial vehicle detection model is obtained by:
acquiring a training image set, wherein each training image comprises a rectangular frame marked with a detection result of the unmanned aerial vehicle;
constructing an optimal training image set, wherein the optimal training image set comprises the training image set;
removing any training image from the training image set;
acquiring a plurality of corresponding pixel points with the infrared radiation intensity value smaller than a given infrared heat source intensity threshold value from the training image, and acquiring a corresponding maximum infrared radiation intensity value from the plurality of pixel points;
judging whether the sum value between the maximum infrared radiation intensity value and a given intensity gradient is smaller than the infrared heat source intensity threshold value or not; if yes, adding the infrared radiation intensity values corresponding to the pixel points with the intensity gradient quantity to obtain a new training image, and storing the new training image into the training image set and the optimal training image set;
judging whether the training image set is an empty set or not; if yes, training the built unmanned aerial vehicle detection model by using the optimal training image set to obtain a trained unmanned aerial vehicle detection model; if not, returning to the step of removing any training image from the training image set.
3. The method for detecting an unmanned aerial vehicle based on infrared thermal imaging according to claim 1, wherein the synchronizing the step of marking each pixel obtained by the traversal according to a given intensity threshold condition comprises:
when each pixel point is obtained in the traversal process of the thermal imaging image to be detected, judging whether the infrared radiation intensity value corresponding to the pixel point is greater than or equal to a given infrared heat source intensity threshold value; if yes, a positive judgment result is given to the pixel point so as to represent that the pixel point meets the given intensity threshold condition; if not, a negative judgment result is given to the pixel point so as to represent that the pixel point does not meet the given intensity threshold condition.
4. The method of claim 1, wherein splitting the set of active pixels into a plurality of sub-sets of active pixels according to a given coordinate proximity condition comprises:
any pixel point is removed from the effective pixel point set;
judging whether a plurality of pixel points with the horizontal coordinate difference value and the vertical coordinate difference value of the pixel points not exceeding the given step length exist in the effective pixel point set; if yes, the plurality of pixel points are moved out and an effective pixel point subset is formed with the pixel points; if not, the pixel points are singly formed into an effective pixel point subset;
and executing the two operations circularly until the effective pixel point set becomes an empty set.
5. The unmanned aerial vehicle detection method based on infrared thermal imaging according to claim 1, wherein the given rectangular construction criteria are: in all pixel points contained in each subset of valid pixel points, there are unequal maximum and minimum abscissa values, and unequal maximum and minimum ordinate values.
6. The method for detecting an unmanned aerial vehicle based on infrared thermal imaging according to claim 1, wherein the acquiring all the largest rectangular frames correspondingly covered by the subset of all the effective pixels comprises:
for each effective pixel point subset meeting a given rectangle construction standard, acquiring a maximum abscissa value, a minimum abscissa value, a maximum ordinate value and a minimum ordinate value from all pixel points according to coordinate information of all pixel points contained in the effective pixel point subset;
determining four different vertex pixels according to the maximum abscissa value, the minimum abscissa value, the maximum ordinate value and the minimum ordinate value;
and determining the largest rectangular frame covered by the effective pixel point subset according to the four vertex pixels.
7. The unmanned aerial vehicle detection method based on infrared thermal imaging according to claim 1, wherein when the effective pixel point set is an empty set, detection result output recording that no unmanned aerial vehicle exists in the thermal imaging image to be detected is generated.
8. An unmanned aerial vehicle detection system based on infrared thermal imaging, the system comprising:
the first module is used for traversing according to a given step length in the thermal imaging image to be detected and marking each pixel point obtained through traversing synchronously according to a given intensity threshold condition;
the second module is used for constructing an effective pixel point set, and storing all the pixel points marked and meeting the given intensity threshold condition into the effective pixel point set;
the third module is used for splitting the effective pixel point set into a plurality of effective pixel point subsets according to a given coordinate proximity condition when the effective pixel point set is a non-empty set;
a fourth module, configured to select, from the plurality of effective pixel point subsets, all effective pixel point subsets that meet a given rectangular construction standard, and obtain all maximum rectangular frames that are correspondingly covered by all effective pixel point subsets;
and a fifth module, configured to intercept all rectangular images corresponding to the all maximum rectangular frames in the thermal imaging image to be detected, and process all rectangular images by using a trained unmanned aerial vehicle detection model to obtain a detection result.
9. A computer device comprising a memory and a processor, the memory storing a computer program, wherein the processor executes the computer program to implement the infrared thermal imaging based drone detection method of any one of claims 1 to 7.
10. A computer readable storage medium, on which a computer program is stored, characterized in that the computer program, when being executed by a processor, implements the infrared thermal imaging based drone detection method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311416483.9A CN117537929B (en) | 2023-10-27 | Unmanned aerial vehicle detection method, system, equipment and medium based on infrared thermal imaging |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311416483.9A CN117537929B (en) | 2023-10-27 | Unmanned aerial vehicle detection method, system, equipment and medium based on infrared thermal imaging |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN117537929A true CN117537929A (en) | 2024-02-09 |
| CN117537929B CN117537929B (en) | 2024-07-16 |
Family
ID=
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106778746A (en) * | 2016-12-23 | 2017-05-31 | 成都赫尔墨斯科技有限公司 | A kind of anti-unmanned plane method of multiple target |
| CN106910203A (en) * | 2016-11-28 | 2017-06-30 | 江苏东大金智信息***有限公司 | The method for quick of moving target in a kind of video surveillance |
| KR20210099450A (en) * | 2020-02-04 | 2021-08-12 | 한국해양대학교 산학협력단 | Far away small drone detection method Using Deep Learning |
| CN113283411A (en) * | 2021-07-26 | 2021-08-20 | 中国人民解放军国防科技大学 | Unmanned aerial vehicle target detection method, device, equipment and medium |
| CN113487517A (en) * | 2021-07-26 | 2021-10-08 | 中国人民解放军国防科技大学 | Unmanned aerial vehicle target detection method, device and equipment based on image enhancement |
| KR20220107494A (en) * | 2021-01-25 | 2022-08-02 | 한국해양대학교 산학협력단 | Drone detection system based on deep learning using SWIR camera |
| CN116188528A (en) * | 2023-01-10 | 2023-05-30 | 中国人民解放军军事科学院国防科技创新研究院 | RGBT unmanned aerial vehicle target tracking method and system based on multi-stage attention mechanism |
| CN116681962A (en) * | 2023-05-05 | 2023-09-01 | 江苏宏源电气有限责任公司 | Power equipment thermal image detection method and system based on improved YOLOv5 |
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106910203A (en) * | 2016-11-28 | 2017-06-30 | 江苏东大金智信息***有限公司 | The method for quick of moving target in a kind of video surveillance |
| CN106778746A (en) * | 2016-12-23 | 2017-05-31 | 成都赫尔墨斯科技有限公司 | A kind of anti-unmanned plane method of multiple target |
| KR20210099450A (en) * | 2020-02-04 | 2021-08-12 | 한국해양대학교 산학협력단 | Far away small drone detection method Using Deep Learning |
| KR20220107494A (en) * | 2021-01-25 | 2022-08-02 | 한국해양대학교 산학협력단 | Drone detection system based on deep learning using SWIR camera |
| CN113283411A (en) * | 2021-07-26 | 2021-08-20 | 中国人民解放军国防科技大学 | Unmanned aerial vehicle target detection method, device, equipment and medium |
| CN113487517A (en) * | 2021-07-26 | 2021-10-08 | 中国人民解放军国防科技大学 | Unmanned aerial vehicle target detection method, device and equipment based on image enhancement |
| CN116188528A (en) * | 2023-01-10 | 2023-05-30 | 中国人民解放军军事科学院国防科技创新研究院 | RGBT unmanned aerial vehicle target tracking method and system based on multi-stage attention mechanism |
| CN116681962A (en) * | 2023-05-05 | 2023-09-01 | 江苏宏源电气有限责任公司 | Power equipment thermal image detection method and system based on improved YOLOv5 |
Non-Patent Citations (3)
| Title |
|---|
| 李青;袁家政;刘宏哲;: "基于目标识别与显著性检测的图像场景多对象分割", 计算机科学, no. 05, 15 May 2017 (2017-05-15) * |
| 苑智玮;黄树彩;熊志刚;胡立忠;: "基于SVM后验概率的红外弱小目标检测", 激光与红外, no. 06, 20 June 2016 (2016-06-20) * |
| 陈冠楠;杨坤涛;谢志明;滕忠坚;陈荣;: "基于深度优先遍历的图像边缘检测方法", 应用光学, no. 01, 15 January 2008 (2008-01-15) * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11004129B2 (en) | Image processing | |
| Wolf et al. | Evaluation of video activity localizations integrating quality and quantity measurements | |
| Boult et al. | Into the woods: Visual surveillance of noncooperative and camouflaged targets in complex outdoor settings | |
| CN102830958B (en) | A kind of method and system for obtaining interface control information | |
| US9087267B2 (en) | Image scene recognition | |
| US8103109B2 (en) | Recognizing hand poses and/or object classes | |
| WO2021196389A1 (en) | Facial action unit recognition method and apparatus, electronic device, and storage medium | |
| WO2021088422A1 (en) | Application message notification method and device | |
| WO2021090771A1 (en) | Method, apparatus and system for training a neural network, and storage medium storing instructions | |
| CN111932544A (en) | Tampered image detection method and device and computer readable storage medium | |
| WO2023165616A1 (en) | Method and system for detecting concealed backdoor of image model, storage medium, and terminal | |
| CN107341139A (en) | Multimedia processing method and device, electronic equipment and storage medium | |
| CN108537820B (en) | Dynamic prediction method, system and applicable equipment | |
| Wang et al. | Enhanced visual analysis for cluster tendency assessment and data partitioning | |
| CN111951283A (en) | Medical image identification method and system based on deep learning | |
| Huang et al. | Detecting shadows in high-resolution remote-sensing images of urban areas using spectral and spatial features | |
| CN117537929B (en) | Unmanned aerial vehicle detection method, system, equipment and medium based on infrared thermal imaging | |
| Zhou et al. | Unsupervised saliency detection via knn mechanism and object-biased prior | |
| CN113037925A (en) | Information processing method, information processing apparatus, electronic device, and readable storage medium | |
| CN117537929A (en) | Unmanned aerial vehicle detection method, system, equipment and medium based on infrared thermal imaging | |
| US11068121B2 (en) | System and method for visual exploration of subnetwork patterns in two-mode networks | |
| CN112148171B (en) | Interface switching method and device and electronic equipment | |
| CN114639058A (en) | Fire smoke image detection method, device, equipment and storage medium | |
| CN113282780A (en) | Picture management method and device, electronic equipment and readable storage medium | |
| US11048713B2 (en) | System and method for visual exploration of search results in two-mode networks |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant |