CN112507865B - Smoke identification method and device - Google Patents
Smoke identification method and device Download PDFInfo
- Publication number
- CN112507865B CN112507865B CN202011415037.2A CN202011415037A CN112507865B CN 112507865 B CN112507865 B CN 112507865B CN 202011415037 A CN202011415037 A CN 202011415037A CN 112507865 B CN112507865 B CN 112507865B
- Authority
- CN
- China
- Prior art keywords
- smoke
- image
- point
- pixel
- detected
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/25—Fusion techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/25—Determination of region of interest [ROI] or a volume of interest [VOI]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
- G06V10/267—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion by performing operations on regions, e.g. growing, shrinking or watersheds
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Physics & Mathematics (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Bioinformatics & Computational Biology (AREA)
- General Engineering & Computer Science (AREA)
- Artificial Intelligence (AREA)
- Life Sciences & Earth Sciences (AREA)
- Multimedia (AREA)
- Fire-Detection Mechanisms (AREA)
- Image Analysis (AREA)
Abstract
A method and a device for identifying visible smoke of a power transmission line channel are used for solving the problem that the smoke detectable rate is low in the existing technology for identifying visible image smoke of the power transmission line channel. Calculating a loss function in the smoke image sample through a target detection algorithm; training the target detection algorithm through a smoke image sample to obtain a smoke image detection model for determining the position of a smoke area in an image to be detected; carrying out image segmentation on a smoke region in an image to be detected, taking a binary image of the segmented image as a Mask image, and calculating the smoke drifting direction; converting a smoke area in the image to be detected from an RGB color space to an HSV color space, and calculating accumulated values of pixel values of the segmented image in a brightness V space, which are horizontally and vertically projected respectively in a Mask image; and determining the smoke concentration according to the smoke drifting direction and the accumulated value. By the method, the smoke detection rate is improved, and the false alarm rate is reduced.
Description
Technical Field
The invention relates to the technical field of information processing, in particular to a method and a device for identifying smoke in a visualized image of a power transmission line channel.
Background
When a fire disaster happens near the overhead transmission line channel, the insulation clearance of the transmission line is extremely easy to reduce to induce the tripping of the transmission line, and further, a large-range long-time power failure accident is caused. Therefore, the fire monitoring is carried out on the transmission line channel, the accuracy of the fire monitoring is improved, and the reliable operation of the power grid is effectively guaranteed.
The existing single-image smoke monitoring method is limited by uncertainty of smoke forms, and the colors, shapes, areas and directions in the smoke forms are unstable, so that the feature extraction is difficult, the smoke monitoring method is easily interfered by atmospheric cloud and mist, and the false alarm rate of smoke detection are high particularly in plum rain seasons or regions such as Tibet and the like. In addition, in a field environment, for example, when straw burning in a field is detected, due to a large number of interference factors, the expected accuracy rate cannot be achieved by using the image processing method alone.
Disclosure of Invention
In order to overcome the defects of the technology, the invention provides a smoke identification method and a smoke identification device for reducing the false alarm rate and the false alarm rate of the smoke identification device during smoke detection.
The technical scheme adopted by the invention for overcoming the technical problems is as follows:
a smoke identification method comprising the steps of:
a) Calculating a loss function of a smoke concentration central point in a visualized smoke image sample of a power transmission line channel, a classification loss function and a regression loss function through a target detection algorithm, wherein the smoke concentration central point is the highest point of smoke concentration marked in the smoke image sample in advance;
b) Training a target detection algorithm through a smoke image sample to obtain a trained smoke image detection model;
c) Determining the position of a smoke area in the image to be detected according to the trained smoke image detection model;
d) Carrying out image segmentation on a smoke area in an image to be detected, taking a binary image of the segmented image as a Mask image, and calculating the smoke drifting direction;
e) Converting a smoke region in an image to be detected from an RGB (Red, green, blue) color space to an HSV (Hue, saturation, value) color space, and calculating the accumulated values of the pixel values of the segmented image in a brightness V space projected horizontally and vertically in the Mask image;
f) And determining the smoke concentration according to the smoke drifting direction and the accumulated value.
When the target detection algorithm is modified, the loss function of the central point in the target detection algorithm is modified, namely, the central point which is labeled in advance is adopted to directly calculate. The calculation steps are reduced, and the calculation time is shortened. And moreover, the concentration value of the smoke is calculated through the floating direction of the smoke and the horizontal and vertical projection of the smoke on the brightness space pixel, and images such as clouds and the fog are distinguished through the calculation of the concentration value, so that the false alarm rate is reduced. The embodiment of the application relates to a deep learning smoke detection method based on a convolutional neural network and a discrimination method based on shape and position correlation. The two methods are adopted to detect the smoke image to be detected successively, so that the higher detection rate of the smoke is ensured, and partial false alarms can be filtered.
Further, after the smoke concentration is determined by the smoke scattering direction and the accumulated value, namely after the step f), the method further comprises the following steps: and under the condition that the concentration values of the area to be detected in the smoke image to be detected from bottom to top are detected to be sequentially reduced, determining that the area to be detected is a smoke area.
Further, training a target detection algorithm through a smoke image sample, wherein in the specific step a), the target detection algorithm is trained through a function:
calculating a classification loss function and a regression loss function toAnd determining a loss function of the smoke concentration center point, wherein the loss L is classified cls Center point Loss L using the Focal local Loss function center-ness Using the BCE loss function, the regression loss L reg With the use of a weighted GIoU loss function,
taking a smoke label value set when the data set is constructed when the image sample is a positive sample, and taking 0 when the image sample is a negative sample, wherein the smoke label value is a numerical value set by determining the smoke density, N pos To mark the number of positive samples, λ 1 And λ 2 Are all weight values, p x,y In order to score the classification of the object,
output the classification score, t, for the neural network x,y In order to obtain the target regression score,
outputting a regression score for the neural network, centerness x,y The score is given to the center point of the target,
and outputting the central point score for the neural network.
According to the embodiment of the application, the characteristic that the concentration of smoke is gradually reduced from bottom to top is utilized, the concentration value of a region to be detected from bottom to top is determined, so that the smoke is distinguished from the cloud or the mist, and the false alarm rate of a smoke image detection model during smoke detection is reduced. Further, step e) comprises the following steps:
e-1) by the formula Sum i ＝pixel+sum i Ifmask =0, calculating an accumulated value Sum of pixel values projected horizontally and vertically in the Mask image, respectively i In the formula, i is a horizontal axis or a vertical axis, pixels are pixels, mask is a mask binary image, pixels =1 when mask =0, and pixels =0 when mask ≠ 0;
e-2) according to nongdu (x, y) = sum x ×sum y Calculating smokeThe density nongdu (x, y) is represented by x, y being the abscissa and ordinate of the pixel, y = x × tan θ, and θ being the angular direction of the divided image.
According to the embodiment of the application, the central point loss function is transformed, the central point does not need to be independently calculated, the point with the highest smoke concentration in the smoke image marked in advance is directly used as the central point in the central point loss function for calculation, the calculation steps are reduced, and the calculation process is simpler.
In one implementation of the present application, the regression loss is a weighted GIoU loss function, specifically, the regression loss is obtained according to the following formula: the regression loss L reg The regression loss was calculated using a weighted GIoU loss function by the following formula:
in the formula w j Representing a weighted weight; a. The wc Is the weighted union area, dist i Is the distance, dist, from the center point to the ith edge point j The distance from the central point to the jth edge point is defined as the central point of network output, and the center-point is the central point with the highest actual smoke concentration; x is a radical of a fluorine atom 1 ，y 1 Respectively represent a pairThe abscissa and ordinate, x, of the upper left corner of the rectangular frame marked by the smoke region 2 ，y 2 Respectively representing the abscissa and the ordinate of the right lower corner point of the marked rectangular frame; x is a radical of a fluorine atom 1 ′，y 1 ' the abscissa and ordinate of the upper left corner point of a rectangular frame representing the predicted smoke region, x, respectively 2 ′，y 2 ' denotes the abscissa and ordinate of the lower right corner of the rectangular frame of the predicted smoke region,
weighted for the abscissa of the upper left corner of the rectangular box,
weighted for the abscissa of the lower right corner of the rectangular box,
weighted for the ordinate of the upper left corner of the rectangular box,
weighting the ordinate of the lower right corner of the rectangular frame; a. The wc Is the weighted union area; u is the union area; ioU is the intersection to union ratio, w 1 For labeling the upper left corner (x) of the rectangular frame 1 ，y 1 ) Distance weight from the center point of the label, w 2 To mark the lower right corner (x) of the rectangular frame 2 ，y 2 ) Distance weight from the center point of the label, w 1 ' to predict the upper left corner of the rectangular box (x) 1 '，y 1 ') distance weight from predicted center point, w 2 ' to predict the lower right corner (x) of the rectangular frame 2 '，y 2 ') distance weight from the predicted center point.
By calculating the weighted regression loss, the accuracy of the training model is made higher. When the trained model is used for detecting the smoke image subsequently, the detection rate of the smoke can be improved, and the position of the rectangular frame for marking the detected smoke position is more accurate.
Further, in step d), the binary image of the segmented image is used as a Mask image, and the direction of the smoke drifting is calculated, specifically comprising:
d-1) establishing a coordinate axis by taking the intersection point of the left edge boundary line and the upper edge boundary line of the image to be detected as an original point, taking the rightward extending direction as an x axis and taking the downward extending direction as a y axis;
d-2) based on the spatial Moment algorithm
Determining the angle direction of a communication area, wherein the communication area is a smoke area after being segmented in the image to be detected, theta is the angle of the communication area, u 11 ，u 20 ，u 02 Is the center distance.
Further, the central point output by the smoke graph detection model is used as an initial seed point coordinate, and the image segmentation is carried out on the smoke area through a flooding filling method.
Further, before training the target detection algorithm through the smoke image samples in step b), the method further includes:
constructing a synthetic image training set by G (x, y) Gaussian weighting according to a function AP (x, y) = G (x, y) xSmoke (x, y) + (1-G (x, y)) × BP (x, y); wherein AP is a synthesized Smoke pixel, BP is a negative sample image pixel value before synthesis, and Smok is a Smoke pixel value.
Further, before training the target detection algorithm through the smoke image samples in step b), the method further includes: the collected smoke images and the power transmission channel hidden danger images are subjected to weighted fusion according to a function pixel _ after = alpha × pixel _ smoke + (1-alpha) × pixel _ before to obtain smoke image training sets with different concentrations, wherein the power transmission channel hidden danger images are images with smoke and/or images without smoke, the pixel _ before is an original pixel value of the power transmission channel hidden danger images, the pixel _ smoke is a pixel value of the smoke images, the pixel _ after is the pixel value of the smoke images after weighted fusion, and the alpha is a set smoke label value.
By carrying out weighted fusion on the smoke image and the power transmission channel hidden danger image, different label values can be taken from the alpha, so that smoke samples with different concentrations after weighted fusion are obtained, and a sample training set is further increased. Meanwhile, a smoke image sample set with power transmission and electrification can be added, and the detection rate of the smoke detection model after training on smoke in a power transmission channel scene is improved.
The invention also relates to a smoke recognition device comprising:
at least one processor; and (c) a second step of,
a memory communicatively coupled to the at least one processor; wherein, the first and the second end of the pipe are connected with each other,
the memory stores instructions executable by the at least one processor to enable the at least one processor to:
calculating a loss function of a smoke concentration central point in a smoke image sample, a classification loss function and a regression loss function through a target detection algorithm, wherein the smoke concentration central point is the highest point of smoke concentration marked in the smoke image sample in advance;
training the target detection algorithm through the smoke image sample to obtain a trained smoke image detection model;
determining the position of a smoke area in the image to be detected according to the trained smoke image detection model;
carrying out image segmentation on the smoke region in the image to be detected, taking a binary image of the segmented image as a Mask image, and calculating the smoke drifting direction;
converting the smoke region in the image to be detected from an RGB color space to an HSV color space, and calculating the accumulated values of the pixel values of the segmented image in a brightness V space projected horizontally and vertically in the Mask image;
and determining the smoke concentration according to the smoke drifting direction and the accumulated value.
The invention has the beneficial effects that: the method and the device for identifying the smoke provided by the embodiment of the application are used for training a target detection algorithm by using the constructed smoke image sample. The point with the highest smoke concentration is directly used as the central point for calculating the loss of the central point, so that the calculation process can be reduced, and the calculation time can be shortened. Meanwhile, the regression loss function is calculated in a weighting mode, so that the trained model can accurately mark the position of the smoke region. In addition, in order to reduce the false alarm rate, the embodiment of the application adopts a smoke filtering method based on the shape and the position correlation at the same time. The method makes clear distinction between the smoke and the cloud, the fog and the like by utilizing the difference between the smoke and the cloud, the fog and other images with higher false alarm rate in direction and concentration, namely, the smoke concentration is gradually reduced from bottom to top. Therefore, the embodiment of the application can improve the detectable rate of the smoke, reduce the false alarm rate in the detection process and improve the overall detectable rate of the smoke.
Drawings
FIG. 1 is a flow chart of a smoke identification method of the present invention;
FIG. 2 is a labeled diagram of a smoke image of the present invention;
FIG. 3 is a negative sample diagram of the present invention depicting cloud misreporting as smoke;
FIG. 4 is a smoke diagram of the present invention;
FIG. 5 is a comparison of a binary image before and after segmentation of an image to be measured according to the present invention;
fig. 6 is a schematic structural view of the smoke recognition device of the present invention.
Detailed Description
The invention is further described with reference to fig. 1 to 6.
The existing method for monitoring the smoke through a single image is limited by the characteristics of changeful smoke forms, colors, areas, directions and the like, so that the extraction of the smoke features in the image is difficult. And is easily affected by the atmospheric cloud and mist, so that the identified smoke accuracy is low. In addition, the existing mode of converting the smoke target image from the RGB color space into the HSV color space and the YCbCr color space, calculating a color histogram, and carrying out equalization and binarization processing on the color histogram has little effect on the detection of the smoke with concealment, so that the detection of the smoke under the field environment is inaccurate. Therefore, the existing detection method for the smoke is influenced by the environment and the weather, and the accuracy rate of smoke identification is not high.
In order to solve the above problem, the embodiments of the present application provide a method and an apparatus for smoke identification. When the target detection algorithm is trained, the regression loss function is calculated in a weighting mode, so that the trained smoke detection model can mark the smoke position more accurately, and the detection rate of smoke is improved. And when the center point loss is calculated, the point with the highest concentration marked out is directly adopted as the center point for calculation, so that the calculation steps are reduced. Meanwhile, the angle and the concentration of the smoke region in the image to be detected are calculated, and the image is distinguished from interference images such as fog and cloud, so that the false alarm rate in the smoke detection process is reduced.
Technical solutions proposed in embodiments of the present application are described in detail below with reference to the accompanying drawings.
Fig. 1 is a flowchart of a method for smoke identification according to an embodiment of the present disclosure. As shown in fig. 1, the smoke recognition method includes the steps of:
s101: the smoke recognition device constructs a training set required for training a target detection algorithm.
In one embodiment of the present application, a set of smoke image data is gathered as much as possible in a real business scene. And marking the smoke area in the collected smoke image, such as a rectangular frame. In one example, the range is marked starting from a higher concentration of smoke until the edges of the smoke are not visible to the naked eye.
It should be noted that a single smoke image may have multiple smoke regions, and therefore, multiple rectangular frames may be used to mark multiple smoke regions in the single smoke image.
In one embodiment of the application, the smoke concentration is determined and different smoke signature values are set.
Specifically, the smoke is divided into two grades according to the concentration of the smoke. The label value of smoke whose background is not visible to a large extent is set to 1, and the label value of smoke whose background is visible is set to 0.8. Meanwhile, the point with the highest smoke concentration can be marked by a round point.
For example, fig. 2 is a labeled diagram of a smoke image provided in an embodiment of the present application, where a range marked by a rectangular box is a smoke region, and a point labeled by a circular point a is a point with the highest smoke concentration.
In one embodiment of the application, in order to make up for the defect that a smoke data set is difficult to collect in a real service scene, a smoke image is synthesized in an image enhancement mode.
In one embodiment of the application, the smog image is collected through a network and the like, and the smog area in the collected smog image is scratched off. And then, according to a function pixel _ after = alpha × pixel _ smoke + (1-alpha) × pixel _ before, performing weighted fusion on the scratched smoke region and the power transmission channel hidden danger image to obtain smoke image training samples with different concentrations. The images of the hidden danger of the power transmission channel can be images with smoke and/or images without smoke.
In an embodiment of the application, in the weighted fusion function, pixel _ before is an original pixel value of the power transmission channel hidden danger image, pixel _ pause is a pixel value of the smoke image, pixel _ after is a pixel value of the smoke image after weighted fusion, and alpha is a weight and is also a set smoke label value.
It should be noted that when the extracted smoke region and the power transmission channel hidden danger image are subjected to weighted fusion, the weight alpha may be different values between 0 and 1. By generating a plurality of smoke images with different pixel values, the smoke effect under different concentrations is simulated.
In an embodiment of the application, when the neural network model before training detects smoke, images of partial fog and white cloud can be wrongly judged as smoke images. And (4) the images of fog, white cloud and the like are wrongly judged as the images of the smoke and are collected. Meanwhile, the smog area in the smog image collected through a network and other channels is scratched off, and the scratched off smog area image is pasted to the data set such as the fog and the white cloud. And the paste does not overlap with the target such as the white cloud, the fog and the like.
According to the embodiment of the application, the smoke area image is pasted on the misjudged negative sample set, and the pasted sample set is used for training the target detection algorithm to obtain the smoke detection model. The discrimination of the smoke sample, the samples such as white cloud, fog and the like is improved to a certain extent, and the false alarm rate can be reduced by more than 30%.
In one embodiment of the present application, the smoke region is pasted into the data set misjudged as smoke by means of gaussian weighting.
Specifically, a synthetic image training set is constructed according to a function AP (x, y) = G (x, y) × Smoke (x, y) + (1-G (x, y)) × BP (x, y). Wherein AP is a synthesized Smoke pixel, BP is a negative sample image pixel value before synthesis, smoke is a Smoke pixel value, and G (x, y) Gaussian weighting is used for constructing a synthesized image training set. G (x, y) may be represented by a Gaussian kernel function
Where σ is the bandwidth used to control the radial range of action of the function, and x, y are the currently calculated point coordinates, respectively.
According to the embodiment of the application, the smog area is pasted to the negative sample training set in a Gaussian kernel function weighting mode, and the problem that the pasted target boundary is obvious can be avoided. The interference generated when the target detection algorithm is trained is avoided, and the detection rate of smoke is further improved.
S102: and training the target detection algorithm by the smoke recognition device so as to obtain a trained smoke image detection model. And training the target detection algorithm by the smoke recognition device so as to obtain a trained smoke image detection model.
In an embodiment of the present application, an FCOS (full relational One-Stage Object Detection, single-Stage Object Detection algorithm based on anchor-free) Object Detection algorithm is selected as a base network, and a loss function thereof is modified.
In one embodiment of the present application, by a function:
three loss functions, L, for FCOS are determined cls To classify the loss, L reg As a regression loss parameter, L center-ness The center point loss parameter.
Among the above loss functions, the classification loss L cls Center point Loss L using the Focal local Loss function center-ness Using the BCE loss function, the regression loss L reg With the use of a weighted GIoU loss function,
taking a smoke label value set when the data set is constructed when the image sample is a positive sample, and taking 0 when the image sample is a negative sample, wherein the smoke label value is a numerical value set by determining the smoke density, N pos To mark the number of positive samples, λ 1 And λ 2 Are all weight values, p x,y In order to score the classification of the object,
output the classification score, t, for the neural network x,y In order to obtain the target regression score,
outputting a regression score for the neural network, centerness x,y The score is given to the center point of the object,
and outputting the central point score for the neural network.
In one embodiment of the present application, in calculating the loss function, the regression loss and the center point loss are calculated only for the smoke samples. The classification loss is calculated in both smoke and non-smoke samples.
In the above loss function, the center point used when calculating the center point loss is the point with the highest smoke concentration previously marked in the sample set. The coordinates of the center point of the image are not required to be calculated independently, and the calculation time and the calculation steps are reduced.
In one embodiment of the present application, the regression loss in the loss function employs a weighted GIoU loss function.
In particular, the pass function
Calculating a weighted regression loss, wherein w j Representing a weighted weight; a. The wc Is the weighted union area, dist i Is the distance, dist, from the center point to the ith edge point j The distance from the central point to the jth edge point is shown, the cornernet _ point is the central point of the network output, and the center-point is the central point with the highest actual smoke concentration; x is a radical of a fluorine atom 1 ，y 1 Respectively representing the abscissa and ordinate, x, of the upper left corner of a rectangular frame marking the smoke region 2 ′，y 2 ' respectively representing the abscissa and the ordinate of the right lower corner point of the marked rectangular frame; x is a radical of a fluorine atom 1 ′，y 1 ' X represents the abscissa and ordinate, respectively, of the upper left corner of the rectangular frame of the predicted smoke region 2 ′，y 2 ' respectively denote the abscissa and ordinate of the lower right corner point of the rectangular frame of the predicted smoke region,
weighted for the abscissa of the upper left corner of the rectangular box,
weighted for the abscissa of the lower right corner of the rectangular box,
weighted for the ordinate of the upper left corner of the rectangular box,
weighting the ordinate of the lower right corner of the rectangular frame; a. The wc Is the weighted union area; u is the union area; ioU is the intersection to union ratio, w 1 For labeling the upper left corner (x) of the rectangular frame 1 ，y 1 ) Distance weight from the center point of the label, w 2 To mark the lower right corner (x) of the rectangular frame 2 ，y 2 ) Distance weight from the center point of the annotation, w 1 ' to predict the upper left corner of the rectangular box (x) 1 '，y 1 ') distance weight from predicted center point, w 2 ' to predict the lower right corner (x) of the rectangular frame 2 '，y 2 ') distance weight from the predicted center point.
In one embodiment of the application, the coordinates of the edge points of the smoke marked area can be known according to the rectangular frame marked with the smoke area, and the coordinates of the central point can be known according to the marked point with the highest concentration.
As shown in equation (1), in the weighted loss function, the edge weight is the relative error from the center point to the edge point, and the edge point closer to the center point is weighted higher. And (3) selecting the marked rectangular frame and the predicted rectangular frame, and respectively corresponding one coordinate value smaller in the abscissa of the weighted upper left-hand corner coordinate point and the corresponding point smaller in the weighted upper left-hand corner ordinate according to the formula (2). And then, selecting the coordinate values with relatively larger weighted abscissa values and ordinate values in the lower right-hand corner coordinate points respectively. According to the formula (3), the weighted union area of the two rectangular frames is obtained according to the selected four coordinate points. In formula (4), a union area U is obtained according to the union of the labeling frame and the prediction frame. And obtaining the difference value between the weighted union area and the ratio 1 between the weighted union areas, wherein the difference value between the intersection ratio of the two rectangular frames and the ratio 1 is the weighted GIoU loss.
S103: and determining the position of the smoke in the image to be detected through the trained smoke detection model.
In an embodiment of the present application, the trained smoke detection model may detect a smoke region in an image to be detected. When the smoke is detected in the image to be detected, the smoke area is marked by using the rectangular frame.
According to the embodiment of the application, the smog detection model can be used for marking the smog area in the image to be detected. However, in practical application, images with radiation characteristics and image texture characteristics similar to those of smoke, such as clouds or fog, can be mistakenly reported as smoke with a certain probability. For example, fig. 3 is a negative sample graph of mistaking a cloud for smoke provided by the embodiment of the present application, and fig. 4 is a smoke graph provided by the embodiment of the present application, and it can be seen that the two graphs are not easily distinguished. Therefore, in order to reduce the false alarm rate, the embodiment of the present application further filters the marked smoke image according to the shape and the position correlation and the density thereof, so as to reduce the false alarm rate, and the specific implementation steps thereof are explained in S104-S106.
S104: and the smoke recognition device is used for segmenting the smoke region in the image to be detected and calculating the smoke drifting direction according to the segmented binary image.
In one embodiment of the application, a water filling method is adopted to segment the smoke area of the image to be detected. The initial seed point during segmentation is a central point coordinate output by the smoke detection model, namely a point coordinate with the highest smoke concentration in the image to be detected. The point pixel value is taken as the pixel value of the initial division.
In an embodiment of the application, the segmented binary image is used as a Mask image, and multi-neighborhood filling is performed on the Mask image to make up for segmentation holes. And calculating the drifting direction of the smoke in the segmented smoke region according to the Mask image.
Specifically, as shown in fig. 5, which is a comparison graph of a binary image before and after segmentation of an image to be measured provided by the embodiment of the present application, a black connected region in a right image is a segmentThe black dot in the lower left corner of the cut smoke region is the highest concentration dot. By spatial Moment algorithm
And calculating the angle between the central axis where the highest concentration point is located and the coordinate axis in the smoke area, namely the drifting direction of the smoke area. In an embodiment of the present application, a coordinate axis is established with an intersection point of a left edge boundary line and an upper edge boundary line of the image to be measured as an origin, a rightward extending direction is an x-axis, a downward extending direction is a y-axis, where θ is an angle of the connected region, u is an angle of the connected region 11 ，u 20 ，u 02 Is the center distance.
S105: and calculating the density of the smoke area of the image to be detected through the smoke recognition device.
In one embodiment of the present application, the original image of the smoke region in the image to be measured is converted from the RGB color space to the HSV color space. And calculating the accumulated value of the horizontal and vertical separate projections of the brightness under a Mask image according to the brightness space V space pixel value.
In particular by the Sum formula Sum i ＝pixel+sum i Ifmask =0, calculating an accumulated value Sum of pixel values projected horizontally and vertically in the Mask image, respectively i In the formula, i is a horizontal axis or a vertical axis, pixel is a pixel, mask is a mask binary image, pixel =1 when mask =0, and pixel =0 when mask ≠ 0. For example, the first line of the Mask image has a pixel value of [1 1 10 0 0 0 0 0 0 0]. According to the above accumulation formula, when mask =0, pixel =1, and when mask ≠ 0, pixel =0. It can be found that the number of pixel values of mask =0 is 10, and thus, the Sum after the accumulation can be obtained by accumulating the 10 pixel values i The value of (2) is 10.
In one embodiment of the present application, when i represents the horizontal axis, the accumulation formula may be Sum x ＝pixel+sum x And the pixel values of any line of the smoke area of the image to be detected are accumulated.
It should be noted that the initial reference number is 0, and the specific numerical value of the number of rows or columns is determined according to the size of the image smoke region to be measured.
S106: and the smoke recognition device determines whether the area to be detected is a smoke area or not according to the detected concentration value of the area to be detected.
In one embodiment of the present application, nongdu (x, y) = sum according to the formula nongdu (x, y) = sum x ×sum y And calculating smoke concentration nongdu (x, y) according to the drifting direction of the smoke calculated in the step S104 and the accumulated value of the pixel values calculated in the step S105, wherein x and y are respectively the abscissa and the ordinate of the pixel point, y = x × tan θ, and θ is the angle direction of the binary image after the smoke region of the image to be measured is divided.
In an embodiment of the application, when the fact that the concentration values of the region to be detected in the image of the smoke to be detected become smaller from bottom to top is detected, the region to be detected is determined to be the smoke region.
The embodiment of the application aims at the characteristics that smoke has directivity and the concentration is gradually thin from bottom to top. It is proposed to distinguish smoke from cloud and fog images according to the difference of direction and concentration. The problem of current smog recognition device can't effectively reduce to cloud, fog production wrong report is solved. Thereby improving the detection rate of the smoke. Fig. 6 is a schematic view of an internal structure of the smoke recognition device according to the embodiment of the present application.
The embodiment of the application provides a smog recognition device includes:
at least one processor; and the number of the first and second groups,
a memory communicatively coupled to the at least one processor; wherein, the first and the second end of the pipe are connected with each other,
the memory stores instructions executable by the at least one processor to enable the at least one processor to:
calculating a loss function of a smoke concentration central point in a smoke image sample, a classification loss function and a regression loss function through a target detection algorithm, wherein the smoke concentration central point is the highest point of smoke concentration marked in the smoke image sample in advance.
And training the target detection algorithm through the smoke image sample to obtain a trained smoke image detection model.
Determining the position of a smoke area in the image to be detected according to the trained smoke image detection model;
and carrying out image segmentation on the smoke region in the image to be detected, taking the binary image of the segmented image as a Mask image, and calculating the smoke drifting direction.
And converting the smoke region in the image to be detected from the RGB color space to the HSV color space, and calculating the accumulated values of the pixel values of the segmented image in the brightness V space projected horizontally and vertically in the Mask image.
And determining the smoke concentration according to the smoke drifting direction and the accumulated value.
Finally, it should be noted that: although the present invention has been described in detail with reference to the foregoing embodiments, it will be apparent to those skilled in the art that modifications may be made to the embodiments described above, or equivalents may be substituted for elements thereof. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims (9)
1. A smoke recognition method, comprising the steps of:
a) Calculating a loss function of a smoke concentration central point in a visualized smoke image sample of a power transmission line channel, a classification loss function and a regression loss function through a target detection algorithm, wherein the smoke concentration central point is the highest point of smoke concentration marked in the smoke image sample in advance;
b) Training a target detection algorithm through a smoke image sample to obtain a trained smoke image detection model;
c) Determining the position of a smoke area in the image to be detected according to the trained smoke image detection model;
d) Carrying out image segmentation on a smoke area in an image to be detected, taking a binary image of the segmented image as a Mask image, and calculating the smoke drifting direction;
e) Converting a smoke area in an image to be detected from an RGB color space to an HSV color space, and calculating accumulated values of pixel values of the segmented image in a brightness V space projected horizontally and vertically in the Mask image;
f) Determining the smoke concentration according to the smoke drifting direction and the accumulated value;
in the step d), the binary image of the segmented image is used as a Mask image, and the direction of smoke drifting is calculated, and the method specifically comprises the following steps:
d-1) establishing a coordinate axis by taking the intersection point of the left edge boundary line and the upper edge boundary line of the image to be detected as an origin, taking the rightward extending direction as an x axis and taking the downward extending direction as a y axis;
d-2) based on the spatial Moment algorithm
Determining the angle direction of a communication area, wherein the communication area is a smoke area after being segmented in the image to be detected, theta is the angle of the communication area, u 11 ，u 20 ，u 02 Is the center distance.
2. The smoke recognition method of claim 1, further comprising the step of, after step f): and under the condition that the concentration values of the area to be detected in the smoke image to be detected from bottom to top are detected to be sequentially reduced, determining that the area to be detected is a smoke area.
3. The smoke recognition method of claim 1, wherein: in step a) by the function:
calculate the scoreClass loss function and regression loss function, and loss function for determining smoke concentration center point, wherein the class loss L is shown in the formula cls Center point Loss L using the Focal local Loss function center-ness Using the BCE loss function, the regression loss L reg With the use of a weighted GIoU loss function,
taking a smoke label value set in constructing the data set when the image sample is a positive sample, and taking 0 when the image sample is a negative sample, the smoke label value being a numerical value set by determining a smoke density, N pos To mark the number of positive samples, λ 1 And λ 2 Are all weight values, p x,y In order to score the classification of the object,
output the classification score, t, for the neural network x,y In order to obtain the target regression score,
outputting a regression score for the neural network, centerness x,y The score is given to the center point of the target,
and outputting the central point score for the neural network.
4. The smoke recognition method of claim 1, wherein step e) comprises the steps of:
e-1) by the formula Sum i ＝pixel+sum i If Mask =0, calculating an accumulated value Sum of pixel values projected horizontally and vertically in the Mask image, respectively i Wherein i is a horizontal axis or a vertical axis, pixel is a pixel, mask is a mask binary image, pixel =1 when mask =0, and pixel =0 when mask ≠ 0;
e-2) according to nongdu (x, y) = sum x ×sum y Calculating the smoke concentration nongdu (x, y), wherein x and y are pixel points respectivelyY = x × tan θ, and θ is an angular direction of the divided image.
5. A smoke identification method according to claim 3, wherein said regression loss L reg The regression loss was calculated using a weighted GIoU loss function by the following formula:
in the formula w j Representing a weighted weight; a. The wc Is the weighted union area, dist i Distance of the center point to the ith edge point, dist j The distance from the central point to the jth edge point is shown, the corner-point is the central point of the network output, and the center-point is the central point with the highest actual smoke concentration; x is a radical of a fluorine atom 1 ，y 1 Respectively representing the abscissa and ordinate, x, of the upper left corner point of the rectangular frame marking the smoke region 2 ，y 2 Respectively representing the abscissa and the ordinate of the right lower corner point of the marked rectangular frame; x is a radical of a fluorine atom 1 ′，y 1 ' respectively denote the abscissa and ordinate of the upper left corner point of the rectangular frame of the predicted smoke region,x 2 ′，y 2 ' respectively denote the abscissa and ordinate of the lower right corner point of the rectangular frame of the predicted smoke region,
weighted for the abscissa of the upper left corner of the rectangular box,
weighted for the abscissa of the lower right corner of the rectangular box,
weighted for the ordinate of the upper left corner of the rectangular box,
weighting the ordinate of the lower right corner of the rectangular frame; u is the union area; ioU is the intersection to union ratio, w 1 For labeling the upper left corner (x) of the rectangular frame 1 ，y 1 ) Distance weight from the center point of the label, w 2 To mark the lower right corner (x) of the rectangular frame 2 ，y 2 ) Distance weight from the center point of the annotation, w 1 ' to predict the upper left corner of the rectangular box (x) 1 '，y 1 ') distance weight from predicted center point, w 2 ' is prediction rectangle frame lower right corner (x) 2 '，y 2 ') distance weight from the predicted center point.
6. The smoke recognition method of claim 1, wherein: and taking the central point output by the smoke image detection model as an initial seed point coordinate, and carrying out image segmentation on the smoke area by a flooding filling method.
7. The smoke recognition method of claim 1, wherein: before training the target detection algorithm through the smoke image sample in the step b), the method further comprises the following steps:
constructing a synthetic image training set by G (x, y) Gaussian weighting according to a function AP (x, y) = G (x, y) xSmoke (x, y) + (1-G (x, y)) × BP (x, y); wherein AP is a synthesized Smoke pixel, BP is a negative sample image pixel value before synthesis, and Smok is a Smoke pixel value.
8. The smoke recognition method according to claim 1, wherein: before training the target detection algorithm through the smoke image sample in the step b), the method further comprises the following steps: the collected smoke images and the power transmission channel hidden danger images are subjected to weighted fusion according to a function pixel _ after = alpha × pixel _ smoke + (1-alpha) × pixel _ before to obtain smoke image training sets with different concentrations, wherein the power transmission channel hidden danger images are images with smoke and/or images without smoke, the pixel _ before is an original pixel value of the power transmission channel hidden danger images, the pixel _ smoke is a pixel value of the smoke images, the pixel _ after is the pixel value of the smoke images after weighted fusion, and the alpha is a set smoke label value.
9. A smoke recognition device, comprising:
at least one processor; and (c) a second step of,
a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,
the memory stores instructions executable by the at least one processor to cause the at least one processor to:
calculating a loss function of a smoke concentration central point in a smoke image sample, a classification loss function and a regression loss function through a target detection algorithm, wherein the smoke concentration central point is the highest point of smoke concentration marked in the smoke image sample in advance;
training the target detection algorithm through the smoke image sample to obtain a trained smoke image detection model;
determining the position of a smoke area in the image to be detected according to the trained smoke image detection model;
using the left edge boundary line and the upper edge boundary line of the image to be measuredThe intersection point is used as an original point to establish a coordinate axis, the rightward extending direction is used as an x axis, and the downward extending direction is used as a y axis; based on the spatial Moment algorithm, according to
Determining the angle direction of a communication area, wherein the communication area is a smoke area after being segmented in the image to be detected, theta is the angle of the communication area, u 11 ，u 20 ，u 02 Is the center distance;
converting the smoke region in the image to be detected from the RGB color space to the HSV color space, and calculating the accumulated values of the pixel values of the segmented image in the brightness V space projected horizontally and vertically in the Mask image;
and determining the smoke concentration according to the smoke drifting direction and the accumulated value.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011415037.2A CN112507865B (en) | 2020-12-04 | 2020-12-04 | Smoke identification method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011415037.2A CN112507865B (en) | 2020-12-04 | 2020-12-04 | Smoke identification method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112507865A CN112507865A (en) | 2021-03-16 |
| CN112507865B true CN112507865B (en) | 2022-11-29 |
Family
ID=74970578
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011415037.2A Active CN112507865B (en) | 2020-12-04 | 2020-12-04 | Smoke identification method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112507865B (en) |
Families Citing this family (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113128422B (en) * | 2021-04-23 | 2024-03-29 | 重庆市海普软件产业有限公司 | Image smoke and fire detection method and system for deep neural network |
| CN116091935B (en) * | 2023-04-07 | 2023-08-01 | 四川三思德科技有限公司 | Forest fire and smoke alarm agriculture operation interference resistance processing method, device and medium |
| CN116798003B (en) * | 2023-08-28 | 2023-11-14 | 苏州科尔珀恩机械科技有限公司 | Visual auxiliary monitoring method for incinerator |
| CN116824514B (en) * | 2023-08-30 | 2023-12-08 | 四川弘和数智集团有限公司 | Target identification method and device, electronic equipment and storage medium |
| CN117036361B (en) * | 2023-10-10 | 2024-02-20 | 云南大学 | Power grid transmission line smoke detection method, system, electronic equipment and medium |
| CN117233119B (en) * | 2023-11-10 | 2024-01-12 | 北京环拓科技有限公司 | Method for identifying and quantifying VOC (volatile organic compound) gas cloud image by combining sensor calibration module |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109977790A (en) * | 2019-03-04 | 2019-07-05 | 浙江工业大学 | A kind of video smoke detection and recognition methods based on transfer learning |
| CN111126230A (en) * | 2019-12-17 | 2020-05-08 | 上海工程技术大学 | Smoke concentration quantitative evaluation method and electronic equipment applying same |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10395498B2 (en) * | 2015-02-19 | 2019-08-27 | Smoke Detective, Llc | Fire detection apparatus utilizing a camera |
-
2020
- 2020-12-04 CN CN202011415037.2A patent/CN112507865B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109977790A (en) * | 2019-03-04 | 2019-07-05 | 浙江工业大学 | A kind of video smoke detection and recognition methods based on transfer learning |
| CN111126230A (en) * | 2019-12-17 | 2020-05-08 | 上海工程技术大学 | Smoke concentration quantitative evaluation method and electronic equipment applying same |
Non-Patent Citations (3)
| Title |
|---|
| An early smoke detection system based on motion estimation;Tung Xuan Truong et al.;《International Forum on Strategic Technology 2010》;20101213;第1-4页 * |
| Research on Video Smoke Recognition Based on Dynamic Image Segmentation Detection Technology;Ethan liu;《2019 12th International Conference on Intelligent Computation Technology and Automation (ICICTA)》;20200302;第240-243页 * |
| 基于图像处理的舰船火灾烟雾检测技术研究;周泊龙等;《舰船科学技术》;20160531;第111-115页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112507865A (en) | 2021-03-16 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112507865B (en) | Smoke identification method and device | |
| CN109977812B (en) | Vehicle-mounted video target detection method based on deep learning | |
| CN113139521B (en) | Pedestrian boundary crossing monitoring method for electric power monitoring | |
| CN100520362C (en) | Method for detecting forest fire fog based on colorful CCD image analysis | |
| CN111222478A (en) | Construction site safety protection detection method and system | |
| CN103442209A (en) | Video monitoring method of electric transmission line | |
| CN110675588B (en) | Forest fire detection device and method | |
| CN112149543B (en) | Building dust recognition system and method based on computer vision | |
| US11145089B2 (en) | Method for measuring antenna downtilt based on multi-scale detection algorithm | |
| CN111639530B (en) | Method and system for detecting and identifying power transmission tower and insulator of power transmission line | |
| CN106408025A (en) | Classification and recognition method of aerial image insulators based on image processing | |
| CN113255590A (en) | Defect detection model training method, defect detection method, device and system | |
| CN114841920A (en) | Flame identification method and device based on image processing and electronic equipment | |
| CN113298077A (en) | Transformer substation foreign matter identification and positioning method and device based on deep learning | |
| CN113657305B (en) | Video-based intelligent detection method for black smoke vehicle and ringeman blackness level | |
| CN112115878A (en) | Forest fire smoke root node detection method based on smoke area density | |
| CN117975360A (en) | Deep learning-based stacked object distinguishing method and system | |
| CN113657250A (en) | Flame detection method and system based on monitoring video | |
| CN117475353A (en) | Video-based abnormal smoke identification method and system | |
| CN107066929B (en) | Hierarchical recognition method for parking events of expressway tunnel integrating multiple characteristics | |
| CN112016641A (en) | Method and device for alarming line short circuit fault caused by foreign matter | |
| CN107992799A (en) | Towards the preprocess method of Smoke Detection application | |
| CN112686105B (en) | Fog concentration grade identification method based on video image multi-feature fusion | |
| CN113610940B (en) | Ocean vector file and image channel threshold based coastal area color homogenizing method | |
| CN114757962A (en) | Overhead line forest fire identification method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |