CN112418005A - Smoke multi-classification identification method based on backward radiation attention pyramid network - Google Patents
Smoke multi-classification identification method based on backward radiation attention pyramid network Download PDFInfo
- Publication number
- CN112418005A CN112418005A CN202011226816.8A CN202011226816A CN112418005A CN 112418005 A CN112418005 A CN 112418005A CN 202011226816 A CN202011226816 A CN 202011226816A CN 112418005 A CN112418005 A CN 112418005A
- Authority
- CN
- China
- Prior art keywords
- smoke
- pyramid
- image
- network
- classification
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 239000000779 smoke Substances 0.000 title claims abstract description 76
- 230000005855 radiation Effects 0.000 title claims abstract description 37
- 238000000034 method Methods 0.000 title claims abstract description 24
- 230000007246 mechanism Effects 0.000 claims abstract description 6
- 238000001914 filtration Methods 0.000 claims abstract description 4
- 238000012549 training Methods 0.000 claims description 18
- 238000005520 cutting process Methods 0.000 claims description 16
- 238000012544 monitoring process Methods 0.000 claims description 13
- 238000011176 pooling Methods 0.000 claims description 11
- 238000010586 diagram Methods 0.000 claims description 7
- 230000004913 activation Effects 0.000 claims description 6
- 238000011156 evaluation Methods 0.000 claims description 6
- 239000000126 substance Substances 0.000 claims description 5
- 238000000605 extraction Methods 0.000 claims description 4
- 230000006870 function Effects 0.000 claims description 4
- 238000013528 artificial neural network Methods 0.000 claims description 3
- 230000002708 enhancing effect Effects 0.000 claims description 3
- 238000010606 normalization Methods 0.000 claims description 3
- 238000007781 pre-processing Methods 0.000 claims description 3
- 239000013598 vector Substances 0.000 claims description 3
- 238000012545 processing Methods 0.000 claims description 2
- 238000010801 machine learning Methods 0.000 abstract description 2
- 238000010248 power generation Methods 0.000 description 15
- 230000007935 neutral effect Effects 0.000 description 6
- 238000013135 deep learning Methods 0.000 description 5
- 230000002159 abnormal effect Effects 0.000 description 3
- 238000001514 detection method Methods 0.000 description 3
- 239000007789 gas Substances 0.000 description 3
- 238000003909 pattern recognition Methods 0.000 description 3
- 238000011160 research Methods 0.000 description 3
- 238000010276 construction Methods 0.000 description 2
- 238000013461 design Methods 0.000 description 2
- 238000004880 explosion Methods 0.000 description 2
- 230000001537 neural effect Effects 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 230000005180 public health Effects 0.000 description 2
- 238000012360 testing method Methods 0.000 description 2
- 230000000007 visual effect Effects 0.000 description 2
- 239000002912 waste gas Substances 0.000 description 2
- 238000003915 air pollution Methods 0.000 description 1
- 239000006229 carbon black Substances 0.000 description 1
- 238000012512 characterization method Methods 0.000 description 1
- 238000002485 combustion reaction Methods 0.000 description 1
- 230000000052 comparative effect Effects 0.000 description 1
- 238000013527 convolutional neural network Methods 0.000 description 1
- 230000007797 corrosion Effects 0.000 description 1
- 238000005260 corrosion Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000005611 electricity Effects 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 239000000284 extract Substances 0.000 description 1
- 239000003517 fume Substances 0.000 description 1
- 230000036541 health Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 230000002265 prevention Effects 0.000 description 1
- 231100000331 toxic Toxicity 0.000 description 1
- 230000002588 toxic effect Effects 0.000 description 1
- 239000002341 toxic gas Substances 0.000 description 1
- 239000010891 toxic waste Substances 0.000 description 1
- 238000013022 venting Methods 0.000 description 1
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Chemical compound O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/40—Scenes; Scene-specific elements in video content
- G06V20/41—Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/40—Scenes; Scene-specific elements in video content
- G06V20/46—Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/40—Scenes; Scene-specific elements in video content
- G06V20/49—Segmenting video sequences, i.e. computational techniques such as parsing or cutting the sequence, low-level clustering or determining units such as shots or scenes
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Software Systems (AREA)
- Computing Systems (AREA)
- Computational Linguistics (AREA)
- General Health & Medical Sciences (AREA)
- Evolutionary Computation (AREA)
- Data Mining & Analysis (AREA)
- Molecular Biology (AREA)
- Biophysics (AREA)
- General Engineering & Computer Science (AREA)
- Biomedical Technology (AREA)
- Mathematical Physics (AREA)
- Artificial Intelligence (AREA)
- Life Sciences & Earth Sciences (AREA)
- Health & Medical Sciences (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a smoke multi-classification identification method based on a back radiation attention pyramid network, which is used for judging the working state of an emptying torch. The network is first composed of three pyramid blocks in series, which consist of 3, 4 and 5 basic convolution modules. Then, an attention mechanism is introduced into each pyramid module for feature filtering. And finally, all feedforward outputs of all pyramid modules are connected through backward radiation, and the system comprehensively fuses low, medium and high-level features. A smoke multi-classification identification method based on a backward radiation attention pyramid network belongs to the field of atmospheric environment protection and the field of machine learning.
Description
Technical Field
The invention relates to a method for identifying the working state of a thermal power generation system, in particular to a smoke multi-classification identification method based on a backward radiation attention pyramid network, and belongs to the field of atmospheric environment protection and the field of machine learning.
Background
The discovery and widespread use of electricity is the result of the second and third technological revolution, bringing a lot of convenience to human life. Common power generation modes mainly include thermal power generation, solar power generation, nuclear power generation, wind power generation and the like. However, the cost of solar power generation is too high, and large-scale use cannot be realized; nuclear power generation easily causes unrecoverable pollution to the environment; wind power generation is limited by wind power and terrain. Therefore, thermal power is still the mainstream method in the power generation mode.
In order to better maintain the safety of life and production and protect the ecological environment, the waste gas generated by the thermal power system is completely combusted and then is discharged into the atmosphere through an emptying torch. Under normal operating conditions, the thermal power generation system should produce white smoke (i.e., water vapor) when the flare is vented. However, when the venting torch discharges black smoke (i.e. carbon black) or colorless smoke (i.e. toxic gas), which exceeds the unrecoverable degree specified in the thermal power generation industry, the operation of the thermal power system is abnormal, and serious safety risks and environmental problems are caused. In particular, black smoke is caused by incomplete combustion of exhaust gas, and is liable to cause fatal damage to public health and ecological environment. Colorless fumes most likely mean that toxic exhaust gases are emitted untreated directly into the air, which often results in serious air pollution. Therefore, a well-designed smoke recognition model is needed to monitor the working state of the thermal power generation system, so as to protect the human health and the ecological environment. In recent years, with the rapid development of deep learning, it has been successfully applied to many important tasks. It has been widely accepted by researchers that deep convolutional network extracted features have better characterization capabilities than traditional manually extracted features.
CN201910319875.0 discloses a transformer substation smoke and fire intelligent recognition monitoring method based on deep learning, wherein the method optimizes a frame of YOLO v3 as a basis, and completes construction and training of an image recognition model by adopting an image data set; in addition, the method also utilizes video data to train a pseudo three-dimensional convolution residual error network to complete the construction and training of a video identification model; the extracted video stream and the image are sent to an image recognition model after being preprocessed, and when smoke is detected, the video recognition model is automatically called to carry out secondary recheck to check the detection result.
CN201611116917.3 discloses a flame/smoke recognition method based on RGB reconstruction after forest image cutting, which destroys a background area by utilizing the difference of RGB three-component values of the flame area, the smoke area and the background area, and continuously reduces the area to be detected; after being processed by a modified gray formula for enhancing flame characteristics, the smoke or the suspected flame area is judged according to the proportion of non-negative elements through filtering, cutting, corrosion and expansion; the invention is applied to the occasions for identifying flame/smoke in forest fire prevention.
While deep learning has achieved significant success in recognition, research in smoke multi-class recognition has been quite limited. Therefore, a new model is urgently needed to be established to solve the problem of multi-classification identification of the smoke concentration. Based on the consideration, the invention firstly researches the multi-classification smoke identification problem, and carefully develops a new smoke multi-classification network based on the backward radiation attention pyramid so as to fully utilize the visual characteristics of the smoke at different levels, thereby improving the classification accuracy of the classifier and the algorithm accuracy.
Disclosure of Invention
The invention researches the working state of a thermal power generation system and designs a smoke multi-classification identification method (IRAP-Net) based on a backward radiation attention pyramid network, aiming at the serious problems that under the abnormal operation condition of a thermal power generation system, the waste gas is not completely combusted or is not treated, and a large amount of black smoke or colorless toxic waste gas is discharged into the atmosphere to cause severe explosion and atmospheric pollution. The method adopts the real-time monitoring image of the vented torch smoke acquired by the monitoring camera of the thermal power plant as the data to be detected, optimizes the image through a deep learning framework based on a pyramid network, and utilizes an image data set to construct and train the image to obtain the whole smoke classification condition of the real-time monitoring image of the smoke so as to complete the real-time monitoring of the smoke of the thermal power plant.
The invention adopts the technical scheme that the smoke multi-classification identification method based on the backward radiation attention pyramid network is mainly established through the following steps:
step 1, obtaining an image to be detected from the smoke real-time monitoring video.
For a video clip obtained by an industrial camera, image data is obtained by extracting a video frame and cutting an image to be used as an input of a model.
Step 1.1, extracting video frames, extracting video current frames at intervals of 1.5s, and directly discarding videos of other parts;
step 1.2, cutting the smoke image segment, cutting each frame image extracted from the video into a smoke image block of 48 multiplied by 48 in a mode of stipulating a cutting position, and abandoning other image areas;
and 2, building a pyramid network based on the backward radiation attention.
And constructing a pyramid learning network based on the backward radiation attention, wherein the network is composed of three pyramid modules connected in series, and the three pyramid layers are respectively composed of 3, 4 and 5 basic convolution modules. And introducing an attention mechanism module in each pyramid block for characteristic filtering. And finally, connecting all feedforward outputs of all pyramid modules through backward radiation, and comprehensively fusing low, medium and high-level features. The input of the method is the image block obtained in the step 1, and the output of the method is smoke monitoring data.
Step 2.1, a basic convolution module consisting of two convolution layers, a normalization layer and an activation layer is adopted to carry out network preprocessing, a smoke image with the input value of (48 multiplied by 3) is subjected to pre-training consisting of the basic convolution module and a maximum pooling layer, and the output value of the smoke image with the input value of (24 multiplied by 16), so that the characteristic is pre-adjusted;
step 2.2, extracting low-level features by using 3 groups of complex convolution modules consisting of 2 unit convolution modules as a first-level pyramid model, inputting the low-level features as a result of (24 multiplied by 16) dimensional network pre-training, and outputting the low-level features with (24 multiplied by 96) dimensions after stacking the features;
and 2.3, enhancing relevant characteristics by adopting an attention mechanism. Firstly, considering the signal mapped to each feature by the output pyramid block, initializing the weight by using global average pooling, wherein the formula is as follows:
wherein the content of the first and second substances,
representing the initialization result of the global average pooling weight;
denotes fcThe characteristic map of the nth smoke image in (1); f. ofcA feature map representing a group c of smoke images;
and
feature map respectively representing smoke image
Height and width of; i and j represent
Height and width coordinates. Then, a two-layer neural network consisting of two complete connection layers and two activation layers is introduced, and the initialized weight is learned and updated, so that the optimal weight is obtained:
wherein the content of the first and second substances,
representing the optimal weight of the smog image multi-classification network; w1，W2Is the weight of two fully connected layers; wherein sigmaReLUAnd σSigmoidRefer to the ReLU function and Sigmoid function used in the active layer, respectively; omegacShows group c smoke diagramsAnd initializing the weight value corresponding to the characteristic diagram of the image. And finally, adding the original feature map into the associated enhanced feature map to generate a new feature map. The input is the low-level feature of (24 × 24 × 96) dimension, and the output is the weighted low-level feature of (24 × 24 × 96);
step 2.4, extracting the middle-layer features by using 4 groups of complex convolution modules consisting of 3 unit convolution modules as a second-layer pyramid model, wherein the input of the complex convolution modules is (24 multiplied by 96) weighted low-layer features, and the output of the complex convolution modules is (24 multiplied by 160) dimensional middle-layer features after the features are stacked;
step 2.5, synchronization step 2.3, its input is the middle layer characteristic (24 × 24 × 160), its output is the weighted middle layer characteristic (24 × 24 × 160);
step 2.6, extracting high-level features by taking 5 groups of complex convolution modules consisting of 4 unit convolution modules as a third-level pyramid model, wherein the input of the complex convolution modules is (24 multiplied by 160) weighted high-level features, and the output of the complex convolution modules is (24 multiplied by 240) dimensional high-level features after the features are stacked;
step 2.7, synchronization step 2.3, its input is the high-level feature of (24 × 24 × 240), its output is the weighted high-level feature of (24 × 24 × 240);
and 2.8, combining the backbone of the pyramid network with the feature map feedforward output by the three pyramid blocks to form a backward radiation connection. Firstly, a unit convolution module is adopted to superpose four groups of output feature maps extracted from a main stem and three pyramid blocks together to obtain a group of new feature maps. The newly created feature map is then stretched to size 128 x 1 using a global average pooling operation. Finally, the vectors are processed by utilizing the full-connection operation, and a multi-classification smoke recognition result is obtained;
and 3, training a pyramid network based on the backward radiation attention.
And 3.1, performing multi-classification training on the smoke based on the back radiation attention pyramid network formed in the step 2 and the step 3 by using the smoke image blocks obtained in the step 1 as a training set. Wherein, the momentum coefficient is set to be 0.9, the initial learning rate is 0.001, and the learning rate attenuation coefficient is 0.00001. A mini-batch cycle of size 16 was used for 300 training periods.
And 4, carrying out multi-classification monitoring on the smoke image.
Video frames of videos obtained by the industrial camera are extracted, images are cut, features are extracted, and smoke classification evaluation is carried out.
Step 4.1, video frame extraction and image cutting are carried out on the video to be tested according to the step 1;
and 4.2, performing multi-classification evaluation on the smoke by using the pyramid network trained in the step 4 and based on the backward radiation attention.
Compared with the prior art, the invention has the following advantages:
(1) aiming at the problems that the abnormal working condition of a thermal power system is easy to cause fatal damage to public health and ecological environment, and severe explosion and atmospheric pollution are caused, the invention provides a novel well-designed smoke multi-classification identification deep network for the first time.
(2) By combining the pyramid body with an attention mechanism and the backward radiation connection, the backward radiation attention pyramid network can effectively fuse various low-grade and high-grade characteristics to realize visual smoke identification.
(3) The present invention performs thorough testing and algorithmic comparisons on a large database of smoke images. The result shows that IRAP-Net obtains superior performance, which is obviously superior to the advanced depth models of other countries.
Drawings
FIG. 1 is a schematic diagram of a backward radiation attention pyramid network design;
fig. 2 is a table of network parameters of the present invention.
Detailed Description
The present invention is described in detail below, and the embodiments and specific operations of the present invention are given in the present examples on the premise of the technical solution of the present invention, but the scope of the present invention is not limited to the following processes.
The implementation mode is shown in figures 1 and 2 and comprises the following steps:
step S10, extracting and cutting the video to obtain image data;
step S20, building a pyramid network based on the back radiation attention;
step S30, training a pyramid network based on the backward radiation attention;
step S40, carrying out multi-classification monitoring on the smoke image;
the step S10 of obtaining image data by video extraction cropping according to the embodiment further includes the steps of:
step S100, extracting video frames, extracting video current frames at constant intervals of 1.5S, and discarding other video segments;
step S110, cutting the smoke image, cutting each frame of smoke image into smoke image blocks of 48 multiplied by 48, and discarding other image areas;
the attention pyramid network building step S20 of the embodiment further includes the steps of:
step S200, preprocessing a network by adopting a basic convolution module and a maximum pooling layer, wherein the basic convolution module consists of two groups of convolution layers, a normalization layer and an activation layer, and the step realizes the pre-adjustment of characteristics;
step S210, extracting low-level features by using 3 groups of complex convolution modules comprising 2 unit convolution modules as a first-level pyramid model;
and step S220, initializing the weight by utilizing global average pooling, introducing a two-layer neural network consisting of two groups of complete connection layers and activation layers to learn and update the initialized weight, adding the original feature graph to the associated enhanced feature graph by using the weight, and generating a new feature graph.
Step S230, taking 4 groups of complex convolution modules consisting of 3 unit convolution modules as a second-layer pyramid model to extract the middle-layer features;
step S240, extracting the weighted middle layer characteristics in the synchronization step 2.3;
step S250, extracting high-level features by taking 5 groups of complex convolution modules consisting of 4 unit convolution modules as a third-level pyramid model;
step S260, the synchronization step 2.3 extracts the weighted high-level features;
and step S270, combining the backbone of the pyramid network and the characteristic diagram feed-forward output by the pyramid block by adopting backward radiation connection. Firstly, a unit convolution module is adopted to superpose four groups of output feature maps extracted from a main stem and three pyramid blocks together to obtain a group of new feature maps. The newly created feature map is then stretched to size 128 x 1 using a global average pooling operation. Finally, processing the vector by utilizing full-connection operation to obtain a multi-classification smoke identification result;
the step S30 of training the pyramid network based on back radiation attention further includes the following steps:
and step S300, performing multi-classification smoke training on the pyramid network based on the back radiation attention, which is formed in the step 2, by using the smoke image blocks obtained in the step 1 as a training set. Wherein, the momentum coefficient is set to be 0.9, the initial learning rate is 0.001, and the learning rate attenuation coefficient is 0.00001. A small batch size of size 16 was used for 300 cycles of training.
The multi-classification monitoring step S40 of the embodiment further includes the steps of:
step S400, video frame extraction and image cutting are carried out on the video to be tested according to the step 1;
and S410, performing multi-classification evaluation on the smoke by using the pyramid network based on the backward radiation attention trained in the step 3.
Step S420, three common evaluation criteria, i.e. accuracy, detection rate and false recognition rate, are used as the indexes for detecting the network performance. The first criterion is Global Accuracy (GAR):
wherein N is+，N0，N-Respectively representing the number of positive samples, the number of neutral samples and the number of negative samples; t is+，T0，T-To be correctThe number of positive samples identified, neutral samples correctly identified, and negative samples correctly identified. The second criterion is Global Detection Rate (GDR):
wherein DR+，DR0，DR-The ratios of the number of correct results in the positive, neural and negative samples to the total positive, neutral and negative samples, respectively, are defined as follows:
the third criterion is the false positive rate (GFAR):
wherein the content of the first and second substances,
in the formula, F+The number of positive samples that are misclassified as neutral samples or negative samples; f0A number of neutral samples that are misclassified as positive or negative samples; f-It is the number of negative samples that are incorrectly classified as positive samples and neutral samples.
The invention compares the performance with the existing Alex-Net [1], ZF-Net [2], VGG-Net [3], Google-Net [4], Res-Net [5], Xception [6], Dense-Net [7], DNCNN [8] and DCNN [9] networks, and gives the experimental results of the application of the invention.
Table 1 comparative test results of the invention
Reference to the literature
[1]Krizhevsky，I.Sutskever，and G.E.Hinton，“ImageNet classification with deep convolutional neural networks，”Proc.Adv.Neural Inf.Process.Syst.(NIPS)，vol.25.pp.1097-1105，Dec.2012.
[2]M.D.Zeiler and R.Fergus，“Visualizing and understanding convolutional networks，”Proc.Eur.Conf.Comp.Vis.(ECCV)，pp.818-833，Sep.2014.
[3]K.Simonyan and A.Zisserman，“Very deep convolutional networks for large-scale image recognition，”arXiv preprint arXiv：1409.1556，Sep.2014.
[4]C.Szegedy,W.Liu,Y.Jia,P.Sermanet,S.E.Reed,D.Anguelov,D.Erhan,V.Vanhoucke,and A.Rabinovich,“Going deeper with convolutions,”IEEE Conf.Computer Vision&Pattern Recognition(CVPR),pp.1-9,2015.
[5]K.He,X.Zhang,S.Ren,and J.Sun,“Deep residual learning for image recognition,”IEEE Conf.Computer Vision&Pattern Recognition(CVPR),pp.770-778,Jun.2016.
[6]F.Chollet,“Xception:Deep Learning with depthwise separable convolutions,”IEEE Conf.Computer Vision&Pattern Recognition(CVPR),pp.1800-1807,Nov.2017.
[7]G.Huang,Z.Liu,L.van der Maaten,and K.Q.Weinberger,“Densely connected convolutional networks,”IEEE Conf.Computer Vision&Pattern Recognition(CVPR),pp.4700-4708,Jul.2017.
[8]Z.Yin,B.Wan,F.Yuan,X.Xia,and J.Shi,“A deep normalization and convolutional neural network for image smoke detection,”IEEEAccess,vol.5,pp.18429-18438,Aug.2017.
[9]K.Gu,Z.Xia,J.Qiao,and W.Lin,“Deep dual-channel neural network for image-based smoke detection,”IEEE Trans.Multimedia,vol.22,no.2,pp.311-323,Feb.2020.
Claims (5)
1. A multi-classification smoke identification method based on a backward radiation attention pyramid network is characterized by comprising the following steps: the method is established by the following steps:
step 1, obtaining an image to be detected from a smoke real-time monitoring video;
for a video clip obtained by an industrial camera, image data is obtained by extracting a video frame and cutting an image and is used as the input of a model;
step 2, building a pyramid network based on the backward radiation attention;
building a pyramid learning network based on the backward radiation attention, wherein the network is composed of three pyramid modules connected in series, and the three pyramid layers are respectively composed of 3, 4 and 5 basic convolution modules; an attention mechanism module is introduced into each pyramid block for feature filtering; finally, all feedforward outputs of all pyramid modules are connected through backward radiation, and low, medium and high-level features are comprehensively fused; inputting the image block obtained in the step (1) and outputting smoke monitoring data;
step 3, training a pyramid network based on the backward radiation attention;
step 4, carrying out multi-classification monitoring on the smoke image;
video frames of videos obtained by the industrial camera are extracted, images are cut, features are extracted, and smoke classification evaluation is carried out.
2. The smoke multi-classification recognition method based on the backward radiation attention pyramid network as claimed in claim 1, wherein: in the step 1, a video frame is extracted, a video current frame is extracted at an interval of 1.5s, and videos of other parts are directly discarded;
step 1.2, cutting the smoke image segment, cutting each frame image extracted from the video into a smoke image block of 48 multiplied by 48 in a mode of stipulating a cutting position, and abandoning other image areas.
3. The smoke multi-classification recognition method based on the backward radiation attention pyramid network as claimed in claim 1, wherein: in step 2, step 2.1, a basic convolution module consisting of two sets of convolution layers, a normalization layer and an activation layer is adopted to carry out network preprocessing, a smoke image with the input of (48 multiplied by 3) is input, and after pre-training consisting of the basic convolution module and a maximum pooling layer, the output of the smoke image is (24 multiplied by 16), so that the characteristic is pre-adjusted;
step 2.2, extracting low-level features by using 3 groups of complex convolution modules consisting of 2 unit convolution modules as a first-level pyramid model, inputting the results of (24 multiplied by 16) dimensional network pre-training, and outputting the low-level features with (24 multiplied by 96) dimensions after stacking the features;
step 2.3, enhancing relevant characteristics by adopting an attention mechanism; firstly, considering the signal mapped to each feature by the output pyramid block, initializing the weight by using global average pooling, wherein the formula is as follows:
wherein the content of the first and second substances,
representing the initialization result of the global average pooling weight;
denotes fcThe characteristic map of the nth smoke image in (1); f. ofcA feature map representing a group c of smoke images;
and
feature map respectively representing smoke image
Height and width of; i and j represent
Coordinates of height and width; then, a two-layer neural network consisting of two complete connection layers and two activation layers is introduced, and the initialized weight is learned and updated, so that the optimal weight is obtained:
wherein the content of the first and second substances,
representing the optimal weight of the smog image multi-classification network; w1，W2Is the weight of two fully connected layers; wherein sigmaReLUAnd σSigmoidRefer to the ReLU function and Sigmoid function used in the active layer, respectively; omegacRepresenting the initialization weight corresponding to the characteristic diagram of the c group of smoke images; finally, adding the original characteristic diagram into the associated enhanced characteristic diagram to generate a new characteristic diagram; the input is the low-level feature of (24 × 24 × 96) dimension, and the output is the weighted low-level feature of (24 × 24 × 96);
step 2.4, extracting the middle-layer features by using 4 groups of complex convolution modules consisting of 3 unit convolution modules as a second-layer pyramid model, wherein the input of the complex convolution modules is (24 multiplied by 96) weighted low-layer features, and the output of the complex convolution modules is (24 multiplied by 160) dimensional middle-layer features after the features are stacked;
step 2.5, synchronization step 2.3, its input is the middle layer characteristic (24 × 24 × 160), its output is the weighted middle layer characteristic (24 × 24 × 160);
step 2.6, extracting high-level features by taking 5 groups of complex convolution modules consisting of 4 unit convolution modules as a third-level pyramid model, wherein the input of the complex convolution modules is (24 multiplied by 160) weighted high-level features, and the output of the complex convolution modules is (24 multiplied by 240) dimensional high-level features after the features are stacked;
step 2.7, synchronization step 2.3, its input is the high-level feature of (24 × 24 × 240), its output is the weighted high-level feature of (24 × 24 × 240);
step 2.8, combining the backbone of the pyramid network and the feature map feedforward output by the three pyramid blocks to form a backward radiation connection; firstly, a unit convolution module is adopted to superpose four groups of output feature maps extracted from a main stem and three pyramid blocks together to obtain a group of new feature maps; then, a global average pooling operation is used on the newly created feature map, stretching it to a size of 128 x 1; and finally, processing the vector by utilizing full-connection operation to obtain a multi-classification smoke identification result.
4. The smoke multi-classification recognition method based on the backward radiation attention pyramid network as claimed in claim 1, wherein: in step 3, step 3.1, the smoke image blocks obtained in step 1 are used as a training set, and smoke multi-classification training is carried out on the basis of the back radiation attention pyramid network formed in step 2 and step 3.
5. The smoke multi-classification recognition method based on the backward radiation attention pyramid network as claimed in claim 1, wherein: step 4, step 4.1, video frame extraction and image cutting are carried out on the video to be tested according to the step 1;
and 4.2, performing multi-classification evaluation on the smoke by using the pyramid network trained in the step 4 and based on the backward radiation attention.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011226816.8A CN112418005B (en) | 2020-11-06 | 2020-11-06 | Smoke multi-classification identification method based on reverse radiation attention pyramid network |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011226816.8A CN112418005B (en) | 2020-11-06 | 2020-11-06 | Smoke multi-classification identification method based on reverse radiation attention pyramid network |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112418005A true CN112418005A (en) | 2021-02-26 |
| CN112418005B CN112418005B (en) | 2024-05-28 |
Family
ID=74827814
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011226816.8A Active CN112418005B (en) | 2020-11-06 | 2020-11-06 | Smoke multi-classification identification method based on reverse radiation attention pyramid network |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112418005B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113096103A (en) * | 2021-04-15 | 2021-07-09 | 北京工业大学 | Intelligent smoke image sensing method for emptying torch |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109376695A (en) * | 2018-11-26 | 2019-02-22 | 北京工业大学 | A kind of smog detection method based on depth hybrid neural networks |
| CN109902693A (en) * | 2019-02-16 | 2019-06-18 | 太原理工大学 | One kind being based on more attention spatial pyramid characteristic image recognition methods |
| CN110059723A (en) * | 2019-03-19 | 2019-07-26 | 北京工业大学 | A kind of robust smog detection method based on integrated depth convolutional neural networks |
| US20200057935A1 (en) * | 2017-03-23 | 2020-02-20 | Peking University Shenzhen Graduate School | Video action detection method based on convolutional neural network |
| CN111539314A (en) * | 2020-04-21 | 2020-08-14 | 上海海事大学 | Cloud and fog shielding-oriented sea surface target significance detection method |
| CN111582483A (en) * | 2020-05-14 | 2020-08-25 | 哈尔滨工程大学 | Unsupervised learning optical flow estimation method based on space and channel combined attention mechanism |
-
2020
- 2020-11-06 CN CN202011226816.8A patent/CN112418005B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20200057935A1 (en) * | 2017-03-23 | 2020-02-20 | Peking University Shenzhen Graduate School | Video action detection method based on convolutional neural network |
| CN109376695A (en) * | 2018-11-26 | 2019-02-22 | 北京工业大学 | A kind of smog detection method based on depth hybrid neural networks |
| CN109902693A (en) * | 2019-02-16 | 2019-06-18 | 太原理工大学 | One kind being based on more attention spatial pyramid characteristic image recognition methods |
| CN110059723A (en) * | 2019-03-19 | 2019-07-26 | 北京工业大学 | A kind of robust smog detection method based on integrated depth convolutional neural networks |
| CN111539314A (en) * | 2020-04-21 | 2020-08-14 | 上海海事大学 | Cloud and fog shielding-oriented sea surface target significance detection method |
| CN111582483A (en) * | 2020-05-14 | 2020-08-25 | 哈尔滨工程大学 | Unsupervised learning optical flow estimation method based on space and channel combined attention mechanism |
Non-Patent Citations (2)
| Title |
|---|
| 郭启帆;刘磊;张珹;徐文娟;靖稳峰;: "基于特征金字塔的多尺度特征融合网络", 工程数学学报, no. 05, 15 October 2020 (2020-10-15) * |
| 陈琴;朱磊;后云龙;邓慧萍;吴谨;: "基于深度中心邻域金字塔结构的显著目标检测", 模式识别与人工智能, no. 06, 15 June 2020 (2020-06-15) * |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113096103A (en) * | 2021-04-15 | 2021-07-09 | 北京工业大学 | Intelligent smoke image sensing method for emptying torch |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112418005B (en) | 2024-05-28 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109543760B (en) | Confrontation sample detection method based on image filter algorithm | |
| CN109376695B (en) | Smoke detection method based on deep hybrid neural network | |
| CN110334660A (en) | A kind of forest fire monitoring method based on machine vision under the conditions of greasy weather | |
| CN116343330A (en) | Abnormal behavior identification method for infrared-visible light image fusion | |
| CN113642606B (en) | Marine ship detection method based on attention mechanism | |
| CN116416576A (en) | Smoke/flame double-light visual detection method based on V3-YOLOX | |
| CN111046728A (en) | Straw combustion detection method based on characteristic pyramid network | |
| CN112418005B (en) | Smoke multi-classification identification method based on reverse radiation attention pyramid network | |
| CN114140750A (en) | Filling station safety helmet wearing real-time detection method based on YOLOv4-Tiny | |
| CN115862073A (en) | Transformer substation harmful bird species target detection and identification method based on machine vision | |
| CN113688948A (en) | Method for identifying attached marine organism species based on YOLO v5 | |
| CN114565891A (en) | Smoke and fire monitoring method and system based on graph generation technology | |
| CN112288026A (en) | Infrared weak and small target detection method based on class activation diagram | |
| CN116152678A (en) | Marine disaster-bearing body identification method based on twin neural network under small sample condition | |
| Antioquia et al. | ZipNet: ZFNet-level accuracy with 48× fewer parameters | |
| CN112990357B (en) | Black box video countermeasure sample generation method based on sparse disturbance | |
| CN112837281B (en) | Pin defect identification method, device and equipment based on cascade convolution neural network | |
| Park et al. | A study on fire detection in ship engine rooms using convolutional neural network | |
| CN110991243A (en) | Straw combustion identification method based on combination of color channel HSV and convolutional neural network | |
| CN110991366A (en) | Shipping monitoring event identification method and system based on three-dimensional residual error network | |
| Zhang et al. | Combining deep learning with R-CDT for solar defect recognition | |
| CN111428649A (en) | Remote sensing intelligent extraction method for wind power generation facility | |
| CN116645727B (en) | Behavior capturing and identifying method based on Openphase model algorithm | |
| CN116071624B (en) | Smoking detection data labeling method based on active learning | |
| Liu et al. | Research on automatic detection method of infrared marine buoy based on CNN |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |