CN112598071A

CN112598071A - Open fire identification method, device, equipment and storage medium

Info

Publication number: CN112598071A
Application number: CN202011577158.7A
Authority: CN
Inventors: 甘伟豪; 王意如; 王栋梁
Original assignee: Beijing Sensetime Technology Development Co Ltd
Current assignee: Beijing Sensetime Technology Development Co Ltd
Priority date: 2020-12-28
Filing date: 2020-12-28
Publication date: 2021-04-02

Abstract

The embodiment of the application provides an open fire identification method, an open fire identification device, open fire identification equipment and a storage medium, wherein the method comprises the following steps: acquiring a target identification network; the target identification network is obtained by training an initial network by adopting a first data set, and the initial network is obtained by training a preset neural network by adopting a second data set; and identifying the object contained in the image to be identified based on the target identification network, and determining whether the image to be identified contains open fire.

Description

Open fire identification method, device, equipment and storage medium

Technical Field

The embodiment of the application relates to the field of machine learning, and relates to but is not limited to an open fire identification method, an open fire identification device, open fire identification equipment and a storage medium.

Background

In the field of computer vision and deep learning, a neural network for detecting an open fire phenomenon in an image generally needs to be trained by using a training image set of more than ten thousand levels, and under the condition that sufficient training image data does not support training, the robustness of the trained neural network cannot be ensured.

Disclosure of Invention

The embodiment of the application provides an open fire identification method, an open fire identification device, open fire identification equipment and a storage medium.

The embodiment of the application provides an open fire identification method, which comprises the following steps:

acquiring a target identification network; the target identification network is obtained by training an initial network by adopting a first data set, and the initial network is obtained by training a preset neural network by adopting a second data set;

and identifying the object contained in the image to be identified based on the target identification network, and determining whether the image to be identified contains open fire. Therefore, the target identification network obtained by adopting two data training is adopted to identify the object contained in the image to be identified so as to confirm whether the image to be identified contains open fire or not, so that misjudgment on open fire identification can be reduced, and the accuracy of open fire identification is improved.

In some embodiments, confirming whether the image to be identified contains an open flame comprises: determining a plurality of frame images which are continuous in time sequence with the image to be identified in the case that a light-emitting object is included in the identified object; determining the duration of the continuous appearance of the light-emitting object in the multi-frame images and/or the moving distance of the light-emitting object in the duration of the continuous appearance of the light-emitting object in the multi-frame images; and when the duration of the continuous appearance of the luminous object is longer than the preset duration and/or the moving distance of the luminous object in the duration of the continuous appearance is shorter than the preset displacement, determining that the luminous object is an open fire. In this way, when the recognition result includes an open fire, the duration and the moving distance of the light-emitting object on the continuous multi-frame image are determined, so that erroneous determination of the open fire can be reduced.

In some embodiments, determining a moving distance of the light-emitting object over a duration of time of continuous presence in the plurality of frames of images comprises: determining the position information of the luminous object in the world coordinate system in each frame of image within the continuous appearing duration; and determining the moving distance of the luminous object within the continuous appearing time length according to the position information. In this way, the displacement of the light emitting object can be determined more accurately.

In some embodiments, the determining the duration of the continuous presence of the light-emitting object in the plurality of frames of images comprises: acquiring a multi-frame image which is continuous with the image to be identified in time sequence and contains the light-emitting object; and determining the duration of the continuous appearance of the light-emitting object in the multi-frame images based on the time difference value of the last frame image and the first frame image in the multi-frame images. Therefore, the duration of the continuous appearance of the luminous object can be conveniently and quickly judged, and the accuracy of open fire identification is further improved.

In some embodiments, confirming whether the image to be identified contains an open flame comprises: in a case where a light-emitting object is included in the recognized object, determining a plurality of frame images chronologically continuous with the image to be recognized: determining a target detection frame of the light-emitting object in the multi-frame image to obtain a target detection frame set; determining the intersection ratio between two target detection frames adjacent in time sequence in the target detection frame set; determining the duration of the intersection ratio which is greater than or equal to a preset ratio; and responding to the condition that the duration is greater than or equal to the preset duration, determining that the moving distance of the luminous object in the duration of continuous occurrence is less than the preset displacement, and confirming that the luminous object is open fire. In this way, by determining the intersection ratio between the detection frames of the light-emitting objects in the multi-frame images, whether the image to be identified includes open fire or not is judged, and abnormal false alarm can be reduced.

In some embodiments, the first data set comprises the second data set and an image data set of an open flame screened through the target recognition network; the second data set includes at least one of: the method comprises the steps of crawling internet data into a data set, generating the data set by a virtual game engine and generating an academic data set. Therefore, the richness of the second data set can be improved, and the first data set with balanced positive and negative samples can be obtained.

In some embodiments, the step of obtaining the first data set comprises: acquiring an image of a real open fire scene; identifying an object contained in an image of the real open fire scene based on the target recognition network; under the condition that the probability that the object is open fire is identified to be greater than or equal to a preset probability threshold value, marking the real open fire scene image; generating the first data set based on the labeled real open fire scene image and the second data set. Therefore, the target recognition network is adopted to label the open fire of the image collected in the real open fire scene, and the optimized first data set can be obtained by combining the second data set.

In some embodiments, after the confirming that the light-emitting object is an open fire, the method further comprises: determining an open fire type of the open fire; and generating and outputting alarm information based on the open fire type. Therefore, through monitoring the open fire in the image to be identified, the associated personnel can be timely reminded of processing the luminous object as the open fire based on the alarm information of the data.

The embodiment of the application provides an open fire recognition device, the device includes:

the first acquisition module is used for acquiring a target identification network; the target identification network is obtained by training an initial network by adopting a first data set, and the initial network is obtained by training a preset neural network by adopting a second data set;

and the first confirmation module is used for identifying the object contained in the image to be identified based on the target identification network and confirming whether the image to be identified contains open fire or not.

In the above apparatus, the first confirmation module includes:

a first determination submodule configured to determine, in a case where a light-emitting object is included in an identified object, a plurality of frame images that are consecutive in time series with the image to be identified;

a second determining submodule for determining a duration of the continuous appearance of the light-emitting object in the multi-frame image and/or a moving distance of the light-emitting object within the duration of the continuous appearance of the light-emitting object in the multi-frame image;

and the third determining submodule is used for determining that the luminous object is an open fire when the continuous appearance time of the luminous object is longer than the preset time and/or the moving distance of the luminous object in the continuous appearance time is shorter than the preset displacement.

In the above apparatus, the second determination submodule includes:

the first determining unit is used for determining the position information of the luminous object in the world coordinate system in each frame of image within the continuous occurrence time;

and the second determining unit is used for determining the moving distance of the luminous object within the continuous appearing time length according to the position information.

In the above apparatus, the second determining sub-module includes:

the first acquisition unit is used for acquiring a plurality of frames of images which are continuous in time sequence with the image to be identified and the image content of which comprises the light-emitting object;

and the third determining unit is used for determining the duration of the continuous appearance of the light-emitting object in the multi-frame images based on the time difference value of the last frame image and the first frame image in the multi-frame images.

In the above apparatus, the apparatus further comprises:

a first determination module configured to determine, in a case where a light-emitting object is included in the identified object, a plurality of frame images that are consecutive in time series with the image to be identified; determining a target detection frame of the light-emitting object in the multi-frame image to obtain a target detection frame set;

a second determining module, configured to determine an intersection ratio between two temporally adjacent target detection frames in the target detection frame set;

the third determining module is used for determining the duration of the intersection ratio which is greater than or equal to a preset ratio;

and the fourth determination module is used for responding to the condition that the duration is greater than or equal to the preset duration, determining that the moving distance of the luminous object in the duration of continuous occurrence is less than the preset displacement, and confirming that the luminous object is an open fire.

In the above apparatus, the first data set includes the second data set and an open flame image data set screened out by the target recognition network;

the second data set includes at least one of: the method comprises the steps of crawling internet data into a data set, generating the data set by a virtual game engine and generating an academic data set.

In the above apparatus, the apparatus further comprises:

a second obtaining module for obtaining the first data set; the second obtaining module includes:

the first acquisition submodule is used for acquiring an image of a real open fire scene;

a first identification submodule for identifying an object contained in an image of the real open fire scene based on the target identification network;

the first labeling submodule is used for labeling the real open fire scene image under the condition that the probability that the object is open fire is identified to be greater than or equal to a preset probability threshold;

and the first generation submodule is used for generating the first data set based on the marked real open fire scene image and the second data set.

In the above apparatus, the apparatus further comprises:

a fifth determining module for determining the open fire type of the open fire;

and the first generation module is used for generating and outputting alarm information based on the open fire type.

Embodiments of the present application provide a computer storage medium, where computer-executable instructions are stored, and after being executed, the computer-executable instructions can implement the above-mentioned method steps.

Embodiments of the present application provide a computer device, where the computer device includes a memory and a processor, where the memory stores computer-executable instructions, and the processor executes the computer-executable instructions on the memory to implement the above-mentioned method steps.

Embodiments of the present application provide a computer program comprising computer instructions for implementing the above-mentioned method steps.

According to the technical scheme provided by the embodiment of the application, for the image to be recognized, firstly, a target recognition network is obtained, the network adopts a second data set to train a preset network to obtain an initial network, and adopts a first data set to train the initial network to obtain the target recognition network capable of recognizing open fire; therefore, the preset neural network is trained by adopting the multi-source first data set in a wider range, the sensitivity of the target recognition network to open fire recognition can be improved, the initial network is trained by adopting the second data set, and the recognition accuracy of the obtained target recognition network in a real open fire scene can be improved. Then, open fire recognition is carried out on the image to be recognized through the target recognition network, so that the fact that open fire is contained in the image to be recognized is confirmed. Therefore, misjudgment of the open fire phenomenon can be reduced, and the accuracy of the identification result is improved.

Drawings

Fig. 1 is a schematic flow chart illustrating an implementation of an open fire identification method according to an embodiment of the present application;

FIG. 2 is a schematic diagram illustrating an implementation process of training a target recognition network according to an embodiment of the present disclosure;

FIG. 3 is a scene diagram of collecting images of an explosion and an open fire through the Internet according to an embodiment of the present application;

FIG. 4 is a scene diagram of collecting images of an explosion fire through a virtual game according to an embodiment of the present application;

FIG. 5 is a scene diagram of an embodiment of the present application for collecting images of an explosion and an open flame from an academic dataset;

fig. 6 is a schematic structural diagram of a RetinaNet provided in the embodiment of the present application;

FIG. 7 is a schematic diagram of an implementation scenario of data collection according to an embodiment of the present application;

FIG. 8 is a schematic diagram illustrating an implementation flow of training of a target recognition network according to an embodiment of the present application;

FIG. 9 is a schematic view of an intelligent transportation system recognizing an abnormal explosion and fire event management according to an embodiment of the present application;

fig. 10 is a schematic structural diagram of the composition of an open fire recognition device according to an embodiment of the present application;

fig. 11 is a schematic structural diagram of a computer device according to an embodiment of the present application.

Detailed Description

In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, specific technical solutions of the present invention will be described in further detail below with reference to the accompanying drawings in the embodiments of the present application. The following examples are intended to illustrate the present application but are not intended to limit the scope of the present application.

In the following description, reference is made to "some embodiments" which describe a subset of all possible embodiments, but it is understood that "some embodiments" may be the same subset or different subsets of all possible embodiments, and may be combined with each other without conflict.

In the following description, references to the terms "first \ second \ third" are only to distinguish similar objects and do not denote a particular order, but rather the terms "first \ second \ third" are used to interchange specific orders or sequences, where appropriate, to enable embodiments of the invention described herein to be practiced in other than the order shown or described herein.

Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. The terminology used herein is for the purpose of describing embodiments of the invention only and is not intended to be limiting of the invention.

Before further detailed description of the embodiments of the present invention, terms and expressions mentioned in the embodiments of the present invention are explained, and the terms and expressions mentioned in the embodiments of the present invention are applied to the following explanations.

1) Cold start: in the related art, cold boot is a boot method of a computer, which is to cut off the power supply of the computer and restart the computer. In the embodiment of the application, cold start refers to a process of developing a first version of a model according to existing data when no algorithm model exists.

2) Iterative training: and performing iterative optimization training through data increase on the basis of the model after cold start.

An exemplary application of the device for open fire identification provided in the embodiments of the present application is described below, and the device provided in the embodiments of the present application may be implemented as various types of user terminals such as a notebook computer with an image capture function, a tablet computer, a desktop computer, a camera, a mobile device (e.g., a personal digital assistant, a dedicated messaging device, a portable game device), and the like, and may also be implemented as a server. In the following, an exemplary application will be explained when the device is implemented as a terminal or a server.

The method may be applied to a computer device, and in some embodiments, the functions performed by the method may be implemented by a processor in the computer device invoking program code, where the program code may be stored in a computer storage medium.

An embodiment of the present application provides an open fire identification method, and fig. 1 is a schematic flow chart illustrating an implementation of the open fire identification method provided in the embodiment of the present application, and is described in detail below with reference to fig. 1.

And step S101, acquiring a target identification network.

In some embodiments, the target recognition network is obtained by training an initial network with a first data set obtained based on a real open fire scene and a cold start data set, the initial network is obtained by training a preset neural network with a second data set obtained from a cold start database, or the target recognition network is obtained by training an initial network with a first data set at least including a real open fire image in a real open fire scene, and the initial network is obtained by training a preset neural network with a second data set in a non-real open fire scene. The target recognition network is a trained neural network and is used for detecting the object in the image to be recognized. The preset neural network can be a neural network with any structure for target detection; examples of such Networks include the retina network (RetinaNet), fast-regional Convolutional Neural Networks (fast-RCNN), and regression detection (You Only Look one, YoLO). The image content included in the image in the first data set is acquired based on a real open fire scene, for example, an image acquired from a monitoring video of a traffic system; the image in the second data set includes a picture content acquired based on an unreal open fire scene, for example, the manner of acquiring the image in the second data set includes: images crawled through internet data, images generated using virtual game scenes, images extracted from academic data, and the like. The images in the second data set are images whose picture content includes an open flame. Therefore, the preset neural network is trained by adopting the image in the second data set including the open fire acquired in the unreal open fire scene to obtain the initial network, the training image is acquired from the multi-source image data in a wider range, and the sensitivity of the initial network obtained by training to the open fire identification can be improved. The picture content of the images in the first data set may or may not include an open flame. Therefore, the training image is optimized by the image at least comprising the first data set acquired in the real open fire scene, the initial network is iterated by the optimized sample image to obtain the target identification network, and therefore the identification accuracy of the obtained target identification network in the real open fire scene can be improved.

And S102, identifying the object contained in the image to be identified based on the target identification network, and confirming whether the image to be identified contains open fire or not.

In some embodiments, the target recognition network is adopted to recognize the object included in the picture content of the image to be recognized, which may be understood as inputting the image to be recognized into the target recognition network, and the target recognition network classifies and locates the object included in the picture content of the image to be recognized, and the obtained recognition result is whether the picture content includes an open fire and the position information of the open fire.

In some possible implementation manners, the image to be recognized may be an image acquired in any scene, may be an image with complex picture content, or may be an image with simple picture content. The image to be identified may be a plurality of frames or a frame of image, for example, a plurality of frames or a frame of image of a monitoring video in a traffic monitoring system, or a plurality of frames or a frame of image in a monitoring video in a shopping mall, or a plurality of frames or a frame of image collected in a scenic spot, etc.

The object included in the image to be recognized is an object which appears in the preset image database less than or equal to the preset number of times, and can also be understood as an event which occurs in a real open flame scene less frequently and participates in non-human. In a specific example, if the image to be recognized is an image of a collected building, after an object in the picture content of the image is recognized through a target recognition network, the object included in the picture content of the image to be recognized is obtained; if the explosion fire occurs in the objects, the display information of the picture of the explosion fire is determined. For example, the time length of the image in which the explosion fire occurs, the moving distance of the image in which the explosion fire occurs within a certain time length, and the like. And under the condition that the presentation information meets the preset condition, confirming that the image to be recognized contains open fire. In some possible implementations, the presence information meeting the preset condition indicates that the recognition result has a high probability of including an open fire. The presentation information is the presentation condition of the luminous object in the identification result in a multi-frame image which is continuous with the image to be identified in time sequence; for example, the presentation duration in the multi-frame images or the moving distance in the multi-frame images are comprehensively considered, and after the presentation information of the light-emitting object in the multi-frame images is confirmed to meet the preset condition, the image to be recognized is determined to include the open fire, so that the false alarm caused by recognizing a single-frame image can be reduced, and the accuracy of open fire recognition is improved.

In the embodiment of the application, for the image to be identified, firstly, a sample image under an unreal open fire scene is adopted for training to obtain an initial network; then, training an initial model by adopting a sample image set at least comprising a real open fire scene to obtain a target recognition network capable of recognizing an image to be recognized; therefore, the preset neural network is trained by adopting a multi-source sample image with a larger range, the sensitivity of the network to open fire identification can be improved, and the initial network is trained by adopting a sample image set at least under a real open fire scene, so that the identification accuracy of the obtained target identification network under the real open fire scene can be improved. And finally, identifying the object contained in the image to be identified by adopting the target identification network so as to confirm whether the image to be identified contains open fire or not. In this way, by identifying the object included in the image to be identified to determine whether the object includes an open fire, erroneous judgment of an open fire scene can be reduced, and the accuracy of the identification result can be improved.

In some embodiments, in a case where an open fire is included in the recognition result, it is determined whether the open fire is included in the image to be recognized, that is, whether the light-emitting object is an open fire, by determining the duration and the moving distance of the light-emitting object in the continuous multi-frame images. That is, the above step S102 may be implemented by the following procedure:

in step S121, in a case where a light-emitting object is included in the recognized object, a multi-frame image chronologically continuous with the image to be recognized is determined.

In some possible implementations, if the recognition result includes the light-emitting object, a multi-frame image that is chronologically continuous with the image to be recognized is determined, and the multi-frame image may be chronologically continuous with the image to be recognized or chronologically continuous with the image to be recognized in the future. Taking an example that the multi-frame image may be continuous with the image to be recognized in a historical time sequence, a continuous multi-frame image with the image to be recognized as the last frame is determined, that is, a multi-frame image continuous with the image to be recognized within a certain historical time period (for example, 2 minutes) is determined with the current time as an end point. In a specific example, if the light-emitting object is included in the recognition result, a continuous multi-frame image having the image to be recognized as the last frame is determined.

In step S122, a time period during which the light-emitting object continuously appears in the multi-frame images and/or a moving distance within the time period during which the light-emitting object continuously appears in the multi-frame images is determined.

In some possible implementations, the duration of the continuous presence is a duration of the continuous presence of the light-emitting object, and the moving distance may be a distance of continuous movement or a distance of non-continuous movement. If the luminous object appears in each frame of image, the continuous appearing time length is equal to the corresponding acquisition time length of the multi-frame image; if the luminous object appears in a part of the images, the continuous appearance time is shorter than the acquisition time corresponding to the multi-frame images. For example, the multi-frame image is an image within 3 minutes before the current time, but the light-emitting object continues to appear in the history image within 2 minutes before the current time, that is, the light-emitting object does not appear in the history images between the 2 nd minute and the 3 rd minute, and then the duration of the continuous appearance of the light-emitting object is 2 minutes. The moving distance of the light-emitting object within the duration of the continuous occurrence may be understood as the moving distance of the light-emitting object in the world coordinate system within the duration of the continuous occurrence, for example, the moving distance of the light-emitting object is determined by detecting the geographical position of the light-emitting object in the multi-frame image.

In some embodiments, the distance of movement of the light emitting object over the duration of the persistent presence may be achieved by: determining the position information of the luminous object in the world coordinate system in each frame of image within the continuous appearing time length; and determining a moving distance of the light-emitting object within the duration of the continuous presence based on the position information. For example, during the continuous occurrence period, the coordinates of the light-emitting object in the world coordinate system in the history image of the 2 nd minute before the current time are determined in the history image for the first time, and the coordinates of the light-emitting object in the world coordinate system in each frame of history image are determined continuously, and the moving distance of the light-emitting object is determined by the coordinates. In this way, by determining the world coordinates of the light-emitting object in the continuous multi-frame images to determine the moving distance of the light-emitting object, whether the light-emitting object moves greatly can be determined more accurately.

In some embodiments, the duration of time that the light-emitting object appears continuously in the plurality of frames of images may be determined by:

firstly, a multi-frame image which is continuous in time sequence with an image to be identified and the picture content of which comprises a light-emitting object is acquired.

In some possible implementation manners, if the identified object of the image to be identified includes a light-emitting object, the image which is continuous with the image to be identified in future time sequence and/or historical time sequence and the picture content of which includes the light-emitting object is acquired, and a multi-frame image is obtained. In a specific example, the image to be identified and the acquired image are continuous in future time sequence and/or historical time sequence, and the picture content comprises all the images of the luminous objects.

Then, based on the time difference value between the last frame image and the first frame image in the multi-frame images, the duration of the continuous appearance of the light-emitting object in the multi-frame images is determined.

In some possible implementation manners, the time sequence length occupied by the multi-frame images is determined, and the time sequence length is determined as the duration of the light-emitting object appearing in the multi-frame images continuously. The time sequence length is the time sequence length occupied by the multiple frames of images, namely the acquisition time length corresponding to the multiple frames of images from the first frame of image to the last frame of image; for example, the first frame of image is acquired in 5 minutes and 30 seconds, and the last frame of image is acquired in 8 minutes and 30 seconds, so that the time sequence length occupied by the plurality of frames of images is 3 minutes.

In some possible implementation manners, after the acquisition time length of the multiple frames of images is determined, since the picture content of each frame of image in the multiple frames of images includes the light-emitting object, the duration of the light-emitting object appearing continuously is the acquisition time length of the multiple frames of images. Therefore, the acquisition duration of the multi-frame images is used as the duration of the continuous appearance of the luminous object in the multi-frame images, the duration of the continuous appearance of the luminous object can be simply and accurately judged, and the accuracy of open fire identification is further improved.

And S123, when the duration of the continuous appearance of the luminous object is longer than the preset duration and/or the moving distance of the luminous object in the duration of the continuous appearance is shorter than the preset displacement, determining that the luminous object is an open fire.

In some possible implementation manners, the identified luminous object with the duration of continuous occurrence being greater than or equal to the preset duration is not a false alarm and is an object which appears at a high probability, for example, in a scene of identifying an explosion open fire picture, if the duration of continuous occurrence of the picture of the luminous object is greater than or equal to the preset duration, it is indicated that the picture has a high probability of being an open fire picture, and it is indicated that the acquisition scene of the image to be identified is a scene in which an open fire occurs; and/or, if the moving distance is less than the preset displacement, it indicates that the light-emitting object is a movable light-emitting object, such as a car light, in a scene of identifying an explosion and fire picture, if the light-emitting object is identified and the moving distance of the light-emitting object is large, it indicates that the identified light-emitting object is not an explosion and fire picture, and may be a car light or the like; if the luminous object is identified and the moving distance of the luminous object is small, the luminous object is indicated to be an open fire picture with high probability; in such a case, the scene of the image to be recognized is determined as the scene of the open fire, that is, the light-emitting object is determined as the open fire, so that false alarm caused by recognition using a single frame image can be reduced, and the accuracy of the recognition result can be improved.

In the above steps S121 to S123, the confirmation of the light-emitting object as the open flame includes the following three cases:

the first condition is as follows: and if the duration of continuous appearance of the light-emitting object in the multi-frame images which are continuous in time sequence with the images to be identified is greater than or equal to the preset duration, determining that the light-emitting object is open fire.

Case two: and if the moving distance of the luminous object in the continuous appearing time length is smaller than the preset displacement, the luminous object is determined to be open fire.

Case three: and if the moving distance of the luminous object in the continuous appearing time length is less than the preset displacement on the basis that the continuous appearing time length of the luminous object is greater than or equal to the preset time length, the luminous object is determined to be open fire.

In some embodiments, the determination of whether the light-emitting object is a real open fire is made by determining the intersection ratio between the detection frames of the light-emitting object in the multi-frame images, that is, the step S123 may be implemented by:

firstly, determining a target detection frame of a luminous object in a multi-frame image to obtain a target detection frame set.

In some possible implementation manners, for a plurality of acquired frames of images which are consecutive in time sequence with the image to be identified, a target detection frame for detecting the light-emitting object in each frame of images of the plurality of frames is determined. And determining whether the luminous object continuously appears for a longer time and moves for a longer distance by judging the intersection ratio among the plurality of target detection frames in the target detection frame set.

And secondly, determining the intersection ratio between two target detection frames adjacent in time sequence in the target detection frame set.

In some possible implementations, for each frame of an image in which a light-emitting object appears, a ratio of an intersection and a union between target detection frames of two frames of images adjacent in time series is determined. The larger the ratio is, the closer the two detection frames are, i.e. the higher the coincidence degree of the two detection frames is, further, the smaller the distance that the light-emitting object moves in the two adjacent frames of images is.

And thirdly, determining the duration of the intersection ratio which is greater than or equal to the preset ratio.

In some possible implementations, the degree of coincidence between the detection frames in adjacent images is determined by calculating a coincidence ratio for the detection frames of each adjacent two-frame image. The preset ratio may be set according to a possible moving distance of the light-emitting object, for example, if the possible moving distance of the light-emitting object is 1 meter, it indicates that the coincidence degree of the detection frames of the light-emitting object in the adjacent frames may not be completely coincident, for example, the coincidence degree is over 90%; based on this, the preset ratio may be set to 0.9. The fact that the coincidence ratio is larger than or equal to the preset ratio indicates that the coincidence degree between the detection frames in the adjacent images is larger, and the duration of the coincidence ratio between the detection frames in the adjacent images can be seen by determining the duration of the coincidence ratio larger than or equal to the preset ratio.

Fourthly, responding to the fact that the duration is longer than or equal to the preset duration, and confirming that the luminous object is open fire; or responding to the condition that the duration is longer than or equal to the preset duration and the moving distance of the luminous object in the duration appearing continuously is smaller than the preset displacement, and determining that the luminous object is open fire.

In some possible implementation manners, if the duration of the intersection ratio greater than or equal to the preset ratio is greater than or equal to the preset duration, it indicates that the coincidence degree between the detection frames of the light-emitting objects in the multi-frame image is greater within the preset duration; that is, within the preset time length, the moving distance of the light-emitting object in the multi-frame images at different times is small, and then the light-emitting object can be determined to be an open fire. In this way, by determining the intersection ratio between the detection frames of the light-emitting objects in the multi-frame images, whether the image to be identified includes open fire or not is judged, and abnormal false alarm can be reduced.

Through the first step to the fourth step, after the luminous object is confirmed to be the open fire, the alarm matched with the type is output by judging the type of the open fire so as to prompt a manager in time, and firstly, the open fire type of the open fire is determined; wherein the open fire types include: explosion open fire, lightning open fire, electric open fire, open fire generated by impact friction, open fire generated by combustible object combustion (such as waste incineration, building fire or tree fire, and the like), and the like. Then, according to the open fire type, alarm information is generated and output. In a specific example, the warning information matching with the open fire type is generated, for example, the open fire type is building fire, and a voice prompt, a text prompt or a video prompt with building fire identification is generated.

In the embodiment of the application, whether the image to be identified comprises open fire or not is determined by comprehensively considering the duration of continuous appearance of the light-emitting object in the multi-frame image and the moving distance in the duration, so that the accuracy of open fire identification can be improved.

In some embodiments, in order to improve the accuracy of the open fire recognition of the target recognition network, at least one of a data set obtained by crawling internet data, a data set generated by a virtual game engine and an academic data set is used as a second data set, and a preset neural network is trained into a target initial network; and the open fire image data set and the second data set screened out by the target identification network are used as the first data set to train the target initial network, so that the target identification network with strong robustness and high identification accuracy is obtained.

In some embodiments, to enable the first data set to better optimize the initial network, the first data set may be obtained by:

firstly, an image of a real open fire scene is acquired.

In some possible implementations, the real open fire scene is a scene where a real open fire occurs, such as a scene where an explosive open fire occurs, a scene where a building fire occurs, or a scene where a tree fire occurs. The image of the real open fire scene can adopt an image acquisition device, and the image is acquired in the real open fire scene; but also video frames extracted from videos monitoring real open fire scenes, and the like.

And secondly, identifying the object contained in the image of the real open fire scene based on the target identification network.

In some possible implementation manners, for an acquired image of a real open fire scene, a trained target recognition network is adopted to recognize an object contained in the image; for example, if the real open fire scene is a building fire scene, the objects included in the image of the real open fire scene include: open fire, buildings and roads on which the buildings are located.

And thirdly, under the condition that the probability that the object is open fire is identified to be greater than or equal to a preset probability threshold value, marking a real open fire scene image.

In some possible implementations, a trained target recognition network is employed to determine the probability that each object contained in the image of the real open fire scene is open fire. If the probability is larger than or equal to a preset probability threshold value, the image is likely to include open fire, namely, the object is likely to be open fire, and the real open fire scene image is labeled.

And fourthly, generating a first data set based on the marked real open fire scene image and the second data set.

In some possible implementation manners, the label matched with the naked flame is adopted to label the real naked flame scene image, so as to obtain the labeled real naked flame scene image. The label matching the open fire is understood to be a true label of the open fire, for example, the open fire is an explosive open fire, and the label is an explosive open fire. In this way, the real open fire scene images screened out by the target identification network and including the large open fire probability are labeled by adopting the truth value labels to obtain the labeled real open fire scene images; and the marked real open fire scene image and the second data set jointly form the first data set, so that the first data set can be optimized.

In the embodiment of the application, the target recognition network is adopted to label the open fire of the image acquired in the real open fire scene, the optimized first data set can be obtained by combining the second data set, and then the optimized first data set is adopted to train the initial network, so that the target recognition network with better performance is obtained.

In some embodiments, an initial network and a target recognition network are trained on the basis of establishing a set of efficient multi-source sample image collection process, and meanwhile, a recognition effect of the target recognition network in an actual application scene is better in a multi-frame time sequence fusion judgment mode, where a training process of the target recognition network is shown in fig. 2, fig. 2 is a schematic diagram of an implementation process of training of the target recognition network provided by an embodiment of the present application, and the following description is performed with reference to steps shown in fig. 2:

step S201, acquiring a second data set including an open fire from at least one preset image library.

In some possible implementations, the at least one preset image library includes, but is not limited to, at least one of: an internet image database, an image library generated by a virtual game, image data in academic data, and the like; in a specific example, the second data set is obtained by acquiring images including an open flame picture (for example, a building ignition picture, a gas stove combustion picture or a tree combustion picture) from a plurality of image data sources (i.e., a plurality of preset image databases), so that the richness of the second data set is improved.

Step S202, recognizing the object contained in the image of the second data set by adopting a preset neural network to obtain a second recognition result.

In some possible implementations, the predetermined neural network may be any convolutional neural network, and in a specific example, the structure of the predetermined neural network is as shown in fig. 6. And training the preset neural network by adopting the second data set, inputting the images in the second data set into the preset neural network, and identifying the objects included in the picture content of the images in the second data set by the preset neural network to obtain the probability that the picture content includes each object. For example, in a scene in which an explosion flame image is recognized, the object is set to be a flame image and a flame image, and the second recognition result is the probability of the flame image and the probability of the flame image.

And step S203, adjusting network parameters of a preset neural network by adopting the loss of the second recognition result to obtain an initial network.

In some possible implementation manners, based on the second recognition result and the truth label of the image in the second data set, the loss of the second recognition result is determined, and the network parameter of the preset neural network is adjusted by using the loss to obtain the initial network. The accuracy of the recognition result of the initial network is greater than that of the preset neural network.

The above steps S202 and S203 may be understood as taking the second data set as an initial training set, and training the preset neural network to obtain an initial network, that is, training the preset neural network to obtain the initial network by using the second data set obtained from the unreal open fire scene.

And step S204, acquiring a real open fire scene image.

In some possible implementations, images of an open flame scene are acquired under a real open flame scene. For example, acquiring an image of an open fire scene under a real open fire scene includes: the method comprises the steps of collecting an image of an open fire picture in a monitoring video of a traffic system, or collecting an image of an open fire picture in a monitoring video of a residential building.

Step S205, labeling the image meeting the preset conditions in the real open fire scene image by using an initial network, obtaining a first data set based on the labeled image and a second data set, and identifying the object included in the picture content of the first data set by using the initial network to obtain a first identification result.

In some possible implementations, the initial network is trained using real open flame scene images. And inputting the real open fire scene image into an initial network, and identifying the objects contained in the real open fire scene image by the initial network to obtain the probability of each object. For example, in a scene in which an explosion and fire picture is identified, the object is set to be an open fire picture and an open fire picture, and the first identification result is the probability of the open fire picture and the probability of the open fire picture.

And step S206, adjusting the network parameters of the initial network by adopting the loss of the first identification result so as to enable the obtained loss of the first identification result output by the target identification network to meet the convergence condition.

In some possible implementation manners, based on the first recognition result and a truth-value label of the real open fire scene image, the loss of the first recognition result is determined, the loss is adopted to adjust the network parameters of the initial network, and the target recognition network with the loss of the output first recognition result meeting the convergence condition is obtained. The accuracy of the recognition result of the target recognition network is greater than that of the initial network.

In the embodiment of the application, the preset neural network is trained by using the images in the second data set collected by the multi-source head to obtain the initial network, and the initial network is trained by using the first data set comprising the images in the real open fire scene and the second data set to obtain the target identification network, so that the robustness of the target identification network obtained by training is improved.

In some embodiments, in order to improve the recognition accuracy and robustness of the trained target recognition network in the actual application scenario, step S205 may be implemented by:

and step S251, determining the probability that the picture content of the real open fire scene image comprises open fire by adopting an initial network.

In some possible implementations, for each real open fire scene image, the initial network is used to identify the object included in the picture content of the image, that is, to determine the probability that the picture content of the real open fire scene image includes an open fire. For example, the initial network is used to determine the probability that the picture content of the image of the real open flame scene includes a picture in which the open flame occurs.

And step S252, determining the real open fire scene image as an image to be annotated under the condition that the probability is greater than or equal to a preset probability threshold.

In some possible implementation manners, if the probability is greater than or equal to a preset probability threshold, it is stated that the training real open fire scene image is likely to include open fire, and such real open fire scene image is taken as the image to be annotated.

And step S253, labeling the image to be labeled by adopting a label matched with the naked fire to obtain a labeled image, and taking the labeled image and the second data set as a first data set.

In some possible implementations, the label matching the open fire may be understood as a true label of the open fire, for example, if the open fire is a picture of the open fire, then the label is a picture of the open fire. Therefore, the images to be annotated screened out by the initial network and including the images with high open fire probability are annotated by adopting the truth-value labels to obtain the annotated images, so that the first data set is optimized.

Step S254, identifying an object included in the screen content of the first data set by using the initial network, and obtaining the first identification result.

In some possible implementation manners, firstly, the marked images and the second data set are used as the optimized first data set, and the initial network is trained to obtain a first recognition result of each marked image; and then, adjusting the initial network by using the loss of the first recognition result to obtain the target recognition network of which the loss of the output first recognition result meets the convergence condition.

In the embodiment of the application, the initial network is used for mining the images in the large-scale unmarked first data set, and iterative training is carried out, so that the target recognition network is more friendly to the environment in the actual application scene.

In the related art, in the field of computer vision and deep learning, target detectors such as fast-RCNN, RetinaNet, YOLO and the like can be used to complete an image target detection task, and there is a great demand for the data size of training images (for example, training images of ten thousand levels or more are required) in the model training process for realizing the image target detection task. If the data volume of the training image is insufficient, the stability and robustness of the trained detector cannot be guaranteed. In the related art, richness of training images is improved by performing image enhancement on existing images. However, due to the limited space for image enhancement, richer training images cannot be provided for network learning optimization. Moreover, image enhancement is processed based on a single image frame, but in practical application, a line system is a time-sequential system, and single frame processing often easily generates a large number of false alarms, so that the use effect of the system is poor.

Based on this, the embodiment of the application provides an open fire identification method, which can be realized through the following processes:

in the first step, image acquisition is performed in a cold start mode.

In some possible implementation manners, in the early stage of algorithm development, in the case that there is no image of an explosion and open fire in a real actual battle scene (e.g., a traffic scene), the embodiment of the present application collects image data in the following three manners:

the method comprises the steps of crawling through internet data, and collecting images of explosion open fire.

Referring to fig. 3, fig. 3 is a scene diagram of collecting images of explosion and open fire through the internet according to the embodiment of the present application, and as shown in fig. 3, in internet data, images of fire of residential buildings 302 in a crawling scene 301 are obtained.

And secondly, generating an explosion naked flame image based on the virtual game.

Referring to fig. 4, fig. 4 is a scene diagram of collecting images of an explosion fire through a virtual game according to an embodiment of the present application, and as shown in fig. 4, images 402 of burning vehicles are collected through a virtual game scene 401 constructed with explosion fire.

And thirdly, collecting images of the explosion open fire based on the academic data set.

Referring to fig. 5, fig. 5 is a scene diagram of an explosion open fire image collected by an academic dataset according to an embodiment of the present application, and as shown in fig. 5, an image 501 including an open fire is obtained from the academic dataset, and the image 501 may be an image including an open fire in any scene, for example, as shown in fig. 5, an image of an open fire of a gas stove burning in a kitchen scene.

In the above three ways, a cold start data set (corresponding to the second data set of the above embodiment) implementing an abnormal explosion open fire identification algorithm is constructed by these three aspects (wherein the explosion open fire images collected by the three ways are each proportional to 1/3). Therefore, data acquisition is carried out on targets such as the explosion open fire in a targeted manner by adopting an internet data crawling mode, and image data of the explosion open fire is generated by adopting a virtual game; therefore, the training set can be acquired from multi-source data in a wider range, and the network obtained by training is more efficient and sensitive to the perception of the explosive naked flame target in actual combat verification.

And secondly, training and developing the initial network.

In some possible implementation manners, the initial network may be a cold start model obtained by initially training the RetinaNet through cold start data, and after the cold start data is provided, the cold start data is used to train the RetinaNet target detection network to obtain the cold start model for explosion open fire recognition. Fig. 6 is a schematic structural diagram of a RetinaNet provided in an embodiment of the present application, where the structure diagram of the RetinaNet includes: a residual Network 601(ResNet), a Feature Pyramid Network 602 (FPN), a classification subnet 603(class subnet), and a detection box subnet 604(box subnet); w and H represent the height and width of the feature map, respectively, where:

the residual error network 601 is configured to perform feature extraction on an input image by using a multi-layer residual error network, and extract semantic features of deeper layers in the image by down-sampling layer by layer, so as to obtain features of different sizes.

The feature pyramid network 602 is configured to perform upsampling on features of different sizes extracted by the residual error network 601, fuse the upsampled features with the features of the layer extracted by the residual error network 601, and input the fused features into the classification sub-network 603 and the detection frame sub-network 604 respectively.

The classification subnetwork 603 is configured to classify the fused features to obtain a probability that the fused features include the open fire features, that is, to classify whether the fused features include the open fire features.

And the detection frame sub-network 604 is used for regressing the position where the open fire characteristic appears in the fusion characteristic, so that the trained RetinaNet can locate the position where the open fire occurs in the image.

And thirdly, optimizing the image data for iterative training.

In some possible implementations, based on the cold start model, an image data mining process is run in a large-scale real open fire scene video, and image data acquisition can be performed in a real open fire scene (e.g., an actual source). The image data mining is realized by the following steps: and judging the posterior probability of each image according to the cold start model, and when the posterior probability is greater than a certain threshold value, determining the image data as the image needing to be marked for collecting so as to perform iterative training on the cold start model by combining the image with the cold start data. In the process, more positive sample images in a real open fire scene can be acquired, more negative sample false reports can be generated, and the negative sample false reports are added into the training process of the cold start model, so that the suppression of the false reports in the real open fire scene by the optimization algorithm is facilitated. Therefore, in the embodiment of the application, a large number of levels of unlabeled images are mined through the cold start model, and iterative training is performed, so that the target recognition network obtained through training is more friendly to the actual combat scene environment. As shown in fig. 7, fig. 7 is a schematic view of an implementation scenario of data collection according to an embodiment of the present application, and is used for labeling an image including an open fire in a real open fire scene (for example, a monitoring video in a traffic scene); for example, for an image 701 representing a real traffic scene, the image 701 includes a scene of burning of an automobile 702, and the image 701 is determined as an image needing to be labeled.

And fourthly, performing logic fusion through the multi-frame time sequence images in the real open fire scene to determine whether alarm information is output or not.

In some possible implementation manners, in an intelligent traffic system, the appearance of night vehicle lamps is similar to that of open fire, so that more false alarms are easily generated on a trained explosion open fire model, and such data are not suitable for being put into network training optimization. Therefore, the car lights are considered to belong to moving objects (moving with the car) in the actual scene, but the accident explosion and smoke basically stay in the picture. The embodiment of the application introduces time sequence multi-frame image fusion confirmation logic, and judges whether the detected explosion and naked flame target appears in a picture for a long time, specifically, whether IoU of the target frame on different time frames continuously exceeds a certain threshold value is evaluated. If yes, the alarm is an explosion open fire event. In the embodiment of the present application, as shown in fig. 8, the following description is made in conjunction with the steps shown in fig. 8 for a process of implementing multi-frame time-series image fusion:

in step S801, a plurality of frames of temporally successive history images are input.

In step S802, object detection is performed on the screen content of each frame of the history image.

Step S803, determine whether the detection result includes an explosion abnormal picture.

In some possible implementations, if the detection result includes the explosion exception screen, step S804 is entered, otherwise, step S802 is continuously executed.

Step S804 determines whether the position of the explosion abnormal picture is output at the same position in the plurality of frames of history images.

In some possible implementations, the position of the explosion exception picture is output at the same position in the multiple frames of history images, and the process proceeds to step S805, otherwise, the process continues to step S802.

And step S805, generating and outputting alarm information of the explosion open fire event.

In this way, by fusing and considering the positions of the explosion abnormal pictures in the history images in which a plurality of frames are chronologically continuous, it is possible to reduce abnormal false alarms, and by multi-frame time-series fusion logic confirmation, it is possible to reduce false alarms due to interfering objects such as night lights.

In one particular example, road safety is a very important concern in an intelligent traffic management scenario. The first time it takes for a vehicle crash event to be alerted to a manager. The embodiment of the application can monitor the road condition in real time, and when the collision accident occurs, serious harm is caused, such as explosion, smoke and fire, the system can generate an alarm signal at the first time without allowing a manager to stare at a monitoring camera for 24 hours, so that manpower is greatly liberated, and the monitoring capability is improved. As shown in fig. 9, fig. 9 is a schematic diagram illustrating management of an abnormal explosion and open fire recognition event of an intelligent traffic system according to an embodiment of the present invention, and as can be seen from fig. 9, when a firework 95 is selected for monitoring in a "task type" 902 (including a congestion 91, a cloud 92, a pedestrian/vehicle intrusion 93, an abnormal parking 94, a firework 95 and a reverse 96) of an interface 901 program, a plurality of videos monitored in a "task name" 903 are sequentially displayed, the firework is displayed in the "task type" 902, the video source "904 shows that the video is originated from one-way monitoring equipment, which open fire images may exist in several places for the monitored video are displayed in an" alarm number "905, and the running state of the monitored video is displayed in a" state "906," running 907 "indicates that the monitored video runs normally; "stop" 908 means that the monitoring device is not accessed or video stops playing, etc.; "anomaly (1-way anomaly)" 909 can indicate that an image of an open flame appears in the video. The alarm log, task details, and whether to restart or terminate the surveillance video are displayed in operation 910.

In the embodiment of the application, a set of efficient multi-source collection flow of the images of the explosion open fire samples is established, a target identification network capable of detecting abnormal explosion open fire is trained on the basis, and meanwhile, the identification effect of the target identification network in an actual application scene is more stable and robust through a mode of fusion judgment of multi-frame time sequence images.

An embodiment of the present application provides an open fire recognition apparatus, fig. 10 is a schematic structural diagram of a composition of the open fire recognition apparatus of the embodiment of the present application, and as shown in fig. 10, an open fire recognition apparatus 1000 includes:

a first obtaining module 1001, configured to obtain a target identification network; the target identification network is obtained by training an initial network by adopting a first data set, and the initial network is obtained by training a preset neural network by adopting a second data set;

the first determining module 1002 is configured to identify an object included in an image to be identified based on the target identification network, and determine whether the image to be identified includes an open fire.

In the above apparatus, the first confirmation module 1002 includes:

In the above apparatus, the second determination submodule includes:

In the above apparatus, the second determining sub-module includes:

In the above apparatus, the apparatus further comprises:

a fifth determining module for determining the open fire type of the open fire;

It should be noted that the above description of the embodiment of the apparatus, similar to the above description of the embodiment of the method, has similar beneficial effects as the embodiment of the method. For technical details not disclosed in the embodiments of the apparatus of the present application, reference is made to the description of the embodiments of the method of the present application for understanding.

It should be noted that, in the embodiment of the present application, if the open fire identification method is implemented in the form of a software functional module and sold or used as a standalone product, it may also be stored in a computer-readable storage medium. Based on such understanding, the technical solutions of the embodiments of the present application may be essentially implemented or portions thereof contributing to the prior art may be embodied in the form of a software product stored in a storage medium, and including several instructions for causing a computer device (which may be a terminal, a server, etc.) to execute all or part of the methods described in the embodiments of the present application. And the aforementioned storage medium includes: various media capable of storing program codes, such as a usb disk, a removable hard disk, a Read Only Memory (ROM), a magnetic disk, or an optical disk. Thus, embodiments of the present application are not limited to any specific combination of hardware and software.

Correspondingly, the embodiment of the present application further provides a computer program product, which includes computer executable instructions for implementing the steps in the open fire identification method provided by the embodiment of the present application.

Accordingly, embodiments of the present application further provide a computer storage medium, where computer-executable instructions are stored on the computer storage medium, and the computer-executable instructions are used to implement the steps of the open fire identification method provided in the foregoing embodiments.

Accordingly, an embodiment of the present application provides a computer device, fig. 11 is a schematic structural diagram of the computer device in the embodiment of the present application, and as shown in fig. 11, the device 1100 includes: a processor 1101, at least one communication bus, a user interface, at least one external communication interface 1102 and memory 1103. Wherein the communication bus is configured to enable connected communication between the components. The user interface may include a display screen, and the communication interface 1102 may include standard wired and wireless interfaces, among others. Wherein the processor 1101 is configured to execute an image processing program in a memory to implement the steps of the open fire identification method provided by the above-mentioned embodiments.

The above description of the computer device and storage medium embodiments is similar to the description of the method embodiments above, with similar beneficial effects as the method embodiments. For technical details not disclosed in the embodiments of the computer device and the storage medium of the present application, reference is made to the description of the embodiments of the method of the present application for understanding.

It should be appreciated that reference throughout this specification to "one embodiment" or "an embodiment" means that a particular feature, structure or characteristic described in connection with the embodiment is included in at least one embodiment of the present application. Thus, the appearances of the phrases "in one embodiment" or "in an embodiment" in various places throughout this specification are not necessarily all referring to the same embodiment. Furthermore, the particular features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. It should be understood that, in the various embodiments of the present application, the sequence numbers of the above-mentioned processes do not mean the execution sequence, and the execution sequence of each process should be determined by its function and inherent logic, and should not constitute any limitation to the implementation process of the embodiments of the present application. The above-mentioned serial numbers of the embodiments of the present application are merely for description and do not represent the merits of the embodiments.

It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.

In the several embodiments provided in the present application, it should be understood that the disclosed apparatus and method may be implemented in other ways. The above-described device embodiments are merely illustrative, for example, the division of the unit is only a logical functional division, and there may be other division ways in actual implementation, such as: multiple units or components may be combined, or may be integrated into another system, or some features may be omitted, or not implemented. In addition, the coupling, direct coupling or communication connection between the components shown or discussed may be through some interfaces, and the indirect coupling or communication connection between the devices or units may be electrical, mechanical or other forms.

The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units; can be located in one place or distributed on a plurality of network units; some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiments of the present application.

In addition, all functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may be separately regarded as one unit, or two or more units may be integrated into one unit; the integrated unit can be realized in a form of hardware, or in a form of hardware plus a software functional unit.

Those of ordinary skill in the art will understand that: all or part of the steps for realizing the method embodiments can be completed by hardware related to program instructions, the program can be stored in a computer readable storage medium, and the program executes the steps comprising the method embodiments when executed; and the aforementioned storage medium includes: various media that can store program codes, such as a removable Memory device, a Read Only Memory (ROM), a magnetic disk, or an optical disk. Alternatively, the integrated units described above in the present application may be stored in a computer-readable storage medium if they are implemented in the form of software functional modules and sold or used as independent products. Based on such understanding, the technical solutions of the embodiments of the present application may be essentially implemented or portions thereof contributing to the prior art may be embodied in the form of a software product stored in a storage medium, and including several instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the methods described in the embodiments of the present application. And the aforementioned storage medium includes: a removable storage device, a ROM, a magnetic or optical disk, or other various media that can store program code.

The above description is only for the specific embodiments of the present application, but the scope of the present application is not limited thereto, and any person skilled in the art can easily conceive of the changes or substitutions within the technical scope of the present application, and shall be covered by the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

1. An open fire identification method, characterized in that the method comprises:

and identifying the object contained in the image to be identified based on the target identification network, and determining whether the image to be identified contains open fire.

2. The method according to claim 1, wherein confirming whether the image to be recognized contains an open flame comprises:

determining a plurality of frame images which are continuous in time sequence with the image to be identified in the case that a light-emitting object is included in the identified object;

determining the duration of the continuous appearance of the light-emitting object in the multi-frame images and/or the moving distance of the light-emitting object in the duration of the continuous appearance of the light-emitting object in the multi-frame images;

and when the duration of the continuous appearance of the luminous object is longer than the preset duration and/or the moving distance of the luminous object in the duration of the continuous appearance is shorter than the preset displacement, determining that the luminous object is an open fire.

3. The method of claim 2, wherein determining the distance traveled by the light-emitting object over the duration of the continuous presence of the light-emitting object in the plurality of frame images comprises:

determining the position information of the luminous object in the world coordinate system in each frame of image within the continuous appearing duration;

and determining the moving distance of the luminous object within the continuous appearing time length according to the position information.

4. The method of claim 2, wherein the determining the duration of the continuous presence of the light-emitting object in the plurality of frames of images comprises:

acquiring a multi-frame image which is continuous with the image to be identified in time sequence and contains the light-emitting object;

and determining the duration of the continuous appearance of the light-emitting object in the multi-frame images based on the time difference value of the last frame image and the first frame image in the multi-frame images.

5. The method according to any one of claims 1 to 4, wherein confirming whether the image to be recognized contains an open fire comprises:

determining a target detection frame of the light-emitting object in the multi-frame image to obtain a target detection frame set;

determining the intersection ratio between two target detection frames adjacent in time sequence in the target detection frame set;

determining the duration of the intersection ratio which is greater than or equal to a preset ratio;

and responding to the condition that the duration is greater than or equal to the preset duration, determining that the moving distance of the luminous object in the duration of continuous occurrence is less than the preset displacement, and confirming that the luminous object is open fire.

6. The method of any one of claims 1 to 5, wherein the first data set comprises the second data set and an image data set of an open flame screened through the object recognition network;

7. The method of any one of claims 1 to 6, wherein the step of obtaining the first data set comprises:

acquiring an image of a real open fire scene;

identifying an object contained in an image of the real open fire scene based on the target recognition network;

under the condition that the probability that the object is open fire is identified to be greater than or equal to a preset probability threshold value, marking the real open fire scene image;

generating the first data set based on the labeled real open fire scene image and the second data set.

8. The method according to any one of claims 1 to 7, wherein after the confirming that the light-emitting object is an open fire, the method further comprises:

determining an open fire type of the open fire;

and generating and outputting alarm information based on the open fire type.

9. An open flame identification device, characterized in that the device comprises:

10. A computer storage medium having computer-executable instructions stored thereon that, when executed, perform the method steps of any of claims 1 to 8.

11. A computer device comprising a memory having computer-executable instructions stored thereon and a processor operable to perform the method steps of any of claims 1 to 8 when the processor executes the computer-executable instructions on the memory.