CN115346060A - Picture abnormity identification method and device, electronic equipment and storage medium - Google Patents
Picture abnormity identification method and device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN115346060A CN115346060A CN202210928102.4A CN202210928102A CN115346060A CN 115346060 A CN115346060 A CN 115346060A CN 202210928102 A CN202210928102 A CN 202210928102A CN 115346060 A CN115346060 A CN 115346060A
- Authority
- CN
- China
- Prior art keywords
- frame image
- time period
- preset time
- current frame
- picture
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/762—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using clustering, e.g. of similar faces in social networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V2201/00—Indexing scheme relating to image or video recognition or understanding
- G06V2201/07—Target detection
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Medical Informatics (AREA)
- Software Systems (AREA)
- Computing Systems (AREA)
- Databases & Information Systems (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Health & Medical Sciences (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Artificial Intelligence (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Image Analysis (AREA)
- Closed-Circuit Television Systems (AREA)
Abstract
A picture abnormity identification method, a device, an electronic device and a storage medium, wherein the method comprises the following steps: acquiring a current frame image and a current background frame image; detecting the picture difference between the current frame image and the background frame image; and if the picture difference reaches a preset threshold value and the occupation ratio of various pixels in the current frame image meets a preset abnormal condition, determining that the current frame image has picture abnormality. By adopting the technical scheme of the application to identify the abnormal picture, the accuracy of identifying the abnormal picture can be effectively improved.
Description
Technical Field
The present application relates to the field of image processing technologies, and in particular, to a method and an apparatus for identifying a picture anomaly, an electronic device, and a storage medium.
Background
In the fields of monitoring and the like, the purpose of monitoring can be realized through pictures collected by the camera equipment. In the process of capturing a picture by the image capturing apparatus, a situation of picture abnormality such as monochrome, blurring, and the like sometimes occurs, for example, a situation in which the image capturing apparatus is intentionally blocked or a malfunction of the image capturing apparatus occurs. If there is an abnormality, normal use of the image pickup apparatus will be affected.
In the existing method for detecting the screen abnormality, whether the screen has the abnormality is generally determined according to the difference between the front frame image and the rear frame image of the screen.
However, the way of identifying the screen abnormality in this way is less accurate.
Disclosure of Invention
In view of the foregoing, it is desirable to provide a screen abnormality recognition method, apparatus, electronic device, and storage medium capable of improving screen abnormality recognition accuracy.
According to a first aspect, an embodiment provides a picture anomaly identification method, including:
acquiring a current frame image and a current background frame image;
detecting the picture difference degree between the current frame image and the background frame image;
and if the picture difference reaches a preset threshold value and the proportion of various pixels in the current frame image meets a preset abnormal condition, determining that the current frame image has picture abnormality.
According to a second aspect, an embodiment provides a picture abnormality recognition apparatus, including:
the acquisition module is used for acquiring a current frame image and a current background frame image;
the detection module is used for detecting the picture difference degree between the current frame image and the background frame image;
and the processing module is used for determining that the current frame image has abnormal pictures if the picture difference reaches a preset threshold value and the occupation ratio of various pixels in the current frame image meets a preset condition.
In an embodiment, the processing module is further configured to obtain a ratio of each type of pixel in the current frame image if the picture difference reaches a preset threshold; and if the proportion of each pixel in the current frame image meets a preset abnormal condition, determining that the current frame image has abnormal pictures.
In one embodiment, the detection module is further configured to determine a picture difference degree between the current frame image and the background frame image according to a difference value between the current frame image and the background frame image.
In one embodiment, the processing module is further configured to perform clustering processing on pixel values of pixels of the current frame image to obtain the number of pixels of each category; and determining the proportion of the number of the pixel points of each category to the number of all the pixel points as the proportion of each pixel in the current frame image.
In one embodiment, the detection module is further configured to detect whether a target object exists in the current frame image, so as to obtain a detection result; and if the detection result indicates that the target object does not exist in the current frame image, detecting the picture difference degree between the current frame image and the background frame image.
In one embodiment, the picture abnormality recognition apparatus further includes a background update module; the detection module is further configured to, if the detection result indicates that a target object exists in the current frame image, continue to detect whether a target object exists in a next frame image, until no target object exists in the frame image, detect a picture difference between the frame image without the target object and the background frame image;
and the background updating module is used for updating the background frame image according to the frame image in the preset time period after the preset time period if the picture difference is smaller than the preset threshold and the occupation ratio of various pixels in the frame image does not meet the preset abnormal condition.
In one embodiment, the background updating module is further configured to update the background frame image according to the frame image in the preset time period every preset time period if the target object is not detected in the frame image in the preset time period.
In an embodiment, the background updating module is further configured to update the background frame image according to the frame image in the preset time period if the target object is not detected in the frame image in the preset time period, the picture difference degree is smaller than a preset threshold, and the occupation ratio of each type of pixel in the frame image in the preset time period does not satisfy the preset abnormal condition.
In one embodiment, the background updating module is further configured to determine, every first preset time period, a first image from the frame images within the first preset time period and update the background frame image to the first image if the target object is not detected in the frame images within the first preset time period;
and/or determining a second image from the frame images in a second preset time period and updating the background frame image into the second image if the target object is not detected in the frame images in the second preset time period every second preset time period; the second preset time period is greater than the first preset time period.
According to a third aspect, an embodiment provides an electronic device, including a memory and a processor, where the memory stores a computer program, and the processor implements the steps of the screen abnormality identification method according to the embodiments of the present application when executing the computer program.
According to a fourth aspect, an embodiment provides a computer-readable storage medium having a program stored thereon, the program being executable by a processor to implement the screen abnormality identification method of the embodiments of the present application.
A computer program product or computer program comprising computer instructions stored in a computer readable storage medium; the processor of the computer device reads the computer instructions from the computer-readable storage medium, and when the processor executes the computer instructions, the steps in the screen abnormality identification method according to the embodiments of the present application are implemented.
According to the method, the device, the electronic equipment and the storage medium for identifying the picture abnormity, the picture abnormity identification is carried out by fully utilizing the characteristics of the abnormal picture, and the picture abnormity identification is carried out through the steps, so that the accuracy of the picture abnormity identification is effectively improved.
Drawings
FIG. 1 is a diagram of an implementation environment of a method for identifying an image anomaly in an embodiment;
FIG. 2 is a flow diagram of a method for picture anomaly identification in one embodiment;
FIG. 3 is a flowchart illustrating an embodiment of determining whether the difference between frames reaches a predetermined threshold and whether the percentage of each pixel in the current frame meets a predetermined abnormal condition;
FIG. 4 is a frame of image taken by the electronic device;
FIG. 5 is a result of clustering performed on FIG. 4;
FIG. 6 is a flowchart of a method for identifying anomalies in a display screen in another embodiment;
FIG. 7 is a diagram illustrating an operation of a method for identifying an image anomaly in an embodiment;
FIG. 8 is a block diagram illustrating an exemplary embodiment of a device for detecting screen abnormalities;
FIG. 9 is a block diagram showing the construction of a screen abnormality recognition apparatus in another embodiment;
fig. 10 is a block diagram of an electronic device in an embodiment.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more apparent, the present application is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of and not restrictive on the broad application. Similar elements in different embodiments have been given the same associated similar element numbers. In the following description, numerous details are set forth in order to provide a better understanding of the present application. However, one skilled in the art will readily recognize that some of the features may be omitted or replaced with other elements, materials, methods in different instances. In some instances, certain operations related to the present application have not been shown or described in this specification in order not to obscure the core of the present application with unnecessary detail, and it is not necessary for those skilled in the art to describe these operations in detail, so that they may be fully understood from the description in the specification and the general knowledge in the art.
Furthermore, the features, operations, or characteristics described in the specification may be combined in any suitable manner to form various embodiments. Also, the various steps or actions in the method descriptions may be transposed or transposed in order, as will be apparent to one of ordinary skill in the art. Thus, the various sequences in the specification and drawings are for the purpose of describing certain embodiments only and are not intended to imply a required sequence unless otherwise indicated where such sequence must be followed.
The numbering of the components as such, e.g., "first", "second", etc., is used herein only to distinguish the objects as described, and does not have any sequential or technical meaning. The term "connected" and "coupled" when used in this application, unless otherwise indicated, includes both direct and indirect connections (couplings).
Referring to fig. 1, fig. 1 is a schematic diagram of an implementation environment according to the present invention. The implementation environment includes a server 100, a router 200, a terminal 300, a gateway 400, a smart device 500, and the like.
The terminal 300 may communicate with the server 100 through a network, and the terminal 300 may be a desktop computer, a notebook computer, a tablet computer, a smart phone, an intelligent control panel, or other electronic devices that can implement network connection, which is not limited herein. The server 100 may be implemented as a stand-alone server or a server cluster composed of a plurality of servers. The terminal 300 may establish a network connection with the router 200, for example, a network connection with the router 200 through WiFi or the like, and access a local area network established by the router 200.
The smart device 500 may include smart lights, smart air conditioners, smart curtains, smart control panels, smart cameras, and so forth. The smart device 500 may communicate with the gateway 400 through its own configured communication module, and thus be controlled by the gateway 400. In one embodiment, the gateway 400 may establish a local area network, and the smart device 500 is connected to the gateway 400 to access the local area network established by the gateway 400. The gateway 400 may establish a lan through ZIGBEE or bluetooth or other communication means. Meanwhile, the gateway 400 may establish a network connection with the router 200 and further access the local area network established by the router 200, so that the terminal 300 and the smart device 500 are in the same local area network, and a user may interact with the smart device 500 accessing the gateway 400 by means of the terminal 300 to control the smart device 500 to perform a corresponding action or view information sent by the smart device 500.
Referring to fig. 2, the method for recognizing a screen abnormality in an embodiment of the present application includes steps 110 to 140, which is described by taking an example that the method is applied to an electronic device, where the electronic device may be an electronic device with a camera function, for example, specifically, an intelligent camera device in the intelligent device 500 in fig. 1, and the intelligent camera device may include a cat-eye camera, a fixed monitoring camera, and the like. The method enables the electronic equipment to perform self-checking work of the picture, ensures normal operation of the functions of the electronic equipment, and can find the abnormal picture of the electronic equipment in time when the abnormal picture (such as being shielded, color cast, abnormal brightness and the like) occurs, which is specifically described below.
Step 110: and acquiring a current frame image and a current background frame image.
The electronic device continuously captures frame images to form a video when capturing the video. The current frame image is a frame image which needs to be processed by the electronic device at the current moment. The background frame image is a reference image for comparing a current frame image with a frame image shot by the electronic device, and the background frame image may be a frame image shot by the electronic device at a previous moment. The background frame image may be updated continuously, for example, every preset time interval, and a picture taken when the electronic device is turned on may be used as an initial background frame image.
When the electronic device recognizes the abnormal picture, it needs to acquire the current frame image and the current background frame image. The abnormal picture can be identified every frame of image, or once every preset time interval, or once when a preset trigger condition is met.
Step 120: and detecting the picture difference degree between the current frame image and the background frame image.
The picture difference is a difference between corresponding attributes of at least two frames of images, and may include, for example, a difference between attributes such as pixel distribution, sharpness, and resolution.
After the electronic equipment acquires the current frame image and the current background frame image, the current frame image is compared with the background frame image, the difference between the current frame image and the background frame image is distinguished, and the picture difference degree is obtained. For example, the current frame image and the background frame image may be operated according to the pixel values to obtain the picture difference. The picture difference degree can be expressed by a value, a percentage, or the like.
Step 130: and judging whether the picture difference between the current frame image and the background frame image reaches a preset threshold value or not and whether the occupation ratio of various pixels in the current frame image meets a preset abnormal condition or not.
Step 140: and determining that the current frame image has picture abnormity.
After the electronic device identifies the difference between the current frame image and the background frame image, if the picture difference between the current frame image and the background frame image reaches the preset threshold and the occupation ratio of each pixel in the current frame image meets the preset abnormal condition, step 140 is executed.
The preset threshold may be preset, and may be specifically set according to experience and actual needs, and in one example, may be specifically 30%.
According to the picture abnormity identification method, the picture difference degree between the current frame image and the background frame image is detected at first, whether the picture difference degree reaches the preset threshold value or not and whether the occupation ratio of various pixels in the current frame image meets the preset abnormity condition or not are judged, when the picture difference degree reaches the preset threshold value and the occupation ratio of various pixels in the current frame image meets the preset abnormity condition, the current frame image is determined to have picture abnormity, the characteristics of an abnormal picture are fully utilized to carry out picture abnormity identification, the picture abnormity identification is carried out through the steps, the accuracy of identifying the picture abnormity is effectively improved, and various picture abnormity conditions can be detected.
Referring to FIG. 3, step 130 includes steps 131-133 in one embodiment, which are described in detail below.
Step 110: and acquiring a current frame image and a current background frame image.
Step 120: and detecting the picture difference degree between the current frame image and the background frame image.
Step 131: it is determined whether the difference between the current frame image and the background frame image reaches a predetermined threshold, and if yes, step 132 is executed.
Step 132: and acquiring the ratio of each pixel in the current frame image.
In an embodiment, the proportion of each type of pixels in the current frame image may be determined through clustering, and reference may be made to the following description of clustering.
Step 133: and judging whether the occupation ratio of various pixels in the current frame image meets a preset abnormal condition, if so, executing the step 140.
In an embodiment, for the above-mentioned occupation ratios of various pixels obtained by clustering the pixel values of the pixels of the current frame image, since the distribution of the pixel values of the image is relatively single and the occupation ratio of one or two types of pixels is relatively large when the conditions of shading, color cast, abnormal brightness, etc. occur in the picture, the preset abnormal condition may be set such that the sum of the occupation ratios of the first two types of pixels with the largest occupation ratio reaches the preset occupation ratio threshold, that is, the step 140 is executed when the sum of the occupation ratios of the first two types of pixels with the largest occupation ratio reaches the preset occupation ratio threshold.
Fig. 4 is a frame of image shot by an electronic device, where an occlusion exists in the image, an area a is an occluded area, and a kmeans clustering algorithm is taken as an example, a category k =5 is set, an occupancy threshold is 50%, and a corresponding clustering result is obtained after clustering processing is performed.
In an embodiment, as shown in fig. 5, a schematic diagram of a clustering result obtained after the clustering process is performed on fig. 4 is shown. In fig. 5, 5 types of pixel results are shown, that is, the pixel types corresponding to the areas 51-55 shown in fig. 5. The first two types of pixels with the largest ratio are the two types of pixels of the category 51 and the category 53 in fig. 5, and the sum of the ratios of the two types of pixels exceeds 50%, so that the frame image can be determined to have a picture abnormality.
In one embodiment, the electronic device may determine the picture difference between the current frame image and the background frame image according to the difference between the current frame image and the background frame image. For example, the difference between the current frame image and the background frame image can be obtained by subtracting the background frame image from the current frame image, and the ratio of the difference is used as the picture difference between the current frame image and the background frame image.
In the embodiment, the image difference between the current frame image and the background frame image is determined by adopting a back difference method, the method is easy to realize, and the image difference between the current frame image and the background frame image can be rapidly and accurately determined due to the fact that the back difference method is sensitive to the difference between the current frame image and the background frame image.
As described above, the electronic device may identify the picture anomaly every frame of image, or perform identification every preset time period (for example, 30 minutes), and/or perform identification once when a preset trigger condition is met.
In one embodiment, the method for identifying an image anomaly further includes: clustering the pixel values of the pixels of the current frame image to obtain the number of the pixels of each category; and determining the proportion of the number of the pixel points of each category to the number of all the pixel points as the proportion of each pixel in the current frame image.
The proportion of each type of pixels is the proportion of each type of pixels in all pixels of the current frame image, and may specifically be the number proportion of the pixels of each type in the image in all pixels of the current frame image, for example.
The ratio of each type of pixels in the current frame image in step 130 can be determined by clustering in one embodiment.
Specifically, the electronic device may perform clustering processing on pixel values of pixels of the current frame image to classify the pixels to obtain the number of pixels of each category, and then determine the ratio of the number of pixels of each category to the number of all pixels as the ratio of the number of pixels of each category to the number of pixels of the current frame image. The clustering process can adopt a kmeans clustering algorithm and the like, and the number of categories can be set according to actual needs.
In this embodiment, the pixel values of the pixel points of the current frame image are clustered to obtain the ratios of various pixels in the current frame image, so that the implementation is simple, the operation speed is high, and the classification is accurate.
In some embodiments, in order to prevent a person, an animal, or the like from affecting the judgment of the screen abnormality recognition, the electronic device may detect the image, and perform the recognition of the screen abnormality only in the case where no person, animal, or the like is present, to exclude their effect.
Referring to fig. 6, the method for identifying abnormal frames in this embodiment includes steps 210 to 260, wherein step 210 refers to step 110 above, and steps 210 to 260 are described in detail below.
Step 210: and acquiring a current frame image and a current background frame image.
Step 220: and detecting whether a target object exists in the current frame image to obtain a detection result.
The target object comprises a human figure, a human face, an animal and the like, and the electronic device can perform detection on the target object in a deep learning manner, for example, for the human figure, a human figure detection model based on a deep neural network can be used for detection. The detection is performed in a deep learning manner, and other hardware devices such as PIRs (Passive Infrared Radiation), distance sensors, depth cameras and the like can be independent.
Step 230: and judging whether the detection result indicates that the target object does not exist in the current frame image.
If yes, that is, if the detection result indicates that the target object does not exist in the current frame image, step 240 is executed: and detecting the picture difference degree between the current frame image and the background frame image.
Step 250: and judging whether the picture difference between the current frame image and the background frame image reaches a preset threshold value or not and whether the occupation ratio of various pixels in the current frame image meets a preset abnormal condition or not.
Step 260: and determining that the current frame image has picture abnormity.
If not, it indicates that the target object exists in the current frame image, the step 240 is not executed, and the next frame image continues to be detected. In short, when the timing for performing screen abnormality recognition is reached, the electronic device does not perform screen abnormality recognition if the target object exists in the image, and performs recognition again until the target object does not exist.
It should be noted that, in this embodiment, the same is true for the case where the above-mentioned abnormal recognition of the screen is triggered when a person is detected, and the abnormal recognition of the screen is not immediately performed when the frame image in which the person is detected until no person is present in the image, that is, the abnormal recognition of the screen is performed when the image is switched from the person to the person.
In the embodiment, the screen abnormity is not identified when the target object exists in the image, and the screen abnormity is identified only when the target object does not exist, so that the interference of the target object can be eliminated, and the screen abnormity identification result is more accurate and reliable.
In one embodiment, if the detection result indicates that a target object exists in the current frame image, the electronic device continues to detect whether the target object exists in the next frame image until the target object does not exist in the frame image, the electronic device detects a picture difference between the frame image without the target object and the background frame image, and if the picture difference is smaller than a preset threshold and the occupation ratio of each type of pixel in the frame image does not satisfy a preset abnormal condition, the background frame image is updated according to the frame image within a preset time period after the preset time period. Namely, the frame image is subjected to image abnormity recognition once from the existence of the target object to the nonexistence of the target object, and the background frame image is updated if no abnormity exists.
Specifically, when a target object is detected, the state of the target object is recorded first, whether the target object exists in the next frame of image is continuously detected, after no target object exists in the frame of image, the picture difference degree between the frame of image without the target object and the background frame of image is detected, if the picture difference degree is smaller than a preset threshold value and the occupation ratio of various pixels in the frame of image does not meet a preset abnormal condition, the background frame of image is updated, the background frame of image can be updated immediately or after a preset time period, and the background frame of image can be updated according to the frame of image in the preset time period during updating, for example, the frame of image with the closest time distance is used as the background frame of image. In this case, if it is set that the background frame image is updated every preset time period, it is not necessary to wait for the inter-arrival time to be updated, for example, it is set that the updating is performed every 30 seconds, if the target object is not detected at the inter-arrival time, the screen abnormality recognition is performed after there is no target object in the frame image, and if there is no abnormality, the background frame image is immediately updated, and it is not necessary to wait for the inter-arrival time to be 30 seconds. Of course, the update may be performed after the inter-arrival time.
In this embodiment, if it is detected that a target object exists in the frame image, the frame abnormality recognition is performed when the target object does not exist, which is equivalent to triggering the frame abnormality recognition by the target object, so that the frame abnormality caused by the target object can be detected in time, the interference of the target object is eliminated, and if there is no abnormality, the background frame image is updated, which can adapt to the scene change caused by the target object.
In an embodiment, every preset time period, if the target object is not detected in the frame image within the preset time period, the electronic device updates the background frame image according to the frame image within the preset time period. The background frame image may be updated every preset time period, for example, every 1 minute, 30 minutes, and the like, and may be updated according to the frame image within the preset time period.
It is to be understood that the background frame image should be an image that does not include the target object. Therefore, in this embodiment, the electronic device may perform target object detection on each frame image captured within a preset time period, and when the background frame image needs to be updated, if the target object is not detected in the frame image within the preset time period, update the background frame image according to the frame image within the preset time period, for example, determine a frame image as the background frame image, or use a frame image with the closest time distance as the background frame image.
In this embodiment, target object detection is performed on all frame images within a preset time period, and if a target object is not detected in the frame images within the preset time period, the background frame image is updated according to the frame images within the preset time period, so that it can be ensured that interference of the target object does not exist in the background frame image.
In an embodiment, if the target object is not detected in the frame image within the preset time period, the picture difference degree is smaller than the preset threshold, and the occupation ratio of each type of pixels in the frame image within the preset time period does not meet the preset abnormal condition, the background frame image is updated according to the frame image within the preset time period.
In this embodiment, it is necessary to ensure that the frame images within the preset time period are not abnormal, and therefore, the electronic device may perform target object detection and image abnormality identification on each frame image captured within the preset time period.
When the target object is not detected in the frame image within the preset time period and no picture is abnormal, the electronic device updates the background frame image according to the frame image within the preset time period, namely if the target object is not detected in the frame image within the preset time period, the picture difference degree is smaller than a preset threshold value, and the occupation ratio of various pixels does not meet a preset abnormal condition, the electronic device can update the background frame image according to the frame image within the preset time period.
In this embodiment, not only the target object detection but also the image anomaly recognition are performed on the frame image within the preset time period, so as to ensure that the frame image within the preset time period has no target object or no anomaly, and then the background frame image is updated according to the frame image within the preset time period, so that the background frame image can be ensured to be a normal image without interference of the target object.
In some embodiments, every second preset time period, if the target object is not detected in the frame image within the first preset time period, the first image is determined from the frame image within the first preset time period, and the background frame image is updated to the first image, and/or every second preset time period, if the target object is not detected in the frame image within the second preset time period, the second image is determined from the frame image within the second preset time period, and the background frame image is updated to the second image, where the second preset time period is greater than the first preset time period.
Specifically, there may be two time periods spaced for updating the background frame image, which are a first preset time period and a second preset time period respectively, the two time periods have different lengths, and the second preset time period is not set to be greater than the first preset time period, for example, the first preset time period may be set to be 1 minute, and the second preset time period may be set to be 30 minutes.
The first image is a frame of image within a first preset time period, for example, a last frame of image captured by the electronic device within the first preset time period, the second image is a frame of image within a second preset time period, for example, a last frame of image captured by the electronic device within the second preset time period, and the first image and the second image are used for updating the background frame image.
When the background frame image is updated, the electronic device may determine, every a first preset time period, a first image from the frame images within the first preset time period as the background frame image if the target object is not detected in the frame images within the first preset time period; or every second preset time period, if the target object is not detected in the frame image in the second preset time period, the electronic device determines the second image as the background frame image from the frame image in the second preset time period, or the two images may be performed simultaneously.
In an embodiment, two frames of candidate background frame images may be set, including a first candidate background frame image and a second candidate background frame image, and one of the candidate background frame images is selected as the background frame image according to different situations when performing abnormal picture recognition, where the first candidate background frame image is updated every first preset time period, and the second candidate background frame image is updated every second preset time period.
In this embodiment, there may be two time periods between updating of the background frame image, the two time periods have different lengths, the background frame image with different interval durations may be selected according to different scenes to perform the image anomaly identification, and the application range is wider.
By updating the background frame image in time, the method can effectively adapt to the change of a shooting scene, prevent alarm abnormity caused by light, outdoor object change and the like, and is favorable for solving the problems of frequent report, false report, failure in report and the like of abnormal pictures.
The screen abnormality recognition method according to the present application will be described below with reference to fig. 7 by way of a specific example. In this example, there are two trigger conditions for the screen abnormality recognition, one is to perform the screen abnormality recognition when a person is detected, and the other is to perform the screen abnormality recognition every 30 minutes. When the fact that people perform abnormal picture recognition is detected, a first alternative background frame image is used as a background frame image, when the fact that people perform abnormal picture recognition at intervals of 30 minutes, a second alternative background frame image is used as a background frame image, the first alternative background frame image is updated every 1 minute, the second alternative background frame image is updated every 30 minutes, and the second alternative background frame image is updated after the fact that the picture is recognized to be abnormal is also used. Both the picture abnormality recognition and the background frame image update need to be performed without a person.
Referring to fig. 7, the electronic device first acquires an image, and then performs human shape detection on each frame of the image captured in real time, where the human shape detection may be human body detection, and may further include detecting a part of a human body, such as detecting a human face or an upper half of the human body, and also regarding as detecting a human shape.
And if the human shape still exists, the next frame of image is continuously detected until the human shape does not exist. And then, performing picture abnormity identification, wherein the first alternative background frame image is used as a background frame image, the picture difference degree between the current frame image and the background frame image is determined by adopting a back difference method, and if the picture difference degree reaches a preset threshold value and the occupation ratio of various pixels in the current frame image meets a preset abnormity condition, the current frame image is determined to have picture abnormity.
If there is no abnormality in the screen, the background is updated, and for example, the update may be performed every 1 minute. And if no human shape is detected in the frame images within 1 minute, taking the last frame image shot by the electronic equipment within the 1 minute as a first candidate background frame image, and if no target object is detected in the frame images within 30 minutes, taking the last frame image shot by the electronic equipment within the 30 minutes as a second candidate background frame image every 30 minutes. In the background updating process, both the two candidate background frame images may be specifically updated, or only one of the two candidate background frame images may be updated. The screen abnormality recognition performed at this time can recognize a screen abnormality caused by a person, for example, a camera is artificially blocked.
Further, the screen abnormality recognition may be performed at intervals of 30 minutes. And when the time interval reaches 30 minutes, acquiring the next frame of image, detecting whether the human figure exists in the frame of image, performing abnormal picture identification when the human figure does not exist, using a second alternative background frame image as a background frame image, and determining the picture difference degree between the current frame of image and the background frame of image by using a back difference method. And if the picture difference reaches a preset threshold value and the occupation ratio of various pixels in the current frame image meets a preset abnormal condition, determining that the current frame image has picture abnormality. If the picture is not abnormal, the background updating is carried out, and both the two alternative background frame images can be updated or only one of the two alternative background frame images can be updated. The screen abnormality recognition performed at this time can recognize a screen abnormality caused by a factor other than a person, for example, strong light irradiation, and the like, and can avoid that other abnormality cannot be recognized when no person is detected.
When the picture is abnormal, the message can be pushed to the APP to be checked by the user, the processing is prompted, and the recorded video can be uploaded to the APP.
In a specific application scene, the method can be particularly applied to the field of intelligent home monitoring. Wherein, electronic equipment specifically can be the camera equipment of fixed camera, and fixed camera specifically still can be the doorbell camera, cat eye etc. promptly, and the doorbell camera is similar surveillance camera head. The purpose is to carry out the self-checking work of camera picture, ensure that camera function can normal operating, when the camera picture appears unusually, if by sheltering from, when the condition such as colour cast, luminance is unusual, in time and initiatively report abnormal information and corresponding recording video.
For pictures collected by the camera, whether the camera is intentionally shielded or not and picture abnormal conditions such as taking monochromatic and fuzzy pictures caused by camera faults need to be detected, and if the abnormal conditions exist, the normal use of the cat-eye camera can be influenced. In general, when an abnormality occurs, other things may be attached to the camera by a person in a malicious way, or the camera may be illuminated by strong light, so that the picture of the camera cannot be recorded normally.
When the user uses the camera, firstly, the camera equipment can be accessed to the network, specifically, the corresponding equipment management page in the APP of the user terminal enters the setting option, the image abnormity detection function of the camera equipment is opened, the number of the backgrounds and the updating duration of each background can be set at the moment, and the video recording function can be started at the same time. After the picture abnormity detection function and the video recording function are started, the camera equipment enters the video recording work. When the camera equipment detects that the picture is abnormal, the camera equipment can push picture abnormal information to a user terminal APP to remind the user that the picture is abnormal, and therefore the user is guided to check and process the picture. The abnormal picture prompt message can remind the user to check until the user confirms to process the prompt message.
In the embodiment, the change degree of the picture is determined by identifying the background difference among the image frames acquired by the camera equipment; and if the change degree reaches a preset threshold value, performing occlusion analysis on the current image based on pixel value distribution, calculating the similarity of the front image and the back image, and identifying the abnormal condition of the image according to the similarity.
In the detection process, dynamic background updating is carried out in a self-adaptive mode through multi-background modeling, the problem that the background updating is not timely due to the fact that outdoor scenes change and the like can be effectively solved, and the problems of frequent reporting, misinformation, non-reporting and the like can be effectively solved through multi-background modeling. The background times can be defined by users, and the updating time can be defined by users.
In the process of image detection, the human shape is detected by adopting the deep learning model, auxiliary judgment is carried out, and other sensing hardware equipment is not relied on, so that the human body in the image can be accurately and efficiently identified. And the abnormal detection and analysis are carried out on the picture through various forms, the picture change detection is carried out by a frame difference method, and the picture pixel analysis is carried out on the current picture.
In the embodiment, the human shape detection is carried out on each frame of image aiming at the image abnormity caused by artificial shielding, and the image abnormity identification is carried out when a person is detected in the image, so that the condition that the camera is shielded artificially can be identified in time; aiming at the abnormal pictures caused by factors except people, the abnormal pictures are identified by a timing detection mechanism and are identified every 30 minutes, so that the abnormal pictures are prevented from being identified only by detecting people in the images and neglecting the abnormal pictures caused by other factors; and different background frame images are respectively adopted for the two situations, so that the method is more targeted. The abnormal picture recognition and the background frame image updating are both carried out under the condition that no person exists, and the interference of the person is eliminated.
Referring to fig. 8, the apparatus in an embodiment includes an obtaining module 1, a detecting module 2, and a processing module 3, which are described below respectively.
The obtaining module 1 is configured to obtain a current frame image and a current background frame image.
The detection module 2 is configured to detect a picture difference between the current frame image and the background frame image.
The processing module 3 is configured to determine that the current frame image has an image anomaly if the image difference reaches a preset threshold and the ratio of each type of pixels in the current frame image meets a preset condition.
In an embodiment, the processing module 3 is further configured to determine whether a picture difference between the current frame image and the background frame image reaches a preset threshold, if so, obtain an occupation ratio of each type of pixel in the current frame image, determine whether the occupation ratio of each type of pixel in the current frame image meets a preset abnormal condition, and if so, determine that the current frame image has a picture abnormality.
In one embodiment, the detection module 2 determines the picture difference between the current frame image and the background frame image according to the difference between the current frame image and the background frame image.
In an embodiment, the processing module 3 is further configured to perform clustering processing on pixel values of pixels of the current frame image to obtain the number of pixels of each category, and determine the ratio of the number of pixels of each category to the number of all pixels as the ratio of the number of pixels of each category in the current frame image.
In an embodiment, the detection module 2 is further configured to, before detecting the picture difference between the current frame image and the background frame image, detect whether a target object exists in the current frame image to obtain a detection result, and if the detection result indicates that the target object does not exist in the current frame image, detect the picture difference between the current frame image and the background frame image.
Referring to fig. 9, the device for recognizing abnormal image in an embodiment further includes a background updating module 4. The detection module 2 is further configured to, if the detection result indicates that a target object exists in the current frame image, continue to detect whether a target object exists in a next frame image, until the target object does not exist in the frame image, detect a picture difference between the frame image in which the target object does not exist and a background frame image, and the background update module 4 is configured to, if the picture difference is smaller than a preset threshold and an occupation ratio of each type of pixels in the frame image does not satisfy a preset abnormal condition, update the background frame image according to the frame image in a preset time period after the preset time period elapses.
In an embodiment, the background updating module 4 is further configured to update the background frame image according to the frame image in the preset time period every preset time period if the target object is not detected in the frame image in the preset time period.
In an embodiment, the background updating module 4 is further configured to update the background frame image according to the frame image in the preset time period if the target object is not detected in the frame image in the preset time period, the picture difference degree is smaller than the preset threshold, and the ratio of each type of pixel in the frame image in the preset time period does not satisfy the preset abnormal condition.
In an embodiment, the background updating module 4 is further configured to determine, every a first preset time period, a first image from the frame images in the first preset time period and update the background frame image to the first image if the target object is not detected in the frame images in the first preset time period, and/or determine, every a second preset time period, a second image from the frame images in the second preset time period and update the background frame image to the second image if the target object is not detected in the frame images in the second preset time period, where the second preset time period is greater than the first preset time period.
For specific limitations of the picture anomaly identification device of the present application, reference may be made to the above limitations of the picture anomaly identification method, which are not described herein again. All or part of the modules in the screen abnormality recognition device can be realized by software, hardware and a combination thereof. The modules can be embedded in a hardware form or independent from a processor in the computer device, and can also be stored in a memory in the computer device in a software form, so that the processor can call and execute operations corresponding to the modules.
In an embodiment of the present application, an electronic device is further provided, which includes a memory and a processor, where the memory stores a computer program, and the processor implements the steps in the foregoing method embodiments when executing the computer program in the memory.
The internal structure of the electronic device may be as shown in fig. 10. The electronic equipment comprises a processor, a memory, a network interface, a display screen, an input device and a camera device which are connected through a system bus. Wherein the processor of the electronic device is configured to provide computing and control capabilities. The memory of the electronic equipment comprises a nonvolatile storage medium and an internal memory. The nonvolatile storage medium stores an operating system and an electronic program. The internal memory provides an environment for the operation of an operating system and an electronic program in the nonvolatile storage medium. The network interface of the electronic device is used for connecting and communicating with an external terminal through a network. The electronic program is executed by a processor to implement the screen abnormality recognition method of the present application. The display screen of the electronic equipment can be a liquid crystal display screen or an electronic ink display screen, and the input device of the electronic equipment can be a touch layer covered on the display screen, a key, a track ball or a touch pad arranged on the shell of the electronic equipment, an external keyboard, a touch pad or a mouse and the like. The camera device of the electronic device is used for shooting images and sending the images to a display screen for displaying, and the camera device can be a CMOS camera or a CCD camera and the like.
In an embodiment of the present application, a computer-readable storage medium is further provided, which stores a computer program, and the computer program, when executed by a processor, implements the steps in the above-mentioned method embodiments.
There is also provided, in an embodiment of the present application, a computer program product or computer program comprising computer instructions stored in a computer readable storage medium. The computer instructions are read by a processor of the computer device from a computer-readable storage medium, and the computer instructions are executed by the processor to cause the computer device to perform the steps of the above-described method embodiments.
It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by hardware instructions of a computer program, which can be stored in a non-volatile computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. Any reference to memory, storage, database, or other medium used in the embodiments provided herein may include non-volatile and/or volatile memory, among others. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically Programmable ROM (EPROM), electrically Erasable Programmable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms such as Static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double Data Rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous Link DRAM (SLDRAM), rambus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).
The technical features of the above embodiments can be arbitrarily combined, and for the sake of brevity, all possible combinations of the technical features in the above embodiments are not described, but should be considered as the scope of the present specification as long as there is no contradiction between the combinations of the technical features.
The above-mentioned embodiments only express several embodiments of the present application, and the description thereof is more specific and detailed, but not construed as limiting the scope of the invention. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the concept of the present application, which falls within the scope of protection of the present application. Therefore, the protection scope of the present patent shall be subject to the appended claims.
Claims (12)
1. A method for recognizing screen abnormality is characterized by comprising the following steps:
acquiring a current frame image and a current background frame image;
detecting the picture difference degree between the current frame image and the background frame image;
and if the picture difference reaches a preset threshold value and the proportion of various pixels in the current frame image meets a preset abnormal condition, determining that the current frame image has picture abnormality.
2. The method of claim 1, wherein the determining that the current frame image has the picture abnormality if the picture difference reaches a preset threshold and the ratio of the pixels in the current frame image satisfies a preset abnormality condition comprises:
if the picture difference reaches a preset threshold value, acquiring the occupation ratio of various pixels in the current frame image;
and if the proportion of each pixel in the current frame image meets a preset abnormal condition, determining that the current frame image has abnormal pictures.
3. The method of claim 2, wherein the detecting the picture difference between the current frame image and the background frame image comprises:
and determining the picture difference degree between the current frame image and the background frame image according to the difference value between the current frame image and the background frame image.
4. The method of claim 1, wherein the method further comprises:
clustering the pixel values of the pixel points of the current frame image to obtain the number of the pixel points of each category;
and determining the proportion of the number of the pixel points of each category to the number of all the pixel points as the proportion of each pixel in the current frame image.
5. The method of claim 1, wherein before detecting the picture difference between the current frame image and the background frame image, further comprising:
detecting whether a target object exists in the current frame image to obtain a detection result;
the detecting the picture difference between the current frame image and the background frame image includes:
and if the detection result indicates that the target object does not exist in the current frame image, detecting the picture difference degree between the current frame image and the background frame image.
6. The method of claim 5, wherein the method further comprises:
if the detection result indicates that the target object exists in the current frame image, whether the target object exists in the next frame image or not is continuously detected until the target object does not exist in the frame image, and the picture difference degree between the frame image without the target object and the background frame image is detected;
and if the picture difference degree is smaller than the preset threshold value and the occupation ratio of various pixels in the frame image does not meet the preset abnormal condition, updating the background frame image according to the frame image in the preset time period after the preset time period.
7. The method of any one of claims 1 to 5, further comprising:
and updating the background frame image according to the frame image in the preset time period every other preset time period if the target object is not detected in the frame image in the preset time period.
8. The method of claim 7, wherein if no target object is detected in the frame image within the preset time period, updating the background frame image according to the frame image within the preset time period comprises:
and if the target object is not detected in the frame image within the preset time period, the picture difference degree is smaller than a preset threshold value, and the occupation ratio of various pixels in the frame image within the preset time period does not meet the preset abnormal condition, updating the background frame image according to the frame image within the preset time period.
9. The method of claim 7, wherein, every preset time period, if no target object is detected in the frame image within the preset time period, updating the background frame image according to the frame image within the preset time period comprises:
every other first preset time period, if no target object is detected in the frame images in the first preset time period, determining a first image from the frame images in the first preset time period, and updating the background frame image into the first image;
and/or determining a second image from the frame images in a second preset time period and updating the background frame image into the second image if the target object is not detected in the frame images in the second preset time period every second preset time period; the second preset time period is greater than the first preset time period.
10. A screen abnormality recognition apparatus comprising:
the acquisition module is used for acquiring a current frame image and a current background frame image;
the detection module is used for detecting the picture difference degree between the current frame image and the background frame image;
and the processing module is used for determining that the current frame image has abnormal pictures if the picture difference reaches a preset threshold value and the occupation ratio of various pixels in the current frame image meets a preset condition.
11. An electronic device, comprising a memory storing a computer program and a processor implementing the steps of the method of any of claims 1 to 9 when the processor executes the computer program.
12. A computer-readable storage medium, characterized in that the medium has stored thereon a program which is executable by a processor to implement the method according to any one of claims 1-9.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210928102.4A CN115346060A (en) | 2022-08-03 | 2022-08-03 | Picture abnormity identification method and device, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210928102.4A CN115346060A (en) | 2022-08-03 | 2022-08-03 | Picture abnormity identification method and device, electronic equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115346060A true CN115346060A (en) | 2022-11-15 |
Family
ID=83950344
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210928102.4A Pending CN115346060A (en) | 2022-08-03 | 2022-08-03 | Picture abnormity identification method and device, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115346060A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116033222A (en) * | 2022-12-22 | 2023-04-28 | 深圳创维－Rgb电子有限公司 | Video playing method, device, display equipment and computer readable storage medium |
| CN116563429A (en) * | 2023-06-14 | 2023-08-08 | 厦门华厦学院 | Animation design processing supervision system and method based on cloud computing |
| CN117745664A (en) * | 2023-12-15 | 2024-03-22 | 苏州智华汽车电子有限公司 | Image dynamic detection method, device, equipment and storage medium |
-
2022
- 2022-08-03 CN CN202210928102.4A patent/CN115346060A/en active Pending
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116033222A (en) * | 2022-12-22 | 2023-04-28 | 深圳创维－Rgb电子有限公司 | Video playing method, device, display equipment and computer readable storage medium |
| CN116563429A (en) * | 2023-06-14 | 2023-08-08 | 厦门华厦学院 | Animation design processing supervision system and method based on cloud computing |
| CN116563429B (en) * | 2023-06-14 | 2023-10-27 | 厦门华厦学院 | Animation design processing supervision system and method based on cloud computing |
| CN117745664A (en) * | 2023-12-15 | 2024-03-22 | 苏州智华汽车电子有限公司 | Image dynamic detection method, device, equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN115346060A (en) | Picture abnormity identification method and device, electronic equipment and storage medium | |
| KR101789690B1 (en) | System and method for providing security service based on deep learning | |
| KR101825045B1 (en) | Alarm method and device | |
| CN110044486B (en) | Method, device and equipment for avoiding repeated alarm of human body inspection and quarantine system | |
| US20190333241A1 (en) | People flow analysis apparatus, people flow analysis system, people flow analysis method, and non-transitory computer readable medium | |
| KR102195706B1 (en) | Method and Apparatus for Detecting Intruder | |
| US20190304272A1 (en) | Video detection and alarm method and apparatus | |
| CN110390229B (en) | Face picture screening method and device, electronic equipment and storage medium | |
| CN110059666B (en) | Attention detection method and device | |
| CN111899470B (en) | Human body falling detection method, device, equipment and storage medium | |
| US20220122360A1 (en) | Identification of suspicious individuals during night in public areas using a video brightening network system | |
| Juhana et al. | Design and implementation of smart home surveillance system | |
| CN111860152A (en) | Method, system, equipment and computer equipment for detecting personnel state | |
| CN107122743A (en) | Security-protecting and monitoring method, device and electronic equipment | |
| CN108230607A (en) | A kind of image fire detection method based on regional characteristics analysis | |
| CN111260869A (en) | Method and device for extracting video frames in monitoring video and computer equipment | |
| KR20090044957A (en) | Theft and left baggage survellance system and meothod thereof | |
| CN109727417B (en) | Method and controller for controlling a video processing unit to facilitate detection of a newcomer | |
| CN114898443A (en) | Face data acquisition method and device | |
| KR20210013865A (en) | Abnormal behavior detection system and method using generative adversarial network | |
| CN110505438B (en) | Queuing data acquisition method and camera | |
| Yuwono et al. | Design and implementation of human detection feature on surveillance embedded IP camera | |
| KR20200059643A (en) | ATM security system based on image analyses and the method thereof | |
| CN111917967A (en) | Door monitoring system and control method thereof | |
| JP2009032116A (en) | Face authentication apparatus, face authentication method, and access management apparatus |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |