CN112634319A - Video background and foreground separation method and system, electronic device and storage medium - Google Patents
Video background and foreground separation method and system, electronic device and storage medium Download PDFInfo
- Publication number
- CN112634319A CN112634319A CN202011580813.4A CN202011580813A CN112634319A CN 112634319 A CN112634319 A CN 112634319A CN 202011580813 A CN202011580813 A CN 202011580813A CN 112634319 A CN112634319 A CN 112634319A
- Authority
- CN
- China
- Prior art keywords
- background
- foreground
- current frame
- points
- background model
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
- G06T7/215—Motion-based segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/194—Segmentation; Edge detection involving foreground-background segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
- G06T7/254—Analysis of motion involving subtraction of images
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10016—Video; Image sequence
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20212—Image combination
- G06T2207/20224—Image subtraction
Landscapes
- Engineering & Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Multimedia (AREA)
- Compression Or Coding Systems Of Tv Signals (AREA)
Abstract
The invention relates to an artificial intelligence technology, and discloses a method for separating a video background from a video foreground, which comprises the following steps: initializing a background model by adopting the image of the first N frames of the video stream, wherein eight neighborhoods of pixel points are adopted in each frame to select sample points; performing foreground detection on the current frame image according to the background model, and judging foreground points and background points; extracting a foreground target and a background image in the current frame image according to a judgment result; and updating the background model according to the judged background point. The invention also provides a video background and foreground separation system, an electronic device and a computer readable storage medium. The method, the system, the electronic device and the computer readable storage medium for separating the background and the foreground of the video can separate the background and the foreground target from the video stream in real time and extract a background image.
Description
Technical Field
The invention relates to the technical field of artificial intelligence, in particular to a method, a system, an electronic device and a computer readable storage medium for separating a video background from a video foreground.
Background
In an application scenario of video data such as surveillance video, it is often necessary to separate background (static) and foreground objects (dynamic) from a video stream. Background and foreground separation methods commonly used in the industry at present are:
the separation method based on the optical flow method can well adapt to the change of a dynamic scene, but has the defect that the method is difficult to use in practice due to high algorithm complexity.
The separation method based on the interframe difference method is to subtract the pixel values of two adjacent frames or two images separated by several frames in a video stream, and threshold the subtracted images to extract the motion area in the images. The method can well cope with the influence of light mutation, but has the defects that the cavity phenomenon is easy to generate and the detection effect is influenced.
A separation method based on background modeling is provided, wherein a Gaussian mixture model and visual background extraction are mainstream methods. The Gaussian mixture model utilizes K Gaussian distributions to construct a background model in a time domain. The visual background extraction is based on the clustering idea and comprises background modeling, foreground detection and background updating. The method has good robustness, small calculation amount of the algorithm and good real-time performance, but the first frame is adopted to initialize the background model, so that ghost is easily generated, and the detection result is influenced.
In summary, how to implement the real-time background and foreground separation of a video stream simply and accurately without generating ghost and void phenomena and effectively coping with sudden changes in illumination becomes a technical problem to be solved urgently.
Disclosure of Invention
In view of the above, the present invention provides a method, a system, an electronic device and a computer-readable storage medium for separating a background and a foreground of a video, so as to solve at least one of the above technical problems.
Firstly, in order to achieve the above object, the present invention provides a method for separating a background from a foreground of a video, the method comprising the steps of:
initializing a background model by adopting the image of the first N frames of the video stream, wherein eight neighborhoods of pixel points are adopted in each frame to select sample points;
performing foreground detection on the current frame image according to the background model, and judging foreground points and background points;
extracting a foreground target and a background image in the current frame image according to a judgment result; and
and updating the background model according to the judged background point.
Optionally, the initializing the background model by using the first N frames of images of the video stream includes:
in each frame of the previous N frames of images, M sample points randomly selected by equal probability in eight neighborhoods of any pixel points are adopted to form a sample set;
and constructing the initialized background model according to M x N sample points of the previous N frames of images, wherein M and N are positive integers.
Optionally, the performing foreground detection on the current frame image according to the background model, and determining foreground points and background points includes:
calculating a plurality of Euclidean distances between any one first pixel point in the current frame image and a corresponding sample point in the background model;
counting the number of Euclidean distances corresponding to the first pixel points, which are smaller than a matching threshold value R;
if the number is larger than or equal to a preset first threshold value, judging the first pixel point as a background point;
and if the number is smaller than the first threshold value, judging the first pixel point as a foreground point.
Optionally, before performing foreground detection on the current frame image according to the background model, the method further includes:
and determining the matching threshold value R by adopting a self-adaptive background disturbance suppression algorithm.
Optionally, the determining the matching threshold R by using an adaptive background disturbance suppression algorithm includes:
calculating the mean value of the difference sum of the pixel values of the pixel points in the current frame image and the pixel values of the corresponding sample points in the background model;
when the calculation result is larger than a preset value, increasing the matching threshold value R;
and when the calculation result is less than or equal to the preset value, keeping the matching threshold value R unchanged.
Optionally, before performing foreground detection on the current frame image according to the background model, the method further includes:
judging whether the current frame image has illumination mutation or not;
when the sudden change of illumination occurs, the background model is reinitialized according to the previous N frames of images from the current frame.
Optionally, the determining whether the current frame image has an illumination mutation includes:
calculating the difference between the pixel value of each pixel point in the current frame image and the pixel value of the corresponding pixel point in the previous frame;
when the difference is larger than a preset second threshold value, determining the corresponding pixel point as a pixel point with sudden illumination change;
counting the number of the pixels with the sudden illumination change determined in the current frame image;
calculating the ratio of the number of the pixels with the illumination mutation to the total number of the pixels in the current frame image;
when the ratio is larger than a preset third threshold value, determining that the current frame image has illumination mutation;
and when the ratio is smaller than or equal to the third threshold, determining that the current frame image has no illumination mutation.
In addition, to achieve the above object, the present invention further provides a system for separating a background from a foreground of a video, the system comprising:
the construction module is used for initializing a background model by adopting the images of the previous N frames of the video stream, wherein eight neighborhoods of pixel points are adopted in each frame to select sample points;
the detection module is used for carrying out foreground detection on the current frame image according to the background model and judging foreground points and background points;
the extraction module is used for extracting a foreground target and a background image in the current frame image according to the judgment result;
and the updating module is used for updating the background model according to the judged background point.
Further, to achieve the above object, the present invention further provides an electronic device, which includes a memory and a processor, where the memory stores a video background and foreground separation program that can be executed on the processor, and the video background and foreground separation program implements the steps of the video background and foreground separation method when executed by the processor.
Further, to achieve the above object, the present invention also provides a computer readable storage medium storing a video background and foreground separation program, which is executable by at least one processor to cause the at least one processor to execute the steps of the video background and foreground separation method as described above.
Compared with the prior art, the method, the system, the electronic device and the computer-readable storage medium for separating the background and the foreground of the video can separate the background (static) and the foreground target (dynamic) from the video stream in real time and extract the background image. In addition, the invention adopts continuous multiframes to establish the background model, thereby avoiding the limitation that the prior scheme only uses the first frame to initialize the background model and is not easy to eliminate the ghost, leading the background model to have more real and reliable background information and accelerating the speed of eliminating the ghost.
Drawings
FIG. 1 is a diagram of an alternative hardware architecture of the electronic device of the present invention;
FIG. 2 is a block diagram of a first embodiment of a video background and foreground separation system according to the present invention;
FIG. 3 is a block diagram of a second embodiment of a video background and foreground separation system according to the present invention;
FIG. 4 is a block diagram of a third embodiment of a video background and foreground separation system according to the present invention;
FIG. 5 is a flowchart illustrating a first embodiment of a method for separating background and foreground from a video according to the present invention;
FIG. 6 is a flowchart illustrating a second embodiment of a method for separating background and foreground from a video according to the present invention;
FIG. 7 is a flowchart illustrating a third embodiment of a method for separating a background from a foreground in a video according to the present invention;
the implementation, functional features and advantages of the objects of the present invention will be further explained with reference to the accompanying drawings.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
It should be noted that the description relating to "first", "second", etc. in the present invention is for descriptive purposes only and is not to be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one such feature. In addition, technical solutions between various embodiments may be combined with each other, but must be realized by a person skilled in the art, and when the technical solutions are contradictory or cannot be realized, such a combination should not be considered to exist, and is not within the protection scope of the present invention.
Fig. 1 is a schematic diagram of an alternative hardware architecture of the electronic device 2 according to the present invention.
In this embodiment, the electronic device 2 may include, but is not limited to, a memory 11, a processor 12, and a network interface 13, which may be communicatively connected to each other through a system bus. It is noted that fig. 1 only shows the electronic device 2 with components 11-13, but it is to be understood that not all of the shown components are required to be implemented, and that more or fewer components may be implemented instead.
The electronic device 2 may be a server or other electronic equipment with computing capability. The server may be a rack server, a blade server, a tower server, a cabinet server, or other computing devices, may be an independent server, or may be a server cluster composed of a plurality of servers.
The memory 11 includes at least one type of readable storage medium including a flash memory, a hard disk, a multimedia card, a card type memory (e.g., SD or DX memory, etc.), a Random Access Memory (RAM), a Static Random Access Memory (SRAM), a Read Only Memory (ROM), an Electrically Erasable Programmable Read Only Memory (EEPROM), a Programmable Read Only Memory (PROM), a magnetic memory, a magnetic disk, an optical disk, etc. In some embodiments, the storage 11 may be an internal storage unit of the electronic device 2, such as a hard disk or a memory of the electronic device 2. In other embodiments, the memory 11 may also be an external storage device of the electronic apparatus 2, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), or the like, provided on the electronic apparatus 2. Of course, the memory 11 may also comprise both an internal memory unit of the electronic apparatus 2 and an external memory device thereof. In this embodiment, the memory 11 is generally used for storing an operating system installed in the electronic device 2 and various types of application software, such as program codes of the video background and foreground separation system 200. Furthermore, the memory 11 may also be used to temporarily store various types of data that have been output or are to be output.
The processor 12 may be a Central Processing Unit (CPU), controller, microcontroller, microprocessor, or other data Processing chip in some embodiments. The processor 12 is typically used to control the overall operation of the electronic device 2. In this embodiment, the processor 12 is configured to execute the program code stored in the memory 11 or process data, such as executing the video background and foreground separation system 200.
The network interface 13 may comprise a wireless network interface or a wired network interface, and the network interface 13 is generally used for establishing a communication connection between the electronic apparatus 2 and other electronic devices.
The hardware structure and functions of the related devices of the present invention have been described in detail so far. Various embodiments of the present invention will be presented based on the above description.
First, the present invention provides a video background and foreground separation system 200.
Referring to fig. 2, a block diagram of a first embodiment of a video background and foreground separation system 200 according to the present invention is shown.
In this embodiment, the video background and foreground separation system 200 includes a series of computer program instructions stored in the memory 11, and when the computer program instructions are executed by the processor 12, the video background and foreground separation operation according to the embodiments of the present invention can be implemented. In some embodiments, the video background and foreground separation system 200 may be divided into one or more modules based on the particular operations implemented by the portions of the computer program instructions. For example, in fig. 2, the video background and foreground separation system 200 may be segmented into a construction module 201, a detection module 202, an extraction module 203, and an update module 204. Wherein:
the building module 201 is configured to initialize a background model by using the first N frames of the video stream.
In order to avoid the defects of ghost and the like caused by only adopting the first frame to initialize the background model, the embodiment adopts the image of the first N frames of the video stream to initialize the background model.
For any pixel point x, the pixel value at the ith frame is Vi(x) In that respect In the ith frame of the video stream, M sample points randomly selected by the equal probability in x eight neighborhoods of pixel points are adopted to form a sample set Bi(x) In that respect Namely:
from the first frame to the nth frame, a total of M × N sample points are used to construct the initialized background model.
Wherein M and N are both positive integers. For example, M and N may take on values of 20 and 6, respectively.
The background model constructed by the embodiment has both space domain information and time domain information, the space domain information enables the background model to have randomness, and the time domain information enables the background model to have diversity.
The detection module 202 is configured to perform foreground detection on the current frame image according to the background model, and determine foreground points and background points.
After the initialization of the background model is completed, foreground detection can be performed on the current frame. In this embodiment, the foreground detection specifically includes: calculating the Euclidean distance between any pixel point in the current frame and the corresponding pixel point in the background model, counting the number of Euclidean distances smaller than the matching threshold value R, and if the number is larger than or equal to the experience threshold value TminAnd judging the pixel points as background points, otherwise, as foreground points. Wherein the threshold value TminThe value may be 2.
The extracting module 203 is configured to extract a foreground object and a background image according to the determination result.
After foreground detection is performed to determine the foreground point and the background point of the current frame, the foreground object and the background image of the current frame can be extracted by any available method, which is not described herein again.
The updating module 204 is configured to update the background model according to the determined background point.
And updating the background model by replacing the sample values in the background model with the pixel points judged as background points. In background model updating, the pixel points judged as background points have
Is substituted for the background modelAny one of the sample values in (b) is,
is a time subsampling factor, also having
Randomly updates the background model of the neighborhood pixels. Wherein
The value may be 16. And after the background model is updated, the next current frame adopts the new background model to carry out foreground detection.
The video background and foreground separation system provided by this embodiment can separate a background (static) and a foreground object (dynamic) from a video stream in real time and extract a background image. Moreover, the system adopts continuous multiframes to establish the background model, avoids the limitation that the prior scheme only uses the first frame to initialize the background model and is not easy to eliminate the ghost, ensures that the background model has more real and reliable background information, and accelerates the speed of eliminating the ghost.
Referring to fig. 3, a block diagram of a second embodiment of a video background and foreground separation system 200 according to the present invention is shown. In this embodiment, the video background and foreground separation system 200 further includes an adjusting module 205 in addition to the constructing module 201, the detecting module 202, the extracting module 203, and the updating module 204 in the first embodiment.
The adjusting module 205 is configured to determine the matching threshold R by using a self-adaptive background disturbance suppression algorithm.
If the pixel point x matches the background model in an Euclidean space with a fixed matching threshold value R as a radius, the whole-course invariable threshold value has no influence on a static background, but a false background point can be detected as a foreground point under the dynamic background disturbance. Therefore, an adaptive background disturbance suppression algorithm is required to determine the matching threshold R to overcome this drawback. That is, the matching threshold R is resized according to the environment, enhancing the adaptive background change capability of the algorithm.
In this embodiment, the index μ (x) for measuring the influence of background disturbance is:
namely, calculating the average value of the difference sum of the pixel value of the pixel point x of the current frame and the pixel value of the background sample in the background model. The larger the mu (x), the more obvious the disturbance of the background is, and the matching threshold R (x) should be increased to suppress the influence of the background disturbance; the smaller μ (x) indicates that the background tends to be flat. The judgment strategy of R (x) is as follows:
wherein, alpha is a self-increasing adaptive parameter, and the value can be 0.05; delta is a scale factor, and the value can be 5; ri(x) And Ri -1(x) Respectively representing the matching threshold values of the pixel point x in the ith frame and the (i-1) th frame. And when the background disturbance degree is large, increasing the background on the basis of the matching threshold of the pixel point of the previous frame, and keeping the original matching threshold when the background tends to be stable.
In addition, consider that R is always greater when the degree of background perturbations is greateri(x) Will increase infinitely, so R needs to be giveni(x) Setting an upper limit Rmax(e.g. R)max＝30)。
After the adjusting module 205 determines the matching threshold R of the current frame, the detecting module 202 is triggered to perform foreground detection on the current frame image according to the background model and the determined matching threshold R, and determine foreground points and background points.
The video background and foreground separation system provided by this embodiment can separate a background (static) and a foreground object (dynamic) from a video stream in real time and extract a background image. In addition, the system adopts a self-adaptive background disturbance suppression algorithm to determine the matching threshold value in the foreground detection, so that the self-adaptive background change capability of the algorithm is enhanced.
Referring to fig. 4, a block diagram of a third embodiment of a video background and foreground separation system 200 according to the present invention is shown. In this embodiment, the video background and foreground separation system 200 further includes a determining module 206 based on the first embodiment or the second embodiment.
The judging module 206 is configured to judge whether a current frame has a sudden illumination change.
In this embodiment, the light mutation is determined according to the following criteria: if the current frame has illumination mutation, the difference between the pixel value of the pixel point and the pixel value of the previous frame is larger.
The difference between the pixel values of the current frame and the previous frame of each pixel point is as follows: f (x) Vi(x)-Vi-1(x) In that respect And (4) setting an empirical threshold eta, and if f (x) is greater than the threshold eta, summarizing the pixel point into a pixel point with sudden illumination change. Then counting the number C of the pixel points with the illumination mutation1And calculating the ratio of the current pixel point to the total number C of the current frame pixel points. When the ratio is greater than the threshold value phi, the illumination mutation of the current frame is indicated. When the ratio is less than or equal to the threshold value phi, the illumination mutation does not occur in the current frame. Wherein eta and phi can be respectively 5 and 0.7.
When a sudden change in illumination occurs, the construction module 201 is triggered to reinitialize the background model starting from the current frame.
Specifically, the background model is initialized again quickly in the background model initialization manner in the first embodiment described above based on the previous N frames from the current frame. That is, M × N sample points are selected from the current frame and the next N-1 frame, and the initialized background model is reconstructed. The adjustment module 205 and the detection module 202 are then triggered for subsequent processing based on the new background model.
When the sudden change of illumination does not occur, the adjustment module 205 and the detection module 202 are directly triggered to perform subsequent processing without reinitializing the background model.
The video background and foreground separation system provided by this embodiment can separate a background (static) and a foreground object (dynamic) from a video stream in real time and extract a background image. In addition, the system adopts an illumination mutation suppression method to effectively and quickly promote the updating of the background model, and can well cope with the influence of the illumination mutation.
In addition, the invention also provides a method for separating the background and the foreground of the video.
Fig. 5 is a schematic flow chart of a video background and foreground separation method according to a first embodiment of the present invention. In this embodiment, the execution order of the steps in the flowchart shown in fig. 5 may be changed and some steps may be omitted according to different requirements. The method comprises the following steps:
and step S400, initializing a background model by adopting the first N frames of the video stream.
In order to avoid the defects of ghost and the like caused by only adopting the first frame to initialize the background model, the embodiment adopts the image of the first N frames of the video stream to initialize the background model.
For any pixel point x, the pixel value at the ith frame is Vi(x) In that respect In the ith frame of the video stream, M sample points randomly selected by the equal probability in x eight neighborhoods of pixel points are adopted to form a sample set Bi(x) In that respect Namely:
from the first frame to the nth frame, a total of M × N sample points are used to construct the initialized background model.
Wherein M and N are both positive integers. For example, M and N may take on values of 20 and 6, respectively.
The background model constructed by the embodiment has both space domain information and time domain information, the space domain information enables the background model to have randomness, and the time domain information enables the background model to have diversity.
And S402, carrying out foreground detection on the current frame image according to the background model, and judging foreground points and background points.
After the initialization of the background model is completed, foreground detection can be performed on the current frame. In this embodiment, the foreground detection specifically includes: calculating the Euclidean distance between any pixel point in the current frame and the corresponding pixel point in the background model, counting the number of Euclidean distances smaller than a matching threshold value R, and if the number is larger than or equal to an empirical threshold value TminAnd judging the pixel points as background points, otherwise, as foreground points. Wherein the threshold value TminThe value may be 2.
And S404, extracting a foreground target and a background image according to the judgment result.
After foreground detection is performed to determine the foreground point and the background point of the current frame, the foreground object and the background image in the current frame image can be extracted by any available method, which is not described herein again.
Step S406, updating the background model according to the determined background point.
And updating the background model by replacing the sample values in the background model with the pixel points judged as background points. In background model updating, the pixel points judged as background points have
Replaces any one sample value in the background model,
is a time subsampling factor, also having
Randomly updates the background model of the neighborhood pixels. Wherein
The value may be 16. And after the background model is updated, the next current frame adopts the new background model to carry out foreground detection.
The method for separating the background and the foreground of the video provided by the embodiment can be used for separating the background (static) and the foreground object (dynamic) from the video stream in real time and extracting the background image. In addition, the method adopts continuous multiframes to establish the background model, avoids the limitation that the prior scheme only uses the first frame to initialize the background model and is not easy to eliminate the ghost, ensures that the background model has more real and reliable background information, and accelerates the speed of eliminating the ghost.
Fig. 6 is a schematic flow chart of a method for separating background from foreground in a video according to a second embodiment of the present invention. In this embodiment, the method for separating the background and the foreground of the video further includes step S502 based on the first embodiment. The execution order of the steps in the flowchart shown in fig. 6 may be changed and some steps may be omitted according to different requirements.
The method comprises the following steps:
and step S500, initializing a background model by adopting the first N frames of the video stream.
In order to avoid the defects of ghost and the like caused by only adopting the first frame to initialize the background model, the embodiment adopts the image of the first N frames of the video stream to initialize the background model.
For any pixel point x, the pixel value at the ith frame is Vi(x) In that respect In the ith frame of the video stream, M sample points randomly selected by the equal probability in x eight neighborhoods of pixel points are adopted to form a sample set Bi(x) In that respect Namely:
from the first frame to the nth frame, a total of M × N sample points are used to construct the initialized background model.
Wherein M and N are both positive integers. For example, M and N may take on values of 20 and 6, respectively.
The background model constructed by the embodiment has both space domain information and time domain information, the space domain information enables the background model to have randomness, and the time domain information enables the background model to have diversity.
Step S502, a matching threshold value R is determined by adopting a self-adaptive background disturbance suppression algorithm.
If the pixel point x matches the background model in an Euclidean space with a fixed matching threshold value R as a radius, the whole-course invariable threshold value has no influence on a static background, but a false background point can be detected as a foreground point under the dynamic background disturbance. Therefore, an adaptive background disturbance suppression algorithm is required to determine the matching threshold R to overcome this drawback. That is, the matching threshold R is resized according to the environment, enhancing the adaptive background change capability of the algorithm.
In this embodiment, the index μ (x) for measuring the influence of background disturbance is:
namely, calculating the average value of the difference sum of the pixel value of the pixel point x of the current frame and the pixel value of the background sample in the background model. The larger the mu (x), the more obvious the disturbance of the background is, and the matching threshold R (x) should be increased to suppress the influence of the background disturbance; the smaller μ (x) indicates that the background tends to be flat. The judgment strategy of R (x) is as follows:
wherein, alpha is a self-increasing adaptive parameter, and the value can be 0.05; delta is a scale factor, and the value can be 5; ri(x) And Ri -1(x) Respectively representing the matching threshold values of the pixel point x in the ith frame and the (i-1) th frame. And when the background disturbance degree is large, increasing the background on the basis of the matching threshold of the pixel point of the previous frame, and keeping the original matching threshold when the background tends to be stable.
In addition, consider that R is always greater when the degree of background perturbations is greateri(x) Will increase infinitely, so R needs to be giveni(x) Setting an upper limit Rmax(e.g. R)max＝30)。
And step S504, performing foreground detection on the current frame image according to the background model, and judging foreground points and background points.
After the initialization of the background model is completed, foreground detection can be performed on the current frame. In this embodiment, the foreground detection specifically includes: calculating the Euclidean distance between any pixel point in the current frame and the corresponding pixel point in the background model, counting the number of Euclidean distances smaller than the matching threshold value R, and if the number is larger than or equal to the experience threshold value TminAnd judging the pixel points as background points, otherwise, as foreground points. Wherein the threshold value TminThe value may be 2.
And step S506, extracting a foreground target and a background image according to the judgment result.
After foreground detection is performed to determine the foreground point and the background point of the current frame, the foreground object and the background image of the current frame can be extracted by any available method, which is not described herein again.
Step S508, updating the background model according to the determined background point.
And updating the background model by replacing the sample values in the background model with the pixel points judged as background points. In background model updating, the pixel points judged as background points have
Replaces any one sample value in the background model,
is a time subsampling factor, also having
Randomly updates the background model of the neighborhood pixels. Wherein
The value may be 16. And after the background model is updated, the next current frame adopts the new background model to carry out foreground detection.
The method for separating the background and the foreground of the video provided by the embodiment can be used for separating the background (static) and the foreground object (dynamic) from the video stream in real time and extracting the background image. In addition, the method adopts a self-adaptive background disturbance suppression algorithm to determine the matching threshold value in the foreground detection, and the self-adaptive background change capability of the algorithm is enhanced.
Fig. 7 is a schematic flow chart of a video background and foreground separation method according to a third embodiment of the present invention. In this embodiment, the method for separating the background and the foreground of the video further includes steps S602 to S604 based on the first embodiment or the second embodiment. The execution order of the steps in the flowchart shown in fig. 7 may be changed and some steps may be omitted according to different requirements.
The method comprises the following steps:
and step S600, initializing a background model by adopting the first N frames of the video stream.
In order to avoid the defects of ghost and the like caused by only adopting the first frame to initialize the background model, the embodiment adopts the image of the first N frames of the video stream to initialize the background model.
For any pixel point x, the pixel value at the ith frame is Vi(x) In that respect In the ith frame of the video stream, M sample points randomly selected by the equal probability in x eight neighborhoods of pixel points are adopted to form a sample set Bi(x) In that respect Namely:
from the first frame to the nth frame, a total of M × N sample points are used to construct the initialized background model.
Wherein M and N are both positive integers. For example, M and N may take on values of 20 and 6, respectively.
The background model constructed by the embodiment has both space domain information and time domain information, the space domain information enables the background model to have randomness, and the time domain information enables the background model to have diversity.
Step S602, determining whether the current frame has an illumination mutation. When the light mutation occurs, executing step S604; when the light mutation does not occur, step S606 is performed.
In this embodiment, the light mutation is determined according to the following criteria: if the current frame has illumination mutation, the difference between the pixel value of the pixel point and the pixel value of the previous frame is larger.
The difference between the pixel values of the current frame and the previous frame of each pixel point is as follows: f (x) Vi(x)-Vi-1(x) In that respect And (4) setting an empirical threshold eta, and if f (x) is greater than the threshold eta, summarizing the pixel point into a pixel point with sudden illumination change. Then counting the number C of the pixel points with the illumination mutation1And calculating the ratio of the current pixel point to the total number C of the current frame pixel points. When the ratio is greater than the threshold value phi, the illumination mutation of the current frame is indicated. When the ratio is less than or equal to the threshold value phi, the illumination mutation does not occur in the current frame. Wherein eta and phi can be respectively 5 and 0.7.
Step S604, the background model is initialized again from the current frame.
Specifically, the background model is initialized again quickly in the background model initialization manner of step S600 described above based on the previous N frames from the current frame. That is, M × N sample points are selected from the current frame and the next N-1 frame, and the initialized background model is reconstructed. Then, step S606 is executed, and the subsequent steps are performed based on the new background model.
Step S606, a matching threshold value R is determined by adopting a self-adaptive background disturbance suppression algorithm.
If the pixel point x matches the background model in an Euclidean space with a fixed matching threshold value R as a radius, the whole-course invariable threshold value has no influence on a static background, but a false background point can be detected as a foreground point under the dynamic background disturbance. Therefore, an adaptive background disturbance suppression algorithm is required to determine the matching threshold R to overcome this drawback. That is, the matching threshold R is resized according to the environment, enhancing the adaptive background change capability of the algorithm.
In this embodiment, the index μ (x) for measuring the influence of background disturbance is:
namely, calculating the average value of the difference sum of the pixel value of the pixel point x of the current frame and the pixel value of the background sample in the background model. The larger the mu (x), the more obvious the disturbance of the background is, and the matching threshold R (x) should be increased to suppress the influence of the background disturbance; the smaller μ (x) indicates that the background tends to be flat. The judgment strategy of R (x) is as follows:
wherein, alpha is a self-increasing adaptive parameter, and the value can be 0.05; delta is a scale factor, and the value can be 5; ri(x) And Ri -1(x) Respectively representing the matching threshold values of the pixel point x in the ith frame and the (i-1) th frame. And when the background disturbance degree is large, increasing the background on the basis of the matching threshold of the pixel point of the previous frame, and keeping the original matching threshold when the background tends to be stable.
In addition, consider that R is always greater when the degree of background perturbations is greateri(x) Will increase infinitely, so R needs to be giveni(x) Setting an upper limit Rmax(e.g. R)max＝30)。
Step S608, performing foreground detection on the current frame image according to the background model, and determining foreground points and background points.
After the initialization of the background model is completed, foreground detection can be performed on the current frame. In this embodiment, the foreground detection specifically includes: calculating any pixel point in the current frame and the background modelThe Euclidean distance between corresponding pixel points is counted, the number of Euclidean distances smaller than the matching threshold value R is counted, and if the number is larger than or equal to the experience threshold value TminAnd judging the pixel points as background points, otherwise, as foreground points. Wherein the threshold value TminThe value may be 2.
And step S610, extracting a foreground target and a background image according to the judgment result.
After foreground detection is performed to determine the foreground point and the background point of the current frame, the foreground object and the background image of the current frame can be extracted by any available method, which is not described herein again.
Step S612, updating the background model according to the determined background point.
And updating the background model by replacing the sample values in the background model with the pixel points judged as background points. In background model updating, the pixel points judged as background points have
Replaces any one sample value in the background model,
is a time subsampling factor, also having
Randomly updates the background model of the neighborhood pixels. Wherein
The value may be 16. And after the background model is updated, the next current frame adopts the new background model to carry out foreground detection.
The method for separating the background and the foreground of the video provided by the embodiment can be used for separating the background (static) and the foreground object (dynamic) from the video stream in real time and extracting the background image. In addition, the method adopts an illumination mutation suppression method to effectively and quickly promote the updating of the background model, and can well cope with the influence of the illumination mutation.
The present invention further provides another embodiment, which is to provide a computer readable storage medium, wherein the computer readable storage medium stores a video background and foreground separation program, and the video background and foreground separation program can be executed by at least one processor, so as to make the at least one processor execute the steps of the video background and foreground separation method as described above.
The above-mentioned serial numbers of the embodiments of the present invention are merely for description and do not represent the merits of the embodiments.
Through the above description of the embodiments, those skilled in the art will clearly understand that the method of the above embodiments can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware, but in many cases, the former is a better implementation manner. Based on such understanding, the technical solutions of the present invention may be embodied in the form of a software product, which is stored in a storage medium (such as ROM/RAM, magnetic disk, optical disk) and includes instructions for enabling a terminal device (such as a mobile phone, a computer, a server, an air conditioner, or a network device) to execute the method according to the embodiments of the present invention.
The above description is only a preferred embodiment of the present invention, and not intended to limit the scope of the present invention, and all modifications of equivalent structures and equivalent processes, which are made by using the contents of the present specification and the accompanying drawings, or directly or indirectly applied to other related technical fields, are included in the scope of the present invention.
Claims (10)
1. A method for separating background and foreground of a video, the method comprising:
initializing a background model by adopting the image of the first N frames of the video stream, wherein eight neighborhoods of pixel points are adopted in each frame to select sample points;
performing foreground detection on the current frame image according to the background model, and judging foreground points and background points;
extracting a foreground target and a background image in the current frame image according to a judgment result; and
and updating the background model according to the judged background point.
2. The method of claim 1, wherein initializing the background model using the first N frames of the video stream comprises:
in each frame of the previous N frames of images, M sample points randomly selected by equal probability in eight neighborhoods of any pixel points are adopted to form a sample set;
and constructing the initialized background model according to M x N sample points of the previous N frames of images, wherein M and N are positive integers.
3. The method of claim 1, wherein the performing foreground detection on the current frame image according to the background model and determining foreground points and background points comprises:
calculating a plurality of Euclidean distances between any one first pixel point in the current frame image and a corresponding sample point in the background model;
counting the number of Euclidean distances corresponding to the first pixel points, which are smaller than a matching threshold value R;
if the number is larger than or equal to a preset first threshold value, judging the first pixel point as a background point;
and if the number is smaller than the first threshold value, judging the first pixel point as a foreground point.
4. The method of claim 3, further comprising, prior to foreground detection of a current frame image according to the background model:
and determining the matching threshold value R by adopting a self-adaptive background disturbance suppression algorithm.
5. The method of claim 4, wherein said determining said matching threshold R using an adaptive background disturbance rejection algorithm comprises:
calculating the mean value of the difference sum of the pixel values of the pixel points in the current frame image and the pixel values of the corresponding sample points in the background model;
when the calculation result is larger than a preset value, increasing the matching threshold value R;
and when the calculation result is less than or equal to the preset value, keeping the matching threshold value R unchanged.
6. The method of claim 1 or 4, further comprising, prior to foreground detection of a current frame image according to the background model:
judging whether the current frame image has illumination mutation or not;
when the sudden change of illumination occurs, the background model is reinitialized according to the previous N frames of images from the current frame.
7. The method of claim 6, wherein the determining whether the current frame image has a sudden illumination change comprises:
calculating the difference between the pixel value of each pixel point in the current frame image and the pixel value of the corresponding pixel point in the previous frame;
when the difference is larger than a preset second threshold value, determining the corresponding pixel point as a pixel point with sudden illumination change;
counting the number of the pixels with the sudden illumination change determined in the current frame image;
calculating the ratio of the number of the pixels with the illumination mutation to the total number of the pixels in the current frame image;
when the ratio is larger than a preset third threshold value, determining that the current frame image has illumination mutation;
and when the ratio is smaller than or equal to the third threshold, determining that the current frame image has no illumination mutation.
8. A video background and foreground separation system, the system comprising:
the construction module is used for initializing a background model by adopting the images of the previous N frames of the video stream, wherein eight neighborhoods of pixel points are adopted in each frame to select sample points;
the detection module is used for carrying out foreground detection on the current frame image according to the background model and judging foreground points and background points;
the extraction module is used for extracting a foreground target and a background image in the current frame image according to the judgment result;
and the updating module is used for updating the background model according to the judged background point.
9. An electronic device, comprising a memory, a processor, the memory having stored thereon a video background and foreground separation program executable on the processor, the video background and foreground separation program when executed by the processor implementing the steps of the video background and foreground separation method according to any one of claims 1-7.
10. A computer-readable storage medium storing a video background and foreground separation program executable by at least one processor to cause the at least one processor to perform the steps of the video background and foreground separation method according to any one of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011580813.4A CN112634319A (en) | 2020-12-28 | 2020-12-28 | Video background and foreground separation method and system, electronic device and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011580813.4A CN112634319A (en) | 2020-12-28 | 2020-12-28 | Video background and foreground separation method and system, electronic device and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112634319A true CN112634319A (en) | 2021-04-09 |
Family
ID=75325634
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011580813.4A Pending CN112634319A (en) | 2020-12-28 | 2020-12-28 | Video background and foreground separation method and system, electronic device and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112634319A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115442668A (en) * | 2022-07-21 | 2022-12-06 | 浙江大华技术股份有限公司 | Target state recognition method, apparatus and computer-readable storage medium |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110599523A (en) * | 2019-09-10 | 2019-12-20 | 江南大学 | ViBe ghost suppression method fused with interframe difference method |
| CN111062974A (en) * | 2019-11-27 | 2020-04-24 | 中国电力科学研究院有限公司 | Method and system for extracting foreground target by removing ghost |
-
2020
- 2020-12-28 CN CN202011580813.4A patent/CN112634319A/en active Pending
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110599523A (en) * | 2019-09-10 | 2019-12-20 | 江南大学 | ViBe ghost suppression method fused with interframe difference method |
| CN111062974A (en) * | 2019-11-27 | 2020-04-24 | 中国电力科学研究院有限公司 | Method and system for extracting foreground target by removing ghost |
Non-Patent Citations (1)
| Title |
|---|
| 刘志豪 等: "基于改进视觉背景提取算法的前景检测", 《小型微型计算机***》, vol. 41, no. 4, pages 758 - 762 * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115442668A (en) * | 2022-07-21 | 2022-12-06 | 浙江大华技术股份有限公司 | Target state recognition method, apparatus and computer-readable storage medium |
| CN115442668B (en) * | 2022-07-21 | 2024-04-12 | 浙江大华技术股份有限公司 | Target state identification method, device and computer readable storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108961303B (en) | Image processing method and device, electronic equipment and computer readable medium | |
| CN109308463B (en) | Video target identification method, device and equipment | |
| CN111062974B (en) | Method and system for extracting foreground target by removing ghost | |
| CN109446061B (en) | Page detection method, computer readable storage medium and terminal device | |
| CN112184759A (en) | Moving target detection and tracking method and system based on video | |
| WO2021013049A1 (en) | Foreground image acquisition method, foreground image acquisition apparatus, and electronic device | |
| CN110147708B (en) | Image data processing method and related device | |
| CN113661497A (en) | Matching method, matching device, electronic equipment and computer-readable storage medium | |
| JP2021135993A (en) | Text recognition method, text recognition apparatus, electronic device, and storage medium | |
| CN113112542A (en) | Visual positioning method and device, electronic equipment and storage medium | |
| CN111476814A (en) | Target tracking method, device, equipment and storage medium | |
| CN111160107B (en) | Dynamic region detection method based on feature matching | |
| CN111383191B (en) | Image processing method and device for vascular fracture repair | |
| CN112634319A (en) | Video background and foreground separation method and system, electronic device and storage medium | |
| JP7014005B2 (en) | Image processing equipment and methods, electronic devices | |
| EP4075381B1 (en) | Image processing method and system | |
| CN112907433A (en) | Digital watermark embedding method, digital watermark extracting device, digital watermark embedding apparatus, digital watermark extracting apparatus, and digital watermark extracting medium | |
| CN112084880A (en) | Image processing method, device, storage medium and equipment | |
| CN115345895B (en) | Image segmentation method and device for visual detection, computer equipment and medium | |
| CN116128922A (en) | Object drop detection method, device, medium and equipment based on event camera | |
| CN111369591A (en) | Method, device and equipment for tracking moving object | |
| CN108520259B (en) | Foreground target extraction method, device, equipment and storage medium | |
| CN116012732A (en) | Video generation method, device, equipment and storage medium | |
| CN113762027B (en) | Abnormal behavior identification method, device, equipment and storage medium | |
| CN110580706A (en) | Method and device for extracting video background model |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |