CN114187245A - Video contamination detection method and device, electronic equipment and storage medium - Google Patents

Video contamination detection method and device, electronic equipment and storage medium Download PDF

Info

Publication number
CN114187245A
CN114187245A CN202111426236.8A CN202111426236A CN114187245A CN 114187245 A CN114187245 A CN 114187245A CN 202111426236 A CN202111426236 A CN 202111426236A CN 114187245 A CN114187245 A CN 114187245A
Authority
CN
China
Prior art keywords
image
detected
video
point
frame
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202111426236.8A
Other languages
Chinese (zh)
Inventor
陈海波
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shenlan Industrial Intelligent Innovation Research Institute Ningbo Co ltd
Original Assignee
Shenlan Industrial Intelligent Innovation Research Institute Ningbo Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shenlan Industrial Intelligent Innovation Research Institute Ningbo Co ltd filed Critical Shenlan Industrial Intelligent Innovation Research Institute Ningbo Co ltd
Priority to CN202111426236.8A priority Critical patent/CN114187245A/en
Publication of CN114187245A publication Critical patent/CN114187245A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G06T7/215—Motion-based segmentation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G06T7/269—Analysis of motion using gradient-based methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10016—Video; Image sequence
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30168—Image quality inspection

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Multimedia (AREA)
  • Quality & Reliability (AREA)
  • Image Processing (AREA)

Abstract

The video contamination detection method provided by the invention carries out processing comprising the following steps on each frame of image to be detected in a video to be detected: extracting an image to be detected and another frame image adjacent to the image to be detected from a video to be detected; calculating optical flow characteristics based on the image to be detected and another frame image, and performing visualization processing and binarization processing on the calculated optical flow characteristics to obtain a first-stage result image; performing morphological filtering on the first-stage result image to obtain a second-stage result image; and removing the connected domain with the area more than the specified area in the two-stage result image, thereby detecting the stained point in the image to be detected. The invention also provides a video contamination detection device, electronic equipment and a storage medium. The present invention can detect a stain point appearing in a video as a moving object based on an optical flow method and image morphological filtering, and can easily detect a stain point in a digital video converted from a film roll.

Description

Video contamination detection method and device, electronic equipment and storage medium
Technical Field
The present application relates to the field of video processing technologies, and in particular, to a method and an apparatus for detecting video contamination, an electronic device, and a computer-readable storage medium.
Background
Earlier filmed films have become valuable historical image data today, with significant research and preservation value. With the development of digital technology, people have been able to convert film films into digital films for storage.
However, the film is easily contaminated by dust during long-term storage, and therefore, after the digital video is turned from the film roll, there may be stained spots (dark spots or bright spots) that randomly appear in the video. When repairing the dirt points caused by the dust, the existing repairing technology mostly depends on manual work, and professional personnel are required to repair the dirt points frame by frame through a repair tool, so that the efficiency is extremely low.
Some image quality detection technologies exist at present, but the image quality detection technologies can only detect the degree of video contamination so as to judge the quality of the image, cannot detect the position of a contamination point, and even cannot repair a contamination area.
Disclosure of Invention
An object of the present invention is to provide a video smear detection method, apparatus, electronic device, and storage medium, which can easily detect a smear point in a digital video converted from a film roll.
The purpose of the application is realized by adopting the following technical scheme:
in a first aspect, the present application provides a video contamination detection method, which performs processing on an image to be detected in each frame of a video to be detected, the processing including the following steps:
extracting an image to be detected and another frame image adjacent to the image to be detected from a video to be detected;
calculating optical flow characteristics based on the image to be detected and the other frame image, and performing visualization processing and binarization processing on the calculated optical flow characteristics to obtain a first-stage result image;
performing morphological filtering on the first-stage result image to obtain a second-stage result image; and
and removing connected domains with the area more than the specified area in the two-stage result image, thereby detecting the stained point in the image to be detected.
The technical scheme has the beneficial effects that: the stain point appearing in the video can be detected as a moving object based on the optical flow method and the image morphological filtering, so that the stain point in the digital video converted from a film can be easily detected.
Optionally, in the video contamination detection method, a frame of image continuous to the image to be detected is extracted from the video to be detected as the other frame of image adjacent to the image to be detected, and a contamination point in the image to be detected is detected based on the image to be detected and the other frame of image.
The technical scheme has the beneficial effects that: the method can detect the stained point in the image to be detected by using the image with shorter time interval with the image to be detected, thereby more accurately detecting the stained point by using an optical flow method.
Optionally, in the video contamination detection method, an image separated from the image to be detected by one frame of image is extracted from the video to be detected as another frame of image adjacent to the image to be detected, and a contamination point in the image to be detected is detected again based on the image to be detected and the another frame of image, so as to verify the contamination point.
The technical scheme has the beneficial effects that: the stain point can be verified using an image spaced one frame from the image to be detected, thereby preventing false detection of the stain point.
Alternatively, in the above-described video smear detection method, the visualization process of the optical flow features is completed by normalizing the optical flow features to numerical values corresponding to brightness,
and assigning a value which is larger than the first threshold value in the values obtained by the optical flow feature normalization to be a first value, and assigning other values in the values obtained by the optical flow feature normalization to be a second value by setting the first threshold value, thereby finishing the binarization processing.
The technical scheme has the beneficial effects that: the method can visualize the stained point detected by the optical flow method, and is convenient for further distinguishing the stained point from the normally moving object in the later step.
Optionally, in the above video contamination detection method, for the detected contamination point, the contamination point is filled with pixels near the contamination point, so that the video is repaired.
The technical scheme has the beneficial effects that: since the stained area can be filled with pixels in the vicinity of the stained point, even if some of the stained points are erroneously detected, the stained repair after the stained point is detected is not affected.
Optionally, in the video contamination detection method, for the detected contamination point, the contamination point is filled with pixels at a position corresponding to the contamination point in a previous frame image or a subsequent frame image of the image to be detected, so as to repair the video.
The technical scheme has the beneficial effects that: the method can repair the stained point by combining the information of the previous and next frame images, so that even if some stained points are detected by mistake, the stained point repair after the stained point is detected cannot be influenced.
With the above-described aspect of the present invention, it is possible to detect a stain point appearing in a video as a moving object based on an optical flow method and image morphological filter detection, and it is possible to easily detect a stain point in a digital video converted from a film roll.
In a second aspect, the present application provides a video contamination detection apparatus having:
the image extraction unit extracts an image to be detected and another frame image adjacent to the image to be detected from a video to be detected;
a first-stage calculation unit which calculates optical flow characteristics based on the image to be detected and the other frame image, and performs visualization processing and binarization processing on the calculated optical flow characteristics to obtain a first-stage result image;
the two-stage calculation unit is used for performing morphological filtering on the first-stage result image to obtain a two-stage result image; and
and an image detection unit that detects an image to be detected by removing a connected domain having a predetermined area or more from the two-stage result image.
Optionally, in the video contamination detection apparatus, the image extraction unit extracts, from a video to be detected, one frame image that is continuous with the image to be detected as the other frame image that is adjacent to the image to be detected, and the one-stage calculation unit and the two-stage calculation unit detect the contamination point in the image to be detected based on the image to be detected and the other frame image.
Alternatively, in the above-described video smear detection apparatus,
the image extraction unit extracts an image which is separated from the image to be detected by one frame of image from the video to be detected as another frame of image adjacent to the image to be detected, and the one-stage calculation unit and the two-stage calculation unit detect the stained point in the image to be detected again based on the image to be detected and the another frame of image, so that the stained point is verified.
Alternatively, in the above-described video smear detection apparatus, the one-stage calculation unit may perform the visualization process of the optical flow features by normalizing the optical flow features to numerical values corresponding to brightness,
the two-stage calculation unit assigns a value larger than the first threshold value among values obtained by normalization of the optical flow features to a first value and assigns other values among the values obtained by normalization of the optical flow features to a second value by setting the first threshold value, thereby completing the binarization processing.
Optionally, in the above video contamination detection apparatus, there are:
and the video repairing unit is used for filling the dirty point with pixels near the dirty point aiming at the detected dirty point so as to repair the video.
Optionally, in the video contamination detection apparatus, the video repair unit fills the contamination point with a pixel at a position corresponding to the contamination point in a previous frame image or a subsequent frame image of the image to be detected, so as to repair the video, with respect to the contamination point detected.
In a third aspect, the present application provides an electronic device comprising a memory and a processor, the memory storing a computer program, the processor implementing the steps of any of the above methods when executing the computer program.
In a fourth aspect, the present application provides a computer readable storage medium having stored thereon a computer program which, when executed by a processor, carries out the steps of any of the methods described above.
Drawings
The present application is further described below with reference to the drawings and examples.
FIG. 1 is a schematic illustration of the steps of a video smear detection method provided herein;
FIG. 2 is a schematic illustration of an image to be detected in the present application;
FIG. 3 is a schematic diagram of another frame of image adjacent to an image to be detected in the present application;
FIG. 4 is a schematic illustration of a stage result image in the present application;
FIG. 5 is a schematic illustration of a two-stage result image in the present application;
FIG. 6 is a stain point detected using the video stain detection method of the present application;
FIG. 7 is a schematic illustration of an image after repair of an insult point;
FIG. 8 is a schematic structural diagram of a video contamination detection apparatus provided by the present application;
fig. 9 is a schematic structural diagram of an electronic device according to an embodiment of the present application;
fig. 10 is a schematic structural diagram of a program product for implementing a video contamination detection method according to an embodiment of the present application.
Detailed Description
The present application is further described with reference to the accompanying drawings and the detailed description, and it should be noted that, in the present application, the embodiments or technical features described below may be arbitrarily combined to form a new embodiment without conflict.
The inventors of the present invention, after studying the repair of a video from the film roll rotation data, have found that the stain points appearing in this type of video are characterized in that, since the stain points are caused by dust adhering to a certain frame film of the film roll, the stain points do not appear continuously at the same position in a plurality of frames of the video although the positions where they appear in the video are random.
The inventors of the present invention studied this feature of the smear point, and found that the smear problem is manifested in that the brightness of a certain area (where dust is located) on the image is abruptly changed and does not continue to appear in this area. Based on this, the inventors of the present invention thought that a stain point appearing in a certain frame image and disappearing in the next frame image can be handled as a moving object, and further thought that a stain point (i.e., a moving object) in a video can be detected by an optical flow method.
The video contamination detection method provided by the invention carries out processing comprising the following steps on each frame of image to be detected in a video to be detected:
step S1: extracting an image to be detected and another frame image adjacent to the image to be detected from a video to be detected;
step S2: calculating optical flow characteristics based on the image to be detected and another frame image, and performing visualization processing and binarization processing on the calculated optical flow characteristics to obtain a first-stage result image;
step S3: performing morphological filtering on the first-stage result image to obtain a second-stage result image; and
step S4: and removing the connected domain with the area more than the specified area in the two-stage result image, thereby detecting the stained point in the image to be detected.
The present invention can detect a stain point appearing in a video as a moving object based on an optical flow method and image morphological filtering, and can easily detect a stain point in a digital video converted from a film roll. In addition, the method and the device can repair the stained points by combining the information of the same frame or the previous and subsequent frame images, thereby improving the quality of the video image and improving the watching experience.
The above steps are specifically described below.
In step S1 of the present invention, an image to be detected and another frame image adjacent to the image to be detected are extracted from a video to be detected.
Here, the video to be detected refers to digital video converted from video stored on film.
In the present application, when the entire video needs to be detected, the image to be detected is called frame by frame starting from the first frame of the video, and the processing including step S2, step S3, and step S4 is performed on each frame of the image to be detected and another frame of image adjacent to the image to be detected.
Specifically, a first frame image and a second frame image of the video may be extracted first, and a stain point in the first frame image may be detected, and then the second frame image and a third frame image may be extracted, and a stain point in the second frame image may be detected. The detection is performed frame by frame in this order, so that a smear point of the entire video can be detected.
However, it is understood that since the smear point randomly appears in a certain frame image, and there is no smear point at the same position in both the previous frame image and the next frame image of the frame image, it is of course possible to detect the smear point in the next frame image using the previous frame image in chronological order. Specifically, the last frame image and the second to last frame image of the video may be extracted, the smear point in the last frame image may be detected, and the detection may be performed sequentially until the first frame image of the video is detected.
Of course, the detection order of the specific image frames is not limited as long as the video contamination detection method of the present invention can detect contamination points of a video.
In the above description, although "another frame image adjacent to the image to be detected" is described, in the description and claims of the present invention, "another frame image adjacent to the image to be detected" is not limited to one frame image continuous to the image to be detected, and any frame image whose time interval with the image to be detected is not so large that the object in the image is greatly displaced may be used as "another frame image adjacent to the image to be detected" described in the description and claims of the present invention. For example, any image whose time interval with the image to be detected is below 1/24s belongs to the "another frame image adjacent to the image to be detected" defined in the present specification and claims.
For example, it is entirely possible to extract, from the video to be detected, an image one frame apart from the image to be detected as "another frame image adjacent to the image to be detected" (for example, when the first frame is extracted as the image to be detected, the third frame is extracted as another frame image adjacent to the image to be detected). Because the time interval between every two frames of video is extremely short when the video is generally shot, even if one frame is separated between two adjacent frames of images, the object in the two adjacent frames of images can not generate huge displacement, and the displacement and the stained point of the object in the two adjacent frames of images can still be detected by using an optical flow method.
In the present invention, one frame image continuous with an image to be detected is extracted from a video to be detected as another frame image adjacent to the image to be detected, and a stain point in the image to be detected is detected based on the image to be detected and the another frame image.
In addition, in the invention, an image separated from the image to be detected by one frame of image is extracted from the video to be detected as another frame of image adjacent to the image to be detected, and the stained point in the image to be detected is detected again based on the image to be detected and the other frame of image, so that the stained point can be verified, and the reliability of detecting the stained point is increased. When an image one frame apart from the image to be detected is used as "another frame image adjacent to the image to be detected", a first threshold value for binarization, which will be described later, may be appropriately enlarged according to the actual situation.
In step S2 of the present invention: and calculating optical flow characteristics based on the image to be detected and the other frame image, and performing visualization processing and binarization processing on the calculated optical flow characteristics to obtain a first-stage result image.
First, the optical flow characteristics will be explained.
On an image plane, the motion of an object is often represented by the difference of the gray-scale distribution of the different images in the image sequence, so that the motion field in space is transferred to the image and is represented as an optical flow field. The optical flow field is a two-dimensional vector field which reflects the change trend of the gray scale of each point on the image and can be regarded as an instantaneous velocity field generated by the movement of a pixel point with the gray scale on an image plane. The information contained in it is the instantaneous motion velocity vector information of each pixel point.
In an ideal case, the optical flow field corresponds to a motion field. By using the optical flow method, a moving object can be detected, and the speed of the moving object can be calculated.
In the present invention, the inventors have studied the characteristics of the stain points that occur when converting a video stored on a film into a digital video, and have found that these stain points can be captured as moving objects because the positions of these spots that occur in the video are random and do not continuously occur at the same position in a plurality of frames of video.
The concrete description is as follows:
consider the light intensity of a pixel I (x, y, t) in the first frame (where t represents the time dimension in which it is located). It moves the distance (dx, dy) to the next frame, taking dt times. Assuming that the light intensity of the pixel before and after the motion is considered to be constant, i.e.:
I(x，y，t)＝I(x+dx，y+dy，t+dt) (1)
carrying out Taylor expansion on the right end of the formula (1) to obtain:
Figure BDA0003374815080000081
where ε represents a second order infinite small term, which can be ignored. Further, when formula (2) is substituted for formula (1), it is possible to obtain:
Figure BDA0003374815080000082
and u and v are velocity vectors of optical flows along an X axis and a Y axis respectively, and the velocity vectors are obtained:
Figure BDA0003374815080000083
if order
Figure BDA0003374815080000084
Then equation (3) can be written as
Ixu+Iyv+It＝0 (5)
Where Ix, Iy, It can be obtained from the image data, and (u, v) is the optical flow vector.
The optical flow vector reflects the change trend of the gray level of each point on the image, and can be regarded as the instantaneous speed generated by the movement of the pixel points with the gray levels on the image plane. According to the study of the present inventors, since a stain point in a video does not continuously appear at the same position in a multi-frame video, it is possible to study the stain point as a moving object and calculate the optical flow feature thereof.
As described above, an optical flow feature is a vector feature, with vector direction representing the instantaneous direction of motion of the point and vector magnitude representing the magnitude of motion. The brightness range of the image is [0, 255], so that the visualization of the optical flow characteristics can be completed only by normalizing the calculated optical flow characteristics to be [0, 255], and the image at this time is a gray scale image. The process of normalizing the optical flow features to the luminance range of the image is the visualization process of the image.
After the optical flow features are visualized, the size of the optical flow features falls between [0, 255], and at this time, a small threshold is selected, and the motion region detected by the optical flow method is assigned 255 and the other regions are assigned 0, so that the motion region (including the stain point regarded as the motion object) can be displayed in white and the other regions can be displayed in black in the image. The above process is the binarization processing.
And performing visualization processing and binarization processing on the optical flow characteristics to obtain a stage result image.
In step S3 of the present application, morphological filtering is performed on the one-stage result image to obtain a two-stage result image.
As is clear from the above description of the binarization processing, in the one-stage result image, in addition to the smear point, the region of another moving object in the image is displayed in white.
At this time, the one-stage result image is morphologically filtered so that the optical flows of the normal motion areas are connected into a large connected domain.
It should be noted that since the distribution of the stained points is random and discrete, the stained area is not changed by the morphological filtering.
In step S4 of the present application, the connected component having a predetermined area or more in the two-stage result image is removed, and the stain point in the image to be detected is detected.
In step S4, since the normally moving object in the two-stage result image has been removed, the remaining white dot is a stain dot. That is, the smear point in the image to be detected is detected.
In this step, detecting a stain point in an image to be detected means detecting a position of the stain point in the image.
With the above-described aspect of the present invention, it is possible to detect a stain point appearing in a video as a moving object based on an optical flow method and image morphological filter detection, and it is possible to easily detect a stain point in a digital video converted from a film roll.
In addition, the method may further include step S5, in which, for the detected contamination point, the contamination point is filled with pixels near the contamination point in the image to be detected, so as to repair the video. Alternatively, the video may be repaired by filling the detected stain point with a pixel at a position corresponding to the stain point in the previous frame image or the subsequent frame image of the image to be detected.
Through the above steps S1 to S5, although some of the contaminated points are erroneously detected, no missing detection is detected. And since the repair method employed by step S5 fills the stained area with pixels near the stained point or pixels at the same position in the previous and next frame images, the false detection area has little effect on the stain repair after the stained point is detected.
According to the invention, the stained points can be repaired by combining the information of the same frame or the previous and subsequent frame images, thereby improving the quality of the video image and improving the watching experience.
Next, an embodiment of the present invention will be described.
First, an image to be detected (see fig. 2) and a next frame image of the image to be detected in the video to be detected (i.e., in the present embodiment, "another frame image adjacent to the image to be detected" is a frame image continuous with the image to be detected, see fig. 3) are extracted from the video to be detected.
Fig. 2 and 3 are two continuous frames of images in a video to be detected, and white dots in the images are smear dots. Since the optical flow method detects a motion feature from a change in luminance, black stain points can be detected similarly.
Optical flow features are calculated based on fig. 2 and 3, and visualization processing and binarization processing are performed on the calculated optical flow features to obtain a one-stage result image.
Fig. 4 is a one-stage result image obtained by performing a visualization process and a binarization process on the optical flow information detected in fig. 2 and 3. In this embodiment, the threshold value of the binarization processing is set to 5. When an image spaced one frame apart from the image to be detected is employed as another frame image adjacent to the image to be detected, the threshold value of the binarization processing may be set to 10, for example.
Then, morphological filtering is performed on the first-stage result image to obtain a second-stage result image, as shown in fig. 5.
Then, connected components having areas larger than or equal to a predetermined area in the two-stage result image are removed, and a stain point (e.g., a white dot in fig. 6) in the image to be detected (fig. 2) is detected. The size of the "connected component region having a predetermined area or more" can be set by the operator in accordance with the actual situation, and in the present embodiment, the "connected component region having a predetermined area or more" is set to be one percent of the image size.
Thereafter, the smear point is filled in using pixels in the vicinity of the smear point in the image to be detected (fig. 2). Thereby obtaining a restored image shown in fig. 7.
The stained points in the whole video can be detected and repaired by taking the image in the video to be detected frame by frame as the image to be detected and processing the image frame by frame.
The present invention can detect a stain point appearing in a video as a moving object based on an optical flow method and image morphological filtering, and can easily detect a stain point in a digital video converted from a film roll. In addition, the method and the device can repair the stained points by combining the information of the same frame or the previous and subsequent frame images, thereby improving the quality of the video image and improving the watching experience.
As shown in fig. 8, the present invention also provides a video contamination detection apparatus having:
an image extraction unit 101, the image extraction unit 101 extracting an image to be detected and another frame image adjacent to the image to be detected from a video to be detected;
a first-stage calculating unit 102, wherein the first-stage calculating unit 102 calculates an optical flow feature based on the image to be detected and the other frame image, and performs visualization processing and binarization processing on the calculated optical flow feature to obtain a first-stage result image;
a two-stage calculating unit 103, wherein the two-stage calculating unit 103 performs morphological filtering on the one-stage result image to obtain a two-stage result image; and
and an image stain detection unit 104, wherein the image stain detection unit 104 removes the connected domain with the area larger than the predetermined area in the two-stage result image, thereby detecting the stain in the image to be detected.
In one embodiment, in the video contamination detection apparatus of the present invention, the image extraction unit 101 extracts one frame image continuous with the image to be detected from the video to be detected as another frame image adjacent to the image to be detected, and the one-stage calculation unit 102 and the two-stage calculation unit 103 detect a contamination point in the image to be detected based on the image to be detected and the another frame image.
In an embodiment, in the video contamination detection apparatus of the present invention, the image extraction unit 101 extracts, from the video to be detected, an image separated from the image to be detected by one frame of image as another frame of image adjacent to the image to be detected, and the one-stage calculation unit 102 and the two-stage calculation unit 103 detect a contamination point in the image to be detected again based on the image to be detected and the another frame of image, thereby verifying the contamination point.
In an embodiment of the present invention, the first-stage calculating unit 102 performs the visualization process of the optical flow features by normalizing the optical flow features to a numerical value corresponding to brightness, and the second-stage calculating unit 103 performs the binarization process by setting a first threshold value, assigning a numerical value greater than the first threshold value among the numerical values obtained by normalizing the optical flow features to a first value, and assigning other numerical values among the numerical values obtained by normalizing the optical flow features to a second value.
The video contamination detection apparatus of the present invention may further have a video repair unit 105 that, for the detected contamination point, fills the contamination point with pixels near the contamination point, thereby repairing the video; alternatively, the video repair unit 105 may fill the dirty point with a pixel at a position corresponding to the dirty point in a previous frame image or a subsequent frame image of the image to be detected, so as to repair the video.
Referring to fig. 9, an embodiment of the present application further provides an electronic device 200, where the electronic device 200 includes at least one memory 210, at least one processor 220, and a bus 230 connecting different platform systems.
The memory 210 may include readable media in the form of volatile memory, such as Random Access Memory (RAM)211 and/or cache memory 212, and may further include Read Only Memory (ROM) 213.
The memory 210 further stores a computer program, and the computer program can be executed by the processor 220, so that the processor 220 executes the steps of the video contamination detection method in the embodiment of the present application, and a specific implementation manner of the method is consistent with the implementation manner and the achieved technical effect described in the embodiment of the video contamination detection method, and details of the method are not repeated.
Memory 210 may also include a utility 214 having at least one program module 215, such program modules 215 including, but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which, or some combination thereof, may comprise an implementation of a network environment.
Accordingly, the processor 220 may execute the computer programs described above, and may execute the utility 214.
Bus 230 may be a local bus representing one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processor, or any other type of bus structure.
The electronic device 200 may also communicate with one or more external devices 240, such as a keyboard, pointing device, bluetooth device, etc., and may also communicate with one or more devices capable of interacting with the electronic device 200, and/or with any devices (e.g., routers, modems, etc.) that enable the electronic device 200 to communicate with one or more other computing devices. Such communication may be through input-output interface 250. Also, the electronic device 200 may communicate with one or more networks (e.g., a Local Area Network (LAN), a Wide Area Network (WAN), and/or a public network such as the Internet) via the network adapter 260. The network adapter 260 may communicate with other modules of the electronic device 200 via the bus 230. It should be appreciated that although not shown in the figures, other hardware and/or software modules may be used in conjunction with the electronic device 200, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID systems, tape drives, and data backup storage platforms, to name a few.
The embodiments of the present application further provide a computer-readable storage medium, where the computer-readable storage medium is used to store a computer program, and when the computer program is executed, the steps of the video contamination detection method in the embodiments of the present application are implemented, and a specific implementation manner of the steps is consistent with the implementation manner and the achieved technical effect described in the embodiments of the video contamination detection method, and some details are not repeated.
Fig. 10 shows a program product 300 for implementing the video contamination detection method according to the present embodiment, which may employ a portable compact disc read only memory (CD-ROM) and include program codes, and may be executed on a terminal device, such as a personal computer. However, the program product 300 of the present invention is not so limited, and in this application, a readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. Program product 300 may employ any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples (a non-exhaustive list) of the readable storage medium include: an electrical connection having one or more wires, a portable disk, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
A computer readable storage medium may include a propagated data signal with readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A readable storage medium may also be any readable medium that can communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device. Program code embodied on a readable storage medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing. Program code for carrying out operations for aspects of the present invention may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, C + + or the like and conventional procedural programming languages, such as the C language or similar programming languages. The program code may execute entirely on the user's computing device, partly on the user's device, as a stand-alone software package, partly on the user's computing device and partly on a remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device may be connected to the user computing device through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computing device (e.g., through the internet using an internet service provider).
While the present application is described in terms of various aspects, including exemplary embodiments, the principles of the invention should not be limited to the disclosed embodiments, but are also intended to cover various modifications, equivalents, and alternatives falling within the spirit and scope of the invention as defined by the appended claims.

Claims (10)

1. The video contamination detection method is characterized in that an image to be detected in each frame of a video to be detected is processed by the following steps:
extracting an image to be detected and another frame image adjacent to the image to be detected from a video to be detected;
calculating optical flow characteristics based on the image to be detected and the other frame image, and performing visualization processing and binarization processing on the calculated optical flow characteristics to obtain a first-stage result image;
performing morphological filtering on the first-stage result image to obtain a second-stage result image; and
and removing connected domains with the area more than the specified area in the two-stage result image, thereby detecting the stained point in the image to be detected.
2. The video smear detection method of claim 1,
and extracting a frame image continuous with the image to be detected from the video to be detected as the other frame image adjacent to the image to be detected, and detecting a stain point in the image to be detected based on the image to be detected and the other frame image.
3. The video smear detection method of claim 2,
and extracting an image which is separated from the image to be detected by one frame of image from the video to be detected as another frame of image adjacent to the image to be detected, and detecting the stained point in the image to be detected again based on the image to be detected and the another frame of image, thereby verifying the stained point.
4. The video smear detection method of claim 1,
performing the visualization of the optical flow features by normalizing the optical flow features to a numerical value corresponding to brightness,
and assigning a value which is larger than the first threshold value in the values obtained by the optical flow feature normalization to be a first value, and assigning other values in the values obtained by the optical flow feature normalization to be a second value by setting the first threshold value, thereby finishing the binarization processing.
5. The video smear detection method of claim 1,
and filling the dirty point with pixels near the dirty point aiming at the detected dirty point, so as to repair the video.
6. The video smear detection method of claim 1,
and filling the stained point with the pixel of the position corresponding to the stained point in the previous frame image or the next frame image of the image to be detected aiming at the detected stained point, so as to repair the video.
7. A video contamination detection apparatus comprising:
the image extraction unit extracts an image to be detected and another frame image adjacent to the image to be detected from a video to be detected;
a first-stage calculation unit which calculates optical flow characteristics based on the image to be detected and the other frame image, and performs visualization processing and binarization processing on the calculated optical flow characteristics to obtain a first-stage result image;
the two-stage calculation unit is used for performing morphological filtering on the first-stage result image to obtain a two-stage result image; and
and an image detection unit that detects an image to be detected by removing a connected domain having a predetermined area or more from the two-stage result image.
8. The video smear detection apparatus according to claim 7, having:
and the video repairing unit is used for filling the dirty point with pixels near the dirty point aiming at the detected dirty point so as to repair the video.
9. An electronic device, comprising a memory storing a computer program and a processor, wherein the processor implements the steps of the video contamination detection method of any one of claims 1-6 when executing the computer program.
10. A computer-readable storage medium, in which a computer program is stored which, when being executed by a processor, carries out the steps of the video contamination detection method according to any one of claims 1 to 6.
CN202111426236.8A 2021-11-25 2021-11-25 Video contamination detection method and device, electronic equipment and storage medium Pending CN114187245A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202111426236.8A CN114187245A (en) 2021-11-25 2021-11-25 Video contamination detection method and device, electronic equipment and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202111426236.8A CN114187245A (en) 2021-11-25 2021-11-25 Video contamination detection method and device, electronic equipment and storage medium

Publications (1)

Publication Number Publication Date
CN114187245A true CN114187245A (en) 2022-03-15

Family

ID=80602788

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202111426236.8A Pending CN114187245A (en) 2021-11-25 2021-11-25 Video contamination detection method and device, electronic equipment and storage medium

Country Status (1)

Country Link
CN (1) CN114187245A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114757947A (en) * 2022-06-14 2022-07-15 苏州魔视智能科技有限公司 Defect detection method, device and system for camera lens

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114757947A (en) * 2022-06-14 2022-07-15 苏州魔视智能科技有限公司 Defect detection method, device and system for camera lens
CN114757947B (en) * 2022-06-14 2022-09-27 苏州魔视智能科技有限公司 Method, device and system for detecting fouling of camera lens

Similar Documents

Publication Publication Date Title
US20200175673A1 (en) Method and device for detecting defect of meal box, server, and storage medium
CN112287875B (en) Abnormal license plate recognition method, device, equipment and readable storage medium
CN109413411B (en) Black screen identification method and device of monitoring line and server
CN113436100B (en) Method, apparatus, device, medium, and article for repairing video
CN107992791A (en) Target following failure weight detecting method and device, storage medium, electronic equipment
CN114220009A (en) Infrared image-based wire windage yaw identification method and system
US10599946B2 (en) System and method for detecting change using ontology based saliency
CN113378834A (en) Object detection method, device, apparatus, storage medium, and program product
CN114187245A (en) Video contamination detection method and device, electronic equipment and storage medium
CN113326721A (en) Image blur detection method and device based on sliding window re-blur
CN110807416A (en) Digital instrument intelligent recognition device and method suitable for mobile detection device
CN112380940B (en) Processing method and device of high-altitude parabolic monitoring image, electronic equipment and storage medium
CN107833185B (en) Image defogging method and device, storage medium and electronic equipment
CN109934185B (en) Data processing method and device, medium and computing equipment
CN108961293B (en) Background subtraction method, device, equipment and storage medium
CN113869163B (en) Target tracking method and device, electronic equipment and storage medium
KR20230043419A (en) Ai-based building defect inspection system
CN113591752A (en) Equipment oil leakage monitoring method based on convolutional neural network and related equipment
CN113628192A (en) Image blur detection method, device, apparatus, storage medium, and program product
CN113117341B (en) Picture processing method and device, computer readable storage medium and electronic equipment
CN111191593A (en) Image target detection method and device, storage medium and sewage pipeline detection device
CN111062337B (en) People stream direction detection method and device, storage medium and electronic equipment
CN113379838B (en) Method for generating roaming path of virtual reality scene and storage medium
CN115861321B (en) Production environment detection method and system applied to industrial Internet
CN117765439A (en) Target object detection method, vehicle control method, device, chip and equipment

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination