CN109064494B - Video floating paper detection method and device and computer readable storage medium - Google Patents
Video floating paper detection method and device and computer readable storage medium Download PDFInfo
- Publication number
- CN109064494B CN109064494B CN201811068698.5A CN201811068698A CN109064494B CN 109064494 B CN109064494 B CN 109064494B CN 201811068698 A CN201811068698 A CN 201811068698A CN 109064494 B CN109064494 B CN 109064494B
- Authority
- CN
- China
- Prior art keywords
- detected
- video
- floating paper
- picture
- paper sheet
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
- G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/431—Generation of visual interfaces for content selection or interaction; Content or additional data rendering
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/431—Generation of visual interfaces for content selection or interaction; Content or additional data rendering
- H04N21/4312—Generation of visual interfaces for content selection or interaction; Content or additional data rendering involving specific graphical features, e.g. screen layout, special fonts or colors, blinking icons, highlights or animations
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10016—Video; Image sequence
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Signal Processing (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Physics & Mathematics (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Evolutionary Computation (AREA)
- General Engineering & Computer Science (AREA)
- Evolutionary Biology (AREA)
- Bioinformatics & Computational Biology (AREA)
- Artificial Intelligence (AREA)
- Life Sciences & Earth Sciences (AREA)
- Image Analysis (AREA)
Abstract
The present disclosure discloses a video floating paper detection method, a video floating paper detection device, a video floating paper detection hardware device, and a computer-readable storage medium. The video floating paper detection method comprises the steps of carrying out floating paper detection on at least one frame of picture to be detected extracted from a video to be detected, wherein the floating paper is a sub-display window which is inserted into the video to be detected and is irrelevant to the content of the video to be detected; and determining whether the video to be detected contains floating paper sheets or not according to the detection result of the at least one frame of picture to be detected. The method and the device for detecting the floating paper sheets in the video comprise the steps of firstly detecting the floating paper sheets of at least one frame of picture to be detected extracted from the video to be detected, wherein the floating paper sheets are sub-display windows which are inserted into the video to be detected and are irrelevant to the content of the video to be detected, and then determining whether the video to be detected contains the floating paper sheets or not according to the detection result of the at least one frame of picture to be detected, so that the video classification accuracy can be improved.
Description
Technical Field
The present disclosure relates to the field of information processing technologies, and in particular, to a method and an apparatus for detecting a video floating paper sheet, and a computer-readable storage medium.
Background
In recent years, with the rapid development of multimedia technology and computer networks, the capacity of digital video is increasing at an alarming rate. In this way, images captured from digital video often contain important text information, which plays an important role in text content-based video database retrieval. The method is convenient for concise description and explanation of main contents of the video to a certain extent, or convenient for video classification, or convenient for identification of illegal videos, and the like.
In the prior art, the author of the video often adds some words, such as a slogan, a introductory phrase, etc., to the video, and in general, the words are printed, and the existing recognition method is relatively easy to recognize. However, in some other scenarios, the author of the video may add some paper effects to the video, and some handwritten characters may appear on the paper effects. Since handwritten words are usually scrawled and not easily recognized, the difficulty brought to video classification is often classified as a no-word video because the handwritten words are not recognized.
Therefore, it is necessary to detect the floating paper contained in the video for this type of video, and then manually recognize the text information contained in the floating paper, so it becomes important to accurately detect whether the video contains the floating paper.
Disclosure of Invention
The technical problem solved by the present disclosure is to provide a method for detecting a video floating paper sheet, so as to at least partially solve the technical problem of inaccurate classification of existing videos. In addition, a video floating paper sheet detection device, a video floating paper sheet detection hardware device, a computer readable storage medium and a video floating paper sheet detection terminal are also provided.
In order to achieve the above object, according to one aspect of the present disclosure, the following technical solutions are provided:
a video floating paper detection method comprises the following steps:
detecting a floating paper sheet of at least one frame of picture to be detected extracted from a video to be detected, wherein the floating paper sheet is a sub-display window which is inserted into the video to be detected and is irrelevant to the content of the video to be detected;
and determining whether the video to be detected contains floating paper sheets or not according to the detection result of the at least one frame of picture to be detected.
Further, the step of determining whether the video to be detected contains a floating paper sheet according to the detection result of the at least one frame of picture to be detected includes:
and if the floating paper sheet is detected to be contained in at least one frame of picture to be detected, determining that the floating paper sheet is contained in the video to be detected.
Further, the step of detecting the floating paper sheet of at least one frame of picture to be detected extracted from the video to be detected includes:
extracting image characteristics of each frame of picture to be detected aiming at a plurality of frames of pictures to be detected;
and comparing the image characteristics of the frames of pictures to be detected, and if the pictures to be detected containing the same image characteristics exist, determining that at least two frames of the pictures to be detected contain floating paper pieces.
Further, the step of detecting the floating paper sheet of at least one frame of picture to be detected extracted from the video to be detected includes:
extracting feature points of a single frame of picture to be detected and adjacent feature points of the feature points;
determining a characteristic region according to the similarity of the characteristic point and the adjacent characteristic points;
and if the picture to be detected contains at least two characteristic regions, determining that the picture to be detected contains a floating paper sheet.
Further, the method further comprises:
using pictures known to contain floating paper pieces and/or pictures known not to contain floating paper pieces as training samples;
labeling the training sample according to whether the training sample contains a floating paper sheet or not;
training and learning the marked training samples by adopting a deep learning classification algorithm to obtain an image classifier;
the step of detecting the floating paper sheet of at least one frame of picture to be detected extracted from the video to be detected comprises the following steps:
and inputting the at least one frame of picture to be detected into the image classifier, and determining a detection result in the at least one frame of picture to be detected according to a classification result of the image classifier.
In order to achieve the above object, according to still another aspect of the present disclosure, the following technical solutions are also provided:
a video floating paper detection device comprising:
the floating paper piece detection module is used for carrying out floating paper piece detection on at least one frame of picture to be detected extracted from a video to be detected, and the floating paper piece is a sub-display window which is inserted into the video to be detected and is irrelevant to the content of the video to be detected;
and the floating paper sheet determining module is used for determining whether the video to be detected contains floating paper sheets or not according to the detection result of the at least one frame of picture to be detected.
Further, the floating paper piece determination module is specifically configured to: and if the floating paper sheet is detected to be contained in at least one frame of picture to be detected, determining that the floating paper sheet is contained in the video to be detected.
Further, the floating paper sheet detection module is specifically configured to: extracting image characteristics of each frame of picture to be detected aiming at a plurality of frames of pictures to be detected; and comparing the image characteristics of the frames of pictures to be detected, and if the pictures to be detected containing the same image characteristics exist, determining that at least two frames of the pictures to be detected contain floating paper pieces.
Further, the floating paper sheet detection module is specifically configured to: extracting feature points of a single frame of picture to be detected and adjacent feature points of the feature points; determining a characteristic region according to the similarity of the characteristic point and the adjacent characteristic points; and if the picture to be detected contains at least two characteristic regions, determining that the picture to be detected contains a floating paper sheet.
Further, the apparatus further comprises:
the image classifier training module is used for taking pictures known to contain floating paper sheets and/or pictures known not to contain floating paper sheets as training samples; labeling the training sample according to whether the training sample contains a floating paper sheet or not; training and learning the marked training samples by adopting a deep learning classification algorithm to obtain an image classifier;
the floating paper sheet detection module is specifically configured to: and inputting the at least one frame of picture to be detected into the image classifier, and determining a detection result in the at least one frame of picture to be detected according to a classification result of the image classifier.
In order to achieve the above object, according to still another aspect of the present disclosure, the following technical solutions are also provided:
a video floating paper detection hardware device, comprising:
a memory for storing non-transitory computer readable instructions; and
and the processor is used for executing the computer readable instructions, so that the processor can realize the steps in any one of the above technical schemes of the video floating paper sheet detection method when being executed.
In order to achieve the above object, according to still another aspect of the present disclosure, the following technical solutions are also provided:
a computer readable storage medium for storing non-transitory computer readable instructions which, when executed by a computer, cause the computer to perform the steps of any of the above-described video floating paper sheet detection method aspects.
In order to achieve the above object, according to still another aspect of the present disclosure, the following technical solutions are also provided:
a video floats paper detection terminal, includes any one video and floats paper detection device.
The disclosed embodiment provides a video floating paper detection method, a video floating paper detection device, a video floating paper detection hardware device, a computer readable storage medium and a video floating paper detection terminal. The video floating paper detection method comprises the steps of carrying out floating paper detection on at least one frame of picture to be detected extracted from a video to be detected, wherein the floating paper is a sub-display window which is inserted into the video to be detected and is irrelevant to the content of the video to be detected; and determining whether the video to be detected contains floating paper sheets or not according to the detection result of the at least one frame of picture to be detected. The method and the device for detecting the floating paper sheets in the video comprise the steps of firstly detecting the floating paper sheets of at least one frame of picture to be detected extracted from the video to be detected, wherein the floating paper sheets are sub-display windows which are inserted into the video to be detected and are irrelevant to the content of the video to be detected, and then determining whether the video to be detected contains the floating paper sheets or not according to the detection result of the at least one frame of picture to be detected, so that the video classification accuracy can be improved.
The foregoing is a summary of the present disclosure, and for the purposes of promoting a clear understanding of the technical means of the present disclosure, the present disclosure may be embodied in other specific forms without departing from the spirit or essential attributes thereof.
Drawings
FIG. 1a is a schematic flow diagram of a video floating paper sheet detection method according to one embodiment of the present disclosure;
FIG. 1b is a schematic flow diagram of a video floating paper sheet detection method according to another embodiment of the present disclosure;
FIG. 1c is a schematic flow diagram of a video floating paper sheet detection method according to another embodiment of the present disclosure;
FIG. 1d is a schematic flow diagram of a video floating paper sheet detection method according to another embodiment of the present disclosure;
FIG. 1e is a schematic flow diagram of a video floating paper sheet detection method according to another embodiment of the present disclosure;
FIG. 2a is a schematic diagram of an apparatus for video floating paper sheet detection according to one embodiment of the present disclosure;
FIG. 2b is a schematic diagram of a video floating paper sheet detection device according to another embodiment of the present disclosure;
FIG. 3 is a schematic diagram of a video floating paper sheet detection hardware device according to one embodiment of the present disclosure;
FIG. 4 is a schematic structural diagram of a computer-readable storage medium according to one embodiment of the present disclosure;
fig. 5 is a schematic structural diagram of a video floating paper sheet detection terminal according to an embodiment of the present disclosure.
Detailed Description
The embodiments of the present disclosure are described below with specific examples, and other advantages and effects of the present disclosure will be readily apparent to those skilled in the art from the disclosure in the specification. It is to be understood that the described embodiments are merely illustrative of some, and not restrictive, of the embodiments of the disclosure. The disclosure may be embodied or carried out in various other specific embodiments, and various modifications and changes may be made in the details within the description without departing from the spirit of the disclosure. It is to be noted that the features in the following embodiments and examples may be combined with each other without conflict. All other embodiments, which can be derived by a person skilled in the art from the embodiments disclosed herein without making any creative effort, shall fall within the protection scope of the present disclosure.
It is noted that various aspects of the embodiments are described below within the scope of the appended claims. It should be apparent that the aspects described herein may be embodied in a wide variety of forms and that any specific structure and/or function described herein is merely illustrative. Based on the disclosure, one skilled in the art should appreciate that one aspect described herein may be implemented independently of any other aspects and that two or more of these aspects may be combined in various ways. For example, an apparatus may be implemented and/or a method practiced using any number of the aspects set forth herein. Additionally, such an apparatus may be implemented and/or such a method may be practiced using other structure and/or functionality in addition to one or more of the aspects set forth herein.
It should be noted that the drawings provided in the following embodiments are only for illustrating the basic idea of the present disclosure, and the drawings only show the components related to the present disclosure rather than the number, shape and size of the components in actual implementation, and the type, amount and ratio of the components in actual implementation may be changed arbitrarily, and the layout of the components may be more complicated.
In addition, in the following description, specific details are provided to facilitate a thorough understanding of the examples. However, it will be understood by those skilled in the art that the aspects may be practiced without these specific details.
In order to solve the technical problem of inaccurate video classification, the embodiment of the disclosure provides a method for detecting a video floating paper sheet. As shown in fig. 1a, the video floating-paper detection method mainly includes the following steps S1 to S2. Wherein:
step S1: and detecting a floating paper sheet of at least one frame of picture to be detected extracted from the video to be detected, wherein the floating paper sheet is a sub-display window which is inserted into the video to be detected and is irrelevant to the content of the video to be detected.
The pictures to be detected can be one or more frames, and when the pictures to be detected are multiple frames, a single frame of the pictures to be detected is detected respectively, or the pictures to be detected are detected by comparing multiple frames of the pictures to be detected.
The sub-display window includes, but is not limited to, inserted advertisements, pornographic information, or handwritten text information.
Step S2: and determining whether the video to be detected contains floating paper sheets or not according to the detection result of at least one frame of picture to be detected.
The detection result includes, but is not limited to, that only one frame of the picture to be detected contains a floating paper sheet, or multiple frames of the picture to be detected contain a floating paper sheet, or no picture to be detected contains a floating paper sheet.
In the embodiment, the floating paper sheet detection is performed on at least one frame of picture to be detected extracted from the video to be detected, wherein the floating paper sheet is a sub-display window which is inserted into the video to be detected and is irrelevant to the content of the video to be detected, and then whether the video to be detected contains the floating paper sheet or not is determined according to the detection result of the at least one frame of picture to be detected, so that the video classification accuracy can be improved.
In an alternative embodiment, as shown in fig. 1b, step S2 specifically includes:
and if the floating paper is detected to be contained in at least one frame of picture to be detected, determining that the floating paper is contained in the video to be detected.
Specifically, when it is detected that only one frame of the to-be-detected picture contains a floating paper sheet, or a plurality of frames of the to-be-detected picture contain a floating paper sheet, it is determined that the to-be-detected video contains a floating paper sheet, and otherwise, it is determined that the to-be-detected video does not contain a floating paper sheet.
In the embodiment, the floating paper piece detection is performed on at least one frame of picture to be detected extracted from the video to be detected, the floating paper piece is a sub-display window which is inserted into the video to be detected and is irrelevant to the content of the video to be detected, and if the floating paper piece is detected to be contained in the at least one frame of picture to be detected, the floating paper piece is determined to be contained in the video to be detected, so that the video classification accuracy can be improved.
In an alternative embodiment, as shown in fig. 1c, step S1 specifically includes:
s11: and extracting the image characteristics of each frame of picture to be detected aiming at multiple frames of pictures to be detected.
The image features may be feature points of the picture to be detected or feature areas of the picture to be detected.
S12: and comparing the image characteristics of each frame of picture to be detected, and if the pictures to be detected containing the same image characteristics exist, determining that at least two frames of pictures to be detected contain floating paper pieces.
Specifically, when the image features are the feature points of the pictures to be detected, extracting shape context features and Scale-invariant feature transform (SIFT) features of the feature points, comparing similarity of the feature points among multiple frames of pictures to be detected according to the shape context features and the SIFT features of the feature points, obtaining a matching result of the similarity of the feature points among the pictures to be detected, and obtaining a matched feature region, wherein the feature region is the same image feature. This example can be used to detect a situation where the position of the floating paper sheet changes in each frame picture of the video.
When the image features are feature point regions of the pictures to be detected, comparing whether each frame of the pictures to be detected contains the same feature region, and specifically determining by adopting a pixel point matching method or a feature region similarity calculation method. This example can be used to detect a situation where the position of the floating paper sheet is fixed in each frame picture of the video.
In the embodiment, the image characteristics of each frame of the picture to be detected are extracted, the image characteristics of each frame of the picture to be detected are compared, and if the picture to be detected containing the same image characteristics exists, it is determined that at least two frames of the picture to be detected contain the floating paper, so that the video to be detected contains the floating paper, and the video classification accuracy can be improved.
In an alternative embodiment, as shown in fig. 1d, step S1 specifically includes:
s13: and extracting the characteristic points of the picture to be detected and the adjacent characteristic points of the characteristic points aiming at the single-frame picture to be detected.
The feature points may be SIFT feature points.
S14: and determining the characteristic region according to the similarity between the characteristic point and the adjacent characteristic point.
S15: and if the picture to be detected contains at least two characteristic areas, determining that the picture to be detected contains the floating paper sheet.
Specifically, according to the characteristics of the detected video, pixel points of a single-frame picture contained in the detected video have great relevance, while the inserted floating paper is often irrelevant to the video content, and the contained pixel points are also greatly different from the extracted pixel points of the single-frame picture.
In the embodiment, the feature points of the picture to be detected and the adjacent feature points of the feature points are extracted, the feature regions are determined according to the similarity between the feature points and the adjacent feature points, and if the picture to be detected contains at least two feature regions, the picture to be detected contains the floating paper, so that the video to be detected contains the floating paper, and the video classification accuracy can be improved.
In an alternative embodiment, as shown in fig. 1e, the method of this embodiment further includes:
s3: pictures known to contain a piece of floating paper and/or pictures known not to contain a piece of floating paper were used as training samples.
S4: the training samples were labeled according to whether they contained a floating paper sheet.
Specifically, before training, in order to distinguish between a picture containing a piece of floating paper and a picture not containing a piece of floating paper, each picture needs to be labeled. For example, a picture containing a piece of floating paper is labeled 1, and a picture not containing a piece of floating paper is labeled 0.
S5: and training and learning the marked training samples by adopting a deep learning classification algorithm to obtain the image classifier.
The deep learning classification algorithm that can be used includes, but is not limited to, any of the following: naive bayes algorithm, artificial neural network algorithm, genetic algorithm, K-nearest neighbor (KNN) classification algorithm, clustering algorithm, etc.
The step S1 specifically includes:
and inputting at least one frame of picture to be detected into an image classifier, and determining a detection result in the at least one frame of picture to be detected according to a classification result of the image classifier.
In the embodiment, the image classifier is trained, at least one frame of picture to be detected is input into the image classifier, and the detection result in the at least one frame of picture to be detected is determined according to the classification result of the image classifier, so that whether the video to be detected contains the floating paper pieces or not is determined according to the detection result of the at least one frame of picture to be detected, and the video classification accuracy can be improved.
It will be appreciated by those skilled in the art that obvious modifications (e.g., combinations of the enumerated modes) or equivalents may be made to the above-described embodiments.
In the above, although the steps in the embodiment of the video floating paper detection method are described in the above sequence, it should be clear to those skilled in the art that the steps in the embodiment of the present disclosure are not necessarily performed in the above sequence, and may also be performed in other sequences such as reverse, parallel, and cross, and further, on the basis of the above steps, those skilled in the art may also add other steps, and these obvious modifications or equivalents should also be included in the protection scope of the present disclosure, and are not described herein again.
For convenience of description, only the relevant parts of the embodiments of the present disclosure are shown, and details of the specific techniques are not disclosed, please refer to the embodiments of the method of the present disclosure.
In order to solve the technical problem of how to improve the user experience effect, the embodiment of the present disclosure provides a video floating paper detection device. The apparatus may perform the steps in the above-described video floating paper sheet detection method embodiments. As shown in fig. 2a, the apparatus mainly comprises: a floating paper piece detection module 21 and a floating paper piece determination module 22; the floating paper sheet detection module 21 is used for performing floating paper sheet detection on at least one frame of picture to be detected extracted from a video to be detected, wherein the floating paper sheet is a sub-display window which is inserted into the video to be detected and is irrelevant to the content of the video to be detected; the floating paper piece determining module 22 is configured to determine whether the video to be detected contains floating paper pieces according to a detection result of at least one frame of picture to be detected.
The pictures to be detected can be one or more frames, and when the pictures to be detected are multiple frames, a single frame of the pictures to be detected is detected respectively, or the pictures to be detected are detected by comparing multiple frames of the pictures to be detected.
The sub-display window includes, but is not limited to, inserted advertisements, pornographic information, or handwritten text information.
The detection result includes, but is not limited to, that only one frame of the picture to be detected contains a floating paper sheet, or multiple frames of the picture to be detected contain a floating paper sheet, or no picture to be detected contains a floating paper sheet.
In the embodiment, the floating paper sheet detection module 21 is used for performing floating paper sheet detection on at least one frame of picture to be detected extracted from the video to be detected, wherein the floating paper sheet is a sub-display window which is inserted into the video to be detected and is irrelevant to the content of the video to be detected, and then the floating paper sheet determination module 22 is used for determining whether the video to be detected contains the floating paper sheet or not according to the detection result of the at least one frame of picture to be detected, so that the video classification accuracy can be improved.
In an alternative embodiment, based on the illustration of fig. 2a, the floating paper determination module 22 is specifically configured to: and if the floating paper is detected to be contained in at least one frame of picture to be detected, determining that the floating paper is contained in the video to be detected.
Specifically, when the floating paper sheet detection module 21 detects that only one frame of the picture to be detected contains a floating paper sheet, or multiple frames of the picture to be detected contain a floating paper sheet, the floating paper sheet determination module 22 determines that the video to be detected contains a floating paper sheet, or determines that the video to be detected does not contain a floating paper sheet.
In the embodiment, the floating paper sheet detection module 21 is used for performing floating paper sheet detection on at least one frame of to-be-detected picture extracted from the to-be-detected video, the floating paper sheet is a sub-display window which is inserted into the to-be-detected video and is unrelated to the content of the to-be-detected video, and if the floating paper sheet determination module 22 detects that the at least one frame of to-be-detected picture contains the floating paper sheet, the floating paper sheet is determined to be contained in the to-be-detected video, so that the video classification accuracy can be improved.
In an alternative embodiment, based on the illustration in fig. 2a, the floating paper sheet detection module 21 is specifically configured to: extracting image characteristics of each frame of picture to be detected aiming at a plurality of frames of pictures to be detected; and comparing the image characteristics of each frame of picture to be detected, and if the pictures to be detected containing the same image characteristics exist, determining that at least two frames of pictures to be detected contain floating paper pieces.
The image features may be feature points of the picture to be detected or feature areas of the picture to be detected.
Specifically, when the image features are the feature points of the pictures to be detected, the shape context features and the SIFT features of the feature points are extracted, the similarity of the feature points among the multiple frames of pictures to be detected is compared according to the shape context features and the SIFT features of the feature points, the matching result of the similarity of the feature points among the pictures to be detected is obtained, and the matched feature regions are obtained, wherein the feature regions are the same image features. This example can be used to detect a situation where the position of the floating paper sheet changes in each frame picture of the video.
When the image features are feature point regions of the pictures to be detected, comparing whether each frame of the pictures to be detected contains the same feature region, and specifically determining by adopting a pixel point matching method or a feature region similarity calculation method. This example can be used to detect a situation where the position of the floating paper sheet is fixed in each frame picture of the video.
In the embodiment, the image characteristics of each frame of picture to be detected are extracted by the floating paper sheet detection module 21, the image characteristics of each frame of picture to be detected are compared, and if the picture to be detected containing the same image characteristics exists, at least two frames of pictures to be detected in the picture to be detected are determined to contain the floating paper sheets, so that the floating paper sheet determination module 22 is used for determining that the video to be detected contains the floating paper sheets, and the video classification accuracy can be improved.
In an alternative embodiment, based on the illustration in fig. 2a, the floating paper sheet detection module 21 is specifically configured to: extracting feature points of the picture to be detected and adjacent feature points of the feature points aiming at the single-frame picture to be detected; determining a characteristic region according to the similarity between the characteristic point and the adjacent characteristic point; and if the picture to be detected contains at least two characteristic areas, determining that the picture to be detected contains the floating paper sheet.
The feature points may be SIFT feature points.
Specifically, according to the characteristics of the detected video, pixel points of a single-frame picture contained in the detected video have great relevance, while the inserted floating paper is often irrelevant to the video content, and the contained pixel points are also greatly different from the extracted pixel points of the single-frame picture.
In this embodiment, the floating paper sheet detection module 21 is used to extract the feature points of the picture to be detected and the neighboring feature points of the feature points, the feature regions are determined according to the similarity between the feature points and the neighboring feature points, and if it is detected that the picture to be detected includes at least two feature regions, the floating paper sheet determination module 22 is used to determine that the picture to be detected includes the floating paper sheet, so that the video to be detected includes the floating paper sheet, and the video classification accuracy can be improved.
In an alternative embodiment, as shown in fig. 2b, the apparatus of this embodiment further includes: an image classifier training module 23; wherein, the image classifier training module 23 is used for taking pictures known to contain floating paper sheets and/or pictures known not to contain floating paper sheets as training samples; labeling the training sample according to whether the training sample contains the floating paper sheet or not; training and learning the marked training samples by adopting a deep learning classification algorithm to obtain an image classifier;
the floating paper piece detection module 21 is specifically configured to: and inputting at least one frame of picture to be detected into an image classifier, and determining a detection result in the at least one frame of picture to be detected according to a classification result of the image classifier.
Specifically, the image classifier training module 23 needs to label each picture before training in order to distinguish the picture containing the floating paper from the picture not containing the floating paper. For example, a picture containing a piece of floating paper is labeled 1, and a picture not containing a piece of floating paper is labeled 0.
The deep learning classification algorithm that can be used includes, but is not limited to, any of the following: naive bayes algorithm, artificial neural network algorithm, genetic algorithm, K-nearest neighbor (KNN) classification algorithm, clustering algorithm, etc.
In this embodiment, the image classifier training module 23 trains the image classifier, at least one frame of picture to be detected is input into the image classifier, and the detection result in the at least one frame of picture to be detected is determined according to the classification result of the image classifier, so that the floating paper piece determining module 22 determines whether the video to be detected contains floating paper pieces according to the detection result of the at least one frame of picture to be detected, thereby improving the video classification accuracy.
For detailed descriptions of the working principle, the realized technical effects, and the like of the embodiment of the video floating paper detection apparatus, reference may be made to the related descriptions in the foregoing embodiment of the video floating paper detection method, and further description is omitted here.
Fig. 3 is a hardware block diagram illustrating a video floating paper sheet detection hardware device according to an embodiment of the present disclosure. As shown in fig. 3, a video floating paper sheet detection hardware device 30 according to an embodiment of the present disclosure includes a memory 31 and a processor 32.
The memory 31 is used to store non-transitory computer readable instructions. In particular, memory 31 may include one or more computer program products that may include various forms of computer-readable storage media, such as volatile memory and/or non-volatile memory. The volatile memory may include, for example, Random Access Memory (RAM), cache memory (cache), and/or the like. The non-volatile memory may include, for example, Read Only Memory (ROM), hard disk, flash memory, etc.
The processor 32 may be a Central Processing Unit (CPU) or other form of processing unit having data processing capabilities and/or instruction execution capabilities, and may control other components in the video floating paper sheet detection hardware device 30 to perform desired functions. In one embodiment of the present disclosure, the processor 32 is configured to execute the computer readable instructions stored in the memory 31, so that the video floating paper sheet detection hardware device 30 performs all or part of the aforementioned steps of the video floating paper sheet detection method according to the embodiments of the present disclosure.
Those skilled in the art should understand that, in order to solve the technical problem of how to obtain a good user experience, the present embodiment may also include well-known structures such as a communication bus, an interface, and the like, and these well-known structures should also be included in the protection scope of the present disclosure.
For the detailed description of the present embodiment, reference may be made to the corresponding descriptions in the foregoing embodiments, which are not repeated herein.
Fig. 4 is a schematic diagram illustrating a computer-readable storage medium according to an embodiment of the present disclosure. As shown in fig. 4, a computer-readable storage medium 40, having non-transitory computer-readable instructions 41 stored thereon, in accordance with an embodiment of the present disclosure. When executed by a processor, the non-transitory computer readable instructions 41 perform all or part of the steps of the aforementioned method for matching video features according to the embodiments of the present disclosure.
The computer-readable storage medium 40 includes, but is not limited to: optical storage media (e.g., CD-ROMs and DVDs), magneto-optical storage media (e.g., MOs), magnetic storage media (e.g., magnetic tapes or removable disks), media with built-in rewritable non-volatile memory (e.g., memory cards), and media with built-in ROMs (e.g., ROM cartridges).
For the detailed description of the present embodiment, reference may be made to the corresponding descriptions in the foregoing embodiments, which are not repeated herein.
Fig. 5 is a diagram illustrating a hardware structure of a terminal according to an embodiment of the present disclosure. As shown in fig. 5, the video floating-paper detecting terminal 50 includes the above-described video floating-paper detecting apparatus embodiment.
The terminal may be implemented in various forms, and the terminal in the present disclosure may include, but is not limited to, mobile terminals such as a mobile phone, a smart phone, a notebook computer, a digital broadcast receiver, a PDA (personal digital assistant), a PAD (tablet computer), a PMP (portable multimedia player), a navigation device, a vehicle-mounted terminal, a vehicle-mounted display terminal, a vehicle-mounted electronic rear view mirror, etc., and fixed terminals such as a digital TV, a desktop computer, etc.
The terminal may also include other components as equivalent alternative embodiments. As shown in fig. 5, the video floating paper piece detecting terminal 50 may include a power supply unit 51, a wireless communication unit 52, an a/V (audio/video) input unit 53, a user input unit 54, a sensing unit 55, an interface unit 56, a controller 57, an output unit 58, a memory 59, and the like. Fig. 5 shows a terminal having various components, but it is to be understood that not all of the shown components are required to be implemented, and that more or fewer components may alternatively be implemented.
The wireless communication unit 52 allows, among other things, radio communication between the terminal 50 and a wireless communication system or network. The a/V input unit 53 is for receiving an audio or video signal. The user input unit 54 may generate key input data according to a command input by a user to control various operations of the terminal. The sensing unit 55 detects a current state of the terminal 50, a position of the terminal 50, presence or absence of a touch input of the terminal 50 by a user, an orientation of the terminal 50, acceleration or deceleration movement and direction of the terminal 50, and the like, and generates a command or signal for controlling an operation of the terminal 50. The interface unit 56 serves as an interface through which at least one external device is connected to the terminal 50. The output unit 58 is configured to provide output signals in a visual, audio, and/or tactile manner. The memory 59 may store software programs or the like for processing and controlling operations performed by the controller 55, or may temporarily store data that has been output or is to be output. The memory 59 may include at least one type of storage medium. Also, the terminal 50 may cooperate with a network storage device that performs a storage function of the memory 59 through a network connection. The controller 57 generally controls the overall operation of the terminal. In addition, the controller 57 may include a multimedia module for reproducing or playing back multimedia data. The controller 57 may perform a pattern recognition process to recognize a handwriting input or a picture drawing input performed on the touch screen as a character or an image. The power supply unit 51 receives external power or internal power and supplies appropriate power required to operate the respective elements and components under the control of the controller 57.
Various embodiments of the video feature comparison method presented in the present disclosure may be implemented using a computer-readable medium, such as computer software, hardware, or any combination thereof. For a hardware implementation, various embodiments of the comparison method of video features proposed by the present disclosure may be implemented by using at least one of an Application Specific Integrated Circuit (ASIC), a Digital Signal Processor (DSP), a Digital Signal Processing Device (DSPD), a Programmable Logic Device (PLD), a Field Programmable Gate Array (FPGA), a processor, a controller, a microcontroller, a microprocessor, an electronic unit designed to perform the functions described herein, and in some cases, various embodiments of the comparison method of video features proposed by the present disclosure may be implemented in the controller 57. For software implementation, various embodiments of the video feature comparison method presented in the present disclosure may be implemented with a separate software module that allows at least one function or operation to be performed. The software codes may be implemented by software applications (or programs) written in any suitable programming language, which may be stored in memory 59 and executed by controller 57.
For the detailed description of the present embodiment, reference may be made to the corresponding descriptions in the foregoing embodiments, which are not repeated herein.
The foregoing describes the general principles of the present disclosure in conjunction with specific embodiments, however, it is noted that the advantages, effects, etc. mentioned in the present disclosure are merely examples and are not limiting, and they should not be considered essential to the various embodiments of the present disclosure. Furthermore, the foregoing disclosure of specific details is for the purpose of illustration and description and is not intended to be limiting, since the disclosure is not intended to be limited to the specific details so described.
The block diagrams of devices, apparatuses, systems referred to in this disclosure are only given as illustrative examples and are not intended to require or imply that the connections, arrangements, configurations, etc. must be made in the manner shown in the block diagrams. These devices, apparatuses, devices, systems may be connected, arranged, configured in any manner, as will be appreciated by those skilled in the art. Words such as "including," "comprising," "having," and the like are open-ended words that mean "including, but not limited to," and are used interchangeably therewith. The words "or" and "as used herein mean, and are used interchangeably with, the word" and/or, "unless the context clearly dictates otherwise. The word "such as" is used herein to mean, and is used interchangeably with, the phrase "such as but not limited to".
Also, as used herein, "or" as used in a list of items beginning with "at least one" indicates a separate list, such that, for example, a list of "A, B or at least one of C" means A or B or C, or AB or AC or BC, or ABC (i.e., A and B and C). Furthermore, the word "exemplary" does not mean that the described example is preferred or better than other examples.
It is also noted that in the systems and methods of the present disclosure, components or steps may be decomposed and/or re-combined. These decompositions and/or recombinations are to be considered equivalents of the present disclosure.
Various changes, substitutions and alterations to the techniques described herein may be made without departing from the techniques of the teachings as defined by the appended claims. Moreover, the scope of the claims of the present disclosure is not limited to the particular aspects of the process, machine, manufacture, composition of matter, means, methods and acts described above. Processes, machines, manufacture, compositions of matter, means, methods, or acts, presently existing or later to be developed that perform substantially the same function or achieve substantially the same result as the corresponding aspects described herein may be utilized. Accordingly, the appended claims are intended to include within their scope such processes, machines, manufacture, compositions of matter, means, methods, or acts.
The previous description of the disclosed aspects is provided to enable any person skilled in the art to make or use the present disclosure. Various modifications to these aspects will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other aspects without departing from the scope of the disclosure. Thus, the present disclosure is not intended to be limited to the aspects shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
The foregoing description has been presented for purposes of illustration and description. Furthermore, this description is not intended to limit embodiments of the disclosure to the form disclosed herein. While a number of example aspects and embodiments have been discussed above, those of skill in the art will recognize certain variations, modifications, alterations, additions and sub-combinations thereof.
Claims (4)
1. A video floating paper detection method is characterized by comprising the following steps:
extracting image characteristics of each frame of picture to be detected aiming at a plurality of frames of pictures to be detected;
comparing the image characteristics of the frames of pictures to be detected, and if the pictures to be detected containing the same image characteristics exist, determining that at least two frames of the pictures to be detected contain floating paper sheets; alternatively, the first and second electrodes may be,
extracting feature points of a single frame of picture to be detected and adjacent feature points of the feature points;
determining a characteristic region according to the similarity of the characteristic point and the adjacent characteristic points;
if the picture to be detected contains at least two characteristic regions, determining that the picture to be detected contains a floating paper sheet; the floating paper sheet is a sub-display window which is inserted into the video to be detected and is irrelevant to the content of the video to be detected;
and if the floating paper sheet is detected to be contained in at least one frame of picture to be detected, determining that the floating paper sheet is contained in the video to be detected.
2. A video floating paper detection device, comprising:
the floating paper sheet detection module is used for extracting the image characteristics of each frame of picture to be detected aiming at a plurality of frames of pictures to be detected; comparing the image characteristics of the frames of pictures to be detected, and if the pictures to be detected containing the same image characteristics exist, determining that at least two frames of the pictures to be detected contain floating paper sheets; or, aiming at a single frame of picture to be detected, extracting feature points of the picture to be detected and adjacent feature points of the feature points; determining a characteristic region according to the similarity of the characteristic point and the adjacent characteristic points; if the picture to be detected contains at least two characteristic regions, determining that the picture to be detected contains a floating paper sheet; the floating paper sheet is a sub-display window which is inserted into the video to be detected and is irrelevant to the content of the video to be detected;
and the floating paper sheet determining module is used for determining that the video to be detected contains the floating paper sheet if the fact that at least one frame of picture to be detected contains the floating paper sheet is detected.
3. A video floating paper detection hardware device, comprising:
a memory for storing non-transitory computer readable instructions; and
a processor for executing the computer readable instructions such that the processor when executed implements the video floating paper sheet detection method of claim 1.
4. A computer readable storage medium storing non-transitory computer readable instructions which, when executed by a computer, cause the computer to perform the video floating paper sheet detection method of claim 1.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811068698.5A CN109064494B (en) | 2018-09-13 | 2018-09-13 | Video floating paper detection method and device and computer readable storage medium |
| PCT/CN2018/117711 WO2020052082A1 (en) | 2018-09-13 | 2018-11-27 | Video floating paperdetection method and device, and computer readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811068698.5A CN109064494B (en) | 2018-09-13 | 2018-09-13 | Video floating paper detection method and device and computer readable storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109064494A CN109064494A (en) | 2018-12-21 |
| CN109064494B true CN109064494B (en) | 2021-09-21 |
Family
ID=64761493
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811068698.5A Active CN109064494B (en) | 2018-09-13 | 2018-09-13 | Video floating paper detection method and device and computer readable storage medium |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN109064494B (en) |
| WO (1) | WO2020052082A1 (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111738173B (en) * | 2020-06-24 | 2023-07-25 | 北京奇艺世纪科技有限公司 | Video clip detection method and device, electronic equipment and storage medium |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101183430A (en) * | 2007-12-13 | 2008-05-21 | 中国科学院合肥物质科学研究院 | Handwriting digital automatic identification method based on module neural network SN9701 rectangular array |
| CN101853398A (en) * | 2010-05-11 | 2010-10-06 | 浙江大学 | Chinese paper cutting identification method based on space constraint characteristic selection and combination thereof |
| CN105184226A (en) * | 2015-08-11 | 2015-12-23 | 北京新晨阳光科技有限公司 | Digital identification method, digital identification device, neural network training method and neural network training device |
| CN107679552A (en) * | 2017-09-11 | 2018-02-09 | 北京飞搜科技有限公司 | A kind of scene classification method and system based on multiple-limb training |
| CN108288077A (en) * | 2018-04-17 | 2018-07-17 | 天津和或节能科技有限公司 | Grading of old paper device establishes device and method, grading of old paper system and method |
Family Cites Families (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101453575B (en) * | 2007-12-05 | 2010-07-21 | 中国科学院计算技术研究所 | Video subtitle information extracting method |
| US9202137B2 (en) * | 2008-11-13 | 2015-12-01 | Google Inc. | Foreground object detection from multiple images |
| CN101448100B (en) * | 2008-12-26 | 2011-04-06 | 西安交通大学 | Method for extracting video captions quickly and accurately |
| CN103186780B (en) * | 2011-12-30 | 2018-01-26 | 乐金电子(中国)研究开发中心有限公司 | Video caption recognition methods and device |
| CN104966097B (en) * | 2015-06-12 | 2019-01-18 | 成都数联铭品科技有限公司 | A kind of complex script recognition methods based on deep learning |
| CN105718861B (en) * | 2016-01-15 | 2019-06-07 | 北京市博汇科技股份有限公司 | A kind of method and device identifying video stream data classification |
| US9996772B2 (en) * | 2016-04-28 | 2018-06-12 | International Business Machines Corporation | Detection of objects in images using region-based convolutional neural networks |
| CN108038850A (en) * | 2017-12-08 | 2018-05-15 | 天津大学 | A kind of drainage pipeline Exception Type automatic testing method based on deep learning |
| CN107948640B (en) * | 2017-12-19 | 2020-09-25 | 百度在线网络技术（北京）有限公司 | Video playing test method and device, electronic equipment and storage medium |
-
2018
- 2018-09-13 CN CN201811068698.5A patent/CN109064494B/en active Active
- 2018-11-27 WO PCT/CN2018/117711 patent/WO2020052082A1/en active Application Filing
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101183430A (en) * | 2007-12-13 | 2008-05-21 | 中国科学院合肥物质科学研究院 | Handwriting digital automatic identification method based on module neural network SN9701 rectangular array |
| CN101853398A (en) * | 2010-05-11 | 2010-10-06 | 浙江大学 | Chinese paper cutting identification method based on space constraint characteristic selection and combination thereof |
| CN105184226A (en) * | 2015-08-11 | 2015-12-23 | 北京新晨阳光科技有限公司 | Digital identification method, digital identification device, neural network training method and neural network training device |
| CN107679552A (en) * | 2017-09-11 | 2018-02-09 | 北京飞搜科技有限公司 | A kind of scene classification method and system based on multiple-limb training |
| CN108288077A (en) * | 2018-04-17 | 2018-07-17 | 天津和或节能科技有限公司 | Grading of old paper device establishes device and method, grading of old paper system and method |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109064494A (en) | 2018-12-21 |
| WO2020052082A1 (en) | 2020-03-19 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109165301B (en) | Video cover selection method, device and computer readable storage medium | |
| Shi et al. | Script identification in the wild via discriminative convolutional neural network | |
| CN108121816B (en) | Picture classification method and device, storage medium and electronic equipment | |
| US8965115B1 (en) | Adaptive multi-modal detection and fusion in videos via classification-based-learning | |
| Escalante et al. | A naive bayes baseline for early gesture recognition | |
| US9691004B2 (en) | Device and method for service provision according to prepared reference images to detect target object | |
| US10152540B2 (en) | Linking thumbnail of image to web page | |
| JP5063632B2 (en) | Learning model generation apparatus, object detection system, and program | |
| CN111598012B (en) | Picture clustering management method, system, device and medium | |
| CN111209827A (en) | OCR (optical character recognition) bill problem recognition method and system based on feature detection | |
| CN104750791A (en) | Image retrieval method and device | |
| US20150269135A1 (en) | Language identification for text in an object image | |
| TW201546636A (en) | Annotation display assistance device and method of assisting annotation display | |
| CN112861842A (en) | Case text recognition method based on OCR and electronic equipment | |
| US20160104052A1 (en) | Text-based thumbnail generation | |
| CN109064494B (en) | Video floating paper detection method and device and computer readable storage medium | |
| KR101773064B1 (en) | Method and apparatus for increasing the readability of the image type contents | |
| CN104636059B (en) | The searching method and its system of account project | |
| WO2020052085A1 (en) | Video text detection method and device, and computer readable storage medium | |
| CN110795918B (en) | Method, device and equipment for determining reading position | |
| US20210374189A1 (en) | Document search device, document search program, and document search method | |
| CN101582118A (en) | Dictionary creating apparatus, recognizing apparatus, and recognizing method | |
| US20160283520A1 (en) | Search device, search method, and computer program product | |
| US20160092729A1 (en) | Information processing device, information processing method, and computer program product | |
| KR101951706B1 (en) | Method, and apparatus for increasing the readability of the image type contents |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CP01 | Change in the name or title of a patent holder |
Address after: 100041 B-0035, 2 floor, 3 building, 30 Shixing street, Shijingshan District, Beijing. Patentee after: Douyin Vision Co.,Ltd. Address before: 100041 B-0035, 2 floor, 3 building, 30 Shixing street, Shijingshan District, Beijing. Patentee before: Tiktok vision (Beijing) Co.,Ltd. Address after: 100041 B-0035, 2 floor, 3 building, 30 Shixing street, Shijingshan District, Beijing. Patentee after: Tiktok vision (Beijing) Co.,Ltd. Address before: 100041 B-0035, 2 floor, 3 building, 30 Shixing street, Shijingshan District, Beijing. Patentee before: BEIJING BYTEDANCE NETWORK TECHNOLOGY Co.,Ltd. |
|
| CP01 | Change in the name or title of a patent holder |