CN111131878A - Video preprocessing and frame loss detection method and device, electronic equipment and storage medium - Google Patents
Video preprocessing and frame loss detection method and device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN111131878A CN111131878A CN201911413278.0A CN201911413278A CN111131878A CN 111131878 A CN111131878 A CN 111131878A CN 201911413278 A CN201911413278 A CN 201911413278A CN 111131878 A CN111131878 A CN 111131878A
- Authority
- CN
- China
- Prior art keywords
- video frame
- anchor point
- current video
- video
- frame
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 48
- 238000007781 pre-processing Methods 0.000 title claims abstract description 42
- 238000003860 storage Methods 0.000 title claims abstract description 17
- 238000012545 processing Methods 0.000 claims abstract description 86
- 238000000034 method Methods 0.000 claims abstract description 53
- 230000008569 process Effects 0.000 claims abstract description 18
- 230000015654 memory Effects 0.000 claims description 19
- 238000000605 extraction Methods 0.000 claims description 13
- 239000000126 substance Substances 0.000 claims description 10
- 238000004364 calculation method Methods 0.000 claims description 4
- 239000000284 extract Substances 0.000 description 24
- 230000009286 beneficial effect Effects 0.000 description 14
- 238000010586 diagram Methods 0.000 description 12
- 230000008901 benefit Effects 0.000 description 7
- 238000011156 evaluation Methods 0.000 description 7
- 230000006870 function Effects 0.000 description 7
- 238000005457 optimization Methods 0.000 description 7
- 230000035945 sensitivity Effects 0.000 description 7
- 238000004891 communication Methods 0.000 description 6
- 238000004590 computer program Methods 0.000 description 6
- 230000000694 effects Effects 0.000 description 6
- 238000007405 data analysis Methods 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 4
- 238000012986 modification Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 230000003993 interaction Effects 0.000 description 2
- 239000004973 liquid crystal related substance Substances 0.000 description 2
- 230000000717 retained effect Effects 0.000 description 2
- 230000002194 synthesizing effect Effects 0.000 description 2
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 230000001413 cellular effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000010295 mobile communication Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000001151 other effect Effects 0.000 description 1
- 238000009877 rendering Methods 0.000 description 1
- 230000001953 sensory effect Effects 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000012549 training Methods 0.000 description 1
- 238000012795 verification Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/431—Generation of visual interfaces for content selection or interaction; Content or additional data rendering
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/60—Network structure or processes for video distribution between server and client or between remote clients; Control signalling between clients, server and network components; Transmission of management data between server and client, e.g. sending from server to client commands for recording incoming content stream; Communication details between server and client
- H04N21/63—Control signaling related to video distribution between client, server and network components; Network processes for video distribution between server and clients or between remote clients, e.g. transmitting basic layer and enhancement layers over different transmission paths, setting up a peer-to-peer communication via Internet between remote STB's; Communication protocols; Addressing
- H04N21/647—Control signaling between network components and server or clients; Network processes for video distribution between server and clients, e.g. controlling the quality of the video stream, by dropping packets, protecting content from unauthorised alteration within the network, monitoring of network load, bridging between two different networks, e.g. between IP and wireless
- H04N21/64784—Data processing by the network
- H04N21/64792—Controlling the complexity of the content stream, e.g. by dropping packets
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Computer Networks & Wireless Communication (AREA)
- Computer Security & Cryptography (AREA)
- Image Analysis (AREA)
Abstract
The application discloses a video preprocessing and frame loss detection method and device, electronic equipment and a storage medium, and relates to the field of computer vision. The video preprocessing scheme is as follows: extracting a video frame from a video to be processed as a current video frame, and determining an anchor point area in the current video frame; allocating anchor point information for marking the current video frame to the anchor point area of the current video frame; adding anchor point information into an anchor point area of the current video frame, and repeatedly executing the operation until each video frame is added with the anchor point information corresponding to the video frame; and combining all the video frames added with the anchor point information corresponding to the video frames into a new video to be processed, and sending the new video to be processed to the video processing equipment so that the video processing equipment processes the new video to be processed. The embodiment of the application can automatically detect the frame loss of the video, has high accuracy and strong reliability, and can effectively reduce the labor cost and the time cost.
Description
Technical Field
The present application relates to the field of image processing technologies, and further relates to computer vision technologies, and in particular, to a method and an apparatus for video preprocessing and frame loss detection, an electronic device, and a storage medium.
Background
Short videos have become a big trend of content creation at present, and apps of many mobile devices have short video production capability. Because the processing modes of the APP video coding and decoding are different, various problems exist in the produced video, and one of the problems is video frame loss. The video frame loss detection technology is a detection technology for comparing a generated new video file with a repeated frame and a lost frame existing in an original video after the original video is subjected to video editing and synthesizing processing of local equipment (or cloud equipment). At present, there are no detection methods for video frame sequence verification in network transmission, no-reference frame loss detection in network video environment, and frame loss detection in video playing and rendering in the industry, but there is no detection method for the influence of video processing tools on the frame sequence of the original video.
At present, no automatic detection method for frame loss caused by video editing and synthesizing processes exists in the industry. At present, whether a video meets the requirement of human eyes on time sensitivity or not is mainly identified by human eyes in the industry, so that whether the video is smooth or not is judged, and whether a frame loss condition exists or not is judged. Under modern video standards, the frame rate of television, movie and network video reaches 24 frames per second and even higher. The difference between two adjacent video images is difficult to distinguish only by human eyes; under the condition that the video is blocked, the fact that the frame is lost or a plurality of repeated frames continuously appear cannot be distinguished; and the frame loss condition can not be accurately quantified only by subjective evaluation and judgment of human eyes, which is not beneficial to further optimization of the products such as the shooting device.
Disclosure of Invention
In view of this, embodiments of the present application provide a method, an apparatus, an electronic device, and a storage medium for video preprocessing and frame loss detection, which can automatically perform frame loss detection on a video, and have high accuracy and strong reliability, thereby effectively reducing labor cost and time cost.
In a first aspect, an embodiment of the present application provides a video preprocessing method, which is applied to a front-end device, and the method includes:
extracting a video frame from a video to be processed as a current video frame, and determining an anchor point area in the current video frame;
allocating anchor point information for marking the current video frame to the anchor point area of the current video frame;
adding the anchor point information into an anchor point area of the current video frame, and repeatedly executing the operation of extracting the video frame until each video frame in the video to be processed is added with the anchor point information corresponding to the video frame;
and combining all the video frames added with the anchor point information corresponding to the video frames into a new video to be processed, and sending the new video to be processed to video processing equipment so that the video processing equipment processes the new video to be processed.
The above embodiment has the following advantages or beneficial effects: before the video is processed by the video processing device, the front-end device may add anchor point information for marking the video frame in an anchor point region of each video frame, and after the video is processed by the video processing device, the back-end device may extract the anchor point information for marking the video frame in each video frame, so that whether the current video frame has a frame loss or not may be detected according to the anchor point information of the current video frame and the anchor point information extracted from the previous video frame. Because the embodiment adopts the technical means that the anchor point information used for marking the current video frame is added in the anchor point area of each video frame and whether the video frame has frame loss or not is detected according to the anchor point information extracted from the anchor point area of each video frame, the technical problem of inaccurate observation by human eyes in the prior art is solved.
In the above embodiment, the determining an anchor point region in the current video frame includes:
determining the coordinates of a fixed point in the anchor point region in the current video frame;
and determining the anchor point region in the current video frame according to the coordinates of the fixed point and the predetermined shape information of the anchor point region.
The above embodiment has the following advantages or beneficial effects: in the above embodiment, the coordinates of a fixed point are determined in the current video frame, and then the anchor point area is determined in the current video frame according to the shape information of the anchor point area determined in advance by using the coordinates of the fixed point as a reference point. The coordinates of the fixed point can be obtained by predetermining the pixel point position of the display interface, and the shape information of the anchor point region can also be obtained by predetermining, so that the anchor point region can be determined quickly and accurately, and the anchor point information corresponding to the current video frame can be added in the anchor point region.
In a second aspect, an embodiment of the present application provides a frame loss detection method, which is applied to a back-end device, and the method includes:
receiving a processed video sent by video processing equipment;
extracting a video frame from the processed video as a current video frame, and determining an anchor point area in the current video frame;
extracting anchor point information used for marking the current video frame from the anchor point area of the current video frame;
detecting whether the current video frame has a frame loss or not according to the anchor point information of the current video frame and the anchor point information of the previous video frame; and repeating the operation of extracting the video frames until detecting whether each video frame in the processed video has a frame loss.
The above embodiment has the following advantages or beneficial effects: after the video is processed by the video processing device, the back-end device can extract anchor point information for marking the video frame from each video frame, so that whether the current video frame has a frame loss or not can be detected according to the anchor point information of the current video frame and the anchor point information extracted from the previous video frame. Because the embodiment adopts the technical means that the anchor point information used for marking the current video frame is added in the anchor point area of each video frame and whether the video frame has frame loss or not is detected according to the anchor point information extracted from the anchor point area of each video frame, the technical problem of inaccurate observation by human eyes in the prior art is solved.
In the above embodiment, the determining an anchor point region in the current video frame includes:
acquiring the transverse resolution and the longitudinal resolution of the current video frame and the transverse resolution and the longitudinal resolution of an original video frame corresponding to the current video frame;
calculating a horizontal resolution factor and a vertical resolution factor of the current video frame according to the horizontal resolution and the vertical resolution of the current video frame and the horizontal resolution and the vertical resolution of an original video frame corresponding to the current video frame;
and determining the anchor point region in the current video frame according to the coordinates of the fixed point in the anchor point region, the shape information of the anchor point region and the transverse resolution factor and the longitudinal resolution factor of the current video frame, which are determined in advance.
The above embodiment has the following advantages or beneficial effects: in the above embodiment, the horizontal resolution factor and the vertical resolution factor of the current video frame are calculated first, and then the anchor point region is determined in the current video frame according to the coordinates of the fixed point in the anchor point region and the shape information of the anchor point region, as well as the horizontal resolution factor and the vertical resolution factor of the current video frame. Because the horizontal resolution and the vertical resolution of the current video frame are changed after the video is processed by the video processing device, the anchor point region can be more accurately determined in the current video frame by calculating the horizontal resolution factor and the vertical resolution factor of the current video frame, so that the anchor point information corresponding to the current video frame can be extracted from the anchor point region.
In the above embodiment, the extracting anchor point information for marking the current video frame from the anchor point region of the current video frame includes:
if the anchor point area of the current video frame is a color image, converting the anchor point area of the current video frame into a gray image;
and extracting anchor point information used for marking the current video frame from the gray-scale image.
The above embodiment has the following advantages or beneficial effects: in the above embodiment, the current video frame is first converted from the color image to the gray image, and then the anchor point information for marking the current video frame is extracted from the gray image. Because the pixel range of the gray-scale image is less than that of the color image, anchor point information can be extracted more easily from the gray-scale image, so that the calculation complexity can be reduced, and the extraction efficiency can be improved.
In a third aspect, the present application further provides a video preprocessing apparatus, including: the device comprises a determining module, a distributing module, an adding module and a sending module; wherein the content of the first and second substances,
the determining module is used for extracting a video frame from a video to be processed as a current video frame and determining an anchor point area in the current video frame;
the allocation module is used for allocating anchor point information for marking the current video frame to the anchor point area of the current video frame;
the adding module is used for adding the anchor point information into the anchor point area of the current video frame and repeatedly executing the operation of extracting the video frame until each video frame in the video to be processed is added with the anchor point information corresponding to the video frame;
the sending module is configured to combine all video frames to which anchor point information corresponding to the video frames is added into a new video to be processed, and send the new video to be processed to video processing equipment, so that the video processing equipment processes the new video to be processed.
In the foregoing embodiment, the determining module is specifically configured to determine, in the current video frame, coordinates of a fixed point in the anchor point region; and determining the anchor point region in the current video frame according to the coordinates of the fixed point and the predetermined shape information of the anchor point region.
In a fourth aspect, the present application further provides a frame loss detection apparatus, including: the device comprises a receiving module, an extracting module and a detecting module; wherein the content of the first and second substances,
the receiving module is used for receiving the processed video sent by the video processing equipment;
the extraction module is used for extracting a video frame from the processed video as a current video frame, and determining an anchor point area in the current video frame; extracting anchor point information used for marking the current video frame from the anchor point area of the current video frame;
the detection module is used for detecting whether the current video frame has a frame loss or not according to the anchor point information of the current video frame and the anchor point information of the previous video frame; and repeating the operation of extracting the video frames until detecting whether each video frame in the processed video has a frame loss.
In the above embodiment, the extraction module includes: the method comprises the following steps of obtaining a submodule, a calculating submodule and a determining submodule; wherein the content of the first and second substances,
the obtaining submodule is used for obtaining the transverse resolution and the longitudinal resolution of the current video frame and the transverse resolution and the longitudinal resolution of the original video frame corresponding to the current video frame;
the calculation submodule is used for calculating a transverse resolution factor and a longitudinal resolution factor of the current video frame according to the transverse resolution and the longitudinal resolution of the current video frame and the transverse resolution and the longitudinal resolution of the original video frame corresponding to the current video frame;
the determining submodule is configured to determine the anchor point region in the current video frame according to the coordinates of the fixed point in the anchor point region and the shape information of the anchor point region, and the horizontal resolution factor and the vertical resolution factor of the current video frame, which are predetermined.
In the above embodiment, the extraction module is specifically configured to convert the anchor point region of the current video frame into a gray image if the anchor point region of the current video frame is a color image; and extracting anchor point information used for marking the current video frame from the gray-scale image.
In a fifth aspect, an embodiment of the present application provides an electronic device, including:
one or more processors;
a memory for storing one or more programs,
when executed by the one or more processors, cause the one or more processors to implement the video pre-processing or frame loss detection method of any embodiment of the present application.
In a sixth aspect, the present application provides a storage medium, on which a computer program is stored, where the computer program, when executed by a processor, implements the video preprocessing or frame loss detection method according to any embodiment of the present application.
One embodiment in the above application has the following advantages or benefits: according to the video preprocessing and frame loss detection method, device, electronic equipment and storage medium, the front-end equipment can extract one video frame from the video frames to be processed as a current video frame, and an anchor point area is determined in the current video frame; allocating anchor point information for marking the current video frame to the anchor point area of the current video frame; adding anchor point information into an anchor point area of a current video frame, and repeatedly executing the operation of extracting the video frame until each video frame in the video to be processed is added with the anchor point information corresponding to the video frame; and combining all the video frames added with the anchor point information corresponding to the video frames into a new video to be processed, and sending the new video to be processed to the video processing equipment so that the video processing equipment processes the new video to be processed. Correspondingly, the back-end device can receive the processed video frame sent by the video processing device, extract a video frame from the processed video as the current video frame, and determine the anchor point area in the current video frame; extracting anchor point information used for marking the current video frame from an anchor point area of the current video frame; determining whether the current video frame has a frame loss according to the anchor point information of the current video frame and the anchor point information of the previous video frame; and repeating the operation of extracting the video frames until detecting whether each video frame in the processed video has a frame loss. That is to say, before the video is processed by the video processing device, the front-end device may add anchor point information for marking the video frame in an anchor point region of each video frame, and after the video is processed by the video processing device, the back-end device may extract anchor point information for marking the video frame in each video frame, so that whether the current video frame has a frame loss or not may be detected according to the anchor point information of the current video frame and the anchor point information extracted from the previous video frame. In the prior art, whether the video meets the requirement of human eyes on time sensitivity is mainly identified through human eyes, so that whether the video is smooth or not and whether a frame loss condition exists or not are judged, but the difference between two adjacent frames of video images is difficult to distinguish only by the human eyes; under the condition that the video is blocked, the fact that the frame is lost or a plurality of repeated frames continuously appear cannot be distinguished; and the frame loss condition can not be accurately quantified only by subjective evaluation and judgment of human eyes, which is not beneficial to further optimization of the products such as the shooting device. Because the technical means that the anchor point information used for marking the current video frame is added in the anchor point area of each video frame and whether the video frame has frame loss or not is detected according to the anchor point information extracted from the anchor point area of each video frame is adopted, the technical problem that the video frame is not accurately observed through human eyes in the prior art is solved, the technical effect of detecting the frame loss of the video can be automatically realized by adopting the technical scheme provided by the application, the accuracy is high, the reliability is high, and the labor cost and the time cost can be effectively reduced; moreover, the technical scheme of the embodiment of the application is simple and convenient to implement, convenient to popularize and wide in application range.
Other effects of the above-described alternative will be described below with reference to specific embodiments.
Drawings
The drawings are included to provide a better understanding of the present solution and are not intended to limit the present application. Wherein:
fig. 1 is a schematic flowchart of a video preprocessing method according to an embodiment of the present application;
fig. 2 is a schematic flowchart of a video preprocessing method according to a second embodiment of the present application;
fig. 3 is a schematic structural diagram of an anchor point region provided in the second embodiment of the present application;
fig. 4 is a schematic flowchart of a frame loss detection method according to a third embodiment of the present application;
fig. 5 is a schematic flowchart of a frame loss detection method according to the fourth embodiment of the present application;
fig. 6 is a schematic structural diagram of an anchor point region after clipping according to the fourth embodiment of the present application;
fig. 7 is a schematic structural diagram of a video preprocessing apparatus according to a fifth embodiment of the present application;
fig. 8 is a schematic structural diagram of a frame loss detection apparatus according to a sixth embodiment of the present application;
fig. 9 is a schematic structural diagram of an extraction module provided in a sixth embodiment of the present application;
fig. 10 is a block diagram of an electronic device for implementing the video preprocessing and frame loss detection method according to the embodiment of the present application.
Detailed Description
The following description of the exemplary embodiments of the present application, taken in conjunction with the accompanying drawings, includes various details of the embodiments of the application for the understanding of the same, which are to be considered exemplary only. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the present application. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.
Example one
Fig. 1 is a schematic flowchart of a video preprocessing method according to an embodiment of the present application. The method can be executed by a video preprocessing device or a front-end device, the device or the front-end device can be realized by software and/or hardware, and the device or the front-end device can be integrated in any intelligent device with network communication function. As shown in fig. 1, the video pre-processing method may include the steps of:
s101, extracting a video frame from a video to be processed as a current video frame, and determining an anchor point area in the current video frame.
In a specific embodiment of the present application, it is assumed that a video to be processed includes N video frames; respectively as follows: video frame 0, video frame 1, …, video frame N-1; wherein N is a natural number of 1 or more. In this step, the front-end device may extract a video frame from the video to be processed as the current video frame. For example, the front-end device may extract video frame 0 from the N video frames as a current video frame, and determine an anchor point region in the current video frame. Specifically, the front-end device may first determine a coordinate of a fixed point in the anchor point region in the current video frame; and then determining the anchor point area in the current video frame according to the coordinates of the fixed point and the shape information of the anchor point area determined in advance. For example, assume that an anchor point region of a current video frame is a rectangular region, a fixed point in the anchor point region is a vertex at the upper left corner of the rectangular region, coordinates of the fixed point are (x, y), and shape information of the anchor point region determined in advance is (w, h); wherein w is the width of the rectangular region, and h is the height of the rectangular region. Therefore, in this step, the front-end device may determine the rectangular region Rect (x, y, w, h) according to the vertex coordinates of the upper left corner of the rectangular region and the width and height of the rectangular region.
S102, allocating anchor point information for marking the current video frame for the anchor point area of the current video frame.
In a specific embodiment of the present application, the front-end device may allocate anchor point information for marking the current video frame to the anchor point region of the current video frame. Specifically, the head-end device may assign a consecutive number to the anchor region of the current video frame. For example, the front-end device may assign number 0 to the anchor region of video frame 0; allocating a number 1 to an anchor point region of a video frame 1; and so on.
S103, adding the anchor point information into the anchor point area of the current video frame, and repeatedly executing the operation of extracting the video frame until each video frame in the video to be processed is added with the anchor point information corresponding to the video frame.
In a specific embodiment of the present application, the front-end device may add anchor point information to an anchor point region of a current video frame, and repeatedly perform the above operation of extracting video frames until each video frame in the video to be processed is added with anchor point information corresponding thereto. Specifically, the front-end device may add number 0 to the anchor region of video frame 0; number 1 is added to the anchor region of video frame 1; number 2 is added to the anchor region of video frame 2; the above operations are repeatedly executed until each video frame in the video to be processed is added with the number corresponding to the video frame.
And S104, combining all the video frames added with the anchor point information corresponding to the video frames into a new video to be processed, and sending the new video to be processed to the video processing equipment so that the video processing equipment processes the new video to be processed.
In a specific embodiment of the present application, the front-end device may combine all video frames to which anchor point information corresponding to the video frames has been added into a new video to be processed, and send the new video to be processed to the video processing device, so that the video processing device processes the new video to be processed. Specifically, it is assumed that the video to be processed includes N video frames, which are respectively: video frame 0, video frame 1, …, video frame N-1; wherein N is a natural number of 1 or more. Assume that video frame 0 has anchor information 0 added thereto, video frame 1 has anchor information 1, … added thereto, and video frame N-1 has anchor information N-1 added thereto. In this step, the front-end device may combine the video frame 0 to which the anchor information 0 has been added, the video frames 1 and … to which the anchor information 1 has been added, and the video frame N-1 to which the anchor information N-1 has been added into a new video to be processed, and send the new video to be processed to the video processing device.
According to the video preprocessing method provided by the embodiment of the application, the front-end equipment can extract one video frame from the video frames to be processed as the current video frame, and an anchor point area is determined in the current video frame; allocating anchor point information for marking the current video frame to the anchor point area of the current video frame; adding anchor point information into an anchor point area of a current video frame, and repeatedly executing the operation of extracting the video frame until each video frame in the video to be processed is added with the anchor point information corresponding to the video frame; and combining all the video frames added with the anchor point information corresponding to the video frames into a new video to be processed, and sending the new video to be processed to the video processing equipment so that the video processing equipment processes the new video to be processed. In the prior art, whether the video meets the requirement of human eyes on time sensitivity is mainly identified through human eyes, so that whether the video is smooth or not and whether a frame loss condition exists or not are judged, but the difference between two adjacent frames of video images is difficult to distinguish only by the human eyes; under the condition that the video is blocked, the fact that the frame is lost or a plurality of repeated frames continuously appear cannot be distinguished; and the frame loss condition can not be accurately quantified only by subjective evaluation and judgment of human eyes, which is not beneficial to further optimization of the products such as the shooting device. Because the technical means that the anchor point information used for marking the current video frame is added in the anchor point area of each video frame and whether the video frame has frame loss or not is detected according to the anchor point information extracted from the anchor point area of each video frame is adopted, the technical problem that the video frame is not accurately observed through human eyes in the prior art is solved, the technical effect of detecting the frame loss of the video can be automatically realized by adopting the technical scheme provided by the application, the accuracy is high, the reliability is high, and the labor cost and the time cost can be effectively reduced; moreover, the technical scheme of the embodiment of the application is simple and convenient to implement, convenient to popularize and wide in application range.
Example two
Fig. 2 is a schematic flowchart of a video preprocessing method according to a second embodiment of the present application. As shown in fig. 2, the video pre-processing method may include the steps of:
s201, extracting a video frame from a video to be processed as a current video frame, and determining the coordinate of a fixed point in an anchor point area in the current video frame.
In a specific embodiment of the present application, the front-end device may extract a video frame from a video to be processed as a current video frame, and determine coordinates of a fixed point in an anchor point region in the current video frame. Specifically, the front-end device may use a pixel point in the current video frame as a reference point, and then determine the coordinate of a fixed point in the anchor point region according to the reference point. For example, the front-end device may use a pixel point at the center of the current video frame as a reference point, and then determine the coordinates of the fixed point in the current video frame according to the predetermined deviation distance and deviation angle.
S202, according to the coordinates of the fixed point and the shape information of the anchor point area which is determined in advance, the anchor point area is determined in the current video frame.
In a specific embodiment of the present application, the front-end device may determine the anchor point region in the current video frame according to the coordinates of the fixed point and the predetermined shape information of the anchor point region. For example, assume that an anchor point region of a current video frame is a rectangular region, a fixed point in the anchor point region is a vertex at the upper left corner of the rectangular region, coordinates of the fixed point are (x1, y1), and shape information of the anchor point region determined in advance is (w, h); wherein w is the width of the rectangular region, and h is the height of the rectangular region. In this step, the front-end device may determine the rectangular region Rect (x1, y1, w, h) according to the coordinates of the top left corner of the rectangular region and the width and height of the rectangular region. For another example, assume that the anchor point region of the current video frame is a rectangular region, the fixed point in the anchor point region is the top right corner fixed point of the rectangular region, the coordinates of the fixed point are (x2, y2), and the shape information of the anchor point region determined in advance is (w, h); wherein w is the width of the rectangular region, and h is the height of the rectangular region. In this step, the front-end device may determine the rectangular region Rect (x2, y2, w, h) according to the coordinates of the top-right corner of the rectangular region and the width and height of the rectangular region. For another example, assume that the anchor point region of the current video frame is a circular region, the fixed point in the anchor point region is the center of the circular region, the coordinates of the fixed point are (x3, y3), and the radius of the anchor point region determined in advance is R. In this step, the front-end device may determine the circular region Rect (x3, y3, R) according to the coordinates of the center of the circular region and the radius of the circular region. In a specific implementation, there may be many implementations, and the implementations are not limited to the three implementations described above.
S203, allocating an anchor point information for marking the current video frame to the anchor point region of the current video frame.
In a specific embodiment of the present application, the front-end device may allocate anchor point information for marking the current video frame to the anchor point region of the current video frame. Fig. 3 is a schematic structural diagram of an anchor point region provided in the second embodiment of the present application. As shown in fig. 3, the anchor point region may be at the upper left corner of the current video frame, and the anchor point region may be a rectangular region, a circular region, or a region with a predetermined shape. Taking a rectangular area as an example, the front-end device may mark a rectangular area in each frame of the video to be processed as an anchor point area of each frame, and in addition, the color displayed in the anchor point area of each frame is different from the color displayed outside the anchor point area, so that the anchor point area may be identified in each frame in a manner of being visible to the naked eye, and thus, the anchor point information may be identified in the anchor point area. For example, the front-end device may assign number 0 to the anchor region of video frame 0; allocating a number 1 to an anchor point region of a video frame 1; and so on.
S204, adding the anchor point information into the anchor point area of the current video frame, and repeatedly executing the operation of extracting the video frame until each video frame in the video to be processed is added with the anchor point information corresponding to the video frame.
In a specific embodiment of the present application, the front-end device may add anchor point information to an anchor point region of a current video frame, and repeatedly perform the above operation of extracting video frames until each video frame in the video to be processed is added with anchor point information corresponding thereto. Specifically, the front-end device may add number 0 to the anchor region of video frame 0; number 1 is added to the anchor region of video frame 1; number 2 is added to the anchor region of video frame 2; the above operations are repeatedly executed until each video frame in the video to be processed is added with the number corresponding to the video frame.
S205, combining all the video frames added with the anchor point information corresponding to the video frames into a new video to be processed, and sending the new video to be processed to the video processing equipment so that the video processing equipment processes the new video to be processed.
In a specific embodiment of the present application, the front-end device may combine all video frames to which anchor point information corresponding to the video frames has been added into a new video to be processed, and send the new video to be processed to the video processing device, so that the video processing device processes the new video to be processed. Specifically, it is assumed that the video to be processed includes N video frames, which are respectively: video frame 0, video frame 1, …, video frame N-1; wherein N is a natural number of 1 or more. Assume that video frame 0 has anchor information 0 added thereto, video frame 1 has anchor information 1, … added thereto, and video frame N-1 has anchor information N-1 added thereto. In this step, the front-end device may combine the video frame 0 to which the anchor information 0 has been added, the video frames 1 and … to which the anchor information 1 has been added, and the video frame N-1 to which the anchor information N-1 has been added into a new video to be processed, and send the new video to be processed to the video processing device.
In a specific embodiment of the present application, the video processing device may be a camera, or may be another video processing device, and the video processing device may perform various types of processing on a new video to be processed, which is not limited herein. It should be noted that the resolution of the new video to be processed may change after being processed by the video processing device.
According to the video preprocessing method provided by the embodiment of the application, the front-end equipment can extract one video frame from the video frames to be processed as the current video frame, and an anchor point area is determined in the current video frame; allocating anchor point information for marking the current video frame to the anchor point area of the current video frame; adding anchor point information into an anchor point area of a current video frame, and repeatedly executing the operation of extracting the video frame until each video frame in the video to be processed is added with the anchor point information corresponding to the video frame; and combining all the video frames added with the anchor point information corresponding to the video frames into a new video to be processed, and sending the new video to be processed to the video processing equipment so that the video processing equipment processes the new video to be processed. In the prior art, whether the video meets the requirement of human eyes on time sensitivity is mainly identified through human eyes, so that whether the video is smooth or not and whether a frame loss condition exists or not are judged, but the difference between two adjacent frames of video images is difficult to distinguish only by the human eyes; under the condition that the video is blocked, the fact that the frame is lost or a plurality of repeated frames continuously appear cannot be distinguished; and the frame loss condition can not be accurately quantified only by subjective evaluation and judgment of human eyes, which is not beneficial to further optimization of the products such as the shooting device. Because the technical means that the anchor point information used for marking the current video frame is added in the anchor point area of each video frame and whether the video frame has frame loss or not is detected according to the anchor point information extracted from the anchor point area of each video frame is adopted, the technical problem that the video frame is not accurately observed through human eyes in the prior art is solved, the technical effect of detecting the frame loss of the video can be automatically realized by adopting the technical scheme provided by the application, the accuracy is high, the reliability is high, and the labor cost and the time cost can be effectively reduced; moreover, the technical scheme of the embodiment of the application is simple and convenient to implement, convenient to popularize and wide in application range.
EXAMPLE III
Fig. 4 is a schematic flow chart of a frame loss detection method according to a third embodiment of the present application. The method can be executed by a frame loss detection apparatus or a backend device, the apparatus or the backend device can be implemented by software and/or hardware, and the apparatus or the backend device can be integrated in any intelligent device with a network communication function. As shown in fig. 4, the frame loss detection method may include the following steps:
s401, receiving the processed video sent by the video processing equipment.
In a specific embodiment of the present application, the backend device may receive the processed video sent by the video processing device. Specifically, the video processing device may perform any form of processing on a new video to be processed, and only anchor point information corresponding to the new video to be processed needs to be retained in each video frame, which is not limited in this application.
S402, extracting a video frame from the processed video as a current video frame, and determining an anchor point area in the current video frame.
In a specific embodiment of the present application, the backend device may extract one video frame from the processed video as a current video frame, and determine an anchor point region in the current video frame. For example, the backend device may extract a video frame 0 from the processed video, use the video frame 0 as a current video frame, and determine an anchor point region in the current video frame. Specifically, the back-end device may first obtain a horizontal resolution and a vertical resolution of a current video frame, and a horizontal resolution and a vertical resolution of an original video frame corresponding to the current video frame; then, according to the transverse resolution and the longitudinal resolution of the current video frame and the transverse resolution and the longitudinal resolution of the original video frame corresponding to the current video frame, calculating a transverse resolution factor and a longitudinal resolution factor of the current video frame; and then determining the anchor point region in the current video frame according to the coordinates of the fixed point in the anchor point region, the shape information of the anchor point region and the transverse resolution factor and the longitudinal resolution factor of the current video frame.
S403, extracting anchor point information for marking the current video frame from the anchor point region of the current video frame.
In a specific embodiment of the present application, the backend device may extract anchor point information for marking the current video frame in an anchor point region of the current video frame. Specifically, the back-end device may perform image recognition on the current video frame first, and recognize an anchor point region in the current video frame; anchor information for marking the current video frame is then extracted from the identified anchor regions. For example, the backend device may extract number 0 for marking video frame 0 in the anchor region of video frame 0; extracting a number 1 for marking the video frame 1 from an anchor point area of the video frame 1; and so on.
Preferably, in the embodiment of the present application, if the anchor point region of the current video frame is a color image, the backend device may first convert the anchor point region of the current video frame into a gray image; anchor point information for marking the current video frame is then extracted from the grayscale image.
S404, detecting whether the current video frame has frame loss or not according to the anchor point information of the current video frame and the anchor point information of the previous video frame; and repeating the operation of extracting the video frames until detecting whether each video frame in the processed video has a frame loss.
In a specific embodiment of the present application, the backend device may detect whether a frame loss occurs in a current video frame according to anchor point information of the current video frame and anchor point information of a previous video frame; and repeating the operation of extracting the video frames until detecting whether each video frame in the processed video has a frame loss. Specifically, the backend device may determine whether anchor point information of a current video frame and anchor point information of a previous video frame satisfy a preset rule; if the anchor point information of the current video frame and the anchor point information of the previous video frame meet the preset rule, the back-end equipment can detect that no frame is lost in the current video frame; if the anchor point information of the current video frame and the anchor point information of the previous video frame do not meet the preset rule; the back-end device may detect that the current video frame has a frame loss. For example, if the anchor point information of the current video frame and the anchor point information of the previous video frame are both numbers, the back-end device may detect that there is no frame loss in the current video frame if the number of the current video frame is consecutive to the number of the previous video frame; if the number of the current video frame is not continuous with the number of the last video frame, the back-end equipment can detect that the current video frame has a frame loss.
According to the frame loss detection method provided by the embodiment of the application, the back-end equipment can receive the processed video frame sent by the video processing equipment, one video frame is extracted from the processed video as the current video frame, and the anchor point area is determined in the current video frame; extracting anchor point information used for marking the current video frame from an anchor point area of the current video frame; determining whether the current video frame has a frame loss according to the anchor point information of the current video frame and the anchor point information of the previous video frame; and repeating the operation of extracting the video frames until detecting whether each video frame in the processed video has a frame loss. That is to say, before the video is processed by the video processing device, the front-end device may add anchor point information for marking the video frame in an anchor point region of each video frame, and after the video is processed by the video processing device, the back-end device may extract anchor point information for marking the video frame in each video frame, so that whether the current video frame has a frame loss or not may be detected according to the anchor point information of the current video frame and the anchor point information extracted from the previous video frame. In the prior art, whether the video meets the requirement of human eyes on time sensitivity is mainly identified through human eyes, so that whether the video is smooth or not and whether a frame loss condition exists or not are judged, but the difference between two adjacent frames of video images is difficult to distinguish only by the human eyes; under the condition that the video is blocked, the fact that the frame is lost or a plurality of repeated frames continuously appear cannot be distinguished; and the frame loss condition can not be accurately quantified only by subjective evaluation and judgment of human eyes, which is not beneficial to further optimization of the products such as the shooting device. Because the technical means that the anchor point information used for marking the current video frame is added in the anchor point area of each video frame and whether the video frame has frame loss or not is detected according to the anchor point information extracted from the anchor point area of each video frame is adopted, the technical problem that the video frame is not accurately observed through human eyes in the prior art is solved, the technical effect of detecting the frame loss of the video can be automatically realized by adopting the technical scheme provided by the application, the accuracy is high, the reliability is high, and the labor cost and the time cost can be effectively reduced; moreover, the technical scheme of the embodiment of the application is simple and convenient to implement, convenient to popularize and wide in application range.
Example four
Fig. 5 is a schematic flow chart of a frame loss detection method according to the fourth embodiment of the present application. As shown in fig. 5, the frame loss detection method may include the following steps:
s501, receiving the processed video sent by the video processing equipment.
In a specific embodiment of the present application, the backend device may receive the processed video sent by the video processing device. Specifically, the video processing device may perform any form of processing on a new video to be processed, and only anchor point information corresponding to the new video to be processed needs to be retained in each video frame, which is not limited in this application.
S502, extracting a video frame from the processed video to be used as a current video frame, and acquiring the transverse resolution and the longitudinal resolution of the current video frame and the transverse resolution and the longitudinal resolution of an original video frame corresponding to the current video frame.
In a specific embodiment of the present application, the back-end device may extract a video frame from the processed video as a current video frame, and obtain a horizontal resolution and a vertical resolution of the current video frame, and a horizontal resolution and a vertical resolution of an original video frame corresponding to the current video frame; the original video frame corresponding to the current video frame is as follows: the video frame in the new video to be processed has the same position or sequence as the current video frame. For example, the backend device may extract video frame 0 from the processed video, and use video frame 0 as the current video frame, assuming that the horizontal resolution and the vertical resolution of video frame 0 in the new video to be processed are R respectively0(w) and R0(h) (ii) a Suppose that the horizontal resolution and vertical resolution of video frame 0 in the processed video are R 'respectively'0(w) and R'0(h) (ii) a In this step, the back-end device may obtain the lateral resolution R of video frame 0 in the new video to be processed0(w) and longitudinal resolution R0(h) And the lateral resolution R 'of video frame 0 in the processed video'0(w) and longitudinal resolution R'0(h)。
S503, calculating the horizontal resolution factor and the vertical resolution factor of the current video frame according to the horizontal resolution and the vertical resolution of the current video frame and the horizontal resolution and the vertical resolution of the original video frame corresponding to the current video frame.
In a specific embodiment of the present application, the back-end device may calculate a horizontal resolution factor and a vertical resolution factor of the current video frame according to the horizontal resolution and the vertical resolution of the current video frame and the horizontal resolution and the vertical resolution of the original video frame corresponding to the current video frame. Specifically, it is assumed that the backend device can use video frame 0 as the current video frame, and the horizontal resolution and the vertical resolution of video frame 0 in the new video to be processed are respectively R0(w) and R0(h) And the horizontal resolution and the vertical resolution of the video frame 0 in the processed video are R 'respectively'0(w) and R'0(h) (ii) a In this step, the horizontal resolution factor of the current video frame is: r is R0(w)/R’0(w); the vertical resolution factor of the current video frame is: RH ═ R0(h)/R’0(h)。
S504, according to the coordinates of the fixed point in the anchor point region, the shape information of the anchor point region, the transverse resolution factor and the longitudinal resolution factor of the current video frame, the anchor point region is determined in the current video frame.
In a specific embodiment of the present application, the backend device may determine the anchor point region in the current video frame according to the coordinates of the fixed point in the anchor point region and the shape information of the anchor point region, and the horizontal resolution factor and the vertical resolution factor of the current video frame. Specifically, it is assumed that the backend device can use the processed video frame 0 as the current video frame, and the coordinates of the fixed point in the anchor point region of the video frame 0 before processing are (x, y); the shape information of the anchor point area is (w, h); in this step, the backend device may determine the anchor point region of the video frame 0 after processing according to the following formula: rect (x/RW, y/RH, w/RW, h/RH).
And S505, extracting anchor point information used for marking the current video frame from the anchor point area of the current video frame.
In a specific embodiment of the present application, the postshield device may extract anchor point information for marking the current video frame in an anchor point region of the current video frame. Specifically, the back-end device may perform image recognition on the current video frame first, and recognize an anchor point region in the current video frame; anchor information for marking the current video frame is then extracted from the identified anchor regions. For example, the backend device may extract number 0 for marking video frame 0 in the anchor region of video frame 0; extracting a number 1 for marking the video frame 1 from an anchor point area of the video frame 1; and so on.
S506, detecting whether the current video frame has a frame loss or not according to the anchor point information of the current video frame and the anchor point information extracted from the previous video frame; and repeating the operation of extracting the video frames until detecting whether each video frame in the processed video has a frame loss.
In a specific embodiment of the present application, the backend device may detect whether a frame loss occurs in a current video frame according to anchor point information of the current video frame and anchor point information of a previous video frame; and repeating the operation of extracting the video frames until detecting whether each video frame in the processed video has a frame loss. Specifically, the backend device may determine whether anchor point information of a current video frame and anchor point information of a previous video frame satisfy a preset rule; if the anchor point information of the current video frame and the anchor point information of the previous video frame meet the preset rule, the back-end equipment can detect that no frame is lost in the current video frame; if the anchor point information of the current video frame and the anchor point information of the previous video frame do not meet the preset rule; the back-end device may detect that the current video frame has a frame loss. For example, if the anchor point information of the current video frame and the anchor point information of the previous video frame are both numbers, the back-end device may detect that there is no frame loss in the current video frame if the number of the current video frame is consecutive to the number of the previous video frame; if the number of the current video frame is not continuous with the number of the last video frame, the back-end equipment can detect that the current video frame has a frame loss.
Preferably, in an embodiment of the present application, when detecting whether a frame loss occurs in a current video frame, the backend device may cut the anchor point region in each video frame, and compare whether a number of the current video frame is consecutive to a number of a previous video frame, so as to detect whether the frame loss occurs in the current video frame. Fig. 6 is a schematic structural diagram of an anchor point region after clipping according to the fourth embodiment of the present application. As shown in fig. 6, it is assumed that the backend apparatus receives 15 video frames transmitted by the video processing apparatus. Specifically, the 1 st video frame received by the back-end device and sent by the video processing device is 0. png; the received 2 nd video frame sent by the video processing equipment is 1. png; the received 3 rd video frame sent by the video processing equipment is 2. png; …, respectively; the 15 th video frame received from the video processing device is 14. png. Wherein the number of the 1 st video frame 0.png0 is 0; number 1 for the 2 nd video frame 1.png 0; number 2 for the 3 rd video frame 2.png 0; number 3 for the 4 th video frame 3.png 0; number 4 for the 5 th video frame 4.png 0; number 5 for the 6 th video frame 5.png 0; number 6 for the 7 th video frame 6.png 0; number 7 for the 8 th video frame 7.png 0; number 9 for the 9 th video frame 8.png 0; number 10 for the 10 th video frame 9.png 0; number 11 for the 11 th video frame 10.png 0; number 12 for the 12 th video frame 01.png 0; number 13 for the 13 th video frame 12.png 0; number 14 for the 14 th video frame 13.png 0; the 15 th video frame 14.png0 is numbered 15. Since the 8 th video frame 7.png0 is numbered 7; number 9 for the 9 th video frame 8.png 0; thus, the back-end device may detect that a video frame is lost after the 8 th video frame and before the 9 th video frame.
It should be noted that, in the specific embodiment of the present application, the front-end device and the back-end device are the same electronic device, or two independent electronic devices. In addition, the video processing device in this application may be the same electronic device as the front-end device, or may be two independent electronic devices, and the video processing device in this application may be the same electronic device as the back-end device, or may be two independent electronic devices, which is not limited herein.
In the embodiment of the present application, the video preprocessing and frame loss detection method can be completed by the following five systems: the system comprises a video preprocessing system, a camera processing system, a video anchor point information extraction system, an anchor point information automatic identification system and a data analysis system; specifically, the video preprocessing and frame loss detection method may include the following steps: (1) the video preprocessing system comprises: the main function of the video preprocessing system is to automatically add specific anchor point information to the designated area of each frame of image of the video A to be processed to form the video B, wherein the anchor point information needs to ensure that each frame has unique anchor point information for the original video, thereby facilitating subsequent detection and identification. (2) A camera processing system: the system mainly realizes that the video shooting device automatically processes the video B produced in the step (1), and automatically obtains the video C produced after the video shooting device processes. (3) Video anchor point information extraction system: the system automatically extracts anchor point information of each frame in the video C, forms an anchor point information picture sequence D, and then performs certain graphic image processing (such as gray processing) on all anchor point pictures in the anchor point information picture sequence D, so that the subsequent automatic identification of the anchor point information is facilitated. (4) Anchor point information automatic identification system: the system aims to realize the text processing of the image anchor point information, perform image recognition on the anchor point information image sequence D generated in the step (3) by using schemes such as machine learning, neural network training and the like, and store the anchor point information E of the recognized text type. (5) A data analysis system: and (4) analyzing the anchor point information E acquired in the step (4) by the data analysis system to obtain the data information of the lost frame and the repeated frame. Preferably, in the embodiment of the present application, the video pre-processing system may be integrated in the front-end device; the camera processing system may be integrated in the video processing device; the video anchor point information extraction system, the anchor point information automatic identification system and the data analysis system can be integrated in the back-end equipment.
According to the frame loss detection method provided by the embodiment of the application, the back-end equipment can receive the processed video frame sent by the video processing equipment, one video frame is extracted from the processed video as the current video frame, and the anchor point area is determined in the current video frame; extracting anchor point information used for marking the current video frame from an anchor point area of the current video frame; determining whether the current video frame has a frame loss according to the anchor point information of the current video frame and the anchor point information of the previous video frame; and repeating the operation of extracting the video frames until detecting whether each video frame in the processed video has a frame loss. That is to say, before the video is processed by the video processing device, the front-end device may add anchor point information for marking the video frame in an anchor point region of each video frame, and after the video is processed by the video processing device, the back-end device may extract anchor point information for marking the video frame in each video frame, so that whether the current video frame has a frame loss or not may be detected according to the anchor point information of the current video frame and the anchor point information extracted from the previous video frame. In the prior art, whether the video meets the requirement of human eyes on time sensitivity is mainly identified through human eyes, so that whether the video is smooth or not and whether a frame loss condition exists or not are judged, but the difference between two adjacent frames of video images is difficult to distinguish only by the human eyes; under the condition that the video is blocked, the fact that the frame is lost or a plurality of repeated frames continuously appear cannot be distinguished; and the frame loss condition can not be accurately quantified only by subjective evaluation and judgment of human eyes, which is not beneficial to further optimization of the products such as the shooting device. Because the technical means that the anchor point information used for marking the current video frame is added in the anchor point area of each video frame and whether the video frame has frame loss or not is detected according to the anchor point information extracted from the anchor point area of each video frame is adopted, the technical problem that the video frame is not accurately observed through human eyes in the prior art is solved, the technical effect of detecting the frame loss of the video can be automatically realized by adopting the technical scheme provided by the application, the accuracy is high, the reliability is high, and the labor cost and the time cost can be effectively reduced; moreover, the technical scheme of the embodiment of the application is simple and convenient to implement, convenient to popularize and wide in application range.
EXAMPLE five
Fig. 7 is a schematic structural diagram of a video preprocessing apparatus according to a fifth embodiment of the present application. As shown in fig. 7, the apparatus 700 includes: a determining module 701, an allocating module 702, an adding module 703 and a sending module 704; wherein the content of the first and second substances,
the determining module 701 is configured to extract a video frame from a video to be processed as a current video frame, and determine an anchor point region in the current video frame;
the allocating module 702 is configured to allocate anchor point information for marking the current video frame to an anchor point region of the current video frame;
an adding module 703, configured to add the anchor point information to the anchor point region of the current video frame, and repeatedly execute the operation of extracting the video frame until each video frame in the video to be processed is added with the anchor point information corresponding to the video frame;
the sending module 704 is configured to combine all video frames to which anchor point information corresponding to the video frames is added into a new video to be processed, and send the new video to be processed to a video processing device, so that the video processing device processes the new video to be processed.
Further, the determining module 701 is specifically configured to determine a coordinate of a fixed point in the anchor point region in the current video frame; and determining the anchor point region in the current video frame according to the coordinates of the fixed point and the predetermined shape information of the anchor point region.
The video preprocessing device can execute the method provided by any embodiment of the application, and has corresponding functional modules and beneficial effects of the execution method. For details of the video pre-processing method provided in any of the embodiments of the present application, reference may be made to the following description.
EXAMPLE six
Fig. 8 is a schematic structural diagram of a frame loss detection apparatus according to a sixth embodiment of the present application. As shown in fig. 8, the apparatus 800 includes: a receiving module 801, an extracting module 802 and a detecting module 803; wherein the content of the first and second substances,
the receiving module 801 is configured to receive a processed video sent by a video processing device;
the extracting module 802 is configured to extract a video frame from the processed video as a current video frame, and determine an anchor point region in the current video frame; extracting anchor point information used for marking the current video frame from the anchor point area of the current video frame;
the detecting module 803 is configured to detect whether there is a frame loss in the current video frame according to the anchor point information of the current video frame and the anchor point information of the previous video frame; and repeating the operation of extracting the video frames until detecting whether each video frame in the processed video has a frame loss.
Fig. 9 is a schematic structural diagram of a receiving module according to a sixth embodiment of the present application. As shown in fig. 9, the extraction module 802 includes: an acquisition sub-module 8021, a calculation sub-module 8022 and a determination sub-module 8023; wherein the content of the first and second substances,
the obtaining sub-module 8021 is configured to obtain a horizontal resolution and a vertical resolution of the current video frame, and a horizontal resolution and a vertical resolution of an original video frame corresponding to the current video frame;
the calculating submodule 8022 is configured to calculate a horizontal resolution factor and a vertical resolution factor of the current video frame according to the horizontal resolution and the vertical resolution of the current video frame and the horizontal resolution and the vertical resolution of the original video frame corresponding to the current video frame;
the determining submodule 8023 is configured to determine the anchor point region in the current video frame according to the predetermined coordinates of the fixed point in the anchor point region, the shape information of the anchor point region, and the horizontal resolution factor and the vertical resolution factor of the current video frame.
Further, the extraction module 802 is specifically configured to convert the anchor point region of the current video frame into a gray image if the anchor point region of the current video frame is a color image; and extracting anchor point information used for marking the current video frame from the gray-scale image.
The frame loss detection device can execute the method provided by any embodiment of the application, and has corresponding functional modules and beneficial effects of the execution method. For details of the technique not described in detail in this embodiment, reference may be made to the frame loss detection method provided in any embodiment of the present application.
EXAMPLE seven
According to an embodiment of the present application, an electronic device and a readable storage medium are also provided.
Fig. 10 is a block diagram of an electronic device for video preprocessing and frame loss detection according to an embodiment of the present application. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be examples only, and are not meant to limit implementations of the present application that are described and/or claimed herein.
As shown in fig. 10, the electronic apparatus includes: one or more processors 1001, memory 1002, and interfaces for connecting the various components, including high-speed interfaces and low-speed interfaces. The various components are interconnected using different buses and may be mounted on a common motherboard or in other manners as desired. The processor may process instructions for execution within the electronic device, including instructions stored in or on the memory to display graphical information of a GUI on an external input/output apparatus (such as a display device coupled to the interface). In other embodiments, multiple processors and/or multiple buses may be used, along with multiple memories and multiple memories, as desired. Also, multiple electronic devices may be connected, with each device providing portions of the necessary operations (e.g., as a server array, a group of blade servers, or a multi-processor system). Fig. 10 illustrates an example of one processor 1001.
The memory 1002 is a non-transitory computer readable storage medium provided herein. Wherein the memory stores instructions executable by at least one processor to cause the at least one processor to perform the video pre-processing and frame loss detection methods provided herein. The non-transitory computer readable storage medium of the present application stores computer instructions for causing a computer to perform the video pre-processing and frame loss detection methods provided herein.
The memory 1002, as a non-transitory computer readable storage medium, may be used for storing non-transitory software programs, non-transitory computer executable programs, and modules, such as program instructions/modules corresponding to the video preprocessing and frame loss detection method in the embodiment of the present application (for example, the determining module 701, the allocating module 702, the adding module 703, and the sending module 704 shown in fig. 7). The processor 1001 executes various functional applications and data processing of the server by running non-transitory software programs, instructions, and modules stored in the memory 1002, that is, implements the video preprocessing and frame loss detection method in the above method embodiment.
The memory 1002 may include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required for at least one function; the storage data area may store data created according to the use of the electronic device of the video preprocessing and frame loss detection method, and the like. Further, the memory 1002 may include high-speed random access memory, and may also include non-transitory memory, such as at least one magnetic disk storage device, flash memory device, or other non-transitory solid state storage device. In some embodiments, the memory 1002 may optionally include memory located remotely from the processor 1001, and such remote memory may be connected to the electronic device of the video pre-processing and frame loss detection method via a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
The electronic device of the video preprocessing and frame loss detection method may further include: an input device 1003 and an output device 1004. The processor 1001, the memory 1002, the input device 1003, and the output device 1004 may be connected by a bus or other means, and the bus connection is exemplified in fig. 10.
The input device 1003 may receive input numeric or character information and generate key signal inputs related to user settings and function control of the electronic apparatus of the video pre-processing and frame loss detection method, such as a touch screen, a keypad, a mouse, a track pad, a touch pad, a pointer, one or more mouse buttons, a track ball, a joystick, or other input devices. The output devices 1004 may include a display device, auxiliary lighting devices (e.g., LEDs), and tactile feedback devices (e.g., vibrating motors), among others. The display device may include, but is not limited to, a Liquid Crystal Display (LCD), a Light Emitting Diode (LED) display, and a plasma display. In some implementations, the display device can be a touch screen.
Various implementations of the systems and techniques described here can be realized in digital electronic circuitry, integrated circuitry, application specific ASICs (application specific integrated circuits), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs that are executable and/or interpretable on a programmable system including at least one programmable processor, which may be special or general purpose, receiving data and instructions from, and transmitting data and instructions to, a storage system, at least one input device, and at least one output device.
These computer programs (also known as programs, software applications, or code) include machine instructions for a programmable processor, and may be implemented using high-level procedural and/or object-oriented programming languages, and/or assembly/machine languages. As used herein, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, apparatus, and/or device (e.g., magnetic discs, optical disks, memory, Programmable Logic Devices (PLDs)) used to provide machine instructions and/or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term "machine-readable signal" refers to any signal used to provide machine instructions and/or data to a programmable processor.
To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which a user can provide input to the computer. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic, speech, or tactile input.
The systems and techniques described here can be implemented in a computing system that includes a back-end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back-end, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), Wide Area Networks (WANs), the internet, and blockchain networks.
The computer system may include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other.
According to the technical scheme of the embodiment of the application, the front-end equipment can extract one video frame from the video frames to be processed as the current video frame, and an anchor point area is determined in the current video frame; allocating anchor point information for marking the current video frame to the anchor point area of the current video frame; adding anchor point information into an anchor point area of a current video frame, and repeatedly executing the operation of extracting the video frame until each video frame in the video to be processed is added with the anchor point information corresponding to the video frame; and combining all the video frames added with the anchor point information corresponding to the video frames into a new video to be processed, and sending the new video to be processed to the video processing equipment so that the video processing equipment processes the new video to be processed. Correspondingly, the back-end device can receive the processed video frame sent by the video processing device, extract a video frame from the processed video as the current video frame, and determine the anchor point area in the current video frame; extracting anchor point information used for marking the current video frame from an anchor point area of the current video frame; determining whether the current video frame has a frame loss according to the anchor point information of the current video frame and the anchor point information of the previous video frame; and repeating the operation of extracting the video frames until detecting whether each video frame in the processed video has a frame loss. That is to say, before the video is processed by the video processing device, the front-end device may add anchor point information for marking the video frame in an anchor point region of each video frame, and after the video is processed by the video processing device, the back-end device may extract anchor point information for marking the video frame in each video frame, so that whether the current video frame has a frame loss or not may be detected according to the anchor point information of the current video frame and the anchor point information extracted from the previous video frame. In the prior art, whether the video meets the requirement of human eyes on time sensitivity is mainly identified through human eyes, so that whether the video is smooth or not and whether a frame loss condition exists or not are judged, but the difference between two adjacent frames of video images is difficult to distinguish only by the human eyes; under the condition that the video is blocked, the fact that the frame is lost or a plurality of repeated frames continuously appear cannot be distinguished; and the frame loss condition can not be accurately quantified only by subjective evaluation and judgment of human eyes, which is not beneficial to further optimization of the products such as the shooting device. Because the technical means that the anchor point information used for marking the current video frame is added in the anchor point area of each video frame and whether the video frame has frame loss or not is detected according to the anchor point information extracted from the anchor point area of each video frame is adopted, the technical problem that the video frame is not accurately observed through human eyes in the prior art is solved, the technical effect of detecting the frame loss of the video can be automatically realized by adopting the technical scheme provided by the application, the accuracy is high, the reliability is high, and the labor cost and the time cost can be effectively reduced; moreover, the technical scheme of the embodiment of the application is simple and convenient to implement, convenient to popularize and wide in application range.
It should be understood that various forms of the flows shown above may be used, with steps reordered, added, or deleted. For example, the steps described in the present application may be executed in parallel, sequentially, or in different orders, and the present invention is not limited thereto as long as the desired results of the technical solutions disclosed in the present application can be achieved.
The above-described embodiments should not be construed as limiting the scope of the present application. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may be made in accordance with design requirements and other factors. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present application shall be included in the protection scope of the present application.
Claims (12)
1. A video preprocessing method applied to a front-end device, the method comprising:
extracting a video frame from a video to be processed as a current video frame, and determining an anchor point area in the current video frame;
allocating anchor point information for marking the current video frame to the anchor point area of the current video frame;
adding the anchor point information into an anchor point area of the current video frame, and repeatedly executing the operation of extracting the video frame until each video frame in the video to be processed is added with the anchor point information corresponding to the video frame;
and combining all the video frames added with the anchor point information corresponding to the video frames into a new video to be processed, and sending the new video to be processed to video processing equipment so that the video processing equipment processes the new video to be processed.
2. The method of claim 1, wherein determining an anchor region in the current video frame comprises:
determining the coordinates of a fixed point in the anchor point region in the current video frame;
and determining the anchor point region in the current video frame according to the coordinates of the fixed point and the predetermined shape information of the anchor point region.
3. A frame loss detection method is applied to a back-end device, and comprises the following steps:
receiving a processed video sent by video processing equipment;
extracting a video frame from the processed video as a current video frame, and determining an anchor point area in the current video frame;
extracting anchor point information used for marking the current video frame from the anchor point area of the current video frame;
detecting whether the current video frame has a frame loss or not according to the anchor point information of the current video frame and the anchor point information of the previous video frame; and repeating the operation of extracting the video frames until detecting whether each video frame in the processed video has a frame loss.
4. The method of claim 3, wherein determining an anchor region in the current video frame comprises:
acquiring the transverse resolution and the longitudinal resolution of the current video frame and the transverse resolution and the longitudinal resolution of an original video frame corresponding to the current video frame;
calculating a horizontal resolution factor and a vertical resolution factor of the current video frame according to the horizontal resolution and the vertical resolution of the current video frame and the horizontal resolution and the vertical resolution of an original video frame corresponding to the current video frame;
and determining the anchor point region in the current video frame according to the coordinates of the fixed point in the anchor point region, the shape information of the anchor point region and the transverse resolution factor and the longitudinal resolution factor of the current video frame, which are determined in advance.
5. The method of claim 3, wherein extracting anchor information for marking the current video frame in the anchor region of the current video frame comprises:
if the anchor point area of the current video frame is a color image, converting the anchor point area of the current video frame into a gray image;
and extracting anchor point information used for marking the current video frame from the gray-scale image.
6. A video pre-processing apparatus, the apparatus comprising: the device comprises a determining module, a distributing module, an adding module and a sending module; wherein the content of the first and second substances,
the determining module is used for extracting a video frame from a video to be processed as a current video frame and determining an anchor point area in the current video frame;
the allocation module is used for allocating anchor point information for marking the current video frame to the anchor point area of the current video frame;
the adding module is used for adding the anchor point information into the anchor point area of the current video frame and repeatedly executing the operation of extracting the video frame until each video frame in the video to be processed is added with the anchor point information corresponding to the video frame;
the sending module is configured to combine all video frames to which anchor point information corresponding to the video frames is added into a new video to be processed, and send the new video to be processed to video processing equipment, so that the video processing equipment processes the new video to be processed.
7. The apparatus of claim 6, wherein:
the determining module is specifically configured to determine a coordinate of a fixed point in the anchor point region in the current video frame; and determining the anchor point region in the current video frame according to the coordinates of the fixed point and the predetermined shape information of the anchor point region.
8. A frame loss detection apparatus, the apparatus comprising: the device comprises a receiving module, an extracting module and a detecting module; wherein the content of the first and second substances,
the receiving module is used for receiving the processed video sent by the video processing equipment;
the extraction module is used for extracting a video frame from the processed video as a current video frame, and determining an anchor point area in the current video frame; extracting anchor point information used for marking the current video frame from the anchor point area of the current video frame;
the detection module is used for detecting whether the current video frame has a frame loss or not according to the anchor point information of the current video frame and the anchor point information of the previous video frame; and repeating the operation of extracting the video frames until detecting whether each video frame in the processed video has a frame loss.
9. The apparatus of claim 8, wherein the extraction module comprises: the method comprises the following steps of obtaining a submodule, a calculating submodule and a determining submodule; wherein the content of the first and second substances,
the obtaining submodule is used for obtaining the transverse resolution and the longitudinal resolution of the current video frame and the transverse resolution and the longitudinal resolution of the original video frame corresponding to the current video frame;
the calculation submodule is used for calculating a transverse resolution factor and a longitudinal resolution factor of the current video frame according to the transverse resolution and the longitudinal resolution of the current video frame and the transverse resolution and the longitudinal resolution of the original video frame corresponding to the current video frame;
the determining submodule is configured to determine the anchor point region in the current video frame according to the coordinates of the fixed point in the anchor point region and the shape information of the anchor point region, and the horizontal resolution factor and the vertical resolution factor of the current video frame, which are predetermined.
10. The apparatus of claim 8, wherein:
the extraction module is specifically configured to convert the anchor point region of the current video frame into a gray image if the anchor point region of the current video frame is a color image; and extracting anchor point information used for marking the current video frame from the gray-scale image.
11. An electronic device, comprising:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,
the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-2 or 3-5.
12. A non-transitory computer readable storage medium having stored thereon computer instructions for causing a computer to perform the method of any one of claims 1-2 or 3-5.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201911413278.0A CN111131878A (en) | 2019-12-31 | 2019-12-31 | Video preprocessing and frame loss detection method and device, electronic equipment and storage medium |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201911413278.0A CN111131878A (en) | 2019-12-31 | 2019-12-31 | Video preprocessing and frame loss detection method and device, electronic equipment and storage medium |
Publications (1)
Publication Number | Publication Date |
---|---|
CN111131878A true CN111131878A (en) | 2020-05-08 |
Family
ID=70506497
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN201911413278.0A Pending CN111131878A (en) | 2019-12-31 | 2019-12-31 | Video preprocessing and frame loss detection method and device, electronic equipment and storage medium |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN111131878A (en) |
Citations (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN101527855A (en) * | 2009-04-21 | 2009-09-09 | 成都索贝数码科技股份有限公司 | Automatic detection method of video signals and system thereof |
CN105611291A (en) * | 2015-12-31 | 2016-05-25 | 北京奇艺世纪科技有限公司 | Method and device for adding mark information and detection frame loss in video frame |
EP3253064A1 (en) * | 2015-02-17 | 2017-12-06 | Huawei Technologies Co., Ltd. | Frame loss method for video frame and video sending apparatus |
CN108495120A (en) * | 2018-01-31 | 2018-09-04 | 华为技术有限公司 | A kind of video frame detection, processing method, apparatus and system |
CN109241885A (en) * | 2018-08-23 | 2019-01-18 | 无锡天脉聚源传媒科技有限公司 | A kind of method and device of special area in intelligent recognition video |
CN110418185A (en) * | 2019-07-22 | 2019-11-05 | 广州市天正科技有限公司 | The localization method and its system of anchor point in a kind of augmented reality video pictures |
-
2019
- 2019-12-31 CN CN201911413278.0A patent/CN111131878A/en active Pending
Patent Citations (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN101527855A (en) * | 2009-04-21 | 2009-09-09 | 成都索贝数码科技股份有限公司 | Automatic detection method of video signals and system thereof |
EP3253064A1 (en) * | 2015-02-17 | 2017-12-06 | Huawei Technologies Co., Ltd. | Frame loss method for video frame and video sending apparatus |
CN105611291A (en) * | 2015-12-31 | 2016-05-25 | 北京奇艺世纪科技有限公司 | Method and device for adding mark information and detection frame loss in video frame |
CN108495120A (en) * | 2018-01-31 | 2018-09-04 | 华为技术有限公司 | A kind of video frame detection, processing method, apparatus and system |
CN109241885A (en) * | 2018-08-23 | 2019-01-18 | 无锡天脉聚源传媒科技有限公司 | A kind of method and device of special area in intelligent recognition video |
CN110418185A (en) * | 2019-07-22 | 2019-11-05 | 广州市天正科技有限公司 | The localization method and its system of anchor point in a kind of augmented reality video pictures |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
WO2020140698A1 (en) | Table data acquisition method and apparatus, and server | |
CN111986178A (en) | Product defect detection method and device, electronic equipment and storage medium | |
US11798193B2 (en) | Multi-dimensional image detection on at least two acquired images | |
US11700417B2 (en) | Method and apparatus for processing video | |
CN110659600B (en) | Object detection method, device and equipment | |
EP3819820B1 (en) | Method and apparatus for recognizing key identifier in video, device and storage medium | |
US11967132B2 (en) | Lane marking detecting method, apparatus, electronic device, storage medium, and vehicle | |
CN111601013B (en) | Method and apparatus for processing video frames | |
CN111275827A (en) | Edge-based augmented reality three-dimensional tracking registration method and device and electronic equipment | |
CN112184837A (en) | Image detection method and device, electronic equipment and storage medium | |
CN111523292B (en) | Method and device for acquiring image information | |
CN111401248B (en) | Sky area identification method and device, electronic equipment and storage medium | |
CN110798681A (en) | Monitoring method and device of imaging equipment and computer equipment | |
CN111696134A (en) | Target detection method and device and electronic equipment | |
CN109461203B (en) | Gesture three-dimensional image generation method and device, computer equipment and storage medium | |
CN111131878A (en) | Video preprocessing and frame loss detection method and device, electronic equipment and storage medium | |
CN113221742B (en) | Video split screen line determining method, device, electronic equipment, medium and program product | |
CN112651983B (en) | Splice graph identification method and device, electronic equipment and storage medium | |
CN112150380B (en) | Method, apparatus, electronic device, and readable storage medium for correcting image | |
CN112558810A (en) | Method, device, equipment and storage medium for detecting fingertip position | |
CN114863143B (en) | Image detection method, image detection device, electronic equipment and storage medium | |
CN113419915A (en) | Cloud terminal desktop stillness determination method and device | |
CN111680628A (en) | Text box fusion method, device, equipment and storage medium | |
CN111858303A (en) | Method, device, equipment and medium for controlling page of applet | |
CN113691866B (en) | Video processing method, device, electronic equipment and medium |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
RJ01 | Rejection of invention patent application after publication | ||
RJ01 | Rejection of invention patent application after publication |
Application publication date: 20200508 |