CN112218119B - Video stream frame rate conversion method, device, terminal equipment and storage medium - Google Patents
Video stream frame rate conversion method, device, terminal equipment and storage medium Download PDFInfo
- Publication number
- CN112218119B CN112218119B CN202011069843.9A CN202011069843A CN112218119B CN 112218119 B CN112218119 B CN 112218119B CN 202011069843 A CN202011069843 A CN 202011069843A CN 112218119 B CN112218119 B CN 112218119B
- Authority
- CN
- China
- Prior art keywords
- video stream
- frame rate
- video
- frame
- key
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000006243 chemical reaction Methods 0.000 title claims abstract description 67
- 238000000034 method Methods 0.000 title claims abstract description 47
- 239000013598 vector Substances 0.000 claims description 31
- 238000006073 displacement reaction Methods 0.000 claims description 30
- 238000000605 extraction Methods 0.000 claims description 8
- 238000012216 screening Methods 0.000 claims description 4
- 238000004891 communication Methods 0.000 description 12
- 238000012545 processing Methods 0.000 description 10
- 238000010586 diagram Methods 0.000 description 6
- 230000002093 peripheral effect Effects 0.000 description 4
- 238000013473 artificial intelligence Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004590 computer program Methods 0.000 description 1
- 238000012217 deletion Methods 0.000 description 1
- 230000037430 deletion Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 230000001788 irregular Effects 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 230000005055 memory storage Effects 0.000 description 1
- 238000010295 mobile communication Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000003672 processing method Methods 0.000 description 1
- 238000009877 rendering Methods 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/234—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
- H04N21/2343—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements
- H04N21/234381—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements by altering the temporal resolution, e.g. decreasing the frame rate by frame skipping
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/01—Conversion of standards, e.g. involving analogue television standards or digital television standards processed at pixel level
- H04N7/0127—Conversion of standards, e.g. involving analogue television standards or digital television standards processed at pixel level by changing the field or frame frequency of the incoming video signal, e.g. frame rate converter
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
Abstract
The invention discloses a video stream frame rate conversion method, which comprises the following steps: acquiring a target video stream, wherein the target video stream comprises a plurality of sub-video streams with preset duration; extracting key video frames of the plurality of sub-video streams to obtain a plurality of key video streams; performing frame rate conversion on the plurality of key video streams to obtain first output video streams with a plurality of preset frame rates; based on the plurality of first output video streams, a first resulting video stream is obtained. The invention also discloses a video stream frame rate conversion device, terminal equipment and a storage medium. The obtained first output video stream moving picture is smooth.
Description
Technical Field
The present invention relates to the field of video processing technologies, and in particular, to a method and an apparatus for converting a frame rate of a video stream, a terminal device, and a storage medium.
Background
Currently, motion compensation is an indispensable performance evaluation index for medium and high-end terminal devices. The fluency and integrity of moving pictures are also key points of increasing consumer attention, and the fluency and integrity of moving pictures are the targets pursued by numerous terminal equipment manufacturers. The terminal device performs motion compensation, that is, performs frame rate conversion on the original video stream to obtain a high frame rate output video stream of 60Hz (60 frames per second), 90Hz, or 120 Hz.
However, the terminal device obtains a moving picture of the high frame rate output video stream not smoothly by the conventional frame rate conversion method.
Disclosure of Invention
The invention mainly aims to provide a video stream frame rate conversion method, a video stream frame rate conversion device, a terminal device and a storage medium, and aims to solve the technical problem that in the prior art, a moving picture of a high-frame-rate video stream obtained by adopting the existing frame rate conversion method is not smooth.
In order to achieve the above object, the present invention provides a method for converting a frame rate of a video stream, the method comprising the following steps:
acquiring a target video stream, wherein the target video stream comprises a plurality of sub-video streams with preset duration;
extracting key video frames of the plurality of sub-video streams to obtain a plurality of key video streams;
performing frame rate conversion on the plurality of key video streams to obtain first output video streams with a plurality of preset frame rates;
based on the plurality of first output video streams, a first resulting video stream is obtained.
Optionally, before the step of extracting key video frames from the plurality of sub-video streams to obtain a plurality of key video streams, the method further includes:
acquiring frame rate information of the plurality of sub-video streams;
judging whether the frame rate information meets a preset condition or not;
the step of extracting key video frames from the plurality of sub-video streams to obtain a plurality of key video streams comprises:
and when the frame rate information does not meet the preset condition, extracting key video frames of the plurality of sub-video streams to obtain a plurality of key video streams.
Optionally, after the step of determining whether the frame rate information meets the preset condition, the method further includes:
when the frame rate information meets the preset condition, performing frame rate conversion on the plurality of sub-video streams to obtain a plurality of second output video streams with preset frame rates;
obtaining a second resulting video stream based on the plurality of second output video streams.
Optionally, when the frame rate information does not satisfy the preset condition, performing key video frame extraction on the plurality of sub-video streams to obtain a plurality of key video streams includes:
when the frame rate information does not meet the preset condition, performing copy frame screening on each sub-video stream in the plurality of sub-video streams to obtain a copy frame of each sub-video stream;
deleting the copied frame of each sub-video stream to obtain a plurality of key video streams.
Optionally, the step of performing frame rate conversion on the multiple key video streams to obtain a first output video stream with multiple preset frame rates includes:
performing image matching on each key video frame in each key video stream to obtain a displacement vector of each key video frame;
and performing frame rate conversion on each key video stream based on the displacement vector to obtain a first output video stream with a preset frame rate corresponding to each key video stream.
Optionally, the step of performing image matching on each key video frame in each key video stream to obtain a displacement vector of each key video frame includes:
determining a plurality of non-overlapping macroblocks in each key video frame of each key video stream;
and obtaining the displacement vector of each key video frame based on the macro block.
Optionally, the step of obtaining the displacement vector of each key video frame based on the macro block includes:
acquiring the gray value of each key video frame;
based on the macro blocks and the gray values, obtaining a displacement vector of each key video frame by using a formula I;
the first formula is as follows:
wherein (i, j) is the displacement vector, M × N is the size of the macroblock, and f k For the gray value of the k-th key video frame in each key video stream, f k-1 Is the gray value of the (k-1) th key video frame in each key video stream.
In addition, to achieve the above object, the present invention further provides a video stream frame rate conversion apparatus, including:
the acquisition module is used for acquiring a target video stream, wherein the target video stream comprises a plurality of sub-video streams with preset duration;
the extraction module is used for extracting key video frames of the plurality of sub-video streams to obtain a plurality of key video streams;
the frame rate conversion module is used for performing frame rate conversion on the plurality of key video streams to obtain a plurality of first output video streams with preset frame rates;
an obtaining module configured to obtain a first resulting video stream based on the plurality of first output video streams.
In addition, to achieve the above object, the present invention further provides a terminal device, including: a memory, a processor and a video stream frame rate conversion program stored on said memory and executable on said processor, said video stream frame rate conversion program being configured to implement the steps of the video stream frame rate conversion method as claimed in any one of the above.
Furthermore, to achieve the above object, the present invention also proposes a storage medium having a video stream frame rate conversion program stored thereon, which when executed by a processor implements the steps of the video stream frame rate conversion method according to any one of the above.
The technical scheme of the invention adopts a video stream frame rate conversion method, and obtains a target video stream, wherein the target video stream comprises a plurality of sub-video streams with preset duration; extracting key video frames of the plurality of sub-video streams to obtain a plurality of key video streams; performing frame rate conversion on the plurality of key video streams to obtain a plurality of first output video streams with preset frame rates; based on the plurality of first output video streams, a first resulting video stream is obtained. Because the obtained plurality of first output video streams are obtained by performing frame rate conversion on the plurality of key video streams, the plurality of key video streams are obtained by performing key video frame extraction on the plurality of sub-video streams, and non-key video frames influencing the target video stream are deleted, the fluency of the obtained plurality of first output video streams is better, and further the motion pictures of the obtained first output video streams are fluent.
Drawings
In order to more clearly illustrate the embodiments or technical solutions of the present invention, the drawings used in the embodiments or technical solutions of the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the structures shown in the drawings without creative efforts.
Fig. 1 is a schematic structural diagram of a terminal device in a hardware operating environment according to an embodiment of the present invention;
FIG. 2 is a flowchart illustrating a video stream frame rate conversion method according to a first embodiment of the present invention;
FIG. 3 is a diagram illustrating a motion vector obtaining process according to the present invention;
fig. 4 is a block diagram of a first embodiment of a video stream frame rate conversion apparatus according to the present invention.
The implementation, functional features and advantages of the objects of the present invention will be further explained with reference to the accompanying drawings.
Detailed Description
The technical solutions in the embodiments of the present invention will be described clearly and completely with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Referring to fig. 1, fig. 1 is a schematic structural diagram of a terminal device in a hardware operating environment according to an embodiment of the present invention.
The terminal device may be a User Equipment (UE) such as a Mobile phone, a smart phone, a notebook computer, a digital broadcast receiver, a Personal Digital Assistant (PDA), a tablet computer (PAD), a handheld device, a vehicle-mounted device, a wearable device, a computing device or other processing device connected to a wireless modem, a Mobile Station (MS), etc., as well as a smart tv, a smart projector, or a smart set-top box, etc. The terminal device may be referred to as a user terminal, a portable terminal, a desktop terminal, etc.
In general, a terminal device includes: at least one processor 301, a memory 302 and a video stream frame rate conversion program stored on said memory and executable on said processor, said video stream frame rate conversion program being configured to implement the steps of the video stream frame rate conversion method as described before.
The processor 301 may include one or more processing cores, such as a 4-core processor, an 8-core processor, and so on. The processor 301 may be implemented in at least one hardware form of a DSP (Digital Signal Processing), an FPGA (Field-Programmable Gate Array), and a PLA (Programmable Logic Array). The processor 301 may also include a main processor and a coprocessor, where the main processor is a processor for processing data in an awake state, and is also called a Central Processing Unit (CPU); a coprocessor is a low power processor for processing data in a standby state. In some embodiments, the processor 301 may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content required to be displayed on the display screen. Processor 301 may also include an AI (Artificial Intelligence) processor for processing video stream frame rate conversion method related operations such that the video stream frame rate conversion method model may be trained and learned autonomously, improving efficiency and accuracy.
In some embodiments, the terminal may further include: a communication interface 303 and at least one peripheral device. The processor 301, the memory 302 and the communication interface 303 may be connected by buses or signal lines. Various peripheral devices may be connected to communication interface 303 via a bus, signal line, or circuit board. Specifically, the peripheral device includes: at least one of radio frequency circuitry 304, a display screen 305, and a power source 306.
The communication interface 303 may be used to connect at least one peripheral device related to I/O (Input/Output) to the processor 301 and the memory 302. . In some embodiments, processor 301, memory 302, and communication interface 303 are integrated on the same chip or circuit board; in some other embodiments, any one or two of the processor 301, the memory 302 and the communication interface 303 may be implemented on a single chip or circuit board, which is not limited in this embodiment.
The Radio Frequency circuit 304 is used for receiving and transmitting RF (Radio Frequency) signals, also called electromagnetic signals. The radio frequency circuitry 304 communicates with communication networks and other communication devices via electromagnetic signals. The rf circuit 304 converts an electrical signal into an electromagnetic signal to transmit, or converts a received electromagnetic signal into an electrical signal. Optionally, the radio frequency circuit 304 comprises: an antenna system, an RF transceiver, one or more amplifiers, a tuner, an oscillator, a digital signal processor, a codec chipset, a subscriber identity module card, and so forth. The radio frequency circuitry 304 may communicate with other terminals via at least one wireless communication protocol. The wireless communication protocols include, but are not limited to: metropolitan area networks, various generation mobile communication networks (2G, 3G, 4G, and 5G), Wireless local area networks, and/or WiFi (Wireless Fidelity) networks. In some embodiments, the radio frequency circuit 304 may further include NFC (Near Field Communication) related circuits, which are not limited in this application.
The display screen 305 is used to display a UI (User Interface). The UI may include graphics, text, icons, video, and any combination thereof. When the display screen 305 is a touch display screen, the display screen 305 also has the ability to capture touch signals on or over the surface of the display screen 305. The touch signal may be input to the processor 301 as a control signal for processing. At this point, the display screen 305 may also be used to provide virtual buttons and/or a virtual keyboard, also referred to as soft buttons and/or a soft keyboard. In some embodiments, the display screen 305 may be one, the front panel of the electronic device; in other embodiments, the display screens 305 may be at least two, respectively disposed on different surfaces of the electronic device or in a folded design; in still other embodiments, the display 305 may be a flexible display disposed on a curved surface or a folded surface of the electronic device. Even further, the display screen 305 may be arranged in a non-rectangular irregular figure, i.e. a shaped screen. The Display 305 may be made of LCD (liquid crystal Display), OLED (Organic Light-Emitting Diode), and the like.
The power supply 306 is used to power various components in the electronic device. The power source 306 may be alternating current, direct current, disposable or rechargeable. When power source 306 comprises a rechargeable battery, the rechargeable battery may support wired or wireless charging. The rechargeable battery can also be used to support fast charge technology. Those skilled in the art will appreciate that the configuration shown in fig. 1 does not constitute a limitation of the video stream frame rate conversion apparatus, and may include more or less components than those shown, or combine some components, or arrange different components.
Furthermore, an embodiment of the present invention further provides a storage medium, on which a video stream frame rate conversion program is stored, and the video stream frame rate conversion program, when executed by a processor, implements the steps of the video stream frame rate conversion method as described above. Therefore, a detailed description thereof will be omitted. In addition, the beneficial effects of the same method are not described in detail. For technical details not disclosed in the embodiments of the storage medium referred to in the present application, reference is made to the description of the embodiments of the method of the present application. Determining by way of example, the program instructions may be deployed to be executed on one terminal device, or on multiple terminal devices located at one site, or distributed across multiple sites and interconnected by a communication network.
It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by a computer program, which can be stored in a storage medium and can include the processes of the embodiments of the methods described above when executed. The storage medium may be a magnetic disk, an optical disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), or the like.
Based on the hardware structure, the embodiment of the video stream frame rate conversion method is provided.
Referring to fig. 2, fig. 2 is a schematic flow chart of a video stream frame rate conversion method according to a first embodiment of the present invention; the method is applied to the terminal equipment and comprises the following steps:
step S11: the method comprises the steps of obtaining a target video stream, wherein the target video stream comprises a plurality of sub-video streams with preset duration.
It should be noted that the execution subject of the present invention is a terminal device, and may be any one of the above terminal devices, especially an intelligent television or an intelligent set-top box.
The target video stream generally refers to a video stream acquired by the terminal device from the server, and may include video streams of various formats, and typically, the frame rate of the target video stream is 24Hz (24 frames per second), 25Hz, or 30Hz, or the like. The video stream with the preset duration refers to that when frame rate conversion is performed on a target video stream, the basic conversion unit presets the duration to be 1s in order to ensure the fluency of a moving picture of an obtained result video stream, that is, the frame rate conversion unit is a video stream of one second.
Generally, the frame rate of an original video stream is 24Hz, in order to adapt to more various playing modes, various platforms convert the frame rate of the original video stream to obtain a video stream with 25Hz, 30Hz or the like, and store the video stream in a server, and when a terminal device acquires a target video stream from the server, the frame rate of the acquired target video stream is 25Hz, 30Hz or the like.
For example, when the server converts an original video stream with a frame rate of 24Hz into a video stream with a frame rate of 25Hz, a duplicated frame (obtained by duplicating any one of the 24 video frames) is inserted into 24 video frames included in the original video stream every second, so as to obtain a video stream with a frame rate of 25 Hz. Similarly, 6 copy frames are inserted to obtain a video stream with a frame rate of 30 Hz.
In general, when a terminal device performs frame rate conversion on a video stream, a video stream with a frame rate of 24Hz is converted into a video stream with a high frame rate, and because the target video stream has more repeated video frames (i.e., duplicated frames) per second, a moving picture of the high frame rate video stream obtained after directly performing frame rate conversion on the video stream with the frame rate of 25Hz or 30Hz is not smooth.
Step S12: and extracting key video frames of the plurality of sub-video streams to obtain a plurality of key video streams.
Before step S12, frame rate information of the plurality of sub-video streams needs to be acquired; and judging whether the frame rate information meets a preset condition or not.
Accordingly, step S12 includes: and when the frame rate information does not meet the preset condition, extracting key video frames of the plurality of sub-video streams to obtain a plurality of key video streams.
The frame rate information refers to a frame rate of a video stream, and a frame rate of a target video green acquired by the terminal device from the server may be 24Hz, 25Hz, or 30Hz, where only the target video stream with the frame rate of 24Hz is a video stream that does not need to be extracted by a key video frame, and for the video stream that does not need to be extracted by the key video frame, frame rate conversion is directly performed on a plurality of sub-video streams corresponding to the target video stream, that is: when the frame rate information meets the preset condition, performing frame rate conversion on the plurality of sub-video streams (the frame rate conversion method refers to a frame rate conversion method of a key video stream described below, which is not described herein), and obtaining a plurality of second output video streams at a preset frame rate, where one sub-video stream corresponds to one second output video stream; obtaining a second resulting video stream based on the plurality of second output video streams.
The preset condition means that the frame rates of the multiple sub-videos are 24Hz, and when the preset condition is not met, the frame rates of the sub-videos are 25Hz or 30Hz and the like; since the frame rate of the same target video stream acquired at the same time is usually constant, when determining whether the frame rates of the plurality of sub-video streams meet the preset condition, only the frame rate information of the first sub-video stream in the plurality of sub-video streams needs to be acquired and determined, and the frame rate information of the subsequent sub-video streams does not need to be acquired again.
It can be understood that when the preset time is 1s, performing key video frame extraction on each second of sub-video streams to obtain key video streams of the sub-videos, and performing key extraction on a plurality of sub-video streams corresponding to a target video respectively to obtain a plurality of key video streams; when the frame rate information meets the preset condition, the plurality of sub-video streams are subjected to frame rate conversion to directly obtain a plurality of corresponding second output video streams, the second result video stream is a video stream formed by combining the plurality of second output video streams according to a time sequence and forming a longer video stream, namely the video stream after the frame rate conversion corresponding to the target video stream, and the frame rates of the second output video stream and the second result video stream are the same.
For example, the preset time duration is 1s, the target video stream is a video stream of 30s, the corresponding sub-video streams are 30 (the sequence of the time of the sub-video streams), the frame rate information of the sub-video streams meets the preset condition, the number of the obtained second output video streams is also 30 (the second output video streams also have a time sequence, and correspond to the sequence of the time of the sub-video streams), and the 30 second output video streams are combined into a second result video stream of 30s according to the time sequence.
Further, step S12 includes: when the frame rate information does not meet the preset condition, performing copy frame screening on each sub-video stream in the plurality of sub-video streams to obtain a copy frame of each sub-video stream; deleting the copied frame of each sub-video stream to obtain a plurality of key video streams.
It should be noted that, generally, after the target video stream is obtained, when the frame rate information of the target video stream does not satisfy the preset condition, a processor (e.g., SOC) of the terminal device performs copy frame screening on each sub-video stream in a plurality of sub-video streams included in the target video stream, that is, a copy frame inserted by the server in each sub-video stream of the target video stream is screened out in each sub-video stream, and the copy frame is deleted, so as to obtain a key video stream, where the frame rate of the key video stream is 24Hz, for example, the frame rate of the sub-video stream is 25Hz, and after the deletion of the copy frame, one of the copy frames is deleted, so as to obtain the frame rate of the key video stream is 24 Hz.
In specific application, when the frame rate of the sub-video stream is 25Hz and the preset time is 1s, the processor of the terminal device compares 25 frames included in each sub-video stream, marks the frame without the repeated frame as 1, marks the frame repeated with other frames as 0, obtains a sequence number, deletes the frame corresponding to the frame marked as 0 in the sequence number, and obtains a key video stream corresponding to the sub-video stream, wherein the key video stream includes 24 non-repeated frames.
For example, when the frame rate of the sub-video stream is 25Hz and the preset time duration is 1s, after the sub-video stream is marked, the obtained sequence number is 1111111111011111111111111, the 11 th frame corresponding to 0 in the sub-video stream is deleted, and the key video stream corresponding to the sub-video stream is obtained, where the key video stream includes 24 non-repeating frames, and each frame is a key video frame.
Step S13: and performing frame rate conversion on the plurality of key video streams to obtain a plurality of first output video streams with preset frame rates.
Typically, after obtaining the key video stream, the processor of the terminal device sends the key video stream to a Frame Rate Converter (FRC), which performs frame rate conversion on the key video stream. The preset frame rate is a high frame rate, and the preset frame rate may be 60Hz, 75Hz, 90Hz, 120Hz, or the like, which is not limited in the present invention.
In the present embodiment, step S13 includes: performing image matching on each key video frame in each key video stream to obtain a displacement vector of each key video frame; and performing frame rate conversion on each key video stream based on the displacement vector to obtain a first output video stream with a preset frame rate corresponding to each key video stream.
The step of performing image matching on each key video frame in each key video stream to obtain a displacement vector of each key video frame includes: determining a plurality of non-overlapping macroblocks in each key video frame of each key video stream; and obtaining the displacement vector of each key video frame based on the macro block.
The step of obtaining the displacement vector of each key video frame based on the macro block comprises: acquiring the gray value of each key video frame; based on the macro blocks and the gray values, obtaining a displacement vector of each key video frame by using a formula I;
the first formula is as follows:
wherein (i, j) is the displacement vector, M × N is the size of the macroblock, and f k For the gray value of the k-th key video frame in each key video stream, f k-1 Is the gray value of the (k-1) th key video frame in each key video stream.
For any one of the plurality of key video streams, step S13 is executed to obtain a corresponding first output video stream.
It should be noted that, image matching is performed on 24 key video frames in each key video stream with a preset duration of 1s, to obtain a displacement vector, specifically, for any key video stream, a plurality of non-overlapping macro blocks are determined in each key video frame in the key video stream, and a corresponding displacement vector is obtained for each macro block, for example, the 3 rd key video frame of the first key video stream includes 7 macro blocks, and the corresponding displacement vector is the 7 displacement vectors; specifically, a corresponding displacement vector is obtained by using a formula one according to the gray values of the macro block and the key video frame.
And calculating the displacement vector for each key video frame in each key video stream with preset duration, and performing frame rate conversion by using the displacement vector to obtain first output video streams corresponding to the key video streams, wherein one key video stream corresponds to one first output video stream.
Referring to fig. 3, fig. 3 is a schematic diagram of the process of obtaining the displacement vector according to the present invention; the P frame is a key video frame currently undergoing frame rate conversion (i.e., frame interpolation compensation) in the key video frame, the reference frame is Pr, B is a macro block of the P frame, and the macro block B is a macro block in the Pr frame, which has the same coordinate position as the macro block B in the P frame in the video frame. In the Pr frame, a matching macro block Br with the minimum residual difference with the macro block B is searched according to the search criterion. Coordinate (x) of upper left corner of macro block Br r ，y r ) And the difference between the coordinate (x, y) of the left upper corner and the coordinate (x, y) of the macro block B is the displacement vector (i, j).
It is understood that the frame rate conversion described herein is motion compensation or frame interpolation compensation, and the three are a processing method.
Step S14: based on the plurality of first output video streams, a first resulting video stream is obtained.
After a plurality of first output video streams are obtained, combining the plurality of first video streams into a first result video stream corresponding to the target video stream according to the corresponding time sequence, wherein the frame rate of the first result video stream is the same as that of the first output video stream; for example, the preset time duration is 1s, the target video stream is a 60s video stream, the corresponding sub-video streams are 60 (the sequence of the sub-video streams is time), the frame rate information of the sub-video streams does not meet the preset condition, the number of the obtained key video streams is 60, the number of the obtained first output video streams is also 60 (the first output video streams also have a time sequence, and correspond to the sequence of the sub-video streams is time), and the 60 first output video streams are combined into a 60s first result video stream according to the time sequence.
According to the technical scheme, a video stream frame rate conversion method is adopted, and a target video stream is obtained, wherein the target video stream comprises a plurality of sub-video streams with preset time duration; extracting key video frames of the plurality of sub-video streams to obtain a plurality of key video streams; performing frame rate conversion on the plurality of key video streams to obtain first output video streams with a plurality of preset frame rates; based on the plurality of first output video streams, a first resulting video stream is obtained. Because the obtained plurality of first output video streams are obtained by performing frame rate conversion on the plurality of key video streams, the plurality of key video streams are obtained by performing key video frame extraction on the plurality of sub-video streams, and non-key video frames influencing the target video stream are deleted, the fluency of the obtained plurality of first output video streams is better, and further the motion pictures of the obtained first output video streams are fluent.
Referring to fig. 4, fig. 4 is a block diagram of a first embodiment of a video stream frame rate conversion apparatus according to the present invention, where the apparatus is applied to a terminal device, and the apparatus includes:
the obtaining module 10 is configured to obtain a target video stream, where the target video stream includes a plurality of sub-video streams with preset duration.
An extracting module 20, configured to perform key video frame extraction on the multiple sub-video streams to obtain multiple key video streams.
The frame rate conversion module 30 is configured to perform frame rate conversion on the plurality of key video streams to obtain a first output video stream with a plurality of preset frame rates.
An obtaining module 40 is configured to obtain a first resulting video stream based on the plurality of first output video streams.
The above description is only an alternative embodiment of the present invention, and not intended to limit the scope of the present invention, and all modifications and equivalents of the present invention, which are made by the contents of the present specification and the accompanying drawings, or directly/indirectly applied to other related technical fields, are included in the scope of the present invention.
Claims (9)
1. A method for frame rate conversion of a video stream, said method comprising the steps of:
acquiring a target video stream, wherein the target video stream comprises a plurality of sub-video streams with preset duration;
extracting key video frames of the plurality of sub-video streams to obtain a plurality of key video streams;
performing frame rate conversion on the plurality of key video streams to obtain first output video streams with a plurality of preset frame rates;
based on the plurality of first output video streams, a first resulting video stream is obtained.
2. The method for frame rate conversion of video streams according to claim 1, wherein before the step of extracting key video frames from the plurality of sub-video streams to obtain a plurality of key video streams, the method further comprises:
acquiring frame rate information of the plurality of sub-video streams;
judging whether the frame rate information meets a preset condition or not; the preset condition is that the frame rate of a plurality of sub-videos corresponding to the frame rate information is 24 Hz;
the step of extracting key video frames from the plurality of sub-video streams to obtain a plurality of key video streams comprises:
and when the frame rate information does not meet the preset condition, extracting key video frames of the plurality of sub-video streams to obtain a plurality of key video streams.
3. The method of converting a frame rate of a video stream according to claim 2, wherein after the step of determining whether the frame rate information satisfies a predetermined condition, the method further comprises:
when the frame rate information meets the preset condition, performing frame rate conversion on the plurality of sub-video streams to obtain a plurality of second output video streams with preset frame rates;
obtaining a second resulting video stream based on the plurality of second output video streams.
4. The method for converting frame rate of video stream according to claim 3, wherein said extracting key video frames from said plurality of sub-video streams when said frame rate information does not satisfy said preset condition, and obtaining a plurality of key video streams comprises:
when the frame rate information does not meet the preset condition, performing copy frame screening on each sub-video stream in the plurality of sub-video streams to obtain a copy frame of each sub-video stream;
deleting the copied frame of each sub-video stream to obtain a plurality of key video streams.
5. The method as claimed in claim 4, wherein the step of performing frame rate conversion on the plurality of key video streams to obtain a first output video stream with a plurality of preset frame rates comprises:
performing image matching on each key video frame in each key video stream to obtain a displacement vector of each key video frame;
performing frame rate conversion on each key video stream based on the displacement vector to obtain a first output video stream of a preset frame rate corresponding to each key video stream; the frame rate conversion of each key video stream is completed by a frame rate converter in a motion compensation or frame interpolation compensation mode;
wherein the step of performing image matching on each key video frame in each key video stream to obtain a displacement vector of each key video frame comprises:
determining a plurality of non-overlapping macroblocks in each key video frame of each key video stream;
and obtaining the displacement vector of each key video frame based on the macro block.
6. The method of claim 5, wherein the step of obtaining the displacement vector of each key video frame based on the macro block comprises:
acquiring the gray value of each key video frame;
based on the macro blocks and the gray values, obtaining a displacement vector of each key video frame by using a formula I;
the first formula is as follows:
wherein (i, j) is the displacement vector, M × N is the size of the macroblock, and f k For the gray value of the k-th key video frame in each key video stream, f k-1 Is the gray value of the (k-1) th key video frame in each key video stream.
7. A video stream frame rate conversion apparatus, the apparatus comprising:
the acquisition module is used for acquiring a target video stream, wherein the target video stream comprises a plurality of sub-video streams with preset duration;
the extraction module is used for extracting key video frames of the sub-video streams to obtain a plurality of key video streams;
the frame rate conversion module is used for performing frame rate conversion on the plurality of key video streams to obtain a plurality of first output video streams with preset frame rates;
an obtaining module configured to obtain a first resulting video stream based on the plurality of first output video streams.
8. A terminal device, characterized in that the terminal device comprises: a memory, a processor and a video stream frame rate conversion program stored on the memory and executable on the processor, the video stream frame rate conversion program being configured to implement the steps of the video stream frame rate conversion method according to any of claims 1 to 6.
9. A storage medium having stored thereon a video stream frame rate conversion program which, when executed by a processor, implements the steps of the video stream frame rate conversion method according to any one of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011069843.9A CN112218119B (en) | 2020-09-30 | 2020-09-30 | Video stream frame rate conversion method, device, terminal equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011069843.9A CN112218119B (en) | 2020-09-30 | 2020-09-30 | Video stream frame rate conversion method, device, terminal equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112218119A CN112218119A (en) | 2021-01-12 |
| CN112218119B true CN112218119B (en) | 2022-09-09 |
Family
ID=74052803
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011069843.9A Active CN112218119B (en) | 2020-09-30 | 2020-09-30 | Video stream frame rate conversion method, device, terminal equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112218119B (en) |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104837031A (en) * | 2015-04-08 | 2015-08-12 | 中国科学院信息工程研究所 | Method for high-speed self-adaptive video keyframe extraction |
| CN110290421A (en) * | 2019-08-01 | 2019-09-27 | 北京达佳互联信息技术有限公司 | Frame per second method of adjustment, device, computer equipment and storage medium |
Family Cites Families (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP4353253B2 (en) * | 2007-01-31 | 2009-10-28 | ソニー株式会社 | FRAME RATE CONTROL DEVICE, FRAME RATE CONTROL METHOD, RECORDING DEVICE, AND IMAGING DEVICE |
| JP2013207466A (en) * | 2012-03-28 | 2013-10-07 | Casio Comput Co Ltd | Video playback apparatus and program |
| CN103826078A (en) * | 2014-03-11 | 2014-05-28 | 深圳瑞信视讯技术有限公司 | Linear storage method, device and terminal for video data |
| CN105072345B (en) * | 2015-08-25 | 2018-12-25 | 深圳市通则技术股份有限公司 | Method for video coding and device |
| CN105554592B (en) * | 2016-02-02 | 2019-03-01 | 深圳市指点信息科技有限公司 | A kind of method and system for realizing high frame-rate video Image Acquisition transmission |
| CN106778686A (en) * | 2017-01-12 | 2017-05-31 | 深圳职业技术学院 | A kind of copy video detecting method and system based on deep learning and graph theory |
| CN110270092A (en) * | 2019-06-27 | 2019-09-24 | 三星电子（中国）研发中心 | The method and device and electronic equipment that frame per second for electronic equipment is promoted |
-
2020
- 2020-09-30 CN CN202011069843.9A patent/CN112218119B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104837031A (en) * | 2015-04-08 | 2015-08-12 | 中国科学院信息工程研究所 | Method for high-speed self-adaptive video keyframe extraction |
| CN110290421A (en) * | 2019-08-01 | 2019-09-27 | 北京达佳互联信息技术有限公司 | Frame per second method of adjustment, device, computer equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112218119A (en) | 2021-01-12 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN105557058B (en) | Electronic device and its communication connecting method | |
| CN113382261A (en) | Method and device for decoding and encoding prediction mode | |
| CN110852951B (en) | Image processing method, device, terminal equipment and computer readable storage medium | |
| CN107948724B (en) | Video transmission control method, device, storage medium and mobile terminal | |
| WO2020134789A1 (en) | Mobile terminal and method for controlling on and off of screen, and computer storage medium | |
| CN113099233A (en) | Video encoding method, video encoding device, video encoding apparatus, and storage medium | |
| CN113014830A (en) | Video blurring method, device, equipment and storage medium | |
| CN111263216B (en) | Video transmission method, device, storage medium and terminal | |
| CN112218119B (en) | Video stream frame rate conversion method, device, terminal equipment and storage medium | |
| CN112347049A (en) | File information display method, terminal device and computer-readable storage medium | |
| CN114598876B (en) | Motion compensation method and device for dynamic image, terminal equipment and storage medium | |
| CN114339966A (en) | Interface control method, device, medium and electronic equipment for data transmission | |
| CN113393809A (en) | Display terminal control method, display terminal control device, display terminal and storage medium | |
| CN112532971A (en) | Image processing method, image processing device, display terminal and computer readable storage medium | |
| CN109040753B (en) | Prediction mode selection method, device and storage medium | |
| CN202218348U (en) | Mobile terminal with self-timer three-dimensional (3D) image effect | |
| CN107682631B (en) | Image processing method and mobile terminal | |
| CN114125346B (en) | Video conversion method and device | |
| CN114495859B (en) | Picture display method, device, display terminal and storage medium | |
| CN113763230B (en) | Image style migration model training method, style migration method and device | |
| CN112423004B (en) | Video data transmission method, device, transmitting end and storage medium | |
| WO2024066834A1 (en) | Vsync signal control method, electronic device, storage medium and chip | |
| CN117724627A (en) | Wallpaper preview interface display method and electronic equipment | |
| CN112346865A (en) | File acquisition method and device, terminal equipment and storage medium | |
| CN117573244A (en) | Screen display method, terminal device and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |