CN103108148B - The frame interpolation method of frame of video and messaging device - Google Patents
The frame interpolation method of frame of video and messaging device Download PDFInfo
- Publication number
- CN103108148B CN103108148B CN201110360319.1A CN201110360319A CN103108148B CN 103108148 B CN103108148 B CN 103108148B CN 201110360319 A CN201110360319 A CN 201110360319A CN 103108148 B CN103108148 B CN 103108148B
- Authority
- CN
- China
- Prior art keywords
- video
- frame
- data
- effective
- video data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Landscapes
- Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
Abstract
The frame interpolation method of frame of video and messaging device, described frame interpolation method comprises: with the predetermined cycle, obtains the first frame of video from camera module; The second frame of video and the 3rd frame of video is produced based on described first frame of video, wherein said second frame of video comprises the effective video data of the odd-numbered line at least partially of described first frame of video, described 3rd frame of video comprises the effective video data of the even number line at least partially of described first frame of video, and the described effective video data of described second frame of video and the 3rd video form the complete effective video data needed for described first frame of video; Utilize other row except described effective video data in described second frame of video of black data filling; Utilize other row except described effective video data in described 3rd frame of video of black data filling; And order inserts described second frame of video and the 3rd frame of video between current first frame of video and next the first frame of video.
Description
Technical field
The present invention relates to a kind of frame interpolation method for frame of video and use the messaging device of the method.
Background technology
Current, universal due to Video chat and video conference application, uses the camera module that such as notebook or mobile phone are arranged to carry out video session and is becoming more and more general.
In darker environment, because camera module is subject to the restriction of its photoperceptivity, if ensure that the object in each frame of video is fully exposed, then need the longer time for exposure, the frame per second (frame number of unit interval shooting) of the video causing the unit interval to be taken thus is very low.Now, user can feel that picture is discontinuous.In addition, in this case, if camera module adopts higher frame per second, then the object in each frame of video is under-exposed, thus causes the picture of frame of video very dim.
In addition, even if in well-lighted situation, due to the hardware capabilities (e.g., 20FPS) of camera module, then the video that this camera module produces also links up not.
Summary of the invention
In order to solve above-mentioned technical problem of the prior art, according to an aspect of the present invention, a kind of frame interpolation method of frame of video being provided, comprising: with the predetermined cycle, obtaining the first frame of video from camera module; The second frame of video and the 3rd frame of video is produced based on described first frame of video, wherein said second frame of video comprises the effective video data of the odd-numbered line at least partially of described first frame of video, described 3rd frame of video comprises the effective video data of the even number line at least partially of described first frame of video, and the described effective video data of described second frame of video and the 3rd video form the complete effective video data needed for described first frame of video; Utilize other row except described effective video data in described second frame of video of black data filling; Utilize other row except described effective video data in described 3rd frame of video of black data filling; And order inserts described second frame of video and the 3rd frame of video between current first frame of video and next the first frame of video.
In addition, according to one embodiment of present invention, the frame interpolation method of frame of video comprises further: produce the 4th frame of video based on described current first frame of video and next first frame of video described.
In addition, according to one embodiment of present invention, wherein
Based on the mathematic interpolation motion vector data between described current first frame of video and next first frame of video described; And produce the 4th frame of video based on described motion vector data.
In addition, according to another embodiment of the invention, the frame interpolation method of frame of video comprises further:
The 5th frame of video and the 6th frame of video is produced based on described 4th frame of video, wherein said 5th frame of video comprises the effective video data of the odd-numbered line at least partially of described 4th frame of video, described 6th frame of video comprises the effective video data of the even number line at least partially of described 4th frame of video, and the described effective video data of described 5th frame of video and the 6th video form the complete effective video data needed for described 4th frame of video; Utilize other row except described effective video data in described 5th frame of video of black data filling; Utilize other row except described effective video data in described 6th frame of video of black data filling; And order inserts described second frame of video, the 3rd frame of video, the 4th frame of video, the 5th frame of video and the 6th frame of video between described current first frame of video and next first frame of video described.
In addition, according to a further aspect in the invention, provide a kind of messaging device, comprising: camera module, configuration carrys out capture video, and produces vision signal, Graphics Processing Unit, configuration receives the vision signal that described camera module produces, and produces the first frame of video based on described vision signal with the predetermined cycle, and processing unit, configuration produces the second frame of video and the 3rd frame of video based on described first frame of video, and order inserts described second frame of video and the 3rd frame of video between current first frame of video and next the first frame of video, wherein said second frame of video comprises the effective video data of the odd-numbered line at least partially of described first frame of video, described 3rd frame of video comprises the effective video data of the even number line at least partially of described first frame of video, and the described effective video data of described second frame of video and the 3rd video form the complete effective video data needed for described first frame of video, other row in described second frame of video except described effective video data is filled by black data, and other row in described 3rd frame of video except described effective video data is filled by black data.
In addition, according to one embodiment of present invention, wherein said processing unit produces the 4th frame of video based on described current first frame of video and next first frame of video described.
In addition, according to one embodiment of present invention, wherein said processing unit based on the mathematic interpolation motion vector data between described current first frame of video and next first frame of video described, and produces the 4th frame of video based on described motion vector data.
In addition, according to one embodiment of present invention, wherein said processing unit produces the 5th frame of video and the 6th frame of video based on described 4th frame of video, and order inserts described second frame of video between described current first frame of video and next first frame of video described, 3rd frame of video, 4th frame of video, 5th frame of video and the 6th frame of video, wherein said 5th frame of video comprises the effective video data of the odd-numbered line at least partially of described 4th frame of video, and described 6th frame of video comprises the effective video data of the even number line at least partially of described 4th frame of video, and the described effective video data of described 5th frame of video and the 6th video form the complete effective video data needed for described 4th frame of video, other row in described 5th frame of video except described effective video data is filled by black data, and other row in described 6th frame of video except described effective video data is filled by black data.
In addition, according to a further aspect in the invention, provide a kind of messaging device, comprising: camera module, configuration carrys out capture video, and produces vision signal, Graphics Processing Unit, configuration receives the vision signal that described camera module produces, and produces the first frame of video based on described vision signal with the predetermined cycle, and frame of video plug-in unit, configuration produces the second frame of video and the 3rd frame of video based on described first frame of video, and order inserts described second frame of video and the 3rd frame of video between current first frame of video and next the first frame of video, wherein said second frame of video comprises the effective video data of the odd-numbered line at least partially of described first frame of video, described 3rd frame of video comprises the effective video data of the even number line at least partially of described first frame of video, the described effective video data of described second frame of video and the 3rd video form the complete effective video data needed for described first frame of video, and other row in described second frame of video except described effective video data is filled by black data, other row in described 3rd frame of video except described effective video data is filled by black data.
By above-mentioned configuration, extract a part for original frame of video, effective video data as odd-numbered line or even number line form new frame of video, and make new frame of video have the size (resolution) of original video frame the remaining rows black data filling in new frame of video.By producing new frame of video from original video frame, and between original video frame, inserting these new frame of video, effectively can improve the frame per second of video.In addition, because new frame of video also comprises original video frame at least partially, therefore user can not be made not feel well when watching, the experience sense which thereby enhancing user is subject to.
Accompanying drawing explanation
Fig. 1 is the schematic block diagram of diagram messaging device according to an embodiment of the invention;
Fig. 2 is the schematic block diagram of diagram messaging device in accordance with another embodiment of the present invention; And
Fig. 3 is the flow chart of the frame interpolation method of diagram frame of video according to an embodiment of the invention.
Embodiment
Describe in detail with reference to the accompanying drawings according to each embodiment of the present invention.Here it is to be noted that it in the accompanying drawings, identical Reference numeral is given there is identical or similar structures and function part substantially, and the repeated description of will omit about them.
Below with reference to Fig. 1, the messaging device according to the embodiment of the present invention is described.Fig. 1 is the schematic block diagram of diagram messaging device according to an embodiment of the invention.
As shown in Figure 1, the messaging device of such as PC, notebook, panel computer or mobile phone and so at least can comprise camera module 1, Graphics Processing Unit 2 and processing unit 3, wherein camera module 1 can be connected with Graphics Processing Unit 2, and Graphics Processing Unit 2 can be connected with processing unit 3.
Here, camera module 1 can be the camera module of any type, and based on the control shooting still image of software or video to produce image or vision signal (e.g., photoreceptor signal), and produced image or vision signal can be sent to Graphics Processing Unit 2.
Graphics Processing Unit 2 can be realized by the arbitrary image signal processor mated with camera module 1, and can based on picture signal or vision signal produce have predetermined format (as, jpeg) image or have predetermined format (as, mpeg2, mpeg4 etc.) video.When the abundant exposure mode capture video of camera module 1, Graphics Processing Unit 2 can based on the time for exposure of camera module 1, multiple original frame of video (being called for short the first frame of video below) is produced, to form continuous print video pictures with the cycle sequences corresponding with the time for exposure.
Processing unit 3 can be realized by arbitrary processor, microprocessor.According to embodiments of the invention, processing unit 3 can under the control of predetermined software, current first frame of video that graphic based processing unit 2 produces produces two new frame of video (being called for short the second frame of video and the 3rd frame of video below), and order inserts this second frame of video and the 3rd frame of video to produce new video flowing between current first frame of video and next the first frame of video.
Such as, according to one embodiment of present invention, the second frame of video can comprise the effective video data of the odd-numbered line of current first frame of video, and the 3rd frame of video can comprise the effective video data of the even number line of current first frame of video.Here, the effective video data in the second frame of video and the 3rd frame of video refer to the initial data in the first frame of video.That is, when being combined by the valid data in the second frame of video and the 3rd frame of video, the effective video data of the second frame of video and the 3rd video form all effective video data needed for the first complete frame of video.In other words, respectively comprise a part of valid data in the first frame of video in the second frame of video and the 3rd frame of video, and the two is complementary.
In addition, in order to ensure that the second frame of video and the 3rd frame of video have and have identical resolution with the first frame of video, by in the second frame of video except effective video data (as, the valid data of odd-numbered line) outer other row (as, even number line) filled by black data, and filled removing effective video data (e.g., the valid data of even number line) other row outward (e.g., odd-numbered line) in the 3rd frame of video by black data.
Particularly, for the second frame of video, the second frame of video can be produced by the effective video data utilizing black data to fill the even number line of the first frame of video.In addition, for the 3rd frame of video, the 3rd frame of video can be produced by the effective video data utilizing black data to fill the odd-numbered line of the first frame of video.In addition, the present invention is not limited thereto, can also by the data of the effective video data of the odd-numbered line of extraction first frame of video as the corresponding odd-numbered line of the second frame of video, and utilize black data to fill even number line and produce the second frame of video, and can by the data of the effective video data of the even number line of extraction first frame of video as the corresponding even number line of the 3rd frame of video, and utilize black data filling odd-numbered line to produce the 3rd frame of video.
After processing unit 3 produces the second frame of video and the 3rd frame of video based on current first frame of video, processing unit 3 sequentially can insert this second frame of video and the 3rd frame of video to produce new video flowing between current first frame of video and next first frame of video.Now, the display (not shown) that new video flowing can be supplied to messaging device shows, or can provide it to the remote equipment that is connected with messaging device to realize Video chat or video conference.
Be described above the second frame of video and comprise the situation that the effective video data of the odd-numbered line of current first frame of video and the 3rd frame of video comprise the effective video data of the even number line of current first frame of video.The second frame of video and the 3rd frame of video can also be produced from the first frame of video based on alternate manner.Such as, can controlled processing unit 3, the effective video data of the effective video data and half even number line that make processing unit 3 can comprise the half odd-numbered line of the first frame of video based on the second frame of video that current first frame of video produces are used as the odd-numbered line of its correspondence and the video data of even number line.In addition, the effective video data of the 3rd frame of video effective video data and half even number line that also can comprise the half odd-numbered line of the first frame of video are used as the odd-numbered line of its correspondence and the video data of even number line.Here, as long as the second frame of video and the 3rd frame of video comprise a part of valid data in the first frame of video respectively on the position of its correspondence, and the two be complementary (namely, the effective video data of the second frame of video and the 3rd video form all effective video data needed for the first complete frame of video), and remaining row black data in the second frame of video and the 3rd frame of video is filled, just at random can determine that the second frame of video and the 3rd frame of video comprise the video data of which part in the first frame of video respectively, and the remainder in the second frame of video and the 3rd frame of video is filled by black data.
By above-mentioned configuration, utilize a part for original video frame (the first frame of video), effective video data as odd-numbered line or even number line formed multiple new frame of video (as, second and the 3rd frame of video), and the remaining rows black data in new frame of video filled make new frame of video have the size (resolution) of original video frame.After producing new frame of video, between original video frame, insert the frame of video that these are new, effectively can improve the frame per second of video.Such as, when light is bad, usual camera module 1 can only produce the video of 10 ~ 15 frames per second, then according to embodiments of the invention, the frame per second of video can be brought up to 30 ~ 45FPS from 10 ~ 15FPS, the situation making user can not experience picture card thus to pause.
In addition, owing to also comprising in new frame of video (real picture) at least partially of original video frame, therefore user can not be made to feel the exception of picture when watching, the experience sense which thereby enhancing user is subject to.
In addition, according to another embodiment of the invention, processing unit 3 current first frame of video that provides of graphic based processing unit 2 and next the first frame of video can also produce intermediate video frame (, being called for short the 4th frame of video below).Particularly, processing unit 3 based on the mathematic interpolation motion vector data between current first frame of video and next the first frame of video, and can produce the 4th frame of video based on motion vector data.Here, because the mode inserting intermediate video frame based on motion vector is known to those skilled in the art, therefore only it is simply introduced here.
Such as, processing unit 3 can produce intermediate video frame based on next frame of video of the first current obtained frame of video and this first frame of video obtained subsequently.Particularly, difference between current first frame of video and next the first frame of video can be extracted to determine moving region, and after determining moving region, motion vector (the direction of moving object is determined in change based on the characteristic value in moving region, distance etc.), and calculate the position of moving object in intermediate video frame (the 4th frame of video) based on obtained motion vector.In addition, current first frame of video and next the first frame of video can also be divided into multiple block, and the difference between comparison blocks determines moving region, and after determining moving region, the motion vector of moving object is determined in the change based on the characteristic value in moving region.Then the position of moving object in the 4th frame of video is calculated based on obtained motion vector.Here the corresponding region that there is not difference in current first frame of video and next the first frame of video is stagnant zone, therefore, after obtaining the position of moving object in the 4th frame of video and stagnant zone, can be easy to produce the 4th frame of video, i.e. intermediate video frame from current first frame of video or next the first frame of video.
Then, similar to the previous description, processing unit 3 produces two new frame of video (being called for short the 5th frame of video and the 6th frame of video below) based on the 4th frame of video, and order inserts the second frame of video, the 3rd frame of video, the 4th frame of video, the 5th frame of video and the 6th frame of video to produce new video flowing between current first frame of video and next the first frame of video.
Such as, according to one embodiment of present invention, the 5th frame of video can comprise the effective video data of the odd-numbered line of the 4th frame of video, and the 6th frame of video can comprise the effective video data of the even number line of the 4th frame of video.Here, the effective video data in the 5th frame of video and the 6th frame of video refer to the initial data in the 4th frame of video.In addition, when being combined by the valid data in the 5th frame of video and the 6th frame of video, the effective video data of the 5th frame of video and the 6th video form all effective video data needed for the 4th complete frame of video.In other words, respectively comprise a part of valid data in the 4th frame of video in the 5th frame of video and the 6th frame of video, and the two is complementary.
In addition, in order to ensure that the 5th frame of video and the 6th frame of video have and have identical resolution with the 4th frame of video, by in the 5th frame of video except effective video data (as, the valid data of odd-numbered line) outer other row (as, even number line) filled by black data, and filled removing effective video data (e.g., the valid data of even number line) other row outward (e.g., odd-numbered line) in the 6th frame of video by black data.
Particularly, for the 5th frame of video, the 5th frame of video can be produced by the effective video data utilizing black data to fill the even number line of the 4th frame of video.In addition, for the 6th frame of video, the 6th frame of video can be produced by the effective video data utilizing black data to fill the odd-numbered line of the 4th frame of video.In addition, can also by the data of the effective video data of the odd-numbered line of extraction the 4th frame of video as the corresponding odd-numbered line of the 5th frame of video, and utilize black data to fill even number line and produce the 5th frame of video, and can by the data of the effective video data of the even number line of extraction the 4th frame of video as the corresponding even number line of the 6th frame of video, and utilize black data filling odd-numbered line to produce the 6th frame of video.
Here, after processing unit 3 produces the 5th frame of video and the 6th frame of video based on the 4th frame of video, processing unit 3 can sequentially insert the second frame of video, the 3rd frame of video, the 4th frame of video, the 5th frame of video and the 6th frame of video to produce new video flowing between current first frame of video and next first frame of video.Now, the display (not shown) that new video flowing can be supplied to messaging device shows, or can provide it to the remote equipment that is connected with messaging device to realize Video chat or video conference.
Be described above the 5th frame of video and comprise the situation that the effective video data of the odd-numbered line of the 4th frame of video and the 6th frame of video comprise the effective video data of the even number line of the 4th frame of video.The 5th frame of video and the 6th frame of video can also be produced from the 4th frame of video based on alternate manner.Here, as long as the 5th frame of video and the 6th frame of video comprise a part of valid data in the 4th frame of video respectively on the position of its correspondence, and the two be complementary (namely, the effective video data of the 5th frame of video and the 6th video form all effective video data needed for the 4th complete frame of video), and remaining row black data in the 5th frame of video and the 6th frame of video is filled, just at random can determine that the 5th frame of video and the 6th frame of video comprise the video data of which part in the first frame of video respectively, and the remainder in the 5th frame of video and the 6th frame of video is filled by black data.
By above-mentioned configuration, even if bad at light, when camera module 1 can only produce the video of 10 ~ 15 frames per second, according to the present embodiment, the frame per second of video can be brought up to 60 ~ 90FPS from 10 ~ 15FPS, the situation making user can not experience picture card completely thus to pause.
In addition, because the present embodiment further increases the frame per second of video, therefore the displaying time of each frame of video shortens accordingly.The time that the benefit shortening the displaying time of each frame of video is to show the frame of video with black data reduces further.Due to the characteristic that human eye vision is residual, shorten the displaying time (e.g., be less than 15ms, namely correspond to 60FPS) with the frame of video of black data, human eye can not discover the black data of filling in frame of video, further increases the impression that user watches video thus.
In addition, the present invention is not limited thereto.Fig. 2 is the schematic block diagram of diagram messaging device in accordance with another embodiment of the present invention.
The difference of the messaging device shown in the messaging device shown in Fig. 2 and Fig. 1 is: the messaging device of Fig. 2 comprises frame of video plug-in unit 4 further.Here, because the 26S Proteasome Structure and Function of camera module 1, Graphics Processing Unit 2 is identical with the corresponding component shown in Fig. 1, therefore here, the repeated description about them is eliminated.
Such as, frame of video plug-in unit 4 can be realized by DSP, and is connected with Graphics Processing Unit 2 and processing unit 3.Here, according to the present embodiment, frame of video plug-in unit 4 can replace processing unit 3 to carry out interleave operation.Particularly, the internal logic (firmware) that can configure frame of video plug-in unit 4 makes frame of video plug-in unit 4 current first frame of video that produces of graphic based processing unit 2 can produce two new frame of video (being called for short the second frame of video and the 3rd frame of video below), and order inserts this second frame of video and the 3rd frame of video to produce new video flowing between current first frame of video and next the first frame of video.
Here, the second frame of video can comprise the effective video data of the odd-numbered line of current first frame of video, and the 3rd frame of video can comprise the effective video data of the even number line of current first frame of video.In addition, in order to ensure that the second frame of video and the 3rd frame of video have and have identical resolution with the first frame of video, by in the second frame of video except effective video data (as, the valid data of odd-numbered line) outer other row (as, even number line) filled by black data, and filled removing effective video data (e.g., the valid data of even number line) other row outward (e.g., odd-numbered line) in the 3rd frame of video by black data.
After frame of video plug-in unit 4 produces the second frame of video and the 3rd frame of video based on current first frame of video, frame of video plug-in unit 4 sequentially can insert this second frame of video and the 3rd frame of video to produce new video flowing between current first frame of video and next first frame of video.Now, produced new video flowing can be supplied to processing unit 3 by frame of video plug-in unit 4.Then the display (not shown) new video flowing being supplied to messaging device by processing unit 3 shows, or can provide it to the remote equipment that is connected with messaging device to realize Video chat or video conference.By arranging frame of video plug-in unit 4, the burden of processing unit 3 can be reduced, thus can ensure to provide produced new video flowing glibly.
Be described above the messaging device according to each embodiment of the present invention, below, with reference to Fig. 3, the frame interpolation method according to the frame of video of the embodiment of the present invention is described.Fig. 3 is the flow chart of the frame interpolation method of diagram frame of video according to an embodiment of the invention.
As shown in Figure 3, in step S301, with the predetermined cycle, obtain the first frame of video from camera module.
Particularly, as previously described, camera module 1 takes still image or video under software control to produce image or vision signal (e.g., photoreceptor signal), and produced image or vision signal are sent to Graphics Processing Unit 2.
Then, Graphics Processing Unit 2 receives this vision signal, and produce the multiple original frame of video (being called for short the first frame of video below) with predetermined format (e.g., mpeg4), to form continuous print video pictures with predetermined cycle sequences.
In step S302, the second frame of video and the 3rd frame of video is produced based on the first frame of video, wherein the second frame of video comprises the effective video data of the odd-numbered line at least partially of described first frame of video, 3rd frame of video comprises the effective video data of the even number line at least partially of described first frame of video, other row in second frame of video except described effective video data is filled by black data, and other row in the 3rd frame of video except described effective video data is filled by black data.
Particularly, processing unit 3 (or frame of video plug-in unit 4) produces the second frame of video and the 3rd frame of video based on current first frame of video.Here, the second frame of video that processing unit 3 produces can comprise the effective video data of the odd-numbered line of current first frame of video, and the 3rd frame of video can comprise the effective video data of the even number line of current first frame of video.In addition, processing unit 3 by the second frame of video except effective video data (as, the valid data of odd-numbered line) outer other row (as, even number line) filled by black data, and by the 3rd frame of video except effective video data (as, the valid data of even number line) outer other row (e.g., odd-numbered line) filled by black data.
Here, the second frame of video can be produced by the effective video data utilizing black data to fill the even number line of the first frame of video, and the 3rd frame of video can be produced by the effective video data utilizing black data to fill the odd-numbered line of the first frame of video.In addition, can also by the data of the effective video data of the odd-numbered line of extraction first frame of video as the corresponding odd-numbered line of the second frame of video, and utilize black data to fill even number line and produce the second frame of video, and can by the data of the effective video data of the even number line of extraction first frame of video as the corresponding even number line of the 3rd frame of video, and utilize black data filling odd-numbered line to produce the 3rd frame of video.
Be described above the second frame of video and comprise the situation that the effective video data of the odd-numbered line of current first frame of video and the 3rd frame of video comprise the effective video data of the even number line of current first frame of video.Here, as long as the second frame of video and the 3rd frame of video comprise a part of valid data in the first frame of video respectively on the position of its correspondence, and the two be complementary (namely, the effective video data of the second frame of video and the 3rd video form all effective video data needed for the first complete frame of video), and remaining row black data in the second frame of video and the 3rd frame of video is filled, just at random can determine that the second frame of video and the 3rd frame of video comprise the video data of which part in the first frame of video respectively, and the remainder in the second frame of video and the 3rd frame of video is filled by black data.
Then, in step S303, between current first frame of video and next the first frame of video, order inserts the second frame of video and the 3rd frame of video.
Particularly, processing unit 3 (or frame of video plug-in unit 4) sequentially can insert this second frame of video and the 3rd frame of video to produce new video flowing between current first frame of video and next first frame of video.Here, the display (not shown) that new video flowing can be supplied to messaging device shows, or can provide it to the remote equipment that is connected with messaging device to realize Video chat or video conference.
In addition, according to another embodiment of the invention, the method shown in Fig. 3 can further include step: produce the 4th frame of video based on current first frame of video and next the first frame of video.
Particularly, processing unit 3 can extract difference between current first frame of video and next the first frame of video to determine moving region, and after determining moving region, motion vector (the direction of moving object is determined in change based on the characteristic value in moving region, distance etc.), and calculate the position of moving object in intermediate video frame (the 4th frame of video) based on obtained motion vector.In addition, current first frame of video and next the first frame of video can also be divided into multiple block, and the difference between comparison blocks determines moving region, and after determining moving region, the motion vector of moving object is determined in the change based on the characteristic value in moving region.Then the position of moving object in the 4th frame of video is calculated based on obtained motion vector.Here, because the corresponding region that there is not difference in current first frame of video and next the first frame of video is stagnant zone, therefore, after obtaining the position of moving object in the 4th frame of video and stagnant zone, can be easy to produce the 4th frame of video, i.e. intermediate video frame from current first frame of video or next the first frame of video.
In this case, the method shown in Fig. 3 can further include step:
The 5th frame of video and the 6th frame of video is produced based on the 4th frame of video, wherein the 5th frame of video comprises the effective video data of the odd-numbered line at least partially of the 4th frame of video, 6th frame of video comprises the effective video data of the even number line at least partially of the 4th frame of video, other row in 5th frame of video except described effective video data is filled out by black data, and other row in the 6th frame of video except effective video data is filled by black data; And order inserts described second frame of video, the 3rd frame of video, the 4th frame of video, the 5th frame of video and the 6th frame of video between current first frame of video and next the first frame of video.
Particularly, processing unit 3 produces the 5th frame of video and the 6th frame of video based on the 4th frame of video, and order inserts the second frame of video, the 3rd frame of video, the 4th frame of video, the 5th frame of video and the 6th frame of video to produce new video flowing between current first frame of video and next the first frame of video.
Here, the 5th frame of video can comprise the effective video data of the odd-numbered line of the 4th frame of video, and the 6th frame of video can comprise the effective video data of the even number line of the 4th frame of video.In addition, when being combined by the valid data in the 5th frame of video and the 6th frame of video, the effective video data of the 5th frame of video and the 6th video form all effective video data needed for the 4th complete frame of video.In other words, respectively comprise a part of valid data in the 4th frame of video in the 5th frame of video and the 6th frame of video, and the two is complementary.In addition, in 5th frame of video except effective video data (as, the valid data of odd-numbered line) outer other row (as, even number line) filled by black data, and in the 6th frame of video except effective video data (as, the valid data of even number line) outer other row (e.g., odd-numbered line) filled by black data.
Particularly, the 5th frame of video can be produced by the effective video data utilizing black data to fill the even number line of the 4th frame of video.In addition, the 6th frame of video can be produced by the effective video data utilizing black data to fill the odd-numbered line of the 4th frame of video.In addition, can also by the data of the effective video data of the odd-numbered line of extraction the 4th frame of video as the corresponding odd-numbered line of the 5th frame of video, and utilize black data to fill even number line and produce the 5th frame of video, and can by the data of the effective video data of the even number line of extraction the 4th frame of video as the corresponding even number line of the 6th frame of video, and utilize black data filling odd-numbered line to produce the 6th frame of video.
Then, after processing unit 3 produces the 5th frame of video and the 6th frame of video based on the 4th frame of video, processing unit 3 sequentially inserts the second frame of video, the 3rd frame of video, the 4th frame of video, the 5th frame of video and the 6th frame of video to produce new video flowing between current first frame of video and next first frame of video.The display (not shown) that the new video stream produced can be supplied to messaging device shows, or can provide it to the remote equipment that is connected with messaging device to realize Video chat or video conference.
Be described above the 5th frame of video and comprise the situation that the effective video data of the odd-numbered line of the 4th frame of video and the 6th frame of video comprise the effective video data of the even number line of the 4th frame of video.The 5th frame of video and the 6th frame of video can also be produced from the 4th frame of video based on alternate manner.Here, as long as the 5th frame of video and the 6th frame of video comprise a part of valid data in the 4th frame of video respectively on the position of its correspondence, and the two be complementary (namely, the effective video data of the 5th frame of video and the 6th video form all effective video data needed for the 4th complete frame of video), and remaining row black data in the 5th frame of video and the 6th frame of video is filled, just at random can determine that the 5th frame of video and the 6th frame of video comprise the video data of which part in the first frame of video respectively, and the remainder in the 5th frame of video and the 6th frame of video is filled by black data.
Each embodiment of the present invention is described in detail above.But, it should be appreciated by those skilled in the art that without departing from the principles and spirit of the present invention, various amendment can be carried out to these embodiments, combination or sub-portfolio, and such amendment should fall within the scope of the present invention.
Claims (9)
1. a frame interpolation method for frame of video, comprising:
With the predetermined cycle, obtain the first frame of video from camera module;
The second frame of video and the 3rd frame of video is produced based on described first frame of video, wherein said second frame of video comprises the effective video data of the odd-numbered line at least partially of described first frame of video, described 3rd frame of video comprises the effective video data of the even number line at least partially of described first frame of video, and the described effective video data of described second frame of video and the 3rd video form the complete effective video data needed for described first frame of video, other row in described second frame of video except described effective video data is filled by black data, and other row in described 3rd frame of video except described effective video data is filled by black data, and
Between current first frame of video and next the first frame of video, order inserts described second frame of video and the 3rd frame of video.
2. the method for claim 1, comprises further:
The 4th frame of video is produced based on described current first frame of video and next first frame of video described.
3. method as claimed in claim 2, wherein
Based on the mathematic interpolation motion vector data between described current first frame of video and next first frame of video described; And
The 4th frame of video is produced based on described motion vector data.
4. method as claimed in claim 3, comprises further:
The 5th frame of video and the 6th frame of video is produced based on described 4th frame of video, wherein said 5th frame of video comprises the effective video data of the odd-numbered line at least partially of described 4th frame of video, described 6th frame of video comprises the effective video data of the even number line at least partially of described 4th frame of video, the described effective video data of described 5th frame of video and the 6th video form the complete effective video data needed for described 4th frame of video, other row in described 5th frame of video except described effective video data is filled by black data, and other row in described 6th frame of video except described effective video data is filled by black data, and
Between described current first frame of video and next first frame of video described, order inserts described second frame of video, the 3rd frame of video, the 4th frame of video, the 5th frame of video and the 6th frame of video.
5. a messaging device, comprising:
Camera module, configuration carrys out capture video, and produces vision signal;
Graphics Processing Unit, configuration receives the vision signal that described camera module produces, and produces the first frame of video based on described vision signal with the predetermined cycle; And
Processing unit, configuration produces the second frame of video and the 3rd frame of video based on described first frame of video, and order inserts described second frame of video and the 3rd frame of video between current first frame of video and next the first frame of video,
Wherein said second frame of video comprises the effective video data of the odd-numbered line at least partially of described first frame of video, described 3rd frame of video comprises the effective video data of the even number line at least partially of described first frame of video, and the described effective video data of described second frame of video and the 3rd video form the complete effective video data needed for described first frame of video;
Other row in described second frame of video except described effective video data is filled by black data; And
Other row in described 3rd frame of video except described effective video data is filled by black data.
6. messaging device as claimed in claim 5, wherein
Described processing unit produces the 4th frame of video based on described current first frame of video and next first frame of video described.
7. messaging device as claimed in claim 6, wherein
Described processing unit based on the mathematic interpolation motion vector data between described current first frame of video and next first frame of video described, and produces the 4th frame of video based on described motion vector data.
8. messaging device as claimed in claim 7, wherein
Described processing unit produces the 5th frame of video and the 6th frame of video based on described 4th frame of video, and order inserts described second frame of video, the 3rd frame of video, the 4th frame of video, the 5th frame of video and the 6th frame of video between described current first frame of video and next first frame of video described
Wherein said 5th frame of video comprises the effective video data of the odd-numbered line at least partially of described 4th frame of video, and described 6th frame of video comprises the effective video data of the even number line at least partially of described 4th frame of video, and the described effective video data of described 5th frame of video and the 6th video form the complete effective video data needed for described 4th frame of video;
Other row in described 5th frame of video except described effective video data is filled by black data; And
Other row in described 6th frame of video except described effective video data is filled by black data.
9. a messaging device, comprising:
Camera module, configuration carrys out capture video, and produces vision signal;
Graphics Processing Unit, configuration receives the vision signal that described camera module produces, and produces the first frame of video based on described vision signal with the predetermined cycle; And
Frame of video plug-in unit, configuration produces the second frame of video and the 3rd frame of video based on described first frame of video, and order inserts described second frame of video and the 3rd frame of video between current first frame of video and next the first frame of video,
Wherein said second frame of video comprises the effective video data of the odd-numbered line at least partially of described first frame of video, described 3rd frame of video comprises the effective video data of the even number line at least partially of described first frame of video, and the described effective video data of described second frame of video and the 3rd video form the complete effective video data needed for described first frame of video; And
Other row in described second frame of video except described effective video data is filled by black data;
Other row in described 3rd frame of video except described effective video data is filled by black data.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201110360319.1A CN103108148B (en) | 2011-11-14 | 2011-11-14 | The frame interpolation method of frame of video and messaging device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201110360319.1A CN103108148B (en) | 2011-11-14 | 2011-11-14 | The frame interpolation method of frame of video and messaging device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN103108148A CN103108148A (en) | 2013-05-15 |
| CN103108148B true CN103108148B (en) | 2016-03-30 |
Family
ID=48315682
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201110360319.1A Active CN103108148B (en) | 2011-11-14 | 2011-11-14 | The frame interpolation method of frame of video and messaging device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN103108148B (en) |
Families Citing this family (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107155016A (en) * | 2017-03-30 | 2017-09-12 | 安徽森度科技有限公司 | Simulate camera digital of digital video data implementation method |
| CN111614945B (en) | 2019-02-22 | 2022-02-08 | 台达电子工业股份有限公司 | Projection display device and display method |
| TWI698119B (en) * | 2019-02-22 | 2020-07-01 | 台達電子工業股份有限公司 | Projection display apparatus and display method |
| KR20210066619A (en) | 2019-11-28 | 2021-06-07 | 삼성전자주식회사 | Electronic apparatus and control method thereof |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1742485A (en) * | 2003-01-21 | 2006-03-01 | 索尼株式会社 | Frame conversion method, frame conversion circuit, and electronic camera |
| CN101246680A (en) * | 2008-02-04 | 2008-08-20 | 炬力集成电路设计有限公司 | Image frame rate conversion method, image zoom conversion device and equipment |
| CN101621652A (en) * | 2009-07-21 | 2010-01-06 | 上海华平信息技术股份有限公司 | Method for transmitting interlaced picture in high quality and changing the interlaced picture into non-interlaced picture in picture transmission system |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP5566133B2 (en) * | 2010-03-05 | 2014-08-06 | キヤノン株式会社 | Frame rate conversion processor |
-
2011
- 2011-11-14 CN CN201110360319.1A patent/CN103108148B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1742485A (en) * | 2003-01-21 | 2006-03-01 | 索尼株式会社 | Frame conversion method, frame conversion circuit, and electronic camera |
| CN101246680A (en) * | 2008-02-04 | 2008-08-20 | 炬力集成电路设计有限公司 | Image frame rate conversion method, image zoom conversion device and equipment |
| CN101621652A (en) * | 2009-07-21 | 2010-01-06 | 上海华平信息技术股份有限公司 | Method for transmitting interlaced picture in high quality and changing the interlaced picture into non-interlaced picture in picture transmission system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN103108148A (en) | 2013-05-15 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108921951B (en) | Virtual reality image display method and device and virtual reality equipment | |
| US8300087B2 (en) | Method and system for response time compensation for 3D video processing | |
| KR100843087B1 (en) | A image generation apparatus and method for the same | |
| Kara et al. | Evaluation of the concept of dynamic adaptive streaming of light field video | |
| JP6062512B2 (en) | Imaging apparatus, information processing system, and image data transmission method | |
| EP2384001A1 (en) | Providing of encoded video applications in a network environment | |
| CN109819232B (en) | Image processing method, image processing device and display device | |
| JP2023513304A (en) | Motion Smoothing in Distributed Systems | |
| CN102905056A (en) | Method and device for processing video image | |
| CN103108148B (en) | The frame interpolation method of frame of video and messaging device | |
| JP6218787B2 (en) | Imaging apparatus, information processing apparatus, display apparatus, information processing system, image data transmission method, and image display method | |
| CN111290725A (en) | Screen projection method, equipment and storage medium | |
| JP2011090079A (en) | Display device, display method and computer program | |
| KR20170106389A (en) | Video transmission based on independently encoded background updates | |
| US9881541B2 (en) | Apparatus, system, and method for video creation, transmission and display to reduce latency and enhance video quality | |
| US20180035076A1 (en) | Video processing apparatus, video processing system, and video processing method | |
| JP6362116B2 (en) | Display device, control method therefor, program, and storage medium | |
| CN113515193B (en) | Model data transmission method and device | |
| CN103139524B (en) | Method for optimizing video and messaging device | |
| CN104601945A (en) | Method for continuously displaying picture parts | |
| CN102026007A (en) | Method and system for processing video | |
| CN105993176A (en) | Image processing apparatus using monitoring of video memory | |
| CN115835035A (en) | Image frame interpolation method, device and equipment and computer readable storage medium | |
| JP2007034222A (en) | Video display controller, video display control method, and program thereof | |
| JP5664356B2 (en) | Generation apparatus and generation method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant |