CN106331823A - Video playing method and device - Google Patents
Video playing method and device Download PDFInfo
- Publication number
- CN106331823A CN106331823A CN201610799113.1A CN201610799113A CN106331823A CN 106331823 A CN106331823 A CN 106331823A CN 201610799113 A CN201610799113 A CN 201610799113A CN 106331823 A CN106331823 A CN 106331823A
- Authority
- CN
- China
- Prior art keywords
- buffering
- image
- panoramic picture
- panoramic
- buffer
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 55
- 239000000872 buffer Substances 0.000 claims abstract description 91
- 230000015572 biosynthetic process Effects 0.000 claims abstract description 19
- 238000000605 extraction Methods 0.000 claims abstract description 18
- 238000003786 synthesis reaction Methods 0.000 claims abstract description 17
- 238000011049 filling Methods 0.000 claims abstract description 4
- 230000003139 buffering effect Effects 0.000 claims description 103
- 238000002372 labelling Methods 0.000 claims description 11
- 230000004886 head movement Effects 0.000 claims description 10
- 230000008676 import Effects 0.000 claims description 10
- 239000000284 extract Substances 0.000 claims description 8
- 238000004422 calculation algorithm Methods 0.000 claims description 5
- 238000002156 mixing Methods 0.000 claims description 5
- 230000002194 synthesizing effect Effects 0.000 claims description 4
- 238000004080 punching Methods 0.000 claims description 3
- 230000003111 delayed effect Effects 0.000 claims description 2
- 238000001514 detection method Methods 0.000 claims description 2
- 230000008569 process Effects 0.000 description 17
- 238000010586 diagram Methods 0.000 description 13
- 238000004590 computer program Methods 0.000 description 6
- 230000006870 function Effects 0.000 description 5
- 230000000007 visual effect Effects 0.000 description 4
- 241000282693 Cercopithecidae Species 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 238000007654 immersion Methods 0.000 description 3
- 230000035807 sensation Effects 0.000 description 3
- 208000012886 Vertigo Diseases 0.000 description 2
- 230000009471 action Effects 0.000 description 2
- 230000008859 change Effects 0.000 description 2
- 208000002173 dizziness Diseases 0.000 description 2
- 230000026676 system process Effects 0.000 description 2
- 230000006399 behavior Effects 0.000 description 1
- 238000005094 computer simulation Methods 0.000 description 1
- 230000004927 fusion Effects 0.000 description 1
- 238000007689 inspection Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 239000011159 matrix material Substances 0.000 description 1
- 239000000203 mixture Substances 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000003860 storage Methods 0.000 description 1
- 239000011800 void material Substances 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/4302—Content synchronisation processes, e.g. decoder synchronisation
- H04N21/4305—Synchronising client clock from received content stream, e.g. locking decoder clock with encoder clock, extraction of the PCR packets
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/4302—Content synchronisation processes, e.g. decoder synchronisation
- H04N21/4307—Synchronising the rendering of multiple content streams or additional data on devices, e.g. synchronisation of audio on a mobile phone with the video output on the TV screen
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
- H04N21/44004—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving video buffer management, e.g. video decoder buffer or video display buffer
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
- H04N21/44016—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving splicing one content stream with another content stream, e.g. for substituting a video clip
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Processing Or Creating Images (AREA)
- Studio Circuits (AREA)
Abstract
The embodiment of the invention provides a video playing method and device. The video playing method comprises the steps of forming panoramic image buffer according to panoramic video data, filling the panoramic image buffer to a buffer queue, and then acquiring the panoramic image buffer from the buffer queue so as to obtain preprocessed image buffer. Extraction is performed on the preprocessed image buffer through a system according to motion sensing data collected by a motion sensor so as to generate scene image buffer, so that the system is enabled to perform synthesis and display after carrying out extraction on the panoramic image buffer according to the latest sensing data, the image delay time caused by motion of a user can be substantially shortened, and thus use feelings of the user are improved.
Description
Technical field
The present embodiments relate to the communications field, particularly relate to a kind of video broadcasting method and device.
Background technology
Along with the fast development of VR (Virtual Reality, virtual reality) technology in recent years, VR product user is not yet
Break and increase.For VR product user, the strongest impression that uses is to the feeling of immersion playing content.
In prior art, generally there is spinning sensation in user in using VR product process, loses feeling of immersion, have a strong impact on
Use impression.Producing this spinning sensation mainly by long causing picture time delay, wherein picture postpones to refer to user
(head) motion starts, and sees, to user, the time interval that post exercise picture is fully displayed on terminal unit.When between the time
When more than 20ms, user will perceive picture and postpone.Picture long meeting time delay makes human body sensing technical ability and human eye
The image seen does not mates, and then produces the sensation of dizziness.
Therefore, how to reduce the picture delay that panoramic video is play, be technical problem the most urgently to be resolved hurrily.
Summary of the invention
The embodiment of the present invention provides a kind of video broadcasting method, and during to solve panoramic video broadcasting in prior art, head turns
The dynamic picture caused postpones excessive problem.
In order to solve the problems referred to above, the invention discloses a kind of video broadcasting method, including:
Panoramic picture buffering is formed according to panoramic video data, and by described panoramic picture buffer fillings to buffer queue
In, described panoramic picture buffering includes panorama labelling.
From described buffer queue, obtain described panoramic picture buffering carry out image management, slow to obtain pretreatment image
Punching.
The motion-sensing data gathered according to motion sensor, are extracted described pretreatment image buffering by system
Generation scene image buffers.
Described scene image buffering is synthesized, to show the scene image of correspondence position.
Preferably, form panoramic picture buffering according to panoramic video data, and by described panoramic picture buffer fillings to slow
Rush in queue, including removing free buffer from buffer queue；The panoramic video data obtained by application program are filled to described
In free buffer, form panoramic picture buffering；Described panoramic picture buffering is moved into described buffer queue.
Preferably, before pretreatment image buffering being carried out extraction generation scene image buffering by system, also include: inspection
Survey whether described pretreatment image buffering exists panorama labelling, to determine that described pretreatment image buffering buffers for panoramic picture.
Preferably, by system, pretreatment image buffering is carried out extraction and generate scene image buffering, including: pass through system
Call blending algorithm panoramic picture buffering is extracted, to generate described scene image buffering.
Wherein, pretreatment image buffering is extracted, synthesize, in same frame synchronization with to scene image buffering
Complete, and in next frame synchronizes, show the scene image of correspondence position.
Further, motion sensor is arranged in virtual reality device, for being tracked user's head movement, obtains
Fetching shows the motion-sensing data of head movement state.
On the other hand, in order to solve the problems referred to above, the invention also discloses a kind of video play device, including:
Image imports module, for forming panoramic picture buffering according to panoramic video data, and is delayed by described panoramic picture
Filling is charged in buffer queue, and described panoramic picture buffering includes panorama labelling.
Image management module, carries out image management for obtaining described panoramic picture buffering from described buffer queue, with
Acquisition pretreatment image buffers.
Abstraction module, for the motion-sensing data gathered according to motion sensor, by system to described pretreatment figure
Scene image buffering is generated as buffering carries out extraction.
Synthesis display module, for synthesizing described scene image buffering, to show the scene image of correspondence position.
To sum up, the embodiment of the present invention buffers by forming panoramic picture according to panoramic video data, is buffered by panoramic picture
Fill to buffer queue.And obtain from buffer queue this panoramic picture buffering carry out image management, to obtain pretreatment figure
As buffering.The motion-sensing data gathered further according to motion sensor, carry out extraction by system to pretreatment image buffering raw
Become scene image buffering, to enable a system to according to up-to-date sensing data, the most permissible after panoramic picture buffering is extracted
Carry out synthesis display, thus significantly shorten the picture time delay caused because of user movement, and then make user cannot perceive picture
Face postpones, and promotes user and uses impression.
Accompanying drawing explanation
In order to be illustrated more clearly that the technical scheme of the embodiment of the present invention, below by institute in the description to the embodiment of the present invention
The accompanying drawing used is needed to be briefly described, it should be apparent that, the accompanying drawing in describing below is only some enforcements of the present invention
Example, for those of ordinary skill in the art, on the premise of not paying creative work, it is also possible to according to these accompanying drawings
Obtain other accompanying drawing.
A kind of video broadcasting method flow chart that Fig. 1 provides for the embodiment of the present invention；
The another kind of video broadcasting method flow chart that Fig. 2 provides for the embodiment of the present invention；
Fig. 3 is that the buffer of a kind of video broadcasting method in prior art manages schematic diagram；
Fig. 4 is the time diagram of a kind of video broadcasting method in prior art；
The buffer of a kind of video broadcasting method that Fig. 5 provides for the embodiment of the present invention manages schematic diagram；
The time diagram of a kind of video broadcasting method that Fig. 6 provides for the embodiment of the present invention；
The structured flowchart of a kind of video play device that Fig. 7 provides for the embodiment of the present invention；
The structured flowchart of the another kind of video play device that Fig. 8 provides for the embodiment of the present invention.
Detailed description of the invention
Below in conjunction with the accompanying drawing in the embodiment of the present invention, the technical scheme in the embodiment of the present invention is carried out clear, complete
Describe, it is clear that described embodiment is a part of embodiment of the present invention rather than whole embodiments wholely.Based on this
Embodiment in bright, the every other enforcement that those of ordinary skill in the art are obtained under not making creative work premise
Example, broadly falls into the scope of protection of the invention.
For the ease of understanding video broadcasting method provided by the present invention, before describing specific embodiment, to embodiment
Technical term involved by explains:
GraphicBuffer: image buffers, android system is described by definition GraphicBuffer data type
One piece of image buffer, this object can transmit with striding course.
Bufferqueue: buffer queue, is the local realization of ISurfaceTexture, it is necessary to each void in reloading interface
Function, such as queueBuffer, dequeueBuffer etc..It is believed that BufferQueue is a service centre, by it
Manage buffer.
SurfaceFlinger: image management module, the main Surface being responsible for synthesizing each window, then passes through
OpenGLES is shown on FrameBuffer.SurfaceFlinger service operation is in the System process of android system
In, it is responsible for the frame buffer zone of android system.
Hwcomposer: image synthesizer, utilizes hardware to complete image data set and merges the function of display.
Surface: drawing object, corresponding one piece of screen buffer, the corresponding Surface of each window, any
View is drawn on Surface.
Queuebuffer: buffering of falling in lines, will buffer immigration queue, it is possible to be expressed as arriving buffer queue
In Bufferqueue.
Dequeuebuffer: buffering of falling out, i.e. removal buffering from queue, it is possible to be expressed as from Bufferqueue
Dequeue goes out buffer.
VR (Virtual Reality, virtual reality): be a kind of can establishment and the Computer Simulation in the experiencing virtual world
System.It utilizes computer to generate a kind of simulated environment, be a kind of Multi-source Information Fusion interactively Three-Dimensional Dynamic what comes into a driver's and
The system emulation of entity behavior, it is possible to make user be immersed in this environment.
FrameBuffer: frame buffers, and is used for by picture output device from the core buffer comprising complete frame data
In drive video display apparatus.Core buffer Plays contains the color-values composition of each pixel on screen.
Overlay: superimposed layer, in the case of not revising framework source code, it is achieved the customization of resource.
The brief explanation implication of technical term herein, above technical term can enter in conjunction with specific embodiments follow-up
One step ground is explained.
Embodiment one
With reference to Fig. 1, give a kind of video broadcasting method flow chart that the embodiment of the present invention provides.
Step 101, forms panoramic picture buffering according to panoramic video data, and by panoramic picture buffer fillings to buffering team
In row.
Wherein, panoramic video is obtained by application program, including 360 degree of panoramic videos, is by catching whole to mm professional camera special
The image information of individual scene or the picture after using modeling software to render, use software to carry out picture split, and with specially
The panoramic picture that plays out of player.Browse for virtual reality, the plane graph of two dimension is modeled to real three-dimensional space
Between, present to user.
In the present embodiment, as a example by being applied to VR field.In order to determine that the video of broadcasting is panoramic video, can be at panorama
Arranging panorama labelling flag in the image buffers of video, if there is flag, correspondence image is panoramic video.
By panoramic video data are filled to free buffer buffer1 of buffer queue Bufferqueue, formed complete
Scape image buffers GraphicBuffer1, makes these panoramic video data be directed in system cache, it is simple to system is to this aphorama
Frequency operates.
Step 102, obtains panoramic picture buffering from buffer queue and carries out image management, slow to obtain pretreatment image
Punching.
Wherein, image management module SurfaceFlinger obtains this GraphicBuffer1 inside Bufferqueue,
And carry out image management, it is thus achieved that pretreatment image buffering GraphicBuffer2.Wherein, image management is specifically for each window
Mouthful measure, layout and drafting, thus to being buffered into Row sum-equal matrix.
Step 103, the motion-sensing data gathered according to motion sensor, by system, pretreatment image buffering is carried out
Extraction generates scene image buffering.
Wherein, motion sensor is used for detecting user when watching this panoramic video, the motion conditions of head.Such as, when
During head rotation, motion sensor is by the visual direction after positioning head rotation, and this information is sent out as motion-sensing data
Deliver in system.From GraphicBuffer2, extract the scene image in visual direction according to these motion-sensing data for system
Buffering GraphicBuffer3.Step 104, synthesizes scene image buffering, to show the scene image of correspondence position.
Utilize image synthesizer Hwcomposer that GraphicBuffer3 is synthesized, thus at virtual reality device
The scene image in visual direction after demonstrating user's head movement on screen.
In the data import system of panoramic video being cached in abovementioned steps, panoramic video data are converted into
GraphicBuffer2.Therefore after motion sensor collects these motion-sensing data, only need to be to GraphicBuffer2
Extract, GraphicBuffer3 just can be directly placed in Hwcomposer synthesis, and show on screen.Without
Panoramic video data are extracted by application program to be passed through, then will extract result import system, and by system to image buffers
After process, just can put in Hwcomposer and synthesize, and show on screen.I.e. the application is in order to reduce acquisition motion-sensing number
According to operation afterwards to shorten the time, in advance all of vedio data is carried out buffered.When motion sensor obtains
During to the position of head rotation, it is only necessary to select from the GraphicBuffer2 completing buffered and motion-sensing number
According to corresponding GraphicBuffer3.Thus be greatly shortened from obtaining motion-sensing data to being finally shown to display screen
The upper required time.
To sum up, the embodiment of the present invention buffers by forming panoramic picture according to panoramic video data, is buffered by panoramic picture
Fill to buffer queue.And obtain from buffer queue this panoramic picture buffering carry out image management, to obtain pretreatment figure
As buffering.The motion-sensing data gathered further according to motion sensor, carry out extraction by system to pretreatment image buffering raw
Become scene image buffering, to enable a system to according to up-to-date sensing data, the most permissible after panoramic picture buffering is extracted
Carry out synthesis display, thus significantly shorten the picture time delay caused because of user movement, and then make user cannot perceive picture
Face postpones, and promotes user and uses impression.
Embodiment two
On the basis of above-described embodiment, the present embodiment is discussed further video broadcasting method.
With reference to Fig. 2, give the another kind of video broadcasting method flow chart that the embodiment of the present invention provides:
Step 201, forms panoramic picture buffering according to panoramic video data, and by panoramic picture buffer fillings to buffering team
In row.
Concrete, application program dequeue from BufferQueue goes out buffer1 with carry image information, will apply journey
After the panoramic video data that sequence obtains are filled to this buffer1 with formation GraphicBuffer1, then should
In GraphicBuffer1queue to BufferQueue, to complete the importing to system of the panoramic video data.
Step 202, obtains panoramic picture buffering from buffer queue and carries out image management.
Concrete, after panoramic video data are filled to BufferQueue, by SurfaceFlinger from
BufferQueue obtains this GraphicBuffer1, and this GraphicBuffer1 is carried out image management, it is thus achieved that
GraphicBuffer2.After GraphicBuffer2 is called by subsequent step, it will buffer1 is discharged extremely
In BufferQueue, wherein SurfaceFlinger service operation is in the System process of Android system, specifically for management
The frame buffering FrameBuffer of Android system, synthesizes the Surface of each window.
Step 203, detects whether described pretreatment image buffering exists panorama labelling.
Concrete, owing to the image buffers that system is panoramic video is provided with flag, therefore can be right
GraphicBuffer2 carries out extraction and generates before GraphicBuffer3, there is flag by detecting whether, it is judged that this image
Whether buffering includes panoramic video.If there is flag, then need this GraphicBuffer2 is extracted.
Step 204, the motion-sensing data gathered according to motion sensor, by system, pretreatment image buffering is carried out
Extraction generates scene image buffering.
In the present embodiment, as a example by being applied to VR field.Motion sensor can be arranged in VR equipment, for example, it is possible to
Being arranged in the VR helmet, wherein, the VR helmet is that one utilizes head mounted display, the vision to external world of people, audition is closed,
User is guided to produce the virtual reality terminal of a kind of sensation in virtual environment.When user's head movement, can to
Family head movement is tracked, thus obtains the motion-sensing data of instruction head movement state.
When motion sensor collects motion-sensing data, owing to caching having existed corresponding scene
GraphicBuffer2, therefore can be according to up-to-date motion-sensing data, directly by system to panoramic video
GraphicBuffer2 extracts, to obtain the GraphicBuffer3 corresponding with motion-sensing data.Wherein, to aphorama
The GraphicBuffer2 of frequency extracts, and can call blending algorithm by system and complete.
Step 205, synthesizes scene image buffering, to show the scene image of correspondence position.
Concrete, by Hwcomposer, GraphicBuffer3 is synthesized, obtain corresponding with motion-sensing data
The scene image of position, and this scene image is shown on the display screen of virtual display device.Wherein, if passing through overlay
Mode, then be directly synthesized to GraphicBuffer2 on overlay by Hwcomposer.If by framebuffer mode,
Then this GraphicBuffer2 is plotted on framebuffer, and is synthesized to framebuffer eventually through Hwcomposer
On.
Unlike the prior art, when using virtual display device to view and admire panoramic video, if head is turned to by position A
Position B.In the prior art, after motion sensor collects these motion-sensing data, need by application program entirely
Scape video data extracts, then will extraction result import system, and by system to image buffers process after, just can put into
Hwcomposer synthesizes, and shows on screen.Can be seen that from obtaining motion-sensing data to last aobvious from this process
Show on display screen, at least need the time of 3-4 frame synchronization.A kind of video broadcasting method in prior art is given with reference to Fig. 3
Buffer manage schematic diagram.The time diagram of a kind of video broadcasting method in prior art is given with reference to Fig. 4.Even if adopting
By " three buffering " technology, the time that picture postpones is also required to the time more than 2 frame synchronization.If the display screen of virtual reality device
Use the LCDs that refresh rate is 60Hz of current main flow, then this display screen shows that the time of a frame is about 16.67ms, aobvious
Show that 2 frames that is 2 frame synchronization just requires more than the time of 33ms, i.e. picture and will exceed well over 20ms time delay, make user clearly
Feel that picture postpones.Such as, in panoramic video, you see a monkey, when monkey from your visual field rapidly,
In tow ghost climb to by the side of one tree upper time, you look up the monkey on tree, have spent the time more than 20ms, picture just with
The head you has turned in the past.User is made to lose feeling of immersion.
And during the embodiment of the present invention is owing to caching the data import system of panoramic video, and generate
GraphicBuffer2.So, after motion sensor collects these motion-sensing data, only need to be to GraphicBuffer2
Extract, the GraphicBuffer3 that extraction obtains just can be directly placed in Hwcomposer synthesis, and show at screen
On curtain.Time that picture postpone is greatly shortened.A kind of video broadcasting method that the embodiment of the present invention provides is given with reference to Fig. 5
Buffer manage schematic diagram.The sequential signal of a kind of video broadcasting method that the embodiment of the present invention provides is given with reference to Fig. 6
Figure.Wherein, GraphicBuffer2 is extracted, synthesize with to GraphicBuffer3, can be in same frame synchronization
In complete, therefore motion sensor collect these motion-sensing data next frame synchronize in, just can demonstrate correspondence
The scene image of position, thus the time that picture postpones is controlled the time in a frame synchronization.Such as, if virtual reality device
Display screen uses the LCDs that refresh rate is 60Hz of current main flow, then this display screen shows that the time of a frame is about
16.67ms, i.e. picture can control within 20ms time delay, and then make user cannot perceive picture delay, it is to avoid
Because picture postpones the dizziness sensation that artificial deliviery is raw, promote user and use impression.
It should be noted that for embodiment of the method, in order to be briefly described, therefore it is all expressed as a series of action group
Closing, but those skilled in the art should know, the embodiment of the present invention is not limited by described sequence of movement, because depending on
According to the embodiment of the present invention, some step can use other orders or carry out simultaneously.Secondly, those skilled in the art also should
Knowing, embodiment described in this description belongs to preferred embodiment, and the involved action not necessarily present invention implements
Necessary to example.
Embodiment three
On the basis of above-described embodiment, the present embodiment additionally provides a kind of video play device, is applied to virtual reality
Terminal.
Give the structured flowchart of a kind of video play device that the embodiment of the present invention provides with reference to Fig. 7, specifically can wrap
Include such as lower module:
Image imports module 701, for forming panoramic picture buffering according to panoramic video data, and by described panoramic picture
Buffer fillings is in buffer queue, and panoramic picture buffering includes panorama labelling.
Image management module 702, carries out image management for obtaining panoramic picture buffering from buffer queue, pre-to obtain
Process image buffers.
Abstraction module 703, for the motion-sensing data gathered according to motion sensor, by system to pretreatment image
Buffering carries out extraction and generates scene image buffering.
Synthesis display module 704, for synthesizing scene image buffering, to show the scene image of correspondence position.
With reference to Fig. 8, in an optional implementation of the present invention, on the basis of Fig. 7, video play device also wraps
Include:
Panorama detection module 705, is used for, before pretreatment image buffering carries out extraction generation scene image buffering, examining
Survey whether pretreatment image buffering exists panorama labelling, to determine that pretreatment image buffering buffers for panoramic picture.
Image imports module 701 and includes: submodule 7011 of falling out, filling submodule 7012 and submodule 7013 of falling in lines.Its
In, submodule 7011 of falling out, for removing free buffer from buffer queue；Fill submodule 7012, for by application program
The panoramic video data obtained are filled to free buffer, form panoramic picture buffering；Fall in lines submodule 7013, for by panorama
Image buffers moves into buffer queue.
Based on this, abstraction module 703, specifically for by system call blending algorithm to panoramic picture buffering take out
Take, to generate scene image buffering.
Further, pretreatment image buffering is extracted by abstraction module 703, with synthesis display module 704 to scene
Image buffers synthesizes, and completes in same frame synchronization；And the display that synthesis display module 704 is to synthesis result, at next
Frame synchronization completes.
Wherein, motion sensor is arranged in virtual reality device, and for being tracked user's head movement, acquisition refers to
Show the motion-sensing data of head movement state.
To sum up, the embodiment of the present invention imports module 701 by image and forms panoramic picture buffering according to panoramic video data,
By in panoramic picture buffer fillings to buffer queue.And from buffer queue, obtain this panorama sketch by image management module 702
As buffering carries out image management, to obtain pretreatment image buffering.The motion-sensing data gathered further according to motion sensor, logical
Cross abstraction module 703 and pretreatment image buffering is carried out extraction generation scene image buffering, aobvious to enable a system to by synthesis
Show that module 704, according to up-to-date sensing data, just can carry out synthesis and show the extraction result of abstraction module 703, thus greatly
Width shortens the picture time delay caused because of user movement, and then makes user cannot perceive picture delay, promotes user and uses
Impression.
For device embodiment, due to itself and embodiment of the method basic simlarity, so describe is fairly simple, relevant
Part sees the part of embodiment of the method and illustrates.
Each embodiment in this specification all uses the mode gone forward one by one to describe, what each embodiment stressed is with
The difference of other embodiments, between each embodiment, identical similar part sees mutually.
Those skilled in the art are it should be appreciated that the embodiment of the embodiment of the present invention can be provided as method, device or calculate
Machine program product.Therefore, the embodiment of the present invention can use complete hardware embodiment, complete software implementation or combine software and
The form of the embodiment of hardware aspect.And, the embodiment of the present invention can use one or more wherein include computer can
With in the computer-usable storage medium (including but not limited to disk memory, CD-ROM, optical memory etc.) of program code
The form of the computer program implemented.
The embodiment of the present invention is with reference to method, terminal unit (system) and computer program according to embodiments of the present invention
The flow chart of product and/or block diagram describe.It should be understood that can be by computer program instructions flowchart and/or block diagram
In each flow process and/or the flow process in square frame and flow chart and/or block diagram and/or the combination of square frame.These can be provided
Computer program instructions sets to general purpose computer, special-purpose computer, Embedded Processor or other programmable data processing terminals
Standby processor is to produce a machine so that held by the processor of computer or other programmable data processing terminal equipment
The instruction of row produces for realizing in one flow process of flow chart or multiple flow process and/or one square frame of block diagram or multiple square frame
The device of the function specified.
These computer program instructions may be alternatively stored in and can guide computer or other programmable data processing terminal equipment
In the computer-readable memory worked in a specific way so that the instruction being stored in this computer-readable memory produces bag
Including the manufacture of command device, this command device realizes in one flow process of flow chart or multiple flow process and/or one side of block diagram
The function specified in frame or multiple square frame.
These computer program instructions also can be loaded on computer or other programmable data processing terminal equipment so that
On computer or other programmable terminal equipment, execution sequence of operations step is to produce computer implemented process, thus
The instruction performed on computer or other programmable terminal equipment provides for realizing in one flow process of flow chart or multiple flow process
And/or the step of the function specified in one square frame of block diagram or multiple square frame.
Although having been described for the preferred embodiment of the embodiment of the present invention, but those skilled in the art once knowing base
This creativeness concept, then can make other change and amendment to these embodiments.So, claims are intended to be construed to
The all changes including preferred embodiment and falling into range of embodiment of the invention and amendment.
Finally, in addition it is also necessary to explanation, in this article, the relational terms of such as first and second or the like be used merely to by
One entity or operation separate with another entity or operating space, and not necessarily require or imply these entities or operation
Between exist any this reality relation or order.And, term " includes ", " comprising " or its any other variant meaning
Containing comprising of nonexcludability, so that include that the process of a series of key element, method, article or terminal unit not only wrap
Include those key elements, but also include other key elements being not expressly set out, or also include for this process, method, article
Or the key element that terminal unit is intrinsic.In the case of there is no more restriction, by wanting that statement " including ... " limits
Element, it is not excluded that there is also other identical element in including the process of key element, method, article or terminal unit.
Above to a kind of video broadcasting method provided by the present invention and device, it is described in detail, used herein
Principle and the embodiment of the present invention are set forth by specific case, and the explanation of above example is only intended to help to understand
The method of the present invention and core concept thereof；Simultaneously for one of ordinary skill in the art, according to the thought of the present invention, at tool
All will change on body embodiment and range of application, to sum up, this specification content should not be construed as the limit to the present invention
System.
Claims (12)
1. a video broadcasting method, it is characterised in that described method includes:
Panoramic picture buffering is formed according to panoramic video data, and by described panoramic picture buffer fillings to buffer queue, institute
State panoramic picture buffering and include panorama labelling；
From described buffer queue, obtain described panoramic picture buffering carry out image management, to obtain pretreatment image buffering；
The motion-sensing data gathered according to motion sensor, carry out extraction by system to described pretreatment image buffering and generate
Scene image buffers；
Described scene image buffering is synthesized, to show the scene image of correspondence position.
Method the most according to claim 1, it is characterised in that described slow according to panoramic video data formation panoramic picture
Punching, and by described panoramic picture buffer fillings to buffer queue, including:
Free buffer is removed from buffer queue；
The panoramic video data obtained by application program are filled to described free buffer, form panoramic picture buffering；
Described panoramic picture buffering is moved into described buffer queue.
Method the most according to claim 1, it is characterised in that described buffering pretreatment image by system is extracted
Before generating scene image buffering, also include:
Detect whether described pretreatment image buffering exists panorama labelling, to determine that described pretreatment image buffers as panoramic picture
Buffering.
Method the most according to claim 1, it is characterised in that described buffering pretreatment image by system is extracted
Generation scene image buffers, including:
Call blending algorithm by system panoramic picture buffering is extracted, to generate described scene image buffering.
Method the most according to claim 1, it is characterised in that
Described buffer pretreatment image is extracted, and synthesizes scene image buffering, in same frame synchronization with described
Complete；
The scene image of described correspondence position is shown in next frame synchronizes.
Method the most according to claim 1, it is characterised in that
Described motion sensor is arranged in virtual reality device, for being tracked user's head movement, obtains instruction head
The motion-sensing data of portion's kinestate.
7. a video play device, it is characterised in that described device includes:
Image imports module, for forming panoramic picture buffering according to panoramic video data, and described panoramic picture buffering is filled out
Being charged in buffer queue, described panoramic picture buffering includes panorama labelling；
Image management module, carries out image management for obtaining described panoramic picture buffering from described buffer queue, to obtain
Pretreatment image buffers；
Abstraction module, for the motion-sensing data gathered according to motion sensor, is delayed described pretreatment image by system
Rush in row extraction and generate scene image buffering；
Synthesis display module, for synthesizing described scene image buffering, to show the scene image of correspondence position.
Device the most according to claim 7, it is characterised in that described image imports module and includes:
Fall out submodule, for removing free buffer from buffer queue；
Filling submodule, the panoramic video data for being obtained by application program are filled to described free buffer, form panorama
Image buffers；
Fall in lines submodule, for described panoramic picture buffering is moved into described buffer queue.
Device the most according to claim 7, it is characterised in that described device also includes:
Panorama detection module, is used for detecting whether described pretreatment image buffering exists panorama labelling, to determine described pretreatment
Image buffers is panoramic picture buffering.
Device the most according to claim 7, it is characterised in that
Described abstraction module, extracts, to generate panoramic picture buffering specifically for calling blending algorithm by system
State scene image buffering.
11. devices according to claim 7, it is characterised in that
Pretreatment image buffering is extracted by described abstraction module, carries out scene image buffering with described synthesis display module
Synthesis, completes in same frame synchronization；
The display to synthesis result of the described synthesis display module, completes in next frame synchronizes.
12. devices according to claim 7, it is characterised in that
Described motion sensor is arranged in virtual reality device, for being tracked user's head movement, obtains instruction head
The motion-sensing data of portion's kinestate.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610799113.1A CN106331823B (en) | 2016-08-31 | 2016-08-31 | A kind of video broadcasting method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610799113.1A CN106331823B (en) | 2016-08-31 | 2016-08-31 | A kind of video broadcasting method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN106331823A true CN106331823A (en) | 2017-01-11 |
| CN106331823B CN106331823B (en) | 2019-08-20 |
Family
ID=57786299
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610799113.1A Active CN106331823B (en) | 2016-08-31 | 2016-08-31 | A kind of video broadcasting method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106331823B (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106896826A (en) * | 2017-02-27 | 2017-06-27 | 张斌 | Moving scene digit synthesis system and method |
| CN108921867A (en) * | 2018-06-01 | 2018-11-30 | 天津煋鸟科技有限公司 | A kind of motion detection and human body recognition method of panoramic picture |
| WO2019014853A1 (en) * | 2017-07-18 | 2019-01-24 | 深圳市柔宇科技有限公司 | Method for detecting delay in head-mounted display data transmission, and head-mounted display |
| WO2019085109A1 (en) * | 2017-10-31 | 2019-05-09 | 歌尔科技有限公司 | Image writing control method and apparatus, and electronic device |
| CN109741463A (en) * | 2019-01-02 | 2019-05-10 | 京东方科技集团股份有限公司 | Rendering method, device and the equipment of virtual reality scenario |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105103034A (en) * | 2013-03-25 | 2015-11-25 | 索尼电脑娱乐欧洲有限公司 | Display |
| CN105323552A (en) * | 2015-10-26 | 2016-02-10 | 北京时代拓灵科技有限公司 | Method and system for playing panoramic video |
| CN105404393A (en) * | 2015-06-30 | 2016-03-16 | 指点无限（美国）有限公司 | Low-latency virtual reality display system |
| US20160228771A1 (en) * | 2015-02-05 | 2016-08-11 | Sony Computer Entertainment Inc. | Motion sickness monitoring and application of supplemental sound to counteract sickness |
| US20160236083A1 (en) * | 2002-12-10 | 2016-08-18 | Sony Interactive Entertainment America Llc | System and Method for Compressing Video Frames or Portions Thereof Based on Feedback Information from a Client Device |
| CN105898338A (en) * | 2015-12-18 | 2016-08-24 | 乐视致新电子科技（天津）有限公司 | Panorama video play method and device |
| CN105898359A (en) * | 2016-04-27 | 2016-08-24 | 乐视控股（北京）有限公司 | Virtual reality terminal and method and device for processing video thereof |
| CN105898138A (en) * | 2015-12-18 | 2016-08-24 | 乐视致新电子科技（天津）有限公司 | Panoramic video play method and device |
-
2016
- 2016-08-31 CN CN201610799113.1A patent/CN106331823B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20160236083A1 (en) * | 2002-12-10 | 2016-08-18 | Sony Interactive Entertainment America Llc | System and Method for Compressing Video Frames or Portions Thereof Based on Feedback Information from a Client Device |
| CN105103034A (en) * | 2013-03-25 | 2015-11-25 | 索尼电脑娱乐欧洲有限公司 | Display |
| US20160228771A1 (en) * | 2015-02-05 | 2016-08-11 | Sony Computer Entertainment Inc. | Motion sickness monitoring and application of supplemental sound to counteract sickness |
| CN105404393A (en) * | 2015-06-30 | 2016-03-16 | 指点无限（美国）有限公司 | Low-latency virtual reality display system |
| CN105323552A (en) * | 2015-10-26 | 2016-02-10 | 北京时代拓灵科技有限公司 | Method and system for playing panoramic video |
| CN105898338A (en) * | 2015-12-18 | 2016-08-24 | 乐视致新电子科技（天津）有限公司 | Panorama video play method and device |
| CN105898138A (en) * | 2015-12-18 | 2016-08-24 | 乐视致新电子科技（天津）有限公司 | Panoramic video play method and device |
| CN105898359A (en) * | 2016-04-27 | 2016-08-24 | 乐视控股（北京）有限公司 | Virtual reality terminal and method and device for processing video thereof |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106896826A (en) * | 2017-02-27 | 2017-06-27 | 张斌 | Moving scene digit synthesis system and method |
| CN106896826B (en) * | 2017-02-27 | 2020-04-28 | 张斌 | Motion scene digital synthesis system and method |
| WO2019014853A1 (en) * | 2017-07-18 | 2019-01-24 | 深圳市柔宇科技有限公司 | Method for detecting delay in head-mounted display data transmission, and head-mounted display |
| WO2019085109A1 (en) * | 2017-10-31 | 2019-05-09 | 歌尔科技有限公司 | Image writing control method and apparatus, and electronic device |
| CN108921867A (en) * | 2018-06-01 | 2018-11-30 | 天津煋鸟科技有限公司 | A kind of motion detection and human body recognition method of panoramic picture |
| CN109741463A (en) * | 2019-01-02 | 2019-05-10 | 京东方科技集团股份有限公司 | Rendering method, device and the equipment of virtual reality scenario |
Also Published As
| Publication number | Publication date |
|---|---|
| CN106331823B (en) | 2019-08-20 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| JP6898430B2 (en) | Continuous Time Warping and Binocular Time Warping and Methods for Virtual Reality and Augmented Reality Display Systems | |
| CN106331823A (en) | Video playing method and device | |
| CN106464854B (en) | Image encodes and display | |
| CN110515452A (en) | Image processing method, device, storage medium and computer equipment | |
| CN108292489A (en) | Information processing unit and image generating method | |
| EP1967016B1 (en) | 3d image display method and apparatus | |
| CN106066701B (en) | A kind of AR and VR data processing equipment and method | |
| CN111051959B (en) | Generating new frames using rendered and non-rendered content from previous perspectives | |
| CN114615486A (en) | Media compositor for computer-generated reality | |
| CN105959666A (en) | Method and device for sharing 3d image in virtual reality system | |
| CN107277485B (en) | Image display method and device based on virtual reality | |
| CN109656367A (en) | Image processing method, device and electronic equipment under a kind of scene applied to VR | |
| US9161012B2 (en) | Video compression using virtual skeleton | |
| EP3991142A1 (en) | Fast hand meshing for dynamic occlusion | |
| CN109901290A (en) | The determination method, apparatus and wearable device of watching area | |
| CN117555426B (en) | Virtual reality interaction system based on digital twin technology | |
| EP3346375B1 (en) | Program, recording medium, content provision device, and control method | |
| CN106412718A (en) | Rendering method and device for subtitles in 3D space | |
| CN113552947A (en) | Virtual scene display method and device and computer readable storage medium | |
| Wu et al. | Reducing the latency in head‐mounted displays by a novel prediction method using grey system theory | |
| CN105867615A (en) | Information processing method and electronic device | |
| CN115048954A (en) | Retina-imitating target detection method and device, storage medium and terminal | |
| JP4214528B2 (en) | Pseudo stereoscopic image generation apparatus, pseudo stereoscopic image generation program, and pseudo stereoscopic image display system | |
| CN206097536U (en) | Cause of earthquake projection arrangement | |
| CN102495907A (en) | Video summary with depth information |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |