WO2017088491A1 - 一种视频的播放方法和装置 - Google Patents
一种视频的播放方法和装置 Download PDFInfo
- Publication number
- WO2017088491A1 WO2017088491A1 PCT/CN2016/089121 CN2016089121W WO2017088491A1 WO 2017088491 A1 WO2017088491 A1 WO 2017088491A1 CN 2016089121 W CN2016089121 W CN 2016089121W WO 2017088491 A1 WO2017088491 A1 WO 2017088491A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- information
- adjustment
- model
- video frame
- panoramic
- Prior art date
Links
- 238000000034 method Methods 0.000 title claims abstract description 52
- 239000011159 matrix material Substances 0.000 claims description 38
- 238000004590 computer program Methods 0.000 claims description 16
- 239000013598 vector Substances 0.000 claims description 12
- 238000006243 chemical reaction Methods 0.000 claims description 8
- 238000004364 calculation method Methods 0.000 claims description 7
- 230000005484 gravity Effects 0.000 claims description 6
- 230000006870 function Effects 0.000 abstract description 13
- 238000010586 diagram Methods 0.000 description 8
- 230000009466 transformation Effects 0.000 description 8
- 230000008569 process Effects 0.000 description 7
- 230000009471 action Effects 0.000 description 3
- 238000013507 mapping Methods 0.000 description 3
- 230000000007 visual effect Effects 0.000 description 3
- 230000008901 benefit Effects 0.000 description 2
- 239000012634 fragment Substances 0.000 description 2
- 230000003993 interaction Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000004422 calculation algorithm Methods 0.000 description 1
- 239000000969 carrier Substances 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/21—Server components or server architectures
- H04N21/218—Source of audio or video content, e.g. local disk arrays
- H04N21/21805—Source of audio or video content, e.g. local disk arrays enabling multiple viewpoints, e.g. using a plurality of cameras
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
- H04N13/20—Image signal generators
- H04N13/275—Image signal generators from 3D object models, e.g. computer-generated stereoscopic image signals
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
- H04N13/20—Image signal generators
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/21—Server components or server architectures
- H04N21/218—Source of audio or video content, e.g. local disk arrays
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
- H04N21/4402—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display
- H04N21/440236—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display by media transcoding, e.g. video is transformed into a slideshow of still pictures, audio is converted into text
Definitions
- Embodiments of the present invention relate to the field of mobile Internet technologies, and in particular, to a method and an apparatus for adjusting a panoramic video.
- the current user uses the terminal device to watch the video in the live broadcast or on-demand system of the video, and can watch the live program, or select the favorite program for playing according to personal preference.
- video data can be viewed in a live or on-demand system of video.
- a panoramic video is a smooth and clear dynamic video image composed of a myriad of panoramic images.
- the maturity of the current panoramic video stitching algorithm and the popularity of the panoramic recording device have made the panoramic video source more and more, and it is possible for the user to watch the panoramic video on the mobile terminal.
- the user can only view the displayed panoramic video screen, but cannot adjust, and the video playback is not flexible.
- the technical problem to be solved by the embodiments of the present invention is to provide a method for adjusting a panoramic video, which solves the problem of inflexible video playback.
- the embodiment of the present invention further provides an apparatus for adjusting a panoramic video to ensure implementation and application of the foregoing method.
- an embodiment of the present invention discloses a method for adjusting a panoramic video, the method comprising: binding a panoramic image frame of a panoramic video to a spherical model to generate an output video frame; and receiving an adjustment instruction, Converting the adjustment instruction into model adjustment information corresponding to the spherical model; adjusting the output video frame according to the model adjustment information to generate an adjusted output video frame.
- an embodiment of the present invention further discloses a device for adjusting a panoramic video, the device comprising: a binding module, performing video on demand or viewing in the mobile terminal During the live broadcast, the adjusted panoramic video is displayed by playing the adjusted output video frame; the conversion module receives the adjustment instruction, converts the adjustment instruction into model adjustment information corresponding to the spherical model; and the adjustment module adjusts according to the model Information is adjusted to the output video frame to generate an adjusted output video frame.
- a computer program comprising computer readable code that, when executed on a mobile terminal, causes the mobile terminal to perform the method described above.
- a computer readable medium wherein the computer program described above is stored.
- a mobile terminal includes: a processor and a memory;
- the memory is for storing program code
- the processor is configured to execute the program code stored by the memory, for:
- the embodiments of the invention include the following advantages:
- the present application provides a method for adjusting a panoramic video, which combines a panoramic image frame of a panoramic video with a spherical model to generate an output video frame, receives an adjustment instruction, and converts the adjustment instruction into a model corresponding to the spherical model. Adjusting information; adjusting the output video frame according to the model adjustment information to generate an adjusted output video frame.
- FIG. 1 is a flow chart showing the steps of an embodiment of a method for adjusting a panoramic video according to the present invention
- FIG. 2 is a flow chart showing the steps of another embodiment of the method for adjusting panoramic video according to the present invention.
- FIG. 3 is a structural block diagram of an embodiment of an apparatus for adjusting a panoramic video according to the present invention.
- FIG. 4 is a structural block diagram of another embodiment of an apparatus for adjusting panoramic video according to the present invention.
- FIG. 5 is a structural block diagram of a video binding sub-module in an optional embodiment of the present invention.
- FIG. 6 is a structural block diagram of a matrix calculation sub-module in an alternative embodiment of the present invention.
- Figure 7 shows schematically a block diagram of a mobile terminal for performing the method according to the invention
- Fig. 8 schematically shows a storage unit for holding or carrying program code implementing the method according to the invention.
- One of the core concepts of the embodiments of the present invention is to generate an output video frame by binding a panoramic image frame of the panoramic video to a spherical model; receiving an adjustment instruction, and converting the adjustment instruction into a model corresponding to the spherical model Adjusting information; adjusting the output video frame according to the model adjustment information to generate an adjusted output video frame.
- FIG. 1 a flow chart of steps of an embodiment of a method for adjusting a panoramic video according to the present invention is shown, which may specifically include the following steps:
- Step S102 binding the panoramic image frame of the panoramic video to the spherical model to generate an output.
- Video frame
- the panoramic source data includes a 720-degree or 360-degree panoramic video source, and the dynamic video can be viewed 360 degrees from the top, bottom, left, and right of the camera position.
- the panoramic source data includes a plurality of panoramic image frames, and the panoramic source data needs to realize a 3D stereoscopic effect of panoramic playback by means of a stereo model such as a spherical model, which can be realized by binding the stereo model with the panoramic image frame of the panoramic source data. .
- the panoramic slice source when performing video on-demand or live broadcast, can be used to play the 3D stereoscopic video, so the spherical model can be bound to each panoramic image frame of the panoramic video. After binding, an output video frame can be generated, and the code stream of the output video frame is played in the mobile terminal to realize corresponding panoramic video playback.
- the mobile terminal refers to a computer device that can be used in mobile, including a smart phone, a tablet computer, a vehicle terminal, and the like.
- Step S104 Receive an adjustment instruction, and convert the adjustment instruction into model adjustment information corresponding to the spherical model.
- the embodiment of the invention can not only play the panoramic video in the mobile terminal, but also realize the interaction with the panoramic video according to the adjustment operation of the user, such as freely switching the user's perspective according to the video scene, freely zooming in or expanding the visual perspective of the video.
- the function of video playback is more abundant.
- the adjustment instruction corresponding to the adjustment operation of the user may be received, and the adjustment content, such as rotation, scaling, etc., is determined according to the adjustment instruction, thereby converting the adjustment instruction, and determining model adjustment information corresponding to the spherical model.
- Step S106 Adjust the output video frame according to the model adjustment information to generate an adjusted output video frame.
- the adjustment information may be mapped to the corresponding output video frame according to the model, so that the output video frame of the panoramic video is adjusted to generate an adjusted output video frame, such as Switch the camera's angle of view, zoom in or zoom in on the viewing angle of the panoramic video.
- the panoramic image frame of the panoramic video is bound to the spherical model to generate an output video frame; receiving an adjustment instruction, converting the adjustment instruction into model adjustment information corresponding to the spherical model;
- the adjustment information adjusts the output video frame to generate an adjusted output video frame.
- FIG. 2 a flow chart of steps of an embodiment of a method for adjusting a panoramic video according to the present invention is shown. Specifically, the method may include the following steps:
- Step S202 Establish a spherical model based on the model information, where the model information includes a vertex, a normal vector, and a spherical texture coordinate of the spherical model.
- Step S204 Parse each panoramic video frame to determine image texture information of each panoramic video frame.
- Binding a panoramic video frame to a spherical model requires first obtaining the data information of the panoramic video frame and determining the model information of the spherical model, and mapping the panoramic video frame to the spherical model according to the data information and the model information, thereby implementing binding. Therefore, before binding, each panoramic video frame can be parsed to determine image texture information of each panoramic video frame.
- texture is an important visual cues, which are ubiquitous in images.
- the image texture information includes the relationship between the tone primitives and the tone primitives that make up the texture, such as the texture ID (Identity).
- the panoramic video frame can then be bound to the spherical model according to the texture information, and the texture information and the spherical model are mapped to bind the panoramic video frame to the spherical model.
- the specific steps are as follows:
- Step S206 determining a position of the camera in the image texture information, and setting a position of the camera as a vertex of the spherical model.
- Step S208 Perform point matching on the image texture information and the spherical texture coordinates according to the normal vector and the vertex of the spherical model.
- Step S210 Bind the panoramic video frame to the spherical model according to the point correspondence.
- the panoramic video frame In order to realize the binding of the panoramic video frame and the spherical model, it is necessary to correspond the image texture information of the panoramic video frame with the model information of the spherical model.
- the position of the camera of the frequency frame The position of the camera is set to the apex of the spherical model, such as setting the position of the camera as a coordinate point (0, 0, 0). Correspondence between the position of the camera in the image texture information and the vertices of the spherical model is achieved.
- the panoramic video frame can then be divided into fragments of a plurality of specific geometries, which are generally divided into a plurality of triangular fragments for ease of division. Determining three vertex information of the triangle according to the texture information, and according to the normal vector and the vertex of the spherical model, the vertex information of the plurality of triangles is correspondingly matched with the spherical texture coordinates such as (0, 0) to (1, 1), Therefore, the panoramic video frame is bound to the spherical model according to the point corresponding, and the binding can be implemented by a function opengl.
- the output video frame can be played to display the panoramic video.
- the specific adjustment steps are as follows:
- Step S212 Calculate the placement state information of the mobile terminal according to the gravity sensing parameter, and determine the motion direction information according to the placement state information of the mobile terminal.
- the movement direction information of the gyroscope and the touch screen in the mobile terminal is determined by the device placement state information, for example, if the touch screen is a horizontal screen, the touch screen correctly inputs the values of x and y; when the touch screen is a vertical screen, the values of x and y are exchanged.
- the touch screen is a reverse screen, the touch screen correctly inputs the values of x and z; the touch screen exchanges the values of x and z when the screen is reversed.
- converting the adjustment instruction to the model adjustment information corresponding to the spherical model includes: calculating a viewpoint matrix according to the adjustment instruction; and determining model adjustment information corresponding to the spherical model according to the viewpoint matrix . That is, after receiving the adjustment instruction, the viewpoint matrix may be calculated according to the adjustment instruction, and then the model adjustment information corresponding to the spherical model is calculated by using the viewpoint matrix, and the specific steps are as follows:
- Step S214 Determine adjustment information according to the adjustment instruction.
- the adjustment instruction includes: a single-finger adjustment instruction and/or a two-finger adjustment instruction
- the adjustment information is determined according to an adjustment instruction corresponding to the adjustment operation of the user. There are three situations that can be included:
- the user's adjustment operation includes the function of single-finger drawing to achieve switching perspective.
- the corresponding operation instruction is also a single-finger adjustment instruction, and the corresponding adjustment information is rotation information.
- the mobile terminal is a gyroscope that measures the switching of the angle of view, that is, the rotation information. That is, the rotation direction and the rotation angle of the gyroscope are determined according to the single-finger adjustment command, and the rotation direction and the rotation angle are used as the rotation information.
- the user's adjustment operation includes pinching, which realizes the function of zooming in or expanding the visual angle of the panoramic video.
- the operation instruction corresponding to the adjustment operation is the two-finger adjustment instruction, and the corresponding adjustment information is the scaling information.
- the mobile terminal determines the zoom-in or enlargement of the angle of view, that is, the zoom information, through the sensing information of the touch screen. That is, the zoom information is determined according to the two-finger adjustment command of the touch screen.
- the user's adjustment operation includes single-finger drawing to switch the angle of view, and pinch-to-finger to achieve a close or wide viewing angle of the panoramic video.
- the operation instruction corresponding to the user's adjustment operation is the single-finger adjustment instruction and the two-finger adjustment instruction, and the corresponding adjustment information is the rotation information and the scaling information. That is, the rotation direction and the rotation angle of the gyroscope are determined according to the single-finger adjustment command, and the rotation direction and the rotation angle are used as rotation information; and the zoom information is determined according to the two-finger adjustment command to the touch screen.
- Step S216 Calculate a viewpoint matrix according to the motion direction information and the adjustment information.
- the viewpoint matrix includes a current transformation matrix, a projection matrix, an orientation matrix, and a final transformation matrix. Firstly, the current transformation matrix of the current output video frame is obtained, and then the orientation matrix is calculated according to the movement direction information of the gyroscope, the touch screen and the rotation information of the gyroscope; the projection matrix is calculated according to the zoom information of the touch screen, and finally the final transformation matrix is obtained.
- Step S218, determining model adjustment information corresponding to the spherical model according to the view matrix.
- Step S220 Adjust the bound output video frame according to the model adjustment information to generate an adjusted output video frame.
- the information of each point in the current transformation matrix such as the coordinate values of each point, is obtained, and the corresponding model adjustment information on the spherical model is determined. If a certain point is selected, the current coordinate value is determined according to the current transformation matrix, the coordinate value after the rotation processing is determined by the transformation toward the matrix, and the coordinate value after the scaling process is obtained by transformation of the projection matrix, and the information is adjusted according to the model. That is, the correspondence between the coordinate values of the four matrices in the view matrix, the coordinate values of the points of the bound output video frame are adjusted to generate an adjusted output video frame.
- Step S222 Display the adjusted panoramic video by playing the adjusted output video frame when performing video on demand or live video broadcast in the mobile terminal.
- the panoramic video can be adjusted by adjusting the instruction. After the adjustment of the output video frame is completed, the adjusted output video frame can be played to display the adjusted panoramic video.
- the panoramic source is adjusted and displayed according to the adjustment operation of the user, thereby realizing effective interaction between the user and the panoramic video source, and embodies the advantage of the panoramic video relative to the ordinary video.
- the image texture information of each panoramic video frame is determined, and the image texture information is correspondingly mapped to the spherical texture coordinates according to the normal vector and the vertex of the spherical model to realize the panoramic video frame and the spherical shape.
- the binding of the model The mapping between the texture information of the image and the spherical texture coordinates makes the binding process simpler and more accurate.
- FIG. 3 a block diagram of a structure of an apparatus for adjusting a panoramic video according to the present invention is shown. Specifically, the following modules may be included: a binding module 302, a conversion module 304, and an adjustment module 306, where:
- the binding module 302 is configured to display the adjusted panoramic video by playing the adjusted output video frame when performing video on demand or live video in the mobile terminal.
- the conversion module 304 is configured to receive an adjustment instruction, and convert the adjustment instruction into model adjustment information corresponding to the spherical model;
- the adjusting module 306 is configured to adjust the output video frame according to the model adjustment information to generate an adjusted output video frame.
- the panoramic image frame of the panoramic video is bound to the spherical model to generate an output video frame; receiving an adjustment instruction, converting the adjustment instruction into model adjustment information corresponding to the spherical model; The adjustment information adjusts the output video frame to generate an adjusted output video frame.
- FIG. 4 a block diagram showing the structure of another embodiment of the apparatus for adjusting panoramic video of the present invention is shown. Specifically, the following modules may be included: a binding module 302, a conversion module 304, an adjustment module 306, and a playback module 308, where:
- the binding module 302 is configured to display the adjusted panoramic video by playing the adjusted output video frame when performing video on demand or live video in the mobile terminal.
- the binding module 302 includes a model establishing submodule 3022, a video parsing submodule 3024, and a video binding submodule 3026.
- the model building sub-module 3022 is configured to establish a spherical model based on the model information, wherein the model information includes a vertex, a normal vector, and a spherical texture coordinate of the spherical model.
- the video parsing sub-module 3024 is configured to parse each panoramic video frame to determine image texture information of each panoramic video frame.
- the video binding sub-module 3026 is configured to bind the panoramic video frame to the spherical model according to the texture information.
- the video binding sub-module 3026 includes: a vertex determining unit 30262, a texture corresponding unit 30264, and a video binding unit 30266.
- the vertex determining unit 30262 is configured to determine a position of the camera in the image texture information, and set a position of the camera as a vertex of the spherical model.
- the texture corresponding unit 30264 is configured to point the image texture information and the spherical texture coordinates according to the normal vector and the vertex of the spherical model.
- the video binding unit 30266 is configured to bind the panoramic video frame to the spherical model according to the point correspondence.
- the conversion module 304 is configured to receive an adjustment instruction, and convert the adjustment instruction into model adjustment information corresponding to the spherical model.
- the conversion module 304 includes:
- the matrix calculation sub-module 3042 is configured to calculate a view matrix according to the adjustment instruction.
- the adjustment information determining sub-module 3044 is configured to determine model adjustment information corresponding to the spherical model according to the view matrix.
- the matrix calculation sub-module 3042 includes: a direction determining unit 30422, an adjustment information determining unit 30424, and a viewpoint matrix calculating unit 30426.
- the direction determining unit 30422 is configured to calculate placement state information of the mobile terminal according to the gravity sensing parameter, and determine motion direction information according to the placement state information of the mobile terminal.
- the adjustment information determining unit 30424 is configured to determine the adjustment information according to the adjustment instruction.
- the viewpoint matrix calculation unit 30426 is configured to calculate a viewpoint matrix according to the motion direction information and the adjustment information.
- the adjustment instruction includes: a single-finger adjustment instruction and/or a two-finger adjustment instruction, where the adjustment information includes: rotation information and/or scaling information;
- the adjustment information determining unit 30424 is configured to determine a rotation direction and a rotation angle of the gyroscope according to the single-finger adjustment instruction, and use the rotation direction and the rotation angle as rotation information; and/or determine the scaling information according to the two-finger adjustment instruction of the touch screen. .
- the adjusting module 306 is configured to adjust the output video frame according to the model adjustment information to generate an adjusted output video frame.
- the playing module 308 is configured to display the adjusted panoramic video by playing the adjusted output video frame when performing video on demand or live video in the mobile terminal.
- the image texture information of each panoramic video frame is determined, and the image texture information is correspondingly mapped to the spherical texture coordinates according to the normal vector and the vertex of the spherical model to realize the panoramic video frame and the spherical shape.
- the binding of the model The mapping between the texture information of the image and the spherical texture coordinates makes the binding process simpler and more accurate. .
- the description is relatively simple, and the relevant parts can be referred to the description of the method embodiment.
- the various component embodiments of the present invention may be implemented in hardware, or in a software module running on one or more processors, or in a combination thereof.
- a microprocessor or digital signal processor may be used in practice to implement some or all of the functionality of some or all of the components of the mobile terminal in accordance with embodiments of the present invention.
- the invention can also be implemented as a device or device program (e.g., a computer program and a computer program product) for performing some or all of the methods described herein.
- a program implementing the invention may be stored on a computer readable medium or may be in the form of one or more signals. Such signals may be downloaded from an Internet website, provided on a carrier signal, or provided in any other form.
- FIG. 7 shows a mobile terminal that can implement a method of playing a video according to the present invention.
- the mobile terminal traditionally includes a processor 710 and a computer program product or computer readable medium in the form of a memory 720.
- Memory 720 can be an electronic memory such as a flash memory, EEPROM (Electrically Erasable Programmable Read Only Memory), EPROM, hard disk, or ROM.
- Memory 720 has a memory space 730 for program code 731 for performing any of the method steps described above.
- storage space 730 for program code may include various program code 731 for implementing various steps in the above methods, respectively.
- the program code can be read from or written to one or more computer program products.
- Such computer program products include program code carriers such as hard disks, compact disks (CDs), memory cards or floppy disks.
- Such a computer program product is typically a portable or fixed storage unit as described with reference to FIG.
- the storage unit may have a storage section, a storage space, and the like arranged similarly to the storage 720 in the mobile terminal of FIG.
- the program code can be compressed, for example, in an appropriate form.
- the storage unit includes computer readable code 731', i.e., code readable by a processor, such as 710, that when executed by the mobile terminal causes the mobile terminal to perform the methods described above Each step.
- a processor such as 710
- embodiments of the invention may be in the form of an entirely hardware embodiment, an entirely software embodiment, or a combination of software and hardware.
- embodiments of the invention may take the form of a computer program product embodied on one or more computer usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) including computer usable program code.
- Embodiments of the invention are described with reference to flowchart illustrations and/or block diagrams of methods, terminal devices (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flowchart illustrations and/or FIG.
- These computer program instructions can be provided to a processor of a general purpose computer, special purpose computer, embedded processor or other programmable data processing terminal device to produce a machine such that instructions are executed by a processor of a computer or other programmable data processing terminal device
- Means are provided for implementing the functions specified in one or more of the flow or in one or more blocks of the flow chart.
- the computer program instructions can also be stored in a computer readable memory that can direct a computer or other programmable data processing terminal device to operate in a particular manner, such that the instructions stored in the computer readable memory produce an article of manufacture comprising the instruction device.
- the instruction device implements the functions specified in one or more blocks of the flowchart or in a flow or block of the flowchart.
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Databases & Information Systems (AREA)
- Controls And Circuits For Display Device (AREA)
- Studio Devices (AREA)
Abstract
本发明实施例提供了一种全景视频的调整方法及装置，使得视频播放更加灵活，功能更加丰富。所述的方法包括：将全景视频的全景图像帧与球形模型进行绑定，生成输出视频帧；接收调整指令，将所述调整指令转换为与所述球形模型对应的模型调整信息；根据所述模型调整信息对所述输出视频帧进行调整，生成调整的输出视频帧。通过根据用户的调整操作对全景片源进行相应的调整并显示，使得全景视频播放更加灵活，丰富了全景视频播放的功能。
Description
本申请要求在2015年11月23日提交中国专利局、申请号为201510818977.9、发明名称为“一种视频的播放方法和装置”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
本发明实施例涉及移动互联网技术领域，尤其是涉及一种全景视频的调整方法和装置。
当前用户采用终端设备，进入视频的直播或点播***中观看视频，既可以看直播节目，也可以根据个人爱好，搜索后选择喜欢的节目进行播放。例如在智能手机、电脑以及智能电视上均可以进入视频的直播或点播***中观看视频数据。
在移动端视频的直播或点播***中，用户所能观看到的视频内容取决于视频源。全景视频是由无数张全景图片串连起来组成的流畅清楚的动态视频图像。当前全景视频拼接算法的成熟和全景录制设备的普及，使得全景视频片源越来越多，用户在移动端观看全景视频成为可能。但目前用户仅能观看显示的全景视频画面，而不能进行调整，视频播放不灵活。
发明内容
本发明实施例所要解决的技术问题是提供一种全景视频的调整方法，解决视频播放不灵活的问题。
相应的，本发明实施例还提供了一种全景视频的调整装置，用以保证上述方法的实现及应用。
根据本发明的一个方面，本发明实施例公开了一种全景视频的调整方法，所述的方法包括：将全景视频的全景图像帧与球形模型进行绑定，生成输出视频帧；接收调整指令，将所述调整指令转换为与所述球形模型对应的模型调整信息；根据所述模型调整信息对所述输出视频帧进行调整，生成调整的输出视频帧。
根据本发明的另一个方面，本发明实施例还公开了一种全景视频的调整装置，所述的装置包括：绑定模块，在移动终端中进行视频点播或视
频直播时，通过播放调整的输出视频帧显示调整的全景视频；转换模块，接收调整指令，将所述调整指令转换为与所述球形模型对应的模型调整信息；调整模块，根据所述模型调整信息对所述输出视频帧进行调整，生成调整的输出视频帧。
根据本发明的又一个方面，提供了一种计算机程序，其包括计算机可读代码，当所述计算机可读代码在移动终端上运行时，导致所述移动终端执行上述的方法。
根据本发明的再一个方面，提供了一种计算机可读介质，其中存储了上述的计算机程序。
本发明的再一个发明，提供了一种移动终端，包括：处理器和存储器；
所述存储器用于存储程序代码；
所述处理器用于执行所述存储器存储的程序代码，用于：
将全景视频的全景图像帧与球形模型进行绑定，生成输出视频帧；
接收调整指令，将所述调整指令转换为与所述球形模型对应的模型调整信息；
根据所述模型调整信息对所述全景视频的输出视频帧进行调整，生成调整的输出视频帧。
与现有技术相比，本发明实施例包括以下优点：
本申请提供了一种全景视频的调整方法，将全景视频的全景图像帧与球形模型进行绑定，生成输出视频帧；接收调整指令，将所述调整指令转换为与所述球形模型对应的模型调整信息；根据所述模型调整信息对所述输出视频帧进行调整，生成调整的输出视频帧。通过根据用户的调整操作对全景片源进行相应的调整并显示，使得全景视频播放更加灵活，丰富了全景视频播放的功能。
为了更清楚地说明本发明实施例或现有技术中的技术方案，下面将对实施例或现有技术描述中所需要使用的附图作一简单地介绍，显而易见地，下面描述中的附图是本发明的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。
图1是本发明的一种全景视频的调整方法实施例的步骤流程图；
图2是本发明的另一种全景视频的调整方法实施例的步骤流程图；
图3是本发明的一种全景视频的调整装置实施例的结构框图；
图4是本发明的另一种全景视频的调整装置实施例的结构框图；
图5是本发明一个可选实施例中视频绑定子模块的结构框图；
图6是本发明一个可选实施例中矩阵计算子模块的结构框图；
图7示意性地示出了用于执行根据本发明的方法的移动终端的框图；以及
图8示意性地示出了用于保持或者携带实现根据本发明的方法的程序代码的存储单元。
为使本发明实施例的目的、技术方案和优点更加清楚，下面将结合本发明实施例中的附图，对本发明实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例是本发明一部分实施例，而不是全部的实施例。基于本发明中的实施例，本领域普通技术人员在没有作出创造性劳动前提下所获得的所有其他实施例，都属于本发明保护的范围。
本发明实施例的核心构思之一在于，通过将全景视频的全景图像帧与球形模型进行绑定，生成输出视频帧；接收调整指令，将所述调整指令转换为与所述球形模型对应的模型调整信息；根据所述模型调整信息对所述输出视频帧进行调整，生成调整的输出视频帧。通过根据用户的调整操作对全景片源进行相应的调整并显示，使得全景视频播放更加灵活，丰富了全景视频播放的功能。
实施例一
详细介绍本发明实施例全景视频的调整方法。
参照图1，示出了本发明的一种全景视频的调整方法实施例的步骤流程图，具体可以包括如下步骤：
步骤S102、将全景视频的全景图像帧与球形模型进行绑定，生成输出
视频帧。
全景片源数据包括720度或者360度全景视频源，可以在摄像机位置的上下左右360度任意的观看动态视频。全景片源数据包括多个全景图像帧，全景片源数据需要借助立体模型如球形模型来实现全景播放的3D立体效果，可以通过将立体模型与其全景片源数据的全景图像帧进行绑定来实现。
本实施例中，在进行视频的点播或直播时，可以采用全景片源来播放3D立体的视频，因此可以将球形模型与全景视频的各全景图像帧进行绑定。绑定后即可生成输出视频帧，在移动终端中播放该输出视频帧的码流即实现相应的全景视频的播放。其中，移动终端指的是可以在移动中使用的计算机设备，包括智能手机、平板电脑、车载终端等。
步骤S104、接收调整指令，将所述调整指令转换为与所述球形模型对应的模型调整信息。
本发明实施例不仅可以实现在移动终端中播放全景视频，而且可以根据用户的调整操作来实现与全景视频的互动，如根据视频场景随意切换用户的视角、自由拉近或者扩大视频的可视视角，从而在直播或点播过程中增加视频播放的灵活性，使得视频播放的功能更丰富。可以接收用户的调整操作对应的调整指令，依据该调整指令确定调整内容，如旋转、缩放等，从而对调整指令进行转换，确定与所述球形模型对应的模型调整信息。
步骤S106、根据所述模型调整信息对所述输出视频帧进行调整，生成调整的输出视频帧。
由于输出视频帧与球形模型是绑定的，因此依据所述模型调整信息可以映射到对应的输出视频帧上，从而对所述全景视频的输出视频帧进行调整，生成调整的输出视频帧，如进行摄像机视角的切换，拉近或者扩大全景视频的可视视角等。
综上所述，将全景视频的全景图像帧与球形模型进行绑定，生成输出视频帧；接收调整指令，将所述调整指令转换为与所述球形模型对应的模型调整信息；根据所述模型调整信息对所述输出视频帧进行调整，生成调整的输出视频帧。通过根据用户的调整操作对全景片源进行相应的调整并
显示，使得全景视频播放更加灵活，丰富了全景视频播放的功能。
实施例二
详细介绍本发明实施例的全景视频的调整方法。
参照图2，示出了本发明的另一种全景视频的调整方法实施例的步骤流程图，具体可以包括如下步骤：
步骤S202、基于模型信息建立球形模型，其中，所述模型信息包括所述球形模型的顶点、法向量及球形纹理坐标。
步骤S204、对各全景视频帧进行解析，确定各全景视频帧的图像纹理信息。
将全景视频帧与球形模型进行绑定，需要先获取全景视频帧的数据信息以及确定球形模型的模型信息，依据数据信息及模型信息将全景视频帧映射到球形模型上，从而实现绑定。因此，在绑定前，可以对各全景视频帧进行解析，确定各全景视频帧的图像纹理信息。其中，纹理是一种重要的视觉线索，在图像中普遍存在，图像纹理信息包括组成纹理的色调基元和色调基元间的相互关系，例如纹理ID(Identity)。根据需要确定球形模型的模型信息，先设置如球形模型的顶点、法向量及球形纹理坐标，再建立球形模型。基于这些模型信息建立球形模型。
然后可以依据所述纹理信息将全景视频帧与所述球形模型进行绑定，将将纹理信息和球形模型进行映射，从而将全景视频帧与所述球形模型进行绑定，具体步骤如下：
步骤S206、在所述图像纹理信息中确定摄像机的位置，将所述摄像机的位置设置为所述球形模型的顶点。
步骤S208、依据所述球形模型的法向量和顶点，将所述图像纹理信息与球形纹理坐标进行点对应。
步骤S210、依据所述点对应将所述全景视频帧与所述球形模型进行绑定。
为了实现全景视频帧与球形模型的绑定，需要将全景视频帧的图像纹理信息与球形模型的模型信息进行对应。通过对图像纹理信息中组成纹理的色调基元和色调基元间的相互关系进行分析，可以确定用于拍摄全景视
频帧的摄像机的位置。将所述摄像机的位置设置为球形模型的顶点，如将所述摄像机的位置设置为坐标点(0，0，0)。实现图像纹理信息中摄像机的位置与球形模型的顶点的对应。
然后，可以将全景视频帧划分成多个特定几何形状的碎片，为了方便划分，一般划分成多个三角形的碎片。根据纹理信息确定该三角形的三个顶点信息，依据所述球形模型的法向量和顶点，将多个三角形的顶点信息与球形纹理坐标如(0，0)～(1，1)进行点对应，从而依据所述点对应将所述全景视频帧与所述球形模型进行绑定，绑定可以通过函数opengl实现。
在移动终端中进行视频点播或视频直播时，通过上述方式将全景图像帧与球形模型进行绑定得到输出视频帧后，可以对输出视频帧进行播放，从而显示全景视频。用户观看过程中，若想要调整观看角度、细节等，可以对输出视频帧进行调整，具体调整步骤如下：
步骤S212、依据重力感应参数计算所述移动终端的摆放状态信息，依据所述移动终端的摆放状态信息确定运动方向信息。
获取移动终端的重力感应参数，通过重力感应参数在球形模型坐标系中xyz三个方向的分量，来计算移动终端的摆放状态信息，如竖屏、反竖屏、横屏或反横屏。通过设备摆放状态信息确定移动终端中陀螺仪和触摸屏的运动方向信息，例如：若触摸屏是横屏时，触摸屏正确输入x和y的值；则触摸屏是竖屏时，交换x和y的值；触摸屏是反横屏时，触摸屏正确输入x和z的值；触摸屏是反竖屏时交换x和z的值。
本实施例中，将所述调整指令转换为与所述球形模型对应的模型调整信息，包括：依据所述调整指令计算视点矩阵；根据所述视点矩阵确定与所述球形模型对应的模型调整信息。即在接收到调整指令之后，可以依据调整指令计算视点矩阵，然后采用该视点矩阵计算与所述球形模型对应的模型调整信息，具体步骤如下：
步骤S214、依据所述调整指令确定调整信息。
本发明的一个优选实施例中，调整指令包括：单指调整指令和/或双指调整指令，所述调整信息包括：旋转信息和/或缩放信息。则依据所述调整指令确定调整信息包括：依据单指调整指令确定陀螺仪的旋转方向和旋转
角度，将所述旋转方向和旋转角度作为旋转信息；和/或，依据对触摸屏的双指调整指令确定缩放信息。
依据用户的调整操作对应的调整指令确定调整信息。可以包括三种情况：
第一种情况是，用户的调整操作包括单指划取实现切换视角的功能。对应的操作指令也就是单指调整指令，对应的调整信息就是旋转信息。而移动终端是通过陀螺仪来测定视角的切换即旋转信息。即依据单指调整指令确定陀螺仪的旋转方向和旋转角度，将所述旋转方向和旋转角度作为旋转信息。
第二种情况是，用户的调整操作包括双指捏合，实现拉近或扩大全景视频的可视视角的功能。调整操作对应的操作指令也就是双指调整指令，对应的调整信息就是缩放信息。而移动终端通过触摸屏的感应信息来确定视角的拉近或扩大即缩放信息。即依据对触摸屏的双指调整指令确定缩放信息。
第三种情况是，用户的调整操作包括单指划取从而切换视角，和双指捏合实现拉近或扩大全景视频的可视视角。用户的调整操作对应的操作指令也就是单指调整指令和双指调整指令，对应的调整信息就是旋转信息和缩放信息。即依据单指调整指令确定陀螺仪的旋转方向和旋转角度，将所述旋转方向和旋转角度作为旋转信息；和依据对触摸屏的双指调整指令确定缩放信息。
步骤S216、依据所述运动方向信息和调整信息计算视点矩阵。
视点矩阵包括当前变换矩阵、投影矩阵、朝向矩阵和最终变换矩阵。首先获取当前输出视频帧的当前变换矩阵，再依据陀螺仪、触摸屏的运动方向信息和陀螺仪的旋转信息计算出朝向矩阵；依据触摸屏的缩放信息来计算投影矩阵，最终得到最终变换矩阵。
步骤S218、根据所述视点矩阵确定与所述球形模型对应的模型调整信息。
步骤S220、根据所述模型调整信息对绑定的输出视频帧进行调整，生成调整的输出视频帧。
获取当前变换矩阵中的各点的信息，如各点的坐标值，确定在球形模型上对应的模型调整信息。如选定某一点，依据当前变换矩阵确定当前坐标值，通过朝向矩阵的变换确定经旋转处理后的坐标值，通过投影矩阵的变换得出经缩放处理后的坐标值，根据所述模型调整信息即视点矩阵中四个矩阵之间的坐标值的对应关系，对绑定的输出视频帧各点的坐标值进行调整，生成调整的输出视频帧。
步骤S222、在移动终端中进行视频点播或视频直播时，通过播放调整的输出视频帧显示调整的全景视频。
在移动终端中进行视频点播或视频直播时，通过调整指令可以对全景视频进行调整，在完成输出视频帧的调整后，可以播放调整的输出视频帧从而显示调整的全景视频。本发明实施例通过根据用户的调整操作对全景片源进行相应的调整并显示，实现了用户与全景视频源的有效互动，体现了全景视频相对于普通视频的优势。
综上所述，通过对全景视频帧进行解析，确定各全景视频帧的图像纹理信息，依据球形模型的法向量和顶点，将图像纹理信息与球形纹理坐标进行点对应，实现全景视频帧与球形模型的绑定。通过图像的纹理信息与球形纹理坐标进行点对应，使得绑定过程更简单并且更准确。
需要说明的是，对于方法实施例，为了简单描述，故将其都表述为一系列的动作组合，但是本领域技术人员应该知悉，本发明实施例并不受所描述的动作顺序的限制，因为依据本发明实施例，某些步骤可以采用其他顺序或者同时进行。其次，本领域技术人员也应该知悉，说明书中所描述的实施例均属于优选实施例，所涉及的动作并不一定是本发明实施例所必须的。
实施例三
参照图3，示出了本发明一种全景视频的调整装置实施例的结构框图，具体可以包括如下模块：绑定模块302，转换模块304，以及调整模块306，其中：
绑定模块302，用于在移动终端中进行视频点播或视频直播时，通过播放调整的输出视频帧显示调整的全景视频。
转换模块304，用于接收调整指令，将所述调整指令转换为与所述球形模型对应的模型调整信息；
调整模块306，用于根据所述模型调整信息对所述输出视频帧进行调整，生成调整的输出视频帧。
综上所述，将全景视频的全景图像帧与球形模型进行绑定，生成输出视频帧；接收调整指令，将所述调整指令转换为与所述球形模型对应的模型调整信息；根据所述模型调整信息对所述输出视频帧进行调整，生成调整的输出视频帧。通过根据用户的调整操作对全景片源进行相应的调整并显示，使得全景视频播放更加灵活，丰富了全景视频播放的功能。
参照图4，示出了本发明另一种全景视频的调整装置实施例的结构框图。具体可以包括如下模块：绑定模块302，转换模块304，调整模块306，以及播放模块308，其中：
绑定模块302，用于在移动终端中进行视频点播或视频直播时，通过播放调整的输出视频帧显示调整的全景视频。
本发明另一个可选实施例中，所述绑定模块302包括：模型建立子模块3022，视频解析子模块3024，以及视频绑定子模块3026。
模型建立子模块3022，用于基于模型信息建立球形模型，其中，所述模型信息包括所述球形模型的顶点、法向量及球形纹理坐标。
视频解析子模块3024，用于对各全景视频帧进行解析，确定各全景视频帧的图像纹理信息。
视频绑定子模块3026，用于依据所述纹理信息将全景视频帧与所述球形模型进行绑定。
如图5所示，本发明另一个可选实施例中，所述视频绑定子模块3026包括：顶点确定单元30262，纹理对应单元30264，以及视频绑定单元30266。
顶点确定单元30262，用于在所述图像纹理信息中确定摄像机的位置，将所述摄像机的位置设置为所述球形模型的顶点。
纹理对应单元30264，用于依据所述球形模型的法向量和顶点，将所述图像纹理信息与球形纹理坐标进行点对应。
视频绑定单元30266，用于依据所述点对应将所述全景视频帧与所述球形模型进行绑定。
转换模块304，用于接收调整指令，将所述调整指令转换为与所述球形模型对应的模型调整信息.
本发明另一个可选实施例中，所述转换模块304包括：
矩阵计算子模块3042，用于依据所述调整指令计算视点矩阵。
调整信息确定子模块3044，用于根据所述视点矩阵确定与所述球形模型对应的模型调整信息。
如图6所示，本发明另一个可选实施例中，所述矩阵计算子模块3042包括：方向确定单元30422，调整信息确定单元30424，以及视点矩阵计算单元30426。
方向确定单元30422，用于依据重力感应参数计算所述移动终端的摆放状态信息，依据所述移动终端的摆放状态信息确定运动方向信息。
调整信息确定单元30424，用于依据所述调整指令确定调整信息。
视点矩阵计算单元30426，用于依据所述运动方向信息和调整信息计算视点矩阵。
其中，本发明另一个可选实施例中，所述调整指令包括：单指调整指令和/或双指调整指令，所述调整信息包括：旋转信息和/或缩放信息；
则调整信息确定单元30424，用于依据单指调整指令确定陀螺仪的旋转方向和旋转角度，将所述旋转方向和旋转角度作为旋转信息；和/或依据对触摸屏的双指调整指令确定缩放信息。
调整模块306，用于根据所述模型调整信息对所述输出视频帧进行调整，生成调整的输出视频帧。
播放模块308，用于在移动终端中进行视频点播或视频直播时，通过播放调整的输出视频帧显示调整的全景视频。
综上所述，通过对全景视频帧进行解析，确定各全景视频帧的图像纹理信息，依据球形模型的法向量和顶点，将图像纹理信息与球形纹理坐标进行点对应，实现全景视频帧与球形模型的绑定。通过图像的纹理信息与球形纹理坐标进行点对应，使得绑定过程更简单并且更准确。。
对于装置实施例而言，由于其与方法实施例基本相似，所以描述的比较简单，相关之处参见方法实施例的部分说明即可。
本说明书中的各个实施例均采用递进的方式描述，每个实施例重点说明的都是与其他实施例的不同之处，各个实施例之间相同相似的部分互相参见即可。
本发明的各个部件实施例可以以硬件实现，或者以在一个或者多个处理器上运行的软件模块实现，或者以它们的组合实现。本领域的技术人员应当理解，可以在实践中使用微处理器或者数字信号处理器(DSP)来实现根据本发明实施例的移动终端中的一些或者全部部件的一些或者全部功能。本发明还可以实现为用于执行这里所描述的方法的一部分或者全部的设备或者装置程序(例如，计算机程序和计算机程序产品)。这样的实现本发明的程序可以存储在计算机可读介质上，或者可以具有一个或者多个信号的形式。这样的信号可以从因特网网站上下载得到，或者在载体信号上提供，或者以任何其他形式提供。
例如，图7示出了可以实现根据本发明的视频的播放方法的移动终端。该移动终端传统上包括处理器710和以存储器720形式的计算机程序产品或者计算机可读介质。存储器720可以是诸如闪存、EEPROM(电可擦除可编程只读存储器)、EPROM、硬盘或者ROM之类的电子存储器。存储器720具有用于执行上述方法中的任何方法步骤的程序代码731的存储空间730。例如，用于程序代码的存储空间730可以包括分别用于实现上面的方法中的各种步骤的各个程序代码731。这些程序代码可以从一个或者多个计算机程序产品中读出或者写入到这一个或者多个计算机程序产品中。这些计算机程序产品包括诸如硬盘，紧致盘(CD)、存储卡或者软盘之类的程序代码载体。这样的计算机程序产品通常为如参考图8所述的便携式或者固定存储单元。该存储单元可以具有与图7的移动终端中的存储器720类似布置的存储段、存储空间等。程序代码可以例如以适当形式进行压缩。通常，存储单元包括计算机可读代码731’，即可以由例如诸如710之类的处理器读取的代码，这些代码当由移动终端运行时，导致该移动终端执行上面所描述的方法中的
各个步骤。本领域内的技术人员应明白，本发明实施例的实施例可提供为方法、装置、或计算机程序产品。因此，本发明实施例可采用完全硬件实施例、完全软件实施例、或结合软件和硬件方面的实施例的形式。而且，本发明实施例可采用在一个或多个其中包含有计算机可用程序代码的计算机可用存储介质(包括但不限于磁盘存储器、CD-ROM、光学存储器等)上实施的计算机程序产品的形式。
本发明实施例是参照根据本发明实施例的方法、终端设备(***)、和计算机程序产品的流程图和/或方框图来描述的。应理解可由计算机程序指令实现流程图和/或方框图中的每一流程和/或方框、以及流程图和/或方框图中的流程和/或方框的结合。可提供这些计算机程序指令到通用计算机、专用计算机、嵌入式处理机或其他可编程数据处理终端设备的处理器以产生一个机器，使得通过计算机或其他可编程数据处理终端设备的处理器执行的指令产生用于实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能的装置。
这些计算机程序指令也可存储在能引导计算机或其他可编程数据处理终端设备以特定方式工作的计算机可读存储器中，使得存储在该计算机可读存储器中的指令产生包括指令装置的制造品，该指令装置实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能。
这些计算机程序指令也可装载到计算机或其他可编程数据处理终端设备上，使得在计算机或其他可编程终端设备上执行一系列操作步骤以产生计算机实现的处理，从而在计算机或其他可编程终端设备上执行的指令提供用于实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能的步骤。
尽管已描述了本发明实施例的优选实施例，但本领域内的技术人员一旦得知了基本创造性概念，则可对这些实施例做出另外的变更和修改。所以，所附权利要求意欲解释为包括优选实施例以及落入本发明实施例范围的所有变更和修改。
最后，还需要说明的是，在本文中，诸如第一和第二等之类的关系术语仅仅用来将一个实体或者操作与另一个实体或操作区分开来，而不一定
要求或者暗示这些实体或操作之间存在任何这种实际的关系或者顺序。而且，术语“包括”、“包含”或者其任何其他变体意在涵盖非排他性的包含，从而使得包括一系列要素的过程、方法、物品或者终端设备不仅包括那些要素，而且还包括没有明确列出的其他要素，或者是还包括为这种过程、方法、物品或者终端设备所固有的要素。在没有更多限制的情况下，由语句“包括一个......”限定的要素，并不排除在包括所述要素的过程、方法、物品或者终端设备中还存在另外的相同要素。
以上对本发明所提供的一种全景视频的调整方法和一种全景视频的调整装置，进行了详细介绍，本文中应用了具体个例对本发明的原理及实施方式进行了阐述，以上实施例的说明只是用于帮助理解本发明的方法及其核心思想；同时，对于本领域的一般技术人员，依据本发明的思想，在具体实施方式及应用范围上均会有改变之处，综上所述，本说明书内容不应理解为对本发明的限制。
Claims (17)
- 一种全景视频的调整方法，其特征在于，所述的方法包括：将全景视频的全景图像帧与球形模型进行绑定，生成输出视频帧；接收调整指令，将所述调整指令转换为与所述球形模型对应的模型调整信息；根据所述模型调整信息对所述全景视频的输出视频帧进行调整，生成调整的输出视频帧。
- 根据权利要求1所述的方法，其特征在于，所述将全景视频的全景图像帧与球形模型进行绑定包括：基于模型信息建立球形模型，其中，所述模型信息包括所述球形模型的顶点、法向量及球形纹理坐标。对全景视频的全景视频帧进行解析，确定所述全景视频帧的图像纹理信息；依据所述纹理信息将所述全景视频帧与所述球形模型进行绑定。
- 根据权利要求2所述的方法，其特征在于，所述依据所述纹理信息将全景视频帧与所述球形模型进行绑定包括：在所述图像纹理信息中确定摄像机的位置，将所述摄像机的位置设置为所述球形模型的顶点；依据所述球形模型的法向量和顶点，将所述图像纹理信息与球形纹理坐标进行点对应；依据所述点对应将所述全景视频帧与所述球形模型进行绑定。
- 根据权利要求1所述的方法，其特征在于，所述将所述调整指令转换为与所述球形模型对应的模型调整信息包括：依据所述调整指令计算视点矩阵；根据所述视点矩阵确定与所述球形模型对应的模型调整信息。
- 根据权利要求4所述的方法，其特征在于，所述依据所述调整指令计算视点矩阵，包括：依据重力感应参数计算所述移动终端的摆放状态信息，依据所述移动终端的摆放状态信息确定运动方向信息；依据所述调整指令确定调整信息；依据所述运动方向信息和调整信息计算视点矩阵。
- 根据权利要求5所述的方法，其特征在于，所述调整指令包括：单指调整指令和/或双指调整指令，所述调整信息包括：旋转信息和/或缩放信息；所述依据所述调整指令确定调整信息，包括：依据单指调整指令确定陀螺仪的旋转方向和旋转角度，将所述旋转方向和旋转角度作为旋转信息；和/或依据对触摸屏的双指调整指令确定缩放信息。
- 根据权利要求1所述的方法，其特征在于，还包括：在移动终端中进行视频点播或视频直播时，通过播放调整的输出视频帧显示调整的全景视频。
- 一种全景视频的调整装置，其特征在于，所述的装置包括：绑定模块，用于在移动终端中进行视频点播或视频直播时，通过播放调整的输出视频帧显示调整的全景视频；转换模块，用于接收调整指令，将所述调整指令转换为与所述球形模型对应的模型调整信息；调整模块，用于根据所述模型调整信息对所述输出视频帧进行调整，生成调整的输出视频帧。
- 根据权利要求8所述的装置，其特征在于，所述绑定模块包括：模型建立子模块，用于基于模型信息建立球形模型，其中，所述模型信息包括所述球形模型的顶点、法向量及球形纹理坐标；视频解析子模块，用于对各全景视频帧进行解析，确定各全景视频帧的图像纹理信息；视频绑定子模块，用于依据所述纹理信息将全景视频帧与所述球形模型进行绑定。
- 根据权利要求9所述的装置，其特征在于，所述视频绑定子模块包括：顶点确定单元，用于在所述图像纹理信息中确定摄像机的位置，将所 述摄像机的位置设置为所述球形模型的顶点；纹理对应单元，用于依据所述球形模型的法向量和顶点，将所述图像纹理信息与球形纹理坐标进行点对应；视频绑定单元，用于依据所述点对应将所述全景视频帧与所述球形模型进行绑定。
- 根据权利要求8所述的装置，其特征在于，所述转换模块包括：矩阵计算子模块，用于依据所述调整指令计算视点矩阵；调整信息确定子模块，用于根据所述视点矩阵确定与所述球形模型对应的模型调整信息。
- 根据权利要求11所述的装置，其特征在于，所述矩阵计算子模块包括：方向确定单元，用于依据重力感应参数计算所述移动终端的摆放状态信息，依据所述移动终端的摆放状态信息确定运动方向信息；调整信息确定单元，用于依据所述调整指令确定调整信息；视点矩阵计算单元，用于依据所述运动方向信息和调整信息计算视点矩阵。
- 根据权利要求12所述的装置，其特征在于，其中，所述调整指令包括：单指调整指令和/或双指调整指令，所述调整信息包括：旋转信息和/或缩放信息；所述调整信息确定单元，用于依据单指调整指令确定陀螺仪的旋转方向和旋转角度，将所述旋转方向和旋转角度作为旋转信息；和/或依据对触摸屏的双指调整指令确定缩放信息。
- 根据权利要求8所述的装置，其特征在于，还包括：播放模块，用于在移动终端中进行视频点播或视频直播时，通过播放调整的输出视频帧显示调整的全景视频。
- 一种计算机程序，包括计算机可读代码，当所述计算机可读代码在移动终端上运行时，导致所述移动终端执行根据权利要求1-7中的任一个所述的方法。
- 一种计算机可读介质，其中存储了如权利要求15所述的计算机程 序。
- 一种移动终端，其特征在于，包括：处理器和存储器；所述存储器用于存储程序代码；所述处理器用于执行所述存储器存储的程序代码，用于：将全景视频的全景图像帧与球形模型进行绑定，生成输出视频帧；接收调整指令，将所述调整指令转换为与所述球形模型对应的模型调整信息；根据所述模型调整信息对所述全景视频的输出视频帧进行调整，生成调整的输出视频帧。
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| US15/245,024 US20170150212A1 (en) | 2015-11-23 | 2016-08-23 | Method and electronic device for adjusting video |
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510818977.9A CN105979242A (zh) | 2015-11-23 | 2015-11-23 | 一种视频的播放方法和装置 |
| CN201510818977.9 | 2015-11-23 |
Related Child Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| US15/245,024 Continuation US20170150212A1 (en) | 2015-11-23 | 2016-08-23 | Method and electronic device for adjusting video |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| WO2017088491A1 true WO2017088491A1 (zh) | 2017-06-01 |
Family
ID=56988419
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| PCT/CN2016/089121 WO2017088491A1 (zh) | 2015-11-23 | 2016-07-07 | 一种视频的播放方法和装置 |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN105979242A (zh) |
| WO (1) | WO2017088491A1 (zh) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108734791A (zh) * | 2018-03-30 | 2018-11-02 | 北京奇艺世纪科技有限公司 | 全景视频的处理方法和装置 |
| CN113038262A (zh) * | 2021-01-08 | 2021-06-25 | 深圳市智胜科技信息有限公司 | 一种全景直播方法和装置 |
| CN113194326A (zh) * | 2021-04-28 | 2021-07-30 | 平安国际智慧城市科技股份有限公司 | 全景直播方法、装置、计算机设备及计算机可读存储介质 |
| CN114691068A (zh) * | 2022-03-23 | 2022-07-01 | 北京优酷科技有限公司 | 基于投屏技术的信息显示方法及装置 |
Families Citing this family (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105959675A (zh) * | 2016-05-25 | 2016-09-21 | 腾讯科技（深圳）有限公司 | 一种视频数据的处理方法和装置 |
| CN106527857A (zh) * | 2016-10-10 | 2017-03-22 | 成都斯斐德科技有限公司 | 一种基于虚拟现实的全景视频交互方法 |
| CN106454401A (zh) * | 2016-10-26 | 2017-02-22 | 乐视网信息技术（北京）股份有限公司 | 一种播放视频的方法及装置 |
| CN106534892A (zh) * | 2016-11-23 | 2017-03-22 | 上海沙塔信息科技有限公司 | 基于可视角度再编码的虚拟现实直播***和方法 |
| CN107659851B (zh) * | 2017-03-28 | 2019-09-17 | 腾讯科技（北京）有限公司 | 全景图像的展示控制方法及装置 |
| CN108319362B (zh) * | 2018-01-02 | 2020-06-23 | 联想(北京)有限公司 | 一种全景信息显示方法、电子设备和计算机存储介质 |
| CN110737336B (zh) * | 2019-10-12 | 2022-05-13 | Oppo广东移动通信有限公司 | 一种视频显示方向调整方法、装置及计算机可读存储介质 |
| CN111641871A (zh) * | 2020-05-29 | 2020-09-08 | 广州华多网络科技有限公司 | 直播视频的展示方法、装置、终端和可读存储介质 |
| CN112801758A (zh) * | 2021-03-17 | 2021-05-14 | 珠海必要工业科技股份有限公司 | 定制图像的方向调整方法、装置、存储介质及终端设备 |
| CN113411684B (zh) * | 2021-06-24 | 2023-05-30 | 广州酷狗计算机科技有限公司 | 视频播放方法和装置、存储介质及电子设备 |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20120262540A1 (en) * | 2011-04-18 | 2012-10-18 | Eyesee360, Inc. | Apparatus and Method for Panoramic Video Imaging with Mobile Computing Devices |
| CN103327217A (zh) * | 2012-03-23 | 2013-09-25 | 杭州普维光电技术有限公司 | 一种视频处理方法及装置 |
| US20140160233A1 (en) * | 2012-12-11 | 2014-06-12 | Sony Corporation | Information processing apparatus, information processing method, and program |
| CN104010225A (zh) * | 2014-06-20 | 2014-08-27 | 合一网络技术(北京)有限公司 | 显示全景视频的方法和*** |
| CN104219584A (zh) * | 2014-09-25 | 2014-12-17 | 广州市联文信息科技有限公司 | 基于增强现实的全景视频交互方法和*** |
| CN104735464A (zh) * | 2015-03-31 | 2015-06-24 | 华为技术有限公司 | 一种全景视频交互传输方法、服务器和客户端 |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR20100126812A (ko) * | 2008-03-17 | 2010-12-02 | 휴렛-팩커드 디벨롭먼트 컴퍼니, 엘.피. | 비디오 이미지 스트림을 디스플레이하는 방법 및 클라이언트 관리 시스템 |
| CN201523430U (zh) * | 2009-06-23 | 2010-07-07 | 长峰科技工业集团公司 | 一种全景视频监控*** |
| CN102945563B (zh) * | 2012-09-26 | 2017-05-24 | 天津游奕科技有限公司 | 一种全景视频的展示与交互***及方法 |
| CN104184961A (zh) * | 2013-05-22 | 2014-12-03 | 辉达公司 | 用于生成全景视频的移动设备和*** |
| CN103455356B (zh) * | 2013-09-05 | 2017-02-08 | 中国计量学院 | 多核移动设备上3d模型的并发加载及渲染方法 |
-
2015
- 2015-11-23 CN CN201510818977.9A patent/CN105979242A/zh active Pending
-
2016
- 2016-07-07 WO PCT/CN2016/089121 patent/WO2017088491A1/zh active Application Filing
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20120262540A1 (en) * | 2011-04-18 | 2012-10-18 | Eyesee360, Inc. | Apparatus and Method for Panoramic Video Imaging with Mobile Computing Devices |
| CN103327217A (zh) * | 2012-03-23 | 2013-09-25 | 杭州普维光电技术有限公司 | 一种视频处理方法及装置 |
| US20140160233A1 (en) * | 2012-12-11 | 2014-06-12 | Sony Corporation | Information processing apparatus, information processing method, and program |
| CN104010225A (zh) * | 2014-06-20 | 2014-08-27 | 合一网络技术(北京)有限公司 | 显示全景视频的方法和*** |
| CN104219584A (zh) * | 2014-09-25 | 2014-12-17 | 广州市联文信息科技有限公司 | 基于增强现实的全景视频交互方法和*** |
| CN104735464A (zh) * | 2015-03-31 | 2015-06-24 | 华为技术有限公司 | 一种全景视频交互传输方法、服务器和客户端 |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108734791A (zh) * | 2018-03-30 | 2018-11-02 | 北京奇艺世纪科技有限公司 | 全景视频的处理方法和装置 |
| CN113038262A (zh) * | 2021-01-08 | 2021-06-25 | 深圳市智胜科技信息有限公司 | 一种全景直播方法和装置 |
| CN113194326A (zh) * | 2021-04-28 | 2021-07-30 | 平安国际智慧城市科技股份有限公司 | 全景直播方法、装置、计算机设备及计算机可读存储介质 |
| CN114691068A (zh) * | 2022-03-23 | 2022-07-01 | 北京优酷科技有限公司 | 基于投屏技术的信息显示方法及装置 |
Also Published As
| Publication number | Publication date |
|---|---|
| CN105979242A (zh) | 2016-09-28 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| WO2017088491A1 (zh) | 一种视频的播放方法和装置 | |
| JP7017866B2 (ja) | 従来型描画機器及び没入型描画機器用の没入型ビデオのフォーマット化の方法、装置及びストリーム | |
| TWI674791B (zh) | 用於發信視埠與感興趣區域的方法與設備 | |
| US10659685B2 (en) | Control of viewing angles for 360-degree video playback | |
| US20190387348A1 (en) | Mixed-order ambisonics (moa) audio data for computer-mediated reality systems | |
| JP6673381B2 (ja) | 画像処理装置 | |
| US20200236283A1 (en) | Method, system and apparatus for stabilising frames of a captured video sequence | |
| JP6309749B2 (ja) | 画像データ再生装置および画像データ生成装置 | |
| US10235795B2 (en) | Methods of compressing a texture image and image data processing system and methods of generating a 360 degree panoramic video thereof | |
| WO2015196937A1 (zh) | 一种录制视频的方法和装置 | |
| US10701282B2 (en) | View interpolation for visual storytelling | |
| TW201730841A (zh) | 圖像資料處理系統和相關方法以及相關圖像融合方法 | |
| US20170150212A1 (en) | Method and electronic device for adjusting video | |
| US10121262B2 (en) | Method, system and apparatus for determining alignment data | |
| KR20190039118A (ko) | 파노라마 영상 압축 방법 및 장치 | |
| JP7177034B2 (ja) | レガシー及び没入型レンダリングデバイスのために没入型ビデオをフォーマットする方法、装置、及びストリーム | |
| EP3526639A1 (en) | Display of visual data with a virtual reality headset | |
| TW201621815A (zh) | 基於深度資訊的模型建構方法、圖形處理裝置與紀錄媒體 | |
| US10616551B2 (en) | Method and system for constructing view from multiple video streams | |
| JP2023519372A (ja) | 三次元ビデオの処理方法、装置、可読記憶媒体及び電子機器 | |
| CN112738361A (zh) | 一种视频直播虚拟演播厅的实现方法 | |
| CN114445600A (zh) | 一种特效道具的展示方法、装置、设备及存储介质 | |
| US10887653B2 (en) | Systems and methods for performing distributed playback of 360-degree video in a plurality of viewing windows | |
| CN111512640B (zh) | 多相机设备 | |
| CN109636917B (zh) | 三维模型的生成方法、装置、硬件装置 |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| 121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 16867718 Country of ref document: EP Kind code of ref document: A1 |
|
| NENP | Non-entry into the national phase |
Ref country code: DE |
|
| 122 | Ep: pct application non-entry in european phase |
Ref document number: 16867718 Country of ref document: EP Kind code of ref document: A1 |