WO2018120474A1 - 一种信息的处理方法及装置 - Google Patents
一种信息的处理方法及装置 Download PDFInfo
- Publication number
- WO2018120474A1 WO2018120474A1 PCT/CN2017/078585 CN2017078585W WO2018120474A1 WO 2018120474 A1 WO2018120474 A1 WO 2018120474A1 CN 2017078585 W CN2017078585 W CN 2017078585W WO 2018120474 A1 WO2018120474 A1 WO 2018120474A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- information
- spatial
- target
- space
- spatial information
- Prior art date
Links
- 230000010365 information processing Effects 0.000 title abstract description 11
- 238000003672 processing method Methods 0.000 title abstract description 7
- 238000000034 method Methods 0.000 claims description 88
- 238000012545 processing Methods 0.000 claims description 39
- 238000010586 diagram Methods 0.000 description 27
- 230000008569 process Effects 0.000 description 25
- 238000003860 storage Methods 0.000 description 21
- 230000005540 biological transmission Effects 0.000 description 18
- 230000011218 segmentation Effects 0.000 description 11
- 230000006870 function Effects 0.000 description 10
- 101100412093 Schizosaccharomyces pombe (strain 972 / ATCC 24843) rec16 gene Proteins 0.000 description 8
- 230000008859 change Effects 0.000 description 8
- 238000004891 communication Methods 0.000 description 8
- 101100238610 Mus musculus Msh3 gene Proteins 0.000 description 6
- 230000003044 adaptive effect Effects 0.000 description 6
- 238000005516 engineering process Methods 0.000 description 6
- 230000006978 adaptation Effects 0.000 description 5
- AWSBQWZZLBPUQH-UHFFFAOYSA-N mdat Chemical compound C1=C2CC(N)CCC2=CC2=C1OCO2 AWSBQWZZLBPUQH-UHFFFAOYSA-N 0.000 description 5
- 230000009471 action Effects 0.000 description 4
- 238000013507 mapping Methods 0.000 description 4
- 230000008447 perception Effects 0.000 description 4
- 238000004088 simulation Methods 0.000 description 4
- 210000003128 head Anatomy 0.000 description 3
- 230000003068 static effect Effects 0.000 description 3
- 230000000153 supplemental effect Effects 0.000 description 3
- 238000004458 analytical method Methods 0.000 description 2
- 230000006399 behavior Effects 0.000 description 2
- 238000004590 computer program Methods 0.000 description 2
- 230000001419 dependent effect Effects 0.000 description 2
- 238000005538 encapsulation Methods 0.000 description 2
- 210000000887 face Anatomy 0.000 description 2
- 230000002452 interceptive effect Effects 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 238000009877 rendering Methods 0.000 description 2
- 239000007787 solid Substances 0.000 description 2
- 230000000007 visual effect Effects 0.000 description 2
- 101100301559 Bacillus anthracis repS gene Proteins 0.000 description 1
- 101100247969 Clostridium saccharobutylicum regA gene Proteins 0.000 description 1
- 101100412434 Escherichia coli (strain K12) repB gene Proteins 0.000 description 1
- 101100184605 Escherichia coli mobA gene Proteins 0.000 description 1
- 101000708578 Milk vetch dwarf virus (isolate N) Para-Rep C3 Proteins 0.000 description 1
- 101100391699 Pseudomonas viridiflava gacA gene Proteins 0.000 description 1
- 101100242901 Quaranfil virus (isolate QrfV/Tick/Afghanistan/EG_T_377/1968) PB2 gene Proteins 0.000 description 1
- 101150082826 Segment-2 gene Proteins 0.000 description 1
- 101100114425 Streptococcus agalactiae copG gene Proteins 0.000 description 1
- 101100194052 Thogoto virus (isolate SiAr 126) Segment 2 gene Proteins 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 230000000295 complement effect Effects 0.000 description 1
- 238000005094 computer simulation Methods 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000004438 eyesight Effects 0.000 description 1
- 230000001815 facial effect Effects 0.000 description 1
- 230000004927 fusion Effects 0.000 description 1
- 238000005304 joining Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000008520 organization Effects 0.000 description 1
- 238000002360 preparation method Methods 0.000 description 1
- -1 rep2 Proteins 0.000 description 1
- 101150044854 repA gene Proteins 0.000 description 1
- 101150055347 repA2 gene Proteins 0.000 description 1
- 101150107738 repB gene Proteins 0.000 description 1
- 238000005096 rolling process Methods 0.000 description 1
- 230000016776 visual perception Effects 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/25—Management operations performed by the server for facilitating the content distribution or administrating data related to end-users or client devices, e.g. end-user or client device authentication, learning user preferences for recommending movies
- H04N21/262—Content or additional data distribution scheduling, e.g. sending additional data at off-peak times, updating software modules, calculating the carousel transmission frequency, delaying a video stream transmission, generating play-lists
- H04N21/26258—Content or additional data distribution scheduling, e.g. sending additional data at off-peak times, updating software modules, calculating the carousel transmission frequency, delaying a video stream transmission, generating play-lists for generating a list of items to be played back in a given order, e.g. playlist, or scheduling item distribution according to such list
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T19/00—Manipulating 3D models or images for computer graphics
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/234—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
- H04N21/2343—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements
- H04N21/23439—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements for generating different versions
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/41—Structure of client; Structure of client peripherals
- H04N21/414—Specialised client platforms, e.g. receiver in car or embedded in a mobile appliance
- H04N21/41407—Specialised client platforms, e.g. receiver in car or embedded in a mobile appliance embedded in a portable device, e.g. video client on a mobile phone, PDA, laptop
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/47—End-user applications
- H04N21/472—End-user interface for requesting content, additional data or services; End-user interface for interacting with content, e.g. for content reservation or setting reminders, for requesting event notification, for manipulating displayed content
- H04N21/4728—End-user interface for requesting content, additional data or services; End-user interface for interacting with content, e.g. for content reservation or setting reminders, for requesting event notification, for manipulating displayed content for selecting a Region Of Interest [ROI], e.g. for requesting a higher resolution version of a selected region
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
- H04N21/81—Monomedia components thereof
- H04N21/816—Monomedia components thereof involving special video data, e.g 3D video
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
- H04N21/83—Generation or processing of protective or descriptive data associated with content; Content structuring
- H04N21/845—Structuring of content, e.g. decomposing content into time segments
- H04N21/8456—Structuring of content, e.g. decomposing content into time segments by decomposing the content in the time domain, e.g. in time segments
Definitions
- the present invention relates to the field of streaming media processing, and in particular, to a method and an apparatus for processing information.
- the DASH technical specification is mainly composed of two major parts: the media presentation description (English: Media Presentation) Description, MPD) and media file format (English: file format).
- the media file format belongs to a file format.
- the server prepares multiple versions of the code stream for the same video content.
- Each version of the code stream is called a representation in the DASH standard (English: representation).
- Representation is a collection and encapsulation of one or more codestreams in a transport format, one representation containing one or more segments.
- Different versions of the code stream may have different coding parameters such as code rate and resolution, and each code stream is divided into a plurality of small files, and each small file is called segmentation (or segmentation, English: segment). It is possible to switch between different media representations during the client requesting media segmentation data.
- the segment may be packaged in accordance with the standard ISO/IEC 14496-12 (ISO BMFF (Base Media File Format)) or may be encapsulated in accordance with ISO/IEC 13818-1 (MPEG-2 TS).
- the media presentation description is called MPD
- the MPD can be an xml file.
- the information in the file is described in a hierarchical manner. As shown in FIG. 1, the information of the upper level is completely inherited by the next level. Some media metadata is described in this file, which allows the client to understand the media content information in the server and can use this information to construct the http-URL of the request segment.
- media presentation is a collection of structured data for presenting media content
- media presentation description English: media presentation description
- a standardized description of media presentation files for providing streaming media services Period English: period
- representation is the description information of one or more code streams in the transmission format Collection and encapsulation, one representation containing one or more segments
- adaptive set English: AdaptationSet
- Subset English: subset
- the segmentation information is the HTTP unified resource in the media presentation description.
- the media unit referenced by the locator, the segmentation information describes the segmentation of the media data, and the media data Segment may be stored in a file, can be stored separately, in one possible
- the related technical concept of the MPEG-DASH technology of the present invention can refer to the relevant provisions in ISO/IEC 23009-1 Information technology--Dynamic adaptive streaming over HTTP (DASH)--Part 1:Media presentation description and segment formats, refer to the relevant provisions in the historical standard version, such as ISO/IEC 23009-1:2013 or ISO/IEC 23009-1:2012.
- Virtual reality technology is a computer simulation system that can create and experience virtual worlds. It uses computer to generate a simulation environment. It is a multi-source information fusion interactive 3D dynamic vision and system simulation of entity behavior. The user is immersed in the environment.
- VR mainly includes simulation environment, perception, natural skills and sensing equipment.
- the simulation environment is a computer-generated, real-time, dynamic, three-dimensional, realistic image. Perception means that the ideal VR should have the perception that everyone has.
- there are also perceptions such as hearing, touch, force, and motion, and even smell and taste, also known as multi-perception.
- Natural skills refer to the rotation of the person's head, eyes, gestures, or other human behaviors.
- a sensing device is a three-dimensional interactive device.
- VR video or 360 degree video, or Omnidirectional video
- only the video image representation and associated audio presentation corresponding to the orientation portion of the user's head are presented.
- VR video is that the entire video content will be presented to the user; VR video is only a subset of the entire video is presented to the user (English: in VRtypically only a subset Of the entire video region represented by the videopictures).
- a Spatial Object is defined as a spatial part of a content component (ega region of interest, or a tile ) and represented by either an adaptation Set or a Sub-Representation.”
- Spatial information is the spatial relationship between spatial objects (Spatial Objects).
- a spatial object is defined as a part of a content component, such as an existing region of interest (ROI) and tiles; spatial relationships can be described in Adaptation Set and Sub-Representation.
- ROI region of interest
- tiles spatial relationships can be described in Adaptation Set and Sub-Representation.
- the spatial information of the spatial object can be described in the MPD.
- a file is composed of a number of boxes and a full box (FullBox).
- Each Box consists of a header (Header) and data (Data).
- Data is the actual data of the Box, which can be pure data or more sub-Boxes.
- the contact type (reference_type) used for the link between the track of the media content and the track of the metadata is 'cdsc', such as parsing the association in the track of the video.
- Track, and the associated type is 'cdsc', indicating that the associated track is the metadata track used to describe the track of the video.
- the client needs to parse all the tracks included in the file, and then according to the type of contact between the track of the media content and the track of the metadata.
- VR video viewing applications such as 360-degree viewing angles are increasingly presented to users.
- the user may change the view angle (English: field of view, FOV) at any time, and each view corresponds to a video object of a spatial object (which can be understood as an area in the VR video), and is presented to the user when the view is switched.
- the VR video image within the perspective should also be switched.
- the video data of the spatial object that can cover the human eye's perspective is presented.
- the spatial object viewed by the user may be the region of interest that most users choose to view, or may be the region specified by the video creator.
- the area will change over time.
- spatial information describing the location of the spatial object in the VR video needs to be encapsulated in a corresponding file. Since the image data in the video data corresponds to a large number of images, a large amount of spatial information of a large number of images causes an excessive amount of data.
- the embodiment of the invention provides a method and a device for processing information of a streaming media, which can save the data amount of the spatial information.
- a method for processing information of a streaming media comprising:
- target space information of the target space object where the target space object is a spatial object of two spatial objects, where the two spatial objects are associated with data of two images included in the target video data, and the target spatial information includes the same attribute space Information, the same attribute space information includes the same information between the spatial information of the two spatial objects, and the spatial information of the other spatial objects other than the target spatial object includes the same attribute spatial information;
- video data to be played is determined.
- the target video data may be a code stream, that is, a target video code stream, or may be original data that has not been encoded or reconstructed reconstructed data, that is, a target video sequence.
- the target video data is the target video code stream
- the data of the two images may be the encoded data of the two images.
- the target video data is the target video sequence
- the data of the two images is the original data or the reconstructed data of the two images. .
- the two images and the two spatial objects may have a one-to-one correspondence.
- the same attribute space information is a group of information, and the spatial information of each of the two space objects shares the group information.
- the acquiring target space information of the target space object may include receiving target space information of the target space object from the server.
- the two images may be two frames in the video sequence, that is, the two images correspond to different moments, and the two images may also be sub-images of the same frame in the video sequence, that is, the two images correspond to the same image. At the moment, the two images can also be sub-images of different frames in the video sequence.
- the redundancy of the spatial information is reduced, thereby reducing the amount of data of the spatial information.
- the determining, according to the target space information, the video data to be played may include: determining, according to the target spatial information, whether the target spatial object includes all or part of a spatial object corresponding to the screen to be played, where the target spatial object includes When all or part of the spatial object corresponding to the picture to be played is used, the target video data is the video data to be played.
- the determining, according to the target space information, the video data to be played may further include: according to the target space information, and a spatial relationship of the spatial object corresponding to the screen to be played relative to the target spatial object (or, the perspective switching) Track), the spatial information of the spatial object corresponding to the picture to be played (or the spatial object after the view switching) is determined, thereby further determining the video data to be played.
- the video data to be played may be a video code stream to be played, and after acquiring the target space information of the target space object, the relative position of the target space object in the panoramic space (or called a panoramic space object) may be determined, and then During the video playing process, the location of the spatial object after the perspective switching is determined according to the target spatial information of the target spatial object and the trajectory of the viewing angle switching. And further requesting, from the server, the video code stream to be played corresponding to the space object corresponding to the picture to be played.
- the request for acquiring the video code stream to be played may be sent to the server according to the URL of the code stream of each spatial object described in the MPD, thereby obtaining the video code stream to be played, decoding and playing the to-be-played video stream.
- Video stream may be sent to the server according to the URL of the code stream of each spatial object described in the MPD, thereby obtaining the video code stream to be played, decoding and playing the to-be-played video stream.
- the target spatial information further includes hetero-information space information of the target spatial object
- the spatial information of the other spatial object further includes the different spatial objects. Attribute space information, the different attribute space information of the target space object is different from the different attribute information of the other space object.
- the different attribute space information of the target space object and the different attribute information of the other space object may refer to different values of the two different attribute space information.
- the target space information includes location information of a center point of the target space object or an upper left of the target space object The location information of the point, the target space information further including the width of the target space object and the height of the target space object.
- the target space information may also include location information of other location points (lower left point, upper right point, lower right point, or preset point) in the target space object instead of the location information of the center point of the target space object or The position information of the upper left point of the target space object.
- the upper left point is a point at which the value of the abscissa and the value of the ordinate are the smallest in the target space object.
- the position information of the center point or the position information of the upper left point may be a pitch angle ⁇ (pitch) and a yaw angle y (yaw), or may be a pitch angle. ⁇ (pitch), yaw angle yaw, and roll angle ⁇ (roll).
- the coordinate system corresponding to the target spatial information is a pixel coordinate system
- the position information of the center point or the position information of the upper left point may be pixels.
- the absc issa of the unit and the ordinate in pixels.
- the target spatial information includes location information of an upper left point of the target spatial object, and the target spatial object The location information of the lower right point.
- the target space information may also include location information of an upper right point of the target spatial object, and location information of a lower left point of the target spatial object.
- the target spatial information includes spatial rotation information of the target spatial object.
- the spatial rotation information of the target space object may be used to indicate a degree of rotation of the target space object relative to a horizontal coordinate axis or a longitudinal coordinate axis of the panoramic space object, where the target space object is located in the panoramic space object.
- the spatial rotation information when the coordinate system corresponding to the target spatial information is an angular coordinate system, the spatial rotation information may be a roll angle ⁇ (roll), and when the coordinate system corresponding to the target spatial information is a pixel coordinate system, the spatial rotation information may be used.
- the motion vector of the position point in the target space object obtained by the roll angle conversion is represented by the motion vector in units of pixels.
- the target spatial information is encapsulated in a space
- the spatial information data is a code stream of the target video data, metadata of the target video data, or a file independent of the target video data, the spatial information track being independent of the target The trajectory of the video data.
- the file independent of the target video data may be a spatial information file for describing spatial information.
- the trajectory independent of the target video data may be a spatial information trajectory for describing spatial information.
- the target spatial information when the target spatial information is encapsulated in the code stream of the target video data, the target spatial information may be encapsulated in an auxiliary enhanced information unit or a parameter set unit in the code stream of the target video data, and the target spatial information may also be encapsulated in The segment of the representation in which the target video data is located. Specifically, it can be encapsulated in a specified box in the segment (for example, trun box or tfhd box).
- the same attribute space information and the different attribute space information of the target space object may be encapsulated in the same box, or may be encapsulated in different boxes. Specifically, the same attribute space information may be encapsulated in a 3dsc box, and the different attribute space information of the target space object may be encapsulated in the mdat box.
- the spatial information data or the spatial information track further includes a spatial information type for indicating a type of the same attribute spatial information.
- the identifier, the spatial information type identifier is used to indicate information belonging to the same attribute spatial information in the target spatial information.
- the spatial information type identifier can also be used to indicate information of the heterogeneous spatial information belonging to the target spatial object in the target spatial information.
- the spatial information type identifier may also be used to indicate the same attribute space information or the spatial information type of the different attribute space information of the target space object.
- the optional spatial information type may include, but is not limited to, including location information of the spatial object but not The spatial information including the width and height information of the spatial object, the spatial information including the width and height information of the spatial object but not the positional information of the spatial object, the spatial information including the width and height information of the spatial object, and the positional information of the spatial object.
- the spatial information type identifier may also be used to indicate the spatial object type of the two spatial objects.
- the optional spatial object types may include, but are not limited to, a spatial object with a constant position and width, a position change, and a width and height constant. Spatial objects, spatial objects with constant position and width and height, and spatial objects with varying positions and widths.
- the spatial information type identifier when the spatial information type identifier is the first preset value, the spatial information type identifier is used to indicate that the information belonging to the same attribute spatial information in the target spatial information is the target space. Position information of a center point of the object or position information of an upper left point of the target space object, and the target space pair The width of the image and the height of the target space object.
- the spatial information type identifier is used to indicate that the information belonging to the same attribute spatial information in the target spatial information is the width of the target spatial object and the height of the target spatial object.
- the spatial information type identifier is the third preset value, the spatial information type identifier is used to indicate that the target spatial information does not belong to the same attribute spatial information.
- the spatial information type identifier when the spatial information type identifier is the first preset value, the spatial information type identifier further indicates that the hetero-attribute space information does not exist.
- the spatial information type identifier when the spatial information type identifier is the second preset value, the spatial information type identifier further indicates that the heterogeneous spatial information of the target spatial object is the location information of the central point of the target spatial object or the location information of the upper left point of the target spatial object.
- the spatial information type identifier when the spatial information type identifier is the third preset value, the spatial information type identifier further indicates that the heterogeneous spatial information of the target spatial object is the location information of the central point of the target spatial object or the location information of the upper left point of the target spatial object. And the width of the target space object and the height of the target space object.
- the spatial information type identifier when the spatial information type identifier is the fourth preset value, the spatial information type identifier is used to indicate that the information belonging to the same attribute spatial information in the target spatial information is the target space.
- the spatial information type identifier is used to indicate that the information belonging to the same attribute spatial information in the target spatial information is the location information of the lower right point of the target spatial object.
- the spatial information type identifier is the sixth preset value, the spatial information type identifier is used to indicate that the target spatial information does not belong to the same attribute spatial information. It should be noted that the position information of the upper left point of the target space object or the position information of the lower right point of the target space object may be replaced by the width of the target space object and the height of the target space object.
- the spatial information type identifier when the spatial information type identifier is the fourth preset value, the spatial information type identifier further indicates that the hetero-attribute space information does not exist.
- the spatial information type identifier when the spatial information type identifier is the fifth preset value, the spatial information type identifier further indicates that the heterogeneous spatial information of the target spatial object is the location information of the central point of the target spatial object or the location information of the upper left point of the target spatial object.
- the spatial information type identifier when the spatial information type identifier is the sixth preset value, the spatial information type identifier further indicates that the heterogeneous spatial information of the target spatial object is the location information of the upper left point of the target spatial object, and the lower right point of the target spatial object location information. It should be noted that the position information of the upper left point of the target space object or the position information of the lower right point of the target space object may be replaced by the width of the target space object and the height of the target space object.
- the spatial information type identifier indicates that the target spatial information does not belong to the same attribute spatial information
- the same attribute The spatial information includes a wide minimum of the target spatial object, a high minimum of the target spatial object, a wide maximum of the target spatial object, and a high maximum of the target spatial object.
- the spatial information type identifier is encapsulated in the same box as the same attribute spatial information.
- the spatial information data or the spatial information trajectory is further A coordinate system identifier for indicating a coordinate system corresponding to the target spatial information, the coordinate system being a pixel coordinate system or an angular coordinate system.
- position information is represented by coordinates of pixels, and width and height are also expressed in units of pixels.
- the position information is represented by an angle, for example, a pitch angle ⁇ (pitch) and a yaw angle y (yaw), or a pitch angle ⁇ (pitch), yaw
- the angle yaw and the roll angle ⁇ (roll) are angles to represent the angle range.
- the coordinate system identifier is encapsulated in the same box as the same attribute space information.
- the implementation manner of the tenth implementation manner of the first aspect in the eleventh possible implementation manner of the first aspect, is further The space rotation information identifier is used to indicate whether the target space information includes spatial rotation information of the target space object.
- the space rotation information identifier may be encapsulated in the same box as the same attribute space information, and the space rotation information identifier may also be encapsulated in the same box as the different attribute space information of the target space object. Specifically, when the spatial rotation information identifier and the heterogeneous spatial information of the target spatial object are encapsulated in the same box, when the spatial rotation information identifier indicates that the target spatial information includes spatial rotation information of the target spatial object, The hetero-attribute space information of the target space object includes spatial rotation information.
- the spatial rotation information identifier indicates that the target spatial information includes spatial rotation information of the target spatial object, it indicates that the target spatial object has a rotation.
- the spatial rotation information identifier indicates that the target spatial information does not include the spatial rotation information of the target spatial object, it indicates that the target spatial object has no rotation.
- a second aspect provides a method for processing information of a streaming media, where the method includes:
- the target space information includes the same attribute space information
- the same attribute space information includes The same information between the spatial information of the two spatial objects, and the spatial information of the other spatial objects except the target spatial object includes the same attribute spatial information.
- the method may further include: sending the target space information to the client.
- the target space information may further include the hetero-attribute space information of the target spatial object, where the spatial information of the other spatial object further includes the other spatial object.
- the different attribute space information, the different attribute space information of the target space object is different from the different attribute information of the other spatial object.
- the target spatial information may include location information of a central point of the target spatial object or the target spatial object
- the target space information may further include a width of the target space object and a height of the target space object.
- the spatial information of the two spatial objects may include location information of respective center points of the two spatial objects or position information of respective upper left points of the two spatial objects, and spatial information of each of the two spatial objects is further The respective widths of the two spatial objects and the respective heights of the two spatial objects may be included.
- the target space information may include location information of an upper left point of the target spatial object, and the target spatial object The location information of the lower right point.
- the spatial information of each of the two spatial objects may include location information of respective upper left points of the two spatial objects and position information of respective lower right points of the two spatial objects.
- the target space information may include the Target empty Spatial rotation information between objects.
- the spatial information of each of the two spatial objects may include spatial rotation information of the two spatial objects.
- the target spatial information may be encapsulated in In the spatial information data or the spatial information track, the spatial information data may be a code stream of the target video data, metadata of the target video data, or a file independent of the target video data, and the spatial information track may be independent.
- the trajectory of the target video data may be independent.
- the spatial information data or the spatial information track may further include spatial information indicating a type of the same attribute spatial information.
- a type identifier, the space information type identifier is used to indicate information in the target space information that belongs to the same attribute space information.
- the spatial information type identifier indicates that the target spatial information does not belong to the same attribute space information
- the same attribute space The information may include a wide minimum of the target spatial object, a high minimum of the target spatial object, a wide maximum of the target spatial object, and a high maximum of the target spatial object.
- the spatial information type identifier may be encapsulated in the same box as the same attribute space information. in.
- the spatial information data or the spatial information trajectory is further A coordinate system identifier for indicating a coordinate system corresponding to the target spatial information may be included, and the coordinate system is a pixel coordinate system or an angular coordinate system.
- the coordinate system identifier may be encapsulated in the same box as the same attribute space information.
- the implementation manner of the tenth implementation manner of the second aspect in the eleventh possible implementation manner of the second aspect, is further A spatial rotation information identifier may be included, the spatial rotation information identifying a spatial rotation information indicating whether the target spatial information includes the target spatial object.
- a device for processing information of a streaming media comprising:
- An acquiring module configured to acquire target space information of a target space object, where the target space object is a spatial object of two spatial objects, where the two spatial objects are associated with data of two images included in the target video data, the target space
- the information includes the same attribute space information, and the same attribute space information includes the same information between the spatial information of the two spatial objects, and the spatial information of the other spatial objects other than the target spatial object includes the same Attribute space information;
- a determining module configured to determine video data to be played according to the target space information acquired by the acquiring module.
- the obtaining module may be configured to receive the target space information from the server.
- the target space information further includes the hetero-attribute space information of the target spatial object
- the spatial information of the other spatial object further includes the different spatial objects. Attribute space information, the different attribute space information of the target space object is different from the different attribute information of the other space object.
- the target space information includes location information of a center point of the target space object or location information of an upper left point of the target space object, where the target space information further includes the target space The width of the object and the height of the target space object.
- the target space information includes location information of an upper left point of the target spatial object, and the target spatial object The location information of the lower right point.
- the target spatial information includes the target Spatial rotation information of a spatial object.
- the target spatial information is encapsulated in the space
- the spatial information data is a code stream of the target video data, metadata of the target video data, or a file independent of the target video data, the spatial information track being independent of the target The trajectory of the video data.
- the spatial information data or the spatial information track further includes a spatial information type indicating a type of the same attribute spatial information.
- the identifier, the spatial information type identifier is used to indicate information belonging to the same attribute spatial information in the target spatial information.
- the spatial information type identifier indicates that the target spatial information does not belong to the same attribute space information
- the same attribute space The information includes a wide minimum of the target spatial object, a high minimum of the target spatial object, a wide maximum of the target spatial object, and a high maximum of the target spatial object.
- the spatial information type identifier is encapsulated in the same box as the same attribute space information.
- the spatial information data or the spatial information trajectory is further A coordinate system identifier for indicating a coordinate system corresponding to the target spatial information, the coordinate system being a pixel coordinate system or an angular coordinate system.
- the coordinate system identifier is encapsulated in the same box as the same attribute space information.
- the implementation manner of the tenth implementation manner of the third aspect in the eleventh possible implementation manner of the third aspect, is further The space rotation information identifier is used to indicate whether the target space information includes spatial rotation information of the target space object.
- a fourth aspect provides a processing device for streaming media information, the device comprising:
- An acquiring module configured to acquire spatial information of each of two spatial objects associated with data of two images in the target video data
- a determining module configured to determine target space information of the target spatial object according to the spatial information of the two spatial objects acquired by the acquiring module, where the target spatial object is a spatial object in the two spatial objects, and the target spatial information includes The same attribute space information, the same attribute space information includes the same information between the spatial information of the two spatial objects, and the spatial information of the other spatial objects except the target spatial object includes The same attribute space information.
- the device may further include a sending module, configured to send, to the client, the target space information determined by the determining module.
- the target space information may further include the hetero-attribute space information of the target spatial object, where the spatial information of the other spatial object further includes the other spatial object.
- the different attribute space information, the different attribute space information of the target space object is different from the different attribute information of the other spatial object.
- the target space information may include location information of a center point of the target spatial object or the target spatial object
- the target space information may further include a width of the target space object and a height of the target space object.
- the spatial information of the two spatial objects may include location information of respective center points of the two spatial objects or position information of respective upper left points of the two spatial objects, and spatial information of each of the two spatial objects is further The respective widths of the two spatial objects and the respective heights of the two spatial objects may be included.
- the target space information may include location information of an upper left point of the target spatial object, and the target spatial object The location information of the lower right point.
- the spatial information of each of the two spatial objects may include location information of respective upper left points of the two spatial objects and position information of respective lower right points of the two spatial objects.
- the target space information may include the The spatial rotation information of the target space object.
- the spatial information of each of the two spatial objects may include spatial rotation information of the two spatial objects.
- the target spatial information may be encapsulated in In the spatial information data or the spatial information track, the spatial information data may be a code stream of the target video data, metadata of the target video data, or a file independent of the target video data, and the spatial information track may be independent. The trajectory of the target video data.
- the spatial information data or the spatial information trajectory may further include spatial information indicating a type of the same attribute spatial information.
- a type identifier, the space information type identifier is used to indicate information in the target space information that belongs to the same attribute space information.
- the spatial information type identifier indicates that the target spatial information does not belong to the same attribute spatial information
- the same attribute space The information may include a wide minimum of the target spatial object, a high minimum of the target spatial object, a wide maximum of the target spatial object, and a high maximum of the target spatial object.
- the spatial information type identifier may be encapsulated in the same box as the same attribute spatial information. in.
- the spatial information data or the spatial information trajectory may further include a coordinate system identifier for indicating a coordinate system corresponding to the target spatial information, where the coordinate system is a pixel coordinate system or Angle coordinate system.
- the coordinate system identifier may be encapsulated in the same box as the same attribute space information.
- the implementation manner of the tenth implementation manner of the fourth aspect in the eleventh possible implementation manner of the fourth aspect, is further A spatial rotation information identifier may be included, the spatial rotation information identifying a spatial rotation information indicating whether the target spatial information includes the target spatial object.
- a fifth aspect provides a processing device for streaming media information, the device comprising a processor and a memory; the memory for storing a code; the processor reading the code stored in the memory for executing the first Aspects of the methods provided.
- a computer storage medium for storing computer software instructions for execution by a processor of the fifth aspect for performing the method provided by the first aspect.
- a processing apparatus for streaming media information comprising a processor and a memory; the memory for storing a code; the processor reading the code stored in the memory for executing the second Aspects of the methods provided.
- a computer storage medium for storing computer software instructions for execution by a processor of the seventh aspect for performing the method provided by the second aspect.
- a description manner of a reference type between media data and metadata is disclosed.
- different contact types are defined for different usage methods of metadata, which can facilitate the client to process according to the contact type.
- the type of association between the media data and the metadata is stored in a track of the media data or a track of the metadata; in a possible implementation manner, The type of connection between media data and metadata is transmitted in the form of a box.
- the definitions of Track and box can refer to the existing MPEG-DASH standard and the relevant provisions of ISO/IEC 14496-12, and will not be described here.
- information about the type of contact can be stored in the "tref" box.
- the media data is video data
- the type of association between the media data and the metadata is stored in the video track.
- the tref box is stored in the metadata describing the video track.
- the track that includes the tref box is the referenced video track, and is associated with the metadata track by the type of association between the media data and the metadata in the tref box.
- the associated metadata track can be determined by way of track id.
- the contact type may be used to describe one or more of the following information:
- a region of interest (ROI) in the media data a spatial region covered by the media data, quality information of the ROI associated in the media data, and quality information associated with the spatial region covered by the media data.
- ROI region of interest
- the contact type can be used to describe one or more of the following information:
- the media data corresponds to the spatial position information of the ROI in the spatial object on the spherical surface, or the 2D plane, or the mapped image;
- the area covered by the media data is on a spherical surface, or a 2D plane, or spatial position information in the mapped image;
- the above ROI information is included in the metadata track of the time attribute of the ROI, and the quality information is included in the metadata track of the quality time attribute.
- the 'tref' box of the track of the media data includes a type of association indicating the media data and the metadata.
- This contact type can be used to describe one or more of the following information:
- the media data corresponds to 2D spatial position information of a region of interest (ROI) in the spatial object, or spatial position information of the ROI in the spherical surface corresponding to the spatial data in the media object, or the ROI in the corresponding spatial object of the media data is Spatial position information in the mapped image, or 2D spatial position information of the spatial object corresponding to the spatial object, or spatial position information of the spatial object corresponding to the spatial object in the spherical surface, or space in the mapped image corresponding to the spatial object of the media data.
- the location information, or the quality information of the 2D spatial position of the ROI in the media data corresponding to the spatial object, or the quality information of the spatial position of the ROI in the spherical space corresponding to the media data, or the ROI in the corresponding spatial object of the media data is mapped
- the value of the reference type is rois, indicating that the associated track includes the region information of the ROI (this track contains the region information of the ROI on the sphere).
- the information is a spatial area that describes an image corresponding to a sample in the associated video track.
- the client can obtain the ROI area information by parsing the samples in the metadata track of the time attribute of the ROI, and use the ROI information to present the ROI of the image corresponding to the samp le in the associated video track (The client can use the sample in this Track to render the ROI on the sphere).
- the value of the reference type is roiq, indicating that the associated track includes the quality information of the ROI of the image corresponding to the sample in the associated video track on the spherical surface (this track contains the quality information of the ROI on the sphere for the referenced video track).
- the client can use the sample i n this track to know the quality of the ROI object on the sphere by parsing the sample in the metadata track with the time attribute of the quality. .
- the value of the reference type is conc, which indicates that the associated track includes the coverage information of the image corresponding to the sample in the video track on the spherical surface (this track provides information on the area on the spherical surface for The referenced video track).
- the associated metadata track may be a track track (a track maybe a recommended viewport timed metadata track), and the associated video track may be a director's perspective video track (the referenced video track maybe a Director's cut video track).
- the contact type describing the metadata associated with the media data is encapsulated.
- the client parses the data of the media track, through the information described by the contact type of the media data and the metadata, the client can understand what processing can be performed on the video data, and the method decouples the analysis between the tracks and reduces the client. Implement the complexity in the process.
- the client can request corresponding track data according to different processing needs.
- the ROI in the embodiment of the present invention may be a viewing angle, or may be a recommended viewing angle, such as an author's perspective.
- the viewing angle or the recommended viewing angle may also refer to a coverage area, where the coverage area refers to a spatial range of spatial objects corresponding to the media data.
- the track of the media data includes a contact type that describes metadata associated with the media data, and the contact type may describe a 2D spatial location of the ROI in the corresponding spatial object of the media data. Or the spatial position of the ROI in the spatial object corresponding to the media data, or the spatial position of the ROI in the spatial image corresponding to the media data in the mapped image.
- the track of the media data includes a ‘tref’ box, where
- the reference_type in 'tref'box takes the value ri2d, which indicates that the media data is associated with a timed metadata track of an ROI.
- the sample in the track of the ROI is the 2D spatial location information of the ROI in the spatial object corresponding to the media data.
- the 2D position information may be a position defined in the existing standard ISO/IEC 23001-10;
- the reference_type value is rois
- the sample in the ROI track is the spatial position information of the ROI in the spherical space corresponding to the media data
- the spatial position information in the spherical surface may be the existing standard ISO/IEC a sample in the timed metadata track on the sphere defined in 23000-20;
- the reference_type value is ri2p, where the value indicates that the media data is associated with a timed metadata track of an ROI, and the sample (sample) in the track of the ROI is the spatial position information of the ROI in the mapped image in the spatial object corresponding to the media data.
- the client can handle the following:
- the client parses the track of the media data.
- the track id of the track of the media data is 1 (can be any non-zero integer), and the 'tref' box in the track is obtained; the reference_type value from the 'tref' box is 'ri2d'.
- the associated track of 'rois' or 'ri2p', the track id of the associated track is 2 (can be any non-zero integer);
- the client determines, according to ‘ri2d’, that the track whose track id is 2 describes the 2D spatial position information of the ROI in the corresponding spatial object of the media data.
- the track whose track id is 2 describes the spatial position information of the ROI in the spherical surface corresponding to the spatial data of the media data.
- the track whose track id is 2 describes the spatial position information of the ROI in the mapped image in the corresponding spatial object of the media data
- the client can provide an ROI option on the user interface, and the user selects whether to view the content of the ROI area. If the user selects to view the content of the ROI area, the client presents the ROI area content; or the client directly presents the ROI content;
- the type of the metadata referenced by the track is clearly described, so that the resolution between the client's tracks is decoupled, and the complexity in the client implementation process is reduced.
- the coverage information describes an area that is intercepted by the media content of the current track in the entire source video content, and the area may be an area in the VR sphere. It may also be an area in a 2D image, or an area that is spherically mapped to a 2D image; as shown in FIG. 17, the spatial position information of the small image on the right side in FIG. 17(a) is a large image on the left side.
- Coverage information in the spherical image (17(a)) the area where the gray area is intercepted on the sphere is the coverage area of the gray area, and the coverage area
- the association type in the volume track adds a description in the 'tref' box, and the description information indicates that the metadata is 2D spatial position information of the spatial object corresponding to the media data, or the spatial position information of the spatial object corresponding to the spatial object in the spherical surface, or It is spatial position information of the media data corresponding to the spatial object in the mapped image.
- the coverage information is described by using a box.
- a specific example is as follows:
- the box provides information for the area on the spherical surface represented by the projected frame associated with the container ProjctedOmnidirectionalVideoBox. (This box provides information on the area on the spherical surface that is represented by the projected frame associated with the container Projcted Omnidirectional VideoBox). If there is no box indicating the coverage information in the data, the projection of the frame is a representation of the full frame.
- the spherical area represented by the projected frame is the area specified by the two yaw circles and the two pitch circles, as shown in FIG. 10 (When the projection format is the equirectangular projection, the spherical The region represented by the projected frame is the region specified by two yaw circles and two pitch circles, as illustrated fig 10).
- the coverage information is described as follows:
- the hor_range element and the ver_range element specify a horizontal and vertical range of the image corresponding to the sample in the video track on the spherical surface, and the unit may be 0.01 degrees.
- hor_range and ver_range specify the extent of the center point of the region.
- the range of hor_range should be between 1 and 36000.
- the range of ver_range should be between 1 and 36000.
- Center_pitch+ver_range ⁇ 2 is no greater than 18000.
- center_pitch-ver_range ⁇ 2 is not less than -18000 (hor_range and ver_rangespecify the range through the center point of the region.
- hor_range shall be in the range of 1to 36000, inclusive.ver_range shall be in the range Of 1to 36000,inclus ive.center_pitch+ver_range ⁇ 2shall not be greater than 18000.center_pitch-ver_range ⁇ 2shall not be less than-18000).
- the value of dynamic_range_flag is equal to 0, indicating that the horizontal and vertical ranges corresponding to all samples involved in the sample entry are not changed (dynamic_range_flag equal to 0specifies that the horizontal and vertical ranges of the region In all samples reference to this sample entry).
- dynamic_range_flag 0specifies that the horizontal and vertical ranges of the region In all samples reference to this sample entry.
- the value of dynamic_range_flag equal to 1 indicates that the horizontal and vertical ranges corresponding to the sample are described in the sample format (dynamic_range_flag equal to 1specifies that the horizonal and vertical ranges of the region is indicated in the sample) Format).
- the coverage area of the image corresponding to the sample in the video track on the spherical surface is described by the overlay information track.
- the coverage timed metadata track indicates the coverage area of the video content on the sphere.
- the entry type of the sample in the coverage timed metadata track is 'covg'.
- the RegionOnSphereSample element can be used to describe the sample syntax in the coverage timed metadata track.
- Specific examples of the RegionOnSphereSample element can be referred to the relevant provisions in the existing standards, for example, with reference to the relevant examples of ISO/IEC 23000-20.
- the value of shape_type in the RegionOnSphereConfigBox in the sample entry is 0.
- the static_hor_range element and the static_ver_range element, or the hor_range element and the ver_range element are used to indicate the coverage of the corresponding horizontal view and the coverage of the vertical view, respectively, and the center_yaw element and the center_pitch element are used to indicate the center point of the coverage.
- the reference_type in the 'tref' box takes the value cv2d, the semantic representation that the media data is associated with a timed metadata track of a coverage area, and the sample in the track of the coverage area is the corresponding spatial object in the media data.
- 2D spatial location information may be location information defined in an existing standard ISO/IEC 23001-10;
- the reference_type value is cvsp
- the semantic representation is that the media data is associated with a timed metadata track of the coverage area, and the sample in the track of the coverage area is spatial position information of the spatial object corresponding to the spatial object in the spherical surface; the spherical surface
- the information in the information may be the sample in the timed metadata track on the sphere defined in the existing standard ISO/IEC 23000-20;
- the reference_type value is cv2p.
- the semantic representation indicates that the media data is associated with a timed metadata track of a coverage area.
- the sample in the track of the coverage area is spatial location information of the spatial object corresponding to the spatial object in the mapped image.
- the client can handle the following:
- the client parses the track of the media data.
- the track id of the track of the media data is 1 (can be any non-zero integer)
- the 'tref' box of the media track is obtained, and the reference_type value is 'cv2d' from the 'tref' box.
- the associated track of 'cvsp' or 'cv2p' the track id of the track is 2 (can be any non-zero integer);
- the client determines that the track whose track id is 2 describes the 2D spatial position information of the spatial object corresponding to the media data, or judges that the track id is 2 according to 'cvsp', and the media data is described.
- the client determines that the track whose track id is 2 describes the spatial position information of the spatial object corresponding to the spatial object in the mapped image;
- the client can judge whether the content of the media is presented or intercepted part of the content according to the coverage information and the rendering capability of the device, or when the user's perspective changes, the user acquires data other than the perspective.
- the track whose track id is 2 is a spatial information description track of the coverage area
- the sample entry type of the track indicates that the current timed metadata track is a spatial information description track of the coverage area
- the sample entry type may take a value of ' Cvvp'(coverage viewport).
- the coverage information of the media may be described by an independent track, such as a timed metadata track description with a sample entry type value of 'cvvp', but if the coverage information of the media is a static case (every frame in the video) The coverage information is the same), then the media does not need to be associated with the coverage area.
- the specific description information is in 'covi' in ISO/IEC 23000-20 (covering information box) ), the shape of the coverage area on the spherical or 2D plane is described in the box.
- the value of the contact type describes a description manner of the association relationship between the metadata track of the quality information and the media data track.
- the type of the metadata referenced by the track is clearly described, so that the resolution between the tracks of the client is decoupled, and the complexity in the implementation process of the client is reduced.
- FIG. 1 is a schematic structural diagram of an MPD transmitted by a DASH standard used for system layer video streaming media transmission;
- FIG. 2 is a schematic diagram of an example of a framework for DASH standard transmission used in system layer video streaming media transmission
- FIG. 3 is a schematic diagram of switching of a code stream segment according to an embodiment of the present invention.
- FIG. 4 is a schematic diagram of a segmentation storage manner in code stream data
- 5 is another schematic diagram of a segmentation storage manner in code stream data
- FIG. 6 is a schematic diagram of a perspective corresponding to a change in viewing angle
- Figure 7 is another schematic diagram of the spatial relationship of spatial objects
- FIG. 8 is a schematic flowchart diagram of a method for processing information of a streaming media according to an embodiment of the present disclosure
- FIG. 9 is a schematic diagram of relative positions of target space objects in a panoramic space
- FIG. 10 is a schematic diagram of a coordinate system according to an embodiment of the present invention.
- FIG. 11 is a schematic diagram of another coordinate system according to an embodiment of the present invention.
- Figure 12 is a schematic illustration of another coordinate system in accordance with an embodiment of the present invention.
- FIG. 13 is a schematic flowchart diagram of a method for processing information of a streaming media according to an embodiment of the present disclosure
- FIG. 14 is a schematic diagram showing the logical structure of a device for processing information of streaming media according to an embodiment of the present invention.
- FIG. 15 is a schematic diagram showing the logical structure of a device for processing information of streaming media according to an embodiment of the present invention.
- FIG. 16 is a schematic structural diagram of hardware of a computer device according to an embodiment of the present invention.
- Figure 17 is a schematic illustration of a coverage area in accordance with an embodiment of the present invention.
- the current client-side system layer video streaming media transmission scheme can adopt the DASH standard framework, as shown in FIG. 2, and FIG. 2 is a schematic diagram of a framework example of DASH standard transmission used in system layer video streaming media transmission.
- the data transmission process of the system layer video streaming media transmission scheme includes two processes: a server side (such as an HTTP server, a media content preparation server, hereinafter referred to as a server) generates media data for video content, responds to a client request process; and a client ( The process of requesting and obtaining media data from a server, such as an HTTP streaming client.
- the media data includes a media presentation description (MPD) and a media stream (eg, a video stream to be played).
- MPD media presentation description
- media stream eg, a video stream to be played.
- the MPD on the server includes a plurality of representations (also called presentations, English: representation), each representation describing a plurality of segments.
- the client's HTTP streaming request control module obtains the service.
- the MPD sent by the server analyzes the MPD, determines the information of each segment of the video code stream described in the MPD, and further determines the segment to be requested, sends a corresponding segmented HTTP request to the server, and passes the media.
- the player performs decoding playback.
- the media data generated by the server for the video content includes a video stream corresponding to different versions of the same video content, and an MPD of the code stream.
- the server generates a low-resolution low-rate low frame rate (such as 360p resolution, 300kbps code rate, 15fps frame rate) for the video content of the same episode, and a medium-rate medium-rate high frame rate (such as 720p).
- Resolution 1200 kbps, 25 fps frame rate, high resolution, high bit rate, high frame rate (such as 1080p resolution, 3000 kbps, 25 fps frame rate).
- FIG. 1 is a schematic structural diagram of an MPD of a system transmission scheme DASH standard.
- each of the information indicating a plurality of segments is described in time series, for example, Initialization Segment, Media Segment 1, Media Segment2, ..., Media Segment20, etc.
- the representation may include segmentation information such as a playback start time, a playback duration, and a network storage address (for example, a network storage address expressed in the form of a Uniform Resource Locator (URL)).
- URL Uniform Resource Locator
- the client In the process of the client requesting and obtaining the media data from the server, when the user selects to play the video, the client obtains the corresponding MPD according to the video content requested by the user to the server.
- the client sends a request for downloading the code stream segment corresponding to the network storage address to the server according to the network storage address of the code stream segment described in the MPD, and the server sends the code stream segment to the client according to the received request.
- the client After the client obtains the stream segment sent by the server, it can perform decoding, playback, and the like through the media player.
- FIG. 3 is a schematic diagram of switching of a code stream segment according to an embodiment of the present invention.
- the server can prepare three different versions of code stream data for the same video content (such as a movie), and describe the three different versions of the code stream data in the MPD using three Representations.
- the above three Representations (hereinafter referred to as rep) can be assumed to be rep1, rep2, rep3, and the like.
- rep1 is a high-definition video with a code rate of 4mbps (megabits per second)
- rep2 is a standard-definition video with a code rate of 2mbps
- rep3 is a normal video with a code rate of 1mbps.
- Each rep segment contains a video stream within a time period.
- each rep describes the segments of each time segment according to the time series, and the segment lengths of the same time period are the same, thereby enabling content switching of segments on different reps.
- the segment marked as shadow in the figure is the segmentation data requested by the client, wherein the first 3 segments requested by the client are segments of rep3, and the client may request rep2 when requesting the 4th segment.
- the fourth segment in the middle can be switched to play on the fourth segment of rep2 after the end of the third segment of rep3.
- the playback end point of the third segment of Rep3 (corresponding to the time end of the playback time) is the playback start point of the fourth segment (corresponding to the time start time of playback), and also rep2 or rep1.
- the playback start point of the 4th segment is used to achieve alignment of segments on different reps. After the client requests the 4th segment of rep2, it switches to rep1, requests the 5th segment and the 6th segment of rep1, and so on. Then you can switch to rep3, request the 7th segment of rep3, then switch to rep1, request the 8th segment of rep1.
- Each rep segment can The files that are connected end to end exist in one file, and can also be stored as small files one by one.
- the segment may be packaged in accordance with the standard ISO/IEC 14496-12 (ISO BMFF (Base Media File Format)) or may be encapsulated in accordance with ISO/IEC 13818-1 (MPEG-2 TS). It can be determined according to the requirements of the actual application scenario, and no limitation is imposed here.
- FIG. 4 is a schematic diagram of a segment storage mode in the code stream data; All the segments on the same rep are stored in one file, as shown in Figure 5.
- Figure 5 is another schematic diagram of the segmentation storage mode in the code stream data.
- each segment in the segment of repA is stored as a file separately, and each segment in the segment of repB is also stored as a file separately.
- the server may describe information such as the URL of each segment in the form of a template or a list in the MPD of the code stream.
- the server may use an index segment (English: index segment, that is, sidx in FIG. 5) in the MPD of the code stream to describe related information of each segment.
- the index segment describes the byte offset of each segment in its stored file, the size of each segment, and the duration of each segment (also known as the duration of each segment).
- the spatial region of the VR video (the spatial region may also be called a spatial object) is a 360-degree panoramic space (or full).
- the azimuth space, or the panoramic space object exceeds the normal visual range of the human eye. Therefore, the user changes the viewing angle (ie, the angle of view, FOV) at any time during the process of watching the video.
- FIG. 6 is a schematic diagram of a perspective corresponding to a change in viewing angle.
- Box 1 and Box 2 are two different perspectives of the user, respectively.
- the video image viewed when the user's perspective is box 1 is a video image presented by the one or more spatial objects corresponding to the perspective at the moment.
- the user's perspective is switched to box 2.
- the video image viewed by the user should also be switched to the video image presented by the space object corresponding to box 2 at that moment.
- the server may divide a panoramic space (or called a panoramic spatial object) within a 360-degree viewing angle range to obtain a plurality of spatial objects, each The spatial object corresponds to a sub-view of the user, and the splicing of the plurality of sub-views forms a complete human eye viewing angle. That is, the human eye angle of view (hereinafter referred to as the angle of view) may correspond to one or more divided spatial objects, and the spatial object corresponding to the perspective is all the spatial objects corresponding to the content objects within the scope of the human eye.
- the angle of view may correspond to one or more divided spatial objects
- the spatial object corresponding to the perspective is all the spatial objects corresponding to the content objects within the scope of the human eye.
- the viewing angle of the human eye can be dynamically changed, but generally the viewing angle range may be 120 degrees*120 degrees, and the spatial object corresponding to the content object within the human eye angle range of 120 degrees*120 degrees may include one or more divided objects.
- the spatial object is, for example, the angle of view 1 corresponding to the frame 1 of FIG. 6 and the angle of view 2 corresponding to the frame 2.
- the client may obtain the spatial information of the video code stream prepared by the server for each spatial object through the MPD, and then request the video code stream corresponding to one or more spatial objects in a certain period of time according to the requirement of the perspective.
- the segment outputs the corresponding spatial object according to the perspective requirements.
- the client outputs the video stream segment corresponding to all the spatial objects within the 360-degree viewing angle range in the same time period, and then displays the complete video image in the entire 360-degree panoramic space.
- the server may first map the spherical surface into a plane, and divide the spatial object on the plane. Specifically, the server may map the spherical surface into a latitude and longitude plan by using a latitude and longitude mapping manner.
- FIG. 7 is a schematic diagram of a spatial object according to an embodiment of the present invention. The server can map the spherical surface into a latitude and longitude plan, and divide the latitude and longitude plan into a plurality of spatial objects such as A to I.
- the server may also map the spherical surface into a cube, expand the plurality of faces of the cube to obtain a plan view, or map the spherical surface to other polyhedrons, and expand the plurality of faces of the polyhedron to obtain a plan view or the like.
- the server can also map the spherical surface to a plane by using more mapping methods, which can be determined according to the requirements of the actual application scenario, and is not limited herein. The following will be described in conjunction with FIG. 7 in a latitude and longitude mapping manner. As shown in FIG. 7, after the server can divide the spherical panoramic space into a plurality of spatial objects such as A to I, a set of DASH video code streams can be prepared for each spatial object.
- the set of DASH video streams corresponding to each spatial object When the client user switches the viewing angle of the video viewing, the client can obtain the code stream corresponding to the new spatial object according to the new perspective selected by the user, and then the video content of the new spatial object code stream can be presented in the new perspective.
- the method and apparatus for processing information provided by the embodiments of the present invention will be described below with reference to FIG. 8 to FIG.
- the system layer video streaming media transmission scheme adopts the DASH standard, and realizes the transmission of video data by analyzing the MPD by the client, requesting the video data to the server as needed, and receiving the data sent by the server.
- a main plot route may be designed for the video playback according to the storyline requirement of the video.
- the user only needs to watch the video image corresponding to the main plot route to understand the storyline, and other video images can be seen or not.
- the client can selectively play the video image corresponding to the storyline, and other video images may not be presented, which can save the transmission resource and storage space resources of the video data, and improve the processing efficiency of the video data. .
- the video image to be presented to the user at each play time during video playback can be set according to the above-mentioned main plot route, and the video sequence of each play time can be obtained by stringing the time series to obtain the above main plot route.
- Storyline The video image to be presented to the user at each of the playing times is a video image presented on a spatial object corresponding to each playing time, that is, a video image to be presented by the spatial object during the time period.
- the angle of view corresponding to the video image to be presented at each of the playing times may be set as the author's perspective
- the spatial object that presents the video image in the perspective of the author may be set as the author space object.
- the code stream corresponding to the author view object can be set as the author view code stream.
- the video stream data of multiple video frames (encoded data of multiple video frames) is included in the code stream of the author.
- Each video frame can be presented as one image, that is, corresponding to multiple images in the author's view code stream.
- the image presented by the author's perspective is only part of the panoramic image (or VR image or omnidirectional image) that the entire video is to present.
- the spatial information of the spatial object associated with the image corresponding to the author's video stream may be different, or may be the same, that is, the spatial information of the spatial object associated with the video data of the author's perspective stream is different.
- the corresponding code stream can be prepared by the server for the author perspective of each play time.
- the code stream corresponding to the author view may be set as the author view code stream.
- the server may encode the author view code stream and transmit it to the client.
- the story scene picture corresponding to the author view code stream may be presented to the user.
- the server does not need to transmit the code stream of other perspectives other than the author's perspective (set to the non-author perspective, that is, the static view stream) to the client, which can save resources such as the transmission bandwidth of the video data.
- the author's perspective is an image of the preset space object set by the author according to the video storyline
- the author space objects at different playing times may be different or the same, thereby knowing that the author's perspective is a With the changing perspective of the playing time
- the author's spatial object is a dynamic space pair with constantly changing positions.
- the image space of the author space object corresponding to each play time is not the same in the panoramic space.
- Each of the spatial objects shown in FIG. 7 is a spatial object divided according to a preset rule, and is a spatial object fixed in a relative position in the panoramic space.
- the author space object corresponding to any play time is not necessarily fixed as shown in FIG. 7.
- One of the spatial objects and a spatial object whose relative position is constantly changing in the global space.
- the content presented by the video obtained by the client from the server is stringed by each author's perspective. It does not contain the spatial object corresponding to the non-author perspective.
- the author view code stream only contains the content of the author space object, and the MPD obtained from the server does not.
- the spatial information of the author's spatial object containing the author's perspective the client can only decode and present the code stream of the author's perspective. If the viewing angle of the viewing is switched to the non-author perspective during the video viewing process, the client cannot present the corresponding video content to the user.
- the identifier information when the server generates the media presentation description, may be added to the media presentation description for identifying the author view code stream of the video, that is, the author view code stream.
- the identifier information may be carried in the attribute information of the code stream set in which the author view code stream is carried in the media presentation description, that is, the identifier information may be carried in the information of the adaptive set in the media presentation description, where the identifier is The information can also be carried in the information contained in the presentation contained in the media presentation description. Further, the foregoing identification information may also be carried in the information of the descriptor in the media presentation description.
- the client can quickly obtain the code stream of the author view code stream and the non-author view by parsing the MPD to obtain the syntax elements added in the MPD. If the spatial information related to the author's view stream is encapsulated in a separate metadata file, the client can parse the MPD and obtain the metadata of the spatial information according to the codec identifier, thereby parsing the spatial information.
- the server may also add spatial information for one or more author space objects in the author view stream.
- each author space object corresponds to one or more images, that is, one or more images may be associated with the same spatial object, or each image may be associated with one spatial object.
- the server can add the spatial information of each author space object in the author view code stream, and can also use the space information as a sample and independently encapsulate it in a track or file.
- the spatial information of an author space object is the spatial relationship between the author space object and its associated content component, that is, the spatial relationship between the author space object and the panoramic space. That is, the space described by the spatial information of the author space object may specifically be a partial space in the panoramic space, such as any one of the spatial objects in FIG. 7 above.
- the server may add the foregoing spatial information to the trun box or the tfhd box included in the segment of the author view code stream in the existing file format, and describe the video frame of the author view code stream.
- the spatial information of the spatial object associated with each frame of image corresponding to the data may be added to the server.
- the spatial information of the spatial objects associated with each frame of image may have the same information, the spatial information of the plurality of author spatial objects is duplicated and redundant, thereby affecting the efficiency of data transmission.
- the modification of the file format provided by the present invention can also be applied to the file format of the ISOBMFF or the MPEG2-TS, and can be determined according to the requirements of the actual application scenario, and is not limited herein.
- FIG. 8 is a schematic flowchart diagram of a method for processing information of a streaming media according to an embodiment of the present invention.
- the information processing method of the streaming media provided by the embodiment of the present invention can be applied to the DASH field, and can also be applied to other streaming media fields, such as streaming media transmission based on the RTP protocol.
- the executor of the method may be a client, and may be a terminal, a user equipment, or a computer device, or may be a network device, such as a gateway, a proxy server, or the like. As shown in FIG. 8, the method may include the following steps:
- Target space information of a target space object where the target space object is a spatial object of two spatial objects, where the two spatial objects are associated with data of two images included in the target video data, and the target spatial information includes the same Attribute space information, including the spatial information of each of the two spatial objects
- the spatial information of the other spatial objects except the target spatial object of the two spatial objects includes the same attribute spatial information.
- the target video data may be the target video code stream or the uncoded video data.
- the data of the two images may be the encoded data of the two images.
- the target video code stream may be an author view code stream or a non-author view code stream.
- the acquiring the target space information of the target space object may be receiving the target space information from the server.
- the two images may correspond to the two spatial objects one by one, or one spatial object may correspond to two images.
- the spatial information of a target space object is a spatial relationship between the target space object and its associated content component, that is, the spatial relationship between the target space object and the panoramic space. That is, the space described by the target space information of the target space object may specifically be a partial space in the panoramic space.
- the target video data may be the above-mentioned author view code stream or a non-author view code stream.
- the target space object may be the author space object described above, or may not be.
- the video data to be played can also be displayed.
- the target space information may further include hetero-attribute space information of the target space object, and the spatial information of the other spatial object further includes hetero-attribute space information of the other spatial object, and the hetero-attribute space of the target spatial object The information is different from the heterogeneous information of the other spatial object.
- the target space information may include location information of a center point of the target spatial object or location information of an upper left point of the target spatial object, and the target spatial information may further include the target space.
- the width of the object and the height of the target space object may include location information of a center point of the target spatial object or location information of an upper left point of the target spatial object, and the target spatial information may further include the target space. The width of the object and the height of the target space object.
- the target spatial information when the coordinate system corresponding to the target spatial information is an angular coordinate system, the target spatial information may be described by using a yaw angle.
- the target spatial information When the coordinate system corresponding to the target spatial information is a pixel coordinate system, the target spatial information may adopt a latitude and longitude map.
- the spatial position description is described by other geometric solid figures, and no limitation is imposed here. It is described by the yaw angle method, such as the pitch angle ⁇ (pitch), the yaw angle yaw, and the roll angle ⁇ (roll), which are used to indicate the width of the angle range and to indicate the height of the angle range.
- FIG. 9 is a schematic diagram of the relative positions of the center points of the target space objects in the panoramic space. In FIG.
- the point O is the center of the 360-degree VR panoramic video spherical image, which can be considered as the position of the human eye when viewing the VR panoramic image.
- Point A is the center point of the target space object
- C and F are the boundary points of the target space object along the horizontal coordinate axis of the target space object
- E and D are the target space objects passing the point A along the target space.
- the boundary point of the longitudinal coordinate axis of the object B is the projection point of the A point along the spherical meridian line on the equator line
- I is the starting coordinate point of the horizontal direction on the equator line.
- Pitch angle the center position of the image of the target space object is mapped to the vertical direction of the point on the panoramic spherical (ie global space) image, such as ⁇ AOB in FIG. 9;
- Yaw angle the center position of the image of the target space object is mapped to the horizontal deflection angle of the point on the panoramic spherical image, as shown in FIG. 9 ⁇ IOB;
- Rolling angle the center position of the image of the yaw angle space object is mapped to the rotation angle of the point on the panoramic spherical image and the connection direction of the spherical center, as shown in FIG. 9 ⁇ DOB;
- the image of the target space object is represented by the maximum field of view of the panoramic spherical image, as shown by ⁇ DOE in Figure 9;
- the width of the angular range (the width of the target space object in the angular coordinate system):
- the image of the target space object is represented by the field of view of the panoramic spherical image, as indicated by the horizontal maximum angle of the field of view, as shown in Figure 9 for COF.
- the target spatial information may include location information of an upper left point of the target spatial object, and location information of a lower right point of the target spatial object.
- the target space information when the target space object is not a rectangle, the target space information may include at least one of a shape type, a radius, and a perimeter of the target space object.
- the target spatial information can include spatial rotation information for the target spatial object.
- the target spatial information may be encapsulated in spatial information data or a spatial information track
- the spatial information data may be a code stream of the target video data, metadata of the target video data, or independent of the A file of target video data, which may be a track independent of the target video data.
- the spatial information data or the spatial information track may further include a spatial information type identifier for indicating the type of the same attribute spatial information, where the spatial information type identifier is used to indicate that the target spatial information belongs to the same attribute spatial information.
- Information for indicating the type of the same attribute spatial information, where the spatial information type identifier is used to indicate that the target spatial information belongs to the same attribute spatial information.
- the same attribute spatial information may include a wide minimum of the target spatial object, and a high minimum of the target spatial object.
- the width maximum of the target space object and the high maximum of the target space object may include a wide minimum of the target spatial object, and a high minimum of the target spatial object.
- the spatial information type identifier and the same attribute space information may be encapsulated in the same box.
- the server when the target space information is encapsulated in a file (a spatial information file) independent of the target video data or a track (a spatial information track) independent of the target video data, the server may be in a 3dsc in an existing file format. Add the same attribute space information in the box, and add the different attribute space information of the target space object in the mdat box in the existing file format.
- the same attribute space information may be part of yaw, pitch, roll, reference_width, and reference_height, but not all, for example, no roll.
- the roll may belong to the different attribute space information of the target space object, or may not be included in the target space information.
- the spatial information type identifier regionType is also added to the 3dsc box, and the sample is an example in the angular coordinate system. Wherein, when the spatial information type identifier is 0, the spatial information type identifier is used to indicate that the information belonging to the same attribute spatial information in the target spatial information is the location information of the central point of the target spatial object or the upper left of the target spatial object The location information of the point, as well as the width of the target space object and the height of the target space object.
- the position information is represented by a pitch angle ⁇ (pitch), a yaw angle yaw (yaw), and a roll angle ⁇ (roll), and the width and height can also be expressed by angles.
- pitch angle ⁇ pitch
- yaw yaw angle
- roll roll
- the width and height can also be expressed by angles.
- the spatial information type identifier is used to indicate that the information belonging to the same attribute spatial information in the target spatial information is the width of the target spatial object and the height of the target spatial object.
- the sizes of the two spatial objects are the same and the positions are different.
- the spatial information type identifier is 2
- the spatial information type identifier is used to indicate that the target spatial information does not belong to the same attribute spatial information. Another way to understand is that when the spatial information type identifier is 2, the size and position of the two spatial objects are different.
- the spatial information type identifier when the spatial information type identifier is 0, it may indicate that the hetero-attribute space information does not exist.
- the spatial information type identifier when the spatial information type identifier is 1, the spatial information type identifier further indicates that the heterogeneous spatial information of the target spatial object is the The position information of the center point of the target space object or the position information of the upper left point of the target space object.
- the spatial information type identifier is 2
- the spatial information type identifier further indicates that the heterogeneous spatial information of the target spatial object is location information of a central point of the target spatial object or location information of an upper left point of the target spatial object, and the target The width of the spatial object and the height of the target space object.
- the sample is a sample in a pixel coordinate system.
- the spatial information type identifier is used to indicate that the information belonging to the same attribute spatial information in the target spatial information is the upper left point of the target spatial object.
- the position information is represented by the abscissa in units of pixels and the ordinate in pixels, and the width and height can also be expressed in units of pixels.
- the abscissa and the ordinate may be coordinates of the position point in the latitude and longitude plan view in FIG. 7, or may be coordinates in the panoramic space (or the panoramic space object).
- the location and size of the two spatial objects are the same. It should be noted that the location information of the center point of the target space object may be used instead of the location information of the upper left point of the target space object.
- the spatial information type identifier is 1
- the spatial information type identifier is used to indicate that the information belonging to the same attribute spatial information in the target spatial information is the width of the target spatial object and the height of the target spatial object. Another way to understand is that when the spatial information type identifier is 1, the two spatial objects have the same size and different positions.
- the spatial information type identifier is 2
- the spatial information type identifier is used to indicate that the target spatial information does not belong to the same attribute spatial information. Another way to understand is that when the spatial information type identifier is 2, the size and position of the two spatial objects are different.
- the spatial information type identifier when the spatial information type identifier is 0, it may indicate that the hetero-attribute space information does not exist.
- the spatial information type identifier when the spatial information type identifier is 1, the spatial information type identifier further indicates that the heterogeneous spatial information of the target spatial object is the location information of the upper left point of the target spatial object.
- the spatial information type identifier is 2 the spatial information type identifier further indicates that the heterogeneous spatial information of the target spatial object is location information of the upper left point of the target spatial object, and the width of the target spatial object and the height of the target spatial object.
- the location information of the center point of the target space object may be used instead of the location information of the upper left point of the target space object.
- Example of adding spatial information (Sample 3):
- the sample is a sample in a pixel coordinate system.
- the spatial information type identifier is used to indicate that the information belonging to the same attribute spatial information in the target spatial information is the upper left point of the target spatial object.
- the positional information is represented by the abscissa in units of pixels and the ordinate in units of pixels.
- the abscissa and the ordinate may be coordinates of the position point in the latitude and longitude plan view in FIG. 7, or may be coordinates in the panoramic space (or the panoramic space object).
- the position information of the lower right point of the target space object may be replaced by the width and height of the target space object.
- the spatial information type identifier is used to indicate that the information belonging to the same attribute spatial information in the target spatial information is the location information of the lower right point of the target spatial object. Another way to understand is that when the spatial information type identifier is 1, the two spatial objects have the same size and different positions. It should be noted that the position information of the lower right point of the target space object may be replaced by the width and height of the target space object.
- the spatial information type identifier is 2
- the spatial information type identifier is used to indicate that the target spatial information does not belong to the same attribute spatial information. Another way to understand is that when the spatial information type identifier is 2, the size and position of the two spatial objects are different.
- the spatial information type identifier when the spatial information type identifier is 0, it may indicate that the hetero-attribute space information does not exist.
- the spatial information type identifier when the spatial information type identifier is 1, the spatial information type identifier further indicates that the heterogeneous spatial information of the target spatial object is the location information of the upper left point of the target spatial object.
- the spatial information type identifier is 2 the spatial information type identifier further indicates that the heterogeneous spatial information of the target spatial object is location information of an upper left point of the target spatial object, and location information of a lower right point of the target spatial object. It should be noted that the position information of the lower right point of the target space object may be replaced by the width and height of the target space object.
- the spatial information data or the spatial information trajectory may further include a coordinate system identifier for indicating a coordinate system corresponding to the target spatial information, and the coordinate system is a pixel coordinate system or an angular coordinate system.
- the coordinate system identifier and the same attribute space information may be encapsulated in the same box.
- the server when the target space information is encapsulated in a file (a spatial information file) independent of the target video data or a track (a spatial information track) independent of the target video data, the server may be in a 3dsc in an existing file format. Add a coordinate system identifier to the box.
- the coordinate system is an angular coordinate system
- the coordinate system identifier is 1, the coordinate system is a pixel coordinate system.
- the spatial information data or the spatial information trajectory may further include a spatial rotation information identifier for indicating whether the target spatial information includes spatial rotation information of the target spatial object.
- the spatial rotation information identifier may be encapsulated in the same box (for example, 3dscbox) as the same attribute space information, and the spatial rotation information identifier may also be encapsulated in the same box with the different attribute space information of the target space object (for example, mdat) Box). Specifically, when the spatial rotation information identifier and the heterogeneous spatial information of the target spatial object are encapsulated in the same box, when the spatial rotation information identifier indicates that the target spatial information includes spatial rotation information of the target spatial object, The hetero-attribute space information of the target space object includes spatial rotation information.
- the server may encapsulate the spatial rotation information identifier and the heterogeneous spatial information of the target spatial object in a same box (for example, an mdat box). Further, the server may encapsulate the spatial rotation information identifier and the heterogeneous spatial information of the target spatial object in the same sample of the same box. One of the samples can encapsulate the different attribute information corresponding to a spatial object.
- Example 1 Example of adding a spatial rotation information identifier (example 1):
- the same attribute space information and the attribute space information of the target space object may also be encapsulated in the track matedata of the video, such as in the same box, for example, a trun box or The tfhd box is either in the new box.
- Example of adding spatial information (Sample 4):
- One spatial information of one spatial object is a sample, and the above sample quantity is used to indicate the number of spatial objects, and each spatial object corresponds to a respective set of different attribute spatial information.
- the steps are as follows:
- Obtain a spatial information file or a spatial information track (the spatial information may be called timed metadata) or a spatial information metadata of the video (or metadata called target video data);
- Parsing the box (space information description box) with the tag 3dsc, parsing the spatial information type identifier, the spatial information type identifier can be used to indicate the spatial object type of the two spatial objects, and the optional spatial object type can include but not Limited to: spatial objects with constant position and size, spatial objects with different position and size, spatial objects with unchanged position and size, and spatial objects with different positions and sizes;
- the spatial object type that is parsed is a spatial object with the same position and size
- the spatial object with the same position and size refers to the spatial position of the spatial object and the size of the spatial object, so it is parsed in the 3dsc box.
- the same attribute space information can be used as the target space information
- the spatial object type indicates that the spatial information of all the two spatial objects are the same, and the values are consistent with the parsed same attribute space information; if it is the same type Attribute space information, in the subsequent parsing, there is no need to parse the box where the heterogeneous spatial information of the target space object is located;
- the spatial object type parsed is a spatial object with a change in position and size
- the same attribute space information in the 3dscbox will carry the size information of the spatial object, such as the width of the spatial object; then in the subsequent analysis
- the obtained information in the hetero-attribute space information of the target space object is the position information of each spatial object.
- the information carried in the heterogeneous spatial information of the target spatial object obtained by subsequent parsing is the position information of each spatial object (for example, The position information of the center point) and the size information of the space object, such as the width and height of the space object;
- the target space information After parsing the target space information, select the presented content object in the obtained VR video according to the spatial object (target space object) described by the target space information; or request the video data corresponding to the spatial object described by the target spatial information to decode Presenting, or determining, based on the target spatial information, the location of the currently viewed video content in the VR video space (or, the panoramic space).
- the manner in which the spatial information is carried may be described by adding a carryType in the MPD: carried in the spatial information file or the spatial information track or the metadata of the target video data.
- the specific MPD sample is as follows:
- Example 1 Spatial information is carried in the metadata of the target video data.
- va l ue 1,0
- 1 is the identity of the source
- 0 indicates that the spatial information is carried in the metadata of the trajectory of the target video data (or metadata called the target video data).
- Example 2 Spatial information is carried in the spatial information track
- va l ue 1,1
- 1 is the identity of the source, and 1 indicates that the spatial information is carried in the independent spatial information track.
- Example 3 Spatial information is carried in a separate spatial information file
- va l ue 1,2
- 1 is the identity of the source
- 2 indicates that the spatial information is carried in a separate spatial information file
- the target video representation associated with the spatial information file or called the target video
- the client can obtain the spatial information carrying manner by parsing the MPD, thereby obtaining the spatial letter according to the carrying manner. interest.
- the spatial information data or the spatial information track may further include a width and height type identifier for indicating the target space object, where the width and height type identifier may be used to indicate the target space object for describing A coordinate system of width and height, or the width and height type identification may be used to indicate a coordinate system for describing a boundary of the target spatial object.
- the width and height type identifier may be an identifier, and may also include a wide type identifier and a high type identifier.
- the space type identifier and the same attribute space information may be encapsulated in the same box (for example, a 3dsc box), and the width and height type identifier may also be encapsulated in the same box as the different attribute space information of the target space object. For example, mdat box).
- the server may encapsulate the width and height type identifier and the same attribute space information in the same box (for example, a 3dsc box). Further, when the target space information is encapsulated in a file (a spatial information file) independent of the target video data or a track (a spatial information track) independent of the target video data, the server may add a width and height type identifier in the 3dsc box. .
- the same attribute space information and the attribute space information of the target space object may also be encapsulated in the track matedata of the video, such as in the same box, for example, a trun box or Tfhd box or tfhd box or new box.
- the coordinate system for describing the width and height of the target space object is as shown in FIG. 10, and the shaded portion of the spherical surface is the target space object, and the four corners of the target space object are The vertices are B, E, G, and I respectively; in FIG.
- O is the center of the sphere corresponding to the 360-degree VR panoramic video spherical image
- the vertices BEGI are respectively the circle of the spherical center (the circle is centered on the center of the sphere O, and The radius of the circle is the radius of the sphere corresponding to the 360 degree VR panoramic video spherical image
- the circle passes the z axis
- the number of the circle is two, one passing point BAIO, one passing point EFGO), and parallel to the coordinate axis x axis
- the circle of the y-axis (the circle is not centered on the center of the sphere O, the number of the circle is two, and the two circles are parallel to each other, one passing point BDE, one passing point IHG) on the spherical surface
- C is the target
- the center point of the spatial object, the angle corresponding to the DH side is represented as the height of the target space object, the angle corresponding to the AF side is represented as the width of the target space
- the vertex of the angle corresponding to the IG side is the intersection of the circle where the IHG is located and the z-axis in the above circle, and the vertex of the angle corresponding to the AF edge.
- the vertices of the corners corresponding to the BI side, the EG side, and the DH side are also O points.
- the target space object can also be two circles passing the x-axis and parallel to the y-axis and the z-axis, but the two circles of the spherical center are intersected, and the target space object can also be The two circles of the y-axis are obtained by intersecting the two circles parallel to the x-axis and the z-axis but not the center of the sphere.
- the width and height type identifier is 1
- the coordinate system for describing the width and height of the target space object is as shown in FIG. 11, and the shaded portion of the spherical surface is the target space object, and the vertices of the four corners of the target space object are B, E, respectively. , G, I; In Fig.
- O is the spherical center corresponding to the 360-degree VR panoramic video spherical image
- the apex BEGI is a circle passing through the z-axis (the circle is centered on the center O, and the radius of the circle is 360)
- the radius of the sphere corresponding to the VR panoramic video spherical image, the number of the circle is two, one passing point BAI, one passing point EFG), and a circle passing the y-axis (the circle is centered on the center O, and the circle
- the radius is the radius of the sphere corresponding to the 360 degree VR panoramic video spherical image.
- the number of the circle is two, one passing point BDE, one passing point IHG) on the spherical surface, C is the center point of the target space object, DH
- the angle corresponding to the edge is represented as the height of the target space object
- the angle corresponding to the AF edge is represented as the width of the target space object
- the DH edge and the AF edge pass the C point, wherein the BI edge, the EG edge and the DH edge correspond to the same angle
- the BE edge The angles corresponding to the IG edge and the AF edge are the same; the top of the corner corresponding to the BE edge
- the point is point J, the point J is the intersection of the circle passing through the two points of BE and parallel to the x-axis and the y-axis, and the apex of the angle corresponding to the IG side is a circle passing through two points of IG and parallel with the x-axis and the y-axis.
- the intersection with the z-axis, the vertex of the angle corresponding to the AF side is the O point, the vertex of the angle corresponding to the BI side is the L point, and the point L is the intersection of the circle parallel to the z-axis and the x-axis and the y-axis.
- the vertex of the angle corresponding to the EG side is the intersection of the circle passing through the EG two points and parallel to the z-axis and the x-axis, and the apex of the angle corresponding to the DH side is also the O-point.
- the target space object can also be obtained by intersecting two circles passing the x-axis and two circles passing the z-axis.
- the target space object can also be two circles passing the x-axis and The two circles of the y-axis are intersected.
- the coordinate system for describing the width and height of the target space object is as shown in FIG. 12, and the shaded portion of the spherical surface is the target space object, and the vertices of the four corners of the target space object are B, E, respectively. , G, I; In FIG.
- O is a spherical center corresponding to a 360-degree VR panoramic video spherical image
- the vertices BEGI are respectively a circle parallel to the coordinate axis x-axis and the z-axis (the circle is not centered on the spherical center O,
- the number of the circles is two, and the two circles are parallel to each other, the number of the circles is two, one passing point BAI, one passing point EFG), and a circle parallel to the coordinate axis x-axis and y-axis (the circle Do not take the center of the sphere O as the center, the number of the circle is two, and the two circles are parallel to each other, one passing point BDE, one passing point I HG) on the spherical surface
- C is the center point of the target space object
- DH The angle corresponding to the edge is represented as the height of the target space object
- the angle corresponding to the AF edge is represented as the width of the target space object
- the target space object may also be parallel to the y-axis and the z-axis and the two circles of the spherical center and the y-axis and the x-axis are parallel, but the two circles of the spherical center intersect.
- the target space object may also be parallel to the y-axis and the z-axis and the two circles of the spherical center are parallel to the z-axis and the x-axis and the two circles of the spherical center intersect.
- the J point and the L point in FIG. 11 are the same as the J point in FIG. 10, the vertex of the corner corresponding to the BE side is the J point, and the vertex of the angle corresponding to the BI side is the L point; in FIG. 12, the BE side The vertices corresponding to the BI side are all O points.
- the same attribute space information and the attribute space information of the target space object may also include description information of the target space object, such as the description information used to describe the target space object as a view area (for example, It may be a spatial object corresponding to the view code stream, or a region of interest, or the description information is used to describe the quality information of the target space object.
- description information of the target space object such as the description information used to describe the target space object as a view area (for example, It may be a spatial object corresponding to the view code stream, or a region of interest, or the description information is used to describe the quality information of the target space object.
- the tfhd box is either a new box syntax (syntax) to add the description information, or the description information (content_type) is added in the SphericalCoordinatesSample for implementing one or more of the following functions: describing the target space object as a view area
- the target space object is described as a region of interest, and quality information for describing the target space object.
- FIG. 13 is a schematic flowchart diagram of a method for processing information of a streaming media according to an embodiment of the present invention.
- the execution body of the method may be a server, and may specifically be a computer device. As shown in FIG. 13, the method may include the following steps:
- S1001 Acquire spatial information of each of two spatial objects associated with data of two images in the target video data
- S1002 Determine, according to spatial information of each of the two spatial objects, target space information of the target spatial object, where the target spatial object is a spatial object in the two spatial objects, where the target spatial information includes the same attribute space information, and the same attribute space
- the information includes the same information between the spatial information of the two spatial objects, and the spatial information of the other spatial objects other than the target spatial object includes the same attribute spatial information.
- the method may further include transmitting the target space information to the client.
- the target space information may further include hetero-attribute space information of the target space object, and the spatial information of the other spatial object further includes hetero-attribute space information of the other spatial object, and the hetero-attribute space of the target spatial object The information is different from the heterogeneous information of the other spatial object.
- the target spatial information may include location information of a central point of the target spatial object or location information of an upper left point of the target spatial object, the target spatial information may further include a width of the target spatial object and the target The height of the space object.
- the spatial information of the two spatial objects may include location information of respective center points of the two spatial objects or position information of respective upper left points of the two spatial objects, and spatial information of each of the two spatial objects is further The respective widths of the two spatial objects and the respective heights of the two spatial objects may be included.
- the target spatial information may include location information of an upper left point of the target spatial object, and location information of a lower right point of the target spatial object.
- the spatial information of each of the two spatial objects may include location information of respective upper left points of the two spatial objects and position information of respective lower right points of the two spatial objects.
- the target spatial information can include spatial rotation information for the target spatial object.
- the spatial information of each of the two spatial objects may include spatial rotation information of the two spatial objects.
- the target spatial information may be encapsulated in spatial information data or a spatial information track
- the spatial information data may be a code stream of the target video data, metadata of the target video data, or independent of the A file of target video data, which may be a track independent of the target video data.
- the spatial information data or the spatial information track may further include a spatial information type identifier for indicating the type of the same attribute spatial information, where the spatial information type identifier is used to indicate that the target spatial information belongs to the same attribute spatial information.
- Information for indicating the type of the same attribute spatial information, where the spatial information type identifier is used to indicate that the target spatial information belongs to the same attribute spatial information.
- the same attribute spatial information may include a wide minimum of the target spatial object, and a high minimum of the target spatial object.
- the width maximum of the target space object and the high maximum of the target space object may include a wide minimum of the target spatial object, and a high minimum of the target spatial object.
- the spatial information type identifier may be encapsulated in the same box as the same attribute spatial information.
- the spatial information data or the spatial information trajectory may further include a coordinate system identifier for indicating a coordinate system corresponding to the target spatial information, and the coordinate system is a pixel coordinate system or an angular coordinate system.
- the coordinate system identifier may be encapsulated in the same box as the same attribute space information.
- the spatial information data or the spatial information trajectory may further include a spatial rotation information identifier, where the spatial rotation information identifier is used to indicate whether the target spatial information includes spatial rotation information of the target spatial object.
- FIG. 14 is a processing device 1100 for streaming information according to an embodiment of the present invention.
- the information processing device 1100 may be a server, and may be a computer device.
- the device 1100 includes an obtaining module 1101 and a determining module 1102, where
- the acquiring module 1101 is configured to acquire target space information of a target space object, where the target space object is a spatial object of two spatial objects, where the two spatial objects are associated with data of two images included in the target video data, the target
- the spatial information includes the same attribute space information
- the same attribute space information includes the same information between the spatial information of the two spatial objects, and the spatial information of the other spatial objects except the target spatial object includes the spatial information.
- Same attribute space information is configured to acquire target space information of a target space object, where the target space object is a spatial object of two spatial objects, where the two spatial objects are associated with data of two images included in the target video data, the target The spatial information includes the same attribute space information, and the same attribute space information includes the same information between the spatial information of the two spatial objects, and the spatial information of the other spatial objects except the target spatial object includes the spatial information. Same attribute space information.
- the determining module 1102 is configured to determine video data to be played according to the target space information acquired by the acquiring module.
- the information processing apparatus 1100 may further include a display module (or a display) for displaying the video data to be played.
- the acquisition module 1101 is configured to receive the target spatial information from the server.
- the acquisition module may be a receiving module (or a receiver, or a transceiver).
- the target space information may further include hetero-attribute space information of the target space object, and the spatial information of the other spatial object further includes hetero-attribute space information of the other spatial object, and the hetero-attribute space of the target spatial object The information is different from the heterogeneous information of the other spatial object.
- the target spatial information may include location information of a central point of the target spatial object or location information of an upper left point of the target spatial object, the target spatial information may further include a width of the target spatial object and the target The height of the space object.
- the spatial information of the two spatial objects may include location information of respective center points of the two spatial objects or position information of respective upper left points of the two spatial objects, and spatial information of each of the two spatial objects is further The respective widths of the two spatial objects and the respective heights of the two spatial objects may be included.
- the target spatial information may include location information of an upper left point of the target spatial object, and location information of a lower right point of the target spatial object.
- the spatial information of each of the two spatial objects may include location information of respective upper left points of the two spatial objects and position information of respective lower right points of the two spatial objects.
- the target spatial information can include spatial rotation information for the target spatial object.
- the spatial information of each of the two spatial objects may include spatial rotation information of the two spatial objects.
- the target spatial information may be encapsulated in spatial information data or spatial information trajectories.
- the spatial information data may be a code stream of the target video data, metadata of the target video data, or a file independent of the target video data
- the spatial information track may be a track independent of the target video data.
- the spatial information data or the spatial information track may further include a spatial information type identifier for indicating the type of the same attribute spatial information, where the spatial information type identifier is used to indicate that the target spatial information belongs to the same attribute spatial information.
- Information for indicating the type of the same attribute spatial information, where the spatial information type identifier is used to indicate that the target spatial information belongs to the same attribute spatial information.
- the same attribute spatial information may include a wide minimum of the target spatial object, and a high minimum of the target spatial object.
- the width maximum of the target space object and the high maximum of the target space object may include a wide minimum of the target spatial object, and a high minimum of the target spatial object.
- the spatial information type identifier may be encapsulated in the same box as the same attribute spatial information.
- the spatial information data or the spatial information trajectory may further include a coordinate system identifier for indicating a coordinate system corresponding to the target spatial information, and the coordinate system is a pixel coordinate system or an angular coordinate system.
- the coordinate system identifier may be encapsulated in the same box as the same attribute space information.
- the spatial information data or the spatial information trajectory may further include a spatial rotation information identifier, where the spatial rotation information identifier is used to indicate whether the target spatial information includes spatial rotation information of the target spatial object.
- the functions of the obtaining module 1101 and the determining module 1102 can be implemented by means of software programming, hardware programming, or circuit implementation, which is not limited herein.
- a processing device 1200 for providing information of a streaming media where the device includes an obtaining module 1201 and a determining module 1202, where
- the obtaining module 1201 is configured to acquire spatial information of each of two spatial objects associated with data of two images in the target video data.
- the determining module 1202 is configured to determine target space information of the target spatial object according to the spatial information of the two spatial objects acquired by the acquiring module, where the target spatial object is a spatial object in the two spatial objects, and the target spatial information is Include the same attribute space information, the same attribute space information includes the same information between the spatial information of the two spatial objects, and the spatial information of the other spatial objects except the target spatial object includes the same attribute Spatial information.
- the apparatus 1200 may further include a sending module (or a transmitter, or a transceiver) for transmitting the target space information determined by the determining module to the client.
- a sending module or a transmitter, or a transceiver for transmitting the target space information determined by the determining module to the client.
- the target space information may further include hetero-attribute space information of the target space object, and the spatial information of the other spatial object further includes hetero-attribute space information of the other spatial object, and the hetero-attribute space of the target spatial object The information is different from the heterogeneous information of the other spatial object.
- the target spatial information may include location information of a central point of the target spatial object or location information of an upper left point of the target spatial object, the target spatial information may further include a width of the target spatial object and the target The height of the space object.
- the spatial information of the two spatial objects may include location information of respective center points of the two spatial objects or position information of respective upper left points of the two spatial objects, and spatial information of each of the two spatial objects is further The respective widths of the two spatial objects and the respective heights of the two spatial objects may be included.
- the target spatial information may include location information of an upper left point of the target spatial object, and location information of a lower right point of the target spatial object.
- the spatial information of each of the two spatial objects may include location information of respective upper left points of the two spatial objects and position information of respective lower right points of the two spatial objects.
- the target spatial information can include spatial rotation information for the target spatial object.
- the spatial information of each of the two spatial objects may include spatial rotation information of the two spatial objects.
- the target spatial information may be encapsulated in spatial information data or a spatial information track
- the spatial information data may be a code stream of the target video data, metadata of the target video data, or independent of the A file of target video data, which may be a track independent of the target video data.
- the spatial information data or the spatial information track may further include a spatial information type identifier for indicating the type of the same attribute spatial information, where the spatial information type identifier is used to indicate that the target spatial information belongs to the same attribute spatial information.
- Information for indicating the type of the same attribute spatial information, where the spatial information type identifier is used to indicate that the target spatial information belongs to the same attribute spatial information.
- the same attribute spatial information may include a wide minimum of the target spatial object, and a high minimum of the target spatial object.
- the width maximum of the target space object and the high maximum of the target space object may include a wide minimum of the target spatial object, and a high minimum of the target spatial object.
- the spatial information type identifier may be encapsulated in the same box as the same attribute spatial information.
- the spatial information data or the spatial information trajectory may further include a coordinate system identifier for indicating a coordinate system corresponding to the target spatial information, and the coordinate system is a pixel coordinate system or an angular coordinate system.
- the coordinate system identifier may be encapsulated in the same box as the same attribute space information.
- the spatial information data or the spatial information trajectory may further include a spatial rotation information identifier, where the spatial rotation information identifier is used to indicate whether the target spatial information includes spatial rotation information of the target spatial object.
- the functions of the obtaining module 1201 and the determining module 1202 can be implemented by means of software programming, and can also be implemented by hardware programming or by a circuit, which is not limited herein.
- FIG. 16 is a schematic diagram showing the hardware structure of a computer device 1300 according to an embodiment of the present invention.
- the computer device 1300 can be used as an implementation of the processing device 1100 for streaming media information, and can also be used as an implementation of the streaming device information processing device 1200.
- the computer device 1300 includes a processor 1302. Memory 1304, input/output interface 1306, communication interface 1308, and bus 1310.
- the processor 1302, the memory 1304, the input/output interface 1306, and the communication interface 1308 implement communication connections with each other through the bus 1310.
- the processor 1302 may be a general-purpose central processing unit (CPU), a microprocessor, an application specific integrated circuit (ASIC), or one or more integrated circuits for executing related programs.
- CPU central processing unit
- ASIC application specific integrated circuit
- the processor 1302 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method may be completed by an integrated logic circuit of hardware in the processor 1302 or an instruction in a form of software.
- the above processing The device 1302 can be a general purpose processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), an off-the-shelf programmable gate array (FPGA), or other programmable logic device, discrete gate or transistor logic device, discrete hardware component.
- DSP digital signal processor
- ASIC application specific integrated circuit
- FPGA off-the-shelf programmable gate array
- the methods, steps, and logical block diagrams disclosed in the embodiments of the present invention may be implemented or carried out.
- the general purpose processor may be a microprocessor or the processor or any conventional processor or the like.
- the steps of the method disclosed in the embodiments of the present invention may be directly implemented by the hardware decoding processor, or may be performed by a combination of hardware and software modules in the decoding processor.
- the software module can be located in a conventional storage medium such as random access memory, flash memory, read only memory, programmable read only memory or electrically erasable programmable memory, registers, and the like.
- the storage medium is located in the memory 1304, and the processor 1302 reads the information in the memory 1304, and combines the hardware to complete the processing device 1100 of the information of the streaming media provided by the embodiment of the present invention or the module included in the processing device 1200 of the streaming media information.
- the function to be performed, or the processing method of the information of the streaming media corresponding to FIG. 8 or FIG. 13 provided by the embodiment of the method of the present invention.
- the memory 1304 may be a read only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). Memory 1304 can store operating systems as well as other applications.
- ROM read only memory
- RAM random access memory
- Memory 1304 can store operating systems as well as other applications.
- the processing code for executing the information of the technical solution provided by the embodiment of the present invention is stored in the memory 1304, and the processing device 1100 for executing the information of the streaming media by the processor 1302 Or an operation performed by a module included in the processing device 1200 for streaming information, or a method for generating a regression model for predicting a key performance indicator provided by an embodiment of the method of the present invention.
- the input/output interface 1306 is configured to receive input data and information, and output data such as operation results. It can be used as the acquisition module 1101 in the device 1100, or the acquisition module 1201 or the transmission module in the device 1200.
- Communication interface 1308 implements communication between computer device 1300 and other devices or communication networks using transceivers such as, but not limited to, transceivers. It can be used as the acquisition module 1101 in the device 1100, or the acquisition module 1201 or the transmission module in the device 1200.
- Bus 1310 can include a path for communicating information between various components of computer device 1300, such as processor 1302, memory 1304, input/output interface 1306, and communication interface 1308.
- the computer device 1300 shown in FIG. 16 only shows the processor 1302, the memory 1304, the input/output interface 1306, the communication interface 1308, and the bus 1310, those skilled in the art will understand in the specific implementation process.
- the computer device 1300 further includes other devices necessary for normal operation, for example, when the processing device 1100 implements information as streaming media, the processing device 1100 for streaming media information may further include a display for displaying the video to be played. data.
- computer device 1300 may also include hardware devices that implement other additional functions, depending on the particular needs.
- computer device 1300 may also only include the components necessary to implement embodiments of the present invention, and does not necessarily include all of the devices shown in FIG.
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- General Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Computer Graphics (AREA)
- Human Computer Interaction (AREA)
- Computer Hardware Design (AREA)
- Software Systems (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
- Information Transfer Between Computers (AREA)
Abstract
Description
Claims (15)
- 一种流媒体的信息的处理方法,其特征在于,所述方法包括:获取目标空间对象的目标空间信息,所述目标空间对象为两个空间对象中的空间对象,所述两个空间对象与目标视频数据中包括的两个图像的数据关联,所述目标空间信息包括同属性空间信息,所述同属性空间信息包括所述两个空间对象各自的空间信息之间相同的信息,所述两个空间对象中除所述目标空间对象外的其他空间对象的空间信息包括所述同属性空间信息;根据所述目标空间信息,确定要播放的视频数据。
- 根据权利要求1所述的方法,其特征在于,所述目标空间信息还包括所述目标空间对象的异属性空间信息,所述其他空间对象的空间信息还包括所述其他空间对象的异属性空间信息,所述目标空间对象的异属性空间信息与所述其他空间对象的异属性信息不同。
- 根据权利要求1或2所述的方法,其特征在于,所述目标空间信息包括所述目标空间对象的中心点的位置信息或者所述目标空间对象的左上点的位置信息,所述目标空间信息还包括所述目标空间对象的宽和所述目标空间对象的高。
- 根据权利要求1或2所述的方法,其特征在于,所述目标空间信息包括所述目标空间对象的左上点的位置信息,和所述目标空间对象的右下点的位置信息。
- 根据权利要求1至4任一项所述的方法,其特征在于,所述目标空间信息包括所述目标空间对象的空间旋转信息。
- 根据权利要求1至5任一项所述的方法,其特征在于,所述目标空间信息封装在空间信息数据或者空间信息轨迹(track)中,所述空间信息数据为所述目标视频数据的码流、所述目标视频数据的元数据或者独立于所述目标视频数据的文件,所述空间信息轨迹为独立于所述目标视频数据的轨迹。
- 根据权利要求6所述的方法,其特征在于,所述空间信息数据或者所述空间信息轨迹中还包括用于指示所述同属性空间信息的类型的空间信息类型标识,所述空间信息类型标识用于指示所述目标空间信息中属于所述同属性空间信息的信息。
- 根据权利要求7所述的方法,其特征在于,在所述空间信息类型标识指示目标空间信息中没有属于所述同属性空间信息的信息时,所述同属性空间信息包括所述目标空间对象的宽的最小值、所述目标空间对象的高的最小值、所述目标空间对象的宽的最大值和所述目标空间对象的高的最大值。
- 根据权利要求7或8所述的方法,其特征在于,所述空间信息类型标识与所述同属性空间信息封装在同一个box中。
- 根据权利要求6至9任一项所述的方法,其特征在于,所述空间信息数据或者所述空间信息轨迹中还包括用于指示所述目标空间信息对应的坐标系的坐标系标识,所述坐标系为像素坐标系或者角度坐标系。
- 根据权利要求10所述的方法,其特征在于,所述坐标系标识与所述同属性空间信息封装在同一个box中。
- 根据权利要求6至11任一项所述的方法,其特征在于,所述空间信息数据或者所述空间信息轨迹还包括空间旋转信息标识,所述空间旋转信息标识用于指示所述目标空间信息是否包括所述目标空间对象的空间旋转信息。
- 一种流媒体的信息的处理装置,其特征在于,所述装置包括:获取模块,用于获取目标空间对象的目标空间信息,所述目标空间对象为两个空间对象中的空间对象,所述两个空间对象与目标视频数据中包括的两个图像的数据关联,所述目标空间信息包括同属性空间信息,所述同属性空间信息包括所述两个空间对象各自的空间信息之间相同的信息,所述两个空间对象中除所述目标空间对象外的其他空间对象的空间信息包括所述同属性空间信息;确定模块,用于根据所述获取模块获取的所述目标空间信息,确定要播放的视频数据。
- 根据权利要求13所述的装置,其特征在于,所述目标空间信息还包括所述目标空间对象的异属性空间信息,所述其他空间对象的空间信息还包括所述其他空间对象的异属性空间信息,所述目标空间对象的异属性空间信息与所述其他空间对象的异属性信息不同。
- 根据权利要求13或14所述的装置,其特征在于,所述目标空间信息包括所述目标空间对象的中心点的位置信息或者所述目标空间对象的左上点的位置信息,所述目标空间信息还包括所述目标空间对象的宽和所述目标空间对象的高。
Priority Applications (9)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
MX2019007926A MX2019007926A (es) | 2016-12-30 | 2017-03-29 | Método y aparato de procesamiento de información. |
EP17888252.8A EP3557534A4 (en) | 2016-12-30 | 2017-03-29 | INFORMATION PROCESSING METHOD AND APPARATUS |
EP23172704.1A EP4287637A1 (en) | 2016-12-30 | 2017-03-29 | Information processing method and apparatus |
CN201780081717.5A CN110121734B (zh) | 2016-12-30 | 2017-03-29 | 一种信息的处理方法及装置 |
JP2019535932A JP7058273B2 (ja) | 2016-12-30 | 2017-03-29 | 情報処理方法および装置 |
BR112019013609A BR112019013609A8 (pt) | 2016-12-30 | 2017-03-29 | Método e aparelho de processamento de informação |
KR1020197022156A KR102261559B1 (ko) | 2016-12-30 | 2017-03-29 | 정보 처리 방법 및 장치 |
CN201710214606.9A CN108271044B (zh) | 2016-12-30 | 2017-04-01 | 一种信息的处理方法及装置 |
US16/458,734 US20190325652A1 (en) | 2016-12-30 | 2019-07-01 | Information Processing Method and Apparatus |
Applications Claiming Priority (4)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201611252815.4 | 2016-12-30 | ||
CN201611252815 | 2016-12-30 | ||
CNPCT/CN2017/070863 | 2017-01-11 | ||
PCT/CN2017/070863 WO2018120294A1 (zh) | 2016-12-30 | 2017-01-11 | 一种信息的处理方法及装置 |
Related Child Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US16/458,734 Continuation US20190325652A1 (en) | 2016-12-30 | 2019-07-01 | Information Processing Method and Apparatus |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2018120474A1 true WO2018120474A1 (zh) | 2018-07-05 |
Family
ID=62706718
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/CN2017/078585 WO2018120474A1 (zh) | 2016-12-30 | 2017-03-29 | 一种信息的处理方法及装置 |
Country Status (1)
Country | Link |
---|---|
WO (1) | WO2018120474A1 (zh) |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US11532128B2 (en) * | 2017-03-23 | 2022-12-20 | Qualcomm Incorporated | Advanced signaling of regions of interest in omnidirectional visual media |
Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20090132931A1 (en) * | 2007-11-15 | 2009-05-21 | International Business Machines Corporation | Method, device and program for automatically generating reference mark in virtual shared space |
CN102930047A (zh) * | 2012-11-15 | 2013-02-13 | 中国科学院深圳先进技术研究院 | 虚拟地球用户化身节点检索方法及*** |
CN103390057A (zh) * | 2013-07-26 | 2013-11-13 | 国家测绘地理信息局卫星测绘应用中心 | 一种历史信息的空间化建模存储方法 |
CN104735464A (zh) * | 2015-03-31 | 2015-06-24 | 华为技术有限公司 | 一种全景视频交互传输方法、服务器和客户端 |
CN105898254A (zh) * | 2016-05-17 | 2016-08-24 | 亿唐都科技(北京)有限公司 | 节省带宽的vr全景视频布局方法、装置及展现方法、*** |
-
2017
- 2017-03-29 WO PCT/CN2017/078585 patent/WO2018120474A1/zh unknown
Patent Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20090132931A1 (en) * | 2007-11-15 | 2009-05-21 | International Business Machines Corporation | Method, device and program for automatically generating reference mark in virtual shared space |
CN102930047A (zh) * | 2012-11-15 | 2013-02-13 | 中国科学院深圳先进技术研究院 | 虚拟地球用户化身节点检索方法及*** |
CN103390057A (zh) * | 2013-07-26 | 2013-11-13 | 国家测绘地理信息局卫星测绘应用中心 | 一种历史信息的空间化建模存储方法 |
CN104735464A (zh) * | 2015-03-31 | 2015-06-24 | 华为技术有限公司 | 一种全景视频交互传输方法、服务器和客户端 |
CN105898254A (zh) * | 2016-05-17 | 2016-08-24 | 亿唐都科技(北京)有限公司 | 节省带宽的vr全景视频布局方法、装置及展现方法、*** |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US11532128B2 (en) * | 2017-03-23 | 2022-12-20 | Qualcomm Incorporated | Advanced signaling of regions of interest in omnidirectional visual media |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
WO2018120294A1 (zh) | 一种信息的处理方法及装置 | |
WO2018214698A1 (zh) | 一种视频信息的呈现方法和装置 | |
KR102241082B1 (ko) | 복수의 뷰포인트들에 대한 메타데이터를 송수신하는 방법 및 장치 | |
WO2018058773A1 (zh) | 一种视频数据的处理方法及装置 | |
KR102305633B1 (ko) | 퀄리티 기반 360도 비디오를 송수신하는 방법 및 그 장치 | |
TWI670973B (zh) | 在iso基本媒體檔案格式推導虛擬實境投影、填充、感興趣區域及視埠相關軌跡並支援視埠滾動訊號之方法及裝置 | |
US20200389640A1 (en) | Method and device for transmitting 360-degree video by using metadata related to hotspot and roi | |
CN109644262A (zh) | 发送全向视频的方法、接收全向视频的方法、发送全向视频的装置和接收全向视频的装置 | |
CN109218755B (zh) | 一种媒体数据的处理方法和装置 | |
WO2018068213A1 (zh) | 一种视频数据的处理方法及装置 | |
WO2018126702A1 (zh) | 一种应用于虚拟现实技术的流媒体的传输方法和客户端 | |
CN107888993B (zh) | 一种视频数据的处理方法及装置 | |
US20210176446A1 (en) | Method and device for transmitting and receiving metadata about plurality of viewpoints | |
US20190199921A1 (en) | Method for transmitting 360-degree video, method for receiving 360-degree video, 360-degree video transmitting device, and 360-degree video receiving device | |
US20200145716A1 (en) | Media information processing method and apparatus | |
CN110798707B (zh) | 传输媒体数据的方法、客户端和服务器 | |
CN107959861B (zh) | 一种数据处理方法、相关设备及*** | |
WO2018058993A1 (zh) | 一种视频数据的处理方法及装置 | |
WO2018120474A1 (zh) | 一种信息的处理方法及装置 | |
CN116248947A (zh) | 处理媒体数据的方法、客户端和服务器 | |
KR20200008631A (ko) | 360도 비디오를 전송하는 방법, 360도 비디오를 수신하는 방법, 360도 비디오 전송 장치, 360도 비디오 수신 장치 | |
CN108271084B (zh) | 一种信息的处理方法及装置 | |
WO2023169003A1 (zh) | 点云媒体的解码方法、点云媒体的编码方法及装置 |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 17888252 Country of ref document: EP Kind code of ref document: A1 |
|
ENP | Entry into the national phase |
Ref document number: 2019535932 Country of ref document: JP Kind code of ref document: A |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
REG | Reference to national code |
Ref country code: BR Ref legal event code: B01A Ref document number: 112019013609 Country of ref document: BR |
|
ENP | Entry into the national phase |
Ref document number: 2017888252 Country of ref document: EP Effective date: 20190715 Ref document number: 20197022156 Country of ref document: KR Kind code of ref document: A |
|
ENP | Entry into the national phase |
Ref document number: 112019013609 Country of ref document: BR Kind code of ref document: A2 Effective date: 20190628 |