WO2011030234A1 - Recommended depth value for overlaying a graphics object on three-dimensional video - Google Patents
Recommended depth value for overlaying a graphics object on three-dimensional video Download PDFInfo
- Publication number
- WO2011030234A1 WO2011030234A1 PCT/IB2010/052664 IB2010052664W WO2011030234A1 WO 2011030234 A1 WO2011030234 A1 WO 2011030234A1 IB 2010052664 W IB2010052664 W IB 2010052664W WO 2011030234 A1 WO2011030234 A1 WO 2011030234A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- depth
- region
- depth value
- sequence
- recommended
- Prior art date
Links
- 238000000034 method Methods 0.000 claims abstract description 64
- 230000008569 process Effects 0.000 claims description 18
- 230000008859 change Effects 0.000 claims description 9
- 230000033001 locomotion Effects 0.000 description 19
- 238000000605 extraction Methods 0.000 description 10
- 238000004891 communication Methods 0.000 description 6
- 230000005540 biological transmission Effects 0.000 description 4
- 238000009877 rendering Methods 0.000 description 4
- 230000001360 synchronised effect Effects 0.000 description 4
- 239000000284 extract Substances 0.000 description 3
- 238000002372 labelling Methods 0.000 description 3
- 230000007246 mechanism Effects 0.000 description 3
- 238000004458 analytical method Methods 0.000 description 2
- 230000002452 interceptive effect Effects 0.000 description 2
- 230000000007 visual effect Effects 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 1
- 238000004590 computer program Methods 0.000 description 1
- 238000007796 conventional method Methods 0.000 description 1
- 230000001934 delay Effects 0.000 description 1
- 238000003708 edge detection Methods 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 238000000611 regression analysis Methods 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 238000000926 separation method Methods 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 230000002123 temporal effect Effects 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/50—Depth or shape recovery
- G06T7/55—Depth or shape recovery from multiple images
- G06T7/593—Depth or shape recovery from multiple images from stereo images
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
- H04N13/10—Processing, recording or transmission of stereoscopic or multi-view image signals
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
- H04N13/20—Image signal generators
- H04N13/293—Generating mixed stereoscopic images; Generating mixed monoscopic and stereoscopic images, e.g. a stereoscopic image overlay window on a monoscopic image background
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
- H04N13/30—Image reproducers
- H04N13/361—Reproducing mixed stereoscopic images; Reproducing mixed monoscopic and stereoscopic images, e.g. a stereoscopic image overlay window on a monoscopic image background
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N5/00—Details of television systems
- H04N5/222—Studio circuitry; Studio devices; Studio equipment
- H04N5/262—Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects
- H04N5/278—Subtitling
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/025—Systems for the transmission of digital non-picture data, e.g. of text during the active part of a television frame
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/08—Systems for the simultaneous or sequential transmission of more than one television signal, e.g. additional information signals, the signals occupying wholly or partially the same frequency band, e.g. by time division
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10004—Still image; Photographic image
- G06T2207/10012—Stereo images
Definitions
- the present invention relates to a method of generating a recommended depth value for use in displaying a graphics item over a three dimensional video; and to a method of operating a display device to display a graphics item over a three dimensional video.
- Graphics items such as subtitles and on-screen display messages (OSDs) are often overlayed on top of video.
- graphics items can simply be placed on top of the video. Whilst this might obscure video of interest, it does not generate any visual conflict.
- 3D three-dimensional
- Some graphics items include "x,y" locations for placement of the graphics item on
- 3D DVDs that are available today all place graphics items at a single, fixed depth (equivalent to the screen depth).
- Some 3D video coding mechanisms utilize a grey-scale depth map. This is generated from views taken from different camera angles, and provides depth information for 25 each pixel location, and is generated automatically by the encoding process.
- a method of generating a recommended depth value for use in displaying a graphics item over a three dimensional video including at a headend: receiving a three dimensional video including video frames; analyzing a sequence of said video frames of the three dimensional video in turn to produce a sequence of depth maps, each depth map in the sequence of depth maps being associated with timing data relating that depth map to a corresponding video frame in the sequence of video frames, each depth map including depth values, each depth value representing a depth of a pixel location in its corresponding video frame; selecting a region of depth maps in the sequence of depth maps; analyzing the region of the depth maps in the sequence to identify a furthest forward depth value for said region in the sequence of depth maps; and transmitting the furthest forward depth value as the recommended depth value for the region to a display device and region information describing the region.
- the method further includes: selecting at least one additional region of depth maps in the sequence of depth maps; analyzing the at least one additional region of the depth maps in the sequence to identify at least one additional furthest forward depth value for the at least one additional region; and transmitting the at least one additional furthest forward depth value as an additional recommended depth value for the at least one additional region and additional region information describing the at least one additional region.
- the method further includes: receiving details of the graphics item, the details comprising a two dimensional screen location where the graphics item is to be displayed and a size of the graphics item; and determining the region from the details of the graphics item.
- the method further includes: receiving a maximum depth value; comparing the furthest forward depth value with the maximum depth value; and transmitting the maximum depth value as the recommended depth value if the furthest forward depth value exceeds the maximum depth value, otherwise transmitting the furthest forward depth value as the recommended depth value.
- the method further includes: selecting an alternate region of depth maps in the sequence of depth maps; analyzing the alternate region of the depth maps in the sequence to identify an alternate furthest forward depth value for the alternate region; and transmitting the alternate furthest forward depth value as an alternate recommended depth value for the alternate region and alternate region information describing the alternate region.
- prediction of a change in depth value is used to identify the furthest forward depth.
- the recommended depth value is transmitted independently of the three dimensional video.
- a method of operating a display device to display a graphics item over a three dimensional video including: receiving the three dimensional video including video frames; receiving the graphics item; receiving a recommended depth value for use in displaying the graphics item over the three dimensional video; and displaying the graphics item over the three dimensional video at the recommended depth value, wherein the recommended depth value is generated according to a recommended depth value generation process executed at a headend, the recommended depth value generation process including: analyzing a sequence of the video frames in turn to produce a sequence of depth maps, each depth map in the sequence of depth maps being associated with timing data relating that depth map to a corresponding video frame in the sequence of video frames, each depth map including depth values, each depth value representing a depth of a pixel location in its corresponding frame; selecting a region of depth maps in the sequence of depth maps; analyzing the region of the depth maps in the sequence of depth maps to identify a furthest forward depth value for the region in the sequence of depth
- the recommended depth value generation process further includes at the headend: selecting an alternate region of depth maps in the sequence of depth maps; analyzing the alternate region of the depth maps in the sequence to identify an alternate furthest forward depth value for the alternate region; and transmitting the alternate furthest forward depth value as an alternate recommended depth value for the alternate region and alternate region information describing the alternate region to the display device; the method further including displaying the graphics item in a region of the three dimensional video described by the region information or in an alternate region of the three dimensional video described by the alternate region information.
- the determination is based on information received from a user.
- the method further includes: comparing the recommended depth value with a maximum depth value; and causing the graphics item to disappear if the recommended depth value exceeds the maximum depth value.
- the method further includes: comparing the recommended depth value and the alternate recommended depth value with a maximum depth value; and displaying the graphics item in the alternate region if the recommended depth value exceeds the maximum depth value but does not exceed the alternate recommended depth value.
- the recommended depth value generation process further includes at the headend: selecting at least one additional region of depth maps in the sequence of depth maps; analyzing the at least one additional region of the depth maps in the sub-sequence to identify at least one additional furthest forward depth value for the at least one additional region; and transmitting the at least one additional furthest forward depth value as an additional recommended depth value for the at least one additional region and additional region information describing the at least one additional region; the method further including: displaying the graphics item at a furthest forward depth value of the recommended depth value and the at least one additional recommended depth value.
- Fig. 1 is a simplified pictorial illustration of a system constructed and operative in accordance with embodiments of the present invention
- Fig. 2 is a flow chart of process executed by the processing module of fig. 1, according to embodiments of the present invention.
- an input stream of 3D video data 101 is received at a headend, typically in an uncompressed format.
- the 3D video can be received as two separate streams of frame accurately synchronized 2D video data, one for the left-eye image and one for the right-eye image, which can then be combined to form 3D video; or can be received as a single stream where the left-eye and right-eye images are encoded as the single stream.
- Different formats can be used to place the left-eye and right-eye images into a single stream.
- Each frame of the 3D video data is associated with a time code that provides a time reference for identification and synchronization.
- each frame may be associated with a SMPTE timecode (as defined in the SMPTE 12M specification of the Society of Motion Picture and Television Engineers (SMPTE)).
- the SMPTE time codes may in turn be mapped to MPEG presentation time stamps (PTS) (as defined in International Standard ISO/IEC 13818-1, Information technology— Generic coding of moving pictures and associated audio information: Systems - a metadata field in an MPEG-2 Transport Stream or Program Stream that is used to achieve synchronization of programs' separate elementary streams (for example video, audio, subtitles) when presented to the viewer) - using synchronized auxiliary data (as defined in ETSI TS 102 823 Digital Video Broadcasting (DVB); Specification for the carriage of synchronized auxiliary data in DVB transport streams.)
- each frame may be associated with an MPEG PTS.
- the 3D video data may also be accompanied by camera metadata defining parameters of the camera(s) used to record the 3D video such as camera location, lens parameters, f-stop settings, focal distance, auto-focus information etc., and stereo camera parameters such as the relative separations and angle of the stereo cameras.)
- the 3D video data and optional camera metadata is passed to depth extraction module 103, which is operable to produce a depth map for each frame of the 3D video data according to any known depth extraction technique (e.g. as described in "Depth Extraction System Using Stereo Pairs ", Ghaffar, R. et al., Lecture Notes in Computer Science, Springer Berlin/Heidelberg, ISSN 0302-9743, Volume 3212/2004).
- a depth map typically provides depth values (typically 24 or 32 bit) for pixel locations of a corresponding frame of 3D video.
- a depth value conveys the extracted estimate of the depth (or distance from the camera) of a pixel location in the 3D video frame.
- Each depth map is associated with timing data (e.g. SMPTE time code or MPEG PTS, as described previously) used to synchronize a depth map with the frame of 3D video from which it was generated.
- a depth map produced for each frame of the 3D video data represents a large amount of information.
- the depth maps and timing data 105 are stored in a memory (not shown) and then passed to processing module 107, together with processing parameters 1 17, when required for processing.
- Processing parameters 1 17 enable the temporal and/or spatial filtering of the depth maps so that the quantity of data being processed can be reduced.
- the depth maps can be temporally filtered by processing selected sequences of depth maps corresponding to a selected sequence of the 3D video (e.g. if a graphics item is only to be displayed for a limited period of time).
- the depth maps can be spatially filtered by processing selected regions of depth maps (e.g. if a graphics item only overlays part of the 3D video). Examples of typical processing parameters 1 17 include:
- Frame sequence specifies a sequence of depth maps (associated with a sequence of frames of 3D video) that are to be processed to determine a recommended graphics item depth.
- the frame sequence parameter typically comprises an identification of the start frame in the sequence; and either a sequence duration or an identification of the end frame in the sequence.
- the sequence of depth maps can be identified and extracted using the timing data that accompanied the depth maps;
- Region location and size specifies a region of the depth map to be processed corresponding to a region of the 3D video that a particular graphics item will overlay.
- Location is typically specified using an (x,y) coordinate location (e.g. where x and y can be specified using pixels; or using numbers in the range 0 to 1 (if the origin (0,0) is defined as being at the bottom left of the screen); or using numbers in the range -1 to +1 (of the origin (0,0) is defined as being at the centre of the screen) or using percentage ranges etc.).
- Size both x and y components
- Region tag specifies a name for the region and allows multiple regions to be specified and processed;
- Alternate region specifies one or more alternate region(s) (typically by location and size) that could be used for the graphics (e.g. to allow display device 1 1 1 or a user of display device 1 1 1 to configure which region to use for a particular graphics item);
- Region movement specifies how far a region defined by the region location and size parameters can move (typically translational movement) in order to avoid a depth conflict (e.g. allowing a region for a particular graphics item to move to the side or up/down if the recommended depth exceeds a maximum value (see below for details of the depth parameter); or if an object in the 3D video moves in front of the graphics item).
- Region movement (both x and y components) is typically specified using the same system as that used for location.
- a maximum limit for how fast the region can move may also be specified. This is typically specified as a maximum change of x and/or y component per frame.
- the second order differential i.e. a maximum change of change of x and/or y component per frame) could also be specified;
- Region size adjustment specifies how much a region defined by the region location and size parameters can change in size as an alternative (or as well as) moving in space and/or depth (see below for details of the depth parameter). Size adjustment (x and/or y components) is typically specified using the same system as that used for location;
- Depth used to specify a backward and forward depth for a region defined by the region location and size parameters.
- Backward depth is typically defined as depth into the screen, backwards and away from a viewer.
- Forward depth is typically defined as depth out of the screen, forwards and towards a viewer.
- Depth (typically a z-component orthogonal to the x and y components) is typically specified using the same system as that used for location. Multiple depth parameters could be set in order to specify a 'preferred' depth and one or more 'alternative' depths. Maximum depth limits can also be specified. Setting a maximum backward depth ensures that the graphics are not placed too far backward (e.g. behind an acceptable value when there is no 3D video behind the graphics item).
- a maximum forward depth ensures that the graphics are not placed too far forward (e.g. so far forward that the graphics item would be uncomfortable to view at a normal viewing distance). If during processing a calculated depth exceeds a specified depth limit, then the region may be moved (typically according to the region movement and/or region size adjustment parameters) or may be temporarily deleted; • Depth movement: specifies a maximum limit for how fast a region defined by the region location and size parameters can move in depth. This is typically specified as a maximum change of z component per frame. The second order differential (i.e. a maximum change of change of z component per frame) could also be specified. Setting the depth movement parameter is useful to prevent a graphics item from oscillating in response to the changes in the 3D video content it is overlaying;
- Depth jump a flag specifying that, if the depth movement limit is exceeded, a region is moved to a different location (e.g. one of the regions specified by the alternate region parameter);
- Depth Extraction Interface Tags some depth extraction systems are able to perform object tracking/face recognition.
- the system may receive additional camera metadata derived, for example, from object tracking and giving specific object screen locations.
- a depth extraction interface tag would enable a region to be described with additional information which would link it with information returned by the depth extraction system.
- Object extraction systems typically provide identifiers for objects that are being tracked. Human interaction may be used to resolve the identifiers to a suitable interface tag as described above, or the system may automatically provide a known interface tag value(s) or name(s) that the depth extraction module and processing module are configured with;
- Minimum Clearance used to provide the minimum "clearance” between the overlaid graphic and the most forward underlying object in the 3D video.
- Processing module 107 processes depth maps in order to extract information that is used to place graphics in the 3D video and convey the information in an effective format to display device 1 1 1. This involves indentifying areas/regions within 3D video frame sequences that will be used for graphics placement and a recommended depth at which graphics could be placed.
- the output of processing module 107 is one or more streams of processed depth data and associated timing data 109 that defines a recommended depth at which to place graphics for one or more frame sequences of the 3D video; or one or more regions of the 3D video.
- Graphics items typically include subtitles; closed captions; asynchronous messages (e.g. event notifications, tuner conflict notifications); emergency announcements; logo bugs; interactive graphics applications; information banners; etc.
- the processed depth data and timing data 109 is transmitted from the headend to a display device 1 1 1 via a network 113.
- the processed depth data and timing data 109 can be compressed prior to transmission.
- Display device 1 1 1 is typically an integrated receiver decoder (IRD); set top box (STB), digital video recorder (DVR) etc.
- Network 1 13 is typically a one-way or two-way communication network, e.g. one or more of a satellite based communication network; a cable based communication network; a terrestrial broadcast television network; a telephony based communication network; a mobile telephony based communication network; an Internet Protocol (IP) television broadcast network; a computer based communication network; etc.
- IP Internet Protocol
- the graphics data and the 3D video data 1 15 are also transmitted from the headend to display device 1 1 1 via network 1 13.
- the graphics data and the 3D video data 1 15 can be transmitted separately from the processed depth data and associated timing data 109.
- the graphics data and the 3D video data 1 15 are compressed prior to transmission.
- Display device 1 1 1 receives the processed depth data and timing data 109 and the 3D video and graphics data 115.
- the graphics data is provided to a rendering engine within display device 1 1 1 together with the processed depth data.
- the rendering engine then generates the appropriate images and combines them with the 3D video at the recommended depth indicated by the processed depth data using standard combination techniques. It will be remembered that the processed depth data is accompanied by timing data allowing it to be associated and synchronized with the 3D video.
- display device 1 1 1 1 identifies a conflict between the event currently be viewed and an event to be recorded.
- display device 1 1 1 consults the processed depth data and timing data 109 and locates the region associated with the OSD message class for the tuner conflict. This provides display device 1 1 1 with screen location and depth information. This depth information is then provided to the graphics rendering software and hardware together with the graphic to be displayed.
- the graphics rendering software converts the graphics and location and depth information into images that are perceived by the viewer as being placed at the intended depth and location.
- processing module 107 processes depth maps in order to extract information from the depth maps that is used to place one or more graphics items in the 3D video and convey the information in an effective format to display device 1 1 1.
- This information is typically includes a recommended depth at which to place each graphics item and information about the region that the recommended depth relates to.
- the region information can be in the form of a region tag (as mentioned previously); or a screen area (defined by region size and location as mentioned previously; or by a vector outline). Other ways of describing a region will be apparent to someone skilled in the art.
- a first stage the details of a graphics item are passed to processing module 107. These details include the screen location of the graphics item, the size of the graphics item and details of which frames of the 3D video the graphics item will appear over.
- processing module 107 uses the details of the graphics item to derive the frame sequence, region location and size processing parameters.
- processing module 107 may not derive the region size and location parameters (e.g. if details of the graphics item were not provided). In such a case, processing module 107 would use the entire depth map rather than just a region of the depth map.
- processing module 107 may alternatively or additionally use the depth maps of all the frames of the 3D video in the processing rather than just the depth maps from a sequence of frames.
- processing module 107 extracts the appropriate depth maps for the (derived) frame sequence and analyses the (derived) region of the extracted depth maps (step 207) in order to calculate a furthest forward depth of pixels in the 3D video where the graphics item is to be placed so as not to conflict with 3D video image. That is, if the graphics item was placed behind (or backwards from) the furthest forward depth, a depth disparity may occur (e.g.
- calculating the furthest forward depth involves calculating the depth of pixels in the derived region for frames in the sequence and extracting the furthest forward depth identified in the calculation for use as the recommended depth for the graphics item.
- processing module 107 packages the processed depth data (including the recommended depth for a graphics item overlaying that region; and the region location and size if appropriate) together with the timing data for the processed depth data and transmits it to display device 1 1 1 (step 211).
- the above described process is suitable, for example, for calculating the recommended depth for subtitles to accompany the 3D video.
- the 3D video scene may have a static depth throughout the video (e.g. a newsreader sitting at a desk).
- the graphics item can be placed in the 3D video without changing in depth on a frame by frame basis.
- processing module 107 may extract all depth maps for the
- each subtitle to be displayed could be considered as a separate graphics item, each associated with a defined sequence of frames of the 3D video. Details of the screen location and size for each subtitle will typically already be known from a subtitle generation process but subtitling of live content (e.g. news) can also be handled. These details can be passed to processing module 107 in step 201.
- processing module 107 can be provided to processing module 107 for processing module 107 to use when comparing depth maps to calculate the processed depth data.
- a depth limit maximum depth value
- the (x,y) positioning of the graphics item could be controlled so that the graphics item could be moved and/or resized according to those parameters in order to find a screen location for the graphics item where the recommended depth (which will be recalculated for the new region location and/or size) is within the depth limit.
- the depth movement parameter could be used to enable the recommended depth to vary across a sequence of frames.
- a subtitle could move slowly towards the camera during its display, minimizing its overall movement and speed of movement whilst also avoiding conflict with any object in the video.
- other information relevant to the placement of the graphics item can also be included as part of the processed depth data. For example, it may be possible to place the graphics item in a different screen location (identified by the alternate region processing parameter) in which case the recommended depth for the graphics item for the alternate region can also be calculated and included in the processed depth data.
- object tracking functionality may be used to track objects in the 3D video. It may be desirable to include a graphics item associated with an object that is being tracked (e.g. labeling the ball as such in a fast moving game such as a golf or hockey; or labeling a player with their name).
- the output of object tracking could be provided to the processing module 107 as the region location parameter (with a suitable offset to avoid conflicting with the actual object being tracked, e.g. using the region movement parameter) enabling the depths for a graphics item labeling the tracked object to be calculated.
- the depth extraction and processing take place at the time the 3D video is ingested into the headend. This is often significantly in advance of the transmission.
- the processed depth data can be provided in advance of the transmission of the 3D video.
- the processed depth data can be provided at the same time as the 3D video.
- the processed depth data is typically provided as a separate stream of data to the 3D video data.
- processing module 107 processes the depth maps in real time. Due to the normal encoding delays at the headend, at least a second (and sometimes up to ten seconds) of the 3D video can be analyzed in order to calculate the processed depth data. Well known techniques such as regression analysis can be used to extrapolate depth motions of objects in the 3D video beyond those obtained from the analyzed depth maps and the extrapolated depth motions used to when calculating the recommended depths.
- the processed depth data is transmitted to display device 1 1 1 at the same time as the 3D video, typically once every second. However, in alternative embodiments the processed depth data may only be sent at less-regular intervals (e.g. every 5s) with a check being performed more regularly (e.g. every Is) to see if the recommended depth has changed. If it has changed then it can be transmitted immediately.
- display device 1 1 1 may be able to adapt where graphics items are placed based on user set preferences. For example, a user may be able to set a user preferred depth for graphics items that overrides the originally received recommended depth (assuming no conflict is indicated (e.g. where the user preferred depth is further forward than the transmitted recommended depth)).
- Display device 1 1 1 1 may also be configured to make certain graphics items disappear if they conflict with other graphics items, or to correctly combine the graphics items at their relative depths if transparency options make such an alternative possible.
- the display device 1 1 1 may also be configured with different depth movement and/or region movement parameters.
- the head-end may employ one of several alternative methods for region identification:
- ⁇ a-priori defined screen areas e.g. corresponding to known graphics assets
- edge detection arbitrary identified regions of common depth or bounded by such mechanisms as edge detection.
- a region can be described using one or more of a range of methods depending on the expected usage of the region. These can include a region tag that is used by display device 1 11, e.g. "N " to indicate a Now-Next banner region, or a screen area from regular shapes such as a square or rectangle to complex shapes defined using a vector outline. As such, each region could include a range of descriptive information.
- display device 1 1 1 1 may receive details of multiple regions (each with a recommended depth), and then, using information about the graphics item(s), identify the region(s) that the graphics item(s) will overlap with. Display device 1 1 1 would then choose an appropriate depth based on an analysis of the recommended depth for all regions. For example, display device 1 1 1 may use the graphics item to identify the screen area that the graphics item will cover, and then consult the set of regions. Display device 1 11 would then identify which region(s) is/are covered by the graphics item, and from this identification process extract the appropriate depth positioning information. Typically, the region(s) include descriptions that allow their screen location/area to be identified.
- a 'near plane' and a 'far plane' can be defined for graphics items.
- a 'near plane' is the nearest plane to a user viewing a three dimensional video (i.e. out of the screen) in which to place a graphics item.
- a 'far plane' is the furthest plane from a user viewing a three dimensional video (i.e. into the screen) in which to place a graphics item.
- software components of the present invention may, if desired, be implemented in ROM (read only memory) form.
- the software components may, generally, be implemented in hardware, if desired, using conventional techniques. It is further appreciated that the software components may be instantiated, for example: as a computer program product; on a tangible medium; or as a signal interpretable by an appropriate computer.
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Testing, Inspecting, Measuring Of Stereoscopic Televisions And Televisions (AREA)
- Processing Or Creating Images (AREA)
Abstract
Description
Claims
Priority Applications (4)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
EP10731579.8A EP2462736B8 (en) | 2009-09-08 | 2010-06-15 | Recommended depth value for overlaying a graphics object on three-dimensional video |
KR1020127009007A KR101210315B1 (en) | 2009-09-08 | 2010-06-15 | Recommended depth value for overlaying a graphics object on three-dimensional video |
US13/394,689 US20120218256A1 (en) | 2009-09-08 | 2010-06-15 | Recommended depth value for overlaying a graphics object on three-dimensional video |
IL218246A IL218246A0 (en) | 2009-09-08 | 2012-02-22 | Rrecommended depth value for overlaying a graphics object on three-dimensional video |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
GB0915658.9 | 2009-09-08 | ||
GB0915658A GB2473282B (en) | 2009-09-08 | 2009-09-08 | Recommended depth value |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2011030234A1 true WO2011030234A1 (en) | 2011-03-17 |
Family
ID=41203354
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/IB2010/052664 WO2011030234A1 (en) | 2009-09-08 | 2010-06-15 | Recommended depth value for overlaying a graphics object on three-dimensional video |
Country Status (6)
Country | Link |
---|---|
US (1) | US20120218256A1 (en) |
EP (1) | EP2462736B8 (en) |
KR (1) | KR101210315B1 (en) |
GB (1) | GB2473282B (en) |
IL (1) | IL218246A0 (en) |
WO (1) | WO2011030234A1 (en) |
Cited By (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20130342526A1 (en) * | 2012-06-26 | 2013-12-26 | Yi-Ren Ng | Depth-assigned content for depth-enhanced pictures |
EP2538683A3 (en) * | 2011-06-21 | 2014-06-11 | LG Electronics Inc. | Method and apparatus for processing broadcast signal for 3-dimensional broadcast service |
US20150062296A1 (en) * | 2012-04-13 | 2015-03-05 | Koninklijke Philips N.V. | Depth signaling data |
US10091495B2 (en) | 2012-12-24 | 2018-10-02 | Thomson Licensing | Apparatus and method for displaying stereoscopic images |
US10129524B2 (en) | 2012-06-26 | 2018-11-13 | Google Llc | Depth-assigned content for depth-enhanced virtual reality images |
US11328446B2 (en) | 2015-04-15 | 2022-05-10 | Google Llc | Combining light-field data with active depth data for depth map generation |
Families Citing this family (17)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN102860019B (en) * | 2010-04-28 | 2015-07-29 | 富士胶片株式会社 | Stereo-picture regenerating unit and method, stereo photographic device, stereoscopic display device |
KR101809479B1 (en) * | 2010-07-21 | 2017-12-15 | 삼성전자주식회사 | Apparatus for Reproducing 3D Contents and Method thereof |
JP5438082B2 (en) * | 2010-11-16 | 2014-03-12 | パナソニック株式会社 | Display device and display method |
WO2012150100A1 (en) * | 2011-05-02 | 2012-11-08 | Thomson Licensing | Smart stereo graphics inserter for consumer devices |
US20120293636A1 (en) * | 2011-05-19 | 2012-11-22 | Comcast Cable Communications, Llc | Automatic 3-Dimensional Z-Axis Settings |
US9113043B1 (en) * | 2011-10-24 | 2015-08-18 | Disney Enterprises, Inc. | Multi-perspective stereoscopy from light fields |
US9165401B1 (en) | 2011-10-24 | 2015-10-20 | Disney Enterprises, Inc. | Multi-perspective stereoscopy from light fields |
CN103416069A (en) * | 2012-01-13 | 2013-11-27 | 索尼公司 | Transmission device, transmission method, receiving device, and receiving method |
US8881209B2 (en) * | 2012-10-26 | 2014-11-04 | Mobitv, Inc. | Feedback loop content recommendation |
US9330171B1 (en) * | 2013-10-17 | 2016-05-03 | Google Inc. | Video annotation using deep network architectures |
GB2553782B (en) * | 2016-09-12 | 2021-10-20 | Niantic Inc | Predicting depth from image data using a statistical model |
CN108965929B (en) * | 2017-05-23 | 2021-10-15 | 华为技术有限公司 | Video information presentation method, video information presentation client and video information presentation device |
US10902825B2 (en) * | 2018-12-21 | 2021-01-26 | Arris Enterprises Llc | System and method for pre-filtering crawling overlay elements for display with reduced real-time processing demands |
US11238604B1 (en) * | 2019-03-05 | 2022-02-01 | Apple Inc. | Densifying sparse depth maps |
CN110990610B (en) * | 2019-11-28 | 2023-04-21 | 北京中网易企秀科技有限公司 | Recommendation method and system for data objects |
GB202104554D0 (en) | 2021-03-31 | 2021-05-12 | British Telecomm | Auto safe zone detection |
CN113538551B (en) * | 2021-07-12 | 2023-08-15 | Oppo广东移动通信有限公司 | Depth map generation method and device and electronic equipment |
Citations (9)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
EP0735784A2 (en) * | 1995-03-29 | 1996-10-02 | Sanyo Electric Co. Ltd | Three-dimensional image display device |
EP0905988A1 (en) * | 1997-09-30 | 1999-03-31 | Kabushiki Kaisha Toshiba | Three-dimensional image display apparatus |
JP2004274125A (en) | 2003-03-05 | 2004-09-30 | Sony Corp | Image processing apparatus and method |
WO2006111893A1 (en) * | 2005-04-19 | 2006-10-26 | Koninklijke Philips Electronics N.V. | Depth perception |
WO2008038205A2 (en) | 2006-09-28 | 2008-04-03 | Koninklijke Philips Electronics N.V. | 3 menu display |
WO2008044191A2 (en) | 2006-10-11 | 2008-04-17 | Koninklijke Philips Electronics N.V. | Creating three dimensional graphics data |
WO2008115222A1 (en) | 2007-03-16 | 2008-09-25 | Thomson Licensing | System and method for combining text with three-dimensional content |
US20090027549A1 (en) | 2004-05-17 | 2009-01-29 | Weisgerber Robert C | Method for processing motion pictures at high frame rates with improved temporal and spatial resolution, resulting in improved audience perception of dimensionality in 2-D and 3-D presentation |
WO2009083885A1 (en) * | 2007-12-26 | 2009-07-09 | Koninklijke Philips Electronics N.V. | Image processor for overlaying a graphics object |
-
2009
- 2009-09-08 GB GB0915658A patent/GB2473282B/en not_active Expired - Fee Related
-
2010
- 2010-06-15 WO PCT/IB2010/052664 patent/WO2011030234A1/en active Application Filing
- 2010-06-15 US US13/394,689 patent/US20120218256A1/en not_active Abandoned
- 2010-06-15 EP EP10731579.8A patent/EP2462736B8/en not_active Not-in-force
- 2010-06-15 KR KR1020127009007A patent/KR101210315B1/en active IP Right Grant
-
2012
- 2012-02-22 IL IL218246A patent/IL218246A0/en unknown
Patent Citations (10)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
EP0735784A2 (en) * | 1995-03-29 | 1996-10-02 | Sanyo Electric Co. Ltd | Three-dimensional image display device |
EP0905988A1 (en) * | 1997-09-30 | 1999-03-31 | Kabushiki Kaisha Toshiba | Three-dimensional image display apparatus |
JP2004274125A (en) | 2003-03-05 | 2004-09-30 | Sony Corp | Image processing apparatus and method |
US20090027549A1 (en) | 2004-05-17 | 2009-01-29 | Weisgerber Robert C | Method for processing motion pictures at high frame rates with improved temporal and spatial resolution, resulting in improved audience perception of dimensionality in 2-D and 3-D presentation |
WO2006111893A1 (en) * | 2005-04-19 | 2006-10-26 | Koninklijke Philips Electronics N.V. | Depth perception |
US20080192067A1 (en) | 2005-04-19 | 2008-08-14 | Koninklijke Philips Electronics, N.V. | Depth Perception |
WO2008038205A2 (en) | 2006-09-28 | 2008-04-03 | Koninklijke Philips Electronics N.V. | 3 menu display |
WO2008044191A2 (en) | 2006-10-11 | 2008-04-17 | Koninklijke Philips Electronics N.V. | Creating three dimensional graphics data |
WO2008115222A1 (en) | 2007-03-16 | 2008-09-25 | Thomson Licensing | System and method for combining text with three-dimensional content |
WO2009083885A1 (en) * | 2007-12-26 | 2009-07-09 | Koninklijke Philips Electronics N.V. | Image processor for overlaying a graphics object |
Cited By (8)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
EP2538683A3 (en) * | 2011-06-21 | 2014-06-11 | LG Electronics Inc. | Method and apparatus for processing broadcast signal for 3-dimensional broadcast service |
US9445077B2 (en) | 2011-06-21 | 2016-09-13 | Lg Electronics Inc. | Method and apparatus for processing broadcast signal for 3-dimensional broadcast service |
US20150062296A1 (en) * | 2012-04-13 | 2015-03-05 | Koninklijke Philips N.V. | Depth signaling data |
US20130342526A1 (en) * | 2012-06-26 | 2013-12-26 | Yi-Ren Ng | Depth-assigned content for depth-enhanced pictures |
US9607424B2 (en) * | 2012-06-26 | 2017-03-28 | Lytro, Inc. | Depth-assigned content for depth-enhanced pictures |
US10129524B2 (en) | 2012-06-26 | 2018-11-13 | Google Llc | Depth-assigned content for depth-enhanced virtual reality images |
US10091495B2 (en) | 2012-12-24 | 2018-10-02 | Thomson Licensing | Apparatus and method for displaying stereoscopic images |
US11328446B2 (en) | 2015-04-15 | 2022-05-10 | Google Llc | Combining light-field data with active depth data for depth map generation |
Also Published As
Publication number | Publication date |
---|---|
GB2473282B (en) | 2011-10-12 |
GB2473282A (en) | 2011-03-09 |
KR20120039767A (en) | 2012-04-25 |
KR101210315B1 (en) | 2012-12-11 |
EP2462736B8 (en) | 2013-12-18 |
EP2462736B1 (en) | 2013-10-30 |
IL218246A0 (en) | 2012-04-30 |
US20120218256A1 (en) | 2012-08-30 |
EP2462736A1 (en) | 2012-06-13 |
GB0915658D0 (en) | 2009-10-07 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
EP2462736B1 (en) | Recommended depth value for overlaying a graphics object on three-dimensional video | |
US10390000B2 (en) | Systems and methods for providing closed captioning in three-dimensional imagery | |
KR101716636B1 (en) | Combining 3d video and auxiliary data | |
JP5429034B2 (en) | Stereo image data transmitting apparatus, stereo image data transmitting method, stereo image data receiving apparatus, and stereo image data receiving method | |
EP1949689B1 (en) | Digital video zooming system | |
US8872976B2 (en) | Identification of 3D format and graphics rendering on 3D displays | |
US20120050154A1 (en) | Method and system for providing 3d user interface in 3d televisions | |
JP2013517677A (en) | System and method for compositing 3D text with 3D content | |
US20130182071A1 (en) | 3d video image encoding apparatus, decoding apparatus and method | |
US10057559B2 (en) | Transferring of 3D image data | |
JP2011146831A (en) | Video processing apparatus, method of processing video, and computer program | |
US20120154382A1 (en) | Image processing apparatus and image processing method | |
JP6391629B2 (en) | System and method for compositing 3D text with 3D content |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 10731579 Country of ref document: EP Kind code of ref document: A1 |
|
WWE | Wipo information: entry into national phase |
Ref document number: 218246 Country of ref document: IL |
|
WWE | Wipo information: entry into national phase |
Ref document number: 2010731579 Country of ref document: EP |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
ENP | Entry into the national phase |
Ref document number: 20127009007 Country of ref document: KR Kind code of ref document: A |
|
WWE | Wipo information: entry into national phase |
Ref document number: 13394689 Country of ref document: US |