WO2011032642A1 - Verfahren und vorrichtung zum erzeugen von teilansichten und/oder einer raumbildvorlage aus einer 2d-ansicht für eine stereoskopische wiedergabe - Google Patents
Verfahren und vorrichtung zum erzeugen von teilansichten und/oder einer raumbildvorlage aus einer 2d-ansicht für eine stereoskopische wiedergabe Download PDFInfo
- Publication number
- WO2011032642A1 WO2011032642A1 PCT/EP2010/005335 EP2010005335W WO2011032642A1 WO 2011032642 A1 WO2011032642 A1 WO 2011032642A1 EP 2010005335 W EP2010005335 W EP 2010005335W WO 2011032642 A1 WO2011032642 A1 WO 2011032642A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- pixel
- pixels
- view
- control value
- original
- Prior art date
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
- H04N13/20—Image signal generators
- H04N13/261—Image signal generators with monoscopic-to-stereoscopic image conversion
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
- H04N13/20—Image signal generators
- H04N13/257—Colour aspects
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
- H04N13/20—Image signal generators
- H04N13/275—Image signal generators from 3D object models, e.g. computer-generated stereoscopic image signals
Definitions
- the invention relates to a method and a device for generating partial views and optionally a spatial image template for a stereoscopic reproduction via an autostereoscopic reproduction device.
- a two-dimensional image which is an original view and images an origin scene from a viewing angle
- several further views which are referred to as partial views, are derived, which represent the original scene from other viewing angles.
- a spatial image template is created which generates a three-dimensional visual impression of the original scene in a reproduction via the autostereoscopic display device in a human observer.
- Illustrations of the real, three-dimensional world, as well as mathematically or artistically generated three-dimensional scenes and objects have always been depicted on two-dimensional surfaces and presented in the form of a projection.
- the depth information is lost in this case, but man already learns in early childhood to reconstruct part of this information based on his experience and object knowledge.
- the real three-dimensional world can be detected stereoscopically.
- the same scene is detected by means of at least two detection devices from different viewing directions or viewing angles.
- the detection must be done simultaneously for moving objects in the scene.
- the detection can be done, for example, photographically by film or digitally by means of a video camera, a CCD, an electronic camera or the like.
- the different views can also be generated computationally or artificially, whereby the scenes and objects are detected from at least two directions and the resulting images are subsequently made visible to the left and right human eye in a suitable manner.
- the human visual apparatus is no longer able to perceive the individual images separately, so that the result is again the impression of a three-dimensional overall image, although the two eyes are supplied with different image information (views) at different times.
- the methods operating on the basis of the various spectacles mentioned have the advantage that even a number of spectacles-equipped observers can at the same time see the displayed image information in an adequate manner, the use of spectacles being subjectively often perceived as a disadvantage.
- the aforementioned methods for generating three-dimensional image impressions in humans presuppose that in each case there are at least two different views of the represented scene or of the objects which have been acquired or calculated from different directions.
- multi-camera systems and corresponding image storage and transfer means are required.
- this task is usually relatively easy to solve by rendering two views of the scenes with different virtual camera positions.
- the determined segments are assigned to individual depth levels; a continuous, continuous depth map for the output image can not be generated. Further, highlighting or dropping in the background of individual image segments is explicitly intended to interprete the image content, i. bound to the detected objects in the scene. Thus, DE 103 48 618 A1 expressly emphasizes the content-dependent segmentation method. Automated recognition of this image content is possible, if at all, only in special cases.
- the object of the invention is therefore to provide a method and an apparatus which automatically generate from a two-dimensional (2D) view of an originating scene further views of the originating scene, representing the scene from different viewing angles or viewing directions and from this, if necessary, combine a spatial image template that at a reproduction via a stereoscopic or autostereoscopic display device as realistic as possible three-dimensional visual impression is caused in a viewer.
- the method and apparatus are preferably designed to be able to automatically process individual two-dimensional views of a sequence in real time, ie. without requiring human intervention during processing.
- the invention is based on the finding that the receptors for different color stimuli on the human retina are not evenly distributed. Therefore, different shades have a different influence on the spatial vision of humans.
- a method is proposed for generating image data for a reproduction that provides a three-dimensional visual impression in a human Obtaining viewer, comprising the steps of receiving and / or reading in 2D image data of an original view, deriving from several other views, representing the information shown in the original view, which are referred to as the original scene, from different viewing angles. These views, which represent the originating scene from different viewing directions or viewing angles, are also referred to below as partial views.
- color values of at least two primary colors of a color system are determined for each pixel of the 2D image data of the original view, each of the pixels is automatically assigned a control value as a function of the at least two determined color values, each of the control values representing disparity information, and during deriving of the plurality of views representing different viewing directions of an original scene represented by the original image 2D image data, the individual pixels of the original view are shifted pixel-by-pixel relative to each other in accordance with a shift vector given to the respective partial view and the control value associated with the single pixel, and from the shifted pixels Origin view the pixels of the respective derived partial view are derived.
- Disparity or disparity information is understood here as a measure which indicates how large an offset of the imaging positions of the same imaged object is in different partial views that represent the same original scene from different viewing directions.
- an apparatus for generating image data for reproduction via a stereoscopic or autostereoscopic display apparatus the rendering of which via the display apparatus produces a three-dimensional visual impression on a human observer
- the apparatus comprising: an interface for receiving 2D digital image data Origin view, and a view determination unit for deriving a plurality of derived subviews that respectively represent the originating scene represented in the original view from a different viewing angle.
- the view determination unit comprises a color value determination unit which determines color values for at least two primary colors of a color system for each of the pixels of the original view; an allocation unit, the each of the pixels depending on the at least two determined color values one Assigns control value, and a shift unit, which shifts for each of the other views every single pixel of the original view depending on the associated control value and a predetermined displacement for the corresponding view vector and derives the pixels of the respective partial view based on the shifted pixels.
- a color value determination unit which determines color values for at least two primary colors of a color system for each of the pixels of the original view
- an allocation unit the each of the pixels depending on the at least two determined color values one Assigns control value
- a shift unit which shifts for each of the other views every single pixel of the original view depending on the associated control value and a predetermined displacement for the corresponding view vector and derives the pixels of the respective partial view based on the shifted pixels.
- the original view also represents a partial view to which the displacement vector 0 is to be assigned. This is the case when one of the partial views is assigned the original viewing direction (or the original viewing angle or the original viewing position). Even if one of the partial views is identical to the original view, this partial view is also considered as a derived partial view. This means that the derived partial views may include but are not necessarily included in the original view. In many cases, all partial views that are derived differ from the original view.
- the representation of the acquired image data is usually via autostereoscopic playback devices. These are usually supplied with a so-called spatial image template. Such can be created by merging the multiple derived partial views, which may include the original view.
- a preferred development thus provides for combining the derived partial views into a spatial image template.
- a corresponding device thus comprises an assembly unit for creating the spatial image template. The way in which the several derived partial views, which optionally comprise the original view, are combined to form a spatial image template, depends on the reproduction apparatus to be used. The necessary information is usually given by the manufacturer of the playback device.
- control values for the 2D image data of the original view are determined at a central location, for example a transmitting station, and assigned to it.
- a method is carried out comprising the steps of: receiving and / or reading in 2D image data of an original view, inferring and Assigning a control value to each pixel of the original view; wherein color values of at least two primary colors of a color system are determined for each pixel of the original image 2D image data and each of the pixels is automatically assigned a control value dependent on the at least two determined color values, each of the control values representing disparity information.
- the 2D image data are transmitted together with the control values to one or more receiving stations, which is indicated by an arrow. It is understood that the transmission can be coded or uncoded depending on the transmission link. When coding a coding can be selected, which makes it possible to detect transmission errors and possibly even correct.
- the individual pixels of the original view are pixel-wise corresponding to a displacement vector and the individual given to each view
- Pixel associated control value are shifted relative to each other and based on the shifted pixels, the pixels of the respective partial view are derived.
- An apparatus for image data processing for generating image data for reproduction via a reproduction apparatus in the reproduction of which via the reproduction apparatus a three-dimensional visual impression is produced by a human observer, comprises in such a receiving station: an interface for receiving 2D digital image data of an original view and of control values, wherein each pixel is assigned exactly one control value and each of the control values represents disparity information and is determined for the associated pixel of the original image 2D image data from color values of at least two primary colors of a color system of that assigned pixel or can be automatically determined from these color values; a view determination unit for deriving a plurality of partial views representing the original scene represented in the original view from different viewing angles; and a Summary unit for creating a room image template from the several
- the view determination unit comprises a shift unit which shifts for each of the partial views each of the individual pixels of the original view depending on the associated control value and a given for the corresponding partial view displacement vector and derived from the shifted pixels, the pixels of the respective partial view.
- a transmitting station comprises an image data processing device for processing image data for playback via a playback device, the playback of which via the playback device produces a three-dimensional visual impression on a human observer, the device comprising: an interface to
- a color value determination unit which determines color values for at least two primary colors of a color system for each of the pixels of the original view, as well as
- the 2D image data and the associated control values can be output or provided via the interface for receiving the 2D image data or a further interface, for example a transmission interface.
- control values assigned to the individual pixels are calculated in a preferred embodiment.
- a calculation unit is provided for this purpose.
- a function is used that depends on the at least two specific color values as variables.
- the function for the calculation of the control values as a function of the determined color values is in a preferred embodiment a function which has an extreme value and is continuous or discrete continuous, but need not be continuously differentiable.
- the function may still depend on parameter values determined by the reproducing apparatus with which the generated image data is to be reproduced. Moreover, in some embodiments, a given optimal viewing distance has an influence on the calculation of the control values.
- a color system is a system which makes it possible to characterize a color using the color values of individual primary colors.
- a well-known color system for Characterization of colors is, for example, the RGB system (red-green-blue system), in which the individual colors can be generated via a color addition of a red color component, a green color component and a blue color component of the primary colors red, green and blue.
- RGB system red-green-blue system
- the proportion of the individual base color with which it participates in the color addition is determined in each case. For example, in the RGB color system, color values between 0 and 100 are determined for each primary color. The color values then indicate a percentage of an intensity with which the respective base color enters into the color addition.
- the result is a white color for the human viewer. If red and green with 100% intensity and blue with 0% intensity are included in the color addition, the result is, for example, a yellow hue.
- control values which then depend exclusively on the at least two color values, only once for all possible color value combinations and stored in a control value memory.
- the control values can then be read from the control value memory, depending on the determined at least two color values.
- control values are each individually calculated currently for each pixel based on the control value function. This is useful if image data is to be processed for only one source view, or if successively image data from different source views are to be processed, but which should be viewed on different display devices or at different distances to the playback device (s).
- V m a * (-1 + 2 * (m -1) / (n-1))
- an equal number of partial views are generated, which are opposite to an original viewing position from which the original view is taken Representing the left-shifted viewing position how views are generated that represent a right-shifted viewing position relative to the viewing position selected for the origin view.
- the shifting of the individual pixels of the original view preferably takes place in such a way that the respective pixels are shifted in accordance with a disparity vector, the disparity vector for each pixel being calculated as a product of the control value associated with the pixel and the shift vector of the corresponding view.
- the disparity vector is thus for each pixel of the corresponding control value weighted shift vector of the corresponding view.
- the pixels of the source view are processed one row at a time, one at a time, in the direction pointed to by the respective displacement vector associated with the partial view being derived.
- the pixels of the original view which are moved to positions which (with regard to the processing direction) remain behind a position to which a pixel of the line has already been moved, are ignored.
- the pixels assigned to a matrix are automatically shifted to a new matrix position during the displacement process, assuming that the distance between the individual matrix positions an integer multiple of the pixel spacing and the control value range and the value range of the lengths of the displacement vectors are selected accordingly. Since a resulting offset of the individual pixels of the original view is exclusively determined by the pixel-related control values and the partial view-related displacement vectors, defects may arise in the pixel structure or matrix assigned to the shifted pixels if disparity vectors of the pixels adjacent in the original view differ from one another. These are partly not shifted or not to be considered pixels of the original view to assign, as already explained above.
- defects also arise when the control value of the pixels increases in the processing direction. A subsequent pixel is thus shifted by a larger amount than the previous pixel, so that between these one or more defects may arise. Irrespective of their origin, the defects can be filled up either by the color values of the shifted pixel, which is arranged with respect to the displacement direction in front of the respective defect or pixel of the pixel structure or matrix, ie is less or not shifted than the pixel that far it is postponed that one or more other pixels having a lower control value are or are considered to be non-visible and therefore remain or remain undisclosed and thus cause or cause the one or more defects.
- the resulting defects are provided in a line with color values, which are determined by an interpolation from the color values in the derivation of the view to be considered displaced pixels that surround or limit the defects after the shift. If the amounts of the disparity vectors formed from the displacement vectors and the control values are not integral, then it is necessary for all or most of the pixels of the partial view matrix to determine the color values for the individual matrix positions either on the basis of the color values of the pixel closest to the respective line or to calculate the color values by interpolation from the color values of the shifted pixels to be considered.
- the interpolation may be performed linearly or performed following another predetermined function. It is also possible to take into account color values of shifted pixels that are neighbors next to, or next to, etc. neighbors.
- each of the pixels of the partial views is assigned a complete color information which comprises color values with respect to all primary colors of the color system.
- the computational effort for generating the individual partial views to be derived from an original view of a sequence can be kept so low that the calculation of the multiple partial views can be done in "video / TV real-time", ie after a possibly necessary start-up phase, the multiple partial views in FIG
- both the individual partial views can be derived in parallel and the individual rows can be processed line by line in parallel independently of each other.This parallelization makes it possible, even with high original viewing rates to calculate the required derived partial views for determining a spatial image template with the clock rate of the image sequence in a reproduction of an image sequence.
- the received 2D image data in the derivation of the partial views and the control values are usually accessed multiple times, in some embodiments these are stored in a first memory area. Likewise, it is provided in some embodiments to deposit the determined partial views or at least parts thereof in second storage areas.
- the individual units of the device for generating the required image data are preferably realized via a software-controlled computer unit. However, it is also possible to integrate the individual units in one or more FPGA units. Yet another embodiment provides that an ASIC circuit converts the individual units, wherein the image data of the original view and, if appropriate, parameter values which depend on the device or viewing distance can be read in via data lines and the generated image data are output via data lines.
- embodiments can be implemented in which the first memory area and the second memory areas can be integrated into the ASIC circuit or can also be located in a memory which is coupled to the ASIC circuit. The same applies to the control value memory, which however is preferably implemented in the ASIC circuit.
- Fig. 1 is a schematic flow of the processing of image data of a 2D image to
- FIG. 2 is a schematic representation for explaining the image point wise
- 3 is a schematic representation of a device for processing 2D
- Fig. 4 shows a further embodiment of a schematic representation of a
- Fig. 5 is a schematic representation to illustrate an optimized
- FIGS. 6a to 6c show exemplary assignment matrices for the combination of partial views into a spatial image template
- FIG. 7 shows a schematic representation of a method in which control values are assigned to the 2D image data in a transmitting station and the partial views are derived at the receiving stations from the received 2D image data of the original view together with the received control data and combined to form room image templates.
- FIG. 1 schematically shows a method 1 of processing 2D image data of an original view 2.
- the 2D image data are received, recorded or read in the form of an image file, for example.
- the image file of the 2D image data here comprises digitized pixels of the original view 2, which represents an original scene.
- the received 2D image data is preferably colored 2D image data.
- For each pixel of the original view 2 is a color information available. For example, a total of 24 bits are available for encoding the color, with 8 bits each being used for coding a color value in an RGB color system.
- the received 2D image data may have gray scale encoding, with each pixel being associated with a gray level.
- the received 2D image data may be the image data of a single image, which originates, for example, from a data carrier 3 or may be a view of an image sequence 4, for example from a receiver / playback device 5 provided.
- the receiving / reproducing apparatus 5 may be, for example, a television receiver, a video recorder, a DVD player or other player.
- the 2D image data is processed in a processing stage, which may also be referred to as a view determination unit 6, to derive therefrom several partial views 7-1, 7-2, 7-3, 7-4.
- the plurality of derived partial views 7-1 to 7-4 each have the same number of pixels as the original view represented by the 2D image data. If a change in format, resolution, etc., is desired in the intended 3D reproduction in comparison with the original view 2, then the original view is adapted accordingly prior to processing, so that this condition applies to this adapted original view and the partial views derived therefrom.
- Each of the derived partial views 7-1 to 7-4 represents the original scene represented by the original view 2 from another viewing position or another viewing angle or another viewing direction.
- the viewer determining unit 6 first makes at least two color values of Basic colors of a color system determined.
- the 2D image data of the original view are provided in the form of an RGB-coded image file
- the corresponding color values for example for the primary colors red and blue of each pixel, can be read out of the 2D image file
- the color values may need to be calculated.
- the color values are converted into the RGB color system. For a conversion from a greyscale coding into an RGB coding or a conversion from one color system to another color system, there are calculation rules known in the prior art.
- the pixels of the original view 2 are individually assigned control values 8 on the basis of the determined color values.
- the control value 8 in each case represents a disparity information which, in the later processing sequence, determines how great the disparity for the respective pixel in the individual derived partial views 7-1 to 7-4 with respect to the original view 2 or between the individual derived partial views 7-1 to 7-4 will be. If one takes a real scene of two slightly shifted viewing positions, one finds that pixels that represent the same object and have a large disparity between the two shots often "obscure" pixels of objects that would have less disparity Thus, the control values 8 also regulate the visibility of individual pixels of the Origin view 2 in the derived partial views 7-1 to 7-4, as will be explained below.
- the individual pixels of the 2D image file can be assigned to a pixel matrix having vertical columns and horizontal rows.
- the 2D image data of the original view 2 are processed plural times line by line to generate respective pixel lines for the plurality of derived views 7-1 to 7-4.
- Each line is processed as often as derived partial views are generated. (If one of the derived partial views is identical to the original view due to the specifications with regard to the viewing directions assigned to the partial views, processing of the image row can, of course, be saved.)
- each line of the original view 2 is thus processed four times. This is assumed in each case from the image line of the original view 2.
- the derived partial views 7-1 to 7-4 to be derived is assigned a displacement vector V m , where m indicates an index of the respective view, ie
- 7-1 to 7-4 respectively represent the original scene represented by the origin view 2 from different viewing positions. These viewing positions are or are calculated in equal numbers equidistantly symmetrically right and left to the viewing position from which the original view is taken.
- the displacement of the viewing position relative to the viewing position of the original view is represented in each case via a displacement vector V m , which is assigned to the respective partial view m.
- a calculation rule for the displacement vector V m a * (-1 + 2 * (m-1) / (n-1)), where a is a constant and preferably results equal to 1 is selected.
- the displacement vector V m associated with the respective partial view m is weighted or scaled with the control value associated with the individual pixel in order to determine a disparity vector for each individual pixel of the original view 2 in the line-by-line processing. For each pixel of the original view 2, as many disparity vectors are thus determined as partial views are derived. While the control values each have the same sign, both positive and negative shift vectors occur.
- the sign of the displacement vector is associated with a shift direction. Within a pixel row, a positive sign is associated with a direction pointing from left to right and a negative sign corresponding to a direction pointing from right to left. The individual pixels of a row are corresponding to that direction processed, which is predetermined by the direction of the displacement vector for the corresponding partial view.
- the pixels of a line are processed from left to right and shifted from left to right according to the respective disparity vector. It should be noted that those pixels whose disparity vector has a length which is shorter than the length of the previously processed pixel, due to the disparity in the corresponding partial view that is calculated, are not visible and therefore not moved or for the Derivation of the pixels of the corresponding partial view are not taken into account. This also means that pixels whose disparity vector would assign them a new position, which is already occupied by a previously shifted pixel with a larger disparity vector, are not visible and therefore will not be moved or taken into account.
- the amounts of the disparity vectors are rounded to integers so that the shifted pixels can be associated with pixels of a matrix-like raster of the partial view to be derived.
- the shifted pixels thus represent directly the derived pixels of the partial view.
- the disparity vectors originally (not necessarily immediately) of adjacent pixels of the original view may differ greatly from each other, as explained above, gaps between the shifted pixels taken into account in the derivation of the pixels of the partial view will appear in the one with the corresponding partial view associated matrix-like raster.
- these imperfections are filled with the color values which correspond to the pixel previously shifted and to be taken into account with respect to the execution direction of the line.
- the color values of the defects are calculated by interpolation of the adjacent color values to be considered.
- the color values for the individual pixels can also be calculated from the pixels to be considered, whose positions do not coincide exactly with the pixel positions of the matrix-like grid of the partial view, by a subpixel-accurate interpolation.
- the pixels of a line can each independently of the pixels in the adjacent lines are determined.
- the pixels of the different derived partial views 7-1 to 7-4 to be generated at the same time can be independently derived from the pixels of the image data representing the original view 2.
- the individual determined lines of pixels of the derived partial views 7-1 to 7-4 are stored in the embodiment described here in each case in corresponding image memory.
- the plurality of derived views 7-1 to 7-4 are assembled into a spatial image original 10 in a so-called recombination stage 11, which is output to the stereoscopic reproduction apparatus 9.
- the original view prior to processing i. a derivation of the plurality of views
- the resolution of the playback device is recalculated to the playback format, if the resolution of the playback device differs from a resolution of the original view. This is not shown in Fig. 1, but may optionally be provided.
- the control values 8 are calculated in a preferred embodiment according to a function that depends on the blue and red color value component of the pixels.
- the function has an extremum which occurs at a red value R lim and at a blue value B l, m .
- the red and blue color values R , im , B lim at which the maximum of the control value function occurs are referred to as the red color threshold and the blue color threshold.
- the control value is the absolute value of the difference of the red color value and If, on the other hand, the absolute value of the color value of the blue color component reduced by the amount of the blue color limit is greater than the absolute value of the color value of the red color component reduced by the amount of the red color limit, the control value becomes the absolute value of the difference of the blue color value and the blue one Color limit determined.
- the absolute values of the corresponding differences are each normalized by a common factor.
- control value S fa is calculated according to the following formula:
- const. stands for a normalization constant and m and f 2 hm are color threshold values of the primary colors at which the control value function S (fi, f 2 ) has an extremum.
- the function is steadily above the area of the color values and f 2 . If the color system is suitably selected, the control values depend on exactly two color values.
- control values for a number of image data sets, which represent different original views of an image sequence remain unchanged, the control values can be stored in and output from a control value memory and only have to be calculated once for the corresponding image sequence. If the reproducing apparatus used and the desired optimum viewing distance remain constant for different image sequences, the control values need only be calculated once.
- the pixel 21-9 to which the control value with the value 6 is assigned should be shifted.
- the pixel 21-9 to be displaced is shifted to the right by six pixel positions in the image line 24 of the view to be generated and thus represents a partial view pixel 25-15.
- this pixel 21-12 is not shifted or taken into account in the definition of the color values for the pixels of the partial view to be generated.
- the pixels 21-13 to 21-17, to each of which a control value 0 is assigned would be moved to a previously defined pixel of the partial view to be generated.
- the pixel 21-18, to which a control value of 0 is assigned is used and shifted by 0 pixel locations, thus defining the pixel 25-18 of the line 24 to be generated of the corresponding partial view.
- the imperfections can also be obtained via an interpolation of the color values, here for example pixels 25-8 and 25-8. 15, done.
- the control values or more precisely the amounts of the disparity vectors, which are generated by a multiplication of the control value by the corresponding shift vector are rounded to integer values.
- the color values can be interpolated since the pixels of the original view are then not shifted by exact image locations or pixel spacings.
- the displacement of the pixels with respect to a matrix-type raster directly results in the pixels of the derived partial view only in such cases in which the disparity vectors used during the displacement are rounded to integers in length (if they do not already have an integer length) and the whole Number shifts are assigned around whole matrix locations.
- the shifted pixels form a pixel structure which, even ignoring the possibly existing "defects" or the pixels which are not to be considered for the displacement, does not necessarily have to have a regular matrix structure , on the basis of which the actual pixel matrix of the partial view is determined. This means that the pixels of the pixel matrix of the partial view are determined on the basis of the color values of the pixel structure (intermediate matrix).
- FIG. 3 schematically shows an apparatus 31 for image data processing for generating image data for a 3D reproduction. This is designed to use 2D color image data for generating a plurality of partial views thereof and, if necessary, to combine these partial views into a spatial image template.
- an interface 32 Via an interface 32, the image data of the original view and, if appropriate, parameters via the reproduction device and / or an optimal viewing distance to the reproduction device are received and recorded.
- the interface 32 is coupled to a central processor unit 33, which in turn is linked to a memory 34.
- the memory 34 comprises a program memory area 35, in which a program code is stored, via which the operation and operation of the central processor unit 33 is controlled.
- the program memory area 35 may include both an operating system and an application software that controls the actual image data processing.
- the memory 34 further comprises a first memory area 36, in which the image data received via the interface 32 are stored.
- a first memory area 36 in which the image data received via the interface 32 are stored.
- the image data is read in the form of a 24-bit RGB signal
- 24-bit memory is reserved for each pixel. This means that 8 bits are reserved for each of the three basic colors red, green and blue.
- control values can be calculated on the basis of two color values, for example based on the color values for the primary colors red and blue, on the basis of a control value function.
- control values are calculated for all possible color value combinations and stored in a control value memory area 37 of the memory 34.
- the control value function may additionally depend on parameters which are dependent on the respective reproduction device for which several partial views of the original view or a spatial image template are generated.
- Optimum viewing distance from the playback device as a parameter in the control value function.
- the program code is further designed such that in each case at least two color values, preferably the color values of the primary color red and the primary color blue, are determined for the individual pixels of the original view, which are stored in the first memory area 36.
- the pixels are transferred in an RGB coding
- only the 8-bit memory areas which are assigned to the two primary colors have to be read out in each case.
- the corresponding associated control value can now be read from the control value memory area 37 and assigned to the corresponding pixel.
- This value can then be stored in the reserved bits of the first memory area 36, for example.
- control value memory area 37 it is possible to design the control value memory area 37 in such a way that the color values already present in the RGB coding, which are included in the calculation of the control value, are themselves used as pointer values in a matrix-type memory organization of the control value memory area 37.
- the image lines of the original view are now processed line by line once for each partial view.
- the lines of pixels here it is assumed that the pixels are organized in horizontal rows and vertical columns
- the execution direction is determined in each case by the direction of the associated displacement vector.
- a disparity vector is formed from the shift vector and the control value associated with the respective pixel, and the Pixels shifted relative to the original view in the generated partial view in accordance with the disparity vector.
- the absolute value of the disparity vector is rounded to an integer value, so that a shift takes place about entire grid positions in the matrix-type grid of picture elements.
- One pixel is shifted by whole column units.
- the shift to non-integer positions within the column is permitted and performed, and then an interpolation of the color value is performed to determine the color values at the individual raster points of the image line.
- the occurring defects which are assigned no pixel in the shift, are either filled with the color values, which has a previously correspondingly less far or possibly zero shifted pixel or determined by an interpolation of the color values of the pixels adjacent (not necessarily but immediately adjacent) to the defects within the row.
- the interpolation may be performed linearly or performed following another predetermined function.
- color values of shifted pixels, which are neighbors next to, or next to, etc. can also be taken into account. Since the individual lines of a partial view and the same lines of different partial views can be calculated independently of each other, it is possible to partially or completely parallelize the image processing. As a result, an acceleration of the image data processing can be achieved.
- the individual generated partial views of the original scene are stored in second memory areas 38 of the memory 34. If an odd number of partial views are calculated, one of the partial views is usually identical to the original view. Accordingly, in some embodiments, the source view is once copied to a second storage area 38. In some embodiments, the partial views stored in the second storage areas 38 are output via the interface 32 or another interface 39 for further processing and / or storage. In a preferred embodiment, however, the partial views stored in the second memory areas 38 are combined to form a spatial image template, which is temporarily stored in a spatial image memory 40.
- a graphics processor 41 calculates the spatial image template on the basis of the partial views stored in the second memory areas 38 and the output of the spatial image template via the further interface 39 are preferred.
- the following is an example of how to create a room image template from the partial views.
- the necessary information is generally provided in order to be able to carry out the summary.
- the reproduction device 24 different image information in each case different
- Viewing direction directions can spend. For the three-dimensional reproduction in the case described here, only four different partial images are used as templates. Since information is to be output in all viewing directions, the same image information is therefore output in several of the spatial directions.
- the manufacturer of the reproduction apparatus has specified three assignment matrices, which respectively indicate one of the three primary colors red, green, and blue of an RGB color system, such as the corresponding color value of the corresponding base color for the individual pixels from the color values of the pixels of the partial views results.
- the three assignment matrices for the primary colors green, blue and red are shown correspondingly in FIGS. 6a to 6c.
- the assignment matrices are each 8x12 matrices.
- the individual matrix entries each indicate an index associated with one of the four partial views m.
- pixels in the partial views and the spatial image template to be generated are each indexed via an index pair (x, y), where x indicates the position within the line and y the line itself.
- a pixel of the spatial image template is represented by the expression:
- the color value of the basic color "basic color" of the pixel is determined by the expression
- Pixel RBV [x, y] .red: pixel TA (assignment matrix ROT [x modul ° 8 ' y modul ° 121 > [x, y] .rot, where pixel TA (m) [x, y] .grunfrabe the color value for Basic color "basic color" of the pixel of the partial view indexed by the index pair (x, y) indicates the partial view m for the other primary colors in the shortened notation corresponding to:
- a modulo b (often also “a mod b"), is defined here by the following calculation rule: where a is a real number and b is a natural number greater than zero and
- Gaussian bracket which gives the real number c the largest integer less than or equal to the number c, i.
- assignment matrices are smaller than the fields or the space image template, the matrix entries are used periodically.
- a further embodiment of a device 51 for image data processing is shown schematically.
- the individual functional units are partially or completely formed in hardware, for example by means of an FPGA or an application-related processor chip (ASIC).
- ASIC application-related processor chip
- the image data of the original view and optionally further parameters are received again and stored in the first memory area 53.
- Subviews are subsequently generated by means of a view determination unit 54.
- the control values are calculated and stored in the control value storage area 56.
- a color value determination unit 57 determines, for each of the pixels, two color values of two primary colors of a color system. For this purpose, it may be necessary to convert the color information of a pixel from one color system to another color system.
- an assignment unit 58 assigns a control value of the control value memory area 56 to the respective pixel.
- the assigned control value can also be stored, for example, in the first memory area 53.
- a partial view determination unit 59 uses the image data stored in the first memory area and the assigned control values to determine a plurality of partial views which are stored in second memory areas 61. This is a shift of the pixels of the original view in a shift unit 60, as this is explained by way of example in connection with FIG. 2.
- a summary unit 62 summarizes the multiple partial views into a spatial image template which is optionally stored in a spatial image storage 63 and is then output via a further interface 64.
- the intermediate storage of the spatial image template and, if appropriate, the creation of the spatial image template may be omitted. In the latter case, the partial views are then output and provided directly via the further interface.
- the image data is acquired by a detection module 81 and written to a removable memory buffer 82. This corresponds to the first storage area.
- the image data is read from the removable memory buffer 82 and processed by a processing unit as indicated above, by assigning control values from a control value memory 84 to the pixels, and then determining and executing the corresponding displacements of the pixels together with the displacement vectors, and so on Partial views are generated, which are stored in further removable storage buffers 85.
- the partial views stored in the further removable storage buffers 85 are then read in as a so-called texture buffer 87 of a graphics processing unit 88.
- the individual texture buffers 87 are associated with punch masks 89.
- the color values of individual pixels for the creation of the spatial image template are selected and combined via OR combinations in a so-called blending unit 90 to form a spatial image template, which in a subsequent image clock of an autostereoscopic reproducing device 91 is output.
- the use of the various memory or memory areas may be limited particularly in embodiments implemented in hardware.
- the calculated partial views or their pixels can be output without buffering. Even if a room image template is to be calculated, it is not necessary to buffer all partial views. It is sufficient to cache only parts, for example parts of the same line of the partial views, in order to determine the individual pixels of the spatial image template. In optimized circuits, a memory requirement can be greatly reduced. 7 schematically shows an embodiment in which the control value assignment to the pixels and the "actual" determination of the partial views take place spatially separated A 2D image data stream 102 is provided to a transmitting station 101.
- the 2D image data stream 102 comprises, for example, successive original views of one
- the transmitting station 101 comprises an image data processing device 103, which assigns disparity information in the form of control values to the individual pixels of the 2D image data, as explained in detail above, the device comprising an allocation unit 104 for this purpose Color evaluation unit, a calculation unit and optionally a control value memory include, which are not shown separately.
- An interface which is formed for example as a transmitting antenna 105, the 2D image data together with the image assigned control values to one or more receiving stations 106, of which only one is shown, transmitted.
- the receiving station 106 comprises an interface formed, for example, as a receiving antenna 107.
- the 2D image data, along with the control values associated with the pixels, are passed on to an image processing apparatus 108 for producing image data for display, which produces a three-dimensional impression on a viewer.
- This device 108 is designed to generate the partial views 7-1 to 7-4 and possibly also a spatial image original 10, as described in detail above.
- the determination of the control values is of course no longer necessary, since these are already transmitted with the 2D image data.
- the device 108 thus comprises a view determination unit 109, which determines partial views 7-1 to 7-4.
- a spatial image template 10 is created from the partial views 7-1 to 7-4 in a summary unit 110, which is output via an output interface 111 to an autostereoscopic reproduction apparatus 112 and reproduced therefrom.
- Image processing apparatus for generating image data which gives a three-dimensional impression upon reproduction
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Testing, Inspecting, Measuring Of Stereoscopic Televisions And Televisions (AREA)
- Image Processing (AREA)
- Processing Or Creating Images (AREA)
Abstract
Description
Claims
Priority Applications (5)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US13/496,396 US8693767B2 (en) | 2009-09-15 | 2010-08-27 | Method and device for generating partial views and/or a stereoscopic image master from a 2D-view for stereoscopic playback |
EP10749808A EP2478705A1 (de) | 2009-09-15 | 2010-08-27 | Verfahren und vorrichtung zum erzeugen von teilansichten und/oder einer raumbildvorlage aus einer 2d-ansicht für eine stereoskopische wiedergabe |
RU2012114692/08A RU2541928C2 (ru) | 2009-09-15 | 2010-08-27 | Способ и устройство для формирования частичных видов и/или оригинала стереоскопического изображения из двухмерного изображения для стереоскопического воспроизведения |
AU2010294914A AU2010294914B2 (en) | 2009-09-15 | 2010-08-27 | Method and device for generating partial views and/or a stereoscopic image master from a 2D-view for stereoscopic playback |
CN201080041094.7A CN102612837B (zh) | 2009-09-15 | 2010-08-27 | 由2d视图产生部分视图和/或立体原图以便立体重现的方法和装置 |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
DE102009041328.6 | 2009-09-15 | ||
DE102009041328A DE102009041328A1 (de) | 2009-09-15 | 2009-09-15 | Verfahren und Vorrichtung zum Erzeugen von Teilansichten und/oder einer Raumbildvorlage aus einer 2D-Ansicht für eine stereoskopische Wiedergabe |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2011032642A1 true WO2011032642A1 (de) | 2011-03-24 |
Family
ID=43128251
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/EP2010/005335 WO2011032642A1 (de) | 2009-09-15 | 2010-08-27 | Verfahren und vorrichtung zum erzeugen von teilansichten und/oder einer raumbildvorlage aus einer 2d-ansicht für eine stereoskopische wiedergabe |
Country Status (7)
Country | Link |
---|---|
US (1) | US8693767B2 (de) |
EP (1) | EP2478705A1 (de) |
CN (1) | CN102612837B (de) |
AU (1) | AU2010294914B2 (de) |
DE (1) | DE102009041328A1 (de) |
RU (1) | RU2541928C2 (de) |
WO (1) | WO2011032642A1 (de) |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN105191298A (zh) * | 2013-02-20 | 2015-12-23 | 股份公司Mooovr | 使用2d-3d多维内容文件的多维内容服务提供***、该服务提供方法以及该多维内容文件 |
Families Citing this family (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
KR20150080003A (ko) * | 2011-09-12 | 2015-07-08 | 인텔 코포레이션 | 모션 패럴랙스를 이용한 2d 이미지로부터의 3d 지각 생성 |
WO2014163466A1 (ko) * | 2013-04-05 | 2014-10-09 | 삼성전자 주식회사 | 정수 픽셀의 위치와 관련하여 비디오의 부호화 및 복호화를 수행하는 방법과 그 장치 |
CN110100444A (zh) * | 2016-12-22 | 2019-08-06 | 株式会社Kt | 视频信号处理方法和装置 |
Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
EP1521483A1 (de) * | 2003-09-30 | 2005-04-06 | Kabushiki Kaisha Toshiba | Verfahren zur Herstellung von Reliefbildern und stereoskopische Bildanzeigevorrichtung |
DE10348618A1 (de) | 2003-09-15 | 2005-04-14 | Armin Grasnick | Verfahren zum Erstellen einer Raumbildvorlage für Abbildungsverfahren mit räumlichen Tiefenwirkungen und Vorrichtung zum Anzeigen einer Raumbildvorlage |
US20070046899A1 (en) | 2005-07-12 | 2007-03-01 | Sony Corporation | Stereoscopic image display apparatus |
Family Cites Families (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP3197801B2 (ja) * | 1995-09-08 | 2001-08-13 | 三洋電機株式会社 | 二次元表示画像生成方法 |
DE19825950C1 (de) * | 1998-06-12 | 2000-02-17 | Armin Grasnick | Anordnung zur dreidimensionalen Darstellung |
US6476805B1 (en) * | 1999-12-23 | 2002-11-05 | Microsoft Corporation | Techniques for spatial displacement estimation and multi-resolution operations on light fields |
RU2287858C2 (ru) * | 2001-11-24 | 2006-11-20 | Тдв Текнолоджиз Корп. | Создание последовательности стереоскопических изображений из последовательности двумерных изображений |
CN1295655C (zh) * | 2001-11-24 | 2007-01-17 | Tdv技术公司 | 从2d图像序列生成立体图像序列 |
EP1665815A2 (de) * | 2003-09-15 | 2006-06-07 | Armin Grasnick | Verfahren zum erstellen einer raumbildvorlage für abbildungsverfahren mit rumlichen tiefenwirkungen und vorrichtung zum anzeigen einer raumbildvorlage |
CA2627999C (en) * | 2007-04-03 | 2011-11-15 | Her Majesty The Queen In Right Of Canada, As Represented By The Minister Of Industry Through The Communications Research Centre Canada | Generation of a depth map from a monoscopic color image for rendering stereoscopic still and video images |
-
2009
- 2009-09-15 DE DE102009041328A patent/DE102009041328A1/de not_active Withdrawn
-
2010
- 2010-08-27 RU RU2012114692/08A patent/RU2541928C2/ru not_active IP Right Cessation
- 2010-08-27 WO PCT/EP2010/005335 patent/WO2011032642A1/de active Application Filing
- 2010-08-27 EP EP10749808A patent/EP2478705A1/de not_active Withdrawn
- 2010-08-27 AU AU2010294914A patent/AU2010294914B2/en not_active Ceased
- 2010-08-27 US US13/496,396 patent/US8693767B2/en not_active Expired - Fee Related
- 2010-08-27 CN CN201080041094.7A patent/CN102612837B/zh not_active Expired - Fee Related
Patent Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
DE10348618A1 (de) | 2003-09-15 | 2005-04-14 | Armin Grasnick | Verfahren zum Erstellen einer Raumbildvorlage für Abbildungsverfahren mit räumlichen Tiefenwirkungen und Vorrichtung zum Anzeigen einer Raumbildvorlage |
EP1521483A1 (de) * | 2003-09-30 | 2005-04-06 | Kabushiki Kaisha Toshiba | Verfahren zur Herstellung von Reliefbildern und stereoskopische Bildanzeigevorrichtung |
US20070046899A1 (en) | 2005-07-12 | 2007-03-01 | Sony Corporation | Stereoscopic image display apparatus |
Non-Patent Citations (1)
Title |
---|
See also references of EP2478705A1 * |
Cited By (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN105191298A (zh) * | 2013-02-20 | 2015-12-23 | 股份公司Mooovr | 使用2d-3d多维内容文件的多维内容服务提供***、该服务提供方法以及该多维内容文件 |
CN105191298B (zh) * | 2013-02-20 | 2017-07-11 | 股份公司Mooovr | 使用2d‑3d多维内容文件的多维内容服务提供***、该服务提供方法以及该多维内容文件 |
Also Published As
Publication number | Publication date |
---|---|
US8693767B2 (en) | 2014-04-08 |
AU2010294914A1 (en) | 2012-03-08 |
US20120170837A1 (en) | 2012-07-05 |
CN102612837B (zh) | 2015-06-03 |
DE102009041328A1 (de) | 2011-03-24 |
RU2012114692A (ru) | 2013-10-27 |
EP2478705A1 (de) | 2012-07-25 |
RU2541928C2 (ru) | 2015-02-20 |
CN102612837A (zh) | 2012-07-25 |
AU2010294914B2 (en) | 2014-05-01 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
DE69621778T2 (de) | Tiefenabhängige parallaktische pixelverschiebung | |
DE69221346T2 (de) | Vorrichtung und Verfahren zur Erstellung stereoskopischer Bilder | |
EP2027728B1 (de) | Verfahren und vorrichtung zur pseudoholographischen bilderzeugung | |
DE69628662T2 (de) | Bildumwandlungs- und kodierungstechniken | |
DE69215058T2 (de) | Dreidimensionale Bildwiedergabeeinrichtung, die elektrisch erzeugte Parallaxstreifen verwendet | |
DE19640936C2 (de) | Positionsadaptiver Autostereoskoper Monitor (PAM) | |
DE69432692T2 (de) | Bildverarbeitungsvorrichtung und -verfahren. | |
DE19953595B4 (de) | Verfahren und Vorrichtung zur Verarbeitung dreidimensionaler Bilder | |
EP0776576B1 (de) | Verfahren und vorrichtung zur darstellung von stereoskopischen videobildern auf einem display | |
EP1451775A1 (de) | Erzeugung einer stereo-bildfolge aus einer 2d-bildfolge | |
DE102006055641A1 (de) | Anordnung und Verfahren zur Aufnahme und Wiedergabe von Bildern einer Szene und/oder eines Objektes | |
DE69529260T2 (de) | Gerät und Verfahren zur Bildverarbeitung | |
DE69601577T2 (de) | Verfahren und einrichtung zur aufnahme stereoskopischer bilder | |
EP2676449B1 (de) | Verfahren und vorrichtung zur stereobasis-erweiterung von stereoskopischen bildern und bildfolgen | |
EP2540089A2 (de) | Verfahren zur visualisierung von dreidimensionalen bildern auf einer 3d-anzeigevorrichtung und 3d-anzeigevorrichtung | |
DE112012000563T5 (de) | Verfahren und Vorrichtung zum Senden/Empfangen eines digitalen Übertragungssignals | |
EP3170307B1 (de) | Verfahren zur darstellung einer dreidimensionalen szene auf einem autostereoskopischen monitor | |
DE102013217826A1 (de) | Verfahren zum erzeugen, übertragen und empfangen stereoskopischer bilder, und zugehörige geräte | |
WO2010128097A2 (de) | Verfahren zur räumlichen darstellung | |
WO2011032642A1 (de) | Verfahren und vorrichtung zum erzeugen von teilansichten und/oder einer raumbildvorlage aus einer 2d-ansicht für eine stereoskopische wiedergabe | |
DE10056978B4 (de) | Verfahren zur Erzeugung eines stereographischen Bildes | |
DE19500699A1 (de) | Personen-adaptiver stereoskoper Video-Schirm (PASS) | |
DE202008011541U1 (de) | Anzeigesystem | |
DE112012002679T5 (de) | Vorrichtung und Verfahren zum Codieren/Decodieren von Mehrfachansicht-Bildern | |
DE10046788B4 (de) | Verfahren und Vorrichtung zur Erzeugung eines Kombinationsbildes aus in einem Datensatz gespeicherten Bildinformationen |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
WWE | Wipo information: entry into national phase |
Ref document number: 201080041094.7 Country of ref document: CN |
|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 10749808 Country of ref document: EP Kind code of ref document: A1 |
|
WWE | Wipo information: entry into national phase |
Ref document number: 2010294914 Country of ref document: AU |
|
ENP | Entry into the national phase |
Ref document number: 2010294914 Country of ref document: AU Date of ref document: 20100827 Kind code of ref document: A |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
WWE | Wipo information: entry into national phase |
Ref document number: 13496396 Country of ref document: US |
|
REEP | Request for entry into the european phase |
Ref document number: 2010749808 Country of ref document: EP |
|
WWE | Wipo information: entry into national phase |
Ref document number: 2010749808 Country of ref document: EP |
|
WWE | Wipo information: entry into national phase |
Ref document number: 2012114692 Country of ref document: RU |