WO2012014494A1 - Image processing apparatus and method and program - Google Patents
Image processing apparatus and method and program Download PDFInfo
- Publication number
- WO2012014494A1 WO2012014494A1 PCT/JP2011/004318 JP2011004318W WO2012014494A1 WO 2012014494 A1 WO2012014494 A1 WO 2012014494A1 JP 2011004318 W JP2011004318 W JP 2011004318W WO 2012014494 A1 WO2012014494 A1 WO 2012014494A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- signal
- image
- unit
- sum
- image signal
- Prior art date
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
- H04N13/10—Processing, recording or transmission of stereoscopic or multi-view image signals
- H04N13/106—Processing image signals
-
- G—PHYSICS
- G03—PHOTOGRAPHY; CINEMATOGRAPHY; ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ELECTROGRAPHY; HOLOGRAPHY
- G03B—APPARATUS OR ARRANGEMENTS FOR TAKING PHOTOGRAPHS OR FOR PROJECTING OR VIEWING THEM; APPARATUS OR ARRANGEMENTS EMPLOYING ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ACCESSORIES THEREFOR
- G03B35/00—Stereoscopic photography
- G03B35/08—Stereoscopic photography by simultaneous recording
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
- H04N13/10—Processing, recording or transmission of stereoscopic or multi-view image signals
- H04N13/106—Processing image signals
- H04N13/161—Encoding, multiplexing or demultiplexing different image signal components
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
- H04N13/20—Image signal generators
- H04N13/204—Image signal generators using stereoscopic image cameras
- H04N13/239—Image signal generators using stereoscopic image cameras using two 2D image sensors having a relative position equal to or related to the interocular distance
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
- H04N19/597—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding specially adapted for multi-view video sequence encoding
Definitions
- the present disclosure relates to image processing apparatuses and methods and programs and, in particular, an image processing apparatus and method and a program capable of more easily suppressing the occurrence of an error in the location of a congestion point, that is, convergence point, in 3D video.
- One characteristic of stereoscopic video obtained using a so-called single-lens stereoscopic 3D camera is that an in-focus object being a subject is oriented at a location on a displayed screen where the stereoscopic image is displayed. That is, when left eye video and right eye video that form stereoscopic video are displayed on a display screen, the same in-focus objects of the left and right video substantially match each other.
- a display apparatus that allows a user to watch stereoscopic video using polarized glasses, shutter glasses, or other glasses, if the user watches video displayed on the display apparatus without the polarized glasses or other glasses, that video is seen as 2D (two-dimensional) video. And, if the user watches video displayed on the display apparatus through polarized glasses or other glasses, that video is seen as 3D (three-dimensional) video (for example, refer to Patent Literature 1). In this way, a display apparatus that can be used with polarized glasses or other glasses has a characteristic of compatibility between 2D video and 3D video.
- the method may including receiving an image signal comprising a left-image signal representing a left image and a right-image signal representing a right image.
- the method may further include generating a sum signal by combining the left-image signal and the right-image signal.
- the method may also include displaying a sum image corresponding to the sum signal, the displayed image including a convergence point and a focus point.
- an electronic device for processing images The electronic device may receive an image signal comprising a left-image signal representing a left image and a right-image signal representing a right image.
- the electronic device may further generate a sum signal by combining the left-image signal and the right-image signal.
- the electronic device may also display a sum image corresponding to the sum signal, the displayed image including a convergence point and a focus point.
- tangibly embodied non-transitory computer-readable storage medium including instructions that, when executed by a processor, perform a method for processing images on an electronic device.
- the method may including receiving an image signal comprising a left-image signal representing a left image and a right-image signal representing a right image.
- the method may further include generating a sum signal by combining the left-image signal and the right-image signal.
- the method may also include displaying a sum image corresponding to the sum signal, the displayed image including a convergence point and a focus point.
- a photographer makes either one of left eye video and right eye video be displayed on a viewer, and carries out lens adjustment on, for example, a focus, zoom, and iris while checking video for the single eye displayed on the viewer.
- lens adjustment on, for example, a focus, zoom, and iris while checking video for the single eye displayed on the viewer.
- the present embodiment is directed to being capable of more easily suppressing an error in the location of a congestion point of 3D video.
- the occurrence of an error in the location of a congestion point in 3D video can be suppressed more easily.
- Fig. 1 illustrates a configuration example of one embodiment of an imaging apparatus to which the present embodiment is applied.
- Fig. 2 is a flowchart for describing an imaging process.
- Fig. 3 illustrates a configuration example of a signal reproducing apparatus.
- Fig. 4 is a flowchart for describing a reproducing process.
- Fig. 5 illustrates another configuration example of a signal reproducing apparatus.
- Fig. 6 is a flowchart for describing a reproducing process.
- Fig. 7 illustrates a configuration example of a signal reproducing unit.
- Fig. 8 is a flowchart for describing an edit-point recording process.
- Fig. 9 illustrates a configuration example of an imaging apparatus.
- Fig. 10 is a flowchart for describing an imaging process.
- Fig. 10 is a flowchart for describing an imaging process.
- FIG. 11 illustrates a configuration example of an imaging apparatus.
- Fig. 12 is a flowchart for describing an imaging process.
- Fig. 13 illustrates another configuration example of a signal reproducing apparatus.
- Fig. 14 is a flowchart for describing a reproducing process.
- Fig. 15 is a block diagram that illustrates a configuration example of a computer.Description of Embodiments
- FIG. 1 illustrates a configuration example of one embodiment of an imaging apparatus.
- An imaging apparatus 11 is a so-called single-lens stereoscopic 3D camera, and receives light from an object and acquires a stereoscopic image signal that includes an L signal being an image signal for a left eye and an R signal being an image signal for a right eye.
- an L signal for a left eye that is, a left-image signal
- an R signal for a right eye that is, a right-image signal
- the stereoscopic image signal may be a moving image signal.
- the imaging apparatus 11 includes a synchronizing (sync) signal generating unit 21, an optical system 22, an imaging unit 23-1, an imaging unit 23-2, a gamma conversion unit 24-1, a gamma conversion unit 24-2, a sum signal computing unit 25, a difference signal computing unit 26, a coding unit 27, a signal transferring unit 28, a recording unit 29, a signal switching unit 30, and a display unit 31.
- a synchronizing (sync) signal generating unit 21 an optical system 22
- an imaging unit 23-1 an imaging unit 23-2
- the sync signal generating unit 21 receives an externally supplied external sync signal of a specific frequency clock, generates a sync signal of the same frequency and phase as those of the supplied external sync signal, and supplies the generated sync signal to the imaging unit 23-1 and the imaging unit 23-2. If no external sync signal is supplied to the sync signal generating unit 21, the sync signal generating unit 21 may generate a sync signal of a frequency previously set in a so-called free-running manner.
- the optical system 22 can include a plurality of lenses, for example, and guides light incident from an object to the imaging unit 23-1 and the imaging unit 23-2.
- an entrance pupil of the optical system 22 is provided with a mirror or other elements for separating light incident from an object into two beams, and the two separated beams are guided to the imaging unit 23-1 and the imaging unit 23-2, respectively.
- light incident on the entrance pupil of the optical system 22 is separated into two beams by two mirrors having different inclination directions arranged on an optical path of the light (for example, refer to Japanese Unexamined Patent Application Publication No. 2010-81580).
- the imaging unit 23-1 and the imaging unit 23-2 generate an L signal and a R signal by photoelectrically converting light incident from the optical system 22 in synchronization with a sync signal supplied from the sync signal generating unit 21 and supply the L and R signals to the gamma conversion unit 24-1 and the gamma conversion unit 24-2.
- the gamma conversion unit 24-1 and the gamma conversion unit 24-2 perform gamma conversion on an L signal and an R signal supplied from the imaging unit 23-1 and the imaging unit 23-2 and supply the signals to the sum signal computing unit 25, the difference signal computing unit 26, and the signal switching unit 30.
- the imaging unit 23-1 and the imaging unit 23-2 are also referred to simply as the imaging unit 23 if it is not necessary to distinguish between them and the gamma conversion unit 24-1 and the gamma conversion unit 24-2 are also referred to simply as the gamma conversion unit 24 if it is not necessary to distinguish between them.
- the sum signal computing unit 25 determines the sum of an L signal and an R signal supplied from the gamma conversion unit 24-1 and the gamma conversion unit 24-2 and supplies the resultant sum signal to the coding unit 27 and the signal switching unit 30.
- the difference signal computing unit 26 determines the difference between an L signal and an R signal supplied from the gamma conversion unit 24-1 and the gamma conversion unit 24-2 and supplies the resultant difference signal to the coding unit 27 and the signal switching unit 30.
- the coding unit 27 includes a sum signal coding unit 41 that codes a sum signal from the sum signal computing unit 25 and a difference signal coding unit 42 that codes a difference signal from the difference signal computing unit 26.
- the coding unit 27 supplies the sum signal and difference signal acquired by coding to the recording unit 29 and the signal transferring unit 28.
- the signal transferring unit 28 transfers (sends) a sum signal and a difference signal supplied from the coding unit 27 to an apparatus (not illustrated) connected over a communication network, such as the Internet, or a cable.
- the recording unit 29 includes a hard disk or other elements and records a sum signal and a difference signal supplied from the coding unit 27.
- the signal switching unit 30 supplies the display unit 31 with any one of an L signal and an R signal supplied from the gamma conversion unit 24, a sum signal supplied from the sum signal computing unit 25, and a difference signal supplied from the difference signal computing unit 26, and the display unit 31 displays an image corresponding to the respective signal supplied.
- the display unit 31 may display a left image corresponding to the left-image signal if the left-image signal is supplied, a right image corresponding to the right-image signal if the right-image signal is supplied, a sum image corresponding to the sum signal if the sum signal is supplied, a difference image corresponding to the difference signal if the difference signal is supplied, or any combination of such images.
- the imaging apparatus 11 when a user operates the imaging apparatus 11 and provides an instruction to start obtaining an image of an object, the imaging apparatus 11 starts an imaging process, obtains the image of the object, and generates a stereoscopic image signal.
- the imaging process performed by the imaging apparatus 11 is described below with reference to the flowchart of Fig. 2.
- the imaging unit 23 obtains an image of an object. That is, the optical system 22 collects light incident from an object, separates it into two beams, and causes them to be incident on the imaging unit 23-1 and the imaging unit 23-2.
- Each of the imaging unit 23-1 and the imaging unit 23-2 obtains an image of an object by photoelectrically converting light incident from the optical system 22 in synchronization with a sync signal supplied from the sync signal generating unit 21. By virtue of the sync signal, images of the same frame of an L signal and an R signal are always obtained at the same time.
- the imaging unit 23-1 and the imaging unit 23-2 supply the L signal and R signal acquired by the photoelectrical conversion to the gamma conversion unit 24-1 and the gamma conversion unit 24-2.
- the gamma conversion unit 24-1 and the gamma conversion unit 24-2 perform gamma conversion on an L signal and an R signal supplied from the imaging unit 23-1 and the imaging unit 23-2. With this, the L signal and R signal are gamma-corrected.
- the gamma conversion unit 24-1 and the gamma conversion unit 24-2 supply the L signal and R signal subjected to the gamma conversion to the sum signal computing unit 25, the difference signal computing unit 26, and the signal switching unit 30.
- a curve that indicates an input-output characteristic of gamma conversion when the horizontal axis represents an input value and the vertical axis represents an output value is a curve that is bowed upward in the vertical axis (convex upward).
- the exponent in gamma conversion is not limited to (1/2.2), and it may be another value.
- gamma conversion unit 24 in addition to gamma conversion, another correction process for improving the image quality, such as defect correction, white balance adjustment, or shading adjustment, may be performed on an L signal and an R signal.
- the sum signal computing unit 25 generates a sum signal by determining the sum of an L signal and an R signal supplied from the gamma conversion unit 24 and supplies it to the sum signal coding unit 41 or the signal switching unit 30. That is, as for an L signal and an R signal of a specific frame, the sum signal computing unit 25 determines the sum of the pixel value of a pixel of an image corresponding to the L signal (hereinafter also referred to as L image) and the pixel value of a pixel of an image corresponding to the R signal (hereinafter also referred to as R image) that is in the same location as the pixel of the L image and sets the determined sum as the pixel value of the pixel of the image corresponding to the sum signal, that is, sum image.
- L image the sum signal computing unit 25 determines the sum of the pixel value of a pixel of an image corresponding to the L signal
- R image pixel value of a pixel of an image corresponding to the R signal
- the pixel value of each pixel of an image corresponding to a sum signal is described as the sum of the pixel value of a pixel of an L image and the pixel value of a pixel of an R image, the pixels being in the same location in the same frame, the pixel value of the pixel of an image corresponding to a sum signal may be a value acquired by normalization of the sum of the pixel values of pixels in the same location of the L image and R image.
- the result is that the image corresponding to the sum signal is an image in which an L image and an R image are overlaid with each other. In other words, only dynamic ranges of the respective images are different.
- the difference signal computing unit 26 generates a difference signal by determining the difference between an L signal and an R signal supplied from the gamma conversion unit 24 and supplies it to the difference signal coding unit 42 and the signal switching unit 30. That is, for an L signal and an R signal of a specific frame, the difference signal computing unit 26 subtracts, from the pixel value of a pixel in the L image, the pixel value of a pixel of the R image that is in the same location as the pixel of the L image and sets the resultant difference of the pixel values as the pixel value of the corresponding pixel of the difference image.
- the pixel value of a pixel in the difference image may be a value in which the difference between an L signal and an R signal is normalized. And, if a coder at a subsequent step (difference signal coding unit 42) cannot take a negative value as its input, a preset offset value may be added so as to prevent a difference signal from having a negative value.
- the signal switching unit 30 supplies the display unit 31 with a user-specified signal among an L signal and an R signal from the gamma conversion unit 24, a sum signal from the sum signal computing unit 25, and a difference signal from the difference signal computing unit 26, and the display unit 31 displays an image corresponding to the respective signal supplied.
- the display unit 31 may display a left image corresponding to the left-image signal if the left-image signal is supplied, a right image corresponding to the right-image signal if the right-image signal is supplied, a sum image corresponding to the sum signal if the sum signal is supplied, a difference image corresponding to the difference signal if the difference signal is supplied, or any combination of such images.
- a user can display an image corresponding to any one signal of an L signal, R signal, sum signal, and difference signal on the display unit 31 when operating the imaging apparatus 11 and obtaining an image of an object. Accordingly, the user can switch a displayed image to a desired one and obtain an image of the object while seeing the displayed image on the display unit 31.
- the image corresponding to the sum signal when displayed on the display unit 31, because the image corresponding to the sum signal may be an image in which an L image and an R image are overlaid with each other, a user can obtain an image while checking to determine that no error occurs between the L image for the left eye and the R image for the right eye.
- One characteristic of the imaging apparatus 11 being a single-lens stereoscopic 3D camera is that a focus position, that is, focus point, of the optical system 22 and a congestion point coincide with each other. Therefore, lens adjustment in the optical system 22 by a user seeing an image corresponding to a sum signal displayed on the display unit 31 such that a congestion point is in a location on a display screen of the display unit 31, in other words, such that the same object contained in the L image and in the R image are overlaid with each other in an image corresponding to a sum signal corresponds to setting a focus position with high precision.
- a user can reliably put an object of interest into focus by an easy operation of performing lens adjustment of the optical system 22 such that left and right images of the object of interest coincide with each other while seeing an image corresponding to a sum signal displayed on the display unit 31.
- the imaging apparatus 11 can orient the object of interest on a display screen when the acquired stereoscopic image is reproduced. In other words, the occurrence of an error in the location of a congestion point of a stereoscopic image can be suppressed more easily.
- displaying an image corresponding to a sum signal on the display unit 31 enables a user to obtain a stereoscopic image while checking for not only a focus position but also for an error in the location of a congestion point for an L image and an R image.
- an image displayed on the display unit 31 is switched to an image corresponding to a L signal or an R signal
- a user can obtain an image of a stereoscopic image while conducting lens operation of focusing video for a single eye and while seeing the L image or the R image, as in a traditional case.
- displaying on the display unit 31 is switched to an image corresponding to the difference signal enables a user to make only a component of an error in the location of a congestion point, between an L image and an R image be displayed and operate a lens of the optical system 22 so as to eliminate an error in the location of a congestion point between the left and right images with high precision.
- step S15 an image corresponding to a user-specified signal is displayed on the display unit 31.
- the coding unit 27 codes a sum signal and a difference signal and supplies them to the signal transferring unit 28 and the recording unit 29.
- the sum signal coding unit 41 codes a sum signal supplied from the sum signal computing unit 25 by a specific coding method
- the difference signal coding unit 42 codes a difference signal supplied from the difference signal computing unit 26 by a specific coding method.
- a coding method used in coding a sum signal and a difference signal can be, for example, moving picture experts group (MPEG), joint photographic experts group (JPEG) 2000, or advanced video coding (AVC).
- MPEG moving picture experts group
- JPEG joint photographic experts group
- AVC advanced video coding
- the signal transferring unit 28 transfers a sum signal and a difference signal supplied from the coding unit 27 to another apparatus.
- the recording unit 29 records the sum signal and the difference signal supplied from the coding unit 27.
- the imaging apparatus 11 determines whether obtainment of an image of the object is to finish. For example, when a user provides an instruction to complete obtaining an image of the object, the imaging apparatus 11 determines that it is to finish.
- step S18 when the obtainment of an image is determined not to finish, the processing returns to step S11 and the above-described processing is repeated. In contrast to this, at step S18, when the obtainment of an image is determined to finish, the units of the imaging apparatus 11 stop their running processing, and the imaging process finishes.
- the imaging apparatus 11 during obtaining an image of an object, the imaging apparatus 11 generates, from an L signal and an R signal acquired by the obtainment of the image of the object, a sum signal of them and displays an image corresponding to the sum signal.
- displaying the image corresponding to the sum signal during obtaining the image of the object enables a user to obtain the image of the object while checking for an error in the location of a congestion point between left and right images, thus allowing focus adjustment to be carried out more easily and with high precision.
- the occurrence of an error in the location of a congestion point of a stereoscopic image acquired by obtainment of an image can be suppressed, and compatibility between 2D video and 3D video can be provided to the stereoscopic image.
- twin-lens 3D camera in which a congestion point of left eye video and right eye video and a focus position match each other at a specific location in a depth direction on a display screen.
- a twin-lens 3D camera independently carries out adjustment of a congestion point and adjustment of a focus position; if video in which left eye video and right eye video are overlaid with each other is made to be displayed and a photographer operates a lens, compatibility between 2D video and 3D video can be provided to a stereoscopic image.
- a sum signal and a difference signal output from the imaging apparatus 11 in Fig. 1 can be received by a signal reproducing apparatus 61 illustrated in Fig. 3 and reproduced, for example.
- the signal reproducing apparatus 61 illustrated in Fig. 3 includes a signal transferring unit 71, a recording/reproducing unit 72, a switching unit 73, a decoding unit 74, an inverse gamma conversion unit 75-1, an inverse gamma conversion unit 75-2, an L signal generating unit 76, an R signal generating unit 77, and a display unit 78.
- the signal transferring unit 71 receives a sum signal and a difference signal transmitted from the imaging apparatus 11 and supplies them to the switching unit 73. Note that a sum signal and a difference signal received by the signal transferring unit 71 may be supplied to the recording/reproducing unit 72 and recorded.
- the recording/reproducing unit 72 supplies a recorded sum signal and difference signal to the switching unit 73.
- the switching unit 73 supplies the sum signal and difference signal supplied from either one of the signal transferring unit 71 and the recording/reproducing unit 72 to the decoding unit 74.
- the decoding unit 74 includes a sum signal decoding unit 81 that decodes a sum signal from the switching unit 73 and a difference signal decoding unit 82 that decodes a difference signal from the switching unit 73 and supplies the decoded sum signal and difference signal to the inverse gamma conversion unit 75-1 and the inverse gamma conversion unit 75-2.
- a decoding method used in the decoding unit 74 corresponds to a coding method used in the imaging apparatus 11.
- the inverse gamma conversion unit 75-1 and the inverse gamma conversion unit 75-2 perform inverse gamma conversion on a sum signal and a difference signal supplied from the decoding unit 74 and supply the resultant signals to the L signal generating unit 76 and the R signal generating unit 77.
- the inverse gamma conversion unit 75-1 and the inverse gamma conversion unit 75-2 are also referred to simply as the inverse gamma conversion unit 75 if it is not necessary to distinguish between them.
- the L signal generating unit 76 generates an L signal from a sum signal and a difference signal supplied from the inverse gamma conversion unit 75-1 and the inverse gamma conversion unit 75-2 and supplies it to the display unit 78.
- the R signal generating unit 77 generates an R signal from a sum signal and a difference signal supplied from the inverse gamma conversion unit 75-1 and the inverse gamma conversion unit 75-2 and supplies it to the display unit 78.
- the display unit 78 stereoscopically displays an image corresponding to the L signal supplied from the L signal generating unit 76 and an R signal supplied from the R signal generating unit 77 by a specific display method that allows a user to watch a stereoscopic image using, for example, polarized glasses. That is, an L image and an R image are displayed such that the R image is observed by the right eye of a user who wears polarized glasses or other glasses and the L image is observed by the left eye.
- the signal reproducing apparatus 61 illustrated in Fig. 3 When a user provides an instruction to display a stereoscopic image, the signal reproducing apparatus 61 illustrated in Fig. 3 performs a reproducing process in response to the instruction and displays the stereoscopic image.
- the reproducing process performed by the signal reproducing apparatus 61 is described below with reference to the flowchart of Fig. 4.
- the switching unit 73 acquires a stereoscopic image signal of a stereoscopic image that a user has provided an instruction to reproduce. That is, the switching unit 73 acquires a sum signal and a difference signal that form a user-specified stereoscopic image signal and supplies them to the decoding unit 74.
- the decoding unit 74 decodes a sum signal and a difference signal supplied from the switching unit 73 and supplies them to the inverse gamma conversion unit 75. Specifically, the sum signal is decoded by the sum signal decoding unit 81 and supplied to the inverse gamma conversion unit 75-1, whereas the difference signal is decoded by the difference signal decoding unit 82 and supplied to the inverse gamma conversion unit 75-2.
- the inverse gamma conversion unit 75-1 and the inverse gamma conversion unit 75-2 perform inverse gamma conversion on a sum signal and a difference signal supplied from the sum signal decoding unit 81 and the difference signal decoding unit 82 and supply the resultant signals to the L signal generating unit 76 and the R signal generating unit 77.
- a curve that indicates an input-output characteristic of inverse gamma conversion when the horizontal axis represents an input value and the vertical axis represents an output value is a curve that is bowed downward in the vertical axis (convex downward).
- the exponent in inverse gamma conversion is not limited to (2.2), and it may be another value.
- the L signal generating unit 76 generates an L signal by dividing the sum of a sum signal and a difference signal supplied from the inverse gamma conversion unit 75 by 2 and supplies the L signal to the display unit 78.
- the R signal generating unit 77 generates an R signal by dividing the difference between a sum signal and a difference signal supplied from the inverse gamma conversion unit 75 by 2 and supplies the R signal to the display unit 78. That is, the difference signal is subtracted from the sum signal and divided by 2.
- the display unit 78 displays a stereoscopic image corresponding to an L signal and an R signal supplied from the L signal generating unit 76 and the R signal generating unit 77, and the reproducing process finishes.
- a method of displaying a stereoscopic image used in the display unit 78 can be any method, such as a polarized-glasses method, a time-division shutter method, or a lenticular system.
- the signal reproducing apparatus 61 decodes a coded sum signal and difference signal, extracts an L signal and an R signal by computation, and displays a stereoscopic image corresponding to the respective signal. Note that, also in the signal reproducing apparatus 61, displaying may be switched so as to display any one of a stereoscopic image, an L image, an R image, an image of a sum signal, and an image of a difference signal.
- the imaging apparatus 11 when the imaging apparatus 11 is remotely controlled or in another case, an image of a sum signal may be displayed in the signal reproducing apparatus 61 for focus operation.
- the signal reproducing apparatus 61 can be configured as illustrated in Fig. 5, for example. Note that in Fig. 5 the same reference numerals are used as in Fig. 3 for corresponding portions and the description thereof is omitted as appropriate.
- the signal reproducing apparatus 61 in Fig. 5 includes a signal transferring unit 71, a sum signal decoding unit 81, an inverse gamma conversion unit 75, and a display unit 78.
- the decoded signal is subjected to inverse gamma conversion by the inverse gamma conversion unit 75 and a resultant image corresponding to the signal is displayed on the display unit 78.
- the signal transferring unit 71 receives a sum signal transmitted from the imaging apparatus 11 and supplies it to the sum signal decoding unit 81.
- the sum signal decoding unit 81 decodes a sum signal supplied from the signal transferring unit 71 and supplies it to the inverse gamma conversion unit 75.
- the sum signal decoding unit 81 carries out decoding using necessary data of the sum signal so as to acquire an image having user-specified resolution. Specifically, a sum signal is decoded using, of a sum signal subjected to progressive coding, data of layers, from data of the lowest layer used in acquiring an image having the lowest resolution to data of a layer used in acquiring an image having specified resolution.
- the signal transferring unit 71 may request coded data of the sum signal from the lowest layer to a specified layer from the imaging apparatus 11 and receive only data of the sum signal necessary for decoding.
- the inverse gamma conversion unit 75 performs inverse gamma conversion on a sum signal supplied from the sum signal decoding unit 81 and supplies it to the display unit 78. Note that, in the inverse gamma conversion at step S73, substantially the same processing is performed as at step S43 in Fig. 4. Then, at step S74, the display unit 78 displays an image corresponding to the sum signal supplied from the inverse gamma conversion unit 75, and the reproducing process finishes.
- a user conducts remote control or other operations of the imaging apparatus 11 while checking an image corresponding to a sum signal displayed on the display unit 78. Also in this case, as in the case of the imaging process described with reference to Fig. 2, during obtaining an image of an object, the user can obtain the image of the object while seeing the displayed image corresponding to the sum signal and checking for an error in the location of a congestion point between the left and right images, and the occurrence of an error in focusing can be more easily suppressed.
- the signal reproducing apparatus 61 in Fig. 5 is configured to decode and display only an image corresponding to a sum signal. Thus, with this signal reproducing apparatus 61, size reduction, cost reduction, power saving, and speed enhancement of processing of the apparatus can be achieved.
- an apparatus that employs a sum signal and a difference signal coded by the imaging apparatus 11 can be an editing apparatus for editing a stereoscopic image (i.e., moving image) that is formed of a sum signal and a difference signal.
- Fig. 7 illustrates a configuration example of a signal reproducing unit incorporated in such an editing apparatus.
- a signal reproducing unit 111 includes an input unit 121, a control unit 122, a recording/reproducing unit 72, a sum signal decoding unit 81, an inverse gamma conversion unit 75, and a display unit 78. Note that in Fig. 7 the same reference numerals are used as in Fig. 3 for corresponding portions and the description thereof is omitted as appropriate.
- the input unit 121 supplies a signal corresponding to that operation to the control unit 122.
- the control unit 122 can instruct the sum signal decoding unit 81 to decode a sum signal and edit a sum signal and a difference signal recorded in the recording/reproducing unit 72.
- the recording/reproducing unit 72 records a sum signal and a difference signal acquired by obtainment of an image of an object by the imaging apparatus 11.
- the signal reproducing unit 111 When a user operates the signal reproducing unit 111 described above and provides an instruction to edit a sum signal and a difference signal recorded in the recording/reproducing unit 72, the signal reproducing unit 111 starts an edit-point recording process.
- the edit-point recording process performed by the signal reproducing unit 111 is described below with reference to the flowchart of Fig. 8.
- the sum signal decoding unit 81 acquires a sum signal of a stereoscopic image to be displayed from the recording/reproducing unit 72 and decodes it. That is, when a user operates the input unit 121, specifies a stereoscopic image, and provides an instruction to start editing the stereoscopic image, the control unit 122 instructs the sum signal decoding unit 81 to decode the sum signal forming the user-specified stereoscopic image. Then, the sum signal decoding unit 81 decodes the sum signal in accordance with the instruction from the control unit 122 and supplies it to the inverse gamma conversion unit 75.
- the sum signal has been subjected to progressive coding and resolution of an image corresponding to the sum signal to be displayed is specified by a user, decoding of the sum signal with needed resolution is conducted.
- the inverse gamma conversion unit 75 performs inverse gamma conversion on a sum signal from the sum signal decoding unit 81 and supplies it to the display unit 78. Note that, in the inverse gamma conversion at step S102, substantially the same processing is performed as at step S43 in Fig. 4. Then, at step S103, the display unit 78 displays an image corresponding to the sum signal on the basis of the sum signal supplied from the inverse gamma conversion unit 75.
- a user when an image corresponding to a sum signal is displayed, a user operates the input unit 121 as appropriate, and specifies an edit point of a stereoscopic image, that is, a starting point and an end point of a scene that the user aims to cut while, for example, fast-forwarding or fast-reproducing the displayed image.
- the control unit 122 determines whether an edit point has been specified by a user.
- an edit point is determined to have been specified
- the control unit 122 records the specified edit point of a stereoscopic image in the recording/reproducing unit 72 on the basis of the signal from the input unit 121. That is, the reproduction time of each of a starting point and an end point of the stereoscopic image specified as an edit point is recorded.
- step S106 the control unit 122 determines whether the process is to finish. For example, when a user specifies all edit points of a stereoscopic image and provides an instruction to end editing, it is determined that the process is to finish.
- step S106 When, at step S106, it is determined that the process is not to finish, the processing returns to step S101, and the above-described processing is repeated. That is, a signal of a next frame of a stereoscopic image is decoded and displayed, and an edit point is recorded in response to an operation of a user.
- step S106 when, at step S106, it is determined that the process is to finish, the edit-point recording process finishes.
- the signal reproducing unit 111 edits a stereoscopic image on the basis of an edit point recorded in the recording/reproducing unit 72. That is, at the time of the completion of the edit-point recording process, only an edit point that identifies each scene to be cut from a stereoscopic image, and the stereoscopic image is not actually edited.
- the signal reproducing unit 111 cuts a scene identified by that edit point from each of a sum signal and a difference signal that form a stereoscopic image recorded in the recording/reproducing unit 72 and edits them. That is, user-specified scenes in a sum signal are cut and combined to form into a new sum signal, whereas user-specified scenes in a difference signal are cut and combined to form into a new difference signal. Then, a moving image corresponding to the new sum signal and difference signal acquired in this way is the stereoscopic image after editing.
- the signal reproducing unit 111 reads and decodes, out of a sum signal and a difference signal that form a recorded stereoscopic image, only the sum signal, displays it, and records an edit point in response to an operation of the user. Then, the signal reproducing unit 111 records all edit points and, after the completion of an edit-point recording process, edits the stereoscopic image on the basis of a recorded edit point independently of an operation of the user.
- an object in an L image and an object in an R image are displayed in an image corresponding to a sum signal, a user can select a scene to be cut while seeing the image corresponding to the sum signal and checking whether no error in the location of a congestion point occurs between the L image for the left eye and the R image for the right eye.
- the signal reproducing unit 111 decodes and displays only an image corresponding to a sum signal, so the throughput required for decoding is half that when both an L image and an R image are decoded, as in a traditional case.
- a sum signal can be decoded and an image corresponding to the sum signal can be displayed with higher speed.
- the signal reproducing unit 111 is configured to decode and display only a sum signal. Thus, with this signal reproducing apparatus 111, size reduction, cost reduction, power saving, and speed enhancement of processing of the apparatus can be achieved.
- a single imaging unit may obtain an L image and an R image by dividing an obtained image of an object.
- an imaging apparatus can be configured as illustrated in Fig. 9, for example.
- Fig. 9 the same reference numerals are used as in Fig. 1 for corresponding portions and the description thereof is omitted as appropriate.
- An imaging apparatus 151 in Fig. 9 include a sync signal generating unit 21, an optical system 161, an imaging unit 162, a video separating unit 163, a sum signal computing unit 25, a difference signal computing unit 26, a coding unit 27, a signal transferring unit 28, a recording unit 29, and a display unit 31.
- the optical system 161 can include a lens and a polarizing element, for example, and guides light from an object to the imaging unit 162.
- the imaging unit 162 obtains an L image and an R image at different observation positions (view positions) of the object by photoelectrically converting light incident from the optical system 161.
- pixels of a light sensing surface of the imaging unit 162 include pixels on which light forming an L image, out of light from an object, is incident and pixels on which light forming an R image is incident.
- a polarizing element forming the optical system 161 separates light from an object into light forming an L image and light forming an R image by extracting only light in a particular polarizing direction and makes the light be incident on corresponding pixels the light sensing surface of the imaging unit 162.
- a polarizing element at the position of an entrance pupil of the optical system 161 and a polarizing element in each pixel on the light sensing surface of the imaging unit 162 enable only either one of light forming an L image and light forming an R image to be incident on each pixel of the imaging unit 162. Accordingly, a single image acquired by obtainment of an image by the imaging unit 162 results in generation of a signal having an L image component and an R image component. A signal generated by the imaging unit 162 is supplied to the video separating unit 163.
- the video separating unit 163 separates the signal from the imaging unit 162 into an L signal and an R signal by extracting an L signal component and an R signal component from the signal supplied from the imaging unit 162 and supplies the L and R signals to the sum signal computing unit 25 and the difference signal computing unit 26.
- the imaging apparatus 151 only an image corresponding to a sum signal formed by an L signal and an R signal is displayed on the display unit 31. And, the imaging apparatus 151 may include a gamma conversion unit that performs gamma conversion on an L signal and an R signal.
- the imaging apparatus 151 When a user operates the imaging apparatus 151 and provides an instruction to start obtaining an image of an object, the imaging apparatus 151 starts an imaging process, obtains the image of the object, and generates a stereoscopic image signal.
- the imaging process performed by the imaging apparatus 151 is described below with reference to the flowchart of Fig. 10.
- the imaging unit 162 generates a signal corresponding to the image of an object. That is, the optical system 161 separates light from an object into light forming an L signal and light forming an R signal and makes the separated light incident on corresponding pixels of the imaging unit 162.
- the imaging unit 162 generates the signal corresponding to the image of the object by photoelectrically converting light incident from the optical system 161 in synchronization with a sync signal supplied from the sync signal generating unit 21 and supplies the resultant signal to the video separating unit 163.
- the video separating unit 163 separates an L signal component and an R signal component of a signal supplied from the imaging unit 162 and performs a correction process as needed, thereby generating an L signal and an R signal and supplying the L and R signals to the sum signal computing unit 25 and the difference signal computing unit 26.
- the sum signal computing unit 25 generates a sum signal from an L signal and an R signal supplied from the video separating unit 163 and supplies the sum signal to the coding unit 27 and the display unit 31.
- the difference signal computing unit 26 generates a difference signal from an L signal and an R signal supplied from the video separating unit 163 and supplies the difference signal to the coding unit 27.
- the display unit 31 displays an image corresponding to a sum signal supplied from the sum signal computing unit 25.
- the coding unit 27 codes a sum signal and a difference signal supplied from the sum signal computing unit 25 and the difference signal computing unit 26 and supplies the coded signals to the signal transferring unit 28 and the recording unit 29.
- step S137 and step S138 is performed, and the imaging process finishes.
- This processing is substantially the same as that at step S17 and step S18 in Fig. 2, so the description thereof is omitted.
- the imaging apparatus 151 generates an L signal and an R signal from a signal corresponding to an image obtained by the single imaging unit 162.
- the optical system 161 described above separates light forming an L image and light forming an R image using a polarizing element
- the right half and the left half of a beam incident on the entrance pupil of the optical system 161 may be made to be alternately incident on the imaging unit 162 in a time division manner using a shutter (for example, refer to Japanese Unexamined Patent Application Publication No. 2001-61165).
- an L signal and an R signal are alternately generated by the imaging unit 162.
- FIG. 11 a stereoscopic image formed of an L image and an R image is obtained.
- a multi-view image in which a stereoscopic image having a different view is displayed depending on a location where a user watches it may be obtained.
- an example imaging apparatus is configured as illustrated in Fig. 11.
- an imaging apparatus 191 in Fig. 11 can be a light-field camera for obtaining an N-view image.
- the imaging apparatus 191 in Fig. 11 includes a sync signal generating unit 21, an optical system 201, an imaging unit 202, a video separating unit 203, an average signal computing unit 204, a difference signal computing unit 205-1 to a difference signal computing unit 205-(N-1), a coding unit 206, a signal transferring unit 28, a recording unit 29, and a display unit 31.
- the same reference numerals are used as in Fig. 1 for corresponding portions and the description thereof is omitted as appropriate.
- the optical system 201 can include a plurality of lenses, for example, and guides light incident from an object to the imaging unit 202.
- the imaging unit 202 generates a multi-view signal that contains signal components for N different views by photoelectrically converting light incident from the optical system 201 in synchronization with a sync signal supplied from the sync signal generating unit 21 and supplies it to the video separating unit 203.
- a beam for a view to be incident on each pixel on the light sensing surface of the imaging unit 202 is previously determined.
- Light from an object is divided into beams for a plurality of views by a microlens array disposed in the optical system 201, and the beams are guided to the pixels of the imaging unit 202.
- the video separating unit 203 separates a multi-view signal supplied from the imaging unit 202 into an image signal for each view on the basis of arrangement of pixels for views in the imaging unit 202 and supplies the image signals to the average signal computing unit 204 and the difference signal computing unit 205-1 to the difference signal computing unit 205-(N-1). Note that image signals for N views separated from a multi-view signal are referred to as image signal P1 to image signal PN, respectively.
- the average signal computing unit 204 determines the average value of pixel values of pixels of an image signal P1 to an image signal PN supplied from the video separating unit 203 and sets the determined average value as the pixel value of a new pixel, thereby generating an average signal.
- Each pixel of an image corresponding to the average signal (hereinafter referred to as average image) is the average of pixels lying in the same location in images for N views.
- the average signal computing unit 204 supplies a generated average signal to the display unit 31, the coding unit 206, and the difference signal computing unit 205-1 to the difference signal computing unit 205-(N-1).
- the difference signal computing unit 205-1 generates a difference signal D1 by determining the difference between an image signal P1 supplied from the video separating unit 203 and an average signal supplied from the average signal computing unit 204. This process is repeated for all of the difference signal computing units 205-1 through 205-(N-1) to generate the difference signals D1 through D(N-1) by determining the difference between the image signals P1 through P(N-1). These generated difference signals D1 through D(N-1) are supplied to the coding unit 206.
- the difference signal computing units 205-1 through 205-(N-1) are also referred to simply as the difference signal computing unit 205 if it is not necessary to distinguish between them.
- the image signals P1 through PN are also referred to simply as the image signal P if it is not necessary to distinguish between them, and the difference signals D1 through D(N-1) are also referred to simply as the difference signal D if it is not necessary to distinguish between them.
- the coding unit 206 includes an average signal coding unit 211 that codes an average signal from the average signal computing unit 204 and difference signal coding units 212-1 through 212-(N-1) that each codes the difference signal D from the difference signal computing unit 205.
- the coding unit 206 supplies the average signal and difference signal D acquired by coding to the recording unit 29 and the signal transferring unit 28.
- difference signal coding units 212-1 through 212-(N-1) are also referred to simply as the difference signal coding unit 212 if it is not necessary to distinguish between them.
- the imaging apparatus 191 when a user operates the imaging apparatus 191 and provides an instruction to start obtaining a signal corresponding to an image of an object, the imaging apparatus 191 starts an imaging process, obtains the signal corresponding to the image of the object, and generates a multi-view signal.
- the imaging process performed by the imaging apparatus 191 is described below with reference to the flowchart of Fig. 12.
- the imaging unit 202 generates a signal corresponding to an image of an object. That is, the optical system 201 collects beams for views incident from an object and causes them to be incident on the imaging unit 202. The imaging unit 202 generates a multi-view signal corresponding to an image of an object by photoelectrically converting the beams incident from the optical system 201. Then, the imaging unit 202 supplies the multi-view signal to the video separating unit 203.
- the video separating unit 203 separates a multi-view signal supplied from the imaging unit 202 into an image signal P for each view and supplies them to the average signal computing unit 204 and the difference signal computing unit 205. Note that in the video separating unit 203 a correction process, such as gamma conversion, defect correction, or white balance adjustment, may be performed on an image signal P for each view.
- the average signal computing unit 204 generates an average signal by determining the average value of an image signal P1 to an image signal PN supplied from the video separating unit 203 and supplies it to the display unit 31, the average signal coding unit 211, and the difference signal computing unit 205. That is, the sum of image signals P is divided by the number N of views (the number of the image signals P) to generate an average signal.
- the difference signal computing unit 205 generates a difference signal D by subtracting an average signal supplied from the average signal computing unit 204 from an image signal P supplied from the video separating unit 203 and supplies it to the difference signal coding unit 212.
- the difference signal computing unit 205-1 determines the difference between the image signal P1 and the average signal and thus generates the difference signal D1.
- the display unit 31 displays an average image corresponding to an average signal supplied from the average signal computing unit 204. Because the average image is an image in which images of an object observed from views are overlaid with each other, a user can obtain an image while seeing the average image displayed on the display unit 31 and checking whether no error in the location of a congestion point occurs between the images for views. With this, the occurrence of an error in the location of a congestion point in a multi-view image can be suppressed more easily.
- the coding unit 206 codes an average signal from the average signal computing unit 204 and a difference signal D from the difference signal computing unit 205 and supplies them to the signal transferring unit 28 and the recording unit 29. That is, the average signal coding unit 211 codes the average signal, and the difference signal coding unit 212 codes the difference signal D.
- the signal transferring unit 28 transfers an average signal and a difference signal D supplied from the coding unit 206 to another apparatus.
- the recording unit 29 records an average signal and a difference signal D supplied from the coding unit 206.
- the imaging apparatus 191 determines whether acquisition of an image of an object is finished. For example, the imaging apparatus 191 may determine that image acquisition is finished, by receiving a user instruction.
- step S168 when it is determined that the process is not finished, the processing returns to step S161 and the above-described processing is repeated. In contrast to this, at step S168, when it is determined that the process is finished, the units of the imaging apparatus 191 stop their running processing, and the imaging process is complete.
- the imaging apparatus 191 generates an average signal from image signals P from acquired views, by obtaining a signal corresponding to an image of an object during obtaining the signal corresponding to the image of the object.
- the imaging apparatus then displays an average image.
- the imaging apparatus 191 is configured to acquire a multi-view signal that contains components for a plurality of views using the single optical system 201 and the single imaging unit 202
- the optical system 201 and the imaging unit 202 may be provided for each view.
- image signals P for N views are directly acquired by obtaining a signal corresponding to an image of an object, the video separating unit 203 is not necessary.
- an average signal and a difference signal D output from the imaging apparatus 191 in Fig. 11 can be received and reproduced by a signal reproducing apparatus 241 illustrated in Fig. 13, for example.
- the signal reproducing apparatus 241 illustrated in Fig. 13 includes a signal transferring unit 71, a recording/reproducing unit 72, a switching unit 73, a decoding unit 251, a signal generating unit 252-1 to a signal generating unit 252-N, and a display unit 253. Note that in Fig. 13 the same reference numerals are used as in Fig. 3 for corresponding portions and the description thereof is omitted as appropriate.
- the decoding unit 251 includes an average signal decoding unit 261 that decodes an average signal from the switching unit 73 and difference signal decoding units 262-1 through 262-(N-1) that decode difference signals D1 through D(N-1) from the switching unit 73.
- the decoding unit 251 supplies the average signal and difference signals D through D(N-1) to the signal generating units 252-1 through 252-N.
- difference signal decoding units 262-1 through 262-(N-1) are also referred to simply as the difference signal decoding unit 262 if it is not necessary to distinguish between them.
- the signal generating units 252-1 through 252-N generate image signals P for views from an average signal and difference signals D supplied from the decoding unit 251 and supply them to the display unit 253. Note that hereinafter the signal generating units 252-1 through 252-N are also referred to simply as the signal generating unit 252 if it is not necessary to distinguish between them.
- the display unit 253 displays an N-view image corresponding to an image signal P for each view supplied from the signal generating unit 252. ⁇ Description of Reproducing Process>
- the signal reproducing apparatus 241 illustrated in Fig. 13 When being instructed by a user to display an N-view image, the signal reproducing apparatus 241 illustrated in Fig. 13 performs a reproducing process in response to the instruction and displays the N-view image. The reproducing process performed by the signal reproducing apparatus 241 is described below with reference to the flowchart of Fig. 14.
- the switching unit 73 acquires an N-view signal in response to a user command. That is, the switching unit 73 acquires a signal of a user-specified N-view image, that is, an average signal and a difference signal D from the signal transferring unit 71 and the recording/reproducing unit 72, and supplies them to the decoding unit 251.
- the decoding unit 251 decodes an average signal and a difference signal D supplied from the switching unit 73 and supplies them to the signal generating unit 252. Specifically, the average signal decoding unit 261 decodes an average signal, and the difference signal decoding unit 262 decodes a difference signal D.
- the signal generating unit 252 generates an image signal P for each view on the basis of an average signal and a difference signal D supplied from the decoding unit 251 and supplies them to the display unit 253.
- the signal generating unit 252-1 generates an image signal P1 by determining the sum of a difference signal D1 and an average signal.
- the signal generating units 252-2 through 252-(N-1) generate image signals P2 through P(N-1) by determining the sums of a difference signals D2 through D(N-1) and an average signal.
- the signal generating unit 252-N generates an image signal PN by subtracting the sum of the difference signals D1 through D(N-1) from the average signal.
- the display unit 253 employs a lenticular method or another method to display an N-view image corresponding to an image signals P1 through PN for views supplied from the signal generating unit 252, and the reproducing process finishes.
- the signal reproducing apparatus 241 decodes a coded average signal and difference signal, extracts an image signal for each view by computation, and displays an N-view image corresponding to the respective image signal.
- all of the units in the above-described imaging apparatus 11, signal reproducing apparatus 61, signal reproducing unit 111, imaging apparatus 151, imaging apparatus 191, and signal reproducing apparatus 241 can be implemented using specialized hardware. In this manner, processes performed in these apparatus may be more easily performed in parallel.
- the above-described series of processes may also be implemented by general-purpose processors executing software. If the series of processes is executed by software, a program forming the software is installed from a program storage medium into one or more processors incorporated in dedicated hardware or into a device that can perform various functions by installation of various kinds of programs, for example, a general-purpose personal computer.
- Fig. 15 is a block diagram that illustrates a configuration example of hardware of a computer that executes the above-described series of processes using a program.
- a central processing unit (CPU) 301 In the computer, a central processing unit (CPU) 301, a read-only memory (ROM) 302, and a random-access memory (RAM) 303 are connected to each other by a bus 304.
- CPU central processing unit
- ROM read-only memory
- RAM random-access memory
- the bus 304 is connected to an input/output interface 305.
- the input/output interface 305 is connected to an input unit 306 including, for example, a keyboard, a mouse, and/or a microphone, an output unit 307 including, for example, a display and/or a speaker, a storage unit 308 including, for example, a hard disk and/or non-volatile memory, a communication unit 309 including, for example, a network interface, and a drive 310 for driving a removable medium 311, such as a magnetic disk, an optical disk, a magneto-optical disk, or semiconductor memory.
- a removable medium 311 such as a magnetic disk, an optical disk, a magneto-optical disk, or semiconductor memory.
- the above-described series of processes is performed by the CPU 301 loading a program stored in the storage unit 308 into the RAM 303 through the input/output interface 305 and the bus 304 and executing the program, for example.
- a program executed by a computer can be provided by being stored in the removable medium 311 being a package medium, such as a magnetic medium (including a flexible disk), an optical disk (e.g., compact-disk read-only memory (CD-ROM) or digital versatile disc (DVD)), a magneto-optical disk, or semiconductor memory, or through a wired or wireless transmission medium, such as a local area network, the Internet, or digital satellite broadcasting.
- a package medium such as a magnetic medium (including a flexible disk), an optical disk (e.g., compact-disk read-only memory (CD-ROM) or digital versatile disc (DVD)), a magneto-optical disk, or semiconductor memory, or through a wired or wireless transmission medium, such as a local area network, the Internet, or digital satellite broadcasting.
- a program can be installed into the storage unit 308 through the input/output interface 305 by attachment of the removable medium 311 to the drive 310. And, a program can be installed into the storage unit 308 by being received by the communication unit 309 through a wired or wireless transmission medium. In addition, a program can be stored in advance in the ROM 302 or the storage unit 308.
- a program executed by a computer may be a program by which processes are executed on a time-series basis in the order described in this specification or may also be a program by which processes are executed in parallel or at a necessary time, such as at the time of calling.
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Testing, Inspecting, Measuring Of Stereoscopic Televisions And Televisions (AREA)
- Studio Devices (AREA)
- Image Processing (AREA)
Abstract
Description
Also disclosed is an electronic device for processing images. The electronic device may receive an image signal comprising a left-image signal representing a left image and a right-image signal representing a right image. The electronic device may further generate a sum signal by combining the left-image signal and the right-image signal. The electronic device may also display a sum image corresponding to the sum signal, the displayed image including a convergence point and a focus point.
Further disclosed is tangibly embodied non-transitory computer-readable storage medium including instructions that, when executed by a processor, perform a method for processing images on an electronic device. The method may including receiving an image signal comprising a left-image signal representing a left image and a right-image signal representing a right image. The method may further include generating a sum signal by combining the left-image signal and the right-image signal. The method may also include displaying a sum image corresponding to the sum signal, the displayed image including a convergence point and a focus point.
Fig. 1 illustrates a configuration example of one embodiment of an imaging apparatus.
<Description of Imaging Process>
<Configuration of Signal Reproducing Apparatus>
<Description of Reproducing Process>
And, for example, when the
<Description of Reproducing Process>
And, one possible example of an apparatus that employs a sum signal and a difference signal coded by the
And, although an example in which two imaging units 23 obtain an L signal and an R signal has been described with reference to Fig. 1, a single imaging unit may obtain an L image and an R image by dividing an obtained image of an object.
<Description of Imaging Process>
And, with reference to Fig. 1, an example is described in which a stereoscopic image formed of an L image and an R image is obtained. However, a multi-view image in which a stereoscopic image having a different view is displayed depending on a location where a user watches it may be obtained. In such a case, an example imaging apparatus is configured as illustrated in Fig. 11.
by photoelectrically converting light incident from the
<Description of Imaging Process>
<Description of Reproducing Process>
Reference Signs List
Claims (15)
- A computer-implemented method for processing images on an electronic device, the method comprising:
receiving an image signal comprising a left-image signal representing a left image and a right-image signal representing a right image;
generating a sum signal for the image signal by combining the left-image signal and the right-image signal;
displaying a sum image corresponding to the sum signal, the displayed image including a convergence point and a focus point. - A method of claim 1, further comprising:
encoding the sum signal; and
outputting the encoded sum signal. - A method of claim 1, further comprising:
separating the received image signal into the left-image signal and the right-image signal. - A method of claim 3, further comprising:
performing a correction on the separated left-image signal and the separated right-image signal. - A method of claim 1, further comprising:
performing a gamma conversion on the left-image signal and the right-image signal. - A method of claim 1, wherein the sum signal represents an image comprising an overlay of the left image and the right image.
- A method of claim 1, wherein the sum signal comprises a sum of pixel values for the left-image signal and the right-image signal when the pixels of the respective images are in a same location in a same frame.
- A method of claim 1, wherein the sum signal comprises a normalization of a sum of pixel values for the left-image signal and the right-image signal when the pixels of the corresponding images are in a same location in a same frame.
- A method of claim 1, further comprising:
generating a difference signal for the image signal by combining the left-image signal and the right-image signal; - A method of claim 2, wherein encoding the sum signal further comprises encoding a difference signal and outputting the encoded sum signal further comprises outputting the encoded difference signal.
- A method of claim 9, further comprising:
displaying a difference image corresponding to the difference signal. - A method of claim 9, wherein the difference signal comprises a difference of pixel values for the left-image signal and the right-image signal when the pixels of the respective images are in a same location in a same frame.
- A method of claim 9, wherein the difference signal comprises a normalization of a difference of pixel values for the left-image signal and the right-image signal when the pixels of the corresponding images are in a same location in a same frame.
- An electronic device for processing images, the device comprising:
an imaging unit configured to receive an image signal comprising a left-image signal and a right-image signal;
a signal computing unit configured to generate a sum signal for the image signal by combining the left-image signal and the right-image signal; and
a display unit configured to display a sum image corresponding to the sum signal, the displayed image including a convergence point and a focus point. - A tangibly embodied non-transitory computer-readable storage medium including instructions that, when executed by a processor, perform a method for processing images, the method comprising:
receiving an image signal comprising a left-image signal representing a left image and a right-image signal representing a right image;
generating a sum signal for the image signal by combining the left-image signal and the right-image signal;
displaying a sum image corresponding to the sum signal, the displayed image including a convergence point and a focus point.
Priority Applications (8)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
EP11812091.4A EP2599320A4 (en) | 2010-07-30 | 2011-07-29 | Image processing apparatus and method and program |
MX2013000939A MX2013000939A (en) | 2010-07-30 | 2011-07-29 | Image processing apparatus and method and program. |
CA2804362A CA2804362A1 (en) | 2010-07-30 | 2011-07-29 | Image processing apparatus and method and program |
RU2013103037/08A RU2013103037A (en) | 2010-07-30 | 2011-07-29 | DEVICE AND METHOD FOR PROCESSING IMAGES AND PROGRAM |
US13/811,752 US20130120530A1 (en) | 2010-07-30 | 2011-07-29 | Image processing apparatus and method and program |
BR112013001730A BR112013001730A2 (en) | 2010-07-30 | 2011-07-29 | computer-implemented image processing method, electronic image processing device, and tangibly incorporated non-temporary computer readable storage media |
AU2011284087A AU2011284087A1 (en) | 2010-07-30 | 2011-07-29 | Image processing apparatus and method and program |
CN2011800361451A CN103026714A (en) | 2010-07-30 | 2011-07-29 | Image processing apparatus and method and program |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
JP2010-172503 | 2010-07-30 | ||
JP2010172503A JP2012034215A (en) | 2010-07-30 | 2010-07-30 | Image processing apparatus and method and program |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2012014494A1 true WO2012014494A1 (en) | 2012-02-02 |
Family
ID=45529720
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/JP2011/004318 WO2012014494A1 (en) | 2010-07-30 | 2011-07-29 | Image processing apparatus and method and program |
Country Status (10)
Country | Link |
---|---|
US (1) | US20130120530A1 (en) |
EP (1) | EP2599320A4 (en) |
JP (1) | JP2012034215A (en) |
CN (1) | CN103026714A (en) |
AU (1) | AU2011284087A1 (en) |
BR (1) | BR112013001730A2 (en) |
CA (1) | CA2804362A1 (en) |
MX (1) | MX2013000939A (en) |
RU (1) | RU2013103037A (en) |
WO (1) | WO2012014494A1 (en) |
Cited By (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN104247413A (en) * | 2012-04-23 | 2014-12-24 | Lg电子株式会社 | Signal processing device and method for 3D service |
US9470820B2 (en) | 2012-07-31 | 2016-10-18 | Asahi Glass Company, Limited | Microlens array, image pickup element package, and method for manufacturing microlens array |
Families Citing this family (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US10244223B2 (en) * | 2014-01-10 | 2019-03-26 | Ostendo Technologies, Inc. | Methods for full parallax compressed light field 3D imaging systems |
EP3264761A1 (en) * | 2016-06-23 | 2018-01-03 | Thomson Licensing | A method and apparatus for creating a pair of stereoscopic images using least one lightfield camera |
KR102682823B1 (en) | 2023-08-03 | 2024-07-08 | (주)캔탑스 | Junction Control System Based on Location of OHT Using Power Communication Line |
Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JPH0746630A (en) * | 1993-07-28 | 1995-02-14 | Victor Co Of Japan Ltd | Stereoscopic video signal compression device |
JPH09116882A (en) * | 1995-10-13 | 1997-05-02 | Ricoh Co Ltd | Audio visual communication equipment |
JPH11355624A (en) * | 1998-06-05 | 1999-12-24 | Fuji Photo Film Co Ltd | Photographing device |
JP2002218501A (en) * | 2001-01-18 | 2002-08-02 | Olympus Optical Co Ltd | Image pickup device |
JP2008289065A (en) * | 2007-05-21 | 2008-11-27 | Fujifilm Corp | Imaging device, image reproducing device, imaging method, and program |
Family Cites Families (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JPS59140788A (en) * | 1983-01-31 | 1984-08-13 | Sony Corp | Display device of pseudo stereoscopic image |
JP4306638B2 (en) * | 2005-04-14 | 2009-08-05 | ソニー株式会社 | Image processing system, imaging apparatus, imaging method, image reproducing apparatus, and image reproducing method |
JP4172504B2 (en) * | 2006-06-22 | 2008-10-29 | ソニー株式会社 | Image processing apparatus, imaging apparatus, and method |
JP4757812B2 (en) * | 2007-02-20 | 2011-08-24 | 富士フイルム株式会社 | Stereoscopic imaging apparatus, method, and program |
JP4851406B2 (en) * | 2007-08-31 | 2012-01-11 | 富士フイルム株式会社 | Image display method for adjustment in multi-view imaging system and multi-view imaging system |
US8300086B2 (en) * | 2007-12-20 | 2012-10-30 | Nokia Corporation | Image processing for supporting a stereoscopic presentation |
-
2010
- 2010-07-30 JP JP2010172503A patent/JP2012034215A/en not_active Withdrawn
-
2011
- 2011-07-29 RU RU2013103037/08A patent/RU2013103037A/en unknown
- 2011-07-29 EP EP11812091.4A patent/EP2599320A4/en not_active Withdrawn
- 2011-07-29 WO PCT/JP2011/004318 patent/WO2012014494A1/en active Application Filing
- 2011-07-29 CA CA2804362A patent/CA2804362A1/en not_active Abandoned
- 2011-07-29 AU AU2011284087A patent/AU2011284087A1/en not_active Abandoned
- 2011-07-29 CN CN2011800361451A patent/CN103026714A/en active Pending
- 2011-07-29 BR BR112013001730A patent/BR112013001730A2/en not_active Application Discontinuation
- 2011-07-29 US US13/811,752 patent/US20130120530A1/en not_active Abandoned
- 2011-07-29 MX MX2013000939A patent/MX2013000939A/en not_active Application Discontinuation
Patent Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JPH0746630A (en) * | 1993-07-28 | 1995-02-14 | Victor Co Of Japan Ltd | Stereoscopic video signal compression device |
JPH09116882A (en) * | 1995-10-13 | 1997-05-02 | Ricoh Co Ltd | Audio visual communication equipment |
JPH11355624A (en) * | 1998-06-05 | 1999-12-24 | Fuji Photo Film Co Ltd | Photographing device |
JP2002218501A (en) * | 2001-01-18 | 2002-08-02 | Olympus Optical Co Ltd | Image pickup device |
JP2008289065A (en) * | 2007-05-21 | 2008-11-27 | Fujifilm Corp | Imaging device, image reproducing device, imaging method, and program |
Non-Patent Citations (1)
Title |
---|
See also references of EP2599320A4 * |
Cited By (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN104247413A (en) * | 2012-04-23 | 2014-12-24 | Lg电子株式会社 | Signal processing device and method for 3D service |
CN104247413B (en) * | 2012-04-23 | 2017-12-29 | Lg 电子株式会社 | Signal handling equipment and method for 3D services |
US9470820B2 (en) | 2012-07-31 | 2016-10-18 | Asahi Glass Company, Limited | Microlens array, image pickup element package, and method for manufacturing microlens array |
Also Published As
Publication number | Publication date |
---|---|
MX2013000939A (en) | 2013-02-15 |
EP2599320A1 (en) | 2013-06-05 |
JP2012034215A (en) | 2012-02-16 |
BR112013001730A2 (en) | 2016-05-31 |
CA2804362A1 (en) | 2012-02-02 |
EP2599320A4 (en) | 2014-07-16 |
CN103026714A (en) | 2013-04-03 |
AU2011284087A1 (en) | 2013-02-07 |
RU2013103037A (en) | 2014-07-27 |
US20130120530A1 (en) | 2013-05-16 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
KR101185870B1 (en) | Apparatus and method for processing 3 dimensional picture | |
US20120050478A1 (en) | Method and System for Utilizing Multiple 3D Source Views for Generating 3D Image | |
US8810565B2 (en) | Method and system for utilizing depth information as an enhancement layer | |
US20120054575A1 (en) | Method and system for error protection of 3d video | |
JP2014502818A (en) | Primary image and secondary image image capturing apparatus and method for image processing | |
JP2006352877A (en) | Image display mode conversion method and apparatus | |
US20120050481A1 (en) | Method and system for creating a 3d video from a monoscopic 2d video and corresponding depth information | |
JP2012186781A (en) | Image processing device and image processing method | |
WO2012147329A1 (en) | Stereoscopic intensity adjustment device, stereoscopic intensity adjustment method, program, integrated circuit, and recording medium | |
US20140334537A1 (en) | Image processing device and image processing method | |
WO2012014494A1 (en) | Image processing apparatus and method and program | |
US10037335B1 (en) | Detection of 3-D videos | |
JP5395911B2 (en) | Stereo image encoding apparatus and method | |
JP2013025649A (en) | Image processing device, image processing method, and program | |
US8941718B2 (en) | 3D video processing apparatus and 3D video processing method | |
EP2485494A1 (en) | Method and system for utilizing depth information as an enhancement layer | |
JP2015019326A (en) | Encoding device, encoding method, decoding device, and decoding method | |
JP2013150071A (en) | Encoder, encoding method, program and storage medium | |
JP2020120322A (en) | Distance image coding device and program of the same, and distance image decoding device and program of the same | |
US9369698B2 (en) | Imaging apparatus and method for controlling same | |
US20140072271A1 (en) | Recording apparatus, recording method, reproduction apparatus, reproduction method, program, and recording reproduction apparatus | |
EP2485495A2 (en) | Method and system for creating a 3D video from a monoscopic 2D video and corresponding depth information | |
JP2013247653A (en) | Image processing apparatus and image processing method | |
EP2485493A2 (en) | Method and system for error protection of 3D video | |
WO2013136832A1 (en) | Stereoscopic image display control device and stereoscopic image display control method |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
WWE | Wipo information: entry into national phase |
Ref document number: 201180036145.1 Country of ref document: CN |
|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 11812091 Country of ref document: EP Kind code of ref document: A1 |
|
WWE | Wipo information: entry into national phase |
Ref document number: 2011812091 Country of ref document: EP |
|
ENP | Entry into the national phase |
Ref document number: 2804362 Country of ref document: CA |
|
WWE | Wipo information: entry into national phase |
Ref document number: 13811752 Country of ref document: US Ref document number: MX/A/2013/000939 Country of ref document: MX |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
ENP | Entry into the national phase |
Ref document number: 2011284087 Country of ref document: AU Date of ref document: 20110729 Kind code of ref document: A |
|
ENP | Entry into the national phase |
Ref document number: 2013103037 Country of ref document: RU Kind code of ref document: A |
|
REG | Reference to national code |
Ref country code: BR Ref legal event code: B01A Ref document number: 112013001730 Country of ref document: BR |
|
ENP | Entry into the national phase |
Ref document number: 112013001730 Country of ref document: BR Kind code of ref document: A2 Effective date: 20130123 |