KR20170096071A - Three dimensional virtual reality video image production method and apparatus thereof - Google Patents
Three dimensional virtual reality video image production method and apparatus thereof Download PDFInfo
- Publication number
- KR20170096071A KR20170096071A KR1020160017006A KR20160017006A KR20170096071A KR 20170096071 A KR20170096071 A KR 20170096071A KR 1020160017006 A KR1020160017006 A KR 1020160017006A KR 20160017006 A KR20160017006 A KR 20160017006A KR 20170096071 A KR20170096071 A KR 20170096071A
- Authority
- KR
- South Korea
- Prior art keywords
- image
- stitching
- images
- right eye
- virtual reality
- Prior art date
Links
Images
Classifications
-
- H04N13/0282—
-
- H04N13/0242—
-
- H04N5/23238—
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N5/00—Details of television systems
- H04N5/222—Studio circuitry; Studio devices; Studio equipment
- H04N5/262—Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects
- H04N5/2628—Alteration of picture size, shape, position or orientation, e.g. zooming, rotation, rolling, perspective, translation
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Image Processing (AREA)
Abstract
There is provided a method for producing a three-dimensional virtual reality image, the method comprising: (a) generating a two-dimensional original image by using a 360-degree omnidirectional imaging apparatus, wherein the multi- Including right eye images; (b) generating a single panoramic left eye image by performing automatic image stitching processing of the plurality of left eye images photographed at the same point in time, and performing automatic image stitching processing of the plurality of right eye images to generate a single panoramic right eye image; (c) performing a second-half image processing for adjusting the distortion of the stitching area superimposed on each of the original images in the automatic image stitching process for each of the panoramic left eye image and the panoramic right eye image, Generating an image and a right eye image; And (d) fabricating a three-dimensional virtual reality image using the distortion-adjusted left eye image and the right eye image.
Description
BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image production method and apparatus, and more particularly, to a method and apparatus for producing a three-dimensional virtual reality image.
Recently, interest in 3D VR (stereoscopic virtual reality) image has been amplified, and research on 3D VR image has been actively carried out.
Generally, it is known that humans feel the most stereoscopic effect due to the parallax between the eyes. And you can feel the immersion feeling by maximizing the image of all directions. In recent years, much research has been conducted on techniques for providing high-resolution 3D VR images without distortion.
In order to produce a 3D VR image, a stitching rendering operation is required to convert several 2D images into one 3D VR image. However, according to the related art, such a stitching rendering operation is complicated.
The present invention provides a 3D VR image production method and apparatus for converting 14 2D input images into one image without distortion, thereby providing high resolution 3D VR images.
According to an aspect of the present invention, there is provided a method for producing a three-
(a) generating a multi-view two-dimensional original image by using a 360-degree omnidirectional image pickup device, wherein the multi-view two-dimensional original image includes a plurality of left eye images and a plurality of right eye images; (b) generating a single panoramic left eye image by performing automatic image stitching processing of the plurality of left eye images photographed at the same point in time, and performing automatic image stitching processing of the plurality of right eye images to generate a single panoramic right eye image; (c) performing a second-half image processing for adjusting the distortion of the stitching area superimposed on each of the original images in the automatic image stitching process for each of the panoramic left eye image and the panoramic right eye image, Generating an image and a right eye image; And (d) fabricating a three-dimensional virtual reality image using the distortion-adjusted left eye image and the right eye image.
In one embodiment, in the step (b), the automatic image stitching processing includes:
Adjusting the distortions generated in each original image based on distortion information corresponding to the focal length and the distortion aberration of each of the plurality of cameras in the omnidirectional image photographing apparatus and stitching the distorted images through inverse adjustment .
In one embodiment, in the step (c), the second-
A blending effect and a manual stitching process may be performed on the image subjected to the automatic image stitching so that the distortion rate and the sharpness of the stitching area overlapping the images are uniform throughout the single panorama image.
In one embodiment, the omnidirectional image capturing apparatus includes two cameras corresponding to the left eye camera and the right eye camera in the 360-degree league (RIG), one pair on the upper side of the league, one image on the lower side of the league, A total of 14 images are acquired at the same time point,
The left eye panorama image and the right eye panorama image can be generated by automatic stitching processing of 7 left eye images and 7 right eye images, respectively.
According to another aspect of the present invention, there is provided a three-dimensional virtual reality image producing apparatus including a 360-degree omnidirectional radiographing apparatus and an image processing apparatus for executing the above-mentioned image producing method.
According to the embodiment of the present invention, a 3D VR image production method and apparatus for converting 14 2D input images into one image without distortion are provided, thereby making it possible to more easily produce 3D VR images of 360 degree high resolution and super high definition There is an effect that can be.
BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 is an overall flowchart of a 3D VR image production method according to an embodiment of the present invention; FIG.
BACKGROUND OF THE INVENTION 1. Field of the Invention [0001] The present invention relates to a 3D VR imaging apparatus, and more particularly,
FIG. 3 illustrates 2D images acquired by the omnidirectional imaging device of FIG. 2. FIG.
4 and 5 illustrate an image after distortion is automatically adjusted from a 2D photographed original image obtained by the omnidirectional image photographing apparatus.
FIG. 6 is a view illustrating a panoramic image generated by an image automatic stitching method according to an embodiment of the present invention, and a sensory image that may occur when the panoramic image is generated.
7 is a view illustrating a corrected image of a heterogeneous image illustrated in FIG. 6 by a second-half image processing process according to an embodiment of the present invention.
FIG. 8 is a view illustrating an image after automatic image stitching and late image processing according to an embodiment of the present invention; FIG.
BRIEF DESCRIPTION OF THE DRAWINGS The present invention is capable of various modifications and various embodiments, and specific embodiments are illustrated in the drawings and described in detail in the detailed description. It is to be understood, however, that the invention is not to be limited to the specific embodiments, but includes all modifications, equivalents, and alternatives falling within the spirit and scope of the invention.
DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, the present invention will be described in detail with reference to the accompanying drawings. In addition, numerals (e.g., first, second, etc.) used in the description of the present invention are merely an identifier for distinguishing one component from another.
Also, throughout the specification, when an element is referred to as being "connected" or "connected" with another element, the element may be directly connected or directly connected to the other element, It should be understood that, unless an opposite description is present, it may be connected or connected via another element in the middle. Also, throughout the specification, when an element is referred to as "including" an element, it is understood that the element may include other elements as well, without departing from the other elements unless specifically stated otherwise. Also, the terms "a "," module ", and the like in the description mean a unit for processing at least one function or operation, which means that it can be implemented by one or more hardware or software or a combination of hardware and software .
1 is an overall flowchart of a 3D VR image production method according to an embodiment of the present invention. 2 is a view for explaining an omniazimetric imaging apparatus used for producing a 3D VR image according to an embodiment of the present invention, and FIG. 3 is a view for explaining 2D images obtained by the omnidirectional imaging apparatus of FIG. 2 FIG. Hereinafter, with reference to FIG. 1, a general procedure of a 3D VR image production method according to an embodiment of the present invention will be described with reference to FIG. 2 and FIG.
Referring to FIG. 1, the image production Production method may be as follows.
First, 14 video cameras are mounted on a 360 RIG (ie, one pair of two cameras on the top, one pair of cameras on the bottom, and five pairs of 10 cameras on an equally spaced angle Here, the pair of cameras are responsible for the left eye image and the right eye image, respectively, see FIG. 2) to obtain a 2D original image of 360 degrees all around the same point (see FIG. 3).
At this time, it is possible to activate the control sharing function for simultaneously controlling the recording of 14 cameras to one Wi-Fi remote controller. When control sharing is completed, simultaneous recording through 14 cameras is started with the Wi-Fi remote control. At this time, a click sound or a slate for synchronization of post-production synchronization of each image may be inserted. Simultaneous recording by 14 cameras can be stopped by using the Wi-Fi remote control after shooting the desired scene or scene. Recorded image data of 14 cameras can be backed up to SSD HDD.
Simultaneous recording may be performed simultaneously with the image acquisition process described above, and the simultaneous recording method may be as follows. First, you can set up a wireless microphone to record the scene sound with the actor's ambassador, and perform simultaneous recording on the spot. In this case, you can record a click sound or a slate sound for post-production synchronization. The recorded audio data can be backed up to the SSD HDD.
The video data and audio data obtained as described above can be stored in a total of three backup storages. Here, the first backup data may be used for delivery to the editing room, the second backup data may be used for delivery to a sound mixing company, a CG (computer graphic) company, a DI company, and the like.
In addition, referring to FIG. 1, the post-processing process of the photographed image and audio data may be performed in the following order.
First, the captured image file name is modified to create a file list for editing (Log Data).
Thereafter, image auto stitching is performed on the 14 captured images for editing, sound work, and C.G operations. Such automatic image stitching is an automatic process that can proceed at a high speed, though the sophistication of the final image is reduced. As a result of such automatic image stitching, a low-capacity proxy file (i.e., a temporary file) can be produced. A specific method of the image automatic stitching method will be described later with reference to FIGS. 4 and 5. FIG.
The synchronized recording sound is synchronized with the proxy file generated as described above and matched. Thereafter, editing and previewing of the proxy file are performed to output the final edited video, and a file list (XML, EDL, etc.) of the final edited video is output. A high-resolution image sequence (DPX, TIFF, TGA, etc.) can be output from the photographed original as a file list.
The proxy file is replaced with the output high-resolution image sequence, and color matching and primary color correction are performed on the high-resolution image sequence. Thereafter, a sophisticated manual stitching operation is performed on the high-resolution image sequence, and final color correction, sound effects and music operations are performed. As a result, the image DPX sequence having been subjected to the final color correction operation can be outputted, and the effect sound and the music WAV file after the mixing operation can be outputted.
When the image processing for each of the left eye image and the right eye image is completed through the above-described process, the 3D stereoscopic image is adjusted to the 3D VR, the alignment adjustment process, the zero point re-adjustment process, the HIT (Horizontal Image Transition) Perform the time difference adjustment process. In this way, MOV (Prores422 HQ) files can be output as Final Mastering VIDEO, and uncompressed WAV (5.1, 3D stereo) files can be output as Final Mastering AUDIO. With respect to the final file, the media codec and specification conversion may be performed according to the download platform, or the streaming media codec and specification conversion may be performed according to the VOD platform.
The overall process of the 3D VR image production method has been described above with reference to FIGS. 1 to 3. FIG. Hereinafter, an image automatic stitching method and a rear image processing process according to an embodiment of the present invention will be described with reference to FIGS. 4 to 8. FIG.
First, among the above-described 3D VR image production methods, an image automatic stitching method will be described with reference to FIGS. 4 and 5. FIG. 4 and 5 are views illustrating an image after distortion is automatically adjusted from a 2D photographed original image acquired by the omniazimetric imaging apparatus.
In the embodiment of the present invention, distortion information is adjusted for each of the fourteen shot original images, and the distortion information is automatically synthesized into a single image to generate a panoramic image. Referring to the top image of FIG. 4, any one of the 14 photographed original images is illustrated. Such a photographed original image is obtained in a concave or convex curved state compared with the actual photographed body itself due to the optical focal length and distortion aberration of the imaging lens of the camera. Therefore, in consideration of the distortion ratio depending on the lens characteristics, it is necessary to straighten the distortion in the opposite direction, which is automatically adjusted according to the embodiment of the present invention. The distorted images thus automatically adjusted are illustrated in the bottom image of FIG. 4 and in FIG.
As described above, the images of which the respective image distortions are automatically adjusted are automatically stitched while forming overlapping areas according to the viewing angle range by each camera. According to such automatic stitching, a single panoramic image as in the top image of FIG. 6 can be generated.
Among the 3D VR image production methods described above, the latter image processing process (mainly manual stitching process) will be described with reference to FIGS. 6 to 8. FIG. 6 is a view illustrating a panoramic image generated by an image automatic stitching method according to an embodiment of the present invention and a heterogeneous image that may occur when the panoramic image is generated. FIG. 8 is a view illustrating an image after auto-stitching and a second-half image processing according to an embodiment of the present invention. FIG. to be.
When the automatic stitching process described above with reference to FIGS. 4 and 5 is completed, a single panoramic image like the top image of FIG. 6 can be generated very quickly and easily. However, referring to the top image of FIG. 6, image distortion may occur as identified in certain image areas (see A, B, and C in FIG. 6). This is because, in the process of automatically stitching two images picked up by different cameras quickly, a sense of heterogeneity may occur in a boundary region where the images overlap each other.
In order to solve the image distortion in the automatic stitching process, the embodiment of the present invention performs the latter image processing including the manual stitching operation. The latter image processing process can be processed by applying various blending effects to the stitching area (i.e., the overlapping areas between images), and also by manual and fine distortion matching by the editor. A good stitching result means that even if a stitching error occurs, the distortion rate is evenly applied to the whole image, and sharpness and horizontal / vertical lines are straightly aligned and there is no slack. Therefore, in the latter image processing process, the result after the stitching has a uniform distortion rate of the image as a whole, or the sharpness of the image becomes high, so that a high-definition high-quality image can be generated. The image after the second half image processing is illustrated in FIG. 7, and the final image after the second half image processing is illustrated in FIG.
The method and apparatus according to embodiments of the present invention may be implemented in the form of program instructions that can be executed through various computer means and recorded in a computer-readable medium. The computer readable medium may include program instructions, data files, data structures, and the like, alone or in combination.
Program instructions to be recorded on a computer-readable medium may be those specially designed and constructed for the present invention or may be available to those skilled in the computer software arts. Examples of computer-readable media include magnetic media such as hard disks, floppy disks and magnetic tape; optical media such as CD-ROMs and DVDs; magnetic media such as floppy disks; Includes hardware devices specifically configured to store and execute program instructions such as magneto-optical media and ROM, RAM, flash memory, and the like. Examples of program instructions include machine language code such as those produced by a compiler, as well as high-level language code that can be executed by a computer using an interpreter or the like.
The hardware devices described above may be configured to operate as one or more software modules to perform the operations of the present invention, and vice versa.
While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it will be understood by those of ordinary skill in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention as defined by the following claims And changes may be made without departing from the spirit and scope of the invention.
Claims (5)
(a) generating a multi-view two-dimensional original image by using a 360-degree omnidirectional image capturing device, wherein the multi-view two-dimensional original image includes a plurality of left eye images and a plurality of right eye images;
(b) generating a single panoramic left eye image by performing automatic image stitching of the plurality of left eye images photographed at the same point in time, and performing automatic image stitching processing of the plurality of right eye images to generate a single panoramic right eye image;
(c) performing a second-half image processing for adjusting the distortion of the stitching area superimposed on each of the original images in the automatic image stitching process for each of the panoramic left eye image and the panoramic right eye image, Generating an image and a right eye image; And
(d) producing a three-dimensional virtual reality image using the distortion-adjusted left eye image and right eye image
Dimensional virtual reality image.
In the step (b), the automatic image stitching process may include:
Adjusting the distortion generated in each original image based on distortion information corresponding to the focal length and the distortion aberration of each imaging lens of each of the plurality of cameras in the omniazimetric imaging apparatus and stitching the distorted images through inverse adjustment Dimensional virtual reality image.
In the step (c), the second-
Wherein the blending effect and the manual stitching process are performed so that the distortion rate and the sharpness of the stitching area overlapping the images are uniform over the entire panorama image, Dimensional virtual reality image.
The omnidirectional image photographing apparatus has two pairs of cameras corresponding to the left eye camera and the right eye camera on a 360-degree league (RIG), one on the upper side of the league, one on the lower side of the league, and five on the circumference of the league. A total of 14 images are acquired at the time point,
Wherein the left panoramic image and the right panoramic image are generated by automatic stitching processing of 7 left eye images and 7 right eye images, respectively.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| KR1020160017006A KR20170096071A (en) | 2016-02-15 | 2016-02-15 | Three dimensional virtual reality video image production method and apparatus thereof |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| KR1020160017006A KR20170096071A (en) | 2016-02-15 | 2016-02-15 | Three dimensional virtual reality video image production method and apparatus thereof |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| KR20170096071A true KR20170096071A (en) | 2017-08-23 |
Family
ID=59759324
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| KR1020160017006A KR20170096071A (en) | 2016-02-15 | 2016-02-15 | Three dimensional virtual reality video image production method and apparatus thereof |
Country Status (1)
| Country | Link |
|---|---|
| KR (1) | KR20170096071A (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108174178A (en) * | 2018-01-09 | 2018-06-15 | 重庆爱奇艺智能科技有限公司 | A kind of method for displaying image, device and virtual reality device |
| WO2020004967A1 (en) * | 2018-06-28 | 2020-01-02 | 주식회사 알파서클 | Virtual reality image reproduction device for reproducing plurality of virtual reality images to improve image quality of specific region, and method for generating virtual reality image |
| KR20200002018A (en) * | 2019-10-21 | 2020-01-07 | 주식회사 알파서클 | Apparaturs for playing plural vr video in sync or async method |
| CN111726566A (en) * | 2019-03-21 | 2020-09-29 | 上海飞猿信息科技有限公司 | Implementation method for correcting splicing anti-shake in real time |
| WO2023059144A1 (en) * | 2021-10-07 | 2023-04-13 | 주식회사 벤타브이알 | Image stitching method and system for relaying 3d vr images in real time |
-
2016
- 2016-02-15 KR KR1020160017006A patent/KR20170096071A/en unknown
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108174178A (en) * | 2018-01-09 | 2018-06-15 | 重庆爱奇艺智能科技有限公司 | A kind of method for displaying image, device and virtual reality device |
| WO2020004967A1 (en) * | 2018-06-28 | 2020-01-02 | 주식회사 알파서클 | Virtual reality image reproduction device for reproducing plurality of virtual reality images to improve image quality of specific region, and method for generating virtual reality image |
| KR20200001750A (en) * | 2018-06-28 | 2020-01-07 | 주식회사 알파서클 | Apparaturs for playing vr video to improve quality of specific area |
| CN112204627A (en) * | 2018-06-28 | 2021-01-08 | 株式会社阿尔法空间 | Virtual reality image reproduction device and virtual reality image generation method for reproducing a plurality of virtual reality images to improve image quality of specific area |
| CN111726566A (en) * | 2019-03-21 | 2020-09-29 | 上海飞猿信息科技有限公司 | Implementation method for correcting splicing anti-shake in real time |
| KR20200002018A (en) * | 2019-10-21 | 2020-01-07 | 주식회사 알파서클 | Apparaturs for playing plural vr video in sync or async method |
| WO2023059144A1 (en) * | 2021-10-07 | 2023-04-13 | 주식회사 벤타브이알 | Image stitching method and system for relaying 3d vr images in real time |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| JP6429829B2 (en) | Image processing system, image processing apparatus, control method, and program | |
| JP6833348B2 (en) | Information processing device, image processing system, information processing device control method, virtual viewpoint image generation method, and program | |
| JP6808357B2 (en) | Information processing device, control method, and program | |
| JP6482498B2 (en) | Control device, control method, and program | |
| JP6742444B2 (en) | Virtual reality 360 degree video camera system for live streaming | |
| KR20170096071A (en) | Three dimensional virtual reality video image production method and apparatus thereof | |
| KR102013978B1 (en) | Method and apparatus for fusion of images | |
| JP6289811B2 (en) | Video processing apparatus and method | |
| US9264695B2 (en) | System and method for multi-viewpoint video capture | |
| JP2017212591A (en) | Control apparatus, control method, and program | |
| US20160191891A1 (en) | Video capturing and formatting system | |
| JP6057570B2 (en) | Apparatus and method for generating stereoscopic panoramic video | |
| WO2020195232A1 (en) | Image processing device, image processing method, and program | |
| CN106292162A (en) | Stereographic device and corresponding control methods | |
| KR20120046802A (en) | Apparatus and method of creating 3 dimension panorama image by using a camera | |
| JP2012054862A (en) | Image processing apparatus and image processing method | |
| US10937462B2 (en) | Using sharding to generate virtual reality content | |
| JP6790145B2 (en) | Control device, control method, and program | |
| JP5156757B2 (en) | System and method for acquiring and editing audio and video data | |
| TW201327019A (en) | Capturing a perspective-flexible, viewpoint-synthesizing panoramic 3D image with a multi-view 3D camera | |
| Wang et al. | A common feature-based disparity control strategy in stereoscopic panorama generation | |
| KR101577671B1 (en) | Method of producing and implementing for dynamic cinema | |
| KR20160102898A (en) | Method and apparatus for generating lens-related metadata | |
| JP2019071650A (en) | Control device, control method, and program | |
| KR102461031B1 (en) | Network-based image processing method and apparatus therefor |