CN109379579B - Processing method for acquiring real three-dimensional data of light field in real time - Google Patents
Processing method for acquiring real three-dimensional data of light field in real time Download PDFInfo
- Publication number
- CN109379579B CN109379579B CN201811386391.XA CN201811386391A CN109379579B CN 109379579 B CN109379579 B CN 109379579B CN 201811386391 A CN201811386391 A CN 201811386391A CN 109379579 B CN109379579 B CN 109379579B
- Authority
- CN
- China
- Prior art keywords
- light field
- acquiring
- dimensional data
- sub
- data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Landscapes
- Testing, Inspecting, Measuring Of Stereoscopic Televisions And Televisions (AREA)
- Studio Devices (AREA)
Abstract
The invention discloses a processing method for acquiring real three-dimensional light field data in real time, which comprises the steps of selecting a light field real three-dimensional data acquisition device with a main lens combined with a sub-lens array, calibrating a central point of the light field real three-dimensional data acquisition device by adopting a white image calibration method, then shooting in real time by using the light field real three-dimensional data acquisition device to acquire all light field data packets of a target object, then acquiring a sub-lens disparity map image by using the light field data packets, converting the sub-lens disparity map image into a light field slice image, and finally performing multi-stream multiplexing compression on the light field slice image. The processing method for acquiring the light field true three-dimensional data in real time can convert the light field data packet of the three-dimensional object into high-density recognizable light field slice data so as to present the data on the true three-dimensional platform.
Description
Technical Field
The invention relates to the technical field of light field true three-dimensional data processing, and provides a method for acquiring light field true three-dimensional data in real time and processing the data to be suitable for the technical field of true three-dimensional display.
Background
The 3D technology is developed through initial three-dimensional display of a volume, wearable 3D display and grating naked eye 3D display until the light field is displayed in a three-dimensional mode. The demand for display reduction of the real world is higher and higher, and the real world of people's life is a world full of light rays which are transmitted around people along each direction to form a continuous light field no matter a solar light source or an artificial light source. The light field is all the light rays in the whole three-dimensional space. If the propagation process of the light can be recorded and restored at the display end, which is the light field three-dimensional display technology, the presented effect is similar to the three-dimensional real space of the real world.
To completely express light field information, it is known from a plenoptic function proposed by alderson (e.h. adelson) that a seven-dimensional function, i.e., three-dimensional coordinates (x, y, z) of light rays, an arbitrary transmission direction (α), and a wavelength (λ) and time (t) of light, is simplified by McMillan (McMillan), and the seven-dimensional function is reduced to four-dimensional, i.e., a path through which light rays pass is confirmed by points on two planes, so as to confirm the direction and position of light rays, as shown in fig. 1.
Disclosure of Invention
The invention provides a mode for acquiring and processing light field data in real time and finally applying the light field data to true three-dimensional display application, and the specific technical scheme is as follows:
a processing method for acquiring real three-dimensional data of a light field in real time comprises the following steps:
selecting a light field true three-dimensional data acquisition device combining a main lens and a sub-lens array, and calibrating a central point of the light field true three-dimensional data acquisition device by adopting a white image calibration method;
shooting in real time by using a light field true three-dimensional data acquisition device to acquire all light field data packets of a target object;
acquiring a sub-lens disparity map image by using the light field data packet;
converting the sub-lens disparity map image into a light-field slice image;
and performing multi-stream multiplexing compression on the light field slice image.
The white image calibration method of the processing method for acquiring the real three-dimensional data of the light field in real time is to shoot the real three-dimensional data acquisition device of the light field towards a pure white scene to acquire image data of a single color, and accurately calibrate the central point of the real three-dimensional data acquisition device of the light field by taking the acquired image data as a reference.
The processing method for acquiring the real three-dimensional data of the light field in real time comprises the step of acquiring the real three-dimensional data of the light field by using a light field camera, a multi-camera light field acquisition system based on a plane or a multi-camera light field acquisition system based on a spherical surface.
The processing method for acquiring the real three-dimensional data of the light field in real time comprises the steps of decoding and post-processing by using a light field data packet of a light field camera and directly shooting and acquiring sub-camera images of a multi-camera light field acquisition system based on a plane or a spherical surface;
the decoding of the processing method for acquiring the real three-dimensional data of the light field in real time refers to that the central point of a calibrated light field camera is taken as a reference, the parameters of the light field camera are combined, images projected by each sub-lens are obtained, and the images are compressed into a common format. Common formats include. jpg,. png,. bmp.
The processing method for acquiring the real three-dimensional data of the light field in real time comprises frequency domain filtering and color correction.
According to the processing method for acquiring the real three-dimensional data of the light field in real time, the light field data packet comprises depth information of a shot target object, rgb information of each sub-lens and partial parameter information of a light field camera. Such as luminous flux, frame rate, etc.
The processing method for acquiring the light field true three-dimensional data in real time comprises a FFmpeg data compression mode and an avs data compression mode.
The processing method for acquiring the light field true three-dimensional data in real time can convert the light field data packet of the three-dimensional object into high-density recognizable light field slice data so as to present the data on the true three-dimensional platform.
Drawings
FIG. 1 is a diagram illustrating the definition of a four-dimensional light field.
Fig. 2 is a schematic diagram of light path propagation of a light field camera.
Fig. 3 is a flow chart of a process for acquiring real-time light field true three-dimensional data.
Fig. 4 is a conversion diagram of a sub-lens disparity map and a light field slice image.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
The data source adopted by the processing method for acquiring the real three-dimensional data of the light field in real time is mainly a light field acquisition device or system, and the acquisition device can determine a straight line for penetrating through a biplane introduced in the technical background based on the main principle. Wherein the biplanes are the main lens plane and the sub-lens plane of the collecting device, and the propagation of the ray path is shown in fig. 2.
Frequency domain filtering, color correction and the like, and belongs to the field of preprocessing of the parallax images of the sub-lenses. Because the light of the sub-lens at the boundary is insufficient, and the color of the obtained sub-lens parallax image is not ideal, the obtained sub-lens parallax image needs to be subjected to corresponding frequency domain filtering and color correction, so that the sub-aperture parallax image with better quality is obtained.
People can feel three-dimensional objects mainly based on the binocular parallax, motion parallax, monocular focusing and binocular convergence principles of human eyes. The three-dimensional object is reproduced by mainly utilizing the binocular disparity principle of human beings to transmit a disparity map to eyes of an observer and generate depth perception through brain fusion. Therefore, after obtaining the sub-lens parallax image of the light field camera, the sub-lens parallax image needs to be converted into a light field slice image in the visual range of human eyes, the three-dimensional information of the object is reproduced through superposition of the light field slice image layer by layer, and when the human eyes observe different head-up parallax images, the three-dimensional object can be reproduced with high reduction degree.
And (3) multi-stream multiplexing compression, namely compressing the obtained disparity map by using FFmpeg and other modes, wherein the mode has the main advantages that the resolution ratio of the light field slice image can be ensured to the maximum extent, the processing efficiency is high, and distortion is avoided.
The invention provides a processing method for acquiring real three-dimensional data of a light field in real time, and the flow is shown in figure 3. The processing method mainly comprises the steps of obtaining real-time light field data, calibrating a camera, decoding the light field data, post-processing images, generating sub-lens images, generating light field slice images, compressing the data and the like. The real-time light field data acquisition device includes, but is not limited to, a light field camera, a planar multi-camera acquisition system, a spherical multi-camera acquisition system, and the like. The generation of the sub-lens image includes, but is not limited to, post-processing of a light field data packet, sub-camera acquisition in a planar multi-camera acquisition system, sub-camera acquisition in a spherical multi-camera acquisition system, and the like. The data compression includes, but is not limited to, FFmpeg, avs and other data compression methods.
Example 1
The light field camera with the number of sub-lenses of 11 × 11 orders of magnitude is taken as the acquisition equipment, the display platform which takes the light field slice image presenting the true three-dimensional scene as the principle is taken as the carrier, and the real three-dimensional data processing mode of the real-time acquisition light field is further described in detail as follows:
a. the acquisition device for acquiring the light field data of the object is in a mode of a main lens and a sub-lens array, wherein the sub-lens array is 11 multiplied by 11. And acquiring all light field data packets of the target object by real-time shooting. The data packet includes depth information of the target object, rgb information of each sub-lens, and partial parameter information of the camera.
b. And acquiring a white image, namely shooting a camera by aiming at a pure white scene such as a wall or white paper. The step is mainly to acquire the light field data of a single color so as to conveniently use the image as a reference and more accurately calibrate the central point of the light field camera.
c. And decoding the light field data, mainly decomposing the acquired original light field data packet, taking the center point of the calibrated light field camera as a reference, combining the parameters of the light field camera to obtain images projected by each sub-lens, and compressing the images into common formats such as jpg, png, bmp and the like.
d. The image post-processing mainly comprises frequency domain filtering, color correction and the like, the processing reason of the step is that the conditions of uneven illuminance and the like can occur when the acquisition system acquires each angle of the image, and through the step, the contrast of the image can be improved, and the extraction of useful information of the image in the later period is facilitated.
e. The generation of the light field slice image mainly carries out the conversion from the disparity map to the light field slice image through the three-dimensional light field information of the existing target object, and the conversion relationship is detailed in an attached figure 4.
f. And multi-stream multiplexing compression mainly considers the phenomenon of resolution loss of three-dimensional display, and adopts a multi-stream multiplexing mode, so that the resolution of a single slice image is not compressed, and the display effect of a true three-dimensional display platform can be greatly ensured.
Although the invention has been described herein with reference to illustrative embodiments thereof, it should be understood that numerous other modifications and embodiments can be devised by those skilled in the art that will fall within the spirit and scope of the principles of this disclosure. More specifically, various variations and modifications may be made to the component parts and/or arrangements of the subject combination arrangement within the scope of the disclosure herein. In addition to variations and modifications in the component parts and/or arrangements, other uses will also be apparent to those skilled in the art.
Claims (5)
1. A processing method for acquiring real three-dimensional data of a light field in real time is characterized by comprising the following steps:
a. selecting a light field true three-dimensional data acquisition device combining a main lens and a sub-lens array, and calibrating a central point of the light field true three-dimensional data acquisition device by adopting a white image calibration method;
b. shooting in real time by using a light field true three-dimensional data acquisition device to acquire all light field data packets of a target object; the light field true three-dimensional data acquisition device is a light field camera, a multi-camera light field acquisition system based on a plane or a multi-camera light field acquisition system based on a spherical surface; the light field data packet comprises depth information of a shot target object, rgb information of each sub-lens and partial parameter information of a light field camera;
c. acquiring a sub-lens disparity map image by using the light field data packet; the mode for acquiring the sub-lens disparity map image comprises the steps of decoding and post-processing by using a light field data packet of a light field camera, and directly shooting and acquiring the sub-camera image of a multi-camera light field acquisition system based on a plane or a spherical surface; the decoding means that the central point of the calibrated light field camera is taken as a reference, the parameters of the light field camera are combined to obtain images projected by each sub-lens, and the images are compressed into a common format;
d. converting the sub-lens disparity map image into a light-field slice image;
e. and performing multi-stream multiplexing compression on the light field slice image.
2. The processing method for acquiring the light field true three-dimensional data in real time according to claim 1, wherein the white image calibration method is to shoot the light field true three-dimensional data acquisition device towards a pure white scene to acquire image data of a single color, and to accurately calibrate the central point of the light field true three-dimensional data acquisition device with the acquired image data as a reference.
3. The processing method for acquiring light field true three-dimensional data in real time according to claim 1, wherein the common format comprises jpg,. png,. bmp.
4. The process of acquiring light field true three-dimensional data in real time according to claim 1, wherein the post-processing comprises frequency domain filtering and color correction.
5. The processing method for acquiring light field true three-dimensional data in real time according to claim 1, wherein the multi-stream multiplexing compression comprises FFmpeg data compression and avs data compression.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811386391.XA CN109379579B (en) | 2018-11-20 | 2018-11-20 | Processing method for acquiring real three-dimensional data of light field in real time |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811386391.XA CN109379579B (en) | 2018-11-20 | 2018-11-20 | Processing method for acquiring real three-dimensional data of light field in real time |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109379579A CN109379579A (en) | 2019-02-22 |
| CN109379579B true CN109379579B (en) | 2020-03-17 |
Family
ID=65376803
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811386391.XA Active CN109379579B (en) | 2018-11-20 | 2018-11-20 | Processing method for acquiring real three-dimensional data of light field in real time |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109379579B (en) |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102520787B (en) * | 2011-11-09 | 2015-05-06 | 浙江大学 | Real-time spatial three-dimensional presentation system and real-time spatial three-dimensional presentation method |
| WO2016140934A2 (en) * | 2015-03-01 | 2016-09-09 | Nextvr Inc. | Methods and apparatus for making environmental measurements and/or using such measurements in 3d image rendering |
| CN107063117A (en) * | 2017-03-15 | 2017-08-18 | 上海大学 | Underwater laser synchronous scanning range of triangle imaging system and method based on optical field imaging |
| CN207218846U (en) * | 2017-05-03 | 2018-04-10 | 丁志宇 | Circulating type mixes light field imaging device |
| CN108776980B (en) * | 2018-05-14 | 2021-07-09 | 南京工程学院 | Calibration method for micro-lens light field camera |
-
2018
- 2018-11-20 CN CN201811386391.XA patent/CN109379579B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN109379579A (en) | 2019-02-22 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US9094675B2 (en) | Processing image data from multiple cameras for motion pictures | |
| JP4188968B2 (en) | Stereoscopic video providing method and stereoscopic video display device | |
| JP4942106B2 (en) | Depth data output device and depth data receiver | |
| KR101538947B1 (en) | The apparatus and method of hemispheric freeviewpoint image service technology | |
| GB2465072A (en) | Combining range information with images to produce new images of different perspective | |
| WO2019085022A1 (en) | Generation method and device for optical field 3d display unit image | |
| CN101277454A (en) | Method for generating real time tridimensional video based on binocular camera | |
| CN108616742B (en) | 3D panoramic shooting system and method | |
| CN108141578A (en) | Camera is presented | |
| US20130202191A1 (en) | Multi-view image generating method and apparatus using the same | |
| US11415935B2 (en) | System and method for holographic communication | |
| CN108377355A (en) | A kind of video data handling procedure, device and equipment | |
| CN105472308A (en) | Multi-view naked eye 3D video conference system | |
| CN111064945B (en) | Naked eye 3D image acquisition and generation method | |
| CN109379579B (en) | Processing method for acquiring real three-dimensional data of light field in real time | |
| KR20140037430A (en) | A system for generating high quality digital holographic video using vertical rig based on depth+rgb camera | |
| CN109961395B (en) | Method, device and system for generating and displaying depth image and readable medium | |
| CN105072434A (en) | Remote transmission system and method of holographic images | |
| KR20050083352A (en) | The apparatus and method for acquisition and displays a panoramic and three-dimensional image using the stereo-camera in a mobile communication terminal. | |
| EP2852149A1 (en) | Method and apparatus for generation, processing and delivery of 3D video | |
| Wadaskar et al. | A rich stereoscopic 3d high dynamic range image & video database of natural scenes | |
| CN111629194B (en) | Method and system for converting panoramic video into 6DOF video based on neural network | |
| CN103581653B (en) | Method for non-interference depth extraction of optical coding depth camera system according to luminous intensity modulation | |
| CN106408598A (en) | Three-dimensional portrait reconstruction printing device based on array lens | |
| JP4764516B1 (en) | Multi-view image encoding device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |