WO2018000892A1 - 一种全景立体影像的成像方法、装置及*** - Google Patents

一种全景立体影像的成像方法、装置及*** Download PDF

Info

Publication number
WO2018000892A1
WO2018000892A1 PCT/CN2017/080195 CN2017080195W WO2018000892A1 WO 2018000892 A1 WO2018000892 A1 WO 2018000892A1 CN 2017080195 W CN2017080195 W CN 2017080195W WO 2018000892 A1 WO2018000892 A1 WO 2018000892A1
Authority
WO
WIPO (PCT)
Prior art keywords
coordinate system
eye
image
spherical
imaging
Prior art date
Application number
PCT/CN2017/080195
Other languages
English (en)
French (fr)
Inventor
刘亚辉
沈靖程
王士博
王超
Original Assignee
深圳市圆周率软件科技有限责任公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 深圳市圆周率软件科技有限责任公司 filed Critical 深圳市圆周率软件科技有限责任公司
Publication of WO2018000892A1 publication Critical patent/WO2018000892A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G03—PHOTOGRAPHY; CINEMATOGRAPHY; ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ELECTROGRAPHY; HOLOGRAPHY
    • G03B—APPARATUS OR ARRANGEMENTS FOR TAKING PHOTOGRAPHS OR FOR PROJECTING OR VIEWING THEM; APPARATUS OR ARRANGEMENTS EMPLOYING ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ACCESSORIES THEREFOR
    • G03B35/00—Stereoscopic photography
    • G03B35/18—Stereoscopic photography by simultaneous viewing
    • G03B35/20—Stereoscopic photography by simultaneous viewing using two or more projectors
    • G—PHYSICS
    • G03—PHOTOGRAPHY; CINEMATOGRAPHY; ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ELECTROGRAPHY; HOLOGRAPHY
    • G03B—APPARATUS OR ARRANGEMENTS FOR TAKING PHOTOGRAPHS OR FOR PROJECTING OR VIEWING THEM; APPARATUS OR ARRANGEMENTS EMPLOYING ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ACCESSORIES THEREFOR
    • G03B37/00—Panoramic or wide-screen photography; Photographing extended surfaces, e.g. for surveying; Photographing internal surfaces, e.g. of pipe
    • G03B37/04—Panoramic or wide-screen photography; Photographing extended surfaces, e.g. for surveying; Photographing internal surfaces, e.g. of pipe with cameras or projectors providing touching or overlapping fields of view
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/20—Image signal generators
    • H04N13/296—Synchronisation thereof; Control thereof

Definitions

  • the invention belongs to the field of panoramic imaging technology, and in particular relates to an imaging method, device and system for panoramic stereoscopic images.
  • the methods of panoramic imaging are mainly divided into two categories: first, using a single camera unit (or multiple camera unit) to obtain all the images in the scene space by rotating around the axis to form a panoramic image; second, using multiple camera units The images are combined in a fixed manner to obtain all the images in the scene space at the same time and then stitched together to form a panoramic image.
  • the advantage of the above first method is that a panoramic image of a static scene can be captured by a smaller number of camera units; the disadvantage is that the imaging effect on the dynamic scene is not good, and the panoramic video cannot be obtained; the advantage of the second method described above is that Instant imaging can effectively image both static and dynamic scenes; the disadvantage is that the consistency of internal parameters of each camera unit and the uniformity of position arrangement between camera units are high. This consistency and uniformity constraints Undoubtedly reduce the flexibility of the shooting method and the possibility of free combination of multiple devices.
  • the present invention provides a method, a device and a system for imaging panoramic stereoscopic images, so as to realize flexible acquisition of panoramic stereoscopic images, and reduce parameters of each camera unit during the construction process of the panoramic imaging device. Consistency of uniformity and uniformity of positional placement.
  • the present invention provides an imaging method for panoramic stereoscopic images, comprising the following steps:
  • Step A arbitrarily arranging a plurality of camera units in the space according to different application scenarios
  • Step B calibrating the internal and external parameters of each camera unit in the step A that has been arranged;
  • Step C controlling the camera units in each of the step B to perform cooperative work, and performing synchronous acquisition and transmission of image point coordinates;
  • Step D arbitrarily designating a left-eye spherical coordinate system O L and a right-eye spherical coordinate system O R in the space;
  • Step E according to the left-eye spherical coordinate system O L and the right-eye spherical coordinate system O R in the step D, respectively, the image point coordinates of the imaging unit in the step C are solved to the left-eye spherical coordinate system O L and the right a mapping function of the spherical coordinate system O R , and respectively generating a left-eye spherical image corresponding to the left-eye spherical coordinate system O L and a right-eye spherical image corresponding to the right-eye spherical coordinate system O R according to the solved mapping function;
  • step F the left-eye spherical image and the right-eye spherical image in the step E are respectively thrown into the left eye and the right eye to complete the stereoscopic display of the panoramic image.
  • the solution method of the mapping function includes the following steps:
  • Step E1 determining the relationship between the object point coordinates and the image point coordinates in the camera unit coordinate system
  • Step E2 calibrating the spherical coordinate system and the camera unit coordinate system, and determining a transformation relationship between the camera unit coordinate system and the spherical coordinate system;
  • Step E3 according to the relationship between the object point coordinates and the image point coordinates in the step E1 and the transformation relationship between the camera unit coordinate system and the spherical coordinate system in the step E2, the coordinates of the object point in the spherical coordinate system are solved;
  • step E4 according to the coordinates of the object point in the step E3 in the spherical coordinate system, the spherical coordinates of the object point projected onto the spherical coordinate system with the radius r are solved.
  • the distance between the center of the spherical image of the left eye and the center of the spherical image of the right eye is equal to the distance between the eyes of the human eye.
  • the present invention provides an imaging apparatus for panoramic stereoscopic images, including:
  • the camera unit is arranged to arrange a plurality of camera units in the space according to the unused application scenarios;
  • a system calibration module configured to calibrate internal and external parameters of each camera unit that has been arranged in the camera unit arrangement module, and is connected to the camera unit arrangement module;
  • a hardware communication module configured to control each camera unit of the camera unit arrangement module to perform cooperative work and perform synchronous acquisition of image point coordinates, receive image point coordinates collected by each camera unit in the camera unit arrangement module, and transmit Connected to the camera unit arrangement module;
  • the left-eye and right-eye spherical coordinate system specifies a module for arbitrarily designating a left-eye spherical coordinate system O L and a right-eye spherical coordinate system O R in the space;
  • a mapping function solving module configured to receive image point coordinates transmitted by the hardware communication module, and specify a left-eye spherical coordinate system O L and a right-eye spherical coordinate system O R according to the left-eye and right-eye spherical coordinate systems respectively Solving a mapping function of the image point coordinates of the camera unit in the camera unit arrangement module to the left-eye spherical coordinate system O L and the right-eye spherical coordinate system O R , and respectively generating a left eyeball according to the solved mapping function a left-eye spherical image corresponding to the surface coordinate system O L and a right-eye spherical image corresponding to the right-eye spherical coordinate system O R , which are connected to the hardware communication module and the left-eye and right-eye spherical coordinate system specifying modules;
  • An image display module configured to cast a left-eye spherical image and a right-eye spherical image in the mapping function solving module into the left eye and the right eye respectively to complete stereoscopic display of the panoramic image, which is compared with the mapping function solving module connection.
  • mapping function solving module includes:
  • a coordinate relationship determining unit configured to determine a relationship between the object point coordinates and the image point coordinates in the camera unit coordinate system
  • a coordinate system transformation relationship determining unit is configured to calibrate the spherical coordinate system and the camera unit coordinate system, and determine a transformation relationship between the camera unit coordinate system and the spherical coordinate system;
  • a spherical coordinate system coordinate solving unit of the object point configured to calculate a coordinate of the object point in the spherical coordinate system according to the relationship between the object point coordinate and the image point coordinate and the transformation relationship between the camera unit coordinate system and the spherical coordinate system And connecting to the coordinate relationship determining unit and the coordinate system transformation relationship determining unit;
  • a spherical coordinate solving unit of the object point configured to calculate a spherical coordinate of the object point projected onto the spherical coordinate system with the radius r according to the coordinate of the object point in the spherical coordinate system, and the spherical coordinate system coordinate of the object point
  • the solution units are connected.
  • the distance between the center of the spherical image of the left eye and the center of the spherical image of the right eye is equal to the distance between the eyes of the human eye.
  • the present invention provides an imaging system for panoramic stereoscopic images, the imaging system of the panoramic stereoscopic image comprising N imaging units randomly distributed or equally spaced on a circumference of the same circle, wherein N is greater than or equal to A positive integer of 2.
  • the present invention provides an imaging system for panoramic stereoscopic images, the imaging system of the panoramic stereoscopic image comprising eight imaging units arbitrarily distributed or symmetrically arranged on four sides of the same quadrilateral, the field of view of the imaging unit More than 180 degrees.
  • the present invention provides an imaging system for panoramic stereoscopic images, the imaging system of the panoramic stereoscopic image comprising four camera units A randomly arranged or equally spaced on the circumference of the same circle, and two backs aligned in an axisymmetric arrangement and The optical axis is connected to the imaging unit B of the center of the plane formed by the four imaging units A and perpendicular to the plane, and the field of view of the imaging unit A is greater than 180 degrees, and the imaging unit B is The field of view is greater than 180 degrees.
  • the invention provides an imaging system for panoramic stereoscopic images
  • the imaging system of the panoramic stereoscopic image comprises eight camera units C arranged at an equal or equal interval on the circumference of the same circle, and the two backs are aligned in an axisymmetric setting.
  • the optical axis is connected to the imaging unit D which is perpendicular to the plane of the plane formed by the eight imaging units C, and the field of view of the imaging unit C is greater than 90 degrees, the imaging unit D The field of view is greater than 180 degrees.
  • the present invention realizes panoramic stereoscopic image acquisition with a large degree of freedom by means of a combination of multiple camera units, and based on this, solves the true multi-view stereoscopic display of the panoramic image by using the method of multiple panoramic ball pairs.
  • the invention effectively reduces the constraint of the consistency of the parameters of the camera unit and the uniformity of the position arrangement during the construction process of the panoramic photographing device, and greatly improves the degree of freedom and flexibility of the photographing unit.
  • FIG. 1 is a schematic flowchart of a method for imaging a panoramic stereoscopic image according to a first embodiment of the present invention
  • FIG. 2 is a schematic flow chart of a solution method of the mapping function in FIG. 1;
  • FIG. 3 is a schematic structural diagram of an imaging apparatus for panoramic stereoscopic images according to a second embodiment of the present invention.
  • FIG. 4 is a schematic structural diagram of a mapping function solving module in FIG. 3;
  • FIG. 5 is a schematic diagram showing a mapping relationship between images formed by a single camera unit in an arbitrary spherical coordinate system
  • FIG. 6 is a schematic diagram showing a mapping relationship between overlapping imaging regions of a plurality of imaging units in an arbitrary spherical coordinate system
  • FIG. 7 is a schematic structural diagram of an imaging system for panoramic stereoscopic images according to a third embodiment of the present invention.
  • FIG. 8 is a schematic structural diagram of an imaging system for panoramic stereoscopic images according to a fourth embodiment of the present invention.
  • FIG. 9 is a schematic structural diagram of an imaging system for panoramic stereoscopic images according to a fifth embodiment of the present invention.
  • FIG. 10 is a schematic structural diagram of an imaging system for panoramic stereoscopic images according to a sixth embodiment of the present invention.
  • FIG. 11 is a schematic structural diagram of an imaging system for panoramic stereoscopic images according to a seventh embodiment of the present invention.
  • FIG. 12 is a schematic structural diagram of an imaging system for panoramic stereoscopic images according to an eighth embodiment of the present invention.
  • FIG. 13 is a schematic structural diagram of an imaging system for panoramic stereoscopic images according to a ninth embodiment of the present invention.
  • FIG. 14 is a schematic diagram showing the positional relationship of a spherical coordinate system pair when the panoramic stereoscopic image forming apparatus of the present invention is viewed from the front;
  • FIG. 15 is a schematic diagram showing the positional relationship of a spherical coordinate system pair when the panoramic stereoscopic image forming apparatus of the present invention is turned to the right front;
  • 16 is a schematic diagram showing the positional relationship of a spherical coordinate system pair when the panoramic stereoscopic image forming apparatus of the present invention is turned to the right side;
  • FIG. 17 is a schematic diagram showing the positional relationship of a spherical coordinate system pair when the panoramic stereoscopic image forming apparatus of the present invention is viewed from an arbitrary viewing angle.
  • FIG. 1 is a schematic flowchart of a method for imaging a panoramic stereoscopic image according to a first embodiment of the present invention. As shown in FIG. 1, the method for imaging a panoramic stereoscopic image includes the following steps:
  • Step S20 arbitrarily arranging a plurality of camera units in space according to different application scenarios
  • Step S21 calibrating the internal and external parameters of the respective image capturing units that have been arranged
  • Step S22 controlling each camera unit to perform cooperative work, and performing synchronous acquisition and transmission of image point coordinates
  • Step S23 arbitrarily designating a left-eye spherical coordinate system O L and a right-eye spherical coordinate system O R in the space;
  • Step S24 the spherical coordinate left eye and the right eye O L O R spherical coordinate system solver imaging unit to image eye point coordinates are spherical coordinates mapping function O L and O R right spherical coordinate system, and according to The solved mapping function respectively generates a left-eye spherical image corresponding to the left-eye spherical coordinate system O L and a right-eye spherical image corresponding to the right-eye spherical coordinate system O R ;
  • step S25 the left-eye spherical image and the right-eye spherical image are respectively thrown into the left eye and the right eye to complete the stereoscopic display of the panoramic image.
  • the distance between the center of the spherical surface of the left-eye spherical image and the center of the spherical image of the right eye is equal to the distance between the human eye and the eye.
  • FIG. 2 is a schematic flowchart of a solution method of the mapping function in FIG. 1.
  • the mapping function solving method includes the following steps:
  • Step S70 determining the relationship between the object point coordinates and the image point coordinates in the camera unit coordinate system
  • Step S71 calibrating the spherical coordinate system and the camera unit coordinate system, and determining a transformation relationship between the camera unit coordinate system and the spherical coordinate system;
  • Step S72 according to the relationship between the object point coordinates and the image point coordinates, and the camera unit coordinate system and the spherical coordinate The transformation relationship between the systems, and the coordinates of the object point in the spherical coordinate system;
  • step S73 the spherical coordinates of the object point projected onto the spherical coordinate system with radius r are solved according to the coordinates of the object point in the spherical coordinate system.
  • FIG. 3 is a schematic structural diagram of an imaging apparatus for a panoramic stereoscopic image according to a second embodiment of the present invention.
  • the imaging apparatus of the panoramic stereoscopic image includes:
  • the camera unit arrangement module 31 is configured to arbitrarily arrange a plurality of camera units in the space according to the unused application scenarios;
  • the system calibration module 32 is configured to calibrate internal and external parameters of the image capturing unit 31 that have been arranged in the camera unit arrangement module 31, and is connected to the camera unit arrangement module 31;
  • the hardware communication module 33 is configured to control the camera units of the camera unit arrangement module 31 to perform coordinated operation and perform synchronous acquisition of image point coordinates, and receive image point coordinates collected by each camera unit in the camera unit arrangement module 31 and transmit the image coordinates. It is connected to the camera unit arrangement module 31;
  • the left-eye and right-eye spherical coordinate system specifying module 34 is configured to arbitrarily designate a left-eye spherical coordinate system O L and a right-eye spherical coordinate system O R in the space;
  • the mapping function solving module 35 is configured to receive the pixel coordinates transmitted by the hardware communication module 33, and specify the left-eye spherical coordinate system O L and the right-eye spherical coordinate system O R in the module 34 according to the left-eye and right-eye spherical coordinate systems. Solving the mapping function of the image point coordinates of the camera unit in the camera unit arrangement module 31 to the left-eye spherical coordinate system O L and the right-eye spherical coordinate system O R respectively, and respectively generating a left eyeball according to the solved mapping function.
  • a left-eye spherical image corresponding to the surface coordinate system O L and a right-eye spherical image corresponding to the right-eye spherical coordinate system O R which are connected to the hardware communication module 33 and the left-eye and right-eye spherical coordinate system specifying modules 34;
  • the image display module 36 is configured to cast the left-eye spherical image and the right-eye spherical image in the mapping function solving module 35 into the left eye and the right eye respectively to complete the stereoscopic display of the panoramic image, which is connected to the mapping function solving module 35. .
  • the distance between the center of the spherical surface of the left-eye spherical image and the center of the spherical image of the right eye is equal to the distance between the human eye and the eye.
  • FIG. 4 is a schematic structural diagram of the mapping function solving module in FIG. 3. As shown in FIG. 4, the mapping function solving module includes:
  • a coordinate relationship determining unit 81 configured to determine a relationship between the object point coordinates and the image point coordinates in the camera unit coordinate system
  • the coordinate system transformation relationship determining unit 82 is configured to calibrate the spherical coordinate system and the camera unit coordinate system, and determine a transformation relationship between the camera unit coordinate system and the spherical coordinate system;
  • the spherical coordinate system coordinate solving unit 83 of the object point is configured to calculate the coordinates of the object point in the spherical coordinate system according to the relationship between the object point coordinates and the image point coordinates and the transformation relationship between the camera unit coordinate system and the spherical coordinate system, and
  • the coordinate relationship determining unit 81 and the coordinate system transformation relationship determining unit 82 are connected;
  • the spherical coordinate solving unit 84 of the object point is configured to solve the spherical coordinate of the object point projected onto the spherical coordinate system with the radius r according to the coordinates of the object point in the spherical coordinate system, and the spherical coordinate system coordinate solving unit 83 of the object point Connected.
  • FIG. 5 is a schematic diagram of a mapping relationship between an image formed by a single imaging unit and an arbitrary spherical coordinate system. As shown in FIG. 5, the space between the arbitrary spherical coordinate system 501 and the imaging unit 502 of an arbitrary spatial posture is shown.
  • the relationship, as well as the internal parameters of the camera unit, can establish a mapping function of the point coordinates I(x 1 ', y 1 ', f), the object distance d to the spherical coordinates, wherein the spherical coordinate system is represented as OXYZ, and the camera unit coordinate system represents For O'X'Y'Z', the internal parameters of the camera unit are recorded as K in the form of vectors, including optical imaging coordinates, focal length, integrated distortion parameters, and lens angle of view.
  • Figure 5 is a simplified example of a spatial mapping relationship in a two-dimensional plane (OXZ). According to the relative positional relationship between the camera unit coordinate system (O'X'Y'Z') and the spherical coordinate system, the optical center in the plane (OXZ) can be obtained.
  • the coordinate O'(x 0 , z 0 ) on the top, and the image coordinate I(x 1 ', y 1 ', f) can be converted into the coordinate I(x 1 , z 1 ) on the plane (OXZ);
  • the point coordinates P(x 3 ', y 3 ', z 3 ') can be obtained by the imaging unit imaging formula and the object distance d; according to the camera unit coordinate system (O'X'Y'Z') and the spherical coordinate system OXYZ For the spatial relationship, find the coordinate value (R, ⁇ ) of the object point P(x 3 ′, y 3 ', z 3 ') in the spherical coordinate system, R is the radius of the sphere, and ⁇ is the vector OP on the OXZ plane. The angle of the X axis.
  • the mapping relationship of the above two-dimensional plane can be extended to the mapping relationship of the three-dimensional space.
  • mapping function can be implemented by the following calculation process:
  • Equation 1 The relationship between the object point P(x 3 ', y 3 ', z 3 ') and the image point coordinate I(x i ', y i ', f) under the camera unit coordinate system O'X'Y'Z' As shown in Equation 1:
  • Equation 2 the transformation relationship between the spherical coordinate system and the camera unit coordinate system is as shown in Equation 2:
  • a rotation matrix R of 3 rows and 3 columns and a translation matrix T of 3 rows and 1 column can be obtained by Equation 2.
  • Equation 3 From the results of Equations 1 and 2, the coordinates P(x 3 , y 3 , z 3 ) of the object point P in the spherical coordinate system can be obtained from Equation 3:
  • Equation 4 the coordinates of the spherical surface of the spherical coordinate system OXYZ projected by the object point P to the radius r.
  • FIG. 6 is a schematic diagram showing the mapping relationship between the overlapping imaging regions of the plurality of imaging units in an arbitrary spherical coordinate system, as shown in FIG. 6, in the overlapping imaging regions of the multiple imaging units, under different camera unit coordinate systems.
  • the object distance d can be obtained by solving the multi-eye stereoscopic relationship. Therefore, in the arbitrary spherical coordinate system 601, according to the spatial relationship between the imaging units 602 and 603 which are mutually superimposed and imaged, the object point P can be solved relative to the imaging unit 602. According to the analysis of FIG.
  • the point P in the case where the object distance and the image point coordinate of P are known, the point P can be obtained in the spherical coordinate system according to the spatial relationship between the spherical coordinate system OXYZ and the imaging unit coordinate system OXY.
  • the spatial relationship between the coordinate system OXYZ and the camera unit coordinate system O 1 X 1 Y, and the coordinates of the point P in the spherical coordinate system are obtained. Therefore, for the two imaging units of the overlapping imaging, the mapping of the object point P to the spherical coordinate system can be performed.
  • Relational representation K 1 and K 2 represent internal parameter vectors of the imaging unit 602 and the imaging unit 603, respectively, (x i1 , y i1 ) and (x i2 , y i2 ) are images of the object point P in the imaging unit 602 and the imaging unit 603, respectively. Point coordinates, the method does not need to predict the object distance d, and can automatically calculate the spatial three-dimensional coordinates of the object point.
  • FIG. 7 is a schematic structural diagram of an imaging system for panoramic stereoscopic images according to a third embodiment of the present invention.
  • the imaging system of the panoramic stereoscopic image includes two cameras with back alignments arranged in an axisymmetric manner.
  • the unit 41 has an angle of view of the camera unit 41 greater than 180 degrees.
  • FIG. 8 is a schematic structural diagram of an imaging system for panoramic stereoscopic images according to a fourth embodiment of the present invention.
  • the imaging system of the panoramic stereoscopic image includes three arbitrary distributions on the circumference of the same circle. Or an imaging unit 42 that is equally spaced, the field of view of the imaging unit 42 is greater than 120 degrees.
  • FIG. 9 is a schematic structural diagram of an imaging system for panoramic stereoscopic images according to a fifth embodiment of the present invention.
  • the imaging system of the panoramic stereoscopic image includes four arbitrary distributions on the circumference of the same circle. Or an imaging unit 43 disposed at equal intervals, the angle of view of the imaging unit 43 being greater than 180 degrees.
  • FIG. 10 is a schematic structural diagram of an imaging system for panoramic stereoscopic images according to a sixth embodiment of the present invention.
  • the imaging system of the panoramic stereoscopic image includes eight randomly distributed on the circumference of the same circle. Or an imaging unit 44 disposed at equal intervals, the angle of view of the imaging unit 44 being greater than 90 degrees.
  • FIG. 11 is a schematic structural diagram of an imaging system for a panoramic stereoscopic image according to a seventh embodiment of the present invention.
  • the imaging system of the panoramic stereoscopic image includes four arbitrary distributions on the circumference of the same circle. Or an equally spaced camera unit A45 and two camera units 46 that are aligned in an axisymmetric manner and whose optical axis is connected through the center of the plane formed by the four camera units A45 and perpendicular to the plane.
  • the angle of view of the imaging unit A45 is greater than 180 degrees
  • the angle of view of the imaging unit B46 is greater than 180 degrees.
  • FIG. 12 is a schematic structural diagram of an imaging system for panoramic stereoscopic images according to an eighth embodiment of the present invention.
  • the imaging system of the panoramic stereoscopic image includes eight randomly distributed on the circumference of the same circle. Or an equally spaced camera unit C47, and two camera units D48 that are aligned in an axisymmetric arrangement and whose optical axis is connected through the center of the plane formed by the eight camera units C47 and perpendicular to the plane,
  • the field of view of the camera unit C47 is greater than 90 degrees
  • the field of view of the camera unit D48 is greater than 180 degrees.
  • FIG. 13 is a schematic structural diagram of an imaging system for a panoramic stereoscopic image according to a ninth embodiment of the present invention.
  • the imaging system of the panoramic stereoscopic image includes eight arbitrary sides on the same side of the same quadrilateral.
  • the imaging unit 49 is symmetrically disposed or symmetrically arranged, and the field of view of the imaging unit 49 is greater than 180 degrees.
  • the panoramic stereoscopic image forming method of the present invention can map the object points captured by the panoramic stereoscopic image forming device to a spherical coordinate system of an arbitrary center to form a panoramic sphere of the image, so that the object point can be mapped to a plurality of sets of spherical distances.
  • FIG. 14 FIG.
  • FIG. 14 is a schematic diagram showing the positional relationship of the spherical coordinate system pair when the panoramic stereoscopic image forming apparatus of the present invention is viewed from the front.
  • O L represents a left-eye panoramic spherical coordinate system
  • O R represents a right-eye panoramic view.
  • the distance D between the left-eye panoramic spherical coordinate system O L and the right-eye panoramic spherical coordinate system O R can be set as the observer's pupil distance according to the demand, and the left-eye panoramic sphere And the radius of the right-eye panoramic sphere is set to the same value R; please refer to FIG. 15, FIG.
  • FIG. 15 is a schematic diagram showing the positional relationship of the spherical coordinate system pair when the panoramic stereoscopic image forming apparatus of the present invention is turned to the right front;
  • FIG. 16 is FIG.
  • FIG. 17 is a schematic diagram showing the positional relationship of a spherical coordinate system pair when the panoramic stereoscopic image forming apparatus of the present invention is viewed from an arbitrary viewing angle;
  • FIG. 17 is a schematic diagram showing a positional relationship of a spherical coordinate system pair when the panoramic stereoscopic image forming apparatus of the present invention is turned to the right side; .
  • the invention realizes the panoramic stereoscopic image acquisition with large degree of freedom by combining multiple camera units, and solves the problem of the real multi-view stereoscopic display of the panoramic image by using the method of multiple panoramic ball pairs, and the invention effectively reduces the problem.
  • the panorama shooting device constrains the consistency of the parameters of each camera unit and the uniformity of position arrangement during the construction process, which greatly improves the freedom and flexibility of the camera unit.

Landscapes

  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Studio Devices (AREA)
  • Testing, Inspecting, Measuring Of Stereoscopic Televisions And Televisions (AREA)
  • Stereoscopic And Panoramic Photography (AREA)

Abstract

一种全景立体影像的成像方法、装置及***，包括：步骤A，在空间中任意排布若干个摄像单元（S21)；步骤B，对各个摄像单元的内外参数进行标定（S22)；步骤C，控制各个摄像单元进行协同工作，并进行像点坐标的同步采集和传输（S23)；步骤D，在空间中任意指定左眼和右眼球面坐标系（S24)；步骤E，分别解算摄像单元的像点坐标到左眼和右眼球面坐标系的映射函数，生成左眼和右眼球面图像（S25)；步骤F，将左眼和右眼球面图像分别投到左眼和右眼中完成全景图像的立体显示(S26)。实现了较大自由度的全景立体影像获取，有效减少了全景拍摄设备在搭建过程中对各摄像单元参数的一致性和位置排布的均匀性的约束，提高了摄像单元拍摄的自由度和灵活性。

Description

一种全景立体影像的成像方法、装置及*** 技术领域
本发明属于全景成像技术领域，特别是涉及一种全景立体影像的成像方法、装置及***。
背景技术
目前全景成像的方式主要分为两类：第一，使用单摄像单元(或多摄像单元)绕轴旋转的方式获取场景空间内的全部影像进而拼接形成全景图像；第二，使用多个摄像单元按固定方式组合拍摄得到同一时刻场景空间内的全部影像进而拼接形成全景图像。上述第一类方法的优点是可用较少数量的摄像单元拍摄出静态场景的全景图像；缺点是对动态场景的成像效果不佳，无法实现全景视频的获取；上述第二类方法的优点是可以瞬间成像，对静态、动态场景均可有效成像；缺点是对各个摄像单元内部参数的一致性和各个摄像单元之间位置排布的均匀性等要求较高，这种一致性和均匀性的约束无疑降低了拍摄方式的灵活性以及多种设备自由组合的可能性。
发明内容
综上所述，为解决上述技术问题，本发明提供了一种全景立体影像的成像方法、装置及***，以实现全景立体影像的灵活获取，减少全景拍摄设备在搭建过程中对各摄像单元参数的一致性和位置排布的均匀性的约束。
第一方面，本发明提供了一种全景立体影像的成像方法，包括下述步骤：
步骤A，根据不同的应用场景，在空间中任意排布若干个摄像单元；
步骤B，对已排布完成的所述步骤A中的各个摄像单元的内外参数进行标定；
步骤C，控制各个所述步骤B中的摄像单元进行协同工作，并进行像点坐标的同步采集并传输；
步骤D，在空间中任意指定一个左眼球面坐标系OL和一个右眼球面坐标系OR；
步骤E，根据所述步骤D中的左眼球面坐标系OL和右眼球面坐标系OR分别解算所述步骤C中的摄像单元的像点坐标到左眼球面坐标系OL和右眼球面坐标系OR的映射函数，并根据解算的映射函数分别生成与左眼球面坐标系OL对应的左眼球面图像和与右眼球面坐标系OR对应的右眼球面图像；
步骤F，将所述步骤E中的左眼球面图像和右眼球面图像分别投到左眼和右眼中完成全景图像的立体显示。
进一步地，所述映射函数的解算方法包括下述步骤：
步骤E1，在摄像单元坐标系下，确定物点坐标与像点坐标的关系；
步骤E2，标定球面坐标系和摄像单元坐标系，确定摄像单元坐标系和球面坐标系之间的变换关系；
步骤E3，根据所述步骤E1中的物点坐标与像点坐标的关系及所述步骤E2中的摄像单元坐标系和球面坐标系之间的变换关系，求解物点在球面坐标系的坐标；
步骤E4，根据所述步骤E3中的物点在球面坐标系的坐标，求解物点投影到半径为r的球面坐标系上的球面坐标。
进一步地，所述左眼球面图像所在的球心和右眼球面图像所在的球心之间的距离等于人眼瞳距。
第二方面，本发明提供了一种全景立体影像的成像装置，包括：
摄像单元排布模块，用于根据不用的应用场景，在空间中任意排布若干个摄像单元；
***标定模块，用于对所述摄像单元排布模块中已排布完成的各个摄像单元的内外参数进行标定，其与所述摄像单元排布模块相连接；
硬件通讯模块，用于控制所述摄像单元排布模块的各个摄像单元进行协同工作并进行像点坐标的同步采集，接收摄像单元排布模块中各个摄像单元采集到的像点坐标并传输，其与所述摄像单元排布模块相连接；
左眼和右眼球面坐标系指定模块，用于在空间中任意指定一个左眼球面坐标系OL和一个右眼球面坐标系OR；
映射函数解算模块，用于接收所述硬件通讯模块传输的像点坐标，并根据左眼和右眼球面坐标系指定模块中的左眼球面坐标系OL和右眼球面坐标系OR分别解算所述摄像单元排布模块中已标定摄像单元的像点坐标到左眼球面坐标系OL和右眼球面坐标系OR的映射函数，并根据解算的映射函数分别生成与左眼球面坐标系OL对应的左眼球面图像和与右眼球面坐标系OR对应的右眼球面图像，其与所述硬件通讯模块和所述左眼和右眼球面坐标系指定模块相连接；
图像显示模块，用于将所述映射函数解算模块中的左眼球面图像和右眼球面图像分别投到左眼和右眼中完成全景图像的立体显示，其与所述映射函数解算模块相连接。
进一步地，所述映射函数解算模块包括：
坐标关系确定单元，用于在摄像单元坐标系下，确定物点坐标与像点坐标的关系；
坐标系变换关系确定单元，用于标定球面坐标系和摄像单元坐标系，确定摄像单元坐标系和球面坐标系之间的变换关系；
物点的球面坐标系坐标求解单元，用于根据所述物点坐标与像点坐标的关系及所述摄像单元坐标系和球面坐标系之间的变换关系，求解物点在球面坐标系的坐标，其与所述坐标关系确定单元和所述坐标系变换关系确定单元相连接；
物点的球面坐标求解单元，用于根据所述物点在球面坐标系的坐标，求解物点投影到半径为r的球面坐标系上的球面坐标，其与所述物点的球面坐标系坐标求解单元相连接。
进一步地，所述左眼球面图像所在的球心和右眼球面图像所在的球心之间的距离等于人眼瞳距。
第三方面，本发明提供了一种全景立体影像的成像***，所述全景立体影像的成像***包括N个在同一圆的圆周上任意分布或等间距设置的摄像单元，其中N为大于或等于2的正整数。
本发明提供了一种全景立体影像的成像***，所述全景立体影像的成像***包括八个在同一四边形的四条边上任意分布或两两对称设置的摄像单元，所述摄像单元的视场角大于180度。
本发明提供了一种全景立体影像的成像***，所述全景立体影像的成像***包括四个在同一圆的圆周上任意分布或等间距设置的摄像单元A及两个背部对齐成轴对称设置且其光轴连线穿过由所述四个摄像单元A构成的平面的中心并与所述平面垂直的摄像单元B，所述摄像单元A的视场角大于180度，所述摄像单元B的视场角大于180度。
本发明提供了一种全景立体影像的成像***，所述全景立体影像的成像***包括八个在同一圆的圆周上任意分布或等间距设置的摄像单元C，及两个背部对齐成轴对称设置且其光轴连线穿过由所述八个摄像单元C构成的平面的中心并与所述平面垂直的摄像单元D，所述摄像单元C的视场角大于90度，所述摄像单元D的视场角大于180度。
与现有技术相比，本发明通过多摄像单元组合的方式实现了较大自由度的全景立体影像获取，并以此为基础通过使用多全景球对的方法解决了全景影像真实多视角立体显示的问题，本发明有效减少了全景拍摄设备在搭建过程中对各摄像单元参数的一致性和位置排布的均匀性的约束，大大提高了摄像单元拍摄的自由度和灵活性。
附图说明
图1为本发明第一实施例提供的全景立体影像的成像方法的流程示意图；
图2为图1中映射函数的解算方法的流程示意图；
图3为本发明第二实施例提供的全景立体影像的成像装置的结构示意图；
图4为图3中映射函数解算模块的结构示意图；
图5为单一摄像单元所成的像在任意球面坐标系下的映射关系示意图；
图6为多个摄像单元的交叠成像区域在任意球面坐标系的映射关系示意图；
图7为本发明第三实施例提供的全景立体影像的成像***的结构示意图；
图8为本发明第四实施例提供的全景立体影像的成像***的结构示意图；
图9为本发明第五实施例提供的全景立体影像的成像***的结构示意图；
图10为本发明第六实施例提供的全景立体影像的成像***的结构示意图；
图11为本发明第七实施例提供的全景立体影像的成像***的结构示意图；
图12为本发明第八实施例提供的全景立体影像的成像***的结构示意图；
图13为本发明第九实施例提供的全景立体影像的成像***的结构示意图；
图14为本发明全景立体影像成像装置向正前方观察时球面坐标系对的位置关系示意图；
图15为本发明全景立体影像成像装置转向右前方观察时球面坐标系对的位置关系示意图；
图16为本发明全景立体影像成像装置转向正右方观察时球面坐标系对的位置关系示意图；
图17为本发明全景立体影像成像装置在任意视角观察时球面坐标系对的位置关系示意图。
具体实施方式
为了使本发明的目的、技术方案及优点更加清楚明白，以下结合附图及实施例，对本发明进行进一步详细说明。应当理解，此处所描述的具体实施例仅仅用以解释本发明，并不用于限定本发明。
以下结合具体实施例对本发明的实现进行详细的描述。
请参阅图1，图1是本发明第一实施例提供的全景立体影像的成像方法的流程示意图，如图1所示，全景立体影像的成像方法包括以下步骤：
步骤S20，根据不同的应用场景，在空间中任意排布若干个摄像单元；
步骤S21，对已排布完成的各个摄像单元的内外参数进行标定；
步骤S22，控制各个摄像单元进行协同工作，并进行像点坐标的同步采集并传输；
步骤S23，在空间中任意指定一个左眼球面坐标系OL和一个右眼球面坐标系OR；
步骤S24，根据左眼球面坐标系OL和右眼球面坐标系OR分别解算摄像单元的像点坐标到左眼球面坐标系OL和右眼球面坐标系OR的映射函数，并根据解算的映射函数分别生成与左眼球面坐标系OL对应的左眼球面图像和与右眼球面坐标系OR对应的右眼球面图像；
步骤S25，将左眼球面图像和右眼球面图像分别投到左眼和右眼中完成全景图像的立体显示。
本发明第一实施例中，左眼球面图像所在的球心和右眼球面图像所在的球心之间的距离等于人眼瞳距。
请参阅图2，图2是图1中映射函数的解算方法的流程示意图，如图2所示，映射函数的解算方法包括以下步骤：
步骤S70，在摄像单元坐标系下，确定物点坐标与像点坐标的关系；
步骤S71，标定球面坐标系和摄像单元坐标系，确定摄像单元坐标系和球面坐标系之间的变换关系；
步骤S72，根据物点坐标与像点坐标的关系及摄像单元坐标系和球面坐标 系之间的变换关系，求解物点在球面坐标系的坐标；
步骤S73，根据物点在球面坐标系的坐标，求解物点投影到半径为r的球面坐标系上的球面坐标。
请参阅图3，图3是本发明第二实施例提供的全景立体影像的成像装置的结构示意图，如图3所示，全景立体影像的成像装置包括：
摄像单元排布模块31，用于根据不用的应用场景，在空间中任意排布若干个摄像单元；
***标定模块32，用于对摄像单元排布模块31中已排布完成的各个摄像单元的内外参数进行标定，其与摄像单元排布模块31相连接；
硬件通讯模块33，用于控制摄像单元排布模块31的各个摄像单元进行协同工作并进行像点坐标的同步采集，接收摄像单元排布模块31中各个摄像单元采集到的像点坐标并传输，其与摄像单元排布模块31相连接；
左眼和右眼球面坐标系指定模块34，用于在空间中任意指定一个左眼球面坐标系OL和一个右眼球面坐标系OR；
映射函数解算模块35，用于接收硬件通讯模块33传输的像点坐标，并根据左眼和右眼球面坐标系指定模块34中的左眼球面坐标系OL和右眼球面坐标系OR分别解算摄像单元排布模块31中已标定摄像单元的像点坐标到左眼球面坐标系OL和右眼球面坐标系OR的映射函数，并根据解算的映射函数分别生成与左眼球面坐标系OL对应的左眼球面图像和与右眼球面坐标系OR对应的右眼球面图像，其与硬件通讯模块33和左眼和右眼球面坐标系指定模块34相连接；
图像显示模块36，用于将映射函数解算模块35中的左眼球面图像和右眼球面图像分别投到左眼和右眼中完成全景图像的立体显示，其与映射函数解算模块35相连接。
本发明第二实施例中，左眼球面图像所在的球心和右眼球面图像所在的球心之间的距离等于人眼瞳距。
请参阅图4，图4是图3中映射函数解算模块的结构示意图，如图4所示，映射函数解算模块包括：
坐标关系确定单元81，用于在摄像单元坐标系下，确定物点坐标与像点坐标的关系；
坐标系变换关系确定单元82，用于标定球面坐标系和摄像单元坐标系，确定摄像单元坐标系和球面坐标系之间的变换关系；
物点的球面坐标系坐标求解单元83，用于根据物点坐标与像点坐标的关系及摄像单元坐标系和球面坐标系之间的变换关系，求解物点在球面坐标系的坐标，其与坐标关系确定单元81和坐标系变换关系确定单元82相连接；
物点的球面坐标求解单元84，用于根据物点在球面坐标系的坐标，求解物点投影到半径为r的球面坐标系上的球面坐标，其与物点的球面坐标系坐标求解单元83相连接。
请参阅图5，图5是单一摄像单元所成的像在任意球面坐标系下的映射关系示意图，如图5所示，根据任意球面坐标系501与任意空间姿态的摄像单元502之间的空间关系，以及摄像单元的内部参数，可以建立像点坐标I(x1′，y1′，f)、物距d到球面坐标的映射函数，其中球面坐标系表示为OXYZ，摄像单元坐标系表示为O’X’Y’Z’，摄像单元内部参数以向量的形式记为K，包括光心成像坐标、焦距、综合畸变参数、镜头视场角等。
图5是空间映射关系在二维平面(OXZ)的简化示例，根据摄像单元坐标系(O’X’Y’Z’)与球面坐标系的相对位置关系，可以得到光心在平面(OXZ)上的坐标O′(x0，z0)，同时可将像点坐标I(x1′，y1′，f)转换为平面(OXZ)上的坐标I(x1，z1)；物点坐标P(x3′，y3′，z3′)可以通过摄像单元成像公式和物距d求得；根据摄像单元坐标系(O’X’Y’Z’)和球面坐标系OXYZ的空间关系，求得物点P(x3′，y3′，z3′)在球面坐标系下的坐标值(R，θ)，R是球的半径，θ是向量OP在OXZ平面上与X轴的夹角。同理，可以将上述二维平面的映射关系扩展为三维空间的映射关系。
由图5所示的几何关系可以将摄像单元坐标系下任意像点I(xi′，yi′，f)映射到球面坐标系中，映射函数可表示为
Figure PCTCN2017080195-appb-000001
其中O表示球面坐标OXYZ，O’表示摄像单元坐标系O’X’Y’Z’，K＝(x0′，y0′，f，Δx′，Δy′)表示摄像单元内部参数向量，其中(x0′，y0′)表示光心在像面的坐标，f表示焦距，(Δx′，Δy′)表示成像综合畸变，d表示物距，(xi′，yi′)表示像点坐标，其中物距d可以人为指定或使用距离传感器测量获得，其余参数通过***标定获得。
具体地，映射函数可以通过以下计算过程实现：
1.物点P(x3′，y3′，z3′)在摄像单元坐标系O’X’Y’Z’下与像点坐标I(xi′，yi′，f)的关系如式1所示：
Figure PCTCN2017080195-appb-000002
2.已标定O和O′的前提下，球面坐标系和摄像单元坐标系之间的变换关系如式2所示：
O＝R×O′+T                (式2)
可通过式2求得一个3行3列的旋转矩阵R，和一个3行1列的平移矩阵T。
3.由式1和式2的结果，物点P在球面坐标系的坐标P(x3，y3，z3)可由式3求得：
Figure PCTCN2017080195-appb-000003
4.综上，物点P投影到半径为r的球面坐标系OXYZ下球面的坐标可由式4求得：
Figure PCTCN2017080195-appb-000004
请参阅图6，图6是多个摄像单元的交叠成像区域在任意球面坐标系的映射关系示意图，如图6所示，在多摄像单元的交叠成像区域，不同摄像单元坐标系下的物距d可由多目立体视觉关系求解得出，所以在任意球面坐标系601下，根据相互交叠成像的摄像单元602和603之间的空间关系，可解得物点P相对摄像单元602的物距，根据图3的分析可知，在P的物距和像点坐标已知的情况下，可以根据球面坐标系OXYZ和摄像单元坐标系OXY的空间关系，求得点P在球面坐标系下的坐标系OXYZ和摄像单元坐标系O1X1Y的空间关系，求得点P在球面坐标系下的坐标，所以对于交叠成像的两个摄像单元，可以将物点P到球面坐标系的映射关系表示
Figure PCTCN2017080195-appb-000005
K1和K2分别表示摄像单元602和摄像单元603的内部参数向量，(xi1，yi1)和(xi2，yi2)分别是物点P在摄像单元602和摄像单元603中的像点坐标，该方法不需要预知物距d，可以自动解算出物点的空间三维坐标。
本发明提供了一种全景立体影像的成像***，包括N个在同一圆的圆周上任意分布或等间距设置的摄像单元，其中N为大于或等于2的正整数。请参阅图7，图7是本发明第三实施例提供的全景立体影像的成像***的结构示意图，如图7所示，该全景立体影像的成像***包括两个背部对齐成轴对称设置的摄像单元41，所述摄像单元41的视场角大于180度。
请参阅图8，图8是本发明第四实施例提供的全景立体影像的成像***的结构示意图，如图8所示，该全景立体影像的成像***包括三个在同一圆的圆周上任意分布或等间距设置的摄像单元42，所述摄像单元42的视场角大于120度。
请参阅图9，图9是本发明第五实施例提供的全景立体影像的成像***的结构示意图，如图9所示，该全景立体影像的成像***包括四个在同一圆的圆周上任意分布或等间距设置的摄像单元43，所述摄像单元43的视场角大于180度。
请参阅图10，图10是本发明第六实施例提供的全景立体影像的成像***的结构示意图，如图10所示，该全景立体影像的成像***包括八个在同一圆的圆周上任意分布或等间距设置的摄像单元44，所述摄像单元44的视场角大于90度。
请参阅图11，图11是本发明第七实施例提供的全景立体影像的成像***的结构示意图，如图11所示，该全景立体影像的成像***包括四个在同一圆的圆周上任意分布或等间距设置的摄像单元A45及两个背部对齐成轴对称设置且其光轴连线穿过由所述四个摄像单元A45构成的平面的中心并与所述平面垂直的摄像单元B46，所述摄像单元A45的视场角大于180度，所述摄像单元B46的视场角大于180度。
请参阅图12，图12是本发明第八实施例提供的全景立体影像的成像***的结构示意图，如图12所示，该全景立体影像的成像***包括八个在同一圆的圆周上任意分布或等间距设置的摄像单元C47，及两个背部对齐成轴对称设置且其光轴连线穿过由所述八个摄像单元C47构成的平面的中心并与所述平面垂直的摄像单元D48，所述摄像单元C47的视场角大于90度，所述摄像单元D48的视场角大于180度。
本发明提供了另一种全景立体影像的成像***，包括M个在同一正多边形的边上任意分布或相互对称设置的摄像单元，其中M为大于或等于2的正整数。请参阅图13，图13是本发明第九实施例提供的全景立体影像的成像***的结构示意图，如图13所示，该全景立体影像的成像***包括八个在同一四边形的四条边上任意分布或两两对称设置的摄像单元49，所述摄像单元49的视场角大于180度。
本发明的全景立体影像成像方法可以将全景立体影像成像装置拍摄到的物点映射到任意中心的球面坐标系下，构成图像的全景球，所以可以通过将物点映射到多组球心距离接近观察者瞳距的球面坐标系对的方式，模拟不同视角情况下左右眼的成像全景球，并将球面坐标系对所对应的左右眼全景球图像通过分屏显示装置分别投射到左右眼中，来实现全景图像的立体显示。请参阅图14，图14是本发明全景立体影像成像装置向正前方观察时球面坐标系对的位置关系示意图，如图14所示OL表示左眼全景球坐标系，OR表示右眼全景球坐标系，为了给观察者以更自然的体验，左眼全景球坐标系OL与右眼全景球坐标系OR球心间的距离D可根据需求设置为观察者瞳距，左眼全景球和右眼全景球的半径设为相同值R；请参阅图15，图15是本发明全景立体影像成像装置转向右前方观察时球面坐标系对的位置关系示意图；请参阅图16，图16是本发明全景立体影像成像装置转向正右方观察时球面坐标系对的位置关系示意图；请参阅图17，图17是本发明全景立体影像成像装置在任意视角观察时球面坐标系对的位置关系示意图。
本发明通过多摄像单元组合的方式实现了较大自由度的全景立体影像获取，并以此为基础通过使用多全景球对的方法解决了全景影像真实多视角立体显示的问题，本发明有效减少了全景拍摄设备在搭建过程中对各摄像单元参数的一致性和位置排布的均匀性的约束，大大提高了摄像单元拍摄的自由度和灵活性。
以上所述仅为本发明的较佳实施例而已，并不用以限制本发明，凡在本发明的精神和原则之内所作的任何修改、等同替换和改进等，均应包含在本发明的保护范围之内。

Claims (10)

  1. 一种全景立体影像的成像方法，其特征在于，包括下述步骤：
    步骤A，根据不同的应用场景，在空间中任意排布若干个摄像单元；
    步骤B，对已排布完成的所述步骤A中的各个摄像单元的内外参数进行标定；
    步骤C，控制各个所述步骤B中的摄像单元进行协同工作，并进行像点坐标的同步采集并传输；
    步骤D，在空间中任意指定一个左眼球面坐标系OL和一个右眼球面坐标系OR；
    步骤E，根据所述步骤D中的左眼球面坐标系OL和右眼球面坐标系OR分别解算所述步骤C中的摄像单元的像点坐标到左眼球面坐标系OL和右眼球面坐标系OR的映射函数，并根据解算的映射函数分别生成与左眼球面坐标系OL对应的左眼球面图像和与右眼球面坐标系OR对应的右眼球面图像；
    步骤F，将所述步骤E中的左眼球面图像和右眼球面图像分别投到左眼和右眼中完成全景图像的立体显示。
  2. 如权利要求1所述的全景立体影像的成像方法，其特征在于，所述映射函数的解算方法包括下述步骤：
    步骤E1，在摄像单元坐标系下，确定物点坐标与像点坐标的关系；
    步骤E2，标定球面坐标系和摄像单元坐标系，确定摄像单元坐标系和球面坐标系之间的变换关系；
    步骤E3，根据所述步骤E1中的物点坐标与像点坐标的关系及所述步骤E2中的摄像单元坐标系和球面坐标系之间的变换关系，求解物点在球面坐标系的坐标；
    步骤E4，根据所述步骤E3中的物点在球面坐标系的坐标，求解物点投影到半径为r的球面坐标系上的球面坐标。
  3. 如权利要求1所述的全景立体影像的成像方法，其特征在于，所述左眼 球面图像所在的球心和右眼球面图像所在的球心之间的距离等于人眼瞳距。
  4. 一种全景立体影像的成像装置，其特征在于，包括：
    摄像单元排布模块，用于根据不用的应用场景，在空间中任意排布若干个摄像单元；
    ***标定模块，用于对所述摄像单元排布模块中已排布完成的各个摄像单元的内外参数进行标定，其与所述摄像单元排布模块相连接；
    硬件通讯模块，用于控制所述摄像单元排布模块的各个摄像单元进行协同工作并进行像点坐标的同步采集，接收摄像单元排布模块中各个摄像单元采集到的像点坐标并传输，其与所述摄像单元排布模块相连接；
    左眼和右眼球面坐标系指定模块，用于在空间中任意指定一个左眼球面坐标系OL和一个右眼球面坐标系OR；
    映射函数解算模块，用于接收所述硬件通讯模块传输的像点坐标，并根据左眼和右眼球面坐标系指定模块中的左眼球面坐标系OL和右眼球面坐标系OR分别解算所述摄像单元排布模块中已标定摄像单元的像点坐标到左眼球面坐标系OL和右眼球面坐标系OR的映射函数，并根据解算的映射函数分别生成与左眼球面坐标系OL对应的左眼球面图像和与右眼球面坐标系OR对应的右眼球面图像，其与所述硬件通讯模块和所述左眼和右眼球面坐标系指定模块相连接；
    图像显示模块，用于将所述映射函数解算模块中的左眼球面图像和右眼球面图像分别投到左眼和右眼中完成全景图像的立体显示，其与所述映射函数解算模块相连接。
  5. 如权利要求4所述的全景立体影像的成像装置，其特征在于，所述映射函数解算模块包括：
    坐标关系确定单元，用于在摄像单元坐标系下，确定物点坐标与像点坐标的关系；
    坐标系变换关系确定单元，用于标定球面坐标系和摄像单元坐标系，确定 摄像单元坐标系和球面坐标系之间的变换关系；
    物点的球面坐标系坐标求解单元，用于根据所述物点坐标与像点坐标的关系及所述摄像单元坐标系和球面坐标系之间的变换关系，求解物点在球面坐标系的坐标，其与所述坐标关系确定单元和所述坐标系变换关系确定单元相连接；
    物点的球面坐标求解单元，用于根据所述物点在球面坐标系的坐标，求解物点投影到半径为r的球面坐标系上的球面坐标，其与所述物点的球面坐标系坐标求解单元相连接。
  6. 如权利要求4所述的全景立体影像的成像装置，其特征在于，所述左眼球面图像所在的球心和右眼球面图像所在的球心之间的距离等于人眼瞳距。
  7. 一种全景立体影像的成像***，其特征在于，所述全景立体影像的成像***包括N个在同一圆的圆周上任意分布或等间距设置的摄像单元，其中N为大于或等于2的正整数。
  8. 一种全景立体影像的成像***，其特征在于，所述全景立体影像的成像***包括八个在同一四边形的四条边上任意分布或两两对称设置的摄像单元，所述摄像单元的视场角大于180度。
  9. 一种全景立体影像的成像***，其特征在于，所述全景立体影像的成像***包括四个在同一圆的圆周上任意分布或等间距设置的摄像单元A及两个背部对齐成轴对称设置且其光轴连线穿过由所述四个摄像单元A构成的平面的中心并与所述平面垂直的摄像单元B，所述摄像单元A的视场角大于180度，所述摄像单元B的视场角大于180度。
  10. 一种全景立体影像的成像***，其特征在于，所述全景立体影像的成像***包括八个在同一圆的圆周上任意分布或等间距设置的摄像单元C，及两个背部对齐成轴对称设置且其光轴连线穿过由所述八个摄像单元C构成的平面的中心并与所述平面垂直的摄像单元D，所述摄像单元C的视场角大于90度，所述摄像单元D的视场角大于180度。
PCT/CN2017/080195 2016-06-27 2017-04-12 一种全景立体影像的成像方法、装置及*** WO2018000892A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201610475449.2A CN106990668B (zh) 2016-06-27 2016-06-27 一种全景立体影像的成像方法、装置及***
CN201610475449.2 2016-06-27

Publications (1)

Publication Number Publication Date
WO2018000892A1 true WO2018000892A1 (zh) 2018-01-04

Family

ID=59413654

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2017/080195 WO2018000892A1 (zh) 2016-06-27 2017-04-12 一种全景立体影像的成像方法、装置及***

Country Status (2)

Country Link
CN (1) CN106990668B (zh)
WO (1) WO2018000892A1 (zh)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110766731A (zh) * 2019-10-21 2020-02-07 武汉中海庭数据技术有限公司 一种全景影像与点云自动配准的方法、装置及存储介质
CN111429529A (zh) * 2020-04-10 2020-07-17 浙江大华技术股份有限公司 一种坐标转换的标定方法、电子设备以及计算机存储介质
CN113012032A (zh) * 2021-03-03 2021-06-22 中国人民解放军战略支援部队信息工程大学 一种自动标注地名的空中全景图像显示方法

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108322730A (zh) * 2018-03-09 2018-07-24 嘀拍信息科技南通有限公司 一种可采集360度场景结构的全景深度相机***

Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101365140A (zh) * 2008-09-25 2009-02-11 浙江大学 基于矩阵分解的摄像机阵列标定方法
US20100328428A1 (en) * 2009-06-26 2010-12-30 Booth Jr Lawrence A Optimized stereoscopic visualization
CN102170576A (zh) * 2011-01-30 2011-08-31 中兴通讯股份有限公司 双摄像头立体拍摄的处理方法及装置
CN103149788A (zh) * 2013-03-22 2013-06-12 天津曙光敬业科技有限公司 空中360°全景照片拍摄装置及方法
CN203069959U (zh) * 2012-12-13 2013-07-17 中国科学院宁波材料技术与工程研究所 一种360度全景摄像机
CN103235478A (zh) * 2012-12-19 2013-08-07 乔楠 用于环幕立体摄像的***、设备、图像处理方法及装置
CN103973944A (zh) * 2013-02-06 2014-08-06 深圳市振华微电子有限公司 半球型全景成像装置及方法
CN204631439U (zh) * 2015-05-29 2015-09-09 杨振贤 一种图像采集设备
CN105187812A (zh) * 2015-09-02 2015-12-23 中国兵器工业计算机应用技术研究所 一种双目视觉立体匹配算法

Patent Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101365140A (zh) * 2008-09-25 2009-02-11 浙江大学 基于矩阵分解的摄像机阵列标定方法
US20100328428A1 (en) * 2009-06-26 2010-12-30 Booth Jr Lawrence A Optimized stereoscopic visualization
CN102170576A (zh) * 2011-01-30 2011-08-31 中兴通讯股份有限公司 双摄像头立体拍摄的处理方法及装置
CN203069959U (zh) * 2012-12-13 2013-07-17 中国科学院宁波材料技术与工程研究所 一种360度全景摄像机
CN103235478A (zh) * 2012-12-19 2013-08-07 乔楠 用于环幕立体摄像的***、设备、图像处理方法及装置
CN103973944A (zh) * 2013-02-06 2014-08-06 深圳市振华微电子有限公司 半球型全景成像装置及方法
CN103149788A (zh) * 2013-03-22 2013-06-12 天津曙光敬业科技有限公司 空中360°全景照片拍摄装置及方法
CN204631439U (zh) * 2015-05-29 2015-09-09 杨振贤 一种图像采集设备
CN105187812A (zh) * 2015-09-02 2015-12-23 中国兵器工业计算机应用技术研究所 一种双目视觉立体匹配算法

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110766731A (zh) * 2019-10-21 2020-02-07 武汉中海庭数据技术有限公司 一种全景影像与点云自动配准的方法、装置及存储介质
CN111429529A (zh) * 2020-04-10 2020-07-17 浙江大华技术股份有限公司 一种坐标转换的标定方法、电子设备以及计算机存储介质
CN111429529B (zh) * 2020-04-10 2024-02-20 浙江大华技术股份有限公司 一种坐标转换的标定方法、电子设备以及计算机存储介质
CN113012032A (zh) * 2021-03-03 2021-06-22 中国人民解放军战略支援部队信息工程大学 一种自动标注地名的空中全景图像显示方法

Also Published As

Publication number Publication date
CN106990668A (zh) 2017-07-28
CN106990668B (zh) 2019-10-11

Similar Documents

Publication Publication Date Title
TWI555378B (zh) 一種全景魚眼相機影像校正、合成與景深重建方法與其系統
WO2018076154A1 (zh) 一种基于鱼眼摄像机空间位姿标定的全景视频生成方法
US11184597B2 (en) Information processing device, image generation method, and head-mounted display
WO2018000892A1 (zh) 一种全景立体影像的成像方法、装置及***
WO2010119852A1 (ja) 任意視点画像合成装置
KR20180101165A (ko) 파노라마 프레임으로의 프레임 스티칭
JP2017505565A (ja) 多面映像の生成方法及びシステム
CN107578450B (zh) 一种用于全景相机装配误差标定的方法及***
CN109285189B (zh) 一种无需双目同步的直线轨迹快速计算方法
JP4928476B2 (ja) 立体像生成装置、その方法およびそのプログラム
CN107545537A (zh) 一种从稠密点云生成3d全景图片的方法
US11812009B2 (en) Generating virtual reality content via light fields
WO2019047847A1 (zh) 一种虚拟现实的六自由度三维重构方法、***及便携式终端
TWI788739B (zh) 3d顯示設備、3d圖像顯示方法
WO2018032841A1 (zh) 绘制三维图像的方法及其设备、***
WO2021104308A1 (zh) 全景深度测量方法、四目鱼眼相机及双目鱼眼相机
CN105139336B (zh) 一种多通道全景影像转换球幕鱼眼影片的方法
TW201632982A (zh) 三維全景影像之產生方法
CN104159099A (zh) 一种3d立体影视制作中双目立体摄像机的设置方法
CN108450031A (zh) 图像捕获装置
CN110060349B (zh) 一种扩展增强现实头戴式显示设备视场角的方法
CN110766752A (zh) 一种带反光标志点的虚拟现实交互眼镜及空间定位方法
Hasmanda et al. The modelling of stereoscopic 3D scene acquisition
JP2013105002A (ja) ３ｄ映像撮影制御システム、３ｄ映像撮影制御方法、およびプログラム
JP2011182003A (ja) パノラマカメラ及び３６０度パノラマ立体映像システム

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 17818889

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 17818889

Country of ref document: EP

Kind code of ref document: A1