WO2023003527A1 - Method for calibration of an optical-digital projection system - Google Patents

Method for calibration of an optical-digital projection system Download PDF

Info

Publication number
WO2023003527A1
WO2023003527A1 PCT/UA2022/000037 UA2022000037W WO2023003527A1 WO 2023003527 A1 WO2023003527 A1 WO 2023003527A1 UA 2022000037 W UA2022000037 W UA 2022000037W WO 2023003527 A1 WO2023003527 A1 WO 2023003527A1
Authority
WO
WIPO (PCT)
Prior art keywords
projection system
dome
digital
optical
point
Prior art date
Application number
PCT/UA2022/000037
Other languages
French (fr)
Inventor
Vitaliy Serhiiovych SLYUSARENKO
Original Assignee
Slyusarenko Vitaliy Serhiiovych
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Slyusarenko Vitaliy Serhiiovych filed Critical Slyusarenko Vitaliy Serhiiovych
Publication of WO2023003527A1 publication Critical patent/WO2023003527A1/en

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N9/00—Details of colour television systems
    • H04N9/12—Picture reproducers
    • H04N9/31—Projection devices for colour picture display, e.g. using electronic spatial light modulators [ESLM]
    • H04N9/3179—Video signal processing therefor
    • H04N9/3185—Geometric adjustment, e.g. keystone or convergence
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N9/00—Details of colour television systems
    • H04N9/12—Picture reproducers
    • H04N9/31—Projection devices for colour picture display, e.g. using electronic spatial light modulators [ESLM]
    • H04N9/3141—Constructional details thereof
    • H04N9/3147—Multi-projection systems
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N9/00—Details of colour television systems
    • H04N9/12—Picture reproducers
    • H04N9/31—Projection devices for colour picture display, e.g. using electronic spatial light modulators [ESLM]
    • H04N9/3191—Testing thereof
    • H04N9/3194—Testing thereof including sensor feedback

Definitions

  • a claimed method of calibration of the optical-digital projection system relates to the means of setting up projection systems or projection-type devices.
  • a proposed technical solution can be applied to calibrate and synchronize optical and digital projection systems of image or video playback that project images onto a single screen or surface.
  • the position of the camera is controlled so that the predetermined height and azimuth orientation on a dome are aligned with the picture center of the camera.
  • the video projecting system makes a cursor projected from the video projector in alignment with the picture center of the video camera and measures the height and an azimuth angle of this point image and projector coordinates of the cursor at a plurality of points.
  • a coordinate transformation is carried out on desired horizontal coordinates and a projected picture in accordance with measured data, set positions and position angles of the projector and the like are calculated; a coordinate transformation is carried out based on these values to display desired images and figures on desired horizontal coordinates.
  • the claimed technical solution is based on the task of creating a method that will provide calibration and synchronization of multiple digital projectors and an optical projection system while providing corrections for the effect of distortion.
  • the task is to combine the projection coordinate systems of the digital projection system and the optical projection system.
  • the problem is solved by proposing a method of calibrating an optical- digital projection system, which includes the use of an optical projection system placed in a demonstration hall to project an image of the starry sky onto the surface of the dome of the demonstration hall.
  • the optical-digital projection system additionally incorporates a digital projection system, which contains multiple digital projectors for playing back an additional image by reproducing a projection on the corresponding fragment of the surface of the demonstration hall dome, and an electronic computer and the optical projection system is connected to an electronic computer, and the optical projection system has a case and contains a camera that is connected to an electronic computer and transmits an image of the fragment of the surface of the dome, which has certain coordinates, and the electronic computer contains a device for visual data output, a device for data input, and a processing unit, moreover, the processing unit contains a stored set of instructions.
  • the method includes the following steps:
  • the digital projection system reproduces a sequence of graphic patterns on the surface of the dome of the demonstration hall, while
  • the problem is solved by a variant of the claimed invention, according to which the method of calibration of the optical-digital projection system, which includes the use of an optical projection system placed in the demonstration hall to project an image of the starry sky onto the surface of the dome of the demonstration hall, is determined.
  • the optical-digital projection system additionally incorporates a digital projection system, which contains multiple digital projectors for playing back an additional image by reproducing a projection on the corresponding fragment of the surface of the demonstration hall dome, and an electronic computer and the optical projection system is connected to an electronic computer, and the optical projection system has a case and contains a camera that is connected to an electronic computer and transmits an image of the fragment of the surface of the dome, which has certain coordinates, and the electronic computer contains a device for visual data output, a device for data input, and a processing unit, moreover, the processing unit contains a stored set of instructions.
  • the variant of the method includes the following steps:
  • the calculation point is located within the surface of the dome on the projection of the digital projection system and not in the blind area of the projection of the digital projection system, determining using the camera whether an offset exists between the calculation point and the control point based on the orientation of the optical projection system for the corresponding position of the optical projection system in the projection coordinate system of the digital projection system, and determining the numerical value of the offset,
  • a variant of the method implies that if the calculation point is located within the surface of the dome on the projection of the digital projection system and not in the blind area of the projection of the digital projection system,
  • the variant of the method claims that, if the calculation point is located within the dome surface on the projection of the digital projection system and not in the blind area of the projection of the digital projection system, then to determine the control point, after each rotation of the optical projection system, the digital projection system reproduces a sequence of graphic patterns on the surface of the dome of the demonstration hall, while
  • the current invention achieves the technical result of shortening the calibration time of the optical-digital projection system compared to its nearest analog counterpart.
  • the additional technical result is a reduction in distortion and an increase in alignment accuracy when projecting the image.
  • Another advantage of the claimed technical solution is an increase in the accuracy of calibration, which is achieved by increasing the number of recognized control points.
  • the method allows calibrating the digital projection system more accurately on the dome screen, which has blind spots not covered by the projection of the digital projection system.
  • blind areas can be expressed as follows:
  • the method allows calibrating the digital projection system more accurately on the dome screen when the projection of the digital projection system extends beyond the dome surface.
  • an obstacle is overcome in which the calculation points at the equator of the projection of the digital projection system are not visible partially or completely since they are projected outside the dome surface.
  • FIG. 1 illustrates the arrangement of the components of the optical projection system relative to the dome.
  • FIG. 2 illustrates an image of the fragment of the surface of the dome covered by the camera field of view obtained from the camera and transmitted to the electronic computer.
  • FIG. 3 illustrates the changes in the coordinate grid of the projection of the digital projection system
  • FIG. 3a illustrates the formed conditional grid of positions of calculation points before combining optical and digital projection systems
  • Fig. 3b illustrates the coordinate grid of the projection of the digital projection system with superimposed calculation points, in which the distortion of the projection of the digital projection system is implemented
  • FIG. 3c illustrates the projection of a digital projection system with implemented distortions and superimposed calculation points.
  • Fig. 1 is a fragment of the surface of the dome
  • 2 is a field of view of the camera
  • 3 is a camera
  • 4 is an optical projection system
  • 5 is the surface of the dome (dome screen) in the demonstration hall.
  • Fig. 2 6 is an image of a fragment of the surface of the dome, 7 is the location where the calculation point is determined, and 8 is the location where the control point is established.
  • “Fragment of the surface of the dome” is an area of the surface of the dome covered by the camera's field of view every time the optical projection system changes orientation.
  • “Plurality of fragments of the surface of the dome” refers to the entirety of all fragments of the surface of the dome, overlapping or otherwise, that can be covered by the camera field of view in all possible orientation positions of the optical projection system.
  • Offset is a numerical value of the distance between the calculated point and the control point, which is determined based on the image received from the camera.
  • “Shift” is a numerical value that indicates the distance the current position of the calculated point has shifted in comparison with its position in the previous iteration.
  • the first variant of the method of the claimed technical solution involves determining the calibration of the optical-digital projection system based on points displayed on the dome surface and visible in the fragment of the surface of the dome covered by the camera field of view.
  • the second variant of the method is aimed at calibrating the optical-digital projection system, assuming that the calculation point cannot be determined directly because it is located in the blind area of the projection of the digital projection system or at the equator of the projection of the digital projection system and partially or completely outside the dome surface.
  • the optical-digital projection system can be used to display the starry sky on the dome of the planetarium.
  • the reproduction of the image can be carried out on any static surface of the demonstration hall.
  • the demonstration hall means some limited area containing the surface on which the image can be reproduced.
  • the surface on which the image is reproduced may be a screen or have a different definition.
  • the demonstration hall should be understood as a planetarium, and the surface on which the image is reproduced is the dome.
  • Optical projection can be accomplished in a planetarium using a star projector, which reproduces an imitation of the starry sky on the dome's surface.
  • This type of system is mainly located in the center of the demonstration hall.
  • the optical projection system projects the image of the starry sky onto the surface of the planetarium dome, where the specified image is taken as a reference.
  • the optical projection system is connected to the electronic computer through communication channels to transmit the image of the fragment of the surface of the dome with defined coordinates and covered by the camera's field of view. Preference is given to wired communication channels, which have higher bandwidth.
  • the optical projection system has a case and comprises a camera that is configured so its field of view covers one fragment of the surface of the dome with defined coordinates of a plurality of fragments of the surface of the dome, to form an image of the fragment of the surface of the dome covered by the camera field of view and transmit it to an electronic computer.
  • the camera is preferably mounted on the equator of the optical projection system and can be placed both outside the case and inside the case.
  • the optical projection system preferably includes a light source that emits a laser beam on a specific fragment of the surface of the dome.
  • said light source may be directed at a point to which coordinates origin (the zero latitude/longitude point) are assigned in the coordinate system of the optical projection system.
  • the electronic computer always contains information about the coordinates of the zero latitude/longitude point of the coordinate system of the optical projection system, regardless of the position of the optical projection system. This is provided by the projection of the laser beam on the surface of the dome of the showroom, where the projection of the laser beam indicates the zero latitude/longitude point of the coordinate system of the optical projection system.
  • the field of view of the camera can cover a corresponding fragment of the surface of the dome with predetermined coordinates.
  • the digital projection system includes a plurality of digital projectors that are pre-calibrated to reproduce the additional image on the surface of the dome of the showroom. Thus, allowing a complete image to be reproduced by the entire plurality of digital projectors. In this case, each digital projector from a plurality of digital projectors reproduces the projection on the corresponding fragment of the surface of the dome of the showroom. Moreover, the digital projection system receives data for reproduction from an electronic computer.
  • the projection coordinate system of the digital projection system needs to be adjusted to match the coordinate system of the optical projection system.
  • the absence of such calibration will be followed by the dealignment of the images of the optical projection system and the digital projection system.
  • images of the starry sky are displayed on the surface of the dome using an optical projection system, and with the help of a digital projection system, connecting lines are drawn between the "stars" to highlight the constellation in the starry sky.
  • the electronic computer is a software-controlled device for information processing, comprising a device for visual data output, a device for data input, and a processing unit.
  • a device for visual data output a device for visual data output
  • a device for data input a device for data input
  • a processing unit a processing unit for processing
  • Such an electronic computer can be a desktop personal computer, laptop, etc., that supports network communication.
  • the electronic computer is configured to issue commands to the optical projection system to perform its rotation at a given angle in a given direction along a given axis.
  • 36 control points are used to determine the angles of rotation.
  • a device for visual data output is understood to be a high-resolution electronic device, such as a monitor, or a screen of a personal electronic computing device such as a laptop, tablet, etc.
  • a device for data input is understood to be: 1) any manual input devices, such as a keyboard, mouse, touch screen, touchpad, etc. 2) information inputs for receiving signals from external sources, such as a camera and a digital projector.
  • a processing unit is understood to be an arithmetic and logic device designed to control an electronic computer.
  • the processing unit contains a stored set of instructions, the execution of which ensures the operation of the proposed method, including calibration and playback control of optical and digital projection systems.
  • the control point must first be determined.
  • a source of laser emission is fixed firmly on the optical projection system.
  • the control point should be determined only once in the frame of the camera. Therefore, the control point must be located a priori in the camera frame.
  • FIG. 1 shows an installed optical projection system in a showroom, preferably on a fixed base.
  • a plurality of digital projectors is installed and calibrated together to produce a single image reproduced by said plurality of digital projection devices.
  • the optical projection system is oriented at a predetermined position and predetermined coordinates associated with the rotation angle of the optical projection system.
  • the camera is calibrated until the zero latitude/longitude point of the coordinate system of the optical projection system is determined.
  • a laser beam projected onto the surface of the dome of the demonstration hall is used to determine the location of the zero latitude/longitude point of the coordinate system of the optical projection system.
  • the location of the zero latitude/longitude point of the coordinate system of the optical projection system is determined by assessing the spot formed by the laser beam on the surface of the dome. By doing so, a primary reference point for further calibration can be set and a reference to a specific point can be created. It should be noted that the point in the camera frame occupies a certain area defined by a set of pixels of the digital image. The mathematical center of this area corresponds to the coordinates of the control point. Then, based on the orientation of the optical projection system for the corresponding position of the optical projection system, the coordinates of the calculation point in the projection coordinate system of the digital projection system are determined.
  • the optical projection system is alternately oriented to the remaining predetermined positions and predetermined coordinates, and during rotation stop in each such position, the camera captures one fragment of the surface of the dome with defined coordinates of a plurality of fragments of the surface of the dome, to form an image of the fragment of the surface of the dome covered by the camera field of view. Then the existence of an offset of the calculation point relative to the control point is determined. This stage is completed when the offsets for all calculation points are determined. The number of calculation points must be equal to the number of predetermined positions of the optical projection system in which the determination of the specified offset is performed. After that, the calculation points are superposed with the control points, thereby distorting the projection of the digital projection system.
  • the "determination of the existence of an offset” should be understood as the initiation of the algorithm for determining the numerical value, namely the numerical value of an offset.
  • the value of the distortions implemented into the projection at a given calculation point is zero.
  • this numerical value is crucial for determining the distortion implemented into the projection at a given calculation point.
  • Offset determination and alignment can be performed by two alternative embodiments, which are refinements of such an offset detennination. According to the first alternative embodiment, the combination of the projection coordinate systems of the optical and digital projection systems occurs due to the implementation of corrective distortions in the projection of the digital projection system at the calculation points.
  • the optical projection system the orientation of which is controlled along all coordinate axes, is sequentially rotated to certain positions. For these positions of the optical projection system (based on orientation data), the coordinates of the corresponding calculation points are predetermined. Thus, the coordinates of the points that correspond to the coordinates of the zero latitude/longitude points of the coordinate system of the optical projection system at the corresponding positions of the optical projection system are determined, and they can now be detennined in the projection coordinate system of the digital projection system.
  • the calculation point is reproduced by the digital projection system in the coordinate system of the projection of the digital projection system with coordinates matching the coordinates of the control point (for the optical projection system), taking into account the rotation of the optical projection system.
  • the fragment of the surface of the dome covered by the camera's field of view is captured by the camera, and this image is transferred to an electronic computer.
  • the laser source and the camera rotate synchronously with the optical projection system.
  • the position of the control point within the camera frame remains in a certain place, regardless of the orientation of the optical projection system.
  • the point in the camera frame occupies a certain area defined by a set of pixels of the digital image.
  • the coordinates of the calculation point associated with the positioning of the optical projection system are determined on the image obtained from the camera of the fragment of the surface of the dome with defined coordinates.
  • the processing unit iteratively shifts the corresponding calculation point on the projection of the digital projection system until it coincides with a previously known (control) point in the camera frame at a specific position of the optical projection system.
  • the position of the calculation point on the projection of the digital projection system is controlled by a camera.
  • An iterative shift should be understood as an incremental shift of the calculation point on the projection of the digital projection system to coincide with the control point, which is performed using a processing unit that contains a stored set of instructions and should be obvious to those skilled in the art from the description, as follows.
  • the first iteration determines the numerical value of offset between the positions of the calculation point and the control point in the camera frame and stores this value.
  • the coordinates of the calculation point are generated using the law of random numbers or similar law.
  • the numerical value of the offset between the positions of the calculation point and the control point in the camera frame is determined and compared with the stored value.
  • the processing unit determines that the position of the calculation point with the generated coordinates in the second iteration has shifted in the wrong direction since the obtained value indicates that the calculation point with the generated coordinates in the second iteration is positioned at a greater distance from the control point than in the previous iteration - the obtained offset value is ignored. If the obtained offset value is less than the stored value, the processing unit determines that the location of the calculation point with the generated coordinates in the second iteration has shifted in the correct direction in relation to the control point.
  • the generated coordinates of the calculation point on the third iteration will become the basis for determining the offset of the calculation point relative to the control point, which is compared with the offset of the calculation point relative to the control point determined on the second iteration.
  • the iteration steps that reduce the offset between the calculation point and the control point are repeated until the calculation point and the control point coincide.
  • the coordinates of its last position are compared with the initially calculated coordinates. The difference between these coordinates represents the numerical value of the distortion that needs to be applied to the projection of the digital projection system at a given calculation point.
  • FIG. 2 which illustrates the first embodiment of the claimed solution, shows an image of a fragment of the surface of the dome in a camera frame, with control and calculation points obtained by an electronic computer.
  • FIG. 2 is provided to understand the essence of the claimed method: in particular, during the iterative shift of the calculation point, the control point will not be visible in the frame.
  • the control point marked by a spot from the intersection of the laser beam with the surface of the dome, is fixed in the frame only when the control point in the camera frame is determined initially. But it is shown in FIG. 2 to facilitate understanding of the essence of the claimed method.
  • the projection image of a digital projection system is a virtual plane, mainly in the form of a circle on which a certain texture is applied.
  • a texture is any graphic image, static or dynamic, that is reproduced on a virtual surface.
  • FIG. 3b shows the projection plane of a digital projection system with superimposed calculation points. At these points, the calculated coordinates in the coordinate system of the digital projection system cause corrective distortions in the projection of the digital projection system.
  • FIG. 3c shows the projection plane of a digital projection system with distortions.
  • the position of any celestial body (projected by the optical projection system) can now be calculated in the projection coordinates of the digital projection system. This allows the projection of a digital projection system to form and display graphic content relative to any celestial body projected by the optical projection system, and to synchronize its movement with the motion of the starry sky formed by the optical projection system.
  • the control point is determined on the image of the fragment of the surface of the dome with defined coordinates received from the camera and covered by the camera field of view, and the coordinates of the control point are determined in the projection coordinate system of the digital projection system, in particular, the zero latitude/longitude point of the coordinate system of the optical projection system for its corresponding orientation is determined using Gray code, where the method includes the steps of:
  • the digital projection system reproduces a sequence of graphic patterns on the surface of the dome of the demonstration hall
  • the claimed method has a second variant, which is described in detail below.
  • the technical solution provides that the calculation point can be located at the equator of the projection of the digital projection system and completely or partially outside the surface of the dome.
  • the coordinates of the desired calculation point are determined by extrapolating the coordinates of adjacent calculation points. When the position of at least one adjacent calculation point changes, the coordinates of the desired calculation point are recalculated.
  • the advantage of this variant is the reduction in the number of zones for which no distortion can be applied to compensate for the deviation between the projections of optical and digital projection systems.
  • the technical solution provides that the calculation point can be located in the blind area of the projection of the digital projection system.
  • the coordinates of the desired calculation point are determined by interpolating the coordinates of adjacent calculation points.
  • the coordinates of the desired calculation point are recalculated. Further calibration is perfonned according to the calculated coordinates of the desired calculation point. This reduces the number of blind areas for which it is impossible to calibrate the optical-digital projection system.
  • the calculation point can also be located within the surface of the dome on the projection of the digital projection system and not within the blind area of the projection.
  • the position of the calculation point on the projection of the digital projection system is monitored by a camera.
  • the points of the projection of the digital projection system are sequentially shifted, whereby the necessary corrective distortions are introduced into the projection of the digital projection system.
  • a control point in the camera frame is determined once. After that, the coordinates of the control point in the camera frame are known and their further determination is unnecessary.
  • the coordinates of the calculation point are then determined based on the orientation of the optical projection system for the corresponding position of the optical projection system in the projection coordinate system of the digital projection system. It should be noted that all the positions of the optical projection system during the calibration process are predetermined and can be defined, for example, by the angles of rotation of the optical projection system.
  • one fragment of the surface of the dome with defined coordinates of a plurality of fragments of the surface of the dome is captured by the camera to form an image of the fragment of the surface of the dome covered by the camera's field of view.
  • the preset parameters for the orientation of the optical projection system make it possible to determine the approximate area on the surface of the dome, to which the point with zero latitude/longitude coordinates in the coordinate system of the optical projection system is shifted, and this area is covered by the camera field of view, and the calculation point is located in this area and corresponds to a shifted point with zero latitude/longitude coordinates in the coordinate system of the optical projection system, but in the coordinate system of the digital projection system.
  • the calculation point is displayed on the surface of the dome by means of a digital projection system.
  • the coordinates of the desired calculation point are determined by extrapolating the coordinates of adjacent calculation points. Then, when the position of at least one adjacent calculation point changes, the coordinates of the desired calculation point are recalculated.
  • the problem to be solved at this stage is that, under the given conditions, the coordinates of the calculation point are technically difficult or impossible to determine from the image captured by the camera.
  • the existence of an offset of the calculation point relative to the control point in the projection coordinate system of the digital projection system for the corresponding orientation of the optical projection system is determined using the camera, and the numerical value of the offset is determined.
  • the value of distortion in the projection of the digital projection system can be determined at most calculation points.
  • the distortion to the projection of the digital projection system at this calculation point by the value of the offset is implemented.
  • the offset of the calculation point relative to the control point in the projection coordinate system of the digital projection system is determined by performing the following.
  • the combination of the projection coordinate systems of the optical and digital projection systems occurs by determining the value of the deviation of the coordinates of a previously known point in the camera frame and calculation points (determined for the corresponding orientations of the optical projection system) and introducing corrective distortions into the projection of the digital projection system at these calculation points.
  • the optical projection system the orientation of which is controlled along all coordinate axes, is sequentially rotated to certain positions.
  • the coordinates of the corresponding calculation points are predetermined. That is, the coordinates of the calculation points correspond to the coordinates of a point of zero latitude/longitude of the coordinate system of the optical projection system at its corresponding orientations, but in the projection coordinate system of the digital projection system.
  • the calculation point is reproduced by the digital projection system in the projection coordinate system of the digital projection system with coordinates matching the coordinates of the control point (for the optical projection system) considering the rotation of the optical projection system.
  • the fragment of the surface of the dome covered by the camera's field of view is captured by the camera, and this image is transferred to an electronic computer.
  • the laser source and the camera rotate synchronously with the optical projection system.
  • the position of the control point within the camera frame remains in a certain place, regardless of the orientation of the optical projection system.
  • the point in the camera frame occupies a certain area defined by a set of pixels of the digital image.
  • the coordinates of the calculation point associated with the positioning of the optical projection system are determined on the image obtained from the camera of the fragment of the surface of the dome with defined coordinates.
  • the processing unit iteratively shifts the corresponding calculation point on the projection of the digital projection system until it coincides with the control point in the camera frame at the corresponding position of the optical projection system as described above.
  • the value of an offset of the calculation point relative to the control point in the projection coordinate system of the digital projection system is determined by performing the following.
  • the digital projection system displays a sequence of graphic patterns (the Structured Light Patterns), and the camera mounted on the case of the optical projection system captures that fragment of each pattern that falls into its field of view.
  • the array of the coordinates is obtained, which is interpolated to determine the actual coordinates of the zero latitude/longitude point of the coordinate system of the optical projection system (for a specific orientation of the optical projection system) in the coordinate space of the equidistant or other projection of the digital projection system.
  • a corresponding calculation point on the projections of the digital projection system is known, as shown in FIG. 3a.
  • FIG. 3a shows the calculation points of the projection of the digital projection system, which form a conditional grid of positions of the calculation points and are used to distort the projection of the digital projection system.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Physics & Mathematics (AREA)
  • Geometry (AREA)
  • Projection Apparatus (AREA)

Abstract

A claimed method of calibration of the optical-digital projection system relates to the means of setting up projection systems or projection-type devices. Calibration involves combining the projections of optical and digital projection systems by matching their projection coordinate systems via the introduction of distortions in the digital projection at certain points. The first variant of the method of the claimed technical solution involves the calibration of the optical - digital projection system based on the points displayed by the digital projection system on the surface of the dome and covered by the field of view of the camera that is fixed on the optical projection system. The second variant of the method involves calibrating the optical-digital projection system when some points, used to determine the amount of distortion, are displayed by the digital projection system in the blind zone or on the equator of the projection and partially or entirely outside the surface of the dome. The current invention achieves the technical result of shortening the calibration time of the optical- digital projection system in comparison with its closest analog. The additional technical result is an increase in alignment precision of projections of optical and digital projection systems. A further advantage of the claimed technical solution is that it increases the accuracy of calibration by expanding the number of points that can be used for calibration.

Description

METHOD FOR CALIBRATION OF AN OPTICAL-DIGITAL
PROJECTION SYSTEM
FIELD OF INVENTION
A claimed method of calibration of the optical-digital projection system relates to the means of setting up projection systems or projection-type devices.
A proposed technical solution can be applied to calibrate and synchronize optical and digital projection systems of image or video playback that project images onto a single screen or surface.
BACKGROUND OF THE INVENTION
From the prior art known video projection system described in Japanese patent JP5339688 (B2) from 13.11.2013, according to which the video projection system, in order to project a precise video, makes the picture center of a camera, which is controllable for arbitrarily setting of its height and azimuth angle, in alignment with a cursor projected from a video projector, measures a positional relationship between the video projector and a fixed star, carries out an interpolation operation or the like with reference to these values to calculate a position of the projector and its posture angle, precisely operates together with a fixed star projector, and projects a precise video, with reference to these values to calculate a position and angle of rotation of the projector, to work in precise unison with the distribution projector.
The position of the camera is controlled so that the predetermined height and azimuth orientation on a dome are aligned with the picture center of the camera. The video projecting system makes a cursor projected from the video projector in alignment with the picture center of the video camera and measures the height and an azimuth angle of this point image and projector coordinates of the cursor at a plurality of points. A coordinate transformation is carried out on desired horizontal coordinates and a projected picture in accordance with measured data, set positions and position angles of the projector and the like are calculated; a coordinate transformation is carried out based on these values to display desired images and figures on desired horizontal coordinates.
The disadvantage of this system is the length of time required to calibrate the system. Moreover, the use of only one projector does not allow covering the entire dome screen and is limited.
THE PROBLEM WHICH THE INVENTION IS AIMED TO SOLVE
The claimed technical solution is based on the task of creating a method that will provide calibration and synchronization of multiple digital projectors and an optical projection system while providing corrections for the effect of distortion. In other words, the task is to combine the projection coordinate systems of the digital projection system and the optical projection system.
BRIEF DESCRIPTION OF THE INVENTION
The problem is solved by proposing a method of calibrating an optical- digital projection system, which includes the use of an optical projection system placed in a demonstration hall to project an image of the starry sky onto the surface of the dome of the demonstration hall. According to the technical solution, the optical-digital projection system additionally incorporates a digital projection system, which contains multiple digital projectors for playing back an additional image by reproducing a projection on the corresponding fragment of the surface of the demonstration hall dome, and an electronic computer and the optical projection system is connected to an electronic computer, and the optical projection system has a case and contains a camera that is connected to an electronic computer and transmits an image of the fragment of the surface of the dome, which has certain coordinates, and the electronic computer contains a device for visual data output, a device for data input, and a processing unit, moreover, the processing unit contains a stored set of instructions.
The method includes the following steps:
- determining the control point by producing the zero latitude/longitude point of the coordinate system of the optical projection system on the surface of the dome using a laser beam projected onto the surface of the dome of the demonstration hall,
- pre-calibrating the camera until the zero latitude/longitude point of the coordinate system of the optical projection system is determined.
- determining a control point in the camera frame once.
- determining the coordinates of the calculation point, based on the orientation of the optical projection system for the corresponding position of the optical projection system in the projection coordinate system of the digital projection system,
- capturing with the camera one fragment of the surface of the dome with defined coordinates of a plurality of fragments of the surface of the dome, to form an image of the fragment of the surface of the dome covered by the camera's field of view,
- determining whether an offset exists between the calculation point and the control point based on the orientation of the optical projection system for the corresponding position of the optical projection system in the projection coordinate system of the digital projection system, and determining the numerical value of the offset,
- implementing distortion to the projection of the digital projection system at a given calculation point by the value of the offset.
Additionally, the calculation point on the surface of the dome is reproduced using a digital projection system, by
- determining the coordinates of the calculation point associated with the positioning of the optical projection system on the image of the fragment of the surface of the dome with defined coordinates received from the camera and covered by the camera field of view,
- shifting the calculation point on the projection of the digital projection system iteratively until it coincides with the control point in each corresponding position of the optical projection system.
Furthermore, alternatively, to determine the control point, after each rotation of the optical projection system, the digital projection system reproduces a sequence of graphic patterns on the surface of the dome of the demonstration hall, while
- capturing on the camera these patterns for further analysis using the Gray code,
- determining the actual coordinates of the control point in the projection coordinate system of the digital projection system in each position of the optical projection system,
- shifting the calculation point until it coincides with the control point.
The problem is solved by a variant of the claimed invention, according to which the method of calibration of the optical-digital projection system, which includes the use of an optical projection system placed in the demonstration hall to project an image of the starry sky onto the surface of the dome of the demonstration hall, is determined. According to this version of the claimed solution, the optical-digital projection system additionally incorporates a digital projection system, which contains multiple digital projectors for playing back an additional image by reproducing a projection on the corresponding fragment of the surface of the demonstration hall dome, and an electronic computer and the optical projection system is connected to an electronic computer, and the optical projection system has a case and contains a camera that is connected to an electronic computer and transmits an image of the fragment of the surface of the dome, which has certain coordinates, and the electronic computer contains a device for visual data output, a device for data input, and a processing unit, moreover, the processing unit contains a stored set of instructions.
The variant of the method includes the following steps:
- determining the control point by producing the zero latitude/longitude point of the coordinate system of the optical projection system on the surface of the dome using a laser beam projected on the surface of the dome of the demonstration hall,
- pre-calibrating the camera until the zero latitude/longitude point of the coordinate system of the optical projection system is determined.
- determining a control point in the camera frame once.
- determining the coordinates of the calculation point, based on the orientation of the optical projection system for the corresponding position of the optical projection system in the projection coordinate system of the digital projection system,
- capturing with the camera one fragment of the surface of the dome with defined coordinates of a plurality of fragments of the surface of the dome, to form an image of the fragment of the surface of the dome covered by the camera's field of view,
- if the calculation point is located at the projection equator of the digital projection system and partially or completely outside the dome surface, determining the coordinates of the desired calculation point by extrapolating the coordinates of adjacent calculation points, then recalculating the coordinates of the desired calculation point when the position of at least one adjacent calculation point has changed,
- if the calculation point is located in the blind area of the projection of the digital projection system, determining the coordinates of the desired calculation point by interpolating the coordinates of adjacent calculation points, then recalculating the coordinates of the desired calculation point when the position of at least one adjacent calculation point has changed,
- if the calculation point is located within the surface of the dome on the projection of the digital projection system and not in the blind area of the projection of the digital projection system, determining using the camera whether an offset exists between the calculation point and the control point based on the orientation of the optical projection system for the corresponding position of the optical projection system in the projection coordinate system of the digital projection system, and determining the numerical value of the offset,
- implementing distortion to the projection of the digital projection system at a given calculation point by the value of the offset.
Additionally, a variant of the method implies that if the calculation point is located within the surface of the dome on the projection of the digital projection system and not in the blind area of the projection of the digital projection system,
- reproducing the calculation point on the surface of the dome using a digital projection system,
- determining the coordinates of the calculation point associated with the positioning of the optical projection system on the image of the fragment of the surface of the dome with defined coordinates received from the camera and covered by the camera field of view,
- shifting the calculation point on the projection of the digital projection system iteratively until it coincides with the control point in each corresponding position of the optical projection system.
Furthermore, according to the current invention, the variant of the method claims that, if the calculation point is located within the dome surface on the projection of the digital projection system and not in the blind area of the projection of the digital projection system, then to determine the control point, after each rotation of the optical projection system, the digital projection system reproduces a sequence of graphic patterns on the surface of the dome of the demonstration hall, while
- capturing on the camera these patterns for further analysis using the Gray code,
- determining the actual coordinates of the control point in the projection coordinate system of the digital projection system in each position of the optical projection system,
- shifting the calculation point until it coincides with the control point.
TECHNICAL RESULT
The current invention achieves the technical result of shortening the calibration time of the optical-digital projection system compared to its nearest analog counterpart. The additional technical result is a reduction in distortion and an increase in alignment accuracy when projecting the image. Another advantage of the claimed technical solution is an increase in the accuracy of calibration, which is achieved by increasing the number of recognized control points.
According to the claimed invention, the method allows calibrating the digital projection system more accurately on the dome screen, which has blind spots not covered by the projection of the digital projection system.
Herewith, blind areas can be expressed as follows:
- technical holes on the surface of the dome, where the projection of the digital projection system cannot be reproduced;
- shadows cast on the dome surface by various objects, that is, objects obstructing the beams of digital projectors that prevent part of the projection from reaching the dome's surface, therefore casting shadows.
It is a further advantage of the claimed invention that the method allows calibrating the digital projection system more accurately on the dome screen when the projection of the digital projection system extends beyond the dome surface. As a result, an obstacle is overcome in which the calculation points at the equator of the projection of the digital projection system are not visible partially or completely since they are projected outside the dome surface.
The calibration of the projection system would either be incomplete or incorrect without the newly introduced features of the claimed method.
BRIEF DESCRIPTION OF THE DRAWINGS
The essence of the proposed method is illustrated in the accompanying drawings, which show schematically how the components of the system are arranged relative to the dome. It is obvious to those skilled in the art from the description of the current invention that the components of the system may have a different arrangement. Hence, the embodiment shown in the drawing does not limit the implementation of the proposed method to the given mutual arrangement of the components. Other options for the mutual arrangement of the components are admissible if they allow the set task to be completed and offer the desired technical outcome.
FIG. 1 illustrates the arrangement of the components of the optical projection system relative to the dome.
FIG. 2 illustrates an image of the fragment of the surface of the dome covered by the camera field of view obtained from the camera and transmitted to the electronic computer.
FIG. 3 illustrates the changes in the coordinate grid of the projection of the digital projection system, where
FIG. 3a illustrates the formed conditional grid of positions of calculation points before combining optical and digital projection systems,
Fig. 3b illustrates the coordinate grid of the projection of the digital projection system with superimposed calculation points, in which the distortion of the projection of the digital projection system is implemented,
FIG. 3c illustrates the projection of a digital projection system with implemented distortions and superimposed calculation points.
In Fig. 1, 1 is a fragment of the surface of the dome, 2 is a field of view of the camera, 3 is a camera, 4 is an optical projection system, and 5 is the surface of the dome (dome screen) in the demonstration hall.
In Fig. 2, 6 is an image of a fragment of the surface of the dome, 7 is the location where the calculation point is determined, and 8 is the location where the control point is established.
DETAILED DESCRIPTION OF THE INVENTION IN REGARD TO THE PREFERRED EMBODIMENT
Hereinafter, a detailed description of the claimed invention in its preferred embodiment will be given with reference to the accompanying drawings.
The definitions of the following terms are intended to simplify the understanding of the essence of the claimed invention:
"Fragment of the surface of the dome" is an area of the surface of the dome covered by the camera's field of view every time the optical projection system changes orientation.
"Plurality of fragments of the surface of the dome" refers to the entirety of all fragments of the surface of the dome, overlapping or otherwise, that can be covered by the camera field of view in all possible orientation positions of the optical projection system.
"Offset" is a numerical value of the distance between the calculated point and the control point, which is determined based on the image received from the camera.
"Shift" is a numerical value that indicates the distance the current position of the calculated point has shifted in comparison with its position in the previous iteration.
In the following description, it is considered that the first variant of the method of the claimed technical solution involves determining the calibration of the optical-digital projection system based on points displayed on the dome surface and visible in the fragment of the surface of the dome covered by the camera field of view. The second variant of the method is aimed at calibrating the optical-digital projection system, assuming that the calculation point cannot be determined directly because it is located in the blind area of the projection of the digital projection system or at the equator of the projection of the digital projection system and partially or completely outside the dome surface.
The optical-digital projection system can be used to display the starry sky on the dome of the planetarium. However, it is obvious to those skilled in the art from the description of the claimed technical solution that the reproduction of the image can be carried out on any static surface of the demonstration hall. Hereinafter, the demonstration hall means some limited area containing the surface on which the image can be reproduced. The surface on which the image is reproduced may be a screen or have a different definition. To simplify the understanding of the essence of the claimed technical solution, it is assumed that the demonstration hall should be understood as a planetarium, and the surface on which the image is reproduced is the dome.
Optical projection can be accomplished in a planetarium using a star projector, which reproduces an imitation of the starry sky on the dome's surface. This type of system is mainly located in the center of the demonstration hall. The optical projection system projects the image of the starry sky onto the surface of the planetarium dome, where the specified image is taken as a reference.
The optical projection system is connected to the electronic computer through communication channels to transmit the image of the fragment of the surface of the dome with defined coordinates and covered by the camera's field of view. Preference is given to wired communication channels, which have higher bandwidth.
The optical projection system has a case and comprises a camera that is configured so its field of view covers one fragment of the surface of the dome with defined coordinates of a plurality of fragments of the surface of the dome, to form an image of the fragment of the surface of the dome covered by the camera field of view and transmit it to an electronic computer. The camera is preferably mounted on the equator of the optical projection system and can be placed both outside the case and inside the case. In addition, the optical projection system preferably includes a light source that emits a laser beam on a specific fragment of the surface of the dome. In particular, said light source may be directed at a point to which coordinates origin (the zero latitude/longitude point) are assigned in the coordinate system of the optical projection system.
The electronic computer always contains information about the coordinates of the zero latitude/longitude point of the coordinate system of the optical projection system, regardless of the position of the optical projection system. This is provided by the projection of the laser beam on the surface of the dome of the showroom, where the projection of the laser beam indicates the zero latitude/longitude point of the coordinate system of the optical projection system. Thus, when the optical projection system is rotated to a certain angle relative to the position of the origin of the coordinate system (zero latitude/longitude), the field of view of the camera can cover a corresponding fragment of the surface of the dome with predetermined coordinates.
The digital projection system includes a plurality of digital projectors that are pre-calibrated to reproduce the additional image on the surface of the dome of the showroom. Thus, allowing a complete image to be reproduced by the entire plurality of digital projectors. In this case, each digital projector from a plurality of digital projectors reproduces the projection on the corresponding fragment of the surface of the dome of the showroom. Moreover, the digital projection system receives data for reproduction from an electronic computer.
However, the projection coordinate system of the digital projection system needs to be adjusted to match the coordinate system of the optical projection system. The absence of such calibration will be followed by the dealignment of the images of the optical projection system and the digital projection system. For example, images of the starry sky are displayed on the surface of the dome using an optical projection system, and with the help of a digital projection system, connecting lines are drawn between the "stars" to highlight the constellation in the starry sky.
When the images projected by optical and digital projection systems are aligned using calibration, the connecting lines will be drawn from star to star. Without such alignment, the lines projected by the digital projector will be drawn with an offset relative to the depicted objects, in this case, "stars".
The electronic computer is a software-controlled device for information processing, comprising a device for visual data output, a device for data input, and a processing unit. Such an electronic computer can be a desktop personal computer, laptop, etc., that supports network communication.
In a general embodiment of the method, the electronic computer is configured to issue commands to the optical projection system to perform its rotation at a given angle in a given direction along a given axis. In most implementations, 36 control points are used to determine the angles of rotation. A device for visual data output is understood to be a high-resolution electronic device, such as a monitor, or a screen of a personal electronic computing device such as a laptop, tablet, etc.
A device for data input is understood to be: 1) any manual input devices, such as a keyboard, mouse, touch screen, touchpad, etc. 2) information inputs for receiving signals from external sources, such as a camera and a digital projector.
A processing unit is understood to be an arithmetic and logic device designed to control an electronic computer. The processing unit contains a stored set of instructions, the execution of which ensures the operation of the proposed method, including calibration and playback control of optical and digital projection systems.
According to the proposed method, the control point must first be determined. To do this, a source of laser emission is fixed firmly on the optical projection system. Preferably, the control point should be determined only once in the frame of the camera. Therefore, the control point must be located a priori in the camera frame.
FIG. 1 shows an installed optical projection system in a showroom, preferably on a fixed base. A plurality of digital projectors is installed and calibrated together to produce a single image reproduced by said plurality of digital projection devices. The optical projection system is oriented at a predetermined position and predetermined coordinates associated with the rotation angle of the optical projection system. The camera is calibrated until the zero latitude/longitude point of the coordinate system of the optical projection system is determined. Preferably, a laser beam projected onto the surface of the dome of the demonstration hall is used to determine the location of the zero latitude/longitude point of the coordinate system of the optical projection system. In the camera frame, the location of the zero latitude/longitude point of the coordinate system of the optical projection system is determined by assessing the spot formed by the laser beam on the surface of the dome. By doing so, a primary reference point for further calibration can be set and a reference to a specific point can be created. It should be noted that the point in the camera frame occupies a certain area defined by a set of pixels of the digital image. The mathematical center of this area corresponds to the coordinates of the control point. Then, based on the orientation of the optical projection system for the corresponding position of the optical projection system, the coordinates of the calculation point in the projection coordinate system of the digital projection system are determined. Next, the optical projection system is alternately oriented to the remaining predetermined positions and predetermined coordinates, and during rotation stop in each such position, the camera captures one fragment of the surface of the dome with defined coordinates of a plurality of fragments of the surface of the dome, to form an image of the fragment of the surface of the dome covered by the camera field of view. Then the existence of an offset of the calculation point relative to the control point is determined. This stage is completed when the offsets for all calculation points are determined. The number of calculation points must be equal to the number of predetermined positions of the optical projection system in which the determination of the specified offset is performed. After that, the calculation points are superposed with the control points, thereby distorting the projection of the digital projection system. It should be noted that the "determination of the existence of an offset" should be understood as the initiation of the algorithm for determining the numerical value, namely the numerical value of an offset. In the case of a zero offset value, the value of the distortions implemented into the projection at a given calculation point is zero. In the case of a non-zero offset numerical value, this numerical value is crucial for determining the distortion implemented into the projection at a given calculation point. Offset determination and alignment can be performed by two alternative embodiments, which are refinements of such an offset detennination. According to the first alternative embodiment, the combination of the projection coordinate systems of the optical and digital projection systems occurs due to the implementation of corrective distortions in the projection of the digital projection system at the calculation points.
The optical projection system, the orientation of which is controlled along all coordinate axes, is sequentially rotated to certain positions. For these positions of the optical projection system (based on orientation data), the coordinates of the corresponding calculation points are predetermined. Thus, the coordinates of the points that correspond to the coordinates of the zero latitude/longitude points of the coordinate system of the optical projection system at the corresponding positions of the optical projection system are determined, and they can now be detennined in the projection coordinate system of the digital projection system.
During the rotation stop of the optical projection system in each such position, the calculation point is reproduced by the digital projection system in the coordinate system of the projection of the digital projection system with coordinates matching the coordinates of the control point (for the optical projection system), taking into account the rotation of the optical projection system.
The fragment of the surface of the dome covered by the camera's field of view is captured by the camera, and this image is transferred to an electronic computer. It should be noted that the laser source and the camera rotate synchronously with the optical projection system. As a result, the position of the control point within the camera frame remains in a certain place, regardless of the orientation of the optical projection system. It should also be noted that the point in the camera frame occupies a certain area defined by a set of pixels of the digital image. The coordinates of the calculation point associated with the positioning of the optical projection system are determined on the image obtained from the camera of the fragment of the surface of the dome with defined coordinates.
Next, using the stored set of instructions, the processing unit iteratively shifts the corresponding calculation point on the projection of the digital projection system until it coincides with a previously known (control) point in the camera frame at a specific position of the optical projection system. The position of the calculation point on the projection of the digital projection system is controlled by a camera.
An iterative shift should be understood as an incremental shift of the calculation point on the projection of the digital projection system to coincide with the control point, which is performed using a processing unit that contains a stored set of instructions and should be obvious to those skilled in the art from the description, as follows. The first iteration determines the numerical value of offset between the positions of the calculation point and the control point in the camera frame and stores this value. In the second iteration, the coordinates of the calculation point are generated using the law of random numbers or similar law. The numerical value of the offset between the positions of the calculation point and the control point in the camera frame is determined and compared with the stored value. If the obtained offset value is greater than the stored value, the processing unit determines that the position of the calculation point with the generated coordinates in the second iteration has shifted in the wrong direction since the obtained value indicates that the calculation point with the generated coordinates in the second iteration is positioned at a greater distance from the control point than in the previous iteration - the obtained offset value is ignored. If the obtained offset value is less than the stored value, the processing unit determines that the location of the calculation point with the generated coordinates in the second iteration has shifted in the correct direction in relation to the control point. In this case, the generated coordinates of the calculation point on the third iteration will become the basis for determining the offset of the calculation point relative to the control point, which is compared with the offset of the calculation point relative to the control point determined on the second iteration. The iteration steps that reduce the offset between the calculation point and the control point are repeated until the calculation point and the control point coincide. When the calculation point coincides with the control point, the coordinates of its last position are compared with the initially calculated coordinates. The difference between these coordinates represents the numerical value of the distortion that needs to be applied to the projection of the digital projection system at a given calculation point.
FIG. 2, which illustrates the first embodiment of the claimed solution, shows an image of a fragment of the surface of the dome in a camera frame, with control and calculation points obtained by an electronic computer. It should be noted that FIG. 2 is provided to understand the essence of the claimed method: in particular, during the iterative shift of the calculation point, the control point will not be visible in the frame. The control point, marked by a spot from the intersection of the laser beam with the surface of the dome, is fixed in the frame only when the control point in the camera frame is determined initially. But it is shown in FIG. 2 to facilitate understanding of the essence of the claimed method.
Thus, the points of the projection of the digital projection system are consistently shifted, whereby the necessary corrective distortions are implemented in the projection of the digital projection system.
It is worth noting that the projection image of a digital projection system is a virtual plane, mainly in the form of a circle on which a certain texture is applied. Essentially, a texture is any graphic image, static or dynamic, that is reproduced on a virtual surface.
FIG. 3b shows the projection plane of a digital projection system with superimposed calculation points. At these points, the calculated coordinates in the coordinate system of the digital projection system cause corrective distortions in the projection of the digital projection system.
The projection of the digital projection system is deformed (as shown in Fig. 3c) and its calculation points now coincide with the control points (in the orientation of the optical projection system). Thus, the calculation points on the projection of the digital projection system are shifted by the value of the deviation of the calculated coordinates from the actual ones. In other words, FIG. 3c shows the projection plane of a digital projection system with distortions.
By knowing the deviation of the calculated coordinates from the actual ones, and pre-distorting the projection of the digital projection system, the position of any celestial body (projected by the optical projection system) can now be calculated in the projection coordinates of the digital projection system. This allows the projection of a digital projection system to form and display graphic content relative to any celestial body projected by the optical projection system, and to synchronize its movement with the motion of the starry sky formed by the optical projection system.
According to the second embodiment of the implementation of the proposed method, the control point is determined on the image of the fragment of the surface of the dome with defined coordinates received from the camera and covered by the camera field of view, and the coordinates of the control point are determined in the projection coordinate system of the digital projection system, in particular, the zero latitude/longitude point of the coordinate system of the optical projection system for its corresponding orientation is determined using Gray code, where the method includes the steps of:
- determining the coordinates of the control point in the coordinate system of the camera frame by pre-calibrating the camera once until the point of zero latitude/longitude of the coordinate system of the optical projection system is determined, in particular until the fixation in the camera frame of the point of zero latitude/longitude of the coordinate system of the optical projection system,
- during each rotation stop in one of the predetermined positions of the optical projection system, the digital projection system reproduces a sequence of graphic patterns on the surface of the dome of the demonstration hall,
- capturing on the camera these patterns for further analysis using the Gray code,
- determining the actual coordinates of the control point in the projection coordinate system of the digital projection system in each position of the optical projection system,
- shifting the calculation point until it coincides with the control point, in other words, shifting the calculation point to the coordinates of the control point.
The claimed method has a second variant, which is described in detail below. The technical solution provides that the calculation point can be located at the equator of the projection of the digital projection system and completely or partially outside the surface of the dome. To perform calibration, the coordinates of the desired calculation point are determined by extrapolating the coordinates of adjacent calculation points. When the position of at least one adjacent calculation point changes, the coordinates of the desired calculation point are recalculated. The advantage of this variant is the reduction in the number of zones for which no distortion can be applied to compensate for the deviation between the projections of optical and digital projection systems.
According to the second variant, the technical solution provides that the calculation point can be located in the blind area of the projection of the digital projection system. To perform calibration, the coordinates of the desired calculation point are determined by interpolating the coordinates of adjacent calculation points. When the position of at least one adjacent calculation point changes, the coordinates of the desired calculation point are recalculated. Further calibration is perfonned according to the calculated coordinates of the desired calculation point. This reduces the number of blind areas for which it is impossible to calibrate the optical-digital projection system.
According to this variant of the claimed method, the calculation point can also be located within the surface of the dome on the projection of the digital projection system and not within the blind area of the projection.
The position of the calculation point on the projection of the digital projection system is monitored by a camera.
As a result of the method, the points of the projection of the digital projection system are sequentially shifted, whereby the necessary corrective distortions are introduced into the projection of the digital projection system.
According to this variant of the claimed method, first, a control point in the camera frame is determined once. After that, the coordinates of the control point in the camera frame are known and their further determination is unnecessary.
The coordinates of the calculation point are then determined based on the orientation of the optical projection system for the corresponding position of the optical projection system in the projection coordinate system of the digital projection system. It should be noted that all the positions of the optical projection system during the calibration process are predetermined and can be defined, for example, by the angles of rotation of the optical projection system.
Then, one fragment of the surface of the dome with defined coordinates of a plurality of fragments of the surface of the dome is captured by the camera to form an image of the fragment of the surface of the dome covered by the camera's field of view. The preset parameters for the orientation of the optical projection system make it possible to determine the approximate area on the surface of the dome, to which the point with zero latitude/longitude coordinates in the coordinate system of the optical projection system is shifted, and this area is covered by the camera field of view, and the calculation point is located in this area and corresponds to a shifted point with zero latitude/longitude coordinates in the coordinate system of the optical projection system, but in the coordinate system of the digital projection system. The calculation point is displayed on the surface of the dome by means of a digital projection system.
In the event that the calculation point is located at the equator of the projection of the digital projection system and completely or partially outside the surface of the dome, the coordinates of the desired calculation point are determined by extrapolating the coordinates of adjacent calculation points. Then, when the position of at least one adjacent calculation point changes, the coordinates of the desired calculation point are recalculated. The problem to be solved at this stage is that, under the given conditions, the coordinates of the calculation point are technically difficult or impossible to determine from the image captured by the camera.
In the event that the calculation point is located in the blind area of the projection of the digital projection system, it is impossible to determine its coordinates directly. To accomplish this, an indirect approach is used, which determines the coordinates of the desired calculation point by interpolating the coordinates of adjacent calculation points. The result is an implied value, which is recalculated when the position of at least one adjacent calculation point changes.
In the event that the calculation point is located within the surface of the dome on the projection of the digital projection system and not within the blind area of the projection of the digital projection system, the existence of an offset of the calculation point relative to the control point in the projection coordinate system of the digital projection system for the corresponding orientation of the optical projection system is determined using the camera, and the numerical value of the offset is determined. Using this method, the value of distortion in the projection of the digital projection system can be determined at most calculation points.
After the value of the offset of the calculation point relative to the control point is determined, the distortion to the projection of the digital projection system at this calculation point by the value of the offset is implemented.
According to the first embodiment of this variant of the claimed method, the offset of the calculation point relative to the control point in the projection coordinate system of the digital projection system is determined by performing the following.
The combination of the projection coordinate systems of the optical and digital projection systems occurs by determining the value of the deviation of the coordinates of a previously known point in the camera frame and calculation points (determined for the corresponding orientations of the optical projection system) and introducing corrective distortions into the projection of the digital projection system at these calculation points.
The optical projection system, the orientation of which is controlled along all coordinate axes, is sequentially rotated to certain positions. For these positions of the optical projection system (based on orientation data), the coordinates of the corresponding calculation points are predetermined. That is, the coordinates of the calculation points correspond to the coordinates of a point of zero latitude/longitude of the coordinate system of the optical projection system at its corresponding orientations, but in the projection coordinate system of the digital projection system.
During the rotation stop of the optical projection system in each such position, the calculation point is reproduced by the digital projection system in the projection coordinate system of the digital projection system with coordinates matching the coordinates of the control point (for the optical projection system) considering the rotation of the optical projection system. The fragment of the surface of the dome covered by the camera's field of view is captured by the camera, and this image is transferred to an electronic computer. It should be noted that the laser source and the camera rotate synchronously with the optical projection system. As a result, the position of the control point within the camera frame remains in a certain place, regardless of the orientation of the optical projection system. It should also be noted that the point in the camera frame occupies a certain area defined by a set of pixels of the digital image.
The coordinates of the calculation point associated with the positioning of the optical projection system are determined on the image obtained from the camera of the fragment of the surface of the dome with defined coordinates. During the rotation stop of the optical projection system in each position, using the stored set of instructions, the processing unit iteratively shifts the corresponding calculation point on the projection of the digital projection system until it coincides with the control point in the camera frame at the corresponding position of the optical projection system as described above.
According to the second embodiment of this variant of the claimed method, if the calculation point is located within the surface of the dome on the projection of the digital projection system and not within the blind area of the projection of the digital projection system, the value of an offset of the calculation point relative to the control point in the projection coordinate system of the digital projection system is determined by performing the following.
During the rotation stop of the optical projection system in each predetermined position, the digital projection system displays a sequence of graphic patterns (the Structured Light Patterns), and the camera mounted on the case of the optical projection system captures that fragment of each pattern that falls into its field of view.
Based on the analysis of images with fragments of graphic patterns captured by the camera using the Structured Light Algorithm, in particular the Gray code, as described above, the array of the coordinates is obtained, which is interpolated to determine the actual coordinates of the zero latitude/longitude point of the coordinate system of the optical projection system (for a specific orientation of the optical projection system) in the coordinate space of the equidistant or other projection of the digital projection system. For each control point with determined coordinates, a corresponding calculation point on the projections of the digital projection system is known, as shown in FIG. 3a. In other words, FIG. 3a shows the calculation points of the projection of the digital projection system, which form a conditional grid of positions of the calculation points and are used to distort the projection of the digital projection system.
It is now possible to calculate the position of any celestial body (projected by an optical projection system) in the projection coordinates of a digital projection system. This allows the projection of a digital projection system to form and display graphic content relative to any celestial body projected by the optical projection system, and to synchronize its movement with the motion of the starry sky formed by the optical projection system.
Thus, the tasks set before the claimed technical solution are solved and Ċ the specified technical result is achieved.

Claims

1. A method of calibrating an optical-digital projection system, which includes the use of an optical projection system placed in a demonstration hall to project an image of the starry sky onto the surface of the dome of the demonstration hall, wherein the optical-digital projection system additionally incorporates a digital projection system, which contains multiple digital projectors for playing back an additional image by reproducing a projection on the corresponding fragment of the surface of the demonstration hall dome, and an electronic computer and the optical projection system is connected to an electronic computer, and the optical projection system has a case and contains a camera that is connected to an electronic computer and transmits an image of the fragment of the surface of the dome, which has certain coordinates, and the electronic computer contains a device for visual data output, a device for data input, and a processing unit, moreover, the processing unit contains a stored set of instructions, where the method includes the following steps:
- determining the control point by producing the zero latitude/longitude point of the coordinate system of the optical projection system on the surface of the dome using a laser beam projected onto the surface of the dome of the demonstration hall,
- pre-calibrating the camera until the zero latitude/longitude point of the coordinate system of the optical projection system is determined.
- determining a control point in the camera frame onCe.
- determining the coordinates of the calculation point, based on the orientation of the optical projection system for the corresponding position of the optical projection system in the projection coordinate system of the digital projection system,
- capturing with the camera one fragment of the surface of the dome with defined coordinates of a plurality of fragments of the surface of the dome, to form an image of the fragment of the surface of the dome covered by the camera's field of view,
- determining whether an offset exists between the calculation point and the control point based on the orientation of the optical projection system for the corresponding position of the optical projection system in the projection coordinate system of the digital projection system, and determining the numerical value of the offset,
- implementing distortion to the projection of the digital projection system at a given calculation point by the value of the offset.
2 The method according to claim 1, wherein the calculation point on the surface of the dome is reproduced using a digital projection system, by
- determining the coordinates of the calculation point associated with the positioning of the optical projection system on the image of the fragment of the surface of the dome with defined coordinates received from the camera and covered by the camera field of view,
- shifting the calculation point on the projection of the digital projection system iteratively until it coincides with the control point in each corresponding position of the optical projection system.
3. The method according to claim 1, wherein to determine the control point, after each rotation of the optical projection system, the digital projection system reproduces a sequence of graphic patterns on the surface of the dome of the demonstration hall, while
- capturing on the camera these patterns for further analysis using the Gray code,
- determining the actual coordinates of the control point in the projection coordinate system of the digital projection system in each position of the optical projection system,
- shifting the calculation point until it coincides with the control point.
4. A method of calibration of the optical-digital projection system, which includes the use of an optical projection system placed in the demonstration hall to project an image of the starry sky onto the surface of the dome of the demonstration hall, is determined, wherein the optical-digital projection system additionally incorporates a digital projection system, which contains multiple digital projectors for playing back an additional image by reproducing a projection on the corresponding fragment of the surface of the demonstration hall dome, and an electronic computer and the optical projection system is connected to an electronic computer, and the optical projection system has a case and contains a camera that is connected to an electronic computer and transmits an image of the fragment of the surface of the dome, which has certain coordinates, and the electronic computer contains a device for visual data output, a device for data input, and a processing unit, moreover, the processing unit contains a stored set of instructions, where the method includes the following steps:
- determining the control point by producing the zero latitude/longitude point of the coordinate system of the optical projection system on the surface of the dome using a laser beam projected on the surface of the dome of the demonstration hall,
- pre-calibrating the camera until the zero latitude/longitude point of the coordinate system of the optical projection system is determined.
- determining a control point in the camera frame once.
- determining the coordinates of the calculation point, based on the orientation of the optical projection system for the corresponding position of the optical projection system in the projection coordinate system of the digital projection system,
- capturing with the camera one fragment of the surface of the dome with defined coordinates of a plurality of fragments of the surface of the dome, to form an image of the fragment of the surface of the dome covered by the camera's field of view,
- if the calculation point is located at the projection equator of the digital projection system and partially or completely outside the dome surface, determining the coordinates of the desired calculation point by extrapolating the coordinates of adjacent calculation points, then recalculating the coordinates of the desired calculation point when the position of at least one adjacent calculation point has changed,
- if the calculation point is located in the blind area of the projection of the digital projection system, determining the coordinates of the desired calculation point by interpolating the coordinates of adjacent calculation points, then recalculating the coordinates of the desired calculation point when the position of at least one adjacent calculation point has changed, - if the calculation point is located within the surface of the dome on the projection of the digital projection system and not in the blind area of the projection of the digital projection system, determining using the camera whether an offset exists between the calculation point and the control point based on the orientation of the optical projection system for the corresponding position of the optical projection system in the projection coordinate system of the digital projection system, and determining the numerical value of the offset,
- implementing distortion to the projection of the digital projection system at a given calculation point by the value of the offset.
5. The method according to claim 4, wherein if the calculation point is located within the surface of the dome on the projection of the digital projection system and not in the blind area of the projection of the digital projection system,
- reproducing the calculation point on the surface of the dome using a digital projection system,
- determining the coordinates of the calculation point associated with the positioning of the optical projection system on the image of the fragment of the surface of the dome with defined coordinates received from the camera and covered by the camera field of view,
- shifting the calculation point on the projection of the digital projection system iteratively until it coincides with the control point in each corresponding position of the optical projection system.
6. The method according to claim 4, wherein if the calculation point is located within the dome surface on the projection of the digital projection system and not in the blind area of the projection of the digital projection system, then to determine the control point, after each rotation of the optical projection system, the digital projection system reproduces a sequence of graphic patterns on the surface of the dome of the demonstration hall, while
- capturing on the camera these patterns for further analysis using the Gray code,
- determining the actual coordinates of the control point in the projection coordinate system of the digital projection system in each position of the optical projection system,
- shifting the calculation point until it coincides with the control point.
PCT/UA2022/000037 2021-07-22 2022-07-19 Method for calibration of an optical-digital projection system WO2023003527A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
UAA202104281 2021-07-22
UAA202104281 2021-07-22

Publications (1)

Publication Number Publication Date
WO2023003527A1 true WO2023003527A1 (en) 2023-01-26

Family

ID=84978858

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/UA2022/000037 WO2023003527A1 (en) 2021-07-22 2022-07-19 Method for calibration of an optical-digital projection system

Country Status (1)

Country Link
WO (1) WO2023003527A1 (en)

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP5036226B2 (en) * 2006-06-13 2012-09-26 貴之 大平 Planetarium control method and planetarium control system
JP5068007B2 (en) * 2005-06-01 2012-11-07 有限会社大平技研 Video projection system
JP5224721B2 (en) * 2007-05-15 2013-07-03 有限会社大平技研 Video projection system
JP5339688B2 (en) 2007-05-07 2013-11-13 有限会社大平技研 Video projection system

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP5068007B2 (en) * 2005-06-01 2012-11-07 有限会社大平技研 Video projection system
JP5036226B2 (en) * 2006-06-13 2012-09-26 貴之 大平 Planetarium control method and planetarium control system
JP5339688B2 (en) 2007-05-07 2013-11-13 有限会社大平技研 Video projection system
JP5224721B2 (en) * 2007-05-15 2013-07-03 有限会社大平技研 Video projection system

Similar Documents

Publication Publication Date Title
US8730130B1 (en) System and method for automatically aligning immersive displays
JP6748961B2 (en) Projection image adjustment system and projection image adjustment method
JP4942752B2 (en) Method and system for aligning projector arrays
US6704000B2 (en) Method for remote computer operation via a wireless optical device
US7131732B2 (en) Projector apparatus, inclination angle obtaining method, and projection image correction method
US9058693B2 (en) Location correction of virtual objects
JP3509652B2 (en) Projector device
CN101572787B (en) Computer vision precision measurement based multi-projection visual automatic geometric correction and splicing method
JP3827662B2 (en) Projection display
CN112399158B (en) Projection image calibration method and device and projection equipment
US10506207B2 (en) Projection system, method for controlling projection system, and projector
JP2015201760A (en) Image projection system and image projection method
US8957964B2 (en) Large-object remote composite image annotation system
US10909748B2 (en) Image projection method and image projection device for three-dimensional object for projection
JP5068007B2 (en) Video projection system
WO2019155903A1 (en) Information processing device and method
WO2023003527A1 (en) Method for calibration of an optical-digital projection system
CN111357284A (en) Method for automatically restoring calibration state of projection system
JP2004140845A (en) Projector
CN116437061A (en) Demonstration image laser projection method, device, computer equipment and storage medium
UA137184U (en) METHOD OF CALIBRATION OF OPTICAL-DIGITAL PROJECTION SYSTEM
US20220187071A1 (en) System and method for controlling a light projector in a construction site
JPH06253241A (en) Projection distortion correction method for projection type display device
JP5159194B2 (en) Liquid crystal projector, control method and control program
JP2004282712A (en) Adjustment device

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22754588

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE