KR20140123418A - Augmented reality system using mirror - Google Patents
Augmented reality system using mirror Download PDFInfo
- Publication number
- KR20140123418A KR20140123418A KR1020140037340A KR20140037340A KR20140123418A KR 20140123418 A KR20140123418 A KR 20140123418A KR 1020140037340 A KR1020140037340 A KR 1020140037340A KR 20140037340 A KR20140037340 A KR 20140037340A KR 20140123418 A KR20140123418 A KR 20140123418A
- Authority
- KR
- South Korea
- Prior art keywords
- image
- mirror
- display
- user
- item
- Prior art date
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N5/00—Details of television systems
- H04N5/222—Studio circuitry; Studio devices; Studio equipment
- H04N5/262—Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
- H04N7/183—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast for receiving images from a single remote source
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Computer Graphics (AREA)
- Geometry (AREA)
- Software Systems (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Processing Or Creating Images (AREA)
Abstract
Description
The present invention relates to an improved augmented reality system using mirrors.
The present invention relates to an improvement of an augmented reality system using a mirror called a holoflector disclosed by Microsoft Corporation. An introductory video on the hollector is available at http://venturebeat.com/2012/02/28/microsoft-holoflector/ . As shown in Fig. 1,
The LCD display (FD) is placed behind the half mirror (HM) and the key (CM) is placed above the half mirror to recognize the gesture of the user (CS) in front of the half mirror (HM) And outputs the virtual object image VO to the LCD display FD in response to the virtual object image VO. Here, matching and outputting the virtual objects means that the user displays his or her appearance in the mirror and the virtual objects as if they exist in the same space. With this system, the user can feel as if his or her own image (RO) and virtual object (VO) in the mirror exist in the same space. For example, the virtual hole VO can be moved to the hand RO of the mirror
. Since the above-mentioned hollet reflector is installed on the upper edge of the semi-mirror (HM), there is a problem that it is difficult to photograph the user when the user approaches the half mirror. To measure the user's three-dimensional position with the kinematics, the gap between the kinem and the user should not be too small. For example, when configuring a digital signage with a single-pole reflector, the user can access the digital signage screen in close contact with the screen in order to view the details of the screen. In this case, There is a problem that it is difficult to do.
In order to solve the problem of the conventional holetirector described above, the present invention provides a configuration in which a key knot CM is installed at an interval IT from the display surface inside a display (FDT) as shown in Fig. It is preferable that the display (FDT) is a transparent display for capturing a user with a mounted keypad as shown in FIG. It is preferable to use a variable reflectance mirror (HMT) which can adjust the reflectance to an electric signal instead of the half mirror. These reflector mirrors are introduced at http://kentoptronics.com/solutions.html as a Switchable Mirror. With this configuration, the image is outputted to the transparent display (FDT) and the transparent state is rapidly reciprocated, and the variable reflectance mirror (HMT) is also driven to reciprocate the transparent state and the mirror state rapidly. It is desirable that the user picks up the kinetics when the transparent display and the variable reflectance mirror are in a transparent state.
In order to match a virtual object to a real object reflected on a mirror in a reflective augmented reality system such as a holetirector and output it to a display, the three-dimensional position of the user's point cloud is converted into a mirror plane, It is necessary to check the collision or contact between the surrounding virtual object and the point-symmetrically transformed point cloud to calculate the movement of the virtual object at the next moment, transform the position and shape of the virtual object according to the movement, and output it to the screen. That is, the kinetect must be able to detect not only the position of the user but also the position of the display surface. In the conventional system having the configuration shown in FIG. 1, the user manually inputs the three-dimensional relative position between the key knot and the display through trial and error several times. However, in the case of the configuration of FIG. 2 according to the present invention, It outputs a square mark of the prescribed shape and size to the position, records the mark with the key, and recognizes the mark size and the distorted shape by the image processing program to calculate the three-dimensional relative position between the key knot and the mark, It is possible to quickly and quickly find the relative position of the robot.
As described above, it is preferable that the process of calculating the relative position between the key and the display is performed only once, and the calculated value is written to the memory device, and then the recorded value is read when used next time.
Calculating the three-dimensional relative position between the camera and the mark from the distorted shape and size of the square mark on the image of a rectangular mark of known shape is a technique known as a solution to the perspective 4 point problem.
If the user is not in front of the screen, the user can change the direction of sight by driving the keyboard with a motor to capture the user. In this case, the relative position and direction between the Kinect and the display are changed. In this manner, the relative position and direction between the Kinect and the display are automatically measured in the above manner, and the reflective ARF system can be driven without interruption. That is, in the configuration of FIG. 2, there is no problem even if the photographing direction of the keynote is changed at any time according to the user's movement. However, such a configuration as shown in FIG. 1 has such a problem that the automatic measurement is impossible and the sight line of the keynote is fixed.
By concealing the fact that the Kinect is hidden inside the display, it is possible to hide the fact that the user is being photographed, thereby reducing the feeling of being watched, and can make the appearance of the system more beautiful and provide more novelty.
1 is a block diagram of an existing system;
Fig. 2 is a block diagram
FIG. 3 is a view showing an embodiment of the present invention
Fig. 4 is a view showing an embodiment of the present invention using a hologram optical element
Example 1
The mirror-based augmented reality system according to the present invention as shown in Fig. 2 is an improvement of an existing system (a holographic reflector disclosed by Microsoft Corporation) as shown in Fig. In Figure 1, the Kinect (CM) is mounted on the upper edge of the Half-Mirror (HM) so that when the user approaches very close to the half-mirror, the distance between the Kinect and the user is too narrow to measure the user's three- There is a problem that it can not. In order to solve this problem, the system of the present invention includes a transparent display (FDT) and a certain distance (IT) inside a reflectivity variable mirror (HMT) Configuration. The transparent display is driven so as to reciprocate rapidly in a state of outputting an image and a transparent state, and the reflectivity variable mirror is made to be in a transparent state at the moment of the transparent state of the display and is driven to be in a mirror state at the moment when the display outputs an image, The user takes a picture at a moment when the display and the reflectivity variable mirror are in a transparent state.
In the case of FIG. 1, there is a problem that the system becomes thick due to the gap (IT) between the key knot and the display. This problem can be solved by installing a mirror (RM) that changes the direction of the line of sight of the keynote between the key and the display as shown in Fig.
Fig. 4 shows a configuration in which the thickness of the system is further reduced by using a hologram optical element instead of the mirror of Fig. The hologram optical element has a reflection type or transmission type holographic film (HO) attached to a mirror or a plastic plate (GL) that totally reflects light incident on an optical element used in a near-eye display, And the like.
Instead of such a hologram optical element, a wedge light guide may be used. The introductory document on wedge light guides is entitled wedge optics in flat panel displays ( http://download.microsoft.com/download/4/B/4/4B49C1C2-4C7A-4CEA-ADB5-EF4E4E7F5F63/Wedge%20optics % 20in% 20flat% 20panel% 20displays.pdf ). The wedge type light guide has a structure in which light incident at one end is totally internally reflected and emitted to the other end.
Various depth cameras or stereo cameras may be used in place of the Kinect to detect the user's three-dimensional position.
CM Kinect
HM anti-mirror
FD LCD display
CS users
VO virtual object image
RO mirror user's appearance
FDT transparent display
HMT variable reflectance mirror
IT interval
RM mirror
GL plastic plate
HO holographic film
Claims (9)
Transparent display;
A reflectivity variable mirror installed in front of or behind the transparent display;
A video photographing means provided behind the display for photographing a user;
And an image processing means for analyzing the user image photographed by the image photographing means to match the virtual object image with the user image reflected on the mirror and outputting the registered user image to the transparent display
The transparent display rapidly reciprocates between a state of outputting an image and a transparent state
Wherein the image capturing means photographs a user at a moment when the transparent display is transparent.
The reflectivity variable mirror quickly reciprocates between a mirror state and a transparent state
Wherein the image capturing means photographs a user at a moment when the reflectivity variable mirror is transparent.
The image processing means
Display a mark of known size and shape on the display
The mark is photographed by the image photographing means
And recognizes the size and the distorted shape of the mark on the photographed image to calculate a three-dimensional relative position between the image photographing means and the display,
And outputs the resultant image to a mirror-based augmented reality system
Wherein the image capturing means includes an optical element for reducing the volume of the system by changing the viewing direction of the image capturing means.
Wherein the optical element is a mirror, a hologram optical element, or a wedge-shaped light guide.
The image capturing means includes a projection unit for projecting an infrared ray pattern
And a camera unit for photographing an image projected on the object by the infrared ray pattern.
Wherein the image capturing means is a stereo camera.
Wherein the image capturing means is a depth camera.
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
KR20130040705 | 2013-04-14 | ||
KR1020130040705 | 2013-04-14 |
Publications (1)
Publication Number | Publication Date |
---|---|
KR20140123418A true KR20140123418A (en) | 2014-10-22 |
Family
ID=51731545
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
KR1020140037340A KR20140123418A (en) | 2013-04-14 | 2014-03-29 | Augmented reality system using mirror |
Country Status (2)
Country | Link |
---|---|
KR (1) | KR20140123418A (en) |
WO (1) | WO2014171644A1 (en) |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
KR102572632B1 (en) * | 2022-08-26 | 2023-08-29 | 강남대학교 산학협력단 | Method and apparatus for providing augmented reality service using user face recognition |
Families Citing this family (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN109255819B (en) * | 2018-08-14 | 2020-10-13 | 清华大学 | Kinect calibration method and device based on plane mirror |
Family Cites Families (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
ES2325374T3 (en) * | 2005-05-03 | 2009-09-02 | Seac02 S.R.L. | INCREASED REALITY SYSTEM WITH IDENTIFICATION OF REAL MARKER OBJECT. |
US20070040033A1 (en) * | 2005-11-18 | 2007-02-22 | Outland Research | Digital mirror system with advanced imaging features and hands-free control |
KR20080102789A (en) * | 2007-05-22 | 2008-11-26 | 김성현 | Virtual mirror |
EP2399245A1 (en) * | 2009-02-23 | 2011-12-28 | Koninklijke Philips Electronics N.V. | Mirror device |
KR20100135001A (en) * | 2009-06-16 | 2010-12-24 | 두원공과대학산학협력단 | A digital virtual mirror system for wearing clothes and the processing method thereof |
-
2014
- 2014-03-29 WO PCT/KR2014/002700 patent/WO2014171644A1/en active Application Filing
- 2014-03-29 KR KR1020140037340A patent/KR20140123418A/en not_active Application Discontinuation
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
KR102572632B1 (en) * | 2022-08-26 | 2023-08-29 | 강남대학교 산학협력단 | Method and apparatus for providing augmented reality service using user face recognition |
Also Published As
Publication number | Publication date |
---|---|
WO2014171644A1 (en) | 2014-10-23 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US10012841B1 (en) | Advanced retroreflecting aerial displays | |
KR101325292B1 (en) | Camera applications in a handheld device | |
EP2716029A1 (en) | Apparatus and method | |
KR101299195B1 (en) | Apparatus for projecting space image | |
Large et al. | Parallel optics in waveguide displays: a flat panel autostereoscopic display | |
US11054652B2 (en) | Augmented reality apparatus | |
KR20140123418A (en) | Augmented reality system using mirror | |
JP2015079201A (en) | Video display system, video display method, and projection type video display device | |
JP2002523790A (en) | Projection system | |
EP3009887B1 (en) | Optical imaging processing system | |
TWI572899B (en) | Augmented reality imaging method and system | |
CN110174975B (en) | Projection touch system | |
US8912483B2 (en) | Display measuring device | |
KR101002071B1 (en) | Apparatus for touching a projection of 3d images on an infrared screen using multi-infrared camera | |
JP2004163528A (en) | Three-dimensional display device | |
WO2018145463A1 (en) | Device and method for implementing augmented reality | |
JP2019179542A (en) | Information processing system and program | |
Tsuchiya et al. | Aerial Imaging by Retro-Reflection for Mid-Air Image Display and Camera Viewpoint Transfer | |
US20220307821A1 (en) | Detection apparatus, detection method, and spatial projection apparatus | |
KR102182580B1 (en) | Hr box | |
US20240184133A1 (en) | Air floating video display apparatus | |
KR101693150B1 (en) | Optical System for Selectively Providing VR and AR | |
WO2018207235A1 (en) | Input/output system, screen set, input/output method, and program | |
JP2024080390A (en) | Space-floating image display device | |
CN116405652A (en) | Three-dimensional display system, three-dimensional display method, electronic device, and storage medium |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
A201 | Request for examination | ||
E902 | Notification of reason for refusal | ||
E601 | Decision to refuse application |