WO2015124957A1

WO2015124957A1 - Augmented reality based handguns targeting system

Info

Publication number: WO2015124957A1
Application number: PCT/HR2015/000005
Authority: WO
Inventors: Damir Krstinic; Ivan LELAS; Karlo LELAS
Original assignee: Damir Krstinic; Lelas Ivan; Lelas Karlo
Priority date: 2014-02-24
Filing date: 2015-02-24
Publication date: 2015-08-27
Also published as: HRP20140173A2

Abstract

A handgun shooter while in motion is unable to hold attention to the target and the point of impact continuously. In order to target more precisely, the shooter needs to stop for a moment, focus on the target and align the front and rear sights with the target, which normally means wasting valuable time. In bright light conditions, not even a built-in laser is of any help, since the laser point is hard to detect. This problem is solved by the augmented reality based handguns targeting system. The system utilises glasses (1) with built-in cameras (3) and a computer (4). Computer processing of the image obtained by cameras (3) discovers the point of impact. Before the shooter's eye, the reticle (12) is outlined on the transparent screen (5), marking the point of impact (16). The reticle on transparent screen (5) of glasses (1) is maintained continuously, as long as the weapon is directed in the watching direction, regardless of the shooter's other activities. The moment the shooter decides to shoot, the reticle marks the point of impact, enabling fast and intuitive shooting, requiring no repositioning of the weapon to the eye level and looking through the targeting devices. The system is used for handguns (6) equipped with a laser (7) and handguns (10) equipped with a camera (11). In handguns with built-in laser (7), the point of impact, marked with laser point (9), is detected by processing the image from cameras (3) on glasses (1). In handguns with built-in camera, the point of impact is detected by comparing images from camera (11) on the handgun (10) and cameras (3) on glasses (1).

Description

AUGMENTED REALITY BASED HANDGUNS TARGETING SYSTEM

INVENTION DESCRIPTION

1. FIELD OF APPLICATION

The invention relates to the system of aiding handguns targeting, based on augmented reality principle.

According to the International Patent Classification, the invention is classified as Section F - Weapons; Class - F41G - Weapon sights, Targeting; Subclasses F41 G 3/00 - Aiming means, Laying means; and 3/16 - Sighting devices adapted for indirect laying of fire; and Field G - Physics; Class G06T - Image data processing or generation, in general.

2. TECHNICAL PROBLEM

While moving and shooting a target, a handgun shooter cannot continuously keep attention to it and target at the same time due to several reasons such as focusing on the very motion, avoiding obstacles, visual tracking of the target in motion itself. The shooter also faces various distractions from the environment, especially if there are other shooters around too. In order to be precise, the shooter needs to stop for a while, focus on the target, and align the front and the rear sights with the target, which actually leads to a waste of valuable time.

The usage of laser sights has helped to make targeting easier and faster, by faster detection of the region of impact and using intuitive targeting, which does not require precise alignment of the eye with the target over the weapon sights. However, the shooter in motion cannot simultaneously keep attention to the target and the point of impact marked by the laser pointer, even when using such systems, due to the simultaneous movements of the target and the weapon on which the laser is mounted. This is particularly the case if the shooter is distracted by the environment. Additional problems when using laser pointers can occur when it's sunny and the light is intensive so that the laser point, which marks the point of impact, is merely visible due to the bright light conditions.

The proposed invention is solving the above described problem. 3. STATE OF THE ART

Classical targeting requires aligning the front and the rear sights on the weapon with the distant target.

Optical sights are based on projecting a reticle image marking the point of impact into the shooter's eye by means of a system of lenses. Such sights may have additional mechanical or electronic components, such as reticle illumination systems, enlargement degree changing systems, etc. Although they enable greater precision when targeting, these sights are not suitable for targeting in motion, because it's virtually impossible to inspect the area and aim through an optical sight simultaneously. Besides, such sights are generally not suitable for mounting on small handguns because of their size.

Holographic or reflexive sights are based on projecting luminous reticle into the space between the shooter's eyes and the target. Due to the stereoscopic nature of human vision, it appears that the dot marking the point of impact is on the target itself, thus minimising the parallax effect, where the dot marking the point of impact moves depending on the angle of viewing into the sight device. Numerous holographic sight varieties have been developed. Such sights are usually embodied as optical devices without enlargement, through which the shooter is looking. Although they enable much more intuitive and faster targeting by using both eyes, you are still required to briefly interrupt visual contact with the target, place the weapon with the sight in front of your eyes, and align the target and the reticle in the targeting system. However, these sights still face the problem of mounting on small handguns.

Laser sights use lasers fitted on weapons to mark the point of impact, which makes it easier for a shooter to detect and follow the point of impact. The basic disadvantage of such sights is that the shooter in motion has difficulties to keep attention to the laser point, which is small and often hard to detect, especially in bright light. When the shooter decides to shoot, he has to stop briefly and focus on detecting the laser point and positioning it on the target. Such sights are sometimes combined with the previously mentioned technologies.

4. DISCLOSURE OF THE INVENTION

The essence of the invention is augmented reality based handguns targeting system. This system uses glasses with built-in cameras and a computer, which can display computer generated information in the field of vision, i.e. on transparent screens which enable users to see the environment. If the weapon is pointed in the direction where the shooter is looking, the point of impact is detected and tracked by the camera mounted on the glasses. The computer processed image is then projected onto the transparent screen and it outlines a reticle, marking the point of impact. The reticle stays outlined as long as the weapon is pointed in the direction where the shooter is looking, regardless of the shooter's other activities. When the shooter decides to shoot, the reticle marks the point of impact, enabling fast and intuitive shooting, which requires no repositioning of the weapon to the eye level or looking through the sight devices.

The system is used for targeting handguns with a built-in laser, where the laser marks the point of impact. If the weapon is directed in the shooter's watching direction, computer processing of the image from the camera mounted on the glasses detects the laser beam that marks the point of impact, and the reticle marking the point of impact is outlined on the glasses transparent screens in the shooter's field of vision. The system is applicable when targeting with handguns fitted with a camera, where the image obtained from the weapon camera is sent to the computer in the glasses using a wireless connection. When comparing the image from the glasses camera and that from the weapon camera, the point of impact is detected and outlined in the form of a reticle, generated on the transparent screen in the shooter's field of vision.

5. ILLUSTRATION DESCRIPTIONS

Figure 1 shows augmented reality glasses, equipped with cameras and computer.

Figure 2 shows handgun equipped with laser.

Figure 3 shows handgun with laser beam showing the point of impact. Figure 4 shows characteristics of the band-pass optical filter.

Figure 5 shows glasses with computer generated reticle on glasses transparent screens through which the shooter sees the environment.

Figure 6 shows handgun equipped with camera.

Figure 7 shows detection of the point of impact, by comparing image from camera mounted on the handgun with image from glasses camera, by identifying details common for both images.

6. DETAILED DESCRIPTION OF THE INVENTION EMBODIMENT

The augmented reality technology is based on feeding artificially generated information directly into the user's field of vision, by means of transparent display devices, used to look at the environment. So far, there have been no technical solutions using augmented reality that would enable handgun intuitive targeting. In the proposed invention, the reticle marking the point of impact is shown on the transparent display device directly in front of the shooter's eye used for looking at the environment, regardless of the weapon's position.

The purpose of the invention is to enable the shooter to detect and hold his attention to the point of impact.

The basic idea of the invention is to feed a computer generated visual information into the shooter's field of vision, whereby attention is directed to the point of impact, without disturbing any other activities.

In a regular usage, the shooter sees the environment through glasses that enable display of computer generated information. If the weapon is directed in the shooter's watching direction, the point of impact is detected and followed electronically, and a reticle marking the point of impact is outlined on the transparent screen before the shooter's eye. The reticle is added into the actual environment in a way that the user always sees it on the point of impact, regardless of the angle of watching, thus eliminating the parallax effect. The reticle marking the point of impact is outlined continuously on the transparent screen regardless of the shooter's other activities, as long as the weapon is directed in the watching direction.

Feeding the computer generated information into the field of vision enables the user to hold attention to the point of impact more easily, without disturbing any other activity, such as movement, avoiding obstacles, maintaining visual contact with the target in motion, etc. The moment the shooter decides to shoot, the reticle marks the point of impact and enables fast and intuitive shooting, requiring no repositioning of the weapon to the eye level.

The augmented reality glasses with built-in cameras and a computer can be added to the helmet worn by the shooter while using the weapon. Other transparent display devices can be used instead of these glasses, such as contact lenses, able to display computer generated information, placed directly onto the shooter's eye.

The described invention can be embodied in several ways. The herein described embodiments are based on analysis of the image received from cameras positioned immediately next to the shooter's eye. The augmented reality glasses are equipped with a computer that can be either integrated into the glasses themselves or made separate and connected to the glasses with wire or a wireless connection.

The system gives an example of the application of augmented reality by means of glasses with integrated cameras and a computer either when shooting with a handgun with a built-in laser or when shooting with a handgun with a built-in camera.

EMBODIMENT AND FUNCTIONING

I. Augmented reality based handguns targeting system with handgun equipped with a laser

Targeting is performed by combining the existing laser targeting technologies and glasses able to present augmented reality with cameras built-in next to the shooter's eyes. Figure 1 shows glasses 1 with cameras 3, computer 4, and transparent screens 5. This invention embodiment is based on analysing the image received from cameras 3 positioned immediately next to the shooter's eyes, by the computer 4 that is integrated in the body 2 of the glasses 1 , or is built separately and connected to the glasses with wire or wireless connection.

Handgun 6 with laser 7, Figure 2, emits laser beam 8 that ends in laser point 9, Figure 3, and marks the point of impact 16 in the Figure 7.

The shooter's field of vision, recorded by the cameras 3 next to the shooter's eyes, is scanned by the computer vision methods, aimed at recognising the laser point 9 emitted by the laser 7, integrated in the handgun 6. Since it's normally hard to detect the laser point 9 which marks the point of impact 16 in bright light, on a sunny day for instance, by either the eye or the computer, to enable a better detection, the glasses mounted cameras are equipped with band-pass optical filters, where the filter's central wavelength corresponds to the wavelength of light of the laser mounted in the weapon. A typical characteristic of a band-pass filter is shown in the Figure 4. A characteristic of a filter of the central wavelength of 532 nm, corresponding to the green light is shown. The built-in optical filter attenuates light of all wavelengths other than that of laser beam 8, whereby the bright light effect is avoided, and the laser point 9 becomes visible.

Analysing the image by both cameras 3 located next to the shooter's eyes, the point of impact 16 is detected, marked by reticle 12 which is outlined on the transparent screens 5 before the shooter's eyes, independently for the left and the right eye. Due to the stereoscopic nature of human vision, this way enables the shooter to have the impression that the reticle 12, marking the point of impact 16, is in the space. The image outlined on the screens shows only the computer generated reticle marking the point of impact, as shown in the Figure 5, while the shooter sees the environment through the transparent screens 5 of the glasses 1.

By using a laser with a wavelength outside the visible part of the spectrum and an appropriate optic filter of the band-pass centred at that wavelength, it is ensured that the reticle 12 marking the point of impact 16 is visible only to the shooter whose glasses 1 are paired with the laser 7 on the handgun 6. Other persons in the area do not see the laser beam 8, showing the point of impact 16, of the wavelength outside the visible part of the spectrum.

II. Augmented reality based handguns targeting system with the handgun equipped with a camera

Augmented reality based handguns targeting is achieved by installing camera 1 1 on handgun 10, as shown in the Figure 6. Coordinates of the centre of the image 13, Figure 7, obtained from the camera 1 1 on the handgun 10 correspond to point of impact 16. Image 13, obtained from the camera 1 1 on the handgun 10, is transferred to computer 4 on glasses 1 by a wireless connection. The point of impact 16 detection algorithm is based on comparing the images obtained from the two cameras by identifying characteristic points present in both images. Based on the characteristic points existing in both images, the projection transformation matrix is calculated, whereby the image from one camera is copied into the plane of the other camera, i.e. image 13, obtained from the camera 11 on the handgun 10 is positioned within the image 14 obtained from the camera 3 positioned next to the shooter's eyes. The image from the handgun camera, positioned within the image obtained from the shooter's eye level camera, is marked by the referral mark 15, Figure 7.

The projection transformation calculation of the image from the camera 11 on the handgun 10 is positioned relating to the coordination system of the camera 3 that corresponds to the shooter's field of vision. This calculation is performed separately for the cameras 3 positioned next to the left and the right eye, wherefore the point of impact 16, corresponding to the centre of the image 13 from the camera 1 1 on the handgun 10, is marked with the reticles 12 on each of the screens 5, before both eyes, separately, as shown in the Figure 5. The stereoscopic nature of the human vision produces the impression that the reticle 12 is in the space, on the very point of impact 16.

This invention embodiment enables that only the shooter, who is carrying the handgun 10 with the camera 1 1 , paired with the cameras 3 on the glasses 1 to view augmented reality, sees the point of impact 16.

LIST OF REFERRAL LETTERS AND NUMBERS IN DESCRIPTIONS AND FIGURES

1 - glasses

2 - body of the glasses

3 - cameras mounted on glasses

4 - computer

5 - glasses transparent screens

6 - handgun equipped with laser

7 - laser

8 - laser beam

9 - laser point

10 - handgun equipped with camera

11 - handgun camera 12 - reticle on glasses transparent screen

13 - image from the handgun camera

14 - image from the camera next to the shooter's eyes

15 - handgun camera image positioned in the image from the camera next to the shooter's eyes

16 - point of impact

7. INVENTION APPLICATION

The invention is used when shooting with a handgun. The invention is a device that may be produced economically, and that has significant improvements relative to the previously known devices used when shooting handguns.

Claims

PATENT CLAIMS

1. Augmented reality based handguns targeting system, wherein the system utilises glasses (1 ) with built-in cameras (3) and a computer (4); the system is used in targeting handguns where images from cameras (3) are processed by the computer (4) to detect a point of impact (16), as long as a weapon is directed in a shooter's watching direction; the point of impact (16) is marked with a reticle (12) generated on transparent screens (5) of glasses (1 ) through which the shooter sees environment; the system enables fast and intuitive targeting, not requiring repositioning of the weapon up to the eye level.

2. The system as claimed in Claim 1 , wherein a handgun (6) is technically characterised by built-in laser (7); the laser (7) emits a laser beam (8) marking the point of impact (16); the point of impact (16) is detected with the laser beam (8) by analysing the image obtained from cameras (3) by the computer (4); the point of impact (16) is marked by the reticle (12) generated on transparent screens (5) of glasses (1 ) through which the shooter sees the environment.

3. The system as claimed in Claim 1 , wherein a handgun (10) is technically characterised by a built-in camera (11 ); the image obtained from the camera (11 ) on the handgun (10) is wirelessly sent to the computer (4); the point of impact (16) is detected by the computer (4) by comparing images from camera (1 1 ) on the handgun (10) and from cameras (3) on glasses (1 ); the point of impact (16) is marked by the reticle (12) generated on transparent screens (5) of glasses (1 ) through which the shooter sees the environment.

4. The system as claimed in Claims 1 and 2, wherein cameras (3) built in glasses (1), are technically characterised by the optical band-passing filter, the central wavelength of which corresponds to the wavelength of the laser (7) built in the handgun (6); processing of image from cameras (3) provided with optical filter recognises only the laser beam (8) emitted by the laser (7), the wavelength of which corresponds to the central wavelength of the optical filter built in cameras (3).

5. The system as claimed in Claims 1 and 4, wherein the wavelength of the laser (7), on the handgun (6), is outside the visible part of the spectrum; the point of impact (16), marked by the laser beam (8), is detected by processing image from cameras (3) with built-in optical band-passing filter, the central wavelength of which corresponds to the wavelength of laser (7) that with glasses (1 ) make a single targeting device; other persons in the environment do not see the laser beam (8) marking the point of impact (16), the wavelength of which is outside the visible part of the spectrum.

6. The system as claimed in Claim 1 , wherein glasses (1 ) with transparent screens (5) to view augmented reality, with built-in cameras (3) and the computer (4), make part of a helmet worn by the shooter while utilising the handgun.