WO2022240003A1 - Omnidirectional external observation system for tank using augmented reality hmd - Google Patents

Omnidirectional external observation system for tank using augmented reality hmd Download PDF

Info

Publication number
WO2022240003A1
WO2022240003A1 PCT/KR2022/005821 KR2022005821W WO2022240003A1 WO 2022240003 A1 WO2022240003 A1 WO 2022240003A1 KR 2022005821 W KR2022005821 W KR 2022005821W WO 2022240003 A1 WO2022240003 A1 WO 2022240003A1
Authority
WO
WIPO (PCT)
Prior art keywords
tank
augmented reality
omnidirectional
driver
external
Prior art date
Application number
PCT/KR2022/005821
Other languages
French (fr)
Korean (ko)
Inventor
최치원
이원석
윤종주
Original Assignee
주식회사 피앤씨솔루션
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 주식회사 피앤씨솔루션 filed Critical 주식회사 피앤씨솔루션
Publication of WO2022240003A1 publication Critical patent/WO2022240003A1/en

Links

Images

Classifications

    • F—MECHANICAL ENGINEERING; LIGHTING; HEATING; WEAPONS; BLASTING
    • F41—WEAPONS
    • F41G—WEAPON SIGHTS; AIMING
    • F41G3/00—Aiming or laying means
    • F41G3/26—Teaching or practice apparatus for gun-aiming or gun-laying
    • F41G3/2605—Teaching or practice apparatus for gun-aiming or gun-laying using a view recording device cosighted with the gun
    • F41G3/2611—Teaching or practice apparatus for gun-aiming or gun-laying using a view recording device cosighted with the gun coacting with a TV-monitor
    • F—MECHANICAL ENGINEERING; LIGHTING; HEATING; WEAPONS; BLASTING
    • F41—WEAPONS
    • F41G—WEAPON SIGHTS; AIMING
    • F41G3/00—Aiming or laying means
    • F—MECHANICAL ENGINEERING; LIGHTING; HEATING; WEAPONS; BLASTING
    • F41—WEAPONS
    • F41G—WEAPON SIGHTS; AIMING
    • F41G3/00—Aiming or laying means
    • F41G3/005—Aiming or laying means with means for correcting the parallax between the sighting means and the muzzle axis
    • F—MECHANICAL ENGINEERING; LIGHTING; HEATING; WEAPONS; BLASTING
    • F41—WEAPONS
    • F41G—WEAPON SIGHTS; AIMING
    • F41G3/00—Aiming or laying means
    • F41G3/26—Teaching or practice apparatus for gun-aiming or gun-laying
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/0093—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00 with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/01—Head-up displays
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/01—Head-up displays
    • G02B27/017—Head mounted
    • G02B27/0172—Head mounted characterised by optical features
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T19/00—Manipulating 3D models or images for computer graphics
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T19/00—Manipulating 3D models or images for computer graphics
    • G06T19/006—Mixed reality
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/20—Image signal generators
    • H04N13/204—Image signal generators using stereoscopic image cameras
    • H04N13/243—Image signal generators using stereoscopic image cameras using three or more 2D image sensors
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/30—Image reproducers
    • H04N13/332—Displays for viewing with the aid of special glasses or head-mounted displays [HMD]
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/01—Head-up displays
    • G02B27/0101—Head-up displays characterised by optical features
    • G02B2027/014—Head-up displays characterised by optical features comprising information/image processing systems
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/01—Head-up displays
    • G02B27/0179—Display position adjusting means not related to the information to be displayed
    • G02B2027/0187—Display position adjusting means not related to the information to be displayed slaved to motion of at least a part of the body of the user, e.g. head, eye

Definitions

  • the present invention relates to an omnidirectional external observation system for a tank, and more particularly, to an omnidirectional external observation system for a tank using an augmented reality HMD.
  • a head mounted display which is a kind of wearable device, refers to various devices that can be worn on a user's head to receive multimedia contents and the like.
  • the head mounted display (HMD) is worn on the user's body and provides images to the user in various environments as the user moves.
  • Head-mounted displays (HMDs) are classified into see-through and see-closed types.
  • the see-through type is mainly used for Augmented Reality (AR), and the closed type is mainly used for virtual reality (VR). reality, VR).
  • a tank is a combat vehicle equipped with a powerful gun or machine gun and equipped with a powerful propulsion engine and driving device that can maneuver even in open roads on a body protected by thick armor. Since the area is minimized, there is a risk of exposing the body to the outside to secure an external view when operating a tank.
  • a method of checking an external situation from the inside through an external image capture device has been developed, it is quite inconvenient to check the image while driving.
  • Patent Registration No. 10-1545336 title of invention: wireless video and control data transmission device for tank shooting training, registration date: August 11, 2015
  • the present invention has been proposed to solve the above problems of the previously proposed methods, and the camera device mounted on the outside of the tram takes omnidirectional shots and the image processing device processes the captured images to generate external image information,
  • the camera device mounted on the outside of the tram takes omnidirectional shots and the image processing device processes the captured images to generate external image information
  • the augmented reality HMD device worn by the driver inside the tank it is possible to secure the outside view from inside the tank without the risk of exposing the body to the outside to secure the outside view of the tank, and the external appearance can be Since it is provided in augmented reality through the driver's field of view, it is possible to grasp the external situation easily and efficiently while driving, and the external image is displayed according to the driver's head movement, which is suitable and intuitive for tank control, and does not require a separate person for external observation.
  • the purpose of this study is to provide an omnidirectional external observation system for tanks using an augmented reality HMD.
  • An omnidirectional external observation system for tanks using an augmented reality HMD according to the features of the present invention for achieving the above object
  • An omnidirectional external observation system for a tank that allows the driver inside the tank to observe the outside of the tank from the inside of the tank,
  • a camera device mounted on the outside of the tank and capable of shooting in all directions;
  • an image processing device that generates external image information to be output to augmented reality from an external image of the tram captured by the camera device
  • augmented reality HMD device It is worn on the head of the driver riding inside the tank, outputs the external image information generated by the image processing device as image light, and provides augmented reality by combining the transmitted light of the real world and the image light. It includes an augmented reality HMD device that
  • the augmented reality HMD device
  • a display unit outputting image light so that the external image information generated by the image processing device can be provided to a driver
  • an optical unit disposed in front of both eyes of the driver wearing the HMD frame, transmitting at least a part of the light of the real world through the driver's field of view, and transmitting the image light output from the display unit to the driver's eye direction;
  • a sensor unit installed on one side of the HMD frame and detecting a head movement of a driver wearing the HMD frame, including an IMU sensor;
  • An image control unit configured to transmit and control external image information generated by the image processing device to be output to the display unit, and to display an external image in a direction the driver is looking in accordance with the movement of the driver's head detected by the sensor unit. It is characterized by its composition.
  • the camera device Preferably, the camera device, the camera device, and
  • It may be composed of a plurality of cameras pointing in different directions to enable omnidirectional shooting.
  • the image processing device More preferably, the image processing device,
  • a 360-degree omnidirectional image may be generated by matching images captured by a plurality of cameras constituting the camera device.
  • the camera device More preferably, the camera device,
  • a disk or cylindrical body Mounted on the outside of the tank, a disk or cylindrical body;
  • It includes a camera module composed of a pair of cameras arranged at an angle of 90 degrees or more and 150 degrees or less,
  • the camera module may be disposed in each direction in four or more directions while surrounding the body.
  • the body Even more preferably, the body,
  • It is rotatably mounted on the outside of the tank and can be controlled by the driver.
  • the camera device Even more preferably, the camera device,
  • the augmented reality HMD device Preferably, the augmented reality HMD device,
  • the video control unit controls the display of
  • Data received from an external server or other augmented reality HMD device through the communication unit may be output and provided as augmented reality through the driver's field of view.
  • the image processing device Preferably, the image processing device, the image processing device, the image processing device, and
  • a 360-degree omnidirectional image can be created by matching images using an artificial intelligence-based deep learning model.
  • a camera device mounted on the outside of a tank performs omnidirectional shooting and an image processing device processes the captured image to generate external image information
  • an image processing device processes the captured image to generate external image information
  • FIG. 1 is a view showing the overall configuration of an omnidirectional external observation system for a tank using an augmented reality HMD according to an embodiment of the present invention.
  • FIG. 2 is a view showing, for example, a state of a tank equipped with a camera device of an omnidirectional external observation system for a tank using an augmented reality HMD according to an embodiment of the present invention.
  • FIG 3 is a view showing a camera device as an example in an omnidirectional external observation system for a tank using an augmented reality HMD according to an embodiment of the present invention.
  • FIG. 4 is a view showing, for example, the inside of a tank in which an omnidirectional external observation system for a tank using an augmented reality HMD according to an embodiment of the present invention is built.
  • FIG. 5 is a view showing the configuration of an augmented reality HMD device in an omnidirectional external observation system for a tank using an augmented reality HMD according to an embodiment of the present invention.
  • FIG. 1 is a diagram showing the overall configuration of an omnidirectional external observation system 10 for a tank using an augmented reality HMD according to an embodiment of the present invention.
  • the omnidirectional external observation system 10 for a tank using an augmented reality HMD according to an embodiment of the present invention allows a driver riding inside the tank to observe the outside of the tank from inside the tank.
  • an omnidirectional external observation system 10 for a tram it may be configured to include a camera device 100, an image processing device 200, and an augmented reality HMD device 300.
  • the camera device 100 is mounted on the outside of the train and may be able to take pictures in all directions. More specifically, the camera device 100 may include a plurality of cameras pointing in different directions to enable omnidirectional photography, and may include a thermal imaging camera, a night vision camera, and the like in addition to a general camera.
  • FIG. 2 is a diagram showing, for example, a state of a tank equipped with a camera device 100 of an omnidirectional external observation system 10 for a tank using an augmented reality HMD according to an embodiment of the present invention.
  • the camera device 100 of the omnidirectional external observation system 10 for a tank using an augmented reality HMD according to an embodiment of the present invention is a vehicle that can best observe the outside. It can be installed on the outer upper side and, depending on the embodiment, can be detachably mounted on the outside of the tank. A specific location where the camera device 100 is installed may differ depending on the shape or structure of a tank, operational environment, and the like.
  • FIG. 3 is a view showing the camera device 100 as an example in the omnidirectional external observation system 10 for a tank using an augmented reality HMD according to an embodiment of the present invention.
  • the camera device 100 of the omnidirectional external observation system 10 for a tank using an augmented reality HMD according to an embodiment of the present invention is mounted on the outside of a tank and has a disk or column shape. It may be configured to include a body 110, and a camera module 120 composed of a pair of cameras disposed at an angle of 90 degrees or more and 150 degrees or less, surrounding the body 110 and in each direction in four or more directions.
  • a pair of camera modules 120 may be disposed. More specifically, the camera modules 120 are arranged at regular intervals in eight directions, and each camera module 120 is configured so that a pair of cameras are disposed at an angle of 120 degrees so that no blind spots occur. can do.
  • the body 110 is rotatably mounted on the outside of the tank and can be rotated and controlled by a driver. Therefore, when some of the camera modules 120 are damaged, such as destruction or breakdown, the undamaged camera module 120 is rotated in a direction required for observation to secure a field of view.
  • the camera device 100 may further include an upper camera 130 disposed upward to observe not only the side surface but also the sky direction.
  • the image processing device 200 may generate external image information to be output in augmented reality from an image of the exterior of the train captured by the camera device 100 . More specifically, the image processing device 200 may generate a 360-degree omnidirectional image by matching images captured by a plurality of cameras constituting the camera device 100 . That is, the image processing device 200 may combine images captured by multiple cameras using image registration technology and configure a single 360-degree external image information.
  • the image processing device 200 may generate a 360-degree omnidirectional image by matching images using an artificial intelligence-based deep learning model.
  • a key-point matching algorithm that extracts a key-point having an important meaning from an image for image matching and matches the image based on the key-point may be used. Images captured in different coordinate systems or viewpoints may be matched through key-point matching.
  • the image processing device 200 may use any one algorithm among SIFT, SURF, KAZE, AKAZE, and ORB for key-point matching. can More specifically, since the image processing device 200 needs to match images in real time to generate external image information, an ORB that is the fastest among several key-point matching algorithms can be used.
  • the augmented reality HMD device 300 is worn on the driver's head inside the train, outputs external image information generated by the image processing device 200 as image light, and transmits light of the real world. It is possible to provide augmented reality by the combination of and image light.
  • FIG. 4 is a view showing, for example, the interior of a train in which the omnidirectional external observation system 10 for a tank using an augmented reality HMD according to an embodiment of the present invention is built.
  • the augmented reality HMD device 300 worn by the driver riding inside the tram Since the external image information generated by the image processing device 200 is provided in augmented reality, the driver can check the outside at a glance through the external image information of the field of view while concentrating on driving the tram without having to look at a separately prepared screen.
  • the omnidirectional external observation system 10 for a tank using an augmented reality HMD includes an HMD frame 310, a display unit 320, an optical unit 330, It may be configured to include a sensor unit 340 and an image controller 350, and may be configured to further include a communication unit 360.
  • the HMD frame 310 is a frame structure that can be worn on the head by a driver.
  • the HMD frame 310 may be configured in the form of a helmet or goggles having a frame structure through which light may enter while worn on the driver's head.
  • it is preferable to configure it in the form of a bulletproof helmet, and a bulletproof helmet (not shown) worn on the driver's head and a display frame (not shown) placed in front of the bulletproof helmet structure can be made.
  • the bulletproof helmet is made of a hard material capable of sufficiently protecting the wearer's head, and an impact relieving member capable of protecting the wearer's head through a buffering action may be included in the open interior space.
  • the shock absorbing member may be made of a material having antibacterial and deodorizing functions, or may be coated with antibacterial and deodorizing functions.
  • a headrest strap and a chin strap may be further included.
  • the display unit 320 may output image light so that external image information generated by the image processing device 200 may be provided to a driver.
  • the display 131 may be configured in various ways such as OLED.
  • the optical unit 330 is disposed in front of both eyes of the pilot wearing the HMD frame 310, transmits at least a part of the light of the real world through the pilot's field of view, and transmits image light output from the display unit 320. It can be configured so that the wearer can experience augmented reality by transmitting it in the direction of the driver's eyes.
  • the optical unit 330 is composed of a plurality of lenses and mirrors and may be implemented in various ways, such as an optical diffraction method, a beam splitter method, a pin mirror method, and the like.
  • the sensor unit 340 is installed on one side of the HMD frame 310 and can detect head movements of a driver wearing the HMD frame 310 including an IMU sensor.
  • the IMU sensor can accurately analyze the direction the driver is looking by detecting the moving direction, speed, and angle of the driver's head.
  • the sensor unit 340 may further include an illuminance sensor, a chemical sensor, and the like, in addition to the IMU sensor.
  • the sensing data collected by the sensor unit 340 may be used by the image controller 350 to provide augmented reality image information.
  • the image controller 350 transmits and controls the external image information generated by the image processing device 200 to be output to the display unit 320, and controls the transmission of the external image information generated by the image processing device 200 according to the driver's head movement detected by the sensor unit 340. It can be controlled so that the external image of the direction is displayed. For example, if the driver turns his head to the left or right, the external image information rotates according to the movement of the head, and the left or right external image viewed by the driver may be displayed in augmented reality.
  • the communication unit 360 may communicate with the image processing device 200, an external server, and other augmented reality HMD devices 300.
  • the image controller 350 may control data received from an external server or other augmented reality HMD device 300 through the communication unit 360 to be output and provided to augmented reality through the driver's field of view.
  • an external situation, instructions, commands, and the like may be transmitted to the operator through augmented reality.
  • the situation can be conveyed intuitively using text, images, colors, and icons.
  • the communication unit 360 is installed on one side of the augmented reality HMD device 300, and the network used by the communication unit 360 includes a local area network (LAN), a wide area network (WAN), and a mobile network.
  • Mobile radio communication network satellite communication network, Bluetooth, Wibro (Wireless Broadband Internet), HSDPA (High Speed Downlink Packet Access), LTE (Long Term Evolution), 3/4/5G (3/4/5th Generation Mobile Telecommunication) can be implemented in all types of wired and wireless networks.
  • the camera device 100 mounted on the outside of the tank performs omnidirectional photography and the image processing device 200 Risk of exposing the body to the outside to secure the external view of the tank Since the outside view is provided in augmented reality through the driver's field of view, the external situation can be grasped easily and efficiently while driving, and the external image is displayed according to the driver's head movement to control the tank. suitable for and intuitive, and does not require a separate person for external observation.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Optics & Photonics (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Radar, Positioning & Navigation (AREA)
  • Computer Graphics (AREA)
  • Computer Hardware Design (AREA)
  • Software Systems (AREA)
  • Theoretical Computer Science (AREA)

Abstract

The present invention provides an omnidirectional external observation system for a tank, wherein the omnidirectional external observation system uses an augmented reality HMD. According to the omnidirectional external observation system, a camera device mounted on the outside of the tank captures omnidirectional images, and an image processing device processes the captured images to generate external image information and provides the external image information using augmented reality through an augmented reality HMD device worn by a driver on board the tank. Accordingly, a field of view outside the tank can be secured from inside the tank without having to risk exposing the body to the outside in order to secure the field of view of the outside, and the external situation can easily and efficiently comprehended while driving since the conditions outside are provided using augmented reality through a driver's field of view. Also, external images are displayed according to the movement of the driver's head, and thus intuitive and suitable for controlling the tank, and there is no need for additional personnel for external observation.

Description

증강현실 HMD를 이용한 전차용 전방위 외부 관측 시스템Omnidirectional external observation system for tanks using augmented reality HMD
본 발명은 전차용 전방위 외부 관측 시스템에 관한 것으로서, 보다 구체적으로는 증강현실 HMD를 이용한 전차용 전방위 외부 관측 시스템에 관한 것이다.The present invention relates to an omnidirectional external observation system for a tank, and more particularly, to an omnidirectional external observation system for a tank using an augmented reality HMD.
디지털 디바이스의 경량화 및 소형화 추세에 따라 다양한 웨어러블 디바이스(wearable device)들이 개발되고 있다. 이러한 웨어러블 디바이스의 일종인 헤드 마운티드 디스플레이(Head Mounted Display)는 사용자가 머리에 착용하여 멀티미디어 컨텐츠 등을 제공받을 수 있는 각종 디바이스를 의미한다. 여기서 헤드 마운티드 디스플레이(HMD)는 사용자의 신체에 착용 되어 사용자가 이동함에 따라서 다양한 환경에서 사용자에게 영상을 제공하게 된다. 이러한 헤드 마운티드 디스플레이(HMD)는 투과(see-through)형과 밀폐(see-closed)형으로 구분되고 있으며, 투과형은 주로 증강현실(Augmented Reality, AR)용으로 사용되고, 밀폐형은 주로 가상현실(Virtual Reality, VR)용으로 사용되고 있다.Various wearable devices are being developed according to the trend of light weight and miniaturization of digital devices. A head mounted display, which is a kind of wearable device, refers to various devices that can be worn on a user's head to receive multimedia contents and the like. Here, the head mounted display (HMD) is worn on the user's body and provides images to the user in various environments as the user moves. Head-mounted displays (HMDs) are classified into see-through and see-closed types. The see-through type is mainly used for Augmented Reality (AR), and the closed type is mainly used for virtual reality (VR). reality, VR).
한편, 전차는 위력이 큰 포나 기관총 등을 탑재하고, 두꺼운 장갑으로 방호된 차체에 도로가 없는 야지(野地)에서도 기동할 수 있는 강력한 추진기관과 주행장치를 지닌 전투차량으로, 안전을 위해 개방된 영역을 최소화하므로 전차 운용 시 외부 시야 확보를 위해 몸을 외부로 노출해야 하는 위험이 있다. 또한, 외부 영상 획득 장치를 통해 내부에서 외부 상황을 확인하는 방법이 개발되기도 했으나, 운전과 동시에 영상 확인을 하는 것은 상당히 불편한 것이 현실이다.On the other hand, a tank is a combat vehicle equipped with a powerful gun or machine gun and equipped with a powerful propulsion engine and driving device that can maneuver even in open roads on a body protected by thick armor. Since the area is minimized, there is a risk of exposing the body to the outside to secure an external view when operating a tank. In addition, although a method of checking an external situation from the inside through an external image capture device has been developed, it is quite inconvenient to check the image while driving.
따라서 탑승자의 안전을 지키면서 더 효율적으로 전차 외부의 상황을 관측하고, 외부 관측 및 전차 운용을 동시에 쉽게 할 수 있는 방법의 개발이 필요한 실정이다.Therefore, it is necessary to develop a method that can more efficiently observe the situation outside the tank while maintaining the safety of passengers, and easily perform external observation and tank operation at the same time.
한편, 본 발명과 관련된 선행기술로서, 등록특허 제10-1545336호(발명의 명칭: 전차 사격 훈련용 무선 영상 및 조종데이터 전송장치, 등록일자: 2015년 08월 11일) 등이 개시된 바 있다.On the other hand, as a prior art related to the present invention, Patent Registration No. 10-1545336 (title of invention: wireless video and control data transmission device for tank shooting training, registration date: August 11, 2015) has been disclosed.
본 발명은 기존에 제안된 방법들의 상기와 같은 문제점들을 해결하기 위해 제안된 것으로서, 전차의 외부에 장착된 카메라 장치가 전방위 촬영을 하고 영상 처리 장치가 촬영된 영상을 처리해 외부 영상 정보를 생성하여, 전차 내부에 탑승한 조종수가 착용한 증강현실 HMD 장치를 통해 증강현실로 제공함으로써, 전차 외부 시야 확보를 위해 몸을 외부로 노출하는 위험 없이 전차 내부에서 외부의 시야를 확보할 수 있고, 외부 모습이 조종수 시야를 통해 증강현실로 제공되므로 운전하면서 외부 상황을 쉽고 효율적으로 파악할 수 있으며, 조종수의 머리 움직임에 따라 외부 영상이 전시되어 전차 조종에 적합하고 직관적이며, 외부 관측을 위한 별도의 인원이 필요 없는, 증강현실 HMD를 이용한 전차용 전방위 외부 관측 시스템을 제공하는 것을 그 목적으로 한다.The present invention has been proposed to solve the above problems of the previously proposed methods, and the camera device mounted on the outside of the tram takes omnidirectional shots and the image processing device processes the captured images to generate external image information, By providing augmented reality through the augmented reality HMD device worn by the driver inside the tank, it is possible to secure the outside view from inside the tank without the risk of exposing the body to the outside to secure the outside view of the tank, and the external appearance can be Since it is provided in augmented reality through the driver's field of view, it is possible to grasp the external situation easily and efficiently while driving, and the external image is displayed according to the driver's head movement, which is suitable and intuitive for tank control, and does not require a separate person for external observation. The purpose of this study is to provide an omnidirectional external observation system for tanks using an augmented reality HMD.
상기한 목적을 달성하기 위한 본 발명의 특징에 따른 증강현실 HMD를 이용한 전차용 전방위 외부 관측 시스템은,An omnidirectional external observation system for tanks using an augmented reality HMD according to the features of the present invention for achieving the above object,
전차 내부에 탑승한 조종수가 상기 전차 내부에서 전차의 외부를 관측할 수 있도록 하는, 전차용 전방위 외부 관측 시스템으로서,An omnidirectional external observation system for a tank that allows the driver inside the tank to observe the outside of the tank from the inside of the tank,
전차의 외부에 장착되며 전방위 촬영이 가능한 카메라 장치;A camera device mounted on the outside of the tank and capable of shooting in all directions;
상기 카메라 장치에서 촬영된 전차 외부의 영상으로부터 증강현실로 출력될 외부 영상 정보를 생성하는 영상 처리 장치; 및an image processing device that generates external image information to be output to augmented reality from an external image of the tram captured by the camera device; and
전차 내부에 탑승한 조종수의 머리에 착용되며, 상기 영상 처리 장치에서 생성한 외부 영상 정보를 영상 광으로 출력해, 투과되는 실제 세계(real world)의 광과 상기 영상 광의 결합에 의한 증강현실을 제공하는 증강현실 HMD 장치를 포함하며,It is worn on the head of the driver riding inside the tank, outputs the external image information generated by the image processing device as image light, and provides augmented reality by combining the transmitted light of the real world and the image light. It includes an augmented reality HMD device that
상기 증강현실 HMD 장치는,The augmented reality HMD device,
상기 조종수가 머리에 착용할 수 있는 HMD 프레임;an HMD frame that the driver can wear on his head;
상기 영상 처리 장치에서 생성한 외부 영상 정보가 조종수에게 제공될 수 있도록 영상 광을 출력하는 디스플레이부;a display unit outputting image light so that the external image information generated by the image processing device can be provided to a driver;
상기 HMD 프레임을 착용한 조종수의 양안의 전방에 배치되며, 조종수의 시야를 통한 실제 세계의 광의 적어도 일부를 투과시키고, 상기 디스플레이부에서 출력되는 영상 광을 상기 조종수의 눈 방향으로 전달하는 광학부;an optical unit disposed in front of both eyes of the driver wearing the HMD frame, transmitting at least a part of the light of the real world through the driver's field of view, and transmitting the image light output from the display unit to the driver's eye direction;
상기 HMD 프레임의 일측에 설치되며, IMU 센서를 포함해 상기 HMD 프레임을 착용한 조종수의 머리 움직임을 감지하는 센서부; 및a sensor unit installed on one side of the HMD frame and detecting a head movement of a driver wearing the HMD frame, including an IMU sensor; and
상기 영상 처리 장치에서 생성한 외부 영상 정보가 상기 디스플레이부로 출력되도록 전송 제어하되, 상기 센서부에서 감지된 조종수의 머리 움직임에 따라 상기 조종수가 바라보는 방향의 외부 영상이 전시되도록 제어하는 영상 제어부를 포함하는 것을 그 구성상의 특징으로 한다.An image control unit configured to transmit and control external image information generated by the image processing device to be output to the display unit, and to display an external image in a direction the driver is looking in accordance with the movement of the driver's head detected by the sensor unit. It is characterized by its composition.
바람직하게는, 상기 카메라 장치는,Preferably, the camera device,
전방위 촬영이 가능하도록 서로 다른 방향을 향하는 복수의 카메라로 구성될 수 있다.It may be composed of a plurality of cameras pointing in different directions to enable omnidirectional shooting.
더욱 바람직하게는, 상기 영상 처리 장치는,More preferably, the image processing device,
상기 카메라 장치를 구성하는 복수의 카메라에서 촬영된 영상을 정합하여 360도 전방위 영상을 생성할 수 있다.A 360-degree omnidirectional image may be generated by matching images captured by a plurality of cameras constituting the camera device.
더욱 바람직하게는, 상기 카메라 장치는,More preferably, the camera device,
상기 전차의 외부에 장착되며, 원반 또는 원기둥 형태의 몸체; 및Mounted on the outside of the tank, a disk or cylindrical body; and
90도 이상 150도 이하 각도로 배치된 한 쌍의 카메라로 구성되는 카메라 모듈을 포함하며,It includes a camera module composed of a pair of cameras arranged at an angle of 90 degrees or more and 150 degrees or less,
상기 몸체를 둘러싸며 4 이상의 방향으로 각 방향마다 상기 카메라 모듈이 배치될 수 있다.The camera module may be disposed in each direction in four or more directions while surrounding the body.
더더욱 바람직하게는, 상기 몸체는,Even more preferably, the body,
상기 전차의 외부에 회전 가능하도록 장착되며, 상기 조종수에 의해 회전 제어될 수 있다.It is rotatably mounted on the outside of the tank and can be controlled by the driver.
더더더욱 바람직하게는, 상기 카메라 장치는,Even more preferably, the camera device,
위쪽으로 배치된 상부 카메라를 더 포함하며, 상기 전차의 외부에 탈착 가능하게 장착될 수 있다.It further includes an upper camera disposed upward, and can be detachably mounted on the outside of the tank.
바람직하게는, 상기 증강현실 HMD 장치는,Preferably, the augmented reality HMD device,
상기 영상 처리 장치, 외부 서버 및 다른 증강현실 HMD 장치와 통신을 수행하는 통신부를 더 포함하며,Further comprising a communication unit that communicates with the image processing device, external server, and other augmented reality HMD devices,
상기 영상 제어부는,The video control unit,
상기 통신부를 통해 외부 서버 또는 다른 증강현실 HMD 장치로부터 수신한 데이터가 출력되어 상기 조종수의 시야를 통해 증강현실로 제공되도록 제어할 수 있다.Data received from an external server or other augmented reality HMD device through the communication unit may be output and provided as augmented reality through the driver's field of view.
바람직하게는, 상기 영상 처리 장치는,Preferably, the image processing device,
인공지능 기반의 딥러닝 모델을 사용해 영상을 정합하여 360도 전방위 영상을 생성할 수 있다.A 360-degree omnidirectional image can be created by matching images using an artificial intelligence-based deep learning model.
본 발명에서 제안하고 있는 증강현실 HMD를 이용한 전차용 전방위 외부 관측 시스템에 따르면, 전차의 외부에 장착된 카메라 장치가 전방위 촬영을 하고 영상 처리 장치가 촬영된 영상을 처리해 외부 영상 정보를 생성하여, 전차 내부에 탑승한 조종수가 착용한 증강현실 HMD 장치를 통해 증강현실로 제공함으로써, 전차 외부 시야 확보를 위해 몸을 외부로 노출하는 위험 없이 전차 내부에서 외부의 시야를 확보할 수 있고, 외부 모습이 조종수 시야를 통해 증강현실로 제공되므로 운전하면서 외부 상황을 쉽고 효율적으로 파악할 수 있으며, 조종수의 머리 움직임에 따라 외부 영상이 전시되어 전차 조종에 적합하고 직관적이며, 외부 관측을 위한 별도의 인원이 필요 없다.According to the omnidirectional external observation system for a tank using an augmented reality HMD proposed in the present invention, a camera device mounted on the outside of a tank performs omnidirectional shooting and an image processing device processes the captured image to generate external image information, By providing augmented reality through the augmented reality HMD device worn by the driver inside the vehicle, it is possible to secure the outside view from inside the tank without the risk of exposing the body to the outside to secure the outside view of the tank, and the external appearance can be seen by the driver Since it is provided in augmented reality through the field of view, it is possible to grasp the external situation easily and efficiently while driving, and the external image is displayed according to the driver's head movement, making it suitable and intuitive for chariot control and does not require a separate person for external observation.
도 1은 본 발명의 일실시예에 따른 증강현실 HMD를 이용한 전차용 전방위 외부 관측 시스템의 전체 구성을 도시한 도면.1 is a view showing the overall configuration of an omnidirectional external observation system for a tank using an augmented reality HMD according to an embodiment of the present invention.
도 2는 본 발명의 일실시예에 따른 증강현실 HMD를 이용한 전차용 전방위 외부 관측 시스템의 카메라 장치가 장착된 전차의 모습을 예를 들어 도시한 도면.2 is a view showing, for example, a state of a tank equipped with a camera device of an omnidirectional external observation system for a tank using an augmented reality HMD according to an embodiment of the present invention.
도 3은 본 발명의 일실시예에 따른 증강현실 HMD를 이용한 전차용 전방위 외부 관측 시스템에서, 카메라 장치를 예를 들어 도시한 도면.3 is a view showing a camera device as an example in an omnidirectional external observation system for a tank using an augmented reality HMD according to an embodiment of the present invention.
도 4는 본 발명의 일실시예에 따른 증강현실 HMD를 이용한 전차용 전방위 외부 관측 시스템이 구축된 전차 내부 모습을 예를 들어 도시한 도면.4 is a view showing, for example, the inside of a tank in which an omnidirectional external observation system for a tank using an augmented reality HMD according to an embodiment of the present invention is built.
도 5는 본 발명의 일실시예에 따른 증강현실 HMD를 이용한 전차용 전방위 외부 관측 시스템에서, 증강현실 HMD 장치의 구성을 도시한 도면.5 is a view showing the configuration of an augmented reality HMD device in an omnidirectional external observation system for a tank using an augmented reality HMD according to an embodiment of the present invention.
<부호의 설명><Description of codes>
10: 본 발명의 특징에 따른 전차용 전방위 외부 관측 시스템10: omnidirectional external observation system for tanks according to the characteristics of the present invention
100: 카메라 장치100: camera device
110: 몸체110: body
120: 카메라 모듈120: camera module
130: 상부 카메라130: upper camera
200: 영상 처리 장치200: image processing device
300: 증강현실 HMD 장치300: augmented reality HMD device
310: HMD 프레임310: HMD frame
320: 디스플레이부320: display unit
330: 광학부330: optics
340: 센서부340: sensor unit
350: 영상 제어부350: video controller
360: 통신부360: communication department
이하, 첨부된 도면을 참조하여 본 발명이 속하는 기술분야에서 통상의 지식을 가진 자가 본 발명을 용이하게 실시할 수 있도록 바람직한 실시예를 상세히 설명한다. 다만, 본 발명의 바람직한 실시예를 상세하게 설명함에 있어, 관련된 공지 기능 또는 구성에 대한 구체적인 설명이 본 발명의 요지를 불필요하게 흐릴 수 있다고 판단되는 경우에는 그 상세한 설명을 생략한다. 또한, 유사한 기능 및 작용을 하는 부분에 대해서는 도면 전체에 걸쳐 동일한 부호를 사용한다.Hereinafter, preferred embodiments will be described in detail so that those skilled in the art can easily practice the present invention with reference to the accompanying drawings. However, in describing a preferred embodiment of the present invention in detail, if it is determined that a detailed description of a related known function or configuration may unnecessarily obscure the gist of the present invention, the detailed description will be omitted. In addition, the same reference numerals are used throughout the drawings for parts having similar functions and actions.
덧붙여, 명세서 전체에서, 어떤 부분이 다른 부분과 ‘연결’ 되어 있다고 할 때, 이는 ‘직접적으로 연결’ 되어 있는 경우뿐만 아니라, 그 중간에 다른 소자를 사이에 두고 ‘간접적으로 연결’ 되어 있는 경우도 포함한다. 또한, 어떤 구성요소를 ‘포함’ 한다는 것은, 특별히 반대되는 기재가 없는 한 다른 구성요소를 제외하는 것이 아니라 다른 구성요소를 더 포함할 수 있다는 것을 의미한다.In addition, throughout the specification, when a part is said to be 'connected' to another part, this is not only the case where it is 'directly connected', but also the case where it is 'indirectly connected' with another element in between. include In addition, 'including' a certain component means that other components may be further included, rather than excluding other components unless otherwise specified.
도 1은 본 발명의 일실시예에 따른 증강현실 HMD를 이용한 전차용 전방위 외부 관측 시스템(10)의 전체 구성을 도시한 도면이다. 도 1에 도시된 바와 같이, 본 발명의 일실시예에 따른 증강현실 HMD를 이용한 전차용 전방위 외부 관측 시스템(10)은, 전차 내부에 탑승한 조종수가 전차 내부에서 전차의 외부를 관측할 수 있도록 하는 전차용 전방위 외부 관측 시스템(10)으로서, 카메라 장치(100), 영상 처리 장치(200) 및 증강현실 HMD 장치(300)를 포함하여 구성될 수 있다.1 is a diagram showing the overall configuration of an omnidirectional external observation system 10 for a tank using an augmented reality HMD according to an embodiment of the present invention. As shown in FIG. 1, the omnidirectional external observation system 10 for a tank using an augmented reality HMD according to an embodiment of the present invention allows a driver riding inside the tank to observe the outside of the tank from inside the tank. As an omnidirectional external observation system 10 for a tram, it may be configured to include a camera device 100, an image processing device 200, and an augmented reality HMD device 300.
카메라 장치(100)는, 전차의 외부에 장착되며 전방위 촬영이 가능할 수 있다. 보다 구체적으로, 카메라 장치(100)는, 전방위 촬영이 가능하도록 서로 다른 방향을 향하는 복수의 카메라로 구성될 수 있으며, 일반적인 카메라 이외에 열 영상 카메라, 야간 투시 카메라 등을 포함할 수 있다.The camera device 100 is mounted on the outside of the train and may be able to take pictures in all directions. More specifically, the camera device 100 may include a plurality of cameras pointing in different directions to enable omnidirectional photography, and may include a thermal imaging camera, a night vision camera, and the like in addition to a general camera.
도 2는 본 발명의 일실시예에 따른 증강현실 HMD를 이용한 전차용 전방위 외부 관측 시스템(10)의 카메라 장치(100)가 장착된 전차의 모습을 예를 들어 도시한 도면이다. 도 2에 도시된 바와 같이, 본 발명의 일실시예에 따른 증강현실 HMD를 이용한 전차용 전방위 외부 관측 시스템(10)의 카메라 장치(100)는, 외부의 모습을 가장 잘 관측할 수 있는 전차의 외부 상측에 설치될 수 있고, 실시예에 따라서는 전차의 외부에 탈착 가능하게 장착될 수 있다. 카메라 장치(100)가 설치되는 구체적인 위치는 전차의 형태나 구조, 작전 환경 등에 따라 상이할 수 있다.FIG. 2 is a diagram showing, for example, a state of a tank equipped with a camera device 100 of an omnidirectional external observation system 10 for a tank using an augmented reality HMD according to an embodiment of the present invention. As shown in FIG. 2, the camera device 100 of the omnidirectional external observation system 10 for a tank using an augmented reality HMD according to an embodiment of the present invention is a vehicle that can best observe the outside. It can be installed on the outer upper side and, depending on the embodiment, can be detachably mounted on the outside of the tank. A specific location where the camera device 100 is installed may differ depending on the shape or structure of a tank, operational environment, and the like.
도 3은 본 발명의 일실시예에 따른 증강현실 HMD를 이용한 전차용 전방위 외부 관측 시스템(10)에서, 카메라 장치(100)를 예를 들어 도시한 도면이다. 도 3에 도시된 바와 같이, 본 발명의 일실시예에 따른 증강현실 HMD를 이용한 전차용 전방위 외부 관측 시스템(10)의 카메라 장치(100)는, 전차의 외부에 장착되며, 원반 또는 원기둥 형태의 몸체(110), 및 90도 이상 150도 이하 각도로 배치된 한 쌍의 카메라로 구성되는 카메라 모듈(120)을 포함하여 구성될 수 있으며, 몸체(110)를 둘러싸며 4 이상의 방향으로 각 방향마다 한 쌍의 카메라 모듈(120)이 배치될 수 있다. 보다 구체적으로, 카메라 모듈(120)은 8개 방향으로 일정한 간격을 두고 배치되며, 한 쌍의 카메라가 120도 각도로 배치되도록 각각의 카메라 모듈(120)을 구성하여 사각(死角)이 발생하지 않도록 할 수 있다.FIG. 3 is a view showing the camera device 100 as an example in the omnidirectional external observation system 10 for a tank using an augmented reality HMD according to an embodiment of the present invention. As shown in FIG. 3, the camera device 100 of the omnidirectional external observation system 10 for a tank using an augmented reality HMD according to an embodiment of the present invention is mounted on the outside of a tank and has a disk or column shape. It may be configured to include a body 110, and a camera module 120 composed of a pair of cameras disposed at an angle of 90 degrees or more and 150 degrees or less, surrounding the body 110 and in each direction in four or more directions. A pair of camera modules 120 may be disposed. More specifically, the camera modules 120 are arranged at regular intervals in eight directions, and each camera module 120 is configured so that a pair of cameras are disposed at an angle of 120 degrees so that no blind spots occur. can do.
또한, 몸체(110)는, 전차의 외부에 회전 가능하도록 장착되며, 조종수에 의해 회전 제어될 수 있다. 따라서 일부 카메라 모듈(120)이 파괴, 고장 등 손상되었을 때 손상되지 않은 카메라 모듈(120)이 관측이 필요한 방향으로 향하도록 회전 제어하여 시야를 확보할 수 있다.In addition, the body 110 is rotatably mounted on the outside of the tank and can be rotated and controlled by a driver. Therefore, when some of the camera modules 120 are damaged, such as destruction or breakdown, the undamaged camera module 120 is rotated in a direction required for observation to secure a field of view.
또한, 카메라 장치(100)는, 위쪽으로 배치된 상부 카메라(130)를 더 포함하여, 측면뿐만 아니라 하늘 방향도 관측하도록 할 수 있다.In addition, the camera device 100 may further include an upper camera 130 disposed upward to observe not only the side surface but also the sky direction.
영상 처리 장치(200)는, 카메라 장치(100)에서 촬영된 전차 외부의 영상으로부터 증강현실로 출력될 외부 영상 정보를 생성할 수 있다. 보다 구체적으로, 영상 처리 장치(200)는, 카메라 장치(100)를 구성하는 복수의 카메라에서 촬영된 영상을 정합하여 360도 전방위 영상을 생성할 수 있다. 즉, 영상 처리 장치(200)는, 영상 정합(Image Registration) 기술을 사용해 여러 카메라에서 촬영된 영상을 결합하고, 360도 전방위의 하나의 외부 영상 정보로 구성할 수 있다.The image processing device 200 may generate external image information to be output in augmented reality from an image of the exterior of the train captured by the camera device 100 . More specifically, the image processing device 200 may generate a 360-degree omnidirectional image by matching images captured by a plurality of cameras constituting the camera device 100 . That is, the image processing device 200 may combine images captured by multiple cameras using image registration technology and configure a single 360-degree external image information.
실시예에 따라서, 영상 처리 장치(200)는, 인공지능 기반의 딥러닝 모델을 사용해 영상을 정합하여 360도 전방위 영상을 생성할 수 있다. 이때, 영상 정합을 위해 영상 내에서 중요한 의미가 있는 키-포인트(key-point)를 추출하고, 이를 기반으로 영상을 매칭하는 키-포인트 매칭 알고리즘을 사용할 수 있다. 키-포인트 매칭을 통해 서로 다른 좌표계나 시점에서 촬영된 영상을 정합할 수 있는데, 영상 처리 장치(200)는 키-포인트 매칭을 위해 SIFT, SURF, KAZE, AKAZE, ORB 중 어느 하나의 알고리즘을 사용할 수 있다. 보다 구체적으로, 영상 처리 장치(200)는 실시간으로 영상을 정합해 외부 영상 정보를 생성해야 하므로, 여러 키-포인트 매칭 알고리즘 중 가장 속도가 빠른 ORB를 사용할 수 있다.According to an embodiment, the image processing device 200 may generate a 360-degree omnidirectional image by matching images using an artificial intelligence-based deep learning model. In this case, a key-point matching algorithm that extracts a key-point having an important meaning from an image for image matching and matches the image based on the key-point may be used. Images captured in different coordinate systems or viewpoints may be matched through key-point matching. The image processing device 200 may use any one algorithm among SIFT, SURF, KAZE, AKAZE, and ORB for key-point matching. can More specifically, since the image processing device 200 needs to match images in real time to generate external image information, an ORB that is the fastest among several key-point matching algorithms can be used.
증강현실 HMD 장치(300)는, 전차 내부에 탑승한 조종수의 머리에 착용되며, 영상 처리 장치(200)에서 생성한 외부 영상 정보를 영상 광으로 출력해, 투과되는 실제 세계(real world)의 광과 영상 광의 결합에 의한 증강현실을 제공할 수 있다.The augmented reality HMD device 300 is worn on the driver's head inside the train, outputs external image information generated by the image processing device 200 as image light, and transmits light of the real world. It is possible to provide augmented reality by the combination of and image light.
도 4는 본 발명의 일실시예에 따른 증강현실 HMD를 이용한 전차용 전방위 외부 관측 시스템(10)이 구축된 전차 내부 모습을 예를 들어 도시한 도면이다. 도 4에 도시된 바와 같이, 본 발명의 일실시예에 따른 증강현실 HMD를 이용한 전차용 전방위 외부 관측 시스템(10)에 따르면, 전차 내부에 탑승한 조종수가 착용한 증강현실 HMD 장치(300)를 통해 영상 처리 장치(200)에서 생성한 외부 영상 정보가 증강현실로 제공되므로, 조종수는 별도로 마련된 화면 등을 볼 필요 없이 전차 운전에 집중하면서 시야의 외부 영상 정보를 통해 외부를 한눈에 확인할 수 있다.4 is a view showing, for example, the interior of a train in which the omnidirectional external observation system 10 for a tank using an augmented reality HMD according to an embodiment of the present invention is built. As shown in FIG. 4, according to the omnidirectional external observation system 10 for a tram using an augmented reality HMD according to an embodiment of the present invention, the augmented reality HMD device 300 worn by the driver riding inside the tram Since the external image information generated by the image processing device 200 is provided in augmented reality, the driver can check the outside at a glance through the external image information of the field of view while concentrating on driving the tram without having to look at a separately prepared screen.
도 5는 본 발명의 일실시예에 따른 증강현실 HMD를 이용한 전차용 전방위 외부 관측 시스템(10)에서, 증강현실 HMD 장치(300)의 구성을 도시한 도면이다. 도 5에 도시된 바와 같이, 본 발명의 일실시예에 따른 증강현실 HMD를 이용한 전차용 전방위 외부 관측 시스템(10)은, HMD 프레임(310), 디스플레이부(320), 광학부(330), 센서부(340) 및 영상 제어부(350)를 포함하여 구성될 수 있으며, 통신부(360)를 더 포함하여 구성될 수 있다.5 is a diagram showing the configuration of the augmented reality HMD device 300 in the omnidirectional external observation system 10 for electric vehicles using the augmented reality HMD according to an embodiment of the present invention. As shown in FIG. 5, the omnidirectional external observation system 10 for a tank using an augmented reality HMD according to an embodiment of the present invention includes an HMD frame 310, a display unit 320, an optical unit 330, It may be configured to include a sensor unit 340 and an image controller 350, and may be configured to further include a communication unit 360.
HMD 프레임(310)은, 조종수가 머리에 착용할 수 있는 프레임 구성이다. 이러한 HMD 프레임(310)은, 조종수가 머리에 착용한 상태에서 빛이 들어올 수 있는 프레임 구조를 갖는 헬멧(helmet) 형태 또는 고글(goggles) 형태로 구성될 수 있다. 다만, 전차에 탑승하는 조종수를 위한 것임을 고려할 때, 방탄 기능이 있는 헬멧 형태로 구성하는 것이 바람직하며, 조종수의 머리에 착용되는 방탄모(미도시)와 방탄모의 전방에 배치되는 디스플레이 프레임(미도시) 구조로 구성될 수 있다.The HMD frame 310 is a frame structure that can be worn on the head by a driver. The HMD frame 310 may be configured in the form of a helmet or goggles having a frame structure through which light may enter while worn on the driver's head. However, considering that it is for the driver riding the tank, it is preferable to configure it in the form of a bulletproof helmet, and a bulletproof helmet (not shown) worn on the driver's head and a display frame (not shown) placed in front of the bulletproof helmet structure can be made.
특히, 방탄모는 착용자 머리를 충분히 보호할 수 있는 단단한 재질로 구성되며, 개방된 내부 공간에는 완충 작용을 통해 착용자의 머리를 보호할 수 있는 충격완화 부재가 포함될 수 있다. 충격완화 부재는 항균 및 탈취 기능이 있는 재질일 수 있으며, 항균 및 탈취 코팅이 된 것일 수도 있다. 그밖에, 머리받이 끈 및 턱 끈 등이 더 포함될 수 있다.In particular, the bulletproof helmet is made of a hard material capable of sufficiently protecting the wearer's head, and an impact relieving member capable of protecting the wearer's head through a buffering action may be included in the open interior space. The shock absorbing member may be made of a material having antibacterial and deodorizing functions, or may be coated with antibacterial and deodorizing functions. In addition, a headrest strap and a chin strap may be further included.
디스플레이부(320)는, 영상 처리 장치(200)에서 생성한 외부 영상 정보가 조종수에게 제공될 수 있도록 영상 광을 출력할 수 있다. 디스플레이(131)는 OLED 등 다양하게 구성될 수 있다.The display unit 320 may output image light so that external image information generated by the image processing device 200 may be provided to a driver. The display 131 may be configured in various ways such as OLED.
광학부(330)는, HMD 프레임(310)을 착용한 조종수의 양안의 전방에 배치되며, 조종수의 시야를 통한 실제 세계의 광의 적어도 일부를 투과시키고, 디스플레이부(320)에서 출력되는 영상 광을 조종수의 눈 방향으로 전달하여, 착용한 조종수가 증강현실을 경험할 수 있도록 구성될 수 있다. 광학부(330)는 복수의 렌즈와 미러 등으로 구성되며 다양한 방식으로 구현될 수 있는데, 예를 들어 광학 회절 방식, 빔 스플리터 방식, 핀 미러 방식 등으로 구현될 수 있다.The optical unit 330 is disposed in front of both eyes of the pilot wearing the HMD frame 310, transmits at least a part of the light of the real world through the pilot's field of view, and transmits image light output from the display unit 320. It can be configured so that the wearer can experience augmented reality by transmitting it in the direction of the driver's eyes. The optical unit 330 is composed of a plurality of lenses and mirrors and may be implemented in various ways, such as an optical diffraction method, a beam splitter method, a pin mirror method, and the like.
센서부(340)는, HMD 프레임(310)의 일측에 설치되며, IMU 센서를 포함해 HMD 프레임(310)을 착용한 조종수의 머리 움직임을 감지할 수 있다. 여기서, IMU 센서는 조종수의 머리의 이동 방향, 속도, 각도 등을 감지해 조종수가 바라보는 방향을 정확하게 분석할 수 있다. 또한, 센서부(340)는, IMU 센서 외에도, 조도 센서, 화학 센서 등을 더 포함할 수 있다. 센서부(340)에서 수집한 센싱 데이터는 영상 제어부(350)가 증강현실 영상 정보를 제공하기 위해 사용할 수 있다.The sensor unit 340 is installed on one side of the HMD frame 310 and can detect head movements of a driver wearing the HMD frame 310 including an IMU sensor. Here, the IMU sensor can accurately analyze the direction the driver is looking by detecting the moving direction, speed, and angle of the driver's head. In addition, the sensor unit 340 may further include an illuminance sensor, a chemical sensor, and the like, in addition to the IMU sensor. The sensing data collected by the sensor unit 340 may be used by the image controller 350 to provide augmented reality image information.
영상 제어부(350)는, 영상 처리 장치(200)에서 생성한 외부 영상 정보가 디스플레이부(320)로 출력되도록 전송 제어하되, 센서부(340)에서 감지된 조종수의 머리 움직임에 따라 조종수가 바라보는 방향의 외부 영상이 전시되도록 제어할 수 있다. 예를 들어, 조종수가 머리를 왼쪽이나 오른쪽으로 향하면, 머리 움직임에 따라 외부 영상 정보가 회전하여 조종수가 바라보는 왼쪽이나 오른쪽의 외부 영상이 증강현실로 전시될 수 있다.The image controller 350 transmits and controls the external image information generated by the image processing device 200 to be output to the display unit 320, and controls the transmission of the external image information generated by the image processing device 200 according to the driver's head movement detected by the sensor unit 340. It can be controlled so that the external image of the direction is displayed. For example, if the driver turns his head to the left or right, the external image information rotates according to the movement of the head, and the left or right external image viewed by the driver may be displayed in augmented reality.
통신부(360)는, 영상 처리 장치(200), 외부 서버 및 다른 증강현실 HMD 장치(300)와 통신을 수행할 수 있다. 영상 제어부(350)는, 통신부(360)를 통해 외부 서버 또는 다른 증강현실 HMD 장치(300)로부터 수신한 데이터가 출력되어 조종수의 시야를 통해 증강현실로 제공되도록 제어할 수 있다. 이와 같이 통신부(360)와 영상 제어부(350)의 상호작용을 통해, 증강현실로 외부 상황, 지시 사항, 명령 등을 조종수에게 전달할 수 있다. 또한, 텍스트, 이미지, 색상, 아이콘 등을 이용해 상황을 직관적으로 전달할 수도 있다.The communication unit 360 may communicate with the image processing device 200, an external server, and other augmented reality HMD devices 300. The image controller 350 may control data received from an external server or other augmented reality HMD device 300 through the communication unit 360 to be output and provided to augmented reality through the driver's field of view. In this way, through the interaction between the communication unit 360 and the image controller 350, an external situation, instructions, commands, and the like may be transmitted to the operator through augmented reality. In addition, the situation can be conveyed intuitively using text, images, colors, and icons.
여기서, 통신부(360)는 증강현실 HMD 장치(300)의 일측에 설치되며, 통신부(360)가 사용하는 네트워크는 근거리 통신망(Local Area Network; LAN), 광역 통신망(Wide Area Network; WAN), 이동 통신망(mobile radio communication network), 위성 통신망, 블루투스(Bluetooth), Wibro(Wireless Broadband Internet), HSDPA(High Speed Downlink Packet Access), LTE(Long Term Evolution), 3/4/5G(3/4/5th Generation Mobile Telecommunication) 등과 같은 모든 종류의 유무선 네트워크로 구현될 수 있다.Here, the communication unit 360 is installed on one side of the augmented reality HMD device 300, and the network used by the communication unit 360 includes a local area network (LAN), a wide area network (WAN), and a mobile network. Mobile radio communication network, satellite communication network, Bluetooth, Wibro (Wireless Broadband Internet), HSDPA (High Speed Downlink Packet Access), LTE (Long Term Evolution), 3/4/5G (3/4/5th Generation Mobile Telecommunication) can be implemented in all types of wired and wireless networks.
전술한 바와 같이, 본 발명에서 제안하고 있는 증강현실 HMD를 이용한 전차용 전방위 외부 관측 시스템(10)에 따르면, 전차의 외부에 장착된 카메라 장치(100)가 전방위 촬영을 하고 영상 처리 장치(200)가 촬영된 영상을 처리해 외부 영상 정보를 생성하여, 전차 내부에 탑승한 조종수가 착용한 증강현실 HMD 장치(300)를 통해 증강현실로 제공함으로써, 전차 외부 시야 확보를 위해 몸을 외부로 노출하는 위험 없이 전차 내부에서 외부의 시야를 확보할 수 있고, 외부 모습이 조종수 시야를 통해 증강현실로 제공되므로 운전하면서 외부 상황을 쉽고 효율적으로 파악할 수 있으며, 조종수의 머리 움직임에 따라 외부 영상이 전시되어 전차 조종에 적합하고 직관적이며, 외부 관측을 위한 별도의 인원이 필요 없다.As described above, according to the omnidirectional external observation system 10 for a tank using the augmented reality HMD proposed in the present invention, the camera device 100 mounted on the outside of the tank performs omnidirectional photography and the image processing device 200 Risk of exposing the body to the outside to secure the external view of the tank Since the outside view is provided in augmented reality through the driver's field of view, the external situation can be grasped easily and efficiently while driving, and the external image is displayed according to the driver's head movement to control the tank. suitable for and intuitive, and does not require a separate person for external observation.
이상 설명한 본 발명은 본 발명이 속한 기술분야에서 통상의 지식을 가진 자에 의하여 다양한 변형이나 응용이 가능하며, 본 발명에 따른 기술적 사상의 범위는 아래의 특허청구범위에 의하여 정해져야 할 것이다.The present invention described above can be variously modified or applied by those skilled in the art to which the present invention belongs, and the scope of the technical idea according to the present invention should be defined by the claims below.

Claims (8)

  1. 전차 내부에 탑승한 조종수가 상기 전차 내부에서 전차의 외부를 관측할 수 있도록 하는, 전차용 전방위 외부 관측 시스템(10)으로서,As an omnidirectional external observation system 10 for a tank, which allows the driver riding inside the tank to observe the outside of the tank from the inside of the tank,
    전차의 외부에 장착되며 전방위 촬영이 가능한 카메라 장치(100);A camera device 100 mounted on the outside of the tank and capable of omnidirectional shooting;
    상기 카메라 장치(100)에서 촬영된 전차 외부의 영상으로부터 증강현실로 출력될 외부 영상 정보를 생성하는 영상 처리 장치(200); 및an image processing device 200 for generating external image information to be output to augmented reality from the image of the outside of the train captured by the camera device 100; and
    전차 내부에 탑승한 조종수의 머리에 착용되며, 상기 영상 처리 장치(200)에서 생성한 외부 영상 정보를 영상 광으로 출력해, 투과되는 실제 세계(real world)의 광과 상기 영상 광의 결합에 의한 증강현실을 제공하는 증강현실 HMD 장치(300)를 포함하며,It is worn on the head of the driver riding inside the train, outputs the external image information generated by the image processing device 200 as image light, and augments by combining the transmitted light of the real world and the image light. It includes an augmented reality HMD device 300 that provides reality,
    상기 증강현실 HMD 장치(300)는,The augmented reality HMD device 300,
    상기 조종수가 머리에 착용할 수 있는 HMD 프레임(310);an HMD frame 310 that the driver can wear on his head;
    상기 영상 처리 장치(200)에서 생성한 외부 영상 정보가 조종수에게 제공될 수 있도록 영상 광을 출력하는 디스플레이부(320);a display unit 320 that outputs image light so that external image information generated by the image processing device 200 can be provided to a driver;
    상기 HMD 프레임(310)을 착용한 조종수의 양안의 전방에 배치되며, 조종수의 시야를 통한 실제 세계의 광의 적어도 일부를 투과시키고, 상기 디스플레이부(320)에서 출력되는 영상 광을 상기 조종수의 눈 방향으로 전달하는 광학부(330);It is disposed in front of both eyes of the driver wearing the HMD frame 310, transmits at least a part of the light of the real world through the driver's field of view, and transmits the image light output from the display unit 320 in the direction of the driver's eyes. Optical unit 330 to transmit to;
    상기 HMD 프레임(310)의 일측에 설치되며, IMU 센서를 포함해 상기 HMD 프레임(310)을 착용한 조종수의 머리 움직임을 감지하는 센서부(340); 및a sensor unit 340 installed on one side of the HMD frame 310 and including an IMU sensor to detect head movement of a pilot wearing the HMD frame 310; and
    상기 영상 처리 장치(200)에서 생성한 외부 영상 정보가 상기 디스플레이부(320)로 출력되도록 전송 제어하되, 상기 센서부(340)에서 감지된 조종수의 머리 움직임에 따라 상기 조종수가 바라보는 방향의 외부 영상이 전시되도록 제어하는 영상 제어부(350)를 포함하는 것을 특징으로 하는, 증강현실 HMD를 이용한 전차용 전방위 외부 관측 시스템(10).The external image information generated by the image processing device 200 is transmitted and controlled to be output to the display unit 320, and the outside of the driver's direction is viewed according to the driver's head movement detected by the sensor unit 340. Characterized in that it includes an image control unit 350 for controlling the image to be displayed, an omnidirectional external observation system for a tank using an augmented reality HMD (10).
  2. 제1항에 있어서, 상기 카메라 장치(100)는,The method of claim 1, wherein the camera device 100,
    전방위 촬영이 가능하도록 서로 다른 방향을 향하는 복수의 카메라로 구성되는 것을 특징으로 하는, 증강현실 HMD를 이용한 전차용 전방위 외부 관측 시스템(10).An omnidirectional external observation system (10) for a tank using an augmented reality HMD, characterized in that it is composed of a plurality of cameras facing in different directions to enable omnidirectional shooting.
  3. 제2항에 있어서, 상기 영상 처리 장치(200)는,The method of claim 2, wherein the image processing device 200,
    상기 카메라 장치(100)를 구성하는 복수의 카메라에서 촬영된 영상을 정합하여 360도 전방위 영상을 생성하는 것을 특징으로 하는, 증강현실 HMD를 이용한 전차용 전방위 외부 관측 시스템(10).An omnidirectional external observation system 10 for a tank using an augmented reality HMD, characterized in that by matching images taken by a plurality of cameras constituting the camera device 100 to generate a 360-degree omnidirectional image.
  4. 제2항에 있어서, 상기 카메라 장치(100)는,The method of claim 2, wherein the camera device 100,
    상기 전차의 외부에 장착되며, 원반 또는 원기둥 형태의 몸체(110); 및Mounted on the outside of the tank, the body 110 in the form of a disk or cylinder; and
    90도 이상 150도 이하 각도로 배치된 한 쌍의 카메라로 구성되는 카메라 모듈(120)을 포함하며,It includes a camera module 120 composed of a pair of cameras arranged at an angle of 90 degrees or more and 150 degrees or less,
    상기 몸체(110)를 둘러싸며 4 이상의 방향으로 각 방향마다 상기 카메라 모듈(120)이 배치된 것을 특징으로 하는, 증강현실 HMD를 이용한 전차용 전방위 외부 관측 시스템(10).Characterized in that the camera module 120 is disposed in each direction in four or more directions surrounding the body 110, an omnidirectional external observation system 10 for a tank using an augmented reality HMD.
  5. 제4항에 있어서, 상기 몸체(110)는,The method of claim 4, wherein the body 110,
    상기 전차의 외부에 회전 가능하도록 장착되며, 상기 조종수에 의해 회전 제어되는 것을 특징으로 하는, 증강현실 HMD를 이용한 전차용 전방위 외부 관측 시스템(10).An omnidirectional external observation system (10) for a tank using an augmented reality HMD, characterized in that it is rotatably mounted on the outside of the tank and the rotation is controlled by the driver.
  6. 제5항에 있어서, 상기 카메라 장치(100)는,The method of claim 5, wherein the camera device 100,
    위쪽으로 배치된 상부 카메라(130)를 더 포함하며, 상기 전차의 외부에 탈착 가능하게 장착되는 것을 특징으로 하는, 증강현실 HMD를 이용한 전차용 전방위 외부 관측 시스템(10).An omnidirectional external observation system (10) for a tank using an augmented reality HMD, further comprising an upper camera (130) disposed upward, characterized in that it is detachably mounted on the outside of the tank.
  7. 제1항에 있어서, 상기 증강현실 HMD 장치(300)는,The method of claim 1, wherein the augmented reality HMD device 300,
    상기 영상 처리 장치(200), 외부 서버 및 다른 증강현실 HMD 장치(300)와 통신을 수행하는 통신부(360)를 더 포함하며,It further includes a communication unit 360 that communicates with the image processing device 200, external servers, and other augmented reality HMD devices 300,
    상기 영상 제어부(350)는,The video controller 350,
    상기 통신부(360)를 통해 외부 서버 또는 다른 증강현실 HMD 장치(300)로부터 수신한 데이터가 출력되어 상기 조종수의 시야를 통해 증강현실로 제공되도록 제어하는 것을 특징으로 하는, 증강현실 HMD를 이용한 전차용 전방위 외부 관측 시스템(10).Characterized in that the data received from an external server or other augmented reality HMD device 300 through the communication unit 360 is output and provided as augmented reality through the driver's field of view, characterized in that for a train using an augmented reality HMD Omnidirectional external observation system (10).
  8. 제1항에 있어서, 상기 영상 처리 장치(200)는,The method of claim 1, wherein the image processing device 200,
    인공지능 기반의 딥러닝 모델을 사용해 영상을 정합하여 360도 전방위 영상을 생성하는 것을 특징으로 하는, 증강현실 HMD를 이용한 전차용 전방위 외부 관측 시스템(10).An omnidirectional external observation system (10) for a tank using an augmented reality HMD, characterized by generating a 360-degree omnidirectional image by matching images using an artificial intelligence-based deep learning model.
PCT/KR2022/005821 2021-05-14 2022-04-23 Omnidirectional external observation system for tank using augmented reality hmd WO2022240003A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
KR1020210063005A KR102495641B1 (en) 2021-05-14 2021-05-14 Omnidirectional external observation system for tanks using ar hmd
KR10-2021-0063005 2021-05-14

Publications (1)

Publication Number Publication Date
WO2022240003A1 true WO2022240003A1 (en) 2022-11-17

Family

ID=84029247

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/KR2022/005821 WO2022240003A1 (en) 2021-05-14 2022-04-23 Omnidirectional external observation system for tank using augmented reality hmd

Country Status (2)

Country Link
KR (1) KR102495641B1 (en)
WO (1) WO2022240003A1 (en)

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20110138410A (en) * 2009-04-14 2011-12-27 배 시스템즈 인포메이션 앤드 일렉트로닉 시스템즈 인티크레이션, 인크. Vehicle-mountable imaging systems and methods
KR102048354B1 (en) * 2019-07-10 2020-01-23 한화시스템(주) Apparatus and method for displaying images in a wireless helmet
KR102125299B1 (en) * 2020-02-05 2020-07-07 한화시스템 주식회사 System and method for battlefield situation recognition for combat vehicle
KR20200107395A (en) * 2019-03-07 2020-09-16 민상규 Foldable virtual reality device
KR20210011665A (en) * 2019-07-23 2021-02-02 주식회사 피앤씨솔루션 A head mounted display apparatus having an emergency navigating function with imu

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20110138410A (en) * 2009-04-14 2011-12-27 배 시스템즈 인포메이션 앤드 일렉트로닉 시스템즈 인티크레이션, 인크. Vehicle-mountable imaging systems and methods
KR20200107395A (en) * 2019-03-07 2020-09-16 민상규 Foldable virtual reality device
KR102048354B1 (en) * 2019-07-10 2020-01-23 한화시스템(주) Apparatus and method for displaying images in a wireless helmet
KR20210011665A (en) * 2019-07-23 2021-02-02 주식회사 피앤씨솔루션 A head mounted display apparatus having an emergency navigating function with imu
KR102125299B1 (en) * 2020-02-05 2020-07-07 한화시스템 주식회사 System and method for battlefield situation recognition for combat vehicle

Also Published As

Publication number Publication date
KR102495641B1 (en) 2023-02-06
KR20220155150A (en) 2022-11-22

Similar Documents

Publication Publication Date Title
EP1588206B1 (en) Helmet for displaying environmental images in critical environments
US7800043B2 (en) Night vision goggles with detachable or reattachable modular components
WO2016137146A1 (en) Mobile communication terminal having unmanned air vehicle
WO2020004966A1 (en) Smart glass display device for both augmented reality and virtual reality comprising plastic lcd shade
US20100128135A1 (en) Modular night vision goggles with pellicle
EP3134764A1 (en) Head mounted display and method for controlling the same
EP1046411B1 (en) Viewing system
WO2021020607A1 (en) Electronic device using augmented reality
WO2021118237A1 (en) Augmented reality visual field-sharing tactical training system optimized for multiple users
US20160253561A1 (en) Face mounted extreme environment thermal sensor
WO2017188740A1 (en) Virtual reality cell phone
WO2022220658A1 (en) Mixed reality industrial helmet linked with digital twin and virtual image
WO2016140384A1 (en) Helmet having camera
KR102125299B1 (en) System and method for battlefield situation recognition for combat vehicle
WO2022240003A1 (en) Omnidirectional external observation system for tank using augmented reality hmd
IL165497A (en) System and method for improving nighttime visual awareness of a pilot flying an aircraft carrying at least one air-to-air missile
WO2022050668A1 (en) Method for detecting hand motion of wearable augmented reality device by using depth image, and wearable augmented reality device capable of detecting hand motion by using depth image
KR102254174B1 (en) A head mounted display apparatus with night vision function
US20020053101A1 (en) Helmet
WO2022050742A1 (en) Method for detecting hand motion of wearable augmented reality device by using depth image and wearable augmented reality device capable of detecting hand motion by using depth image
CN110180102A (en) A kind of external display module of setting FPC and the sky of infrared photography exhale mask
CN108322705A (en) The special vehicle shown based on visual angle observing system and method for processing video frequency out of my cabin
WO2021182859A1 (en) Optical device including plurality of pinholes, and head-mounted display device using same
WO2021246634A1 (en) Augmented reality optical system comprising reflective surface, and head-mounted display device using same
WO2022231224A1 (en) Augmented reality glasses providing panoramic multi-screens and panoramic multi-screen provision method for augmented reality glasses

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22807651

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 22807651

Country of ref document: EP

Kind code of ref document: A1