JP5549436B2 - Vehicle monitoring device - Google Patents

Vehicle monitoring device Download PDF

Info

Publication number
JP5549436B2
JP5549436B2 JP2010153887A JP2010153887A JP5549436B2 JP 5549436 B2 JP5549436 B2 JP 5549436B2 JP 2010153887 A JP2010153887 A JP 2010153887A JP 2010153887 A JP2010153887 A JP 2010153887A JP 5549436 B2 JP5549436 B2 JP 5549436B2
Authority
JP
Japan
Prior art keywords
vehicle
image
virtual viewpoint
viewed
curved surface
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
JP2010153887A
Other languages
Japanese (ja)
Other versions
JP2012019274A (en
Inventor
柳　　拓良
政康 鈴木
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Nissan Motor Co Ltd
Original Assignee
Nissan Motor Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Nissan Motor Co Ltd filed Critical Nissan Motor Co Ltd
Priority to JP2010153887A priority Critical patent/JP5549436B2/en
Publication of JP2012019274A publication Critical patent/JP2012019274A/en
Application granted granted Critical
Publication of JP5549436B2 publication Critical patent/JP5549436B2/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Landscapes

  • Image Processing (AREA)
  • Closed-Circuit Television Systems (AREA)
  • Image Analysis (AREA)

Description

本発明は、一つ以上のカメラにより自車周囲の映像を撮像して表示する車両用監視装置に関する。   The present invention relates to a vehicle monitoring apparatus that captures and displays an image around a host vehicle with one or more cameras.

従来から、車載カメラによる撮影画像を曲面形状に投影し、この曲面形状を仮想視点から見た合成画像を生成する視点変換技術が知られている（特許文献１参照）。この視点変換技術において、上記した合成画像に自車の画像を含ませない場合、周囲の状況と自車との位置関係が分かりにくい。そこで、特許文献１では、この合成映像に自車の画像を加えることで、より違和感の少ない合成画像を生成している。   2. Description of the Related Art Conventionally, a viewpoint conversion technique for projecting an image captured by an in-vehicle camera onto a curved surface shape and generating a composite image in which the curved surface shape is viewed from a virtual viewpoint is known (see Patent Document 1). In this viewpoint conversion technique, when the image of the own vehicle is not included in the composite image, the positional relationship between the surrounding situation and the own vehicle is difficult to understand. Therefore, in Patent Document 1, a composite image with less sense of incongruity is generated by adding an image of the vehicle to the composite video.

特許第３２８６３０６号公報（特に、[0145]〜[0147]参照）Japanese Patent No. 3286306 (especially, see [0145] to [0147])

しかし、合成画像の中に自車を立体物として描くと、仮想視点から見て自車の奥側が死角となってしまう。また、自車の手前側の仮想視点に近い投影曲面も死角となってしまう。   However, if the subject vehicle is drawn as a three-dimensional object in the composite image, the rear side of the subject vehicle becomes a blind spot when viewed from a virtual viewpoint. Further, a projected curved surface close to the virtual viewpoint on the near side of the own vehicle also becomes a blind spot.

本発明は上記課題に鑑みて成されたものであり、その目的は、仮想視点から見た自車の周囲の死角を減らす車両用監視装置及び車両用監視方法を提供することである。   The present invention has been made in view of the above problems, and an object of the present invention is to provide a vehicle monitoring device and a vehicle monitoring method that reduce blind spots around the host vehicle viewed from a virtual viewpoint.

本発明の特徴は、一つ以上のカメラにより自車の周囲の映像を撮像し、自車の周囲に形成される曲面の形状情報を生成し、自車の形状情報及び少なくとも第１の仮想視点に関する情報を読み出し、合成映像を生成して表示する車両用監視装置及び車両用監視方法である。先ず、カメラによる映像を前記した曲面に投影し、この曲面を第１の仮想視点から見た投影画像を生成する。第１の仮想視点から見た、自車の形状を示す自車の映像を生成する。カメラによる映像を投影した曲面を自車の車体外板に投影し、この車体外板を第１の仮想視点から見た二重投影映像を生成する。そして、投影画像、自車の映像及び二重投影映像を合成することにより、前記した合成映像を生成する。 Feature of the present invention captures an image of the peripheral area of the vehicle by one or more cameras to generate shape information of the curved surfaces formed on the peripheral area of the vehicle, the vehicle of the shape information and at least a first virtual viewpoint This is a vehicle monitoring apparatus and a vehicle monitoring method for reading out information related to the above and generating and displaying a composite image. First, a video image from the camera is projected onto the curved surface, and a projection image in which the curved surface is viewed from the first virtual viewpoint is generated. An image of the own vehicle showing the shape of the own vehicle viewed from the first virtual viewpoint is generated. A curved surface onto which an image from the camera is projected is projected onto the vehicle body outer plate of the own vehicle, and a double projection image in which the vehicle body outer plate is viewed from the first virtual viewpoint is generated. Then, the above-described synthesized video is generated by synthesizing the projection image, the video of the own vehicle , and the double projection video.

本発明の車両用監視装置及び車両用監視方法によれば、第１の仮想視点から見て自車の周囲の死角となっていた領域の映像を、第１の仮想視点から見える自車の車体外板に投影して表示することにより、死角を減らすことができる。   According to the vehicle monitoring apparatus and the vehicle monitoring method of the present invention, the vehicle body of the host vehicle viewed from the first virtual viewpoint can be viewed from the first virtual viewpoint of the image of the area around the host vehicle viewed from the first virtual viewpoint. By projecting and displaying on the outer plate, blind spots can be reduced.

本発明の実施の形態に係わる車両用監視装置の構成を示すブロック図である。It is a block diagram which shows the structure of the monitoring apparatus for vehicles concerning embodiment of this invention. 図２（ａ）及び図２（ｂ）は、図１のカメラＣａ１〜ＣａＮの配置例を示す上面図及び側面図である。FIGS. 2A and 2B are a top view and a side view showing an arrangement example of the cameras Ca1 to CaN in FIG. 曲面生成部１２により生成された立体的な曲面ＣＳの一例を示す斜視図である。It is a perspective view which shows an example of the three-dimensional curved surface CS produced | generated by the curved surface production | generation part 12. FIG. 自車２１の進行方向に垂直な面における図３の曲面ＣＳ及び自車２１の切断面を示す断面図である。FIG. 4 is a cross-sectional view showing the curved surface CS of FIG. 図１の二重投影映像生成部１９の動作を説明するための図であって、図４と同じ切断面を示す断面図である。It is a figure for demonstrating operation | movement of the double projection image generation part 19 of FIG. 1, Comprising: It is sectional drawing which shows the same cut surface as FIG. 全周囲の曲面ＣＳ１、ＣＳ２、ＣＳ３の映像を表示する実施例を示す上面図である。It is a top view which shows the Example which displays the image | video of all the curved surfaces CS1, CS2, and CS3. 自車２１の奥側の投影面の死角を減らすその他の実施の形態を示す断面図である。It is sectional drawing which shows other embodiment which reduces the blind spot of the projection surface of the back side of the own vehicle. 図１の車両用監視装置を用いた車両用監視方法を示すフローチャートである。It is a flowchart which shows the monitoring method for vehicles using the monitoring apparatus for vehicles of FIG. 二重投影映像生成部３９のその他の実施の形態を示すブロック図である。It is a block diagram which shows other embodiment of the double projection image generation part 39. FIG. 第１の仮想視点ＰＶ１から見て自車２１の奥側が死角となる例を示す断面図である。It is sectional drawing which shows the example from which the back | inner side of the own vehicle 21 becomes a blind spot seeing from the 1st virtual viewpoint PV1. 自車２１の手前側の第１の仮想視点ＰＶ１に近い曲面部分Ａ１が死角となる例を示す断面図である。It is sectional drawing which shows the example from which the curved surface part A1 near the 1st virtual viewpoint PV1 of the near side of the own vehicle becomes a blind spot.

以下図面を参照して、本発明の実施の形態を説明する。図面の記載において同一部分には同一符号を付している。   Embodiments of the present invention will be described below with reference to the drawings. In the description of the drawings, the same parts are denoted by the same reference numerals.

先ず、図１を参照して、本発明の実施の形態に係わる車両用監視装置の構成を説明する。本発明の実施の形態に係わる車両用監視装置は、一つ以上のカメラＣａ１〜ＣａＮにより自車周囲の映像を撮像する撮像部１１と、自車周囲に形成される曲面の形状情報を生成する曲面生成部１２と、自車の形状情報を記憶する自車形状記憶部１３と、少なくとも第１の仮想視点を決定する視点決定部１４と、合成映像を生成する描画部１５と、合成映像を表示する表示部１６とを備える。   First, with reference to FIG. 1, the structure of the monitoring apparatus for vehicles concerning embodiment of this invention is demonstrated. The vehicle monitoring apparatus according to the embodiment of the present invention generates an image capturing unit 11 that captures an image of the surroundings of the own vehicle with one or more cameras Ca1 to CaN, and shape information of a curved surface formed around the own vehicle. A curved surface generation unit 12, a host vehicle shape storage unit 13 that stores shape information of the host vehicle, a viewpoint determination unit 14 that determines at least a first virtual viewpoint, a drawing unit 15 that generates a composite video, and a composite video And a display unit 16 for displaying.

描画部１５は、撮像部１１による映像を曲面生成部１２により生成された曲面に投影し、この曲面を第１の仮想視点から見た投影画像を生成する投影映像生成部１７と、第１の仮想視点から見た、自車の形状を示す自車映像を生成する自車映像生成部１８と、撮像部１１による映像を投影した曲面を自車の車体外板に投影し、この車体外板を第１の仮想視点から見た二重投影映像を生成する二重投影映像生成部１９と、投影画像、自車映像及び二重投影映像を合成した合成映像を生成する映像合成部２０とを有する。   The drawing unit 15 projects a video image from the imaging unit 11 onto the curved surface generated by the curved surface generation unit 12, and generates a projection image obtained by viewing the curved surface from the first virtual viewpoint. A vehicle image generation unit 18 that generates a vehicle image indicating the shape of the vehicle viewed from a virtual viewpoint, and a curved surface on which the image of the imaging unit 11 is projected is projected onto the vehicle body outer plate. A double projection video generation unit 19 that generates a double projection video viewed from the first virtual viewpoint, and a video synthesis unit 20 that generates a composite video obtained by combining the projection image, the vehicle image, and the double projection video. Have.

撮像部１１は、Ｎ個のカメラＣａ１〜ＣａＮを備え、各カメラＣａ１〜ＣａＮは、自車２１に搭載され、自車周囲の映像を撮像する。撮像された映各像データは描画部１５へ転送される。   The imaging unit 11 includes N cameras Ca <b> 1 to CaN, and each camera Ca <b> 1 to CaN is mounted on the own vehicle 21 and images an image around the own vehicle. The captured image data is transferred to the drawing unit 15.

視点決定部１４は、表示部１６に表示される合成映像の仮想的な視点として、第１の仮想視点を決定する。具体的には、第１の仮想視点の位置、向き、視野角、歪曲特性を決定する。曲面生成部１２は、各カメラＣａ１〜ＣａＮの視点から第１の仮想視点への変換を行う際に基準となる立体的な曲面の形状を生成する。自車形状記憶部１３は、第１の仮想視点から見た自車の映像を描画するするために使用される自車の立体的な形状を記憶する。   The viewpoint determination unit 14 determines a first virtual viewpoint as a virtual viewpoint of the composite video displayed on the display unit 16. Specifically, the position, orientation, viewing angle, and distortion characteristics of the first virtual viewpoint are determined. The curved surface generation unit 12 generates a three-dimensional curved surface shape that serves as a reference when performing conversion from the viewpoints of the cameras Ca1 to CaN to the first virtual viewpoint. The own vehicle shape memory | storage part 13 memorize | stores the three-dimensional shape of the own vehicle used in order to draw the image | video of the own vehicle seen from the 1st virtual viewpoint.

投影映像生成部１７は、各カメラＣａ１〜ＣａＮの特性を示すカメラパラメータのテーブルを格納したメモリを備える。各カメラＣａ１〜ＣａＮの特性には、各カメラＣａ１〜ＣａＮの位置、向き、視野角、歪曲特性などが含まれる。また、投影映像生成部１７は、このカメラパラメータに基づいて、各カメラＣａ１〜ＣａＮにより撮像された映像を３次元空間の空間モデルにマッピングした空間データを作成する。そして、この空間データ及び視点決定部１４により決定された第１の仮想視点の位置、向き、視野角、歪曲特性を参照して、撮像部１１による映像を投影した曲面を第１の仮想視点から見た投影画像を生成する。このようにして、投影映像生成部１７は、各カメラＣａ１〜ＣａＮにより撮像された映像から、第１の仮想視点から見た投影画像へ、視点変換することできる。   The projection video generation unit 17 includes a memory that stores a camera parameter table indicating the characteristics of the cameras Ca1 to CaN. The characteristics of each camera Ca1-CaN include the position, orientation, viewing angle, distortion characteristics, etc. of each camera Ca1-CaN. Further, the projection video generation unit 17 creates spatial data in which videos captured by the cameras Ca1 to CaN are mapped to a three-dimensional space model based on the camera parameters. Then, with reference to the position, orientation, viewing angle, and distortion characteristics of the first virtual viewpoint determined by the spatial data and the viewpoint determination unit 14, the curved surface on which the image by the imaging unit 11 is projected is referred to from the first virtual viewpoint. Generate the projected image. In this way, the projected video generation unit 17 can convert the viewpoint from the video captured by each of the cameras Ca1 to CaN into a projected image viewed from the first virtual viewpoint.

自車映像生成部１８は、自車形状記憶部１３に記憶されている自車の立体的な形状及び第１の仮想視点の位置、向き、視野角、歪曲特性に基づいて、第１の仮想視点から見た自車映像を生成する。自車映像生成部１８の基本的な動作は、ＣＡＤ（Computer Aided Design）やＣＧ（コンピュータ・グラフィックス）などの技術として一般的に知られているものであり、立体的な形状のデータと視点が与えられているときに、その形状をその視点から見たときの映像を生成する。   The own vehicle image generation unit 18 generates a first virtual image based on the three-dimensional shape of the own vehicle stored in the own vehicle shape storage unit 13 and the position, orientation, viewing angle, and distortion characteristics of the first virtual viewpoint. Generates a vehicle image viewed from the viewpoint. The basic operation of the vehicle image generation unit 18 is generally known as a technology such as CAD (Computer Aided Design) or CG (Computer Graphics). Is generated, an image when the shape is viewed from the viewpoint is generated.

なお、曲面生成部１２、視点決定部１４及び描画部１５は、データの演算装置、データを一時的に記憶するレジスタやメモリ装置、及び周辺機器とのインターフェースを行う入出力装置を備えるマイクロプロセッシングユニット（ＭＰＵ）によって実行される機能的な構成を示している。また、自車形状記憶部１３は、半導体メモリなどの記憶装置として構成すればよく、表示部１６は、液晶表示装置などの表示装置として構成すればよい。   The curved surface generation unit 12, the viewpoint determination unit 14, and the drawing unit 15 include a data processing device, a register or memory device that temporarily stores data, and a microprocessing unit including an input / output device that interfaces with peripheral devices. The functional structure performed by (MPU) is shown. The vehicle shape storage unit 13 may be configured as a storage device such as a semiconductor memory, and the display unit 16 may be configured as a display device such as a liquid crystal display device.

図２（ａ）及び図２（ｂ）を参照して、図１のカメラＣａ１〜ＣａＮの配置例を説明する。撮像部１１は、Ｎ（＝６）個のカメラＣａ１〜Ｃａ６を備え、各カメラＣａ１〜Ｃａ６は、自車２１における所定の位置及び所定の角度に搭載され、自車周囲の映像を撮像する。カメラＣａ１は自車２１の右斜め後方の領域を撮像し、カメラＣａ２は自車２１の左斜め後方の領域を撮像し、カメラＣａ３は自車２１の左側の領域を撮像する。そして、カメラＣａ４は自車２１の左斜め前方の領域を撮像し、カメラＣａ５は自車２１の右斜め前方の領域を撮像し、カメラＣａ６は自車２１の右側の領域を撮像する。このように、自車２１に複数個のカメラＣａ１〜Ｃａ６を搭載し、各カメラＣａ１〜Ｃａ６の位置及び角度を調整することにより、自車周囲のうち広い範囲の映像を取得することができる。   An arrangement example of the cameras Ca1 to CaN in FIG. 1 will be described with reference to FIGS. 2 (a) and 2 (b). The imaging unit 11 includes N (= 6) cameras Ca <b> 1 to Ca <b> 6, and each camera Ca <b> 1 to Ca <b> 6 is mounted at a predetermined position and a predetermined angle in the own vehicle 21 and images an image around the own vehicle. The camera Ca <b> 1 images a region diagonally to the right rear of the host vehicle 21, the camera Ca <b> 2 images a region diagonally to the left rear of the host vehicle 21, and the camera Ca <b> 3 images a region on the left side of the host vehicle 21. The camera Ca4 images a region diagonally forward to the left of the host vehicle 21, the camera Ca5 images a region diagonally forward to the right of the host vehicle 21, and the camera Ca6 captures a region on the right side of the host vehicle 21. As described above, by mounting a plurality of cameras Ca1 to Ca6 on the own vehicle 21 and adjusting the positions and angles of the cameras Ca1 to Ca6, it is possible to acquire a wide range of images around the own vehicle.

図３に示すように、曲面生成部１２により生成された立体的な曲面ＣＳは、自車２１を取り囲む半球状の形状を有し、自車２１は、この半球状の形状の内側に位置している。自車２１のタイヤは曲面ＣＳの底部に接し、当該底部が、滑らかな曲面よって、自車２１を取り囲む側面につながっている。   As shown in FIG. 3, the three-dimensional curved surface CS generated by the curved surface generation unit 12 has a hemispherical shape that surrounds the own vehicle 21, and the own vehicle 21 is located inside the hemispherical shape. ing. The tire of the own vehicle 21 is in contact with the bottom of the curved surface CS, and the bottom is connected to the side surface surrounding the own vehicle 21 by a smooth curved surface.

図４は、自車２１の進行方向に垂直な面における図３の曲面ＣＳ及び自車２１の切断面を示す断面図である。自車２１は半球状の曲面ＣＳの内側に配置されている。自車２１のベルトラインには複数個のカメラＣａ１〜Ｃａ６が搭載され、図４のＰＣは、このうちの１つのカメラの視点を示している。また、図４のＰＶ１は、描画部１５により生成される合成映像における第１の仮想視点を示している。   FIG. 4 is a cross-sectional view showing the curved surface CS of FIG. 3 and the cut surface of the host vehicle 21 in a plane perpendicular to the traveling direction of the host vehicle 21. The own vehicle 21 is disposed inside the hemispherical curved surface CS. A plurality of cameras Ca1 to Ca6 are mounted on the belt line of the own vehicle 21, and the PC in FIG. 4 shows the viewpoint of one of these cameras. Also, PV1 in FIG. 4 indicates a first virtual viewpoint in the composite video generated by the drawing unit 15.

投影映像生成部１７は、各カメラＣａ１〜Ｃａ６により撮像された映像を曲面ＣＳに投影して映像ＢＳを生成する。そして、曲面ＣＳに投影した映像ＢＳを第１の仮想視点ＰＶ１から見た投影画像を生成する。すなわち、カメラの視点ＰＣから見た映像ＢＳを、第１の仮想視点ＰＶ１から見た投影画像へ視点変換する。映像ＢＳの生成及び視点変換の際、投影映像生成部１７は、カメラＣａ１〜Ｃａ６に関する情報（自車２１に対する位置、向き、視野角、歪曲特性）、及び第１の仮想視点ＰＶ１に関する情報（自車２１に対する位置、向きＤＶ、視野角ＦＶ１、歪曲特性）を参照する。   The projection video generation unit 17 generates a video BS by projecting video captured by the cameras Ca1 to Ca6 onto the curved surface CS. Then, a projection image in which the video BS projected on the curved surface CS is viewed from the first virtual viewpoint PV1 is generated. That is, the viewpoint of the video BS viewed from the camera viewpoint PC is converted into a projected image viewed from the first virtual viewpoint PV1. During the generation of the video BS and the viewpoint conversion, the projection video generation unit 17 performs information on the cameras Ca1 to Ca6 (position, orientation, viewing angle, distortion characteristics with respect to the own vehicle 21), and information on the first virtual viewpoint PV1 (self Reference is made to the position, direction DV, viewing angle FV1, distortion characteristics with respect to the vehicle 21).

図５を参照して、図１の二重投影映像生成部１９による二重投影映像の生成方法について説明する。先ず、図１の視点決定部１４は、第２の仮想視点ＰＶ２を決定する。具体的には、第２の仮想視点ＰＶ２の位置、向き、視野角、歪曲特性を決定する。図５の例において、第２の仮想視点ＰＶ２は自車２１の略中心にある。   With reference to FIG. 5, a method of generating a double projection image by the double projection image generation unit 19 of FIG. 1 will be described. First, the viewpoint determination unit 14 in FIG. 1 determines the second virtual viewpoint PV2. Specifically, the position, orientation, viewing angle, and distortion characteristics of the second virtual viewpoint PV2 are determined. In the example of FIG. 5, the second virtual viewpoint PV <b> 2 is substantially at the center of the own vehicle 21.

そして、二重投影映像生成部１９は、先ず、投影映像生成部１７と同様にして、各カメラＣａ１〜ＣａＮにより撮像された映像を曲面ＣＳに投影し、この曲面ＣＳを第２の仮想視点ＰＶ２から見た中間映像Ｂ１を生成する。そして、中間映像Ｂ１を自車２１の車体外板に投影した映像Ｂ２を生成し、この車体外板に投影された映像Ｂ２を第１の仮想視点ＰＶ１から見た二重投影映像を生成する。すなわち、二重投影映像生成部１９は、カメラの視点から第２の仮想視点ＰＶ２へ視点変換し、さらに、第２の仮想視点ＰＶ２から第１の仮想視点ＰＶ１へ視点変換する。これにより、車両のボディに反射している風景を模擬して広い視野を実現する二重投影映像を生成することができる。   Then, the double projection video generation unit 19 first projects the images captured by the cameras Ca1 to CaN onto the curved surface CS in the same manner as the projection video generation unit 17, and this curved surface CS is projected to the second virtual viewpoint PV2. The intermediate video B1 viewed from the above is generated. Then, an image B2 obtained by projecting the intermediate image B1 onto the vehicle body outer plate of the host vehicle 21 is generated, and a double projection image obtained by viewing the image B2 projected onto the vehicle body outer plate from the first virtual viewpoint PV1 is generated. That is, the double projection video generation unit 19 converts the viewpoint from the camera viewpoint to the second virtual viewpoint PV2, and further converts the viewpoint from the second virtual viewpoint PV2 to the first virtual viewpoint PV1. As a result, it is possible to generate a double projection image that realizes a wide field of view by simulating the scenery reflected on the vehicle body.

映像合成部２０は、この二重投影映像を、投影画像、自車映像とともに合成して、合成映像を生成する。   The video synthesizing unit 20 synthesizes the double projection video together with the projection image and the own vehicle video to generate a synthesized video.

なお、二重投影映像生成部１９は、曲面ＣＳに投影された映像Ｂ１を自車２１の車体外板のうちベルトラインＢＬから下方の部分に投影し、この下方部分に投影された映像Ｂ２を第１の仮想視点ＰＶ１から見た二重投影映像へ変換する。   The double projection image generation unit 19 projects the image B1 projected on the curved surface CS onto the lower part of the vehicle body outer plate of the vehicle 21 from the belt line BL, and the image B2 projected on the lower part. The image is converted into a double projection image viewed from the first virtual viewpoint PV1.

次に、図８を参照して、図１の車両用監視装置を用いた車両用監視方法について説明する。ここでは、図１における構成のうち、ＭＰＵ（データ演算装置）としての動作を説明する。 Next, a vehicle monitoring method using the vehicle monitoring apparatus of FIG. 1 will be described with reference to FIG. Here, an operation as an MPU (data arithmetic unit) in the configuration in FIG. 1 will be described.

（イ）先ず、Ｓ０１段階において、カメラＣａ１〜Ｃａ６を備える撮像部１１を用いて自車周囲の映像を撮像する。Ｓ０３段階に進み、自車周囲に形成される曲面ＣＳの形状情報を取得する。Ｓ０５段階に進み、自車形状情報記憶部１３から自車２１の形状情報を読み出す。Ｓ０７段階に進み、第１及び第２の仮想視点ＰＶ１、ＶＰ２を決定する。なお、前記したＳ０１〜Ｓ０７段階はどの順番に実施してもかまわない。   (A) First, in step S01, an image around the host vehicle is captured using the imaging unit 11 including the cameras Ca1 to Ca6. Proceeding to step S03, the shape information of the curved surface CS formed around the host vehicle is acquired. Proceeding to step S05, the shape information of the own vehicle 21 is read from the own vehicle shape information storage unit 13. Proceeding to step S07, the first and second virtual viewpoints PV1 and VP2 are determined. The steps S01 to S07 described above may be performed in any order.

（ロ）Ｓ０９段階に進み、投影画像、自車映像及び二重投影映像を合成した合成映像を生成する。詳細には、先ず、Ｓ９１段階において、撮像部１１により撮像した映像を曲面ＣＳに投影し、この曲面ＣＳに投影された映像を第１の仮想視点ＰＶ１から見た投影画像を生成する。Ｓ９３段階に進み、第１の仮想視点ＰＶ１から見た、自車２１の形状を示す自車映像を生成する。Ｓ９５段階に進み、撮像部１１により撮像した映像を投影した曲面ＣＳを自車２１の車体外板に投影し、この車体外板を第１の仮想視点ＰＶ１から見た二重投影映像を生成する。そして、Ｓ９７段階に進み、投影画像、自車映像及び二重投影映像を合成して合成映像を生成する。   (B) Proceeding to step S09, a composite image is generated by combining the projection image, the vehicle image, and the double projection image. Specifically, first, in step S91, the video imaged by the imaging unit 11 is projected onto the curved surface CS, and a projection image obtained by viewing the video image projected on the curved surface CS from the first virtual viewpoint PV1 is generated. Proceeding to step S93, the vehicle image showing the shape of the vehicle 21 viewed from the first virtual viewpoint PV1 is generated. Proceeding to step S95, the curved surface CS on which the image captured by the imaging unit 11 is projected is projected onto the vehicle body outer plate of the host vehicle 21, and a double projection image in which the vehicle body outer plate is viewed from the first virtual viewpoint PV1 is generated. . In step S97, the projection image, the vehicle image, and the double projection image are combined to generate a combined image.

（ハ）最後に、Ｓ１１段階に進み、合成映像を表示部１６に表示する。   (C) Finally, proceeding to step S11, the synthesized video is displayed on the display unit 16.

以上説明したように、本発明の実施の形態によれば、以下の作用効果が得られる。   As described above, according to the embodiment of the present invention, the following effects can be obtained.

図１の映像合成部２０が、投影映像生成部１７により生成された投影画像及び自車映像生成部１８により生成された自車映像を合成した場合、図１０に示すように、第１の仮想視点ＰＶ１から見て、自車２１の奥側が死角となる。また、図１１に示すように、自車２１の手前側の第１の仮想視点ＰＶ１に近い曲面部分Ａ１も死角となってしまう。例えば、第１の仮想視点ＰＶ１を自車２１の真上に置けば、周囲全体が見渡せるようになるが、周囲の映像の歪曲などによりそのような第１の仮想視点ＰＶ１の設定が望ましくない場合がある。   When the video composition unit 20 in FIG. 1 synthesizes the projection image generated by the projection video generation unit 17 and the vehicle image generated by the vehicle image generation unit 18, as shown in FIG. When viewed from the viewpoint PV1, the back side of the host vehicle 21 is a blind spot. Moreover, as shown in FIG. 11, the curved surface portion A1 near the first virtual viewpoint PV1 on the near side of the host vehicle 21 also becomes a blind spot. For example, if the first virtual viewpoint PV1 is placed directly above the host vehicle 21, the entire surrounding can be seen, but such a setting of the first virtual viewpoint PV1 is not desirable due to distortion of surrounding images. There is.

そこで、第１の仮想視点ＰＶ１から見て自車２１の周囲の死角となっていた領域の映像を、第１の仮想視点ＰＶ１から見える自車２１の車体外板に投影して表示することにより、死角を減らすことができる。   Therefore, by projecting and displaying an image of an area that has become a blind spot around the host vehicle 21 when viewed from the first virtual viewpoint PV1, it is projected onto the vehicle body outer plate of the host vehicle 21 that can be viewed from the first virtual viewpoint PV1. , Can reduce blind spots.

二重投影映像生成部１９は、図５に示したように、曲面ＣＳを第２の仮想視点ＰＶ２に向かって自車２１の車体外板に投影し、この車体外板を第１の仮想視点ＰＶ１から見た二重投影映像を生成する。これにより、自車２１の周囲の死角となっていた映像を自車２１に投影できるので、自車２１の周囲の死角となっていた領域Ｂ１が、自車２１の車体外板Ｂ２に反射しているように見え、高い視認性を実現できる。   As shown in FIG. 5, the double projection video generation unit 19 projects the curved surface CS toward the second virtual viewpoint PV2 onto the vehicle body outer plate of the host vehicle 21, and the vehicle body outer plate is projected to the first virtual viewpoint. A double projection image viewed from PV1 is generated. As a result, an image that has become a blind spot around the host vehicle 21 can be projected onto the host vehicle 21, so that the area B <b> 1 that is a blind spot around the host vehicle 21 is reflected on the vehicle body outer plate B <b> 2 of the host vehicle 21. It is possible to realize high visibility.

図５の例において、第２の仮想視点ＰＶ２は自車２１の略中心にある。これにより、自車２１の周囲の映像を大きく、高い視認性で表示できる。第２の仮想視点ＰＶ２が映像Ｂ２の近い位置にある場合、映像Ｂ２は小さくなってしまう。   In the example of FIG. 5, the second virtual viewpoint PV <b> 2 is substantially at the center of the own vehicle 21. Thereby, the image | video around the own vehicle 21 can be enlarged and displayed with high visibility. When the second virtual viewpoint PV2 is at a position close to the video B2, the video B2 becomes small.

二重投影映像生成部１９は、曲面ＣＳを自車２１の車体外板のうちベルトラインＢＬから下方の部分に投影し、この下方部分を第１の仮想視点ＰＶ１から見た二重投影映像を生成する。これにより、ベルトラインよりも下の実際の車両では塗装面に当たる部分に投影することができる。よって、自車２１の周囲の二重投影映像を、略垂直な面に高い視認性で表示できる。なお、実際の車両では、ベルトラインより上のガラス面に当たる部分では、室内の形状などの映像とも重なってしまうため、そこに表示しても分かりにくい。   The double projection video generation unit 19 projects the curved surface CS onto the lower part of the vehicle body outer plate of the vehicle 21 from the belt line BL, and displays the double projection video when the lower part is viewed from the first virtual viewpoint PV1. Generate. As a result, in an actual vehicle below the belt line, it can be projected onto a portion that hits the painted surface. Therefore, a double projection image around the host vehicle 21 can be displayed with high visibility on a substantially vertical surface. In an actual vehicle, the portion that hits the glass surface above the belt line overlaps with the image of the room shape and the like, so it is difficult to understand even if it is displayed there.

（その他の実施の形態）
上記のように、本発明は、１つの実施形態によって記載したが、この開示の一部をなす論述及び図面はこの発明を限定するものであると理解すべきではない。この開示から当業者には様々な代替実施の形態、実施例及び運用技術が明らかとなろう。 (Other embodiments)
As described above, the present invention has been described by way of one embodiment, but it should not be understood that the discussion and drawings that form part of this disclosure limit the present invention. From this disclosure, various alternative embodiments, examples and operational techniques will be apparent to those skilled in the art.

例えば、中間映像Ｂ１を自車２１の車体外板に投影する際に、実際の光の反射を模擬するレイ・トレーシング手法を用いてもよい。そして、曲率が強いボディ形状における反射映像を計算するときのみ曲率が少ないボディ形状に置き換えて計算を行ってもよい。すなわち、二重投影映像生成部１９は、第２の仮想視点ＰＶ２から見た鏡面反射映像よりも上下方向に拡大された二重投影映像を生成してもよい。これにより、自車２１の周囲の二重投影映像を更に大きく、高い視認性で表示できる。   For example, when the intermediate video B1 is projected onto the vehicle body outer plate of the host vehicle 21, a ray tracing method that simulates actual light reflection may be used. Then, the calculation may be performed by replacing the body shape with a small curvature only when calculating the reflection image in the body shape with a strong curvature. That is, the double projection video generation unit 19 may generate a double projection video that is enlarged in the vertical direction from the specular reflection video viewed from the second virtual viewpoint PV2. Thereby, the double-projection image around the own vehicle 21 can be further enlarged and displayed with high visibility.

同様にして、映像の縦方向の大きさだけでなく、横方向の大きさや位置についても調整してもかまわない。例えば、図６の上面図に示すように、二重投影映像を計算するための第２の仮想視点として、２つの仮想視点ＰＶ２、ＰＶ３を設定する。第２の仮想視点ＰＶ２から見た曲面ＣＳ２を自車２１の車体外板ＦＢ２に投影する。もう一つの第２の仮想視点ＰＶ３から見た曲面ＣＳ３を自車２１の車体外板ＦＢ３に投影する。そして、第１の仮想視点から見た、車体外板ＦＢ２及び車体外板ＦＢ３に投影された二重投影映像と、曲面ＣＳ１に投影された投影画像と、自車映像とを合成する。   Similarly, not only the vertical size but also the horizontal size and position of the image may be adjusted. For example, as shown in the top view of FIG. 6, two virtual viewpoints PV2 and PV3 are set as the second virtual viewpoint for calculating the double projection image. The curved surface CS2 viewed from the second virtual viewpoint PV2 is projected onto the vehicle body outer plate FB2 of the host vehicle 21. The curved surface CS3 viewed from another second virtual viewpoint PV3 is projected onto the vehicle body outer plate FB3 of the host vehicle 21. Then, the double projection image projected on the vehicle body outer plate FB2 and the vehicle body outer plate FB3, the projection image projected on the curved surface CS1, and the own vehicle image viewed from the first virtual viewpoint are synthesized.

ここで、車体外板ＦＢ２及び車体外板ＦＢ３に投影された二重投影映像は、第２の仮想視点ＰＶ２、ＰＶ３から見た鏡面反射映像と比べて横方向にオフセットしてもよい。二重投影映像の横方向の大きさや位置を調整することにより、全周囲の曲面ＣＳ１、ＣＳ２、ＣＳ３の映像を第１の仮想視点ＰＶ１から直接にまたは間接的に確認することができる。   Here, the double projection images projected on the vehicle body outer plate FB2 and the vehicle body outer plate FB3 may be offset in the lateral direction as compared with the specular reflection images viewed from the second virtual viewpoints PV2 and PV3. By adjusting the size and position of the double projection image in the horizontal direction, it is possible to directly or indirectly confirm the images of the entire surrounding curved surfaces CS1, CS2, and CS3 from the first virtual viewpoint PV1.

以上では、最終的な映像の視点となる第１の仮想視点ＰＶ１から見て、自車２１の手前側の曲面ＣＳの死角を減らす方法を述べたが、本発明によれば、自車２１の奥側の投影面の死角を減らすこともできる。例えば、第２の仮想視点ＰＶ２を図７のように、第１の仮想視点ＰＶ１よりも上方の位置に設定し、曲面Ｃ１を自車２１の車体外板Ｃ２の位置に投影してもよい。自車２１からみて第１の仮想視点ＰＶ１と反対側の死角を車体外板Ｃ２の位置に表示できる。   In the above, the method of reducing the blind spot of the curved surface CS on the near side of the host vehicle 21 as viewed from the first virtual viewpoint PV1 as the final video viewpoint has been described. It is also possible to reduce the blind spot on the back projection surface. For example, the second virtual viewpoint PV2 may be set at a position above the first virtual viewpoint PV1 as shown in FIG. 7, and the curved surface C1 may be projected on the position of the vehicle body outer plate C2 of the host vehicle 21. The blind spot on the side opposite to the first virtual viewpoint PV1 as viewed from the host vehicle 21 can be displayed at the position of the vehicle body outer plate C2.

二重投影映像生成部１９は、曲面Ｃ１を自車２１の車体外板のうちベルトラインから上方の部分Ｃ２に投影し、この上方部分Ｃ２を第１の仮想視点ＰＶ１から見た二重投影映像を生成してもい。自車２１からみて第１の仮想視点ＰＶ１と反対側の死角を直感的に表示できる。このように、第１の仮想視点ＰＶ１と反対側の死角を自車２１の車体外板上に投影する場合、実際の世界では車両のガラスの部分であるベルトラインの上の領域に投影するのが自然である。自車２１からみて第１の仮想視点ＰＶ１と反対側の死角を直感的に表示できる。   The double projection video generation unit 19 projects the curved surface C1 onto a portion C2 above the belt line of the vehicle body outer plate of the own vehicle 21, and the double projection video when the upper portion C2 is viewed from the first virtual viewpoint PV1. May be generated. The blind spot on the side opposite to the first virtual viewpoint PV1 as viewed from the host vehicle 21 can be intuitively displayed. Thus, when the blind spot opposite to the first virtual viewpoint PV1 is projected onto the vehicle body outer plate of the own vehicle 21, it is projected onto the area above the belt line that is the glass portion of the vehicle in the actual world. Is natural. The blind spot on the side opposite to the first virtual viewpoint PV1 as viewed from the host vehicle 21 can be intuitively displayed.

また、図９に示すように、二重投影映像生成部３９は、自車２１とその周囲の状況から注意すべき周囲の物体または方向を求める状況取得部４１を備え、状況取得部４１が求めた物体または方向の映像を含む二重投影映像を生成してもよい。自車２１とその周囲の状況をセンシングし、特定の方向にて注意すべきものが存在するときのみ、自車２１の車体外板上に二重投影映像を表示する。運転者の注意を特にひきつけることができる。特に注意すべき物体を強調して、運転者の注意を特にひきつけることができる。また、周囲の死角を減らすことができる。   Further, as shown in FIG. 9, the double projection video generation unit 39 includes a situation acquisition unit 41 that obtains a surrounding object or direction to be noted from the own vehicle 21 and the surrounding situation, and the situation acquisition unit 41 obtains the situation. A double projection image may be generated that includes an image of the selected object or direction. The self-projecting vehicle 21 and surrounding conditions are sensed, and only when there is something to be noted in a specific direction, a double projection image is displayed on the vehicle body outer plate of the own vehicle 21. The driver's attention can be particularly attracted. It is possible to emphasize a driver's attention by emphasizing an object to be particularly careful. In addition, surrounding blind spots can be reduced.

また、実施の形態では、撮像部１１が備えるカメラの数が６個である場合を示したが、１〜５、あるいは７以上であってもかまわない。   In the embodiment, the case where the number of cameras included in the imaging unit 11 is six has been described, but may be 1 to 5, or 7 or more.

１１ 撮像部
１２ 曲面生成部
１３ 自車形状記憶部
１４ 視点決定部
１５ 描画部
１６ 表示部
１７ 投影映像生成部
１８ 自車映像生成部
１９、３９ 二重投影映像生成部
２０ 合成部
２１ 自車
４１ 状況取得部
ＰＶ１ 第１の仮想視点
ＰＶ２、ＰＶ３ 第２の仮想視点
Ｃａ１〜ＣａＮ カメラ
ＣＳ 曲面
ＢＬ ベルトライン DESCRIPTION OF SYMBOLS 11 Image pick-up part 12 Curved surface production | generation part 13 Own vehicle shape memory | storage part 14 Viewpoint determination part 15 Drawing part 16 Display part 17 Projection image generation part 18 Own vehicle image generation part 19, 39 Double projection image generation part 20 Composition part 21 Own vehicle 41 Situation acquisition part PV1 1st virtual viewpoint PV2, PV3 2nd virtual viewpoint Ca1-CaN Camera CS Curved surface BL Belt line

Claims (10)

一つ以上のカメラにより自車の周囲の映像を撮像する撮像部と、
前記自車の周囲に形成される曲面の形状情報を生成する曲面生成部と、
前記自車の形状情報及び少なくとも第１の仮想視点に関する情報を記憶する自車形状記憶部と、
合成映像を生成する描画部と、
前記合成映像を表示する表示部と、を備え、
前記描画部は、
前記撮像部による映像を前記曲面に投影し、この曲面を前記第１の仮想視点から見た投影画像を生成する投影映像生成部と、
前記第１の仮想視点から見た、前記自車の形状を示す前記自車の映像を生成する自車映像生成部と、
前記撮像部による映像を投影した前記曲面を前記自車の車体外板に投影し、この車体外板を前記第１の仮想視点から見た二重投影映像を生成する二重投影映像生成部と、
前記投影画像、前記自車の映像及び前記二重投影映像を合成した前記合成映像を生成する映像合成部と、を有する
ことを特徴とする車両用監視装置。 An imaging unit that captures an image of the surroundings of the vehicle with one or more cameras;
And surface generation unit for generating shape information of the curved surfaces formed on the periphery of the vehicle,
A host vehicle shape storage unit for storing information on the host vehicle shape information and at least information on the first virtual viewpoint ;
A drawing unit for generating a composite video;
A display unit for displaying the composite video,
The drawing unit
A projection video generation unit configured to project a video image from the imaging unit onto the curved surface and generate a projection image of the curved surface viewed from the first virtual viewpoint;
Viewed from the first virtual viewpoint, and the vehicle image generation unit for generating an image of the vehicle showing the vehicle shape,
Projecting the curved surface obtained by projecting an image by the imaging unit on the vehicle body outer panel of the vehicle, and a double projection image generator for generating a dual projection image viewed this body shell from said first virtual viewpoint ,
The projection image, the vehicle monitoring apparatus characterized by comprising: a video synthesizing unit, the generating the composite image obtained by combining the vehicle image and the double projection image. 前記自車形状記憶部は、前記第１の仮想視点とは異なる第２の仮想視点に関する情報を更に記憶し、
前記二重投影映像生成部は、前記曲面を前記第２の仮想視点に向かって前記自車の車体外板に投影し、この車体外板を前記第１の仮想視点から見た前記二重投影映像を生成する
ことを特徴とする請求項１に記載の車両用監視装置。 The vehicle shape storage unit further stores information on a second virtual viewpoint different from the first virtual viewpoint,
The double projection video generation unit projects the curved surface onto the vehicle body outer plate of the host vehicle toward the second virtual viewpoint, and the double projection when the vehicle body outer plate is viewed from the first virtual viewpoint. The vehicle monitoring device according to claim 1, wherein an image is generated. 前記第２の仮想視点が前記自車の略中心にあることを特徴とする請求項２に記載の車両用監視装置。 The vehicle monitoring system according to claim 2, wherein the second virtual viewpoint is at the approximate center of the vehicle. 前記二重投影映像生成部は、前記曲面を前記自車の車体外板のうちベルトラインから下方の部分に投影し、この下方の部分を前記第１の仮想視点から見た前記二重投影映像を生成することを特徴とする請求項１〜３のいずれか一項に記載の車両用監視装置。 The dual projection image generating unit projects the said curved lower portion from the belt line of the vehicle body shell of the vehicle, wherein the double-projected images viewed portions of the downwardly from said first virtual viewpoint The vehicle monitoring device according to any one of claims 1 to 3, wherein the vehicle monitoring device is generated. 前記二重投影映像生成部は、前記第２の仮想視点から見た鏡面反射映像よりも上下方向に拡大された二重投影映像を生成することを特徴とする請求項２または３のいずれか一項に記載の車両用監視装置。 The dual projection image generation unit according to claim 2 or 3 of any one and generates a dual projection image is enlarged in the vertical direction than the specular reflection image viewed from the second virtual viewpoint The vehicle monitoring device according to the item. 前記二重投影映像生成部は、前記第２の仮想視点から見た鏡面反射映像と比べて横方向にオフセットしている二重投影映像を生成することを特徴とする請求項２、３または５のいずれか一項に記載の車両用監視装置。 The dual projection image generation unit according to claim 2, 3 or 5, characterized in that to produce a double projection image that is laterally offset in comparison with the specular image viewed from the second virtual viewpoint The vehicle monitoring device according to any one of the above. 前記第２の仮想視点が、前記自車と前記第１の仮想視点よりも上方にあることを特徴とする請求項２に記載の車両用監視装置。   The vehicular monitoring apparatus according to claim 2, wherein the second virtual viewpoint is located above the host vehicle and the first virtual viewpoint. 前記二重投影映像生成部は、前記曲面を前記自車の車体外板のうちベルトラインから上方の部分に投影し、この上方の部分を前記第１の仮想視点から見た前記二重投影映像を生成することを特徴とする請求項１〜７のいずれか一項に記載の車両用監視装置。 The dual projection image generating unit projects the said curved surface above the portion from the belt line of the vehicle body shell of the vehicle, wherein the double-projected images viewed portions of the upwardly from said first virtual viewpoint The vehicle monitoring device according to any one of claims 1 to 7, wherein the vehicle monitoring device is generated. 前記二重投影映像生成部は、前記自車とその周囲の状況から注意すべき周囲の物体または方向を求める状況取得部を備え、前記状況取得部が求めた物体または方向の映像を含む前記二重投影映像を生成することを特徴とする請求項１〜８のいずれか一項に記載の車両用監視装置。   The double projection image generation unit includes a situation acquisition unit that obtains a surrounding object or direction to be noted from the situation of the host vehicle and its surroundings, and includes the image of the object or direction obtained by the situation acquisition unit. The vehicular monitoring apparatus according to claim 1, wherein a multi-projection image is generated. 一つ以上のカメラにより自車の周囲の映像を撮像し、
前記自車の周囲に形成される曲面の形状情報を取得し、
前記自車の形状情報及び少なくとも第１の仮想視点に関する情報を読み出し、
前記映像を前記曲面に投影し、この曲面を前記第１の仮想視点から見た投影画像を生成し、
前記第１の仮想視点から見た、前記自車の形状を示す前記自車の映像を生成し、
前記映像を投影した前記曲面を前記自車の車体外板に投影し、この車体外板を前記第１の仮想視点から見た二重投影映像を生成し、
前記投影画像、前記自車の映像及び前記二重投影映像を合成した合成映像を生成し、
前記合成映像を表示する
ことを特徴とする車両用監視方法。 The image of the peripheral area of the vehicle captured by the one or more cameras,
Acquires shape information of the curved surfaces formed on the periphery of the vehicle,
Read out information about the shape of the vehicle and at least the first virtual viewpoint ,
Projecting the video onto the curved surface, and generating a projected image of the curved surface viewed from the first virtual viewpoint;
Generating an image of the vehicle showing the shape of the vehicle viewed from the first virtual viewpoint;
Projecting the curved surface obtained by projecting the image on the vehicle body outer panel of the vehicle, it generates a double projection image viewed this body shell from said first virtual viewpoint,
The projection image, the generating a combined image obtained by combining the vehicle image and the double projection images,
The vehicle monitoring method, comprising displaying the composite image.
JP2010153887A 2010-07-06 2010-07-06 Vehicle monitoring device Active JP5549436B2 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
JP2010153887A JP5549436B2 (en) 2010-07-06 2010-07-06 Vehicle monitoring device

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
JP2010153887A JP5549436B2 (en) 2010-07-06 2010-07-06 Vehicle monitoring device

Publications (2)

Publication Number Publication Date
JP2012019274A JP2012019274A (en) 2012-01-26
JP5549436B2 true JP5549436B2 (en) 2014-07-16

Family

ID=45604211

Family Applications (1)

Application Number Title Priority Date Filing Date
JP2010153887A Active JP5549436B2 (en) 2010-07-06 2010-07-06 Vehicle monitoring device

Country Status (1)

Country Link
JP (1) JP5549436B2 (en)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP6785480B2 (en) * 2019-01-28 2020-11-18 パナソニックＩｐマネジメント株式会社 Image synthesizer and control method

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP1115250B1 (en) * 1998-07-31 2012-06-06 Panasonic Corporation Method and apparatus for displaying image
JP3753681B2 (en) * 2000-07-19 2006-03-08 松下電器産業株式会社 Monitoring system
JP5439890B2 (en) * 2009-03-25 2014-03-12 富士通株式会社 Image processing method, image processing apparatus, and program

Also Published As

Publication number Publication date
JP2012019274A (en) 2012-01-26

Similar Documents

Publication Publication Date Title
US8817079B2 (en) Image processing apparatus and computer-readable recording medium
CN106464847B (en) Image compounding system and image synthesizing device and image synthesis method for it
JP3286306B2 (en) Image generation device and image generation method
CN104871084B (en) Self adaptation projector
JP3871614B2 (en) Driving assistance device
US9591280B2 (en) Image processing apparatus, image processing method, and computer-readable recording medium
JP6149676B2 (en) Image processing apparatus, image processing method, and program
US20200241731A1 (en) Virtual reality vr interface generation method and apparatus
JP5299296B2 (en) Vehicle periphery image display device and vehicle periphery image display method
US20170011267A1 (en) Image processing apparatus and method that synthesizes an all-round image of a vehicle&#39;s surroundings
US20130002809A1 (en) Image generating apparatus, synthesis table generating apparatus, and computer readable storage medium
JP7182920B2 (en) Image processing device, image processing method and program
US20180182160A1 (en) Virtual object lighting
CN110419061A (en) Mixed reality system with the synthesis of multi-source virtual content and the method using system generation virtual content
JP2015074436A (en) Image processing device, image processing method, and program
JP2009151524A (en) Image display method and image display apparatus
JP5521750B2 (en) Simulation program, simulation apparatus, and simulation method
US11140364B2 (en) Sensor fusion based perceptually enhanced surround view
JPWO2014033886A1 (en) Image processing apparatus, image processing method, and program
JP7040485B2 (en) Display control device and display control program
CN109764888A (en) Display system and display methods
JP7084400B2 (en) 3D surround view system
JP5549436B2 (en) Vehicle monitoring device
CN112896045A (en) Vehicle A-pillar blind area perspective method and system and vehicle
JP2015032936A (en) Imaging apparatus

Legal Events

Date Code Title Description
A621 Written request for application examination

Free format text: JAPANESE INTERMEDIATE CODE: A621

Effective date: 20130530

A977 Report on retrieval

Free format text: JAPANESE INTERMEDIATE CODE: A971007

Effective date: 20140128

A131 Notification of reasons for refusal

Free format text: JAPANESE INTERMEDIATE CODE: A131

Effective date: 20140304

A521 Written amendment

Free format text: JAPANESE INTERMEDIATE CODE: A523

Effective date: 20140331

TRDD Decision of grant or rejection written
A01 Written decision to grant a patent or to grant a registration (utility model)

Free format text: JAPANESE INTERMEDIATE CODE: A01

Effective date: 20140422

A61 First payment of annual fees (during grant procedure)

Free format text: JAPANESE INTERMEDIATE CODE: A61

Effective date: 20140505

R151 Written notification of patent or utility model registration

Ref document number: 5549436

Country of ref document: JP

Free format text: JAPANESE INTERMEDIATE CODE: R151