JP6202240B2 - Overhead image presentation device - Google Patents
Overhead image presentation device Download PDFInfo
- Publication number
- JP6202240B2 JP6202240B2 JP2013020115A JP2013020115A JP6202240B2 JP 6202240 B2 JP6202240 B2 JP 6202240B2 JP 2013020115 A JP2013020115 A JP 2013020115A JP 2013020115 A JP2013020115 A JP 2013020115A JP 6202240 B2 JP6202240 B2 JP 6202240B2
- Authority
- JP
- Japan
- Prior art keywords
- image
- vehicle
- projection plane
- plane
- virtual projection
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 240000004050 Pentaglottis sempervirens Species 0.000 claims description 54
- 235000004522 Pentaglottis sempervirens Nutrition 0.000 claims description 54
- 238000006243 chemical reaction Methods 0.000 claims description 36
- 238000003384 imaging method Methods 0.000 claims description 10
- 239000007787 solid Substances 0.000 claims description 7
- 238000010586 diagram Methods 0.000 description 27
- 230000014509 gene expression Effects 0.000 description 12
- 238000000034 method Methods 0.000 description 8
- 230000008569 process Effects 0.000 description 7
- 238000012790 confirmation Methods 0.000 description 4
- 230000008859 change Effects 0.000 description 3
- 238000001514 detection method Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 230000000007 visual effect Effects 0.000 description 2
- 238000005452 bending Methods 0.000 description 1
- 230000008901 benefit Effects 0.000 description 1
- 230000002596 correlated effect Effects 0.000 description 1
- 230000008878 coupling Effects 0.000 description 1
- 238000010168 coupling process Methods 0.000 description 1
- 238000005859 coupling reaction Methods 0.000 description 1
- 238000009434 installation Methods 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 230000004304 visual acuity Effects 0.000 description 1
Landscapes
- Image Processing (AREA)
- Closed-Circuit Television Systems (AREA)
- Image Analysis (AREA)
Description
この発明は俯瞰画像提示装置に係り、特に、運転支援等を目的として車載カメラで撮像した画像を仮想視点から見下ろした俯瞰画像に変換してモニタに表示する俯瞰画像提示装置に関する。 The present invention relates to a bird's-eye view image presentation device, and more particularly, to a bird's-eye view image presentation device that converts an image captured by a vehicle-mounted camera into a bird's-eye view image viewed from a virtual viewpoint for driving support and the like and displays the image on a monitor.
近年、車両後退時の運転支援を目的として、車両の上方から見下ろしたような俯瞰画像を運転者に提示する俯瞰画像提示装置が一般に普及されつつある。俯瞰画像提示装置は、車両後部に搭載した車載カメラ(バックカメラ)で撮像した画像を変換し、仮想的に車両の上方の視点から目標とする駐車枠と車両後端とを見下ろす俯瞰画像を作成し、作成した俯瞰画像をモニタに表示して運転者に提示する機能を有している。 In recent years, for the purpose of driving assistance when the vehicle is moving backward, an overhead image presentation device that presents a bird's-eye view image looking down from above the vehicle to the driver is becoming popular. The bird's-eye view image presentation device converts the image captured by the in-vehicle camera (back camera) mounted on the rear of the vehicle, and creates a bird's-eye view image that virtually looks down on the target parking frame and the rear end of the vehicle from the viewpoint above the vehicle And has a function of displaying the created overhead image on the monitor and presenting it to the driver.
この運転支援等を目的とした俯瞰画像を作成し、提示する機能は、次の2点を満たすことが望まれる。
1.車両周囲の物体を運転者が視認可能な大きさ、解像度で周辺表示視野を拡大して、死角をなくすこと
2.駐車枠等の元々地面上に描画されている物、ガイドライン等、仮想的に地面上に描画する物を変形させることなく表示すること(例えば、直線を直線として表示すること)
It is desirable that the function of creating and presenting an overhead image for the purpose of driving support satisfies the following two points.
1. 1. Enlarge the peripheral display field of view with a size and resolution that allows the driver to see objects around the vehicle to eliminate blind spots. Displaying objects drawn on the ground virtually, such as parking frames and objects originally drawn on the ground, guide lines, etc. without deforming them (for example, displaying straight lines as straight lines)
このような俯瞰画像提示装置としては、特開2008−141643号公報、特開2005−167638号公報に記載された装置がある。
特開2008−141643号公報には、近景面と遠景面とをつないだ仮想立体面を設定し、仮想立体面を用いた視点変換により、車載カメラが撮像した画像データから近景用と遠景用との2つの表示データを作成して表示する俯瞰画像提示装置が記載されている。
特開2005−167638号公報には、周囲360度の視野領域の映像を得て、視野中心付近では車両の一部を含む路面を俯瞰する画像データに変換し、視野中心から離れた部分では車両の上方を横方向に見た画像データに変換し、俯瞰と横方向との2つの画像データを表示する俯瞰画像提示装置が記載されている。
As such a bird's-eye view image presenting apparatus, there are apparatuses described in Japanese Patent Application Laid-Open Nos. 2008-141643 and 2005-167638.
In Japanese Patent Application Laid-Open No. 2008-141634, a virtual three-dimensional surface connecting a foreground surface and a far-field surface is set, and by using viewpoint conversion using the virtual three-dimensional surface, image data captured by an in-vehicle camera is used for near and far views. A bird's-eye view image presentation device that creates and displays the two display data is described.
Japanese Patent Laid-Open No. 2005-167638 obtains an image of a 360-degree field of view and converts it into image data overlooking the road surface including a part of the vehicle near the center of the field of view, and the vehicle away from the center of the field of view. A bird's-eye view image presentation device that converts the image data into the image data viewed in the horizontal direction and displays two image data in the bird's-eye view and the horizontal direction is described.
しかしながら、特開2008−141643号公報に記載された装置においては、近景用投影面と遠景用投影面とで別々に仮想投影面を作成し、近景用投影面と遠景用投影面との境界を結合している。また、特開2005−167638号公報に記載された装置においては、俯瞰用投影面と横方向用投影面とで別々に仮想投影面を作成し、俯瞰用投影面と横方向用投影面との境界を結合している。
つまり、特開2008−141643号公報、特開2005−167638号公報に記載された装置においては、仮想投影面は、2つの投影面で別々に仮想投影面を作成し、底面と壁面との境界を結合している。このため、2つの投影面の境界で不整合が生じるおそれがあり、運転者に違和感を与えることになる。また、不整合を回避するために、仮想投影面を作成する処理が複雑になる。
However, in the apparatus described in Japanese Patent Application Laid-Open No. 2008-141463, virtual projection planes are created separately for the foreground projection plane and the foreground projection plane, and the boundary between the foreground projection plane and the foreground projection plane is defined. Are connected. Further, in the apparatus described in Japanese Patent Laid-Open No. 2005-167638, a virtual projection plane is created separately for the overhead view projection plane and the horizontal projection plane, and the overhead view projection plane and the horizontal projection plane are Join the boundary.
In other words, in the apparatuses described in Japanese Patent Application Laid-Open Nos. 2008-141634 and 2005-167638, the virtual projection plane is created separately on two projection planes, and the boundary between the bottom surface and the wall surface is created. Are combined. For this reason, mismatching may occur at the boundary between the two projection planes, which gives the driver a sense of incongruity. In addition, in order to avoid inconsistencies, the process of creating a virtual projection plane becomes complicated.
この発明は、俯瞰画像をシームレスに表示して運転者に違和感を与えないようにすることができ、仮想投影面を容易に設定することができ、また、表示範囲を拡大して死角を減らすとともに、地面に描画されている物を変形させることなく表示することができ、遠方の景観の湾曲を抑制することができる俯瞰画像提示装置を提供することを目的とする。 The present invention can seamlessly display a bird's-eye view image so as not to give the driver a sense of incongruity, can easily set a virtual projection plane, and expands the display range to reduce blind spots. An object of the present invention is to provide a bird's-eye view image presentation device that can display an object drawn on the ground without being deformed and can suppress the curvature of a distant landscape.
この発明は、車両周囲を撮像する複数の撮像手段と、前記撮像手段により撮像された画像を車両上方の仮想視点から見下ろした俯瞰画像に変換する画像変換手段と、前記画像変換手段により変換された俯瞰画像を表示する表示手段とを備えた俯瞰画像提示装置において、前記俯瞰画像を投影する仮想投影面は、車両を平面上に置いた状態で、平面に対して水平であり任意の高さに位置する水平面によって仮想投影面を切断した場合に仮想投影面の水平方向切断面が超楕円形状で表されるとともに、超楕円の中心を通り平面に対して垂直であり任意の向きである垂直面によって仮想投影面を切断した場合に仮想投影面の垂直方向切断面がバスタブ曲線で表される立体曲面であり、かつ、1つの数式で表され、前記俯瞰画像は前記超楕円の長半径または短半径位置で分割し、前記複数の撮像手段により撮像された画像を該分割面で結合することを特徴とする。 The present invention includes a plurality of imaging means for imaging a vehicle periphery, an image conversion means for converting an image captured by the imaging means into a bird's-eye view image looking down from a virtual viewpoint above the vehicle, and the image conversion means. In a bird's-eye view image display device comprising a display means for displaying a bird's-eye view image, the virtual projection plane for projecting the bird's-eye view image is horizontal to the plane and at an arbitrary height with the vehicle placed on the plane. When the virtual projection plane is cut by the horizontal plane, the horizontal cut plane of the virtual projection plane is represented by a super-elliptical shape, and the vertical plane that passes through the center of the super-ellipse and is perpendicular to the plane and in any orientation a three-dimensional curved vertical cutting plane of the virtual projection plane is represented by a bathtub curve when cut a virtual projection plane by, and is represented by a single formula, the overhead image is the hyperelliptic long radius Other splits a short radial position, the image captured by the plurality of imaging means, characterized in that bind with the dividing plane.
この発明の仮想投影面は、水平方向切断面が超楕円形状で表されるとともに垂直方向切断面がバスタブ曲線で表される立体曲面であるため、平面状の底面と底面周囲から斜め上方に延びる壁面とから構成される。このため、この発明は、表示範囲を拡大して死角を減らすとともに、地面に描画されている物を変形させることなく表示することができる。
この発明の仮想投影面は、底面に平行でかつ底面と仮想視点との間に位置する任意の平面による切断面が底面の形状を拡大した形状であるため、遠方の景観の湾曲を抑制することができる。
この発明の仮想投影面は、1つの数式で表される曲面であるため、俯瞰画像をシームレスに表示して運転者に違和感を与えないようにすることができ、また、仮想投影面を容易に設定することができる。
The virtual projection plane according to the present invention is a solid curved surface in which the horizontal cut surface is represented by a super-elliptical shape and the vertical cut surface is represented by a bathtub curve, and thus extends obliquely upward from the planar bottom surface and the periphery of the bottom surface. It consists of a wall. For this reason, this invention can display without expanding the display range and reducing the blind spot and without deforming the object drawn on the ground.
The virtual projection plane according to the present invention suppresses the curvature of a distant landscape because the cut surface formed by an arbitrary plane that is parallel to the bottom surface and located between the bottom surface and the virtual viewpoint is an enlarged shape of the bottom surface. Can do.
Since the virtual projection surface of the present invention is a curved surface represented by a single mathematical expression, it is possible to seamlessly display a bird's-eye view image so as not to give the driver a sense of incongruity. Can be set.
以下図面に基づいて、この発明の実施例を説明する。 Embodiments of the present invention will be described below with reference to the drawings.
図1〜図28は、この発明の実施例を示すものである。図1に示すように、車両1の俯瞰画像提示装置2は、車両1の周囲を撮像する撮像手段のカメラ3と、カメラ3により撮像されて画像Gcを車両上方の仮想視点Svから見下ろした俯瞰画像Gpに変換する画像変換手段4と、画像変換手段4により変換された俯瞰画像Gpを表示する表示手段のモニタ5とを備えている。カメラ3は、車両1の後部に取り付けられ、車両1の後方を撮像する。画像変換手段4は、俯瞰画像提示装置2の処理手段6に設けられている。
処理手段6には、前記カメラ3とモニタ5とを接続するとともに、後退確認スイッチ7と俯瞰画像表示選択スイッチ8と操舵角検出手段の操舵角センサ9とを接続している。後退確認スイッチ7は、俯瞰画像提示装置2を起動してモニタ5に俯瞰画像Gpを表示させる。俯瞰画像表示選択スイッチ8は、モニタ5に表示させる画像をカメラ3が撮像した画像Gcと俯瞰画像Gpとに選択して切り替える。操舵角センサ9は、車両1のステアリングハンドルの操舵角を検出する。
前記仮想視点Svは、図2(B)、(C)、(D)に示すように、車両1の上方から見下ろすように配置した仮想カメラ10の撮像位置に設定されている。前記俯瞰画像Gpは、平面11上に車両1を置いた場合、車両1の後部に取り付けられたカメラ3が撮像した画像Gcを、車両1の上方に配置した仮想カメラ10の仮想視点Svから見下ろす方向に設定した仮想投影面Psに投影した画像である。
前記仮想投影面Psは、図2(A)〜(D)に示すように、平面視で楕円形状をしている。この仮想投影面Psは、図3に示すように、仮想投影面Psの高さye方向の設定にアレニウス(Arrehnius)化学反応式(以下、「アレニウス式」と記す。)を応用し、楕円形状とバスタブ曲線で表される立体曲面(バスタブ投影面)からなり、平面11上の底面12と底面12の周囲から斜め上方に延びる壁面13とから構成される。
1 to 28 show an embodiment of the present invention. As shown in FIG. 1, the overhead image presentation device 2 of the vehicle 1 includes a camera 3 that is an imaging unit that captures the surroundings of the vehicle 1, and an overhead view obtained by looking down the image Gc taken from the virtual viewpoint Sv above the vehicle. The image conversion means 4 which converts into the image Gp, and the monitor 5 of the display means which displays the bird's-eye view image Gp converted by the image conversion means 4 are provided. The camera 3 is attached to the rear part of the vehicle 1 and images the rear of the vehicle 1. The image conversion means 4 is provided in the processing means 6 of the overhead image presentation device 2.
The processing means 6 is connected to the camera 3 and the monitor 5, and is connected to a reverse confirmation switch 7, an overhead image display selection switch 8, and a steering angle sensor 9 of the steering angle detection means. The reverse confirmation switch 7 activates the overhead view image presentation device 2 to display the overhead view image Gp on the monitor 5. The overhead image display selection switch 8 selects and switches the image to be displayed on the monitor 5 between the image Gc captured by the camera 3 and the overhead image Gp. The steering angle sensor 9 detects the steering angle of the steering handle of the vehicle 1.
As shown in FIGS. 2B, 2C, and 2D, the virtual viewpoint Sv is set at an imaging position of the virtual camera 10 arranged so as to look down from above the vehicle 1. When the vehicle 1 is placed on the plane 11, the bird's-eye view image Gp looks down from the virtual viewpoint Sv of the virtual camera 10 disposed above the vehicle 1, the image Gc captured by the camera 3 attached to the rear portion of the vehicle 1. It is the image projected on the virtual projection plane Ps set to the direction.
The virtual projection plane Ps has an elliptical shape in plan view as shown in FIGS. The virtual projection plane Ps, as shown in FIG. 3, Arrhenius in the height y e direction of setting a virtual projection plane Ps (Arrehnius) Chemical reaction formula (hereinafter, referred to as "Arrhenius equation".) Applying, oval It consists of a solid curved surface (bathtub projection plane) represented by a shape and a bathtub curve, and is composed of a bottom surface 12 on the plane 11 and a wall surface 13 extending obliquely upward from the periphery of the bottom surface 12.
図4に示すように、仮想投影面Psの座標系を[xe,ye,ze]、底面12の長半径aと短半径bの比率nとし、底面12上のYe=Ye0=0で、底面12周囲(壁面13)までの点[xe,ye,ze]=[xe0,0,ze0](底面12の形状)を「高さyeで平面11に対して水平に切断した楕円面の長半径a、短半径bで表される楕円一般式」(式1)でモデル化し、b=n・aで展開して「長半径aを媒介変数とするアレニウス式に基づく高さye」が相関するモデル式(式2)で示す立体曲面の仮想投影面Psとする。
このモデル式(式2)で示す仮想投影面Psは、図5に示すように、平面11に対して水平に切断した楕円面の長半径方向(X方向)の曲率を変化させて短半径方向(Z方向)の曲率を小さくし、高さyeに関係なく曲率が上方に一定の相似楕円形状とすることで、図6に示すように、車両1の後方の遠方景観に対して、湾曲変形を抑えた状態で直線度を確保した画像を表示できる。
ただし、仮想投影面Psの底面12の形状を、平面11に対して水平に切断した楕円面の短半径方向(Z方向)の曲率を小さくした楕円形状とした場合は、車両1の左右方向(X方向)の曲率が大きくなり、「左右視野の確保」と「車両1の後方の遠方景観に関する湾曲の軽減・直線度の向上」との効果を両立させることが困難となる。すなわち、図7に示すように、仮想投影面Psの底面12の形状を楕円形状とした場合は、壁面13の全周が曲率の大きな曲面となり、湾曲変形が強かった。
As shown in FIG. 4, the coordinate system of the virtual projection plane Ps is [x e , y e , z e ], the ratio n of the major radius a to the minor radius b of the bottom surface 12, and Y e = Y e0 on the bottom surface 12. = 0, the point [x e , y e , z e ] = [x e0 , 0, z e0 ] (the shape of the bottom surface 12) up to the periphery (wall surface 13) of the bottom surface 12 is set to the plane 11 at the height y e On the other hand, the model is modeled by an elliptical general expression represented by a major radius a and a minor radius b of the ellipse horizontally cut (formula 1), and developed by b = n · a, with “major radius a as a parameter. A virtual projection plane Ps of a three-dimensional curved surface represented by a model formula (Formula 2) correlated with a height y e based on the Arrhenius formula.
As shown in FIG. 5, the virtual projection plane Ps represented by this model formula (Formula 2) is obtained by changing the curvature in the major radius direction (X direction) of the ellipsoid cut horizontally with respect to the plane 11. to reduce the curvature of the (Z-direction), by curvature constant similar elliptical upward regardless height y e, as shown in FIG. 6, relative to the rear of the distant landscape of the vehicle 1, curved An image in which straightness is ensured while suppressing deformation can be displayed.
However, when the shape of the bottom surface 12 of the virtual projection plane Ps is an elliptical shape in which the curvature in the short radius direction (Z direction) of the ellipsoid cut horizontally with respect to the plane 11 is reduced, The curvature in the X direction becomes large, and it becomes difficult to achieve both the effects of “securing the left and right visual field” and “reducing the curvature and improving the straightness of the distant landscape behind the vehicle 1”. That is, as shown in FIG. 7, when the shape of the bottom surface 12 of the virtual projection plane Ps is an elliptical shape, the entire circumference of the wall surface 13 is a curved surface having a large curvature, and the curved deformation is strong.
そこで、この俯瞰画像提示装置2は、俯瞰画像Gpを投影する仮想投影面Psを以下のように設定している。
仮想投影面Psは、図8(B)、(C)、(D)に示すように、車両1を平面11上に置いた状態で、平面11に対して水平であり任意の高さに位置する水平面Phによって仮想投影面Psを切断した場合に仮想投影面Psの水平方向切断面が超楕円形状で表されるとともに、超楕円の中心Mを通り平面11に対して垂直であり任意の向きである垂直面Pvによって仮想投影面Psを切断した場合に仮想投影面Psの垂直方向切断面がバスタブ曲線(故障率曲線)で表される立体曲面(超楕円形状のバスタブ投影面)であり、かつ、1つの数式(図9参照)で表される。
前記仮想投影面Psは、図8(A)〜(D)に示すように、平面視で超楕円形状をしている。
この仮想投影面Psは、図9に示すように、仮想投影面Psの座標系を[xe,ye,ze]、底面12の長半径aと短半径bの比率nとし、さらに、図4のアレニウス式を応用した楕円の一般式(式2)に関して変換次数(k乗)を調整可能とする以下の公式で代替する。超楕円形状の底面12上のYe=Ye0=0で、底面12周囲(壁面13)までの点[xe,ye,ze]=[xe0,0,ze0](底面12の形状)を超楕円の一般式(式3)で定義し、b=n・aで展開して超楕円形状とバスタブ曲線とで表される立体曲面を表す1つの数式(式4)を得る。
つまり楕円の公式を、ガブリエル・ラメ(Gabriel Lame)の提唱する超楕円形状(角丸四角形状)の式(ガブリエル・ラメの超楕円式)に置き換えて、超楕円形状とバスタブ曲線とで表される立体曲面を表す1つの数式を得る。ガブリエル・ラメの超楕円式は、モデル式の次数kを大きくするほど「楕円が超楕円(角丸四角)に近づく」、また、「楕円も超楕円も次数を問わず縦・横各半径(長半径・短半径)が共通」という特徴を有している。前記1つの数式により、断面における長半径aを媒介変数とする仮想投影面Ps:Ye=A・(−1+EXP[B・(a)])を設定する。
この仮想投影面Psは、図10、図11に示すように、底面12の形状を平面11に対して水平に切断した断面を超楕円形状(角丸四角形状)とし、壁面13の形状を高さyeに関係なく曲率が一定の相似超楕円形状としたので、図12に示すように、車両1の後方の直線度を維持でき、遠方景観の湾曲を回避できる。
Therefore, this overhead image presentation device 2 sets the virtual projection plane Ps on which the overhead image Gp is projected as follows.
As shown in FIGS. 8B, 8 </ b> C, and 8 </ b> D, the virtual projection plane Ps is horizontal with respect to the plane 11 and positioned at an arbitrary height with the vehicle 1 placed on the plane 11. When the virtual projection plane Ps is cut by the horizontal plane Ph, the horizontal cut plane of the virtual projection plane Ps is expressed in a super-elliptical shape, passes through the center M of the super-ellipse, and is perpendicular to the plane 11 and has an arbitrary orientation. When the virtual projection plane Ps is cut by the vertical plane Pv, the vertical cut plane of the virtual projection plane Ps is a solid curved surface (super elliptical bathtub projection plane) represented by a bathtub curve (failure rate curve), And it is represented by one mathematical expression (see FIG. 9).
As shown in FIGS. 8A to 8D, the virtual projection plane Ps has a super-elliptical shape in plan view.
As shown in FIG. 9, the virtual projection plane Ps has a coordinate system [x e , y e , z e ] of the virtual projection plane Ps, a ratio n between the major radius a and the minor radius b of the bottom surface 12, Instead of the general formula (formula 2) of the ellipse applying the Arrhenius formula of FIG. 4, the conversion formula (kth power) can be adjusted by the following formula. When Y e = Y e0 = 0 on the bottom surface 12 of the super-elliptical shape, the point [x e , y e , z e ] = [x e0 , 0, z e0 ] (bottom surface 12) to the periphery of the bottom surface 12 (wall surface 13). ) Is defined by a general formula of the super ellipse (formula 3), and developed by b = n · a to obtain one mathematical formula (formula 4) representing the solid curved surface represented by the superelliptical shape and the bathtub curve. .
In other words, the ellipse formula is replaced by the super-elliptical shape (rounded square shape) proposed by Gabriel Lame (Gabriel Lame's super-elliptical formula), and expressed by the super-elliptical shape and bathtub curve. One mathematical expression representing a solid curved surface is obtained. Gabriel Lame's super-elliptic formula is that the ellipse gets closer to the super-ellipse (rounded rectangle) as the order k of the model formula increases. The long radius and short radius are common. The virtual projection plane Ps: Y e = A · (−1 + EXP [B · (a)]) using the long radius a in the cross section as a parameter is set by the one mathematical formula.
As shown in FIGS. 10 and 11, the virtual projection plane Ps has a cross section obtained by cutting the shape of the bottom surface 12 horizontally with respect to the plane 11 to a super elliptical shape (rounded square shape), and the shape of the wall surface 13 is high. since the y e in regardless of curvature is constant for similar super elliptical shape, as shown in FIG. 12, can be maintained behind the straightness of the vehicle 1, it can be avoided curvature of distant scenery.
このように、この俯瞰画像提示装置2の仮想投影面Psは、水平方向切断面が超楕円形状で表されるとともに垂直方向切断面がバスタブ曲線で表される立体曲面であるため、平面状の底面12と底面12の周囲から斜め上方に延びる壁面13とから構成される。このため、この俯瞰画像提示装置2は、表示範囲を拡大して死角を減らすとともに、地面に描画されている物を変形させることなく表示することができる。
この俯瞰画像提示装置2の仮想投影面Psは、底面12に平行でかつ底面12と仮想視点Svとの間に位置する任意の平面による切断面が底面12の形状を拡大した形状であるため、遠方の景観の湾曲を抑制することができる。
この俯瞰画像提示装置2の仮想投影面Psは、1つの数式(式4)で表される曲面であるため、俯瞰画像Gpをシームレスに表示して運転者に違和感を与えないようにすることができ、また、仮想投影面Psを容易に設定することができる。
なお、仮想投影面Psの高さye方向の計算式では、媒介変数a(断面長半径)を適用するアレニウス式を応用したが、図13に示すように、アレニウス式以外にも、媒介変数a(断面長半径)を適用するシグモイド曲線、双曲線関数tanhを応用して同様のバスタブ曲線を求めることができる。但し、シグモイド曲線、双曲線関数tanhによって仮想投影面Psの高さye方向の曲面を設定する場合は、ともに高さye方向に限界点(収束点)が発生する。よって、俯瞰画像Gpの場合は、収束高さが仮想カメラ10の設置高さ、つまり仮想視点Svの高さ位置よりも上方になるようパラメータ設定を行う必要がある。
As described above, the virtual projection plane Ps of the overhead view image presentation device 2 is a solid curved surface in which the horizontal cut surface is represented by a super-elliptical shape and the vertical cut surface is represented by a bathtub curve. It comprises a bottom surface 12 and a wall surface 13 extending obliquely upward from the periphery of the bottom surface 12. For this reason, this bird's-eye view image presentation device 2 can display the object displayed on the ground without deforming it while expanding the display range to reduce the blind spot.
Since the virtual projection plane Ps of the overhead view image presentation device 2 is a shape obtained by enlarging the shape of the bottom surface 12 with a cut surface formed by an arbitrary plane parallel to the bottom surface 12 and positioned between the bottom surface 12 and the virtual viewpoint Sv. The curvature of the distant landscape can be suppressed.
Since the virtual projection plane Ps of the overhead image presentation device 2 is a curved surface represented by one mathematical expression (Equation 4), the overhead image Gp may be displayed seamlessly so as not to give the driver a sense of incongruity. In addition, the virtual projection plane Ps can be easily set.
In the formula for the height y e direction of the virtual projection plane Ps, it has been applied Arrhenius equation to apply parametric a (sectional length radius), as shown in FIG. 13, in addition to the Arrhenius equation, parametric A similar bathtub curve can be obtained by applying the sigmoid curve to which a (cross-section radius) is applied and the hyperbolic function tanh. However, when setting the height y e direction of the curved surface of the virtual projection plane Ps sigmoidal curve, the hyperbolic function tanh is the limit point (convergence point) is generated both in the height y e direction. Therefore, in the case of the bird's-eye view image Gp, it is necessary to set parameters so that the convergence height is higher than the installation height of the virtual camera 10, that is, the height position of the virtual viewpoint Sv.
前記俯瞰画像提示装置2は、図14に示すように、車両1を駐車枠(ガイドライン)14に移動させる状況において、位置A付近で車両1を直進後退させている場合、モニタ5に超楕円形状とバスタブ曲線とで表される立体曲面からなる仮想投影面Psにより変換された俯瞰画像Gpを表示することで、図15に示すように、視野の確保(死角削減)と画面表示の湾曲変形の改善を図ることができる。
一方、この俯瞰画像提示装置2は、図14に示すように、車両1を駐車枠(ガイドライン)14に移動させる状況において、位置B付近で車両1を旋回後退させている場合、図16(A)に示すように、モニタ5に表示される俯瞰画像Gpの超楕円形状のコーナ部(角丸四角の角部)付近では、図16(B)に矢印で示すように、背景画像が車両1の動きと一致しない直角に近い軌跡で移動する。このような背景画像の移動は、運転者に違和感を与える場合がある。
ところで、前述のようにガブリエル・ラメの超楕円式は、「楕円も超楕円も次数を問わず縦・横各半径(長半径・短半径)が共通」という特徴を有しているので、図9に示す超楕円形状を表す1つの数式(式4)の次数kと長半径aおよび短半径bとは、それぞれ独立したパラメータであり、値の大きさが相互に影響することはない。よって、異なる乗数(次数k)の底面12、あるいは壁面13(バスタブ断面)であっても、バスタブ座標系[xe,ye,ze]におけるye軸回り[0,90,180,270]deg位置では、長半径aおよび短半径bに起因する同じ周回位置で必ず結像することになる。
そこで、画像変換手段4には、図17(A)、(B)に示すように、俯瞰画像Gpを左右に分割し、俯瞰画像Gpの左右で異なる次数kを持ち長半径・短半径が共通の投影面(バスタブ投影面)を予め複数準備しておき、車両1の旋回走行中に操舵角センサ9が検出するステアリングハンドルの操舵角に基づき、図18(A)に示すように、左右の投影面を組み合わせた仮想投影面Psを切り替えることで、図18(B)に示すように、超楕円形状のコーナ部付近での背景の移動を見た目に自然な表示に近づけることができる。
なお、実際の仮想投影面Psの切り替え処理では、予め画像変換処理を行うための画像変換テーブル(オフセットテーブル)をモニタ5の画面の左半分・右半分サイズ単位でガブリエル・ラメ式の乗数(次数k)のパターン分だけ準備しておき、直進後退中は同じ乗数(次数k)のテーブルを組み合わせ、旋回後退中は操舵角に応じて異なる乗数(次数k)のテーブルを組み合わせる。これにより、旋回後退中は、車両1の旋回に沿った自然な背景画像の移動を実現できるテーブルの組み合わせに切り替えて、より車両1の動きに沿った俯瞰画像Gpを表示することができる。
As shown in FIG. 14, when the vehicle 1 is moved straight back and forth in the vicinity of the position A in the situation where the vehicle 1 is moved to the parking frame (guideline) 14 as shown in FIG. By displaying the bird's-eye view image Gp converted by the virtual projection plane Ps composed of a three-dimensional curved surface represented by a bathtub curve, as shown in FIG. 15, securing of the visual field (reducing blind spots) and curving deformation of the screen display are achieved. Improvements can be made.
On the other hand, as shown in FIG. 14, the bird's-eye view image presentation device 2 is configured to turn the vehicle 1 around the position B in a situation where the vehicle 1 is moved to the parking frame (guideline) 14 and turn back in FIG. As shown in FIG. 16B, the background image is the vehicle 1 in the vicinity of the super-elliptical corner portion (cornered corner) of the overhead image Gp displayed on the monitor 5. It moves with a trajectory close to a right angle that does not match the movement of. Such movement of the background image may give an uncomfortable feeling to the driver.
By the way, as described above, the super-elliptical formula of Gabriel Lame has a feature that “the ellipse and the super-ellipse have the same vertical and horizontal radii (major radius and minor radius) regardless of the order”. The order k, the major radius a, and the minor radius b of one mathematical formula (Formula 4) representing the super-elliptical shape shown in FIG. 9 are independent parameters, and the magnitudes of the values do not affect each other. Therefore, even in the bottom surface 12 of the different multipliers (order k) or the wall surface 13, (bathtub section), Bath coordinates [x e, y e, z e] y e axis in [0, 90, 180, 270 ] In the deg position, an image is always formed at the same rounding position caused by the major radius a and the minor radius b.
Therefore, as shown in FIGS. 17A and 17B, the image conversion means 4 divides the overhead view image Gp into left and right parts, and has different degrees k on the left and right sides of the overhead view image Gp, and has a common major radius and minor radius. A plurality of projection planes (bathtub projection planes) are prepared in advance, and based on the steering angle of the steering wheel detected by the steering angle sensor 9 during the turning of the vehicle 1, as shown in FIG. By switching the virtual projection plane Ps that combines the projection planes, as shown in FIG. 18B, it is possible to bring the background movement near the corner portion of the super-elliptical shape closer to a natural display.
In the actual switching process of the virtual projection plane Ps, an image conversion table (offset table) for performing image conversion processing in advance is a Gabriel-Lame multiplier (order) in units of the left half and right half sizes of the screen of the monitor 5. k) patterns are prepared, and tables of the same multiplier (order k) are combined during straight reverse, and tables of different multipliers (order k) are combined according to the steering angle during reverse turning. Thereby, during turning backward, it is possible to display a bird's-eye view image Gp along the movement of the vehicle 1 by switching to a combination of tables that can realize natural movement of the background image along the turning of the vehicle 1.
このように、画像変換手段4は、車両1の状態に基づいて超楕円で表される投影面の次数kを変化させるため、運転者に与える違和感を抑制することができる。具体的には、俯瞰画像提示装置2に車両1の操舵角を検出する操舵角センサ9を備え、画像変換手段4は操舵角センサ9により検出された操舵角に基づいて超楕円の次数kを変化させるため、車両1の旋回中には車両1の旋回に合わせて自然に背景が変化するように表示して、運転者に違和感を与えないようにすることができる。
さらに、画像変換手段4は、俯瞰画像Gpを左右に分割し、操舵角センサ9により検出された操舵角に基づいて左右の画像の投影面の次数kを変化させるため、車両1の旋回中には車両1の旋回に合わせて自然に背景が変化するように表示して、運転者に違和感を与えないようにすることができる一方、左右の表示範囲を確保し、さらに遠方の景観の湾曲を軽減することができる。
また、この俯瞰画像提示装置2は、仮想投影面Gpを構成する壁面13が超楕円形状であるため、左右の画像の次数を変化させても画像の接合部でずれが生じないようにすることができる。
Thus, since the image conversion means 4 changes the order k of the projection surface represented by a super ellipse based on the state of the vehicle 1, it can suppress a sense of discomfort given to the driver. Specifically, the overhead view image presentation device 2 includes a steering angle sensor 9 that detects the steering angle of the vehicle 1, and the image conversion means 4 calculates the degree k of the super ellipse based on the steering angle detected by the steering angle sensor 9. In order to change, it can display so that a background may change naturally according to the turning of the vehicle 1 during the turning of the vehicle 1, and it can prevent a driver from feeling uncomfortable.
Further, the image conversion means 4 divides the overhead image Gp into left and right and changes the order k of the left and right image projection planes based on the steering angle detected by the steering angle sensor 9, so that the vehicle 1 is turning. Can be displayed so that the background changes naturally with the turning of the vehicle 1 so as not to give the driver a sense of incongruity. Can be reduced.
In addition, since the wall surface 13 constituting the virtual projection plane Gp has a super-elliptical shape, this overhead view image presentation device 2 is configured so that no deviation occurs at the joint portion of the image even if the order of the left and right images is changed. Can do.
ところで、地面投影面に基づく従来の俯瞰画像では、図19(A)に示すように、変換画像に関して「表示変形の小さい領域」だけを抽出し、モニタの画面サイズに合わせて拡大する処理が行なわれていた。この場合、モニタの画面サイズに合わせて拡大する処理は仮想カメラを低い位置に設置し、地面付近を接写する状態としてモデルを定義することができる。その結果、画面上で表示される駐車枠(ガイドライン)14の表示サイズは大きく、視力に衰えを覚える高齢者などを対象とした駐車支援画像の仕様としては見やすいものとなっていた。
逆に、前述超楕円形状とバスタブ曲線で表される仮想投影面Psによる広範囲俯瞰画像では、仮想カメラ10を上方に設置して仮想視点Svの位置を高くしているため、図19(B)に示すように、駐車枠(ガイドライン)14等は画面に対して比較的小さな表示サイズとなり、少々視認しづらい場合が想定される。
よって、広範囲俯瞰画像で示される駐車枠(ガイドライン)14付近の表示サイズを、従来の俯瞰画像寄りに「大きく」表示する広範囲俯瞰画像を設定するためには、仮想カメラ10を下方に設置して仮想視点Svを低い位置に設定しなければならない。
駐車枠14付近の表示サイズの拡大を目的に、仮想視点Svを低く設定した変換画像を図20に示す。図20(A)に示すように、仮想視点Svを低く設定した場合、図20(B)に示すように、変換画像上の左右表示範囲が、地面投影による従来の俯瞰画像(図19(B))と殆ど差がないため、立体投影面を利用するメリットが失われる。
そこで、左右の視野を拡大するために、図21(A)に示すように、超楕円形状とバスタブ曲線で表される仮想投影面Psの底面12のX軸方法のRx半径の値を縮小する処理、つまり長半径aと短半径bの比率nを拡大する処理を行なうことで、図21(C)に示すように、左右表示範囲を拡大することができる。ところが、図21(C)の画像では、従来の俯瞰画像(図21(B))よりも左右表示範囲を拡大し、死角を削減することができた一方で、車両1の後方遠方部分の駐車枠14の直線度が立体曲面に引き寄せられて湾曲し、平行度が異なるハの字(遠方に向かって先窄まり)に変形した俯瞰画像となるため、駐車支援の効果が低下してしまうことになる。
このように、仮想視点Svを低い位置に設定した俯瞰画像Gpに関して、半球や楕円体の特徴が投影面のプロフィール要素に反映された投影面では、表示上の地面付近の解像度が向上してしまうため、投影面曲率による湾曲の影響が、より強く変換画像に反映されてしまうことになる。また、方形のモニタ5の表示画像に対して曲面を持つ投影面で俯瞰処理を試みる場合、画面上部コーナ部(左上・右上隅角部)の補正を均等に行なうことは難しく、やはり違和感を伴う表示変形が発生する。
By the way, in the conventional bird's-eye view image based on the ground projection plane, as shown in FIG. 19 (A), only the “region with small display deformation” is extracted from the converted image, and the process of enlarging it according to the screen size of the monitor is performed. It was. In this case, in the process of enlarging in accordance with the screen size of the monitor, the model can be defined as a state where the virtual camera is installed at a low position and the vicinity of the ground is photographed. As a result, the display size of the parking frame (guideline) 14 displayed on the screen is large, and it is easy to see the specification of the parking assistance image for elderly people who are weak in visual acuity.
On the other hand, in the wide-range overhead view image with the virtual projection plane Ps represented by the above-mentioned super-elliptical shape and bathtub curve, the virtual camera 10 is installed upward and the position of the virtual viewpoint Sv is raised, so FIG. As shown in FIG. 4, the parking frame (guideline) 14 and the like have a relatively small display size with respect to the screen, and it is assumed that the parking frame (guideline) 14 is slightly difficult to view.
Therefore, in order to set a wide range overhead view image that displays the display size near the parking frame (guideline) 14 shown in the wide range overhead view image closer to the conventional overhead view image, the virtual camera 10 is installed below. The virtual viewpoint Sv must be set to a low position.
FIG. 20 shows a converted image in which the virtual viewpoint Sv is set low for the purpose of enlarging the display size near the parking frame 14. As shown in FIG. 20A, when the virtual viewpoint Sv is set low, as shown in FIG. 20B, the left and right display range on the converted image is a conventional overhead view image by ground projection (FIG. 19B). Since there is almost no difference from)), the advantage of using the stereoscopic projection surface is lost.
Therefore, in order to enlarge the left and right fields of view, as shown in FIG. 21A, the value of the Rx radius of the X-axis method of the bottom surface 12 of the virtual projection plane Ps represented by the super-elliptical shape and the bathtub curve is reduced. By performing the process, that is, the process of enlarging the ratio n between the long radius a and the short radius b, the left and right display range can be expanded as shown in FIG. However, in the image of FIG. 21 (C), the left and right display range can be enlarged and the blind spots can be reduced as compared with the conventional overhead view image (FIG. 21 (B)). Since the linearity of the frame 14 is attracted to the three-dimensional curved surface and is curved, and the overhead view image is deformed into a letter C having different parallelism (constricted toward the distance), the parking support effect is reduced. become.
As described above, regarding the overhead image Gp with the virtual viewpoint Sv set at a low position, the resolution near the ground on the display is improved on the projection plane in which the features of the hemisphere and the ellipsoid are reflected in the profile elements of the projection plane. Therefore, the influence of the curvature due to the projection surface curvature is more strongly reflected in the converted image. In addition, when trying to perform a bird's-eye view process on a projection surface having a curved surface with respect to the display image of the square monitor 5, it is difficult to evenly correct the upper corner portion (upper left corner and upper right corner portion) of the screen, which is also uncomfortable. Display deformation occurs.
このような問題について、ガブリエル・ラメの提唱する超楕円形状に基づく底面12を持つ仮想投影面Ps(バスタブ投影面)に関しては、方形のモニタ5の表示画像に対して方形のプロフィールを要素に持つことから、湾曲変形および画面上部の2つの隅角部の補完についても、充分に対応可能である。
そこで、この俯瞰画像提示装置2は、画像変換手段4によって、仮想視点Svの位置を低くした時には、図22(A)に示す従来の楕円形状に基づく仮想投影面に対して、図22(B)に示すように超楕円形状に基づく底面12を持つ仮想投影面Psの車両1の横方向(X軸方向)の長さを短くするとともに、図22(C)に示すように超楕円の次数kを大きくする。
このように、画像変換手段4は、仮想視点Svの位置を低くした時には、仮想投影面Psの車両1の横方向(X軸方向)の長さを短くするとともに、超楕円の次数kを大きくすることことで、仮想視点Psの位置を低くしても、図22(A)に示す従来の画像に対して、左右の表示範囲をより広く確保することができるとともに遠方の景観の湾曲を軽減することができ、より駐車しやすい俯瞰画像Gpに変換することができる。
Regarding such a problem, with respect to the virtual projection plane Ps (bathtub projection plane) having the bottom surface 12 based on the super-elliptical shape proposed by Gabriel Lame, the display image of the square monitor 5 has a square profile as an element. Therefore, it is possible to sufficiently cope with the curved deformation and the complementation of the two corners at the top of the screen.
In view of this, the overhead view image presentation device 2 is configured so that when the position of the virtual viewpoint Sv is lowered by the image conversion means 4, the virtual projection plane based on the conventional elliptical shape shown in FIG. ), The length of the virtual projection plane Ps having the bottom surface 12 based on the super-elliptical shape in the lateral direction (X-axis direction) of the vehicle 1 is shortened, and the order of the super-ellipse is represented as shown in FIG. Increase k.
As described above, when the position of the virtual viewpoint Sv is lowered, the image conversion unit 4 shortens the length of the virtual projection plane Ps in the lateral direction (X-axis direction) of the vehicle 1 and increases the degree k of the super ellipse. By doing this, even if the position of the virtual viewpoint Ps is lowered, the left and right display ranges can be secured wider than the conventional image shown in FIG. Can be converted into an overhead image Gp that is easier to park.
前述の俯瞰画像提示装置2は、1つのカメラ3により撮像した車両1の後方の画像を俯瞰画像Gpに変換してモニタ5に表示させたが、車両1に取り付けた複数のカメラ3により撮像した車両1の周囲の画像を全周囲俯瞰画像に変換してモニタ5に表示させることもできる。この場合、図23に示すように、仮想投影面Psの座標系[xe,ye,ze]の原点は、ワールド座標系の車両1の中心位置付近に設定するのが望ましい。
例えば、図24に示すように、車両全周の俯瞰画像を提示する俯瞰画像提示装置2は、車両1の前方の景観を撮像する前景カメラ3fと、車両1の後方の景観を撮像する後景カメラ3bと、前景カメラ3fが撮像した前景画像Gcfと後景カメラ3bが撮像した後景画像Gcbとを車両全周の俯瞰画像Gpwに変換する画像変換手段4と、俯瞰画像Gpwを表示するモニタ5とを備えている。俯瞰画像提示装置2の処理手段6には、前述カメラ3f、3bとモニタ5とを接続するとともに、後退確認スイッチ7と俯瞰画像表示選択スイッチ8と操舵角検出手段の操舵角センサ9とを接続している。
画像変換手段4は、図25(A)に示すように前景カメラ3fが撮像した前景画像Gcfおよび後景カメラ3bが撮像した後景画像Gcbを入力して前景画像Gcfおよび後景画像Gcbと結合し、図25(B)に示すように超楕円の次数kを大きくした仮想投影面Psに投影して車両全周の俯瞰画像Gpwに変換し、あるいは、図25(C)に示すように超楕円の次数kを小さくした仮想投影面Psに投影して車両全周の俯瞰画像Gpwに変換し、モニタ5に表示させる。
これにより、図24、図25に示す俯瞰画像提示装置2は、前方の景観に関しても、後方の景観に関しても、駐車枠(ガイドライン)などの自車両近傍の路面上描画物の直線性は確保され、さらに仮想投影面Ps(バスタブ投影面)の側面部分を使うことでより広範囲の遠方周囲景観を違和感なく表示できる。さらに、複数のカメラ3f、3bの画像Gcf、Gcbを連結した全周囲俯瞰画像Gpwについても、仮想投影面Psの底面12と側面13との連結境界付近の連続性を確保(継ぎ目なくシームレスな画像を作成)できる。
なお、俯瞰画像提示装置2は、前述前景カメラ3fおよび後景カメラ3bだけでなく、図24に破線で示すように、車両1の左側方の景観を撮像する左景カメラ3lと、車両1の右側方の景観を撮像する右景カメラ3rとを処理手段6に接続し、画像変換手段4によって、左景カメラ3lが撮像した左景画像Gclと右景カメラ3rが撮像した右景画像Gcrとを入力して前述前景画像Gcfおよび後景画像Gcbと組み合わせて結合し、車両全周の俯瞰画像Gpwに変換し、モニタ5に表示させることもできる。
The above-described bird's-eye view image presentation device 2 converts the rear image of the vehicle 1 captured by one camera 3 into the bird's-eye view image Gp and displays it on the monitor 5, but the images are captured by a plurality of cameras 3 attached to the vehicle 1. An image around the vehicle 1 can be converted into an all-around bird's-eye view image and displayed on the monitor 5. In this case, as shown in FIG. 23, the origin of the coordinate system [x e , y e , z e ] of the virtual projection plane Ps is desirably set near the center position of the vehicle 1 in the world coordinate system.
For example, as shown in FIG. 24, the bird's-eye view image presentation device 2 that presents a bird's-eye view image of the entire circumference of the vehicle has a foreground camera 3 f that captures a landscape in front of the vehicle 1 and a background that captures a landscape in the rear of the vehicle 1. The camera 3b, the image conversion means 4 for converting the foreground image Gcf captured by the foreground camera 3f and the background image Gcb captured by the background camera 3b into the bird's-eye view image Gpw around the entire vehicle, and a monitor for displaying the bird's-eye view image Gpw And 5. The processing means 6 of the bird's-eye view image presentation device 2 is connected to the cameras 3f and 3b and the monitor 5, and is connected to the reverse confirmation switch 7, the bird's-eye view image display selection switch 8, and the steering angle sensor 9 of the steering angle detection means. doing.
As shown in FIG. 25A, the image conversion means 4 inputs the foreground image Gcf picked up by the foreground camera 3f and the foreground image Gcb picked up by the background camera 3b, and combines them with the foreground image Gcf and the foreground image Gcb. Then, as shown in FIG. 25 (B), the image is projected onto a virtual projection plane Ps with the degree k of the super ellipse increased and converted into a bird's-eye view image Gpw around the entire vehicle, or as shown in FIG. 25 (C). The image is projected onto a virtual projection plane Ps having a reduced ellipse order k, converted into a bird's-eye view image Gpw around the entire vehicle, and displayed on the monitor 5.
Accordingly, the overhead image presentation device 2 shown in FIGS. 24 and 25 ensures the linearity of the drawing on the road surface in the vicinity of the host vehicle, such as a parking frame (guideline), for both the front landscape and the rear landscape. Furthermore, by using the side portion of the virtual projection plane Ps (bathtub projection plane), a wider range of distant surrounding scenery can be displayed without a sense of incongruity. Furthermore, also for the all-around bird's-eye view image Gpw obtained by connecting the images Gcf and Gcb of the plurality of cameras 3f and 3b, continuity near the connection boundary between the bottom surface 12 and the side surface 13 of the virtual projection plane Ps is ensured (seamless and seamless image) Can be created).
Note that the overhead image presentation device 2 includes not only the foreground camera 3f and the background camera 3b described above, but also a left view camera 3l that captures a landscape on the left side of the vehicle 1, as shown by a broken line in FIG. A right view camera 3r that captures a landscape on the right side is connected to the processing unit 6, and a left view image Gcl captured by the left view camera 3l and a right view image Gcr captured by the right view camera 3r are connected to the processing unit 6. Can be combined and combined with the foreground image Gcf and the background image Gcb described above, converted into a bird's-eye view image Gpw around the entire vehicle, and displayed on the monitor 5.
また、上述のように複数のカメラ3により撮像した車両1の周囲の画像を全周囲俯瞰画像Gpwに変換して提示する俯瞰画像提示装置2は、俯瞰画像Gpwを分割し、分割した各画像の次数kを操舵角センサ9により検出された操舵角に基づいて変化させることで、車両1が旋回中にモニタ5に表示される全周囲画像の移動を自然な表示に近づけることができる。
俯瞰画像提示装置2は、図26に示すように、仮想投影面Psの座標系[xe,ye,ze]の原点をワールド座標系の車両1の中心位置付近に設定し、図27(A)(B)に示すように、俯瞰画像Gpwを長半径、短半径位置で4つの第1〜第4象限Q1〜Q4に分割し、第1〜第4象限Q1〜Q4毎に大きな次数kで超楕円形状(角丸四角形状)の投影面の変換テーブルT1(図27(A))と小さな次数kで楕円形状の投影面の変換テーブルT2(図27(B))とを画像変換手段4に予め準備しておく。画像変換手段4は、車両1の走行中に操舵角センサ9が検出するステアリングハンドルの操舵角に基づき、第1〜第4象限Q1〜Q4に設定した投影面の組み合わせを設定する。
例えば、画像変換手段4は、車両1が直進走行している場合に、第1〜第4象限Q1〜Q4の全てを図27(A)に示す大きな次数kで超楕円形状の投影面に設定し、俯瞰画像Gpwをモニタ5に表示する。これにより、俯瞰画像Gpwは、湾曲変形が少なく自然な表示にすることができる。
一方、画像変換手段4は、車両1が旋回走行している場合に、図28に示すように、旋回方向で前方内側(第2象限Q2)を大きな次数kで超楕円形状の投影面(図27(A))に設定するとともに、旋回方向で前方外側(第1象限Q1)を小さな次数kで楕円形状の投影面(図27(B))に設定し、俯瞰画像Gpwをモニタ5に表示する。これにより、俯瞰画像Gpwは、旋回方向内側を湾曲変形が少なく自然な表示にでき、旋回方向外側を背景の移動が自然な表示にでき、旋回中の画像のコーナ部付近での背景の移動を見た目に自然な表示に近づけることができる。
なお、車両1が旋回走行している場合には、前述のように第1象限Q1、第2象限Q2だけでなく、第3象限Q3、第4象限Q4についても、図28に示すように、第3象限Q3を小さな次数kで楕円形状の投影面(図27(B))に設定するとともに、第4象限Q4を大きな次数kで超楕円形状の投影面(図27(A))に設定し、俯瞰画像Gpwをモニタ5に表示する。
つまり、操舵角に応じて複数のカメラ3別の最適なテーブルの組合せをデータベース化しておき、車両1の直進移動・旋回移動別に変換テーブルの組合せを可変することで、動的全周囲画像をつくることができる。
このように、俯瞰画像提示装置2は、画像変換手段4によって、複数のカメラ3により撮像された画像を結合して、車両1を中心とした車両全周の俯瞰画像Gpwを生成することで、車両1が旋回中であっても、画像の結合部にギャップを生じることなく、全周囲俯瞰画像Gpwをモニタ5に表示することができる。
Moreover, the overhead image presentation device 2 that converts and presents the surrounding image of the vehicle 1 captured by the plurality of cameras 3 as described above into the all-around overhead image Gpw, divides the overhead image Gpw, and By changing the order k based on the steering angle detected by the steering angle sensor 9, the movement of the all-around image displayed on the monitor 5 while the vehicle 1 is turning can be brought close to a natural display.
As shown in FIG. 26, the overhead image presentation device 2 sets the origin of the coordinate system [x e , y e , z e ] of the virtual projection plane Ps near the center position of the vehicle 1 in the world coordinate system. (A) As shown in (B), the overhead image Gpw is divided into four first to fourth quadrants Q1 to Q4 at the positions of the long radius and the short radius, and a large order for each of the first to fourth quadrants Q1 to Q4. A conversion table T1 (FIG. 27A) for a projection surface having a super-elliptical shape (rectangular rectangle) with k and a conversion table T2 for a projection surface having an elliptical shape with a small order k (FIG. 27B) are converted. The means 4 is prepared in advance. The image conversion means 4 sets the combination of projection planes set in the first to fourth quadrants Q1 to Q4 based on the steering angle of the steering wheel detected by the steering angle sensor 9 while the vehicle 1 is traveling.
For example, when the vehicle 1 is traveling straight, the image conversion means 4 sets all of the first to fourth quadrants Q1 to Q4 to a super-elliptical projection plane with a large degree k shown in FIG. Then, the overhead image Gpw is displayed on the monitor 5. Thereby, the bird's-eye view image Gpw can be displayed in a natural manner with little bending deformation.
On the other hand, when the vehicle 1 is turning, the image conversion means 4 has a super-elliptical projection plane (Fig. 28) with a large degree k on the front inner side (second quadrant Q2) in the turning direction. 27 (A)), the front outer side (first quadrant Q1) in the turning direction is set to an elliptical projection surface (FIG. 27B) with a small order k, and the overhead image Gpw is displayed on the monitor 5 To do. As a result, the bird's-eye view image Gpw can have a natural display with little curved deformation on the inner side in the turning direction, a natural display of the background movement on the outer side in the turning direction, and the background movement in the vicinity of the corner portion of the turning image. It can be close to the natural display.
When the vehicle 1 is turning, as shown in FIG. 28, not only in the first quadrant Q1 and the second quadrant Q2, but also in the third quadrant Q3 and the fourth quadrant Q4 as described above. The third quadrant Q3 is set to an elliptical projection plane with a small degree k (FIG. 27B), and the fourth quadrant Q4 is set to a superelliptical projection plane with a large degree k (FIG. 27A). Then, the overhead image Gpw is displayed on the monitor 5.
That is, an optimal table combination for each of the plurality of cameras 3 is made into a database according to the steering angle, and a dynamic all-around image is created by varying the combination of the conversion tables for each straight movement / turning movement of the vehicle 1. be able to.
Thus, the overhead image presentation device 2 combines the images captured by the plurality of cameras 3 by the image conversion means 4 to generate the overhead image Gpw of the entire vehicle around the vehicle 1, Even when the vehicle 1 is turning, the all-around overhead image Gpw can be displayed on the monitor 5 without causing a gap in the image coupling portion.
この発明は、俯瞰画像をシームレスに表示して運転者に違和感を与えないようにし、仮想投影面を容易に設定でき、また、表示範囲を拡大して死角を減らすとともに、地面に描画されている物を変形させることなく表示することができ、各種車両の運転支援を目的とする俯瞰画像提示装置に適用することができる。 The present invention seamlessly displays a bird's-eye view image so as not to give the driver a sense of incongruity, can easily set a virtual projection plane, expands the display range to reduce blind spots, and is drawn on the ground An object can be displayed without being deformed, and can be applied to a bird's-eye view image presentation device for driving assistance of various vehicles.
1 車両
2 俯瞰画像提示装置
3 カメラ
4 画像変換手段
5 モニタ
6 処理手段
7 後退確認スイッチ
8 俯瞰画像表示選択スイッチ
9 操舵角センサ
10 仮想カメラ
11 平面
12 底面
13 壁面
14 駐車枠(ガイドライン)
DESCRIPTION OF SYMBOLS 1 Vehicle 2 Overhead image presentation apparatus 3 Camera 4 Image conversion means 5 Monitor 6 Processing means 7 Backward confirmation switch 8 Overhead image display selection switch 9 Steering angle sensor 10 Virtual camera 11 Plane 12 Bottom surface 13 Wall surface 14 Parking frame (guideline)
Claims (6)
前記俯瞰画像を投影する仮想投影面は、車両を平面上に置いた状態で、平面に対して水平であり任意の高さに位置する水平面によって仮想投影面を切断した場合に仮想投影面の水平方向切断面が超楕円形状で表されるとともに、
超楕円の中心を通り平面に対して垂直であり任意の向きである垂直面によって仮想投影面を切断した場合に仮想投影面の垂直方向切断面がバスタブ曲線で表される立体曲面であり、かつ、1つの数式で表され、
前記俯瞰画像は前記超楕円の長半径または短半径位置で分割し、前記複数の撮像手段により撮像された画像を該分割面で結合することを特徴とする俯瞰画像提示装置。 A plurality of imaging means for imaging the periphery of the vehicle, an image conversion means for converting an image captured by the imaging means into an overhead image viewed from a virtual viewpoint above the vehicle, and an overhead image converted by the image conversion means A bird's-eye view image presentation device comprising display means for
The virtual projection plane for projecting the bird's-eye view image is horizontal to the virtual projection plane when the vehicle is placed on a plane and the virtual projection plane is cut by a horizontal plane that is horizontal to the plane and positioned at an arbitrary height. The direction cut surface is represented by a super-elliptical shape,
When the virtual projection plane is cut by a vertical plane that passes through the center of the super ellipse and is perpendicular to the plane and in any orientation, the vertical cut plane of the virtual projection plane is a solid curved surface represented by a bathtub curve, and Represented by a single formula ,
The bird's-eye view image presenting apparatus, wherein the bird's-eye view image is divided at a position of a major radius or a minor radius of the super ellipse, and images taken by the plurality of imaging means are combined on the division plane .
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
JP2013020115A JP6202240B2 (en) | 2013-02-05 | 2013-02-05 | Overhead image presentation device |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
JP2013020115A JP6202240B2 (en) | 2013-02-05 | 2013-02-05 | Overhead image presentation device |
Publications (2)
Publication Number | Publication Date |
---|---|
JP2014154904A JP2014154904A (en) | 2014-08-25 |
JP6202240B2 true JP6202240B2 (en) | 2017-09-27 |
Family
ID=51576397
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
JP2013020115A Active JP6202240B2 (en) | 2013-02-05 | 2013-02-05 | Overhead image presentation device |
Country Status (1)
Country | Link |
---|---|
JP (1) | JP6202240B2 (en) |
Families Citing this family (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
DE102015205507B3 (en) * | 2015-03-26 | 2016-09-29 | Zf Friedrichshafen Ag | Rundsichtsystem for a vehicle |
JP7013751B2 (en) * | 2017-09-15 | 2022-02-01 | 株式会社アイシン | Image processing equipment |
KR102177878B1 (en) * | 2019-03-11 | 2020-11-12 | 현대모비스 주식회사 | Apparatus and method for processing image |
Family Cites Families (9)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JPH02202286A (en) * | 1989-01-31 | 1990-08-10 | Victor Co Of Japan Ltd | Orthogonal transformation encoding method for picture data |
JPH0594524A (en) * | 1991-10-01 | 1993-04-16 | Nippon Telegr & Teleph Corp <Ntt> | Processing method for dividedly describing projected shape object |
JP5044204B2 (en) * | 2006-12-18 | 2012-10-10 | クラリオン株式会社 | Driving assistance device |
JP5369465B2 (en) * | 2008-03-25 | 2013-12-18 | 富士通株式会社 | VEHICLE IMAGE PROCESSING DEVICE, VEHICLE IMAGE PROCESSING METHOD, AND VEHICLE IMAGE PROCESSING PROGRAM |
JP2010081385A (en) * | 2008-09-26 | 2010-04-08 | Sanyo Electric Co Ltd | Driving assistance device |
JP5182042B2 (en) * | 2008-11-28 | 2013-04-10 | 富士通株式会社 | Image processing apparatus, image processing method, and computer program |
JP5087051B2 (en) * | 2009-07-02 | 2012-11-28 | 富士通テン株式会社 | Image generating apparatus and image display system |
JP5716389B2 (en) * | 2010-12-24 | 2015-05-13 | 日産自動車株式会社 | Vehicle display device |
JP5681569B2 (en) * | 2011-05-31 | 2015-03-11 | 富士通テン株式会社 | Information processing system, server device, and in-vehicle device |
-
2013
- 2013-02-05 JP JP2013020115A patent/JP6202240B2/en active Active
Also Published As
Publication number | Publication date |
---|---|
JP2014154904A (en) | 2014-08-25 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
JP4257356B2 (en) | Image generating apparatus and image generating method | |
US9269124B2 (en) | Image processing device, image processing method, program for image processing device, recording medium, and image display device | |
WO2015043507A1 (en) | Image processing method and apparatus for cars, method for generating car surround view image, and car surround view system | |
JP4560716B2 (en) | Vehicle periphery monitoring system | |
US9436986B2 (en) | Image processing apparatus and image processing method | |
JP6167824B2 (en) | Parking assistance device | |
JP4315968B2 (en) | Image processing apparatus and visibility support apparatus and method | |
US9852494B2 (en) | Overhead image generation apparatus | |
JP5451497B2 (en) | Driving support display device | |
JP5724446B2 (en) | Vehicle driving support device | |
JP7156937B2 (en) | Image processing device and image processing method | |
US10075634B2 (en) | Method and system for generating a surround view | |
JP6202240B2 (en) | Overhead image presentation device | |
JP2017050616A5 (en) | ||
CN108038887A (en) | Based on binocular RGB-D camera depth profile methods of estimation | |
US20130308826A1 (en) | Image processing apparatus, non-transitory computer readable recording medium, and image processing method | |
JP2020126208A5 (en) | ||
JP2013026801A (en) | Vehicle periphery monitoring system | |
JP5861871B2 (en) | Overhead image presentation device | |
JP2010183234A (en) | Drawing apparatus | |
JP5305750B2 (en) | Vehicle periphery display device and display method thereof | |
JP2006127083A (en) | Image processing method, and image processor | |
JP2009064373A (en) | Monitor image simulation device, monitor image simulation method, and monitor image simulation program | |
JP5049304B2 (en) | Device for displaying an image around a vehicle | |
KR101609188B1 (en) | Depth camera system of optimal arrangement to improve the field of view |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
A621 | Written request for application examination |
Free format text: JAPANESE INTERMEDIATE CODE: A621 Effective date: 20150807 |
|
A977 | Report on retrieval |
Free format text: JAPANESE INTERMEDIATE CODE: A971007 Effective date: 20160616 |
|
A131 | Notification of reasons for refusal |
Free format text: JAPANESE INTERMEDIATE CODE: A131 Effective date: 20160706 |
|
A521 | Request for written amendment filed |
Free format text: JAPANESE INTERMEDIATE CODE: A523 Effective date: 20160805 |
|
A131 | Notification of reasons for refusal |
Free format text: JAPANESE INTERMEDIATE CODE: A131 Effective date: 20161202 |
|
A521 | Request for written amendment filed |
Free format text: JAPANESE INTERMEDIATE CODE: A523 Effective date: 20170124 |
|
RD02 | Notification of acceptance of power of attorney |
Free format text: JAPANESE INTERMEDIATE CODE: A7422 Effective date: 20170427 |
|
TRDD | Decision of grant or rejection written | ||
A01 | Written decision to grant a patent or to grant a registration (utility model) |
Free format text: JAPANESE INTERMEDIATE CODE: A01 Effective date: 20170802 |
|
A61 | First payment of annual fees (during grant procedure) |
Free format text: JAPANESE INTERMEDIATE CODE: A61 Effective date: 20170815 |
|
R151 | Written notification of patent or utility model registration |
Ref document number: 6202240 Country of ref document: JP Free format text: JAPANESE INTERMEDIATE CODE: R151 |