WO2020021842A1 - Vehicle display control device, vehicle display control method, control program, and persistent tangible computer-readable medium - Google Patents
Vehicle display control device, vehicle display control method, control program, and persistent tangible computer-readable medium Download PDFInfo
- Publication number
- WO2020021842A1 WO2020021842A1 PCT/JP2019/021029 JP2019021029W WO2020021842A1 WO 2020021842 A1 WO2020021842 A1 WO 2020021842A1 JP 2019021029 W JP2019021029 W JP 2019021029W WO 2020021842 A1 WO2020021842 A1 WO 2020021842A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- vehicle
- distance
- respect
- dimensional position
- point
- Prior art date
Links
- 238000000034 method Methods 0.000 title claims description 23
- 230000002085 persistent effect Effects 0.000 title claims description 4
- 230000008859 change Effects 0.000 claims description 8
- 238000003384 imaging method Methods 0.000 claims description 6
- 239000000523 sample Substances 0.000 claims 12
- 230000006870 function Effects 0.000 description 27
- 238000010586 diagram Methods 0.000 description 14
- 238000012545 processing Methods 0.000 description 14
- 238000012544 monitoring process Methods 0.000 description 12
- 230000008569 process Effects 0.000 description 12
- 238000001514 detection method Methods 0.000 description 8
- 230000002093 peripheral effect Effects 0.000 description 5
- 230000001133 acceleration Effects 0.000 description 4
- 239000000284 extract Substances 0.000 description 4
- 230000005540 biological transmission Effects 0.000 description 2
- 238000004590 computer program Methods 0.000 description 2
- 238000006073 displacement reaction Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000000605 extraction Methods 0.000 description 2
- 230000004907 flux Effects 0.000 description 2
- 210000003128 head Anatomy 0.000 description 2
- 241001465754 Metazoa Species 0.000 description 1
- 230000003044 adaptive effect Effects 0.000 description 1
- 230000001174 ascending effect Effects 0.000 description 1
- 230000003190 augmentative effect Effects 0.000 description 1
- 230000015572 biosynthetic process Effects 0.000 description 1
- 238000002485 combustion reaction Methods 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 239000011521 glass Substances 0.000 description 1
- 238000009434 installation Methods 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 239000000203 mixture Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
Images
Classifications
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60K—ARRANGEMENT OR MOUNTING OF PROPULSION UNITS OR OF TRANSMISSIONS IN VEHICLES; ARRANGEMENT OR MOUNTING OF PLURAL DIVERSE PRIME-MOVERS IN VEHICLES; AUXILIARY DRIVES FOR VEHICLES; INSTRUMENTATION OR DASHBOARDS FOR VEHICLES; ARRANGEMENTS IN CONNECTION WITH COOLING, AIR INTAKE, GAS EXHAUST OR FUEL SUPPLY OF PROPULSION UNITS IN VEHICLES
- B60K35/00—Instruments specially adapted for vehicles; Arrangement of instruments in or on vehicles
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
- G01C21/00—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
- G01C21/26—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 specially adapted for navigation in a road network
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/01—Head-up displays
-
- G—PHYSICS
- G08—SIGNALLING
- G08G—TRAFFIC CONTROL SYSTEMS
- G08G1/00—Traffic control systems for road vehicles
- G08G1/16—Anti-collision systems
Definitions
- the present disclosure relates to a display control device for a vehicle, a display control method for a vehicle, a control program, and a tangible computer-readable medium.
- a head-up display (hereinafter, HUD) that superimposes and displays a virtual image on a foreground of a vehicle by projecting an image on a projection member such as a windshield is known.
- HUD head-up display
- a relative position of a target object with respect to the vehicle is detected based on a difference in a distance to the target object measured by two distance measuring sensors arranged at positions apart from each other in a vehicle, and a head is detected.
- a technology is disclosed in which an annular indicator is projected on a front windshield by an up display so as to surround an object of interest.
- Patent Document 1 as a distance measuring sensor, an ultrasonic sensor, a laser radar, or a millimeter-wave radar that measures the time required from the emission of a signal to the return of the signal by reflection from an object of interest may be used. It has been disclosed.
- the height of the object ahead of the vehicle may be greatly changed by the driver of the vehicle due to the gradient difference between the point where the vehicle is located and the point where the object is located.
- a ranging sensor such as an ultrasonic sensor, a laser radar, or a millimeter-wave radar to accurately measure the height of an object of interest with respect to the host vehicle. Therefore, it is difficult to specify the shift of the target object in the height direction with respect to the own vehicle, and the annular index surrounding the target object is easily shifted in the height direction.
- superimposed display that is significantly shifted in the height direction from the intended position is performed on the target object, and the driver may erroneously recognize the target object or have a distrust of the system.
- the present disclosure relates to a vehicle display that enables a driver to reduce erroneous recognition of a target object and a driver's distrust when a virtual image that emphasizes the target object in a vehicle foreground is superimposed and displayed by a head-up display. It is an object to provide a control device, a display control method for a vehicle, a control program, and a tangible computer-readable medium.
- a vehicle that is used in a vehicle and controls a head-up display that superimposes and displays a virtual image that emphasizes an object in the foreground of the vehicle by projecting a display image on a projection member.
- the display control device transmits a search wave, and the search wave is detected by a search wave sensor that detects a distance and an azimuth of the object with respect to the vehicle by receiving a reflected wave reflected by the object.
- a distance azimuth obtaining unit to obtain a high-accuracy map obtaining unit to obtain a high-accuracy map including height information for each point, and the distance and the azimuth obtained by the distance azimuth obtaining unit, Using the high-accuracy map obtained by the high-accuracy map obtaining unit, a three-dimensional position specifying unit that specifies a three-dimensional position of the object with respect to the vehicle, and specifying the three-dimensional position specifying unit, In accordance with the three-dimensional position of said object with respect to both, and a display control unit that superimposes a display highlighting virtual image of the object.
- a vehicle that is used in a vehicle and controls a head-up display that superimposes and displays a virtual image that emphasizes an object in the foreground of the vehicle by projecting a display image on a projection member.
- the display control method transmits a search wave, and the search wave is detected by a search wave sensor that detects a distance and an orientation of the object with respect to the vehicle by receiving a reflected wave reflected by the object.
- Obtain the distance and direction obtain a high-accuracy map including height information for each point, use the obtained distance and azimuth, and the obtained high-accuracy map to obtain the object with respect to the vehicle.
- a three-dimensional position is specified, and a virtual image that emphasizes the object is superimposed and displayed in accordance with the three-dimensional position of the object with respect to the vehicle to be specified.
- a vehicle that is used in a vehicle and controls a head-up display that superimposes and displays a virtual image that emphasizes an object in the foreground of the vehicle by projecting a display image on a projection member.
- the control program for causing the display control device to function, the computer transmits a search wave, the search wave receives the reflected wave reflected by the object, the distance and the azimuth of the object with respect to the vehicle by receiving the reflected wave.
- a distance and orientation acquisition unit that acquires the distance and orientation detected by a search wave sensor to be detected, a high-accuracy map acquisition unit that acquires a high-accuracy map including height information for each point, and the distance orientation acquisition unit Using the distance and the azimuth obtained in the above and the high-accuracy map obtained by the high-accuracy map obtaining unit, a tertiary that specifies a three-dimensional position of the object with respect to the vehicle A position specifying unit to specify at the three-dimensional position specifying unit, in accordance with the three-dimensional position of the object relative to the vehicle, to function as a display control unit that superimposes a display highlighting virtual image of the object.
- a persistent tangible computer-readable medium comprising computer-implemented instructions, wherein the instructions are used in a vehicle, the method comprising: projecting a display image onto a projection member; For controlling a head-up display that superimposes and displays a virtual image that emphasizes an object in the foreground, wherein the command transmits a search wave and receives a reflected wave that is reflected by the object.
- the search wave sensor By detecting the distance and azimuth of the object with respect to the vehicle by the search wave sensor, to obtain the distance and azimuth, and to obtain a high-accuracy map including height information for each point and Using the acquired distance and orientation and the acquired high-accuracy map to identify and identify a three-dimensional position of the object with respect to the vehicle, In accordance with the three-dimensional position of the object, and a possible to superimpose emphasizes virtual image of the object.
- the three-dimensional position of the target object with respect to the vehicle is specified using the high-precision map including information on the height of each point, in addition to the distance and direction of the target object with respect to the vehicle detected by the search wave sensor. Therefore, it is possible to more accurately specify the height of the target object with respect to the vehicle. Further, since the virtual image for emphasizing the object is superimposed and displayed in accordance with the three-dimensional position, it is possible to suppress the virtual image for emphasizing the object from significantly deviating in the height direction from the intended position. As a result, when the head-up display superimposes and displays a virtual image that emphasizes an object in the foreground of the vehicle, it is possible to reduce erroneous recognition of the object by the driver and distrust of the driver.
- FIG. 1 is a diagram illustrating an example of a schematic configuration of a vehicle system 1, It is a diagram showing an example of mounting the HUD device 230 on the vehicle, FIG.
- FIG. 2 is a diagram illustrating an example of a schematic configuration of an HCU 20, It is a flowchart which shows an example of the flow of the virtual image display control related processing in HCU20, It is a diagram for explaining an example of specifying the position of the target object in the height direction with respect to the own vehicle when the target point corresponds to a map data point, It is a diagram for explaining an example of a search for a gradient reference point for the target point, It is a diagram for explaining an example of estimation of the invisible area, It is a diagram showing an example of superimposed display, It is a diagram showing an example of superimposed display, It is a diagram showing an example of superimposed display, It is a diagram for explaining an example of specifying the position in the height direction of the target object with respect to the vehicle from the captured image, It is a diagram showing an example of superimposed display, It is a figure showing an example of superimposition display.
- the vehicle system 1 is used for a vehicle running on a road such as an automobile.
- the vehicle system 1 includes an HMI (Human Machine Interface) system 2, an ADAS (Advanced Driver Assistance Systems) locator 3, a peripheral monitoring sensor 4, a vehicle control ECU 5, and a driving support ECU 6, as shown in FIG. I have.
- the HMI system 2, the ADAS locator 3, the periphery monitoring sensor 4, the vehicle control ECU 5, and the driving support ECU 6 are assumed to be connected to, for example, an in-vehicle LAN.
- the $ ADAS locator 3 includes a GNSS (Global Navigation Satellite System) receiver 30, an inertial sensor 31, and a map database (hereinafter, map DB) 32.
- the GNSS receiver 30 receives positioning signals from a plurality of artificial satellites.
- the inertial sensor 31 includes, for example, a gyro sensor and an acceleration sensor.
- the ADAS locator 3 sequentially measures the position of the own vehicle by combining the positioning signal received by the GNSS receiver 30 and the measurement result of the inertial sensor 31.
- the present embodiment will be described on the assumption that the vehicle position is represented by coordinates of latitude and longitude.
- the vehicle position may be measured using a travel distance or the like obtained from detection results sequentially output from a vehicle speed sensor mounted on the own vehicle. Then, the measured vehicle position is output to the in-vehicle LAN.
- the map DB 32 is a non-volatile memory, and stores map data such as link data, node data, and road shapes.
- the link data includes data such as a link ID for specifying the link, a link length indicating the length of the link, a link direction, a link travel time, node coordinates of the start and end of the link, and road attributes.
- the node data includes data such as a node ID assigned with a unique number for each node on the map, node coordinates, a node name, a node type, a connection link ID describing a link ID of a link connected to the node, and an intersection type. Consists of It is assumed that the data on the road shape includes data such as altitude, cross slope, and vertical slope. Therefore, this altitude corresponds to height information, and the map data corresponds to a high-accuracy map. Data such as altitude, cross slope, vertical slope, etc. may be at least for each point on the road, for example, for each observation point of map data.
- a configuration may be used in which a three-dimensional map composed of a point group of feature points of a road shape and a structure is used.
- the coordinates of the point group in the height direction are set to the height. It may be used as information.
- the ADAS locator 3 does not use the GNSS receiver 30 and detects the three-dimensional map and a LIDAR (Light Detection and Ranging / Laser) that detects a point group of feature points of a road shape and a structure.
- a configuration may be used in which the position of the own vehicle is specified using the detection result of the surrounding monitoring sensor 4 such as Imaging Detection and Ranging). It should be noted that the map data may be obtained from outside the own vehicle using an in-vehicle communication module mounted on the own vehicle.
- the peripheral monitoring sensor 4 is an autonomous sensor that monitors the surrounding environment of the vehicle.
- the surrounding monitoring sensor 4 is a moving dynamic target such as a pedestrian, an animal other than a human, a vehicle other than the own vehicle, and a stationary still object such as a falling object on a road, a guardrail, a curb, and a tree. Detects objects around the vehicle, such as target objects. In addition, road markings such as lane markings around the vehicle are detected.
- the peripheral monitoring sensor 4 includes a peripheral monitoring camera that captures an image of a predetermined area around the vehicle, a search wave sensor such as a millimeter-wave radar, a sonar, and a LIDAR that transmits a search wave to a predetermined area around the vehicle.
- the peripheral monitoring camera sequentially outputs the captured images sequentially captured to the in-vehicle LAN as sensing information.
- the search wave sensor sequentially outputs a scanning result based on a reception signal obtained when a reflected wave reflected by the object is received to the in-vehicle LAN as sensing information.
- the search wave sensor measures the distance from the search wave sensor to the target based on the time taken from transmitting the search wave to receiving the reflected wave reflected by the target. .
- the search wave sensor scans the search wave, and measures the direction of the target object with respect to the search wave sensor from the transmission angle of the search wave when receiving the reflected wave.
- the azimuth may be represented by an azimuth angle.
- clockwise rotation may be represented by a positive azimuth angle
- counterclockwise rotation may be represented by a negative azimuth angle with respect to the front direction.
- the exploration wave sensor is provided with a plurality of transmitters for transmitting the exploration wave at different places of the vehicle, and the target for the exploration wave sensor depends on which transmission unit transmits the exploration wave to receive the reflected wave. May be measured.
- At least a front camera 41 whose imaging range is a predetermined range in front of the own vehicle and a millimeter wave radar 42 that transmits a search wave to a predetermined range in front of the own vehicle are used as the surrounding monitoring sensor 4.
- the front camera 41 corresponds to an imaging device.
- the sensing ranges of the front camera 41 and the millimeter wave radar 42 at least partially overlap, but need not be the same.
- the front camera 41 may be provided on a rearview mirror of an own vehicle, an upper surface of an instrument panel, or the like.
- the millimeter-wave radar 42 may be provided on a front grill, a front bumper, or the like of the vehicle.
- the millimeter-wave radar 42 transmits a millimeter wave or a quasi-millimeter wave while scanning a predetermined range in front of the own vehicle, and receives a reflected wave reflected by the target object, so that the millimeter wave radar 42 Measure the distance and azimuth of the object.
- the distance and the azimuth of the object with respect to the millimeter-wave radar 42 that is, the distance and the azimuth of the object with respect to the own vehicle are simply referred to as the distance and the azimuth of the object.
- the surrounding monitoring sensor 4 may be configured to use a camera that captures an image other than in front of the own vehicle, or use a search wave sensor such as sonar or LIDAR (Light Detection and Ranging / Laser Imaging Detection and Ranging). Good.
- a search wave sensor such as sonar or LIDAR (Light Detection and Ranging / Laser Imaging Detection and Ranging). Good.
- the vehicle control ECU 5 is an electronic control unit that performs acceleration / deceleration control and / or steering control of the own vehicle.
- Examples of the vehicle control ECU 5 include a steering ECU that performs steering control, a power unit control ECU that performs acceleration / deceleration control, and a brake ECU.
- the vehicle control ECU 5 acquires detection signals output from various sensors such as an accelerator position sensor, a brake pedal force sensor, a steering angle sensor, and a wheel speed sensor mounted on the own vehicle, and performs electronic control throttle, a brake actuator, an EPS (Electric) Power Steering) Outputs control signals to each drive control device such as a motor. Further, the vehicle control ECU 5 can output detection signals of the above-described sensors to the in-vehicle LAN.
- the driving support ECU 6 controls the vehicle control ECU 5 to execute an automatic driving function of substituting a driving operation by a driver.
- the driving support ECU 6 recognizes the traveling environment of the own vehicle based on the vehicle position and the map data of the own vehicle acquired from the ADAS locator 3 and the sensing information of the surrounding monitoring sensor 4. As an example, from the sensing information of the periphery monitoring sensor 4, the shape and the moving state of the object around the own vehicle are recognized, and the shape of the road marking around the own vehicle is recognized. Then, by combining with the vehicle position of the own vehicle and the map data, a virtual space in which the actual traveling environment is reproduced in three dimensions is generated.
- the driving support ECU 6 generates a traveling plan for automatically traveling the own vehicle by the automatic driving function based on the recognized traveling environment.
- a long- and medium-term travel plan and a short-term travel plan may be generated.
- the generated virtual space around the own vehicle is used, for example, for acceleration / deceleration to maintain a target inter-vehicle distance with a preceding vehicle, steering for lane following and lane changing, and collision avoidance. Sudden braking or the like is determined.
- a long- and medium-term traveling plan a recommended route for causing the own vehicle to go to the destination is generated.
- the driving support ECU 6 may be configured to generate only a short-term traveling plan of the short-term traveling plan and the long- and medium-term traveling plan.
- an ACC Adaptive Cruise Control
- AEB Automatic Emergency Breaking
- the HMI system 2 includes an HCU (Human Machine Interface Control Unit) 20, an operation device 21, a DSM (Driver Status Monitor) 22, and a display device 23, and receives an input operation from a driver who is a user of the own vehicle. And present information to the driver of the own vehicle.
- the operation device 21 is a group of switches operated by a driver of the own vehicle.
- the operation device 21 is used for performing various settings. For example, as the operation device 21, there is a steering switch or the like provided on a spoke portion of the steering of the own vehicle.
- the DSM 22 includes a near-infrared light source and a near-infrared camera, and a control unit and the like for controlling these.
- the DSM 22 is disposed, for example, on the steering column cover, the upper surface of the instrument panel 12 (see FIG. 2), or the like, with the near-infrared camera facing the driver's seat side of the vehicle.
- the DSM 22 uses a near-infrared camera to photograph the driver's head irradiated with near-infrared light by the near-infrared light source. The image captured by the near-infrared camera is analyzed by the control unit.
- the control unit extracts, for example, the driver's eyes from the captured image, and detects the position of the eyes with respect to a reference position in the vehicle interior (hereinafter, a viewpoint position).
- the reference position may be, for example, an installation position of the near-infrared camera.
- the DSM 22 outputs information on the detected viewpoint position to the HCU 20.
- HUD 230 is used as the display device 23.
- the HUD device 230 will be described with reference to FIG.
- HUD 230 is provided on instrument panel 12 of the host vehicle.
- the HUD 230 forms a display image based on image data output from the HCU 20 by a projector 231 of, for example, a liquid crystal type or a scanning type.
- Examples of the display image include an image for highlighting an object such as an image for highlighting a preceding vehicle followed by the own vehicle by the ACC function.
- the target object to be highlighted is not limited to the preceding vehicle that the own vehicle follows with the ACC function, and may be an obstacle or the like to be avoided by the own vehicle.
- the display image is not limited to an image for highlighting the target object, but may include an image indicating a vehicle state such as a vehicle speed and an operation state of an automatic driving function, or may include an image indicating a planned course of the vehicle. It is good also as a structure which rubs.
- the HUD 230 projects a display image formed by the projector 231 through an optical system 232 such as a concave mirror, for example, onto a projection area defined on the front windshield 10 as a projection member.
- the projection area is located in front of the driver's seat.
- the light flux of the display image reflected by the front windshield 10 toward the vehicle interior is perceived by the driver sitting in the driver's seat.
- the light flux from the foreground, which is a scene existing in front of the own vehicle, transmitted through the front windshield 10 formed of the translucent glass is also perceived by the driver sitting in the driver's seat.
- This allows the driver to visually recognize the virtual image 100 of the display image formed in front of the front windshield 10 so as to overlap a part of the foreground. That is, the HUD 230 superimposes and displays the virtual image 100 on the foreground of the own vehicle, and realizes so-called AR (Augmented Reality) display.
- the projection member on which the HUD 230 projects the display image is not limited to the front windshield 10 and may be a light-transmitting combiner. Further, as the display device 23, a device that displays an image in addition to the HUD 220 may be used. As an example, there are a combination meter, a CID (Center @ Information @ Display), and the like.
- the HCU 20 is mainly configured by a microcomputer including a processor, a volatile memory, a non-volatile memory, an I / O, and a bus connecting these, and is connected to the HUD 230 and the in-vehicle LAN.
- the HCU 20 controls the display by the HUD 230 by executing a control program stored in the nonvolatile memory.
- the HCU 20 corresponds to a vehicle display control device. Executing the control program by the processor corresponds to executing the vehicle display control method corresponding to the control program.
- the memory here is a non-transitory tangible storage medium that temporarily stores programs and data that can be read by a computer. Further, the non-transitional substantive storage medium is realized by a semiconductor memory, a magnetic disk, or the like.
- the configuration of the HCU 20 related to display control by the HUD device 230 will be described in detail below.
- the HCU 20 acquires a captured image acquisition unit 200, a distance and orientation acquisition unit 201, a vehicle position acquisition unit 202, a high-accuracy map acquisition unit 203, and a three-dimensional position identification unit. 204, a viewpoint position identification unit 205, an invisible area estimation unit 206, and a display generation unit 207 are provided as functional blocks. Note that some or all of the functions performed by the HCU 20 may be configured in hardware by one or more ICs or the like. Further, some or all of the functional blocks included in the HCU 20 may be realized by a combination of execution of software by a processor and hardware members.
- the captured image acquisition unit 200 acquires captured images sequentially captured by the front camera 41 as sensing information.
- the distance and azimuth obtaining unit 201 obtains, as sensing information, the distance and azimuth of an object sequentially measured by the millimeter wave radar 42.
- the own vehicle position acquisition unit 202 acquires the vehicle position of the own vehicle that is sequentially measured by the ADAS locator 3.
- the high-accuracy map acquisition unit 203 acquires map data stored in the map DB 32 of the ADAS locator 3.
- the high-accuracy map acquiring unit 203 acquires a high-accuracy map including information on altitude and cross slope for each observation point, that is, for each of a plurality of points.
- the three-dimensional position specifying unit 204 specifies the three-dimensional position of the target object with respect to the own vehicle.
- the three-dimensional position specifying unit 204 may specify a three-dimensional position by selecting an object from the driving environment recognized by the driving support ECU 6.
- the configuration may be such that an obstacle to be avoided by the own vehicle is selected as an object or a preceding vehicle to be followed by the ACC function is selected as an object.
- a preceding vehicle to be followed by the ACC function is selected as an object will be described as an example.
- the link between the object in the driving environment recognized by the driving support ECU 6, the object in the captured image acquired by the captured image acquisition unit 200, and the object whose distance and direction are acquired by the distance and orientation acquisition unit 201 is: What is necessary is just to carry out based on the substantially coincidence of the distance and the azimuth of the own vehicle.
- the three-dimensional position specifying unit 204 obtains the target distance and the azimuth obtained by the distance azimuth obtaining unit 201 and the high-accuracy map obtained by the high-accuracy map obtaining unit Is used to specify the three-dimensional position of the target object with respect to the own vehicle. Details will be described later.
- the predetermined distance may be a distance that is estimated to start to be insufficient in the accuracy of identifying the height of the object in the captured image based on the captured image of the front camera 41, and may be arbitrarily set. Possible values.
- the three-dimensional position specifying unit 204 obtains the distance and the azimuth obtained by the distance and azimuth obtaining unit 201 and obtains the image by the captured image obtaining unit 200.
- the three-dimensional position of the object with respect to the own vehicle is specified using the captured image to be performed. Details will be described later. Note that the three-dimensional position specifying unit 204 may use the high-accuracy map acquired by the high-accuracy map acquiring unit 203 even when the distance of the target object to the own vehicle is shorter than the above-described predetermined distance.
- the viewpoint position specifying unit 205 specifies the viewpoint position of the driver based on the vehicle position of the own vehicle from the information on the viewpoint positions sequentially detected by the DSM 22. For example, the viewpoint position specifying unit 205 determines the viewpoint position detected by the DSM 22 based on the deviation between the reference position of the viewpoint position in the DSM 22 and the reference position of the vehicle position in the own vehicle. The viewpoint position of the driver of the own vehicle is specified by converting the position into the viewpoint position based on the position.
- the invisible area estimating unit 206 obtains the altitude information of a point between the own vehicle and the target object among the altitude information included in the high-accuracy map acquired by the high-accuracy map acquiring unit 203, and the viewpoint position. Based on the viewpoint position specified by the specifying unit 205, an invisible area that is invisible to the driver of the own vehicle due to a change in the gradient of the road surface is estimated. Details will be described later.
- the display generation unit 207 generates image data for superimposing and displaying a virtual image emphasizing the target object in accordance with the three-dimensional position of the target object with respect to the own vehicle, and outputs the image data to the HUD 230 to output the image data to the HUD 230.
- a virtual image that emphasizes the target object is superimposed and displayed in accordance with the three-dimensional position of the target object.
- the display generation unit 207 corresponds to a display control unit. Marks for highlighting the object (hereinafter, highlight marks) are drawn on individual frames constituting the image data. The drawing position of the highlighting mark is set such that a virtual image based on the display image of the highlighting mark is displayed at an intended position in the foreground when the display image of the highlighting mark is projected on the projection area.
- the display generation unit 207 stores the three-dimensional position of the object with respect to the own vehicle specified by the three-dimensional position specification unit 204, the viewpoint position specified by the viewpoint position specification unit 205, and the non-volatile memory of the HCU 20 in advance.
- the positional relationship between the object, the viewpoint, and the projection area is grasped based on the set position of the projection area.
- the display generation unit 207 determines the projection position of the display image of the highlighting mark in the projection area, that is, the formation of a virtual image based on the display image of the highlighting mark.
- the image position is calculated by a geometric operation.
- the display generation unit 207 superimposes and displays a virtual image that emphasizes the target object, avoiding the invisible area. The details of the superimposed display of the virtual image that emphasizes the object will be described later.
- FIG. 4 may be configured to start when the power of the HUD 230 is turned on and the function of the HUD 230 is turned on.
- the function of the HUD 230 may be switched on / off in accordance with an input operation received by the operation device 21.
- the power supply of the HUD 230 may be switched on and off according to the on / off of a switch (hereinafter, a power switch) for starting the internal combustion engine or the motor generator of the own vehicle.
- a switch hereinafter, a power switch
- step S1 the distance and azimuth acquisition unit 201 acquires the distance and azimuth of the object measured by the millimeter wave radar 42 as sensing information.
- step S2 the three-dimensional position identification unit 204 calculates the position coordinates of the target in a coordinate system based on the vehicle position of the host vehicle based on the distance and the azimuth of the target obtained in S1.
- the distance and azimuth measured by the millimeter-wave radar 42 which is a polar coordinate system, are measured by using a horizontal axis (hereinafter, x-axis) and a front-rear axis (hereinafter, y-axis) based on the vehicle position of the vehicle.
- step S3 if the distance of the object acquired in S1 is equal to or longer than the predetermined distance (YES in S3), the process proceeds to step S4. On the other hand, if the distance of the object acquired in S1 is shorter than the above-described predetermined distance (NO in S3), the process proceeds to step S12.
- the high-accuracy map acquisition unit 203 acquires a high-accuracy map stored in the map DB 32.
- the high-accuracy map acquisition unit 203 may be configured to acquire a high-accuracy map by focusing on the vicinity of the vehicle position of the own vehicle acquired by the own vehicle position acquisition unit 202.
- step S5 the three-dimensional position specifying unit 204 determines the point (hereinafter, the target point) corresponding to the (x, y) coordinates of the target object calculated in S2 on the high-accuracy map acquired in S4 by the height and the cross slope. It is determined whether or not a point (hereinafter, a map data point) in which the information is included.
- the target point may be specified by offsetting the latitude and longitude of the vehicle position of the own vehicle in the geographic coordinate system by the (x, y) coordinates of the target object calculated in S2.
- the process proceeds to step S6.
- the process proceeds to step S7.
- step S6 the three-dimensional position specifying unit 204 specifies the position of the target object in the height direction with respect to the own vehicle, and proceeds to step S9. Specifically, the z-coordinate of an axis in the height direction (hereinafter, z-axis) based on the vehicle position of the own vehicle is specified.
- z-axis an axis in the height direction
- FIG. Ov in FIG. 5 indicates the own vehicle
- Va indicates a preceding vehicle as an object.
- black circles indicate map data points
- Op indicates a point of the map data points where the vehicle is located
- Ap indicates a point of the map data points where the preceding vehicle is located.
- the three-dimensional position specifying unit 204 indicates the altitude of the point on the high-accuracy map where the vehicle is located (see Op in FIG. 5) and the (x, y) coordinates of the object calculated in S2 on the high-accuracy map.
- the z coordinate of the target is specified from the difference (see Di in FIG. 5) from the altitude of the point (see Ap in FIG. 5).
- the three-dimensional position identification unit 204 performs, for example, map matching of the vehicle position of the own vehicle acquired by the own vehicle position acquisition unit 202 with the map data points of the high-accuracy map acquired in S4. The position may be used.
- step S7 the three-dimensional position identification unit 204 searches the map data points of the high-accuracy map acquired in S4 for a gradient reference point serving as a reference for the crossing gradient for the target point.
- a gradient reference point serving as a reference for the crossing gradient for the target point.
- FIG. Ov in FIG. 6 indicates the own vehicle
- Va indicates a preceding vehicle as an object.
- black circles are map data points
- Op is a point of the map data points where the vehicle is located
- Ap is a point of the map data points where the preceding vehicle is located
- Scp is a gradient reference point for the target point. Is shown.
- the three-dimensional position identification unit 204 calculates a route approximation curve (see a broken line in FIG. 6) that approximates a route ahead of the own vehicle from map data points ahead of the vehicle position of the own vehicle. As an example, a spline curve may be calculated. Subsequently, the three-dimensional position specifying unit 204 draws a perpendicular from the target point (see Ap in FIG. 6) to this route approximation curve, and determines a map data point located at the shortest distance from the intersection of the perpendicular and the route approximation curve as a gradient reference. A point (see Scp in FIG. 6). In other words, the three-dimensional position identification unit 204 performs map matching of the target point with the high-accuracy map and searches for a map data point located at the shortest distance from the target point as a gradient reference point.
- step S8 the height of the target point is specified using the transverse gradient at the gradient reference point searched in S7 and the distance between the gradient reference point and the target point, and the process proceeds to step S9.
- step S9 an example of specifying the position of the target object in the height direction with respect to the own vehicle when the target point does not correspond to the map data point will be described with reference to FIG.
- the three-dimensional position specifying unit 204 calculates a distance from the gradient reference point to the target point from the (x, y) coordinates of the gradient reference point and the (x, y) coordinates of the target point.
- the (x, y) coordinates of the gradient reference point are specified by offsetting the longitude and latitude coordinates of the gradient reference point in the geographic coordinate system by the latitude and longitude coordinates of the vehicle position of the own vehicle in the geographic coordinate system. do it.
- step S9 the three-dimensional position specifying unit 204 combines the (x, y) coordinates of the target object calculated in S2 and the z coordinates of the target object specified in S6 or S8, and performs three-dimensional The original position, that is, (x, y, z) coordinates is specified.
- the invisible area estimation unit 206 determines the altitude information of the map data points between the vehicle and the target object among the map data points of the high-accuracy map acquired in S4, Based on the specified viewpoint position, an invisible area invisible to a driver of the own vehicle due to a change in the gradient of the road surface is estimated.
- an example of estimating the invisible area will be described with reference to FIG. Ov in FIG. 7 indicates the own vehicle, and Va indicates a preceding vehicle as an object.
- black circles are map data points
- Vp is a viewpoint position
- Los is a line segment starting from the viewpoint position
- Op is a point of the map data point where the vehicle is located
- Ap is a map data point. The point where the preceding vehicle is located is shown.
- the invisible area estimation unit 206 extracts altitude information from map data points between the position of the vehicle (see Op in FIG. 7) and a target point (see Ap in FIG. 7), and spline-interpolates the altitude information. To obtain an approximate curve (hereinafter, a gradient approximate curve). If the target point does not correspond to the map data point, the above-described gradient reference point may be used instead of the target point. Subsequently, with the viewpoint position specified by the viewpoint position specifying unit 205 (see Vp in FIG. 7) as a starting point, a line segment that is in contact with the obtained approximate slope curve is obtained, and the (x, y, z) coordinates of the contact point are calculated.
- an area equal to or less than this height is estimated as an invisible area.
- a region higher than this height is a visible region.
- the invisible area estimation unit 206 may estimate that there is no invisible area.
- step S11 if the invisible area is estimated in S10, that is, if there is an invisible area (YES in S11), the process proceeds to step S13. On the other hand, when there is no invisible area in S10 (NO in S11), the process proceeds to step S12. In step S12, the display generation unit 207 performs an invisible area avoiding process of excluding the invisible area from the drawing target of the display image of the highlighting mark, and proceeds to step S13.
- step S13 the display generation unit 207 generates image data of the display image of the highlighting mark in accordance with the three-dimensional position specified in S9.
- step S14 the display generation unit 207 outputs the image data generated in S13 to the HUD 230, thereby superimposing a virtual image of a highlighting mark that emphasizes the target object on the foreground in accordance with the three-dimensional position of the target object with respect to the own vehicle. Display. Then, the process proceeds to step S20.
- the display generation unit 207 determines the three-dimensional position specified in S9, the viewpoint position specified by the viewpoint position specification unit 205, and the set position of the projection area stored in the nonvolatile memory of the HCU 20 in advance. Then, a virtual image (see Hl in FIG. 8) of a linear highlighting mark along the lower end of the object (see Va in FIG. 8) as shown in FIG. 8 is superimposed and displayed on the foreground.
- FIG. 8 is a diagram illustrating an example of the superimposed display, in which Vi indicates a projection area. It should be noted that any shape other than the line shape may be used as long as the shape is along the lower end of the object.
- the display generation unit 207 avoids the invisible area, and Is superimposed and displayed.
- the line-shaped highlighting mark (see Hl in FIG. 9) along the lower end of the object (see Va in FIG. 9) is translated to the height of the visible region, What is necessary is just to superimpose and display along the lower end in the visible region of the object.
- FIG. 9 is a diagram illustrating an example of superimposed display, in which Vi indicates a projection area.
- step S15 the captured image acquisition unit 200 acquires a captured image captured by the front camera 41 as sensing information.
- step S16 the three-dimensional position specifying unit 204 specifies the position of the target object in the height direction with respect to the own vehicle using the captured image acquired in step S15.
- the three-dimensional position specifying unit 204 specifies the position of the target object with respect to the vehicle in the height direction based on the amount of change of the vanishing point detected by image recognition from the captured image from the vanishing point on a flat road. I just need.
- FIG. 10A illustrates an example of a captured image of a road surface having a forward ascending slope
- FIG. 10B illustrates an example of a captured image of a flat road.
- the position of the reference point (see BP in FIG. 10), which is a vanishing point in the captured image of the flat road, is obtained in advance at the time of manufacturing the vehicle, at the time of attaching the HUD 230 to the vehicle, and the like. It is assumed that the data is already stored in the nonvolatile memory. Further, it is preferable that the position of the reference point is, for example, for each output value of the height sensor of the vehicle so that a position corresponding to the pitching state of the vehicle can be selected and used.
- the three-dimensional position identification unit 204 extracts a line segment by performing edge extraction on the captured image acquired in S14 and performing Hough transform on the white line candidate from which edge extraction has been performed.
- the three-dimensional position identification unit 204 extracts a white line by comparing the extracted line segment with color information.
- the three-dimensional position specifying unit 204 calculates the position of the intersection of the left and right white lines by extending the left and right white lines extracted from the captured image, and sets the calculated intersection as a vanishing point (see FOE in FIG. 10).
- the above-mentioned reference point may be calculated in advance in the same manner.
- the three-dimensional position identification unit 204 compares the vanishing point with the reference point, and identifies the longitudinal gradient in front of the vehicle from the amount of change of the vanishing point with respect to the reference point.
- the correspondence between the change amount and the vertical gradient may be stored in a nonvolatile memory of the HCU 20 in advance, and the specification may be made by referring to this correspondence.
- the three-dimensional position specifying unit 204 assumes that the vertical gradient from the own vehicle to the target is constant at this vertical gradient.
- the z coordinate of the object is specified from the difference between the vehicle position of the own vehicle and the altitude of the target point. Further, it is preferable that the three-dimensional position specifying unit 204 also specifies the height of the target object such as the vehicle height by image recognition.
- step S17 the three-dimensional position identification unit 204 combines the (x, y) coordinates of the target calculated in S2 and the z coordinates of the target estimated in S16, and determines the three-dimensional position of the target with respect to the own vehicle. That is, the (x, y, z) coordinates are specified.
- step S18 the display generation unit 207 generates image data of the display image of the highlighting mark in accordance with the three-dimensional position specified in S17.
- step S19 the display generation unit 207 outputs the image data generated in S18 to the HUD 230, thereby superimposing a virtual image of a highlighting mark that emphasizes the target object on the foreground in accordance with the three-dimensional position of the target object with respect to the own vehicle. Display.
- the display generation unit 207 sets the three-dimensional position and the height of the target object specified in S17, the viewpoint position specified by the viewpoint position specification unit 205, and the setting of the projection area stored in the nonvolatile memory of the HCU 20 in advance. Based on the position, a virtual image (see Hl in FIG. 11) of a frame-shaped highlighting mark surrounding the object (see Va in FIG. 11) as shown in FIG. 11 is superimposed and displayed on the foreground.
- FIG. 11 is a diagram illustrating an example of superimposed display, in which Vi indicates a projection area. As long as the shape surrounds the object, the shape may be a shape other than the frame shape, or may be a shape lacking a part.
- step S20 if it is the end timing of the virtual image display control-related processing (YES in S20), the virtual image display control-related processing ends. On the other hand, if it is not the end timing of the virtual image display control-related processing (NO in S20), the process returns to S1 and repeats the processing. Examples of the end timing of the virtual image display control-related processing include a case where the power switch of the own vehicle is turned off, a case where the function of the HUD 230 is turned off, and the like.
- the height of the target object with respect to the own vehicle can be specified with high accuracy. Since it is difficult to accurately specify the height of the object itself, a virtual image having a shape along the lower end of the object is superimposed and displayed. According to this, it is possible to perform the display for highlighting the target object while suppressing the displacement in the height direction. Further, in the processing of S15 to S19, since the image captured by the front camera 41 is also used, the height of the target object with respect to the own vehicle can be specified with high accuracy, and the height of the target object itself can be specified with high accuracy. It is possible to do.
- a virtual image of a shape surrounding the object is superimposed and displayed. According to this, it is possible to perform the display for emphasizing the target object while surrounding the target object and making it easy for the driver to recognize the target object, while suppressing the displacement in the height direction.
- ⁇ Summary of Embodiment 1> in addition to the distance and the azimuth of the target object with respect to the own vehicle, which is detected by the millimeter-wave radar 42 serving as the exploration wave sensor, a high-precision map including height information for each point is used. Since the three-dimensional position of the object with respect to the own vehicle is specified, the height of the object with respect to the own vehicle can be specified with higher accuracy. In addition, since the virtual image for emphasizing the object is superimposed and displayed in accordance with the three-dimensional position, it is possible to suppress the virtual image for emphasizing the object from being significantly shifted from the intended position in the height direction. As a result, when the HUD 230 superimposes and displays a virtual image that emphasizes an object in the foreground of the own vehicle, it is possible to reduce erroneous recognition of the object by the driver and distrust of the driver.
- the target object when the target object is at a short distance less than a predetermined distance from the own vehicle, it is possible to accurately specify the height of the target object with respect to the own vehicle at this short distance.
- the image captured by the front camera 41 By using the image captured by the front camera 41, the height of the target object with respect to the own vehicle can be specified particularly accurately.
- an image picked up by the front camera 41 which is difficult to accurately determine the height of the target object with respect to the own vehicle at this distance is used. Instead, by using the high-accuracy map, it is possible to accurately specify the height of the target object with respect to the own vehicle.
- the gradient reference point serving as the reference of the cross slope for the target point is searched for and searched.
- the height of the target point can be specified using the cross slope at the gradient reference point and the distance between the gradient reference point and the target point. Therefore, even when the target point does not correspond to the map data point of the high-precision map, the height of the target object with respect to the own vehicle can be specified with higher accuracy.
- the invisible area estimation unit 206 estimates the invisible area
- the display generation unit 207 superimposes and displays the virtual image of the highlighting mark while avoiding the invisible area. It is not necessary to superimpose and display a virtual image that emphasizes the object at a position far away from a portion that is not hidden by the forward gradient of the object. Therefore, it is possible to further reduce erroneous recognition of the target object by the driver and distrust of the driver.
- a configuration may be used in which the height of an object with respect to the own vehicle is specified using a high-accuracy map whether the object is at a short distance or a long distance.
- the image captured by the front camera 41 may be configured to specify the height of the object itself when the object is at a short distance and to superimpose and display a virtual image of the shape surrounding the object.
- the invisible area estimating unit 206 may estimate the invisible area using a high-accuracy map, and superimpose and display a virtual image of the highlighting mark while avoiding the invisible area. As a specific example, as shown in FIG.
- FIG. 12 is a diagram illustrating an example of the superimposed display, in which Vi indicates a projection area.
- the height of the target object with respect to the own vehicle can be specified with higher accuracy. Therefore, when a virtual image that emphasizes an object in the foreground of the own vehicle is superimposed and displayed by the HUD 230, it is possible to reduce erroneous recognition of the object and distrust of the driver by the driver.
- the height of the object with respect to the own vehicle is specified using a high-precision map, but the target point does not correspond to a map data point.
- the height of the target object with respect to the own vehicle may be specified using the image captured by the front camera 41.
- the target point does not correspond to the map data point and the target object is at a short distance
- the height of the target object with respect to the own vehicle is specified using the image captured by the front camera 41.
- the target point does not correspond to the map data point and the distance of the target object is short, it is possible to accurately specify the height of the target object with respect to the own vehicle at this short distance.
- the captured image captured by the front camera 41 it is possible to accurately specify the height of the target object with respect to the own vehicle while suppressing the processing load for searching for the gradient reference point.
- the target point does not correspond to the map data point and the distance of the target object is long, it is difficult to accurately specify the height of the target object with respect to the own vehicle at this distance.
- the image captured by the camera 41 it is possible to search for the gradient reference point and accurately specify the height of the target object with respect to the own vehicle.
- a virtual image of a shape surrounding the target object is superimposed or displayed at the lower end of the target object.
- the present invention is not necessarily limited to this.
- a virtual image of the same shape may be superimposed and displayed regardless of whether or not the height of the target object with respect to the own vehicle is specified using the captured image captured by the front camera 41.
- the configuration is described in which the invisible area estimation unit 206 estimates the invisible area and the display generation unit 207 superimposes and displays the virtual image of the highlighting mark while avoiding the invisible area.
- the present invention is not limited to this.
- the HCU 20 may not include the invisible area estimation unit 206 and may not be configured to estimate the invisible area.
- the viewpoint position specifying unit 205 specifies the viewpoint position of the driver based on the vehicle position of the own vehicle based on the information on the viewpoint position detected by the DSM 22, but is not limited thereto.
- a configuration may be used in which a set position stored in advance in the nonvolatile memory of the HCU 20 is used as a driver's viewpoint position with reference to the vehicle position of the own vehicle.
- This set position is a virtual point set for each vehicle type, and is a point representing the position of the driver's eyes in a normal driving state.
- the setting position may be set to, for example, a height of 635 mm immediately above the seating reference point.
- composition which specifies a viewpoint position.
- control unit and the technique according to the present disclosure are realized by a dedicated computer provided by configuring a processor and a memory programmed to execute one or a plurality of functions embodied by a computer program. May be done.
- control unit and the technique described in the present disclosure may be implemented by a dedicated computer provided by configuring a processor with one or more dedicated hardware logic circuits.
- control unit and the method described in the present disclosure may be implemented by a combination of a processor and a memory programmed to perform one or more functions and a processor configured with one or more hardware logic circuits. It may be realized by one or more dedicated computers configured.
- the computer program may be stored in a computer-readable non-transitional tangible recording medium as instructions to be executed by a computer.
- each section is expressed as, for example, S1. Further, each section can be divided into multiple subsections, while multiple sections can be combined into one section. Further, each section so configured can be referred to as a device, module, means.
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Remote Sensing (AREA)
- General Physics & Mathematics (AREA)
- Radar, Positioning & Navigation (AREA)
- Automation & Control Theory (AREA)
- Optics & Photonics (AREA)
- Chemical & Material Sciences (AREA)
- Combustion & Propulsion (AREA)
- Transportation (AREA)
- Mechanical Engineering (AREA)
- Traffic Control Systems (AREA)
- Navigation (AREA)
- Instrument Panels (AREA)
Abstract
This vehicle display control device is for controlling a head-up display (230) that is used in a vehicle and superimposes and displays a virtual image for emphasizing an object in the view in front of the vehicle by projecting a display image onto a projection member (10). The vehicle display control device comprises: a distance and direction acquisition unit (201) for acquiring a detected distance and detected direction from a probing wave sensor (42) for detecting the distance and direction of an object using probing waves; a high-precision map acquisition unit (203) for acquiring a high-precision map including point-by-point height information; a three-dimensional position identification unit (204) for using the distance, direction, and high-precision map to identify the three-dimensional position of the object in relation to the vehicle; and a display control unit (207) for causing a virtual image for emphasizing the object to be superimposed and displayed according to the three-dimensional position of the object in relation to the vehicle.
Description
本出願は、2018年7月25日に出願された日本特許出願番号2018-139469号に基づくもので、ここにその記載内容を援用する。
This application is based on Japanese Patent Application No. 2018-139469 filed on July 25, 2018, the contents of which are incorporated herein by reference.
本開示は、車両用表示制御装置、車両用表示制御方法、制御プログラム、および持続的有形コンピュータ読み取り媒体に関するものである。
The present disclosure relates to a display control device for a vehicle, a display control method for a vehicle, a control program, and a tangible computer-readable medium.
従来、ウインドシールド等の投影部材へ画像を投影することによって車両の前景に虚像を重畳表示させるヘッドアップディスプレイ(以下、HUD)が知られている。例えば特許文献1には、車両において互いに離れた位置に配置される2つの測距センサでそれぞれ測定する注目物体までの距離の違いから、車両を基準とした注目物体の相対位置を検出し、ヘッドアップディスプレイによってフロントウインドシールド上に、その注目物体の周囲を囲むようにして環状の指標を映出する技術が開示されている。特許文献1では、測距センサとして、信号を発してからその信号が注目物体で反射して戻ってくるまでに要する時間を測定する超音波センサ、レーザレーダ、又はミリ波レーダを使用することが開示されている。
Conventionally, a head-up display (hereinafter, HUD) that superimposes and displays a virtual image on a foreground of a vehicle by projecting an image on a projection member such as a windshield is known. For example, in Patent Document 1, a relative position of a target object with respect to the vehicle is detected based on a difference in a distance to the target object measured by two distance measuring sensors arranged at positions apart from each other in a vehicle, and a head is detected. A technology is disclosed in which an annular indicator is projected on a front windshield by an up display so as to surround an object of interest. In Patent Document 1, as a distance measuring sensor, an ultrasonic sensor, a laser radar, or a millimeter-wave radar that measures the time required from the emission of a signal to the return of the signal by reflection from an object of interest may be used. It has been disclosed.
車両の前方の物体は、その車両が位置する地点とその物体が位置する地点との勾配差によって、その車両のドライバから見える高さが大きく変動する場合がある。しかしながら、特許文献1に開示の技術では、超音波センサ、レーザレーダ、又はミリ波レーダといった測距センサは、自車に対する注目物体の高さまでを精度良く測定することが難しい。よって、自車に対する注目物体の高さ方向のずれを特定しにくく、注目物体を囲む環状の指標が高さ方向にずれやすくなる。その結果、注目物体に対し、意図した位置から高さ方向に大幅にずれた重畳表示が行われ、運転者が注目物体の誤認識をしたりシステムに対する不信感を抱いたりするおそれがある。
物体 The height of the object ahead of the vehicle may be greatly changed by the driver of the vehicle due to the gradient difference between the point where the vehicle is located and the point where the object is located. However, according to the technology disclosed in Patent Document 1, it is difficult for a ranging sensor such as an ultrasonic sensor, a laser radar, or a millimeter-wave radar to accurately measure the height of an object of interest with respect to the host vehicle. Therefore, it is difficult to specify the shift of the target object in the height direction with respect to the own vehicle, and the annular index surrounding the target object is easily shifted in the height direction. As a result, superimposed display that is significantly shifted in the height direction from the intended position is performed on the target object, and the driver may erroneously recognize the target object or have a distrust of the system.
本開示は、ヘッドアップディスプレイによって車両の前景中の対象物を強調する虚像を重畳表示させる際に、ドライバによるその対象物の誤認識及びドライバの不信感を低減させることを可能にする車両用表示制御装置、車両用表示制御方法、制御プログラム、および持続的有形コンピュータ読み取り媒体を提供することを目的とする。
The present disclosure relates to a vehicle display that enables a driver to reduce erroneous recognition of a target object and a driver's distrust when a virtual image that emphasizes the target object in a vehicle foreground is superimposed and displayed by a head-up display. It is an object to provide a control device, a display control method for a vehicle, a control program, and a tangible computer-readable medium.
本開示の第一の態様に基づいて、車両で用いられ、投影部材に表示画像を投影することによって前記車両の前景中の対象物を強調する虚像を重畳表示させるヘッドアップディスプレイを制御する車両用表示制御装置は、探査波を送信し、その探査波が前記対象物で反射される反射波を受信することで前記車両に対する前記対象物の距離及び方位を検出する探査波センサで検出する、その距離及び方位を、取得する距離方位取得部と、地点別の高さの情報を含む高精度地図を取得する高精度地図取得部と、前記距離方位取得部で取得する前記距離及び前記方位と、前記高精度地図取得部で取得する前記高精度地図とを用いて、前記車両に対する前記対象物の三次元位置を特定する三次元位置特定部と、前記三次元位置特定部で特定する、前記車両に対する前記対象物の三次元位置に合わせて、前記対象物を強調する虚像を重畳表示させる表示制御部とを備える。
According to a first aspect of the present disclosure, for a vehicle that is used in a vehicle and controls a head-up display that superimposes and displays a virtual image that emphasizes an object in the foreground of the vehicle by projecting a display image on a projection member. The display control device transmits a search wave, and the search wave is detected by a search wave sensor that detects a distance and an azimuth of the object with respect to the vehicle by receiving a reflected wave reflected by the object. Distance and azimuth, a distance azimuth obtaining unit to obtain, a high-accuracy map obtaining unit to obtain a high-accuracy map including height information for each point, and the distance and the azimuth obtained by the distance azimuth obtaining unit, Using the high-accuracy map obtained by the high-accuracy map obtaining unit, a three-dimensional position specifying unit that specifies a three-dimensional position of the object with respect to the vehicle, and specifying the three-dimensional position specifying unit, In accordance with the three-dimensional position of said object with respect to both, and a display control unit that superimposes a display highlighting virtual image of the object.
本開示の第二の態様に基づいて、車両で用いられ、投影部材に表示画像を投影することによって前記車両の前景中の対象物を強調する虚像を重畳表示させるヘッドアップディスプレイを制御する車両用表示制御方法は、探査波を送信し、その探査波が前記対象物で反射される反射波を受信することで前記車両に対する前記対象物の距離及び方位を検出する探査波センサで検出する、その距離及び方位を、取得し、地点別の高さの情報を含む高精度地図を取得し、取得する前記距離及び方位と、取得する前記高精度地図とを用いて、前記車両に対する前記対象物の三次元位置を特定し、特定する、前記車両に対する前記対象物の三次元位置に合わせて、前記対象物を強調する虚像を重畳表示させる。
Based on a second aspect of the present disclosure, for a vehicle that is used in a vehicle and controls a head-up display that superimposes and displays a virtual image that emphasizes an object in the foreground of the vehicle by projecting a display image on a projection member. The display control method transmits a search wave, and the search wave is detected by a search wave sensor that detects a distance and an orientation of the object with respect to the vehicle by receiving a reflected wave reflected by the object. Obtain the distance and direction, obtain a high-accuracy map including height information for each point, use the obtained distance and azimuth, and the obtained high-accuracy map to obtain the object with respect to the vehicle. A three-dimensional position is specified, and a virtual image that emphasizes the object is superimposed and displayed in accordance with the three-dimensional position of the object with respect to the vehicle to be specified.
本開示の第三の態様に基づいて、車両で用いられ、投影部材に表示画像を投影することによって前記車両の前景中の対象物を強調する虚像を重畳表示させるヘッドアップディスプレイを制御する車両用表示制御装置として機能させるための制御プログラムは、コンピュータを、探査波を送信し、その探査波が前記対象物で反射される反射波を受信することで前記車両に対する前記対象物の距離及び方位を検出する探査波センサで検出する、その距離及び方位を、取得する距離方位取得部と、地点別の高さの情報を含む高精度地図を取得する高精度地図取得部と、前記距離方位取得部で取得する前記距離及び前記方位と、前記高精度地図取得部で取得する前記高精度地図とを用いて、前記車両に対する前記対象物の三次元位置を特定する三次元位置特定部と、前記三次元位置特定部で特定する、前記車両に対する前記対象物の三次元位置に合わせて、前記対象物を強調する虚像を重畳表示させる表示制御部として機能させる。
Based on a third aspect of the present disclosure, for a vehicle that is used in a vehicle and controls a head-up display that superimposes and displays a virtual image that emphasizes an object in the foreground of the vehicle by projecting a display image on a projection member. The control program for causing the display control device to function, the computer transmits a search wave, the search wave receives the reflected wave reflected by the object, the distance and the azimuth of the object with respect to the vehicle by receiving the reflected wave. A distance and orientation acquisition unit that acquires the distance and orientation detected by a search wave sensor to be detected, a high-accuracy map acquisition unit that acquires a high-accuracy map including height information for each point, and the distance orientation acquisition unit Using the distance and the azimuth obtained in the above and the high-accuracy map obtained by the high-accuracy map obtaining unit, a tertiary that specifies a three-dimensional position of the object with respect to the vehicle A position specifying unit to specify at the three-dimensional position specifying unit, in accordance with the three-dimensional position of the object relative to the vehicle, to function as a display control unit that superimposes a display highlighting virtual image of the object.
本開示の第四の態様に基づいて、コンピュータによって実施される命令を備える持続的有形コンピュータ読み取り媒体であって、当該命令は、車両で用いられ、投影部材に表示画像を投影することによって前記車両の前景中の対象物を強調する虚像を重畳表示させるヘッドアップディスプレイを制御するものであり、当該命令は、探査波を送信し、その探査波が前記対象物で反射される反射波を受信することで前記車両に対する前記対象物の距離及び方位を検出する探査波センサで検出する、その距離及び方位を、取得することと、地点別の高さの情報を含む高精度地図を取得することと、取得する前記距離及び方位と、取得する前記高精度地図とを用いて、前記車両に対する前記対象物の三次元位置を特定し、特定する、前記車両に対する前記対象物の三次元位置に合わせて、前記対象物を強調する虚像を重畳表示させることとを備える。
According to a fourth aspect of the present disclosure, a persistent tangible computer-readable medium comprising computer-implemented instructions, wherein the instructions are used in a vehicle, the method comprising: projecting a display image onto a projection member; For controlling a head-up display that superimposes and displays a virtual image that emphasizes an object in the foreground, wherein the command transmits a search wave and receives a reflected wave that is reflected by the object. By detecting the distance and azimuth of the object with respect to the vehicle by the search wave sensor, to obtain the distance and azimuth, and to obtain a high-accuracy map including height information for each point and Using the acquired distance and orientation and the acquired high-accuracy map to identify and identify a three-dimensional position of the object with respect to the vehicle, In accordance with the three-dimensional position of the object, and a possible to superimpose emphasizes virtual image of the object.
上記によれば、探査波センサで検出する、車両に対する対象物の距離及び方位に加え、地点別の高さの情報を含む高精度地図も用いて、車両に対する対象物の三次元位置を特定するので、車両に対する対象物の高さをより精度良く特定することが可能になる。また、この三次元位置に合わせて、対象物を強調する虚像を重畳表示させるので、対象物を強調する虚像が、意図した位置から高さ方向に大幅にずれることを抑えることが可能になる。その結果、ヘッドアップディスプレイによって車両の前景中の対象物を強調する虚像を重畳表示させる際に、ドライバによるその対象物の誤認識及びドライバの不信感を低減させることが可能になる。
According to the above, the three-dimensional position of the target object with respect to the vehicle is specified using the high-precision map including information on the height of each point, in addition to the distance and direction of the target object with respect to the vehicle detected by the search wave sensor. Therefore, it is possible to more accurately specify the height of the target object with respect to the vehicle. Further, since the virtual image for emphasizing the object is superimposed and displayed in accordance with the three-dimensional position, it is possible to suppress the virtual image for emphasizing the object from significantly deviating in the height direction from the intended position. As a result, when the head-up display superimposes and displays a virtual image that emphasizes an object in the foreground of the vehicle, it is possible to reduce erroneous recognition of the object by the driver and distrust of the driver.
本開示についての上記目的およびその他の目的、特徴や利点は、添付の図面を参照しながら下記の詳細な記述により、より明確になる。その図面は、
車両システム1の概略的な構成の一例を示す図であり、
HUD装置230の車両への搭載例を示す図であり、
HCU20の概略的な構成の一例を示す図であり、
HCU20での虚像表示制御関連処理の流れの一例を示すフローチャートであり、
対象地点が地図データ点にあたる場合の、自車に対する対象物の高さ方向の位置の特定の一例について説明するための図であり、
対象地点にとっての勾配基準点の探索の一例について説明するための図であり、
不可視領域の推定の一例について説明するための図であり、
重畳表示の一例を示す図であり、
重畳表示の一例を示す図であり、
撮像画像からの自車に対する対象物の高さ方向の位置の特定の一例について説明するための図であり、
重畳表示の一例を示す図であり、
重畳表示の一例を示す図である。
The above and other objects, features and advantages of the present disclosure will become more apparent from the following detailed description with reference to the accompanying drawings. The drawing is
FIG. 1 is a diagram illustrating an example of a schematic configuration of a vehicle system 1, It is a diagram showing an example of mounting the HUD device 230 on the vehicle, FIG. 2 is a diagram illustrating an example of a schematic configuration of an HCU 20, It is a flowchart which shows an example of the flow of the virtual image display control related processing in HCU20, It is a diagram for explaining an example of specifying the position of the target object in the height direction with respect to the own vehicle when the target point corresponds to a map data point, It is a diagram for explaining an example of a search for a gradient reference point for the target point, It is a diagram for explaining an example of estimation of the invisible area, It is a diagram showing an example of superimposed display, It is a diagram showing an example of superimposed display, It is a diagram for explaining an example of specifying the position in the height direction of the target object with respect to the vehicle from the captured image, It is a diagram showing an example of superimposed display, It is a figure showing an example of superimposition display.
(実施形態1)
<車両システム1の概略構成>
以下、本実施形態について図面を用いて説明する。車両システム1は、自動車といった路上を走行する車両で用いられるものである。車両システム1は、一例として、図1に示すように、HMI(Human Machine Interface)システム2、ADAS(Advanced Driver Assistance Systems)ロケータ3、周辺監視センサ4、車両制御ECU5、及び運転支援ECU6を含んでいる。HMIシステム2、ADASロケータ3、周辺監視センサ4、車両制御ECU5、及び運転支援ECU6は、例えば車内LANに接続されているものとする。 (Embodiment 1)
<Schematic configuration ofvehicle system 1>
Hereinafter, the present embodiment will be described with reference to the drawings. Thevehicle system 1 is used for a vehicle running on a road such as an automobile. As an example, the vehicle system 1 includes an HMI (Human Machine Interface) system 2, an ADAS (Advanced Driver Assistance Systems) locator 3, a peripheral monitoring sensor 4, a vehicle control ECU 5, and a driving support ECU 6, as shown in FIG. I have. The HMI system 2, the ADAS locator 3, the periphery monitoring sensor 4, the vehicle control ECU 5, and the driving support ECU 6 are assumed to be connected to, for example, an in-vehicle LAN.
<車両システム1の概略構成>
以下、本実施形態について図面を用いて説明する。車両システム1は、自動車といった路上を走行する車両で用いられるものである。車両システム1は、一例として、図1に示すように、HMI(Human Machine Interface)システム2、ADAS(Advanced Driver Assistance Systems)ロケータ3、周辺監視センサ4、車両制御ECU5、及び運転支援ECU6を含んでいる。HMIシステム2、ADASロケータ3、周辺監視センサ4、車両制御ECU5、及び運転支援ECU6は、例えば車内LANに接続されているものとする。 (Embodiment 1)
<Schematic configuration of
Hereinafter, the present embodiment will be described with reference to the drawings. The
ADASロケータ3は、図1に示すように、GNSS(Global Navigation Satellite System)受信機30、慣性センサ31、及び地図データベース(以下、地図DB)32を備えている。GNSS受信機30は、複数の人工衛星からの測位信号を受信する。慣性センサ31は、例えばジャイロセンサ及び加速度センサを備える。ADASロケータ3は、GNSS受信機30で受信する測位信号と、慣性センサ31の計測結果とを組み合わせることにより、自車の車両位置を逐次測位する。一例として、本実施形態では、車両位置が経緯度の座標で表されるものとして説明を行う。経緯度の座標ではx軸が経度,y軸が緯度を示す。なお、車両位置の測位には、自車に搭載された車速センサから逐次出力される検出結果から求めた走行距離等を用いる構成としてもよい。そして、測位した車両位置を車内LANへ出力する。
As shown in FIG. 1, the $ ADAS locator 3 includes a GNSS (Global Navigation Satellite System) receiver 30, an inertial sensor 31, and a map database (hereinafter, map DB) 32. The GNSS receiver 30 receives positioning signals from a plurality of artificial satellites. The inertial sensor 31 includes, for example, a gyro sensor and an acceleration sensor. The ADAS locator 3 sequentially measures the position of the own vehicle by combining the positioning signal received by the GNSS receiver 30 and the measurement result of the inertial sensor 31. As an example, the present embodiment will be described on the assumption that the vehicle position is represented by coordinates of latitude and longitude. In the coordinates of longitude and latitude, the x axis indicates longitude, and the y axis indicates latitude. It should be noted that the vehicle position may be measured using a travel distance or the like obtained from detection results sequentially output from a vehicle speed sensor mounted on the own vehicle. Then, the measured vehicle position is output to the in-vehicle LAN.
地図DB32は、不揮発性メモリであって、リンクデータ、ノードデータ、道路形状等の地図データを格納している。リンクデータは、リンクを特定するリンクID、リンクの長さを示すリンク長、リンク方位、リンク旅行時間、リンクの始端と終端とのノード座標、及び道路属性等の各データから構成される。ノードデータは、地図上のノード毎に固有の番号を付したノードID、ノード座標、ノード名称、ノード種別、ノードに接続するリンクのリンクIDが記述される接続リンクID、交差点種別等の各データから構成される。道路形状のデータには、高度、横断勾配、縦断勾配等のデータを含むものとする。よって、この高度が高さの情報に相当し、地図データが高精度地図に相当する。高度、横断勾配、縦断勾配等のデータは、少なくとも道路上の地点別であればよく、例えば地図データの観測点別とすればよい。
The map DB 32 is a non-volatile memory, and stores map data such as link data, node data, and road shapes. The link data includes data such as a link ID for specifying the link, a link length indicating the length of the link, a link direction, a link travel time, node coordinates of the start and end of the link, and road attributes. The node data includes data such as a node ID assigned with a unique number for each node on the map, node coordinates, a node name, a node type, a connection link ID describing a link ID of a link connected to the node, and an intersection type. Consists of It is assumed that the data on the road shape includes data such as altitude, cross slope, and vertical slope. Therefore, this altitude corresponds to height information, and the map data corresponds to a high-accuracy map. Data such as altitude, cross slope, vertical slope, etc. may be at least for each point on the road, for example, for each observation point of map data.
また、地図データとして、道路形状及び構造物の特徴点の点群からなる三次元地図を用いる構成としてもよく、この三次元地図を用いる場合はこの点群の高さ方向の座標を高さの情報として用いてもよい。ADASロケータ3は、この三次元地図を用いる場合、GNSS受信機30を用いずに、この三次元地図と、道路形状及び構造物の特徴点の点群を検出するLIDAR(Light Detection and Ranging/Laser Imaging Detection and Ranging)等の周辺監視センサ4での検出結果とを用いて、自車の車両位置を特定する構成としてもよい。なお、地図データは、自車に搭載された車載通信モジュールを用いて自車の外部から取得する構成としてもよい。
Further, as the map data, a configuration may be used in which a three-dimensional map composed of a point group of feature points of a road shape and a structure is used. In the case of using this three-dimensional map, the coordinates of the point group in the height direction are set to the height. It may be used as information. When using the three-dimensional map, the ADAS locator 3 does not use the GNSS receiver 30 and detects the three-dimensional map and a LIDAR (Light Detection and Ranging / Laser) that detects a point group of feature points of a road shape and a structure. A configuration may be used in which the position of the own vehicle is specified using the detection result of the surrounding monitoring sensor 4 such as Imaging Detection and Ranging). It should be noted that the map data may be obtained from outside the own vehicle using an in-vehicle communication module mounted on the own vehicle.
周辺監視センサ4は、自車の周辺環境を監視する自律センサである。一例として、周辺監視センサ4は、歩行者,人間以外の動物、自車以外の車両等の移動する動的物標、及び路上の落下物,ガードレール、縁石、及び樹木等の静止している静的物標といった自車周辺の対象物を検出する。他にも、自車周辺の走行区画線等の路面標示を検出する。例えば周辺監視センサ4としては、自車周囲の所定範囲を撮像する周辺監視カメラ、自車周囲の所定範囲に探査波を送信するミリ波レーダ、ソナー、LIDAR等の探査波センサがある。周辺監視カメラは、逐次撮像する撮像画像をセンシング情報として車内LANへ逐次出力する。探査波センサは、対象物によって反射された反射波を受信した場合に得られる受信信号に基づく走査結果をセンシング情報として車内LANへ逐次出力する。
(4) The peripheral monitoring sensor 4 is an autonomous sensor that monitors the surrounding environment of the vehicle. As an example, the surrounding monitoring sensor 4 is a moving dynamic target such as a pedestrian, an animal other than a human, a vehicle other than the own vehicle, and a stationary still object such as a falling object on a road, a guardrail, a curb, and a tree. Detects objects around the vehicle, such as target objects. In addition, road markings such as lane markings around the vehicle are detected. For example, the peripheral monitoring sensor 4 includes a peripheral monitoring camera that captures an image of a predetermined area around the vehicle, a search wave sensor such as a millimeter-wave radar, a sonar, and a LIDAR that transmits a search wave to a predetermined area around the vehicle. The peripheral monitoring camera sequentially outputs the captured images sequentially captured to the in-vehicle LAN as sensing information. The search wave sensor sequentially outputs a scanning result based on a reception signal obtained when a reflected wave reflected by the object is received to the in-vehicle LAN as sensing information.
より詳しくは、探査波センサは、探査波を送信してから、対象物で反射される反射波を受信するまでにかかった時間をもとに、探査波センサから対象物までの距離を測定する。また、探査波センサは、探査波を走査することで、反射波を受信する際の探査波の送信角度から、探査波センサに対する対象物の方位を測定する。方位は方位角で表せばよく、例えば正面方向を基準として時計回りを正の方位角、反時計回りを負の方位角で表せばよい。なお、探査波センサは、探査波を送信する送信部を自車の異なる箇所に複数設けることで、どの送信部から探査波を送信する場合に反射波を受信するかによって探査波センサに対する対象物の方位角を測定する構成としてもよい。
More specifically, the search wave sensor measures the distance from the search wave sensor to the target based on the time taken from transmitting the search wave to receiving the reflected wave reflected by the target. . The search wave sensor scans the search wave, and measures the direction of the target object with respect to the search wave sensor from the transmission angle of the search wave when receiving the reflected wave. The azimuth may be represented by an azimuth angle. For example, clockwise rotation may be represented by a positive azimuth angle and counterclockwise rotation may be represented by a negative azimuth angle with respect to the front direction. The exploration wave sensor is provided with a plurality of transmitters for transmitting the exploration wave at different places of the vehicle, and the target for the exploration wave sensor depends on which transmission unit transmits the exploration wave to receive the reflected wave. May be measured.
本実施形態では、周辺監視センサ4として、少なくとも、自車の前方の所定範囲を撮像範囲とする前方カメラ41と、自車の前方の所定範囲に探査波を送信するミリ波レーダ42とを用いる構成とする。この前方カメラ41が撮像装置に相当する。前方カメラ41とミリ波レーダ42とのセンシング範囲は、少なくとも一部が重なるが、同一である必要はない。例えば、前方カメラ41は、自車のルームミラー,インストルメントパネル上面等に設ける構成とすればよい。また、ミリ波レーダ42は、自車のフロントグリル,フロントバンパ等に設ける構成とすればよい。ミリ波レーダ42は、ミリ波又は準ミリ波を自車の前方の所定範囲を走査しながら送信し、対象物によって反射された反射波を受信することで、ミリ波レーダ42に対しての対象物の距離及び方位角を測定する。以降では、ミリ波レーダ42に対する対象物の距離及び方位角、つまり自車に対する対象物の距離及び方位角を、単に対象物の距離及び方位角と呼ぶ。
In the present embodiment, at least a front camera 41 whose imaging range is a predetermined range in front of the own vehicle and a millimeter wave radar 42 that transmits a search wave to a predetermined range in front of the own vehicle are used as the surrounding monitoring sensor 4. Configuration. The front camera 41 corresponds to an imaging device. The sensing ranges of the front camera 41 and the millimeter wave radar 42 at least partially overlap, but need not be the same. For example, the front camera 41 may be provided on a rearview mirror of an own vehicle, an upper surface of an instrument panel, or the like. The millimeter-wave radar 42 may be provided on a front grill, a front bumper, or the like of the vehicle. The millimeter-wave radar 42 transmits a millimeter wave or a quasi-millimeter wave while scanning a predetermined range in front of the own vehicle, and receives a reflected wave reflected by the target object, so that the millimeter wave radar 42 Measure the distance and azimuth of the object. Hereinafter, the distance and the azimuth of the object with respect to the millimeter-wave radar 42, that is, the distance and the azimuth of the object with respect to the own vehicle are simply referred to as the distance and the azimuth of the object.
なお、周辺監視センサ4としては、自車の前方以外を撮像するカメラを用いたり、ソナー,LIDAR(Light Detection and Ranging/Laser Imaging Detect ion and Ranging)等の探査波センサを用いたりする構成としてもよい。
Note that the surrounding monitoring sensor 4 may be configured to use a camera that captures an image other than in front of the own vehicle, or use a search wave sensor such as sonar or LIDAR (Light Detection and Ranging / Laser Imaging Detection and Ranging). Good.
車両制御ECU5は、自車の加減速制御及び/又は操舵制御を行う電子制御装置である。車両制御ECU5としては、操舵制御を行う操舵ECU、加減速制御を行うパワーユニット制御ECU及びブレーキECU等がある。車両制御ECU5は、自車に搭載されたアクセルポジションセンサ、ブレーキ踏力センサ、舵角センサ、車輪速センサ等の各センサから出力される検出信号を取得し、電子制御スロットル、ブレーキアクチュエータ、EPS(Electric Power Steering)モータ等の各走行制御デバイスへ制御信号を出力する。また、車両制御ECU5は、上述の各センサの検出信号を車内LANへ出力可能である。
The vehicle control ECU 5 is an electronic control unit that performs acceleration / deceleration control and / or steering control of the own vehicle. Examples of the vehicle control ECU 5 include a steering ECU that performs steering control, a power unit control ECU that performs acceleration / deceleration control, and a brake ECU. The vehicle control ECU 5 acquires detection signals output from various sensors such as an accelerator position sensor, a brake pedal force sensor, a steering angle sensor, and a wheel speed sensor mounted on the own vehicle, and performs electronic control throttle, a brake actuator, an EPS (Electric) Power Steering) Outputs control signals to each drive control device such as a motor. Further, the vehicle control ECU 5 can output detection signals of the above-described sensors to the in-vehicle LAN.
運転支援ECU6は、車両制御ECU5を制御することにより、ドライバによる運転操作の代行を行う自動運転機能を実行する。運転支援ECU6は、ADASロケータ3から取得する自車の車両位置及び地図データ,周辺監視センサ4でのセンシング情報をもとに、自車の走行環境を認識する。一例としては、周辺監視センサ4でのセンシング情報から、自車周辺の物体の形状及び移動状態を認識したり、自車周辺の路面標示の形状を認識したりする。そして、自車の車両位置及び地図データと組み合わせることで、実際の走行環境を三次元で再現した仮想空間を生成する。
(4) The driving support ECU 6 controls the vehicle control ECU 5 to execute an automatic driving function of substituting a driving operation by a driver. The driving support ECU 6 recognizes the traveling environment of the own vehicle based on the vehicle position and the map data of the own vehicle acquired from the ADAS locator 3 and the sensing information of the surrounding monitoring sensor 4. As an example, from the sensing information of the periphery monitoring sensor 4, the shape and the moving state of the object around the own vehicle are recognized, and the shape of the road marking around the own vehicle is recognized. Then, by combining with the vehicle position of the own vehicle and the map data, a virtual space in which the actual traveling environment is reproduced in three dimensions is generated.
また、運転支援ECU6は、認識した走行環境に基づき、自動運転機能によって自車を自動走行させるための走行計画を生成する。走行計画としては、長中期の走行計画と、短期の走行計画とを生成する等すればよい。短期の走行計画では、生成した自車の周囲の仮想空間を用いて、例えば先行車との目標車間距離を維持するための加減速,車線追従及び車線変更のための操舵,並びに衝突回避のための急制動等が決定される。長中期の走行計画としては、自車を目的地へ向かわせるための推奨経路を生成する。なお、運転支援ECU6は、短期の走行計画と長中期の走行計画とのうちの短期の走行計画のみを生成する構成としてもよい。
{Circle around (5)} The driving support ECU 6 generates a traveling plan for automatically traveling the own vehicle by the automatic driving function based on the recognized traveling environment. As the travel plan, a long- and medium-term travel plan and a short-term travel plan may be generated. In a short-term traveling plan, the generated virtual space around the own vehicle is used, for example, for acceleration / deceleration to maintain a target inter-vehicle distance with a preceding vehicle, steering for lane following and lane changing, and collision avoidance. Sudden braking or the like is determined. As a long- and medium-term traveling plan, a recommended route for causing the own vehicle to go to the destination is generated. The driving support ECU 6 may be configured to generate only a short-term traveling plan of the short-term traveling plan and the long- and medium-term traveling plan.
運転支援ECU6で実行する自動運転機能の一例としては、駆動力及び制動力を調整することで、先行車との目標車間距離を維持するように自車の走行速度を制御するACC(Adaptive Cruise Control)機能がある。また、前方のセンシング情報をもとに制動力を発生させることで、自車を強制的に減速させるAEB(Autonomous Emergency Braking)機能がある。なお、ここで述べたのは、あくまで一例であり、自動運転の機能として他の機能を備えている構成としてもよい。
As an example of the automatic driving function executed by the driving support ECU 6, an ACC (Adaptive Cruise Control) that adjusts the driving force and the braking force to control the traveling speed of the own vehicle so as to maintain the target inter-vehicle distance with the preceding vehicle. ) There is a function. There is also an AEB (Autonomous Emergency Breaking) function that forcibly decelerates the vehicle by generating a braking force based on sensing information in front. It should be noted that what has been described here is merely an example, and a configuration having another function as a function of automatic driving may be adopted.
HMIシステム2は、HCU(Human Machine Interface Control Unit)20、操作デバイス21、DSM(Driver Status Monitor)22、及び表示装置23を備えており、自車のユーザであるドライバからの入力操作を受け付けたり、自車のドライバに向けて情報を提示したりする。操作デバイス21は、自車のドライバが操作するスイッチ群である。操作デバイス21は、各種の設定を行うために用いられる。例えば、操作デバイス21としては、自車のステアリングのスポーク部に設けられたステアリングスイッチ等がある。
The HMI system 2 includes an HCU (Human Machine Interface Control Unit) 20, an operation device 21, a DSM (Driver Status Monitor) 22, and a display device 23, and receives an input operation from a driver who is a user of the own vehicle. And present information to the driver of the own vehicle. The operation device 21 is a group of switches operated by a driver of the own vehicle. The operation device 21 is used for performing various settings. For example, as the operation device 21, there is a steering switch or the like provided on a spoke portion of the steering of the own vehicle.
DSM22は、近赤外光源及び近赤外カメラと、これらを制御する制御ユニット等とによって構成されている。DSM22は、近赤外カメラを自車の運転席側に向けた姿勢にて、例えばステアリングコラムカバー,インストルメントパネル12(図2参照)の上面等に配置される。DSM22は、近赤外光源によって近赤外光を照射されたドライバの頭部を、近赤外カメラによって撮影する。近赤外カメラによる撮像画像は、制御ユニットによって画像解析される。制御ユニットは、例えばドライバの目を撮像画像から抽出し、車室内の基準位置に対する目の位置(以下、視点位置)を検出する。基準位置は例えば近赤外カメラの設置位置等とすればよい。DSM22は、検出した視点位置の情報をHCU20へ出力する。
The DSM 22 includes a near-infrared light source and a near-infrared camera, and a control unit and the like for controlling these. The DSM 22 is disposed, for example, on the steering column cover, the upper surface of the instrument panel 12 (see FIG. 2), or the like, with the near-infrared camera facing the driver's seat side of the vehicle. The DSM 22 uses a near-infrared camera to photograph the driver's head irradiated with near-infrared light by the near-infrared light source. The image captured by the near-infrared camera is analyzed by the control unit. The control unit extracts, for example, the driver's eyes from the captured image, and detects the position of the eyes with respect to a reference position in the vehicle interior (hereinafter, a viewpoint position). The reference position may be, for example, an installation position of the near-infrared camera. The DSM 22 outputs information on the detected viewpoint position to the HCU 20.
表示装置23としては、ヘッドアップディスプレイ(HUD)装置230を用いる。ここで、図2を用いてHUD装置230について説明を行う。図2に示すようにHUD230は、自車のインストルメントパネル12に設けられる。HUD230は、例えば液晶式又は走査式等のプロジェクタ231により、HCU20から出力される画像データに基づく表示画像を形成する。表示画像としては、ACC機能によって自車が追従する先行車を強調表示するための画像等の対象物を強調表示するための画像がある。強調表示の対象となる対象物は、ACC機能によって自車が追従する先行車に限らず、自車の回避対象となる障害物等であってもよい。なお、表示画像としては、対象物を強調表示するための画像に限らず、車速,自動運転機能の動作状態等の自車状態を示す画像を含んだり、自車の予定進路を示す画像を含んだりする構成としてもよい。
ヘ ッ ド A head-up display (HUD) device 230 is used as the display device 23. Here, the HUD device 230 will be described with reference to FIG. As shown in FIG. 2, HUD 230 is provided on instrument panel 12 of the host vehicle. The HUD 230 forms a display image based on image data output from the HCU 20 by a projector 231 of, for example, a liquid crystal type or a scanning type. Examples of the display image include an image for highlighting an object such as an image for highlighting a preceding vehicle followed by the own vehicle by the ACC function. The target object to be highlighted is not limited to the preceding vehicle that the own vehicle follows with the ACC function, and may be an obstacle or the like to be avoided by the own vehicle. The display image is not limited to an image for highlighting the target object, but may include an image indicating a vehicle state such as a vehicle speed and an operation state of an automatic driving function, or may include an image indicating a planned course of the vehicle. It is good also as a structure which rubs.
HUD230は、プロジェクタ231によって形成される表示画像を、例えば凹面鏡等の光学系232を通じて、投影部材としてのフロントウインドシールド10に既定された投影領域に投影する。投影領域は、運転席前方に位置するものとする。フロントウインドシールド10によって車室内側に反射された表示画像の光束は、運転席に着座するドライバによって知覚される。また、透光性ガラスにより形成されるフロントウインドシールド10を透過した、自車の前方に存在する風景としての前景からの光束も、運転席に着座するドライバによって知覚される。これにより、ドライバは、フロントウインドシールド10の前方にて結像される表示画像の虚像100を、前景の一部と重ねて視認可能となる。つまり、HUD230は、自車の前景に虚像100を重畳表示し、所謂AR(Augmented Reality)表示を実現する。
The HUD 230 projects a display image formed by the projector 231 through an optical system 232 such as a concave mirror, for example, onto a projection area defined on the front windshield 10 as a projection member. The projection area is located in front of the driver's seat. The light flux of the display image reflected by the front windshield 10 toward the vehicle interior is perceived by the driver sitting in the driver's seat. Further, the light flux from the foreground, which is a scene existing in front of the own vehicle, transmitted through the front windshield 10 formed of the translucent glass is also perceived by the driver sitting in the driver's seat. This allows the driver to visually recognize the virtual image 100 of the display image formed in front of the front windshield 10 so as to overlap a part of the foreground. That is, the HUD 230 superimposes and displays the virtual image 100 on the foreground of the own vehicle, and realizes so-called AR (Augmented Reality) display.
なお、HUD230が表示画像を投影する投影部材は、フロントウインドシールド10に限らず、透光性コンバイナであっても構わない。また、表示装置23として、HUD220の他にも、画像を表示する装置を用いる構成としてもよい。一例としては、コンビネーションメータ、CID(Center Information Display)等がある。
The projection member on which the HUD 230 projects the display image is not limited to the front windshield 10 and may be a light-transmitting combiner. Further, as the display device 23, a device that displays an image in addition to the HUD 220 may be used. As an example, there are a combination meter, a CID (Center @ Information @ Display), and the like.
HCU20は、プロセッサ、揮発性メモリ、不揮発性メモリ、I/O、これらを接続するバスを備えるマイクロコンピュータを主体として構成され、HUD230と車内LANとに接続されている。HCU20は、不揮発性メモリに記憶された制御プログラムを実行することにより、HUD230による表示を制御する。このHCU20が車両用表示制御装置に相当する。プロセッサがこの制御プログラムを実行することは、制御プログラムに対応する車両用表示制御方法が実行されることに相当する。ここで言うところのメモリは、コンピュータによって読み取り可能なプログラム及びデータを非一時的に格納する非遷移的実体的記憶媒体(non-transitory tangible storage medium)である。また、非遷移的実体的記憶媒体は、半導体メモリ又は磁気ディスクなどによって実現される。なお、HUD装置230による表示の制御に関するHCU20の構成については、以下で詳述する。
The HCU 20 is mainly configured by a microcomputer including a processor, a volatile memory, a non-volatile memory, an I / O, and a bus connecting these, and is connected to the HUD 230 and the in-vehicle LAN. The HCU 20 controls the display by the HUD 230 by executing a control program stored in the nonvolatile memory. The HCU 20 corresponds to a vehicle display control device. Executing the control program by the processor corresponds to executing the vehicle display control method corresponding to the control program. The memory here is a non-transitory tangible storage medium that temporarily stores programs and data that can be read by a computer. Further, the non-transitional substantive storage medium is realized by a semiconductor memory, a magnetic disk, or the like. The configuration of the HCU 20 related to display control by the HUD device 230 will be described in detail below.
<HCU20の概略構成>
ここで、図3を用いてHCU20の概略構成についての説明を行う。HCU20は、HUD装置230での表示制御に関して、図3に示すように、撮像画像取得部200、距離方位取得部201、自車位置取得部202、高精度地図取得部203、三次元位置特定部204、視点位置特定部205、不可視領域推定部206、及び表示生成部207を機能ブロックとして備える。なお、HCU20が実行する機能の一部又は全部を、一つ或いは複数のIC等によりハードウェア的に構成してもよい。また、HCU20が備える機能ブロックの一部又は全部は、プロセッサによるソフトウェアの実行とハードウェア部材の組み合わせによって実現されてもよい。 <Schematic configuration ofHCU 20>
Here, a schematic configuration of theHCU 20 will be described with reference to FIG. Regarding display control in the HUD device 230, as shown in FIG. 3, the HCU 20 acquires a captured image acquisition unit 200, a distance and orientation acquisition unit 201, a vehicle position acquisition unit 202, a high-accuracy map acquisition unit 203, and a three-dimensional position identification unit. 204, a viewpoint position identification unit 205, an invisible area estimation unit 206, and a display generation unit 207 are provided as functional blocks. Note that some or all of the functions performed by the HCU 20 may be configured in hardware by one or more ICs or the like. Further, some or all of the functional blocks included in the HCU 20 may be realized by a combination of execution of software by a processor and hardware members.
ここで、図3を用いてHCU20の概略構成についての説明を行う。HCU20は、HUD装置230での表示制御に関して、図3に示すように、撮像画像取得部200、距離方位取得部201、自車位置取得部202、高精度地図取得部203、三次元位置特定部204、視点位置特定部205、不可視領域推定部206、及び表示生成部207を機能ブロックとして備える。なお、HCU20が実行する機能の一部又は全部を、一つ或いは複数のIC等によりハードウェア的に構成してもよい。また、HCU20が備える機能ブロックの一部又は全部は、プロセッサによるソフトウェアの実行とハードウェア部材の組み合わせによって実現されてもよい。 <Schematic configuration of
Here, a schematic configuration of the
撮像画像取得部200は、前方カメラ41で逐次撮像する撮像画像をセンシング情報として取得する。距離方位取得部201は、ミリ波レーダ42で逐次測定する対象物の距離及び方位角をセンシング情報として取得する。自車位置取得部202は、ADASロケータ3で逐次測位する自車の車両位置を取得する。高精度地図取得部203は、ADASロケータ3の地図DB32に格納されている地図データを取得する。高精度地図取得部203は、観測点別、つまり複数の地点別の高度及び横断勾配の情報を含む高精度地図を取得することになる。
The captured image acquisition unit 200 acquires captured images sequentially captured by the front camera 41 as sensing information. The distance and azimuth obtaining unit 201 obtains, as sensing information, the distance and azimuth of an object sequentially measured by the millimeter wave radar 42. The own vehicle position acquisition unit 202 acquires the vehicle position of the own vehicle that is sequentially measured by the ADAS locator 3. The high-accuracy map acquisition unit 203 acquires map data stored in the map DB 32 of the ADAS locator 3. The high-accuracy map acquiring unit 203 acquires a high-accuracy map including information on altitude and cross slope for each observation point, that is, for each of a plurality of points.
三次元位置特定部204は、自車に対する対象物の三次元位置を特定する。例えば、三次元位置特定部204は、運転支援ECU6で認識する走行環境のうちから対象物を選定して三次元位置を特定すればよい。一例としては、自車の回避対象とする障害物を対象物として選定したり、ACC機能で追従させる先行車を対象物として選定したりする構成とすればよい。以下では、ACC機能で追従させる先行車を対象物として選定する場合を例に挙げて説明を行う。運転支援ECU6で認識する走行環境のうちの対象物と、撮像画像取得部200で取得する撮像画像中の対象物及び距離方位取得部201で距離及び方位を取得する対象物との紐付けは、自車に体する距離及び方位の略一致をもとに行う等すればよい。
The three-dimensional position specifying unit 204 specifies the three-dimensional position of the target object with respect to the own vehicle. For example, the three-dimensional position specifying unit 204 may specify a three-dimensional position by selecting an object from the driving environment recognized by the driving support ECU 6. As an example, the configuration may be such that an obstacle to be avoided by the own vehicle is selected as an object or a preceding vehicle to be followed by the ACC function is selected as an object. In the following, a case where a preceding vehicle to be followed by the ACC function is selected as an object will be described as an example. The link between the object in the driving environment recognized by the driving support ECU 6, the object in the captured image acquired by the captured image acquisition unit 200, and the object whose distance and direction are acquired by the distance and orientation acquisition unit 201 is: What is necessary is just to carry out based on the substantially coincidence of the distance and the azimuth of the own vehicle.
三次元位置特定部204は、自車に対する対象物の距離が所定距離以上の場合には、距離方位取得部201で取得する対象距離及び方位角と、高精度地図取得部で取得する高精度地図とを用いて、自車に対する対象物の三次元位置を特定する。詳細については後述する。ここで言うところの所定距離とは、前方カメラ41の撮像画像をもとに撮像画像中の物体の高さを特定する精度が十分でなくなり始めると推定される距離とすればよく、任意に設定可能な値である。
When the distance of the target object to the own vehicle is equal to or longer than a predetermined distance, the three-dimensional position specifying unit 204 obtains the target distance and the azimuth obtained by the distance azimuth obtaining unit 201 and the high-accuracy map obtained by the high-accuracy map obtaining unit Is used to specify the three-dimensional position of the target object with respect to the own vehicle. Details will be described later. Here, the predetermined distance may be a distance that is estimated to start to be insufficient in the accuracy of identifying the height of the object in the captured image based on the captured image of the front camera 41, and may be arbitrarily set. Possible values.
一方、三次元位置特定部204は、自車に対する対象物の距離が前述の所定距離未満の場合には、距離方位取得部201で取得する距離及び方位角に加え、撮像画像取得部200で取得する撮像画像を用いて、自車に対する対象物の三次元位置を特定する。詳細については後述する。なお、三次元位置特定部204は、自車に対する対象物の距離が前述の所定距離未満の場合にも高精度地図取得部203で取得する高精度地図を用いる構成としてもよい。
On the other hand, when the distance of the target object to the own vehicle is shorter than the predetermined distance, the three-dimensional position specifying unit 204 obtains the distance and the azimuth obtained by the distance and azimuth obtaining unit 201 and obtains the image by the captured image obtaining unit 200. The three-dimensional position of the object with respect to the own vehicle is specified using the captured image to be performed. Details will be described later. Note that the three-dimensional position specifying unit 204 may use the high-accuracy map acquired by the high-accuracy map acquiring unit 203 even when the distance of the target object to the own vehicle is shorter than the above-described predetermined distance.
視点位置特定部205は、DSM22で逐次検出する視点位置の情報から自車の車両位置を基準とするドライバの視点位置を特定する。例えば、視点位置特定部205は、DSM22で検出する視点位置を、DSM22での視点位置の基準とする位置と自車における車両位置の基準となる位置とのずれをもとに、自車の車両位置を基準とする視点位置に変換することで、自車のドライバの視点位置を特定する。
The viewpoint position specifying unit 205 specifies the viewpoint position of the driver based on the vehicle position of the own vehicle from the information on the viewpoint positions sequentially detected by the DSM 22. For example, the viewpoint position specifying unit 205 determines the viewpoint position detected by the DSM 22 based on the deviation between the reference position of the viewpoint position in the DSM 22 and the reference position of the vehicle position in the own vehicle. The viewpoint position of the driver of the own vehicle is specified by converting the position into the viewpoint position based on the position.
不可視領域推定部206は、高精度地図取得部203で取得する高精度地図に高度の情報が含まれている地点のうち、自車と対象物との間の地点の高度の情報と、視点位置特定部205で特定する視点位置とをもとに、路面の勾配変化によって自車のドライバから不可視となる不可視領域を推定する。詳細については後述する。
The invisible area estimating unit 206 obtains the altitude information of a point between the own vehicle and the target object among the altitude information included in the high-accuracy map acquired by the high-accuracy map acquiring unit 203, and the viewpoint position. Based on the viewpoint position specified by the specifying unit 205, an invisible area that is invisible to the driver of the own vehicle due to a change in the gradient of the road surface is estimated. Details will be described later.
表示生成部207は、自車に対する対象物の三次元位置に合わせて対象物を強調する虚像を重畳表示させるための画像データを生成し、この画像データをHUD230に出力することで、自車に対する対象物の三次元位置に合わせて対象物を強調する虚像を重畳表示させる。この表示生成部207が表示制御部に相当する。画像データを構成する個々のフレームには、対象物を強調表示するためのマーク(以下、強調表示マーク)が描画されている。強調表示マークの描画位置は、強調表示マークの表示画像が投影領域に投影されたときに、強調表示マークの表示画像に基づく虚像が前景中の意図する位置に表示されるように設定される。
The display generation unit 207 generates image data for superimposing and displaying a virtual image emphasizing the target object in accordance with the three-dimensional position of the target object with respect to the own vehicle, and outputs the image data to the HUD 230 to output the image data to the HUD 230. A virtual image that emphasizes the target object is superimposed and displayed in accordance with the three-dimensional position of the target object. The display generation unit 207 corresponds to a display control unit. Marks for highlighting the object (hereinafter, highlight marks) are drawn on individual frames constituting the image data. The drawing position of the highlighting mark is set such that a virtual image based on the display image of the highlighting mark is displayed at an intended position in the foreground when the display image of the highlighting mark is projected on the projection area.
一例としては、表示生成部207は、三次元位置特定部204で特定する自車に対する対象物の三次元位置と、視点位置特定部205で特定する視点位置と、予めHCU20の不揮発性メモリに格納されている投影領域の設定位置とをもとに、対象物、視点、及び投影領域の位置関係を把握する。そして、表示生成部207は、対象物、視点、及び投影領域の位置関係に基づき、強調表示マークの表示画像の投影領域内での投影位置、つまり、強調表示マークの表示画像に基づく虚像の結像位置を、幾何学的な演算によって算出する。
As an example, the display generation unit 207 stores the three-dimensional position of the object with respect to the own vehicle specified by the three-dimensional position specification unit 204, the viewpoint position specified by the viewpoint position specification unit 205, and the non-volatile memory of the HCU 20 in advance. The positional relationship between the object, the viewpoint, and the projection area is grasped based on the set position of the projection area. Then, based on the positional relationship between the target object, the viewpoint, and the projection area, the display generation unit 207 determines the projection position of the display image of the highlighting mark in the projection area, that is, the formation of a virtual image based on the display image of the highlighting mark. The image position is calculated by a geometric operation.
また、表示生成部207は、不可視領域推定部206で推定される不可視領域が存在する場合には、この不可視領域を避けて、対象物を強調する虚像を重畳表示させる。対象物を強調する虚像の重畳表示の詳細については後述する。
{Circle around (4)} If the invisible area estimated by the invisible area estimation unit 206 exists, the display generation unit 207 superimposes and displays a virtual image that emphasizes the target object, avoiding the invisible area. The details of the superimposed display of the virtual image that emphasizes the object will be described later.
<HCU20での虚像表示制御関連処理>
続いて、図4のフローチャートを用いて、HCU20でのHUD230による重畳表示の制御に関連する処理(以下、虚像表示制御関連処理)の流れの一例について説明を行う。図4のフローチャートでは、HUD230の電源がオン且つHUD230の機能がオンになった場合に開始する構成とすればよい。HUD230の機能のオンオフは、操作デバイス21で受け付ける入力操作に応じて切り替えられる構成とすればよい。また、HUD230の電源のオンオフは、自車の内燃機関又はモータジェネレータを始動させるためのスイッチ(以下、パワースイッチ)のオンオフに応じて切り替えられる構成とすればよい。 <Virtual image display control related processing inHCU 20>
Next, an example of a flow of processing (hereinafter, virtual image display control-related processing) related to control of superimposed display by theHUD 230 in the HCU 20 will be described using the flowchart of FIG. The flowchart in FIG. 4 may be configured to start when the power of the HUD 230 is turned on and the function of the HUD 230 is turned on. The function of the HUD 230 may be switched on / off in accordance with an input operation received by the operation device 21. The power supply of the HUD 230 may be switched on and off according to the on / off of a switch (hereinafter, a power switch) for starting the internal combustion engine or the motor generator of the own vehicle.
続いて、図4のフローチャートを用いて、HCU20でのHUD230による重畳表示の制御に関連する処理(以下、虚像表示制御関連処理)の流れの一例について説明を行う。図4のフローチャートでは、HUD230の電源がオン且つHUD230の機能がオンになった場合に開始する構成とすればよい。HUD230の機能のオンオフは、操作デバイス21で受け付ける入力操作に応じて切り替えられる構成とすればよい。また、HUD230の電源のオンオフは、自車の内燃機関又はモータジェネレータを始動させるためのスイッチ(以下、パワースイッチ)のオンオフに応じて切り替えられる構成とすればよい。 <Virtual image display control related processing in
Next, an example of a flow of processing (hereinafter, virtual image display control-related processing) related to control of superimposed display by the
まず、ステップS1では、距離方位取得部201が、ミリ波レーダ42で測定する対象物の距離及び方位角をセンシング情報として取得する。ステップS2では、三次元位置特定部204が、S1で取得する対象物の距離及び方位角をもとに、自車の車両位置を基準とする座標系における対象物の位置座標を算出する。詳しくは、極座標系であるミリ波レーダ42で測定する距離及び方位角を、自車の車両位置を基準とする左右方向の軸(以下、x軸)と前後方向の軸(以下、y軸)とで表されるxy座標系に座標変換する。自車に対する対象物の距離r,自車に対する対象物の方位角θとする場合、x=r・cosθ,y=r・sinθの演算式によって、対象物の(x,y)座標を算出する。
First, in step S1, the distance and azimuth acquisition unit 201 acquires the distance and azimuth of the object measured by the millimeter wave radar 42 as sensing information. In step S2, the three-dimensional position identification unit 204 calculates the position coordinates of the target in a coordinate system based on the vehicle position of the host vehicle based on the distance and the azimuth of the target obtained in S1. Specifically, the distance and azimuth measured by the millimeter-wave radar 42, which is a polar coordinate system, are measured by using a horizontal axis (hereinafter, x-axis) and a front-rear axis (hereinafter, y-axis) based on the vehicle position of the vehicle. Is converted to an xy coordinate system represented by Assuming that the distance r of the target object to the own vehicle and the azimuth angle θ of the target object to the own vehicle, the (x, y) coordinates of the target object are calculated by an arithmetic expression of x = r · cos θ and y = r · sin θ. .
ステップS3では、S1で取得する対象物の距離が前述の所定距離以上である場合(S3でYES)には、ステップS4に移る。一方、S1で取得する対象物の距離が前述の所定距離未満である場合(S3でNO)には、ステップS12に移る。
In step S3, if the distance of the object acquired in S1 is equal to or longer than the predetermined distance (YES in S3), the process proceeds to step S4. On the other hand, if the distance of the object acquired in S1 is shorter than the above-described predetermined distance (NO in S3), the process proceeds to step S12.
ステップS4では、高精度地図取得部203が、地図DB32に格納されている高精度地図を取得する。例えば高精度地図取得部203は、自車位置取得部202で取得する自車の車両位置周辺に絞って高精度地図を取得する構成とすればよい。
In step S4, the high-accuracy map acquisition unit 203 acquires a high-accuracy map stored in the map DB 32. For example, the high-accuracy map acquisition unit 203 may be configured to acquire a high-accuracy map by focusing on the vicinity of the vehicle position of the own vehicle acquired by the own vehicle position acquisition unit 202.
ステップS5では、三次元位置特定部204が、S2で算出する対象物の(x,y)座標に対応する地点(以下、対象地点)が、S4で取得する高精度地図に高さ及び横断勾配の情報が含まれている地点(以下、地図データ点)にあたるか否かを判別する。一例として、対象地点は、地理座標系における自車の車両位置の経緯度を、S2で算出する対象物の(x,y)座標分だけオフセットすることで特定すればよい。そして、対象地点が、地図データ点にあたる場合(S5でYES)には、ステップS6に移る。一方、対象地点が、地図データ点にあたらない場合(S5でNO)には、ステップS7に移る。
In step S5, the three-dimensional position specifying unit 204 determines the point (hereinafter, the target point) corresponding to the (x, y) coordinates of the target object calculated in S2 on the high-accuracy map acquired in S4 by the height and the cross slope. It is determined whether or not a point (hereinafter, a map data point) in which the information is included. As an example, the target point may be specified by offsetting the latitude and longitude of the vehicle position of the own vehicle in the geographic coordinate system by the (x, y) coordinates of the target object calculated in S2. When the target point corresponds to the map data point (YES in S5), the process proceeds to step S6. On the other hand, when the target point does not correspond to the map data point (NO in S5), the process proceeds to step S7.
ステップS6では、三次元位置特定部204が、自車に対する対象物の高さ方向の位置を特定して、ステップS9に移る。詳しくは、自車の車両位置を基準とする高さ方向の軸(以下、z軸)のz座標を特定する。ここで、図5を用いて、対象地点が地図データ点にあたる場合の、自車に対する対象物の高さ方向の位置の特定の一例について説明する。図5のOvが自車を示しており、Vaが対象物としての先行車を示している。また、図5の黒丸が地図データ点,Opが地図データ点のうちの自車が位置する地点,Apが地図データ点のうちの先行車が位置する地点を示している。
In step S6, the three-dimensional position specifying unit 204 specifies the position of the target object in the height direction with respect to the own vehicle, and proceeds to step S9. Specifically, the z-coordinate of an axis in the height direction (hereinafter, z-axis) based on the vehicle position of the own vehicle is specified. Here, an example of specifying the position of the target object in the height direction with respect to the own vehicle when the target point corresponds to a map data point will be described with reference to FIG. Ov in FIG. 5 indicates the own vehicle, and Va indicates a preceding vehicle as an object. In FIG. 5, black circles indicate map data points, Op indicates a point of the map data points where the vehicle is located, and Ap indicates a point of the map data points where the preceding vehicle is located.
三次元位置特定部204は、高精度地図上の自車が位置する地点(図5のOp参照)の高度と、高精度地図上のS2で算出する対象物の(x,y)座標が示す地点(図5のAp参照)の高度との差分(図5のDi参照)から対象物のz座標を特定する。なお、三次元位置特定部204は、例えば自車の車両位置としては、自車位置取得部202で取得する自車の車両位置を、S4で取得する高精度地図の地図データ点にマップマッチングさせた位置を用いればよい。
The three-dimensional position specifying unit 204 indicates the altitude of the point on the high-accuracy map where the vehicle is located (see Op in FIG. 5) and the (x, y) coordinates of the object calculated in S2 on the high-accuracy map. The z coordinate of the target is specified from the difference (see Di in FIG. 5) from the altitude of the point (see Ap in FIG. 5). The three-dimensional position identification unit 204 performs, for example, map matching of the vehicle position of the own vehicle acquired by the own vehicle position acquisition unit 202 with the map data points of the high-accuracy map acquired in S4. The position may be used.
ステップS7では、三次元位置特定部204が、S4で取得する高精度地図の地図データ点から、対象地点にとっての横断勾配の基準となる勾配基準点を探索する。ここで、図6を用いて、三次元位置特定部204での対象地点にとっての勾配基準点の探索の一例について説明する。図6のOvが自車を示しており、Vaが対象物としての先行車を示している。また、図6の黒丸が地図データ点,Opが地図データ点のうちの自車が位置する地点,Apが地図データ点のうちの先行車が位置する地点,Scpが対象地点にとっての勾配基準点を示している。
In step S7, the three-dimensional position identification unit 204 searches the map data points of the high-accuracy map acquired in S4 for a gradient reference point serving as a reference for the crossing gradient for the target point. Here, an example of the search of the gradient reference point for the target point in the three-dimensional position specifying unit 204 will be described with reference to FIG. Ov in FIG. 6 indicates the own vehicle, and Va indicates a preceding vehicle as an object. In FIG. 6, black circles are map data points, Op is a point of the map data points where the vehicle is located, Ap is a point of the map data points where the preceding vehicle is located, and Scp is a gradient reference point for the target point. Is shown.
三次元位置特定部204は、自車の車両位置よりも前方の地図データ点から、自車前方の経路を近似する経路近似曲線(図6の破線参照)を算出する。一例としては、スプライン曲線を算出すればよい。続いて、三次元位置特定部204は、対象地点(図6のAp参照)からこの経路近似曲線に垂線を下ろし、垂線と経路近似曲線との交わる点から最短距離にある地図データ点を勾配基準点(図6のScp参照)とする。なお、三次元位置特定部204は、対象地点を高精度地図にマップマッチングさせて、対象地点から最短距離にある地図データ点を勾配基準点として探索すると言い換えることもできる。
The three-dimensional position identification unit 204 calculates a route approximation curve (see a broken line in FIG. 6) that approximates a route ahead of the own vehicle from map data points ahead of the vehicle position of the own vehicle. As an example, a spline curve may be calculated. Subsequently, the three-dimensional position specifying unit 204 draws a perpendicular from the target point (see Ap in FIG. 6) to this route approximation curve, and determines a map data point located at the shortest distance from the intersection of the perpendicular and the route approximation curve as a gradient reference. A point (see Scp in FIG. 6). In other words, the three-dimensional position identification unit 204 performs map matching of the target point with the high-accuracy map and searches for a map data point located at the shortest distance from the target point as a gradient reference point.
ステップS8では、S7で探索する勾配基準点における横断勾配と、この勾配基準点と対象地点との距離とを用いて、対象地点の高さを特定してステップS9に移る。ここで、図6を用いて、対象地点が地図データ点にあたらない場合の、自車に対する対象物の高さ方向の位置の特定の一例について説明する。
In step S8, the height of the target point is specified using the transverse gradient at the gradient reference point searched in S7 and the distance between the gradient reference point and the target point, and the process proceeds to step S9. Here, an example of specifying the position of the target object in the height direction with respect to the own vehicle when the target point does not correspond to the map data point will be described with reference to FIG.
三次元位置特定部204は、勾配基準点の(x,y)座標と対象地点の(x,y)座標とから、勾配基準点から対象地点までの距離を算出する。一例として、勾配基準点の(x,y)座標は、地理座標系における勾配基準点の経緯度の座標を、地理座標系における自車の車両位置の経緯度の座標分だけオフセットすることで特定すればよい。続いて、勾配基準点から対象地点までの距離と、勾配基準点における横断勾配とをもとに、勾配基準点から対象地点までこの横断勾配で一定と仮定した上で三角関数を用いることで、勾配基準点と対象地点との高度の差分(図6のDi2参照)を算出する。そして、S6と同様にして算出する自車の車両位置と勾配基準点との高度の差分(図6のDi1参照)と、勾配基準点と対象地点との高度の差分(図6のDi2参照)とを足し合わせて対象物のz座標を特定する。
The three-dimensional position specifying unit 204 calculates a distance from the gradient reference point to the target point from the (x, y) coordinates of the gradient reference point and the (x, y) coordinates of the target point. As an example, the (x, y) coordinates of the gradient reference point are specified by offsetting the longitude and latitude coordinates of the gradient reference point in the geographic coordinate system by the latitude and longitude coordinates of the vehicle position of the own vehicle in the geographic coordinate system. do it. Then, based on the distance from the slope reference point to the target point and the cross slope at the slope reference point, by using a trigonometric function on the assumption that the slope reference point to the target point is constant at this cross slope, calculating the altitude difference between the slope reference point and target point (see Di 2 in FIG. 6). Then, a high degree of difference between the vehicle of the vehicle position and slope reference point calculated in the same manner as S6 (see Di 1 in FIG. 6), a high degree of difference between the slope reference point and the target point (Di in FIG 2 ) To specify the z coordinate of the object.
ステップS9では、三次元位置特定部204が、S2で算出する対象物の(x,y)座標と、S6若しくはS8で特定する対象物のz座標とを合わせて、自車に対する対象物の三次元位置、つまり(x,y,z)座標を特定する。
In step S9, the three-dimensional position specifying unit 204 combines the (x, y) coordinates of the target object calculated in S2 and the z coordinates of the target object specified in S6 or S8, and performs three-dimensional The original position, that is, (x, y, z) coordinates is specified.
ステップS10では、不可視領域推定部206が、S4で取得する高精度地図の地図データ点のうちの、自車と対象物との間の地図データ点の高度の情報と、視点位置特定部205で特定する視点位置とをもとに、路面の勾配変化によって自車のドライバから不可視となる不可視領域を推定する。ここで、図7を用いて、不可視領域の推定の一例について説明する。図7のOvが自車を示しており、Vaが対象物としての先行車を示している。また、図7の黒丸が地図データ点,Vpが視点位置,Losが視点位置を起点とする線分,Opが地図データ点のうちの自車が位置する地点,Apが地図データ点のうちの先行車が位置する地点を示している。
In step S10, the invisible area estimation unit 206 determines the altitude information of the map data points between the vehicle and the target object among the map data points of the high-accuracy map acquired in S4, Based on the specified viewpoint position, an invisible area invisible to a driver of the own vehicle due to a change in the gradient of the road surface is estimated. Here, an example of estimating the invisible area will be described with reference to FIG. Ov in FIG. 7 indicates the own vehicle, and Va indicates a preceding vehicle as an object. In FIG. 7, black circles are map data points, Vp is a viewpoint position, Los is a line segment starting from the viewpoint position, Op is a point of the map data point where the vehicle is located, and Ap is a map data point. The point where the preceding vehicle is located is shown.
不可視領域推定部206は、自車の車両位置(図7のOp参照)と対象地点(図7のAp参照)との間の地図データ点から高度の情報を抽出し、高度の情報をスプライン補間して近似曲線(以下、勾配近似曲線)を求める。なお、対象地点が、地図データ点にあたらない場合には、前述の勾配基準点を対象地点の代わりに用いる構成とすればよい。続いて、視点位置特定部205で特定する視点位置(図7のVp参照)を起点とし、求めた勾配近似曲線と接する線分を求め、接点の(x,y,z)座標を算出する。そして、求めた座標点のz座標、つまり自車に対する高さに基づき、この高さ以下の領域を不可視領域として推定する。この高さよりも高い領域が可視領域となる。なお、例えば、不可視領域推定部206は、視点位置から対象地点までの間で勾配近似曲線に接する線分を引けない場合は、不可視領域なしと推定すればよい。
The invisible area estimation unit 206 extracts altitude information from map data points between the position of the vehicle (see Op in FIG. 7) and a target point (see Ap in FIG. 7), and spline-interpolates the altitude information. To obtain an approximate curve (hereinafter, a gradient approximate curve). If the target point does not correspond to the map data point, the above-described gradient reference point may be used instead of the target point. Subsequently, with the viewpoint position specified by the viewpoint position specifying unit 205 (see Vp in FIG. 7) as a starting point, a line segment that is in contact with the obtained approximate slope curve is obtained, and the (x, y, z) coordinates of the contact point are calculated. Then, based on the z-coordinate of the obtained coordinate point, that is, the height with respect to the own vehicle, an area equal to or less than this height is estimated as an invisible area. A region higher than this height is a visible region. In addition, for example, when it is not possible to draw a line segment that is in contact with the approximate gradient curve from the viewpoint position to the target point, the invisible area estimation unit 206 may estimate that there is no invisible area.
ステップS11では、S10で不可視領域が推定される場合、つまり不可視領域ありの場合(S11でYES)には、ステップS13に移る。一方、S10で不可視領域なしの場合(S11でNO)には、ステップS12に移る。ステップS12では、表示生成部207が、不可視領域を強調表示マークの表示画像の描画対象から除外する不可視領域回避処理を行って、ステップS13に移る。
In step S11, if the invisible area is estimated in S10, that is, if there is an invisible area (YES in S11), the process proceeds to step S13. On the other hand, when there is no invisible area in S10 (NO in S11), the process proceeds to step S12. In step S12, the display generation unit 207 performs an invisible area avoiding process of excluding the invisible area from the drawing target of the display image of the highlighting mark, and proceeds to step S13.
ステップS13では、表示生成部207が、S9で特定する三次元位置に合わせて、強調表示マークの表示画像の画像データを生成する。ステップS14では、表示生成部207が、S13で生成する画像データをHUD230に出力することで、自車に対する対象物の三次元位置に合わせて対象物を強調する強調表示マークの虚像を前景に重畳表示させる。そして、ステップS20に移る。
In step S13, the display generation unit 207 generates image data of the display image of the highlighting mark in accordance with the three-dimensional position specified in S9. In step S14, the display generation unit 207 outputs the image data generated in S13 to the HUD 230, thereby superimposing a virtual image of a highlighting mark that emphasizes the target object on the foreground in accordance with the three-dimensional position of the target object with respect to the own vehicle. Display. Then, the process proceeds to step S20.
一例として、表示生成部207は、S9で特定する三次元位置と、視点位置特定部205で特定する視点位置と、予めHCU20の不揮発性メモリに格納されている投影領域の設定位置とをもとに、図8に示すような対象物(図8のVa参照)の下端に沿うライン状の強調表示マークの虚像(図8のHl参照)を前景に重畳表示させる。図8は、重畳表示の一例を示す図であって、Viが投影領域を示している。なお、対象物の下端に沿う形状であれば、ライン状以外の形状であってもよい。
As an example, the display generation unit 207 determines the three-dimensional position specified in S9, the viewpoint position specified by the viewpoint position specification unit 205, and the set position of the projection area stored in the nonvolatile memory of the HCU 20 in advance. Then, a virtual image (see Hl in FIG. 8) of a linear highlighting mark along the lower end of the object (see Va in FIG. 8) as shown in FIG. 8 is superimposed and displayed on the foreground. FIG. 8 is a diagram illustrating an example of the superimposed display, in which Vi indicates a projection area. It should be noted that any shape other than the line shape may be used as long as the shape is along the lower end of the object.
また、表示生成部207は、S12で不可視領域回避処理を行っている場合、つまり、不可視領域推定部206で推定される不可視領域が存在する場合には、この不可視領域を避けて、強調表示マークの虚像を重畳表示させる。具体例としては、図9に示すように、対象物(図9のVa参照)の下端に沿うライン状の強調表示マーク(図9のHl参照)を、可視領域の高さまで平行移動させて、対象物の可視領域中の下端に沿うように重畳表示させればよい。図9は、重畳表示の一例を示す図であって、Viが投影領域を示している。
When the invisible area avoiding process is performed in S12, that is, when there is an invisible area estimated by the invisible area estimating unit 206, the display generation unit 207 avoids the invisible area, and Is superimposed and displayed. As a specific example, as shown in FIG. 9, the line-shaped highlighting mark (see Hl in FIG. 9) along the lower end of the object (see Va in FIG. 9) is translated to the height of the visible region, What is necessary is just to superimpose and display along the lower end in the visible region of the object. FIG. 9 is a diagram illustrating an example of superimposed display, in which Vi indicates a projection area.
ステップS15では、撮像画像取得部200が、前方カメラ41で撮像する撮像画像をセンシング情報として取得する。ステップS16では、三次元位置特定部204が、S15で取得する撮像画像を用いて、自車に対する対象物の高さ方向の位置を特定する。三次元位置特定部204は、自車に対する対象物の高さ方向の位置については、撮像画像から画像認識によって検出する消失点の、平坦路での消失点からの変化量をもとに特定すればよい。
In step S15, the captured image acquisition unit 200 acquires a captured image captured by the front camera 41 as sensing information. In step S16, the three-dimensional position specifying unit 204 specifies the position of the target object in the height direction with respect to the own vehicle using the captured image acquired in step S15. The three-dimensional position specifying unit 204 specifies the position of the target object with respect to the vehicle in the height direction based on the amount of change of the vanishing point detected by image recognition from the captured image from the vanishing point on a flat road. I just need.
ここで、図10を用いて、撮像画像からの自車に対する対象物の高さ方向の位置の特定の一例について説明する。図10のAが前方上り勾配の路面の撮像画像の例を示しており、Bが平坦路の撮像画像の例を示している。なお、本実施形態では、平坦路の撮像画像中における消失点である基準点(図10のBP参照)の位置は、車両の製造時,車両へのHUD230の取り付け時等に予め求めてHCU20の不揮発性メモリに格納済みとする。また、基準点の位置は、車両のピッチング状態に応じた位置を選択して用いることができるように、例えば車両のハイトセンサの出力値別とすることが好ましい。
Here, an example of specifying the position in the height direction of the target object with respect to the own vehicle from the captured image will be described with reference to FIG. FIG. 10A illustrates an example of a captured image of a road surface having a forward ascending slope, and FIG. 10B illustrates an example of a captured image of a flat road. In the present embodiment, the position of the reference point (see BP in FIG. 10), which is a vanishing point in the captured image of the flat road, is obtained in advance at the time of manufacturing the vehicle, at the time of attaching the HUD 230 to the vehicle, and the like. It is assumed that the data is already stored in the nonvolatile memory. Further, it is preferable that the position of the reference point is, for example, for each output value of the height sensor of the vehicle so that a position corresponding to the pitching state of the vehicle can be selected and used.
まず、三次元位置特定部204は、S14で取得する撮像画像に対してエッジ抽出を行い、エッジ抽出を行った白線候補に対してHough変換を行うことで線分を抽出する。三次元位置特定部204は、抽出する線分を色情報と比較することで白線を抽出する。三次元位置特定部204は、撮像画像から抽出する左右の白線を延長することで左右の白線の交点の位置を算出し、算出する交点を消失点(図10のFOE参照)とする。なお、前述の基準点についても予め同様にして算出しておく構成とすればよい。続いて、三次元位置特定部204は、この消失点を基準点と比較し、基準点に対する消失点の変化量から自車前方の縦断勾配を特定する。変化量からの縦断勾配の特定については、予め変化量と縦断勾配との対応関係をHCU20の不揮発性メモリに格納しておき、この対応関係を参照することで特定可能とすればよい。
{First, the three-dimensional position identification unit 204 extracts a line segment by performing edge extraction on the captured image acquired in S14 and performing Hough transform on the white line candidate from which edge extraction has been performed. The three-dimensional position identification unit 204 extracts a white line by comparing the extracted line segment with color information. The three-dimensional position specifying unit 204 calculates the position of the intersection of the left and right white lines by extending the left and right white lines extracted from the captured image, and sets the calculated intersection as a vanishing point (see FOE in FIG. 10). The above-mentioned reference point may be calculated in advance in the same manner. Subsequently, the three-dimensional position identification unit 204 compares the vanishing point with the reference point, and identifies the longitudinal gradient in front of the vehicle from the amount of change of the vanishing point with respect to the reference point. Regarding the specification of the vertical gradient from the change amount, the correspondence between the change amount and the vertical gradient may be stored in a nonvolatile memory of the HCU 20 in advance, and the specification may be made by referring to this correspondence.
そして、三次元位置特定部204は、S1で取得する対象物の距離と、特定する自車前方の縦断勾配とをもとに、自車から対象物までこの縦断勾配で一定と仮定した上で三角関数を用いることで、自車の車両位置と対象地点との高度の差分から対象物のz座標を特定する。また、三次元位置特定部204は、画像認識によって車高といった対象物自体の高さも特定することが好ましい。
Then, based on the distance of the target acquired in S1 and the vertical gradient in front of the own vehicle to be specified, the three-dimensional position specifying unit 204 assumes that the vertical gradient from the own vehicle to the target is constant at this vertical gradient. By using a trigonometric function, the z coordinate of the object is specified from the difference between the vehicle position of the own vehicle and the altitude of the target point. Further, it is preferable that the three-dimensional position specifying unit 204 also specifies the height of the target object such as the vehicle height by image recognition.
ステップS17では、三次元位置特定部204が、S2で算出する対象物の(x,y)座標と、S16で推定する対象物のz座標とを合わせて、自車に対する対象物の三次元位置、つまり(x,y,z)座標を特定する。ステップS18では、表示生成部207が、S17で特定する三次元位置に合わせて、強調表示マークの表示画像の画像データを生成する。ステップS19では、表示生成部207が、S18で生成する画像データをHUD230に出力することで、自車に対する対象物の三次元位置に合わせて対象物を強調する強調表示マークの虚像を前景に重畳表示させる。
In step S17, the three-dimensional position identification unit 204 combines the (x, y) coordinates of the target calculated in S2 and the z coordinates of the target estimated in S16, and determines the three-dimensional position of the target with respect to the own vehicle. That is, the (x, y, z) coordinates are specified. In step S18, the display generation unit 207 generates image data of the display image of the highlighting mark in accordance with the three-dimensional position specified in S17. In step S19, the display generation unit 207 outputs the image data generated in S18 to the HUD 230, thereby superimposing a virtual image of a highlighting mark that emphasizes the target object on the foreground in accordance with the three-dimensional position of the target object with respect to the own vehicle. Display.
一例として、表示生成部207は、S17で特定する三次元位置及び対象物の高さと、視点位置特定部205で特定する視点位置と、予めHCU20の不揮発性メモリに格納されている投影領域の設定位置とをもとに、図11に示すような対象物(図11のVa参照)を囲う枠状の強調表示マークの虚像(図11のHl参照)を前景に重畳表示させる。図11は、重畳表示の一例を示す図であって、Viが投影領域を示している。なお、対象物を囲う形状であれば、枠状以外の形状であってもよく、一部を欠いている形状であってもよい。
As an example, the display generation unit 207 sets the three-dimensional position and the height of the target object specified in S17, the viewpoint position specified by the viewpoint position specification unit 205, and the setting of the projection area stored in the nonvolatile memory of the HCU 20 in advance. Based on the position, a virtual image (see Hl in FIG. 11) of a frame-shaped highlighting mark surrounding the object (see Va in FIG. 11) as shown in FIG. 11 is superimposed and displayed on the foreground. FIG. 11 is a diagram illustrating an example of superimposed display, in which Vi indicates a projection area. As long as the shape surrounds the object, the shape may be a shape other than the frame shape, or may be a shape lacking a part.
ステップS20では、虚像表示制御関連処理の終了タイミングであった場合(S20でYES)には、虚像表示制御関連処理を終了する。一方、虚像表示制御関連処理の終了タイミングでなかった場合(S20でNO)には、S1に戻って処理を繰り返す。虚像表示制御関連処理の終了タイミングの一例としては、自車のパワースイッチがオフになった場合,HUD230の機能がオフになった場合等がある。
In step S20, if it is the end timing of the virtual image display control-related processing (YES in S20), the virtual image display control-related processing ends. On the other hand, if it is not the end timing of the virtual image display control-related processing (NO in S20), the process returns to S1 and repeats the processing. Examples of the end timing of the virtual image display control-related processing include a case where the power switch of the own vehicle is turned off, a case where the function of the HUD 230 is turned off, and the like.
図4のフローチャートで説明した例では、S4~S14の処理では、前方カメラ41で撮像する撮像画像を用いないため、自車に対する対象物の高さは精度良く特定することが可能となる一方、対象物自体の高さを精度良く特定することが難しいため、対象物の下端に沿う形状の虚像を重畳表示させる。これによれば、対象物を強調する表示を、高さ方向のずれを抑えつつ行うことが可能になる。また、S15~S19の処理では、前方カメラ41で撮像する撮像画像も用いるため、自車に対する対象物の高さを精度良く特定することが可能となるとともに、対象物自体の高さも精度良く特定することが可能となる。よって、対象物を囲う形状の虚像を重畳表示させる。これによれば、対象物を強調する表示を、対象物を囲って対象物をドライバに認識しやすくしながらも、高さ方向のずれを抑えて行うことが可能になる。
In the example described with reference to the flowchart of FIG. 4, in the processing of S4 to S14, since the captured image captured by the front camera 41 is not used, the height of the target object with respect to the own vehicle can be specified with high accuracy. Since it is difficult to accurately specify the height of the object itself, a virtual image having a shape along the lower end of the object is superimposed and displayed. According to this, it is possible to perform the display for highlighting the target object while suppressing the displacement in the height direction. Further, in the processing of S15 to S19, since the image captured by the front camera 41 is also used, the height of the target object with respect to the own vehicle can be specified with high accuracy, and the height of the target object itself can be specified with high accuracy. It is possible to do. Therefore, a virtual image of a shape surrounding the object is superimposed and displayed. According to this, it is possible to perform the display for emphasizing the target object while surrounding the target object and making it easy for the driver to recognize the target object, while suppressing the displacement in the height direction.
<実施形態1のまとめ>
実施形態1の構成によれば、探査波センサであるミリ波レーダ42で検出する、自車に対する対象物の距離及び方位角に加え、地点別の高さの情報を含む高精度地図も用いて、自車に対する対象物の三次元位置を特定するので、自車に対する対象物の高さをより精度良く特定することが可能になる。また、この三次元位置に合わせて、対象物を強調する虚像を重畳表示させるので、対象物を強調する虚像が、意図した位置から高さ方向に大幅にずれることを抑えることが可能になる。その結果、HUD230によって自車の前景中の対象物を強調する虚像を重畳表示させる際に、ドライバによるその対象物の誤認識及びドライバの不信感を低減させることが可能になる。 <Summary ofEmbodiment 1>
According to the configuration of the first embodiment, in addition to the distance and the azimuth of the target object with respect to the own vehicle, which is detected by the millimeter-wave radar 42 serving as the exploration wave sensor, a high-precision map including height information for each point is used. Since the three-dimensional position of the object with respect to the own vehicle is specified, the height of the object with respect to the own vehicle can be specified with higher accuracy. In addition, since the virtual image for emphasizing the object is superimposed and displayed in accordance with the three-dimensional position, it is possible to suppress the virtual image for emphasizing the object from being significantly shifted from the intended position in the height direction. As a result, when the HUD 230 superimposes and displays a virtual image that emphasizes an object in the foreground of the own vehicle, it is possible to reduce erroneous recognition of the object by the driver and distrust of the driver.
実施形態1の構成によれば、探査波センサであるミリ波レーダ42で検出する、自車に対する対象物の距離及び方位角に加え、地点別の高さの情報を含む高精度地図も用いて、自車に対する対象物の三次元位置を特定するので、自車に対する対象物の高さをより精度良く特定することが可能になる。また、この三次元位置に合わせて、対象物を強調する虚像を重畳表示させるので、対象物を強調する虚像が、意図した位置から高さ方向に大幅にずれることを抑えることが可能になる。その結果、HUD230によって自車の前景中の対象物を強調する虚像を重畳表示させる際に、ドライバによるその対象物の誤認識及びドライバの不信感を低減させることが可能になる。 <Summary of
According to the configuration of the first embodiment, in addition to the distance and the azimuth of the target object with respect to the own vehicle, which is detected by the millimeter-
また、実施形態1の構成によれば、対象物が自車から所定距離未満の近距離の場合には、この近距離において自車に対する対象物の高さを精度良く特定することが可能な、前方カメラ41で撮像する撮像画像を用いることで、自車に対する対象物の高さを特に精度良く特定することが可能になる。一方、対象物が自車から所定距離以上の遠距離の場合には、この距離において自車に対する対象物の高さを精度良く特定することが困難な、前方カメラ41で撮像する撮像画像を用いず、高精度地図を用いることで、自車に対する対象物の高さを精度良く特定することが可能になる。
Further, according to the configuration of the first embodiment, when the target object is at a short distance less than a predetermined distance from the own vehicle, it is possible to accurately specify the height of the target object with respect to the own vehicle at this short distance. By using the image captured by the front camera 41, the height of the target object with respect to the own vehicle can be specified particularly accurately. On the other hand, when the target object is far from the own vehicle by a predetermined distance or more, an image picked up by the front camera 41 which is difficult to accurately determine the height of the target object with respect to the own vehicle at this distance is used. Instead, by using the high-accuracy map, it is possible to accurately specify the height of the target object with respect to the own vehicle.
さらに、実施形態1の構成によれば、対象地点が高精度地図の地図データ点にあたらない場合であっても、この対象地点にとっての横断勾配の基準となる勾配基準点を探索し、探索する勾配基準点における横断勾配と、この勾配基準点と対象地点との距離とを用いて、対象地点の高さを特定することが可能になる。よって、対象地点が高精度地図の地図データ点にあたらない場合であっても、自車に対する対象物の高さをより精度良く特定することが可能になる。
Further, according to the configuration of the first embodiment, even when the target point does not correspond to the map data point of the high-precision map, the gradient reference point serving as the reference of the cross slope for the target point is searched for and searched. The height of the target point can be specified using the cross slope at the gradient reference point and the distance between the gradient reference point and the target point. Therefore, even when the target point does not correspond to the map data point of the high-precision map, the height of the target object with respect to the own vehicle can be specified with higher accuracy.
他にも、実施形態1の構成によれば、不可視領域推定部206が不可視領域を推定し、表示生成部207が不可視領域を避けて強調表示マークの虚像を重畳表示させるので、対象物のうちの前方勾配で隠れていない部分から大きく離れた位置に対象物を強調する虚像を重畳表示させずに済む。よって、ドライバによるその対象物の誤認識及びドライバの不信感をさらに低減させることが可能になる。
In addition, according to the configuration of the first embodiment, the invisible area estimation unit 206 estimates the invisible area, and the display generation unit 207 superimposes and displays the virtual image of the highlighting mark while avoiding the invisible area. It is not necessary to superimpose and display a virtual image that emphasizes the object at a position far away from a portion that is not hidden by the forward gradient of the object. Therefore, it is possible to further reduce erroneous recognition of the target object by the driver and distrust of the driver.
(実施形態2)
実施形態1では、対象物が自車から所定距離未満の近距離の場合には、対象物の三次元位置の特定に前方カメラ41で撮像する撮像画像を用いるが高精度地図は用いない一方、対象物が自車から所定距離以上の遠距離の場合には、対象物の三次元位置の特定に高精度地図は用いるが前方カメラ41で撮像する撮像画像を用いない構成を示したが、必ずしもこれに限らない。 (Embodiment 2)
In the first embodiment, when the target object is at a short distance less than a predetermined distance from the own vehicle, the captured image captured by thefront camera 41 is used to specify the three-dimensional position of the target object, but the high-precision map is not used. In the case where the target object is a predetermined distance or more from the own vehicle, a configuration in which a high-accuracy map is used to specify the three-dimensional position of the target object but does not use a captured image captured by the front camera 41 has been described. Not limited to this.
実施形態1では、対象物が自車から所定距離未満の近距離の場合には、対象物の三次元位置の特定に前方カメラ41で撮像する撮像画像を用いるが高精度地図は用いない一方、対象物が自車から所定距離以上の遠距離の場合には、対象物の三次元位置の特定に高精度地図は用いるが前方カメラ41で撮像する撮像画像を用いない構成を示したが、必ずしもこれに限らない。 (Embodiment 2)
In the first embodiment, when the target object is at a short distance less than a predetermined distance from the own vehicle, the captured image captured by the
例えば、対象物が近距離の場合であっても遠距離の場合であっても高精度地図を用いて自車に対する対象物の高さを特定する構成としてもよい。この場合、例えば前方カメラ41で撮像する撮像画像は、対象物が近距離の場合において対象物自体の高さを特定し、対象物を囲う形状の虚像を重畳表示させるのに用いる構成としてもよい。さらに、この場合、実施形態1と同様にして不可視領域推定部206が不可視領域を、高精度地図を用いて推定し、不可視領域を避けて強調表示マークの虚像を重畳表示させる構成としてもよい。具体例としては、図12に示すように、対象物(図12のVa参照)を囲う枠状の強調表示マーク(図12のHl参照)の不可視領域にあたる部分をマスク処理して、対象物の可視領域中の部分を囲うように重畳表示させればよい。図12は、重畳表示の一例を示す図であって、Viが投影領域を示している。
For example, a configuration may be used in which the height of an object with respect to the own vehicle is specified using a high-accuracy map whether the object is at a short distance or a long distance. In this case, for example, the image captured by the front camera 41 may be configured to specify the height of the object itself when the object is at a short distance and to superimpose and display a virtual image of the shape surrounding the object. . Further, in this case, as in the first embodiment, the invisible area estimating unit 206 may estimate the invisible area using a high-accuracy map, and superimpose and display a virtual image of the highlighting mark while avoiding the invisible area. As a specific example, as shown in FIG. 12, a portion corresponding to an invisible area of a frame-shaped highlighting mark (see Hl in FIG. 12) surrounding the object (see Va in FIG. 12) is subjected to a mask process, and What is necessary is just to superimpose and display so that the part in a visible region may be enclosed. FIG. 12 is a diagram illustrating an example of the superimposed display, in which Vi indicates a projection area.
以上の構成であっても、探査波センサであるミリ波レーダ42で検出する、自車に対する対象物の距離及び方位角に加え、地点別の高さの情報を含む高精度地図も用いて、自車に対する対象物の三次元位置を特定するので、自車に対する対象物の高さをより精度良く特定することが可能になる。よって、HUD230によって自車の前景中の対象物を強調する虚像を重畳表示させる際に、ドライバによるその対象物の誤認識及びドライバの不信感を低減させることが可能になる。
Even with the above configuration, in addition to the distance and azimuth of the target object with respect to the vehicle detected by the millimeter-wave radar 42 that is a search wave sensor, using a high-accuracy map including height information for each point, Since the three-dimensional position of the target object with respect to the own vehicle is specified, the height of the target object with respect to the own vehicle can be specified with higher accuracy. Therefore, when a virtual image that emphasizes an object in the foreground of the own vehicle is superimposed and displayed by the HUD 230, it is possible to reduce erroneous recognition of the object and distrust of the driver by the driver.
また、対象物が近距離の場合であっても遠距離の場合であっても高精度地図を用いて自車に対する対象物の高さを特定する一方、対象地点が地図データ点にあたらない場合に、前方カメラ41で撮像する撮像画像を用いて自車に対する対象物の高さを特定する構成としてもよい。この構成において、対象地点が地図データ点にあたらない場合であって、対象物が近距離の場合に、前方カメラ41で撮像する撮像画像を用いて自車に対する対象物の高さを特定する一方、対象地点が地図データ点にあたらない場合であって、対象物が遠距離の場合に、S7~S8の処理と同様にして自車に対する対象物の高さを特定する構成とすることが好ましい。
In addition, whether the object is located at a short distance or a long distance, the height of the object with respect to the own vehicle is specified using a high-precision map, but the target point does not correspond to a map data point. Alternatively, the height of the target object with respect to the own vehicle may be specified using the image captured by the front camera 41. In this configuration, when the target point does not correspond to the map data point and the target object is at a short distance, the height of the target object with respect to the own vehicle is specified using the image captured by the front camera 41. When the target point does not correspond to the map data point and the target object is at a long distance, it is preferable to specify the height of the target object with respect to the own vehicle in the same manner as the processing of S7 to S8. .
これによれば、対象地点が地図データ点にあたらない場合であって、対象物の距離が近距離の場合には、この近距離において自車に対する対象物の高さを精度良く特定することが可能な、前方カメラ41で撮像する撮像画像を用いることで、勾配基準点を探索する処理負荷を抑えつつも自車に対する対象物の高さを精度良く特定することが可能になる。一方、対象地点が地図データ点にあたらない場合であって、対象物の距離が遠距離の場合には、この距離において自車に対する対象物の高さを精度良く特定することが困難な、前方カメラ41で撮像する撮像画像を用いず、勾配基準点を探索して自車に対する対象物の高さを精度良く特定することが可能になる。
According to this, when the target point does not correspond to the map data point and the distance of the target object is short, it is possible to accurately specify the height of the target object with respect to the own vehicle at this short distance. By using the captured image captured by the front camera 41, it is possible to accurately specify the height of the target object with respect to the own vehicle while suppressing the processing load for searching for the gradient reference point. On the other hand, when the target point does not correspond to the map data point and the distance of the target object is long, it is difficult to accurately specify the height of the target object with respect to the own vehicle at this distance. Instead of using the image captured by the camera 41, it is possible to search for the gradient reference point and accurately specify the height of the target object with respect to the own vehicle.
(実施形態3)
実施形態1では、前方カメラ41で撮像する撮像画像を用いて自車に対する対象物の高さを特定するか否かに応じて、対象物を囲う形状の虚像を重畳表示させるか対象物の下端に沿う形状の虚像を重畳表示させるかを切り替える構成を示したが、必ずしもこれに限らない。例えば、前方カメラ41で撮像する撮像画像を用いて自車に対する対象物の高さを特定するか否かにかかわらず、同様の形状の虚像を重畳表示させる構成としてもよい。 (Embodiment 3)
In the first embodiment, depending on whether the height of the target object with respect to the own vehicle is specified using the image captured by thefront camera 41, a virtual image of a shape surrounding the target object is superimposed or displayed at the lower end of the target object. Although the configuration for switching whether to superimpose and display the virtual image having the shape along the line has been described, the present invention is not necessarily limited to this. For example, a virtual image of the same shape may be superimposed and displayed regardless of whether or not the height of the target object with respect to the own vehicle is specified using the captured image captured by the front camera 41.
実施形態1では、前方カメラ41で撮像する撮像画像を用いて自車に対する対象物の高さを特定するか否かに応じて、対象物を囲う形状の虚像を重畳表示させるか対象物の下端に沿う形状の虚像を重畳表示させるかを切り替える構成を示したが、必ずしもこれに限らない。例えば、前方カメラ41で撮像する撮像画像を用いて自車に対する対象物の高さを特定するか否かにかかわらず、同様の形状の虚像を重畳表示させる構成としてもよい。 (Embodiment 3)
In the first embodiment, depending on whether the height of the target object with respect to the own vehicle is specified using the image captured by the
(実施形態4)
実施形態1では、不可視領域推定部206が不可視領域を推定し、表示生成部207が不可視領域を避けて強調表示マークの虚像を重畳表示させる構成を示したが、必ずしもこれに限らない。例えば、HCU20に不可視領域推定部206を備えず、不可視領域を推定しない構成としてもよい。 (Embodiment 4)
In the first embodiment, the configuration is described in which the invisiblearea estimation unit 206 estimates the invisible area and the display generation unit 207 superimposes and displays the virtual image of the highlighting mark while avoiding the invisible area. However, the present invention is not limited to this. For example, the HCU 20 may not include the invisible area estimation unit 206 and may not be configured to estimate the invisible area.
実施形態1では、不可視領域推定部206が不可視領域を推定し、表示生成部207が不可視領域を避けて強調表示マークの虚像を重畳表示させる構成を示したが、必ずしもこれに限らない。例えば、HCU20に不可視領域推定部206を備えず、不可視領域を推定しない構成としてもよい。 (Embodiment 4)
In the first embodiment, the configuration is described in which the invisible
(実施形態5)
実施形態1では、DSM22で検出した視点位置の情報から視点位置特定部205が自車の車両位置を基準とするドライバの視点位置を特定する構成を示したが、必ずしもこれに限らない。例えば、予めHCU20の不揮発性メモリに格納される設定位置を、自車の車両位置を基準とするドライバの視点位置として用いる構成としてもよい。この設定位置は、車種別に設定される仮想の点であって、通常の運転状態におけるドライバの目の位置を代表する点である。設定位置は、例えばシーティングリファレンスポイントの直上635mmの高さに設定される構成とすればよい。 (Embodiment 5)
In the first embodiment, the configuration is described in which the viewpointposition specifying unit 205 specifies the viewpoint position of the driver based on the vehicle position of the own vehicle based on the information on the viewpoint position detected by the DSM 22, but is not limited thereto. For example, a configuration may be used in which a set position stored in advance in the nonvolatile memory of the HCU 20 is used as a driver's viewpoint position with reference to the vehicle position of the own vehicle. This set position is a virtual point set for each vehicle type, and is a point representing the position of the driver's eyes in a normal driving state. The setting position may be set to, for example, a height of 635 mm immediately above the seating reference point.
実施形態1では、DSM22で検出した視点位置の情報から視点位置特定部205が自車の車両位置を基準とするドライバの視点位置を特定する構成を示したが、必ずしもこれに限らない。例えば、予めHCU20の不揮発性メモリに格納される設定位置を、自車の車両位置を基準とするドライバの視点位置として用いる構成としてもよい。この設定位置は、車種別に設定される仮想の点であって、通常の運転状態におけるドライバの目の位置を代表する点である。設定位置は、例えばシーティングリファレンスポイントの直上635mmの高さに設定される構成とすればよい。 (Embodiment 5)
In the first embodiment, the configuration is described in which the viewpoint
他にも、設定位置をリクライニング角度,スライド位置といった運転席のシート位置別に予め格納しておくことで、シート位置に応じた設定位置を選択して、自車の車両位置を基準とするドライバの視点位置を特定する構成としてもよい。
In addition, by storing the set position in advance for each driver's seat position such as a reclining angle and a slide position, a set position corresponding to the seat position is selected, and the driver's position based on the vehicle position of the own vehicle is selected. It is good also as composition which specifies a viewpoint position.
(実施形態6)
実施形態1では、高精度地図取得部203が地図データを取得する地図DB32がADASロケータ3に備えられている構成を示したが、必ずしもこれに限らない。例えば、地図DB32は、ADASロケータ3以外に備えられる構成としてもよい。 (Embodiment 6)
In the first embodiment, the configuration in which themap DB 32 from which the high-accuracy map acquisition unit 203 acquires the map data is provided in the ADAS locator 3 has been described. However, the configuration is not limited to this. For example, the map DB 32 may be provided in a configuration other than the ADAS locator 3.
実施形態1では、高精度地図取得部203が地図データを取得する地図DB32がADASロケータ3に備えられている構成を示したが、必ずしもこれに限らない。例えば、地図DB32は、ADASロケータ3以外に備えられる構成としてもよい。 (Embodiment 6)
In the first embodiment, the configuration in which the
(実施形態7)
実施形態1では、自車が自動運転機能を有している場合に適用した例を挙げて説明を行ったが、必ずしもこれに限らない。例えば、自車が自動運転機能を有していない構成であってもよい。この場合には、自車のドライバが注意すべき対象物を強調する虚像を重畳表示させることで、この対象物に対する注意喚起を行う構成等とすればよい。 (Embodiment 7)
In the first embodiment, an example has been described in which the present invention is applied to a case where the own vehicle has an automatic driving function, but the present invention is not necessarily limited to this. For example, the configuration may be such that the own vehicle does not have an automatic driving function. In this case, a configuration may be adopted in which the driver of the host vehicle alerts the target object by superimposing and displaying a virtual image that emphasizes the target object.
実施形態1では、自車が自動運転機能を有している場合に適用した例を挙げて説明を行ったが、必ずしもこれに限らない。例えば、自車が自動運転機能を有していない構成であってもよい。この場合には、自車のドライバが注意すべき対象物を強調する虚像を重畳表示させることで、この対象物に対する注意喚起を行う構成等とすればよい。 (Embodiment 7)
In the first embodiment, an example has been described in which the present invention is applied to a case where the own vehicle has an automatic driving function, but the present invention is not necessarily limited to this. For example, the configuration may be such that the own vehicle does not have an automatic driving function. In this case, a configuration may be adopted in which the driver of the host vehicle alerts the target object by superimposing and displaying a virtual image that emphasizes the target object.
(実施形態8)
実施形態1では、HUD230での表示の制御に関する機能をHUD230とは別体のHCU20が担う構成を示したが、必ずしもこれに限らない。例えば、HUD230での表示の制御に関する機能をHUD230に設けられる制御回路が担う構成としてもよいし、コンビネーションメータに設けられる制御回路等が担う構成としてもよい。 (Embodiment 8)
In the first embodiment, the configuration is described in which theHCU 20 separate from the HUD 230 has a function related to display control in the HUD 230, but the present invention is not limited to this. For example, a configuration may be adopted in which a control circuit provided in the HUD 230 has a function related to display control in the HUD 230, or a configuration in which a control circuit or the like provided in the combination meter has a function.
実施形態1では、HUD230での表示の制御に関する機能をHUD230とは別体のHCU20が担う構成を示したが、必ずしもこれに限らない。例えば、HUD230での表示の制御に関する機能をHUD230に設けられる制御回路が担う構成としてもよいし、コンビネーションメータに設けられる制御回路等が担う構成としてもよい。 (Embodiment 8)
In the first embodiment, the configuration is described in which the
本開示に記載の制御部及びその手法は、コンピュータプログラムにより具体化された一つ乃至は複数の機能を実行するようにプログラムされたプロセッサ及びメモリーを構成することによって提供された専用コンピュータにより、実現されてもよい。あるいは、本開示に記載の制御部及びその手法は、一つ以上の専用ハードウエア論理回路によってプロセッサを構成することによって提供された専用コンピュータにより、実現されてもよい。もしくは、本開示に記載の制御部及びその手法は、一つ乃至は複数の機能を実行するようにプログラムされたプロセッサ及びメモリーと一つ以上のハードウエア論理回路によって構成されたプロセッサとの組み合わせにより構成された一つ以上の専用コンピュータにより、実現されてもよい。また、コンピュータプログラムは、コンピュータにより実行されるインストラクションとして、コンピュータ読み取り可能な非遷移有形記録媒体に記憶されていてもよい。
The control unit and the technique according to the present disclosure are realized by a dedicated computer provided by configuring a processor and a memory programmed to execute one or a plurality of functions embodied by a computer program. May be done. Alternatively, the control unit and the technique described in the present disclosure may be implemented by a dedicated computer provided by configuring a processor with one or more dedicated hardware logic circuits. Alternatively, the control unit and the method described in the present disclosure may be implemented by a combination of a processor and a memory programmed to perform one or more functions and a processor configured with one or more hardware logic circuits. It may be realized by one or more dedicated computers configured. Further, the computer program may be stored in a computer-readable non-transitional tangible recording medium as instructions to be executed by a computer.
ここで、この出願に記載されるフローチャート、あるいは、フローチャートの処理は、複数のセクション(あるいはステップと言及される)から構成され、各セクションは、たとえば、S1と表現される。さらに、各セクションは、複数のサブセクションに分割されることができる、一方、複数のセクションが合わさって一つのセクションにすることも可能である。さらに、このように構成される各セクションは、デバイス、モジュール、ミーンズとして言及されることができる。
Here, the flowchart described in this application or the processing of the flowchart is composed of a plurality of sections (or referred to as steps), and each section is expressed as, for example, S1. Further, each section can be divided into multiple subsections, while multiple sections can be combined into one section. Further, each section so configured can be referred to as a device, module, means.
本開示は、実施例に準拠して記述されたが、本開示は当該実施例や構造に限定されるものではないと理解される。本開示は、様々な変形例や均等範囲内の変形をも包含する。加えて、様々な組み合わせや形態、さらには、それらに一要素のみ、それ以上、あるいはそれ以下、を含む他の組み合わせや形態をも、本開示の範疇や思想範囲に入るものである。
Although the present disclosure has been described based on the embodiments, it is understood that the present disclosure is not limited to the embodiments and the structure. The present disclosure also encompasses various modifications and variations within an equivalent range. In addition, various combinations and forms, and other combinations and forms including only one element, more or less, are also included in the scope and concept of the present disclosure.
Claims (10)
- 車両で用いられ、投影部材(10)に表示画像を投影することによって前記車両の前景中の対象物を強調する虚像を重畳表示させるヘッドアップディスプレイ(230)を制御する車両用表示制御装置であって、
探査波を送信し、その探査波が前記対象物で反射される反射波を受信することで前記車両に対する前記対象物の距離及び方位を検出する探査波センサ(42)で検出する、その距離及び方位を、取得する距離方位取得部(201)と、
地点別の高さの情報を含む高精度地図を取得する高精度地図取得部(203)と、
前記距離方位取得部で取得する前記距離及び前記方位と、前記高精度地図取得部で取得する前記高精度地図とを用いて、前記車両に対する前記対象物の三次元位置を特定する三次元位置特定部(204)と、
前記三次元位置特定部で特定する、前記車両に対する前記対象物の三次元位置に合わせて、前記対象物を強調する虚像を重畳表示させる表示制御部(207)とを備える車両用表示制御装置。 A display control device for a vehicle, which is used in a vehicle and controls a head-up display (230) for superimposing and displaying a virtual image that emphasizes an object in a foreground of the vehicle by projecting a display image on a projection member (10). hand,
A probe wave is transmitted, and the probe wave is detected by a probe wave sensor (42) that detects a distance and an azimuth of the target object with respect to the vehicle by receiving a reflected wave reflected by the target object. A distance direction obtaining unit (201) for obtaining the direction;
A high-accuracy map acquisition unit (203) for acquiring a high-accuracy map including height information for each point;
Using the distance and the azimuth obtained by the distance azimuth obtaining unit and the high-accuracy map obtained by the high-accuracy map obtaining unit, a three-dimensional position specification that specifies a three-dimensional position of the object with respect to the vehicle. Part (204),
A display control unit for a vehicle, comprising: a display control unit (207) configured to superimpose and display a virtual image emphasizing the object in accordance with a three-dimensional position of the object with respect to the vehicle specified by the three-dimensional position specifying unit. - 前記車両の前方を撮像する撮像装置(41)で撮像する撮像画像を取得する撮像画像取得部(200)をさらに備え、
前記三次元位置特定部は、前記車両に対する前記対象物の距離が所定距離以上の場合には、前記距離方位取得部で取得する前記距離及び前記方位と、前記高精度地図取得部で取得する前記高精度地図とを用いて、前記車両に対する前記対象物の三次元位置を特定し、
前記三次元位置特定部は、前記車両に対する前記対象物の距離が所定距離未満の場合には、前記距離方位取得部で取得する前記距離及び前記方位と、前記撮像画像取得部で取得する前記撮像画像を用いて、前記車両に対する前記対象物の三次元位置を特定する請求項1に記載の車両用表示制御装置。 A captured image acquisition unit (200) that acquires a captured image captured by an imaging device (41) that captures an image of the front of the vehicle;
The three-dimensional position identification unit, when the distance of the object to the vehicle is a predetermined distance or more, the distance and the azimuth acquired by the distance azimuth acquisition unit, and the high accuracy map acquisition unit to acquire the Using a high-precision map, identify the three-dimensional position of the object with respect to the vehicle,
When the distance of the object to the vehicle is less than a predetermined distance, the three-dimensional position specifying unit may obtain the distance and the azimuth obtained by the distance azimuth obtaining unit and the imaging obtained by the captured image obtaining unit. The vehicle display control device according to claim 1, wherein a three-dimensional position of the object with respect to the vehicle is specified using an image. - 前記表示制御部は、前記三次元位置特定部で、前記撮像画像を用いて、前記車両に対する前記対象物の三次元位置を特定する場合には、前記対象物を強調する虚像として、前記前景中の前記対象物を囲う形状の虚像を重畳表示させ、
前記表示制御部は、前記撮像画像を用いず、前記高精度地図取得部で取得する前記高精度地図を用いて、前記車両に対する前記対象物の三次元位置を特定する場合には、前記対象物を強調する虚像として、前記前景中の前記対象物の下端に沿う形状の虚像を重畳表示させる請求項2に記載の車両用表示制御装置。 The display control unit, when the three-dimensional position specifying unit specifies the three-dimensional position of the target object with respect to the vehicle using the captured image, as a virtual image that emphasizes the target object in the foreground. Superimposing and displaying a virtual image of a shape surrounding the object,
The display control unit does not use the captured image but uses the high-accuracy map acquired by the high-accuracy map acquisition unit to specify a three-dimensional position of the object with respect to the vehicle. The display control device for a vehicle according to claim 2, wherein a virtual image having a shape along a lower end of the object in the foreground is superimposed and displayed as a virtual image that emphasizes the image. - 前記三次元位置特定部は、前記撮像画像を用いて、前記車両に対する前記対象物の三次元位置を特定する場合、前記車両に対する前記対象物の高さ方向の位置については、前記撮像画像から画像認識によって検出する消失点の、平坦路での消失点からの変化量をもとに特定する請求項2又は3に記載の車両用表示制御装置。 The three-dimensional position identification unit, using the captured image, when identifying the three-dimensional position of the object with respect to the vehicle, for the position in the height direction of the object with respect to the vehicle, the image from the captured image The display control device for a vehicle according to claim 2, wherein the vanishing point detected by the recognition is specified based on an amount of change from the vanishing point on a flat road.
- 前記三次元位置特定部は、前記高精度地図取得部で取得する前記高精度地図を用いて、前記車両に対する前記対象物の三次元位置を特定する場合、前記車両に対する前記対象物の高さ方向の位置については、前記高精度地図上の前記車両が位置する地点の高さと、前記車両に対する前記対象物の前記距離及び前記方位が示す地点の高さとの差分から特定する請求項1~4のいずれか1項に記載の車両用表示制御装置。 The three-dimensional position identification unit, using the high-accuracy map acquired by the high-accuracy map acquisition unit, when identifying the three-dimensional position of the object with respect to the vehicle, the height direction of the object with respect to the vehicle The position of (1) is specified from the difference between the height of the point where the vehicle is located on the high-precision map and the height of the point indicated by the distance and the azimuth of the object with respect to the vehicle. The display control device for a vehicle according to claim 1.
- 前記高精度地図は、少なくとも道路上の複数地点についての高さ及び横断勾配の情報を含むものであり、
前記三次元位置特定部は、前記高精度地図取得部で取得する前記高精度地図を用いて、前記車両に対する前記対象物の三次元位置を特定する場合であって、前記車両に対する前記対象物の距離及び方位が示す地点である対象地点が、前記高精度地図に前記高さ及び前記横断勾配の情報が含まれている地点にあたらない場合には、前記高精度地図に前記高さ及び前記横断勾配の情報が含まれている地点のうちから、前記対象地点にとっての横断勾配の基準となる勾配基準点を探索し、前記勾配基準点での横断勾配と、前記勾配基準点と前記対象地点との距離とから、前記車両に対する前記対象物の前記距離及び前記方位が示す地点の高さを特定する請求項5に記載の車両用表示制御装置。 The high-precision map includes at least height and cross slope information for a plurality of points on the road,
The three-dimensional position identification unit, when using the high-accuracy map acquired by the high-accuracy map acquisition unit, to identify the three-dimensional position of the object with respect to the vehicle, the object of the object with respect to the vehicle If the target point, which is a point indicated by the distance and the azimuth, does not correspond to the point where the information of the height and the traverse gradient is included in the high-accuracy map, the height and the traverse are displayed in the high-accuracy map. From the points that include the information on the gradient, search for a gradient reference point that serves as a reference for the traverse gradient for the target point, the traverse gradient at the gradient reference point, and the gradient reference point and the target point. The display control device for a vehicle according to claim 5, wherein the height of a point indicated by the distance and the azimuth of the object with respect to the vehicle is specified from the distance of the vehicle. - 前記車両のドライバの視点位置を特定する視点位置特定部(205)と、
前記高精度地図に前記高さの情報が含まれている地点のうち、前記車両と前記対象物との間の地点の前記高さの情報と、前記視点位置特定部で特定する前記視点位置とをもとに、路面の勾配変化によって前記ドライバから不可視となる不可視領域を推定する不可視領域推定部(206)とをさらに備え、
前記表示制御部は、前記不可視領域推定部で推定する前記不可視領域を避けて、前記対象物を強調する虚像を重畳表示させる請求項1~6のいずれか1項に記載の車両用表示制御装置。 A viewpoint position specifying unit (205) for specifying a viewpoint position of the driver of the vehicle;
Among the points where the information on the height is included in the high-precision map, the information on the height of the point between the vehicle and the object, and the viewpoint position specified by the viewpoint position specifying unit. And an invisible area estimating unit (206) for estimating an invisible area that is invisible to the driver due to a change in road gradient based on
The display control device for a vehicle according to any one of claims 1 to 6, wherein the display control unit superimposes and displays a virtual image that emphasizes the object, avoiding the invisible region estimated by the invisible region estimation unit. . - 車両で用いられ、投影部材(10)に表示画像を投影することによって前記車両の前景中の対象物を強調する虚像を重畳表示させるヘッドアップディスプレイ(230)を制御する車両用表示制御方法であって、
探査波を送信し、その探査波が前記対象物で反射される反射波を受信することで前記車両に対する前記対象物の距離及び方位を検出する探査波センサ(42)で検出する、その距離及び方位を、取得し、
地点別の高さの情報を含む高精度地図を取得し、
取得する前記距離及び方位と、取得する前記高精度地図とを用いて、前記車両に対する前記対象物の三次元位置を特定し、
特定する、前記車両に対する前記対象物の三次元位置に合わせて、前記対象物を強調する虚像を重畳表示させる車両用表示制御方法。 A display control method for a vehicle, which is used in a vehicle and controls a head-up display (230) that superimposes and displays a virtual image that emphasizes an object in a foreground of the vehicle by projecting a display image on a projection member (10). hand,
A probe wave is transmitted, and the probe wave is detected by a probe wave sensor (42) that detects a distance and an azimuth of the target object with respect to the vehicle by receiving a reflected wave reflected by the target object. To get the bearing,
Obtain a high-precision map that contains height information for each point,
Using the acquired distance and orientation and the acquired high-accuracy map, specify the three-dimensional position of the object with respect to the vehicle,
A display control method for a vehicle in which a virtual image that emphasizes the object is superimposed and displayed in accordance with a specified three-dimensional position of the object with respect to the vehicle. - 車両で用いられ、投影部材(10)に表示画像を投影することによって前記車両の前景中の対象物を強調する虚像を重畳表示させるヘッドアップディスプレイ(230)を制御する車両用表示制御装置として機能させるための制御プログラムであって、
コンピュータを、
探査波を送信し、その探査波が前記対象物で反射される反射波を受信することで前記車両に対する前記対象物の距離及び方位を検出する探査波センサ(42)で検出する、その距離及び方位を、取得する距離方位取得部(201)と、
地点別の高さの情報を含む高精度地図を取得する高精度地図取得部(203)と、
前記距離方位取得部で取得する前記距離及び前記方位と、前記高精度地図取得部で取得する前記高精度地図とを用いて、前記車両に対する前記対象物の三次元位置を特定する三次元位置特定部(204)と、
前記三次元位置特定部で特定する、前記車両に対する前記対象物の三次元位置に合わせて、前記対象物を強調する虚像を重畳表示させる表示制御部(207)として機能させるための制御プログラム。 Function as a vehicle display control device for controlling a head-up display (230) that is used in a vehicle and that superimposes and displays a virtual image that emphasizes an object in the foreground of the vehicle by projecting a display image on a projection member (10). A control program for causing
Computer
A probe wave is transmitted, and the probe wave is detected by a probe wave sensor (42) that detects a distance and an azimuth of the target object with respect to the vehicle by receiving a reflected wave reflected by the target object. A distance direction obtaining unit (201) for obtaining the direction;
A high-accuracy map acquisition unit (203) for acquiring a high-accuracy map including height information for each point;
Using the distance and the azimuth obtained by the distance azimuth obtaining unit and the high-accuracy map obtained by the high-accuracy map obtaining unit, a three-dimensional position specification for specifying a three-dimensional position of the object with respect to the vehicle. Part (204),
A control program for functioning as a display control unit (207) for superimposing and displaying a virtual image emphasizing the object in accordance with a three-dimensional position of the object with respect to the vehicle specified by the three-dimensional position specifying unit. - コンピュータによって実施される命令を備える持続的有形コンピュータ読み取り媒体であって、当該命令は、車両で用いられ、投影部材(10)に表示画像を投影することによって前記車両の前景中の対象物を強調する虚像を重畳表示させるヘッドアップディスプレイ(230)を制御するものであり、当該命令は、
探査波を送信し、その探査波が前記対象物で反射される反射波を受信することで前記車両に対する前記対象物の距離及び方位を検出する探査波センサ(42)で検出する、その距離及び方位を、取得することと、
地点別の高さの情報を含む高精度地図を取得することと、
取得する前記距離及び方位と、取得する前記高精度地図とを用いて、前記車両に対する前記対象物の三次元位置を特定し、
特定する、前記車両に対する前記対象物の三次元位置に合わせて、前記対象物を強調する虚像を重畳表示させることと、
を備える持続的有形コンピュータ読み取り媒体。
A persistent tangible computer-readable medium comprising computer-implemented instructions for use in a vehicle for highlighting an object in a foreground of the vehicle by projecting a display image on a projection member (10). And a head-up display (230) for superimposing and displaying a virtual image to be displayed.
A probe wave is transmitted, and the probe wave is detected by a probe wave sensor (42) that detects a distance and an azimuth of the target object with respect to the vehicle by receiving a reflected wave reflected by the target object. Obtaining the bearing,
Obtaining a high-accuracy map containing height information for each point,
Using the acquired distance and orientation and the acquired high-accuracy map, specify the three-dimensional position of the object with respect to the vehicle,
Identifying, in accordance with the three-dimensional position of the object with respect to the vehicle, superimposing and displaying a virtual image that emphasizes the object,
A persistent tangible computer readable medium comprising:
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
JP2018-139469 | 2018-07-25 | ||
JP2018139469A JP6981377B2 (en) | 2018-07-25 | 2018-07-25 | Vehicle display control device, vehicle display control method, and control program |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2020021842A1 true WO2020021842A1 (en) | 2020-01-30 |
Family
ID=69182211
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/JP2019/021029 WO2020021842A1 (en) | 2018-07-25 | 2019-05-28 | Vehicle display control device, vehicle display control method, control program, and persistent tangible computer-readable medium |
Country Status (2)
Country | Link |
---|---|
JP (1) | JP6981377B2 (en) |
WO (1) | WO2020021842A1 (en) |
Cited By (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN113341942A (en) * | 2020-02-14 | 2021-09-03 | 本田技研工业株式会社 | Vehicle device and acceleration correction method |
WO2021250876A1 (en) * | 2020-06-12 | 2021-12-16 | 三菱電機株式会社 | Road shape estimation device, road shape estimation method, and road shape estimation program |
EP3967538A1 (en) * | 2020-09-09 | 2022-03-16 | Volkswagen Ag | Method for depicting a virtual element |
Families Citing this family (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2021191648A (en) | 2020-06-05 | 2021-12-16 | トヨタ自動車株式会社 | Position estimation device and position estimation method |
JP7279689B2 (en) | 2020-06-16 | 2023-05-23 | トヨタ自動車株式会社 | Data processing device and data processing system |
JP7322855B2 (en) | 2020-10-23 | 2023-08-08 | トヨタ自動車株式会社 | Road surface information creation device and vehicle control system |
JP7314902B2 (en) | 2020-10-29 | 2023-07-26 | トヨタ自動車株式会社 | VEHICLE CONTROL METHOD AND CONTROL DEVICE |
JP2022155162A (en) | 2021-03-30 | 2022-10-13 | ミネベアミツミ株式会社 | Pump system and electronic equipment |
JP7501463B2 (en) | 2021-07-15 | 2024-06-18 | トヨタ自動車株式会社 | Vehicle display control device, display method, and display program |
Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2001331787A (en) * | 2000-05-19 | 2001-11-30 | Toyota Central Res & Dev Lab Inc | Road shape estimating device |
JP2005258941A (en) * | 2004-03-12 | 2005-09-22 | Toyota Central Res & Dev Lab Inc | Device for detecting obstacle |
JP2005343351A (en) * | 2004-06-04 | 2005-12-15 | Olympus Corp | Drive supporting device |
JP2010120617A (en) * | 2008-11-24 | 2010-06-03 | Denso Corp | Image forming device, head-up display, and vehicle display |
JP2014092935A (en) * | 2012-11-02 | 2014-05-19 | Aisin Aw Co Ltd | Preceding vehicle recognition system, preceding vehicle recognition method and preceding vehicle recognition program |
-
2018
- 2018-07-25 JP JP2018139469A patent/JP6981377B2/en active Active
-
2019
- 2019-05-28 WO PCT/JP2019/021029 patent/WO2020021842A1/en active Application Filing
Patent Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2001331787A (en) * | 2000-05-19 | 2001-11-30 | Toyota Central Res & Dev Lab Inc | Road shape estimating device |
JP2005258941A (en) * | 2004-03-12 | 2005-09-22 | Toyota Central Res & Dev Lab Inc | Device for detecting obstacle |
JP2005343351A (en) * | 2004-06-04 | 2005-12-15 | Olympus Corp | Drive supporting device |
JP2010120617A (en) * | 2008-11-24 | 2010-06-03 | Denso Corp | Image forming device, head-up display, and vehicle display |
JP2014092935A (en) * | 2012-11-02 | 2014-05-19 | Aisin Aw Co Ltd | Preceding vehicle recognition system, preceding vehicle recognition method and preceding vehicle recognition program |
Cited By (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN113341942A (en) * | 2020-02-14 | 2021-09-03 | 本田技研工业株式会社 | Vehicle device and acceleration correction method |
WO2021250876A1 (en) * | 2020-06-12 | 2021-12-16 | 三菱電機株式会社 | Road shape estimation device, road shape estimation method, and road shape estimation program |
EP3967538A1 (en) * | 2020-09-09 | 2022-03-16 | Volkswagen Ag | Method for depicting a virtual element |
Also Published As
Publication number | Publication date |
---|---|
JP2020016541A (en) | 2020-01-30 |
JP6981377B2 (en) | 2021-12-15 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
WO2020021842A1 (en) | Vehicle display control device, vehicle display control method, control program, and persistent tangible computer-readable medium | |
US10293748B2 (en) | Information presentation system | |
JP7052786B2 (en) | Display control device and display control program | |
US11982539B2 (en) | Display system, display control device, and display control program product | |
WO2020162109A1 (en) | Display control device, display control program, and persistent physical computer-readable medium | |
US11325470B2 (en) | Method, device and computer-readable storage medium with instructions for controlling a display of an augmented-reality head-up display device for a transportation vehicle | |
US20210089020A1 (en) | Remote parking system | |
WO2020189238A1 (en) | Vehicular display control device, vehicular display control method, and vehicular display control program | |
WO2020070996A1 (en) | Travel lane estimation device, travel lane estimation method, control program, and computer-readable non-temporary storage medium | |
JP7024619B2 (en) | Display control device for mobile body, display control method for mobile body, and control program | |
JP7400242B2 (en) | Vehicle display control device and vehicle display control method | |
JP6855759B2 (en) | Self-driving vehicle control system | |
CN110888432B (en) | Display system, display control method, and storage medium | |
JP7416114B2 (en) | Display control device and display control program | |
JP7302311B2 (en) | Vehicle display control device, vehicle display control method, vehicle display control program | |
CN114764429A (en) | Route data conversion method, non-transitory computer-readable storage medium, and route data conversion device | |
JP2020138609A (en) | Display control device for vehicle, display control method for vehicle, and display control program for vehicle | |
JP7487713B2 (en) | Vehicle display control device, vehicle display device, vehicle display control method and program | |
JP7172730B2 (en) | Vehicle display control device, vehicle display control method, vehicle display control program | |
JP2021028587A (en) | In-vehicle display control device | |
JP2023050414A (en) | Preceding vehicle determination system | |
CN118043236A (en) | Parking support device and parking support method | |
JP2020091148A (en) | Display controller and display control program |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 19840788 Country of ref document: EP Kind code of ref document: A1 |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
122 | Ep: pct application non-entry in european phase |
Ref document number: 19840788 Country of ref document: EP Kind code of ref document: A1 |