WO2024111324A1 - Display control device - Google Patents

Display control device Download PDF

Info

Publication number
WO2024111324A1
WO2024111324A1 PCT/JP2023/038361 JP2023038361W WO2024111324A1 WO 2024111324 A1 WO2024111324 A1 WO 2024111324A1 JP 2023038361 W JP2023038361 W JP 2023038361W WO 2024111324 A1 WO2024111324 A1 WO 2024111324A1
Authority
WO
WIPO (PCT)
Prior art keywords
vehicle
virtual viewpoint
display
control device
display control
Prior art date
Application number
PCT/JP2023/038361
Other languages
French (fr)
Japanese (ja)
Inventor
英昭 古瀬
康平 前島
Original Assignee
株式会社デンソー
トヨタ自動車株式会社
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 株式会社デンソー, トヨタ自動車株式会社 filed Critical 株式会社デンソー
Publication of WO2024111324A1 publication Critical patent/WO2024111324A1/en

Links

Images

Classifications

    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60K—ARRANGEMENT OR MOUNTING OF PROPULSION UNITS OR OF TRANSMISSIONS IN VEHICLES; ARRANGEMENT OR MOUNTING OF PLURAL DIVERSE PRIME-MOVERS IN VEHICLES; AUXILIARY DRIVES FOR VEHICLES; INSTRUMENTATION OR DASHBOARDS FOR VEHICLES; ARRANGEMENTS IN CONNECTION WITH COOLING, AIR INTAKE, GAS EXHAUST OR FUEL SUPPLY OF PROPULSION UNITS IN VEHICLES
    • B60K35/00—Instruments specially adapted for vehicles; Arrangement of instruments in or on vehicles
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R1/00—Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
    • B60R1/20—Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
    • B60R1/22—Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles for viewing an area outside the vehicle, e.g. the exterior of the vehicle
    • B60R1/28—Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles for viewing an area outside the vehicle, e.g. the exterior of the vehicle with an adjustable field of view
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00—Television systems
    • H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast

Definitions

  • This disclosure relates to a display control device mounted on a vehicle.
  • a display control device that synthesizes images captured by one or more cameras mounted on a vehicle to create an image viewed from a virtual viewpoint in the sky, and displays the image on a display.
  • the display control device described in Patent Document 1 changes the display range of an image to be displayed on a display in accordance with the vehicle speed. Specifically, the display control device described in Patent Document 1 calculates the viewpoint position, angle of view, and gaze direction to determine a virtual viewpoint so that the faster the vehicle speed becomes, the more scenery farther away from the vehicle is included in the display range.
  • the display area of a display mounted on a vehicle is limited. Within that area, the points that the driver should pay attention to in the relative positions of the vehicle and other vehicles, i.e., the points that the driver wants to display as an image, will differ depending on the situation.
  • the display control device described in Patent Document 1 does not display in a way that allows the driver to easily check the points that the driver should pay attention to in the relative positions of the vehicle and other vehicles.
  • the purpose of this disclosure is to provide a display control device that can provide appropriate display according to the relative positions of the vehicle and other vehicles.
  • a display control device is mounted on a vehicle together with one or more cameras that capture images of the surroundings of the vehicle, an other vehicle position detection unit that detects the positions of other vehicles present around the vehicle, and a display that displays the images.
  • the display control device includes a virtual viewpoint determination unit and an image creation unit.
  • the virtual viewpoint determination unit calculates the viewpoint position, angle of view, and gaze direction of the images to be displayed on the display so that at least a part of the vehicle and at least a part of the other vehicles are displayed, and determines a virtual viewpoint.
  • the image creation unit creates an image of the display range viewed from the virtual viewpoint by synthesizing images captured by the cameras.
  • the virtual viewpoint determination unit determines the virtual viewpoint to be at a higher position so that the display range is narrowed and enlarged as the position of the other vehicle detected by the other vehicle position detection unit is closer to the host vehicle, and determines the virtual viewpoint to be at a lower position so that the display range is widened and reduced as the position of the other vehicle detected by the other vehicle position detection unit is farther from the host vehicle.
  • the display control device narrows the display range and enlarges the display of the host vehicle and the other vehicle as the vehicle-to-vehicle distance (hereinafter simply referred to as "vehicle distance") becomes closer.
  • vehicle distance the vehicle-to-vehicle distance
  • the display control device widens the display range and reduces the display of the host vehicle and the other vehicle as the vehicle-to-vehicle distance becomes greater. This allows the driver to simultaneously see the other vehicle and its surrounding environment and easily confirm the dangers around it in addition to the other vehicle.
  • This display control device is therefore capable of providing appropriate display so that the driver can easily see areas requiring attention depending on the positional relationship between the host vehicle and the other vehicle.
  • FIG. 1 is a block diagram of a vehicle system including a display control device according to a first embodiment.
  • FIG. 2 is a diagram showing a situation in which another vehicle is present far away from the host vehicle.
  • FIG. 3 is a diagram showing an example of an image displayed on a display in the situation shown in FIG. 2 .
  • FIG. 2 is a diagram showing a situation in which another vehicle is present near the host vehicle;
  • FIG. 5 is a diagram showing an example of an image displayed on a display in the situation shown in FIG. 4 .
  • 4 is a flowchart illustrating an example of a control process executed by a vehicle system including the display control device.
  • 7 is a flowchart illustrating in detail the step of "detection of another vehicle" in FIG. 6.
  • FIG. 7 is a flowchart illustrating in detail the "determination of an optimal virtual viewpoint" in FIG. 6.
  • 11 is a graph showing the relationship between the inter-vehicle distance and the height of the virtual viewpoint.
  • 10 is a flowchart illustrating in detail “determination of an optimal virtual viewpoint” in a display control device according to a second embodiment.
  • a display control device 10 of this embodiment constitutes a vehicle system 100 together with a camera 20, sensors 21 to 27, various control devices 30, 40, and a display 50 mounted on a vehicle. These devices and equipment are connected by an in-vehicle LAN using CAN communication, a wire harness, wireless communication, or the like.
  • CAN is an abbreviation for Controller Area Network
  • LAN is an abbreviation for Local Area Network.
  • One or more cameras 20 are mounted on the vehicle and capture images of the surroundings of the vehicle.
  • four cameras 20 are mounted on the vehicle, and the four cameras 20 capture images of the front, right side, left side, and rear of the vehicle, respectively.
  • the number of cameras 20 mounted on the vehicle is not limited to four, and may be one or more.
  • a digital camera using a solid-state imaging element such as a CCD or CMOS is used as the camera 20.
  • CCD Charge Coupled Device
  • CMOS Complementary Metal Oxide Semiconductor.
  • the vehicle is equipped with a distance measuring device that detects the distance between the vehicle and objects, including other vehicles, around the vehicle, such as a LiDAR sensor 21, a radar sensor 22, a sonar sensor 23, and a vehicle-to-vehicle communication device 28.
  • a LiDAR sensor 21 LiDAR is an abbreviation for Light Detection and Ranging, or Laser Imaging Detection and Ranging.
  • the vehicle needs to be equipped with at least one of the above distance measuring devices, and may also be equipped with other distance measuring devices.
  • the LiDAR sensor 21 measures scattered light in response to laser irradiation and detects the distance to an object, etc.
  • the radar sensor 22 detects the distance to an object, etc. by emitting radio waves such as millimeter waves and measuring the radio waves reflected by the object (i.e., the reflected waves).
  • the sonar sensor 23 detects the distance to an object, etc. by using ultrasonic waves.
  • the vehicle-to-vehicle communication device 28 wirelessly transmits and receives information such as the position and speed of the vehicle and other vehicles.
  • the image data captured by the camera 20 and the information detected by each of the distance measuring devices are input to the other vehicle position detection unit 30.
  • the other vehicle position detection unit 30 is an electronic control device mainly composed of a computer having a processor, memory, etc.
  • the other vehicle position detection unit 30 detects the position of other vehicles present around the vehicle based on the image data input from the camera 20 or the information input from each distance measuring device. Specifically, the other vehicle position detection unit 30 can identify the position and direction, etc. of other vehicles present in front of, to the side of, or behind the vehicle.
  • the other vehicle position detection unit 30 may also be configured to identify the position and direction, etc. of the other vehicle present closest to the vehicle. Information on the position and direction, etc. of other vehicles identified by the other vehicle position detection unit 30 is output to the display control device 10.
  • the driving control device 40 is an electronic control device mainly composed of a computer having a processor, memory, etc.
  • the driving control device 40 receives vehicle speed information from, for example, a vehicle speed sensor 24 that detects the vehicle speed of the vehicle.
  • the driving control device 40 may also receive acceleration information from, for example, an acceleration sensor 25 that detects the acceleration of the vehicle.
  • the driving control device 40 may also receive angular velocity information from, for example, a yaw rate sensor 26 that detects the angular velocity around the vertical axis of the vehicle.
  • the driving control device 40 may also receive direction information from, for example, a direction sensor 27 that detects the direction of the vehicle.
  • the information including the vehicle speed of the vehicle input to the driving control device 40 is output to the display control device 10.
  • the display control device 10 is an electronic control device mainly composed of a computer having a processor, memory, etc.
  • the display control device 10 is equipped with a virtual viewpoint determination unit 11 and an image creation unit 12.
  • the display control device 10 functions as the virtual viewpoint determination unit 11 and the image creation unit 12 by the processor executing each program stored in the memory.
  • the display control device 10 receives inputs of image data captured by the camera 20, information such as the position and direction of other vehicles identified by the other vehicle position detection unit 30, and vehicle speed information of the vehicle itself from the driving control device 40. Based on this information, the display control device 10 creates and outputs an image to be displayed on the display 50 using the virtual viewpoint determination unit 11 and the image creation unit 12.
  • the virtual viewpoint determination unit 11 calculates the viewpoint position, angle of view, and gaze direction of the image to be displayed on the display 50 so that at least a part of the vehicle itself and at least a part of the other vehicle are displayed, and determines the virtual viewpoint.
  • the virtual viewpoint includes the viewpoint position, angle of view, and gaze direction.
  • the image creation unit 12 creates an image of the display range seen from a virtual viewpoint by synthesizing images captured by the cameras 20. Specifically, the image creation unit 12 identifies one or more cameras 20 that have captured images of other vehicles, and creates an image to be displayed on the display 50 by synthesizing the images from the identified cameras 20. The image data created by the image creation unit 12 is output to the display 50. Note that if the host vehicle is not sufficiently visible in the images captured by the cameras 20 that have captured the other vehicles, the image of the host vehicle may be displayed using CG. CG stands for computer graphics.
  • FIG. 2 shows a situation where there is another vehicle, such as an oncoming vehicle, far away from the host vehicle
  • FIG. 4 shows a situation where there is another vehicle, such as an oncoming vehicle, close to the host vehicle.
  • the virtual viewpoint (more specifically, the viewpoint position) is given the symbol VP
  • the host vehicle is given the symbol M1
  • the other vehicle is given the symbol M2.
  • the dashed lines extending from the virtual viewpoint VP indicate the angle of view and the direction of gaze.
  • the virtual viewpoint determination unit 11 determines the virtual viewpoint VP to be at a lower position so that the display range is expanded and reduced as the position of the other vehicle M2 detected by the other vehicle position detection unit 30 becomes farther from the host vehicle M1.
  • a low position means a relatively low vertical height from the ground.
  • the image creation unit 12 creates an image viewed from the virtual viewpoint VP, and outputs the image data to the display 50 for display.
  • FIG. 3 shows an example of an image displayed on the display 50 in the situation shown in FIG. 2.
  • the display control device 10 expands the display range and reduces the display of the host vehicle M1 and the other vehicle M2 as the distance between the host vehicle M1 and the other vehicle M2 increases.
  • This allows the display 50 to show a wide range of the surrounding environment in addition to the host vehicle M1 and the other vehicle M2. Therefore, the driver of the host vehicle M1 can visually check the surrounding environment including the other vehicle M2 and easily confirm dangers around the other vehicle M2 in addition to the other vehicle M2.
  • the distance between the host vehicle M1 and the other vehicle M2 is simply referred to as the "distance between the vehicles.”
  • the driver of the host vehicle M1 is simply referred to as the "driver.”
  • the virtual viewpoint determination unit 11 determines the virtual viewpoint VP to be at a higher position so that the display range is narrowed and enlarged as the position of the other vehicle M2 detected by the other vehicle position detection unit 30 is closer to the host vehicle M1.
  • a high position means a relatively high vertical height from the ground.
  • the image creation unit 12 creates an image viewed from the virtual viewpoint VP, and outputs the image data to the display 50 for display.
  • FIG. 5 shows an example of an image displayed on the display 50 in the situation shown in FIG. 4.
  • a part of the host vehicle M1 and a part of the other vehicle M2 are enlarged and displayed on the display 50. This allows the driver to simultaneously view the host vehicle M1 and the other vehicle M2, and easily check the risk of contact and the sense of distance.
  • FIG. 1 Note that while Figures 2 to 5 have described the control processing executed by the display control device 10 in relation to the host vehicle M1 and an oncoming vehicle, the control processing executed by the display control device 10 is the same whether the other vehicle M2 is a leading vehicle, a vehicle running alongside, a following vehicle, or a stopped vehicle.
  • an oncoming vehicle refers to the other vehicle M2 traveling from in front of the host vehicle M1 toward the host vehicle M1
  • "leading vehicle,” “parallel vehicle,” and “following vehicle” refer to the other vehicle M2 traveling in the same direction as the host vehicle M1, ahead of, to the side of, and behind the host vehicle M1, respectively.
  • the control process shown in the flowcharts of FIGS. 6 to 8 is started at the same time that the vehicle running switch is turned on.
  • the cruise control device 40 acquires the speed of the host vehicle from the vehicle speed sensor 24.
  • the other vehicle position detection unit 30 detects other vehicles present in the vicinity of the vehicle itself.
  • the process executed by the other vehicle position detection unit 30 in S20 is shown in detail in FIG. 7.
  • the other vehicle position detection unit 30 acquires the camera image captured by the camera 20, i.e., the image data.
  • the other vehicle position detection unit 30 uses image recognition technology to detect the position and direction, etc., of other vehicles contained in the camera image, and then in S25, determines the distance between the vehicles.
  • the other vehicle position detection unit 30 may acquire information input from a distance measurement sensor such as the LiDAR sensor 21, the radar sensor 22, or the sonar sensor 23, instead of or in addition to the video data captured by the camera 20. In this case, too, in S25, the other vehicle position detection unit 30 determines the inter-vehicle distance based on that information.
  • the other vehicle position detection unit 30 may acquire information input from other technologies, such as the vehicle-to-vehicle communication device 28, instead of or in addition to the video data captured by the camera 20 or the information input from the distance measurement sensor. In this case as well, in S25, the other vehicle position detection unit 30 determines the inter-vehicle distance based on that information.
  • the display control device 10 determines whether the vehicle speed of the vehicle is within a specified vehicle speed range based on the vehicle speed information obtained by the driving control device 40 from the vehicle speed sensor 24.
  • This specified vehicle speed range is stored in the memory of the display control device 10. For example, when the display control device 10 is used on a highway, this specified vehicle speed range is set to the vehicle speed range when the vehicle is generally traveling on the highway. Alternatively, for example, when the display control device 10 is used on an ordinary road, this specified vehicle speed range is set to the vehicle speed range when the vehicle is generally traveling on an ordinary road.
  • the process proceeds to S70.
  • the display control device 10 outputs the video data of the default viewpoint to the display 50 to display it.
  • the process proceeds to S40.
  • the display control device 10 determines whether the vehicle distance is within a specified distance range.
  • This specified vehicle speed range is stored in the memory of the display control device 10. For example, when the display control device 10 is used on a highway, this specified distance range is generally set to the vehicle distance at which the vehicle should pay attention to other vehicles when traveling on the highway. Also, for example, when the display control device 10 is used on an ordinary road, this specified vehicle speed range is generally set to the vehicle distance at which the vehicle should pay attention to other vehicles when traveling on an ordinary road.
  • the process proceeds to S70.
  • the display control device 10 outputs the video data of the default viewpoint to the display 50 to display it.
  • the process proceeds to S50.
  • the display control device 10 determines an optimal virtual viewpoint depending on the vehicle distance.
  • the process executed by the display control device 10 in S50 is shown in detail in FIG. 8.
  • the virtual viewpoint determination unit 11 provided in the display control device 10 determines the display range to be displayed on the display 50 based on the vehicle distance. Specifically, the virtual viewpoint determination unit 11 widens the display range as the vehicle distance increases, and reduces the display of the host vehicle and the other vehicle. On the other hand, the virtual viewpoint determination unit 11 narrows the display range as the vehicle distance decreases, and enlarges the display of the host vehicle and the other vehicle.
  • the virtual viewpoint determination unit 11 calculates the viewpoint position, angle of view, and viewing direction that will cause the display range to fit within the video display area, and determines the optimal virtual viewpoint. Specifically, the wider the display range, the lower the virtual viewpoint determination unit 11 determines the virtual viewpoint to be, and the narrower the display range, the higher the virtual viewpoint determination unit 11 determines the virtual viewpoint to be.
  • the image creation unit 12 of the display control device 10 creates an image of the display range as seen from the optimal virtual viewpoint determined in S53.
  • the display control device 10 outputs image data of the display range viewed from the optimal virtual viewpoint to the display 50, and causes the image to be displayed on the display 50.
  • the vehicle system 100 including the display control device 10 repeatedly executes the above-mentioned processes of S10 to S70 at a predetermined control cycle.
  • the horizontal axis indicates the distance between vehicles
  • the vertical axis indicates the height of the virtual viewpoint determined by the virtual viewpoint determination unit 11.
  • solid line A shows the relationship between the distance between vehicles and the virtual viewpoint when the distance between vehicles is decreasing over time.
  • Solid line B shows the relationship between the distance between vehicles and the virtual viewpoint when the distance between vehicles remains constant over time.
  • Solid line C shows the relationship between the distance between vehicles and the virtual viewpoint when the distance between vehicles is increasing over time.
  • the virtual viewpoint determination unit 11 determines the virtual viewpoint to be at a lower position as the distance between the vehicles increases. This allows the display control device 10 to expand the display range and display a reduced image of the vehicle's own vehicle and other vehicles as the distance between the vehicles increases. On the other hand, the virtual viewpoint determination unit 11 determines the virtual viewpoint to be at a higher position as the distance between the vehicles decreases. This allows the display control device 10 to narrow the display range and display a larger image of the vehicle's own vehicle and other vehicles as the distance between the vehicles decreases.
  • the virtual viewpoint determination unit 11 reduces the amount of change by which the virtual viewpoint is raised as the vehicle distance increases. This is because, when the virtual viewpoint is relatively low, it is possible to significantly change the display range even if the amount of change in the virtual viewpoint relative to the vehicle distance is small. On the other hand, the virtual viewpoint determination unit 11 increases the amount of change by which the virtual viewpoint is raised as the vehicle distance decreases. This is because, when the virtual viewpoint is relatively high, it is possible to appropriately change the display range by increasing the amount of change in the virtual viewpoint relative to the vehicle distance.
  • the virtual viewpoint determination unit 11 sets the position of the virtual viewpoint when the inter-vehicle distance shown by solid line A is decreasing, regardless of the inter-vehicle distance, higher than the position of the virtual viewpoint when the inter-vehicle distance shown by solid line B is constant. According to this, when the inter-vehicle distance is decreasing, that is, when the vehicle-to-vehicle distance approaches the other vehicle, the virtual viewpoint determination unit 11 sets the position of the virtual viewpoint relatively higher. This narrows the display range in the image on the display 50, and the vehicle-to-vehicle and the other vehicle are displayed in an enlarged manner. Therefore, the driver can simultaneously view the vehicle-to-vehicle and the other vehicle from the image, and can reliably confirm the risk of contact with the other vehicle approaching the vehicle-to-vehicle and the sense of distance.
  • the virtual viewpoint determination unit 11 lowers the position of the virtual viewpoint when the inter-vehicle distance is increasing, as shown by solid line C, for any inter-vehicle distance, compared to the position of the virtual viewpoint when the inter-vehicle distance is constant, as shown by solid line B.
  • the virtual viewpoint determination unit 11 lowers the position of the virtual viewpoint relatively when the inter-vehicle distance is increasing, that is, when the vehicle's own vehicle and the other vehicle are moving away from each other. This widens the display range in the image on the display 50, and the vehicle's own vehicle and the other vehicle are displayed in a reduced size. Therefore, the driver can simultaneously view the other vehicle and its surrounding environment through the image, and can reliably confirm not only the other vehicle moving away from the vehicle's own vehicle, but also any dangers in the surrounding area.
  • the virtual viewpoint determination unit 11 included in the display control device 10 narrows the display range as the position of the other vehicle detected by the other vehicle position detection unit 30 gets closer to the host vehicle, and determines the virtual viewpoint to be at a higher position so that at least a part of the host vehicle and at least a part of the other vehicle are displayed in an enlarged manner. Also, the virtual viewpoint determination unit 11 widens the display range as the position of the other vehicle detected by the other vehicle position detection unit 30 gets farther from the host vehicle, and determines the virtual viewpoint to be at a lower position so that at least a part of the host vehicle and at least a part of the other vehicle are displayed in a reduced manner.
  • this display control device 10 is capable of providing appropriate display so that the driver can easily see areas requiring attention depending on the relative positions of the vehicle and the other vehicle.
  • the other vehicle position detection unit 30 can identify the position and direction of another vehicle that is in front of, to the side of, or behind the vehicle.
  • the image creation unit 12 identifies one or more cameras 20 that have captured images of the other vehicle, and creates an image to be displayed on the display 50 by combining images from the identified cameras 20. According to this, the image creation unit 12 can synthesize images from one or more cameras 20 to create an image of the display range viewed from the virtual viewpoint determined by the virtual viewpoint determination unit 11, i.e., an image to be displayed on the display 50.
  • the virtual viewpoint determination unit 11 sets the position of the virtual viewpoint when the inter-vehicle distance tends to decrease over time to be higher than the position of the virtual viewpoint when the inter-vehicle distance tends to increase over time. According to this, when the inter-vehicle distance tends to decrease with the passage of time, i.e., when the host vehicle and the other vehicle approach each other, the virtual viewpoint determination unit 11 sets the position of the virtual viewpoint relatively high. This narrows the display range of the image on the display 50, and the host vehicle and the other vehicle are displayed in an enlarged manner. Therefore, the driver can simultaneously view the host vehicle and the other vehicle from the image, and can reliably confirm the risk of contact with the other vehicle approaching the host vehicle and the sense of distance.
  • the virtual viewpoint determination unit 11 sets the position of the virtual viewpoint relatively low. This widens the display range of the image on the display 50, and the vehicle and the other vehicle are displayed in a reduced size. This allows the driver to simultaneously view the other vehicle and its surrounding environment through the image, and to reliably confirm not only the other vehicle moving away from the vehicle, but also any dangers in the surrounding area.
  • the virtual viewpoint determination unit 11 increases the amount of change by which the virtual viewpoint is raised as the inter-vehicle distance decreases. According to this, when the vehicle distance is long and the virtual viewpoint is relatively low, the display range can be changed significantly even if the change in the virtual viewpoint with respect to the vehicle distance is small. On the other hand, when the vehicle distance is short and the virtual viewpoint is relatively high, the display range can be changed appropriately by increasing the change in the virtual viewpoint with respect to the vehicle distance. Therefore, this display control device 10 can perform appropriate display so that the driver can easily visually recognize areas that require attention according to the positional relationship between the vehicle and other vehicles.
  • Second Embodiment A second embodiment will be described.
  • a part of the control process executed by the vehicle system 100 in the first embodiment is changed, and the rest is the same as in the first embodiment, so only the parts that are different from the first embodiment will be described.
  • the process of S50 "Determining an optimal virtual viewpoint" in FIG. 6 referred to in the first embodiment is changed.
  • the process of S50 "Determining the optimal virtual viewpoint” in FIG. 6 is shown in detail in FIG. 10.
  • the virtual viewpoint determination unit 11 provided in the display control device 10 determines the display range to be displayed on the display 50 based on the inter-vehicle distance. Specifically, the virtual viewpoint determination unit 11 widens the display range as the inter-vehicle distance increases, and reduces the display of the host vehicle and the other vehicle. On the other hand, the virtual viewpoint determination unit 11 narrows the display range as the inter-vehicle distance decreases, and enlarges the display of the host vehicle and the other vehicle.
  • the virtual viewpoint determination unit 11 determines a virtual viewpoint from a group of virtual viewpoints prepared in advance according to the display range. That is, in the second embodiment, a plurality of virtual viewpoints corresponding to the display range are stored in advance in the memory of the display control device 10. Note that the plurality of virtual viewpoints are set to lower positions as the display range becomes wider, and are set to higher positions as the display range becomes narrower. The virtual viewpoint determination unit 11 selects and determines an appropriate virtual viewpoint according to the display range from among the plurality of virtual viewpoints, i.e., from among the group of virtual viewpoints.
  • the image creation unit 12 of the display control device 10 creates an image of the display range viewed from the optimal virtual viewpoint determined in S53.
  • control process described in the second embodiment can achieve the same effects as the first embodiment.
  • the vehicle is described as being equipped with a distance measuring device such as the LiDAR sensor 21, the radar sensor 22, the sonar sensor 23, the camera 20, the vehicle-to-vehicle communication device 28, etc., but this is not limited to this.
  • a distance measuring device such as the LiDAR sensor 21, the radar sensor 22, the sonar sensor 23, the camera 20, the vehicle-to-vehicle communication device 28, etc.
  • the other vehicle position detection unit 30 can detect the position of other vehicles present around the vehicle using the video data captured by the camera 20, the distance measuring device and the vehicle-to-vehicle communication device 28, etc., do not have to be equipped in the vehicle.
  • the present disclosure is not limited to the above-described embodiments, and can be modified as appropriate.
  • the above-described embodiments and parts thereof are not unrelated to each other, and can be combined as appropriate, except when the combination is clearly impossible.
  • the elements constituting the embodiments are not necessarily essential, except when it is specifically stated that they are essential or when it is clearly considered essential in principle.
  • the numbers, values, amounts, ranges, etc. of the components of the embodiments are mentioned, they are not limited to the specific numbers, except when it is specifically stated that they are essential or when it is clearly limited to a specific number in principle.
  • the shapes, positional relationships, etc. of the components are mentioned, they are not limited to the shapes, positional relationships, etc., except when it is specifically stated that they are essential or when it is clearly limited to a specific shape, positional relationship, etc. in principle.
  • control unit and the method described in the present disclosure may be realized by a dedicated computer provided by configuring a processor and memory programmed to execute one or more functions embodied in a computer program.
  • control unit and the method described in the present disclosure may be realized by a dedicated computer provided by configuring a processor with one or more dedicated hardware logic circuits.
  • control unit and the method described in the present disclosure may be realized by one or more dedicated computers configured by combining a processor and memory programmed to execute one or more functions with a processor configured with one or more hardware logic circuits.
  • the computer program may be stored in a computer-readable non-transient tangible recording medium as instructions executed by the computer.
  • the memory is a non-transient tangible recording medium.
  • a display control device is mounted on a vehicle together with one or more cameras (20) that capture images of the surroundings of the vehicle (M1), a vehicle position detection unit (30) that detects the positions of other vehicles (M2) that exist around the vehicle (M1), and a display (50) that displays the images, a virtual viewpoint determination unit (11) for calculating a viewpoint position, a field angle, and a gaze direction of an image to be displayed on the display and determining a virtual viewpoint (VP); an image creation unit (12) that creates an image of the display range viewed from the virtual viewpoint by synthesizing the images captured by the camera; The virtual viewpoint determination unit determining the virtual viewpoint to be at a higher position so that the display range is narrowed and enlarged as the position of the other vehicle detected by the other vehicle position detection unit approaches the host vehicle; A display control device configured to determine the virtual viewpoint to be at a lower position so that the display range is expanded and reduced as
  • the other vehicle position detection unit is capable of identifying a position and a direction of the other vehicle that is present in front of, behind, or to the side of the host vehicle;
  • the display control device described in the first aspect wherein the image creation unit identifies one or more of the cameras that captured the other vehicle, and creates an image to be displayed on the display by combining images from the identified cameras.
  • the virtual viewpoint determination unit sets the position of the virtual viewpoint when the inter-vehicle distance between the host vehicle and the other vehicle tends to decrease over time to be higher than the position of the virtual viewpoint when the inter-vehicle distance between the host vehicle and the other vehicle tends to increase over time.
  • the virtual viewpoint determination unit increases an amount of change by which the virtual viewpoint is elevated as the inter-vehicle distance between the host vehicle and the other vehicle becomes shorter.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Mechanical Engineering (AREA)
  • Signal Processing (AREA)
  • Chemical & Material Sciences (AREA)
  • Combustion & Propulsion (AREA)
  • Transportation (AREA)
  • Closed-Circuit Television Systems (AREA)

Abstract

A display control device (10) comprises a virtual viewpoint determination unit (11) and an image creation unit (12). The virtual viewpoint determination unit (11) calculates the viewpoint position, angle of view, and gaze direction of an image to be displayed on a display (50) such that at least part of a vehicle (M1) and at least part of another vehicle (M2) are displayed, and determines a virtual viewpoint (VP). The image creation unit (12) creates an image of a display range viewed from the virtual viewpoint (VP) by compositing images captured by a camera (20). The virtual viewpoint determination unit (11) determines the virtual viewpoint (VP) at a high position such that the display range is narrowed and displayed in expanded form as the position of the other vehicle (M2) detected by an other-vehicle position detection unit (30) becomes closer to the vehicle (M1). The virtual viewpoint determination unit (11) determines the virtual viewpoint (VP) at a low position such that the display range is widened and displayed in reduced form as the position of the other vehicle (M2) detected by the other-vehicle position detection unit (30) becomes farther from the vehicle (M1).

Description

表示制御装置Display Control Device 関連出願への相互参照CROSS-REFERENCE TO RELATED APPLICATIONS
　本出願は、２０２２年１１月２３日に出願された日本特許出願番号２０２２－１８６９７２号に基づくもので、ここにその記載内容が参照により組み入れられる。 This application is based on Japanese Patent Application No. 2022-186972, filed on November 23, 2022, the contents of which are incorporated herein by reference.
　本開示は、車両に搭載される表示制御装置に関するものである。 This disclosure relates to a display control device mounted on a vehicle.
　従来、車両に搭載された１つ以上のカメラで撮影された映像を合成して上空の仮想視点から見下ろした映像を作成し、ディスプレイに表示する表示制御装置が知られている。
　特許文献１に記載の表示制御装置は、自車両の車速に応じてディスプレイに表示する映像の表示範囲を変更するものである。具体的には、特許文献１に記載の表示制御装置は、自車両の速度が速くなるほど、自車両からより遠方の景色が表示範囲に含まれるように視点位置、画角および注視方向を計算し仮想視点を決定するものである。 2. Description of the Related Art A display control device is known that synthesizes images captured by one or more cameras mounted on a vehicle to create an image viewed from a virtual viewpoint in the sky, and displays the image on a display.
The display control device described in Patent Document 1 changes the display range of an image to be displayed on a display in accordance with the vehicle speed. Specifically, the display control device described in Patent Document 1 calculates the viewpoint position, angle of view, and gaze direction to determine a virtual viewpoint so that the faster the vehicle speed becomes, the more scenery farther away from the vehicle is included in the display range.
特許第６９９０２４８号公報Japanese Patent No. 6990248
　ところで、車両に搭載されるディスプレイの表示領域は限られている。その中で、自車両と他車両との位置関係において運転者が注意すべき箇所、即ち、映像として表示したい箇所は、状況に応じて異なるものとなる。しかしながら、特許文献１に記載の表示制御装置は、自車両と他車両との位置関係において注意すべき箇所を運転者が容易に確認できるような表示になっていない。 The display area of a display mounted on a vehicle is limited. Within that area, the points that the driver should pay attention to in the relative positions of the vehicle and other vehicles, i.e., the points that the driver wants to display as an image, will differ depending on the situation. However, the display control device described in Patent Document 1 does not display in a way that allows the driver to easily check the points that the driver should pay attention to in the relative positions of the vehicle and other vehicles.
　本開示は、自車両と他車両との位置関係に応じた適切な表示を行うことの可能な表示制御装置を提供することを目的とする。 The purpose of this disclosure is to provide a display control device that can provide appropriate display according to the relative positions of the vehicle and other vehicles.
　本開示の１つの観点によれば、表示制御装置は、自車両周辺を撮影する１つ以上のカメラ、自車両周辺に存在する他車両の位置を検知する他車両位置検知部、および映像を表示するディスプレイと共に車両に搭載される。表示制御装置は、仮想視点決定部と映像作成部を備える。仮想視点決定部は、自車両の少なくとも一部および他車両の少なくとも一部が表示されるように、ディスプレイに表示する映像の視点位置、画角および注視方向を計算し仮想視点を決定する。映像作成部は、カメラで撮影した映像を合成することで仮想視点から視た表示範囲の映像を作成する。
　そして、仮想視点決定部は、他車両位置検知部で検知される他車両の位置が自車両に近いほど表示範囲を狭めて拡大表示するよう仮想視点を高い位置に決定する。また、仮想視点決定部は、他車両位置検知部で検知される他車両の位置が自車両から遠いほど表示範囲を広げて縮小表示するよう仮想視点を低い位置に決定する。 According to one aspect of the present disclosure, a display control device is mounted on a vehicle together with one or more cameras that capture images of the surroundings of the vehicle, an other vehicle position detection unit that detects the positions of other vehicles present around the vehicle, and a display that displays the images. The display control device includes a virtual viewpoint determination unit and an image creation unit. The virtual viewpoint determination unit calculates the viewpoint position, angle of view, and gaze direction of the images to be displayed on the display so that at least a part of the vehicle and at least a part of the other vehicles are displayed, and determines a virtual viewpoint. The image creation unit creates an image of the display range viewed from the virtual viewpoint by synthesizing images captured by the cameras.
The virtual viewpoint determination unit determines the virtual viewpoint to be at a higher position so that the display range is narrowed and enlarged as the position of the other vehicle detected by the other vehicle position detection unit is closer to the host vehicle, and determines the virtual viewpoint to be at a lower position so that the display range is widened and reduced as the position of the other vehicle detected by the other vehicle position detection unit is farther from the host vehicle.
　これによれば、表示制御装置は、自車両と他車両との車間距離（以下、単に「車間距離」という）が近いほど表示範囲を狭めて自車両と他車両とを拡大表示する。そのため、自車両の運転者（以下、単に「運転者」という）は、自車両と他車両とを同時に視認し、接触の危険性や距離感を容易に確認できる。一方、表示制御装置は、車間距離が遠いほど表示範囲を広げて自車両と他車両とを縮小表示する。そのため、運転者は、他車両とその周囲の環境を同時に視認し、他車両に加えてその周囲の危険性を容易に確認できる。したがって、この表示制御装置は、自車両と他車両との位置関係に応じて注意すべき箇所を運転者が容易に視認できるよう適切な表示を行うことが可能である。 According to this, the display control device narrows the display range and enlarges the display of the host vehicle and the other vehicle as the vehicle-to-vehicle distance (hereinafter simply referred to as "vehicle distance") becomes closer. This allows the driver of the host vehicle (hereinafter simply referred to as "driver") to simultaneously see the host vehicle and the other vehicle and easily confirm the risk of contact and the sense of distance. On the other hand, the display control device widens the display range and reduces the display of the host vehicle and the other vehicle as the vehicle-to-vehicle distance becomes greater. This allows the driver to simultaneously see the other vehicle and its surrounding environment and easily confirm the dangers around it in addition to the other vehicle. This display control device is therefore capable of providing appropriate display so that the driver can easily see areas requiring attention depending on the positional relationship between the host vehicle and the other vehicle.
　なお、各構成要素等に付された括弧付きの参照符号は、その構成要素等と後述する実施形態に記載の具体的な構成要素等との対応関係の一例を示すものである。 The reference symbols in parentheses attached to each component indicate an example of the correspondence between the component and the specific components described in the embodiments described below.
第１実施形態に係る表示制御装置を含む車両システムのブロック図である。1 is a block diagram of a vehicle system including a display control device according to a first embodiment. 自車両の遠方に他車両が存在する状況を示す図である。FIG. 2 is a diagram showing a situation in which another vehicle is present far away from the host vehicle. 図２に示した状況でディスプレイに表示される映像の一例を示す図である。FIG. 3 is a diagram showing an example of an image displayed on a display in the situation shown in FIG. 2 . 自車両の近傍に他車両が存在する状況を示す図である。FIG. 2 is a diagram showing a situation in which another vehicle is present near the host vehicle; 図４に示した状況でディスプレイに表示される映像の一例を示す図である。FIG. 5 is a diagram showing an example of an image displayed on a display in the situation shown in FIG. 4 . 表示制御装置を含む車両システムが実行する制御処理の一例を示すフローチャートである。4 is a flowchart illustrating an example of a control process executed by a vehicle system including the display control device. 図６中の「他車両の検知」を詳細に説明したフローチャートである。7 is a flowchart illustrating in detail the step of "detection of another vehicle" in FIG. 6. 図６中の「最適な仮想視点の決定」を詳細に説明したフローチャートである。7 is a flowchart illustrating in detail the "determination of an optimal virtual viewpoint" in FIG. 6. 車間距離と仮想視点の高さとの関係を示すグラフである。11 is a graph showing the relationship between the inter-vehicle distance and the height of the virtual viewpoint. 第２実施形態に係る表示制御装置において「最適な仮想視点の決定」を詳細に説明したフローチャートである。10 is a flowchart illustrating in detail “determination of an optimal virtual viewpoint” in a display control device according to a second embodiment.
　以下、本開示の実施形態について図面を参照しつつ説明する。なお、以下の各実施形態相互において、互いに同一もしくは均等である部分には、同一符号を付し、その説明を省略する。 Below, embodiments of the present disclosure will be described with reference to the drawings. Note that in the following embodiments, parts that are identical or equivalent to each other will be given the same reference numerals and their description will be omitted.
　（第１実施形態）
　第１実施形態について図面を参照しつつ説明する。図１に示すように、本実施形態の表示制御装置１０は、車両に搭載されるカメラ２０、センサ類２１～２７、各種制御装置３０、４０およびディスプレイ５０と共に車両システム１００を構成している。それらの機器や装置は、例えばＣＡＮ通信などによる車内ＬＡＮ、ワイヤーハーネスまたは無線通信等により接続されている。なお、ＣＡＮはController Area Networkの略であり、ＬＡＮはLocal Area Networkの略である。 First Embodiment
The first embodiment will be described with reference to the drawings. As shown in Fig. 1, a display control device 10 of this embodiment constitutes a vehicle system 100 together with a camera 20, sensors 21 to 27, various control devices 30, 40, and a display 50 mounted on a vehicle. These devices and equipment are connected by an in-vehicle LAN using CAN communication, a wire harness, wireless communication, or the like. CAN is an abbreviation for Controller Area Network, and LAN is an abbreviation for Local Area Network.
　カメラ２０は、車両に１つ以上搭載され、自車両の周辺を撮影する。本実施形態では、車両に例えば４つのカメラ２０が搭載されており、その４つのカメラ２０がそれぞれ車両の前方、右側方、左側方、後方を撮影する。なお、車両に搭載されるカメラ２０の数は４つに限らず、１つ以上であればよい。カメラ２０は、例えば、ＣＣＤまたはＣＭＯＳ等の固体撮像素子を利用したデジタルカメラが用いられる。なお、ＣＣＤはCharge Coupled Deviceの略であり、ＣＭＯＳはComplementary Metal Oxide Semiconductorの略である。 One or more cameras 20 are mounted on the vehicle and capture images of the surroundings of the vehicle. In this embodiment, for example, four cameras 20 are mounted on the vehicle, and the four cameras 20 capture images of the front, right side, left side, and rear of the vehicle, respectively. Note that the number of cameras 20 mounted on the vehicle is not limited to four, and may be one or more. For example, a digital camera using a solid-state imaging element such as a CCD or CMOS is used as the camera 20. Note that CCD is an abbreviation for Charge Coupled Device, and CMOS is an abbreviation for Complementary Metal Oxide Semiconductor.
　車両には、自車両周辺の他車両を含む物体と自車両との距離を検出する測距装置として、例えば、ＬｉＤＡＲセンサ２１、レーダーセンサ２２、ソナーセンサ２３、車車間通信装置２８などが搭載されている。なお、ＬｉＤＡＲは、Light Detection and Ranging、またはLaser Imaging Detection and Rangingの略である。なお、車両には、上記の測距装置のうち少なくとも１つが搭載されていればよく、また、それ以外の測距装置が搭載されていてもよい。 The vehicle is equipped with a distance measuring device that detects the distance between the vehicle and objects, including other vehicles, around the vehicle, such as a LiDAR sensor 21, a radar sensor 22, a sonar sensor 23, and a vehicle-to-vehicle communication device 28. Note that LiDAR is an abbreviation for Light Detection and Ranging, or Laser Imaging Detection and Ranging. Note that the vehicle needs to be equipped with at least one of the above distance measuring devices, and may also be equipped with other distance measuring devices.
　ＬｉＤＡＲセンサ２１は、レーザー照射に対する散乱光を測定し、物体までの距離等を検出するものである。レーダーセンサ２２は、ミリ波などの電波を放射し、物体によって反射された電波（即ち、反射波）を測定することにより、物体までの距離等を検出するものである。ソナーセンサ２３は、超音波により物体までの距離等を検出するものである。車車間通信装置２８は、自車両と他車両とが互いの位置や速度といった情報を無線で送受信するものである。 The LiDAR sensor 21 measures scattered light in response to laser irradiation and detects the distance to an object, etc. The radar sensor 22 detects the distance to an object, etc. by emitting radio waves such as millimeter waves and measuring the radio waves reflected by the object (i.e., the reflected waves). The sonar sensor 23 detects the distance to an object, etc. by using ultrasonic waves. The vehicle-to-vehicle communication device 28 wirelessly transmits and receives information such as the position and speed of the vehicle and other vehicles.
　カメラ２０が撮影した映像データ、および、上記各測距装置が検出した情報は、他車両位置検知部３０に入力される。他車両位置検知部３０は、プロセッサおよびメモリー等を有するコンピュータを主体に構成された電子制御装置である。他車両位置検知部３０は、カメラ２０から入力される映像データまたは各測距装置から入力される情報に基づき、自車両の周辺に存在する他車両の位置を検知する。具体的には、他車両位置検知部３０は、自車両の前方、側方または後方に存在する他車両の位置および方向等を特定することが可能である。なお、他車両位置検知部３０は、自車両の最も近くに存在する他車両の位置および方向等を特定するように構成してもよい。他車両位置検知部３０が特定した他車両の位置および方向等の情報は、表示制御装置１０に出力される。 The image data captured by the camera 20 and the information detected by each of the distance measuring devices are input to the other vehicle position detection unit 30. The other vehicle position detection unit 30 is an electronic control device mainly composed of a computer having a processor, memory, etc. The other vehicle position detection unit 30 detects the position of other vehicles present around the vehicle based on the image data input from the camera 20 or the information input from each distance measuring device. Specifically, the other vehicle position detection unit 30 can identify the position and direction, etc. of other vehicles present in front of, to the side of, or behind the vehicle. The other vehicle position detection unit 30 may also be configured to identify the position and direction, etc. of the other vehicle present closest to the vehicle. Information on the position and direction, etc. of other vehicles identified by the other vehicle position detection unit 30 is output to the display control device 10.
　走行制御装置４０は、プロセッサおよびメモリー等を有するコンピュータを主体に構成された電子制御装置である。走行制御装置４０には、例えば、自車両の車速を検出する車速センサ２４から車速情報が入力される。また、走行制御装置４０には、例えば、自車両の加速度を検出する加速度センサ２５から加速度情報が入力されてもよい。また、走行制御装置４０には、例えば、自車両の鉛直軸周りの角速度を検出するヨーレートセンサ２６から角速度情報が入力されてもよい。また、走行制御装置４０には、例えば、自車両の向きを検出する方位センサ２７から方位情報が入力されてもよい。走行制御装置４０に入力される自車両の車速を含む情報は、表示制御装置１０に出力される。 The driving control device 40 is an electronic control device mainly composed of a computer having a processor, memory, etc. The driving control device 40 receives vehicle speed information from, for example, a vehicle speed sensor 24 that detects the vehicle speed of the vehicle. The driving control device 40 may also receive acceleration information from, for example, an acceleration sensor 25 that detects the acceleration of the vehicle. The driving control device 40 may also receive angular velocity information from, for example, a yaw rate sensor 26 that detects the angular velocity around the vertical axis of the vehicle. The driving control device 40 may also receive direction information from, for example, a direction sensor 27 that detects the direction of the vehicle. The information including the vehicle speed of the vehicle input to the driving control device 40 is output to the display control device 10.
　表示制御装置１０は、プロセッサおよびメモリー等を有するコンピュータを主体に構成された電子制御装置である。表示制御装置１０は、仮想視点決定部１１および映像作成部１２を備えている。具体的には、表示制御装置１０は、メモリーに格納された各プログラムをプロセッサが実行することで仮想視点決定部１１および映像作成部１２として機能する。表示制御装置１０には、カメラ２０が撮影した映像データ、他車両位置検知部３０が特定した他車両の位置および方向等の情報、走行制御装置４０から自車両の車速情報等が入力される。表示制御装置１０は、それらの情報に基づき、仮想視点決定部１１および映像作成部１２により、ディスプレイ５０に表示する映像を作成し出力する。 The display control device 10 is an electronic control device mainly composed of a computer having a processor, memory, etc. The display control device 10 is equipped with a virtual viewpoint determination unit 11 and an image creation unit 12. Specifically, the display control device 10 functions as the virtual viewpoint determination unit 11 and the image creation unit 12 by the processor executing each program stored in the memory. The display control device 10 receives inputs of image data captured by the camera 20, information such as the position and direction of other vehicles identified by the other vehicle position detection unit 30, and vehicle speed information of the vehicle itself from the driving control device 40. Based on this information, the display control device 10 creates and outputs an image to be displayed on the display 50 using the virtual viewpoint determination unit 11 and the image creation unit 12.
　仮想視点決定部１１は、自車両の少なくとも一部および他車両の少なくとも一部が表示されるように、ディスプレイ５０に表示する映像の視点位置、画角および注視方向を計算し、仮想視点を決定する。即ち、仮想視点とは、視点位置、画角および注視方向を含んでいる。 The virtual viewpoint determination unit 11 calculates the viewpoint position, angle of view, and gaze direction of the image to be displayed on the display 50 so that at least a part of the vehicle itself and at least a part of the other vehicle are displayed, and determines the virtual viewpoint. In other words, the virtual viewpoint includes the viewpoint position, angle of view, and gaze direction.
　映像作成部１２は、カメラ２０で撮影した映像を合成することで仮想視点から視た表示範囲の映像を作成する。具体的には、映像作成部１２は、他車両を撮影した１つ以上のカメラ２０を特定し、その特定したカメラ２０の映像を合成してディスプレイ５０に表示させる映像を作成する。映像作成部１２で作成された映像データは、ディスプレイ５０に出力される。なお、他車両を撮影したカメラ２０の映像に自車両が十分に映っていない場合、自車両の映像はＣＧにて表示してもよい。ＣＧはcomputer graphicsの略である。 The image creation unit 12 creates an image of the display range seen from a virtual viewpoint by synthesizing images captured by the cameras 20. Specifically, the image creation unit 12 identifies one or more cameras 20 that have captured images of other vehicles, and creates an image to be displayed on the display 50 by synthesizing the images from the identified cameras 20. The image data created by the image creation unit 12 is output to the display 50. Note that if the host vehicle is not sufficiently visible in the images captured by the cameras 20 that have captured the other vehicles, the image of the host vehicle may be displayed using CG. CG stands for computer graphics.
　ここで、図２は、自車両の遠方に他車両として例えば対向車両が存在する状況を示し、図４は、自車両の近傍に他車両として例えば対向車両が存在する状況を示している。なお、図２～図５およびその説明では、仮想視点（詳細には視点位置）に符号ＶＰを付し、自車両に符号Ｍ１を付し、他車両に符号Ｍ２を付している。また、仮想視点ＶＰから延びる破線は、画角および注視方向を示している。 Here, FIG. 2 shows a situation where there is another vehicle, such as an oncoming vehicle, far away from the host vehicle, and FIG. 4 shows a situation where there is another vehicle, such as an oncoming vehicle, close to the host vehicle. Note that in FIGS. 2 to 5 and their explanations, the virtual viewpoint (more specifically, the viewpoint position) is given the symbol VP, the host vehicle is given the symbol M1, and the other vehicle is given the symbol M2. Also, the dashed lines extending from the virtual viewpoint VP indicate the angle of view and the direction of gaze.
　図２に示すように、仮想視点決定部１１は、他車両位置検知部３０で検知される他車両Ｍ２の位置が自車両Ｍ１から遠いほど、表示範囲を広げて縮小表示するよう、仮想視点ＶＰを低い位置に決定する。なお、低い位置とは、地面からの鉛直方向の高さが比較的低いことをいう。映像作成部１２は、その仮想視点ＶＰから視た映像を作成し、その映像データをディスプレイ５０に出力して表示させる。 As shown in FIG. 2, the virtual viewpoint determination unit 11 determines the virtual viewpoint VP to be at a lower position so that the display range is expanded and reduced as the position of the other vehicle M2 detected by the other vehicle position detection unit 30 becomes farther from the host vehicle M1. Note that a low position means a relatively low vertical height from the ground. The image creation unit 12 creates an image viewed from the virtual viewpoint VP, and outputs the image data to the display 50 for display.
　図３は、図２に示した状況において、ディスプレイ５０に表示される映像の一例を示している。図３に示すように、表示制御装置１０は、自車両Ｍ１と他車両Ｍ２との車間距離が遠いほど、表示範囲を広げて自車両Ｍ１と他車両Ｍ２とを縮小表示する。これにより、ディスプレイ５０には、自車両Ｍ１と他車両Ｍ２の他に、周囲の環境が広く映し出される。そのため、自車両Ｍ１の運転者は、他車両Ｍ２を含む周囲の環境を視認し、他車両Ｍ２に加えてその周囲の危険性を容易に確認できる。以下の説明では、自車両Ｍ１と他車両Ｍ２との車間距離を、単に「車間距離」という。また、自車両Ｍ１の運転者を、単に「運転者」という。 FIG. 3 shows an example of an image displayed on the display 50 in the situation shown in FIG. 2. As shown in FIG. 3, the display control device 10 expands the display range and reduces the display of the host vehicle M1 and the other vehicle M2 as the distance between the host vehicle M1 and the other vehicle M2 increases. This allows the display 50 to show a wide range of the surrounding environment in addition to the host vehicle M1 and the other vehicle M2. Therefore, the driver of the host vehicle M1 can visually check the surrounding environment including the other vehicle M2 and easily confirm dangers around the other vehicle M2 in addition to the other vehicle M2. In the following description, the distance between the host vehicle M1 and the other vehicle M2 is simply referred to as the "distance between the vehicles." Also, the driver of the host vehicle M1 is simply referred to as the "driver."
　一方、図４に示すように、仮想視点決定部１１は、他車両位置検知部３０で検知される他車両Ｍ２の位置が自車両Ｍ１に近いほど、表示範囲を狭めて拡大表示するよう、仮想視点ＶＰを高い位置に決定する。なお、高い位置とは、地面からの鉛直方向の高さが比較的高いことをいう。映像作成部１２は、その仮想視点ＶＰから視た映像を作成し、その映像データをディスプレイ５０に出力して表示させる。 On the other hand, as shown in FIG. 4, the virtual viewpoint determination unit 11 determines the virtual viewpoint VP to be at a higher position so that the display range is narrowed and enlarged as the position of the other vehicle M2 detected by the other vehicle position detection unit 30 is closer to the host vehicle M1. Note that a high position means a relatively high vertical height from the ground. The image creation unit 12 creates an image viewed from the virtual viewpoint VP, and outputs the image data to the display 50 for display.
　図５は、図４に示した状況において、ディスプレイ５０に表示される映像の一例を示している。図５に示すように、表示制御装置１０は、車間距離が近いほど、表示範囲を狭めて自車両Ｍ１と他車両Ｍ２とを拡大表示する。これにより、ディスプレイ５０には、自車両Ｍ１の一部と、他車両Ｍ２の一部（この例では、他車両の前方部分）とが拡大表示される。そのため、運転者は、自車両Ｍ１と他車両Ｍ２とを同時に視認し、接触の危険性や距離感を容易に確認できる。 FIG. 5 shows an example of an image displayed on the display 50 in the situation shown in FIG. 4. As shown in FIG. 5, the closer the vehicle-to-vehicle distance is, the narrower the display range is, and the display control device 10 enlarges and displays the host vehicle M1 and the other vehicle M2. As a result, a part of the host vehicle M1 and a part of the other vehicle M2 (in this example, the front part of the other vehicle) are enlarged and displayed on the display 50. This allows the driver to simultaneously view the host vehicle M1 and the other vehicle M2, and easily check the risk of contact and the sense of distance.
　なお、図２～図５では、自車両Ｍ１と対向車両との関係において表示制御装置１０が実行する制御処理を説明したが、表示制御装置１０が実行する制御処理は、他車両Ｍ２が先行車両、並走車両、後続車両または停止車両であっても同じである。なお、対向車両とは、自車両Ｍ１の前方から自車両Ｍ１に向かって走行する他車両Ｍ２をいい、「先行車両」、「並走車両」、「後続車両」とはそれぞれ、自車両Ｍ１の前方、側方、後方を自車両Ｍ１と同一進行方向に走行する他車両Ｍ２をいう。 Note that while Figures 2 to 5 have described the control processing executed by the display control device 10 in relation to the host vehicle M1 and an oncoming vehicle, the control processing executed by the display control device 10 is the same whether the other vehicle M2 is a leading vehicle, a vehicle running alongside, a following vehicle, or a stopped vehicle. Note that an oncoming vehicle refers to the other vehicle M2 traveling from in front of the host vehicle M1 toward the host vehicle M1, and "leading vehicle," "parallel vehicle," and "following vehicle" refer to the other vehicle M2 traveling in the same direction as the host vehicle M1, ahead of, to the side of, and behind the host vehicle M1, respectively.
　次に、本実施形態の表示制御装置１０を含む車両システム１００が実行する制御処理について、図６～図８に示したフローチャートを参照して説明する。なお、以下の説明および図６～図８では、ステップを単に「Ｓ」と表記する。 Next, the control process executed by the vehicle system 100 including the display control device 10 of this embodiment will be described with reference to the flowcharts shown in Figures 6 to 8. Note that in the following description and in Figures 6 to 8, steps are simply represented as "S."
　図６～図８のフローチャートに示す制御処理は、車両の走行スイッチがオンされると同時に開始される。
　まず、図６のＳ１０で、走行制御装置４０は、車速センサ２４から自車両の車速を取得する。 The control process shown in the flowcharts of FIGS. 6 to 8 is started at the same time that the vehicle running switch is turned on.
First, in S10 of FIG. 6, the cruise control device 40 acquires the speed of the host vehicle from the vehicle speed sensor 24.
　次に、Ｓ２０で、他車両位置検知部３０は、自車両の周辺に存在する他車両を検知する。このＳ２０で他車両位置検知部３０が実行する処理を、図７に詳細に示す。図７に示すように、Ｓ２１で、他車両位置検知部３０は、カメラ２０が撮影したカメラ映像、即ち、映像データを所得する。次に、他車両位置検知部３０は、Ｓ２２で、映像認識技術を使用してカメラ映像に含まれる他車両の位置および方向等を検出し、続いてＳ２５で、車間距離を判定する。 Next, in S20, the other vehicle position detection unit 30 detects other vehicles present in the vicinity of the vehicle itself. The process executed by the other vehicle position detection unit 30 in S20 is shown in detail in FIG. 7. As shown in FIG. 7, in S21, the other vehicle position detection unit 30 acquires the camera image captured by the camera 20, i.e., the image data. Next, in S22, the other vehicle position detection unit 30 uses image recognition technology to detect the position and direction, etc., of other vehicles contained in the camera image, and then in S25, determines the distance between the vehicles.
　なお、Ｓ２３で、他車両位置検知部３０は、カメラ２０が撮影した映像データに代えて、又はそれと共に、例えばＬｉＤＡＲセンサ２１、レーダーセンサ２２、ソナーセンサ２３などの測距センサから入力される情報を取得してもよい。その場合も、Ｓ２５で、他車両位置検知部３０は、それらの情報に基づき車間距離を判定する。 In addition, in S23, the other vehicle position detection unit 30 may acquire information input from a distance measurement sensor such as the LiDAR sensor 21, the radar sensor 22, or the sonar sensor 23, instead of or in addition to the video data captured by the camera 20. In this case, too, in S25, the other vehicle position detection unit 30 determines the inter-vehicle distance based on that information.
　或いは、Ｓ２４で、他車両位置検知部３０は、カメラ２０が撮影した映像データまたは測距センサから入力される情報に代えて、又はそれと共に、例えば車車間通信装置２８など、他の技術から入力される情報を取得してもよい。その場合も、Ｓ２５で、他車両位置検知部３０は、その情報に基づき車間距離を判定する。 Alternatively, in S24, the other vehicle position detection unit 30 may acquire information input from other technologies, such as the vehicle-to-vehicle communication device 28, instead of or in addition to the video data captured by the camera 20 or the information input from the distance measurement sensor. In this case as well, in S25, the other vehicle position detection unit 30 determines the inter-vehicle distance based on that information.
　次に、図６のＳ３０で、表示制御装置１０は、走行制御装置４０が車速センサ２４から得た車速情報に基づき、自車両の車速が規定車速範囲内であるか否かを判定する。この規定車速範囲は、表示制御装置１０のメモリーに記憶されている。例えば、表示制御装置１０を高速道路で使用する場合、この規定車速範囲は、一般に車両が高速道路を走行する際の車速範囲に設定される。或いは、例えば、表示制御装置１０を一般道で使用する場合、この規定車速範囲は、一般に車両が一般道を走行する際の車速範囲に設定される。 Next, in S30 of FIG. 6, the display control device 10 determines whether the vehicle speed of the vehicle is within a specified vehicle speed range based on the vehicle speed information obtained by the driving control device 40 from the vehicle speed sensor 24. This specified vehicle speed range is stored in the memory of the display control device 10. For example, when the display control device 10 is used on a highway, this specified vehicle speed range is set to the vehicle speed range when the vehicle is generally traveling on the highway. Alternatively, for example, when the display control device 10 is used on an ordinary road, this specified vehicle speed range is set to the vehicle speed range when the vehicle is generally traveling on an ordinary road.
　Ｓ３０で表示制御装置１０は、自車両の車速が規定車速範囲外にあると判定した場合、処理をＳ７０に進める。Ｓ７０で、表示制御装置１０は、デフォルト視点の映像データをディスプレイ５０に出力し、表示させる。
　一方、Ｓ３０で表示制御装置１０は、自車両の車速が規定車速範囲内にあると判定した場合、処理をＳ４０に進める。 If the display control device 10 determines in S30 that the vehicle speed is outside the specified vehicle speed range, the process proceeds to S70. In S70, the display control device 10 outputs the video data of the default viewpoint to the display 50 to display it.
On the other hand, if the display control device 10 determines in S30 that the vehicle speed is within the specified vehicle speed range, the process proceeds to S40.
　Ｓ４０で、表示制御装置１０は、車間距離が規定距離範囲内であるか否かを判定する。この規定車速範囲は、表示制御装置１０のメモリーに記憶されている。例えば、表示制御装置１０を高速道路で使用する場合、この規定距離範囲は、一般に車両が高速道路を走行する際に他車両を注視すべき車間距離に設定される。また、例えば、表示制御装置１０を一般道で使用する場合、この規定車速範囲は一般に車両が一般道を走行する際に他車両を注視すべき車間距離に設定される。 In S40, the display control device 10 determines whether the vehicle distance is within a specified distance range. This specified vehicle speed range is stored in the memory of the display control device 10. For example, when the display control device 10 is used on a highway, this specified distance range is generally set to the vehicle distance at which the vehicle should pay attention to other vehicles when traveling on the highway. Also, for example, when the display control device 10 is used on an ordinary road, this specified vehicle speed range is generally set to the vehicle distance at which the vehicle should pay attention to other vehicles when traveling on an ordinary road.
　Ｓ４０で表示制御装置１０は、車間距離が規定距離範囲外にあると判定した場合、処理をＳ７０に進める。Ｓ７０で、表示制御装置１０は、デフォルト視点の映像データをディスプレイ５０に出力し、表示させる。
　一方、Ｓ４０で表示制御装置１０は、車間距離が規定距離範囲内にあると判定した場合、処理をＳ５０に進める。 If the display control device 10 determines in S40 that the inter-vehicle distance is outside the specified distance range, the process proceeds to S70. In S70, the display control device 10 outputs the video data of the default viewpoint to the display 50 to display it.
On the other hand, if the display control device 10 determines in S40 that the inter-vehicle distance is within the specified distance range, the process proceeds to S50.
　Ｓ５０で、表示制御装置１０は、車間距離に応じて最適な仮想視点を決定する。このＳ５０で表示制御装置１０が実行する処理を、図８に詳細に示す。図８に示すように、Ｓ５１で、表示制御装置１０の備える仮想視点決定部１１は、車間距離に基づいてディスプレイ５０に表示する表示範囲を決定する。具体的には、仮想視点決定部１１は、車間距離が遠いほど表示範囲を広くして自車両と他車両とを縮小表示する。一方、仮想視点決定部１１は、車間距離が近いほど表示範囲を狭くして自車両と他車両とを拡大表示する。 In S50, the display control device 10 determines an optimal virtual viewpoint depending on the vehicle distance. The process executed by the display control device 10 in S50 is shown in detail in FIG. 8. As shown in FIG. 8, in S51, the virtual viewpoint determination unit 11 provided in the display control device 10 determines the display range to be displayed on the display 50 based on the vehicle distance. Specifically, the virtual viewpoint determination unit 11 widens the display range as the vehicle distance increases, and reduces the display of the host vehicle and the other vehicle. On the other hand, the virtual viewpoint determination unit 11 narrows the display range as the vehicle distance decreases, and enlarges the display of the host vehicle and the other vehicle.
　次に、Ｓ５２で、仮想視点決定部１１は、表示範囲が映像表示領域に収まる視点位置、画角、注視方向を計算し、最適な仮想視点を決定する。具体的には、仮想視点決定部１１は、表示範囲が広いほど仮想視点を低い位置に決定し、表示範囲が狭いほど仮想視点を高い位置に決定する。 Next, in S52, the virtual viewpoint determination unit 11 calculates the viewpoint position, angle of view, and viewing direction that will cause the display range to fit within the video display area, and determines the optimal virtual viewpoint. Specifically, the wider the display range, the lower the virtual viewpoint determination unit 11 determines the virtual viewpoint to be, and the narrower the display range, the higher the virtual viewpoint determination unit 11 determines the virtual viewpoint to be.
　続いて、Ｓ５３で、表示制御装置１０の備える映像作成部１２は、Ｓ５３で決めた最適な仮想視点から視た表示範囲の映像を作成する。 Next, in S53, the image creation unit 12 of the display control device 10 creates an image of the display range as seen from the optimal virtual viewpoint determined in S53.
　続いて、図６のＳ６０で、表示制御装置１０は、最適な仮想視点から視た表示範囲の映像データをディスプレイ５０に出力し、その映像をディスプレイ５０に表示させる。表示制御装置１０を含む車両システム１００は、上述したＳ１０～Ｓ７０の処理を所定の制御周期で繰り返し実行する。 Next, in S60 of FIG. 6, the display control device 10 outputs image data of the display range viewed from the optimal virtual viewpoint to the display 50, and causes the image to be displayed on the display 50. The vehicle system 100 including the display control device 10 repeatedly executes the above-mentioned processes of S10 to S70 at a predetermined control cycle.
　次に、車間距離と仮想視点の高さとの関係を、図９のグラフを参照して説明する。 Next, the relationship between the vehicle distance and the height of the virtual viewpoint will be explained with reference to the graph in Figure 9.
　図９のグラフにおいて、横軸は、車間距離を示し、縦軸は、仮想視点決定部１１によって決められる仮想視点の高さを示している。また、実線Ａは、時間の経過に伴って車間距離が減少傾向にあるときの車間距離と仮想視点との関係を示したものである。実線Ｂは、時間の経過に伴って車間距離が一定の状態にあるときの車間距離と仮想視点との関係を示したものである。実線Ｃは、時間の経過に伴って車間距離が増加傾向にあるときの車間距離と仮想視点との関係を示したものである。 In the graph of FIG. 9, the horizontal axis indicates the distance between vehicles, and the vertical axis indicates the height of the virtual viewpoint determined by the virtual viewpoint determination unit 11. Furthermore, solid line A shows the relationship between the distance between vehicles and the virtual viewpoint when the distance between vehicles is decreasing over time. Solid line B shows the relationship between the distance between vehicles and the virtual viewpoint when the distance between vehicles remains constant over time. Solid line C shows the relationship between the distance between vehicles and the virtual viewpoint when the distance between vehicles is increasing over time.
　実線Ａ～Ｃのいずれにおいても、仮想視点決定部１１は、車間距離が遠いほど仮想視点を低い位置に決定する。これにより、表示制御装置１０は、車間距離が遠いほど表示範囲を広げて自車両と他車両とを縮小表示することが可能となる。一方、仮想視点決定部１１は、車間距離が近いほど仮想視点を高い位置に決定する。これにより、表示制御装置１０は、車間距離が近いほど表示範囲を狭めて自車両と他車両とを拡大表示することが可能となる。 For all of the solid lines A to C, the virtual viewpoint determination unit 11 determines the virtual viewpoint to be at a lower position as the distance between the vehicles increases. This allows the display control device 10 to expand the display range and display a reduced image of the vehicle's own vehicle and other vehicles as the distance between the vehicles increases. On the other hand, the virtual viewpoint determination unit 11 determines the virtual viewpoint to be at a higher position as the distance between the vehicles decreases. This allows the display control device 10 to narrow the display range and display a larger image of the vehicle's own vehicle and other vehicles as the distance between the vehicles decreases.
　また、実線Ａ～Ｃのいずれにおいても、仮想視点決定部１１は、車間距離が遠いほど仮想視点を高くする変化量を小さくしている。これは、仮想視点が比較的低いときは、車間距離に対する仮想視点の変化量が小さくても表示範囲を大きく変えることが可能であるからである。一方、仮想視点決定部１１は、車間距離が近いほど仮想視点を高くする変化量を大きくしている。これは、仮想視点が比較的高いときは、車間距離に対する仮想視点の変化量を大きくすることで表示範囲を適切に変えることが可能であるからである。 Furthermore, for all of the solid lines A to C, the virtual viewpoint determination unit 11 reduces the amount of change by which the virtual viewpoint is raised as the vehicle distance increases. This is because, when the virtual viewpoint is relatively low, it is possible to significantly change the display range even if the amount of change in the virtual viewpoint relative to the vehicle distance is small. On the other hand, the virtual viewpoint determination unit 11 increases the amount of change by which the virtual viewpoint is raised as the vehicle distance decreases. This is because, when the virtual viewpoint is relatively high, it is possible to appropriately change the display range by increasing the amount of change in the virtual viewpoint relative to the vehicle distance.
　さらに、実線Ａおよび実線Ｂに示すように、仮想視点決定部１１は、いずれの車間距離においても、実線Ａに示す車間距離が減少傾向にあるときの仮想視点の位置を、実線Ｂに示す車間距離が一定のときの仮想視点の位置よりも高くしている。これによれば、仮想視点決定部１１は、車間距離が減少傾向にあるとき、即ち、自車両と他車両とが近づくときは、仮想視点の位置を比較的高くする。これにより、ディスプレイ５０の映像において表示範囲が狭くなり自車両と他車両とが拡大表示される。そのため、運転者はその映像により自車両と他車両とを同時に視認し、自車両に近づく他車両との接触の危険性や距離感を確実に確認できる。 Furthermore, as shown by solid lines A and B, the virtual viewpoint determination unit 11 sets the position of the virtual viewpoint when the inter-vehicle distance shown by solid line A is decreasing, regardless of the inter-vehicle distance, higher than the position of the virtual viewpoint when the inter-vehicle distance shown by solid line B is constant. According to this, when the inter-vehicle distance is decreasing, that is, when the vehicle-to-vehicle distance approaches the other vehicle, the virtual viewpoint determination unit 11 sets the position of the virtual viewpoint relatively higher. This narrows the display range in the image on the display 50, and the vehicle-to-vehicle and the other vehicle are displayed in an enlarged manner. Therefore, the driver can simultaneously view the vehicle-to-vehicle and the other vehicle from the image, and can reliably confirm the risk of contact with the other vehicle approaching the vehicle-to-vehicle and the sense of distance.
　一方、実線Ｂおよび実線Ｃに示すように、仮想視点決定部１１は、いずれの車間距離においても、実線Ｂに示す車間距離が一定のときの仮想視点の位置よりも、実線Ｃに示す車間距離が増加傾向にあるときの仮想視点の位置を低くしている。これによれば、仮想視点決定部１１は、車間距離が増加傾向にあるとき、即ち、自車両と他車両とが遠ざかるときは、仮想視点の位置を比較的低くする。これにより、ディスプレイ５０の映像において表示範囲が広くなり自車両と他車両とが縮小表示される。そのため、運転者はその映像により他車両とその周囲の環境を同時に視認し、自車両から遠ざかる他車両に加えてその周囲の危険性を確実に確認できる。 On the other hand, as shown by solid lines B and C, the virtual viewpoint determination unit 11 lowers the position of the virtual viewpoint when the inter-vehicle distance is increasing, as shown by solid line C, for any inter-vehicle distance, compared to the position of the virtual viewpoint when the inter-vehicle distance is constant, as shown by solid line B. As a result, the virtual viewpoint determination unit 11 lowers the position of the virtual viewpoint relatively when the inter-vehicle distance is increasing, that is, when the vehicle's own vehicle and the other vehicle are moving away from each other. This widens the display range in the image on the display 50, and the vehicle's own vehicle and the other vehicle are displayed in a reduced size. Therefore, the driver can simultaneously view the other vehicle and its surrounding environment through the image, and can reliably confirm not only the other vehicle moving away from the vehicle's own vehicle, but also any dangers in the surrounding area.
　以上説明した第１実施形態の表示制御装置１０は、次の作用効果を奏するものである。
　（１）第１実施形態では、表示制御装置１０の備える仮想視点決定部１１は、他車両位置検知部３０で検知される他車両の位置が自車両に近いほど表示範囲を狭め、自車両の少なくとも一部及び他車両の少なくとも一部を拡大表示するよう、仮想視点を高い位置に決定する。また、仮想視点決定部１１は、他車両位置検知部３０で検知される他車両の位置が自車両から遠いほど表示範囲を広げ、自車両の少なくとも一部及び他車両の少なくとも一部を縮小表示するよう仮想視点を低い位置に決定する。 The display control device 10 of the first embodiment described above provides the following advantageous effects.
(1) In the first embodiment, the virtual viewpoint determination unit 11 included in the display control device 10 narrows the display range as the position of the other vehicle detected by the other vehicle position detection unit 30 gets closer to the host vehicle, and determines the virtual viewpoint to be at a higher position so that at least a part of the host vehicle and at least a part of the other vehicle are displayed in an enlarged manner. Also, the virtual viewpoint determination unit 11 widens the display range as the position of the other vehicle detected by the other vehicle position detection unit 30 gets farther from the host vehicle, and determines the virtual viewpoint to be at a lower position so that at least a part of the host vehicle and at least a part of the other vehicle are displayed in a reduced manner.
　これによれば、車間距離が近いほどディスプレイ５０の映像において表示範囲が狭くなり自車両と他車両とが拡大表示されるので、運転者は、自車両と他車両とを同時に視認し、接触の危険性や距離感を容易に確認できる。一方、車間距離が遠いほどディスプレイ５０の映像において表示範囲が広くなり自車両と他車両とが縮小表示されるので、運転者は、他車両とその周囲の環境を同時に視認し、他車両に加えてその周囲の危険性を容易に確認できる。したがって、この表示制御装置１０は、自車両と他車両との位置関係に応じて注意すべき箇所を運転者が容易に視認できるよう適切な表示を行うことが可能である。 According to this, the closer the distance between the vehicles, the narrower the display range in the image on display 50 and the enlarged view of the vehicle's own vehicle and the other vehicle, allowing the driver to see both the vehicle and the other vehicle simultaneously and easily check the risk of contact and the sense of distance. On the other hand, the farther the distance between the vehicles, the wider the display range in the image on display 50 and the reduced view of the vehicle's own vehicle and the other vehicle, allowing the driver to see the other vehicle and its surrounding environment simultaneously and easily check the dangers around it in addition to the other vehicle. Therefore, this display control device 10 is capable of providing appropriate display so that the driver can easily see areas requiring attention depending on the relative positions of the vehicle and the other vehicle.
　（２）第１実施形態では、他車両位置検知部３０は、自車両の前方、側方または後方に存在する他車両の位置および方向を特定することが可能である。映像作成部１２は、他車両を撮影した１つ以上のカメラ２０を特定し、その特定したカメラ２０の映像を合成してディスプレイ５０に表示させる映像を作成する。
　これによれば、映像作成部１２は、１つ以上のカメラ２０の映像を合成することで、仮想視点決定部１１が決定した仮想視点から視た表示範囲の映像、即ち、ディスプレイ５０に表示させる映像を作成することが可能である。 (2) In the first embodiment, the other vehicle position detection unit 30 can identify the position and direction of another vehicle that is in front of, to the side of, or behind the vehicle. The image creation unit 12 identifies one or more cameras 20 that have captured images of the other vehicle, and creates an image to be displayed on the display 50 by combining images from the identified cameras 20.
According to this, the image creation unit 12 can synthesize images from one or more cameras 20 to create an image of the display range viewed from the virtual viewpoint determined by the virtual viewpoint determination unit 11, i.e., an image to be displayed on the display 50.
　（３）第１実施形態では、仮想視点決定部１１は、時間の経過に伴って車間距離が減少傾向にあるときの仮想視点の位置を、時間の経過に伴って車間距離が増加傾向にあるときの仮想視点の位置よりも高くする。
　これによれば、仮想視点決定部１１は、時間の経過に伴って車間距離が減少傾向にあるとき、即ち、自車両と他車両とが近づくときは、仮想視点の位置を比較的高くする。これにより、ディスプレイ５０の映像において表示範囲が狭くなり自車両と他車両とが拡大表示される。そのため、運転者はその映像により自車両と他車両とを同時に視認し、自車両に近づく他車両との接触の危険性や距離感を確実に確認できる。
　一方、仮想視点決定部１１は、時間の経過に伴って車間距離が増加傾向にあるとき、即ち、自車両と他車両とが遠ざかるときは、仮想視点の位置を比較的低くする。これにより、ディスプレイ５０の映像において表示範囲が広くなり自車両と他車両とが縮小表示される。そのため、運転者はその映像により他車両とその周囲の環境を同時に視認し、自車両から遠ざかる他車両に加えてその周囲の危険性を確実に確認できる。 (3) In the first embodiment, the virtual viewpoint determination unit 11 sets the position of the virtual viewpoint when the inter-vehicle distance tends to decrease over time to be higher than the position of the virtual viewpoint when the inter-vehicle distance tends to increase over time.
According to this, when the inter-vehicle distance tends to decrease with the passage of time, i.e., when the host vehicle and the other vehicle approach each other, the virtual viewpoint determination unit 11 sets the position of the virtual viewpoint relatively high. This narrows the display range of the image on the display 50, and the host vehicle and the other vehicle are displayed in an enlarged manner. Therefore, the driver can simultaneously view the host vehicle and the other vehicle from the image, and can reliably confirm the risk of contact with the other vehicle approaching the host vehicle and the sense of distance.
On the other hand, when the inter-vehicle distance is increasing over time, i.e., when the vehicle is moving away from the other vehicle, the virtual viewpoint determination unit 11 sets the position of the virtual viewpoint relatively low. This widens the display range of the image on the display 50, and the vehicle and the other vehicle are displayed in a reduced size. This allows the driver to simultaneously view the other vehicle and its surrounding environment through the image, and to reliably confirm not only the other vehicle moving away from the vehicle, but also any dangers in the surrounding area.
　（４）第１実施形態では、仮想視点決定部１１は、車間距離が近くなるほど仮想視点を高くする変化量を大きくする。
　これによれば、車間距離が遠く、仮想視点が比較的低いときは、車間距離に対する仮想視点の変化量が小さくても表示範囲を大きく変えることが可能である。一方、車間距離が近く、仮想視点が比較的高いときは、車間距離に対する仮想視点の変化量を大きくすることで表示範囲を適切に変えることが可能である。したがって、この表示制御装置１０は、自車両と他車両との位置関係に応じて注意すべき箇所を運転者が容易に視認できるよう適切な表示を行うことが可能である。 (4) In the first embodiment, the virtual viewpoint determination unit 11 increases the amount of change by which the virtual viewpoint is raised as the inter-vehicle distance decreases.
According to this, when the vehicle distance is long and the virtual viewpoint is relatively low, the display range can be changed significantly even if the change in the virtual viewpoint with respect to the vehicle distance is small. On the other hand, when the vehicle distance is short and the virtual viewpoint is relatively high, the display range can be changed appropriately by increasing the change in the virtual viewpoint with respect to the vehicle distance. Therefore, this display control device 10 can perform appropriate display so that the driver can easily visually recognize areas that require attention according to the positional relationship between the vehicle and other vehicles.
　（第２実施形態）
　第２実施形態について説明する。第２実施形態は、第１実施形態に対して車両システム１００が実行する制御処理の一部を変更したものであり、その他については第１実施形態と同様であるため、第１実施形態と異なる部分についてのみ説明する。詳細には、第２実施形態は、第１実施形態で参照した図６中のＳ５０「最適な仮想視点の決定」の処理を変更したものである。 Second Embodiment
A second embodiment will be described. In the second embodiment, a part of the control process executed by the vehicle system 100 in the first embodiment is changed, and the rest is the same as in the first embodiment, so only the parts that are different from the first embodiment will be described. In detail, in the second embodiment, the process of S50 "Determining an optimal virtual viewpoint" in FIG. 6 referred to in the first embodiment is changed.
　第２実施形態において、図６中のＳ５０「最適な仮想視点の決定」の処理を、図１０に詳細に示す。図１０に示すように、Ｓ５５で、表示制御装置１０の備える仮想視点決定部１１は、車間距離に基づいてディスプレイ５０に表示する表示範囲を決定する。具体的には、仮想視点決定部１１は、車間距離が遠いほど表示範囲を広くして自車両と他車両とを縮小表示する。一方、仮想視点決定部１１は、車間距離が近いほど表示範囲を狭くして自車両と他車両とを拡大表示する。 In the second embodiment, the process of S50 "Determining the optimal virtual viewpoint" in FIG. 6 is shown in detail in FIG. 10. As shown in FIG. 10, in S55, the virtual viewpoint determination unit 11 provided in the display control device 10 determines the display range to be displayed on the display 50 based on the inter-vehicle distance. Specifically, the virtual viewpoint determination unit 11 widens the display range as the inter-vehicle distance increases, and reduces the display of the host vehicle and the other vehicle. On the other hand, the virtual viewpoint determination unit 11 narrows the display range as the inter-vehicle distance decreases, and enlarges the display of the host vehicle and the other vehicle.
　次に、Ｓ５６で、仮想視点決定部１１は、表示範囲に応じて事前に準備した仮想視点群から仮想視点を決定する。すなわち、第２実施形態では、表示制御装置１０のメモリーに、予め、表示範囲に応じた仮想視点が複数記憶されている。なお、その複数の仮想視点は、表示範囲が広いほど低い位置に設定され、表示範囲が狭いほど高い位置に設定されている。仮想視点決定部１１は、その複数の仮想視点の中から、即ち、仮想視点群の中から、表示範囲に応じた適切な仮想視点を選択し決定する。 Next, in S56, the virtual viewpoint determination unit 11 determines a virtual viewpoint from a group of virtual viewpoints prepared in advance according to the display range. That is, in the second embodiment, a plurality of virtual viewpoints corresponding to the display range are stored in advance in the memory of the display control device 10. Note that the plurality of virtual viewpoints are set to lower positions as the display range becomes wider, and are set to higher positions as the display range becomes narrower. The virtual viewpoint determination unit 11 selects and determines an appropriate virtual viewpoint according to the display range from among the plurality of virtual viewpoints, i.e., from among the group of virtual viewpoints.
　続いて、Ｓ５７で、表示制御装置１０の備える映像作成部１２は、Ｓ５３で決めた最適な仮想視点から視た表示範囲の映像を作成する。 Next, in S57, the image creation unit 12 of the display control device 10 creates an image of the display range viewed from the optimal virtual viewpoint determined in S53.
　第２実施形態で説明した制御処理においても、第１実施形態と同様の作用効果を奏することが可能である。 The control process described in the second embodiment can achieve the same effects as the first embodiment.
　（他の実施形態）
　上記各実施形態では、車両には、ＬｉＤＡＲセンサ２１、レーダーセンサ２２、ソナーセンサ２３等の測距装置、カメラ２０、車車間通信装置２８などが搭載されているものとして説明したが、これに限らない。他車両位置検知部３０がカメラ２０が撮影した映像データにより自車両の周辺に存在する他車両の位置を検知することが可能であれば、測距装置および車車間通信装置２８などは車両に搭載されていなくてもよい。 Other Embodiments
In each of the above embodiments, the vehicle is described as being equipped with a distance measuring device such as the LiDAR sensor 21, the radar sensor 22, the sonar sensor 23, the camera 20, the vehicle-to-vehicle communication device 28, etc., but this is not limited to this. As long as the other vehicle position detection unit 30 can detect the position of other vehicles present around the vehicle using the video data captured by the camera 20, the distance measuring device and the vehicle-to-vehicle communication device 28, etc., do not have to be equipped in the vehicle.
　本開示は上記した実施形態に限定されるものではなく、適宜変更が可能である。また、上記各実施形態およびその一部は、互いに無関係なものではなく、組み合わせが明らかに不可な場合を除き、適宜組み合わせが可能である。また、上記各実施形態において、実施形態を構成する要素は、特に必須であると明示した場合および原理的に明らかに必須であると考えられる場合等を除き、必ずしも必須のものではないことは言うまでもない。また、上記各実施形態において、実施形態の構成要素の個数、数値、量、範囲等の数値が言及されている場合、特に必須であると明示した場合および原理的に明らかに特定の数に限定される場合等を除き、その特定の数に限定されるものではない。また、上記各実施形態において、構成要素等の形状、位置関係等に言及するときは、特に明示した場合および原理的に特定の形状、位置関係等に限定される場合等を除き、その形状、位置関係等に限定されるものではない。 The present disclosure is not limited to the above-described embodiments, and can be modified as appropriate. The above-described embodiments and parts thereof are not unrelated to each other, and can be combined as appropriate, except when the combination is clearly impossible. In the above-described embodiments, it goes without saying that the elements constituting the embodiments are not necessarily essential, except when it is specifically stated that they are essential or when it is clearly considered essential in principle. In the above-described embodiments, when the numbers, values, amounts, ranges, etc. of the components of the embodiments are mentioned, they are not limited to the specific numbers, except when it is specifically stated that they are essential or when it is clearly limited to a specific number in principle. In the above-described embodiments, when the shapes, positional relationships, etc. of the components are mentioned, they are not limited to the shapes, positional relationships, etc., except when it is specifically stated that they are essential or when it is clearly limited to a specific shape, positional relationship, etc. in principle.
　本開示に記載の制御部及びその手法は、コンピュータプログラムにより具体化された一つ乃至は複数の機能を実行するようにプログラムされたプロセッサ及びメモリーを構成することによって提供された専用コンピュータにより、実現されてもよい。あるいは、本開示に記載の制御部及びその手法は、一つ以上の専用ハードウエア論理回路によってプロセッサを構成することによって提供された専用コンピュータにより、実現されてもよい。もしくは、本開示に記載の制御部及びその手法は、一つ乃至は複数の機能を実行するようにプログラムされたプロセッサ及びメモリーと一つ以上のハードウエア論理回路によって構成されたプロセッサとの組み合わせにより構成された一つ以上の専用コンピュータにより、実現されてもよい。また、コンピュータプログラムは、コンピュータにより実行されるインストラクションとして、コンピュータ読み取り可能な非遷移有形記録媒体に記憶されていてもよい。上記メモリーは非遷移的実体的記録媒体である。 The control unit and the method described in the present disclosure may be realized by a dedicated computer provided by configuring a processor and memory programmed to execute one or more functions embodied in a computer program. Alternatively, the control unit and the method described in the present disclosure may be realized by a dedicated computer provided by configuring a processor with one or more dedicated hardware logic circuits. Alternatively, the control unit and the method described in the present disclosure may be realized by one or more dedicated computers configured by combining a processor and memory programmed to execute one or more functions with a processor configured with one or more hardware logic circuits. Furthermore, the computer program may be stored in a computer-readable non-transient tangible recording medium as instructions executed by the computer. The memory is a non-transient tangible recording medium.
　（本開示の観点）
　上記した本開示については、例えば以下に示す観点として把握することができる。
　　［第１の観点］
　自車両（Ｍ１）周辺を撮影する１つ以上のカメラ（２０）、前記自車両周辺に存在する他車両（Ｍ２）の位置を検知する他車両位置検知部（３０）、および映像を表示するディスプレイ（５０）と共に、車両に搭載される表示制御装置であって、
　前記ディスプレイに表示する映像の視点位置、画角および注視方向を計算し仮想視点（ＶＰ）を決定する仮想視点決定部（１１）と、
　前記カメラで撮影した映像を合成することで前記仮想視点から視た表示範囲の映像を作成する映像作成部（１２）と、を備え、
　前記仮想視点決定部は、
　前記他車両位置検知部で検知される前記他車両の位置が前記自車両に近いほど表示範囲を狭めて拡大表示するよう前記仮想視点を高い位置に決定し、
　前記他車両位置検知部で検知される前記他車両の位置が前記自車両から遠いほど表示範囲を広げて縮小表示するよう前記仮想視点を低い位置に決定するように構成されている、表示制御装置。
　　［第２の観点］
　前記他車両位置検知部は、前記自車両の前方、後方または側方に存在する前記他車両の位置および方向を特定することが可能であり、
　前記映像作成部は、前記他車両を撮影した１つ以上の前記カメラを特定し、その特定した前記カメラの映像を合成して前記ディスプレイに表示させる映像を作成する、第１の観点に記載の表示制御装置。
　　［第３の観点］
　前記仮想視点決定部は、前記自車両と前記他車両との車間距離が時間の経過に伴って減少傾向にあるときの前記仮想視点の位置を、前記自車両と前記他車両との車間距離が時間の経過に伴って増加傾向にあるときの前記仮想視点の位置よりも高くする、第１または第２の観点に記載の表示制御装置。
　　［第４の観点］
　前記仮想視点決定部は、前記自車両と前記他車両との車間距離が近くなるほど前記仮想視点を高くする変化量を大きくする、第１ないし第３の観点のいずれか１つに記載の表示制御装置。 (Aspects of the present disclosure)
The present disclosure described above can be understood from the following viewpoints, for example.
[First viewpoint]
A display control device is mounted on a vehicle together with one or more cameras (20) that capture images of the surroundings of the vehicle (M1), a vehicle position detection unit (30) that detects the positions of other vehicles (M2) that exist around the vehicle (M1), and a display (50) that displays the images,
a virtual viewpoint determination unit (11) for calculating a viewpoint position, a field angle, and a gaze direction of an image to be displayed on the display and determining a virtual viewpoint (VP);
an image creation unit (12) that creates an image of the display range viewed from the virtual viewpoint by synthesizing the images captured by the camera;
The virtual viewpoint determination unit
determining the virtual viewpoint to be at a higher position so that the display range is narrowed and enlarged as the position of the other vehicle detected by the other vehicle position detection unit approaches the host vehicle;
A display control device configured to determine the virtual viewpoint to be at a lower position so that the display range is expanded and reduced as the position of the other vehicle detected by the other vehicle position detection unit becomes farther from the host vehicle.
[Second viewpoint]
the other vehicle position detection unit is capable of identifying a position and a direction of the other vehicle that is present in front of, behind, or to the side of the host vehicle;
The display control device described in the first aspect, wherein the image creation unit identifies one or more of the cameras that captured the other vehicle, and creates an image to be displayed on the display by combining images from the identified cameras.
[Third viewpoint]
The display control device described in the first or second aspect, wherein the virtual viewpoint determination unit sets the position of the virtual viewpoint when the inter-vehicle distance between the host vehicle and the other vehicle tends to decrease over time to be higher than the position of the virtual viewpoint when the inter-vehicle distance between the host vehicle and the other vehicle tends to increase over time.
[Fourth viewpoint]
The display control device according to any one of the first to third aspects, wherein the virtual viewpoint determination unit increases an amount of change by which the virtual viewpoint is elevated as the inter-vehicle distance between the host vehicle and the other vehicle becomes shorter.

Claims (4)

  1. 　自車両（Ｍ１）周辺を撮影する１つ以上のカメラ（２０）、前記自車両周辺に存在する他車両（Ｍ２）の位置を検知する他車両位置検知部（３０）、および映像を表示するディスプレイ（５０）と共に、車両に搭載される表示制御装置であって、
    　前記ディスプレイに表示する映像の視点位置、画角および注視方向を計算し仮想視点（ＶＰ）を決定する仮想視点決定部（１１）と、
    　前記カメラで撮影した映像を合成することで前記仮想視点から視た表示範囲の映像を作成する映像作成部（１２）と、を備え、
    　前記仮想視点決定部は、
    　前記他車両位置検知部で検知される前記他車両の位置が前記自車両に近いほど表示範囲を狭めて拡大表示するよう前記仮想視点を高い位置に決定し、
    　前記他車両位置検知部で検知される前記他車両の位置が前記自車両から遠いほど表示範囲を広げて縮小表示するよう前記仮想視点を低い位置に決定するように構成されている、表示制御装置。     A display control device is mounted on a vehicle together with one or more cameras (20) that capture images of the surroundings of the vehicle (M1), a vehicle position detection unit (30) that detects the positions of other vehicles (M2) that exist around the vehicle (M1), and a display (50) that displays the images,
    a virtual viewpoint determination unit (11) for calculating a viewpoint position, a field angle, and a gaze direction of an image to be displayed on the display and determining a virtual viewpoint (VP);
    and an image creation unit (12) that creates an image of the display range viewed from the virtual viewpoint by synthesizing the images captured by the camera,
    The virtual viewpoint determination unit
    determining the virtual viewpoint to be at a higher position so that the display range is narrowed and enlarged as the position of the other vehicle detected by the other vehicle position detection unit approaches the host vehicle;
    A display control device configured to determine the virtual viewpoint to be at a lower position so that the display range is expanded and reduced as the position of the other vehicle detected by the other vehicle position detection unit becomes farther from the host vehicle.
  2. 　前記他車両位置検知部は、前記自車両の前方、後方または側方に存在する前記他車両の位置および方向を特定することが可能であり、
    　前記映像作成部は、前記他車両を撮影した１つ以上の前記カメラを特定し、その特定した前記カメラの映像を合成して前記ディスプレイに表示させる映像を作成する、請求項１に記載の表示制御装置。     the other vehicle position detection unit is capable of identifying a position and a direction of the other vehicle that is present in front of, behind, or to the side of the host vehicle;
    The display control device according to claim 1 , wherein the image creation unit identifies one or more of the cameras that have captured the other vehicle, and creates an image to be displayed on the display by combining images from the identified cameras.
  3. 　前記仮想視点決定部は、前記自車両と前記他車両との車間距離が時間の経過に伴って減少傾向にあるときの前記仮想視点の位置を、前記自車両と前記他車両との車間距離が時間の経過に伴って増加傾向にあるときの前記仮想視点の位置よりも高くする、請求項１または２に記載の表示制御装置。 The display control device according to claim 1 or 2, wherein the virtual viewpoint determination unit sets the position of the virtual viewpoint when the inter-vehicle distance between the host vehicle and the other vehicle is decreasing over time to be higher than the position of the virtual viewpoint when the inter-vehicle distance between the host vehicle and the other vehicle is increasing over time.
  4. 　前記仮想視点決定部は、前記自車両と前記他車両との車間距離が近くなるほど前記仮想視点を高くする変化量を大きくする、請求項１または２に記載の表示制御装置。 The display control device according to claim 1 or 2, wherein the virtual viewpoint determination unit increases the amount of change by which the virtual viewpoint is elevated as the inter-vehicle distance between the host vehicle and the other vehicle decreases.
PCT/JP2023/038361 2022-11-23 2023-10-24 Display control device WO2024111324A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2022-186972 2022-11-23
JP2022186972 2022-11-23

Publications (1)

Publication Number Publication Date
WO2024111324A1 true WO2024111324A1 (en) 2024-05-30

Family

ID=91195446

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2023/038361 WO2024111324A1 (en) 2022-11-23 2023-10-24 Display control device

Country Status (1)

Country Link
WO (1) WO2024111324A1 (en)

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2010081245A (en) * 2008-09-25 2010-04-08 Nissan Motor Co Ltd Display device for vehicle, and display method
KR20100134154A (en) * 2009-06-15 2010-12-23 현대자동차일본기술연구소 Device and method for display image around a vehicle
JP2012253428A (en) * 2011-05-31 2012-12-20 Fujitsu Ten Ltd Information processing system, server device, and on-vehicle device
JP2017069852A (en) * 2015-10-01 2017-04-06 日産自動車株式会社 Information display device and information display method
JP2017182585A (en) * 2016-03-31 2017-10-05 株式会社Ｓｕｂａｒｕ Display device

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2010081245A (en) * 2008-09-25 2010-04-08 Nissan Motor Co Ltd Display device for vehicle, and display method
KR20100134154A (en) * 2009-06-15 2010-12-23 현대자동차일본기술연구소 Device and method for display image around a vehicle
JP2012253428A (en) * 2011-05-31 2012-12-20 Fujitsu Ten Ltd Information processing system, server device, and on-vehicle device
JP2017069852A (en) * 2015-10-01 2017-04-06 日産自動車株式会社 Information display device and information display method
JP2017182585A (en) * 2016-03-31 2017-10-05 株式会社Ｓｕｂａｒｕ Display device

Similar Documents

Publication Publication Date Title
US20230110116A1 (en) Advanced driver assist system, method of calibrating the same, and method of detecting object in the same
US10394237B2 (en) Perceiving roadway conditions from fused sensor data
US10449900B2 (en) Video synthesis system, video synthesis device, and video synthesis method
US9151626B1 (en) Vehicle position estimation system
US8879050B2 (en) Method for dynamically adjusting the operating parameters of a TOF camera according to vehicle speed
EP3229204A1 (en) Around view provision apparatus and vehicle including the same
JP5953824B2 (en) Vehicle rear view support apparatus and vehicle rear view support method
US10875452B2 (en) Driving assistance device and driving assistance method
US20150055120A1 (en) Image system for automotive safety applications
JP2018022234A (en) Image processing device and environment recognition device
JP5959264B2 (en) Image processing apparatus and method, and computer program
WO2019150552A1 (en) Falling object sensing device, vehicle-mounted system, vehicle, and falling object sensing program
US20220118910A1 (en) Imaging system and method
JP2013053962A (en) Camera system, collision prediction system, and parking guide system
US20220044032A1 (en) Dynamic adjustment of augmented reality image
JP6406175B2 (en) In-vehicle display control device, in-vehicle display control method
WO2024111324A1 (en) Display control device
JP7047291B2 (en) Information processing equipment, image pickup equipment, equipment control system, mobile body, information processing method and program
JP7436331B2 (en) Image processing device
CN115195591A (en) Head-up display device and display method for vehicle
WO2023189568A1 (en) Image generation device, method, and program
US11541900B2 (en) Surrounding vehicle display system
US11380106B2 (en) System, apparatus, and method for vehicle control and image sensor
US20230101538A1 (en) Method and apparatus for assisting camera-based bca
WO2016002418A1 (en) Information processing device