EP3389026A1 - Appareil et procédé d'assistance au conducteur d'un véhicule routier - Google Patents
Appareil et procédé d'assistance au conducteur d'un véhicule routier Download PDFInfo
- Publication number
- EP3389026A1 EP3389026A1 EP17166138.2A EP17166138A EP3389026A1 EP 3389026 A1 EP3389026 A1 EP 3389026A1 EP 17166138 A EP17166138 A EP 17166138A EP 3389026 A1 EP3389026 A1 EP 3389026A1
- Authority
- EP
- European Patent Office
- Prior art keywords
- vehicle
- critical
- image
- display
- bird
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 15
- 235000004522 Pentaglottis sempervirens Nutrition 0.000 claims abstract description 76
- 230000003993 interaction Effects 0.000 claims description 24
- 239000003550 marker Substances 0.000 claims description 16
- 230000001133 acceleration Effects 0.000 claims description 13
- 230000002194 synthesizing effect Effects 0.000 claims description 8
- 230000005540 biological transmission Effects 0.000 description 2
- 238000006243 chemical reaction Methods 0.000 description 1
- 239000003086 colorant Substances 0.000 description 1
- 230000000295 complement effect Effects 0.000 description 1
- 239000002131 composite material Substances 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- B60W30/00—Purposes of road vehicle drive control systems not related to the control of a particular sub-unit, e.g. of systems using conjoint control of vehicle sub-units
- B60W30/08—Active safety systems predicting or avoiding probable or impending collision or attempting to minimise its consequences
- B60W30/095—Predicting travel path or likelihood of collision
- B60W30/0953—Predicting travel path or likelihood of collision the prediction being responsive to vehicle dynamic parameters
-
- G—PHYSICS
- G08—SIGNALLING
- G08G—TRAFFIC CONTROL SYSTEMS
- G08G1/00—Traffic control systems for road vehicles
- G08G1/09—Arrangements for giving variable traffic instructions
- G08G1/0962—Arrangements for giving variable traffic instructions having an indicator mounted inside the vehicle, e.g. giving voice messages
- G08G1/0968—Systems involving transmission of navigation instructions to the vehicle
- G08G1/096855—Systems involving transmission of navigation instructions to the vehicle where the output is provided in a suitable form to the driver
-
- G—PHYSICS
- G08—SIGNALLING
- G08G—TRAFFIC CONTROL SYSTEMS
- G08G1/00—Traffic control systems for road vehicles
- G08G1/16—Anti-collision systems
- G08G1/165—Anti-collision systems for passive traffic, e.g. including static obstacles, trees
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
- B60R1/00—Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- B60W30/00—Purposes of road vehicle drive control systems not related to the control of a particular sub-unit, e.g. of systems using conjoint control of vehicle sub-units
- B60W30/08—Active safety systems predicting or avoiding probable or impending collision or attempting to minimise its consequences
- B60W30/09—Taking automatic action to avoid collision, e.g. braking and steering
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- B60W30/00—Purposes of road vehicle drive control systems not related to the control of a particular sub-unit, e.g. of systems using conjoint control of vehicle sub-units
- B60W30/08—Active safety systems predicting or avoiding probable or impending collision or attempting to minimise its consequences
- B60W30/095—Predicting travel path or likelihood of collision
- B60W30/0956—Predicting travel path or likelihood of collision the prediction being responsive to traffic or environmental parameters
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/20—Scenes; Scene-specific elements in augmented reality scenes
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
- G06V20/58—Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
-
- G—PHYSICS
- G08—SIGNALLING
- G08G—TRAFFIC CONTROL SYSTEMS
- G08G1/00—Traffic control systems for road vehicles
- G08G1/16—Anti-collision systems
- G08G1/166—Anti-collision systems for active traffic, e.g. moving vehicles, pedestrians, bikes
-
- G—PHYSICS
- G08—SIGNALLING
- G08G—TRAFFIC CONTROL SYSTEMS
- G08G1/00—Traffic control systems for road vehicles
- G08G1/16—Anti-collision systems
- G08G1/168—Driving aids for parking, e.g. acoustic or visual feedback on parking space
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N5/00—Details of television systems
- H04N5/222—Studio circuitry; Studio devices; Studio equipment
- H04N5/262—Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects
- H04N5/268—Signal distribution or switching
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
- H04N7/181—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast for receiving images from a plurality of remote sources
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- B60W2420/00—Indexing codes relating to the type of sensors based on the principle of their operation
- B60W2420/40—Photo, light or radio wave sensitive means, e.g. infrared sensors
- B60W2420/403—Image sensing, e.g. optical camera
Definitions
- the present application relates to an apparatus for road vehicle driver assistance and a method therefore.
- a parking assistance system When navigating a road vehicle, e.g. through narrow passages or when performing tight parking maneuvers therewith, it can be difficult to judge whether the vehicle will collide with surrounding objects, e.g. another vehicle.
- surrounding objects e.g. another vehicle.
- currently available parking assistance systems will sometimes show color coded lines and zones within a camera view provided by a forward or rearwards directed camera attached to the vehicle, in order to show how the vehicle will move based on a current steering command. This is done in order to aid a driver of the road vehicle to determine the distance to surrounding objects and to judge whether there is a risk that a collision with such a surrounding object will occur.
- current driver assistance systems sometimes utilizes color coded lines and zones in the camera view in order to indicate the how the vehicle will move based on the current steering commands.
- the color coded lines and zones are a means to aid the driver to determine the distance to surrounding objects and whether there is a risk of collisions with such surrounding objects.
- the camera are normally arranged to show the surrounding view behind or in front of the vehicle, usually depending on direction of the vehicle's motion, and how the vehicle will move based on the current steering command and distance to surrounding objects.
- a 360o degree bird's eye view is sometimes a preferred option that some road vehicle drivers select to use.
- the presented view usually is a composite of camera images from a plurality of cameras arranged to capture images in different directions covering the entire vehicle circumference.
- Embodiments herein aim to provide an improved road vehicle driver assistance apparatus.
- an apparatus comprising: a vehicle control command acquisition unit arranged to acquire at least one of current vehicle control commands relating to steering angle, acceleration, deceleration, velocity and shift position; an image acquisition unit arranged to acquire true-images at the circumference of the vehicle from a plurality of on-board cameras disposed on front, rear and sides of the vehicle; an image processing unit arranged to synthesize a plurality of the images taken by the plurality of on-board cameras to thereby create a bird's eye image looking down on the vehicle from above the vehicle; a display device arranged to display the created bird's eye image, an obstacle identification device arranged to identify obstacles around the vehicle, the apparatus further comprising: a processing unit arranged to calculate a maneuver corridor of the vehicle based on at least one of the acquired current vehicle control commands relating to steering angle, acceleration, deceleration and velocity, to represent the space that the vehicle extension, both front and rear, will traverse when travelling in accordance with the acquired current vehicle control commands; and in that: the processing unit further is arranged to
- a processing unit arranged to determine critical obstacles that are within a predetermined proximity of a calculated maneuver corridor, representative of the full vehicle extension, and determine corresponding critical points of the vehicle calculated to come into the predetermined proximity of the identified obstacles when traversing the calculated maneuver corridor, as well as a display device arranged to display, separate from the bird's eye image, at least one of respective images acquired from determined best camera angles and positions to navigate the respective critical obstacles provides a driver of the associated road vehicle with a clear overview of a current situation and provides immediate access to detailed views of one or more of the critical areas to navigate.
- the processing unit further is arranged to calculate the maneuver corridor of the vehicle based on the front left corner and right rear wheel of the vehicle when a current vehicle control command relating to steering angle is a control command for a right turn, and to calculate the maneuver corridor of the vehicle based on the front right corner and left rear wheel of the vehicle when a current vehicle control command relating to steering angle is a control command for a left turn.
- the provision of calculating the maneuver corridor of the vehicle based on the front left corner and right rear wheel of the vehicle for a right turn, and to calculate the maneuver corridor of the vehicle based on the front right corner and left rear wheel of the vehicle for a left turn ensures a maneuver corridor that properly reflects the space that the vehicle will actually traverse.
- the display device when no critical obstacles are determined, further is arranged to display, separate from the bird's eye image, a true-image forward view acquired from an on-board camera disposed on the front of the vehicle when a current vehicle control command relating to shift position is a control command for forward travel, and a true-image rearward view acquired from an on-board camera disposed on the rear of the vehicle when a current vehicle control command relating to shift position is a control command for reverse travel.
- the image processing unit further is arranged to synthesize symbols corresponding to the marks for critical points of the vehicle; and that the display device further is arranged to display, overlaid each image acquired from a determined best camera angle and position to navigate a critical obstacle, a synthesized symbol corresponding to the mark for a critical point of the vehicle associated with that critical obstacle.
- the processing unit further is arranged to determine, corresponding critical points of the critical objects calculated to come into the predetermined proximity of the critical point of the vehicle when traversing the calculated maneuver corridor, and the image processing unit further is arranged to synthesize marks for critical points of the critical objects, and the display device further being arranged to display, overlaid the bird's eye image, the marks for critical points of the critical objects.
- the provision of displaying, overlaid the bird's eye image, the marks for critical points of the critical objects provides a clear identification of such critical points and also for increased vehicle driver assistance through such identification.
- the image processing unit further is arranged to synthesize position markers for each respective of the on-board cameras disposed on front, rear and sides of the vehicle, and that the display device further is arranged to display, overlaid the bird's eye image, the synthesized camera position markers.
- synthesizing position markers for each respective of the on-board cameras and displaying, overlaid the bird's eye image, the synthesized camera position markers provides a clear identification of such camera positions and also for increased vehicle driver assistance through such identification.
- the image processing unit further is arranged to synthesize highlighted position markers for each respective of the on-board cameras disposed on front, rear and sides of the vehicle for which an image acquired from a determined best camera angle and position to navigate a critical obstacle is determined
- the display device further is arranged to display, overlaid the bird's eye image, the highlighted synthesized camera position markers for each displayed image acquired from a determined best camera angle and position to navigate a critical obstacle.
- the display device comprises one or more touch screen displays, and that the processing unit further is arranged to determine selective user interaction trough clicking for selecting a camera position marker on a touch screen display displaying the bird's eye image, and that the display device further is arranged to display, separate from the bird's eye image, an image acquired from the selected camera.
- the provision of the display device comprising one or more touch screen displays, and that the processing unit further is arranged to determine selective user interaction trough clicking for selecting a camera position marker on a touch screen display displaying the bird's eye image, and that the display device further is arranged to display, separate from the bird's eye image, an image acquired from the selected camera provides for manual selection of which camera view to display and thereby for an interactively adaptable vehicle driver assistance.
- the display device comprises one or more touch screen displays
- the processing unit further is arranged to determine selective user interaction trough clicking for selecting and dragging and dropping a camera position marker to a new position on a touch screen display displaying the bird's eye image, and to calculate based on such selective user interaction a new best camera angle and position
- the display device further is arranged to display, separate from the bird's eye image, an image acquired from the camera angle and position calculated based on that selective user interaction.
- the image processing unit further is arranged to synthesize, from true-images acquired from one or more of the on-board cameras disposed on front, rear and sides of the vehicle, the image acquired from the camera angle and position calculated based on the selective user interaction.
- the display device comprises one or more touch screen displays
- the processing unit further is arranged to determine selective user interaction trough clicking for selecting and dragging and dropping a camera position marker to a display position, separate from a display position used to display the bird's eye image, for selecting an image from the camera associated with that camera position marker, and that the display device further is arranged to display, to that display position, separate from the display position used to display the bird's eye image, an image acquired from the selected camera.
- the provision of selective user interaction trough clicking for selecting and dragging and dropping a camera position marker to a display position, separate from a display position used to display the bird's eye image, for displaying an image acquired from the selected camera to that display position provides for provides for intuitive manual selection of a new camera angle and position image to display and thereby for further interactively adaptable vehicle driver assistance.
- a road vehicle that comprises an apparatus as above.
- a road vehicle that comprises an apparatus as above provides detailed views useful for improved identification of one or more of the critical areas to navigate and thus for improved road vehicle driver assistance.
- a method for road vehicle driver assistance comprises: acquiring at least one of current vehicle control commands relating to steering angle, acceleration, deceleration, velocity and shift position; acquiring true-images at the circumference of the vehicle from a plurality of on-board cameras disposed on front, rear and sides of the vehicle; synthesizing a plurality of the images taken by the plurality of on-board cameras to thereby create a bird's eye image looking down on the vehicle from above the vehicle; displaying the created bird's eye image, identifying obstacles around the vehicle; calculating a maneuver corridor of the vehicle based on at least one of the acquired current vehicle control commands relating to steering angle, acceleration, deceleration and velocity, to represent the space that the vehicle extension, both front and rear, will traverse when travelling in accordance with the acquired current vehicle control commands; determining, from the identified obstacles, critical obstacles that are within a predetermined proximity of the calculated maneuver corridor as well as corresponding critical points of the vehicle calculated to come into the predetermined proximity of the identified obstacles
- a method as above provides a driver of the associated road vehicle with a clear overview of a current situation and provides immediate access to detailed views of one or more of the critical areas to navigate.
- This disclosure is based on the realization that it should be possible to provide an improved road vehicle 2 driver assistance apparatus.
- a maneuver corridor reflecting the predicted future motion of the vehicle 2 should be calculated to properly reflect the space which the vehicle 2 actually will traverse, and secondly that the provision of a 360 degree bird's eye view, even if combined with a rear or front view, is not always the best option when navigating a road vehicle 2, e.g. through narrow passages or when performing tight parking maneuvers therewith.
- an improved road vehicle 2 driver assistance apparatus 1 is provided through an apparatus 1 configured as described in the following.
- a vehicle 2 control command C acquisition unit of the apparatus is arranged to acquire at least one of current vehicle control commands C relating to steering angle, acceleration, deceleration, velocity and shift position.
- the shift position information may e.g. be acquired from a controller (not shown) arranged to control a drive system of the vehicle 2 via a CAN bus, for example.
- Such shift position information will usually, in the case of a manual transmission, contain information on the position of a shift lever (not shown) of the vehicle 2, and in the case of an automatic transmission, the position of a selector lever (not shown).
- the steering angle information may e.g. be acquired from a steering angle sensor (not shown) that senses a steering angle of the vehicle 2.
- Acceleration and deceleration information may be acquired from on-board accelerometers (not-shown).
- Vehicle 2 velocity information may e.g. be acquired from an on-board speedometer (not shown) or alternatively from a vehicle 2 borne satellite based navigational system, such as a GPS system, if available.
- An image acquisition unit 4 of the apparatus 1 is arranged to acquire true-images at the circumference of the vehicle 2 from a plurality of on-board cameras 5 disposed on front, rear and sides of the vehicle 2, e.g. a front camera 5a, a rear camera 5b, a right side camera 5c, and a left side camera 5d that are attached on the front, the rear, the right side and the left side, respectively, of the vehicle 2, as illustrated in figure 2 .
- the on-board cameras 5 may e.g. be composed of wide-angle CCD or CMOS cameras, each of which has an angle of view, for example, of approximately 180 degrees mounted on appropriate spots of the vehicle 2 so as to be capable of taking images of the entire area surrounding the vehicle 2.
- An image processing unit 6 of the apparatus 1 is arranged to synthesize a plurality of the images taken by the plurality of on-board cameras 5 to thereby create a bird's eye image 7 looking down on the vehicle 2 from above the vehicle 2, usually a computer graphic image 7 of the vehicle 2, as illustrated in figure 3a .
- the bird's eye view image may e.g. be generated by performing the viewpoint conversion for the images taken by the on-board cameras 5 to one another, thereby creating a bird's eye image 7 looking down on the entire periphery of the vehicle 2 as a center from a virtual viewpoint above the vehicle 2.
- Such a bird's eye view image 7 is generated using well-known techniques and thus not described in more detail herein.
- the apparatus 1 further comprises a display device 8 arranged to display the created bird's eye image 7 to first position on a display 9 arranged in a driver compartment of the vehicle 2, such as e.g. a liquid crystal display or similar, which may be arranged as e.g. a center stack display, a head-up-display (HUD) or as part of an instrument cluster display.
- a display device 8 arranged to display the created bird's eye image 7 to first position on a display 9 arranged in a driver compartment of the vehicle 2, such as e.g. a liquid crystal display or similar, which may be arranged as e.g. a center stack display, a head-up-display (HUD) or as part of an instrument cluster display.
- a display 9 arranged in a driver compartment of the vehicle 2
- a display 9 arranged in a driver compartment of the vehicle 2
- a display 9 arranged in a driver compartment of the vehicle 2
- a display 9 arranged in a driver compartment of the vehicle 2
- An obstacle 11 identification device 10 is arranged to identify obstacles 11 around the vehicle 2.
- the obstacles 11 around the vehicle 2 may be identified e.g. through image processing for identifying objects from the images acquired around the vehicle 2, which are taken by the plurality of on-board cameras 5.
- obstacles 11 around the vehicle 2 may be identified using on-board sensors (not shown), such as e.g. one or more ultrasonic sensors arranged to output an ultrasonic wave to the periphery of the vehicle 2 and detect a reflected wave thereof, radar or lidar sensors operating similarly but relying on electromagnetic waves or laser light.
- the apparatus 1 further comprises a processing unit 12, e.g. comprising one or more microprocessors, arranged to calculate a maneuver corridor 13 of the vehicle 2 based on at least one of the acquired current vehicle control commands C relating to steering angle, acceleration, deceleration and velocity.
- the maneuver corridor 13 is calculated to represent the space that the vehicle 2 extension, both front and rear, will traverse when travelling in accordance with the acquired current vehicle control commands C, i.e. a band-like track corresponding to a real width traversed by the vehicle 2.
- the processing unit 12 is further arranged to determine, from the identified obstacles 11, critical obstacles 11 that are within a predetermined proximity of the calculated maneuver corridor 13, e.g. within 0.3 m thereof or other suitable predetermined distance.
- the processing unit 12 is further arranged to determine corresponding critical points 15 of the vehicle 2 calculated to come into the predetermined proximity of the identified obstacles 11 when traversing the calculated maneuver corridor 13.
- the image processing unit 6 is further arranged to synthesize borders 13a, 13b (shown as dashed lines in the figures) of the calculated maneuver corridor 13, e.g. based at the rear of vehicle 2 if the vehicle 2 is in forward motion, and based at the front of the vehicle 2 if the vehicle 2 is in rearward motion, representations of the critical obstacles 11, and marks 14 for critical points 15 of the vehicle 2, and the display device 8 is further arranged to display, overlaid the bird's eye image 7, the synthesized borders 13a, 13b of the calculated maneuver corridor 13, representations of the critical obstacles 11, and the marks 14 for critical points 15 of the vehicle 2.
- the borders 13a, 13b of the calculated maneuver corridor 13, the representations of the critical obstacles 11 as well as the marks 14 for critical points 15 of the vehicle 2 may e.g. be presented using graphics in different colors, having different outlines or different appearances.
- the processing unit 12 of the apparatus 1 is further arranged to determine, from the true-images acquired at the circumference of the vehicle 2, best camera angles and positions to navigate critical points 15 of the vehicle 2 past critical obstacles 11 and the display device 8 is further arranged to display, separate from the bird's eye image 7, at least one image 16 acquired from a determined best camera angle and position to navigate a critical obstacle 11.
- Separate from the bird's eye image 7 is meant to include displaying the at least one image 16 acquired from a determined best camera angle and position to navigate a critical obstacle 11 to a separate position, display area, at one and the same display screen 9, i.e. next to the bird's eye view, but also to include displaying the at least one image 16 acquired from a determined best camera angle and position to navigate a critical obstacle 11 to a separate position, display area, at one or more additional display screens 9 (not shown).
- a driver of the associated road vehicle 2 can be provided with a clear overview of a current situation and immediate access to detailed views 16 of one or more of the critical areas to navigate.
- the processing unit 12 is, according to some embodiments, further arranged to calculate the maneuver corridor 13 of the vehicle 2 based on the front left corner and right rear wheel of the vehicle 2 when a current vehicle 2 control command C relating to steering angle is a control command C for a right turn, and to calculate the maneuver corridor 13 of the vehicle 2 based on the front right corner and left rear wheel of the vehicle 2 when a current vehicle 2 control command C relating to steering angle is a control command C for a left turn.
- a maneuver corridor 13 that properly reflects the space that the vehicle 2 will actually traverse.
- the display device 8 is further arranged to display, separate from the bird's eye image 7, a true-image forward view acquired from an on-board camera 5a disposed on the front of the vehicle 2 when a current vehicle 2 control command C relating to shift position is a control command C for forward travel, and a true-image rearward view acquired from an on-board camera 5b disposed on the rear of the vehicle 2 when a current vehicle 2 control command C relating to shift position is a control command C for reverse travel.
- the image processing unit 6 is further arranged to synthesize symbols corresponding to the marks 14 for critical points 15 of the vehicle 2.
- the display device 8 is further arranged to display, overlaid each image 16 acquired from a determined best camera angle and position to navigate a critical obstacle 11, a synthesized symbol 17 corresponding to the mark 14 for a critical point 15 of the vehicle 2 associated with that critical obstacle 11. This is done in order to facilitate identification of which critical point 15 of the vehicle 2 the displayed view 16 relates to.
- the apparatus 1 detects that there are objects 11 close to the maneuver corridor 13, e.g. within 30 cm
- the image 16 is switched to provide a projection of the best possible camera angle and position to get past the obstacle 11.
- the camera image 16 is marked with a small symbol 17 in e.g. a corner of the image 16 with a corresponding marker 14 with the same color or shape on the vehicle 2 presented in the bird's eye image 7.
- the processing unit 12 is further arranged to determine, corresponding critical points 18 of the critical objects 11 calculated to come into the predetermined proximity of the critical point 15 of the vehicle 2 when traversing the calculated maneuver corridor 13.
- the image processing unit 6 is further arranged to synthesize marks 19 for critical points 18 of the critical objects 11, and the display device 8 is further arranged to display, overlaid the bird's eye image 7, the marks 19 for critical points 18 of the critical objects 11 which provides a clear identification of such critical points 18.
- the image processing unit 6 is further arranged to synthesize position markers 20 for each respective of the on-board cameras 5 disposed on front, rear and sides of the vehicle 2, and the display device 8 is further arranged to display, overlaid the bird's eye image 7, the synthesized camera position markers 20 to provide a clear identification of such camera positions and also increased vehicle 2 driver assistance through such identification.
- the image processing unit 6 is further arranged to synthesize highlighted position markers 20 for each respective of the on-board cameras 5 disposed on front, rear and sides of the vehicle 2 for which an image 16 acquired from a determined best camera angle and position to navigate a critical obstacle 11 is determined.
- the display device 8 is further arranged to display, overlaid the bird's eye image 7, the highlighted synthesized camera position markers 20 for each displayed image 16 acquired from a determined best camera angle and position to navigate a critical obstacle 11. This provides a clear identification of the positions of cameras 5 for which an image 16 acquired from a determined best camera angle and position to navigate a critical obstacle 11 is determined, and thereby also provides for increased vehicle 2 driver assistance through such identification.
- the display device 8 comprises one or more touch screen displays 9.
- the processing unit 12 is further arranged to determine selective user interaction trough clicking for selecting a camera position marker 20 on a touch screen display 9 displaying the bird's eye image 7, and the display device 8 is further arranged to display, separate from the bird's eye image 7, an image 16 acquired from the selected camera 5, hereby allowing manual selection of which camera view to display and thereby providing for an interactively adaptable vehicle 2 driver assistance.
- the display device 8 comprises one or more touch screen displays 9, and the processing unit 12 is further arranged to determine selective user interaction trough clicking for selecting and dragging and dropping a camera position marker 20 to a new position on a touch screen display 9 displaying the bird's eye image 7.
- the processing unit 12 is further arranged to calculate, based on such selective user interaction, a new best camera angle and position.
- the display device 8 is arranged to display, separate from the bird's eye image 7, an image 16 acquired from the camera angle and position calculated based on that selective user interaction. This provides for manual selection of such a new best camera angle and position view to display and thereby for interactively adaptable vehicle 2 driver assistance.
- the image processing unit 6 further is arranged to synthesize, from true-images acquired from one or more of the on-board cameras 5 disposed on front, rear and sides of the vehicle 2, the image 16 acquired from the camera angle and position calculated based on the selective user interaction in order to provide for full freedom in manual selection of such a new best camera angle and position synthesized view 16 to display.
- the display device 8 comprises one or more touch screen displays 9, and the processing unit 12 is further arranged to determine selective user interaction trough clicking for selecting and dragging and dropping a camera position marker 20 to a display position, separate from a display position used to display the bird's eye image 7, for selecting an image from the camera 5 associated with that camera position marker 20.
- the display device 8 is further arranged to display, to that display position, separate from the display position used to display the bird's eye image 7, an image 16 acquired from the thus selected camera 5, providing for intuitive manual selection of a new camera angle and position image 16 to display.
- a road vehicle 2 as illustrated in figure 4 , which comprises an apparatus 1 as above.
- Such a road vehicle 2 will thus be able to provide detailed views useful for improved identification of one or more of the critical areas to navigate.
- the method comprises: 21 - acquiring at least one of current vehicle 2 control commands C relating to steering angle, acceleration, deceleration, velocity and shift position; 22- acquiring true-images at the circumference of the vehicle 2 from a plurality of on-board cameras 5 disposed on front, rear and sides of the vehicle 2; 23- synthesizing a plurality of the images taken by the plurality of on-board cameras 5 to thereby create a bird's eye image 7 looking down on the vehicle 2 from above the vehicle 2; 24- displaying the created bird's eye image 7; 25- identifying obstacles 11 around the vehicle 2; 26- calculating a maneuver corridor 13 of the vehicle 2 based on at least one of the acquired current vehicle 2 control commands C relating to steering angle, acceleration, deceleration and velocity, to represent the space that the vehicle 2 extension, both front and rear, will traverse when travelling in accordance with the acquired current vehicle 2 control commands C; 27- determining, from the identified obstacles 11, critical obstacles 11 that are within a predetermined proximity of the calculated maneuver corridor 13 as well as
- a method as above provides a driver of the associated road vehicle 2 with a clear overview of a current situation and provides immediate access to detailed views 16 of one or more of the critical areas to navigate.
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Mechanical Engineering (AREA)
- Transportation (AREA)
- Automation & Control Theory (AREA)
- Radar, Positioning & Navigation (AREA)
- Signal Processing (AREA)
- Theoretical Computer Science (AREA)
- Remote Sensing (AREA)
- Closed-Circuit Television Systems (AREA)
- Traffic Control Systems (AREA)
- Image Processing (AREA)
- Fittings On The Vehicle Exterior For Carrying Loads, And Devices For Holding Or Mounting Articles (AREA)
Priority Applications (3)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
EP17166138.2A EP3389026A1 (fr) | 2017-04-12 | 2017-04-12 | Appareil et procédé d'assistance au conducteur d'un véhicule routier |
CN201810296248.5A CN109367536B (zh) | 2017-04-12 | 2018-04-04 | 用于道路车辆驾驶员辅助的设备及方法 |
US15/944,964 US10692372B2 (en) | 2017-04-12 | 2018-04-04 | Appartus and method for road vehicle driver assistance |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
EP17166138.2A EP3389026A1 (fr) | 2017-04-12 | 2017-04-12 | Appareil et procédé d'assistance au conducteur d'un véhicule routier |
Publications (1)
Publication Number | Publication Date |
---|---|
EP3389026A1 true EP3389026A1 (fr) | 2018-10-17 |
Family
ID=58547343
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
EP17166138.2A Pending EP3389026A1 (fr) | 2017-04-12 | 2017-04-12 | Appareil et procédé d'assistance au conducteur d'un véhicule routier |
Country Status (3)
Country | Link |
---|---|
US (1) | US10692372B2 (fr) |
EP (1) | EP3389026A1 (fr) |
CN (1) | CN109367536B (fr) |
Cited By (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN111845725A (zh) * | 2019-04-30 | 2020-10-30 | 北京车和家信息技术有限公司 | 图像显示方法、装置、车辆及计算机可读存储介质 |
CN113243028A (zh) * | 2019-01-30 | 2021-08-10 | 沃尔沃卡车集团 | 用于调整视觉辅助***的显示视图的方法 |
GB2609482A (en) * | 2021-08-05 | 2023-02-08 | Continental Automotive Gmbh | Method and system for creating a virtual lane for a vehicle |
Families Citing this family (9)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN109691088B (zh) * | 2016-08-22 | 2022-04-15 | 索尼公司 | 图像处理设备、图像处理方法、以及程序 |
US11186295B2 (en) * | 2017-12-22 | 2021-11-30 | Veripat, LLC | Vehicle with escape feature using synthesized vehicle view |
CN109624854A (zh) * | 2018-12-03 | 2019-04-16 | 浙江明航智能科技有限公司 | 一种适用于特种车辆的360°全景辅助可视*** |
JP7016900B2 (ja) * | 2019-04-11 | 2022-02-07 | 三菱ロジスネクスト株式会社 | 制御装置、制御方法及びプログラム |
EP3722522B1 (fr) * | 2019-04-11 | 2024-03-27 | Mitsubishi Logisnext Co., Ltd. | Dispositif de commande, procédé de commande et programme |
CN110435648B (zh) * | 2019-07-26 | 2021-02-26 | 中国第一汽车股份有限公司 | 车辆的行驶控制方法、装置、车辆和存储介质 |
CN110789533B (zh) * | 2019-09-25 | 2021-08-13 | 华为技术有限公司 | 一种数据呈现的方法及终端设备 |
CN112631307A (zh) * | 2020-12-28 | 2021-04-09 | 西咸新区鸿通管廊投资有限公司 | 一种轨道式巡检机器人的多传感器辅助定位模块及方法 |
CN117278697B (zh) * | 2023-11-21 | 2024-03-15 | 中汽传媒(天津)有限公司 | 一种用于汽车碰撞全景视频采集的***和方法 |
Citations (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
EP1725035A1 (fr) * | 2005-05-20 | 2006-11-22 | Nissan Motor Co., Ltd. | Dispositif et procédé d'affichage d'image pour automobile |
EP1731366A1 (fr) * | 2005-06-07 | 2006-12-13 | Nissan Motor Company Limited | Appareil et procédé d'affichage d'images |
US20150073664A1 (en) * | 2013-09-12 | 2015-03-12 | Ford Global Technologies, Llc | Collision warning for a vehicle |
EP2927863A1 (fr) * | 2012-11-27 | 2015-10-07 | Clarion Co., Ltd. | Dispositif de traitement d'image embarqué |
US20160236616A1 (en) * | 2014-04-25 | 2016-08-18 | Komatsu Ltd. | Surroundings monitoring system, work vehicle, and surroundings monitoring method |
US20170026618A1 (en) * | 2011-06-07 | 2017-01-26 | Komatsu Ltd. | Perimeter monitoring device for work vehicle |
Family Cites Families (22)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP4071463B2 (ja) * | 2001-07-16 | 2008-04-02 | 株式会社デンソー | 車両周辺画像処理装置 |
JP4005483B2 (ja) * | 2002-11-20 | 2007-11-07 | 日産自動車株式会社 | 車線逸脱防止装置 |
JP3823924B2 (ja) * | 2003-01-31 | 2006-09-20 | 日産自動車株式会社 | 車両挙動制御装置 |
JP3870911B2 (ja) * | 2003-02-10 | 2007-01-24 | 日産自動車株式会社 | 車線逸脱防止装置 |
JP3900099B2 (ja) * | 2003-03-20 | 2007-04-04 | 日産自動車株式会社 | 車線逸脱防止装置 |
JP2004322787A (ja) * | 2003-04-23 | 2004-11-18 | Nissan Motor Co Ltd | 車線逸脱防止装置 |
JP3979339B2 (ja) * | 2003-05-12 | 2007-09-19 | 日産自動車株式会社 | 車線逸脱防止装置 |
DE602004027699D1 (de) * | 2003-11-26 | 2010-07-29 | Nissan Motor | Spurhaltevorrichtung für ein Kraftfahrzeug |
JP4389567B2 (ja) * | 2003-12-03 | 2009-12-24 | 日産自動車株式会社 | 車線逸脱防止装置 |
US8805601B2 (en) * | 2006-02-28 | 2014-08-12 | Toyota Jidosha Kabushiki Kaisha | Object path prediction method, apparatus, and program, and automatic operation system |
DE102010002105A1 (de) * | 2010-02-18 | 2011-08-18 | Robert Bosch GmbH, 70469 | Verfahren zur Unterstützung eines Fahrers eines Fahrzeugs bei einem Fahrmanöver |
WO2012120076A1 (fr) * | 2011-03-09 | 2012-09-13 | Continental Teves Ag & Co. Ohg | Dispositif de sécurité pour véhicule automobile et procédé permettant de faire fonctionner un véhicule automobile |
US8775006B2 (en) * | 2011-07-14 | 2014-07-08 | GM Global Technology Operations LLC | System and method for enhanced vehicle control |
US8849515B2 (en) * | 2012-07-24 | 2014-09-30 | GM Global Technology Operations LLC | Steering assist in driver initiated collision avoidance maneuver |
US9499168B2 (en) * | 2013-01-09 | 2016-11-22 | Mitsubishi Electric Corporation | Vehicle periphery display device |
US9207095B2 (en) * | 2013-03-15 | 2015-12-08 | Harman International Industries, Incorporated | Integrated navigation and collision avoidance systems |
DE102013205882A1 (de) * | 2013-04-03 | 2014-10-09 | Robert Bosch Gmbh | Verfahren und Vorrichtung zum Führen eines Fahrzeugs im Umfeld eines Objekts |
JP6138655B2 (ja) * | 2013-10-10 | 2017-05-31 | 日立オートモティブシステムズ株式会社 | 車両の運動制御装置 |
US9199668B2 (en) * | 2013-10-28 | 2015-12-01 | GM Global Technology Operations LLC | Path planning for evasive steering maneuver employing a virtual potential field technique |
US9174672B2 (en) * | 2013-10-28 | 2015-11-03 | GM Global Technology Operations LLC | Path planning for evasive steering maneuver in presence of target vehicle and surrounding objects |
GB2560110B (en) * | 2015-05-18 | 2019-08-28 | Roadmetric Ltd | Detection and documentation of tailgating and speeding violations |
DE102015226217A1 (de) * | 2015-12-21 | 2017-06-22 | Robert Bosch Gmbh | Verfahren und System zum assistierten Notbremsen |
-
2017
- 2017-04-12 EP EP17166138.2A patent/EP3389026A1/fr active Pending
-
2018
- 2018-04-04 US US15/944,964 patent/US10692372B2/en active Active
- 2018-04-04 CN CN201810296248.5A patent/CN109367536B/zh active Active
Patent Citations (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
EP1725035A1 (fr) * | 2005-05-20 | 2006-11-22 | Nissan Motor Co., Ltd. | Dispositif et procédé d'affichage d'image pour automobile |
EP1731366A1 (fr) * | 2005-06-07 | 2006-12-13 | Nissan Motor Company Limited | Appareil et procédé d'affichage d'images |
US20170026618A1 (en) * | 2011-06-07 | 2017-01-26 | Komatsu Ltd. | Perimeter monitoring device for work vehicle |
EP2927863A1 (fr) * | 2012-11-27 | 2015-10-07 | Clarion Co., Ltd. | Dispositif de traitement d'image embarqué |
US20150073664A1 (en) * | 2013-09-12 | 2015-03-12 | Ford Global Technologies, Llc | Collision warning for a vehicle |
US20160236616A1 (en) * | 2014-04-25 | 2016-08-18 | Komatsu Ltd. | Surroundings monitoring system, work vehicle, and surroundings monitoring method |
Cited By (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN113243028A (zh) * | 2019-01-30 | 2021-08-10 | 沃尔沃卡车集团 | 用于调整视觉辅助***的显示视图的方法 |
CN113243028B (zh) * | 2019-01-30 | 2023-07-28 | 沃尔沃卡车集团 | 用于调整视觉辅助***的显示视图的方法 |
CN111845725A (zh) * | 2019-04-30 | 2020-10-30 | 北京车和家信息技术有限公司 | 图像显示方法、装置、车辆及计算机可读存储介质 |
GB2609482A (en) * | 2021-08-05 | 2023-02-08 | Continental Automotive Gmbh | Method and system for creating a virtual lane for a vehicle |
Also Published As
Publication number | Publication date |
---|---|
CN109367536B (zh) | 2021-10-22 |
US10692372B2 (en) | 2020-06-23 |
CN109367536A (zh) | 2019-02-22 |
US20180301029A1 (en) | 2018-10-18 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US10692372B2 (en) | Appartus and method for road vehicle driver assistance | |
US11486726B2 (en) | Overlaying additional information on a display unit | |
EP3272586B1 (fr) | Véhicule de travail | |
JP5212748B2 (ja) | 駐車支援装置 | |
US10789490B2 (en) | Method for calculating a display of additional information for an advertisement, a display unit, apparatus for carrying out the method, and transportation vehicle and computer program | |
EP3650285B1 (fr) | Procédé et dispositif d'aide au stationnement | |
EP2464113B1 (fr) | Dispositif de génération d'images périphériques du véhicule | |
CN102740056B (zh) | 图像显示*** | |
US20060080005A1 (en) | Method for displaying a vehicle driving space | |
US20190031206A1 (en) | Viewing System With Field Of Vision Superimposition Depending On The Driving Situation | |
US9845092B2 (en) | Method and system for displaying probability of a collision | |
EP3971006B1 (fr) | Procédé de commande d'affichage et dispositif de commande d'affichage | |
CN113784861B (zh) | 显示控制方法及显示控制装置 | |
CN108791062B (zh) | 动态信息***及操作方法 | |
WO2012039234A1 (fr) | Dispositif d'assistance au stationnement | |
JP2007272350A (ja) | 車両用運転支援装置 | |
US20150054636A1 (en) | Method for assisting a driver of a motor vehicle | |
JP5654284B2 (ja) | 車両用ディスプレイ装置 | |
JP2012006590A (ja) | 自動車の運転者を支援するための方法 | |
JP5213578B2 (ja) | 運転支援装置 | |
JP5617396B2 (ja) | 運転支援装置 | |
JP2005069800A (ja) | 車両用表示装置 | |
CN113386783B (zh) | 用于机动车辆中的自动拖车倒车***的方法和设备 | |
US20210178969A1 (en) | Vehicle surrounding area monitoring device, vehicle surrounding area monitoring method, vehicle, and storage medium storing program for the vehicle surrounding area monitoring device | |
JP7400242B2 (ja) | 車両用表示制御装置および車両用表示制御方法 |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PUAI | Public reference made under article 153(3) epc to a published international application that has entered the european phase |
Free format text: ORIGINAL CODE: 0009012 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: THE APPLICATION HAS BEEN PUBLISHED |
|
AK | Designated contracting states |
Kind code of ref document: A1 Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR |
|
AX | Request for extension of the european patent |
Extension state: BA ME |
|
RBV | Designated contracting states (corrected) |
Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: REQUEST FOR EXAMINATION WAS MADE |
|
17P | Request for examination filed |
Effective date: 20190422 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: EXAMINATION IS IN PROGRESS |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: EXAMINATION IS IN PROGRESS |
|
17Q | First examination report despatched |
Effective date: 20210518 |