WO2019151156A1 - Object detection device and parking assistance device - Google Patents
Object detection device and parking assistance device Download PDFInfo
- Publication number
- WO2019151156A1 WO2019151156A1 PCT/JP2019/002568 JP2019002568W WO2019151156A1 WO 2019151156 A1 WO2019151156 A1 WO 2019151156A1 JP 2019002568 W JP2019002568 W JP 2019002568W WO 2019151156 A1 WO2019151156 A1 WO 2019151156A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- road surface
- dimensional coordinate
- coordinate information
- vehicle
- value
- Prior art date
Links
Images
Classifications
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
- B60R21/00—Arrangements or fittings on vehicles for protecting or preventing injuries to occupants or pedestrians in case of accidents or other traffic risks
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01B—MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
- G01B11/00—Measuring arrangements characterised by the use of optical techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/50—Depth or shape recovery
- G06T7/55—Depth or shape recovery from multiple images
- G06T7/579—Depth or shape recovery from multiple images from motion
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/60—Analysis of geometric attributes
Definitions
- the present disclosure relates to an object detection device configured to detect an object existing around the host vehicle by being mounted on the host vehicle, and a parking support device including the object detection device.
- Patent Document 1 Various devices that acquire three-dimensional coordinates of an object around a vehicle using an image captured by a camera attached to the vehicle are known (see, for example, Patent Document 1).
- this indication provides the composition which can detect the object better than before even if the reflected image of the object existing around the vehicle is generated on the road surface.
- the object detection device is configured to detect an object present around the host vehicle by being mounted on the host vehicle.
- This object detection device Based on the image information acquired by an imaging unit provided to acquire image information corresponding to an image around the host vehicle, a ground plane of the host vehicle at a feature point on the object is a plane.
- a coordinate acquisition unit provided to acquire three-dimensional coordinate information including a ground height value that is a height with reference to a reference plane that is the plane when assumed; For the three-dimensional coordinate information acquired by the coordinate acquisition unit and having a negative value lower than the reference surface, the ground high value is corrected to be a positive value higher than the reference surface.
- a coordinate correction unit provided as follows: It has.
- the parking support device is configured to support parking of the host vehicle.
- This parking assistance device The object detection device; Based on the detection result by the object detection device, a parking area detector provided to detect a parking area in which the host vehicle can be parked, It has.
- the coordinate acquisition unit acquires the three-dimensional coordinate information at the feature point on the object based on the image information acquired by the imaging unit.
- the three-dimensional coordinate information includes the ground high value.
- the ground height value is a height based on the reference plane.
- the reference plane is a plane when the ground plane of the host vehicle is assumed to be a plane.
- the image information corresponding to the reflected image can be acquired by the imaging unit.
- the ground high value is the negative value lower than the reference plane.
- the two-dimensional coordinate information other than the ground high value in the three-dimensional coordinate information can be an accurate value corresponding to the relative position of the object with respect to the host vehicle.
- the coordinate correction unit is configured such that the ground height value is higher than the reference surface with respect to the three-dimensional coordinate information in which the ground height value is the negative value lower than the reference surface. Correct so that In this configuration, the three-dimensional coordinate information in which the ground high value corresponding to the reflected image is the negative value is corrected to the positive value, so that it is not removed as noise, but the detection of the object. Can be actively utilized for. Therefore, according to such a configuration, even if the reflected image of the object existing around the vehicle is generated on the road surface, the object can be detected better than before.
- reference numerals in parentheses attached to each element indicate an example of a correspondence relationship between the element and specific means described in an embodiment described later. Therefore, the present disclosure is not limited by the description of the above reference numerals.
- FIG. 2 is a functional block diagram illustrating a schematic system configuration of the vehicle illustrated in FIG. 1.
- FIG. 3 is a block diagram illustrating a schematic functional configuration of a control unit illustrated in FIG. 2. It is the schematic for demonstrating the principle of operation of the object detection apparatus shown by FIG. 3 is a flowchart illustrating an operation example of the object detection device illustrated in FIG. 1. It is a flowchart which shows an example of the road surface reflection determination in the object detection apparatus shown by FIG. It is a flowchart which shows another example of the road surface reflection determination in the object detection apparatus shown by FIG. 6 is a flowchart showing still another example of road surface reflection determination in the object detection apparatus shown in FIG. 1.
- a vehicle 10 is a so-called four-wheeled vehicle and includes a vehicle body 11 having a substantially rectangular shape in plan view.
- a vehicle center axis X an imaginary straight line passing through the center of the vehicle 10 in the vehicle width direction and parallel to the vehicle full length direction of the vehicle 10 is referred to as a vehicle center axis X.
- the vehicle full length direction is a direction orthogonal to the vehicle width direction and orthogonal to the vehicle height direction.
- the vehicle height direction is a direction that defines the vehicle height of the vehicle 10, and is a direction parallel to the direction of gravity when the vehicle 10 is placed on a horizontal plane.
- the full length direction of the vehicle is the vertical direction in the figure
- the vehicle width direction is the horizontal direction in the figure.
- the “front”, “rear”, “left”, and “right” in the vehicle 10 are defined as indicated by arrows in FIG. That is, the vehicle full length direction is synonymous with the front-rear direction.
- the vehicle width direction is synonymous with the left-right direction. Note that the vehicle height direction may not be parallel to the gravitational action direction depending on the mounting condition or traveling condition of the vehicle 10.
- a front bumper 13 is attached to the front surface portion 12 which is the front end portion of the vehicle body 11.
- a rear bumper 15 is attached to a rear surface portion 14 which is a rear end portion of the vehicle body 11.
- a door panel 17 is attached to the side surface portion 16 of the vehicle body 11. In the specific example shown in FIG. 1, a total of four door panels 17 are provided, two on each side.
- a door mirror 18 is attached to each of the pair of left and right door panels 17 on the front side.
- the vehicle 10 is equipped with an object detection device 20.
- the object detection device 20 is configured to detect the object B existing outside the vehicle 10 by being mounted on the vehicle 10.
- the vehicle 10 equipped with the object detection device 20 may be abbreviated as “own vehicle”.
- the object detection device 20 includes an imaging unit 21, a sonar sensor 22, a radar sensor 23, a vehicle speed sensor 24, a shift position sensor 25, a steering angle sensor 26, a control unit 27, and a display unit 28. And an audio output unit 29.
- an imaging unit 21 a sonar sensor 22, a radar sensor 23, a vehicle speed sensor 24, a shift position sensor 25, a steering angle sensor 26, a control unit 27, and a display unit 28.
- an audio output unit 29 the details of each part which constitutes object detection device 20 are explained. For simplification of illustration, the electrical connection relationship between the respective parts constituting the object detection device 20 is omitted in FIG.
- the imaging unit 21 is provided so as to capture an image around the host vehicle and acquire image information corresponding to the image.
- the imaging unit 21 is a digital camera device and includes an image sensor such as a CCD.
- CCD Charge Coupled Device.
- the vehicle 10 is equipped with a plurality of imaging units 21, that is, a front camera CF, a rear camera CB, a left camera CL, and a right camera CR.
- a singular expression “imaging unit 21” or “a plurality of imaging units” 21 may be used.
- the front camera CF is provided so as to acquire image information corresponding to an image ahead of the host vehicle. Specifically, in the present embodiment, the front camera CF is attached to a room mirror (not shown) disposed in the vehicle interior of the vehicle 10.
- the rear camera CB is mounted on the rear surface portion 14 of the vehicle body 11 so as to acquire image information corresponding to an image behind the host vehicle.
- the left camera CL is attached to the left door mirror 18 so as to acquire image information corresponding to the left image of the host vehicle.
- the right camera CR is attached to the right door mirror 18 so as to obtain image information corresponding to the right image of the host vehicle.
- Each of the plurality of imaging units 21 is electrically connected to the control unit 27. That is, each of the plurality of imaging units 21 acquires image information under the control of the control unit 27 and transmits the acquired image information to the control unit 27.
- the sonar sensor 22 outputs a signal corresponding to the distance to the object B by transmitting a search wave toward the outside of the host vehicle and receiving a received wave including a reflected wave of the search wave from the wall surface of the object B. It is provided as follows. Specifically, in the present embodiment, the sonar sensor 22 is a so-called ultrasonic sensor, and is configured to transmit an exploration wave that is an ultrasonic wave and to receive a reception wave including the ultrasonic wave.
- the object detection device 20 includes at least one sonar sensor 22.
- a plurality of sonar sensors 22 are attached to the vehicle body 11.
- the plurality of sonar sensors 22 are each shifted from the vehicle center axis X to either side in the vehicle width direction.
- at least a part of the plurality of sonar sensors 22 is provided so as to transmit an exploration wave along a direction intersecting the vehicle center axis X.
- the front bumper 13 is equipped with a first front sonar SF1, a second front sonar SF2, a third front sonar SF3, and a fourth front sonar SF4 as sonar sensors 22.
- the rear bumper 15 is equipped with a first rear sonar SR1, a second rear sonar SR2, a third rear sonar SR3, and a fourth rear sonar SR4 as sonar sensors 22.
- a first side sonar SS1, a second side sonar SS2, a third side sonar SS3, and a fourth side sonar SS4 as sonar sensors 22 are mounted on the side surface portion 16 of the vehicle body 11.
- a singular expression “sonar sensor 22” or “a plurality of sonar sensors” The expression “22” may be used.
- One sonar sensor 22 is called a “first sonar sensor”, another one sonar sensor 22 is called a “second sonar sensor”, and “direct wave” and “indirect wave” are expressed as follows: Defined in A received wave that is received by the first sonar sensor and that is caused by a reflected wave from the object B of the exploration wave transmitted from the first sonar sensor is referred to as a “direct wave”. On the other hand, the received wave received by the second sonar sensor and caused by the reflected wave from the object B of the exploration wave transmitted from the first sonar sensor is referred to as “indirect wave”.
- the first front sonar SF1 is provided at the left end of the front surface of the front bumper 13 so as to transmit an exploration wave to the left front of the host vehicle.
- the second front sonar SF2 is provided at the right end of the front surface of the front bumper 13 so as to transmit an exploration wave to the right front of the host vehicle.
- the first front sonar SF1 and the second front sonar SF2 are arranged symmetrically with respect to the vehicle center axis X.
- the third front sonar SF3 and the fourth front sonar SF4 are arranged in the vehicle width direction at positions near the center on the front surface of the front bumper 13.
- the third front sonar SF3 is disposed between the first front sonar SF1 and the vehicle center axis X in the vehicle width direction so as to transmit an exploration wave substantially in front of the host vehicle.
- the fourth front sonar SF4 is disposed between the second front sonar SF2 and the vehicle center axis X in the vehicle width direction so as to transmit an exploration wave substantially in front of the host vehicle.
- the third front sonar SF3 and the fourth front sonar SF4 are arranged symmetrically with respect to the vehicle center axis X.
- the first front sonar SF1 and the third front sonar SF3 are arranged at different positions in plan view. Further, the first front sonar SF1 and the third front sonar SF3 that are adjacent to each other in the vehicle width direction have a positional relationship in which the reflected wave from the object B of the exploration wave transmitted by one can be received as the received wave at the other. Is provided.
- the first front sonar SF1 is arranged to be able to receive both a direct wave corresponding to the exploration wave transmitted by itself and an indirect wave corresponding to the exploration wave transmitted by the third front sonar SF3.
- the third front sonar SF3 is arranged to be able to receive both a direct wave corresponding to the exploration wave transmitted by itself and an indirect wave corresponding to the exploration wave transmitted by the first front sonar SF1.
- the third front sonar SF3 and the fourth front sonar SF4 are arranged at different positions in plan view. Further, the third front sonar SF3 and the fourth front sonar SF4 adjacent to each other in the vehicle width direction have a positional relationship in which the reflected wave from the object B of the exploration wave transmitted by one can be received as the received wave in the other. Is provided.
- the second front sonar SF2 and the fourth front sonar SF4 are arranged at different positions in plan view. Further, the second front sonar SF2 and the fourth front sonar SF4 that are adjacent to each other in the vehicle width direction are in a positional relationship such that a reflected wave from the object B of the exploration wave transmitted by one can be received as a received wave on the other Is provided.
- the first rear sonar SR1 is provided at the left end of the rear surface of the rear bumper 15 so as to transmit an exploration wave to the left rear of the host vehicle.
- the second rear sonar SR2 is provided at the right end portion on the rear surface of the rear bumper 15 so as to transmit an exploration wave to the right rear of the host vehicle.
- the first rear sonar SR1 and the second rear sonar SR2 are arranged symmetrically with respect to the vehicle center axis X.
- the third rear sonar SR3 and the fourth rear sonar SR4 are arranged in the vehicle width direction at a position closer to the center on the rear surface of the rear bumper 15.
- the third rear sonar SR3 is disposed between the first rear sonar SR1 and the vehicle center axis X in the vehicle width direction so as to transmit an exploration wave substantially behind the host vehicle.
- the fourth rear sonar SR4 is disposed between the second rear sonar SR2 and the vehicle center axis X in the vehicle width direction so as to transmit an exploration wave substantially behind the host vehicle.
- the third rear sonar SR3 and the fourth rear sonar SR4 are arranged symmetrically with respect to the vehicle center axis X.
- the first rear sonar SR1 and the third rear sonar SR3 are arranged at different positions in plan view. Further, the first rear sonar SR1 and the third rear sonar SR3 adjacent to each other in the vehicle width direction are provided in a positional relationship such that a reflected wave from the object B of the exploration wave transmitted by one can be received as a received wave in the other. ing.
- the first rear sonar SR1 is disposed so as to be able to receive both a direct wave corresponding to the exploration wave transmitted by itself and an indirect wave corresponding to the exploration wave transmitted by the third rear sonar SR3.
- the third rear sonar SR3 is arranged to be able to receive both a direct wave corresponding to the exploration wave transmitted by itself and an indirect wave corresponding to the exploration wave transmitted by the first rear sonar SR1.
- the third rear sonar SR3 and the fourth rear sonar SR4 are arranged at different positions in plan view. Further, the third rear sonar SR3 and the fourth rear sonar SR4 which are adjacent to each other in the vehicle width direction are provided in a positional relationship such that the reflected wave from the object B of the exploration wave transmitted by one can be received as the received wave in the other. ing.
- the second rear sonar SR2 and the fourth rear sonar SR4 are arranged at different positions in plan view. Further, the second rear sonar SR2 and the fourth rear sonar SR4 which are adjacent to each other in the vehicle width direction are provided in such a positional relationship that a reflected wave from the object B of the exploration wave transmitted by one can be received as a received wave in the other. ing.
- the first side sonar SS1, the second side sonar SS2, the third side sonar SS3, and the fourth side sonar SS4 are configured to transmit an exploration wave to the side of the host vehicle from the vehicle side surface that is the outer surface of the side surface portion 16. Is provided.
- the first side sonar SS1, the second side sonar SS2, the third side sonar SS3, and the fourth side sonar SS4 are provided so as to receive only direct waves.
- the first side sonar SS1 is arranged between the left-side door mirror 18 and the first front sonar SF1 in the front-rear direction so as to transmit an exploration wave to the left of the host vehicle.
- the second side sonar SS2 is disposed between the door mirror 18 on the right side and the second front sonar SF2 in the front-rear direction so as to transmit an exploration wave to the right of the host vehicle.
- the first side sonar SS1 and the second side sonar SS2 are provided symmetrically with respect to the vehicle center axis X.
- the third side sonar SS3 is disposed between the left rear door panel 17 and the first rear sonar SR1 in the front-rear direction so as to transmit an exploration wave to the left of the host vehicle.
- the fourth side sonar SS4 is disposed between the right rear door panel 17 and the second rear sonar SR2 in the front-rear direction so as to transmit a search wave to the right of the host vehicle.
- the third side sonar SS3 and the fourth side sonar SS4 are provided symmetrically with respect to the vehicle center axis X.
- Each of the plurality of sonar sensors 22 is electrically connected to the control unit 27. That is, each of the plurality of sonar sensors 22 transmits a search wave under the control of the control unit 27, generates a signal corresponding to the reception result of the received wave, and transmits the signal to the control unit 27.
- Information included in the signal corresponding to the reception result of the reception wave is hereinafter referred to as “reception information”.
- the reception information includes information related to the reception intensity of the received wave and distance measurement information.
- the “ranging information” is information related to the distance between each of the plurality of sonar sensors 22 and the object B. Specifically, for example, the distance measurement information includes information related to a time difference from the transmission of the exploration wave to the reception of the reception wave.
- the radar sensor 23 is a laser radar sensor or a millimeter wave radar sensor that transmits and receives radar waves, and is mounted on the front surface portion 12 of the vehicle body 11.
- the radar sensor 23 is configured to output a signal corresponding to the position of the reflection point and the relative speed.
- the “reflection point” is a point where the radar wave is reflected on the surface of the object B.
- the “relative speed” is the relative speed of the object B that reflects the reflection point, that is, the radar wave, with respect to the host vehicle.
- the vehicle speed sensor 24, the shift position sensor 25, and the rudder angle sensor 26 are electrically connected to the control unit 27.
- the vehicle speed sensor 24 is provided so as to generate a signal corresponding to the traveling speed of the host vehicle and transmit the signal to the control unit 27.
- the traveling speed of the host vehicle is hereinafter simply referred to as “vehicle speed”.
- the shift position sensor 25 is provided to generate a signal corresponding to the shift position of the host vehicle and transmit the signal to the control unit 27.
- the steering angle sensor 26 is provided so as to generate a signal corresponding to the steering angle of the host vehicle and transmit the signal to the control unit 27.
- the control unit 27 is disposed inside the vehicle body 11.
- the control unit 27 is a so-called in-vehicle microcomputer and includes a CPU, a ROM, a RAM, a nonvolatile RAM, and the like (not shown).
- the non-volatile RAM is, for example, a flash ROM.
- the CPU, ROM, RAM, and nonvolatile RAM of the control unit 27 are hereinafter simply referred to as “CPU”, “ROM”, “RAM”, and “nonvolatile RAM”.
- the control unit 27 is configured such that various control operations can be realized by the CPU reading and executing the program from the ROM or the nonvolatile RAM.
- This program includes a program corresponding to each routine described later.
- the RAM and the nonvolatile RAM are configured to be able to temporarily store processing data when the CPU executes a program.
- various data used when executing the program is stored in advance in the ROM or the nonvolatile RAM.
- Various types of data include, for example, initial values, look-up tables, maps, and the like.
- the control unit 27 performs an object detection operation based on signals and information received from each of the plurality of sonar sensors 22, each of the plurality of imaging units 21, the vehicle speed sensor 24, the shift position sensor 25, the steering angle sensor 26, and the like. Is configured to do.
- the control unit 27 controls the operations of the display unit 28 and the audio output unit 29 to perform a notification operation associated with the object detection state.
- the display unit 28 and the audio output unit 29 are disposed in the vehicle interior of the vehicle 10.
- the display unit 28 and the audio output unit 29 are electrically connected to the control unit 27.
- the display unit 28 is configured to perform a notification operation associated with the object detection state by display using a display screen or an indicator.
- the sound output unit 29 is configured to perform a notification operation associated with the object detection state by sound output.
- a raindrop sensor 31 and a wiper mechanism 41 are mounted on the vehicle 10.
- the raindrop sensor 31 is provided so as to output a signal corresponding to the raindrop attached to the raindrop detection region in the vehicle body 11.
- the wiper mechanism 41 is driven based on the raindrop signal output from the raindrop sensor 31.
- the vehicle 10 is equipped with a parking assistance device 50.
- the parking assist device 50 is configured to assist parking of the host vehicle.
- the object detection device 20 is provided so as to constitute a part of the parking assistance device 50.
- the control unit 27 that forms the main part of the object detection device 20 is an in-vehicle microcomputer that constitutes a parking assistance ECU, and is configured to control the overall operation of the parking assistance device 50.
- ECU is an abbreviation for Electronic Control Unit.
- the parking assistance apparatus 50 is comprised so that the automatic parking of the own vehicle is realizable.
- the parking assist device 50 includes a power control ECU 51, a braking control ECU 52, a steering control ECU 53, and an input operation unit 54 in addition to the object detection device 20.
- the power control ECU 51 is provided to control the travel output of the vehicle 10 by controlling the operation of a travel motor and / or engine (not shown).
- the braking control ECU 52 is provided so as to control the braking force of the vehicle 10 by controlling the driving of a brake actuator (not shown).
- the steering control ECU 53 is provided so as to control the steering amount of the vehicle 10 by controlling the driving of a steering motor (not shown).
- the input operation unit 54 is disposed in the passenger compartment of the vehicle 10.
- the input operation unit 54 includes switches and / or a voice input unit for receiving an input operation by an operator who is an occupant of the vehicle 10.
- control unit 27 includes an object recognition unit 270, a parking area detection unit 271, a route calculation unit 272, and a control amount calculation unit 273 as functional configurations realized on the microcomputer. Have.
- object recognition unit 270 the control unit 27 includes an object recognition unit 270, a parking area detection unit 271, a route calculation unit 272, and a control amount calculation unit 273 as functional configurations realized on the microcomputer. Have.
- control amount calculation unit 273 the functional configuration of the control unit 27 in the present embodiment will be described.
- the object recognition unit 270 is based on the image information acquired using the imaging unit 21, the distance information acquired using the sonar sensor 22 or the like, and the traveling state information of the host vehicle acquired using the vehicle speed sensor 24 or the like. The three-dimensional shape of the object B existing around the host vehicle is recognized. Details of the object recognition unit 270 will be described later.
- the parking area detection unit 271 is provided to detect a parking area around the host vehicle based on the detection result by the object recognition unit 270.
- the “parking area” is an area where the host vehicle can be parked.
- the route calculation unit 272 is provided so as to calculate the guidance route based on the relative position between the parking area and the host vehicle detected by the parking area detection unit 271 and the recognition result by the object recognition unit 270.
- the “guidance route” is a route that guides the host vehicle to the detected parking area, that is, a travel route to the parking area of the host vehicle.
- the control amount calculation unit 273 calculates the vehicle control amount based on the relative position between the parking area detected by the parking area detection unit 271 and the host vehicle and the guidance route calculated by the route calculation unit 272. Is provided. “Vehicle control amount” is a travel parameter necessary for traveling the host vehicle to the parking area.
- the “travel parameters” include travel output, shift position, braking amount, and steering amount.
- the object recognizing unit 270 is configured to recognize the object B existing around the host vehicle using a moving stereo method, specifically, a so-called monocular moving stereo method.
- Moving stereo is also referred to as SFM.
- SFM is an abbreviation for Structure from Motion. That is, the object recognition unit 270 exists in the direction in which the imaging unit 21 is directed based on the history of image information in one imaging unit 21 directed in a specific direction and the amount of movement of the host vehicle.
- the object B is recognized.
- the object recognition unit 270 recognizes the three-dimensional shape of the object B existing in front of the host vehicle based on the history of the image information acquired using the front camera CF and the movement amount of the host vehicle. It is supposed to be. Similarly, the object recognition unit 270 recognizes the three-dimensional shape of the object B existing behind the host vehicle based on the history of image information acquired using the rear camera CB and the movement amount of the host vehicle. It has become. Similarly, the object recognition unit 270 recognizes the three-dimensional shape of the object B existing on the left side of the host vehicle based on the history of image information acquired using the left camera CL and the movement amount of the host vehicle. It is like that. Similarly, the object recognition unit 270 recognizes the three-dimensional shape of the object B present on the right side of the host vehicle based on the history of image information acquired using the right camera CR and the movement amount of the host vehicle. It is like that.
- the object recognition unit 270 includes an image information acquisition unit 701, a movement amount acquisition unit 702, a coordinate acquisition unit 703, a distance acquisition unit 704, a road surface state determination unit 705, a road surface reflection determination unit 706, and a coordinate correction unit 707. And have.
- the details of the object recognition unit 270 will be described with reference to FIGS.
- FIG. 4 shows an example of an image captured by the imaging unit 21.
- the road surface RS is assumed to be a reflective road surface.
- the reflective road surface is a road surface RS in such a state that the reflected image VR of the object B is projected to the extent that it can be acquired by the imaging unit 21.
- a road surface RS or the like in which a water pool has occurred corresponds to a reflective road surface.
- the real image RR and the reflected image VR of the pole-shaped object B are within the field of view of the imaging unit 21 in addition to the diamond-shaped road marking M.
- the image information acquisition unit 701 receives image information acquired using the imaging unit 21 from the imaging unit 21 and stores the image information in time series.
- the movement amount acquisition unit 702 acquires the movement direction and movement amount of the host vehicle based on the outputs of the vehicle speed sensor 24, the shift position sensor 25, and the rudder angle sensor 26.
- the coordinate acquisition unit 703 is based on the image information stored in the image information acquisition unit 701, the movement direction and the movement amount acquired by the movement amount acquisition unit 702, and the three-dimensional coordinate information on the feature point FP on the object B. To get to.
- the feature point FP is a characteristic point or pixel in the image.
- the feature point FP is, for example, a point having a large brightness difference from the surroundings, or a point that can be stably extracted because of a feature in the shape.
- the feature point FP may include a corner portion of the object B, a boundary portion between the object B and the surroundings, and the like.
- the feature point FP can be extracted, for example, by detecting a pixel in which the gradient value of the gradation change is two-dimensionally increased using a known Harris operator. For the Harris operator, see, for example, the following documents: C.HARRIS and M.STEPHENS, “A combined corner and edge detector.”, Proc. Alvey Vision Conf., Pp.147-151,1988.
- the three-dimensional coordinate information at the extracted feature point FP can be acquired by the moving stereo method described above.
- Such three-dimensional coordinate information includes the ground high value.
- the ground height is a height with reference to the reference plane.
- the reference plane is the plane when the ground plane of the host vehicle is assumed to be a plane.
- the ground high value is 0 when it is the same as the reference plane, a positive value when it is higher than the reference plane, and a negative value when it is lower than the reference plane.
- the feature point FP on the real image RR in which the ground high value is a positive value is referred to as “ground detection point FP1”.
- the feature point FP where the ground high value is a negative value is referred to as “underground detection point FP2”.
- the feature point FP on the reflected image VR becomes the underground detection point FP2.
- the distance acquisition unit 704 acquires the distance from the object B based on the output of the sonar sensor 22. That is, the distance acquisition unit 704 stores the distance information corresponding to the distance measuring points in time series.
- the “ranging point” is a point on the surface of the object B that is estimated to have reflected the exploration wave transmitted from the sonar sensor 22.
- the road surface state determination unit 705 determines the state of the road surface RS around the vehicle 10. Specifically, in the present embodiment, the road surface state determination unit 705 determines whether or not the road surface RS may be a reflective road surface based on the output of the raindrop sensor 31 and / or the operating state of the wiper mechanism 41. It is to judge whether.
- the road surface reflection determination unit 706 determines whether or not the three-dimensional coordinate information obtained by the coordinate acquisition unit 703 and having a negative ground high value is based on the reflected image VR displayed on the road surface RS. It has become.
- the road surface reflection determination unit 706 performs a determination operation when the road surface state determination unit 705 determines that the road surface RS may be a reflective road surface that reflects the reflected image VR of the object B. It is supposed to be. A specific example of actual determination contents by the road surface reflection determination unit 706 will be described later.
- the coordinate correcting unit 707 corrects the three-dimensional coordinate information acquired by the coordinate acquiring unit 703 so that the ground high value becomes a negative value so that the ground high value becomes a positive value.
- the coordinate correction unit 707 determines that the three-dimensional coordinate is determined when the road surface reflection determination unit 706 determines that the three-dimensional coordinate information having a negative ground high value is based on the reflected image VR by the reflective road surface.
- the ground height value in the coordinate information is corrected to a positive value. Specifically, for example, the coordinate correction unit 707 reverses the sign to a positive value while maintaining the absolute value of the ground high value at the underground detection point FP2.
- the control unit 27 constituting the parking support ECU operates as a parking support operation when an occupant (for example, a driver) of the host vehicle operates the input operation unit 54. To start.
- the image information acquired by the imaging unit 21 is sequentially transmitted from the imaging unit 21 to the control unit 27.
- the image information received by the control unit 27 is acquired by the image information acquisition unit 701 and stored in time series.
- the distance measurement information acquired by the sonar sensor 22 is sequentially transmitted from the sonar sensor 22 to the control unit 27.
- the distance measurement information received by the control unit 27 is acquired by the distance acquisition unit 704 and stored in time series.
- the movement amount acquisition unit 702 acquires the movement direction and movement amount of the host vehicle based on the outputs of the vehicle speed sensor 24, the shift position sensor 25, and the rudder angle sensor 26.
- the coordinate acquisition unit 703 performs feature points FP on the object B based on the image information stored in the image information acquisition unit 701 in time series and the movement direction and movement amount of the host vehicle acquired by the movement amount acquisition unit 702.
- the three-dimensional coordinate information at is acquired. That is, the three-dimensional coordinate information at the feature point FP on the object B is acquired based on the history of the image information acquired by the imaging unit 21 accompanying the movement of the host vehicle.
- a reflected image VR of the object B existing around the host vehicle may be generated on the reflective road surface.
- image information corresponding to the reflected image VR can be acquired by the imaging unit 21.
- the ground high value is a negative value. That is, the three-dimensional coordinate information corresponding to the underground detection point FP2 that is the feature point FP on the reflected image VR has a ground high value with a negative sign.
- the feature point FP where the ground high value is a negative value is not necessarily based on the reflected image VR generated on the reflective road surface. That is, for example, the feature point FP corresponding to a hole such as a manhole or a concave portion such as a groove formed on the road surface RS also has a negative ground high value.
- the surface of the water film acts as a reflecting mirror. That is, the reflected image VR generated on the water film is a mirror image of the real image RR of the object B. Therefore, normally, the ground detection point FP1 that is the feature point FP on the real image RR and the underground detection point FP2 that is the feature point FP on the reflected image VR corresponding to the real image RR are detected as a pair.
- the two-dimensional coordinate information excluding the ground high value becomes substantially the same value for the real image RR and the reflected image VR.
- the two-dimensional coordinate information other than the ground high value in the three-dimensional coordinate information corresponding to the underground detection point FP2 can be an accurate value corresponding to the relative position of the object B with respect to the host vehicle.
- the ground detection point FP1 which is the feature point FP on the real image RR
- the underground detection point FP2 which is the feature point FP on the reflected image VR corresponding to the real image RR
- the coordinate acquisition unit 703 acquires three-dimensional coordinate information with a positive ground height within a predetermined range from a position obtained by inverting the sign of the ground high value in the three-dimensional coordinate information with a negative ground high value. Should be. Therefore, in this case, the road surface reflection determination unit 706 can determine that the three-dimensional coordinate information having a negative ground high value is based on the reflected image VR. According to this determination method, only the underground detection point FP2 corresponding to the actually existing object B can be set as the correction target by the coordinate correction unit 707, and noise detection is reduced.
- the underground detection point FP2 appears below the ground detection point FP1, that is, on the imaging unit 21 side. More specifically, the underground detection point FP2 exists on the virtual line segment VL connecting the predetermined reference point RP and the ground detection point FP1 or in the vicinity of the virtual line segment VL.
- the reference point RP is, for example, a point corresponding to the imaging unit 21 on the road surface RS, specifically, an intersection of a straight line passing through the arrangement position of the imaging unit 21 while being orthogonal to the reference plane and the reference plane. Alternatively, the reference point RP is a point where, for example, a plurality of straight lines orthogonal to the reference plane converge in the captured image.
- the road surface reflection determination unit 706 can determine that the three-dimensional coordinate information whose ground high value is a negative value is based on the reflected image VR.
- the road surface reflection determination unit 706 can determine that the three-dimensional coordinate information having a negative ground high value is based on the reflected image VR. is there. Thereby, it becomes possible to determine more accurately that the underground detection point FP2 is based on the reflected image VR.
- Condition X In the captured image, that is, image information, the underground detection point FP2 exists on or near the virtual line segment VL.
- the distance to the underground detection point FP2 can be grasped.
- the underground detection point FP2 is a feature point FP on the reflected image VR due to road surface reflection of the real image RR of the object B
- distance measurement information using the sonar sensor 22 is acquired for the object B corresponding to the real image RR.
- the distance measurement point corresponding to the distance from the underground detection point FP2 is recognized by the distance acquisition unit 704. That is, in this case, there is a feature point FP corresponding to the three-dimensional coordinate information having a negative ground high value within a predetermined distance range from the distance acquired by the distance acquisition unit 704.
- the road surface reflection determination unit 706 can determine that the three-dimensional coordinate information having a negative ground high value is based on the reflected image VR. Thereby, it becomes possible to determine more accurately that the underground detection point FP2 is based on the reflected image VR.
- the coordinate correcting unit 707 determines that the ground high value is a negative value. Is corrected so that the ground high value becomes a positive value. Specifically, for example, the coordinate correction unit 707 inverts the sign of the ground high value to the three-dimensional coordinate information corresponding to the underground detection point FP2 while maintaining the absolute value of the ground high value.
- the three-dimensional coordinate information with a negative ground high value corresponding to the reflected image VR is corrected to a positive value, so that it is not removed as noise but actively detected for the object B. It can be utilized. Therefore, according to such a configuration, even if a reflection image VR of the object B existing around the host vehicle is generated on the road surface RS, the object B can be detected better than before.
- the number of obstacle detection points can be increased by using the detection points obtained by reflection, which had been removed as noise in the past, to improve the obstacle judgment performance in rainy weather. Can do.
- the coordinate correction unit 707 when the road surface reflection determination unit 706 determines that the three-dimensional coordinate information with a negative ground high value is based on the reflected image VR, the ground surface in the three-dimensional coordinate information. Correct high value to positive value. Accordingly, erroneous correction of the three-dimensional coordinate information corresponding to the feature point FP on the concave portion such as a hole such as a manhole or a groove can be satisfactorily suppressed.
- the road surface reflection determination unit 706 performs a determination operation when the road surface state determination unit 705 determines that the road surface RS is a reflective road surface. Therefore, generation of unnecessary determination processing load in a situation where no reflective road surface is generated can be avoided satisfactorily. In addition, erroneous correction of the three-dimensional coordinate information corresponding to the feature point FP on the concave portion such as a hole such as a manhole or a groove can be suppressed satisfactorily.
- the coordinate acquisition unit 703 acquires three-dimensional coordinate information based on the history of image information acquired by the imaging unit 21 and the movement amount of the host vehicle acquired by the movement amount acquisition unit 702. To do. Therefore, the so-called monocular moving stereo method can be used. That is, the object B existing in the direction in which the imaging unit 21 is directed is recognized based on the history of image information in one imaging unit 21 directed in a specific direction and the movement amount of the host vehicle. It becomes possible. Thereby, apparatus cost can be reduced rather than the case where the stereo camera method is used.
- the recognition result of the object B by the object detection device 20 is used for the parking support operation by the parking support device 50. Therefore, according to the present embodiment, it is possible to realize a good parking assist operation regardless of conditions such as the weather.
- FIG. 5 is a flowchart showing an example of the detection operation of the object B based on the image information acquired by the imaging unit 21.
- the object detection routine shown in FIG. 5 is repeatedly activated by the CPU at predetermined time intervals after a predetermined activation condition is established.
- step 501 the CPU acquires image information from the imaging unit 21 and stores it in the nonvolatile RAM in time series. In addition, the CPU holds the latest image information acquired this time in the RAM.
- the processing in step 501 corresponds to the operation of the image information acquisition unit 701.
- step 502 the CPU extracts a feature point FP in the captured image based on the acquired image information.
- step 503 the CPU acquires the amount of movement of the host vehicle.
- the CPU acquires three-dimensional coordinate information corresponding to the extracted feature point FP and temporarily stores it in the RAM.
- step 503 corresponds to the operation of the movement amount acquisition unit 702, and the remainder corresponds to the operation of the coordinate acquisition unit 703.
- steps 502 to 504 correspond to known moving stereo or SFM. For this reason, the details of the specific processing contents in steps 502 to 504 are omitted in this specification.
- step 510 the CPU acquires road surface state information.
- the road surface state information is information related to the state of the road surface RS around the host vehicle, that is, information for determining the state of the road surface RS.
- the CPU acquires the output of the raindrop sensor 31 and / or the operating state of the wiper mechanism 41.
- step 520 the CPU determines whether or not the road surface RS may be a reflective road surface.
- the processing of step 510 and step 520 corresponds to the operation of the road surface state determination unit 705.
- step 530 the CPU stores the three-dimensional coordinate information once held in the RAM in the nonvolatile RAM.
- the processing in step 530 corresponds to the operation of the coordinate acquisition unit 703. In this case, the correction of the ground high value is not executed for the three-dimensional coordinate information acquired in step 503. That is, the three-dimensional coordinate information once held in the RAM in step 504 is stored in the nonvolatile RAM as it is in step 530.
- step 540 the CPU determines whether or not there is three-dimensional coordinate information having a negative ground high value.
- step 540 NO
- the CPU advances the process to step 530 and then ends this routine once.
- step 530 the CPU stores the three-dimensional coordinate information once held in the RAM in the nonvolatile RAM. Also in this case, the correction of the ground high value is not performed on the three-dimensional coordinate information acquired in step 503. That is, the three-dimensional coordinate information once held in the RAM in step 504 is stored in the nonvolatile RAM as it is in step 530.
- step 540 YES
- the CPU advances the process to step 550, advances the process to step 530, and then ends this routine once.
- step 550 the CPU executes coordinate correction processing.
- the coordinate correction process is a process for correcting the three-dimensional coordinate information. A specific example of the processing content of step 550 will be described later.
- step 530 the CPU stores the three-dimensional coordinate information that has undergone the processing in step 550 in the nonvolatile RAM.
- FIG. 6A shows a subroutine corresponding to a specific example of the processing content of step 550. This specific example corresponds to the first determination method in the road surface reflection determination unit 706.
- step 611 the CPU extracts one underground detection point FP2 and reads out its three-dimensional coordinate information.
- step 621 the CPU inverts the sign positive while maintaining the absolute value with respect to the ground high value in the read three-dimensional coordinate information.
- step 631 the CPU searches for the ground detection point FP1 corresponding to the three-dimensional coordinate information whose sign is inverted in step 621. Specifically, the CPU determines whether or not the feature point FP on the real image RR exists within a predetermined range centered on the position corresponding to the three-dimensional coordinate information whose sign is inverted in step 621. Thereafter, the CPU proceeds with the process to step 641.
- step 641 as a result of the search in step 631, the CPU determines whether or not there is a set of the ground detection point FP1 and the underground detection point FP2 having a relationship in which the sign of the ground high value is inverted. If there is a set of the ground detection point FP1 and the underground detection point FP2 in the relationship in which the sign of the ground high value is inverted, the determination in step 641 is “YES”.
- the underground detection point FP2 is a feature point FP on the reflected image VR generated by road surface reflection of the real image RR of the object B. That is, the underground detection point FP2 having three-dimensional coordinate information whose ground high value is negative is based on the reflected image VR by road surface reflection.
- step 641 determines whether the road surface reflection determination unit 706.
- step 650 the CPU corrects the three-dimensional coordinate information obtained by the coordinate obtaining unit 703 so that the ground high value becomes a negative value so that the ground high value becomes a positive value.
- the processing in step 650 corresponds to the operation of the coordinate correction unit 707.
- the determination in step 641 is “NO”.
- the underground detection point FP2 is not the feature point FP on the reflected image VR generated by the road surface reflection of the real image RR of the object B. That is, the underground detection point FP2 is a feature point FP corresponding to a hole or a recess, and the ground high value should not be corrected to a positive value. Therefore, in this case, the CPU skips the process of step 650 and advances the process to step 660.
- FIG. 6B shows a subroutine corresponding to another specific example of the processing content of step 550. This specific example corresponds to the second determination method in the road surface reflection determination unit 706.
- step 612 the CPU extracts one underground detection point FP2 and reads out its three-dimensional coordinate information.
- the processing content of step 612 in FIG. 6B is the same as the processing content of step 611 in FIG. 6A.
- step 622 the CPU calculates a virtual half line from the end point to the underground detection point FP2 with the reference point RP as an end point. Subsequently, in step 632, the CPU searches for the ground detection point FP1 on the virtual half line calculated in step 622 or in the vicinity of the virtual half line. Thereafter, the CPU proceeds with the process to step 642.
- step 642 the CPU determines whether or not the corresponding ground detection point FP1 has been found as a result of the search in step 632.
- the determination in step 642 is “YES”.
- the underground detection point FP2 exists on the virtual line segment VL connecting the reference point RP and the ground detection point FP1 or in the vicinity of the virtual line segment VL. That is, in this case, the underground detection point FP2 having the three-dimensional coordinate information having a negative ground high value is based on the reflected image VR by road surface reflection.
- step 642 determines whether the road surface reflection determination unit 706.
- step 632 if the corresponding ground detection point FP1 is not found, the determination in step 642 is “NO”.
- the underground detection point FP2 is not the feature point FP on the reflected image VR generated by the road surface reflection of the real image RR of the object B. That is, the underground detection point FP2 is a feature point FP corresponding to a hole or a recess, and the ground high value should not be corrected to a positive value. Therefore, in this case, the CPU skips the process of step 650 and advances the process to step 660.
- the processing content of step 660 in FIG. 6B is the same as in FIG. 6A.
- FIG. 6C shows a subroutine corresponding to yet another specific example of the processing content of step 550. This specific example corresponds to the third determination method in the road surface reflection determination unit 706.
- step 613 the CPU extracts the underground detection point FP2 and reads out its three-dimensional coordinate information.
- the processing content of step 613 in FIG. 6C is the same as the processing content of step 611 in FIG. 6A and step 612 in FIG. 6B.
- step 623 the CPU reads a distance measuring point.
- step 633 the CPU compares the read distance measuring point with the underground detection point FP2. That is, the CPU collates the three-dimensional coordinate information corresponding to the distance measuring point with the three-dimensional coordinate information corresponding to the underground detection point FP2. Thereafter, the CPU proceeds with the process to step 643.
- step 643 the CPU determines whether or not the underground detection point FP2 exists within a predetermined distance range from the distance measuring point.
- the determination in step 643 is “YES”.
- the underground detection point FP2 is a feature point FP on the reflected image VR generated by the road surface reflection of the real image RR of the object B. That is, the underground detection point FP2 having three-dimensional coordinate information whose ground high value is negative is based on the reflected image VR by road surface reflection.
- step 643 determines whether the road surface reflection determination unit 706.
- step 643 If the underground detection point FP2 does not exist within a predetermined distance range from the distance measurement point, the determination in step 643 is “NO”. In this case, the underground detection point FP2 is not the feature point FP on the reflected image VR generated by the road surface reflection of the real image RR of the object B. That is, the underground detection point FP2 is a feature point FP corresponding to a hole or a recess, and the ground high value should not be corrected to a positive value. Therefore, in this case, the CPU skips the process of step 650 and advances the process to step 660.
- the processing content of step 660 in FIG. 6C is the same as in FIG. 6A and FIG. 6B.
- the vehicle 10 on which the object detection device 20 is mounted is not limited to a four-wheeled vehicle.
- the vehicle 10 on which the object detection device 20 is mounted may be a three-wheeled vehicle or a six-wheeled or eight-wheeled vehicle such as a cargo truck.
- the type of the vehicle 10 on which the object detection device 20 is mounted may be an automobile having only an internal combustion engine, an electric vehicle or a fuel cell vehicle not having an internal combustion engine, or a hybrid vehicle. There may be.
- the shape and structure of the vehicle body 11 are not limited to a box shape, that is, a substantially rectangular shape in plan view.
- the number of door panels 17 is not particularly limited.
- the application target of the object detection device 20 is not limited to parking assistance. That is, for example, the object detection device 20 can be suitably applied to semi-automatic driving or automatic driving corresponding to levels 2 to 5 in the definition of automatic driving.
- the image sensor constituting the imaging unit 21 is not limited to a CCD sensor. That is, for example, a CMOS sensor can be used instead of the CCD sensor.
- CMOS is an abbreviation for Complementary MOS.
- the front camera CF can be disposed in the vehicle interior.
- the front camera CF can be mounted in a vehicle interior, for example, a room mirror.
- the front camera CF may be one or two.
- the object detection device 20 may have a compound eye stereo camera configuration.
- the left camera CL and the right camera CR may be arranged at positions different from the door mirror 18.
- the left camera CL and the right camera CR can be omitted.
- the arrangement and number of sonar sensors 22 are not limited to the above specific example. That is, for example, referring to FIG. 1, when the third front sonar SF3 is disposed at the center position in the vehicle width direction, the fourth front sonar SF4 is omitted. Similarly, when the third rear sonar SR3 is arranged at the center position in the vehicle width direction, the fourth rear sonar SR4 is omitted. The third side sonar SS3 and the fourth side sonar SS4 may be omitted.
- the control unit 27 constitutes a main part of the object detection device 20 in the above embodiment.
- the imaging unit 21, the sonar sensor 22, the radar sensor 23, the vehicle speed sensor 24, the shift position sensor 25, the steering angle sensor 26, the display unit 28, and the audio output unit 29 are not constituent elements of the object detection device 20, It can be grasped as an incidental element of the detection device 20. That is, the configuration according to the present disclosure can be interpreted as being provided only in the control unit 27.
- control unit 27 has a configuration in which the CPU reads and starts the program from the ROM or the like.
- the present disclosure is not limited to such a configuration. That is, for example, the control unit 27 may be a digital circuit configured to be able to operate as described above, for example, an ASIC such as a gate array.
- ASIC is an abbreviation of APPLICATION SPECIFIC INTEGRATED CIRCUIT.
- ⁇ Holes such as manholes or recesses such as grooves also become obstacles during parking assistance, driving assistance, or automatic driving. This is the same as the object B that is an obstacle protruding upward from the road surface RS. For this reason, even if the ground high value is corrected to a positive value as in the case of the underground detection point FP2 on the reflected image VR with respect to the feature point FP corresponding to the hole or the recess, no particular problem occurs in parking assistance or the like. there is a possibility. Therefore, the road surface state determination unit 705 and the road surface reflection determination unit 706 can be omitted.
- the output of an acceleration sensor (not shown) can be used for acquisition of the movement amount of the host vehicle by the movement amount acquisition unit 702.
- the processing content in the coordinate acquisition unit 703 is not limited to monocular moving stereo. Specifically, for example, compound eye stereo processing or integration processing of SFM and compound eye stereo can be used. Compound eye stereo processing or integration processing of SFM and compound eye stereo is already known or known at the time of filing of the present application.
- the distance acquisition unit 704 may acquire distance measurement information based on the output of the radar sensor 23 instead of the output of the sonar sensor 22.
- the processing content in the road surface state determination unit 705 is not limited to the above example. That is, for example, the road surface state determination unit 705 determines that the road surface RS is reflective when water droplets attached to the imaging unit 21 are confirmed in the captured image by the imaging unit 21 present on the current traveling direction side of the host vehicle. It may be determined that there is a possibility of a road surface.
- the road surface state determination unit 705 may determine that the road surface RS may be a reflective road surface when the adhesion of raindrops to the radar sensor 23 is detected.
- the road surface condition determination unit 705 receives from the braking control ECU 52 that the ABS or ESC has been operated and the outside air temperature is lower than a predetermined temperature, the road surface RS may be a reflective road surface. It may be determined that there is.
- ABS is an abbreviation for Antilock Brake System.
- ESC stands for Electronic Stability Control.
- the road surface state determination unit 705 may determine the road surface state based on the combination of the above elements.
- the above elements are the output of the raindrop sensor 31, the operating state of the wiper mechanism 41, the raindrop detection state by the imaging unit 21, the raindrop detection state by the radar sensor 23, the output of the outside air temperature sensor (not shown), the ABS operation state, ESC Including operating conditions, etc.
- the road surface state determination unit 705 may determine the road surface RS around the vehicle 10 based on information (for example, weather information) acquired from outside the host vehicle using vehicle-to-vehicle communication or vehicle-to-vehicle communication. The state may be determined.
- Car-to-vehicle communication may also be referred to as “V2V”.
- V2V is an abbreviation for Vehicle-to-Vehicle.
- the car-outside communication may also be referred to as “V2I”.
- V2I is an abbreviation for Vehicle-to-roadside-Infrastructure, and is also referred to as “road-to-vehicle communication”.
- the concept encompassing both is referred to as “V2X”.
- “V2X” may also be referred to as “vehicle-to-any”.
- the determination method in the road surface reflection determination unit 706 is not limited to the above specific examples. That is, for example, the road surface reflection determination unit 706 may determine that the underground detection point FP2 is based on the reflected image VR when the underground detection point FP2 exists immediately below the ground detection point FP1. “The underground detection point FP2 exists immediately below the ground detection point FP1” means that the two-dimensional coordinate information is the same or substantially the same at the ground detection point FP1 and the underground detection point FP2. As described above, the two-dimensional coordinate information is information other than the ground high value in the three-dimensional coordinate information.
- the determination in the road surface reflection determination unit 706 may use a combination of a plurality of determination methods. That is, for example, when at least two of the following conditions (1) to (3) are satisfied, the road surface reflection determination unit 706 determines that the underground detection point FP2 having three-dimensional coordinate information having a negative ground high value is the road surface. It may be determined that it is based on the reflection image VR by reflection.
- Condition (1) A pair of ground detection point FP1 and underground detection point FP2 having a relationship in which the sign of the ground high value is inverted exists.
- Condition (2) An underground detection point FP2 exists on the virtual line segment VL connecting the reference point RP and the ground detection point FP1 or in the vicinity of the virtual line segment VL.
- Condition (3) An underground detection point FP2 exists within a predetermined distance range from the distance measuring point.
- Condition (4) An underground detection point FP2 exists immediately below the ground detection point FP1.
- Condition (2) is equivalent to the following condition (5).
- Condition (5) The ground detection point FP1 exists on or near the virtual half line from the end point to the underground detection point FP2 with the reference point RP as an end point.
- the expression “acquisition” and similar expressions such as “estimation”, “detection”, “detection”, and “calculation” can be appropriately replaced within a technically consistent range.
- the inequality sign in each determination process may be with or without an equal sign. That is, for example, “less than a predetermined value” and “below a predetermined value” can be replaced with each other within a technically consistent range.
- the elements constituting the above-described embodiment are not necessarily indispensable except for the case where it is clearly indicated that the element is essential and the case where the element is clearly considered to be essential in principle.
- numerical values such as the number, numerical value, quantity, range, etc. of a component are mentioned, the specifics are specified unless explicitly stated as being essential, and clearly limited to a specific number in principle.
- the present disclosure is not limited to any number.
- shape, direction, positional relationship, etc. of a component are mentioned, except when clearly stated as essential and in principle limited to a specific shape, direction, positional relationship, etc.
- the present disclosure is not limited to the shape, direction, positional relationship, and the like.
- modified examples are not limited to the above examples. A plurality of modifications may be combined with each other. Furthermore, all or a part of the above-described embodiment and all or a part of the modified examples can be combined with each other.
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Theoretical Computer Science (AREA)
- Geometry (AREA)
- Mechanical Engineering (AREA)
- Traffic Control Systems (AREA)
- Length Measuring Devices By Optical Means (AREA)
- Image Analysis (AREA)
Abstract
An object detection device (20) is configured to detect an object present around a vehicle when installed in said vehicle. The object detection device is provided with a coordinate acquisition unit (703) and a coordinate correction unit (707). The coordinate acquisition unit acquires, on the basis of image information acquired by an image capturing unit (21) that is provided so as to acquire image information corresponding to an image of the vehicle surroundings, three-dimensional coordinate information at feature points on the object. The three-dimensional coordinate information includes an above-ground height value. The above-ground height value indicates a height from a reference surface, said reference surface being the plane when the vehicle ground-contact surface is considered as a plane. When the three-dimensional coordinate information acquired by the coordinate acquisition unit includes an above-ground height value that is a negative value lower than the reference surface, the coordinate correction unit corrects the above-ground height value to a positive value higher than the reference surface.
Description
本出願は、2018年1月30日に出願された日本特許出願番号2018-13865号に基づくもので、ここにその記載内容が参照により組み入れられる。
This application is based on Japanese Patent Application No. 2018-13865 filed on January 30, 2018, the contents of which are incorporated herein by reference.
本開示は、自車両に搭載されることで当該自車両の周囲に存在する物体を検知するように構成された物体検知装置、および、かかる物体検知装置を備えた駐車支援装置に関する。
The present disclosure relates to an object detection device configured to detect an object existing around the host vehicle by being mounted on the host vehicle, and a parking support device including the object detection device.
車両に取り付けられたカメラにより捕捉された画像を用いて、車両周辺物の3次元座標を取得する装置が、種々知られている(例えば特許文献1等参照)。
Various devices that acquire three-dimensional coordinates of an object around a vehicle using an image captured by a camera attached to the vehicle are known (see, for example, Patent Document 1).
従来のこの種の装置においては、例えば、降雨等により路面に水溜りが発生すると、水面による反射像がカメラに写り込むことで物体検知精度が低下するという課題があった。本開示は、上記に例示した事情等に鑑みてなされたものである。すなわち、本開示は、車両の周囲に存在する物体の反射像が路面に生じても、当該物体を従来よりも良好に検知することが可能な構成を提供する。
In the conventional apparatus of this type, for example, when a puddle is generated on the road surface due to rain or the like, there is a problem that an object detection accuracy is lowered because a reflected image of the water surface is reflected on the camera. The present disclosure has been made in view of the circumstances exemplified above. That is, this indication provides the composition which can detect the object better than before even if the reflected image of the object existing around the vehicle is generated on the road surface.
本開示の1つの観点によれば、物体検知装置は、自車両に搭載されることで、当該自車両の周囲に存在する物体を検知するように構成されている。
この物体検知装置は、
前記自車両の周囲の画像に対応する画像情報を取得するように設けられた撮像部により取得された前記画像情報に基づいて、前記物体上の特徴点における、前記自車両の接地面を平面と仮定した場合の当該平面である基準面を基準とした高さである地上高値を含む3次元座標情報を取得するように設けられた、座標取得部と、
前記座標取得部によって取得された、前記地上高値が前記基準面よりも低い負値となる前記3次元座標情報に対して、前記地上高値が前記基準面よりも高い正値となるように補正するように設けられた、座標補正部と、
を備えている。
本開示の他の1つの観点によれば、駐車支援装置は、前記自車両の駐車を支援するように構成されている。
この駐車支援装置は、
前記物体検知装置と、
前記物体検知装置による検知結果に基づいて、前記自車両が駐車可能な駐車領域を検出するように設けられた、駐車領域検出部と、
を備えている。 According to one aspect of the present disclosure, the object detection device is configured to detect an object present around the host vehicle by being mounted on the host vehicle.
This object detection device
Based on the image information acquired by an imaging unit provided to acquire image information corresponding to an image around the host vehicle, a ground plane of the host vehicle at a feature point on the object is a plane. A coordinate acquisition unit provided to acquire three-dimensional coordinate information including a ground height value that is a height with reference to a reference plane that is the plane when assumed;
For the three-dimensional coordinate information acquired by the coordinate acquisition unit and having a negative value lower than the reference surface, the ground high value is corrected to be a positive value higher than the reference surface. A coordinate correction unit provided as follows:
It has.
According to another aspect of the present disclosure, the parking support device is configured to support parking of the host vehicle.
This parking assistance device
The object detection device;
Based on the detection result by the object detection device, a parking area detector provided to detect a parking area in which the host vehicle can be parked,
It has.
この物体検知装置は、
前記自車両の周囲の画像に対応する画像情報を取得するように設けられた撮像部により取得された前記画像情報に基づいて、前記物体上の特徴点における、前記自車両の接地面を平面と仮定した場合の当該平面である基準面を基準とした高さである地上高値を含む3次元座標情報を取得するように設けられた、座標取得部と、
前記座標取得部によって取得された、前記地上高値が前記基準面よりも低い負値となる前記3次元座標情報に対して、前記地上高値が前記基準面よりも高い正値となるように補正するように設けられた、座標補正部と、
を備えている。
本開示の他の1つの観点によれば、駐車支援装置は、前記自車両の駐車を支援するように構成されている。
この駐車支援装置は、
前記物体検知装置と、
前記物体検知装置による検知結果に基づいて、前記自車両が駐車可能な駐車領域を検出するように設けられた、駐車領域検出部と、
を備えている。 According to one aspect of the present disclosure, the object detection device is configured to detect an object present around the host vehicle by being mounted on the host vehicle.
This object detection device
Based on the image information acquired by an imaging unit provided to acquire image information corresponding to an image around the host vehicle, a ground plane of the host vehicle at a feature point on the object is a plane. A coordinate acquisition unit provided to acquire three-dimensional coordinate information including a ground height value that is a height with reference to a reference plane that is the plane when assumed;
For the three-dimensional coordinate information acquired by the coordinate acquisition unit and having a negative value lower than the reference surface, the ground high value is corrected to be a positive value higher than the reference surface. A coordinate correction unit provided as follows:
It has.
According to another aspect of the present disclosure, the parking support device is configured to support parking of the host vehicle.
This parking assistance device
The object detection device;
Based on the detection result by the object detection device, a parking area detector provided to detect a parking area in which the host vehicle can be parked,
It has.
上記構成においては、前記座標取得部は、前記撮像部により取得された前記画像情報に基づいて、前記物体上の前記特徴点における、前記3次元座標情報を取得する。前記3次元座標情報には、前記地上高値が含まれる。前記地上高値は、前記基準面を基準とした高さである。前記基準面は、前記自車両の前記接地面を平面と仮定した場合の、当該平面である。
前記車両の周囲に存在する前記物体の反射像が路面に生じた場合、かかる反射像に対応する前記画像情報が前記撮像部により取得され得る。この場合、前記反射像に基づいて前記座標取得部によって取得された前記3次元座標情報においては、前記地上高値が、前記基準面よりも低い前記負値となる。一方、前記3次元座標情報における、前記地上高値以外の2次元座標情報については、前記物体の前記自車両に対する相対位置に対応した正確な値となり得る。 In the above configuration, the coordinate acquisition unit acquires the three-dimensional coordinate information at the feature point on the object based on the image information acquired by the imaging unit. The three-dimensional coordinate information includes the ground high value. The ground height value is a height based on the reference plane. The reference plane is a plane when the ground plane of the host vehicle is assumed to be a plane.
When a reflected image of the object existing around the vehicle is generated on the road surface, the image information corresponding to the reflected image can be acquired by the imaging unit. In this case, in the three-dimensional coordinate information acquired by the coordinate acquisition unit based on the reflected image, the ground high value is the negative value lower than the reference plane. On the other hand, the two-dimensional coordinate information other than the ground high value in the three-dimensional coordinate information can be an accurate value corresponding to the relative position of the object with respect to the host vehicle.
前記車両の周囲に存在する前記物体の反射像が路面に生じた場合、かかる反射像に対応する前記画像情報が前記撮像部により取得され得る。この場合、前記反射像に基づいて前記座標取得部によって取得された前記3次元座標情報においては、前記地上高値が、前記基準面よりも低い前記負値となる。一方、前記3次元座標情報における、前記地上高値以外の2次元座標情報については、前記物体の前記自車両に対する相対位置に対応した正確な値となり得る。 In the above configuration, the coordinate acquisition unit acquires the three-dimensional coordinate information at the feature point on the object based on the image information acquired by the imaging unit. The three-dimensional coordinate information includes the ground high value. The ground height value is a height based on the reference plane. The reference plane is a plane when the ground plane of the host vehicle is assumed to be a plane.
When a reflected image of the object existing around the vehicle is generated on the road surface, the image information corresponding to the reflected image can be acquired by the imaging unit. In this case, in the three-dimensional coordinate information acquired by the coordinate acquisition unit based on the reflected image, the ground high value is the negative value lower than the reference plane. On the other hand, the two-dimensional coordinate information other than the ground high value in the three-dimensional coordinate information can be an accurate value corresponding to the relative position of the object with respect to the host vehicle.
そこで、上記構成においては、前記座標補正部は、前記地上高値が前記基準面よりも低い前記負値となる前記3次元座標情報に対して、前記地上高値が前記基準面よりも高い前記正値となるように補正する。かかる構成においては、前記反射像に対応する前記地上高値が前記負値の前記3次元座標情報は、前記正値に補正されることで、ノイズとして除去されるのではなく、前記物体の検知のために積極的に活用され得る。したがって、かかる構成によれば、前記車両の周囲に存在する前記物体の前記反射像が前記路面に生じても、当該物体を従来よりも良好に検知することが可能となる。
Therefore, in the above configuration, the coordinate correction unit is configured such that the ground height value is higher than the reference surface with respect to the three-dimensional coordinate information in which the ground height value is the negative value lower than the reference surface. Correct so that In this configuration, the three-dimensional coordinate information in which the ground high value corresponding to the reflected image is the negative value is corrected to the positive value, so that it is not removed as noise, but the detection of the object. Can be actively utilized for. Therefore, according to such a configuration, even if the reflected image of the object existing around the vehicle is generated on the road surface, the object can be detected better than before.
なお、出願書類における、各要素に付された括弧付きの参照符号は、同要素と後述する実施形態に記載の具体的手段との対応関係の一例を示すものである。よって、本開示は、上記の参照符号の記載によって、何ら限定されるものではない。
In the application documents, reference numerals in parentheses attached to each element indicate an example of a correspondence relationship between the element and specific means described in an embodiment described later. Therefore, the present disclosure is not limited by the description of the above reference numerals.
(実施形態)
以下、実施形態を、図面に基づいて説明する。なお、一つの実施形態に対して適用可能な各種の変形例については、当該実施形態に関する一連の説明の途中に挿入されると、当該実施形態の理解が妨げられるおそれがある。このため、変形例については、当該実施形態に関する一連の説明の途中には挿入せず、その後にまとめて説明する。 (Embodiment)
Hereinafter, embodiments will be described with reference to the drawings. In addition, about the various modifications applicable with respect to one embodiment, when it inserts in the middle of the series of description regarding the said embodiment, there exists a possibility that understanding of the said embodiment may be prevented. For this reason, about a modification, it does not insert in the middle of the series of description regarding the said embodiment, but it demonstrates collectively after that.
以下、実施形態を、図面に基づいて説明する。なお、一つの実施形態に対して適用可能な各種の変形例については、当該実施形態に関する一連の説明の途中に挿入されると、当該実施形態の理解が妨げられるおそれがある。このため、変形例については、当該実施形態に関する一連の説明の途中には挿入せず、その後にまとめて説明する。 (Embodiment)
Hereinafter, embodiments will be described with reference to the drawings. In addition, about the various modifications applicable with respect to one embodiment, when it inserts in the middle of the series of description regarding the said embodiment, there exists a possibility that understanding of the said embodiment may be prevented. For this reason, about a modification, it does not insert in the middle of the series of description regarding the said embodiment, but it demonstrates collectively after that.
(車両の全体構成)
図1を参照すると、車両10は、いわゆる四輪自動車であって、平面視にて略矩形状の車体11を備えている。以下、車両10の車幅方向における中心を通り、且つ車両10における車両全長方向と平行な仮想直線を、車両中心軸線Xと称する。車両全長方向は、車幅方向と直交し且つ車高方向と直交する方向である。車高方向は、車両10の車高を規定する方向であって、車両10を水平面に載置した場合の重力作用方向と平行な方向である。図1において、車両全長方向は図中上下方向であり、車幅方向は図中左右方向である。 (Overall configuration of vehicle)
Referring to FIG. 1, avehicle 10 is a so-called four-wheeled vehicle and includes a vehicle body 11 having a substantially rectangular shape in plan view. Hereinafter, an imaginary straight line passing through the center of the vehicle 10 in the vehicle width direction and parallel to the vehicle full length direction of the vehicle 10 is referred to as a vehicle center axis X. The vehicle full length direction is a direction orthogonal to the vehicle width direction and orthogonal to the vehicle height direction. The vehicle height direction is a direction that defines the vehicle height of the vehicle 10, and is a direction parallel to the direction of gravity when the vehicle 10 is placed on a horizontal plane. In FIG. 1, the full length direction of the vehicle is the vertical direction in the figure, and the vehicle width direction is the horizontal direction in the figure.
図1を参照すると、車両10は、いわゆる四輪自動車であって、平面視にて略矩形状の車体11を備えている。以下、車両10の車幅方向における中心を通り、且つ車両10における車両全長方向と平行な仮想直線を、車両中心軸線Xと称する。車両全長方向は、車幅方向と直交し且つ車高方向と直交する方向である。車高方向は、車両10の車高を規定する方向であって、車両10を水平面に載置した場合の重力作用方向と平行な方向である。図1において、車両全長方向は図中上下方向であり、車幅方向は図中左右方向である。 (Overall configuration of vehicle)
Referring to FIG. 1, a
車両10における「前」「後」「左」「右」を、図1中にて矢印で示された通りに定義する。すなわち、車両全長方向は、前後方向と同義である。また、車幅方向は、左右方向と同義である。なお、車高方向は、車両10の載置条件または走行条件により、重力作用方向と平行とはならない場合があり得る。
The “front”, “rear”, “left”, and “right” in the vehicle 10 are defined as indicated by arrows in FIG. That is, the vehicle full length direction is synonymous with the front-rear direction. The vehicle width direction is synonymous with the left-right direction. Note that the vehicle height direction may not be parallel to the gravitational action direction depending on the mounting condition or traveling condition of the vehicle 10.
車体11における前側の端部である前面部12には、フロントバンパー13が装着されている。車体11における後側の端部である後面部14には、リアバンパー15が装着されている。車体11における側面部16には、ドアパネル17が装着されている。図1に示す具体例においては、左右にそれぞれ2枚ずつ、合計4枚のドアパネル17が設けられている。前側の左右一対のドアパネル17のそれぞれには、ドアミラー18が装着されている。
A front bumper 13 is attached to the front surface portion 12 which is the front end portion of the vehicle body 11. A rear bumper 15 is attached to a rear surface portion 14 which is a rear end portion of the vehicle body 11. A door panel 17 is attached to the side surface portion 16 of the vehicle body 11. In the specific example shown in FIG. 1, a total of four door panels 17 are provided, two on each side. A door mirror 18 is attached to each of the pair of left and right door panels 17 on the front side.
車両10には、物体検知装置20が搭載されている。物体検知装置20は、車両10に搭載されることで、当該車両10の外側に存在する物体Bを検知するように構成されている。以下、物体検知装置20を搭載した車両10を、「自車両」と略称することがある。
The vehicle 10 is equipped with an object detection device 20. The object detection device 20 is configured to detect the object B existing outside the vehicle 10 by being mounted on the vehicle 10. Hereinafter, the vehicle 10 equipped with the object detection device 20 may be abbreviated as “own vehicle”.
具体的には、物体検知装置20は、撮像部21と、ソナーセンサ22と、レーダーセンサ23と、車速センサ24と、シフトポジションセンサ25と、舵角センサ26と、制御部27と、表示部28と、音声出力部29とを備えている。以下、物体検知装置20を構成する各部の詳細について説明する。なお、図示の簡略化のため、物体検知装置20を構成する各部の間の電気接続関係は、図1においては省略されている。
Specifically, the object detection device 20 includes an imaging unit 21, a sonar sensor 22, a radar sensor 23, a vehicle speed sensor 24, a shift position sensor 25, a steering angle sensor 26, a control unit 27, and a display unit 28. And an audio output unit 29. Hereinafter, the details of each part which constitutes object detection device 20 are explained. For simplification of illustration, the electrical connection relationship between the respective parts constituting the object detection device 20 is omitted in FIG.
撮像部21は、自車両の周囲の画像を撮像して、当該画像に対応する画像情報を取得するように設けられている。本実施形態においては、撮像部21は、デジタルカメラ装置であって、CCD等のイメージセンサを備えている。CCDはCharge Coupled Deviceの略である。
The imaging unit 21 is provided so as to capture an image around the host vehicle and acquire image information corresponding to the image. In the present embodiment, the imaging unit 21 is a digital camera device and includes an image sensor such as a CCD. CCD stands for Charge Coupled Device.
本実施形態においては、車両10には、複数の撮像部21、すなわち、フロントカメラCF、リアカメラCB、左側カメラCL、および右側カメラCRが搭載されている。フロントカメラCF、リアカメラCB、左側カメラCL、および右側カメラCRのうちの、いずれかであることを特定しない場合に、以下、「撮像部21」という単数形の表現、または「複数の撮像部21」という表現が用いられることがある。
In the present embodiment, the vehicle 10 is equipped with a plurality of imaging units 21, that is, a front camera CF, a rear camera CB, a left camera CL, and a right camera CR. In the case where it is not specified that any one of the front camera CF, the rear camera CB, the left camera CL, and the right camera CR, hereinafter, a singular expression “imaging unit 21” or “a plurality of imaging units” 21 "may be used.
フロントカメラCFは、自車両の前方の画像に対応する画像情報を取得するように設けられている。具体的には、本実施形態においては、フロントカメラCFは、車両10における車室内に配置された不図示のルームミラーに装着されている。
The front camera CF is provided so as to acquire image information corresponding to an image ahead of the host vehicle. Specifically, in the present embodiment, the front camera CF is attached to a room mirror (not shown) disposed in the vehicle interior of the vehicle 10.
リアカメラCBは、自車両の後方の画像に対応する画像情報を取得するように、車体11の後面部14に装着されている。左側カメラCLは、自車両の左方の画像に対応する画像情報を取得するように、左側のドアミラー18に装着されている。右側カメラCRは、自車両の右方の画像に対応する画像情報を取得するように、右側のドアミラー18に装着されている。
The rear camera CB is mounted on the rear surface portion 14 of the vehicle body 11 so as to acquire image information corresponding to an image behind the host vehicle. The left camera CL is attached to the left door mirror 18 so as to acquire image information corresponding to the left image of the host vehicle. The right camera CR is attached to the right door mirror 18 so as to obtain image information corresponding to the right image of the host vehicle.
複数の撮像部21の各々は、制御部27に電気接続されている。すなわち、複数の撮像部21の各々は、制御部27の制御下で画像情報を取得するとともに、取得した画像情報を制御部27に送信するようになっている。
Each of the plurality of imaging units 21 is electrically connected to the control unit 27. That is, each of the plurality of imaging units 21 acquires image information under the control of the control unit 27 and transmits the acquired image information to the control unit 27.
ソナーセンサ22は、探査波を自車両の外側に向けて発信するとともに、物体Bの壁面による探査波の反射波を含む受信波を受信することで、物体Bとの距離に対応する信号を出力するように設けられている。具体的には、本実施形態においては、ソナーセンサ22は、いわゆる超音波センサであって、超音波である探査波を発信するとともに、超音波を含む受信波を受信可能に構成されている。
The sonar sensor 22 outputs a signal corresponding to the distance to the object B by transmitting a search wave toward the outside of the host vehicle and receiving a received wave including a reflected wave of the search wave from the wall surface of the object B. It is provided as follows. Specifically, in the present embodiment, the sonar sensor 22 is a so-called ultrasonic sensor, and is configured to transmit an exploration wave that is an ultrasonic wave and to receive a reception wave including the ultrasonic wave.
物体検知装置20は、少なくとも1個のソナーセンサ22を備えている。具体的には、本実施形態においては、複数のソナーセンサ22が、車体11に装着されている。複数のソナーセンサ22は、それぞれ、車両中心軸線Xから、車幅方向におけるいずれか一方側にシフトして配置されている。また、複数のソナーセンサ22のうちの少なくとも一部は、車両中心軸線Xと交差する方向に沿って探査波を発信するように設けられている。
The object detection device 20 includes at least one sonar sensor 22. Specifically, in the present embodiment, a plurality of sonar sensors 22 are attached to the vehicle body 11. The plurality of sonar sensors 22 are each shifted from the vehicle center axis X to either side in the vehicle width direction. In addition, at least a part of the plurality of sonar sensors 22 is provided so as to transmit an exploration wave along a direction intersecting the vehicle center axis X.
具体的には、フロントバンパー13には、ソナーセンサ22としての、第一フロントソナーSF1、第二フロントソナーSF2、第三フロントソナーSF3、および第四フロントソナーSF4が装着されている。同様に、リアバンパー15には、ソナーセンサ22としての、第一リアソナーSR1、第二リアソナーSR2、第三リアソナーSR3、および第四リアソナーSR4が装着されている。
Specifically, the front bumper 13 is equipped with a first front sonar SF1, a second front sonar SF2, a third front sonar SF3, and a fourth front sonar SF4 as sonar sensors 22. Similarly, the rear bumper 15 is equipped with a first rear sonar SR1, a second rear sonar SR2, a third rear sonar SR3, and a fourth rear sonar SR4 as sonar sensors 22.
また、車体11の側面部16には、ソナーセンサ22としての、第一サイドソナーSS1、第二サイドソナーSS2、第三サイドソナーSS3、および第四サイドソナーSS4が装着されている。第一フロントソナーSF1、第二フロントソナーSF2、第三フロントソナーSF3、第四フロントソナーSF4、第一リアソナーSR1、第二リアソナーSR2、第三リアソナーSR3、第四リアソナーSR4、第一サイドソナーSS1、第二サイドソナーSS2、第三サイドソナーSS3、および第四サイドソナーSS4のうちの、いずれかであることを特定しない場合に、以下、「ソナーセンサ22」という単数形の表現、または「複数のソナーセンサ22」という表現が用いられることがある。
Also, a first side sonar SS1, a second side sonar SS2, a third side sonar SS3, and a fourth side sonar SS4 as sonar sensors 22 are mounted on the side surface portion 16 of the vehicle body 11. First front sonar SF1, second front sonar SF2, third front sonar SF3, fourth front sonar SF4, first rear sonar SR1, second rear sonar SR2, third rear sonar SR3, fourth rear sonar SR4, first side sonar SS1, In the case where it is not specified that any one of the second side sonar SS2, the third side sonar SS3, and the fourth side sonar SS4, hereinafter, a singular expression “sonar sensor 22” or “a plurality of sonar sensors” The expression “22” may be used.
或る1個のソナーセンサ22を「第一ソナーセンサ」と称し、別の1個の1個のソナーセンサ22を「第二ソナーセンサ」と称して、「直接波」および「間接波」を、以下のように定義する。第一ソナーセンサに受信される受信波であって、第一ソナーセンサから発信された探査波の物体Bによる反射波に起因する受信波を、「直接波」と称する。これに対し、第二ソナーセンサに受信される受信波であって、第一ソナーセンサから発信された探査波の物体Bによる反射波に起因する受信波を、「間接波」と称する。
One sonar sensor 22 is called a “first sonar sensor”, another one sonar sensor 22 is called a “second sonar sensor”, and “direct wave” and “indirect wave” are expressed as follows: Defined in A received wave that is received by the first sonar sensor and that is caused by a reflected wave from the object B of the exploration wave transmitted from the first sonar sensor is referred to as a “direct wave”. On the other hand, the received wave received by the second sonar sensor and caused by the reflected wave from the object B of the exploration wave transmitted from the first sonar sensor is referred to as “indirect wave”.
第一フロントソナーSF1は、自車両の左前方に探査波を発信するように、フロントバンパー13の前側表面における左端部に設けられている。第二フロントソナーSF2は、自車両の右前方に探査波を発信するように、フロントバンパー13の前側表面における右端部に設けられている。第一フロントソナーSF1と第二フロントソナーSF2とは、車両中心軸線Xを挟んで対称に配置されている。
The first front sonar SF1 is provided at the left end of the front surface of the front bumper 13 so as to transmit an exploration wave to the left front of the host vehicle. The second front sonar SF2 is provided at the right end of the front surface of the front bumper 13 so as to transmit an exploration wave to the right front of the host vehicle. The first front sonar SF1 and the second front sonar SF2 are arranged symmetrically with respect to the vehicle center axis X.
第三フロントソナーSF3と第四フロントソナーSF4とは、フロントバンパー13の前側表面における中央寄りの位置にて、車幅方向に配列されている。第三フロントソナーSF3は、自車両の略前方に探査波を発信するように、車幅方向について第一フロントソナーSF1と車両中心軸線Xとの間に配置されている。第四フロントソナーSF4は、自車両の略前方に探査波を発信するように、車幅方向について第二フロントソナーSF2と車両中心軸線Xとの間に配置されている。第三フロントソナーSF3と第四フロントソナーSF4とは、車両中心軸線Xを挟んで対称に配置されている。
The third front sonar SF3 and the fourth front sonar SF4 are arranged in the vehicle width direction at positions near the center on the front surface of the front bumper 13. The third front sonar SF3 is disposed between the first front sonar SF1 and the vehicle center axis X in the vehicle width direction so as to transmit an exploration wave substantially in front of the host vehicle. The fourth front sonar SF4 is disposed between the second front sonar SF2 and the vehicle center axis X in the vehicle width direction so as to transmit an exploration wave substantially in front of the host vehicle. The third front sonar SF3 and the fourth front sonar SF4 are arranged symmetrically with respect to the vehicle center axis X.
上記の通り、第一フロントソナーSF1と第三フロントソナーSF3とは、平面視にて互いに異なる位置に配置されている。また、車幅方向について互いに隣接する第一フロントソナーSF1と第三フロントソナーSF3とは、相互に、一方が送信した探査波の物体Bによる反射波が他方における受信波として受信可能な位置関係に設けられている。
As described above, the first front sonar SF1 and the third front sonar SF3 are arranged at different positions in plan view. Further, the first front sonar SF1 and the third front sonar SF3 that are adjacent to each other in the vehicle width direction have a positional relationship in which the reflected wave from the object B of the exploration wave transmitted by one can be received as the received wave at the other. Is provided.
すなわち、第一フロントソナーSF1は、自己が発信した探査波に対応する直接波と、第三フロントソナーSF3が発信した探査波に対応する間接波との双方を受信可能に配置されている。同様に、第三フロントソナーSF3は、自己が発信した探査波に対応する直接波と、第一フロントソナーSF1が発信した探査波に対応する間接波との双方を受信可能に配置されている。
That is, the first front sonar SF1 is arranged to be able to receive both a direct wave corresponding to the exploration wave transmitted by itself and an indirect wave corresponding to the exploration wave transmitted by the third front sonar SF3. Similarly, the third front sonar SF3 is arranged to be able to receive both a direct wave corresponding to the exploration wave transmitted by itself and an indirect wave corresponding to the exploration wave transmitted by the first front sonar SF1.
同様に、第三フロントソナーSF3と第四フロントソナーSF4とは、平面視にて互いに異なる位置に配置されている。また、車幅方向について互いに隣接する第三フロントソナーSF3と第四フロントソナーSF4とは、相互に、一方が送信した探査波の物体Bによる反射波が他方における受信波として受信可能な位置関係に設けられている。
Similarly, the third front sonar SF3 and the fourth front sonar SF4 are arranged at different positions in plan view. Further, the third front sonar SF3 and the fourth front sonar SF4 adjacent to each other in the vehicle width direction have a positional relationship in which the reflected wave from the object B of the exploration wave transmitted by one can be received as the received wave in the other. Is provided.
同様に、第二フロントソナーSF2と第四フロントソナーSF4とは、平面視にて互いに異なる位置に配置されている。また、車幅方向について互いに隣接する第二フロントソナーSF2と第四フロントソナーSF4とは、相互に、一方が送信した探査波の物体Bによる反射波が他方における受信波として受信可能な位置関係に設けられている。
Similarly, the second front sonar SF2 and the fourth front sonar SF4 are arranged at different positions in plan view. Further, the second front sonar SF2 and the fourth front sonar SF4 that are adjacent to each other in the vehicle width direction are in a positional relationship such that a reflected wave from the object B of the exploration wave transmitted by one can be received as a received wave on the other Is provided.
第一リアソナーSR1は、自車両の左後方に探査波を発信するように、リアバンパー15の後側表面における左端部に設けられている。第二リアソナーSR2は、自車両の右後方に探査波を発信するように、リアバンパー15の後側表面における右端部に設けられている。第一リアソナーSR1と第二リアソナーSR2とは、車両中心軸線Xを挟んで対称に配置されている。
The first rear sonar SR1 is provided at the left end of the rear surface of the rear bumper 15 so as to transmit an exploration wave to the left rear of the host vehicle. The second rear sonar SR2 is provided at the right end portion on the rear surface of the rear bumper 15 so as to transmit an exploration wave to the right rear of the host vehicle. The first rear sonar SR1 and the second rear sonar SR2 are arranged symmetrically with respect to the vehicle center axis X.
第三リアソナーSR3と第四リアソナーSR4とは、リアバンパー15の後側表面における中央寄りの位置にて、車幅方向に配列されている。第三リアソナーSR3は、自車両の略後方に探査波を発信するように、車幅方向について第一リアソナーSR1と車両中心軸線Xとの間に配置されている。第四リアソナーSR4は、自車両の略後方に探査波を発信するように、車幅方向について第二リアソナーSR2と車両中心軸線Xとの間に配置されている。第三リアソナーSR3と第四リアソナーSR4とは、車両中心軸線Xを挟んで対称に配置されている。
The third rear sonar SR3 and the fourth rear sonar SR4 are arranged in the vehicle width direction at a position closer to the center on the rear surface of the rear bumper 15. The third rear sonar SR3 is disposed between the first rear sonar SR1 and the vehicle center axis X in the vehicle width direction so as to transmit an exploration wave substantially behind the host vehicle. The fourth rear sonar SR4 is disposed between the second rear sonar SR2 and the vehicle center axis X in the vehicle width direction so as to transmit an exploration wave substantially behind the host vehicle. The third rear sonar SR3 and the fourth rear sonar SR4 are arranged symmetrically with respect to the vehicle center axis X.
上記の通り、第一リアソナーSR1と第三リアソナーSR3とは、平面視にて互いに異なる位置に配置されている。また、車幅方向について互いに隣接する第一リアソナーSR1と第三リアソナーSR3とは、相互に、一方が送信した探査波の物体Bによる反射波が他方における受信波として受信可能な位置関係に設けられている。
As described above, the first rear sonar SR1 and the third rear sonar SR3 are arranged at different positions in plan view. Further, the first rear sonar SR1 and the third rear sonar SR3 adjacent to each other in the vehicle width direction are provided in a positional relationship such that a reflected wave from the object B of the exploration wave transmitted by one can be received as a received wave in the other. ing.
すなわち、第一リアソナーSR1は、自己が発信した探査波に対応する直接波と、第三リアソナーSR3が発信した探査波に対応する間接波との双方を受信可能に配置されている。同様に、第三リアソナーSR3は、自己が発信した探査波に対応する直接波と、第一リアソナーSR1が発信した探査波に対応する間接波との双方を受信可能に配置されている。
That is, the first rear sonar SR1 is disposed so as to be able to receive both a direct wave corresponding to the exploration wave transmitted by itself and an indirect wave corresponding to the exploration wave transmitted by the third rear sonar SR3. Similarly, the third rear sonar SR3 is arranged to be able to receive both a direct wave corresponding to the exploration wave transmitted by itself and an indirect wave corresponding to the exploration wave transmitted by the first rear sonar SR1.
同様に、第三リアソナーSR3と第四リアソナーSR4とは、平面視にて互いに異なる位置に配置されている。また、車幅方向について互いに隣接する第三リアソナーSR3と第四リアソナーSR4とは、相互に、一方が送信した探査波の物体Bによる反射波が他方における受信波として受信可能な位置関係に設けられている。
Similarly, the third rear sonar SR3 and the fourth rear sonar SR4 are arranged at different positions in plan view. Further, the third rear sonar SR3 and the fourth rear sonar SR4 which are adjacent to each other in the vehicle width direction are provided in a positional relationship such that the reflected wave from the object B of the exploration wave transmitted by one can be received as the received wave in the other. ing.
同様に、第二リアソナーSR2と第四リアソナーSR4とは、平面視にて互いに異なる位置に配置されている。また、車幅方向について互いに隣接する第二リアソナーSR2と第四リアソナーSR4とは、相互に、一方が送信した探査波の物体Bによる反射波が他方における受信波として受信可能な位置関係に設けられている。
Similarly, the second rear sonar SR2 and the fourth rear sonar SR4 are arranged at different positions in plan view. Further, the second rear sonar SR2 and the fourth rear sonar SR4 which are adjacent to each other in the vehicle width direction are provided in such a positional relationship that a reflected wave from the object B of the exploration wave transmitted by one can be received as a received wave in the other. ing.
第一サイドソナーSS1、第二サイドソナーSS2、第三サイドソナーSS3、および第四サイドソナーSS4は、側面部16の外側表面である車両側面から探査波を自車両の側方に発信するように設けられている。第一サイドソナーSS1、第二サイドソナーSS2、第三サイドソナーSS3、および第四サイドソナーSS4は、それぞれ、直接波のみを受信可能に設けられている。
The first side sonar SS1, the second side sonar SS2, the third side sonar SS3, and the fourth side sonar SS4 are configured to transmit an exploration wave to the side of the host vehicle from the vehicle side surface that is the outer surface of the side surface portion 16. Is provided. The first side sonar SS1, the second side sonar SS2, the third side sonar SS3, and the fourth side sonar SS4 are provided so as to receive only direct waves.
第一サイドソナーSS1は、自車両の左方に探査波を発信するように、前後方向について左側のドアミラー18と第一フロントソナーSF1との間に配置されている。第二サイドソナーSS2は、自車両の右方に探査波を発信するように、前後方向について右側のドアミラー18と第二フロントソナーSF2との間に配置されている。第一サイドソナーSS1と第二サイドソナーSS2とは、車両中心軸線Xを挟んで対称に設けられている。
The first side sonar SS1 is arranged between the left-side door mirror 18 and the first front sonar SF1 in the front-rear direction so as to transmit an exploration wave to the left of the host vehicle. The second side sonar SS2 is disposed between the door mirror 18 on the right side and the second front sonar SF2 in the front-rear direction so as to transmit an exploration wave to the right of the host vehicle. The first side sonar SS1 and the second side sonar SS2 are provided symmetrically with respect to the vehicle center axis X.
第三サイドソナーSS3は、自車両の左方に探査波を発信するように、前後方向について左後側のドアパネル17と第一リアソナーSR1との間に配置されている。第四サイドソナーSS4は、自車両の右方に探査波を発信するように、前後方向について右後側のドアパネル17と第二リアソナーSR2との間に配置されている。第三サイドソナーSS3と第四サイドソナーSS4とは、車両中心軸線Xを挟んで対称に設けられている。
The third side sonar SS3 is disposed between the left rear door panel 17 and the first rear sonar SR1 in the front-rear direction so as to transmit an exploration wave to the left of the host vehicle. The fourth side sonar SS4 is disposed between the right rear door panel 17 and the second rear sonar SR2 in the front-rear direction so as to transmit a search wave to the right of the host vehicle. The third side sonar SS3 and the fourth side sonar SS4 are provided symmetrically with respect to the vehicle center axis X.
複数のソナーセンサ22の各々は、制御部27に電気接続されている。すなわち、複数のソナーセンサ22の各々は、制御部27の制御下で探査波を発信するとともに、受信波の受信結果に対応する信号を発生して制御部27に送信するようになっている。受信波の受信結果に対応する信号に含まれる情報を、以下「受信情報」と称する。受信情報には、受信波の受信強度に関連する情報、および、測距情報が含まれる。「測距情報」は、複数のソナーセンサ22の各々と物体Bとの距離に関連する情報である。具体的には、例えば、測距情報には、探査波の発信から受信波の受信までの時間差に関連する情報が含まれる。
Each of the plurality of sonar sensors 22 is electrically connected to the control unit 27. That is, each of the plurality of sonar sensors 22 transmits a search wave under the control of the control unit 27, generates a signal corresponding to the reception result of the received wave, and transmits the signal to the control unit 27. Information included in the signal corresponding to the reception result of the reception wave is hereinafter referred to as “reception information”. The reception information includes information related to the reception intensity of the received wave and distance measurement information. The “ranging information” is information related to the distance between each of the plurality of sonar sensors 22 and the object B. Specifically, for example, the distance measurement information includes information related to a time difference from the transmission of the exploration wave to the reception of the reception wave.
レーダーセンサ23は、レーダー波を送受信するレーザーレーダーセンサまたはミリ波レーダーセンサであって、車体11の前面部12に装着されている。レーダーセンサ23は、反射点の位置および相対速度に対応する信号を出力するように構成されている。「反射点」は、物体Bの表面上における、レーダー波を反射した点である。「相対速度」は、反射点すなわちレーダー波を反射した物体Bの、自車両に対する相対速度である。
The radar sensor 23 is a laser radar sensor or a millimeter wave radar sensor that transmits and receives radar waves, and is mounted on the front surface portion 12 of the vehicle body 11. The radar sensor 23 is configured to output a signal corresponding to the position of the reflection point and the relative speed. The “reflection point” is a point where the radar wave is reflected on the surface of the object B. The “relative speed” is the relative speed of the object B that reflects the reflection point, that is, the radar wave, with respect to the host vehicle.
車速センサ24、シフトポジションセンサ25、および舵角センサ26は、制御部27に電気接続されている。車速センサ24は、自車両の走行速度に対応する信号を発生して、制御部27に送信するように設けられている。自車両の走行速度を、以下単に「車速」と称する。シフトポジションセンサ25は、自車両のシフトポジションに対応する信号を発生して、制御部27に送信するように設けられている。舵角センサ26は、自車両の操舵角に対応する信号を発生して、制御部27に送信するように設けられている。
The vehicle speed sensor 24, the shift position sensor 25, and the rudder angle sensor 26 are electrically connected to the control unit 27. The vehicle speed sensor 24 is provided so as to generate a signal corresponding to the traveling speed of the host vehicle and transmit the signal to the control unit 27. The traveling speed of the host vehicle is hereinafter simply referred to as “vehicle speed”. The shift position sensor 25 is provided to generate a signal corresponding to the shift position of the host vehicle and transmit the signal to the control unit 27. The steering angle sensor 26 is provided so as to generate a signal corresponding to the steering angle of the host vehicle and transmit the signal to the control unit 27.
制御部27は、車体11の内側に配置されている。制御部27は、いわゆる車載マイクロコンピュータであって、図示しないCPU、ROM、RAM、不揮発性RAM、等を備えている。不揮発性RAMは、例えば、フラッシュROM等である。制御部27のCPU、ROM、RAMおよび不揮発性RAMを、以下単に「CPU」、「ROM」、「RAM」および「不揮発性RAM」と略称する。
The control unit 27 is disposed inside the vehicle body 11. The control unit 27 is a so-called in-vehicle microcomputer and includes a CPU, a ROM, a RAM, a nonvolatile RAM, and the like (not shown). The non-volatile RAM is, for example, a flash ROM. The CPU, ROM, RAM, and nonvolatile RAM of the control unit 27 are hereinafter simply referred to as “CPU”, “ROM”, “RAM”, and “nonvolatile RAM”.
制御部27は、CPUがROMまたは不揮発性RAMからプログラムを読み出して実行することで、各種の制御動作を実現可能に構成されている。このプログラムには、後述の各ルーチンに対応するものが含まれている。また、RAMおよび不揮発性RAMは、CPUがプログラムを実行する際の処理データを一時的に格納可能に構成されている。さらに、ROMまたは不揮発性RAMには、プログラムの実行の際に用いられる各種のデータが、あらかじめ格納されている。各種のデータには、例えば、初期値、ルックアップテーブル、マップ、等が含まれている。
The control unit 27 is configured such that various control operations can be realized by the CPU reading and executing the program from the ROM or the nonvolatile RAM. This program includes a program corresponding to each routine described later. The RAM and the nonvolatile RAM are configured to be able to temporarily store processing data when the CPU executes a program. Furthermore, various data used when executing the program is stored in advance in the ROM or the nonvolatile RAM. Various types of data include, for example, initial values, look-up tables, maps, and the like.
制御部27は、複数のソナーセンサ22の各々、複数の撮像部21の各々、車速センサ24、シフトポジションセンサ25、舵角センサ26、等から受信した信号および情報に基づいて、物体検知動作を実行するように構成されている。また、制御部27は、表示部28および音声出力部29の動作を制御することで、物体検知状態に伴う報知動作を行うようになっている。
The control unit 27 performs an object detection operation based on signals and information received from each of the plurality of sonar sensors 22, each of the plurality of imaging units 21, the vehicle speed sensor 24, the shift position sensor 25, the steering angle sensor 26, and the like. Is configured to do. In addition, the control unit 27 controls the operations of the display unit 28 and the audio output unit 29 to perform a notification operation associated with the object detection state.
表示部28および音声出力部29は、車両10における車室内に配置されている。また、表示部28および音声出力部29は、制御部27に電気接続されている。表示部28は、物体検知状態に伴う報知動作を、表示画面またはインジケータによる表示により行うように構成されている。音声出力部29は、物体検知状態に伴う報知動作を、音声出力により行うように構成されている。
The display unit 28 and the audio output unit 29 are disposed in the vehicle interior of the vehicle 10. The display unit 28 and the audio output unit 29 are electrically connected to the control unit 27. The display unit 28 is configured to perform a notification operation associated with the object detection state by display using a display screen or an indicator. The sound output unit 29 is configured to perform a notification operation associated with the object detection state by sound output.
図2を参照すると、車両10には、雨滴センサ31と、ワイパー機構41とが搭載されている。雨滴センサ31は、車体11における雨滴検出領域に付着した雨滴に応じた信号を出力するように設けられている。ワイパー機構41は、雨滴センサ31から出力された雨滴信号に基づいて駆動されるようになっている。
Referring to FIG. 2, a raindrop sensor 31 and a wiper mechanism 41 are mounted on the vehicle 10. The raindrop sensor 31 is provided so as to output a signal corresponding to the raindrop attached to the raindrop detection region in the vehicle body 11. The wiper mechanism 41 is driven based on the raindrop signal output from the raindrop sensor 31.
図2を参照すると、車両10には、駐車支援装置50が搭載されている。駐車支援装置50は、自車両の駐車を支援するように構成されている。本実施形態においては、物体検知装置20は、駐車支援装置50の一部を構成するように設けられている。すなわち、物体検知装置20の主要部をなす制御部27は、駐車支援ECUを構成する車載マイクロコンピュータであって、駐車支援装置50の全体の動作を制御するように構成されている。ECUはElectronic Control Unitの略である。
Referring to FIG. 2, the vehicle 10 is equipped with a parking assistance device 50. The parking assist device 50 is configured to assist parking of the host vehicle. In the present embodiment, the object detection device 20 is provided so as to constitute a part of the parking assistance device 50. That is, the control unit 27 that forms the main part of the object detection device 20 is an in-vehicle microcomputer that constitutes a parking assistance ECU, and is configured to control the overall operation of the parking assistance device 50. ECU is an abbreviation for Electronic Control Unit.
また、本実施形態においては、駐車支援装置50は、自車両の自動駐車を実現可能に構成されている。具体的には、駐車支援装置50は、物体検知装置20に加えて、動力制御ECU51と、制動制御ECU52と、操舵制御ECU53と、入力操作部54とを備えている。
Moreover, in this embodiment, the parking assistance apparatus 50 is comprised so that the automatic parking of the own vehicle is realizable. Specifically, the parking assist device 50 includes a power control ECU 51, a braking control ECU 52, a steering control ECU 53, and an input operation unit 54 in addition to the object detection device 20.
動力制御ECU51は、不図示の走行用モータおよび/またはエンジンの動作を制御して、車両10の走行出力を制御するように設けられている。制動制御ECU52は、不図示のブレーキアクチュエータ等の駆動を制御して、車両10の制動力を制御するように設けられている。操舵制御ECU53は、不図示の操舵用モータの駆動を制御して、車両10の操舵量を制御するように設けられている。
The power control ECU 51 is provided to control the travel output of the vehicle 10 by controlling the operation of a travel motor and / or engine (not shown). The braking control ECU 52 is provided so as to control the braking force of the vehicle 10 by controlling the driving of a brake actuator (not shown). The steering control ECU 53 is provided so as to control the steering amount of the vehicle 10 by controlling the driving of a steering motor (not shown).
入力操作部54は、車両10における車室内に配置されている。入力操作部54は、車両10の乗員である操作者による入力操作を受け付けるための、スイッチ類および/または音声入力部を備えている。
The input operation unit 54 is disposed in the passenger compartment of the vehicle 10. The input operation unit 54 includes switches and / or a voice input unit for receiving an input operation by an operator who is an occupant of the vehicle 10.
(制御部の機能構成)
図3を参照すると、制御部27は、マイクロコンピュータ上に実現される機能上の構成として、物体認識部270と、駐車領域検出部271と、経路算出部272と、制御量算出部273とを有している。以下、本実施形態における、制御部27の機能構成について説明する。 (Functional configuration of control unit)
Referring to FIG. 3, thecontrol unit 27 includes an object recognition unit 270, a parking area detection unit 271, a route calculation unit 272, and a control amount calculation unit 273 as functional configurations realized on the microcomputer. Have. Hereinafter, the functional configuration of the control unit 27 in the present embodiment will be described.
図3を参照すると、制御部27は、マイクロコンピュータ上に実現される機能上の構成として、物体認識部270と、駐車領域検出部271と、経路算出部272と、制御量算出部273とを有している。以下、本実施形態における、制御部27の機能構成について説明する。 (Functional configuration of control unit)
Referring to FIG. 3, the
物体認識部270は、撮像部21を用いて取得した画像情報と、ソナーセンサ22等を用いて取得した距離情報と、車速センサ24等を用いて取得した自車両の走行状態情報とに基づいて、自車両の周囲に存在する物体Bの3次元形状を認識するように構成されている。物体認識部270の詳細については後述する。
The object recognition unit 270 is based on the image information acquired using the imaging unit 21, the distance information acquired using the sonar sensor 22 or the like, and the traveling state information of the host vehicle acquired using the vehicle speed sensor 24 or the like. The three-dimensional shape of the object B existing around the host vehicle is recognized. Details of the object recognition unit 270 will be described later.
駐車領域検出部271は、物体認識部270による検知結果に基づいて、自車両の周囲にて駐車領域を検出するように設けられている。「駐車領域」とは、自車両が駐車可能な領域である。
The parking area detection unit 271 is provided to detect a parking area around the host vehicle based on the detection result by the object recognition unit 270. The “parking area” is an area where the host vehicle can be parked.
経路算出部272は、駐車領域検出部271によって検出された駐車領域と自車両との相対位置と、物体認識部270による認識結果とに基づいて、誘導経路を算出するように設けられている。「誘導経路」とは、検出された駐車領域に自車両を誘導する経路、すなわち、自車両の駐車領域までの走行経路である。
The route calculation unit 272 is provided so as to calculate the guidance route based on the relative position between the parking area and the host vehicle detected by the parking area detection unit 271 and the recognition result by the object recognition unit 270. The “guidance route” is a route that guides the host vehicle to the detected parking area, that is, a travel route to the parking area of the host vehicle.
制御量算出部273は、駐車領域検出部271によって検出された駐車領域と自車両との相対位置と、経路算出部272によって算出された誘導経路とに基づいて、車両制御量を算出するように設けられている。「車両制御量」とは、自車両を駐車領域まで走行させるために必要な走行パラメータである。「走行パラメータ」には、走行出力、シフトポジション、制動量、および操舵量が含まれる。
The control amount calculation unit 273 calculates the vehicle control amount based on the relative position between the parking area detected by the parking area detection unit 271 and the host vehicle and the guidance route calculated by the route calculation unit 272. Is provided. “Vehicle control amount” is a travel parameter necessary for traveling the host vehicle to the parking area. The “travel parameters” include travel output, shift position, braking amount, and steering amount.
物体認識部270は、移動ステレオの手法、具体的には、いわゆる単眼移動ステレオの手法を用いて、自車両の周囲に存在する物体Bを認識するように構成されている。移動ステレオは、SFMとも称される。SFMはStructure from Motionの略である。すなわち、物体認識部270は、特定の方向に向けられた1個の撮像部21における画像情報の履歴と、自車両の移動量とに基づいて、当該撮像部21の向けられた方向に存在する物体Bを認識するようになっている。
The object recognizing unit 270 is configured to recognize the object B existing around the host vehicle using a moving stereo method, specifically, a so-called monocular moving stereo method. Moving stereo is also referred to as SFM. SFM is an abbreviation for Structure from Motion. That is, the object recognition unit 270 exists in the direction in which the imaging unit 21 is directed based on the history of image information in one imaging unit 21 directed in a specific direction and the amount of movement of the host vehicle. The object B is recognized.
具体的には、物体認識部270は、フロントカメラCFを用いて取得した画像情報の履歴と、自車両の移動量とに基づいて、自車両の前方に存在する物体Bの3次元形状を認識するようになっている。同様に、物体認識部270は、リアカメラCBを用いて取得した画像情報の履歴と、自車両の移動量とに基づいて、自車両の後方に存在する物体Bの3次元形状を認識するようになっている。同様に、物体認識部270は、左側カメラCLを用いて取得した画像情報の履歴と、自車両の移動量とに基づいて、自車両の左側方に存在する物体Bの3次元形状を認識するようになっている。同様に、物体認識部270は、右側カメラCRを用いて取得した画像情報の履歴と、自車両の移動量とに基づいて、自車両の右側方に存在する物体Bの3次元形状を認識するようになっている。
Specifically, the object recognition unit 270 recognizes the three-dimensional shape of the object B existing in front of the host vehicle based on the history of the image information acquired using the front camera CF and the movement amount of the host vehicle. It is supposed to be. Similarly, the object recognition unit 270 recognizes the three-dimensional shape of the object B existing behind the host vehicle based on the history of image information acquired using the rear camera CB and the movement amount of the host vehicle. It has become. Similarly, the object recognition unit 270 recognizes the three-dimensional shape of the object B existing on the left side of the host vehicle based on the history of image information acquired using the left camera CL and the movement amount of the host vehicle. It is like that. Similarly, the object recognition unit 270 recognizes the three-dimensional shape of the object B present on the right side of the host vehicle based on the history of image information acquired using the right camera CR and the movement amount of the host vehicle. It is like that.
なお、移動ステレオあるいはSFMについては、本願の出願時点において、すでに周知技術となっている。例えば、米国特許第7,433,494号明細書、同第8,027,514号明細書、等参照。必要に応じ、これらの記載は、技術的に矛盾しない限り、国内法令の許容する範囲において、記載内容が適宜参照により組み入れられる。したがって、本明細書においては、移動ステレオについての詳細については、説明を省略する。
Note that mobile stereo or SFM is already a well-known technology at the time of filing this application. For example, see US Pat. Nos. 7,433,494 and 8,027,514. As necessary, these descriptions are incorporated by reference as appropriate within the scope allowed by national laws and regulations, unless technically contradictory. Therefore, in this specification, description about the moving stereo is omitted.
物体認識部270は、画像情報取得部701と、移動量取得部702と、座標取得部703と、距離取得部704と、路面状態判定部705と、路面反射判定部706と、座標補正部707とを有している。以下、図1~図4を参照しつつ、物体認識部270の詳細について説明する。
The object recognition unit 270 includes an image information acquisition unit 701, a movement amount acquisition unit 702, a coordinate acquisition unit 703, a distance acquisition unit 704, a road surface state determination unit 705, a road surface reflection determination unit 706, and a coordinate correction unit 707. And have. Hereinafter, the details of the object recognition unit 270 will be described with reference to FIGS.
図4は、撮像部21による撮影画像の一例を示す。図4の例においては、路面RSは、反射性路面であるものとする。反射性路面とは、物体Bの反射像VRを、撮像部21により取得可能な程度に映し出すような状態の路面RSである。典型的には、例えば、水溜りが生じた路面RS等が、反射性路面に該当する。具体的には、図4の例においては、菱形の路面標示Mの他に、ポール状の物体Bにおける実像RRおよび反射像VRが、撮像部21の視野内に入っているものとする。
FIG. 4 shows an example of an image captured by the imaging unit 21. In the example of FIG. 4, the road surface RS is assumed to be a reflective road surface. The reflective road surface is a road surface RS in such a state that the reflected image VR of the object B is projected to the extent that it can be acquired by the imaging unit 21. Typically, for example, a road surface RS or the like in which a water pool has occurred corresponds to a reflective road surface. Specifically, in the example of FIG. 4, it is assumed that the real image RR and the reflected image VR of the pole-shaped object B are within the field of view of the imaging unit 21 in addition to the diamond-shaped road marking M.
画像情報取得部701は、撮像部21を用いて取得した画像情報を、撮像部21から受信して時系列で格納するようになっている。移動量取得部702は、車速センサ24、シフトポジションセンサ25、および舵角センサ26の出力に基づいて、自車両の移動方向および移動量を取得するようになっている。座標取得部703は、画像情報取得部701に格納された画像情報と、移動量取得部702によって取得された移動方向および移動量とに基づいて、物体B上の特徴点FPにおける3次元座標情報を取得するようになっている。
The image information acquisition unit 701 receives image information acquired using the imaging unit 21 from the imaging unit 21 and stores the image information in time series. The movement amount acquisition unit 702 acquires the movement direction and movement amount of the host vehicle based on the outputs of the vehicle speed sensor 24, the shift position sensor 25, and the rudder angle sensor 26. The coordinate acquisition unit 703 is based on the image information stored in the image information acquisition unit 701, the movement direction and the movement amount acquired by the movement amount acquisition unit 702, and the three-dimensional coordinate information on the feature point FP on the object B. To get to.
特徴点FPとは、画像中における、特徴的な点あるいは画素をいう。特徴点FPは、例えば、周囲との明度差が大きい点、あるいは、形状に特徴があるために安定して抽出可能な点である。具体的には、特徴点FPには、物体Bにおける角部、物体Bと周囲との境界部、等が含まれ得る。特徴点FPは、例えば、周知のHarrisオペレータを用いて、2次元的に濃淡変化の勾配値が大きくなる画素を検出することにより抽出することが可能である。なお、Harrisオペレータについては、例えば、以下の文献を参照のこと:C.HARRIS and M.STEPHENS, “A combined corner and edge detector.”, Proc. Alvey Vision Conf., pp.147-151,1988。
The feature point FP is a characteristic point or pixel in the image. The feature point FP is, for example, a point having a large brightness difference from the surroundings, or a point that can be stably extracted because of a feature in the shape. Specifically, the feature point FP may include a corner portion of the object B, a boundary portion between the object B and the surroundings, and the like. The feature point FP can be extracted, for example, by detecting a pixel in which the gradient value of the gradation change is two-dimensionally increased using a known Harris operator. For the Harris operator, see, for example, the following documents: C.HARRIS and M.STEPHENS, “A combined corner and edge detector.”, Proc. Alvey Vision Conf., Pp.147-151,1988.
抽出された特徴点FPにおける3次元座標情報は、上記の移動ステレオの手法によって取得され得る。かかる3次元座標情報には、地上高値が含まれる。地上高値は、基準面を基準とした高さである。基準面は、自車両の接地面を平面と仮定した場合の、当該平面である。地上高値は、基準面と同一の場合は0となり、基準面よりも高い場合は正値となり、基準面よりも低い場合は負値となる。なお、図4においては、地上高値が正値となる、実像RR上の特徴点FPを、「地上検知点FP1」と称する。また、地上高値が負値となる特徴点FPを「地下検知点FP2」と称する。反射像VR上の特徴点FPは、地下検知点FP2となる。
The three-dimensional coordinate information at the extracted feature point FP can be acquired by the moving stereo method described above. Such three-dimensional coordinate information includes the ground high value. The ground height is a height with reference to the reference plane. The reference plane is the plane when the ground plane of the host vehicle is assumed to be a plane. The ground high value is 0 when it is the same as the reference plane, a positive value when it is higher than the reference plane, and a negative value when it is lower than the reference plane. In FIG. 4, the feature point FP on the real image RR in which the ground high value is a positive value is referred to as “ground detection point FP1”. Further, the feature point FP where the ground high value is a negative value is referred to as “underground detection point FP2”. The feature point FP on the reflected image VR becomes the underground detection point FP2.
距離取得部704は、ソナーセンサ22の出力に基づいて、物体Bとの距離を取得するようになっている。すなわち、距離取得部704は、測距点に対応する距離情報を時系列で格納するようになっている。「測距点」は、物体Bの表面上における、ソナーセンサ22から発信された探査波を反射したと推定される点である。
The distance acquisition unit 704 acquires the distance from the object B based on the output of the sonar sensor 22. That is, the distance acquisition unit 704 stores the distance information corresponding to the distance measuring points in time series. The “ranging point” is a point on the surface of the object B that is estimated to have reflected the exploration wave transmitted from the sonar sensor 22.
路面状態判定部705は、車両10の周囲の路面RSの状態を判定するようになっている。具体的には、本実施形態においては、路面状態判定部705は、雨滴センサ31の出力および/またはワイパー機構41の作動状態に基づいて、路面RSが反射性路面である可能性があるか否かを判定するようになっている。
The road surface state determination unit 705 determines the state of the road surface RS around the vehicle 10. Specifically, in the present embodiment, the road surface state determination unit 705 determines whether or not the road surface RS may be a reflective road surface based on the output of the raindrop sensor 31 and / or the operating state of the wiper mechanism 41. It is to judge whether.
路面反射判定部706は、座標取得部703によって取得された、地上高値が負値の3次元座標情報が、路面RSに映し出された反射像VRに基づくものであるか否かを判定するようになっている。本実施形態においては、路面反射判定部706は、路面RSが物体Bの反射像VRを映し出す反射性路面である可能性があることを路面状態判定部705が判定した場合に、判定動作を実行するようになっている。路面反射判定部706による実際の判定内容の具体例については後述する。
The road surface reflection determination unit 706 determines whether or not the three-dimensional coordinate information obtained by the coordinate acquisition unit 703 and having a negative ground high value is based on the reflected image VR displayed on the road surface RS. It has become. In the present embodiment, the road surface reflection determination unit 706 performs a determination operation when the road surface state determination unit 705 determines that the road surface RS may be a reflective road surface that reflects the reflected image VR of the object B. It is supposed to be. A specific example of actual determination contents by the road surface reflection determination unit 706 will be described later.
座標補正部707は、座標取得部703によって取得された、地上高値が負値となる3次元座標情報に対して、地上高値が正値となるように補正するようになっている。本実施形態においては、座標補正部707は、地上高値が負値の3次元座標情報が反射性路面による反射像VRに基づくものであると路面反射判定部706が判定した場合に、当該3次元座標情報における地上高値を正値に補正するようになっている。具体的には、例えば、座標補正部707は、地下検知点FP2における地上高値を、絶対値を維持しつつ符号を正値に反転させるようになっている。
The coordinate correcting unit 707 corrects the three-dimensional coordinate information acquired by the coordinate acquiring unit 703 so that the ground high value becomes a negative value so that the ground high value becomes a positive value. In the present embodiment, the coordinate correction unit 707 determines that the three-dimensional coordinate is determined when the road surface reflection determination unit 706 determines that the three-dimensional coordinate information having a negative ground high value is based on the reflected image VR by the reflective road surface. The ground height value in the coordinate information is corrected to a positive value. Specifically, for example, the coordinate correction unit 707 reverses the sign to a positive value while maintaining the absolute value of the ground high value at the underground detection point FP2.
(動作概要)
以下、物体検知装置20すなわち制御部27における動作の概要について、本実施形態の構成により奏される効果とともに、図1~図4を参照しつつ説明する。 (Overview of operation)
Hereinafter, an outline of the operation of theobject detection device 20, that is, the control unit 27 will be described with reference to FIGS. 1 to 4 together with the effects achieved by the configuration of the present embodiment.
以下、物体検知装置20すなわち制御部27における動作の概要について、本実施形態の構成により奏される効果とともに、図1~図4を参照しつつ説明する。 (Overview of operation)
Hereinafter, an outline of the operation of the
車速が所定速度未満である等の所定条件の成立下で、自車両の乗員(例えば運転者)が入力操作部54を操作することにより、駐車支援ECUを構成する制御部27は、駐車支援動作を開始する。
When a predetermined condition such as the vehicle speed is less than a predetermined speed is satisfied, the control unit 27 constituting the parking support ECU operates as a parking support operation when an occupant (for example, a driver) of the host vehicle operates the input operation unit 54. To start.
駐車支援動作が開始すると、撮像部21により取得された画像情報が、撮像部21から制御部27に順次送信される。制御部27が受信した画像情報は、画像情報取得部701により取得され、時系列で格納される。また、ソナーセンサ22により取得された測距情報は、ソナーセンサ22から制御部27に順次送信される。制御部27が受信した測距情報は、距離取得部704により取得され、時系列で格納される。
When the parking support operation starts, the image information acquired by the imaging unit 21 is sequentially transmitted from the imaging unit 21 to the control unit 27. The image information received by the control unit 27 is acquired by the image information acquisition unit 701 and stored in time series. The distance measurement information acquired by the sonar sensor 22 is sequentially transmitted from the sonar sensor 22 to the control unit 27. The distance measurement information received by the control unit 27 is acquired by the distance acquisition unit 704 and stored in time series.
また、移動量取得部702は、車速センサ24、シフトポジションセンサ25、および舵角センサ26の出力に基づいて、自車両の移動方向および移動量を取得する。座標取得部703は、画像情報取得部701に時系列で格納された画像情報と、移動量取得部702により取得した自車両の移動方向および移動量とに基づいて、物体B上の特徴点FPにおける3次元座標情報を取得する。すなわち、物体B上の特徴点FPにおける3次元座標情報が、撮像部21により取得された画像情報の、自車両の移動に伴う履歴に基づいて取得される。
Also, the movement amount acquisition unit 702 acquires the movement direction and movement amount of the host vehicle based on the outputs of the vehicle speed sensor 24, the shift position sensor 25, and the rudder angle sensor 26. The coordinate acquisition unit 703 performs feature points FP on the object B based on the image information stored in the image information acquisition unit 701 in time series and the movement direction and movement amount of the host vehicle acquired by the movement amount acquisition unit 702. The three-dimensional coordinate information at is acquired. That is, the three-dimensional coordinate information at the feature point FP on the object B is acquired based on the history of the image information acquired by the imaging unit 21 accompanying the movement of the host vehicle.
(路面反射判定手法)
図4に示されているように、自車両の周囲に存在する物体Bの反射像VRが、反射性路面に生じる場合がある。この場合、反射像VRに対応する画像情報が、撮像部21により取得され得る。反射像VRに基づいて座標取得部703によって取得された3次元座標情報においては、地上高値が負値となる。すなわち、反射像VR上の特徴点FPである地下検知点FP2に対応する3次元座標情報は、符号が負となる地上高値を有する。 (Road reflection judgment method)
As shown in FIG. 4, a reflected image VR of the object B existing around the host vehicle may be generated on the reflective road surface. In this case, image information corresponding to the reflected image VR can be acquired by theimaging unit 21. In the three-dimensional coordinate information acquired by the coordinate acquisition unit 703 based on the reflected image VR, the ground high value is a negative value. That is, the three-dimensional coordinate information corresponding to the underground detection point FP2 that is the feature point FP on the reflected image VR has a ground high value with a negative sign.
図4に示されているように、自車両の周囲に存在する物体Bの反射像VRが、反射性路面に生じる場合がある。この場合、反射像VRに対応する画像情報が、撮像部21により取得され得る。反射像VRに基づいて座標取得部703によって取得された3次元座標情報においては、地上高値が負値となる。すなわち、反射像VR上の特徴点FPである地下検知点FP2に対応する3次元座標情報は、符号が負となる地上高値を有する。 (Road reflection judgment method)
As shown in FIG. 4, a reflected image VR of the object B existing around the host vehicle may be generated on the reflective road surface. In this case, image information corresponding to the reflected image VR can be acquired by the
ところで、地上高値が負値となる特徴点FPは、反射性路面に生じる反射像VRに基づくものとは限らない。すなわち、例えば、路面RSに形成された、マンホール等の穴あるいは溝等の凹部に対応する、特徴点FPも、地上高値が負値となる。
Incidentally, the feature point FP where the ground high value is a negative value is not necessarily based on the reflected image VR generated on the reflective road surface. That is, for example, the feature point FP corresponding to a hole such as a manhole or a concave portion such as a groove formed on the road surface RS also has a negative ground high value.
図4に示されているように、路面RS上の水膜への物体Bの映り込みに際しては、当該水膜の表面は、反射鏡として作用する。すなわち、当該水膜上に生じる反射像VRは、物体Bの実像RRの鏡像となる。したがって、通常は、実像RR上の特徴点FPである地上検知点FP1と、実像RRに対応する反射像VR上の特徴点FPである地下検知点FP2とが、対となって検知される。
As shown in FIG. 4, when the object B is reflected on the water film on the road surface RS, the surface of the water film acts as a reflecting mirror. That is, the reflected image VR generated on the water film is a mirror image of the real image RR of the object B. Therefore, normally, the ground detection point FP1 that is the feature point FP on the real image RR and the underground detection point FP2 that is the feature point FP on the reflected image VR corresponding to the real image RR are detected as a pair.
実像RRおよび反射像VRに対応する3次元座標情報が移動ステレオの手法により取得されると、実像RRと反射像VRとで、地上高値を除く2次元座標情報は略同一値となる。このため、地下検知点FP2に対応する3次元座標情報における、地上高値以外の2次元座標情報については、物体Bの自車両に対する相対位置に対応した正確な値となり得る。
When the three-dimensional coordinate information corresponding to the real image RR and the reflected image VR is acquired by the moving stereo method, the two-dimensional coordinate information excluding the ground high value becomes substantially the same value for the real image RR and the reflected image VR. For this reason, the two-dimensional coordinate information other than the ground high value in the three-dimensional coordinate information corresponding to the underground detection point FP2 can be an accurate value corresponding to the relative position of the object B with respect to the host vehicle.
(第一判定手法)
上記の通り、通常の場合、実像RR上の特徴点FPである地上検知点FP1と、実像RRに対応する反射像VR上の特徴点FPである地下検知点FP2とが、対となって検知される。この場合、地上高値が負値の3次元座標情報における地上高値の符号を正に反転させた位置から所定範囲内にて、地上高値が正値の3次元座標情報が座標取得部703によって取得されているはずである。したがって、この場合、路面反射判定部706は、地上高値が負値の3次元座標情報が反射像VRに基づくものであると判定することが可能である。かかる判定方法によれば、実際に存在する物体Bと対応する地下検知点FP2のみを座標補正部707による補正対象とすることができ、以てノイズ検出が低減される。 (First judgment method)
As described above, in the normal case, the ground detection point FP1, which is the feature point FP on the real image RR, and the underground detection point FP2, which is the feature point FP on the reflected image VR corresponding to the real image RR, are detected as a pair. Is done. In this case, the coordinateacquisition unit 703 acquires three-dimensional coordinate information with a positive ground height within a predetermined range from a position obtained by inverting the sign of the ground high value in the three-dimensional coordinate information with a negative ground high value. Should be. Therefore, in this case, the road surface reflection determination unit 706 can determine that the three-dimensional coordinate information having a negative ground high value is based on the reflected image VR. According to this determination method, only the underground detection point FP2 corresponding to the actually existing object B can be set as the correction target by the coordinate correction unit 707, and noise detection is reduced.
上記の通り、通常の場合、実像RR上の特徴点FPである地上検知点FP1と、実像RRに対応する反射像VR上の特徴点FPである地下検知点FP2とが、対となって検知される。この場合、地上高値が負値の3次元座標情報における地上高値の符号を正に反転させた位置から所定範囲内にて、地上高値が正値の3次元座標情報が座標取得部703によって取得されているはずである。したがって、この場合、路面反射判定部706は、地上高値が負値の3次元座標情報が反射像VRに基づくものであると判定することが可能である。かかる判定方法によれば、実際に存在する物体Bと対応する地下検知点FP2のみを座標補正部707による補正対象とすることができ、以てノイズ検出が低減される。 (First judgment method)
As described above, in the normal case, the ground detection point FP1, which is the feature point FP on the real image RR, and the underground detection point FP2, which is the feature point FP on the reflected image VR corresponding to the real image RR, are detected as a pair. Is done. In this case, the coordinate
(第二判定手法)
図4に示されているように、撮像部21による撮影画像中において、地下検知点FP2は、地上検知点FP1よりも、下側すなわち撮像部21側に現れる。より詳細には、地下検知点FP2は、所定の基準点RPと地上検知点FP1とを結ぶ仮想線分VL上、あるいは、仮想線分VLの近傍に存在する。基準点RPは、例えば、路面RS上における撮像部21に対応する点、具体的には、基準面と直交しつつ撮像部21の配置位置を通る直線と基準面との交点である。あるいは、基準点RPは、例えば、基準面と直交する複数の直線が撮影画像中にて収束する点である。 (Second determination method)
As shown in FIG. 4, in the captured image by theimaging unit 21, the underground detection point FP2 appears below the ground detection point FP1, that is, on the imaging unit 21 side. More specifically, the underground detection point FP2 exists on the virtual line segment VL connecting the predetermined reference point RP and the ground detection point FP1 or in the vicinity of the virtual line segment VL. The reference point RP is, for example, a point corresponding to the imaging unit 21 on the road surface RS, specifically, an intersection of a straight line passing through the arrangement position of the imaging unit 21 while being orthogonal to the reference plane and the reference plane. Alternatively, the reference point RP is a point where, for example, a plurality of straight lines orthogonal to the reference plane converge in the captured image.
図4に示されているように、撮像部21による撮影画像中において、地下検知点FP2は、地上検知点FP1よりも、下側すなわち撮像部21側に現れる。より詳細には、地下検知点FP2は、所定の基準点RPと地上検知点FP1とを結ぶ仮想線分VL上、あるいは、仮想線分VLの近傍に存在する。基準点RPは、例えば、路面RS上における撮像部21に対応する点、具体的には、基準面と直交しつつ撮像部21の配置位置を通る直線と基準面との交点である。あるいは、基準点RPは、例えば、基準面と直交する複数の直線が撮影画像中にて収束する点である。 (Second determination method)
As shown in FIG. 4, in the captured image by the
このため、路面反射判定部706は、以下の条件Xが成立する場合、地上高値が負値の3次元座標情報が反射像VRに基づくものであると判定することが可能である。あるいは、路面反射判定部706は、かかる条件を満たすような地上検知点FP1が存在する場合、地上高値が負値の3次元座標情報が反射像VRに基づくものであると判定することが可能である。これにより、地下検知点FP2が反射像VRに基づくものであることを、より正確に判定することが可能となる。
・条件X:撮影画像すなわち画像情報において、仮想線分VL上または仮想線分VLの近傍に、地下検知点FP2が存在する。 For this reason, when the following condition X is satisfied, the road surfacereflection determination unit 706 can determine that the three-dimensional coordinate information whose ground high value is a negative value is based on the reflected image VR. Alternatively, when there is a ground detection point FP1 that satisfies such a condition, the road surface reflection determination unit 706 can determine that the three-dimensional coordinate information having a negative ground high value is based on the reflected image VR. is there. Thereby, it becomes possible to determine more accurately that the underground detection point FP2 is based on the reflected image VR.
Condition X: In the captured image, that is, image information, the underground detection point FP2 exists on or near the virtual line segment VL.
・条件X:撮影画像すなわち画像情報において、仮想線分VL上または仮想線分VLの近傍に、地下検知点FP2が存在する。 For this reason, when the following condition X is satisfied, the road surface
Condition X: In the captured image, that is, image information, the underground detection point FP2 exists on or near the virtual line segment VL.
(第三判定手法)
地下検知点FP2に対応する3次元座標情報から、地下検知点FP2との距離が把握可能である。地下検知点FP2が、物体Bの実像RRの路面反射による反射像VR上の特徴点FPである場合、実像RRに対応する物体Bに対しては、ソナーセンサ22を用いた測距情報が取得されているはずである。この場合、地下検知点FP2との距離に対応した測距点が、距離取得部704により認識されていることとなる。すなわち、この場合、距離取得部704によって取得された距離から所定距離範囲内に、地上高値が負値の3次元座標情報に対応する特徴点FPが存在する。したがって、この場合、路面反射判定部706は、地上高値が負値の3次元座標情報が反射像VRに基づくものであると判定することが可能である。これにより、地下検知点FP2が反射像VRに基づくものであることを、より正確に判定することが可能となる。 (Third determination method)
From the three-dimensional coordinate information corresponding to the underground detection point FP2, the distance to the underground detection point FP2 can be grasped. When the underground detection point FP2 is a feature point FP on the reflected image VR due to road surface reflection of the real image RR of the object B, distance measurement information using thesonar sensor 22 is acquired for the object B corresponding to the real image RR. Should be. In this case, the distance measurement point corresponding to the distance from the underground detection point FP2 is recognized by the distance acquisition unit 704. That is, in this case, there is a feature point FP corresponding to the three-dimensional coordinate information having a negative ground high value within a predetermined distance range from the distance acquired by the distance acquisition unit 704. Therefore, in this case, the road surface reflection determination unit 706 can determine that the three-dimensional coordinate information having a negative ground high value is based on the reflected image VR. Thereby, it becomes possible to determine more accurately that the underground detection point FP2 is based on the reflected image VR.
地下検知点FP2に対応する3次元座標情報から、地下検知点FP2との距離が把握可能である。地下検知点FP2が、物体Bの実像RRの路面反射による反射像VR上の特徴点FPである場合、実像RRに対応する物体Bに対しては、ソナーセンサ22を用いた測距情報が取得されているはずである。この場合、地下検知点FP2との距離に対応した測距点が、距離取得部704により認識されていることとなる。すなわち、この場合、距離取得部704によって取得された距離から所定距離範囲内に、地上高値が負値の3次元座標情報に対応する特徴点FPが存在する。したがって、この場合、路面反射判定部706は、地上高値が負値の3次元座標情報が反射像VRに基づくものであると判定することが可能である。これにより、地下検知点FP2が反射像VRに基づくものであることを、より正確に判定することが可能となる。 (Third determination method)
From the three-dimensional coordinate information corresponding to the underground detection point FP2, the distance to the underground detection point FP2 can be grasped. When the underground detection point FP2 is a feature point FP on the reflected image VR due to road surface reflection of the real image RR of the object B, distance measurement information using the
(実施形態の構成による作用効果)
上記のようにして、路面反射判定部706により、地上高値が負値の3次元座標情報が反射像VRに基づくものであると判定されると、座標補正部707は、地上高値が負値となる3次元座標情報に対して、地上高値が正値となるように補正する。具体的には、例えば、座標補正部707は、地下検知点FP2に対応する3次元座標情報に対して、地上高値の絶対値を維持しつつ、地上高値の符号を正に反転させる。 (Operational effects of the configuration of the embodiment)
As described above, when the road surfacereflection determining unit 706 determines that the three-dimensional coordinate information having a negative ground high value is based on the reflected image VR, the coordinate correcting unit 707 determines that the ground high value is a negative value. Is corrected so that the ground high value becomes a positive value. Specifically, for example, the coordinate correction unit 707 inverts the sign of the ground high value to the three-dimensional coordinate information corresponding to the underground detection point FP2 while maintaining the absolute value of the ground high value.
上記のようにして、路面反射判定部706により、地上高値が負値の3次元座標情報が反射像VRに基づくものであると判定されると、座標補正部707は、地上高値が負値となる3次元座標情報に対して、地上高値が正値となるように補正する。具体的には、例えば、座標補正部707は、地下検知点FP2に対応する3次元座標情報に対して、地上高値の絶対値を維持しつつ、地上高値の符号を正に反転させる。 (Operational effects of the configuration of the embodiment)
As described above, when the road surface
本実施形態においては、反射像VRに対応する地上高値が負値の3次元座標情報は、正値に補正されることで、ノイズとして除去されるのではなく物体Bの検知のために積極的に活用され得る。したがって、かかる構成によれば、自車両の周囲に存在する物体Bの反射像VRが路面RSに生じても、当該物体Bを従来よりも良好に検知することが可能となる。また、従来ノイズとして除去されていた、映り込みにより得られた検知点を、補正により活用することで、障害物の検知点数を増加させられるため、雨天時の障害物の判定性能を向上することができる。
In the present embodiment, the three-dimensional coordinate information with a negative ground high value corresponding to the reflected image VR is corrected to a positive value, so that it is not removed as noise but actively detected for the object B. It can be utilized. Therefore, according to such a configuration, even if a reflection image VR of the object B existing around the host vehicle is generated on the road surface RS, the object B can be detected better than before. In addition, the number of obstacle detection points can be increased by using the detection points obtained by reflection, which had been removed as noise in the past, to improve the obstacle judgment performance in rainy weather. Can do.
本実施形態においては、座標補正部707は、地上高値が負値の3次元座標情報が反射像VRに基づくものであると路面反射判定部706が判定した場合に、当該3次元座標情報における地上高値を正値に補正する。したがって、マンホール等の穴あるいは溝等の凹部上の特徴点FPに対応する3次元座標情報を誤って補正することが、良好に抑制され得る。
In the present embodiment, the coordinate correction unit 707, when the road surface reflection determination unit 706 determines that the three-dimensional coordinate information with a negative ground high value is based on the reflected image VR, the ground surface in the three-dimensional coordinate information. Correct high value to positive value. Accordingly, erroneous correction of the three-dimensional coordinate information corresponding to the feature point FP on the concave portion such as a hole such as a manhole or a groove can be satisfactorily suppressed.
本実施形態においては、路面反射判定部706は、路面RSが反射性路面であることを路面状態判定部705が判定した場合に判定動作を実行する。したがって、反射性路面が発生していないような状況における不必要な判定処理負荷の発生が、良好に回避され得る。また、マンホール等の穴あるいは溝等の凹部上の特徴点FPに対応する3次元座標情報を誤って補正することが、良好に抑制され得る。
In the present embodiment, the road surface reflection determination unit 706 performs a determination operation when the road surface state determination unit 705 determines that the road surface RS is a reflective road surface. Therefore, generation of unnecessary determination processing load in a situation where no reflective road surface is generated can be avoided satisfactorily. In addition, erroneous correction of the three-dimensional coordinate information corresponding to the feature point FP on the concave portion such as a hole such as a manhole or a groove can be suppressed satisfactorily.
本実施形態においては、座標取得部703は、撮像部21により取得された画像情報の履歴と、移動量取得部702により取得された自車両の移動量とに基づいて、3次元座標情報を取得する。したがって、いわゆる単眼移動ステレオ法の利用が可能となる。すなわち、特定の方向に向けられた1個の撮像部21における画像情報の履歴と自車両の移動量とに基づいて、当該撮像部21の向けられた方向に存在する物体Bを認識することが可能となる。これにより、ステレオカメラ法を用いた場合よりも、装置コストが低減され得る。
In the present embodiment, the coordinate acquisition unit 703 acquires three-dimensional coordinate information based on the history of image information acquired by the imaging unit 21 and the movement amount of the host vehicle acquired by the movement amount acquisition unit 702. To do. Therefore, the so-called monocular moving stereo method can be used. That is, the object B existing in the direction in which the imaging unit 21 is directed is recognized based on the history of image information in one imaging unit 21 directed in a specific direction and the movement amount of the host vehicle. It becomes possible. Thereby, apparatus cost can be reduced rather than the case where the stereo camera method is used.
駐車支援動作においては、道路走行中における自動運転動作とは異なり、細いポール等の検知しづらい物体Bに対しても、良好に検知する必要がある。雨天時にて路面RSに水溜り等が生じることで物体Bの反射像VRが撮像部21に映り込む場合、物体Bの良好な検知がさらに困難となり得る。
In the parking assistance operation, it is necessary to detect the object B that is difficult to detect, such as a thin pole, unlike the automatic driving operation while driving on the road. When the reflected image VR of the object B is reflected on the imaging unit 21 due to a puddle or the like on the road surface RS in the rain, good detection of the object B may be further difficult.
この点、本実施形態においては、物体検知装置20による物体Bの認識結果は、駐車支援装置50による駐車支援動作に用いられる。したがって、本実施形態によれば、天候等の条件にかかわらず、良好な駐車支援動作を実現することが可能となる。
In this respect, in this embodiment, the recognition result of the object B by the object detection device 20 is used for the parking support operation by the parking support device 50. Therefore, according to the present embodiment, it is possible to realize a good parking assist operation regardless of conditions such as the weather.
(動作例)
以下、本実施形態の構成による、上記の動作概要に対応する具体的な動作例について、フローチャートを用いて説明する。なお、図面において、「ステップ」を単に「S」と略記する。 (Operation example)
Hereinafter, a specific operation example corresponding to the above operation outline according to the configuration of the present embodiment will be described with reference to a flowchart. In the drawings, “step” is simply abbreviated as “S”.
以下、本実施形態の構成による、上記の動作概要に対応する具体的な動作例について、フローチャートを用いて説明する。なお、図面において、「ステップ」を単に「S」と略記する。 (Operation example)
Hereinafter, a specific operation example corresponding to the above operation outline according to the configuration of the present embodiment will be described with reference to a flowchart. In the drawings, “step” is simply abbreviated as “S”.
図5は、撮像部21により取得された画像情報に基づく、物体Bの検知動作の一例を示すフローチャートである。図5に示された物体検知ルーチンは、所定の起動条件が成立した後、所定時間間隔で、CPUにより繰り返し起動される。
FIG. 5 is a flowchart showing an example of the detection operation of the object B based on the image information acquired by the imaging unit 21. The object detection routine shown in FIG. 5 is repeatedly activated by the CPU at predetermined time intervals after a predetermined activation condition is established.
図5に示された物体検知ルーチンが起動されると、まず、ステップ501にて、CPUは、画像情報を撮像部21から取得し、不揮発性RAMに時系列で格納する。また、CPUは、今回取得した最新の画像情報を、RAMに保持する。ステップ501の処理は、画像情報取得部701の動作に対応する。次に、ステップ502にて、CPUは、取得した画像情報に基づいて、撮影画像中の特徴点FPを抽出する。続いて、ステップ503にて、CPUは、自車両の移動量を取得する。さらに、ステップ504にて、CPUは、抽出した特徴点FPに対応する3次元座標情報を取得し、RAMに一旦保持する。
When the object detection routine shown in FIG. 5 is started, first, in step 501, the CPU acquires image information from the imaging unit 21 and stores it in the nonvolatile RAM in time series. In addition, the CPU holds the latest image information acquired this time in the RAM. The processing in step 501 corresponds to the operation of the image information acquisition unit 701. Next, in step 502, the CPU extracts a feature point FP in the captured image based on the acquired image information. Subsequently, in step 503, the CPU acquires the amount of movement of the host vehicle. Further, in step 504, the CPU acquires three-dimensional coordinate information corresponding to the extracted feature point FP and temporarily stores it in the RAM.
ステップ502~ステップ504の処理のうち、ステップ503の処理は移動量取得部702の動作に対応し、その余は座標取得部703の動作に対応する。なお、ステップ502~ステップ504の処理内容は、周知の移動ステレオあるいはSFMに相当する。このため、ステップ502~ステップ504における具体的な処理内容の詳細については、本明細書においては説明を省略する。
Of the processing from step 502 to step 504, the processing at step 503 corresponds to the operation of the movement amount acquisition unit 702, and the remainder corresponds to the operation of the coordinate acquisition unit 703. Note that the processing contents of steps 502 to 504 correspond to known moving stereo or SFM. For this reason, the details of the specific processing contents in steps 502 to 504 are omitted in this specification.
続いて、ステップ510にて、CPUは、路面状態情報を取得する。路面状態情報は、自車両の周囲の路面RSの状態に関連する情報、すなわち、路面RSの状態を判定するための情報である。具体的には、CPUは、雨滴センサ31の出力および/またはワイパー機構41の作動状態を取得する。続いて、ステップ520にて、CPUは、路面RSが反射性路面である可能性があるか否かを判定する。ステップ510およびステップ520の処理は、路面状態判定部705の動作に対応する。
Subsequently, in step 510, the CPU acquires road surface state information. The road surface state information is information related to the state of the road surface RS around the host vehicle, that is, information for determining the state of the road surface RS. Specifically, the CPU acquires the output of the raindrop sensor 31 and / or the operating state of the wiper mechanism 41. Subsequently, in step 520, the CPU determines whether or not the road surface RS may be a reflective road surface. The processing of step 510 and step 520 corresponds to the operation of the road surface state determination unit 705.
路面RSが反射性路面である可能性がない場合(すなわちステップ520=NO)、CPUは、処理をステップ530に進行させた後、本ルーチンを一旦終了する。ステップ530にて、CPUは、RAMに一旦保持した3次元座標情報を、不揮発性RAMに格納する。ステップ530の処理は、座標取得部703の動作に対応する。この場合、ステップ503にて取得された3次元座標情報に対しては、地上高値の補正は実行されない。すなわち、ステップ504にてRAMに一旦保持された3次元座標情報は、ステップ530にて、不揮発性RAMにそのまま格納される。
When there is no possibility that the road surface RS is a reflective road surface (that is, step 520 = NO), the CPU advances the process to step 530 and then ends this routine once. In step 530, the CPU stores the three-dimensional coordinate information once held in the RAM in the nonvolatile RAM. The processing in step 530 corresponds to the operation of the coordinate acquisition unit 703. In this case, the correction of the ground high value is not executed for the three-dimensional coordinate information acquired in step 503. That is, the three-dimensional coordinate information once held in the RAM in step 504 is stored in the nonvolatile RAM as it is in step 530.
一方、路面RSが反射性路面である可能性がある場合(すなわちステップ520=YES)、CPUは、処理をステップ540に進行させる。ステップ540にて、CPUは、地上高値が負値の3次元座標情報が存在するか否かを判定する。
On the other hand, when the road surface RS may be a reflective road surface (that is, step 520 = YES), the CPU proceeds with the process to step 540. In step 540, the CPU determines whether or not there is three-dimensional coordinate information having a negative ground high value.
地上高値が負値の3次元座標情報が存在しない場合(すなわちステップ540=NO)、CPUは、処理をステップ530に進行させた後、本ルーチンを一旦終了する。ステップ530にて、CPUは、RAMに一旦保持した3次元座標情報を、不揮発性RAMに格納する。この場合も、ステップ503にて取得された3次元座標情報に対しては、地上高値の補正は実行されない。すなわち、ステップ504にてRAMに一旦保持された3次元座標情報は、ステップ530にて、不揮発性RAMにそのまま格納される。
If there is no three-dimensional coordinate information with a negative ground high value (ie, step 540 = NO), the CPU advances the process to step 530 and then ends this routine once. In step 530, the CPU stores the three-dimensional coordinate information once held in the RAM in the nonvolatile RAM. Also in this case, the correction of the ground high value is not performed on the three-dimensional coordinate information acquired in step 503. That is, the three-dimensional coordinate information once held in the RAM in step 504 is stored in the nonvolatile RAM as it is in step 530.
地上高値が負値の3次元座標情報が存在する場合(すなわちステップ540=YES)、CPUは、処理をステップ550に進行させた後、処理をステップ530に進行させ、その後、本ルーチンを一旦終了する。ステップ550にて、CPUは、座標補正処理を実行する。座標補正処理は、3次元座標情報を補正するための処理である。ステップ550の処理内容の具体例は後述する。ステップ530にて、CPUは、ステップ550の処理を経た3次元座標情報を、不揮発性RAMに格納する。
When the three-dimensional coordinate information having a negative ground high value exists (that is, step 540 = YES), the CPU advances the process to step 550, advances the process to step 530, and then ends this routine once. To do. In step 550, the CPU executes coordinate correction processing. The coordinate correction process is a process for correcting the three-dimensional coordinate information. A specific example of the processing content of step 550 will be described later. In step 530, the CPU stores the three-dimensional coordinate information that has undergone the processing in step 550 in the nonvolatile RAM.
(第一具体例)
図6Aは、ステップ550の処理内容の一具体例に対応するサブルーチンを示す。本具体例は、路面反射判定部706における第一判定手法に対応する。 (First example)
FIG. 6A shows a subroutine corresponding to a specific example of the processing content of step 550. This specific example corresponds to the first determination method in the road surfacereflection determination unit 706.
図6Aは、ステップ550の処理内容の一具体例に対応するサブルーチンを示す。本具体例は、路面反射判定部706における第一判定手法に対応する。 (First example)
FIG. 6A shows a subroutine corresponding to a specific example of the processing content of step 550. This specific example corresponds to the first determination method in the road surface
かかるサブルーチンの処理が開始されると、まず、ステップ611にて、CPUは、1個の地下検知点FP2を抽出して、その3次元座標情報を読み出す。次に、ステップ621にて、CPUは、読み出した3次元座標情報における地上高値に対して、絶対値を維持しつつ、符号を正に反転させる。
When the processing of this subroutine is started, first, in step 611, the CPU extracts one underground detection point FP2 and reads out its three-dimensional coordinate information. Next, in step 621, the CPU inverts the sign positive while maintaining the absolute value with respect to the ground high value in the read three-dimensional coordinate information.
続いて、ステップ631にて、CPUは、ステップ621にて符号を反転した3次元座標情報に対応する地上検知点FP1を探索する。具体的には、CPUは、ステップ621にて符号を反転した3次元座標情報に対応する位置を中心とした所定範囲内に、実像RR上の特徴点FPが存在するか否かを判定する。その後、CPUは、処理をステップ641に進行させる。
Subsequently, in step 631, the CPU searches for the ground detection point FP1 corresponding to the three-dimensional coordinate information whose sign is inverted in step 621. Specifically, the CPU determines whether or not the feature point FP on the real image RR exists within a predetermined range centered on the position corresponding to the three-dimensional coordinate information whose sign is inverted in step 621. Thereafter, the CPU proceeds with the process to step 641.
ステップ641にて、CPUは、ステップ631による探索の結果、地上高値の符号を反転した関係の地上検知点FP1と地下検知点FP2との組が存在するか否かを判定する。地上高値の符号を反転した関係の地上検知点FP1と地下検知点FP2との組が存在する場合、ステップ641の判定が「YES」となる。この場合、地下検知点FP2は、物体Bの実像RRの路面反射により生じた反射像VR上の特徴点FPである。すなわち、地上高値が負値の3次元座標情報を有する地下検知点FP2は、路面反射による反射像VRに基づくものである。
In step 641, as a result of the search in step 631, the CPU determines whether or not there is a set of the ground detection point FP1 and the underground detection point FP2 having a relationship in which the sign of the ground high value is inverted. If there is a set of the ground detection point FP1 and the underground detection point FP2 in the relationship in which the sign of the ground high value is inverted, the determination in step 641 is “YES”. In this case, the underground detection point FP2 is a feature point FP on the reflected image VR generated by road surface reflection of the real image RR of the object B. That is, the underground detection point FP2 having three-dimensional coordinate information whose ground high value is negative is based on the reflected image VR by road surface reflection.
そこで、ステップ641の判定が「YES」の場合、CPUは、処理をステップ650に進行させた後、処理をステップ660に進行させる。ステップ611~ステップ641の処理は、路面反射判定部706の動作に対応する。
Therefore, if the determination in step 641 is “YES”, the CPU advances the process to step 650 and then advances the process to step 660. The processing from step 611 to step 641 corresponds to the operation of the road surface reflection determination unit 706.
ステップ650にて、CPUは、座標取得部703によって取得された、地上高値が負値となる3次元座標情報に対して、地上高値が正値となるように補正する。ステップ650の処理は、座標補正部707の動作に対応する。
In step 650, the CPU corrects the three-dimensional coordinate information obtained by the coordinate obtaining unit 703 so that the ground high value becomes a negative value so that the ground high value becomes a positive value. The processing in step 650 corresponds to the operation of the coordinate correction unit 707.
ステップ660にて、CPUは、すべての地下検知点FP2に対する処理が終了したか否かを判定する。未処理の地下検知点FP2が存在する場合(すなわちステップ660=NO)、CPUは、処理をステップ611に戻す。一方、すべての地下検知点FP2に対する処理が終了した場合(すなわちステップ660=YES)、CPUは、本サブルーチンを一旦終了する。
In step 660, the CPU determines whether or not the processing for all the underground detection points FP2 has been completed. When there is an unprocessed underground detection point FP2 (ie, step 660 = NO), the CPU returns the process to step 611. On the other hand, when the processing for all the underground detection points FP2 is completed (ie, step 660 = YES), the CPU once ends this subroutine.
一方、地上高値の符号を反転した関係の地上検知点FP1と地下検知点FP2との組が存在しない場合、ステップ641の判定が「NO」となる。この場合、地下検知点FP2は、物体Bの実像RRの路面反射により生じた反射像VR上の特徴点FPではない。すなわち、地下検知点FP2は、穴あるいは凹部に対応する特徴点FPであり、地上高値を正値に補正すべきではない。そこで、この場合、CPUは、ステップ650の処理をスキップして、処理をステップ660に進行させる。
On the other hand, if there is no set of the ground detection point FP1 and the underground detection point FP2 in the relationship in which the sign of the ground high value is inverted, the determination in step 641 is “NO”. In this case, the underground detection point FP2 is not the feature point FP on the reflected image VR generated by the road surface reflection of the real image RR of the object B. That is, the underground detection point FP2 is a feature point FP corresponding to a hole or a recess, and the ground high value should not be corrected to a positive value. Therefore, in this case, the CPU skips the process of step 650 and advances the process to step 660.
(第二具体例)
図6Bは、ステップ550の処理内容の他の一具体例に対応するサブルーチンを示す。本具体例は、路面反射判定部706における第二判定手法に対応する。 (Second example)
FIG. 6B shows a subroutine corresponding to another specific example of the processing content of step 550. This specific example corresponds to the second determination method in the road surfacereflection determination unit 706.
図6Bは、ステップ550の処理内容の他の一具体例に対応するサブルーチンを示す。本具体例は、路面反射判定部706における第二判定手法に対応する。 (Second example)
FIG. 6B shows a subroutine corresponding to another specific example of the processing content of step 550. This specific example corresponds to the second determination method in the road surface
かかるサブルーチンの処理が開始されると、まず、ステップ612にて、CPUは、1個の地下検知点FP2を抽出して、その3次元座標情報を読み出す。図6Bにおけるステップ612の処理内容は、図6Aにおけるステップ611の処理内容と同様である。
When the processing of this subroutine is started, first, in step 612, the CPU extracts one underground detection point FP2 and reads out its three-dimensional coordinate information. The processing content of step 612 in FIG. 6B is the same as the processing content of step 611 in FIG. 6A.
次に、ステップ622にて、CPUは、基準点RPを端点として当該端点から地下検知点FP2に向かう仮想半直線を算出する。続いて、ステップ632にて、CPUは、ステップ622にて算出した仮想半直線上、あるいは、かかる仮想半直線の近傍にて、地上検知点FP1を探索する。その後、CPUは、処理をステップ642に進行させる。
Next, in step 622, the CPU calculates a virtual half line from the end point to the underground detection point FP2 with the reference point RP as an end point. Subsequently, in step 632, the CPU searches for the ground detection point FP1 on the virtual half line calculated in step 622 or in the vicinity of the virtual half line. Thereafter, the CPU proceeds with the process to step 642.
ステップ642にて、CPUは、ステップ632による探索の結果、該当する地上検知点FP1が発見されたか否かを判定する。算出した仮想半直線上あるいはその近傍に地上検知点FP1が存在する場合、ステップ642の判定が「YES」となる。この場合、図4に示されているように、地下検知点FP2は、基準点RPと地上検知点FP1とを結ぶ仮想線分VL上、あるいは、仮想線分VLの近傍に存在する。すなわち、この場合、地上高値が負値の3次元座標情報を有する地下検知点FP2は、路面反射による反射像VRに基づくものである。
In step 642, the CPU determines whether or not the corresponding ground detection point FP1 has been found as a result of the search in step 632. When the ground detection point FP1 exists on or near the calculated virtual half line, the determination in step 642 is “YES”. In this case, as shown in FIG. 4, the underground detection point FP2 exists on the virtual line segment VL connecting the reference point RP and the ground detection point FP1 or in the vicinity of the virtual line segment VL. That is, in this case, the underground detection point FP2 having the three-dimensional coordinate information having a negative ground high value is based on the reflected image VR by road surface reflection.
そこで、ステップ642の判定が「YES」の場合、CPUは、処理をステップ650に進行させた後、本サブルーチンを一旦終了する。図6Bにおけるステップ650の処理内容は、図6Aと同様である。ステップ612~ステップ642の処理は、路面反射判定部706の動作に対応する。
Therefore, if the determination in step 642 is “YES”, the CPU advances the process to step 650 and then ends this subroutine once. The processing content of step 650 in FIG. 6B is the same as in FIG. 6A. The processing from step 612 to step 642 corresponds to the operation of the road surface reflection determination unit 706.
ステップ632による探索の結果、該当する地上検知点FP1が発見されなかった場合、ステップ642の判定が「NO」となる。この場合、地下検知点FP2は、物体Bの実像RRの路面反射により生じた反射像VR上の特徴点FPではない。すなわち、地下検知点FP2は、穴あるいは凹部に対応する特徴点FPであり、地上高値を正値に補正すべきではない。そこで、この場合、CPUは、ステップ650の処理をスキップして、処理をステップ660に進行させる。図6Bにおけるステップ660の処理内容は、図6Aと同様である。
As a result of the search in step 632, if the corresponding ground detection point FP1 is not found, the determination in step 642 is “NO”. In this case, the underground detection point FP2 is not the feature point FP on the reflected image VR generated by the road surface reflection of the real image RR of the object B. That is, the underground detection point FP2 is a feature point FP corresponding to a hole or a recess, and the ground high value should not be corrected to a positive value. Therefore, in this case, the CPU skips the process of step 650 and advances the process to step 660. The processing content of step 660 in FIG. 6B is the same as in FIG. 6A.
(第三具体例)
図6Cは、ステップ550の処理内容のさらに他の一具体例に対応するサブルーチンを示す。本具体例は、路面反射判定部706における第三判定手法に対応する。 (Third example)
FIG. 6C shows a subroutine corresponding to yet another specific example of the processing content of step 550. This specific example corresponds to the third determination method in the road surfacereflection determination unit 706.
図6Cは、ステップ550の処理内容のさらに他の一具体例に対応するサブルーチンを示す。本具体例は、路面反射判定部706における第三判定手法に対応する。 (Third example)
FIG. 6C shows a subroutine corresponding to yet another specific example of the processing content of step 550. This specific example corresponds to the third determination method in the road surface
かかるサブルーチンの処理が開始されると、まず、ステップ613にて、CPUは、地下検知点FP2を抽出して、その3次元座標情報を読み出す。図6Cにおけるステップ613の処理内容は、図6Aにおけるステップ611、および図6Bにおけるステップ612の処理内容と同様である。
When the processing of this subroutine is started, first, in step 613, the CPU extracts the underground detection point FP2 and reads out its three-dimensional coordinate information. The processing content of step 613 in FIG. 6C is the same as the processing content of step 611 in FIG. 6A and step 612 in FIG. 6B.
次に、ステップ623にて、CPUは、測距点を読み出す。続いて、ステップ633にて、CPUは、読み出した測距点と地下検知点FP2とを対比する。すなわち、CPUは、測距点に対応する3次元座標情報と、地下検知点FP2に対応する3次元座標情報とを照合する。その後、CPUは、処理をステップ643に進行させる。
Next, in step 623, the CPU reads a distance measuring point. Subsequently, in step 633, the CPU compares the read distance measuring point with the underground detection point FP2. That is, the CPU collates the three-dimensional coordinate information corresponding to the distance measuring point with the three-dimensional coordinate information corresponding to the underground detection point FP2. Thereafter, the CPU proceeds with the process to step 643.
ステップ643にて、CPUは、測距点から所定距離範囲内に地下検知点FP2が存在するか否かを判定する。測距点から所定距離範囲内に地下検知点FP2が存在する場合、ステップ643の判定が「YES」となる。この場合、この場合、地下検知点FP2は、物体Bの実像RRの路面反射により生じた反射像VR上の特徴点FPである。すなわち、地上高値が負値の3次元座標情報を有する地下検知点FP2は、路面反射による反射像VRに基づくものである。
In step 643, the CPU determines whether or not the underground detection point FP2 exists within a predetermined distance range from the distance measuring point. When the underground detection point FP2 exists within a predetermined distance range from the distance measurement point, the determination in step 643 is “YES”. In this case, in this case, the underground detection point FP2 is a feature point FP on the reflected image VR generated by the road surface reflection of the real image RR of the object B. That is, the underground detection point FP2 having three-dimensional coordinate information whose ground high value is negative is based on the reflected image VR by road surface reflection.
そこで、ステップ643の判定が「YES」の場合、CPUは、処理をステップ650に進行させた後、本サブルーチンを一旦終了する。図6Cにおけるステップ650の処理内容は、図6Aおよび図6Bと同様である。ステップ613~ステップ643の処理は、路面反射判定部706の動作に対応する。
Therefore, if the determination in step 643 is “YES”, the CPU advances the process to step 650 and then ends this subroutine once. The processing content of step 650 in FIG. 6C is the same as in FIG. 6A and FIG. 6B. The processing from step 613 to step 643 corresponds to the operation of the road surface reflection determination unit 706.
測距点から所定距離範囲内に地下検知点FP2が存在しない場合、ステップ643の判定が「NO」となる。この場合、地下検知点FP2は、物体Bの実像RRの路面反射により生じた反射像VR上の特徴点FPではない。すなわち、地下検知点FP2は、穴あるいは凹部に対応する特徴点FPであり、地上高値を正値に補正すべきではない。そこで、この場合、CPUは、ステップ650の処理をスキップして、処理をステップ660に進行させる。図6Cにおけるステップ660の処理内容は、図6Aおよび図6Bと同様である。
If the underground detection point FP2 does not exist within a predetermined distance range from the distance measurement point, the determination in step 643 is “NO”. In this case, the underground detection point FP2 is not the feature point FP on the reflected image VR generated by the road surface reflection of the real image RR of the object B. That is, the underground detection point FP2 is a feature point FP corresponding to a hole or a recess, and the ground high value should not be corrected to a positive value. Therefore, in this case, the CPU skips the process of step 650 and advances the process to step 660. The processing content of step 660 in FIG. 6C is the same as in FIG. 6A and FIG. 6B.
(変形例)
本開示は、上記実施形態に限定されるものではない。故に、上記実施形態に対しては、適宜変更が可能である。以下、代表的な変形例について説明する。以下の変形例の説明においては、上記実施形態との相違点を主として説明する。また、上記実施形態と変形例とにおいて、相互に同一または均等である部分には、同一符号が付されている。したがって、以下の変形例の説明において、上記実施形態と同一の符号を有する構成要素に関しては、技術的矛盾または特段の追加説明なき限り、上記実施形態における説明が適宜援用され得る。 (Modification)
The present disclosure is not limited to the above embodiment. Therefore, it can change suitably with respect to the said embodiment. Hereinafter, typical modifications will be described. In the following description of the modification, differences from the above embodiment will be mainly described. Moreover, in the said embodiment and a modification, the same code | symbol is attached | subjected to the part which is mutually the same or equivalent. Therefore, in the following description of the modified example, regarding the components having the same reference numerals as those in the above embodiment, the description in the above embodiment can be incorporated as appropriate unless there is a technical contradiction or special additional explanation.
本開示は、上記実施形態に限定されるものではない。故に、上記実施形態に対しては、適宜変更が可能である。以下、代表的な変形例について説明する。以下の変形例の説明においては、上記実施形態との相違点を主として説明する。また、上記実施形態と変形例とにおいて、相互に同一または均等である部分には、同一符号が付されている。したがって、以下の変形例の説明において、上記実施形態と同一の符号を有する構成要素に関しては、技術的矛盾または特段の追加説明なき限り、上記実施形態における説明が適宜援用され得る。 (Modification)
The present disclosure is not limited to the above embodiment. Therefore, it can change suitably with respect to the said embodiment. Hereinafter, typical modifications will be described. In the following description of the modification, differences from the above embodiment will be mainly described. Moreover, in the said embodiment and a modification, the same code | symbol is attached | subjected to the part which is mutually the same or equivalent. Therefore, in the following description of the modified example, regarding the components having the same reference numerals as those in the above embodiment, the description in the above embodiment can be incorporated as appropriate unless there is a technical contradiction or special additional explanation.
本開示は、上記実施形態にて示された具体的な装置構成に限定されない。すなわち、例えば、物体検知装置20を搭載する車両10は、四輪自動車に限定されない。具体的には、物体検知装置20を搭載する車両10は、三輪自動車であってもよいし、貨物トラック等の六輪または八輪自動車でもよい。また、物体検知装置20を搭載する車両10の種類は、内燃機関のみを備えた自動車であってもよいし、内燃機関を備えない電気自動車または燃料電池車であってもよいし、ハイブリッド自動車であってもよい。車体11の形状および構造も、箱状すなわち平面視における略矩形状に限定されない。ドアパネル17の数も、特段の限定はない。
The present disclosure is not limited to the specific apparatus configuration shown in the above embodiment. That is, for example, the vehicle 10 on which the object detection device 20 is mounted is not limited to a four-wheeled vehicle. Specifically, the vehicle 10 on which the object detection device 20 is mounted may be a three-wheeled vehicle or a six-wheeled or eight-wheeled vehicle such as a cargo truck. Further, the type of the vehicle 10 on which the object detection device 20 is mounted may be an automobile having only an internal combustion engine, an electric vehicle or a fuel cell vehicle not having an internal combustion engine, or a hybrid vehicle. There may be. The shape and structure of the vehicle body 11 are not limited to a box shape, that is, a substantially rectangular shape in plan view. The number of door panels 17 is not particularly limited.
物体検知装置20の適用対象は、駐車支援に限定されない。すなわち、例えば、物体検知装置20は、自動運転の定義におけるレベル2~レベル5に相当する、半自動運転または自動運転に対しても、好適に適用可能である。
The application target of the object detection device 20 is not limited to parking assistance. That is, for example, the object detection device 20 can be suitably applied to semi-automatic driving or automatic driving corresponding to levels 2 to 5 in the definition of automatic driving.
撮像部21を構成するイメージセンサは、CCDセンサに限定されない。すなわち、例えば、CCDセンサに代えて、CMOSセンサが用いられ得る。CMOSはComplementary MOSの略である。
The image sensor constituting the imaging unit 21 is not limited to a CCD sensor. That is, for example, a CMOS sensor can be used instead of the CCD sensor. CMOS is an abbreviation for Complementary MOS.
撮像部21の配置および個数は、上記の例に限定されない。すなわち、例えば、フロントカメラCFは、車室内に配置され得る。具体的には、例えば、フロントカメラCFは、車室内、例えばルームミラーに装着され得る。フロントカメラCFは、1個であってもよいし、2個であってもよい。すなわち、物体検知装置20は、複眼ステレオカメラ構成を有していてもよい。例えば、左側カメラCLおよび右側カメラCRは、ドアミラー18とは異なる位置に配置され得る。あるいは、左側カメラCLおよび右側カメラCRは、省略され得る。
The arrangement and number of the imaging units 21 are not limited to the above example. That is, for example, the front camera CF can be disposed in the vehicle interior. Specifically, for example, the front camera CF can be mounted in a vehicle interior, for example, a room mirror. The front camera CF may be one or two. That is, the object detection device 20 may have a compound eye stereo camera configuration. For example, the left camera CL and the right camera CR may be arranged at positions different from the door mirror 18. Alternatively, the left camera CL and the right camera CR can be omitted.
ソナーセンサ22の配置および個数は、上記の具体例に限定されない。すなわち、例えば、図1を参照すると、第三フロントソナーSF3が車幅方向における中央位置に配置される場合、第四フロントソナーSF4は省略される。同様に、第三リアソナーSR3が車幅方向における中央位置に配置される場合、第四リアソナーSR4は省略される。第三サイドソナーSS3および第四サイドソナーSS4は、省略され得る。
The arrangement and number of sonar sensors 22 are not limited to the above specific example. That is, for example, referring to FIG. 1, when the third front sonar SF3 is disposed at the center position in the vehicle width direction, the fourth front sonar SF4 is omitted. Similarly, when the third rear sonar SR3 is arranged at the center position in the vehicle width direction, the fourth rear sonar SR4 is omitted. The third side sonar SS3 and the fourth side sonar SS4 may be omitted.
制御部27は、上記実施形態における物体検知装置20の主要部を構成する。このため、撮像部21、ソナーセンサ22、レーダーセンサ23、車速センサ24、シフトポジションセンサ25、舵角センサ26、表示部28、および音声出力部29は、物体検知装置20の構成要素ではなく、物体検知装置20の付随的要素であるものと把握され得る。すなわち、本開示に係る構成は、制御部27にのみ設けられているものとも解釈され得る。
The control unit 27 constitutes a main part of the object detection device 20 in the above embodiment. For this reason, the imaging unit 21, the sonar sensor 22, the radar sensor 23, the vehicle speed sensor 24, the shift position sensor 25, the steering angle sensor 26, the display unit 28, and the audio output unit 29 are not constituent elements of the object detection device 20, It can be grasped as an incidental element of the detection device 20. That is, the configuration according to the present disclosure can be interpreted as being provided only in the control unit 27.
上記実施形態においては、制御部27は、CPUがROM等からプログラムを読み出して起動する構成であった。しかしながら、本開示は、かかる構成に限定されない。すなわち、例えば、制御部27は、上記のような動作を可能に構成されたデジタル回路、例えばゲートアレイ等のASICであってもよい。ASICはAPPLICATION SPECIFIC INTEGRATED CIRCUITの略である。
In the above embodiment, the control unit 27 has a configuration in which the CPU reads and starts the program from the ROM or the like. However, the present disclosure is not limited to such a configuration. That is, for example, the control unit 27 may be a digital circuit configured to be able to operate as described above, for example, an ASIC such as a gate array. ASIC is an abbreviation of APPLICATION SPECIFIC INTEGRATED CIRCUIT.
マンホール等の穴あるいは溝等の凹部も、駐車支援、運転支援、あるいは自動運転に際して、障害物となる。この点では、路面RSから上方に突出した障害物である物体Bと同様である。このため、穴あるいは凹部に対応する特徴点FPに対して、反射像VR上の地下検知点FP2と同様に地上高値を正値に補正しても、駐車支援等に際しては特段の問題は生じない可能性がある。したがって、路面状態判定部705および路面反射判定部706は、省略され得る。
¡Holes such as manholes or recesses such as grooves also become obstacles during parking assistance, driving assistance, or automatic driving. This is the same as the object B that is an obstacle protruding upward from the road surface RS. For this reason, even if the ground high value is corrected to a positive value as in the case of the underground detection point FP2 on the reflected image VR with respect to the feature point FP corresponding to the hole or the recess, no particular problem occurs in parking assistance or the like. there is a possibility. Therefore, the road surface state determination unit 705 and the road surface reflection determination unit 706 can be omitted.
本開示は、上記実施形態にて示された具体的な動作例に限定されない。すなわち、例えば、移動量取得部702による、自車両の移動量の取得には、不図示の加速度センサの出力が用いられ得る。
This disclosure is not limited to the specific operation example shown in the above embodiment. In other words, for example, the output of an acceleration sensor (not shown) can be used for acquisition of the movement amount of the host vehicle by the movement amount acquisition unit 702.
座標取得部703における処理内容は、単眼移動ステレオに限定されない。具体的には、例えば、複眼ステレオ処理、またはSFMと複眼ステレオとの統合処理が用いられ得る。複眼ステレオ処理、またはSFMと複眼ステレオとの統合処理については、本願の出願時において、すでに公知または周知である。
The processing content in the coordinate acquisition unit 703 is not limited to monocular moving stereo. Specifically, for example, compound eye stereo processing or integration processing of SFM and compound eye stereo can be used. Compound eye stereo processing or integration processing of SFM and compound eye stereo is already known or known at the time of filing of the present application.
距離取得部704は、ソナーセンサ22の出力に代えて、レーダーセンサ23の出力に基づいて、測距情報を取得してもよい。
The distance acquisition unit 704 may acquire distance measurement information based on the output of the radar sensor 23 instead of the output of the sonar sensor 22.
路面状態判定部705における処理内容は、上記の例に限定されない。すなわち、例えば、路面状態判定部705は、現在の自車両の進行方向側に存在する撮像部21による撮影画像中に、撮像部21に付着した水滴が確認された場合に、路面RSが反射性路面である可能性があると判定してもよい。
The processing content in the road surface state determination unit 705 is not limited to the above example. That is, for example, the road surface state determination unit 705 determines that the road surface RS is reflective when water droplets attached to the imaging unit 21 are confirmed in the captured image by the imaging unit 21 present on the current traveling direction side of the host vehicle. It may be determined that there is a possibility of a road surface.
あるいは、例えば、路面状態判定部705は、レーダーセンサ23に対する雨滴の付着を検知した場合に、路面RSが反射性路面である可能性があると判定してもよい。
Alternatively, for example, the road surface state determination unit 705 may determine that the road surface RS may be a reflective road surface when the adhesion of raindrops to the radar sensor 23 is detected.
反射性路面としては、水膜の他にも、凍結路面の可能性がある。このため、例えば、路面状態判定部705は、ABSあるいはESCが作動したことを制動制御ECU52等から受信し、且つ外気温が所定温度未満である場合に、路面RSが反射性路面である可能性があると判定してもよい。ABSはAntilock Brake Systemの略である。ESCはElectronic Stability Controlの略である。
As a reflective road surface, there is a possibility of a frozen road surface in addition to a water film. Therefore, for example, when the road surface condition determination unit 705 receives from the braking control ECU 52 that the ABS or ESC has been operated and the outside air temperature is lower than a predetermined temperature, the road surface RS may be a reflective road surface. It may be determined that there is. ABS is an abbreviation for Antilock Brake System. ESC stands for Electronic Stability Control.
路面状態判定部705は、上記の各要素の組み合わせに基づいて、路面状態を判定してもよい。上記の各要素とは、雨滴センサ31の出力、ワイパー機構41の作動状態、撮像部21による雨滴検知状態、レーダーセンサ23による雨滴検知状態、不図示の外気温センサの出力、ABS作動状態、ESC作動状態、等を含む。
The road surface state determination unit 705 may determine the road surface state based on the combination of the above elements. The above elements are the output of the raindrop sensor 31, the operating state of the wiper mechanism 41, the raindrop detection state by the imaging unit 21, the raindrop detection state by the radar sensor 23, the output of the outside air temperature sensor (not shown), the ABS operation state, ESC Including operating conditions, etc.
あるいは、例えば、路面状態判定部705は、車-車間通信あるいは車-外間通信を用いて自車両の外部から取得された情報(例えば気象情報等)に基づいて、車両10の周囲の路面RSの状態を判定してもよい。車-車間通信は「V2V」とも称され得る。V2VはVehicle-to-Vehicleの略である。車-外間通信は「V2I」とも称され得る。V2IはVehicle-to-roadside-Infrastructureの略であり、「路車間通信」とも称される。両者を包含する概念は、「V2X」と称される。「V2X」は「vehicle-to-any」とも称され得る。
Alternatively, for example, the road surface state determination unit 705 may determine the road surface RS around the vehicle 10 based on information (for example, weather information) acquired from outside the host vehicle using vehicle-to-vehicle communication or vehicle-to-vehicle communication. The state may be determined. Car-to-vehicle communication may also be referred to as “V2V”. V2V is an abbreviation for Vehicle-to-Vehicle. The car-outside communication may also be referred to as “V2I”. V2I is an abbreviation for Vehicle-to-roadside-Infrastructure, and is also referred to as “road-to-vehicle communication”. The concept encompassing both is referred to as “V2X”. “V2X” may also be referred to as “vehicle-to-any”.
路面反射判定部706における判定手法も、上記の各具体例に限定されない。すなわち、例えば、路面反射判定部706は、地上検知点FP1の真下に地下検知点FP2が存在する場合に、地下検知点FP2が反射像VRに基づくものであると判定してもよい。「地上検知点FP1の真下に地下検知点FP2が存在する」とは、地上検知点FP1と地下検知点FP2とで、2次元座標情報が互いに同一あるいは略同一であることをいう。2次元座標情報とは、上記の通り、3次元座標情報における地上高値以外の情報である。
The determination method in the road surface reflection determination unit 706 is not limited to the above specific examples. That is, for example, the road surface reflection determination unit 706 may determine that the underground detection point FP2 is based on the reflected image VR when the underground detection point FP2 exists immediately below the ground detection point FP1. “The underground detection point FP2 exists immediately below the ground detection point FP1” means that the two-dimensional coordinate information is the same or substantially the same at the ground detection point FP1 and the underground detection point FP2. As described above, the two-dimensional coordinate information is information other than the ground high value in the three-dimensional coordinate information.
路面反射判定部706における判定は、複数の判定手法の組み合わせを用いてもよい。すなわち、例えば、路面反射判定部706は、以下の条件(1)~(3)のうちの少なくとも2つが成立する場合に、地上高値が負値の3次元座標情報を有する地下検知点FP2が路面反射による反射像VRに基づくものと判定してもよい。
・条件(1):地上高値の符号を反転した関係の地上検知点FP1と地下検知点FP2との組が存在すること。
・条件(2):基準点RPと地上検知点FP1とを結ぶ仮想線分VL上、あるいは、仮想線分VLの近傍に、地下検知点FP2が存在すること。
・条件(3):測距点から所定距離範囲内に地下検知点FP2が存在すること。 The determination in the road surfacereflection determination unit 706 may use a combination of a plurality of determination methods. That is, for example, when at least two of the following conditions (1) to (3) are satisfied, the road surface reflection determination unit 706 determines that the underground detection point FP2 having three-dimensional coordinate information having a negative ground high value is the road surface. It may be determined that it is based on the reflection image VR by reflection.
Condition (1): A pair of ground detection point FP1 and underground detection point FP2 having a relationship in which the sign of the ground high value is inverted exists.
Condition (2): An underground detection point FP2 exists on the virtual line segment VL connecting the reference point RP and the ground detection point FP1 or in the vicinity of the virtual line segment VL.
Condition (3): An underground detection point FP2 exists within a predetermined distance range from the distance measuring point.
・条件(1):地上高値の符号を反転した関係の地上検知点FP1と地下検知点FP2との組が存在すること。
・条件(2):基準点RPと地上検知点FP1とを結ぶ仮想線分VL上、あるいは、仮想線分VLの近傍に、地下検知点FP2が存在すること。
・条件(3):測距点から所定距離範囲内に地下検知点FP2が存在すること。 The determination in the road surface
Condition (1): A pair of ground detection point FP1 and underground detection point FP2 having a relationship in which the sign of the ground high value is inverted exists.
Condition (2): An underground detection point FP2 exists on the virtual line segment VL connecting the reference point RP and the ground detection point FP1 or in the vicinity of the virtual line segment VL.
Condition (3): An underground detection point FP2 exists within a predetermined distance range from the distance measuring point.
なお、上記の条件(1)は、下記の条件(4)と置換され得る。
・条件(4):地上検知点FP1の真下に地下検知点FP2が存在すること。 The above condition (1) can be replaced with the following condition (4).
Condition (4): An underground detection point FP2 exists immediately below the ground detection point FP1.
・条件(4):地上検知点FP1の真下に地下検知点FP2が存在すること。 The above condition (1) can be replaced with the following condition (4).
Condition (4): An underground detection point FP2 exists immediately below the ground detection point FP1.
また、上記の条件(2)は、下記の条件(5)と等価である。
・条件(5):基準点RPを端点として当該端点から地下検知点FP2に向かう仮想半直線上、あるいは、かかる仮想半直線の近傍に、地上検知点FP1が存在すること。 The above condition (2) is equivalent to the following condition (5).
Condition (5): The ground detection point FP1 exists on or near the virtual half line from the end point to the underground detection point FP2 with the reference point RP as an end point.
・条件(5):基準点RPを端点として当該端点から地下検知点FP2に向かう仮想半直線上、あるいは、かかる仮想半直線の近傍に、地上検知点FP1が存在すること。 The above condition (2) is equivalent to the following condition (5).
Condition (5): The ground detection point FP1 exists on or near the virtual half line from the end point to the underground detection point FP2 with the reference point RP as an end point.
「取得」という表現と、「推定」「検出」「検知」「算出」等の類似の表現とは、技術的に矛盾しない範囲内において、適宜置換可能である。各判定処理における不等号は、等号付きであってもよいし、等号無しであってもよい。すなわち、例えば、「所定値未満」と「所定値以下」とは、技術的に矛盾しない範囲内において、互いに置換され得る。
The expression “acquisition” and similar expressions such as “estimation”, “detection”, “detection”, and “calculation” can be appropriately replaced within a technically consistent range. The inequality sign in each determination process may be with or without an equal sign. That is, for example, “less than a predetermined value” and “below a predetermined value” can be replaced with each other within a technically consistent range.
上記実施形態を構成する要素は、特に必須であると明示した場合および原理的に明らかに必須であると考えられる場合等を除き、必ずしも必須のものではないことは言うまでもない。また、構成要素の個数、数値、量、範囲等の数値が言及されている場合、特に必須であると明示した場合および原理的に明らかに特定の数に限定される場合等を除き、その特定の数に本開示が限定されることはない。同様に、構成要素等の形状、方向、位置関係等が言及されている場合、特に必須であると明示した場合および原理的に特定の形状、方向、位置関係等に限定される場合等を除き、その形状、方向、位置関係等に本開示が限定されることはない。
It goes without saying that the elements constituting the above-described embodiment are not necessarily indispensable except for the case where it is clearly indicated that the element is essential and the case where the element is clearly considered to be essential in principle. In addition, when numerical values such as the number, numerical value, quantity, range, etc. of a component are mentioned, the specifics are specified unless explicitly stated as being essential, and clearly limited to a specific number in principle. The present disclosure is not limited to any number. Similarly, when the shape, direction, positional relationship, etc. of a component are mentioned, except when clearly stated as essential and in principle limited to a specific shape, direction, positional relationship, etc. The present disclosure is not limited to the shape, direction, positional relationship, and the like.
変形例も、上記の例示に限定されない。また、複数の変形例が、互いに組み合わされ得る。さらに、上記実施形態の全部または一部と、変形例の全部または一部とが、互いに組み合わされ得る。
The modified examples are not limited to the above examples. A plurality of modifications may be combined with each other. Furthermore, all or a part of the above-described embodiment and all or a part of the modified examples can be combined with each other.
Claims (8)
- 自車両(10)に搭載されることで当該自車両の周囲に存在する物体(B)を検知するように構成された、物体検知装置(20)であって、
前記自車両の周囲の画像に対応する画像情報を取得するように設けられた撮像部(21)により取得された前記画像情報に基づいて、前記物体上の特徴点における、前記自車両の接地面を平面と仮定した場合の当該平面である基準面を基準とした高さである地上高値を含む3次元座標情報を取得するように設けられた、座標取得部(703)と、
前記座標取得部によって取得された、前記地上高値が前記基準面よりも低い負値となる前記3次元座標情報に対して、前記地上高値が前記基準面よりも高い正値となるように補正するように設けられた、座標補正部(707)と、
を備えた物体検知装置。 An object detection device (20) configured to detect an object (B) existing around the host vehicle by being mounted on the host vehicle (10),
Based on the image information acquired by the imaging unit (21) provided to acquire image information corresponding to an image around the host vehicle, the ground plane of the host vehicle at a feature point on the object A coordinate acquisition unit (703) provided to acquire three-dimensional coordinate information including a ground height value that is a height with reference to a reference plane that is the plane when
For the three-dimensional coordinate information acquired by the coordinate acquisition unit and having a negative value lower than the reference surface, the ground high value is corrected to be a positive value higher than the reference surface. A coordinate correction unit (707) provided as follows:
An object detection device comprising: - 前記座標取得部によって取得された、前記地上高値が前記負値の前記3次元座標情報が、路面(RS)に映し出された反射像(VR)に基づくものであることを判定するように設けられた、路面反射判定部(706)をさらに備え、
前記座標補正部は、前記地上高値が前記負値の前記3次元座標情報が前記反射像に基づくものであると前記路面反射判定部が判定した場合に、当該3次元座標情報における前記地上高値を前記基準面よりも高い正値に補正するように設けられた、
請求項1に記載の物体検知装置。 Provided to determine that the three-dimensional coordinate information acquired by the coordinate acquisition unit and having a negative ground high value is based on a reflected image (VR) projected on a road surface (RS). A road surface reflection determination unit (706);
The coordinate correction unit, when the road surface reflection determination unit determines that the three-dimensional coordinate information with the ground height value being the negative value is based on the reflected image, determines the ground height value in the three-dimensional coordinate information. Provided to correct to a higher positive value than the reference surface,
The object detection apparatus according to claim 1. - 前記路面反射判定部は、前記地上高値が前記負値の前記3次元座標情報における前記地上高値の符号を正に反転させた位置から所定範囲内にて、前記地上高値が前記正値の前記3次元座標情報が前記座標取得部によって取得されている場合に、前記地上高値が前記負値の前記3次元座標情報が前記反射像に基づくものであると判定するように設けられた、
請求項2に記載の物体検知装置。 The road surface reflection determination unit is configured so that the ground high value is the positive value within a predetermined range from a position where the sign of the ground high value in the three-dimensional coordinate information in which the ground high value is the negative value is reversed positively. When the three-dimensional coordinate information is acquired by the coordinate acquisition unit, the ground high value is provided to determine that the three-dimensional coordinate information of the negative value is based on the reflected image,
The object detection apparatus according to claim 2. - 前記路面反射判定部は、画像情報にて、所定の基準点(RP)と前記地上高値が前記正値の前記3次元座標情報に対応する前記特徴点(FP1)とを結ぶ仮想線分(VL)上に、前記地上高値が前記負値の前記3次元座標情報に対応する前記特徴点(FP2)が存在する場合に、前記地上高値が前記負値の前記3次元座標情報が前記反射像に基づくものであると判定するように設けられた、
請求項2に記載の物体検知装置。 The road surface reflection determination unit uses a virtual line segment (VL) connecting a predetermined reference point (RP) and the feature point (FP1) corresponding to the three-dimensional coordinate information whose ground high value is the positive value in the image information. ), When the feature point (FP2) corresponding to the three-dimensional coordinate information with the ground high value being the negative value is present, the three-dimensional coordinate information with the ground high value being the negative value is included in the reflected image. Provided to determine that it is based on,
The object detection apparatus according to claim 2. - 前記物体との距離を取得するように設けられた、距離取得部(704)をさらに備え、
前記路面反射判定部は、前記距離取得部によって取得された前記距離から所定距離範囲内に、前記地上高値が前記負値の前記3次元座標情報に対応する前記特徴点が存在する場合に、前記地上高値が前記負値の前記3次元座標情報が前記反射像に基づくものであると判定するように設けられた、
請求項2に記載の物体検知装置。 A distance acquisition unit (704) provided to acquire a distance to the object;
The road surface reflection determination unit, when the feature point corresponding to the three-dimensional coordinate information in which the ground high value is the negative value exists within a predetermined distance range from the distance acquired by the distance acquisition unit, Provided to determine that the three-dimensional coordinate information with the ground high value being the negative value is based on the reflected image,
The object detection apparatus according to claim 2. - 前記路面の状態を判定するように設けられた、路面状態判定部(705)をさらに備え、
前記路面反射判定部は、前記路面が前記物体の前記反射像を映し出す反射性路面であることを前記路面状態判定部が判定した場合に判定動作を実行するように設けられた、
請求項2~5のいずれか1つに記載の物体検知装置。 A road surface state determination unit (705) provided to determine the state of the road surface;
The road surface reflection determination unit is provided to perform a determination operation when the road surface state determination unit determines that the road surface is a reflective road surface that reflects the reflected image of the object.
The object detection device according to any one of claims 2 to 5. - 前記自車両の移動量を取得するように設けられた、移動量取得部(702)をさらに備え、
前記座標取得部は、前記画像情報の履歴と前記移動量とに基づいて、前記3次元座標情報を取得するように設けられた、
請求項1~6のいずれか1つに記載の物体検知装置。 A movement amount acquisition unit (702) provided to acquire the movement amount of the host vehicle;
The coordinate acquisition unit is provided to acquire the three-dimensional coordinate information based on a history of the image information and the movement amount.
The object detection device according to any one of claims 1 to 6. - 前記自車両の駐車を支援するように構成された、駐車支援装置(50)であって、
請求項1~7のいずれか1つに記載の物体検知装置(20)と、
前記物体検知装置による検知結果に基づいて、前記自車両が駐車可能な駐車領域を検出するように設けられた、駐車領域検出部(271)と、
を備えた駐車支援装置。 A parking assistance device (50) configured to assist parking of the host vehicle,
The object detection device (20) according to any one of claims 1 to 7,
A parking area detection unit (271) provided to detect a parking area where the host vehicle can be parked based on a detection result by the object detection device;
Parking assistance device with
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
JP2018013865A JP2019133324A (en) | 2018-01-30 | 2018-01-30 | Object detection device and parking assist device |
JP2018-013865 | 2018-01-30 |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2019151156A1 true WO2019151156A1 (en) | 2019-08-08 |
Family
ID=67478132
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/JP2019/002568 WO2019151156A1 (en) | 2018-01-30 | 2019-01-25 | Object detection device and parking assistance device |
Country Status (2)
Country | Link |
---|---|
JP (1) | JP2019133324A (en) |
WO (1) | WO2019151156A1 (en) |
Cited By (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN111583707A (en) * | 2020-05-28 | 2020-08-25 | 大洋泊车股份有限公司 | Intelligent guiding method for three-dimensional parking lot |
CN112644471A (en) * | 2019-10-11 | 2021-04-13 | 丰田自动车株式会社 | Vehicle parking assist apparatus |
Families Citing this family (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP7358933B2 (en) * | 2019-11-19 | 2023-10-11 | 株式会社アイシン | camera calibration device |
Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2007256029A (en) * | 2006-03-23 | 2007-10-04 | Denso It Laboratory Inc | Stereo image processing device |
JP2014013453A (en) * | 2012-07-03 | 2014-01-23 | Clarion Co Ltd | In-vehicle surrounding environment recognition apparatus |
JP2016009429A (en) * | 2014-06-26 | 2016-01-18 | 富士重工業株式会社 | Vehicle exterior environment recognition device |
Family Cites Families (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP6760298B2 (en) * | 2015-09-30 | 2020-09-23 | ソニー株式会社 | Information acquisition device and information acquisition method |
-
2018
- 2018-01-30 JP JP2018013865A patent/JP2019133324A/en active Pending
-
2019
- 2019-01-25 WO PCT/JP2019/002568 patent/WO2019151156A1/en active Application Filing
Patent Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2007256029A (en) * | 2006-03-23 | 2007-10-04 | Denso It Laboratory Inc | Stereo image processing device |
JP2014013453A (en) * | 2012-07-03 | 2014-01-23 | Clarion Co Ltd | In-vehicle surrounding environment recognition apparatus |
JP2016009429A (en) * | 2014-06-26 | 2016-01-18 | 富士重工業株式会社 | Vehicle exterior environment recognition device |
Cited By (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN112644471A (en) * | 2019-10-11 | 2021-04-13 | 丰田自动车株式会社 | Vehicle parking assist apparatus |
CN111583707A (en) * | 2020-05-28 | 2020-08-25 | 大洋泊车股份有限公司 | Intelligent guiding method for three-dimensional parking lot |
Also Published As
Publication number | Publication date |
---|---|
JP2019133324A (en) | 2019-08-08 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN110573905B (en) | Obstacle detection device | |
US20170267237A1 (en) | Vehicle travel control device | |
US20120101704A1 (en) | Method for operating at least one sensor of a vehicle and vehicle having at least one sensor | |
KR20200029050A (en) | Driving control method and driving control device for driving support vehicles | |
CN108216033B (en) | Vehicle periphery monitoring device | |
WO2018079252A1 (en) | Object detecting device | |
WO2019151156A1 (en) | Object detection device and parking assistance device | |
CN102725168A (en) | Method for assisting a driver in detecting lateral objects | |
JP6673181B2 (en) | Parking assistance device and its control unit | |
CN112995584B (en) | Display device and parking assistance system for vehicle | |
CN110730735B (en) | Parking assist method and parking assist apparatus | |
WO2018225347A1 (en) | Driving-obstacle detecting device and vehicle navigation system | |
CN112977426A (en) | Parking assist system | |
CN111918803B (en) | Driving support control device, driving support system, and driving support control method | |
JP2018130994A (en) | Parking support device | |
KR20210005439A (en) | Advanced Driver Assistance System, Vehicle having the same and method for controlling the vehicle | |
JP2019156308A (en) | Parking support device | |
JP7135579B2 (en) | Object detection device | |
CN113753069A (en) | Automatic driving control method and device | |
CN110462706B (en) | Obstacle detection device | |
JP4768499B2 (en) | In-vehicle peripheral other vehicle detection device | |
WO2021172532A1 (en) | Parking assistance device and parking assistance method | |
JP6958485B2 (en) | Object detection device | |
WO2021075377A1 (en) | Object detection device, object detection method, and object detection program | |
JP7334572B2 (en) | Object detection device and object detection program |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 19746991 Country of ref document: EP Kind code of ref document: A1 |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
122 | Ep: pct application non-entry in european phase |
Ref document number: 19746991 Country of ref document: EP Kind code of ref document: A1 |