WO2017022080A1 - 段差検出装置及び段差検出方法 - Google Patents
段差検出装置及び段差検出方法 Download PDFInfo
- Publication number
- WO2017022080A1 WO2017022080A1 PCT/JP2015/072085 JP2015072085W WO2017022080A1 WO 2017022080 A1 WO2017022080 A1 WO 2017022080A1 JP 2015072085 W JP2015072085 W JP 2015072085W WO 2017022080 A1 WO2017022080 A1 WO 2017022080A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- road surface
- vehicle
- curve
- area
- circuit
- Prior art date
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 48
- 238000005259 measurement Methods 0.000 claims abstract description 52
- 238000000034 method Methods 0.000 description 23
- 238000012545 processing Methods 0.000 description 12
- 238000012937 correction Methods 0.000 description 7
- 238000011156 evaluation Methods 0.000 description 5
- 238000012986 modification Methods 0.000 description 5
- 230000004048 modification Effects 0.000 description 5
- 238000004364 calculation method Methods 0.000 description 4
- 230000007423 decrease Effects 0.000 description 4
- 238000003384 imaging method Methods 0.000 description 4
- 230000010365 information processing Effects 0.000 description 4
- 238000006243 chemical reaction Methods 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 238000007429 general method Methods 0.000 description 2
- 238000004590 computer program Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000006073 displacement reaction Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000001678 irradiating effect Effects 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 238000004441 surface measurement Methods 0.000 description 1
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G08—SIGNALLING
- G08G—TRAFFIC CONTROL SYSTEMS
- G08G1/00—Traffic control systems for road vehicles
- G08G1/16—Anti-collision systems
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01B—MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
- G01B11/00—Measuring arrangements characterised by the use of optical techniques
- G01B11/24—Measuring arrangements characterised by the use of optical techniques for measuring contours or curvatures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
- G06T7/285—Analysis of motion using a sequence of stereo image pairs
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/50—Depth or shape recovery
- G06T7/55—Depth or shape recovery from multiple images
- G06T7/593—Depth or shape recovery from multiple images from stereo images
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
- G06V20/58—Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
- G06V20/588—Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road
-
- G—PHYSICS
- G08—SIGNALLING
- G08G—TRAFFIC CONTROL SYSTEMS
- G08G1/00—Traffic control systems for road vehicles
- G08G1/01—Detecting movement of traffic to be counted or controlled
- G08G1/056—Detecting movement of traffic to be counted or controlled with provision for distinguishing direction of travel
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
- H04N13/20—Image signal generators
- H04N13/204—Image signal generators using stereoscopic image cameras
- H04N13/207—Image signal generators using stereoscopic image cameras using a single 2D image sensor
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
- H04N13/20—Image signal generators
- H04N13/204—Image signal generators using stereoscopic image cameras
- H04N13/239—Image signal generators using stereoscopic image cameras using two 2D image sensors having a relative position equal to or related to the interocular distance
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10016—Video; Image sequence
- G06T2207/10021—Stereoscopic video; Stereoscopic image sequence
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30248—Vehicle exterior or interior
- G06T2207/30252—Vehicle exterior; Vicinity of vehicle
- G06T2207/30256—Lane; Road marking
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30248—Vehicle exterior or interior
- G06T2207/30252—Vehicle exterior; Vicinity of vehicle
- G06T2207/30261—Obstacle
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/50—Depth or shape recovery
- G06T7/521—Depth or shape recovery from laser ranging, e.g. using interferometry; from the projection of structured light
-
- G—PHYSICS
- G08—SIGNALLING
- G08G—TRAFFIC CONTROL SYSTEMS
- G08G1/00—Traffic control systems for road vehicles
- G08G1/16—Anti-collision systems
- G08G1/165—Anti-collision systems for passive traffic, e.g. including static obstacles, trees
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
- H04N2013/0074—Stereoscopic image analysis
- H04N2013/0081—Depth or disparity estimation from stereoscopic image signals
Definitions
- the present invention relates to a step detecting device and a step detecting method for detecting a step on a road surface.
- the roadside object detection device described above recognizes a step when the amount of change in the height of the road surface is equal to or greater than a threshold. Therefore, when the roadway is inclined, the height of the road surface changes according to the position of the roadway. Therefore, the amount of change in the height of the road surface may not be detected correctly depending on how the roadway is positioned.
- the present invention has been made in view of the above problems, and an object of the present invention is to provide a step detecting device and a step detecting method capable of accurately detecting a step on the road surface even when the roadway is inclined. It is.
- a level difference detection device includes a distance measuring unit that detects a height of a road surface around a vehicle, and curves a road surface inclination in a vehicle width direction of a travelable region based on the road surface height. Based on the degree of coincidence of the height of the measurement point with respect to the curve and the degree of coincidence of the height of the measurement point with respect to the curve approximating the slope of the road surface in the extended area obtained by extending the travelable area in the vehicle width direction, Detects steps on the road surface.
- a step on the road surface can be accurately detected even if the roadway is inclined.
- FIG. 1 is a block diagram illustrating an overall configuration of a level difference detection device 1 according to the embodiment.
- FIG. 2 is a perspective view showing an example of a linear step determination position (Pa) set on the road surface around the vehicle.
- FIG. 3 is an overhead view corresponding to FIG.
- FIG. 4 is a graph showing the height distribution of the road surface (Froad) at the step determination position (Pa).
- the area (G 1 to G 4 ) approximating the road surface (Froad) in a curve is expanded step by step by a predetermined range (Kc), and the measurement points (I m1 to I m7 ) and the curve (Croad) It is a figure which shows a series of processing cycles which judge the change of a coincidence degree.
- FIG. 5 shows a series of processing cycles which judge the change of a coincidence degree.
- FIG. 6 is a flowchart showing an example of a step detection method using the step detection device 1 of FIG.
- FIG. 7 is a flowchart showing an example of a detailed procedure of steps S09 to S13 of FIG.
- FIG. 8 is a perspective view showing an example of the step determination position (Pa) when a stereo camera is installed on the side of the vehicle (Vc).
- FIG. 9 is an overhead view showing an example of a 360 degree LRF in which the LRF (12) is set at the center of the roof of the vehicle (Vc) and the entire area around the vehicle is an irradiation range.
- FIG. 10 is a flowchart illustrating an example of a step detection method when a laser range finder (LRF) is used as the distance measuring sensor 12.
- FIG. 11 is a block diagram illustrating an overall configuration of the level difference detection device 2 according to the third and fourth modifications.
- the level difference detection device 1 detects the height of the road surface (hereinafter referred to as “road surface”) around the vehicle, and approximates the slope of the road surface with a curve. Then, a step on the road surface is detected based on the degree of coincidence between the approximated curve and the measurement point on the road surface.
- road surface the road surface
- a step on the road surface is detected based on the degree of coincidence between the approximated curve and the measurement point on the road surface.
- the level difference detection device 1 includes a distance measurement sensor 12 that detects the height of a road surface around the vehicle, and a series of information processing that detects a level difference on the road surface from height data of measurement points measured by the distance measurement sensor 12. And a microcomputer 13 for executing
- An example of the distance measuring sensor 12 can record information on the depth direction (distance from the sensor 12) of the object around the vehicle by simultaneously photographing the object around the vehicle from a plurality of different directions. It is a stereo camera. By performing predetermined image processing on the stereo image obtained by the stereo camera, it is possible to acquire three-dimensional information about the object image shown in the stereo image of the object around the vehicle. Objects around the vehicle include roads and curbs. Details will be described later.
- the microcomputer 13 is composed of, for example, a general-purpose microcontroller including a CPU, a memory, and an input / output unit, and configures a plurality of information processing circuits included in the step detection device by executing a computer program installed in advance.
- the microcomputer 13 repeatedly executes a series of information processing cycles for detecting a step on the road surface from measurement data obtained by the distance measuring sensor 12 at predetermined time intervals.
- the microcomputer 13 may also be used as an electronic control unit (ECU) used for other control related to the vehicle.
- ECU electronice control unit
- the plurality of information processing circuits configured by the microcomputer 13 include an arithmetic circuit 14, a step determination position circuit 15, a travelable area circuit 16, a road surface inclination circuit 17, and a step detection circuit 18.
- the arithmetic circuit 14 constitutes a distance measuring unit 11 together with the distance measuring sensor 12 and acquires a series of stereo images for acquiring three-dimensional information on the image of the object shown in the stereo image of the object around the vehicle from the stereo image obtained by the stereo camera. Perform the process.
- the arithmetic circuit 14 performs lens distortion correction processing for correcting lens distortion on the stereo image, and performs parallelization correction processing (parallel equivalence processing) for correcting the vertical position between the stereo images. Then, a stereo matching process for estimating the correspondence of each pixel between stereo images is performed. Thereby, not only the two-dimensional coordinates of the object on the imaging surface of the stereo camera but also the distance from the imaging surface of the stereo camera to the object can be calculated. Therefore, the distance and direction to the object around the vehicle can be detected.
- the arithmetic circuit 14 can acquire three-dimensional information of an object around the vehicle on the coordinates of the distance measurement data by further performing a coordinate conversion process.
- the three-dimensional information of the object around the vehicle includes three-dimensional information of the road surface around the vehicle. Therefore, the arithmetic circuit 14 can acquire the height data of the measurement points on the road surface around the vehicle.
- this processing may be a general method for correcting lens distortion, and is not particularly limited in the present embodiment.
- Parallelization correction processing is performed by taking a flat plate representing a black and white checkered pattern with both cameras of the stereo camera, for example, so that the positions of the checkered lattice points are the same vertical position on each camera image.
- the spatial position parameter and the angle parameter are estimated.
- this process may be a general technique for performing the parallelization correction process, and is not particularly limited in the present embodiment.
- the stereo matching process calculates, for example, which pixel of the left camera image is associated with which pixel of the right camera image on the basis of the left camera image. For example, the absolute value of the luminance value of each pixel of the left camera image and the luminance value of each pixel of the right camera image is calculated as an evaluation value, and the pixel of the right camera image having the smallest evaluation value is associated with the pixel. calculate.
- the evaluation value calculation method includes, for example, a method using a sum of absolute differences (SAD: Sum of Absolute Differences) or a sum of squared differences (SSD: Sum of Squared Differences), and a range of evaluation value calculation for each pixel. There is a method including peripheral pixels of each pixel instead of one point.
- the calculation method of the evaluation value may be another general method and is not particularly limited in the present embodiment.
- the step determination position circuit 15 sets a linear step determination position in the vehicle width direction on the road surface around the vehicle on the coordinates of the distance measurement data detected by the distance measurement sensor 12. For example, as shown in FIG. 2 and FIG. 3, a step determination position (Pa) extending from the distance measuring sensor 12 by a predetermined distance in the first predetermined direction (Da) and extending in a direction orthogonal to the first predetermined direction (Da). ) Is set on the road surface on the coordinates of the distance measurement data. 2 and 3 show an example in which the distance measuring sensor 12 is installed at the front of the vehicle (Vc) and the traveling direction of the vehicle (Vc) is the first predetermined direction (Da).
- the step determination position (Pa) extending in the vehicle width direction is set in front of the vehicle (Vc) on the coordinates of the distance measurement data.
- the step determination position (Pa) is set within the imaging range of the stereo camera.
- the first predetermined direction (Da) is not limited to the traveling direction of the vehicle (Vc).
- a step (LD) in which the height of the road surface changes discontinuously is formed on the shoulder of the road in the vehicle width direction of the roadway on which the vehicle (Vc) can travel.
- a step portion (for example, a sidewalk or a road shoulder) that is one step higher than the roadway is provided outside the roadway with the step (LD) as a boundary.
- the road is composed of a roadway and a stepped portion (a sidewalk or a shoulder), and a step (LD) is formed at the boundary between the roadway and the stepped portion (a sidewalk or a shoulder).
- the linear step determination position (Pa) extends in a direction crossing the roadway, the step (LD), and the step portion (sidewalk or shoulder).
- the step determination position circuit 15 calculates the height data distribution (ranging data) of the measurement points on the road surface at the step determination position (Pa) from the road surface height acquired by the arithmetic circuit 14.
- the vertical axis in FIG. 4 indicates the height of the road surface (Froad), and the horizontal axis indicates the step determination position (Pa) extending in the vehicle width direction.
- a step (LD) is formed at the boundary between the roadway (Rr) and the step portion (Rd).
- the road surface (Froad) of the roadway (Rr) is provided with a slope (cant) whose height decreases from the center toward the road shoulder at both ends.
- This is a general road design and structure that improves the drainage of the roadway and prevents water from accumulating on the roadway.
- the road surface (Froad) increases rapidly, and the road surface (Froad) of the level difference portion (Rd) forms a flat surface that is one step higher than the roadway (Rr). For example, a sidewalk is provided.
- the travelable area circuit 16 estimates a travelable area (G 1 ) in the traveling direction of the vehicle (Vc) at the step determination position (Pa) based on the speed of the vehicle (Vc).
- the travelable region (G 1 ) is a region in which the vehicle (Vc) can travel and indicates a boundary of the roadway, for example, a region that does not include a step.
- the travelable region circuit 16 sets a region in which a predetermined travel margin region is added to the width of the vehicle (Vc) as a travelable region (G 1 ) in the traveling direction of the vehicle (Vc). presume.
- the predetermined travel margin region changes according to the speed of the vehicle.
- the speed of the vehicle (Vc) can be obtained by continuously acquiring the vehicle speed information of the CAN, the wheel speed pulse information, and the position information of the vehicle using GPS.
- the travelable area circuit 16 determines that it is traveling on a main road and sets a travelable area of 8 m width.
- the travelable area circuit 16 determines that it is traveling on a narrow road such as a residential area and sets a travelable area having a width of 4 m or less.
- the road surface inclination circuit 17 approximates the inclination of the road surface (Froad) in the vehicle width direction of the travelable area (G 1 ) with a curve based on the height of the road surface. For example, a quadratic curve is used as the approximate curve. In addition, a linear curve, a spline curve, or a combination thereof can also be used. Specifically, as shown in FIG. 5, the road surface inclination circuit 17 selects a plurality of measurement points (I m1 , I m2 , I m3 ) on the step determination position (Pa) in the travelable region (G 1 ). To do.
- the slope of the road surface (Froad) in the vehicle width direction of the roadway (Rr) is approximated by a curve (Croad).
- the number and position of measurement points can be arbitrarily selected.
- the travelable region (G 1 ) is too narrow compared to the actual roadway (Rr) region, the end of the travelable region (G 1 ) to the end (road shoulder) of the roadway (Rr). The distance gets longer. For this reason, the approximated curve (Croad) is separated from the actual inclination of the roadway (Rr), and it becomes difficult to detect the step (LD) with high accuracy. Therefore, the travelable area (G 1 ) is gradually expanded in the vehicle width direction, and the curve approximation is repeated to bring the curve (Croad) closer to the slope of the actual roadway (Rr), and the curve (Croad) and the measurement point Can be detected with high accuracy.
- the degree of coincidence between the curve (Croad) and the measurement point is, for example, a deviation or a standard deviation between the curve (Croad) and the measurement point. The smaller the deviation or standard deviation, the higher the degree of match.
- the road surface inclination circuit 17 is a road surface in the vehicle width direction (G 2 , G 3 , G 4 ,...) Obtained by expanding the travelable region (G 1 ) at the level difference determination position (Pa) ( Froad) is approximated by a curve.
- the level difference detection circuit 18 calculates the first matching degree of the heights of the measurement points (I m1 , I m2 , I m3 ) set in the travelable area (G 1 ) with respect to the curve (Croad).
- the second degree of coincidence of the height data of the measurement points (I m1 , I m2 ,..., I m6 , I m7 ) set to is calculated.
- the level difference detection circuit 18 detects a level difference on the road surface based on the first matching level and the second matching level.
- the travelable area circuit 16 sets an extended area (G 2 ) that extends the travelable area (G 1 ) by a predetermined range (Kc) in the vehicle width direction.
- the road surface inclination circuit 17 approximates the inclination of the road surface (Froad) with a curve in the vehicle width direction of the expansion region (G 2 ).
- the level difference detection circuit 18 calculates the degree of coincidence of the height data of the measurement points (I m1 , I m2 , I m3 , I m4 ) with respect to the curve.
- travelable area circuit 16 sets an extension area (G 2) a predetermined range (Kc) extended area widened in the vehicle width direction by (G 3).
- the road surface inclination circuit 17 approximates the inclination of the road surface (Froad) with a curve in the vehicle width direction of the extension region (G 3 ), and the step detection circuit 18 measures the measurement points (I m1 , I m2 , I m3 , I for the curve). The degree of coincidence of the height data of m4 , Im5 ) is calculated. The level difference detection device 1 repeats this series of processing cycles. Then, the step detection circuit 18 determines that there is a step at a location where the approximate error (an example of the degree of match) between the curve and the measurement point changes greatly.
- the extended region (G 4 ) includes a step (LD).
- the degree of coincidence between the approximate curve in the extended region (G 4 ) and the height data of the measurement points (I m1 to I m7 ) changes greatly compared to the previous processing cycle. This is because the measurement points (I m6 , I m7 ) set on the step (LD) are included in the predetermined range (Kc) that is the difference between the expansion region (G 3 ) and the expansion region (G 4 ). .
- the level difference detection circuit 18 detects the level difference based on the degree of coincidence repeatedly calculated by this series of processing cycles.
- the extended area for approximating the road surface curve is gradually expanded in the vehicle width direction by a predetermined range (Kc), and the extended area (G 1 to G 3 ) until the degree of coincidence decreases can be regarded as a roadway. . Therefore, the position of the step can be obtained with high accuracy. Since the curve approximation can be performed widely using the roadway, the step difference can be accurately determined.
- step S01 a stereo image is acquired using a stereo camera which is an example of the distance measuring sensor 12.
- step S03 the arithmetic circuit 14 performs lens distortion correction processing for correcting lens distortion on the stereo image, and performs parallelization correction processing for correcting the vertical position between the stereo images.
- step S05 the arithmetic circuit 14 performs a stereo matching process for estimating the correspondence of each pixel between stereo images. Thereby, the distance and direction to the object around the vehicle can be detected. Furthermore, the arithmetic circuit 14 can acquire the three-dimensional information of the road surface around the vehicle on the coordinates of the distance measurement data by performing the coordinate conversion process.
- the step determining position circuit 15 is arranged in a vehicle width direction on the road surface around the vehicle on the coordinates of the distance measurement data detected by the distance measurement sensor 12, as shown in FIGS.
- the step determination position (Pa) is set.
- the step determination position circuit 15 uses the three-dimensional information of the road surface acquired by the arithmetic circuit 14 to provide height data (distance measurement data) of the road surface measurement point at the step determination position (Pa). Is calculated.
- the travelable area circuit 16 sets an area obtained by adding a predetermined travel margin area to the width of the vehicle (Vc) as the vehicle (Vc) at the step determination position (Pa). Is estimated as a travelable region (G 1 ).
- the road surface inclination circuit 17 sets a plurality of measurement points (I m1 , I m2 , I m3 ) on the step determination position (Pa) in the travelable area (G 1 ).
- the slope in the height direction of the road surface (Froad) is approximated by a curve (Croad).
- the road surface inclination circuit 17, extended region extends the travelable area (G 1) at the step determining a position (Pa) (G 2, G 3, G 4, ⁇ ) road surface in the vehicle width direction (Froad) Is approximated by a curve.
- the level difference detection circuit 18 determines the first matching degree of the height data of the measurement points (I m1 , I m2 , I m3 ) set in the travelable region (G 1 ) with respect to the curve (Croad). calculate.
- the second degree of coincidence of the height data of the measurement points (I m1 , I m2 ,..., I m6 , I m7 ) set to is calculated.
- the level difference detection circuit 18 detects a level difference on the road surface based on the first matching level and the second matching level.
- step S101 the travelable area circuit 16 estimates a travelable area (G 1 ) that is an initial value of a range approximating a road surface (Froad) curve in the vehicle width direction. Then, the slope of the road surface in the vehicle width direction is approximated by a curve (Croad) in the travelable region (G 1 ) (S103), and the height of the measurement points (I m1 , I m2 , I m3 ) with respect to the curve (Croad) The degree of match is calculated (S105). The level difference detection circuit 18 determines whether or not the degree of coincidence is a predetermined reference value or less.
- step S109 the travelable area (G 1) set the extension area widened by a predetermined range (Kc) in the vehicle width direction (G 2). Note that, for example, 10 cm can be set as the predetermined reference value in S107.
- the level difference detection device 1 compares the upper limit value set for the size of the extension area with the extension area (G 2 ). If the extended region (G 2 ) exceeds the upper limit value (YES in S111), the detection of the step forming the road boundary is abandoned and the process is terminated. If the expansion area (G 2 ) does not exceed the upper limit value (NO in S111), the process returns to step S103, and S103 to S111 are repeatedly performed until the degree of coincidence falls below a predetermined reference value.
- the step detection circuit 18 determines that there is a step in the predetermined range (Kc) added in the last step S109 (S113). ).
- the step is not detected, and when the amount of change in the height of the road surface is estimated to be large, the step is erroneously detected. Therefore, the slope of the road surface in the vehicle width direction is approximated by a curve, and a step on the road surface is detected from the displacement of the road surface height with respect to this curve.
- the degree of coincidence in the extended area where the runnable area is extended in the vehicle width direction at the step judgment position greatly changes with the degree of coincidence between the curve and the measurement point in the runnable area, it increases due to expansion. It can be determined that there is a sudden change in the road surface height, that is, a level difference at the spot. Even if the roadway is inclined, it is possible to accurately detect a step on the road surface.
- the travelable area circuit 16 estimates the travelable area (G 1 ) wider as the speed of the vehicle (Vc) is higher. If the vehicle speed is high, it is estimated that the vehicle is traveling on a wide road such as an expressway or a main road. On the other hand, when the vehicle speed is low, it is estimated that the vehicle is traveling on a narrow road such as a residential area. Therefore, since the region where the vehicle can travel without including the step (LD) can be set appropriately, the detection accuracy of the step (LD) is improved.
- the level difference detecting device 1 expands the travelable area (G 1 ) by a predetermined range (Kc) in the vehicle width direction and expands the predetermined range (Kc) by a predetermined range (Kc) (G 2 , G 3 , G 4 ,).
- a road surface (Froad) slope is approximated by a curve, and a cycle for calculating the degree of coincidence of the height of measurement points (I m1 , I m2 ,...) With respect to the curve is repeatedly performed.
- the level difference detection circuit 18 detects a level difference based on the repeatedly calculated degree of match.
- the stereo camera as the distance measuring sensor 12 is installed not on the front part of the vehicle (Vc) but on the side part of the vehicle (Vc), and the imaging range of the stereo camera is set on the vehicle (Vc) side. It doesn't matter as you go. Also in this case, the traveling direction of the vehicle (Vc) is the first predetermined direction (Da).
- step difference determination position circuit 15 can set the linear level
- a step determination position (Pa) extending from the distance measuring sensor 12 by a predetermined distance in the first predetermined direction (Da) and extending in a direction orthogonal to the first predetermined direction (Da) can be set. .
- LRF laser range finder
- the LRF irradiates a laser toward an object around the vehicle, and observes the laser reflected and returned to the object.
- the LRF measures the azimuth in which the object is located based on the laser irradiation direction, and measures the distance to the object based on the time from the laser irradiation to the observation of the reflected laser.
- LRF is also called a laser scanner.
- the LRF irradiation range can be arbitrarily set.
- FIG. 9 shows an example of 360-degree LRF in which the LRF (12) is set at the center of the roof of the vehicle (Vc) and the entire surroundings are irradiated.
- the step determination position circuit 15 sets a step determination position (Pa) that is separated from the distance measuring sensor 12 by a predetermined distance in the first predetermined direction (Da) and extends in a direction orthogonal to the first predetermined direction (Da). Accordingly, the traveling direction of the vehicle (Vc) is set to the first predetermined direction (Da) in the same manner as the example of FIG. 2 in which the distance measuring sensor 12 is installed at the front portion of the vehicle (Vc).
- the extending step determination position (Pa) is set in front of the vehicle (Vc).
- the step difference detection method when a laser range finder (LRF) is used as the distance measuring sensor 12 is different in that the following steps S21 and S23 are performed instead of S01 to S05 in FIG. Other steps S07 to S13 are the same as those in FIG.
- LRF laser range finder
- step S21 the LRF (12) measures the distance to the object together with the direction of the object irradiated with the laser.
- the measurement data is transmitted to the arithmetic circuit 14.
- step S23 the arithmetic circuit 14 converts the measurement data in the polar coordinate system with the LRF (12) as the origin into an orthogonal coordinate system with the vehicle (Vc) as the origin. Thereby, the arithmetic circuit 14 can acquire the three-dimensional information of the road surface around the vehicle. Thereafter, the process proceeds to step S07.
- the travelable area circuit 16 determines the size of the travelable area (G 1 ) based on the speed of the vehicle (Vc).
- the width of the travelable region (G 1 ) can be determined using not only the speed of the vehicle (Vc) but also other criteria.
- the travelable area circuit 16 may determine the width of the travelable area (G 1 ) based on information (roadway information) regarding the roadway (Rr) on which the vehicle (Vc) is traveling.
- the travelable area can be set within a range 1 m narrower than the width information included in the road information so as not to include the actual road boundary.
- the microcomputer 13 further includes a roadway information acquisition circuit 19.
- the roadway information acquisition circuit 19 acquires information on the roadway (Rr) on which the vehicle (Vc) is traveling from the GPS 20 and the map database 21.
- the travelable area circuit 16 can set the distance in the vehicle width direction of the travelable area (G 1 ) based on the information on the roadway (Rr). By using information on the roadway (Rr), it is possible to appropriately set an area in which the vehicle (Vc) can travel without including the step (LD), so that the detection accuracy of the step (LD) is improved.
- Information regarding the roadway (Rr) acquired by the roadway information acquisition circuit 19 can be used not only as a reference for the width of the travelable region (G 1 ) but also as a reference for setting other parameters.
- the level difference detection device 2 sets an upper limit value used in S111 of FIG. 7 based on information on the roadway (Rr). For example, in the case of LRF, since the irradiation range is about 10 m, the upper limit value can be set to 10 m. If the extension area exceeds the upper limit, the step detection process can be terminated. Therefore, the step detection process can be completed in a certain time even on a road without a clear step.
- Step detection device DESCRIPTION OF SYMBOLS 1, 2 Step detection device 11 Distance measuring part 12 Distance sensor 13 Microcomputer 14 Calculation circuit 15 Step judging position circuit 16 Travelable area circuit 17 Road surface inclination circuit 18 Step detection circuit 19 Roadway information acquisition circuit 20 GPS (global positioning system) 21 Map database G 1 Travelable area G 2 to G 4 Extended area I m1 to I m7 Measurement points Croad Curve Froad Road surface Kc Predetermined range Pa Step judgment position LD Step
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Multimedia (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Signal Processing (AREA)
- Traffic Control Systems (AREA)
- Length Measuring Devices With Unspecified Measuring Means (AREA)
- Image Analysis (AREA)
Abstract
Description
次に、図面を参照して、実施形態を詳細に説明する。
測距センサ12としてのステレオカメラを、車両(Vc)の前部ではなく、図8に示すように、車両(Vc)の側部に設置し、ステレオカメラの撮像範囲を車両(Vc)の側方としても構わない。この場合も、車両(Vc)の進行方向を第1の所定方向(Da)とする。これにより、段差判定位置回路15は、車両の周囲(側方)の路面に、段差(LD)と交わる線状の段差判定位置(Pa)を設定することができる。具体的には、測距センサ12から第1の所定方向(Da)に所定距離だけ離れ、第1の所定方向(Da)に直交する方向に延びる段差判定位置(Pa)を設定することができる。
測距センサ12の他の例として、レーザレンジファインダ(LRF)がある。LRFは、車両周囲の物体に向けてレーザーを照射し、物体に反射して戻ってきたレーザーを観測する。そして、LRFは、レーザーの照射方向に基づいて物体が位置する方位を計測すると共に、レーザーの照射から反射レーザーの観測までの時間に基づいて物体までの距離を計測する。LRFは、レーザースキャナとも呼ばれる。LRFの照射範囲は任意に設定可能である。図9は、車両(Vc)のルーフ中央部にLRF(12)を設定し、周囲全体を照射範囲とする360度LRFの例を示す。段差判定位置回路15は、測距センサ12から第1の所定方向(Da)に所定距離だけ離れ、第1の所定方向(Da)に直交する方向に延びる段差判定位置(Pa)を設定する。よって、車両(Vc)の前部に測距センサ12を設置した図2の例と同様にして、車両(Vc)の進行方向を第1の所定方向(Da)とするので、車幅方向に延びる段差判定位置(Pa)が、車両(Vc)の前方に設定される。
実施形態で、走行可能領域回路16は、車両(Vc)の速度に基づいて、走行可能領域(G1)の広さを定めていた。車両(Vc)の速度に限らず、その他の基準を用いても、走行可能領域(G1)の広さを定めることはできる。例えば、走行可能領域回路16は、車両(Vc)が走行している車道(Rr)に関する情報(車道情報)に基づいて、走行可能領域(G1)の広さを定めてもよい。例えば、実際の道路境界を含まないように、車道情報に含まれる幅情報よりも1m狭い範囲で走行可能領域を設定することができる。
車道情報取得回路19が取得する車道(Rr)に関する情報は、走行可能領域(G1)の広さの基準としてのみならず、その他のパラメータを設定するための基準として用いることができる。例えば、段差検出装置2は、車道(Rr)に関する情報に基づいて、図7のS111において用いる上限値を設定する。例えば、LRFの場合、照射範囲は10m程度であるので、上限値を10mと設定することができる。拡張領域の広さが上限値を超える場合には段差検出処理を終了することができる。よって、明確な段差のない道路においても一定時間で段差検出処理を終了することが可能となる。
11 測距部
12 測距センサ
13 マイクロコンピュータ
14 演算回路
15 段差判定位置回路
16 走行可能領域回路
17 路面傾斜回路
18 段差検出回路
19 車道情報取得回路
20 GPS(グローバルポジショニングシステム)
21 地図データベース
G1 走行可能領域
G2~G4 拡張領域
Im1~Im7 測定点
Croad 曲線
Froad 路面
Kc 所定範囲
Pa 段差判定位置
LD 段差
Claims (6)
- 車両の周囲における路面の高さを検出する測距部と、
前記車両の進行方向の走行可能領域を推定する走行可能領域回路と、
前記測距部により検出された前記路面の高さに基づいて、前記走行可能領域の車幅方向における前記路面の傾斜を曲線で近似する路面傾斜回路と、
前記曲線に対する前記走行可能領域に設定された測定点の高さの合致度と、前記走行可能領域を車幅方向に拡張した拡張領域における前記路面の傾斜を近似した曲線に対する前記拡張領域に設定された測定点の高さの合致度とに基づいて、前記路面上の段差を検出する段差検出回路と、
を備えたことを特徴とする段差検出装置。 - 前記走行可能領域回路は、前記車両の速度が速いほど、前記走行可能領域を車幅方向に広く推定することを特徴とする請求項1に記載の段差検出装置。
- 前記車両が走行している車道に関する情報を取得する車道情報取得回路を更に備え、
前記走行可能領域回路は、前記車道に関する情報に基づいて、前記走行可能領域の車幅方向の距離を設定する
ことを特徴とする請求項1に記載の段差検出装置。 - 前記段差検出装置は、前記走行可能領域を車幅方向に所定範囲だけ広げ、前記所定範囲だけ広げた拡張領域において前記路面の傾斜を曲線で近似し、前記曲線に対する前記測定点の高さの合致度を算出するサイクルを繰り返し実施し、
前記段差検出回路は、繰り返し算出される前記合致度に基づいて、前記段差を検出する
ことを特徴とする請求項1~3の何れか一項に記載の段差検出装置。 - 前記車両が走行している車道に関する情報を取得する車道情報取得回路を更に備え、
前記段差検出装置は、前記車道に関する情報に基づいて、前記拡張領域の広さに上限値を設定する
ことを特徴とする請求項4に記載の段差検出装置。 - 車両の周囲における路面の高さを検出し、
前記車両の進行方向の走行可能領域を推定し、
前記路面の高さに基づいて、前記走行可能領域の車幅方向における前記路面の傾斜を曲線で近似し、
前記曲線に対する前記走行可能領域に設定された測定点の高さの合致度と、前記走行可能領域を車幅方向に拡張した拡張領域における前記路面の傾斜を近似した曲線に対する前記拡張領域に設定された測定点の高さの合致度とに基づいて、前記路面上の段差を検出する
ことを特徴とする段差検出方法。
Priority Applications (10)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US15/749,867 US10410070B2 (en) | 2015-08-04 | 2015-08-04 | Step detection device and step detection method |
KR1020187003012A KR102079882B1 (ko) | 2015-08-04 | 2015-08-04 | 단차 검출 장치 및 단차 검출 방법 |
RU2018107670A RU2686279C1 (ru) | 2015-08-04 | 2015-08-04 | Устройство обнаружения уступов и способ обнаружения уступов |
CA2994652A CA2994652C (en) | 2015-08-04 | 2015-08-04 | Step detection device and step detection method |
JP2017532300A JP6451858B2 (ja) | 2015-08-04 | 2015-08-04 | 段差検出装置及び段差検出方法 |
MX2018001343A MX370052B (es) | 2015-08-04 | 2015-08-04 | Dispositivo de deteccion de escalon y metodo de deteccion de escalon. |
EP15900393.8A EP3333829B1 (en) | 2015-08-04 | 2015-08-04 | Step detection device and step detection method |
CN201580082100.6A CN107851390B (zh) | 2015-08-04 | 2015-08-04 | 台阶检测装置及台阶检测方法 |
KR1020197024474A KR20190101491A (ko) | 2015-08-04 | 2015-08-04 | 단차 검출 장치 및 단차 검출 방법 |
PCT/JP2015/072085 WO2017022080A1 (ja) | 2015-08-04 | 2015-08-04 | 段差検出装置及び段差検出方法 |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
PCT/JP2015/072085 WO2017022080A1 (ja) | 2015-08-04 | 2015-08-04 | 段差検出装置及び段差検出方法 |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2017022080A1 true WO2017022080A1 (ja) | 2017-02-09 |
Family
ID=57942691
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/JP2015/072085 WO2017022080A1 (ja) | 2015-08-04 | 2015-08-04 | 段差検出装置及び段差検出方法 |
Country Status (9)
Country | Link |
---|---|
US (1) | US10410070B2 (ja) |
EP (1) | EP3333829B1 (ja) |
JP (1) | JP6451858B2 (ja) |
KR (2) | KR20190101491A (ja) |
CN (1) | CN107851390B (ja) |
CA (1) | CA2994652C (ja) |
MX (1) | MX370052B (ja) |
RU (1) | RU2686279C1 (ja) |
WO (1) | WO2017022080A1 (ja) |
Cited By (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JPWO2017022079A1 (ja) * | 2015-08-04 | 2018-06-21 | 日産自動車株式会社 | 段差検出装置及び段差検出方法 |
WO2019031137A1 (ja) * | 2017-08-07 | 2019-02-14 | 日立オートモティブシステムズ株式会社 | 路側物検出装置、路側物検出方法及び路側物検出システム |
JP2020134367A (ja) * | 2019-02-21 | 2020-08-31 | 日立オートモティブシステムズ株式会社 | 走行路認識装置 |
Families Citing this family (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
KR20200090527A (ko) * | 2019-01-21 | 2020-07-29 | 현대자동차주식회사 | 차선 인식 장치 및 그 방법 |
KR102176376B1 (ko) * | 2019-06-24 | 2020-11-10 | 현대오트론 주식회사 | 물체 거리 측정 장치 및 방법 |
Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2004098717A (ja) * | 2002-09-04 | 2004-04-02 | Fuji Heavy Ind Ltd | 車外監視装置、及び、この車外監視装置を備えた走行制御装置 |
JP2009053818A (ja) * | 2007-08-24 | 2009-03-12 | Toshiba Corp | 画像処理装置及びその方法 |
JP2013205130A (ja) * | 2012-03-28 | 2013-10-07 | Zenrin Co Ltd | 路面形状データの生成方法、路面形状データ生成装置およびコンピュータプログラム |
Family Cites Families (15)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP4523095B2 (ja) * | 1999-10-21 | 2010-08-11 | 富士通テン株式会社 | 情報処理装置、情報統合装置および情報処理方法 |
JP3733875B2 (ja) * | 2000-09-29 | 2006-01-11 | 日産自動車株式会社 | 道路白線認識装置 |
JP3956926B2 (ja) * | 2003-09-24 | 2007-08-08 | アイシン精機株式会社 | 路面走行レーン検出装置 |
JP4609695B2 (ja) * | 2003-10-21 | 2011-01-12 | 日本精機株式会社 | 車両用表示装置 |
RU53766U1 (ru) * | 2005-07-11 | 2006-05-27 | Государственное образовательное учреждение высшего профессионального образования "Хабаровский государственный технический университет" | Прибор для измерения шероховатости дорожного покрытия |
JP5089545B2 (ja) * | 2008-09-17 | 2012-12-05 | 日立オートモティブシステムズ株式会社 | 道路境界検出判断装置 |
JP5094658B2 (ja) * | 2008-09-19 | 2012-12-12 | 日立オートモティブシステムズ株式会社 | 走行環境認識装置 |
CN101644023B (zh) * | 2009-08-21 | 2011-01-26 | 赵怀志 | 一种路面平整度检测方法 |
JP5829980B2 (ja) * | 2012-06-19 | 2015-12-09 | トヨタ自動車株式会社 | 路側物検出装置 |
CN103760569B (zh) * | 2013-12-31 | 2016-03-30 | 西安交通大学 | 一种基于激光雷达的可行驶区域检测方法 |
JP5989701B2 (ja) * | 2014-03-24 | 2016-09-07 | トヨタ自動車株式会社 | 境界検出装置および境界検出方法 |
JP2016001464A (ja) * | 2014-05-19 | 2016-01-07 | 株式会社リコー | 処理装置、処理システム、処理プログラム、及び、処理方法 |
JP6434743B2 (ja) | 2014-08-19 | 2018-12-05 | 日野自動車株式会社 | 走路端部推定装置 |
CN104636763B (zh) * | 2014-12-01 | 2018-10-09 | 北京工业大学 | 一种基于无人驾驶车的道路与障碍物检测方法 |
JP6233345B2 (ja) * | 2015-04-17 | 2017-11-22 | トヨタ自動車株式会社 | 路面勾配検出装置 |
-
2015
- 2015-08-04 WO PCT/JP2015/072085 patent/WO2017022080A1/ja active Application Filing
- 2015-08-04 KR KR1020197024474A patent/KR20190101491A/ko active Application Filing
- 2015-08-04 RU RU2018107670A patent/RU2686279C1/ru active
- 2015-08-04 JP JP2017532300A patent/JP6451858B2/ja active Active
- 2015-08-04 MX MX2018001343A patent/MX370052B/es active IP Right Grant
- 2015-08-04 KR KR1020187003012A patent/KR102079882B1/ko active IP Right Grant
- 2015-08-04 EP EP15900393.8A patent/EP3333829B1/en active Active
- 2015-08-04 CA CA2994652A patent/CA2994652C/en active Active
- 2015-08-04 CN CN201580082100.6A patent/CN107851390B/zh active Active
- 2015-08-04 US US15/749,867 patent/US10410070B2/en active Active
Patent Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2004098717A (ja) * | 2002-09-04 | 2004-04-02 | Fuji Heavy Ind Ltd | 車外監視装置、及び、この車外監視装置を備えた走行制御装置 |
JP2009053818A (ja) * | 2007-08-24 | 2009-03-12 | Toshiba Corp | 画像処理装置及びその方法 |
JP2013205130A (ja) * | 2012-03-28 | 2013-10-07 | Zenrin Co Ltd | 路面形状データの生成方法、路面形状データ生成装置およびコンピュータプログラム |
Non-Patent Citations (1)
Title |
---|
See also references of EP3333829A4 * |
Cited By (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JPWO2017022079A1 (ja) * | 2015-08-04 | 2018-06-21 | 日産自動車株式会社 | 段差検出装置及び段差検出方法 |
WO2019031137A1 (ja) * | 2017-08-07 | 2019-02-14 | 日立オートモティブシステムズ株式会社 | 路側物検出装置、路側物検出方法及び路側物検出システム |
JPWO2019031137A1 (ja) * | 2017-08-07 | 2020-02-27 | 日立オートモティブシステムズ株式会社 | 路側物検出装置、路側物検出方法及び路側物検出システム |
US11151395B2 (en) | 2017-08-07 | 2021-10-19 | Hitachi Astemo, Ltd. | Roadside object detection device, roadside object detection method, and roadside object detection system |
JP2020134367A (ja) * | 2019-02-21 | 2020-08-31 | 日立オートモティブシステムズ株式会社 | 走行路認識装置 |
JP7257814B2 (ja) | 2019-02-21 | 2023-04-14 | 日立Astemo株式会社 | 走行路認識装置 |
Also Published As
Publication number | Publication date |
---|---|
EP3333829B1 (en) | 2019-10-09 |
CA2994652C (en) | 2019-09-10 |
RU2686279C1 (ru) | 2019-04-24 |
EP3333829A4 (en) | 2018-08-08 |
US20180225525A1 (en) | 2018-08-09 |
MX370052B (es) | 2019-11-29 |
KR20180022954A (ko) | 2018-03-06 |
EP3333829A1 (en) | 2018-06-13 |
CN107851390A (zh) | 2018-03-27 |
CN107851390B (zh) | 2020-06-30 |
KR102079882B1 (ko) | 2020-02-20 |
CA2994652A1 (en) | 2017-02-09 |
MX2018001343A (es) | 2018-06-15 |
JP6451858B2 (ja) | 2019-01-16 |
US10410070B2 (en) | 2019-09-10 |
KR20190101491A (ko) | 2019-08-30 |
JPWO2017022080A1 (ja) | 2018-05-24 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
JP6753527B2 (ja) | 地図データ補正方法および装置 | |
JP6451858B2 (ja) | 段差検出装置及び段差検出方法 | |
US10482615B2 (en) | Image processing device and image processing method | |
JP6593088B2 (ja) | 車両位置推定装置及びプログラム | |
JP6552448B2 (ja) | 車両位置検出装置、車両位置検出方法及び車両位置検出用コンピュータプログラム | |
JP6561670B2 (ja) | 段差検出装置及び段差検出方法 | |
KR102464581B1 (ko) | 스테레오 점군 처리장치 및 그 방법 | |
WO2018150921A1 (ja) | 車載環境認識装置 | |
JP6477340B2 (ja) | 道路境界検出装置、自己位置推定装置及び道路境界検出方法 | |
JP6493533B2 (ja) | 段差検出装置及び段差検出方法 | |
JP6485280B2 (ja) | 段差検出装置及び段差検出方法 | |
JP2018036225A (ja) | 状態推定装置 | |
JP6370234B2 (ja) | 地図データ生成装置、地図データ生成方法及び地図データ生成用コンピュータプログラムならびに車両位置検出装置 | |
JP6451544B2 (ja) | 道路境界検出装置、自己位置推定装置及び道路境界検出方法 | |
JP2016004019A (ja) | 物体検出装置及び物体検出方法 |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 15900393 Country of ref document: EP Kind code of ref document: A1 |
|
ENP | Entry into the national phase |
Ref document number: 2017532300 Country of ref document: JP Kind code of ref document: A |
|
ENP | Entry into the national phase |
Ref document number: 20187003012 Country of ref document: KR Kind code of ref document: A |
|
WWE | Wipo information: entry into national phase |
Ref document number: MX/A/2018/001343 Country of ref document: MX |
|
ENP | Entry into the national phase |
Ref document number: 2994652 Country of ref document: CA |
|
WWE | Wipo information: entry into national phase |
Ref document number: 15749867 Country of ref document: US |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
WWE | Wipo information: entry into national phase |
Ref document number: 2018107670 Country of ref document: RU |
|
REG | Reference to national code |
Ref country code: BR Ref legal event code: B01A Ref document number: 112018002338 Country of ref document: BR |
|
ENP | Entry into the national phase |
Ref document number: 112018002338 Country of ref document: BR Kind code of ref document: A2 Effective date: 20180202 |