CN112489131A - Method, device, medium and robot for constructing cost map based on road surface detection - Google Patents
Method, device, medium and robot for constructing cost map based on road surface detection Download PDFInfo
- Publication number
- CN112489131A CN112489131A CN202011570427.7A CN202011570427A CN112489131A CN 112489131 A CN112489131 A CN 112489131A CN 202011570427 A CN202011570427 A CN 202011570427A CN 112489131 A CN112489131 A CN 112489131A
- Authority
- CN
- China
- Prior art keywords
- road
- preset
- road surface
- point
- confidence
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 58
- 238000001514 detection method Methods 0.000 title claims abstract description 43
- 239000011159 matrix material Substances 0.000 claims abstract description 25
- 230000011218 segmentation Effects 0.000 claims abstract description 24
- 238000004364 calculation method Methods 0.000 claims abstract description 15
- 230000004888 barrier function Effects 0.000 claims abstract description 11
- 230000000007 visual effect Effects 0.000 claims abstract description 10
- 238000004422 calculation algorithm Methods 0.000 claims description 25
- 238000001914 filtration Methods 0.000 claims description 16
- 238000009499 grossing Methods 0.000 claims description 12
- 238000004590 computer program Methods 0.000 claims description 9
- 238000010276 construction Methods 0.000 claims description 9
- 238000011426 transformation method Methods 0.000 claims description 6
- 230000009466 transformation Effects 0.000 claims description 4
- 230000002146 bilateral effect Effects 0.000 claims description 3
- 239000000126 substance Substances 0.000 claims description 3
- 230000008569 process Effects 0.000 abstract description 6
- 230000000694 effects Effects 0.000 abstract description 5
- 239000011521 glass Substances 0.000 abstract description 5
- 230000006870 function Effects 0.000 description 16
- 230000015654 memory Effects 0.000 description 15
- 238000010586 diagram Methods 0.000 description 4
- 230000007613 environmental effect Effects 0.000 description 3
- 238000004891 communication Methods 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 230000018109 developmental process Effects 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000013507 mapping Methods 0.000 description 2
- 238000005259 measurement Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 238000012545 processing Methods 0.000 description 2
- 230000003068 static effect Effects 0.000 description 2
- 230000009471 action Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004140 cleaning Methods 0.000 description 1
- 238000011217 control strategy Methods 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 230000004927 fusion Effects 0.000 description 1
- 230000004807 localization Effects 0.000 description 1
- 238000010295 mobile communication Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
- G06T7/77—Determining position or orientation of objects or cameras using statistical methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
- G06T17/05—Geographic models
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/70—Denoising; Smoothing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20024—Filtering details
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Software Systems (AREA)
- Geometry (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Life Sciences & Earth Sciences (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Evolutionary Biology (AREA)
- Probability & Statistics with Applications (AREA)
- Remote Sensing (AREA)
- Computer Graphics (AREA)
- Image Analysis (AREA)
- Image Processing (AREA)
Abstract
The invention discloses a method, a device, a medium and a robot for constructing a cost map based on road surface detection, wherein the method comprises the following steps: collecting a front visual field image, calculating the confidence coefficient of each pixel point in the front visual field image, which is a road surface, based on a preset semantic segmentation network, and generating a confidence coefficient matrix; acquiring the spatial position coordinates of at least one preset road point on a spatial road surface, and calculating the road surface confidence of each preset road point through a confidence matrix; and constructing a cost function corresponding to the barrier map layer according to the road confidence corresponding to all the preset road points so as to generate a local cost map. When the robot moves in an indoor flat layer, the road segmentation of images is realized by means of the robot camera, the confidence coefficient of the road is projected to the cost function of the obstacle map layer constructed under the space coordinate system, the calculation process is simple and quick, and the method has a very good application effect in scenes such as strong light reflection like glass mirrors, road collapse, small stones and the like.
Description
Technical Field
The invention relates to the field of robots, in particular to a method, a device, a medium and a robot for constructing a cost map based on road surface detection.
Background
With the rapid development of the robot industry, various service robots emerge endlessly, and the robots are widely applied in life and work. When the robot moves indoors, the robot senses the surrounding environment by relying on sensors of the robot, wherein the sensors comprise a laser radar, ultrasonic waves, a camera and the like. The single-line laser radar is a common economic and reliable solution, and local path planning is realized by detecting the obstacle distance between the robot and the obstacle. However, in the scenes of strong light reflection, road surface collapse and small stones such as glass mirrors, the distance data obtained by the laser radar is easy to lose effectiveness, so that the local path planning result is unreliable. In the prior art, path planning can be performed by adopting a multi-data fusion mode of various sensor data, such as laser radar data, inertial measurement unit data, gyroscope data and the like, but the requirement on communication resources is high, and the calculation process is complex.
Disclosure of Invention
The invention provides a method, a device, a medium and a robot for constructing a cost map based on road surface detection, which solve the technical problems of high requirement on communication resources and complex calculation process in the prior art.
The technical scheme for solving the technical problems is as follows: a method for constructing a cost map based on road surface detection comprises the following steps:
step 1, collecting a front visual field image, calculating the confidence coefficient of each pixel point in the front visual field image, which is a road surface, based on a preset semantic segmentation network, and generating a confidence coefficient matrix;
step 2, acquiring the spatial position coordinates of at least one preset road point on the spatial road surface corresponding to the front visual field image, and calculating the initial road surface confidence coefficient of each preset road point through the confidence coefficient matrix;
and 3, constructing a cost function corresponding to the barrier map layer according to the initial road confidence corresponding to all the preset road points so as to generate a local cost map.
In a preferred embodiment, the calculating the initial road surface confidence of each preset road surface point through the confidence matrix specifically includes the following steps:
s201, acquiring the spatial position coordinates of at least one preset road point on a spatial road corresponding to the front view image;
s202, calculating a corresponding target pixel point of each preset road point in the front view image by adopting a preset coordinate transformation method;
s203, generating four adjacent pixel points corresponding to each target pixel point in the front view image by adopting a bilinear interpolation method;
s204, inquiring the confidence matrix, and acquiring a first adjacent point confidence coefficient S1, a second adjacent point confidence coefficient S2, a third adjacent point confidence coefficient S3 and a fourth adjacent point confidence coefficient S4 which correspond to the four adjacent pixel points respectively;
s205, calculating the initial road confidence of each preset road point by adopting a first preset formula, wherein the first preset formula is as follows:
score＝(1-dυ)*f1+dυ*f2，
f1＝(1-du)*s1+du*s2，
f2＝(1-du)*s3+du*s4，
the score is an initial road confidence of a preset road point, u and v are coordinates of corresponding target pixel points in the front view image, du is equal to u- [ u ], dv is equal to v- [ v ], and a symbol [ ] represents rounding-down.
In a preferred embodiment, the constructing a cost function of a corresponding obstacle map layer by using the initial road surface confidence degrees of all preset road surface points specifically includes the following steps:
filtering the initial pavement confidence coefficient of each preset pavement point by adopting a preset smooth filtering algorithm to generate an optimized pavement confidence coefficient;
constructing a cost function corresponding to the barrier map layer by adopting the optimized road confidence;
the preset smoothing filter algorithm comprises any one of a mean smoothing filter algorithm, a field smoothing filter algorithm, a bilateral filter algorithm and a median filter algorithm.
In a preferred embodiment, a mean smoothing filter algorithm is used to filter the initial road confidence of each preset road point, and the adopted filter formula is as follows:
wherein the content of the first and second substances,
i. j is the coordinate of the preset road point.
In a preferred embodiment, the cost function for constructing the corresponding obstacle map layer by using the optimized road confidence corresponding to all the preset road points is as follows:
costob[i，j]＝-log(score[i，j])。
a second aspect of the embodiments of the present invention provides an apparatus for constructing a cost map based on road surface detection, including a segmentation module, a calculation module, and a construction module,
the segmentation module is used for acquiring a front view image, calculating the confidence coefficient that each pixel point in the front view image is a road surface respectively based on a preset semantic segmentation network, and generating a confidence coefficient matrix;
the calculation module is used for acquiring the spatial position coordinates of at least one preset road point on the spatial road surface corresponding to the front view image and calculating the initial road surface confidence coefficient of each preset road point according to the confidence coefficient matrix;
the construction module is used for constructing a cost function corresponding to the barrier map layer according to the initial road confidence corresponding to all the preset road points so as to generate a local cost map.
In a preferred embodiment, the calculation module specifically includes:
the acquisition unit is used for acquiring the spatial position coordinates of at least one preset road point on the spatial road corresponding to the front view image;
the coordinate transformation unit is used for calculating a corresponding target pixel point of each preset road point in the front view image by adopting a preset coordinate transformation method;
the interpolation unit is used for generating four adjacent pixel points corresponding to each target pixel point in the front view image by adopting a bilinear interpolation method;
the query unit is used for querying the confidence coefficient matrix and acquiring a first adjacent point confidence coefficient s1, a second adjacent point confidence coefficient s2, a third adjacent point confidence coefficient s3 and a fourth adjacent point confidence coefficient s4 which correspond to the four adjacent pixel points respectively;
a calculating unit, configured to calculate an initial road confidence of each preset road point by using a first preset formula, where the first preset formula is:
score＝(1-dυ)*f1+dυ*f2，
f1＝(1-du)*s1+du*s2，
f2＝(1-du)*s3+du*s4，
the score is an initial road confidence of a preset road point, u and v are coordinates of corresponding target pixel points in the front view image, du is equal to u- [ u ], dv is equal to v- [ v ], and a symbol [ ] represents rounding-down.
In a preferred embodiment, the building block specifically includes:
the filtering unit is used for filtering the initial road surface confidence coefficient of each preset road surface point by adopting a mean smooth filtering algorithm to generate an optimized road surface confidence coefficient;
and the construction unit is used for constructing a cost function corresponding to the barrier map layer by adopting the optimized road confidence corresponding to all the preset road points.
A third aspect of embodiments of the present invention provides a robot, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above method for constructing a cost map based on road surface detection when executing the computer program.
A fourth aspect of embodiments of the present invention provides a computer-readable storage medium storing a computer program which, when executed by a processor, implements the above-described steps of the method of constructing a cost map based on road surface detection.
The invention provides a method, a device, a medium and a robot for constructing a cost map based on road surface detection, when the robot moves in an indoor flat layer, the road surface segmentation of an image is realized by means of a robot camera, the road surface confidence coefficient is projected to a cost function for constructing a barrier map layer under a space coordinate system, the calculation process is simple and quick, and the method has a very good application effect in scenes such as strong light reflection, road surface collapse, small stones and the like of a glass mirror and the like.
In order to make the aforementioned and other objects, features and advantages of the present invention comprehensible, preferred embodiments accompanied with figures are described in detail below.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings needed to be used in the embodiments will be briefly described below, it should be understood that the following drawings only illustrate some embodiments of the present invention and therefore should not be considered as limiting the scope, and for those skilled in the art, other related drawings can be obtained according to the drawings without inventive efforts.
Fig. 1 is a schematic flowchart of a method for constructing a cost map based on road surface detection according to embodiment 1;
fig. 2 is a schematic structural diagram of an apparatus for constructing a cost map based on road surface detection according to embodiment 2;
fig. 3 is a schematic circuit diagram of a controller provided in embodiment 3.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
It should be noted that, if not conflicted, the various features of the embodiments of the invention may be combined with each other within the scope of protection of the invention. Additionally, while functional block divisions are performed in apparatus schematics, with logical sequences shown in flowcharts, in some cases, steps shown or described may be performed in sequences other than block divisions in apparatus or flowcharts. The terms "first", "second", "third", and the like used in the present invention do not limit data and execution order, but distinguish the same items or similar items having substantially the same function and action.
The robot of embodiments of the present invention may be configured in any suitable shape to perform a particular business function operation, for example, the robot of embodiments of the present invention may be a delivery robot, a transfer robot, a care robot, and the like.
The robot generally includes a housing, a sensor unit, a drive wheel assembly, a memory assembly, and a controller. The housing may be substantially circular in shape, and in some embodiments, the housing may be substantially oval, triangular, D-shaped, cylindrical, or otherwise shaped.
The sensor unit is used for collecting some motion parameters of the robot and various data of the environment space. In some embodiments, the sensor unit comprises a lidar mounted above the housing at a mounting height above a top deck height of the housing, the lidar being for detecting an obstacle distance between the robot and an obstacle. In some embodiments, the sensor unit may also include an Inertial Measurement Unit (IMU), a gyroscope, a magnetic field meter, an accelerometer or velocimeter, an optical camera, and so forth.
The driving wheel component is arranged on the shell and drives the robot to move on various spaces, and in some embodiments, the driving wheel component comprises a left driving wheel, a right driving wheel and an omnidirectional wheel, and the left driving wheel and the right driving wheel are respectively arranged on two opposite sides of the shell. The left and right drive wheels are configured to be at least partially extendable and retractable into the bottom of the housing. The omni-directional wheel is arranged at the position, close to the front, of the bottom of the shell and is a movable caster wheel which can rotate 360 degrees horizontally, so that the robot can flexibly steer. The left driving wheel, the right driving wheel and the omnidirectional wheel are arranged to form a triangle, so that the walking stability of the robot is improved. Of course, in some embodiments, the driving wheel component may also adopt other structures, for example, the omni wheel may be omitted, and only the left driving wheel and the right driving wheel may be left to drive the robot to normally walk.
In some embodiments, the robot is further configured with a storage component that is mounted within the receiving slot to accomplish a delivery task or the like.
The controller is respectively and electrically connected with the left driving wheel, the right driving wheel, the omnidirectional wheel and the laser radar. The controller is used as a control core of the robot and is used for controlling the robot to walk, retreat and some business logic processing.
In some embodiments, the controller may be a general purpose processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA), a single chip, ar (aconri scmachine) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination of these components. Also, the controller may be any conventional processor, controller, microcontroller, or state machine. A controller may also be implemented as a combination of computing devices, e.g., a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors in conjunction with a DSP, and/or any other such configuration.
In some embodiments, during the activity of the robot, the controller employs SLAM (simultaneous localization and mapping) technology to construct a map and a position according to the environmental data, so as to move to a target location to complete a delivery task, a cleaning task, and the like. The controller instructs the robot to completely traverse an environmental space through a full coverage path planning algorithm based on the established map and the position of the robot. For example, during the robot traversal, the sensor unit acquires an image of a traversal region, wherein the image of the traversal region may be an image of the entire traversal region or an image of a local traversal region in the entire traversal region. The controller generates a map from the image of the traversal area, the map having indicated an area that the robot needs to traverse and coordinate locations at which obstacles located in the traversal area are located. After each location or area traversed by the robot, the robot marks that the location or area has been traversed based on the map. In addition, as the obstacle is marked in a coordinate mode in the map, when the robot passes, the distance between the robot and the obstacle can be judged according to the coordinate point corresponding to the current position and the coordinate point related to the obstacle, and therefore the robot can pass around the obstacle. Similarly, after the position or the area is traversed and marked, when the next position of the robot moves to the position or the area, the robot makes a strategy of turning around or stopping traversing based on the map and the mark of the position or the area.
It will be appreciated that the controller may also identify traversed locations or areas, or identify obstacles, in a variety of ways to develop a control strategy that meets product needs.
The method for constructing the cost map adopts a semantic segmentation network model. Semantic segmentation is a technology for performing pixel-level classification on an image, and with the development of deep learning, more and more lightweight semantic segmentation network models such as Bisenet and Enet can segment a travelable region in a camera data picture captured by a robot camera to guide the robot to perform path planning in a local region. The robot path planning is usually realized by means of a global static map and a local cost map, and the generation of the local cost map mainly senses the surrounding environment of the robot by means of a robot sensor and constructs the cost value in the cost map according to the environmental obstacle information. The method of the invention is a method for constructing a local cost map by utilizing a semantic segmentation network model.
Referring to fig. 1, a schematic flow chart of a method for constructing a cost map based on road surface detection is provided for embodiment 1 of the present invention, and as shown in fig. 1, the method includes the following steps:
step 1, collecting a front view image, calculating the confidence coefficient of each pixel point in the front view image, which is a road surface, based on a preset semantic segmentation network, and generating a confidence coefficient matrix. In the embodiment, a real-time lightweight semantic segmentation network Bisenet is adopted, a front view image is acquired from a front camera of a robot, the image is placed in a pavement segmentation network, and the confidence Score [ u, v ] that each pixel point in the image is an indoor pavement is calculated.
And 2, acquiring the spatial position coordinates of at least one preset road point on the spatial road surface corresponding to the front view image, and calculating the initial road surface confidence coefficient of each preset road point according to the confidence coefficient matrix. The method specifically comprises the following steps:
s201, obtaining the space position coordinates of at least one preset road surface point on the space road surface corresponding to the front view image. When preset pavement points are set on a spatial pavement, X represents an abscissa relative to the robot, and Z represents an ordinate relative to the robot, for example, X is from-2 meters to 2 meters, one coordinate point is taken every 0.1 meter, Z is from 0 meter to 5 meters, and one coordinate point is taken every 0.1 meter, so that a grid map is generated.
S202, calculating a corresponding target pixel point of each preset road point in the front view image by adopting a preset coordinate transformation method. The invention adopts the principle of single mapping to carry out coordinate transformation.
Specifically, the internal and external parameters (K, RT) of the robot camera are calibrated in a checkerboard calibration mode, and an internal parameter matrix K of the camera is as follows:
external reference matrix RT:
the robot camera is mounted on the robot body at the height Hc in a head-up manner, so that R11 ═ R22 ═ R33 ═ 1, R12 ═ R13 ═ R21 ═ R23 ═ R31 ═ R32 ═ 0, the robot coordinate system is converted into, t1 ═ t3 ═ 0, t2 ═ Hc (camera _ height) (i.e. camera mounting height, fixed value). The camera matrix KRT can then be expressed as:
in the road surface segmentation scene, the spatial height coordinate Y of the road surface is 0, so the relationship between the image coordinate [ u, v ] of the road surface and the spatial position [ X,0, Z ] of the road surface is as follows:
from the above formula, the road surface [ X,0, Z ] in the space can be mapped to the corresponding pixel point [ u, v ] in the visual picture.
And S203, generating four adjacent pixel points corresponding to each target pixel point in the front view image by adopting a bilinear interpolation method. Any path point [ X,0, Z ] is considered and mapped to a corresponding point [ u, v ] in the visual picture, wherein u and v are not integers usually, and the approximate calculation of surrounding pixel points is needed. The method adopts a bilinear interpolation method to calculate the confidence coefficient of the road surface, and takes four points ([ u ], [ v ]), ([ u +1], [ v ]), ([ u ], [ v +1]), ([ u +1], [ v +1]) near the target point u, v, wherein the symbol [ ] represents the lower rounding.
And S204, querying the confidence coefficient matrix, and acquiring a first adjacent point confidence coefficient S1, a second adjacent point confidence coefficient S2, a third adjacent point confidence coefficient S3 and a fourth adjacent point confidence coefficient S4 which correspond to the four adjacent pixel points respectively.
S205, calculating the initial road confidence of each preset road point by adopting a first preset formula, wherein the first preset formula is as follows:
score＝(1-dυ)*f1+dυ*f2，
f1＝(1-du)*s1+du*s2，
f2＝(1-du)*s3+du*s4，
the score is an initial road confidence of a preset road point, u and v are coordinates of corresponding target pixel points in the front view image, du is equal to u- [ u ], dv is equal to v- [ v ], and a symbol [ ] represents rounding-down.
And then, executing step 3, and constructing a cost function corresponding to the obstacle map layer according to the initial road confidence corresponding to all the preset road points so as to generate a local cost map. The method specifically comprises the following steps:
s301, filtering the initial road surface confidence coefficient of each preset road surface point by adopting a preset smooth filtering algorithm to generate an optimized road surface confidence coefficient. The indoor road surface is usually a smooth connected region, so the generated cost map should be smooth, and the confidence coefficient directly calculated by the above steps has noise interference, so that confidence filtering is needed. The preset smoothing filter algorithm comprises any one of a mean smoothing filter algorithm, a field smoothing filter algorithm, a bilateral filter algorithm and a median filter algorithm. In a preferred embodiment, a mean smoothing filter algorithm may be used to filter the initial road confidence of each preset road point, and the adopted filter formula is as follows:
wherein the content of the first and second substances,
i. j is the coordinate of the preset road point.
S302, constructing a cost function of a corresponding obstacle map layer by adopting the optimized road confidence corresponding to all the preset road points as follows:
costob[i，j]＝-log(score[i，j])。
and finally, combining the global static map layer to generate a cost map for a subsequent planning module to use.
According to the method for constructing the cost map based on the pavement detection, when a robot moves in an indoor flat layer, the pavement segmentation of an image is realized by means of a robot camera, the confidence coefficient of the pavement is projected to the cost function for constructing the obstacle map layer under a space coordinate system, the calculation process is simple and quick, and the method has a good application effect in scenes such as strong light reflection, pavement collapse, small stones and the like of a glass mirror.
It should be noted that, in the foregoing embodiments, a certain order does not necessarily exist between the foregoing steps, and it can be understood by those skilled in the art from the description of the embodiments of the present invention that, in different embodiments, the foregoing steps may have different execution orders, that is, may be executed in parallel, may also be executed in an exchange manner, and the like.
As another aspect of the embodiment of the present invention, an embodiment of the present invention further provides a device for constructing a cost map based on road surface detection. The device for constructing the cost map based on the road surface detection can be a software module, the software module comprises a plurality of instructions which are stored in a memory, and the processor can access the memory and call the instructions to execute the instructions so as to complete the method for constructing the cost map based on the road surface detection, which is set forth in each embodiment.
In some embodiments, the device for constructing the cost map based on road surface detection may also be constructed by hardware devices, for example, the device for constructing the cost map based on road surface detection may be constructed by one or more than two chips, and the chips may work in coordination with each other to complete the method for constructing the cost map based on road surface detection described in the above embodiments. For another example, the device for constructing the cost map based on road surface detection may also be constructed by various logic devices, such as a general processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA), a single chip microcomputer, an arm (aconris cmachine) or other programmable logic devices, discrete gate or transistor logic, discrete hardware components, or any combination of these components.
Fig. 2 is a schematic structural diagram of an apparatus for constructing a cost map based on road surface detection according to embodiment 2 of the present invention, which includes a segmentation module 100, a calculation module 200 and a construction module 300,
the segmentation module 100 is configured to collect a front view image, calculate a confidence level that each pixel point in the front view image is a road surface based on a preset semantic segmentation network, and generate a confidence matrix;
the calculation module 200 is configured to acquire a spatial position coordinate of at least one preset road point on a spatial road corresponding to the front view image, and calculate an initial road confidence of each preset road point according to the confidence matrix;
the building module 300 is configured to build a cost function corresponding to the obstacle map layer according to the initial road confidence corresponding to all the preset road points, so as to generate a local cost map.
In a preferred embodiment, the computing module 200 specifically includes:
the acquisition unit is used for acquiring the spatial position coordinates of at least one preset road point on the spatial road corresponding to the front view image;
the coordinate transformation unit is used for calculating a corresponding target pixel point of each preset road point in the front view image by adopting a preset coordinate transformation method;
the interpolation unit is used for generating four adjacent pixel points corresponding to each target pixel point in the front view image by adopting a bilinear difference method;
the query unit is used for querying the confidence coefficient matrix and acquiring a first adjacent point confidence coefficient s1, a second adjacent point confidence coefficient s2, a third adjacent point confidence coefficient s3 and a fourth adjacent point confidence coefficient s4 which correspond to the four adjacent pixel points respectively;
a calculating unit, configured to calculate an initial road confidence of each preset road point by using a first preset formula, where the first preset formula is:
score＝(1-dυ)*f1+dυ*f2，
f1＝(1-du)*s1+du*s2，
f2＝(1-du)*s3+du*s4，
the score is an initial road confidence of a preset road point, u and v are coordinates of corresponding target pixel points in the front view image, du is equal to u- [ u ], dv is equal to v- [ v ], and a symbol [ ] represents rounding-down.
In a preferred embodiment, the building module 300 specifically includes:
the filtering unit is used for filtering the initial road surface confidence coefficient of each preset road surface point by adopting a mean smooth filtering algorithm to generate an optimized road surface confidence coefficient;
and the construction unit is used for constructing a cost function corresponding to the barrier map layer by adopting the optimized road confidence corresponding to all the preset road points.
According to the device for constructing the cost map based on the pavement detection, when a robot moves in an indoor flat layer, the pavement segmentation of an image is realized by means of a robot camera, the confidence coefficient of the pavement is projected to the cost function for constructing the ground layer of the obstacle under a space coordinate system, the calculation process is simple and quick, and the device has a good application effect in scenes such as strong reflection of light, pavement collapse, small stones and the like of a glass mirror.
It should be noted that the device for constructing the cost map based on the road surface detection can execute the method for constructing the cost map based on the road surface detection provided by the embodiment of the invention, and has corresponding functional modules and beneficial effects of the execution method. For technical details that are not described in detail in the embodiment of the apparatus for constructing the cost map based on road surface detection, reference may be made to the method for constructing the cost map based on road surface detection provided in the embodiment of the present invention.
Fig. 3 is a schematic circuit diagram of a controller according to an embodiment of the present invention. As shown in fig. 3, the controller 600 includes one or more processors 61 and a memory 62. In fig. 3, one processor 61 is taken as an example.
The processor 61 and the memory 62 may be connected by a bus or other means, such as the bus connection in fig. 3.
The memory 62, which is a non-volatile computer-readable storage medium, may be used to store non-volatile software programs, non-volatile computer-executable programs, and modules, such as program instructions/modules corresponding to the method for constructing a cost map based on road surface detection in the embodiment of the present invention. The processor 61 executes various functional applications and data processing of the device for constructing the cost map based on road surface detection by running the nonvolatile software program, instructions and modules stored in the memory 62, that is, the method for constructing the cost map based on road surface detection provided by the above method embodiment and the functions of the various modules or units of the above device embodiment are realized.
The memory 62 may include high speed random access memory and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other non-volatile solid state storage device. In some embodiments, the memory 62 may optionally include memory located remotely from the processor 61, and these remote memories may be connected to the processor 61 via a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
The program instructions/modules stored in the memory 62, when executed by the one or more processors 61, perform a method of constructing a cost map based on road surface detection in any of the method embodiments described above.
Embodiments of the present invention further provide a non-volatile computer storage medium, where the computer storage medium stores computer-executable instructions, which are executed by one or more processors, such as one of the processors 61 in fig. 3, so that the one or more processors may execute the method for constructing the cost map based on road surface detection in any of the above-mentioned method embodiments.
Embodiments of the present invention also provide a computer program product, which includes a computer program stored on a non-volatile computer-readable storage medium, where the computer program includes program instructions, and when the program instructions are executed by an electronic device, the electronic device is caused to execute any one of the methods for constructing a cost map based on road surface detection.
The above-described embodiments of the apparatus or device are merely illustrative, wherein the unit modules described as separate parts may or may not be physically separate, and the parts displayed as module units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network module units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment.
Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented by software plus a general hardware platform, and certainly can also be implemented by hardware. Based on such understanding, the above technical solutions substantially or contributing to the related art may be embodied in the form of a software product, which may be stored in a computer-readable storage medium, such as ROM/RAM, magnetic disk, optical disk, etc., and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the method according to the embodiments or some parts of the embodiments.
Finally, it should be noted that: the above examples are only intended to illustrate the technical solution of the present invention, but not to limit it; within the idea of the invention, also technical features in the above embodiments or in different embodiments may be combined, steps may be implemented in any order, and there are many other variations of the different aspects of the invention as described above, which are not provided in detail for the sake of brevity; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and the modifications or the substitutions do not make the essence of the corresponding technical solutions depart from the scope of the technical solutions of the embodiments of the present invention.
Claims (10)
1. A method for constructing a cost map based on road surface detection is characterized by comprising the following steps:
step 1, collecting a front visual field image, calculating the confidence coefficient of each pixel point in the front visual field image, which is a road surface, based on a preset semantic segmentation network, and generating a confidence coefficient matrix;
step 2, acquiring the spatial position coordinates of at least one preset road point on the spatial road surface corresponding to the front visual field image, and calculating the initial road surface confidence coefficient of each preset road point through the confidence coefficient matrix;
and 3, constructing a cost function corresponding to the barrier map layer according to the initial road confidence corresponding to all the preset road points so as to generate a local cost map.
2. The method for constructing the cost map based on the road surface detection according to claim 1, wherein the initial road surface confidence coefficient of each preset road surface point is calculated through a confidence coefficient matrix, and the method specifically comprises the following steps:
s201, acquiring the spatial position coordinates of at least one preset road point on a spatial road corresponding to the front view image;
s202, calculating a corresponding target pixel point of each preset road point in the front view image by adopting a preset coordinate transformation method;
s203, generating four adjacent pixel points corresponding to each target pixel point in the front view image by adopting a bilinear interpolation method;
s204, inquiring the confidence matrix, and acquiring a first adjacent point confidence coefficient S1, a second adjacent point confidence coefficient S2, a third adjacent point confidence coefficient S3 and a fourth adjacent point confidence coefficient S4 which correspond to the four adjacent pixel points respectively;
s205, calculating the initial road confidence of each preset road point by adopting a first preset formula, wherein the first preset formula is as follows:
score＝(1-dv)*f1+dv*f2，
f1＝(1-du)*s1+du*s2，
f2＝(1-du)*s3+du*s4，
the score is an initial road confidence of a preset road point, du is u- [ u ], dv is v- [ v ], u and v are corresponding target pixel point coordinates in the front view image, and a symbol [ ] represents a downward rounding.
3. The method for constructing the cost map based on the road surface detection according to claim 1 or 2, wherein the step of constructing the cost function of the corresponding obstacle map layer by using the initial road surface confidence degrees of all the preset road surface points specifically comprises the following steps:
filtering the initial pavement confidence coefficient of each preset pavement point by adopting a preset smooth filtering algorithm to generate an optimized pavement confidence coefficient;
constructing a cost function corresponding to the barrier map layer by adopting the optimized road confidence;
the preset smoothing filter algorithm comprises any one of a mean smoothing filter algorithm, a field smoothing filter algorithm, a bilateral filter algorithm and a median filter algorithm.
4. The method for constructing the cost map based on the road surface detection according to claim 3, wherein the initial road surface confidence of each preset road surface point is filtered by adopting a mean smoothing filter algorithm, and the adopted filter formula is as follows:
wherein the content of the first and second substances,
i. j is the coordinate of the preset road point.
5. The method for constructing the cost map based on the road surface detection as claimed in claim 4, wherein the cost function for constructing the corresponding obstacle map layer by adopting the optimized road surface confidence corresponding to all the preset road surfaces is as follows:
costob[i，j]＝-log(score[i，j])。
6. a device for constructing a cost map based on road surface detection is characterized by comprising a segmentation module, a calculation module and a construction module,
the segmentation module is used for acquiring a front view image, calculating the confidence coefficient that each pixel point in the front view image is a road surface respectively based on a preset semantic segmentation network, and generating a confidence coefficient matrix;
the calculation module is used for acquiring the spatial position coordinates of at least one preset road point on the spatial road surface corresponding to the front view image and calculating the initial road surface confidence coefficient of each preset road point according to the confidence coefficient matrix;
the construction module is used for constructing a cost function corresponding to the barrier map layer according to the initial road confidence corresponding to all the preset road points so as to generate a local cost map.
7. The device for constructing the cost map based on the road surface detection according to claim 6, wherein the calculation module specifically comprises:
the acquisition unit is used for acquiring the spatial position coordinates of at least one preset road point on the spatial road corresponding to the front view image;
the coordinate transformation unit is used for calculating a corresponding target pixel point of each preset road point in the front view image by adopting a preset coordinate transformation method;
the interpolation unit is used for generating four adjacent pixel points corresponding to each target pixel point in the front view image by adopting a bilinear interpolation method;
the query unit is used for querying the confidence coefficient matrix and acquiring a first adjacent point confidence coefficient s1, a second adjacent point confidence coefficient s2, a third adjacent point confidence coefficient s3 and a fourth adjacent point confidence coefficient s4 which correspond to the four adjacent pixel points respectively;
a calculating unit, configured to calculate an initial road confidence of each preset road point by using a first preset formula, where the first preset formula is:
score＝(1-dv)*f1+dv*f2，
f1＝(1-du)*s1+du*s2，
f2＝(1-du)*s3+du*s4，
wherein, score is the initial road surface confidence of the preset road surface point, u and v are the corresponding target pixel points in the front view image, du is equal to u- [ u ], dv is equal to v- [ v ], and the symbol [ ] represents rounding-down.
8. The device for constructing the cost map based on the road surface detection according to claim 6 or 7, wherein the construction module specifically comprises:
the filtering unit is used for filtering the initial road surface confidence coefficient of each preset road surface point by adopting a mean smooth filtering algorithm to generate an optimized road surface confidence coefficient;
and the construction unit is used for constructing a cost function corresponding to the barrier map layer by adopting the optimized road confidence corresponding to all the preset road points.
9. A computer-readable storage medium, storing a computer program, wherein the computer program, when executed by a processor, implements the method for constructing a cost map based on road surface detection according to any one of claims 1 to 5.
10. A robot comprising the computer-readable storage medium of claim 9 and a processor which, when executing a computer program on the computer-readable storage medium, carries out the steps of the method of constructing a cost map based on road surface detection of any one of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011570427.7A CN112489131B (en) | 2020-12-26 | 2020-12-26 | Method, device, medium and robot for constructing cost map based on pavement detection |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011570427.7A CN112489131B (en) | 2020-12-26 | 2020-12-26 | Method, device, medium and robot for constructing cost map based on pavement detection |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112489131A true CN112489131A (en) | 2021-03-12 |
| CN112489131B CN112489131B (en) | 2024-04-05 |
Family
ID=74914437
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011570427.7A Active CN112489131B (en) | 2020-12-26 | 2020-12-26 | Method, device, medium and robot for constructing cost map based on pavement detection |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112489131B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114648697A (en) * | 2022-03-16 | 2022-06-21 | 东南大学 | Robot travelable path identification method based on improved BiSeNet network |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109241855A (en) * | 2018-08-10 | 2019-01-18 | 西安交通大学 | Intelligent vehicle based on stereoscopic vision can travel area detection method |
| CN109682381A (en) * | 2019-02-22 | 2019-04-26 | 山东大学 | Big visual field scene perception method, system, medium and equipment based on omnidirectional vision |
| CN109765901A (en) * | 2019-02-18 | 2019-05-17 | 华南理工大学 | Dynamic cost digital map navigation method based on line laser and binocular vision |
| CN109903346A (en) * | 2019-05-13 | 2019-06-18 | 长沙智能驾驶研究院有限公司 | Camera attitude detecting method, device, equipment and storage medium |
| CN111178253A (en) * | 2019-12-27 | 2020-05-19 | 深圳佑驾创新科技有限公司 | Visual perception method and device for automatic driving, computer equipment and storage medium |
| WO2020119684A1 (en) * | 2018-12-14 | 2020-06-18 | 中国科学院深圳先进技术研究院 | 3d navigation semantic map update method, apparatus and device |
| US20200218908A1 (en) * | 2019-01-04 | 2020-07-09 | Qualcomm Incorporated | Real-time simultaneous detection of lane marker and raised pavement marker for optimal estimation of multiple lane boundaries |
| CN112015186A (en) * | 2020-09-09 | 2020-12-01 | 上海有个机器人有限公司 | Robot path planning method and device with social attributes and robot |
-
2020
- 2020-12-26 CN CN202011570427.7A patent/CN112489131B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109241855A (en) * | 2018-08-10 | 2019-01-18 | 西安交通大学 | Intelligent vehicle based on stereoscopic vision can travel area detection method |
| WO2020119684A1 (en) * | 2018-12-14 | 2020-06-18 | 中国科学院深圳先进技术研究院 | 3d navigation semantic map update method, apparatus and device |
| US20200218908A1 (en) * | 2019-01-04 | 2020-07-09 | Qualcomm Incorporated | Real-time simultaneous detection of lane marker and raised pavement marker for optimal estimation of multiple lane boundaries |
| CN109765901A (en) * | 2019-02-18 | 2019-05-17 | 华南理工大学 | Dynamic cost digital map navigation method based on line laser and binocular vision |
| CN109682381A (en) * | 2019-02-22 | 2019-04-26 | 山东大学 | Big visual field scene perception method, system, medium and equipment based on omnidirectional vision |
| CN109903346A (en) * | 2019-05-13 | 2019-06-18 | 长沙智能驾驶研究院有限公司 | Camera attitude detecting method, device, equipment and storage medium |
| CN111178253A (en) * | 2019-12-27 | 2020-05-19 | 深圳佑驾创新科技有限公司 | Visual perception method and device for automatic driving, computer equipment and storage medium |
| CN112015186A (en) * | 2020-09-09 | 2020-12-01 | 上海有个机器人有限公司 | Robot path planning method and device with social attributes and robot |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114648697A (en) * | 2022-03-16 | 2022-06-21 | 东南大学 | Robot travelable path identification method based on improved BiSeNet network |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112489131B (en) | 2024-04-05 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109084732B (en) | Positioning and navigation method, device and processing equipment | |
| JP7073315B2 (en) | Vehicles, vehicle positioning systems, and vehicle positioning methods | |
| US9329598B2 (en) | Simultaneous localization and mapping for a mobile robot | |
| CN110488818B (en) | Laser radar-based robot positioning method and device and robot | |
| CN103150748B (en) | Reversing image three-dimensional (3D) scene reconstruction method and system | |
| CN103592944A (en) | Supermarket shopping robot and advancing path planning method thereof | |
| CN111220148A (en) | Mobile robot positioning method, system and device and mobile robot | |
| CN113741438A (en) | Path planning method and device, storage medium, chip and robot | |
| CN111399516A (en) | Robot path planning method and device and robot | |
| TWI754808B (en) | Vehicle, vehicle positioning system, and vehicle positioning method | |
| CN112015186A (en) | Robot path planning method and device with social attributes and robot | |
| CN115328153A (en) | Sensor data processing method, system and readable storage medium | |
| CN114077249B (en) | Operation method, operation equipment, device and storage medium | |
| CN112489131A (en) | Method, device, medium and robot for constructing cost map based on road surface detection | |
| CN112146620B (en) | Target object ranging method and device | |
| CN112033423B (en) | Robot path planning method and device based on road consensus and robot | |
| JP2010225126A (en) | Moving path planning apparatus, mobile body control apparatus and mobile body | |
| Hu et al. | A small and lightweight autonomous laser mapping system without GPS | |
| CN113470121B (en) | Autonomous mobile platform, external parameter optimization method, device and storage medium | |
| CN112045654B (en) | Detection method and device for unmanned closed space and robot | |
| US11797013B2 (en) | Collision avoidance method and mobile machine using the same | |
| CN112528728B (en) | Image processing method and device for visual navigation and mobile robot | |
| CN112284387A (en) | Positioning method for cleaning device, and storage medium | |
| Kowsari et al. | Map-based lane and obstacle-free area detection | |
| CN112597915B (en) | Method, device, medium and robot for identifying indoor close-distance pedestrians |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |