CN112415998B - Obstacle classification obstacle avoidance control system based on TOF camera - Google Patents
Obstacle classification obstacle avoidance control system based on TOF camera Download PDFInfo
- Publication number
- CN112415998B CN112415998B CN202011159235.7A CN202011159235A CN112415998B CN 112415998 B CN112415998 B CN 112415998B CN 202011159235 A CN202011159235 A CN 202011159235A CN 112415998 B CN112415998 B CN 112415998B
- Authority
- CN
- China
- Prior art keywords
- obstacle
- mobile robot
- type
- tof camera
- module
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000004804 winding Methods 0.000 claims description 43
- 238000000034 method Methods 0.000 claims description 25
- 238000001514 detection method Methods 0.000 claims description 12
- 230000008859 change Effects 0.000 claims description 10
- 230000000007 visual effect Effects 0.000 claims description 7
- 230000006854 communication Effects 0.000 claims description 4
- 238000004891 communication Methods 0.000 claims description 4
- 230000003287 optical effect Effects 0.000 claims description 4
- 238000003384 imaging method Methods 0.000 claims description 3
- 230000007175 bidirectional communication Effects 0.000 claims description 2
- 230000009471 action Effects 0.000 abstract description 5
- 238000012549 training Methods 0.000 abstract description 4
- 238000004519 manufacturing process Methods 0.000 abstract description 3
- 238000003672 processing method Methods 0.000 abstract 1
- 230000008569 process Effects 0.000 description 6
- 238000009434 installation Methods 0.000 description 4
- 230000001960 triggered effect Effects 0.000 description 4
- 230000000694 effects Effects 0.000 description 3
- 238000011065 in-situ storage Methods 0.000 description 3
- 238000005070 sampling Methods 0.000 description 3
- 238000013459 approach Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 2
- 239000004568 cement Substances 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000005259 measurement Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000010408 sweeping Methods 0.000 description 2
- 239000002023 wood Substances 0.000 description 2
- 230000001133 acceleration Effects 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 238000004140 cleaning Methods 0.000 description 1
- 239000003086 colorant Substances 0.000 description 1
- 239000002131 composite material Substances 0.000 description 1
- 238000012937 correction Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 239000011521 glass Substances 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 238000000926 separation method Methods 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
- G05D1/02—Control of position or course in two dimensions
- G05D1/021—Control of position or course in two dimensions specially adapted to land vehicles
- G05D1/0212—Control of position or course in two dimensions specially adapted to land vehicles with means for defining a desired trajectory
- G05D1/0214—Control of position or course in two dimensions specially adapted to land vehicles with means for defining a desired trajectory in accordance with safety or protection criteria, e.g. avoiding hazardous areas
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
- G05D1/02—Control of position or course in two dimensions
- G05D1/021—Control of position or course in two dimensions specially adapted to land vehicles
- G05D1/0231—Control of position or course in two dimensions specially adapted to land vehicles using optical position detecting means
- G05D1/0246—Control of position or course in two dimensions specially adapted to land vehicles using optical position detecting means using a video camera in combination with image processing means
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Aviation & Aerospace Engineering (AREA)
- Radar, Positioning & Navigation (AREA)
- Remote Sensing (AREA)
- General Physics & Mathematics (AREA)
- Automation & Control Theory (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Multimedia (AREA)
- Electromagnetism (AREA)
- Control Of Position, Course, Altitude, Or Attitude Of Moving Bodies (AREA)
- Manipulator (AREA)
Abstract
The invention discloses an obstacle classifying obstacle avoidance control system based on a TOF camera, which adopts obstacle size information, position information and brightness information acquired by the TOF camera to effectively classify and identify whether obstacles which should be spanned and collided in an indoor environment, timely trigger a collision warning signal according to the type characteristics of the identified obstacles and the size information of the identified obstacles so as to push a mobile robot to plan a passable area before moving to the corresponding obstacles, and is more suitable for planning an obstacle-free passable area in an indoor actual moving environment compared with the obstacle classifying and processing method in the prior art, thereby ensuring the effectiveness of an obstacle avoidance path and realizing collision-free movement as much as possible. Compared with the prior art, the robot adopts a plurality of cameras or a plurality of laser heads and performs excessive image characteristic point fitting classification training, so that the production cost is reduced, and the real-time performance of obstacle avoidance actions of the robot is improved.
Description
Technical Field
The invention relates to the technical field of intelligent robots, in particular to an obstacle classification obstacle avoidance control system based on a TOF camera.
Background
At present, SLAM robots based on inertial navigation, vision and laser are more and more popular, and a household floor sweeping cleaning robot is strong in representativeness, and the indoor environment is positioned and built in real time by combining data of vision, laser, a gyroscope, acceleration and a wheel odometer, and positioning navigation is realized according to the built map. However, the current pain point is that the robot often has a movable obstacle such as a toy, an electric wire and the like on the ground in a complex obstacle environment, and when the robot collides with the obstacle, the robot can push the obstacle or is wound by the electric wire type obstacle, the sofa type obstacle is also present in a family environment, if the height under the sofa is just lower than the height of the top surface of the machine, the machine can be blocked when entering, because the current sweeping robot is in consideration of cost, the used laser is a single line, the obstacle of the type cannot be detected, the vision is generally realized by using a single camera, and the distance calculation cannot be accurately performed in time, so that the obstacle cannot be accurately detected in real time, and the robot can be classified.
The Chinese patent CN110622085A published in the application of the year 12 and 27 in 2019 relates to the adoption of at least one camera device to acquire a depth image of an obstacle, and then the second image feature of the depth image of the obstacle and the first image feature of the color image are combined to extract and identify the type of the obstacle, but the identified obstacle is not effectively classified according to the passable condition in the indoor actual activity environment of the robot, and excessive image feature point fitting classification training can influence the instantaneity of obstacle avoidance actions of the robot.
Disclosure of Invention
In order to solve the technical problems, the TOF camera module disclosed by the invention acquires the depth information of the obstacle in the detectable visual angle range and the brightness information of the depth image, and can detect the relative position of the obstacle and identify the effective type characteristics of the obstacle to distinguish whether the obstacle should be crossed or not and whether the obstacle should be collided or not, so that when a robot approaches to the obstacle to be collided, the robot can avoid the obstacle in advance to realize a collision-free function or walk around the obstacle or along the edge. The specific technical scheme is as follows:
An obstacle classification obstacle avoidance control system based on a TOF camera is arranged on a mobile robot and comprises an obstacle classification module, an obstacle positioning triggering module and an obstacle avoidance module; the obstacle classification module comprises a TOF camera arranged in the advancing direction of the mobile robot, and is used for calculating the relative position relation of at least one obstacle and the size of the same obstacle in the detection visual angle range of the TOF camera according to the depth image acquired by the TOF camera in real time, and then selecting and combining the brightness image data which is output by the TOF camera in real time and is matched with the corresponding obstacle to identify the type of the corresponding obstacle; the obstacle positioning triggering module is used for receiving the relative position relation of the obstacle of the type currently identified by the obstacle classification module and the size of the same obstacle, and deciding whether the obstacle of the type correspondingly identified triggers a collision warning signal or not based on the data information so as to enable the obstacle to be identified: the mobile robot plans a passable area before moving to a corresponding obstacle; the obstacle avoidance module is used for adjusting the current pose of the mobile robot according to the relative position relation of the obstacle triggering the collision warning signal transmitted by the obstacle positioning triggering module and the size of the same obstacle, and planning an obstacle avoidance path of the mobile robot according to the relative position relation of the obstacle of the current identified type transmitted by the obstacle positioning triggering module and the size of the same obstacle, so that the mobile robot avoids the obstacle triggering the collision warning signal, bypasses the obstacle triggering the collision warning signal, or keeps the original pose to walk continuously.
According to the technical scheme, the obstacle depth information, the size information and the brightness information acquired by the TOF camera are adopted to effectively classify and identify whether obstacles which need to be spanned and collided in an indoor environment, and collision warning signals are triggered timely according to the type characteristics, the relative positions and the size information of the identified obstacles so as to push the mobile robot to plan a passable area before moving to the corresponding obstacle. Compared with the prior art, the laser head adopting a plurality of cameras or multiple lines, and performing excessive image characteristic points and fitting classification training are too large, the technical scheme reduces the production cost of the system and improves the real-time performance of obstacle avoidance actions of the robot.
Further, in the obstacle locating triggering module, the method for deciding whether the obstacle of the type is correspondingly identified to trigger the collision warning signal based on the data information at least comprises the following steps: defining the identified type of obstacle as a target obstacle; types of obstacles include winds, island obstacles, straddlable sills, and furniture that can be traversed by a mobile robot; when the type of the target obstacle is a winding object, if the depth data of the depth image of the target obstacle shot by the TOF camera is smaller than a first safety depth threshold, controlling the obstacle positioning triggering module to trigger a collision warning signal; when the type of the target obstacle is an island obstacle, if the depth data of the depth image of the target obstacle shot by the TOF camera is smaller than a second safety depth threshold, controlling the obstacle positioning triggering module to trigger a collision warning signal; when the type of the target obstacle is a wall body, if the depth data of the depth image of the target obstacle shot by the TOF camera is smaller than a third safety depth threshold, controlling the obstacle positioning triggering module to trigger a collision warning signal; when the type of the target obstacle is a threshold which can be crossed or furniture which can be crossed by the mobile robot, controlling the obstacle positioning triggering module not to trigger a collision warning signal; the first safety depth threshold is larger than the second safety depth threshold, and the second safety depth threshold is larger than the third safety depth threshold. According to the technical scheme, before the identified target obstacle in the visual angle range is approached, the requirements of the matched type obstacle on collision obstacle avoidance are met by setting different safety distances, so that an obstacle-free passable area is pre-judged, and an effective obstacle avoidance path is conveniently planned in the follow-up process.
Further, in the obstacle avoidance module, the method for adjusting the current pose of the mobile robot according to the relative position relationship of the obstacle triggering the collision warning signal and the size of the same obstacle transmitted by the obstacle positioning triggering module at least includes: when the type of the target obstacle is a winding object and the obstacle positioning triggering module triggers the collision warning signal, controlling a mobile robot to reduce the real-time speed and change the current advancing direction according to the outline width of the target obstacle so as to enable a straight line path in the advancing direction after the change to avoid the target obstacle; when the type of the target obstacle is a spanable threshold or furniture which can be traversed by the mobile robot, controlling the mobile robot to keep the current advancing direction; when the type of the target obstacle is an island obstacle and the obstacle positioning triggering module triggers the collision warning signal, controlling the mobile robot to change the current advancing direction so as to enable the mobile robot to start to walk around the outline of the island obstacle; when the type of the target obstacle is a wall body and the obstacle positioning triggering module triggers the collision warning signal, controlling the mobile robot to adjust the optimal edge direction so as to enable the mobile robot to start to enter an edge mode. The method is beneficial to optimizing the current pose of the mobile robot, so that the current motion state (including normal straight walking, in-situ turning, radian turning and edge turning) of the mobile robot is optimized, and the probability of contact of the mobile robot with the obstacle of the identified type is reduced.
Further, in the obstacle avoidance module, the method for planning the obstacle avoidance path of the mobile robot at least includes: firstly, planning a passable area between the mobile robot and the obstacles according to the relative position relation of the obstacles of the type which are currently identified and the size of the same obstacle, and then planning an obstacle avoidance path of the mobile robot in the passable area; the obstacle positioning triggering module is used for transmitting a current recognized type of obstacle to the TOF camera, wherein the current recognized type of obstacle transmitted by the obstacle positioning triggering module comprises an obstacle triggering collision warning signals and other target obstacles in a detection view angle range of the TOF camera; when the type of the target obstacle is a winding object and the obstacle positioning triggering module triggers the collision warning signal, planning an initial obstacle avoidance direction of the mobile robot according to the outline width of the target obstacle, and simultaneously adjusting the initial obstacle avoidance direction of the mobile robot in real time according to position coordinates corresponding to all points on the outline of the identified winding object and other position coordinates of the target obstacle so as to avoid touching the identified winding object, and keeping and adjusting depth data corresponding to all points on the outline of the winding object acquired in real time to be larger than a first safety depth threshold; when the type of the target obstacle is a spanable threshold or furniture which can be traversed by the mobile robot, controlling the mobile robot to walk linearly along the current advancing direction so as to span or traverse the target obstacle; when the type of the target obstacle is an island obstacle and the obstacle positioning triggering module triggers the collision warning signal, controlling the mobile robot to walk along the extending direction of the outline of the island obstacle, and keeping adjusting depth data corresponding to each point on the outline line of the island obstacle which is acquired and identified in real time to be larger than a second safety depth threshold so as to realize obstacle-surrounding walking of the mobile robot; when the type of the target obstacle is a wall body and the obstacle positioning triggering module triggers the collision warning signal, controlling the mobile robot to adjust the optimal edge direction, and starting to execute the walking along the wall along the optimal edge direction.
Thereby do: the TOF camera can control the mobile robot to stride across the threshold when recognizing the threshold, the TOF camera can control the mobile robot to enter the sofa bottom when recognizing the sofa (when the height of the sofa bottom is lower than that of the machine), the TOF camera can prohibit the mobile robot from colliding with the toy and walking around the obstacle when recognizing the toy, the TOF camera can allow the mobile robot to collide with the wall body to walk along the edge when recognizing the wall, and prohibit the mobile robot from crossing wound objects such as wires and avoiding the obstacle in other paths when recognizing the wire, so as to plan an effective obstacle avoidance path.
Further, the TOF camera is installed in front of the mobile robot, the installation height of the TOF camera relative to the traveling plane is 6.7 cm, the optical axis of the TOF camera forms an inclined angle of 10 degrees relative to the top surface of the robot, and a driving wheel at the bottom of the mobile robot is in contact with the traveling plane. So that the brightness value of the target obstacle in the detection view angle range is effective and the contour line of the target obstacle is complete to meet the depth positioning requirement.
Further, the obstacle classification module and the obstacle avoidance module are respectively connected with the obstacle positioning triggering module through a two-way communication interface so as to establish a communication relation of receiving and transmitting responses between signal data. The real-time performance of the depth image information feedback of each obstacle is guaranteed, and the efficiency of obstacle avoidance path planning is improved.
Further, the size of the same obstacle comprises the height of each position point of the contour line of the corresponding obstacle within the preset horizontal width; the preset horizontal width is the length of a horizontal line segment in a preselected communicable domain corresponding to the surface of the obstacle, and the horizontal projection of each position point of the contour line of the corresponding obstacle in the preset horizontal width is on the horizontal line segment. The technical scheme is to identify the shape characteristics of the obstacle and the position points with representative contour lines, which can effectively access characteristic sampling.
Further, the method for identifying the type of the corresponding obstacle by reselecting the brightness image data matched with the corresponding obstacle and output by the TOF camera in real time at least comprises the following steps: when the obstacle classification module judges that the heights of all position points of the contour line of the corresponding obstacle are within the threshold height range in the preset horizontal width, the corresponding obstacle is identified as a spanable threshold; wherein, all heights in the threshold height range are smaller than the body height of the mobile robot; when the obstacle classification module judges that the heights of the lowest position points of the contour lines of the corresponding obstacles within the preset horizontal width are higher than the body height of the mobile robot, the corresponding obstacles are identified as furniture which can be traversed by the mobile robot; when the obstacle classification module judges that the contour line of the corresponding obstacle meets the rectangular characteristic condition, the depth value corresponding to each pixel point on the corresponding depth image is equal, and the height of the position point with the contour line within the preset horizontal width is higher than the preset passable height threshold value, the corresponding obstacle is identified as a wall; the method comprises the steps that a passable height threshold is preset to be higher than the body height of the mobile robot; when the obstacle classification module judges that the variance of the height of each position point of the contour line of the corresponding obstacle within the preset horizontal width meets the first winding condition and/or judges that the variance of the product of the depth of the pixel point of the depth image corresponding to each position point of the contour line of the corresponding obstacle within the preset horizontal width and the gray level of the pixel point of the matched brightness image meets the second winding condition, the corresponding obstacle is identified as a winding; when the obstacle classification module judges that the horizontal width of the space occupied by the corresponding obstacle is smaller than the preset island width and the vertical height of the space occupied by the corresponding obstacle is smaller than the preset island height, identifying the corresponding obstacle as an island obstacle; wherein the preset horizontal width is larger than the body width of the mobile robot; the preset island height falls into the threshold height range; the preset island width is less than or equal to the preset horizontal width.
Thereby do: TOF camera discernment high less threshold, TOF camera discernment get into at the bottom of sofa (the furniture of the high below machine of sofa bottom) control mobile robot gets into at the bottom of the sofa, TOF camera discernment high less toy, TOF camera discernment wall gallery, TOF camera discernment high not big and possess curve characteristic's electric wire. Therefore, the obstacle avoidance device can effectively detect and identify large obstacles, small obstacles, crossing obstacles and prohibiting contact with the obstacles, and further can plan an effective accessible area for the mobile robot to avoid the obstacle.
Further, when the obstacle classification module judges that the brightness image data matched with the communicable surface area of the corresponding obstacle is in a first preset medium gray threshold range, and/or when the obstacle classification module judges that the product of the depth of each position point of the communicable surface area of the corresponding obstacle corresponding to the pixel point of the depth image and the gray level of the pixel point of the matched brightness image is in a second preset medium gray threshold range, the obstacle classification module determines that the surface medium of the corresponding obstacle is a flat plane medium allowing the mobile robot to move without obstacle. According to the technical scheme, the surface medium for identifying the obstacle is judged by utilizing the brightness information of the reflected light on the surface of the obstacle and the matched pixel point depth data in the depth image, so that a more specific obstacle type can be identified, and a traveling plane allowing the mobile robot to walk across without obstacle can be pre-judged.
Further, the brightness image data matched with the corresponding obstacle is light brightness information reflected back to the imaging plane of the TOF camera from the surface of the corresponding obstacle, and is matched with the depth of the pixel point of the depth image of the corresponding obstacle acquired by the TOF camera in a one-to-one correspondence manner. The judgment effect of the first winding condition and the second winding condition is reduced, and fitting operation is reduced. The accuracy of obstacle type recognition is improved.
Drawings
Fig. 1 is a frame diagram of an obstacle classification obstacle avoidance control system based on a TOF camera according to an embodiment of the present invention.
Detailed Description
The following describes the technical solution in the embodiment of the present application in detail with reference to the drawings in the embodiment of the present application. It should be noted that the present application introduces the whole text of chinese patent CN111624997a into the text of the present application, and completes the description of calculating the relative position information of the obstacle region from the depth image acquired by the TOF camera, and the description of the map calibration marking method.
The embodiment of the invention discloses an obstacle classifying obstacle avoidance control system based on a TOF camera, which is arranged on a mobile robot, and comprises an obstacle classifying module, an obstacle positioning triggering module and an obstacle avoidance module as shown in figure 1; the obstacle classification module comprises a TOF camera arranged in the advancing direction of the mobile robot, the TOF camera can be a 3d-TOF camera, the 3d-TOF camera is arranged in front of the robot in a non-shielding manner, the placement angle and the placement position of the 3d-TOF camera can be adjusted according to actual environments, the obstacle classification module is used for calculating the relative position relation of at least one obstacle and the size of the same obstacle in the detection visual angle range of the TOF camera according to depth images acquired by the TOF camera in real time, and in some implementation scenes, the relative position relation of the obstacle and the size of the same obstacle are calculated by combining the temperature calibration coefficient of a sensor and the related depth calibration coefficient so as to output more accurate depth data, and then the type of the corresponding obstacle is identified by combining the brightness image data matched with the corresponding obstacle output by the TOF camera in real time.
It should be noted that, TOF is an abbreviation of Time of Flight (Time of Flight) technology, that is, a sensor emits modulated near infrared light, and reflects after encountering an object, the sensor converts a distance of a photographed object by calculating a Time difference or a phase difference between light emission and reflection so as to generate depth information, and in addition, by combining with conventional camera photographing, a three-dimensional contour of the object can be presented in a manner of representing topographic images of different distances by different colors, so that a stereoscopic 3D model is obtained, and a TOF camera is a camera for data acquisition by adopting the TOF technology.
The depth image is also called a distance image, and refers to an image in which the distance between each pixel point of the depth image and the actual measurement point of the photographed corresponding obstacle is taken as a pixel value. The offset angle between each pixel point and the corresponding measurement point is determined based on the setting parameters of the image pickup device. The depth image directly reflects the geometric shape of the visible surface of each obstacle in the photographed physical scene, and the depth image can be converted into space point cloud data through coordinate conversion. Each obstacle described by the depth data in the depth image can be used as an image of the obstacle to be identified for processing in a subsequent step. Wherein the obstacle should be taken to broadly include objects temporarily placed on a plane of travel and objects that are not easily moved. Depending on the actual application, the travel plane includes, but is not limited to, cement floors, painted floors, floors for laying composite floors, floors for laying solid wood floors, floors for laying carpets, table tops, glass surfaces, and the like. Examples of the object temporarily placed on the traveling plane include a threshold (which can be crossed), a toy (which can inhibit collision), an electric wire (which can inhibit crossing), and the like; examples of the object which is not easy to move include a sofa (when the height of the sofa bottom is lower than the height of the machine, the machine cannot be controlled to enter), a wall, and the like.
The obstacle positioning triggering module is used for receiving the relative position relation (mainly the three-dimensional coordinate position of each position point on the contour line of the obstacle and the angle information under the global coordinate system) of the obstacle of the type recognized by the obstacle classification module and the size of the same obstacle, and deciding whether the obstacle of the type recognized correspondingly triggers a collision warning signal or not based on the data information, wherein some obstacles allow the mobile robot to cross and pass through without triggering the collision warning signal, and for the obstacles which obstruct the normal walking of the mobile robot, the collision warning signal needs to be triggered before the contact occurs, so that: the mobile robot plans a passable area before moving to the corresponding obstacle.
In addition, the obstacle positioning triggering module receives the relative position relation (mainly the three-dimensional coordinate position of each position point on the contour line of the obstacle and the angle information under the global coordinate system) of the obstacle of the type currently identified by the obstacle classification module and the size of the same obstacle, stores the data in real time and matches the analysis data in real time (carries out simple geometric feature fitting, and outputs a type indication signal representing the shape feature of the obstacle on the premise of meeting the obstacle type identification condition), thereby realizing further correction of the received obstacle type and optimizing the identification result of the obstacle classification module. Then, the obstacle positioning triggering module marks on a map constructed in real time by the mobile robot according to the relative position relation of the obstacle calculated by the obstacle classifying module, positions the relative position of the identified obstacle and the obstacle to be identified (the obstacle to be identified but the relative position relation calculated) in the view angle range of the identified obstacle in the map, and conveniently plans an obstacle avoidance path.
The obstacle avoidance module is used for adjusting the current pose of the mobile robot according to the relative position relation of the obstacle triggering the collision warning signal transmitted by the obstacle positioning triggering module and the size of the same obstacle after the collision warning signal is triggered by the obstacle positioning triggering module, and planning an obstacle avoidance path of the mobile robot according to the relative position relation of the obstacle of the current identified type transmitted by the obstacle positioning triggering module and the size of the same obstacle so as to enable the mobile robot to avoid the obstacle triggering the collision warning signal or bypass the obstacle triggering the collision warning signal. The obstacle avoidance module is used for keeping the original pose to continue walking according to the type of the currently identified obstacle and the size information of the same obstacle transmitted by the obstacle positioning triggering module when the obstacle positioning triggering module does not send out a collision warning signal, so that the mobile robot directly passes through the obstacle and spans the obstacle.
In this embodiment, the obstacle avoidance module plans an obstacle avoidance path closer to the actual environment according to the relative position of the identified obstacle (including the obstacle triggering the collision warning signal) marked in the obstacle positioning triggering module and the relative position of the obstacle to be identified in the detection view angle range of the TOF camera and the size of the same obstacle, so that the mobile robot avoids the obstacle (including the identified obstacle, the obstacle to be identified but the relative position relation calculated) in the view angle range of the TOF camera in time, or walks around the corresponding obstacle according to the type of the identified obstacle, or walks across the corresponding obstacle according to the type of the identified obstacle. At least in the implementation process, the method comprises the following steps: the obstacle avoidance module decides and adjusts the current pose of the mobile robot according to the current motion state (normal straight line walking, in-situ turning, radian turning and edge) of the mobile robot and the type of the obstacle, so that the mobile robot can linearly surmount the obstacle before crossing the obstacle, or the current pose of the mobile robot is adjusted to enable the mobile robot to walk around the obstacle or linearly avoid the obstacle before the small obstacle (including small winding) but not touch the obstacle, or the current pose of the mobile robot is adjusted to enable the mobile robot to avoid the obstacle along the edge when the mobile robot approaches the wall. This of course also has a relation to the shape features of the identified obstacle, which are based on geometric shapes, geometric combinations, etc. composed or abstracted by contour lines and/or feature points, for matching the respective obstacle type. Wherein the geometry, geometry combination may be based on the entire contour or partial representation of the contour of the identified obstacle. For example, the shape features provided based on island type include one or more of a circle, sphere, arc, square, cube, pi, etc. in combination. For example, the shape features of the shoe include a plurality of arc shapes connected end to end, the shape features of the chair including pi-shape, eight-claw shape, etc. The shape features provided based on the type of wrap include at least one or more of a curved shape, a serpentine shape, a coiled shape, and the like. The shape features provided based on the type of spatial separation include at least one or more combinations of straight line shapes, broken line shapes, rectangles, and the like.
In summary, the embodiment of the invention effectively classifies and identifies whether the obstacle should be spanned or not and whether the obstacle should collide or not in the indoor environment by adopting the obstacle depth information, the size information and the brightness information acquired by the TOF camera, and timely triggers a collision warning signal according to the type characteristics, the relative position and the size information of the identified obstacle so as to push the mobile robot to plan a passable area before moving to the corresponding obstacle. Compared with the prior art, the system adopts a plurality of cameras or multi-line laser heads and performs excessive image feature point fitting classification training, the production cost of the system disclosed by the embodiment is reduced, and the real-time performance of obstacle avoidance actions of the robot is improved.
Preferably, a camera or a single-wire laser head is arranged on the mobile robot top carrier of the embodiment of the invention and is used for correcting the map after the robot map slips; the mobile robot is also internally provided with a gyroscope for detecting a rotation angle, an odometer for detecting a travel distance, and a sensor capable of detecting a wall surface distance, wherein the sensor for detecting the wall surface distance can be an ultrasonic distance sensor, an infrared intensity detection sensor, an infrared distance sensor, a physical switch detection collision sensor, a capacitance or resistance change detection sensor and the like.
As an embodiment, in the obstacle locating triggering module, the method for deciding whether the obstacle of the type identified correspondingly triggers the collision warning signal based on the data information at least includes: defining the identified type of obstacle as a target obstacle; types of obstacles include windings, island obstacles, straddlable thresholds, and furniture that can be traversed by a mobile robot, with pixel depth data of depth images of portions of the obstacles being continuous (walls), but with portions of the pixel depth data being discontinuous (windings).
When the type of the target obstacle is a winding object, if the depth data of the depth image of the target obstacle shot by the TOF camera is smaller than a first safety depth threshold, controlling the obstacle positioning triggering module to trigger a collision warning signal; when the type of the target obstacle is an island obstacle, if the depth data of the depth image of the target obstacle shot by the TOF camera is smaller than a second safety depth threshold, controlling the obstacle positioning triggering module to trigger a collision warning signal; when the type of the target obstacle is a wall body, if the depth data of the depth image of the target obstacle shot by the TOF camera is smaller than a third safety depth threshold, controlling the obstacle positioning triggering module to trigger a collision warning signal, and allowing the mobile robot to collide with the wall body in the process of continuously controlling the mobile robot to walk along the wall; when the type of the target obstacle is a straddlable threshold or furniture which can be traversed by the mobile robot, the obstacle positioning triggering module is controlled not to trigger a collision warning signal, because the threshold or the traversed furniture which is controlled by the mobile robot is an optimal planning mode so as to minimize the influence on the working traversing mode of the mobile robot, although the mobile robot may walk around a supporting part at the bottom of the furniture in the process; the first safety depth threshold is larger than the second safety depth threshold, the two safety depth thresholds are used for limiting the mobile robot not to touch the target obstacle in the process of decelerating to zero, the second safety depth threshold is larger than the third safety depth threshold, and the third safety depth threshold can limit the mobile robot to collide with the obstacle which is characterized as a wall body before decelerating to zero. Before approaching an identified target obstacle in a visual angle range, the method meets the requirements of the matched type obstacle on collision obstacle avoidance by setting different safety distances so as to pre-judge an obstacle-free passable area and facilitate the subsequent planning of an effective obstacle avoidance path.
As an embodiment, in the obstacle avoidance module, the method for adjusting the current pose of the mobile robot according to the relative position relationship of the obstacle triggering the collision warning signal and the size of the same obstacle transmitted by the obstacle positioning triggering module at least includes: when the type of the target obstacle is a winding object and the obstacle positioning triggering module triggers the collision warning signal, according to the outline width of the target obstacle, the mobile robot is controlled to reduce the real-time speed and change the current advancing direction, so that the changed straight line path in the advancing direction avoids the target obstacle, the mobile robot is facilitated to directly avoid the obstacle, and the mobile robot is not required to walk around the winding object and is easy to be blocked under the condition of misdetecting the relative position of the winding object. When the type of the target obstacle is a threshold which can be spanned or furniture which can be traversed by the mobile robot, the mobile robot is controlled to keep the current advancing direction, namely the collision warning signal is not required to be triggered, and the obstacle avoidance action is not required to be executed, so that the influence degree of the target obstacle on the normal working behavior of the mobile robot is reduced to the minimum. When the type of the target obstacle is an island obstacle and the obstacle positioning triggering module triggers the collision warning signal, the mobile robot is controlled to change the current advancing direction so that the mobile robot starts to walk around the outline of the island obstacle, but the mobile robot cannot collide with the island obstacle. When the type of the target obstacle is a wall body and the obstacle positioning triggering module triggers the collision warning signal, the mobile robot is controlled to adjust the optimal edge direction so that the mobile robot starts to enter an edge mode, but the mobile robot can collide with the wall body in the process of edge walking. The embodiment is beneficial to optimizing the current pose of the mobile robot, so that the current motion state (including normal straight walking, in-situ turning, radian turning and edge turning) of the mobile robot is optimized, and the probability of contact of the mobile robot with the obstacle of the identified type is reduced.
Based on the above embodiment, in the obstacle avoidance module, the method for planning the obstacle avoidance path of the mobile robot according to the relative position relationship of the currently identified type of obstacle and the size of the same obstacle transmitted by the obstacle positioning triggering module, so that the mobile robot avoids the obstacle triggering the collision warning signal, bypasses the obstacle triggering the collision warning signal, or keeps the original pose to walk continuously at least includes: firstly, planning a passable area between the mobile robot and the obstacles according to the relative position relation of the obstacles of the type which are currently identified and the size of the same obstacle, and then planning an obstacle avoidance path of the mobile robot in the passable area; when the outline of the identified obstacle is wider, the area of the planned passable area is larger, the obstacle avoidance path is longer, and otherwise, the area of the planned passable area is smaller; the currently recognized type of obstacle transmitted by the obstacle positioning triggering module comprises an obstacle triggering collision warning signals and other target obstacles in the detection view angle range of the TOF camera.
When the type of the target obstacle is a winding object and the obstacle positioning triggering module triggers the collision warning signal, planning an initial obstacle avoidance direction of the mobile robot according to the outline width of the target obstacle, and simultaneously adjusting the initial obstacle avoidance direction of the mobile robot in real time according to position coordinates corresponding to all points on the outline of the identified winding object and other position coordinates of the target obstacle so as to avoid touching the identified winding object, and keeping and adjusting depth data corresponding to all points on the outline of the winding object acquired in real time to be larger than a first safety depth threshold; when the type of the target obstacle is a spanable threshold or furniture which can be traversed by the mobile robot, controlling the mobile robot to walk linearly along the current advancing direction so as to span or traverse the target obstacle; when the type of the target obstacle is an island obstacle and the obstacle positioning triggering module triggers the collision warning signal, controlling the mobile robot to walk along the extending direction of the outline of the island obstacle, avoiding other obstacles with recognized or marked coordinate positions, and keeping the depth data corresponding to each point on the outline of the island obstacle which is acquired and recognized in real time to be adjusted to be larger than a second safety depth threshold so as to realize the obstacle-surrounding walking of the mobile robot but not to collide with the island obstacle; when the type of the target obstacle is a wall body and the obstacle positioning triggering module triggers the collision warning signal, controlling the mobile robot to adjust the optimal edge direction, and starting to execute the walking along the wall along the optimal edge direction. Thereby do: the TOF camera can control the mobile robot to stride across the threshold when recognizing the threshold, the TOF camera can control the mobile robot to enter the sofa bottom when recognizing the sofa (when the height of the sofa bottom is lower than that of the machine), the TOF camera can prohibit the mobile robot from colliding with the toy and walking around the obstacle when recognizing the toy, the TOF camera can allow the mobile robot to collide with the wall body to walk along the edge when recognizing the wall, and prohibit the mobile robot from crossing wound objects such as wires and avoiding the obstacle in other paths when recognizing the wire, so as to plan an effective obstacle avoidance path.
Preferably, the TOF camera is mounted in front of the mobile robot, and an optical axis of the TOF camera is inclined downward or horizontally arranged relative to a top surface of the robot, so that a brightness value of the target obstacle in a detection view angle range is effective and an outline of the target obstacle is complete to meet a depth positioning requirement. Comprehensively considering the distance between the vertical height of the robot and the distance to be detected by the TOF camera, the optical axis of the TOF camera is obliquely arranged below the top surface of the mobile robot, the inclination angle of the TOF camera is 10 degrees, and meanwhile, the installation height of the TOF camera is 6.7cm relative to the advancing plane of the mobile robot, so that the TOF camera is the optimal installation height and the installation angle obtained through testing.
In the foregoing embodiment, the obstacle classification module and the obstacle avoidance module are respectively connected with the obstacle positioning triggering module through a bidirectional communication interface, so as to establish a communication relationship of transceiving responses between signal data. The real-time performance of the depth image information feedback of each obstacle is guaranteed, and the efficiency of obstacle avoidance path planning is improved. As shown in fig. 1, the information transmitted by the obstacle classification module to the obstacle positioning triggering module includes the type of the obstacle identified by the TOF camera acquisition, the relative position and the size of the obstacle; the information transmitted by the obstacle positioning triggering module to the obstacle avoidance module comprises collision warning signals, the type of the obstacle identified by the TOF camera, the relative position of the obstacle and the size of the obstacle.
As an embodiment, the size of the same obstacle includes the height of each position point of the contour line of the corresponding obstacle within the preset horizontal width; the preset horizontal width is the length of a horizontal line segment in a preselected communicable area corresponding to the surface of the obstacle, the horizontal projection of each position point of the contour line of the corresponding obstacle in the preset horizontal width is on the horizontal line segment, and the selected contour line is not necessarily complete but can cover the upper or lower part of the horizontal line segment. The present embodiment is to identify the shape features of an obstacle and the location points of representative contour lines where accessible feature sampling is effective.
As an embodiment, the method for identifying the type of the corresponding obstacle by reselecting the luminance image data matched with the corresponding obstacle and output by the TOF camera in real time at least includes:
When the obstacle classification module judges that the heights of all position points of the contour line of the corresponding obstacle are within the threshold height range in the preset horizontal width, the corresponding obstacle is identified as a spanable threshold; all heights in the threshold height range are smaller than the body height of the mobile robot, and all heights in the threshold height range can support the mobile robot to climb.
When the obstacle classification module judges that the heights of the lowest position points of the contour lines of the corresponding obstacles within the preset horizontal width are higher than the body height of the mobile robot, the corresponding obstacles are identified as furniture which can be traversed by the mobile robot, the contour lines of the obstacles are generally in the shape of a door frame, a trapezoid, possibly a sofa bottom, a bed bottom and a table and chair bottom, the gap parts of the bottoms of the furniture are higher and wider, so that the heights of the gap parts are larger than the body height of the mobile robot, and the widths of the gap parts are also larger than the body diameter of the mobile robot.
When the obstacle classification module judges that the contour line of the corresponding obstacle meets the rectangular characteristic condition, the depth value corresponding to each pixel point on the corresponding depth image is equal, and the height of the position point with the contour line within the preset horizontal width is higher than the preset passable height threshold value, the corresponding obstacle is identified as a wall; because the surface of the wall body is flat, the shape of the contour line of the wall body accords with rectangular characteristics, the contour line is longer, the height of the wall body is higher than that of the body of the mobile robot, meanwhile, the obstacle identified as the wall body also supports the collision of the mobile robot, and even if the obstacle is misjudged as the hollow part at the bottom of the furniture, the identification result of the type of the obstacle can be corrected after the mobile robot collides with the obstacle; the preset passable height threshold is higher than the body height of the mobile robot.
When the obstacle classification module judges that the variance of the height of each position point of the contour line of the corresponding obstacle within the preset horizontal width meets the first winding condition and/or judges that the variance of the product of the depth of the pixel point of the depth image corresponding to each position point of the contour line of the corresponding obstacle within the preset horizontal width and the gray level of the pixel point of the matched brightness image meets the second winding condition, the corresponding obstacle is identified as a winding; the variance in the winding judgment method can be replaced by variables with statistical significance such as standard deviation, mean square deviation and the like, namely, the statistical variables corresponding to the heights of sampling position points on the contour line of the obstacle, and curve change characteristics of windings (such as wires and cables wound into a group in indoor environment) can be effectively reflected. When the variance of the height falls within a first preset range, the first winding condition is considered to be satisfied; and when the variance of the product falls within a second preset range, the second winding condition is considered to be satisfied. It should be noted that the heights of the windings are smaller, generally smaller than the body height of the mobile robot, and may fall within the threshold height range, but in combination with the recognition result of the winding condition, the mobile robot is controlled not to cross the winding, so as to avoid erroneous judgment.
When the obstacle classification module judges that the horizontal width of the space occupied by the corresponding obstacle is smaller than the preset island width and the vertical height of the space occupied by the corresponding obstacle is smaller than the preset island height, identifying the corresponding obstacle as an island obstacle; the preset horizontal width is larger than the body width of the mobile robot, and the preset island width is smaller than or equal to the preset horizontal width. In some implementation scenarios, the preset island height is smaller than the body height of the mobile robot, and the preset island height even falls within the threshold height range, so that the mobile robot can be effectively avoided from crossing the island obstacle by identifying the obstacle with lower height as the island obstacle rather than misjudging as the threshold, such as controlling the mobile robot not to collide and cross the small toy.
Thereby do: TOF camera discernment high less threshold, TOF camera discernment get into at the bottom of sofa (the furniture of the high below machine of sofa bottom) control mobile robot gets into at the bottom of the sofa, TOF camera discernment high less toy, TOF camera discernment wall gallery, TOF camera discernment high not big and possess curve characteristic's electric wire. Therefore, the obstacle avoidance device can effectively detect and identify large obstacles, small obstacles, crossing obstacles and prohibiting contact with the obstacles, and further can pre-judge an effective accessible area for the mobile robot to avoid the obstacle. Further improving the functional integration level of the recognition obstacle avoidance algorithm of the obstacle classification obstacle avoidance control system.
In the above embodiment, when the obstacle classification module determines that the luminance image data matched with the communicable surface area of the corresponding obstacle is within the first preset medium gray-level threshold range, and/or when the obstacle classification module determines that the product of the depths of the pixels of the depth image corresponding to the respective position points of the communicable surface area of the corresponding obstacle and the pixel gray-level of the matched luminance image is within the second preset medium gray-level threshold range, the obstacle classification module determines that the surface medium corresponding to the obstacle is a flat plane medium allowing the mobile robot to move without obstacle. According to the technical scheme, the surface medium for identifying the obstacle is judged by utilizing the brightness information of the reflected light on the surface of the obstacle and the matched pixel point depth data in the depth image, so that more specific obstacle types (such as a carpet surface, a cement floor and a wood board) can be identified, and a traveling plane allowing the mobile robot to walk across without obstacle can be predicted.
The characteristic of the depth image collected by the TOF camera indicates that the brightness image data matched with the corresponding obstacle is light brightness information reflected back to the imaging plane of the TOF camera from the surface of the corresponding obstacle, and the brightness image data is matched with the depth of the pixel point of the depth image of the corresponding obstacle collected by the TOF camera in a one-to-one correspondence manner. And performing obstacle classification judgment by utilizing the matching relation between the characteristics of the depth image and the characteristics of the brightness image, so that the judgment effect of the first winding condition and the second winding condition is reduced, and fitting operation is reduced. The accuracy of obstacle type recognition is improved.
It is apparent that the above examples are given by way of illustration only and are not limiting of the embodiments. Other variations or modifications of the above teachings will be apparent to those of ordinary skill in the art. It is not necessary here nor is it exhaustive of all embodiments. While still being apparent from variations or modifications that may be made by those skilled in the art are within the scope of the invention.
Claims (10)
1. The obstacle classifying obstacle avoidance control system based on the TOF camera is characterized in that the obstacle classifying obstacle avoidance control system is arranged on a mobile robot and comprises an obstacle classifying module, an obstacle positioning triggering module and an obstacle avoidance module;
The obstacle classification module comprises a TOF camera arranged in the advancing direction of the mobile robot, and is used for calculating the relative position relation of at least one obstacle and the size of the same obstacle in the detection visual angle range of the TOF camera according to the depth image acquired by the TOF camera in real time, and then selecting and combining the brightness image data which is output by the TOF camera in real time and is matched with the corresponding obstacle to identify the type of the corresponding obstacle;
The obstacle positioning triggering module is used for receiving the relative position relation of the obstacle of the type currently identified by the obstacle classification module and the size of the same obstacle, and deciding whether the obstacle of the type correspondingly identified triggers a collision warning signal or not based on the data information so as to enable the obstacle to be identified: the mobile robot plans a passable area before moving to a corresponding obstacle;
The obstacle avoidance module is used for adjusting the current pose of the mobile robot according to the relative position relation of the obstacle triggering the collision warning signal transmitted by the obstacle positioning triggering module and the size of the same obstacle, and planning an obstacle avoidance path of the mobile robot according to the relative position relation of the obstacle of the current identified type transmitted by the obstacle positioning triggering module and the size of the same obstacle, so that the mobile robot avoids the obstacle triggering the collision warning signal, bypasses the obstacle triggering the collision warning signal, or keeps the original pose to walk continuously.
2. The obstacle avoidance control system of claim 1 wherein in the obstacle locating trigger module the method of deciding whether the respective identified type of obstacle triggers a collision warning signal based on the data information comprises at least:
defining the identified type of obstacle as a target obstacle; types of obstacles include winds, island obstacles, straddlable sills, and furniture that can be traversed by a mobile robot;
When the type of the target obstacle is a winding object, if the depth data of the depth image of the target obstacle shot by the TOF camera is smaller than a first safety depth threshold, controlling the obstacle positioning triggering module to trigger a collision warning signal;
when the type of the target obstacle is an island obstacle, if the depth data of the depth image of the target obstacle shot by the TOF camera is smaller than a second safety depth threshold, controlling the obstacle positioning triggering module to trigger a collision warning signal;
when the type of the target obstacle is a wall body, if the depth data of the depth image of the target obstacle shot by the TOF camera is smaller than a third safety depth threshold, controlling the obstacle positioning triggering module to trigger a collision warning signal;
When the type of the target obstacle is a threshold which can be crossed or furniture which can be crossed by the mobile robot, controlling the obstacle positioning triggering module not to trigger a collision warning signal;
the first safety depth threshold is larger than the second safety depth threshold, and the second safety depth threshold is larger than the third safety depth threshold.
3. The obstacle avoidance control system according to claim 2, wherein in the obstacle avoidance module, the method for adjusting the current pose of the mobile robot according to the relative positional relationship of the obstacles and the size of the same obstacle, which trigger the collision warning signal transmitted from the obstacle locating and triggering module, at least comprises:
When the type of the target obstacle is a winding object and the obstacle positioning triggering module triggers the collision warning signal, controlling a mobile robot to reduce the real-time speed and change the current advancing direction according to the outline width of the target obstacle so as to enable a straight line path in the advancing direction after the change to avoid the target obstacle;
When the type of the target obstacle is a spanable threshold or furniture which can be traversed by the mobile robot, controlling the mobile robot to keep the current advancing direction;
when the type of the target obstacle is an island obstacle and the obstacle positioning triggering module triggers the collision warning signal, controlling the mobile robot to change the current advancing direction so as to enable the mobile robot to start to walk around the outline of the island obstacle;
When the type of the target obstacle is a wall body and the obstacle positioning triggering module triggers the collision warning signal, controlling the mobile robot to adjust the optimal edge direction so as to enable the mobile robot to start to enter an edge mode.
4. The obstacle avoidance control system according to claim 3, wherein in the obstacle avoidance module, the method for planning the obstacle avoidance path of the mobile robot according to the relative positional relationship of the currently identified type of obstacle and the size of the same obstacle transmitted from the obstacle positioning triggering module at least comprises:
Firstly, planning a passable area between the mobile robot and the obstacles according to the relative position relation of the obstacles of the type which are currently identified and the size of the same obstacle, and then planning an obstacle avoidance path of the mobile robot in the passable area; the obstacle positioning triggering module is used for transmitting a current recognized type of obstacle to the TOF camera, wherein the current recognized type of obstacle transmitted by the obstacle positioning triggering module comprises a target obstacle triggering collision warning signals and other target obstacles within a detection visual angle range of the TOF camera;
When the type of the target obstacle is a winding object and the obstacle positioning triggering module triggers the collision warning signal, planning an initial obstacle avoidance direction of the mobile robot according to the outline width of the target obstacle, and simultaneously adjusting the initial obstacle avoidance direction of the mobile robot in real time according to position coordinates corresponding to all points on the outline of the identified winding object and other position coordinates of the target obstacle so as to avoid touching the identified winding object, and keeping and adjusting depth data corresponding to all points on the outline of the winding object acquired in real time to be larger than a first safety depth threshold;
When the type of the target obstacle is a spanable threshold or furniture which can be traversed by the mobile robot, controlling the mobile robot to walk linearly along the current advancing direction so as to span or traverse the target obstacle;
When the type of the target obstacle is an island obstacle and the obstacle positioning triggering module triggers the collision warning signal, controlling the mobile robot to walk along the extending direction of the outline of the island obstacle, and keeping adjusting depth data corresponding to each point on the outline line of the island obstacle which is acquired and identified in real time to be larger than a second safety depth threshold so as to realize obstacle-surrounding walking of the mobile robot;
when the type of the target obstacle is a wall body and the obstacle positioning triggering module triggers the collision warning signal, controlling the mobile robot to adjust the optimal edge direction, and starting to execute the walking along the wall along the optimal edge direction.
5. The obstacle classifying obstacle avoidance control system of claim 4 wherein the TOF camera is mounted in front of the mobile robot, the TOF camera being mounted at a height of 6.7 cm relative to the plane of travel, the optical axis of the TOF camera being at an oblique angle of 10 degrees relative to the top surface of the robot, wherein the drive wheel of the bottom of the mobile robot is in contact with the plane of travel.
6. The obstacle classifying obstacle avoidance control system of claim 5 wherein the obstacle classifying module and the obstacle avoidance module are each coupled to the obstacle locating trigger module via a bi-directional communication interface to establish a communication relationship of the transception responses between the signal data.
7. The obstacle-classifying obstacle avoidance control system of any of claims 2 to 6, wherein the size of the same obstacle comprises the height of each location point of the outline of the corresponding obstacle within a predetermined horizontal width;
The preset horizontal width is the length of a horizontal line segment in a preselected communicable domain corresponding to the surface of the obstacle, and the horizontal projection of each position point of the contour line of the corresponding obstacle in the preset horizontal width is on the horizontal line segment.
8. The obstacle-classifying obstacle avoidance control system of claim 7 wherein the method of identifying the type of obstacle in combination with the real-time output of the TOF camera from the luminance image data matching the corresponding obstacle comprises at least:
When the obstacle classification module judges that the heights of all position points of the contour line of the corresponding obstacle are within the threshold height range in the preset horizontal width, the corresponding obstacle is identified as a spanable threshold; wherein, all heights in the threshold height range are smaller than the body height of the mobile robot;
when the obstacle classification module judges that the heights of the lowest position points of the contour lines of the corresponding obstacles within the preset horizontal width are higher than the body height of the mobile robot, the corresponding obstacles are identified as furniture which can be traversed by the mobile robot;
When the obstacle classification module judges that the contour line of the corresponding obstacle meets the rectangular characteristic condition, the depth value corresponding to each pixel point on the corresponding depth image is equal, and the height of the position point with the contour line within the preset horizontal width is higher than the preset passable height threshold value, the corresponding obstacle is identified as a wall; the method comprises the steps that a passable height threshold is preset to be higher than the body height of the mobile robot;
When the obstacle classification module judges that the variance of the height of each position point of the contour line of the corresponding obstacle within the preset horizontal width meets the first winding condition and/or judges that the variance of the product of the depth of the pixel point of the depth image corresponding to each position point of the contour line of the corresponding obstacle within the preset horizontal width and the gray level of the pixel point of the matched brightness image meets the second winding condition, the corresponding obstacle is identified as a winding;
When the obstacle classification module judges that the horizontal width of the space occupied by the corresponding obstacle is smaller than the preset island width and the vertical height of the space occupied by the corresponding obstacle is smaller than the preset island height, identifying the corresponding obstacle as an island obstacle;
Wherein the preset horizontal width is larger than the body width of the mobile robot; the preset island height falls into the threshold height range, and the preset island width is smaller than or equal to the preset horizontal width.
9. The obstacle classification obstacle avoidance control system of claim 8, wherein the obstacle classification module determines the surface medium of the corresponding obstacle as a planar medium that allows the mobile robot to move without obstacle when the obstacle classification module determines that the luminance image data of the corresponding obstacle for a communicable surface area is within a first predetermined medium gray level threshold range, and/or when the obstacle classification module determines that the product of the depths of the pixels of the depth image corresponding to the respective location points of the communicable surface area of the corresponding obstacle and the pixel gray level of the matched luminance image is within a second predetermined medium gray level threshold range.
10. The obstacle-classifying obstacle avoidance control system of claim 9 wherein the luminance image data matching the corresponding obstacle is light luminance information reflected back from the surface of the corresponding obstacle to the imaging plane of the TOF camera, and is matched in one-to-one correspondence to the depths of the pixels of the depth image of the corresponding obstacle acquired by the TOF camera.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011159235.7A CN112415998B (en) | 2020-10-26 | 2020-10-26 | Obstacle classification obstacle avoidance control system based on TOF camera |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011159235.7A CN112415998B (en) | 2020-10-26 | 2020-10-26 | Obstacle classification obstacle avoidance control system based on TOF camera |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112415998A CN112415998A (en) | 2021-02-26 |
| CN112415998B true CN112415998B (en) | 2024-06-18 |
Family
ID=74840561
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011159235.7A Active CN112415998B (en) | 2020-10-26 | 2020-10-26 | Obstacle classification obstacle avoidance control system based on TOF camera |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112415998B (en) |
Families Citing this family (17)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115202330A (en) * | 2021-04-09 | 2022-10-18 | 美智纵横科技有限责任公司 | Control method for cleaning robot to move along obstacle and cleaning robot |
| CN113232585B (en) * | 2021-05-07 | 2023-02-10 | 广州小鹏汽车科技有限公司 | Anti-collision method and device for vehicle door opening, vehicle and storage medium |
| CN113311837A (en) * | 2021-05-26 | 2021-08-27 | 上海十方生态园林股份有限公司 | Unmanned ship obstacle avoidance system based on overwater and underwater environment recognition |
| CN113341978B (en) * | 2021-06-10 | 2023-03-07 | 西北工业大学 | Intelligent trolley path planning method based on ladder-shaped barrier |
| CN113475978B (en) * | 2021-06-23 | 2023-01-31 | 深圳乐动机器人股份有限公司 | Robot recognition control method and device, robot and storage medium |
| CN113625704A (en) * | 2021-06-30 | 2021-11-09 | 北京旷视科技有限公司 | Obstacle avoidance method and device and automatic navigation device |
| CN113741467B (en) * | 2021-09-07 | 2023-10-13 | 深圳大方智能科技有限公司 | Vertical wall surface construction method and construction robot |
| CN113558536B (en) * | 2021-09-24 | 2022-01-25 | 莱克电气绿能科技(苏州)有限公司 | Intelligent calibration method, device and system of intelligent sweeping robot |
| CN114047753B (en) * | 2021-11-03 | 2023-02-03 | 哈尔滨鹏路智能科技有限公司 | Obstacle recognition and obstacle avoidance method of sweeping robot based on deep vision |
| CN114077252B (en) * | 2021-11-16 | 2023-09-12 | 中国人民解放军陆军工程大学 | Robot collision obstacle distinguishing device and method |
| CN114089756B (en) * | 2021-11-16 | 2024-04-30 | 合肥瓦力觉启机器人科技有限公司 | Robot obstacle avoidance method and system |
| CN116416518A (en) * | 2021-12-22 | 2023-07-11 | 广东栗子科技有限公司 | Intelligent obstacle avoidance method and device |
| CN114185356B (en) * | 2022-02-16 | 2022-05-31 | 莱陆科技(天津)有限公司 | Method and device for intelligently identifying obstacles and controlling robot to pass through obstacles |
| CN115525047B (en) * | 2022-03-21 | 2023-07-25 | 江苏集萃清联智控科技有限公司 | Vehicle local track planning method and system with multi-type obstacle avoidance mode |
| CN114895688B (en) * | 2022-05-31 | 2023-06-20 | 合肥新思路智能科技有限公司 | Mobile robot line planning system and method based on artificial intelligence |
| CN114831356B (en) * | 2022-07-05 | 2022-11-01 | 深圳市恒尔创科技有限公司 | Electronic cigarette control method based on mobile terminal and related equipment |
| CN117249832A (en) * | 2023-10-25 | 2023-12-19 | 深圳市天鹤科技有限公司 | Method, device and storage medium for robot indoor navigation |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110622085A (en) * | 2019-08-14 | 2019-12-27 | 珊口（深圳）智能科技有限公司 | Mobile robot and control method and control system thereof |
Family Cites Families (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| DE102016011378A1 (en) * | 2016-09-21 | 2017-04-13 | Daimler Ag | Method for self-localization of a vehicle |
| CN107544501A (en) * | 2017-09-22 | 2018-01-05 | 广东科学技术职业学院 | A kind of intelligent robot wisdom traveling control system and its method |
| CN116578068A (en) * | 2018-06-15 | 2023-08-11 | 科沃斯机器人股份有限公司 | Robot positioning method, robot, and storage medium |
| CN109284348B (en) * | 2018-10-30 | 2021-02-05 | 百度在线网络技术（北京）有限公司 | Electronic map updating method, device, equipment and storage medium |
| CN110727270A (en) * | 2019-10-24 | 2020-01-24 | 常州格力博有限公司 | Automatic working system and method for establishing working area control map thereof |
| CN111624997A (en) * | 2020-05-12 | 2020-09-04 | 珠海市一微半导体有限公司 | Robot control method and system based on TOF camera module and robot |
-
2020
- 2020-10-26 CN CN202011159235.7A patent/CN112415998B/en active Active
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110622085A (en) * | 2019-08-14 | 2019-12-27 | 珊口（深圳）智能科技有限公司 | Mobile robot and control method and control system thereof |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112415998A (en) | 2021-02-26 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112415998B (en) | Obstacle classification obstacle avoidance control system based on TOF camera | |
| CN112327878B (en) | Obstacle classification and obstacle avoidance control method based on TOF camera | |
| CN111067439B (en) | Obstacle processing method and cleaning robot | |
| US9836653B2 (en) | Systems and methods for capturing images and annotating the captured images with information | |
| CN112004645A (en) | Intelligent cleaning robot | |
| US10620636B2 (en) | Method of identifying functional region in 3-dimensional space, and robot implementing the method | |
| US20200409382A1 (en) | Intelligent cleaning robot | |
| CN107041718B (en) | Cleaning robot and control method thereof | |
| CN110147106A (en) | Has the intelligent Mobile Service robot of laser and vision fusion obstacle avoidance system | |
| CN106037591B (en) | Limiting motion of a mobile robot | |
| CN112363513B (en) | Obstacle classification obstacle avoidance control method based on depth information | |
| CN106227212B (en) | The controllable indoor navigation system of precision and method based on grating map and dynamic calibration | |
| US8209074B2 (en) | Robot and method for controlling the same | |
| KR101395089B1 (en) | System and method for detecting obstacle applying to vehicle | |
| EP3104194B1 (en) | Robot positioning system | |
| CN110622085A (en) | Mobile robot and control method and control system thereof | |
| KR100901311B1 (en) | Autonomous mobile platform | |
| CN110275538A (en) | Intelligent cruise vehicle navigation methods and systems | |
| CN111624997A (en) | Robot control method and system based on TOF camera module and robot | |
| CN112347876B (en) | Obstacle recognition method based on TOF camera and cleaning robot | |
| CN216454808U (en) | Docking station for receiving and charging a mobile cleaning robot | |
| CN112327879A (en) | Edge obstacle avoidance method based on depth information | |
| WO2014173290A1 (en) | Automatic walking device and method for determining working area thereof | |
| US11400593B2 (en) | Method of avoiding collision, robot and server implementing thereof | |
| WO2020038155A1 (en) | Autonomous movement device, control method and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information | ||
| CB02 | Change of applicant information |
Address after: 519000 2706, No. 3000, Huandao East Road, Hengqin new area, Zhuhai, Guangdong Applicant after: Zhuhai Yiwei Semiconductor Co.,Ltd. Address before: Room 105-514, No.6 Baohua Road, Hengqin New District, Zhuhai City, Guangdong Province Applicant before: AMICRO SEMICONDUCTOR Co.,Ltd. |
|
| GR01 | Patent grant |