CN114815810A - Unmanned aerial vehicle-cooperated overwater cleaning robot path planning method and equipment - Google Patents
Unmanned aerial vehicle-cooperated overwater cleaning robot path planning method and equipment Download PDFInfo
- Publication number
- CN114815810A CN114815810A CN202210286539.2A CN202210286539A CN114815810A CN 114815810 A CN114815810 A CN 114815810A CN 202210286539 A CN202210286539 A CN 202210286539A CN 114815810 A CN114815810 A CN 114815810A
- Authority
- CN
- China
- Prior art keywords
- cleaning robot
- aerial vehicle
- unmanned aerial
- path
- network
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000004140 cleaning Methods 0.000 title claims abstract description 128
- 238000000034 method Methods 0.000 title claims abstract description 81
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 claims abstract description 95
- 238000013461 design Methods 0.000 claims abstract description 28
- 238000013527 convolutional neural network Methods 0.000 claims abstract description 23
- 239000003344 environmental pollutant Substances 0.000 claims abstract description 23
- 231100000719 pollutant Toxicity 0.000 claims abstract description 23
- 239000011159 matrix material Substances 0.000 claims abstract description 17
- 230000011218 segmentation Effects 0.000 claims abstract description 15
- 238000004064 recycling Methods 0.000 claims abstract description 8
- 230000006870 function Effects 0.000 claims description 20
- 238000004891 communication Methods 0.000 claims description 13
- 230000002068 genetic effect Effects 0.000 claims description 13
- 238000000605 extraction Methods 0.000 claims description 12
- 238000001514 detection method Methods 0.000 claims description 10
- 230000000007 visual effect Effects 0.000 claims description 8
- 238000003860 storage Methods 0.000 claims description 7
- 230000002708 enhancing effect Effects 0.000 claims description 5
- 230000004927 fusion Effects 0.000 claims description 5
- 238000011084 recovery Methods 0.000 claims description 5
- 238000005070 sampling Methods 0.000 claims description 5
- 238000011160 research Methods 0.000 claims description 4
- 238000000638 solvent extraction Methods 0.000 claims 1
- 230000008569 process Effects 0.000 abstract description 16
- 238000005265 energy consumption Methods 0.000 abstract description 4
- 238000012549 training Methods 0.000 description 9
- 238000010586 diagram Methods 0.000 description 7
- 230000000694 effects Effects 0.000 description 3
- 238000003709 image segmentation Methods 0.000 description 3
- 238000013507 mapping Methods 0.000 description 3
- 210000004027 cell Anatomy 0.000 description 2
- 210000000349 chromosome Anatomy 0.000 description 2
- 238000011156 evaluation Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 230000008447 perception Effects 0.000 description 2
- 238000011176 pooling Methods 0.000 description 2
- 238000012545 processing Methods 0.000 description 2
- 238000004088 simulation Methods 0.000 description 2
- 241000192710 Microcystis aeruginosa Species 0.000 description 1
- 230000003044 adaptive effect Effects 0.000 description 1
- 230000004888 barrier function Effects 0.000 description 1
- 238000004590 computer program Methods 0.000 description 1
- 230000008094 contradictory effect Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 238000002372 labelling Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000035772 mutation Effects 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 238000002360 preparation method Methods 0.000 description 1
- 108090000623 proteins and genes Proteins 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
- 230000007704 transition Effects 0.000 description 1
- 239000003643 water by type Substances 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
- G05D1/02—Control of position or course in two dimensions
- G05D1/0206—Control of position or course in two dimensions specially adapted to water vehicles
Landscapes
- Engineering & Computer Science (AREA)
- Aviation & Aerospace Engineering (AREA)
- Radar, Positioning & Navigation (AREA)
- Remote Sensing (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Automation & Control Theory (AREA)
- Image Analysis (AREA)
Abstract
The invention provides a method and equipment for planning paths of an unmanned aerial vehicle-cooperated water cleaning robot, which comprises the following steps: the unmanned aerial vehicle firstly cruises over a river channel, records a moving path, adopts a U-Net semantic segmentation network to realize the division of the water surface and the road surface, and marks the division; after the convolutional neural network is adopted to identify the garbage pollutants, the position information of the garbage pollutants is transmitted to the water cleaning robot, the water cleaning robot obtains the position information, the distance between target points is calculated by adopting an A-x algorithm, a cost matrix is established, and the aim of shortest motion path and recycling corresponding garbage is achieved by applying a dynamically planned path design; the overwater cleaning robot is combined with an image recognition system to carry out dynamic obstacle avoidance design. The invention effectively solves the problems of limited cleaning range, low cleaning efficiency and large energy consumption in the cleaning process of the intelligent water cleaning robot in the autonomous navigation process.
Description
Technical Field
The embodiment of the invention relates to the technical field of mobile robots, in particular to a method and equipment for planning paths of an unmanned aerial vehicle-cooperated water cleaning robot.
Background
In the field of mobile robots, path planning is an important link. For an intelligent water cleaning robot, the path planning of the intelligent water cleaning robot meets the following requirements: the cleaning robot can avoid the above-water obstacles and ensure the safety and stability of the cleaning robot in the driving process; can clean on water effectively and thoroughly in the region. Because the surface of water cleanness has higher complexity, the path planning of surface of water cleaning robot often has the cleaning efficiency not enough, and clean scope is lower, and intelligent degree is not enough problem. Therefore, developing a method and a device for planning a path of an unmanned aerial vehicle-cooperated water cleaning robot can effectively overcome the defects in the related art, and becomes a technical problem to be solved in the industry.
Disclosure of Invention
Aiming at the problems in the prior art, the embodiment of the invention provides a method and equipment for planning the path of an unmanned aerial vehicle-cooperated water cleaning robot.
In a first aspect, an embodiment of the present invention provides a method for planning a path of a water cleaning robot cooperated with an unmanned aerial vehicle, including: the system comprises an unmanned aerial vehicle image identification system based on a convolutional neural network, an unmanned aerial vehicle genetic algorithm path planning system, an overwater cleaning robot visual system, an unmanned aerial vehicle and overwater cleaning robot communication system and an overwater cleaning robot path planning system based on an A-algorithm; the unmanned aerial vehicle firstly cruises over a river channel, records a moving path, adopts a U-Net semantic segmentation network to realize the division of the water surface and the road surface, and marks the division; after the convolutional neural network is adopted to identify the garbage pollutants, the position information of the garbage pollutants is transmitted to the water cleaning robot, the water cleaning robot obtains the position information, the distance between target points is calculated by adopting an A-x algorithm, a cost matrix is established, and the aim of shortest motion path and recycling corresponding garbage is achieved by applying a dynamically planned path design; the overwater cleaning robot is combined with an image recognition system to carry out dynamic obstacle avoidance design.
On the basis of the content of the embodiment of the method, the unmanned aerial vehicle cooperative overwater cleaning robot path planning method provided by the embodiment of the invention adopts a U-Net semantic segmentation network to realize the division of the water surface and the road surface, and marks the division, and comprises the following steps: the method comprises the following steps: five primary effective network characteristic layers can be obtained by utilizing the trunk extraction network, and after the characteristics of the water surface and the road surface target are obtained; step two: the U-Net enters a stage of enhancing feature extraction, and a total effective feature layer after feature fusion is obtained by utilizing an up-sampling function of a network; step three: after the characteristic identification in the first step and the second step, the network classifies the characteristic identification results, which is equivalent to classifying each pixel point; step four: and building a U-Net network, programming according to the composition principle of the U-Net network, and realizing the input and detection input of the image by utilizing Opencv.
On the basis of the content of the embodiment of the method, the unmanned aerial vehicle-cooperated water cleaning robot path planning method provided by the embodiment of the invention adopts a convolutional neural network to identify garbage pollutants and then transmits the position information of the garbage pollutants to the water cleaning robot, and the water cleaning robot obtains the position information, and the method comprises the following steps: and a convolutional neural network is adopted, a target identification framework is created based on the obtained research object, a YoloV4 target detection network is adopted for a main framework, and an identification algorithm based on a DarkNet network main body is designed for further obtaining scene information of a target water area.
On the basis of the content of the embodiment of the method, the method for planning the path of the unmanned aerial vehicle-cooperated water cleaning robot provided by the embodiment of the invention adopts an A-x algorithm to calculate the distance between target points, establishes a cost matrix, and applies a dynamically planned path design to achieve the aims of shortest motion path and recycling corresponding garbage, and comprises the following steps: the method comprises the following steps: calculating the distance between the target points of the preprocessed garbage, the departure point and the distance between each target point, which are identified by the unmanned aerial vehicle, by using an A-x algorithm; step two: selecting a plurality of target points which are closest to the linear distance of the water surface cleaning robot as objects of the cleaning, then establishing a cost matrix, planning a path and establishing a dynamic planning function; step three: the water surface cleaning robot is combined with an image recognition system to carry out dynamic obstacle avoidance design.
On the basis of the content of the embodiment of the method, the method for planning the path of the unmanned aerial vehicle-cooperated water cleaning robot provided by the embodiment of the invention comprises the following steps:
d(i,V')＝min{cik+d(k,V-{k})}(k∈V')
d(k,{})＝cki(k≠i)
wherein d represents the distance from the starting point i to each target point once, only once and finally returning to i; v is all point sets; v' is a set representing passing points; cik is the distance from point i to point k; min is the minimum value; d (k, { }) ═ cki is the last step back to the origin.
In a second aspect, an embodiment of the present invention provides a path planning apparatus for a water cleaning robot cooperated with an unmanned aerial vehicle, including: the first main module is used for an unmanned aerial vehicle convolutional neural network-based image recognition system, an unmanned aerial vehicle genetic algorithm path planning system, a water cleaning robot visual system, an unmanned aerial vehicle and water cleaning robot communication system and a water cleaning robot path planning system based on an A-algorithm; the second main module is used for the unmanned aerial vehicle to cruise above a river channel, recording a moving path, and adopting a U-Net semantic segmentation network to divide the water surface and the road surface and mark the division; the third main module is used for identifying garbage pollutants by adopting a convolutional neural network and then transmitting the position information of the garbage pollutants to the water cleaning robot, the water cleaning robot obtains the position information, the distance between target points is calculated by adopting an A-star algorithm, a cost matrix is established, and the dynamic planning path design is applied to achieve the purposes of shortest motion path and corresponding garbage recovery; and the fourth main module is used for the overwater cleaning robot to perform dynamic obstacle avoidance design by combining with the image recognition system.
In a third aspect, an embodiment of the present invention provides an electronic device, including:
at least one processor; and
at least one memory communicatively coupled to the processor, wherein:
the storage stores program instructions executable by the processor, and the processor calls the program instructions to execute the unmanned aerial vehicle-cooperated marine cleaning robot path planning method provided by any one of the various implementation manners of the first aspect.
In a fourth aspect, embodiments of the present invention provide a non-transitory computer-readable storage medium storing computer instructions for causing a computer to perform the unmanned aerial vehicle-coordinated path planning method for an aquatic cleaning robot provided in any one of the various implementations of the first aspect.
According to the unmanned aerial vehicle cooperative overwater cleaning robot path planning method and device, the moving path is recorded, the division of the water surface and the road surface is achieved by adopting a U-Net semantic segmentation network, the overwater cleaning robot obtains position information, the goal that the moving path is shortest and corresponding garbage is recycled is achieved by applying a dynamically planned path design, and the problems that the intelligent overwater cleaning robot is limited in cleaning range, low in cleaning efficiency and large in energy consumption in the cleaning process in the autonomous navigation process are effectively solved.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, a brief description will be given below to the drawings required for the description of the embodiments or the prior art, and it is obvious that the drawings in the following description are some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to these drawings without creative efforts.
Fig. 1 is a flowchart of a method for planning a path of an unmanned aerial vehicle-assisted water cleaning robot according to an embodiment of the present invention;
fig. 2 is a schematic structural view of a path planning device of an unmanned aerial vehicle-cooperated water cleaning robot provided by an embodiment of the present invention;
fig. 3 is a schematic physical structure diagram of an electronic device according to an embodiment of the present invention;
FIG. 4 is a diagram illustrating an effect of an image prediction result according to an embodiment of the present invention;
FIG. 5 is a flow chart of visual SLAM modeling provided by an embodiment of the present invention;
fig. 6 is a schematic diagram of a simulation effect of a genetic algorithm three-dimensional path planning provided by an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, but not all, embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention. In addition, technical features of various embodiments or individual embodiments provided by the present invention may be arbitrarily combined with each other to form a feasible technical solution, and such combination is not limited by the sequence of steps and/or the structural composition mode, but must be realized by a person skilled in the art, and when the technical solution combination is contradictory or cannot be realized, such a technical solution combination should not be considered to exist and is not within the protection scope of the present invention.
The embodiment of the invention provides a path planning method for an unmanned aerial vehicle-cooperated water cleaning robot, and referring to fig. 1, the method comprises the following steps: the system comprises an unmanned aerial vehicle image recognition system based on a convolutional neural network, an unmanned aerial vehicle genetic algorithm path planning system, a water cleaning robot visual system, an unmanned aerial vehicle and water cleaning robot communication system and a water cleaning robot path planning system based on an A-algorithm; the unmanned aerial vehicle firstly cruises over a river channel, records a moving path, adopts a U-Net semantic segmentation network to realize the division of the water surface and the road surface, and marks the division; after the convolutional neural network is adopted to identify the garbage pollutants, the position information of the garbage pollutants is transmitted to the water cleaning robot, the water cleaning robot obtains the position information, the distance between target points is calculated by adopting an A-x algorithm, a cost matrix is established, and the aim of shortest motion path and recycling corresponding garbage is achieved by applying a dynamically planned path design; the overwater cleaning robot is combined with an image recognition system to carry out dynamic obstacle avoidance design.
Based on the content of the above method embodiment, as an optional embodiment, the unmanned aerial vehicle-cooperated waterborne cleaning robot path planning method provided in the embodiment of the present invention, which adopts a U-Net semantic segmentation network to realize the division of the water surface and the road surface, and marks the division, includes: the method comprises the following steps: five primary effective network characteristic layers can be obtained by utilizing the trunk extraction network, and after the characteristics of the water surface and the road surface target are obtained; step two: the U-Net enters a stage of enhancing feature extraction, and a total effective feature layer after feature fusion is obtained by utilizing an up-sampling function of a network; step three: after the characteristic identification in the first step and the second step, the network classifies the characteristic identification results, which is equivalent to classifying each pixel point; step four: and building a U-Net network, programming according to the composition principle of the U-Net network, and realizing the input and detection input of the image by utilizing Opencv.
Based on the content of the above method embodiment, as an optional embodiment, the unmanned aerial vehicle-cooperated marine cleaning robot path planning method provided in the embodiment of the present invention, where the position information of the garbage pollutant is transmitted to the marine cleaning robot after the garbage pollutant is identified by using the convolutional neural network, and the marine cleaning robot obtains the position information, includes: and a convolutional neural network is adopted, a target identification framework is created based on the obtained research object, a YoloV4 target detection network is adopted for a main framework, and an identification algorithm based on a DarkNet network main body is designed for further obtaining scene information of a target water area.
Based on the content of the above method embodiment, as an optional embodiment, the method for planning a path of an unmanned aerial vehicle-cooperated water cleaning robot provided in the embodiment of the present invention, which calculates a distance between target points by using an a-x algorithm, establishes a cost matrix, and applies a dynamically planned path design to achieve a target that a movement path is shortest and corresponding garbage is recovered, includes: the method comprises the following steps: calculating the distance between the target points of the preprocessed garbage, the departure point and the distance between each target point, which are identified by the unmanned aerial vehicle, by using an A-x algorithm; step two: selecting a plurality of target points which are closest to the linear distance of the water surface cleaning robot as objects of the cleaning, then establishing a cost matrix, planning a path and establishing a dynamic planning function; step three: the water surface cleaning robot is combined with an image recognition system to carry out dynamic obstacle avoidance design.
Based on the content of the above method embodiment, as an optional embodiment, the method for planning a path of an unmanned aerial vehicle-cooperated water cleaning robot provided in the embodiment of the present invention, wherein the establishing of the dynamic planning function includes:
d(i,V')＝min{cik+d(k,V-{k})}(k∈V') (1)
d(k,{})＝cki(k≠i) (2)
wherein d represents the distance from the starting point i to each target point once, only once and finally returning to i; v is all point sets; v' is a set representing passing points; cik is the distance from point i to point k; min is the minimum value; d (k, { }) ═ cki is the last step back to the origin.
According to the unmanned aerial vehicle cooperative water cleaning robot path planning method provided by the embodiment of the invention, the division of the water surface and the road surface is realized by recording the moving path and adopting the U-Net semantic segmentation network, the water cleaning robot obtains the position information, and the goal of shortest moving path and recycling corresponding garbage is achieved by applying the dynamically planned path design, so that the problems of limited cleaning range, low cleaning efficiency and high energy consumption in the cleaning process of the intelligent water cleaning robot during autonomous navigation are effectively solved.
In another embodiment, the genetic algorithm unmanned aerial vehicle path planning process comprises the following specific steps:
(1) establishing a theoretical model
The method comprises the steps of marking coordinate information such as road signs and the like through unmanned aerial vehicle vision, and updating the map state according to self-acquired GPS data and sensing data of a sensor on the surrounding environment at any time. The path planning needs to be updated iteratively in a dynamic scene, and a three-dimensional space path scheme based on a genetic algorithm is adopted to solve the path planning problem in a dynamic environment.
(2) Design algorithm
The genetic algorithm is characterized in that a possible solution of an optimization function is expressed into an individual, each individual forms a gene in a certain coding mode, and a population is evolved by means of genetic operators, selection, crossing and mutation operations, so that the population which is more adaptive to the environment is selected. In path planning, each path is planned into an individual, each population has n individuals, namely n paths, and each individual has m chromosomes, namely the number of intermediate transition points, each point (chromosome) has two dimensions (x, y), and a population is represented by genx and geny in a code. And (4) performing genetic operator operation on the population through evolution of each generation, and selecting a proper individual (optimal path). The genetic algorithm three-dimensional path planning simulation is shown in fig. 6.
In another embodiment, in the process of identifying pollutants by the unmanned aerial vehicle, the vision part is realized by mainly using vision to circulate the target area: image segmentation and image recognition, namely recognition and processing of an external environment under a high-definition camera with a holder by using a visual algorithm.
(1) U-Net image segmentation
Based on the requirement of image segmentation training on the water surface, the division of the water surface and the road surface is realized by utilizing a U-Net semantic segmentation network, and the division is marked. The U-Net is mainly divided into three parts, the first part is a trunk characteristic extraction network which mainly comprises the mutual superposition of a convolution layer and a pooling layer, and five primary effective network characteristic layers can be obtained by utilizing the trunk extraction network. After the characteristics of the water surface target and the road surface target are obtained, the U-Net enters a second stage which is a stage of enhancing characteristic extraction. The main effect of this stage is to obtain the total effective feature layer after feature fusion by using the network up-sampling function. The third stage is a prediction part, and after the previous feature identification, the network can classify by using the result of the feature identification, which is equivalent to classifying each pixel point.
Firstly, a data set is manufactured for target characteristics by using a water surface navigation data set, and a video is divided into 3000 images by frames. And labeling by using Lableme, and marking down a label. And after the completion, building a network by the U-Net. The U-Net network is mainly constructed according to the composition principle of the U-Net network, and the Opencv is used for realizing the input, detection input and image display of the image.
(2) Convolutional neural network image recognition
Based on the acquisition of the study objects, a target recognition framework is created, and the relevant subject framework adopts a YoloV4 target detection network. The YoloV4 mainly comprises three parts, namely a trunk extraction network, a characteristic pyramid and a classification regression layer. The backhaul of the network is formed by a CSPDarknet53 Backbone network, and the characteristic pyramid part comprises an SPP network and a PANet network. The Yoolohead section is mainly used for prediction
An identification algorithm based on a DarkNet network main body is designed for further acquiring scene information of a target water area. The unmanned aerial vehicle utilizes the high-altitude camera, discerns the target waters, obtains the environmental status in the field of view through marking the target object. The algorithm mainly identifies garbage, water bloom and natural buildings.
And combining data obtained after part of low-quality images are preprocessed with data in the original image to form a pollutant data set in the navigation channel, wherein the training input image size is 608 × 608, and 3510 effective frames are provided. Before training, the training set and test set ratios are set to 9:1, the batch size for each iteration is set to 2, 1580 batches are required to be completed for each iteration, and epoch is 50 for the total iterations. The training of the target is mainly divided into two parts, a training preparation stage and a training process.
In the training process, the training of the convolutional neural network is mainly the iterative operation and updating of the convolutional layer, the pooling layer and the full-connection layer, and the obtained prediction result is shown in fig. 4.
In another embodiment, after the unmanned aerial vehicle transmits the signal to the cleaning robot, the robot automatically drives through three-dimensional reconstruction, and the position and depth perception of surrounding scenes is realized through binocular cameras by adopting a mapping algorithm of ORB-SLAM2 on the algorithm. Algorithmically, the robot mapping needs to be accurate enough by taking into account the obstacles encountered in the path. And (3) planning the project group by adopting radar auxiliary mapping, and acquiring surrounding point cloud data through a laser radar. The flow is shown in fig. 5.
In another embodiment, after obtaining the map data and the garbage position, the water surface cleaning robot needs to plan its own motion path to achieve the purpose of cleaning the garbage. Aiming at the characteristics that the water surface cleaning robot needs to clean garbage at a plurality of places and returns to a starting point, the project group analyzes and calculates the distance between the target points by adopting an A-x algorithm, establishes a cost matrix, and applies a dynamically planned path design to achieve the aims of shortest motion path and recycling corresponding garbage.
Establishing a theoretical model
The scheme starts from a path planning problem based on hypothesis, marks of coordinate information such as road signs are made through unmanned aerial vehicle vision, and in unknown terrain, the map state is updated at any time according to self-obtained GPS data and perception data of a sensor to the surrounding environment. And sequentially going to a target point according to the planned movement path to clean the garbage. As the time complexity of the dynamic programming method is increased along with the increase of the number of target points, and the number of garbage collected by the water surface cleaning robot in a single cleaning process is limited, the number of the target points in the single cleaning process is set to be at most n, so that the time complexity of an algorithm is reduced, and the cleaning efficiency is improved.
Algorithm design
Firstly, the distance between the target points of the pre-processed garbage identified by the unmanned aerial vehicle, the distance between the departure point and each target point are calculated by an A-star algorithm. Assuming that the distance between any two points AB is calculated on the gridding map, the evaluation is carried out by using the A-x algorithm and the evaluation formula
F＝G+H
G is the movement cost of moving from the starting point a to the designated cell, and follows the path generated to reach the cell. We agree that the move straight one time cost is 10 and the move diagonal cost is 14. (the actual diagonal shift distance is the square root of 2, or approximately 1.414 times the lateral or longitudinal shift penalty). H is the estimated cost of moving from the designated square to the end point B. The number of squares passed from the current square to the target, either laterally or longitudinally, is counted, the diagonal movement is ignored, and the total is then multiplied by 10.
FGH was marked in three positions, top left, bottom left and bottom right, in a grid to observe the results of each assessment.
Coordinate access and parent node lookup convention order: right, top left, bottom right, with the direction of increase along the X-axis being right and the direction of increase along the Y-axis being up, there may be multiple parent nodes where the last searched parent node is not accessible by the barrier.
And after calculating all distance data, selecting at most n target points closest to the linear distance of the water surface cleaning robot as the cleaning object. Then, a cost matrix is established, a path is planned, and a dynamic planning function is established as shown in the formula (1) and the formula (2).
The implementation basis of the various embodiments of the present invention is realized by programmed processing performed by a device having a processor function. Therefore, in engineering practice, the technical solutions and functions thereof of the embodiments of the present invention can be packaged into various modules. Based on the above practical situation, on the basis of the above embodiments, the embodiments of the present invention provide an unmanned aerial vehicle-cooperated marine cleaning robot path planning apparatus, which is used for executing the unmanned aerial vehicle-cooperated marine cleaning robot path planning method in the above method embodiments. Referring to fig. 2, the apparatus includes: the first main module is used for an unmanned aerial vehicle convolutional neural network-based image recognition system, an unmanned aerial vehicle genetic algorithm path planning system, a water cleaning robot visual system, an unmanned aerial vehicle and water cleaning robot communication system and a water cleaning robot path planning system based on an A-algorithm; the second main module is used for the unmanned aerial vehicle to cruise above a river channel, recording a moving path, and adopting a U-Net semantic segmentation network to divide the water surface and the road surface and mark the division; the third main module is used for identifying garbage pollutants by adopting a convolutional neural network and then transmitting the position information of the garbage pollutants to the water cleaning robot, the water cleaning robot obtains the position information, the distance between target points is calculated by adopting an A-star algorithm, a cost matrix is established, and the dynamic planning path design is applied to achieve the purposes of shortest motion path and corresponding garbage recovery; and the fourth main module is used for the overwater cleaning robot to perform dynamic obstacle avoidance design by combining with the image recognition system.
The unmanned aerial vehicle cooperative overwater cleaning robot path planning device provided by the embodiment of the invention adopts a plurality of modules in fig. 2, realizes the division of water surface and road surface by recording a moving path and adopting a U-Net semantic segmentation network, obtains position information by the overwater cleaning robot, and achieves the aims of shortest moving path and recycling corresponding garbage by applying a dynamically planned path design, thereby effectively solving the problems of limited cleaning range, low cleaning efficiency and large energy consumption in the cleaning process of an intelligent overwater cleaning robot during autonomous navigation.
It should be noted that, the apparatus in the apparatus embodiment provided by the present invention may be used for implementing methods in other method embodiments provided by the present invention, except that corresponding function modules are provided, and the principle of the apparatus embodiment provided by the present invention is basically the same as that of the apparatus embodiment provided by the present invention, so long as a person skilled in the art obtains corresponding technical means by combining technical features on the basis of the apparatus embodiment described above, and obtains a technical solution formed by these technical means, on the premise of ensuring that the technical solution has practicability, the apparatus in the apparatus embodiment described above may be modified, so as to obtain a corresponding apparatus class embodiment, which is used for implementing methods in other method class embodiments. For example:
based on the content of the above device embodiment, as an optional embodiment, the unmanned aerial vehicle-cooperated marine cleaning robot path planning device provided in the embodiment of the present invention further includes: the first submodule is used for realizing the division of the water surface and the road surface by adopting the U-Net semantic segmentation network and marking the division, and comprises: the method comprises the following steps: five preliminary effective network characteristic layers can be obtained by utilizing the trunk extraction network, and after the characteristics of the water surface and the pavement target are obtained; step two: the U-Net enters a stage of enhancing feature extraction, and a total effective feature layer after feature fusion is obtained by utilizing an up-sampling function of a network; step three: after the characteristic identification in the first step and the second step, the network classifies the characteristic identification results, which is equivalent to classifying each pixel point; step four: and building a U-Net network, programming according to the composition principle of the U-Net network, and realizing the input and detection input of the image by utilizing Opencv.
Based on the content of the above device embodiment, as an optional embodiment, the unmanned aerial vehicle-cooperated marine cleaning robot path planning device provided in the embodiment of the present invention further includes: the second submodule is used for realizing that the position information of the garbage pollutant is transmitted to the water cleaning robot after the garbage pollutant is identified by adopting the convolutional neural network, and the water cleaning robot obtains the position information, and the method comprises the following steps: and a convolutional neural network is adopted, a target identification framework is created based on the obtained research object, a YoloV4 target detection network is adopted for a main framework, and an identification algorithm based on a DarkNet network main body is designed for further obtaining scene information of a target water area.
Based on the content of the above device embodiment, as an optional embodiment, the unmanned aerial vehicle-cooperated marine cleaning robot path planning device provided in the embodiment of the present invention further includes: the third sub-module is used for calculating the distance between the target points by adopting an A-x algorithm, establishing a cost matrix, and applying a dynamically planned path design to achieve the purposes of shortest motion path and corresponding garbage recovery, and comprises the following steps: the method comprises the following steps: calculating the distance between the target points of the preprocessed garbage, the departure point and the distance between each target point, which are identified by the unmanned aerial vehicle, by using an A-x algorithm; step two: selecting a plurality of target points which are closest to the linear distance of the water surface cleaning robot as objects of the cleaning, then establishing a cost matrix, planning a path and establishing a dynamic planning function; step three: the water surface cleaning robot is combined with an image recognition system to carry out dynamic obstacle avoidance design.
Based on the content of the above device embodiment, as an optional embodiment, the unmanned aerial vehicle-cooperated marine cleaning robot path planning device provided in the embodiment of the present invention further includes: a fourth sub-module, configured to implement the establishing of the dynamic programming function, including:
d(i,V')＝min{cik+d(k,V-{k})}(k∈V')
d(k,{})＝cki(k≠i)
wherein d represents the distance from the starting point i to each target point once, only once and finally returning to i; v is all point sets; v' is a set representing passing points; cik is the distance from point i to point k; min is the minimum value; d (k, { }) ═ cki is the last step back to the origin.
The method of the embodiment of the invention is realized by depending on the electronic equipment, so that the related electronic equipment is necessarily introduced. To this end, an embodiment of the present invention provides an electronic apparatus, as shown in fig. 3, including: the system comprises at least one processor (processor), a communication Interface (communication Interface), at least one memory (memory) and a communication bus, wherein the at least one processor, the communication Interface and the at least one memory are communicated with each other through the communication bus. The at least one processor may invoke logic instructions in the at least one memory to perform all or a portion of the steps of the methods provided by the various method embodiments described above.
In addition, the logic instructions in the at least one memory may be implemented in software functional units and stored in a computer readable storage medium when sold or used as a stand-alone product. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the method embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disk, and various media capable of storing program codes.
The above-described embodiments of the apparatus are merely illustrative, and the units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one position, or may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment. One of ordinary skill in the art can understand and implement it without inventive effort.
Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware. Based on the understanding, the above technical solutions substantially or otherwise contributing to the prior art may be embodied in the form of a software product, which may be stored in a computer-readable storage medium, such as ROM/RAM, magnetic disk, optical disk, etc., and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the method according to the various embodiments or some parts of the embodiments.
The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present invention. Based on this recognition, each block in the flowchart or block diagrams may represent a module, a program segment, or a portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
In this patent, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.
Finally, it should be noted that: the above examples are only intended to illustrate the technical solution of the present invention, but not to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.
Claims (8)
1. A method for planning the path of a water cleaning robot cooperated by an unmanned aerial vehicle is characterized by comprising the following steps: the system comprises an unmanned aerial vehicle image identification system based on a convolutional neural network, an unmanned aerial vehicle genetic algorithm path planning system, an overwater cleaning robot visual system, an unmanned aerial vehicle and overwater cleaning robot communication system and an overwater cleaning robot path planning system based on an A-algorithm; the unmanned aerial vehicle firstly cruises over a river channel, records a moving path, adopts a U-Net semantic segmentation network to realize the division of the water surface and the road surface, and marks the division; after the convolutional neural network is adopted to identify the garbage pollutants, the position information of the garbage pollutants is transmitted to the water cleaning robot, the water cleaning robot obtains the position information, the distance between target points is calculated by adopting an A-x algorithm, a cost matrix is established, and the aim of shortest motion path and recycling corresponding garbage is achieved by applying a dynamically planned path design; the overwater cleaning robot is combined with an image recognition system to carry out dynamic obstacle avoidance design.
2. The unmanned aerial vehicle-cooperative waterborne cleaning robot path planning method according to claim 1, wherein the partitioning and marking of the water surface and the road surface are realized by adopting a U-Net semantic segmentation network, and comprises the following steps: the method comprises the following steps: five primary effective network characteristic layers can be obtained by utilizing the trunk extraction network, and after the characteristics of the water surface and the road surface target are obtained; step two: the U-Net enters a stage of enhancing feature extraction, and a total effective feature layer after feature fusion is obtained by utilizing an up-sampling function of a network; step three: after the characteristic identification in the first step and the second step, the network classifies the characteristic identification results, which is equivalent to classifying each pixel point; step four: and building a U-Net network, programming according to the composition principle of the U-Net network, and realizing the input and detection input of the image by utilizing Opencv.
3. The unmanned-aerial-vehicle-cooperative path planning method for the aquatic cleaning robot according to claim 2, wherein the step of recognizing the garbage pollutants by using the convolutional neural network and then transmitting the position information to the aquatic cleaning robot obtains the position information comprises the steps of: and a convolutional neural network is adopted, a target identification framework is created based on the obtained research object, a YoloV4 target detection network is adopted for a main framework, and an identification algorithm based on a DarkNet network main body is designed for further obtaining scene information of a target water area.
4. The unmanned aerial vehicle-cooperative path planning method for an aquatic cleaning robot according to claim 3, wherein the calculating of the distance between the target points by using the a-x algorithm, the establishment of the cost matrix, and the application of the dynamically planned path design to achieve the goal of shortest motion path and corresponding garbage recovery comprise: the method comprises the following steps: calculating the distance between the target points of the preprocessed garbage, the departure point and the distance between each target point, which are identified by the unmanned aerial vehicle, by using an A-x algorithm; step two: selecting a plurality of target points which are closest to the linear distance of the water surface cleaning robot as objects of the cleaning, then establishing a cost matrix, planning a path and establishing a dynamic planning function; step three: the water surface cleaning robot is combined with an image recognition system to carry out dynamic obstacle avoidance design.
5. The unmanned-aerial-vehicle-cooperative marine cleaning robot path planning method of claim 4, wherein the establishing a dynamic planning function comprises:
d(i,V')＝min{cik+d(k,V-{k})}(k∈V')
d(k,{})＝cki(k≠i)
wherein d represents the distance from the starting point i to each target point once, only once and finally returning to i; v is all point sets; v' is a set representing passing points; cik is the distance from point i to point k; min is the minimum value; d (k, { }) ═ cki is the last step back to the origin.
6. The utility model provides a clean robot path planning device on water that unmanned aerial vehicle is cooperative which characterized in that includes: the first main module is used for an unmanned aerial vehicle convolutional neural network-based image recognition system, an unmanned aerial vehicle genetic algorithm path planning system, a water cleaning robot visual system, an unmanned aerial vehicle and water cleaning robot communication system and a water cleaning robot path planning system based on an A-algorithm; the second main module is used for the unmanned aerial vehicle to cruise above a river channel, recording a moving path, and adopting a U-Net semantic segmentation network to divide the water surface and the road surface and mark the division; the third main module is used for identifying garbage pollutants by adopting a convolutional neural network and then transmitting the position information of the garbage pollutants to the water cleaning robot, the water cleaning robot obtains the position information, the distance between target points is calculated by adopting an A-star algorithm, a cost matrix is established, and the dynamic planning path design is applied to achieve the purposes of shortest motion path and corresponding garbage recovery; and the fourth main module is used for the overwater cleaning robot to perform dynamic obstacle avoidance design by combining with the image recognition system.
7. An electronic device, comprising:
at least one processor, at least one memory, and a communication interface; wherein,
the processor, the memory and the communication interface are communicated with each other;
the memory stores program instructions executable by the processor, the processor invoking the program instructions to perform the method of any of claims 1 to 5.
8. A non-transitory computer-readable storage medium storing computer instructions for causing a computer to perform the method of any one of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210286539.2A CN114815810A (en) | 2022-03-22 | 2022-03-22 | Unmanned aerial vehicle-cooperated overwater cleaning robot path planning method and equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210286539.2A CN114815810A (en) | 2022-03-22 | 2022-03-22 | Unmanned aerial vehicle-cooperated overwater cleaning robot path planning method and equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114815810A true CN114815810A (en) | 2022-07-29 |
Family
ID=82530742
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210286539.2A Pending CN114815810A (en) | 2022-03-22 | 2022-03-22 | Unmanned aerial vehicle-cooperated overwater cleaning robot path planning method and equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114815810A (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115049825A (en) * | 2022-08-16 | 2022-09-13 | 北京大学 | Water surface cleaning method, device, equipment and computer readable storage medium |
| CN115421494A (en) * | 2022-09-19 | 2022-12-02 | 西安交通大学 | Cleaning robot path planning method, system, computer device and storage medium |
| CN116382328A (en) * | 2023-03-09 | 2023-07-04 | 南通大学 | Dam intelligent detection method based on cooperation of multiple robots in water and air |
| CN116611602A (en) * | 2023-07-17 | 2023-08-18 | 国家电投集团江西电力有限公司 | Photovoltaic panel cleaning path planning method and system |
-
2022
- 2022-03-22 CN CN202210286539.2A patent/CN114815810A/en active Pending
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115049825A (en) * | 2022-08-16 | 2022-09-13 | 北京大学 | Water surface cleaning method, device, equipment and computer readable storage medium |
| CN115049825B (en) * | 2022-08-16 | 2022-11-01 | 北京大学 | Water surface cleaning method, device, equipment and computer readable storage medium |
| CN115421494A (en) * | 2022-09-19 | 2022-12-02 | 西安交通大学 | Cleaning robot path planning method, system, computer device and storage medium |
| CN116382328A (en) * | 2023-03-09 | 2023-07-04 | 南通大学 | Dam intelligent detection method based on cooperation of multiple robots in water and air |
| CN116382328B (en) * | 2023-03-09 | 2024-04-12 | 南通大学 | Dam intelligent detection method based on cooperation of multiple robots in water and air |
| CN116611602A (en) * | 2023-07-17 | 2023-08-18 | 国家电投集团江西电力有限公司 | Photovoltaic panel cleaning path planning method and system |
| CN116611602B (en) * | 2023-07-17 | 2023-11-03 | 国家电投集团江西电力有限公司 | Photovoltaic panel cleaning path planning method and system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Wigness et al. | A rugd dataset for autonomous navigation and visual perception in unstructured outdoor environments | |
| CN110956651B (en) | Terrain semantic perception method based on fusion of vision and vibrotactile sense | |
| US11276230B2 (en) | Inferring locations of 3D objects in a spatial environment | |
| CN114815810A (en) | Unmanned aerial vehicle-cooperated overwater cleaning robot path planning method and equipment | |
| JP6272217B2 (en) | Method and apparatus for estimating free space in the vicinity of a moving object | |
| Zhu et al. | 3d lidar point cloud based intersection recognition for autonomous driving | |
| EP3822852B1 (en) | Method, apparatus, computer storage medium and program for training a trajectory planning model | |
| Paz et al. | Probabilistic semantic mapping for urban autonomous driving applications | |
| CN111213155A (en) | Image processing method, device, movable platform, unmanned aerial vehicle and storage medium | |
| CN111563415A (en) | Binocular vision-based three-dimensional target detection system and method | |
| CN113705636B (en) | Method and device for predicting track of automatic driving vehicle and electronic equipment | |
| Niranjan et al. | Deep learning based object detection model for autonomous driving research using carla simulator | |
| EP3690744A1 (en) | Method for integrating driving images acquired from vehicles performing cooperative driving and driving image integrating device using same | |
| CN110986945B (en) | Local navigation method and system based on semantic altitude map | |
| JP2020123346A (en) | Method and device for performing seamless parameter switching by using location based algorithm selection to achieve optimized autonomous driving in each of regions | |
| CN113126115A (en) | Semantic SLAM method and device based on point cloud, electronic equipment and storage medium | |
| CN110705385B (en) | Method, device, equipment and medium for detecting angle of obstacle | |
| US20220373354A1 (en) | Automatic generation of vector map for vehicle navigation | |
| CN112257668A (en) | Main and auxiliary road judging method and device, electronic equipment and storage medium | |
| CN111098850A (en) | Automatic parking auxiliary system and automatic parking method | |
| CN111508252A (en) | Method and device for updating high-definition map by using V2X information integration technology | |
| Zhang et al. | Hierarchical road topology learning for urban mapless driving | |
| CN113012191B (en) | Laser mileage calculation method based on point cloud multi-view projection graph | |
| CN113671522B (en) | Dynamic environment laser SLAM method based on semantic constraint | |
| Han et al. | Fully convolutional neural networks for road detection with multiple cues integration |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |