CN112327923B - Multi-unmanned aerial vehicle collaborative path planning method - Google Patents
Multi-unmanned aerial vehicle collaborative path planning method Download PDFInfo
- Publication number
- CN112327923B CN112327923B CN202011303855.3A CN202011303855A CN112327923B CN 112327923 B CN112327923 B CN 112327923B CN 202011303855 A CN202011303855 A CN 202011303855A CN 112327923 B CN112327923 B CN 112327923B
- Authority
- CN
- China
- Prior art keywords
- unmanned aerial
- path
- point
- aerial vehicle
- particle
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 36
- 239000002245 particle Substances 0.000 claims abstract description 67
- 238000005457 optimization Methods 0.000 claims abstract description 21
- 230000006870 function Effects 0.000 claims description 15
- 230000009471 action Effects 0.000 claims description 14
- 230000035772 mutation Effects 0.000 claims description 8
- 238000003780 insertion Methods 0.000 claims description 4
- 230000037431 insertion Effects 0.000 claims description 4
- 230000001133 acceleration Effects 0.000 claims description 2
- 238000004364 calculation method Methods 0.000 claims description 2
- 230000003247 decreasing effect Effects 0.000 claims description 2
- 230000009191 jumping Effects 0.000 claims description 2
- 230000008901 benefit Effects 0.000 abstract description 6
- 210000004027 cell Anatomy 0.000 description 5
- 230000008569 process Effects 0.000 description 4
- 238000010586 diagram Methods 0.000 description 3
- 230000002068 genetic effect Effects 0.000 description 2
- 239000003016 pheromone Substances 0.000 description 2
- 201000004569 Blindness Diseases 0.000 description 1
- 241000257303 Hymenoptera Species 0.000 description 1
- 101100264172 Oryza sativa subsp. japonica XIAO gene Proteins 0.000 description 1
- 230000003044 adaptive effect Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 210000000349 chromosome Anatomy 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 238000009499 grossing Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000010187 selection method Methods 0.000 description 1
- 241000894007 species Species 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
- G05D1/10—Simultaneous control of position or course in three dimensions
- G05D1/101—Simultaneous control of position or course in three dimensions specially adapted for aircraft
- G05D1/104—Simultaneous control of position or course in three dimensions specially adapted for aircraft involving a plurality of aircrafts, e.g. formation flying
Landscapes
- Engineering & Computer Science (AREA)
- Aviation & Aerospace Engineering (AREA)
- Radar, Positioning & Navigation (AREA)
- Remote Sensing (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Automation & Control Theory (AREA)
- Control Of Position, Course, Altitude, Or Attitude Of Moving Bodies (AREA)
Abstract
The invention provides a multi-unmanned aerial vehicle collaborative path planning method, which comprises the following steps: the unmanned aerial vehicle group obtains a starting point position, an obstacle point and existing radar coordinates in a map environment, and establishes a two-dimensional grid map of a combat environment; according to the obtained two-dimensional grid map, establishing an optimization target of the multiple unmanned aerial vehicles according to the path length of each unmanned aerial vehicle and the threat strength; initializing the population by combining random generation path points and Q learning to obtain an initial population of each unmanned aerial vehicle path; optimizing the initial population by adopting an improved particle swarm algorithm according to the optimization target; the method takes the overall benefits of the multiple unmanned aerial vehicles as an optimization target, and ensures the safety and the high efficiency of the flight path of each unmanned aerial vehicle.
Description
Technical Field
The invention relates to the technical field of unmanned aerial vehicles, in particular to a multi-unmanned aerial vehicle collaborative path planning method.
Background
The cooperation of many unmanned aerial vehicles is the main style of future unmanned aerial vehicle application, and many unmanned aerial vehicles cooperate the performance of route planning has decided many unmanned aerial vehicles synergy effect to a great extent, is the focus of unmanned aerial vehicle route planning research at present.
The cooperative path planning of the multiple unmanned aerial vehicles refers to planning paths for the multiple unmanned aerial vehicles under the condition of considering cooperative constraint, so that the multiple unmanned aerial vehicles can cooperatively complete tasks at minimum cost. Compared with a single unmanned aerial vehicle, the implementation of path planning of multiple unmanned aerial vehicles is more limited and more difficult, which is a key point and a difficult point of current path planning.
The traditional algorithms such as A, Voronoi and the like have high calculation efficiency and simple planning. MA P B et al use Voronoi diagrams and collaborative strategies to achieve collaborative track planning that satisfies time constraints. Lidewei et al, by improving the search sequence and optimizing the valuation function, changes the undirected search in the A-algorithm into a directed search, changes the global valuation into the local valuation, and improves the algorithm efficiency. However, the traditional algorithm has many limitations in solving the path planning problem, for example, the precision method is only suitable for small-scale path planning, and when the objective function and the constraint condition are complicated, the precision method is difficult to provide an effective solution. The heuristic algorithm is easy to fall into local optimization and is also not suitable for the problem of large scale.
In view of the limitation of solving the path planning of multiple unmanned aerial vehicles by the traditional algorithm, more and more learners utilize the intelligent optimization algorithm to solve the unmanned aerial vehicle cluster path planning, the intelligent optimization algorithm has strong self-organization, strong robustness and strong expandability, and can be used for solving the unmanned aerial vehicle cooperation problem under complex constraint. The ant colony algorithm, the particle swarm algorithm and the genetic algorithm are the most widely used 3 types of methods.
All quantum ants complete path search according to tabu search and quantum pheromone updating node selection probability, quantum rotation angles are updated according to comprehensive cost of optimal paths, quantum pheromone is updated by using simulated quantum rotating gates, the output optimal paths are stored in a path set, whether the number of the paths in the path set reaches the maximum number of the paths is judged, and the paths in the path set are sorted according to length for selection of the unmanned aerial vehicle. Yu et al generated a solution sequence of the unmanned aerial vehicle cluster path using the track point numbers, and the multiple sequences divided by the division points represented multiple unmanned aerial vehicle paths, and introduced differential evolution operation to maintain the diversity of particles for solving the problem of particle precocity, and solved by a hybrid particle swarm algorithm in combination with an adaptive adjustment inertial weight strategy. XIAO et al, in order to solve the problem of path planning of multiple unmanned aerial vehicles traversing multiple target points with obstacle area restriction, aiming at the lowest total flight cost, design a new crossover operator in the process of solving by using a genetic algorithm, randomly select a sub-path from a parent generation and lead the sub-path, and generate paths traversing other target points according to a certain rule, thereby improving the convergence speed while retaining the excellent chromosomes of the parent generation.
In the invention, the particle swarm algorithm based on single route point evaluation can realize learning more potential better route points in the unmanned aerial vehicle cluster, but cross route points can be generated, and in this case, the obtained routes need to be subjected to conflict processing, so that the convergence speed is slowed.
The UUV path planning method based on the particle swarm algorithm does not consider threats possibly encountered in the running process of the unmanned ship, and is a single-target path planning method.
The unmanned aerial vehicle cluster path planning based on the improved Q learning algorithm only plans a path for each unmanned aerial vehicle independently, and does not consider the overall benefit of the unmanned aerial vehicle cluster.
The path planning method based on the ant colony algorithm judges whether the cooperation can be achieved or not by smoothing the initial path of each unmanned aerial vehicle, does not consider the cooperation problem among the unmanned aerial vehicles in the algorithm process, and only plans the path for each unmanned aerial vehicle independently.
Aiming at the current existing unmanned aerial vehicle path planning method, the characteristics of multi-target and cooperative optimization in the multi-unmanned aerial vehicle path planning in the actual situation are combined, the cooperative nature of the multi-unmanned aerial vehicle path planning needs to be deeply researched, and the overall benefit of the unmanned aerial vehicle cluster can be increased under the condition that the unmanned aerial vehicle can reach the destination smoothly.
The invention provides a multi-unmanned aerial vehicle collaborative path planning method based on an improved particle swarm algorithm. In the initialization of particle swarm, the invention adopts a mode of combining random initialization and Q learning initialization, thereby ensuring the diversity of the initial population, accelerating the convergence speed of the algorithm, and improving the position and speed updating mode of the particle swarm algorithm, so that the method is suitable for the path planning problem in a coordinate form; the path lengths and threat strengths of the multiple unmanned aerial vehicles are considered in the flight process of the unmanned aerial vehicles, so that the high efficiency and safety of the multiple unmanned aerial vehicles for executing tasks are ensured; the weighted path length and the threat intensity of the multiple unmanned aerial vehicles are used as optimization targets, and the overall benefit of the unmanned aerial vehicle cluster is increased.
Disclosure of Invention
In view of the above, the present invention provides a method for planning a collaborative path of multiple drones, including the following steps:
s1, acquiring radar coordinates including a starting point position, an obstacle point and an existing radar in a map environment by the unmanned aerial vehicle cluster, and establishing a two-dimensional grid map of a battle environment;
s2, constructing an optimization target of the multiple unmanned aerial vehicles according to the two-dimensional grid map established in the S1 through the path length of each unmanned aerial vehicle and the threat strength;
s3, initializing the population by combining random generation path points and Q learning to obtain an initial population of each unmanned aerial vehicle path;
s4, optimizing the initial population obtained in the S3 by adopting an improved particle swarm optimization according to the optimization target constructed in the step S2;
and S5, obtaining the optimal particles of the population, and outputting the air routes of all unmanned aerial vehicles in the unmanned aerial vehicle cluster.
The technical scheme provided by the invention has the beneficial effects that: 1. the mixed mode of the random generation path and the Q learning generation path is adopted, so that the superiority of the initial solution of the population is ensured;
2. the variation operation is based on the distance and the included angle between the point to be selected and the end point, different selection probabilities are given to the points to be selected, and the convergence speed of the algorithm is accelerated;
3. the overall benefit of the multiple unmanned aerial vehicles is taken as an optimization target, and the safety and the high efficiency of flight paths of the unmanned aerial vehicles are guaranteed.
Drawings
Fig. 1 is a flow chart of a multi-drone collaborative path planning method of the present invention;
FIG. 2 is a grid environment diagram of a multi-UAV collaborative path planning method according to the present invention;
fig. 3 is a schematic diagram illustrating an action variation of a multi-drone collaborative path planning method according to the present invention;
fig. 4 is a cross operation mode 1 of optimal particle and individual for the multi-drone collaborative path planning method of the present invention;
fig. 5 is a cross operation mode 2 of the optimal particle and individual for the multi-unmanned aerial vehicle collaborative path planning method of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, embodiments of the present invention will be further described with reference to the accompanying drawings.
Referring to fig. 1, the present invention provides a method for planning a collaborative path of multiple drones, including the following steps:
s1, acquiring radar coordinates including a starting point position, an obstacle point and an existing radar in a map environment by the unmanned aerial vehicle cluster, and establishing a two-dimensional grid map of a battle environment;
the path points are represented in a form of coordinates in a two-dimensional grid graph, and the flight environment of the unmanned aerial vehicle is described as follows: { (x, y) | xmin≤x≤xmax,ymin≤y≤ymax}；xmin、xmax、yminAnd ymaxRepresented as the boundary of the drone flight;
as shown in fig. 2, the environment is divided into 20 × 20 cells having the same area, each cell carries different parameter information of 0, -1, 1, and 2, and when the cell parameter is 0, it indicates that the area has no obstacle, when the cell parameter is 1, it indicates that the area contains an obstacle or a threat source, when the cell parameter is-1, and it indicates that the area has an end point, and by constructing a two-dimensional grid map, it is possible to obtain environment information well.
S2, constructing an optimization target of the multiple unmanned aerial vehicles according to the two-dimensional grid map established in the S1 through the path length of each unmanned aerial vehicle and the threat strength;
number of steps f taken by i unmanned aerial vehiclesstepConsidering the constraint of the turning angle of the unmanned aerial vehicle as step, if the unmanned aerial vehicle has oblique flight in the path, a penalty f is imposedpenaltyAnd obtaining the path length expression of the single unmanned aerial vehicle:
fi＝fstep+fpenalty
the path comprehensive weighting length of the multiple unmanned aerial vehicles is calculated by the following formula:
wherein m is the number of unmanned aerial vehicles;
dividing a complete path from a starting point to a target point into three segments, and taking each threat point to 1/4, 2/4 and 3/4 of each segment for summation to obtain the threat intensity of the threat point j to the unmanned aerial vehicle i as represented by
Where t is the three parts of the path, itFor the t-th segment of the drone path,
and
for the fourth power of the distance from threat point j to the t-th paths 1/4, 2/4, and 3/4, K is the adjustment factor, and the threat strength of a single drone under the path is:
wherein N is the number of threat points, and the comprehensive threat intensity that the unmanned aerial vehicle received of definition is:
constructing an overall optimization target f of multiple unmanned aerial vehicles: f ═ w1*f1+w2*f2Wherein w is1And w2Respectively, the total weighted path length and the weight of the total weighted threat strength.
S3, initializing the population by combining random generation path points and Q learning to obtain an initial population of each unmanned aerial vehicle path;
the invention adopts a random path point generation mode to obtain part of initial population, the random path point generation mode is that when the initial species group is generated, if a continuous path is obtained, the unmanned aerial vehicle can select a free grid from 8 adjacent grids around the unmanned aerial vehicle as a next grid, the selection method can ensure that the path is continuous, the generated path has diversity, but the generated path is longer, the path threat intensity is higher, and simultaneously, the evolution algebra is increased to influence the convergence speed.
Therefore, the invention obtains another part of the initial population by using a Q learning algorithm, the purpose of the Q learning algorithm is to obtain the maximum accumulated reward value in the interaction with the environment, and therefore, the construction of a reasonable reward function is very important. In the present invention, the reward function is designed as shown in table 1;
setting of a report-back function
| Unmanned plane status | Starting point | Target point | Obstacle | Threat point | Feasible |
| Reward value | |||||
| 0 | 20 | -1 | 1 | r |
In order to accelerate the convergence rate of Q learning, a feasible point reward function based on a sparse function is constructed in Q learning:
wherein r is0Based on the prize value, rmaxTo take into account the maximum reward value at the time of threat, rthreatAs a threat coefficient, xUAVAnd yUAVCoordinates for unmanned aerial vehicles, xgoalAnd ygoalCoordinate of end point, xithreatAnd yithreatAs coordinates of each threat source, kgoalAnd kthreatTo adjust the control coefficient of the reward function, r is guaranteed during the parameter setting>0；
In order to ensure good exploration, the action selection strategy needs to give consideration to exploration and application, the action selection strategy adopted by Q learning in the invention is a Boltzmann distribution method, and the expression of the action selection strategy is as follows:
wherein ViRepresenting state s or state-action pairs (s, a)i) Any of the value functions of p (a)i| s) represents action a under state siThe selected probability, T is a control parameter, and k is the iteration number;
each unmanned aerial vehicle path obtained through the Q learning algorithm has the characteristics of shorter path length and smaller threat intensity compared with the path obtained in a random generation mode.
Selecting a path as a candidate initial solution of a population from the unmanned aerial vehicle path solutions obtained by the Q learning algorithm, prescribing the number of path solutions of each unmanned aerial vehicle in different initial directions by using priori knowledge, and combining the initial solution obtained by the Q learning and the initial solution obtained by randomly generating path points, namely obtaining the initial population of the particle swarm algorithm, wherein the form of the initial solution of the population is a coordinate.
S4, optimizing the initial population obtained in the S3 by adopting an improved particle swarm optimization according to the optimization target constructed in the step S2;
aiming at the problem that a speed and position updating operator of a basic particle swarm algorithm is difficult to express a discrete domain such as coordinate-form-based collaborative path planning, therefore, a position and speed updating formula of the particle swarm algorithm is analyzed and an improvement strategy is proposed;
the updating formula of the position and the speed of the particle swarm algorithm is as follows:
wherein xi,j(t)＝(xi,j(1),xi,j(2),Λ,xi,j(t)) is the position of the ith particle at the t-th iteration, pi,j,best(t)＝(pi,j,best(1),pi,j,best(2),Λ,pi,j,best(3) G) is the optimum value of the particle in t iterationsi,j,best(t)＝(gi,j,best(1),gi,j,best(2),Λ,gi,j,best(3) Is the global optimum of the particle in t iterations, w is the inertial weight, c1And c2Is a non-negative acceleration coefficient, f1As a function of the influence of the particle on itself, f2Is a particle xi,j(t) learning of the optimal value of its individual history, f3Is a particle xi,j(t) learning of its global historical optimum;
in an evolution operation F1Defined as a mutation operation, a particle i mutates itself with a probability of w, with a probability of 1-w remaining unchanged, expressed as:
the convergence of the algorithm is accelerated by adopting a linear time-varying inertia weight strategy:
w has a value of from wmaxLinearly decreasing to a final value wminT is the current iteration number of the algorithm, itermaxThe maximum number of iterations allowed for the algorithm to continue;
the mutation operation comprises the following specific steps:
s41, randomly selecting a point in the path coordinate, and removing the starting point and the end point;
s42, as shown in fig. 3, the next action of the unmanned aerial vehicle has 8 choices according to the point where the unmanned aerial vehicle is located, and if the original coordinate point in the path is 1, the coordinate point to be selected is 2 to 8, and there are 7 coordinate points to be selected. In order to reduce the search blindness, the distance D between each point i to be selected and the target point is calculatediCalculating the probability P that the point i is selectedi：
Wherein k isiThe adjustment coefficient is determined according to the included angle between the action point to be selected and the target point, and particularly, when the point to be selected is an obstacle point, the probability of the point to be selected is 0;
s43, randomly generating a number P of 0-1 if P1+P2+...Pi-1＜P≤P1+P2+...PiIf yes, selecting the point i as a next action point;
s44, adding the current grid into a search tabu table to avoid repeated selection, and replacing the current grid serial number with the next grid serial number;
s45, if all the coordinate points to be selected are obstacle points, canceling the variation operation of the point, and directly jumping to S41 to reselect the variation coordinates;
s46, judging whether the new action point is continuous with the original path, if not, inserting a new non-obstacle coordinate point at the discontinuous position, and judging whether the path is continuous according to the following formula;
δ＝max{abs(xi+1-xi),abs(yi+1-yi)}
wherein (x)i+1,yi+1) Is the coordinate point of the original path, (x)i,yi) If δ is 1, the path is continuous, and the mutation operation is finished; when delta>When 1, the path is discontinuous, new coordinates are directly inserted into the two coordinate points through an average value method, and the coordinate expression of the inserted points is as follows;
and judging whether the coordinates of the insertion point have obstacles or not, if so, sequentially selecting the coordinates in the order of the upper right part, the lower left part and the lower right part of the insertion point to judge whether the coordinates are the obstacles or not until the free coordinate point is selected, judging whether the inserted point is continuous with the original path point again, and if not, repeating the step S45 until the path is continuous.
F is to be2Defined as particles i and c1Is interleaved with its individual optimum values by 1-c1The probability of (c) remains unchanged, which is expressed as:
f is to be3Defined as particles i and c2Is interleaved with its global optimum by 1-c2The probability of (c) remains unchanged, which is expressed as:
step F of cross operation of particles in population and globally optimal particles2Similarly;
and obtaining the optimal particles of each sub-population through the evolution operation, judging whether the optimal particles meet the cycle ending condition, if so, storing the optimal particles of each sub-population, terminating the evolution operation, and if not, re-performing the evolution operation.
S401, judging whether the particle i and the individual optimal particle have the same coordinate point, if so, carrying out the next step, otherwise, ending the mutation operation;
s402, if there is only one identical point, exchanging coordinates of a section from the point to the end point of the individual optimal particle with the particle i, as shown in fig. 4, if there are two or more identical points, randomly selecting two points, and exchanging a section between the two points of the individual optimal particle with the particle i, as shown in fig. 5.
And S5, obtaining the optimal particles of the population, and outputting the air routes of all unmanned aerial vehicles in the unmanned aerial vehicle cluster.
The previous description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the present invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the invention. Thus, the present invention is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims (6)
1. A multi-unmanned aerial vehicle collaborative path planning method is characterized by comprising the following steps:
s1, acquiring radar coordinates including a starting point position, an obstacle point and an existing radar in a map environment by the unmanned aerial vehicle cluster, and establishing a two-dimensional grid map of a battle environment;
s2, constructing an optimization target of the multiple unmanned aerial vehicles according to the two-dimensional grid map established in the S1 through the path length and the threat strength of each unmanned aerial vehicle:
number of steps f taken by i unmanned aerial vehiclesstepConsidering the constraint of the turning angle of the unmanned aerial vehicle as step, if the unmanned aerial vehicle has oblique flight in the path, a penalty f is imposedpenaltyAnd obtaining the path length expression of the single unmanned aerial vehicle:
fi＝fstep+fpenalty
the specific calculation formula of the path comprehensive weighting length of the multiple unmanned aerial vehicles is as follows:
wherein m is the number of unmanned aerial vehicles;
dividing a complete path from a starting point to a target point into three segments, and taking each threat point to 1/4, 2/4 and 3/4 of each segment for summation to obtain the threat intensity of the threat point j to the unmanned aerial vehicle i as represented by
Wherein t is the three parts of the path, it is the t-th segment of the unmanned aerial vehicle path,
and
for the fourth power of the distance from threat point j to the t-th paths 1/4, 2/4, and 3/4, K is the adjustment factor, and the threat strength of a single drone under the path is:
wherein N is the number of threat points, and the comprehensive threat intensity that the unmanned aerial vehicle received of definition is:
constructing an overall optimization target f of multiple unmanned aerial vehicles: f ═ w1*f1+w2*f2Wherein w is1And w2Weights for the total weighted path length and the total weighted threat strength, respectively;
s3, initializing the population by combining random generation path points and Q learning to obtain an initial population of each unmanned aerial vehicle path;
s4, optimizing the initial population obtained in the S3 by adopting an improved particle swarm optimization according to the optimization target constructed in the step S2;
and S5, obtaining the optimal particles of the population, and outputting the air routes of all unmanned aerial vehicles in the unmanned aerial vehicle cluster.
2. The method for planning the collaborative path for multiple unmanned aerial vehicles according to claim 1, wherein the S1 is specifically as follows:
the path points are represented in a form of coordinates in a two-dimensional grid graph, and the flight environment of the unmanned aerial vehicle is described as follows: { (x, y) | xmin≤x≤xmax,ymin≤y≤ymax}；xmin、xmax、yminAnd ymaxRepresented as the boundary of the drone flight;
dividing the flight environment into 20 × 20 grids with the same area, wherein each grid carries different parameter information of 0, -1, 1 and 2, and when the grid parameter is 0, the grid parameter represents the position information of a starting point; when the grid parameter is 1, the area is free of obstacles, when the grid parameter is-1, the area contains obstacles or threat sources, and when the grid parameter is 2, the area represents the position information of the end point.
3. The method for planning the collaborative path for the multiple unmanned aerial vehicles according to claim 1, wherein the specific method for obtaining the initial population in S3 is as follows:
firstly, obtaining a part of initial population by adopting a random path point generation mode, then obtaining a part of initial population by adopting Q learning, and constructing a feasible point reward function based on a sparse function in the Q learning:
wherein r is0Based on the prize value, rmaxTo take into account the maximum reward value at the time of threat, rthreatAs a threat coefficient, xUAVAnd yUAVCoordinates for unmanned aerial vehicles, xgoalAnd ygoalCoordinate of end point, xithreatAnd yithreatAs coordinates of each threat source, kgoalAnd kthreatTo adjust the control coefficient of the reward function, r is guaranteed during the parameter setting>0；
The action selection strategy adopted in Q learning is a Boltzmann distribution method, and the expression of the action selection strategy is as follows:
wherein ViRepresenting state s or state-action pairs (s, a)i) Any of the value functions of p (a)i| s) represents action a under state siSelecting probability, wherein T is a control parameter, and k is iteration times;
selecting a path as a candidate initial solution of a population in the unmanned aerial vehicle path solutions obtained by the Q learning algorithm, prescribing the number of path solutions of each unmanned aerial vehicle in different initial directions by using prior knowledge, combining the initial population obtained by the Q learning and the initial population obtained by randomly generating path points, namely the initial population of the particle swarm algorithm, wherein the form of the initial solution of the population is a coordinate.
4. The method for planning the collaborative path of the multiple unmanned aerial vehicles according to claim 1, wherein an initial population obtained in step S3 is optimized by using an improved particle swarm algorithm according to the optimization target constructed in step S2, and the update formula of the position and the speed of the particle swarm algorithm is as follows:
wherein xi,j(t)＝(xi,j(1),xi,j(2),Λ,xi,j(t)) is the position of the ith particle at the t-th iteration, pi,j,best(t)＝(pi,j,best(1),pi,j,best(2),Λ,pi,j,best(3) G) is the optimum value of the particle in t iterationsi,j,best(t)＝(gi,j,best(1),gi,j,best(2),Λ,gi,j,best(3) Is the global optimum of the particle in t iterations, w is the inertial weight, c1And c2Is a non-negative acceleration coefficient, f1As a function of the influence of the particle on itself, f2Is a particle xi,j(t) learning of the optimal value of its individual history, f3Is a particle xi,j(t) learning of its global historical optimum;
in an evolution operation F1Defined as a mutation operation, a particle i mutates itself with a probability of w, with a probability of 1-w remaining unchanged, expressed as:
the convergence of the algorithm is accelerated by adopting a linear time-varying inertia weight strategy:
w has a value of from wmaxLinearly decreasing to a final value wminT is the current iteration number of the algorithm, itermaxThe maximum number of iterations allowed for the algorithm to continue;
f is to be2Defined as particles i and c1Is interleaved with its individual optimum values by 1-c1The probability of (c) remains unchanged, which is expressed as:
f is to be3Defined as particles i and c2Is interleaved with its global optimum by 1-c2The probability of (c) remains unchanged, which is expressed as:
particle and global optimum particle cross operation step and F2Similarly;
and obtaining the optimal particles of each sub-population through the evolution operation, judging whether the optimal particles meet the cycle ending condition, if so, storing the optimal particles of each sub-population, terminating the evolution operation, and if not, re-performing the evolution operation.
5. The method for planning the collaborative path of multiple unmanned aerial vehicles according to claim 4, wherein the mutation operation comprises the following steps:
s41, randomly selecting a point in the path coordinates;
s42, calculating the distance D between each point i to be selected and the target pointiCalculating the probability P that the point i is selectedi：
Wherein k isiAn adjustment coefficient determined according to prior knowledge, in particular, when the point to be selected is an obstacle point, the probability is 0;
s43, randomly generating a number P of 0-1 if P1+P2+...Pi-1＜P≤P1+P2+...PiSelecting the point i as the next action point;
s44, adding the current grid into a search tabu table to avoid repeated selection, and replacing the current grid serial number with the next grid serial number;
s45, if all the coordinate points to be selected are obstacle points, canceling the variation operation of the point, and directly jumping to S41 to reselect the variation coordinates;
s46, judging whether the new action point is continuous with the original path, if not, inserting a new non-obstacle coordinate point at the discontinuous position, and judging whether the path is continuous according to the following formula;
δ＝max{abs(xi+1-xi),abs(yi+1-yi)}
wherein (x)i+1,yi+1) Is the coordinate point of the original path, (x)i,yi) If δ is 1, the path is continuous, and the mutation operation is finished; when delta>When 1, the path is discontinuous, new coordinates are directly inserted into the two coordinate points through an average value method, and the coordinate expression of the inserted points is as follows;
and judging whether the coordinates of the insertion point have obstacles or not, if so, sequentially selecting the coordinates in the order of the upper right part, the lower left part and the lower right part of the insertion point to judge whether the coordinates are the obstacles or not until the free coordinate point is selected, judging whether the inserted point is continuous with the original path point again, and if not, repeating the step S45 until the path is continuous.
6. The method of claim 4, wherein the particle i is expressed as c1The specific steps of the cross operation of the probability and the individual optimal particles are as follows:
s401, judging whether the particle i and the individual optimal particle have the same coordinate point, if so, carrying out the next step, otherwise, ending the mutation operation;
s402, if only one identical point exists, the coordinates of the interval from the point to the end point of the individual optimal particle are exchanged with the particle i, if two or more identical points exist, two points are randomly selected, and the interval between the two points of the individual optimal particle is exchanged with the particle i.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011303855.3A CN112327923B (en) | 2020-11-19 | 2020-11-19 | Multi-unmanned aerial vehicle collaborative path planning method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011303855.3A CN112327923B (en) | 2020-11-19 | 2020-11-19 | Multi-unmanned aerial vehicle collaborative path planning method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112327923A CN112327923A (en) | 2021-02-05 |
| CN112327923B true CN112327923B (en) | 2022-04-01 |
Family
ID=74321588
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011303855.3A Active CN112327923B (en) | 2020-11-19 | 2020-11-19 | Multi-unmanned aerial vehicle collaborative path planning method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112327923B (en) |
Families Citing this family (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113052720A (en) * | 2021-03-12 | 2021-06-29 | 山东联合电力产业发展有限公司 | MOEA/D evolution multi-objective optimization-based path navigation system and method |
| CN113342008B (en) * | 2021-05-21 | 2023-12-26 | 武汉理工大学 | Path planning system and method for sea-air cooperative underwater target tracking |
| CN113639750B (en) * | 2021-07-20 | 2023-05-26 | 中国地质大学（武汉） | Method and device for planning monitoring path of unmanned aerial vehicle in peak period in consideration of time-varying demand |
| CN114138005B (en) * | 2021-10-26 | 2023-10-13 | 南京航空航天大学 | Urban mass logistics unmanned aerial vehicle flight path planning method and device based on improved A-algorithm |
| CN114326792A (en) * | 2021-12-09 | 2022-04-12 | 中国电子科技集团公司第五十四研究所 | Unmanned aerial vehicle collaborative track control method based on mixed group intelligent algorithm |
| CN114492718A (en) * | 2022-01-25 | 2022-05-13 | 南方科技大学 | Flight decision generation method and device, computer equipment and storage medium |
| CN115308233B (en) * | 2022-09-29 | 2023-05-05 | 中国科学院过程工程研究所 | Radiographic imaging apparatus, radiographic imaging method, and storage medium |
| CN116088396B (en) * | 2023-03-06 | 2023-06-02 | 北京理工大学 | Unmanned cluster double-layer cooperative task control method and system |
| CN116542468B (en) * | 2023-05-06 | 2023-10-20 | 中国人民解放军32370部队 | Unmanned aerial vehicle cluster task planning method |
| CN117494919B (en) * | 2023-11-13 | 2024-04-19 | 广州力生机器人技术有限公司 | Path planning method and device based on multi-robot collaborative stacking operation |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105700549A (en) * | 2016-01-21 | 2016-06-22 | 北京理工大学 | Unmanned plane multi-track planning method based on sequence ecological niche PSO (particle swarm optimization) algorithm |
| CN107479575A (en) * | 2017-08-25 | 2017-12-15 | 中国地质大学（武汉） | A kind of multi-rotor unmanned aerial vehicle flight control method and system |
| CN108021144A (en) * | 2017-12-29 | 2018-05-11 | 中国地质大学（武汉） | A kind of unmanned aerial vehicle flight path planning and dynamic threats evade emulator |
| CN108153331A (en) * | 2017-12-29 | 2018-06-12 | 中国地质大学（武汉） | Unmanned plane dynamic threats bypassing method, equipment and storage device based on Kalman filter |
| CN109992000A (en) * | 2019-04-04 | 2019-07-09 | 北京航空航天大学 | A kind of multiple no-manned plane path collaborative planning method and device based on Hierarchical reinforcement learning |
| CN110544296A (en) * | 2019-07-31 | 2019-12-06 | 中国矿业大学 | intelligent planning method for three-dimensional global flight path of unmanned aerial vehicle in environment with uncertain enemy threat |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104036332A (en) * | 2014-06-17 | 2014-09-10 | 中国地质大学（武汉） | Average gradient value and improved multi-objective particle swarm optimization based robust optimization system |
| IL256934B (en) * | 2015-07-27 | 2022-07-01 | Genghiscomm Holdings Llc | Airborne relays in cooperative-mimo systems |
| CN106970648B (en) * | 2017-04-19 | 2019-05-14 | 北京航空航天大学 | Unmanned plane multi-goal path plans combined method for searching under the environment of city low latitude |
-
2020
- 2020-11-19 CN CN202011303855.3A patent/CN112327923B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105700549A (en) * | 2016-01-21 | 2016-06-22 | 北京理工大学 | Unmanned plane multi-track planning method based on sequence ecological niche PSO (particle swarm optimization) algorithm |
| CN107479575A (en) * | 2017-08-25 | 2017-12-15 | 中国地质大学（武汉） | A kind of multi-rotor unmanned aerial vehicle flight control method and system |
| CN108021144A (en) * | 2017-12-29 | 2018-05-11 | 中国地质大学（武汉） | A kind of unmanned aerial vehicle flight path planning and dynamic threats evade emulator |
| CN108153331A (en) * | 2017-12-29 | 2018-06-12 | 中国地质大学（武汉） | Unmanned plane dynamic threats bypassing method, equipment and storage device based on Kalman filter |
| CN109992000A (en) * | 2019-04-04 | 2019-07-09 | 北京航空航天大学 | A kind of multiple no-manned plane path collaborative planning method and device based on Hierarchical reinforcement learning |
| CN110544296A (en) * | 2019-07-31 | 2019-12-06 | 中国矿业大学 | intelligent planning method for three-dimensional global flight path of unmanned aerial vehicle in environment with uncertain enemy threat |
Non-Patent Citations (2)
| Title |
|---|
| Multiple UAVs Routes Planning Based on particle swarm optimization Algorithm;Wang Guoshi,等;《IEEE》;20101231;第1-5页 * |
| 基于不规则障碍物环境下无人机的改进几何路径规划算法;凌富园，等;《航空电子技术》;20191231;第40-46页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112327923A (en) | 2021-02-05 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112327923B (en) | Multi-unmanned aerial vehicle collaborative path planning method | |
| CN109945881B (en) | Mobile robot path planning method based on ant colony algorithm | |
| CN107515618B (en) | Heterogeneous unmanned aerial vehicle cooperative task allocation method considering time window | |
| CN107330560B (en) | Heterogeneous aircraft multi-task cooperative allocation method considering time sequence constraint | |
| CN113741508B (en) | Unmanned aerial vehicle task allocation method based on improved wolf pack algorithm | |
| CN110766254A (en) | Multi-unmanned aerial vehicle cooperative task allocation method based on improved genetic algorithm | |
| CN111337931A (en) | AUV target searching method | |
| CN113159432A (en) | Multi-agent path planning method based on deep reinforcement learning | |
| CN112733251B (en) | Collaborative flight path planning method for multiple unmanned aerial vehicles | |
| CN113836803A (en) | Improved wolf colony algorithm-based unmanned aerial vehicle cluster resource scheduling method | |
| CN115454067A (en) | Path planning method based on fusion algorithm | |
| Sallam et al. | A hybrid differential evolution with cuckoo search for solving resource constrained project scheduling problems | |
| Liu et al. | A hybrid mobile robot path planning scheme based on modified gray wolf optimization and situation assessment | |
| CN112926825B (en) | Multi-unmanned aerial vehicle task allocation method based on multi-target quantum krill group mechanism | |
| CN116164753B (en) | Mine unmanned vehicle path navigation method and device, computer equipment and storage medium | |
| Li et al. | Mobile Robot Navigation Algorithm Based on Ant Colony Algorithm with A* Heuristic Method | |
| Panda et al. | Autonomous mobile robot path planning using hybridization of particle swarm optimization and Tabu search | |
| Zhou et al. | A novel mission planning method for UAVs’ course of action | |
| CN115963731A (en) | Command control system network structure optimization method based on improved genetic algorithm | |
| CN113141272B (en) | Network security situation analysis method based on iteration optimization RBF neural network | |
| CN112148030B (en) | Underwater glider path planning method based on heuristic algorithm | |
| Li et al. | Improved genetic algorithm for multi-agent task allocation with time windows | |
| CN111796597A (en) | Robot multi-target path optimization method based on novel firefly group optimization | |
| Chittineni et al. | A parallel hybridization of clonal selection with shuffled frog leaping algorithm for solving global optimization problems (P-AISFLA) | |
| CN114510876B (en) | Multi-platform weapon target distribution method based on symbiotic search biological geography optimization |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |