CN117452827B - Under-actuated unmanned ship track tracking control method - Google Patents
Under-actuated unmanned ship track tracking control method Download PDFInfo
- Publication number
- CN117452827B CN117452827B CN202311758281.2A CN202311758281A CN117452827B CN 117452827 B CN117452827 B CN 117452827B CN 202311758281 A CN202311758281 A CN 202311758281A CN 117452827 B CN117452827 B CN 117452827B
- Authority
- CN
- China
- Prior art keywords
- representing
- unmanned ship
- neural network
- under
- indicating
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 99
- 238000013528 artificial neural network Methods 0.000 claims description 137
- 230000006870 function Effects 0.000 claims description 49
- 230000008569 process Effects 0.000 claims description 36
- 238000013016 damping Methods 0.000 claims description 18
- 230000002787 reinforcement Effects 0.000 claims description 13
- 230000003044 adaptive effect Effects 0.000 claims description 12
- 230000001133 acceleration Effects 0.000 claims description 9
- 238000001914 filtration Methods 0.000 claims description 6
- 230000001960 triggered effect Effects 0.000 claims description 5
- 238000005265 energy consumption Methods 0.000 abstract description 5
- 238000004891 communication Methods 0.000 abstract description 4
- 230000009467 reduction Effects 0.000 abstract description 3
- 230000000694 effects Effects 0.000 description 14
- 238000013461 design Methods 0.000 description 5
- 238000004088 simulation Methods 0.000 description 5
- 238000004422 calculation algorithm Methods 0.000 description 4
- 238000010586 diagram Methods 0.000 description 4
- 238000013178 mathematical model Methods 0.000 description 3
- 230000004044 response Effects 0.000 description 3
- 230000008878 coupling Effects 0.000 description 2
- 238000010168 coupling process Methods 0.000 description 2
- 238000005859 coupling reaction Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 210000002569 neuron Anatomy 0.000 description 2
- 238000005457 optimization Methods 0.000 description 2
- 241000282414 Homo sapiens Species 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000011217 control strategy Methods 0.000 description 1
- 230000003111 delayed effect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 238000011478 gradient descent method Methods 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 230000009897 systematic effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
- G05B13/00—Adaptive control systems, i.e. systems automatically adjusting themselves to have a performance which is optimum according to some preassigned criterion
- G05B13/02—Adaptive control systems, i.e. systems automatically adjusting themselves to have a performance which is optimum according to some preassigned criterion electric
- G05B13/04—Adaptive control systems, i.e. systems automatically adjusting themselves to have a performance which is optimum according to some preassigned criterion electric involving the use of models or simulators
- G05B13/042—Adaptive control systems, i.e. systems automatically adjusting themselves to have a performance which is optimum according to some preassigned criterion electric involving the use of models or simulators in which a parameter or coefficient is automatically adjusted to optimise the performance
Landscapes
- Engineering & Computer Science (AREA)
- Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Computation (AREA)
- Medical Informatics (AREA)
- Software Systems (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Automation & Control Theory (AREA)
- Control Of Position, Course, Altitude, Or Attitude Of Moving Bodies (AREA)
- Feedback Control In General (AREA)
Abstract
The invention discloses an under-actuated unmanned ship track tracking control method, which belongs to the technical field of track tracking and comprises the following steps: s1, constructing a kinematic equation and a kinetic equation of an underactuated unmanned ship; s2, determining an error variable set based on a kinematic equation and a kinetic equation of the under-actuated unmanned ship; s3, determining the virtual control rate of the underactuated unmanned ship according to the error variable set; s4, determining the actual control rate of the under-actuated unmanned ship according to the virtual control rate of the under-actuated unmanned ship; s5, constructing a fixed threshold event triggering strategy according to the actual control rate of the under-actuated unmanned ship; s6, completing track tracking by using a fixed threshold event trigger strategy. Based on the specificity of the under-actuated unmanned ship operation environment, the reduction of the communication resources and energy consumption of the system is considered.
Description
Technical Field
The invention belongs to the technical field of track tracking, and particularly relates to an under-actuated unmanned ship track tracking control method.
Background
Ocean occupies about 71% of the earth's surface area with abundant resources. With the continuous development of science and technology, the exploration and utilization of the ocean has attracted attention to human beings, and the phenomenon of using unmanned boats to replace manual operations for carrying out many important applications such as ocean rescue actions, resource exploration and environmental monitoring has become more and more common. Aiming at the practical problem of the under-actuated unmanned ship, the under-actuated unmanned ship stands out by virtue of the advantages of providing only control force of surge and yaw movement, having low energy consumption and simple structure. Therefore, the research on the underactuated unmanned ship has important practical significance.
The control of the existing under-actuated unmanned ship faces great challenges, and the stability of the control system is greatly influenced due to strong nonlinearity and coupling of the control method, uncertainty of model parameters, external disturbance and other uncertain factors, so that the control method applied to the under-actuated unmanned ship has the capabilities of resisting disturbance and reducing uncertain factors, and stability and control precision of the under-actuated unmanned ship in the actual operation process are ensured. At present, a plurality of control methods such as PID control, sliding mode control, fuzzy control, model prediction control and reinforcement learning and deep reinforcement learning control methods based on intelligent algorithms appear for the control of the under-actuated unmanned ship. PID control is a classical feedback control method based on adjusting the proportional, integral and derivative operations of the systematic error and bringing the system to the desired state by continuously adjusting the controller output, but the response of PID control is delayed and less robust. The sliding mode control has strong robustness, can be used for solving the problems of parameter uncertainty, external interference, nonlinear characteristics and the like of a system, realizes the rapid tracking and stable control of the system state by introducing a sliding mode surface, but has the buffeting phenomenon, and the effect of the sliding mode control depends on a mathematical model of the system to a great extent. The fuzzy control is a control method based on the fuzzy logic principle, and the uncertainty and the ambiguity of the system are processed by establishing a fuzzy rule, so that the control of the complex system is realized. But its fuzzy rule base design is subjective and requires a lot of expert knowledge and experience in the design of fuzzy rules. Model predictive control is an advanced control method based on a mathematical model, which optimizes the controller output and achieves optimal control of the system by establishing a mathematical model of the system and utilizing the model to predict. The model predictive control does not depend on a model of the system, so that the problem of uncertain model parameters is solved, but the model predictive control is complex in calculation and has poor response effect on the influence of external disturbance, parameter change or faults and other factors.
At present, a control method based on an intelligent algorithm is widely paid attention to by researchers. Optimal control is an optimization problem aimed at finding a control strategy that allows the system to achieve optimal performance under given constraints. It involves controlling the system over a period of time to achieve a particular goal. However, because of the difficulty in solving the HJB equation, numerical methods, approximation and optimization techniques are generally used, which are not only computationally complex but also computationally intensive. The optimal control output can be obtained by combining reinforcement learning with an optimal control method, and the unknown complex continuous function in the HJB equation is solved by a self-adaptive dynamic programming method, so that the computational complexity is greatly reduced, and the robustness is high. Based on the characteristics of the under-actuated unmanned ship in sea operation, the energy consumption and the communication resource greatly influence the working efficiency of the under-actuated unmanned ship, and in order to realize a control target by utilizing the existing communication resource, researchers design a plurality of transmission schemes to improve the utilization rate of network resources, such as a time trigger control scheme and an event trigger control scheme.
Through the analysis, the under-actuated unmanned ship track tracking control method based on reinforcement learning and event triggering optimal control is provided for the under-actuated unmanned ship track tracking problem, the control precision is realized and the energy consumption is saved by minimizing the cost function through a gradient descent method through the cost function consisting of system errors and control output, and the network resource utilization rate is further improved by adopting a fixed threshold event triggering strategy.
Disclosure of Invention
The invention provides an under-actuated unmanned ship track tracking control method, which aims to solve the problems that the existing control algorithm has poor processing effect on various uncertain factors such as strong nonlinearity and coupling, uncertainty of model parameters, external disturbance and the like, has poor robustness and control effect and is slow in response of a control system in the under-actuated unmanned ship track tracking problem.
The technical scheme of the invention is as follows: the under-actuated unmanned ship track tracking control method comprises the following steps:
s1, constructing a kinematic equation and a kinetic equation of an underactuated unmanned ship;
s2, determining an error variable set based on a kinematic equation and a kinetic equation of the under-actuated unmanned ship;
s3, determining the virtual control rate of the underactuated unmanned ship according to the error variable set;
s4, determining the actual control rate of the under-actuated unmanned ship according to the virtual control rate of the under-actuated unmanned ship;
s5, constructing a fixed threshold event triggering strategy according to the actual control rate of the under-actuated unmanned ship;
s6, completing track tracking by using a fixed threshold event trigger strategy.
Further, in S1, the expression of the kinematic equation of the under-actuated unmanned ship is:
in the method, in the process of the invention,representing the speed in the abscissa direction of an under-actuated unmanned ship in the earth's fixed coordinate system, +.>Representing the speed in the ordinate direction of the under-actuated unmanned ship in the earth's fixed coordinate system, +.>Representing angular velocity on an under-actuated unmanned boat in the earth's fixed coordinate system, +.>Representing the abscissa of an under-actuated unmanned ship in the earth's fixed coordinate system, < >>Representing the ordinate of the under-actuated unmanned ship in the earth's fixed coordinate system, < >>Representing yaw angle in the earth's fixed coordinate system, < >>Indicating the speed of the unmanned ship in the direction of heave, +.>Indicating the speed of the unmanned ship in the direction of swaying,/-or->Indicating the speed of the unmanned ship in the yaw direction.
Further, in S1, the expression of the kinetic equation of the under-actuated unmanned ship is:
in the method, in the process of the invention,indicating the acceleration of the unmanned ship in the direction of heave, +.>Indicating the acceleration of the unmanned ship in the direction of the sway +.>Representing the acceleration of the unmanned ship in the yaw direction, +.>Representing the inertia of the first unmanned ship containing additional mass,/->Representing the inertia of the second unmanned ship comprising additional mass,/->Representing the inertia of a third unmanned ship comprising additional mass,/->Represents an indeterminate first hydrodynamic damping term,/->Represents an indeterminate second hydrodynamic damping term,/->Represents an indeterminate third hydrodynamic damping term,/->Control input representing the direction of heave of the unmanned ship,/->Control input representing the heading direction of the unmanned ship,/->Indicating external disturbance to the first unmanned boat, < ->Indicating external disturbances to the second unmanned boat,indicating the external disturbance to the third unmanned ship, < >>Indicating the speed of the unmanned ship in the direction of heave, +.>Indicating the speed of the unmanned ship in the direction of swaying,/-or->Indicating the speed of the unmanned ship in the yaw direction.
Further, in S2, the set of error variables includes a first position error variable, a second position error variable, a third position error variable, and a fourth position error variable;
first position error variableThe expression of (2) is:
in the method, in the process of the invention,representing yaw angle in the earth's fixed coordinate system, < >>Representing a desired azimuth;
second position error variableThe expression of (2) is:
in the method, in the process of the invention,representing the speed of the unmanned ship in the yaw direction,/->Representing a first filtering virtual control;
third position error variableThe expression of (2) is:
in the method, in the process of the invention,indicating that unmanned ship is->Difference in axial direction, +.>Indicating that unmanned ship is->Difference in the axial direction;
fourth position error variableThe expression of (2) is:
in the method, in the process of the invention,indicating the speed of the unmanned ship in the direction of heave, +.>Representing a second filtering virtual control.
Further, S3 comprises the following sub-steps:
s31, constructing a virtual optimal control rate function;
s32, performing reinforcement learning on the virtual optimal control rate function by utilizing the critic neural network and the actor neural network, and determining the virtual control rate of the underactuated unmanned ship.
Further, in S31, the expression of the virtual optimal control rate function is:
in the method, in the process of the invention,virtual optimal control law representing the first controller, < ->Virtual optimal control law representing the second controller, < >>Representing a first positive constant, ">Representing a third positive constant, ">Representing a first position error variable,/->Representing a third position error variable,>indicating angular error changeQuantity (S)>A first weight vector estimator representing an actor neural network,>representing a first learning rate of the actor neural network, < >>A third learning rate representing an actor neural network, < >>A second weight vector estimator representing an actor neural network,>representing the radial basis function of the first RBF neural network used, +.>Representing the radial basis functions of the third RBF neural network.
Further, in S3, the update law expression of the critic neural network is:
in the method, in the process of the invention,first weight vector adaptive law representing critic neural network, +.>Third weight vector adaptive law representing critic neural network, +.>Representing a first positive constant, ">Representing a third positive constant, ">A first learning rate indicative of critic neural network, <' > and>third learning rate representing critic neural network, < ->Representing a first intermediate variable related to the angle error variable,/for>First weight vector estimator representing critic neural network, +.>Third weight vector estimator representing critic neural network, +.>A first weight vector estimator representing an actor neural network,>a third weight vector estimator representing an actor neural network,>representing a first position error variable,/->Representing a third position error variable,>angular velocity indicative of the desired trajectory, +.>Representing the radial basis functions of the first RBF neural network, and (2)>Representing the radial basis function of the third RBF neural network,>representing a first intermediate variable,/->Representing a third intermediate variable, ">Representing a third intermediate variable related to the angle error variable,/->Representing the angle error variable, ++>Representing a bounded function;
the update law expression of the actor neural network is:
in the method, in the process of the invention,first weight vector adaptive law representing actor neural network, +.>Third weight vector adaptive law representing actor neural network, +.>Representing a first learning rate of the actor neural network, < >>And a third learning rate representing an actor neural network.
Further, S4 comprises the sub-steps of:
s41, constructing an actual optimal control rate function;
s42, performing reinforcement learning on the actual optimal control rate function by utilizing the critic neural network and the actor neural network, and determining the actual control rate of the underactuated unmanned ship.
Further, in S41, the expression of the actual optimal control rate function is:
in the method, in the process of the invention,represents an optimal control law of the first controller, < ->Indicating the optimal control law of the second controller,representing an external disturbance on the second controller, and (2)>Indicating an external disturbance at the first controller, < >>Representing a second positive constant, ">Representing the fourth positive constant, ">Representing a second position error variable, ">Representing a fourth position error variable,>a first RBF neural network representing uncertainty for fitting the model,>a second RBF neural network representing uncertainty for the fitting model,>representing the radial basis function of the fifth RBF neural network,>representing the radial basis function of the sixth RBF neural network,>an optimal weight vector estimator representing a second RBF neural network,/a>An optimal weight vector estimator representing a fourth neural network,/->Representing the radial basis functions of the second RBF neural network, and (2)>Representing the radial basis function of the fourth RBF neural network,>representing a first approximation error,/->Representing a second approximation error.
Further, in S5, a fixed threshold event trigger strategy τ nt The expression of (t) is:
in the method, in the process of the invention,controller output indicative of time triggered strategy derived, +.>Indicating controller update time, +_>Indicate controller->Time of secondary update->Indicate controller->The time of the secondary update.
The beneficial effects of the invention are as follows:
(1) According to the invention, by defining error variables in the tracking process, solving an HJB equation and obtaining a self-adaptive law of neural network weight update by designing a cost function and using an actor-critic method in reinforcement learning, the optimal control of the neural network can be realized by introducing the neural network;
(2) Because the under-actuated unmanned ship has model uncertainty and unknown external disturbance, the invention improves the robustness and accuracy of a control system by using the RBF neural network to fit the model uncertainty and the external disturbance;
(3) Based on the specificity of the under-actuated unmanned ship operation environment, the reduction of the communication resources and energy consumption of the system is considered.
Drawings
FIG. 1 is a flow chart of an under-actuated unmanned boat trajectory tracking control method.
FIG. 2 is a trace plot of an under-actuated unmanned boat;
FIG. 3 is a tracking error plot of under-actuated unmanned boat position error;
FIG. 4 is a tracking error plot of an under-actuated unmanned boat angle error;
FIG. 5 is an under-actuated unmanned boat control input diagram;
FIG. 6 is a graph of the two-norm convergence effect for neural network weights Wa1-Wa 4;
FIG. 7 is a graph of the two-norm convergence effect for neural network weights Wc1-Wc 4;
FIG. 8 is a graph of the effect of two-norm convergence of the weights Wf1-Wf4 for a neural network
Fig. 9 is an event-triggered needle diagram.
Detailed Description
Embodiments of the present invention are further described below with reference to the accompanying drawings.
As shown in fig. 1, the invention provides an under-actuated unmanned ship track tracking control method, which comprises the following steps:
s1, constructing a kinematic equation and a kinetic equation of an underactuated unmanned ship;
s2, determining an error variable set based on a kinematic equation and a kinetic equation of the under-actuated unmanned ship;
s3, determining the virtual control rate of the underactuated unmanned ship according to the error variable set;
s4, determining the actual control rate of the under-actuated unmanned ship according to the virtual control rate of the under-actuated unmanned ship;
s5, constructing a fixed threshold event triggering strategy according to the actual control rate of the under-actuated unmanned ship;
s6, completing track tracking by using a fixed threshold event trigger strategy.
In the embodiment of the present invention, in S1, the expression of the kinematic equation of the under-actuated unmanned ship is:
in the method, in the process of the invention,representing the speed in the abscissa direction of an under-actuated unmanned ship in the earth's fixed coordinate system, +.>Representing the speed in the ordinate direction of the under-actuated unmanned ship in the earth's fixed coordinate system, +.>Representing angular velocity on an under-actuated unmanned boat in the earth's fixed coordinate system, +.>Representing the abscissa of an under-actuated unmanned ship in the earth's fixed coordinate system, < >>Representing the ordinate of the under-actuated unmanned ship in the earth's fixed coordinate system, < >>Representing yaw angle in the earth's fixed coordinate system, < >>Indicating the speed of the unmanned ship in the direction of heave, +.>Indicating the speed of the unmanned ship in the direction of swaying,/-or->Indicating the speed of the unmanned ship in the yaw direction.
In the embodiment of the present invention, in S1, the expression of the kinetic equation of the under-actuated unmanned ship is:
in the method, in the process of the invention,indicating the acceleration of the unmanned ship in the direction of heave, +.>Indicating the acceleration of the unmanned ship in the direction of the sway +.>Representing the acceleration of the unmanned ship in the yaw direction, +.>Representing the inertia of the first unmanned ship containing additional mass,/->Representing the inertia of the second unmanned ship comprising additional mass,/->Representing the inertia of a third unmanned ship comprising additional mass,/->Represents an indeterminate first hydrodynamic damping term,/->Represents an indeterminate second hydrodynamic damping term,/->Represents an indeterminate third hydrodynamic damping term,/->Control input representing the direction of heave of the unmanned ship,/->Control input representing the heading direction of the unmanned ship,/->Indicating external disturbance to the first unmanned boat, < ->Indicating external disturbances to the second unmanned boat,indicating the external disturbance to the third unmanned ship, < >>Indicating the speed of the unmanned ship in the direction of heave, +.>Indicating the speed of the unmanned ship in the direction of swaying,/-or->Indicating the speed of the unmanned ship in the yaw direction.
In the embodiment of the present invention, in S2, the set of error variables includes a first position error variable, a second position error variable, a third position error variable, and a fourth position error variable;
first position error variableThe expression of (2) is:
in the method, in the process of the invention,representing yaw angle in the earth's fixed coordinate system, < >>Representing a desired azimuth;
second position error variableThe expression of (2) is:
in the method, in the process of the invention,representing the speed of the unmanned ship in the yaw direction,/->Representing a first filtering virtual control;
third position error variableThe expression of (2) is:
in the method, in the process of the invention,indicating that unmanned ship is->Difference in axial direction, +.>Indicating that unmanned ship is->Difference in the axial direction;
fourth position error variableThe expression of (2) is:
in the method, in the process of the invention,indicating the speed of the unmanned ship in the direction of heave, +.>Representing a second filtering virtual control.
In an embodiment of the present invention, S3 comprises the following sub-steps:
s31, constructing a virtual optimal control rate function;
s32, performing reinforcement learning on the virtual optimal control rate function by utilizing the critic neural network and the actor neural network, and determining the virtual control rate of the underactuated unmanned ship.
In the embodiment of the present invention, in S31, the expression of the virtual optimal control rate function is:
in the method, in the process of the invention,virtual optimal control law representing the first controller, < ->Virtual optimal control law representing the second controller, < >>Representing a first positive constant, ">Representing a third positive constant, ">Representing a first position error variable,/->Representing a third position error variable,>representing the angle error variable, ++>A first weight vector estimator representing an actor neural network,>representing a first learning rate of the actor neural network, < >>A third learning rate representing an actor neural network, < >>A second weight vector estimator representing an actor neural network,>representing the radial basis function of the first RBF neural network used, +.>Representing the radial basis functions of the third RBF neural network.
In the embodiment of the present invention, in S3, the update law expression of the critic neural network is:
in the method, in the process of the invention,first weight vector adaptive law representing critic neural network, +.>Third weight vector adaptive law representing critic neural network, +.>Representing a first positive constant, ">Representing a third positive constant, ">A first learning rate indicative of critic neural network, <' > and>third learning rate representing critic neural network, < ->Representing a first intermediate variable related to the angle error variable,/for>First weight vector estimator representing critic neural network, +.>Third weight vector estimator representing critic neural network, +.>A first weight vector estimator representing an actor neural network,>a third weight vector estimator representing an actor neural network,>representing a first position error variable,/->Representing a third position error variable,>angular velocity indicative of the desired trajectory, +.>Representing the radial basis functions of the first RBF neural network, and (2)>Representing the radial basis function of the third RBF neural network,>representing a first intermediate variable,/->Representing a third intermediate variable, ">Representing a third intermediate variable related to the angle error variable,/->Representing the angle error variable, ++>Representing a bounded function;
the update law expression of the actor neural network is:
in the method, in the process of the invention,first weight vector adaptive law representing actor neural network, +.>Third weight vector adaptive law representing actor neural network, +.>Representing a first learning rate of the actor neural network, < >>And a third learning rate representing an actor neural network.
In an embodiment of the present invention, S4 comprises the following sub-steps:
s41, constructing an actual optimal control rate function;
s42, performing reinforcement learning on the actual optimal control rate function by utilizing the critic neural network and the actor neural network, and determining the actual control rate of the underactuated unmanned ship.
In the embodiment of the present invention, in S41, the expression of the actual optimal control rate function is:
in the method, in the process of the invention,represents an optimal control law of the first controller, < ->Indicating the optimal control law of the second controller,representing an external disturbance on the second controller, and (2)>Indicating an external disturbance at the first controller, < >>Representing a second positive constant, ">Representing the fourth positive constant, ">Representing a second position error variable, ">Representing a fourth position error variable,>a first RBF neural network representing uncertainty for fitting the model,>a second RBF neural network representing uncertainty for the fitting model,>representing the radial basis function of the fifth RBF neural network,>representing the radial basis function of the sixth RBF neural network,>an optimal weight vector estimator representing a second RBF neural network,/a>An optimal weight vector estimator representing a fourth neural network,/->Representing the radial basis functions of the second RBF neural network, and (2)>Representing the radial basis function of the fourth RBF neural network,>representing a first approximation error,/->Representing a second approximation error.
In the embodiment of the present invention, in S5, a fixed threshold event trigger policy τ nt The expression of (t) is:
in the method, in the process of the invention,controller output indicative of time triggered strategy derived, +.>Indicating controller update time, +_>Indicate controller->Time of secondary update->Indicate controller->The time of the secondary update.
In order to verify the effectiveness of the invention, simulation experiments were performed, the specific experiments are as follows:
simulation experiments are carried out on the under-actuated unmanned ship through the selected model parameters, and the effectiveness of the provided under-actuated unmanned ship track tracking control method based on reinforcement learning and event triggering optimal control is verified. The model parameters are shown in Table 1, the controllerThe design parameters are shown in table 2 and the neural network design parameters are shown in table 3. In the table 1, the contents of the components,representing a first unmanned boat inertia containing additional mass,representing a second unmanned boat inertia containing additional mass,representing a third unmanned boat inertia containing additional mass,representing a first system parameter in a first hydrodynamic damping term,representing a first system parameter in a second hydrodynamic damping term,representing a first system parameter in a third hydrodynamic damping term,a first positive constant is indicated and is indicated,indicating the speed of the desired reference trajectory in the heave direction,representing a second system parameter in the first hydrodynamic damping term,representing a second system parameter in a second hydrodynamic damping term,representing a second system parameter in a third hydrodynamic damping term,representing a third system parameter in the first hydrodynamic damping term,representing a third system parameter in the second hydrodynamic damping term,representing a third system parameter in a third hydrodynamic damping term,indicating the speed of the desired reference trajectory in the yaw direction. In the table 2 of the description of the present invention,representing a first learning rate of the actor neural network,a second learning rate representing an actor neural network,a third learning rate representing an actor neural network,a fourth learning rate representing an actor neural network,a first positive constant is indicated and is indicated,a third positive constant is indicated and is used to indicate,a first filter parameter is indicated and a second filter parameter is indicated,a first learning rate representing a critic neural network,a second learning rate representing a critic neural network,a third learning rate representing a critic neural network,a fourth learning rate representing a critic neural network,a second positive constant is indicated and is used to represent,a fourth positive constant is indicated and is indicated,representing the second filter parameters. In the table 3, the contents of the components,a central value representing a radial basis of the first RBF neural network,representing the central value of the radial basis of the second RBF neural network,represents the central value of the radial basis of the third RBF neural network,the center value of the radial basis of the fourth RBF neural network,a central value representing a radial basis of the fifth RBF neural network,the center value of the radial basis of the sixth RBF neural network,representing the width of the center point of the radial basis of the first RBF neural network,representing the center point width of the radial basis of the second RBF neural network,representing the center point width of the radial basis of the third RBF neural network,representing the center point width of the radial basis of the fourth RBF neural network,the center point width of the radial basis of the fifth RBF neural network is represented,the width of the center point of the radial basis of the sixth RBF neural network is represented, A represents the first scheme of RBF neural network neuron node distribution, and B represents the second scheme of RBF neural network neuron node distribution.
TABLE 1
TABLE 2
TABLE 3 Table 3
The initial position of the underactuated unmanned ship is selected as in the simulation experimentAnd the initial speeds are all 0. The desired trajectory of the under-actuated unmanned boat is set to +.>And->And the initial conditions were all 0. In Table 3->The representative neural network has 72 nodes with a range ofSimilarly->All initial weights of the neural network are 0.
The simulation results are shown in fig. 2-9. FIG. 2 shows a track diagram of the controller in tracking the desired track, which can intuitively see that the unmanned ship is reachingWhen the unmanned ship is used, the unmanned ship can be well fitted with the set expected track, and the proposed controller has good tracking effect and high robustness and accuracy. Fig. 3 shows the effect diagram of the position error variable of the controller with time in the tracking process, and it can be seen that the position error converges at about 18s and the error value converges to about 0, which indicates that the controller designed by the invention has a good tracking effect. Fig. 4 shows the angular error of the under-actuated unmanned boat during the trajectory tracking, the error converged at around 18s and the error value converged to 0.4. Fig. 5 shows the control outputs of the control algorithm according to the present invention, which are respectively represented as the output of the first controller and the output of the second controller, and it is not difficult to see that the values thereof all converge near 18s and the robustness is good. Fig. 6-8 show the two-norm convergence effect of the weight estimation values of the neural network used in the present invention, and as shown in fig. 6-8, the weight estimation values of the neural network used in the present invention all converge, which indicates that the fitting effect is better and stable. FIG. 9 shows an event-triggered needle graph of the present invention, in which the ordinate indicates the time each control input signal remains, and the control signal can be seenAndall have different degrees of retaining effect, control signalThe updating frequency is fast, the maintaining time is short, and the average value is about 0.15 s; control signalThe general updating frequency is slow, the maintaining time is longer, and the average value is about 1 s; since the desired trajectory is selected to be circular, the overall resource is reduced, effectively controlling the signalIs superior to。
The experimental result well shows that the proposed under-actuated unmanned ship track tracking control method based on reinforcement learning and event triggering optimal control has good robustness and accuracy, high convergence rate and good neural network fitting effect, and the resource consumption reduced by the proposed event triggering is shown in table 4.
From the simulation data Table 4, the control output of the controller can be obtainedCompared to the +.f. of unused event trigger strategy>76.03% less, the control output of the controller +.>Compared to the +.f. of unused event trigger strategy>The reduction is 91.45 percent.
TABLE 4 Table 4
Those of ordinary skill in the art will recognize that the embodiments described herein are for the purpose of aiding the reader in understanding the principles of the present invention and should be understood that the scope of the invention is not limited to such specific statements and embodiments. Those of ordinary skill in the art can make various other specific modifications and combinations from the teachings of the present disclosure without departing from the spirit thereof, and such modifications and combinations remain within the scope of the present disclosure.
Claims (7)
1. The under-actuated unmanned ship track tracking control method is characterized by comprising the following steps of:
s1, constructing a kinematic equation and a kinetic equation of an underactuated unmanned ship;
s2, determining an error variable set based on a kinematic equation and a kinetic equation of the under-actuated unmanned ship;
s3, determining the virtual control rate of the underactuated unmanned ship according to the error variable set;
s4, determining the actual control rate of the under-actuated unmanned ship according to the virtual control rate of the under-actuated unmanned ship;
s5, constructing a fixed threshold event triggering strategy according to the actual control rate of the under-actuated unmanned ship;
s6, completing track tracking by using a fixed threshold event trigger strategy;
the step S3 comprises the following substeps:
s31, constructing a virtual optimal control rate function;
s32, performing reinforcement learning on the virtual optimal control rate function by utilizing a critic neural network and an actor neural network, and determining the virtual control rate of the underactuated unmanned ship;
in S31, the expression of the virtual optimal control law function is:
；
in the method, in the process of the invention,virtual optimal control law representing the first controller, < ->Representing a virtual optimal control law for the second controller,representing a first positive constant, ">Representing a third positive constant, ">Representing a first position error variable,/->Representing a third position error variable,>representing the angle error variable, ++>A first weight vector estimator representing an actor neural network,>representing a first learning rate of the actor neural network, < >>A third learning rate representing an actor neural network, < >>A second weight vector estimator representing an actor neural network,>representing the radial basis function of the first RBF neural network used, +.>Representing a radial basis function of a third RBF neural network;
in the step S3, the update law expression of the critic neural network is:
；
in the method, in the process of the invention,first weight vector adaptive law representing critic neural network, +.>Third weight vector adaptive law representing critic neural network, +.>Representing a first positive constant, ">Representing a third positive constant, ">A first learning rate indicative of critic neural network, <' > and>third learning rate representing critic neural network, < ->Representing a first intermediate variable related to the angle error variable,/for>First weight vector estimator representing critic neural network, +.>Third weight vector estimator representing critic neural network, +.>A first weight vector estimator representing an actor neural network,>a third weight vector estimator representing an actor neural network,>representing a first position error variable,/->A third position error variable is represented and,angular velocity indicative of the desired trajectory, +.>Representing the radial basis functions of the first RBF neural network, and (2)>Representing the radial basis function of the third RBF neural network,>representing a first intermediate variable,/->Representing a third intermediate variable, ">Representing a third intermediate variable related to the angle error variable,/->The variable of the angle error is indicated,/>representing a bounded function;
the update law expression of the actor neural network is as follows:
；
in the method, in the process of the invention,first weight vector adaptive law representing actor neural network, +.>Third weight vector adaptive law representing actor neural network, +.>Representing a first learning rate of the actor neural network, < >>And a third learning rate representing an actor neural network.
2. The method for tracking and controlling the trajectory of the under-actuated unmanned ship according to claim 1, wherein in S1, the expression of the kinematic equation of the under-actuated unmanned ship is:
；
in the method, in the process of the invention,representing the speed in the abscissa direction of an under-actuated unmanned ship in the earth's fixed coordinate system, +.>Representing earth's fixed sittingSpeed in ordinate direction of under-actuated unmanned ship in standard system, +.>Representing angular velocity on an under-actuated unmanned boat in the earth's fixed coordinate system, +.>Representing the abscissa of an under-actuated unmanned ship in the earth's fixed coordinate system, < >>Representing the ordinate of the under-actuated unmanned ship in the earth's fixed coordinate system, < >>Representing yaw angle in the earth's fixed coordinate system, < >>Indicating the speed of the unmanned ship in the direction of heave, +.>Indicating the speed of the unmanned ship in the direction of swaying,/-or->Indicating the speed of the unmanned ship in the yaw direction.
3. The method for tracking and controlling the track of the unmanned under-actuated vehicle according to claim 1, wherein in S1, the expression of the kinetic equation of the unmanned under-actuated vehicle is:
；
in the method, in the process of the invention,indicating the acceleration of the unmanned ship in the direction of heave, +.>Indicating the acceleration of the unmanned ship in the direction of the sway +.>Representing the acceleration of the unmanned ship in the yaw direction, +.>Representing the inertia of the first unmanned ship containing additional mass,/->Representing the inertia of the second unmanned ship comprising additional mass,/->Representing the inertia of a third unmanned ship comprising additional mass,/->Represents an indeterminate first hydrodynamic damping term,/->Represents an indeterminate second hydrodynamic damping term,/->Represents an indeterminate third hydrodynamic damping term,/->Control input representing the direction of heave of the unmanned ship,/->Control input representing the heading direction of the unmanned ship,/->Indicating external disturbance to the first unmanned boat, < ->Indicating the external disturbance to the second unmanned boat, < >>Indicating the external disturbance to the third unmanned ship, < >>Indicating the speed of the unmanned ship in the direction of heave, +.>Indicating the speed of the unmanned ship in the direction of swaying,/-or->Indicating the speed of the unmanned ship in the yaw direction.
4. The method of claim 1, wherein in S2, the set of error variables includes a first position error variable, a second position error variable, a third position error variable, and a fourth position error variable;
the first position error variableThe expression of (2) is:
；
in the method, in the process of the invention,representing yaw angle in the earth's fixed coordinate system, < >>Representing a desired azimuth;
the second position error variableThe expression of (2) is:
；
in the method, in the process of the invention,representing the speed of the unmanned ship in the yaw direction,/->Representing a first filtering virtual control;
the third position error variableThe expression of (2) is:
；
in the method, in the process of the invention,indicating that unmanned ship is->Difference in axial direction, +.>Indicating that unmanned ship is->Difference in the axial direction;
the fourth position error variableThe expression of (2) is:
；
in the method, in the process of the invention,indicating the speed of the unmanned ship in the direction of heave, +.>Representing a second filtering virtual control.
5. The under-actuated unmanned ship track following control method according to claim 1, wherein S4 comprises the sub-steps of:
s41, constructing an actual optimal control rate function;
s42, performing reinforcement learning on the actual optimal control rate function by utilizing the critic neural network and the actor neural network, and determining the actual control rate of the underactuated unmanned ship.
6. The method for tracking and controlling the trajectory of the unmanned aerial vehicle according to claim 5, wherein in S41, the expression of the actual optimal control rate function is:
；
in the method, in the process of the invention,represents an optimal control law of the first controller, < ->Indicating the optimal control law of the second controller, < >>Representing an external disturbance on the second controller, and (2)>Indicating an external disturbance at the first controller, < >>A second positive constant is indicated and is used to represent,representing the fourth positive constant, ">Representing a second position error variable, ">Representing a fourth position error variable,>a first RBF neural network representing uncertainty for fitting the model,>a second RBF neural network representing uncertainty for the fitting model,>representing the radial basis function of the fifth RBF neural network,>representing the radial basis function of the sixth RBF neural network,>an optimal weight vector estimator representing a second RBF neural network,/a>An optimal weight vector estimator representing a fourth neural network,/->Representing the secondRadial basis function of RBF neural network, +.>Representing the radial basis function of the fourth RBF neural network,>representing a first approximation error,/->Representing a second approximation error.
7. The method for tracking and controlling the trajectory of an under-actuated unmanned ship according to claim 1, wherein in S5, a fixed threshold event trigger strategy τ is used nt The expression of (t) is:
；
in the method, in the process of the invention,controller output indicative of time triggered strategy derived, +.>Indicating controller update time, +_>Indicate controller->Time of secondary update->Indicate controller->The time of the secondary update.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311758281.2A CN117452827B (en) | 2023-12-20 | 2023-12-20 | Under-actuated unmanned ship track tracking control method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311758281.2A CN117452827B (en) | 2023-12-20 | 2023-12-20 | Under-actuated unmanned ship track tracking control method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN117452827A CN117452827A (en) | 2024-01-26 |
| CN117452827B true CN117452827B (en) | 2024-04-05 |
Family
ID=89589418
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311758281.2A Active CN117452827B (en) | 2023-12-20 | 2023-12-20 | Under-actuated unmanned ship track tracking control method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117452827B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117784619B (en) * | 2024-02-26 | 2024-05-31 | 广东海洋大学 | Under-actuated unmanned ship fault-tolerant control method based on zero and differential game |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108008628A (en) * | 2017-11-17 | 2018-05-08 | 华南理工大学 | A kind of default capabilities control method of uncertain drive lacking unmanned boat system |
| CN108319138A (en) * | 2018-01-29 | 2018-07-24 | 哈尔滨工程大学 | A kind of sliding formwork of drive lacking unmanned boat-contragradience double loop Trajectory Tracking Control method |
| CN113821030A (en) * | 2021-09-08 | 2021-12-21 | 哈尔滨工程大学 | Fixed time trajectory tracking control method of under-actuated unmanned ship |
| CN115167481A (en) * | 2022-08-27 | 2022-10-11 | 华中科技大学 | Under-actuated unmanned ship preset performance path tracking control method and system |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111694365B (en) * | 2020-07-01 | 2021-04-20 | 武汉理工大学 | Unmanned ship formation path tracking method based on deep reinforcement learning |
-
2023
- 2023-12-20 CN CN202311758281.2A patent/CN117452827B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108008628A (en) * | 2017-11-17 | 2018-05-08 | 华南理工大学 | A kind of default capabilities control method of uncertain drive lacking unmanned boat system |
| CN108319138A (en) * | 2018-01-29 | 2018-07-24 | 哈尔滨工程大学 | A kind of sliding formwork of drive lacking unmanned boat-contragradience double loop Trajectory Tracking Control method |
| CN113821030A (en) * | 2021-09-08 | 2021-12-21 | 哈尔滨工程大学 | Fixed time trajectory tracking control method of under-actuated unmanned ship |
| CN115167481A (en) * | 2022-08-27 | 2022-10-11 | 华中科技大学 | Under-actuated unmanned ship preset performance path tracking control method and system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN117452827A (en) | 2024-01-26 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108803321B (en) | Autonomous underwater vehicle track tracking control method based on deep reinforcement learning | |
| CN117452827B (en) | Under-actuated unmanned ship track tracking control method | |
| CN112612209B (en) | Full-drive ship track tracking control method and system based on instruction filtering neural network controller | |
| CN104898688A (en) | UUV four degree-of-freedom dynamic positioning adaptive anti-interference sliding mode control system and control method | |
| CN111948937B (en) | Multi-gradient recursive reinforcement learning fuzzy control method and system of multi-agent system | |
| CN114115262B (en) | Multi-AUV actuator saturation cooperative formation control system and method based on azimuth information | |
| Su et al. | Fixed-time formation of AUVs with disturbance via event-triggered control | |
| CN111176122A (en) | Underwater robot parameter self-adaptive backstepping control method based on double BP neural network Q learning technology | |
| CN114442640A (en) | Track tracking control method for unmanned surface vehicle | |
| Gao et al. | Online optimal control for dynamic positioning of vessels via time-based adaptive dynamic programming | |
| Fu et al. | A cross‐coupling control approach for coordinated formation of surface vessels with uncertain disturbances | |
| CN116360470A (en) | Multi-underwater helicopter cooperative formation control method | |
| CN113110512A (en) | Benthonic AUV self-adaptive trajectory tracking control method for weakening unknown interference and buffeting influence | |
| CN115903820A (en) | Multi-unmanned-boat pursuit and escape game control method | |
| CN116224798A (en) | Autonomous underwater vehicle track tracking control method based on event triggering | |
| CN110703792B (en) | Underwater robot attitude control method based on reinforcement learning | |
| Li et al. | Adaptive fixed-time fuzzy formation control for multiple AUV systems considering time-varying tracking error constraints and asymmetric actuator saturation | |
| Mu et al. | Design of robust adaptive course controller for unmanned surface vehicle with input saturation | |
| Hou et al. | Robust nonlinear model predictive control for ship dynamic positioning using Laguerre function | |
| CN117784619B (en) | Under-actuated unmanned ship fault-tolerant control method based on zero and differential game | |
| CN112859891A (en) | AUV course angle control method for optimizing self-adaptive sliding mode control parameters based on particle swarm optimization | |
| CN117150901B (en) | Design method of dynamic positioning ship position observer capable of saving communication resources | |
| Jianfei et al. | Nonsingular fast terminal sliding mode trajectory tracking control of surface vessel with disturbances | |
| Xu et al. | Neural Network Control Using Composite Learning for USVs with Output Error Constraints | |
| Gu et al. | Path-guided Collision-free Formation Guidance Law for a Fleet of Under-actuated Autonomous Surface Vehicles |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |