CN116977902A - Target tracking method and system for on-board photoelectric stabilized platform of coastal defense - Google Patents
Target tracking method and system for on-board photoelectric stabilized platform of coastal defense Download PDFInfo
- Publication number
- CN116977902A CN116977902A CN202311018578.5A CN202311018578A CN116977902A CN 116977902 A CN116977902 A CN 116977902A CN 202311018578 A CN202311018578 A CN 202311018578A CN 116977902 A CN116977902 A CN 116977902A
- Authority
- CN
- China
- Prior art keywords
- target
- tracked
- tracking
- template
- control signal
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 59
- 230000007123 defense Effects 0.000 title claims abstract description 31
- 230000000007 visual effect Effects 0.000 claims abstract description 47
- 238000000605 extraction Methods 0.000 claims abstract description 43
- 238000004422 calculation algorithm Methods 0.000 claims abstract description 15
- 238000007781 pre-processing Methods 0.000 claims abstract description 9
- 238000004364 calculation method Methods 0.000 claims description 27
- 238000013528 artificial neural network Methods 0.000 claims description 23
- 238000001514 detection method Methods 0.000 claims description 20
- 230000008569 process Effects 0.000 claims description 19
- 230000008859 change Effects 0.000 claims description 18
- 230000006870 function Effects 0.000 claims description 12
- 238000012545 processing Methods 0.000 claims description 10
- 238000011217 control strategy Methods 0.000 claims description 7
- 230000004044 response Effects 0.000 claims description 7
- 230000003247 decreasing effect Effects 0.000 claims description 6
- 238000003708 edge detection Methods 0.000 claims description 6
- 238000012216 screening Methods 0.000 claims description 6
- 210000001258 synovial membrane Anatomy 0.000 claims description 6
- 238000012937 correction Methods 0.000 claims description 3
- 230000007274 generation of a signal involved in cell-cell signaling Effects 0.000 claims description 3
- 230000005764 inhibitory process Effects 0.000 claims description 2
- 238000010586 diagram Methods 0.000 description 13
- 230000000694 effects Effects 0.000 description 8
- 238000012549 training Methods 0.000 description 8
- 230000006641 stabilisation Effects 0.000 description 7
- 238000011105 stabilization Methods 0.000 description 7
- 238000002474 experimental method Methods 0.000 description 5
- 238000012544 monitoring process Methods 0.000 description 4
- 238000007493 shaping process Methods 0.000 description 4
- 238000013461 design Methods 0.000 description 3
- 238000011160 research Methods 0.000 description 3
- 230000000087 stabilizing effect Effects 0.000 description 3
- 238000009825 accumulation Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000001914 filtration Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012360 testing method Methods 0.000 description 2
- 239000013598 vector Substances 0.000 description 2
- 241000764238 Isis Species 0.000 description 1
- 230000003044 adaptive effect Effects 0.000 description 1
- 230000003321 amplification Effects 0.000 description 1
- 230000006399 behavior Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000033228 biological regulation Effects 0.000 description 1
- 238000012512 characterization method Methods 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000004069 differentiation Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000002955 isolation Methods 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 238000012806 monitoring device Methods 0.000 description 1
- 238000003199 nucleic acid amplification method Methods 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 230000005693 optoelectronics Effects 0.000 description 1
- 238000004451 qualitative analysis Methods 0.000 description 1
- 238000005096 rolling process Methods 0.000 description 1
- 238000004088 simulation Methods 0.000 description 1
- 230000001629 suppression Effects 0.000 description 1
- 238000012795 verification Methods 0.000 description 1
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/40—Scenes; Scene-specific elements in video content
- G06V20/48—Matching video sequences
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02A—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
- Y02A10/00—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE at coastal zones; at river basins
- Y02A10/40—Controlling or monitoring, e.g. of flood or hurricane; Forecasting, e.g. risk assessment or mapping
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- Software Systems (AREA)
- Computing Systems (AREA)
- Databases & Information Systems (AREA)
- Medical Informatics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Life Sciences & Earth Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Molecular Biology (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Closed-Circuit Television Systems (AREA)
Abstract
The invention provides a target tracking method and a target tracking system for an on-board photoelectric stabilized platform for coastal defense, and belongs to the technical field of photoelectric stabilized platform tracking. Firstly, searching a target to be tracked and obtaining a target image to be tracked; secondly, carrying out preprocessing operation and feature extraction operation on the image of the target to be tracked in sequence to obtain a feature extraction template of the target to be tracked; then tracking the target feature extraction template to be tracked by adopting a target tracking strategy to obtain the predicted position of the target to be tracked; then calculating the off-target quantity of the predicted position and the visual axis pointing position of the target to be tracked; and finally, generating a servo control signal according to the off-target quantity, and correcting the visual axis orientation in real time according to the servo control signal. According to the invention, the current position of the moving target is obtained by utilizing a target tracking algorithm to calculate the off-target quantity, the motor is servo-controlled by adopting a servo control algorithm to drive the load to correct the visual axis to point to the position capable of tracking the target in real time, and the target accurate tracking of the photoelectric stabilized platform is realized.
Description
Technical Field
The invention belongs to the technical field of photoelectric stabilized platform tracking, and particularly relates to a method and a system for tracking an on-board photoelectric stabilized platform target of an offshore defense.
Background
With the high-speed development of modern informatization technology, the photoelectric stabilized platform is gradually applied to various aspects such as military field, civil field and industrial field, especially in the military field, such as guided control system of missile, unmanned plane reconnaissance system and radar antenna control system, so to speak, the continuous progress of photoelectric technology has changed the form of military warfare. Because the regional situation of the border coastal defense area of China is numerous and miscellaneous and the climate is bad, fighters who fight on the border coastal defense for a long time are hard to work and often face life hazards. In recent years, the photoelectric stabilized platform is gradually applied to border line defense work, can lighten the workload of workers, improve the safety of the workers and efficiently finish monitoring work of suspicious people and ships near the border line.
At present, most of existing photoelectric platforms adopt related filtering algorithms, siamRPN, daSiamRPN and the like as target tracking algorithms, and the control performance, tracking effect and stability of the platform system cannot be well considered; the photoelectric stabilized platform designed for the national border defense engineering takes a border defense patrol car as a carrier, and adopts an optical-mechanical integrated monitoring device designed for detecting and monitoring the condition of a sea target as a target, which is used for tracking the illegal behaviors of suspected ships and recording videos to provide effective evidence, and particularly in the monitoring task of nearby suspicious targets, the target tracking and servo control of the photoelectric stabilized platform have high requirements, so that a scientific border defense vehicle-mounted photoelectric stabilized platform target tracking method needs to be designed, and the high-standard, high-quality and high-precision monitoring tracking of border defense military is realized, thereby achieving the aim of maintaining border safety.
Disclosure of Invention
Based on the technical problems, the invention provides a target tracking method and a target tracking system for an on-board photoelectric stabilized platform for coastal defense, and designs an update tracking algorithm and a fuzzy sliding mode variable structure controller based on a DaSiamRPN improved model, which give consideration to the control performance, the tracking effect and the stabilizing capability of the platform system.
The invention provides a target tracking method for an on-board photoelectric stabilized platform for coastal defense, which comprises the following steps:
step S1: searching a target to be tracked and obtaining a target image to be tracked;
step S2: sequentially performing preprocessing operation and feature extraction operation on the image of the target to be tracked to obtain a feature extraction template of the target to be tracked;
step S3: tracking the target feature extraction template to be tracked by adopting a target tracking strategy to obtain a predicted position of the target to be tracked;
step S4: calculating the off-target quantity of the predicted position and the visual axis pointing position of the target to be tracked;
step S5: generating a servo control signal according to the off-target quantity, and correcting the visual axis orientation in real time according to the servo control signal.
Optionally, the searching the target to be tracked and obtaining the target image to be tracked specifically includes:
the method comprises the steps of searching a target to be tracked, setting a manual searching mode and an automatic searching mode, wherein the manual searching mode is used for manually searching the target to be tracked through a ground station rocker, and the automatic searching mode is used for searching the target to be tracked through a self detection strategy. In operation, a working instruction is sent to the main controller through ground station display control, a working mode of the photoelectric stabilized platform is set, and an image of a target to be tracked is obtained through photoelectric load.
Optionally, the automatic searching mode searches the target to be tracked through a self-detection strategy, and specifically includes:
the self-detection strategy sequentially carries out denoising operation, image enhancement operation, edge detection operation and size standardization operation on the tracking target video frame to obtain a standard tracking target video frame;
analyzing the standard tracking target video frame by using a target detection algorithm to generate candidate target areas, and extracting and classifying the characteristics of each candidate target area to obtain the category and the confidence of the target to be tracked;
position adjustment is carried out on the candidate target area by using bounding box regression, and the real boundary of the target to be tracked is met;
and screening the overlapped or redundant candidate target areas by using non-maximum value inhibition, removing areas with low confidence coefficient or repetition, screening out an optimal detection result, and searching out the target to be tracked.
Optionally, the tracking the feature extraction template of the target to be tracked by using a target tracking policy to obtain a predicted position of the target to be tracked specifically includes:
the target updating strategy is based on DaSiamRPN, a shallow neural network updating strategy is provided, a template for the next frame cross correlation calculation is obtained through a neural network phi, and the specific formula is as follows:
In the method, in the process of the invention,template calculated for next frame cross correlation, phi is neural network, < ->Extracting templates for initial frame features of a video sequence, +.>For the last obtained accumulated template, T i Extracting a template for the target position characteristic of the current frame, +.>And T i All are based on the initial frame feature extraction template +.>Is determined by the position of the predicted position of (2);
in the course of the target update policy,and->And performing jump connection to form residual learning, and performing cross-correlation calculation on the obtained current template (template of cross-correlation calculation of the next frame) and the next frame of search image, wherein the position corresponding to the highest scoring position in the response graph is the predicted position of the target to be tracked.
Optionally, the calculating the off-target amount between the predicted position and the visual axis pointing position of the target to be tracked specifically includes:
the off-target amount comprises an angle off-target amount and a distance off-target amount, and the angle off-target amount comprises a horizontal angle off-target amount and a vertical angle off-target amount;
acquiring pixel coordinates (x target ，y target ) And the pixel coordinates (x target ，y current ) Calculating x along the x-axis target -x current To obtain Deltax, and calculating y along the y-axis target -y current To obtain deltay;
the horizontal angle off-target amount and the vertical angle off-target amount are respectively calculated, and the specific formula is as follows:
Wherein Deltax is the difference value of the pixel coordinate of the predicted position of the target to be tracked and the pixel coordinate of the visual axis pointing position in the x-axis, deltay is the difference value of the pixel coordinate of the predicted position of the target to be tracked and the pixel coordinate of the visual axis pointing position in the y-axis, and x focallength Is the horizontal focal length of the camera, y focallength X is the vertical focal length of the camera angle For the horizontal angle off-target amount, y angle The target off-target amount is a vertical angle;
calculating the distance miss distance, wherein the specific formula is as follows:
where dis is the distance miss distance and λ is the scale factor of the pixel to the actual distance.
Optionally, the generating a servo control signal according to the off-target amount, correcting the visual axis direction in real time according to the servo control signal specifically includes:
processing the off-target quantity by adopting a fuzzy sliding mode control strategy to obtain a servo control signal, wherein the method specifically comprises the following steps of:
fuzzifying the off-target quantity and the derivative of the off-target quantity into a fuzzy input variable, wherein the reasoning rule of the fuzzy controller is as follows: if it isNB, then ak is NB; if->NM, then ΔK is NM; if->Is ZO, then Δk is ZO; if it isPM, then ΔK is PM; if->PB, then ΔK is PB;
in the rule of the rule,to blur the input variables, the rate and direction of change of s is described, when +. >When the change rate of the input variable s is Negative, that is, the trend of the input variable s is changed toward a decreasing direction, NB is Negative large (Negative Big), NM is Negative medium (Negative Middle), PM is Positive medium, PB is Positive large (Positive Big), ZO is Zero (Zero), and Δk is the output variable (change amount);
inputting delta K into a synovial membrane variable structure controller to adjust switching gain K (t) in real time, wherein the specific formula is as follows:
where D is a scaling factor greater than 0, K (t) is a gain that varies in the switching control system according to changes in system state or other conditions,for the switching gain after the real-time adjustment, t is the time in the time domain;
switching gain after the off-target quantity and the real-time adjustmentGenerating a servo control signal via a control function u (t), correcting the visual axis orientation in real time in accordance with the servo control signal.
The invention also provides a target tracking system of the on-board photoelectric stabilized platform for the coastal defense, which comprises the following components:
the target to be tracked searching module is used for searching the target to be tracked and obtaining a target image to be tracked;
the feature extraction template generation module is used for sequentially carrying out preprocessing operation and feature extraction operation on the image of the target to be tracked to obtain a feature extraction template of the target to be tracked;
The target to be tracked prediction module is used for tracking the target feature extraction template to be tracked by adopting a target tracking strategy to obtain the predicted position of the target to be tracked;
the off-target calculating module is used for calculating off-target amounts of the predicted position and the visual axis pointing position of the target to be tracked;
and the visual axis correction module is used for generating a servo control signal according to the off-target quantity and correcting the visual axis direction in real time according to the servo control signal.
Optionally, the target prediction module to be tracked specifically includes:
the next frame cross-correlation calculation template generation sub-module provides a shallow neural network update strategy based on DaSiamRPN, and a template for next frame cross-correlation calculation is obtained through a neural network phi, wherein the specific formula is as follows:
in the method, in the process of the invention,template calculated for next frame cross correlation, phi is neural network, < ->Extracting templates for initial frame features of a video sequence, +.>For the last obtained accumulated template, T i Extracting a template for the target position characteristic of the current frame, +.>And T i All are based on the initial frame feature extraction template +.>Is determined by the position of the predicted position of (2);
a cross-correlation calculation sub-module for, in the target update strategy, And->And performing jump connection to form residual learning, and performing cross-correlation calculation on the obtained current template (template of cross-correlation calculation of the next frame) and the next frame of search image, wherein the position corresponding to the highest scoring position in the response graph is the predicted position of the target to be tracked.
Optionally, the off-target calculating module specifically includes:
a pixel coordinate difference sub-module for obtaining the pixel coordinate (x) of the predicted position of the target to be tracked target ，y target ) And the pixel coordinates (x current ，y current ) Calculating x along an axis target -x current To obtain Deltax, and calculating y along the y-axis target -y current To obtain deltay;
the horizontal angle off-target amount and the vertical angle off-target amount are respectively calculated, and the specific formula is as follows:
wherein Deltax is the difference value of the pixel coordinate of the predicted position of the target to be tracked and the pixel coordinate of the pointing position of the visual axis in the x-axis, and Deltay isDifference value of pixel coordinates of predicted position of target to be tracked and pixel coordinates of visual axis pointing position on axis, x focallength Is the horizontal focal length of the camera, y focallength X is the vertical focal length of the camera angle For the horizontal angle off-target amount, y angle The target off-target amount is a vertical angle;
calculating the distance miss distance, wherein the specific formula is as follows:
where dis is the distance miss distance and λ is the scale factor of the pixel to the actual distance.
Optionally, the off-target calculating module specifically includes:
the off-target amount processing submodule is used for processing the off-target amount by adopting a fuzzy sliding mode control strategy to obtain a servo control signal, and specifically comprises the following steps:
the fuzzy rule generation submodule is used for fuzzifying the off-target quantity and the derivative of the off-target quantity into a fuzzy input variable, and the reasoning rule of the fuzzy controller is as follows: if it isNB, then ak is NB; if->NM, then ΔK is NM; if it isIs ZO, then Δk is ZO; if->PM, then ΔK is PM; if->PB, then ΔK is PB;
in the method, in the process of the invention,to blur the input variables, the variation of s is describedRate and direction, when->When the change rate of the input variable s is Negative, that is, the trend of the input variable s is changed toward a decreasing direction, NB is Negative large (Negative Big), NM is Negative medium (Negative Middle), PM is Positive medium, PB is Positive large (Positive Big), ZO is Zero (Zero), and Δk is the output variable (change amount);
the switching gain adjustment submodule is used for inputting delta K into the synovial membrane variable structure controller to adjust the switching gain K (t) in real time, and the specific formula is as follows:
where D is a scaling factor greater than 0, K (t) is a gain that varies in the switching control system according to changes in system state or other conditions, For the switching gain after the real-time adjustment, t is the time in the time domain;
a control signal generation sub-module for generating the off-target quantity and the switching gain after real-time adjustmentGenerating a servo control signal via a control function u (t), correcting the visual axis orientation in real time in accordance with the servo control signal.
Compared with the prior art, the invention has the following beneficial effects:
aiming at the problems of ensuring the response capability and the anti-interference capability of a platform system while considering the control performance and the tracking effect of the platform system in the complex environment of the coastal defense area when the photoelectric stabilized platform is applied to the coastal defense vehicle, the invention carries out intensive research on a platform tracking algorithm and a control algorithm of a servo system, designs an update tracking algorithm based on a DaSiamRPN improved model and a fuzzy sliding mode variable structure controller, and researches a target tracking method of the coastal defense vehicle-mounted photoelectric stabilized platform. The DaSiamRPN improved model updating tracking algorithm which introduces the shallow neural network is provided, so that the tracking precision of a system platform is remarkably improved; the designed sliding mode variable structure model based on the fuzzy control theory enables the controlled photoelectric stabilized platform to be accurately controlled, gives consideration to the control performance, tracking effect and stabilizing capability of the platform system, and has positive reference effect on the design research of the photoelectric platform in the related field of national defense.
Drawings
FIG. 1 is a flow chart of the method of the present invention;
FIG. 2 is a diagram of a tracking process of the shallow neural network update strategy of the present invention;
FIG. 3 is a diagram of a shallow neural network training process according to the present invention;
FIG. 4 is a schematic diagram of the operation of the fuzzy sliding mode controller of the present invention;
FIG. 5 is a system block diagram of the present invention;
FIG. 6 is a frame construction diagram of a two-axis four-frame photovoltaic stabilization platform according to an embodiment of the present invention;
FIG. 7 is a block diagram of an experimental system of an optoelectronic stabilization platform according to an embodiment of the present invention;
FIG. 8 is a schematic diagram of the experimental system of the photovoltaic stabilization platform according to an embodiment of the present invention;
FIG. 9 is a graph of the target tracking results according to an embodiment of the present invention;
FIG. 10 is a graph of tracking results of adding disturbance signals in the pitch direction according to an embodiment of the present invention;
FIG. 11 is a graph of tracking results of adding disturbance signals in the roll direction according to an embodiment of the present invention.
Detailed Description
The invention is further described below in connection with specific embodiments and the accompanying drawings, but the invention is not limited to these embodiments.
Example 1
As shown in fig. 1, the invention discloses a target tracking method for an on-board photoelectric stabilized platform of an offshore defense, which comprises the following steps:
step S1: searching a target to be tracked and obtaining an image of the target to be tracked.
Step S2: and sequentially carrying out preprocessing operation and feature extraction operation on the image of the target to be tracked to obtain a feature extraction template of the target to be tracked.
Step S3: and tracking the target feature extraction template to be tracked by adopting a target tracking strategy to obtain the predicted position of the target to be tracked.
Step S4: and calculating the off-target quantity of the predicted position and the visual axis pointing position of the target to be tracked.
Step S5: generating a servo control signal according to the off-target quantity, and correcting the visual axis orientation in real time according to the servo control signal.
The steps are discussed in detail below:
step S1: searching a target to be tracked and obtaining an image of the target to be tracked, specifically comprising:
the method comprises the steps that a target to be tracked is searched, wherein the target to be tracked is set to be a manual searching mode and an automatic searching mode, the manual searching mode is used for manually searching the target to be tracked through a ground station rocker, and the automatic searching mode is used for searching the target to be tracked through a self detection strategy, and specifically comprises the following steps:
and (3) sequentially carrying out denoising operation, image enhancement operation, edge detection operation and size standardization operation on the tracking target video frame by using a self detection strategy to obtain a standard tracking target video frame.
Standard tracking target video frames are analyzed using a target detection algorithm to generate regions of interest (Region of Interest, ROI), which is to narrow the search, focusing only on regions that may contain targets. Some methods use heuristic methods, such as Selective Search (Selective Search), to aggregate regions based on similarity between pixels; deep learning methods such as regional advice networks (Region Proposal Network, RPN) may also be used.
In this embodiment, an improved area proposal network (ImRPN) is used, which specifically includes:
in the task of detecting the target, the regional proposal network RPN is the key for improving the accuracy of detecting the target by the network aiming at the optimization of the regional proposal network, and meanwhile, the detection rate can be improved, and the tracking rate is improved laterally. The generation of the interest domain of the regional proposal network mainly depends on an anchor point generation strategy, so that a target candidate region is generated rapidly, and then the anchor point is screened and primarily regressed through a convolution layer. RPN is much better than other candidate region generation methods in terms of generation speed, quality of the region of interest, etc. However, in the process of generating the anchor frames by the RPN, most of the generated anchor frames are redundant sometimes, which causes that the positive and negative sample sizes of the RPN are unbalanced during training, thereby reducing the detection performance of the network. While ImRPN produces more positive samples relative to RPN, redundant anchor box generation can also be reduced and the proposed boxes of high IoU are more scaled.
In this embodiment, the ImRPN predicts the positions and shapes of the anchor frames by using two branches, and then combines them to obtain an anchor frame, and adjusts the features of the anchor frame by using the feature adaptive module to obtain a new feature map for later prediction. The ImRPN is mainly divided into two modules, namely an anchor frame generation module and a characteristic self-adaptation module. The anchor frame generation module is divided into a positioning branch and a shaping branch. The positioning branch is used for judging whether each characteristic point is a target point on the input characteristic diagram; the shaped branches are used to determine the width and height of the anchor boxes that are the target feature points. And the characteristic self-adaptation module resamples the characteristics in the anchor frame area according to the anchor frame result generated by the shaping branch, and adjusts the characteristic receptive field in the anchor frame area according to the size of the interest area. And carrying out position adjustment on the candidate target area by using bounding box regression, wherein the position adjustment accords with the real boundary of the target to be tracked, and specifically comprises the following steps:
The anchor box locates the branch, predicts which regions should be the center point to generate the anchor box. The method comprises the steps of marking a small area in the center of a real frame corresponding to an area on a feature map as an object center area, setting the area as a positive sample during training, setting other areas as negative samples or neglecting the areas according to the distance from the center area, and finally positioning the area where the object activity possibly exists by selecting the position with the corresponding probability value higher than a preset threshold value. After the input characteristic diagram F is convolved by 1X 1, a probability diagram F with the same scale is obtained 1 Value table of points thereinShown in the feature map F, each feature point is a probability of a target. Setting a threshold value when F 1 When the value of (b) is larger than the threshold value, the feature point in the corresponding F is judged to be the target center point. By locating the branches, a part of the area can be screened out as the candidate center point position of the anchor frame, so that the number of the anchor frames can be reduced, and the subsequent calculation amount is reduced.
The anchor frame shaping branch determines the position of the center of the anchor frame, and the anchor frame shaping branch is used for predicting an optimal anchor frame with the width and the height for all anchor points. Assuming a width w and a height h, a two-dimensional 1×1 convolutional network is adopted, a two-channel feature map with the same size as F is input and output, each channel represents the best possible anchor frame size of each position, and is denoted as dw and dh, and the space range [0, 1000] is mapped into [ -1,1] and can be expressed as:
w＝γ×s×e dw
h＝γ×s×e dh
In the formula, s is a convolution step length, gamma is an experience factor, and the width and height of the anchor frame are predicted according to anchor point positioning information, so that the correlation between the position and the shape of the anchor frame is improved, and a higher recall rate can be obtained.
The characteristic self-adaptive module has a certain scale of an anchor frame generated by the RPN, the ImRPN is different from the RPN, and the obtained anchor frame type sample is flexible. When different targets have different scales, the required receptive fields also have different scales, so the ImRPN adjusts the receptive fields corresponding to the anchor frames through characteristic self-adaption. The method is to combine the characteristic diagram with the anchor frame shape information to obtain a new characteristic diagram, and adapt the anchor frame shape of each position by using the new characteristic diagram. Here, a 3×3 deformable convolution operation is used to adjust the initial feature map by an amount of change obtained by a 1×1 convolution layer according to w and h of the anchor frame. The adjusted characteristic value f' i Can be expressed as:
f′ i ＝N t (f i ，w i ，h i )
wherein f' i Characterization of the ith anchor frameCorresponding eigenvalues, w, on graph F i And h i Width and height of the predicted ith anchor frame, N t By means of a 3 x 3 deformable convolution.
Candidate regions are predicted from the convolution feature map. And extracting and classifying the characteristics of each candidate target area to obtain the category and the confidence of the target to be tracked.
And screening the overlapped or redundant candidate target areas by using non-maximum suppression, removing the areas with low confidence coefficient or redundancy, screening out the optimal detection result, and searching out the target to be tracked.
In operation, a working instruction is sent to the main controller through ground station display control, a working mode of the photoelectric stabilized platform is set, and an image of a target to be tracked is obtained through photoelectric load.
Step S2: the method comprises the steps of sequentially carrying out preprocessing operation and feature extraction operation on images of a target to be tracked to obtain a target feature extraction template to be tracked, and specifically comprises the following steps:
preprocessing, which refers to a series of operations performed on an image before it enters feature extraction, is intended to enhance image quality, reduce noise, highlight target features, and the like. The pretreatment operation specifically comprises the following steps:
and denoising, namely reducing noise in the image by adopting a filter (such as Gaussian filtering) so as to improve the stability and accuracy of subsequent processing.
Image enhancement, which uses histogram equalization, contrast enhancement, etc. to enhance the visibility of the object in the image for better feature extraction.
Color space conversion, converting an image from an RGB color space to a HSV, grayscale, etc., color space for better capture of target features.
And the feature extraction operation is to extract representative information or features from the image so as to facilitate subsequent tasks such as target identification, tracking and the like. The common feature extraction method comprises the following steps:
edge detection, which uses an edge detection algorithm (such as Canny edge detection) to identify the boundary of the target and capture the shape and contour information of the target.
And (3) corner detection, namely searching for corners in the image (such as Harris corner detection), and capturing corner features of the target.
And extracting texture features, analyzing the texture mode of the image, and obtaining texture information of the target surface.
Histogram features, statistics of color distribution information of image pixels, and differentiation of color features of different targets or backgrounds.
The feature extraction template is integrated with the obtained image information into a feature extraction template after pretreatment and feature extraction operation. This template contains key features of the object, which may be a series of numbers, vectors or matrices, representing important information of the object in the image. The feature extraction template is generated to determine the position and state of the target by using the key features in the subsequent target tracking process.
Step S3: and tracking the target feature extraction template to be tracked by adopting a target tracking strategy to obtain the predicted position of the target to be tracked.
As shown in fig. 2, step S3 specifically includes:
the DaSiamRPN algorithm provides an interference sensing module, integrates negative samples in a video sequence into similarity, utilizes non-maximum value to inhibit and select potential interferents to form an interferent set, utilizes an interference sensing objective function to reorder m candidate areas P closest to the samples, and finally selects a tracking target q, wherein the specific formula is as follows:
wherein q is the last selected tracking target, f is the cross-correlation operation, z is the position and size of the target in the current frame, and P m As a candidate box with vector projection,d, as an influence factor of interference items on learning i Alpha, for intra-class interference in each frame i An influence factor for each interference term.
The target updating strategy is based on DaSiamRPN, a shallow neural network updating strategy is provided, a template for the next frame cross correlation calculation is obtained through a neural network phi, and the specific formula is as follows:
in the method, in the process of the invention,template calculated for next frame cross correlation, phi is neural network, < ->Extracting templates for initial frame features of a video sequence, +.>For the last obtained accumulated template, T i Extracting a template for the target position characteristic of the current frame, +.>And T i All are based on the initial frame feature extraction template +. >Is used for the prediction of the position of the object.
In the course of the target update policy,and->And performing jump connection to form residual learning, and performing cross-correlation calculation on the obtained current template (template of cross-correlation calculation of the next frame) and the next frame of search image, wherein the position corresponding to the highest scoring position in the response graph is the predicted position of the target to be tracked.
As shown in fig. 3, the shallow neural network training process specifically includes:
wherein L is 2 The loss function representing L2 regularization, also known as the ridge regression loss function. It is used to minimize the sum of squares of model parameters in object tracking or other machine learning tasks. L2 regularization can help prevent overfitting and promote generalization ability of the model.
Currently obtained next frame cross-correlation calculation templateMarking position corresponding to the target of the next frame>Matching, i.e. let->Is to introduce Euclidean distance as a loss function in the optimized template in the next frame search image, so that +.>And->The loss function between them is minimized. Use of accumulated templates in training>To produce an imperfect feature template T i (the accumulation template of the first frame is replaced by the information extracted by the target features), so that the error information actually appearing in the tracking process is presented and trained, the loss function of the actual accumulation template and the ideal template is minimized, and the best model updating effect is obtained.
In order to avoid the responsibility and inefficiency of the training program, the training process is split into a multi-stage training mode, and the shallow neural network is iteratively optimized stage by stage, wherein the specific formula is as follows:
in the method, in the process of the invention,for the template update rate, ++>Accumulating templates for each frame, +.>Features are extracted for realism.
Where K is the number of training phases, K e {1, 2..once, K },for the accumulated templates generated in the previous stage +.>Features are extracted for realism.
Step S4: calculating the off-target quantity of the predicted position and the visual axis pointing position of the target to be tracked, which specifically comprises the following steps:
the pixel coordinate difference submodule acquires pixel coordinates (x) of a predicted position of an object to be tracked target ，y target ) And the pixel coordinates (x current ，y current ) Calculating x along the x-axis target -x current To obtain Deltax, and calculating y along the y-axis target -y current And Δy is obtained.
The horizontal angle off-target amount and the vertical angle off-target amount are respectively calculated, and the specific formula is as follows:
wherein Deltax is the pixel coordinate and visual axis of the predicted position of the target to be trackedDifference value of pixel coordinate of pointing position in x-axis, delta y is difference value of pixel coordinate of predicted position of target to be tracked and pixel coordinate of visual axis pointing position in y-axis, x focallength Is the horizontal focal length of the camera, y focallength X is the vertical focal length of the camera angle For the horizontal angle off-target amount, y angle Is the vertical angle off-target amount.
Calculating the distance miss distance, wherein the specific formula is as follows:
where dis is the distance miss distance and λ is the scale factor of the pixel to the actual distance.
Step S5: generating a servo control signal according to the off-target quantity, and correcting the visual axis orientation in real time according to the servo control signal.
As shown in fig. 4, step S5 specifically includes:
processing the off-target quantity by adopting a fuzzy sliding mode control strategy to obtain a servo control signal, wherein the method specifically comprises the following steps of:
fuzzifying the off-target quantity and the derivative of the off-target quantity into a fuzzy input variable, wherein the reasoning rule of the fuzzy controller is as follows: if it isNB, then ak is NB; if->NM, then ΔK is NM; if->Is ZO, then Δk is ZO; if->PM, then ΔK is PM; if->If PB is found, then ΔK is PB.
In the rule of the rule,to blur the input variables, the rate and direction of change of s is described, when +.>When the change rate of the input variable s is Negative, that is, the trend of the input variable s is changed toward a decreasing direction, NB is Negative large (Negative Big), NM is Negative medium (Negative Middle), PM is Positive medium, PB is Positive large (Positive Big), ZO is Zero (Zero), and Δk is the output variable (change amount).
Inputting delta K into a synovial membrane variable structure controller to adjust switching gain K (t) in real time, wherein the specific formula is as follows:
where D is a scaling factor greater than 0, K (t) is a gain that varies in the switching control system according to changes in system state or other conditions,and t is the time in the time domain for the switching gain after the real-time adjustment.
Switching gain after off-target quantity and real-time adjustmentGenerating a servo control signal through a control function u (t), and correcting the visual axis orientation in real time according to the servo control signal.
Example 2
As shown in fig. 5, the invention discloses a target tracking system of an on-board photoelectric stabilized platform for an offshore defense, which comprises:
and the target to be tracked searching module is used for searching the target to be tracked and obtaining a target image to be tracked.
The feature extraction template generation module is used for sequentially carrying out preprocessing operation and feature extraction operation on the image of the target to be tracked to obtain a feature extraction template of the target to be tracked.
And the target to be tracked prediction module is used for tracking the target feature extraction template to be tracked by adopting a target tracking strategy to obtain the predicted position of the target to be tracked.
And the off-target calculating module is used for calculating the off-target amount of the predicted position and the visual axis pointing position of the target to be tracked.
And the visual axis correction module is used for generating a servo control signal according to the off-target quantity and correcting the visual axis direction in real time according to the servo control signal.
As an optional implementation manner, the target prediction module to be tracked in the invention specifically comprises:
the next frame cross-correlation calculation template generation sub-module provides a shallow neural network update strategy based on DaSiamRPN, and a template for next frame cross-correlation calculation is obtained through a neural network phi, wherein the specific formula is as follows:
in the method, in the process of the invention,template calculated for next frame cross correlation, phi is neural network, < ->Extracting templates for initial frame features of a video sequence, +.>For the last obtained accumulated template, T i Extracting a template for the target position characteristic of the current frame, +.>And T i All are based on the initial frame feature extraction template +.>Is used for the prediction of the position of the object.
A cross-correlation computation sub-module for, in a target update strategy,and->And performing jump connection to form residual learning, and performing cross-correlation calculation on the obtained current template (template of cross-correlation calculation of the next frame) and the next frame of search image, wherein the position corresponding to the highest scoring position in the response graph is the predicted position of the target to be tracked.
As an alternative embodiment, the off-target amount calculating module of the present invention specifically includes:
The pixel coordinate difference submodule acquires pixel coordinates (x) of a predicted position of an object to be tracked target ，y target ) And the pixel coordinates (x current ，y current ) Calculating x along the x-axis target -x current To obtain Deltax, and calculating y along the y-axis target -y current And Δy is obtained.
The horizontal angle off-target amount and the vertical angle off-target amount are respectively calculated, and the specific formula is as follows:
wherein Deltax is the difference value of the pixel coordinate of the predicted position of the target to be tracked and the pixel coordinate of the visual axis pointing position in the x-axis, deltay is the difference value of the pixel coordinate of the predicted position of the target to be tracked and the pixel coordinate of the visual axis pointing position in the y-axis, and x focallength Is the horizontal focal length of the camera, y focallength X is the vertical focal length of the camera angle For the horizontal angle off-target amount, y angle Is the vertical angle off-target amount.
Calculating the distance miss distance, wherein the specific formula is as follows:
where dis is the distance miss distance and λ is the scale factor of the pixel to the actual distance.
As an alternative embodiment, the off-target amount calculating module of the present invention specifically includes:
the off-target amount processing submodule is used for processing the off-target amount by adopting a fuzzy sliding mode control strategy to obtain a servo control signal, and specifically comprises the following steps:
the fuzzy rule generation submodule is used for fuzzifying the off-target quantity and the derivative of the off-target quantity into a fuzzy input variable, and the reasoning rule of the fuzzy controller is as follows: if it is NB, then ak is NB; if->NM, then ΔK is NM; if->Is ZO, then Δk is ZO; if->PM, then ΔK is PM; if->If PB is found, then ΔK is PB.
In the method, in the process of the invention,to blur the input variables, the rate and direction of change of s is described, when +.>When the change rate of the input variable s is Negative, that is, the trend of the input variable s is changed toward a decreasing direction, NB is Negative large (Negative Big), NM is Negative medium (Negative Middle), PM is Positive medium, PB is Positive large (Positive Big), ZO is Zero (Zero), and Δk is the output variable (change amount).
The switching gain adjustment submodule is used for inputting delta K into the synovial membrane variable structure controller to adjust the switching gain K (t) in real time, and the specific formula is as follows:
where D is a scaling factor greater than 0, K (t) is a gain that varies in the switching control system according to changes in system state or other conditions,and t is the time in the time domain for the switching gain after the real-time adjustment.
Control signal generation submodule for adjusting off-target quantity and switching gain in real timeGenerating a servo control signal through a control function u (t), and correcting the visual axis orientation in real time according to the servo control signal.
Example 3
The embodiment develops a two-axis four-frame photoelectric stabilization platform through the embodiment 1 and the embodiment 2, and performs principle explanation, test and verification on the developed photoelectric stabilization platform, and specifically comprises the following steps:
as shown in fig. 6, the structural part adopts a two-axis four-frame platform, which has excellent motion isolation, can eliminate disturbance velocity components caused by geometric constraint, and effectively improves the stability precision of the platform.
In the figure, 1-10 are respectively an outer azimuth axis motor, an outer azimuth axis encoder, an inner azimuth axis encoder, an outer pitching axis encoder, an inner azimuth axis motor, a laser detector, an infrared camera, an inner pitching axis motor, an outer pitching axis motor and a visible light camera.
As shown in FIG. 7, the photoelectric stabilized platform experiment system mainly comprises a photoelectric stabilized platform system, a ground station display control and a swing platform. The working mode of the photoelectric stabilized platform system is set to be a manual searching mode and an automatic searching mode, wherein the manual searching mode is used for manually searching a target through a ground station rocker, and the automatic searching mode is used for searching the target to be tracked through a platform self-detection strategy. In operation, a working instruction is sent to a main controller through ground station display control, a working mode of a photoelectric stabilized platform is set, a picture to be tracked is obtained through photoelectric load, data are sent to a communication controller, an image processor processes the picture to be tracked, detection of a target to be tracked is completed, real-time tracking is conducted on the detected target by utilizing a target tracking strategy, pixel difference values between the current position of the target and the pointing position of a current visual axis are calculated, target position miss quantity is obtained, a servo controller obtains a control signal by means of the miss quantity to control a driving motor, a platform frame shafting is driven by the motor, the visual axis of the photoelectric load on an inner frame points to the current target position, and continuous tracking of the target by the photoelectric stabilized platform is completed. Meanwhile, the swing frequency and the angle of the swing table are set through the swing table control cabinet, disturbance of the photoelectric stabilization platform in practical application is simulated, and the disturbance resistance of the photoelectric stabilization platform in a severe environment is checked.
As shown in fig. 8, in the target tracking part, in order to meet the performance requirements of high precision, real-time performance and strong robustness of the photoelectric stabilized platform, the platform control system adopts a three-closed loop control scheme consisting of a current loop, a speed loop and a position loop. The current loop is used for reducing the influence of current fluctuation on the torque output of the motor; the speed loop improves steady-state precision by compensating speed difference, which is the key of stabilizing the visual axis of the platform and improving the anti-interference performance of the system; the position ring compensates the distance between the controlled target and the visual axis by taking the off-target quantity as the control quantity, so as to realize accurate tracking.
In the work of the photoelectric stabilized platform system, firstly, an instruction is sent to an image sensor according to a working mode set by the platform, an object to be tracked is identified in a manual or automatic mode, image information of the object to be tracked is obtained and transmitted to an image tracker, the image tracker realizes continuous tracking of the object by means of an object tracking strategy, the off-target quantity between the position of the current frame object and the pointing position of a visual axis is calculated, the position deviation quantity of the current object in an actual space is obtained by combining frame angle information measured by a photoelectric encoder, and the position deviation quantity of the current object in the actual space is obtained by combining frame angle information measured by the photoelectric encoder and is input to a position controller to output a corresponding control signal. The speed ring takes a control signal output by the position controller as input, combines the angular speed of the platform frame relative to the inertia space, which is measured by the rate gyro, outputs a control signal for controlling the driven motor through the speed controller, and completes the control of the motor through power amplification. And the current loop at the innermost ring measures the current of the driving motor by the current sensor, and completes current regulation through the feedback loop, so that the current is stably input into the driving motor, and the influence of current fluctuation on the motor is reduced.
The invention is applied to a lake surface tracking experiment, and specifically comprises the following steps:
(1) Target tracking experiment
As shown in fig. 9, in order to check the effectiveness of the target tracking strategy of the proposed photoelectric stabilized platform, a simulation experiment is performed on a water surface ship in an outdoor scene, and the algorithm effect is intuitively described in a qualitative analysis mode. In the target tracking experiment, the ship to be tracked is selected through the rocker of the upper computer operation interface, and it can be seen that the tracking frame of the upper computer display interface continuously tracks the target ship in the process of moving the ship, so that the task of continuously tracking the target in real time in the photoelectric stabilized platform can be completed, and the tracking requirement of the photoelectric stabilized platform in the coastal and marine defense scene is met.
(2) Servo control experiment
As shown in fig. 9 and fig. 10, in order to test the stability of the servo control strategy of the photoelectric stabilized platform provided by the invention, the platform is simulated by the swing platform, and the target distance tracked by the side sea defense scene is far, the swing angle observed by the platform is small, and the swing frequency is generally below 3Hz, so that the stability of the platform is verified by four disturbance modes, namely, a disturbance signal in the pitching direction of 3 °, a disturbance signal in the 2Hz, a disturbance signal in the 3 °, a disturbance signal in the 3Hz, and a disturbance signal in the rolling direction of 3 °, a disturbance signal in the 2Hz, and a disturbance signal in the 3 °, 3 Hz. Experimental results show that the selected tracking targets can overcome most of disturbance of the swinging table, and tracking errors and folded angle values are shown in table 1.
TABLE 1 tracking error and reduced angle results
The above is only a preferred embodiment of the present invention, and is not intended to limit the present invention, but various modifications and variations can be made to the present invention by those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims (10)
1. The method for tracking the targets of the on-board photoelectric stabilized platform of the coastal defense is characterized by comprising the following steps:
step S1: searching a target to be tracked and obtaining a target image to be tracked;
step S2: sequentially performing preprocessing operation and feature extraction operation on the image of the target to be tracked to obtain a feature extraction template of the target to be tracked;
step S3: tracking the target feature extraction template to be tracked by adopting a target tracking strategy to obtain a predicted position of the target to be tracked;
step S4: calculating the off-target quantity of the predicted position and the visual axis pointing position of the target to be tracked;
step S5: generating a servo control signal according to the off-target quantity, and correcting the visual axis orientation in real time according to the servo control signal.
2. The method for tracking the target of the on-board photoelectric stabilized platform for the coastal defense according to claim 1, wherein the searching the target to be tracked and obtaining the image of the target to be tracked specifically comprises:
The method comprises the steps of searching a target to be tracked, setting a manual searching mode and an automatic searching mode, wherein the manual searching mode is used for manually searching the target to be tracked through a ground station rocker, and the automatic searching mode is used for searching the target to be tracked through a self detection strategy. In operation, a working instruction is sent to the main controller through ground station display control, a working mode of the photoelectric stabilized platform is set, and an image of a target to be tracked is obtained through photoelectric load.
3. The method for tracking the target of the on-board photoelectric stabilized platform for the coastal defense according to claim 2, wherein the automatic searching mode is used for searching the target to be tracked through a self-detection strategy, and specifically comprises the following steps:
the self-detection strategy sequentially carries out denoising operation, image enhancement operation, edge detection operation and size standardization operation on the tracking target video frame to obtain a standard tracking target video frame;
analyzing the standard tracking target video frame by using a target detection algorithm to generate candidate target areas, and extracting and classifying the characteristics of each candidate target area to obtain the category and the confidence of the target to be tracked;
position adjustment is carried out on the candidate target area by using bounding box regression, and the real boundary of the target to be tracked is met;
And screening the overlapped or redundant candidate target areas by using non-maximum value inhibition, removing areas with low confidence coefficient or repetition, screening out an optimal detection result, and searching out the target to be tracked.
4. The method for tracking the target of the on-board photoelectric stabilized platform for the coastal defense according to claim 1, wherein the target tracking strategy is adopted to track the feature extraction template of the target to be tracked, so as to obtain the predicted position of the target to be tracked, and the method specifically comprises the following steps:
the target updating strategy is based on DaSiamRPN, a shallow neural network updating strategy is provided, a template for the next frame cross correlation calculation is obtained through a neural network phi, and the specific formula is as follows:
in the method, in the process of the invention,template calculated for next frame cross correlation, phi is neural network, < ->Extracting templates for initial frame features of a video sequence, +.>For the last obtained accumulated template, T i Extracting a template for the target position characteristic of the current frame, +.>And T i All are based on the initial frame feature extraction template +.>Is determined by the position of the predicted position of (2);
in the course of the target update policy,and->And performing jump connection to form residual learning, and performing cross-correlation calculation on the obtained current template (template of cross-correlation calculation of the next frame) and the next frame of search image, wherein the position corresponding to the highest scoring position in the response graph is the predicted position of the target to be tracked.
5. The method for tracking the target of the on-board photoelectric stabilized platform for the coastal defense according to claim 1, wherein the calculating the off-target amount between the predicted position and the visual axis pointing position of the target to be tracked specifically comprises:
the off-target amount comprises an angle off-target amount and a distance off-target amount, and the angle off-target amount comprises a horizontal angle off-target amount and a vertical angle off-target amount;
acquiring pixel coordinates (x target ，y target ) And the pixel coordinates (x current ，y current ) Calculating x along the x-axis target -x current To obtain Deltax, and calculating y along the y-axis target -y current To obtain deltay;
the horizontal angle off-target amount and the vertical angle off-target amount are respectively calculated, and the specific formula is as follows:
wherein Deltax is the difference value of the pixel coordinate of the predicted position of the target to be tracked and the pixel coordinate of the visual axis pointing position in the x-axis, deltay is the difference value of the pixel coordinate of the predicted position of the target to be tracked and the pixel coordinate of the visual axis pointing position in the y-axis, and x focallength Is the horizontal focal length of the camera, y focallength X is the vertical focal length of the camera angle For the horizontal angle off-target amount, y angle The target off-target amount is a vertical angle;
calculating the distance miss distance, wherein the specific formula is as follows:
where dis is the distance miss distance and λ is the scale factor of the pixel to the actual distance.
6. The method for tracking the target of the on-board photoelectric stabilized platform for the coastal defense according to claim 1, wherein the generating a servo control signal according to the off-target amount, and correcting the visual axis direction in real time according to the servo control signal, comprises the following steps:
processing the off-target quantity by adopting a fuzzy sliding mode control strategy to obtain a servo control signal, wherein the method specifically comprises the following steps of:
fuzzifying the off-target quantity and the derivative of the off-target quantity into a fuzzy input variable, wherein the reasoning rule of the fuzzy controller is as follows: if it isNB, then ak is NB; if->NM, then ΔK is NM; if->Is ZO, then Δk is ZO; if->PM, then ΔK is PM; if->PB, then ΔK is PB;
in the rule of the rule,to blur the input variables, the rate and direction of change of s is described, when +.>When the change rate of the input variable s is Negative, that is, the trend of the input variable s is changed toward a decreasing direction, NB is Negative large (Negative Big), NM is Negative medium (Negative Middle), PM is Positive medium, PB is Positive large (Positive Big), ZO is Zero (Zero), and Δk is the output variable (change amount);
inputting delta K into a synovial membrane variable structure controller to adjust switching gain K (t) in real time, wherein the specific formula is as follows:
Where D is a scaling factor greater than 0, K (t) is a gain that varies in the switching control system according to changes in system state or other conditions,for the switching gain after the real-time adjustment, t is the time in the time domain;
switching gain after the off-target quantity and the real-time adjustmentGenerating a servo control signal via a control function u (t), correcting the visual axis orientation in real time in accordance with the servo control signal.
7. An offshore defending vehicle-mounted photoelectric stabilized platform target tracking system, the system comprising:
the target to be tracked searching module is used for searching the target to be tracked and obtaining a target image to be tracked;
the feature extraction template generation module is used for sequentially carrying out preprocessing operation and feature extraction operation on the image of the target to be tracked to obtain a feature extraction template of the target to be tracked;
the target to be tracked prediction module is used for tracking the target feature extraction template to be tracked by adopting a target tracking strategy to obtain the predicted position of the target to be tracked;
the off-target calculating module is used for calculating off-target amounts of the predicted position and the visual axis pointing position of the target to be tracked;
and the visual axis correction module is used for generating a servo control signal according to the off-target quantity and correcting the visual axis direction in real time according to the servo control signal.
8. The on-board photoelectric stabilized platform target tracking system for the offshore defense system according to claim 7, wherein the target prediction module to be tracked specifically comprises:
the next frame cross-correlation calculation template generation sub-module provides a shallow neural network update strategy based on DaSiamRPN, and a template for next frame cross-correlation calculation is obtained through a neural network phi, wherein the specific formula is as follows:
in the method, in the process of the invention,template calculated for next frame cross correlation, phi is neural network, < ->Extracting templates for initial frame features of a video sequence, +.>For the last obtained accumulated template, T i Extracting a template for the target position characteristic of the current frame, +.>And T i All are based on the initial frame feature extraction template +.>Is determined by the position of the predicted position of (2);
a cross-correlation calculation sub-module for, in the target update strategy,and->Making a jump connection to formResidual learning, and performing cross-correlation calculation on the obtained current template (template of cross-correlation calculation of the next frame) and the next frame of search image, wherein the position corresponding to the highest scoring position in the response graph is the predicted position of the target to be tracked.
9. The on-board photoelectric stabilized platform target tracking system for the offshore defense system according to claim 7, wherein the off-target calculation module specifically comprises:
A pixel coordinate difference sub-module for obtaining the pixel coordinate (x) of the predicted position of the target to be tracked target ，y target ) And the pixel coordinates (x current ，y current ) Calculating x along the x-axis target -x current To obtain Deltax, and calculating y along the y-axis target -y current To obtain deltay;
the horizontal angle off-target amount and the vertical angle off-target amount are respectively calculated, and the specific formula is as follows:
wherein Deltax is the difference value of the pixel coordinate of the predicted position of the target to be tracked and the pixel coordinate of the visual axis pointing position in the x-axis, deltay is the difference value of the pixel coordinate of the predicted position of the target to be tracked and the pixel coordinate of the visual axis pointing position in the y-axis, and x focallength Is the horizontal focal length of the camera, y focallength X is the vertical focal length of the camera angle For the horizontal angle off-target amount, y angle The target off-target amount is a vertical angle;
calculating the distance miss distance, wherein the specific formula is as follows:
where dis is the distance miss distance and λ is the scale factor of the pixel to the actual distance.
10. The on-board photoelectric stabilized platform target tracking system for the offshore defense system according to claim 7, wherein the off-target calculation module specifically comprises:
the off-target amount processing submodule is used for processing the off-target amount by adopting a fuzzy sliding mode control strategy to obtain a servo control signal, and specifically comprises the following steps:
The fuzzy rule generation submodule is used for fuzzifying the off-target quantity and the derivative of the off-target quantity into a fuzzy input variable, and the reasoning rule of the fuzzy controller is as follows: if it isNB, then ak is NB; if->NM, then ΔK is NM; if->Is ZO, then Δk is ZO; if->PM, then ΔK is PM; if->PB, then ΔK is PB;
in the method, in the process of the invention,to blur the input variables, the rate and direction of change of s is described, when +.>When the change rate of the input variable s is Negative, that is, the trend of the input variable s is changed toward a decreasing direction, NB is Negative large (Negative Big), NM is Negative medium (Negative Middle), PM is Positive medium, PB is Positive large (Positive Big), ZO is Zero (Zero), and Δk is the output variable (change amount);
the switching gain adjustment submodule is used for inputting delta K into the synovial membrane variable structure controller to adjust the switching gain K (t) in real time, and the specific formula is as follows:
where D is a scaling factor greater than 0, K (t) is a gain that varies in the switching control system according to changes in system state or other conditions,for the switching gain after the real-time adjustment, t is the time in the time domain;
a control signal generation sub-module for generating the off-target quantity and the switching gain after real-time adjustment Generating a servo control signal via a control function u (t), correcting the visual axis orientation in real time in accordance with the servo control signal.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311018578.5A CN116977902B (en) | 2023-08-14 | 2023-08-14 | Target tracking method and system for on-board photoelectric stabilized platform of coastal defense |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311018578.5A CN116977902B (en) | 2023-08-14 | 2023-08-14 | Target tracking method and system for on-board photoelectric stabilized platform of coastal defense |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN116977902A true CN116977902A (en) | 2023-10-31 |
| CN116977902B CN116977902B (en) | 2024-01-23 |
Family
ID=88476597
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311018578.5A Active CN116977902B (en) | 2023-08-14 | 2023-08-14 | Target tracking method and system for on-board photoelectric stabilized platform of coastal defense |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116977902B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117553756A (en) * | 2024-01-10 | 2024-02-13 | 中国人民解放军32806部队 | Off-target amount calculating method, device, equipment and storage medium based on target tracking |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2012167301A1 (en) * | 2011-06-10 | 2012-12-13 | Navisens Pty Ltd | Positioning, tracking and trajectory estimation of a mobile object |
| CN111161309A (en) * | 2019-11-19 | 2020-05-15 | 北航航空航天产业研究院丹阳有限公司 | Searching and positioning method for vehicle-mounted video dynamic target |
| CN111161311A (en) * | 2019-12-09 | 2020-05-15 | 中车工业研究院有限公司 | Visual multi-target tracking method and device based on deep learning |
| CN111192294A (en) * | 2019-12-30 | 2020-05-22 | 华中科技大学 | Target tracking method and system based on target detection |
| CN113838098A (en) * | 2021-09-10 | 2021-12-24 | 北京理工大学 | Intelligent tracking shooting system for remote high-speed moving target |
| CN114743150A (en) * | 2022-04-29 | 2022-07-12 | 北京达佳互联信息技术有限公司 | Target tracking method and device, electronic equipment and storage medium |
-
2023
- 2023-08-14 CN CN202311018578.5A patent/CN116977902B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2012167301A1 (en) * | 2011-06-10 | 2012-12-13 | Navisens Pty Ltd | Positioning, tracking and trajectory estimation of a mobile object |
| CN111161309A (en) * | 2019-11-19 | 2020-05-15 | 北航航空航天产业研究院丹阳有限公司 | Searching and positioning method for vehicle-mounted video dynamic target |
| CN111161311A (en) * | 2019-12-09 | 2020-05-15 | 中车工业研究院有限公司 | Visual multi-target tracking method and device based on deep learning |
| CN111192294A (en) * | 2019-12-30 | 2020-05-22 | 华中科技大学 | Target tracking method and system based on target detection |
| CN113838098A (en) * | 2021-09-10 | 2021-12-24 | 北京理工大学 | Intelligent tracking shooting system for remote high-speed moving target |
| CN114743150A (en) * | 2022-04-29 | 2022-07-12 | 北京达佳互联信息技术有限公司 | Target tracking method and device, electronic equipment and storage medium |
Non-Patent Citations (3)
| Title |
|---|
| QI XU ET.AL: "Study on Disturbance Suppression Strategy for Visual Axis Stabilization System of Photoelectric Stabilization Platform", 《2023 9TH INTERNATIONAL CONFERENCE ON CONTROL SCIENCE AND SYSTEMS ENGINEERING (ICCSSE)》, pages 1 - 6 * |
| YANGYANG LIU ET.AL: "Comparative Study on Correlation Filtering Algorithms for Sea Surface Target Tracking", 《2022 5TH INTERNATIONAL CONFERENCE ON ROBOTICS, CONTROL AND AUTOMATION ENGINEERING (RCAE)》, pages 1 - 5 * |
| 韩顺杰: "无标定图像视觉伺服控制方法研究", 《计算机应用与软件》, vol. 40, no. 04, pages 236 - 240 * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117553756A (en) * | 2024-01-10 | 2024-02-13 | 中国人民解放军32806部队 | Off-target amount calculating method, device, equipment and storage medium based on target tracking |
| CN117553756B (en) * | 2024-01-10 | 2024-03-22 | 中国人民解放军32806部队 | Off-target amount calculating method, device, equipment and storage medium based on target tracking |
Also Published As
| Publication number | Publication date |
|---|---|
| CN116977902B (en) | 2024-01-23 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110232350B (en) | Real-time water surface multi-moving-object detection and tracking method based on online learning | |
| CN109035204B (en) | Real-time detection method for weld joint target | |
| CN101511022B (en) | Method for implementing machine-carried video compression and target tracking unitedly | |
| CN111932588A (en) | Tracking method of airborne unmanned aerial vehicle multi-target tracking system based on deep learning | |
| CN116977902B (en) | Target tracking method and system for on-board photoelectric stabilized platform of coastal defense | |
| CN108492324B (en) | Airplane tracking method based on full-connection network and Kalman filter | |
| CN111723632B (en) | Ship tracking method and system based on twin network | |
| CN108108697A (en) | A kind of real-time UAV Video object detecting and tracking method | |
| CN109213204A (en) | AUV sub-sea floor targets based on data-driven search navigation system and method | |
| CN113593035A (en) | Motion control decision generation method and device, electronic equipment and storage medium | |
| CN110889353B (en) | Space target identification method based on primary focus large-visual-field photoelectric telescope | |
| CN114578807A (en) | Active target detection and obstacle avoidance method for unmanned target vehicle radar vision fusion | |
| CN116912715A (en) | Unmanned aerial vehicle vision servo control method and system for fan blade inspection | |
| CN113781523B (en) | Football detection tracking method and device, electronic equipment and storage medium | |
| CN109815871B (en) | Target ship detection and tracking method based on optical remote sensing image | |
| CN113721615B (en) | Navigation path planning method and system based on machine vision | |
| Alla et al. | Vision-based Deep Learning algorithm for Underwater Object Detection and Tracking | |
| CN109558877B (en) | KCF-based offshore target tracking algorithm | |
| CN113837924A (en) | Water bank line detection method based on unmanned ship sensing system | |
| CN116665097A (en) | Self-adaptive target tracking method combining context awareness | |
| CN116185049A (en) | Unmanned helicopter autonomous landing method based on visual guidance | |
| CN116363693A (en) | Automatic following method and device based on depth camera and vision algorithm | |
| CN115471555A (en) | Unmanned aerial vehicle infrared inspection pose determination method based on image feature point matching | |
| CN115373383A (en) | Autonomous obstacle avoidance method and device for garbage recovery unmanned boat and related equipment | |
| Tsai et al. | The robust and fast approach for vision-based shadowy road boundary detection |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |