CN109859202A - A kind of deep learning detection method based on the tracking of USV water surface optical target - Google Patents
A kind of deep learning detection method based on the tracking of USV water surface optical target Download PDFInfo
- Publication number
- CN109859202A CN109859202A CN201910120133.5A CN201910120133A CN109859202A CN 109859202 A CN109859202 A CN 109859202A CN 201910120133 A CN201910120133 A CN 201910120133A CN 109859202 A CN109859202 A CN 109859202A
- Authority
- CN
- China
- Prior art keywords
- target
- usv
- frame
- tracking
- distance
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Landscapes
- Image Analysis (AREA)
Abstract
The invention belongs to the unmanned platform environment perception of the water surface and control system interleaving techniques fields, more particularly to a kind of deep learning detection method based on the tracking of USV water surface optical target: USV loads video camera and acquires video in real time, and vision signal is transmitted to USV by image pick-up card and is internally embedded formula computer, computer first selectes a key frame every n frame, de-fuzzy processing is carried out to it again, finally detects waterborne target using convolutional neural networks；The present invention is in waterborne target detection and position fixing process, it joined the Smear-eliminated technique of image based on convolutional neural networks, improve objective fuzzy problem caused by seawater fluctuation, object movement and USV are navigated by water, and the recurrence mode based on 24 convolutional layers and 2 full articulamentums detects target has the advantages that speed is fast, background false detection rate is low, make USV target detection that there is real-time, and not vulnerable to interference such as sea sunlight refractions.
Description
Technical field
The invention belongs to the unmanned platform environment perception of the water surface and control system interleaving techniques fields, and in particular to one kind is based on
The deep learning detection method of USV water surface optical target tracking.
Background technique
Unmanned water surface ship (Unmanned Surface Vehicle, abbreviation USV), is increasingly becoming the supplement of Ship platform
Or substitute, have many advantages, such as small in size, high speed, preferable stealthy, intelligent, unmanned injures and deaths, sea can be completed at lower cost
On a large scale, a variety of military and non-military operations also can be performed in scientific investigation and engineering duty for a long time, as sea area searches and rescues, leads
Boat and hydro_geography prospecting；Hydrographic information monitoring, marine meterologal prediction, aquatic organism research, exploration of ocean resources and region
Sea chart is drawn；Coastal waters zone defence；Scouting, search, detection and the removal of mines of specified sea areas；Anti-submarine warfare, anti-special operations and
Hit pirate, anti-terrorism attack etc..
USV needs autonomous navigation in the case where unmanned intervene and completes various tasks, so must have to ocean ring
Border and the entirely autonomous perception and understandability of all kinds of realizations of goal therein, wherein target detection is to can be USV with tracking
The information such as position, posture and the motion profile of waterborne target are provided.Video camera detected by receiving target ontology radiation energy and
Target is tracked, there is anti-reconnaissance capability and the stronger advantage of anti-electronic jamming capability, and the target information observed is abundant, directly
The property seen and reliability are relatively high.Currently, countries in the world, which have had a profound understanding of, develops the object detecting and tracking based on light vision
The importance and urgency of technology.It searched and rescued for sea area, specified sea areas investigation, hit the tasks such as pirate, need closely to detect
Even strike target.Therefore, the target tracking strategy for studying USV also has having very important significance.
Do detection and tracking problems faced using light vision system in USV at present to be mainly reflected in: illumination and weather become
Change will affect picture quality, the movement of object often results in objective fuzzy, this just largely reduces target detection tracking
Accuracy rate；The shake of camera caused by seawater fluctuation and USV high speed operation, not only increases challenge to target detection, to USV
Motion control will also result in very big influence.
Summary of the invention
Present invention aim to address deficiency in the prior art, provide a kind of based on the tracking of USV water surface optical target
Deep learning detection method solves the problems, such as the high false alarm rate of target detection when distant object tracking, reduces objective fuzzy and light
Interference according to variation to object detecting and tracking.
A kind of deep learning detection method based on the tracking of USV water surface optical target is detected by waterborne target and is positioned, closes
Three target following of key interframe, USV path planning and behaviour control system compositions, comprising the following steps:
Waterborne target detection and positioning system step are as follows:
(1.1) USV loads video camera and acquires video in real time, and vision signal is transmitted to inside USV by image pick-up card
Embedded computer, computer first selectes a key frame every n frame, then carries out de-fuzzy processing to it, finally utilizes volume
Product neural network detects waterborne target；
(1.2) it is calculated using the binocular camera binocular distance measuring method on USV in the target detected in step (1.1)
Heart point to left lens camera distance, by this position of the distance as target relative to video camera；The IMU carried further according to USV
Installation site of the Angle of Trim and roll angle, left lens camera of measured USV on USV, it is opposite by calculating acquisition target
The position of USV and angle information；
(1.3) communication protocol comprising object location information is sent to USV path planning with network communication；
Crucial interframe Target Tracking System step are as follows:
(2.1) target detected according to step (1.1) extracts the LBP feature in key frame target region, according to feature
The probability density of model establishes object module；
(2.2) the LBP feature submodel probability density of present frame candidate target is calculated, and with similarity function metric objective
Similarity between model and target candidate model；
(2.3) target position of Meanshift vector iterative search present frame is used；
(2.4) when the drift distance of target's center position is less than threshold value or the number of iterations is more than threshold value, the frame mesh is exported
Cursor position；
(2.5) communication protocol comprising object location information is sent to USV path planning with network communication；
(2.6) step (2.2)-(2.5) are repeated until the tracking of crucial interframe terminates, after return step (1.1)；
USV path planning and behaviour control system step are as follows:
(3.1) path planning layer circulation waiting step (1.3), (2.5) send communication protocol, once receiving, are just added
And verification, by the location information for extracting object after verification；
(3.2) information such as target range, angle extracted according to step (3.1), start to carry out path planning, and by result
It is sent to behaviour control layer；
(3.3) USV behaviour control layer adjusts speed of the bow to angle and navigation according to the instruction of path planning layer；
The USV loads video camera and acquires video in real time, and it is embedding that vision signal by image pick-up card is transmitted to the inside USV
Enter formula computer, computer first selectes a key frame every n frame, then carries out de-fuzzy processing to it, finally utilizes convolution
Neural network detects waterborne target, comprising:
(1.1.1) generates the principle of blurred picture, the M frame image captured during exposure, after being averaged according to sensor
It is mapped as blurred picture with camera response function, clear image and blurred picture pair are produced with this, passes through training convolutional nerve net
Network learns the intrinsic function relationship of image pair, realizes waterborne target image deblurring；
(1.1.2) makes waterborne target data set, 24 layer convolutional Neural net of the end-to-end training based on GoogLeNeT in advance
Network and 2 full articulamentums, key frame is input in trained neural network, output object boundary frame coordinate, confidence level and
The prediction of classification；
(1.1.3) first rejects confidence level and is less than the output of threshold value as a result, carrying out non-maximum restraining to remaining result again: to institute
Bounding box sorts from large to small formation sequence by confidence level, selects the highest bounding box of confidence level, if itself and any remaining frame
IOU be greater than threshold value the residue frame is then rejected from sequence；Selected from sequence next bounding box repeat the above process up to
Terminate.
The target detected according to step (1.1) extracts the LBP feature in key frame target region, according to feature
The probability density of model establishes object module, comprising:
According to the target that step (1.1) detects, the LBP feature of target area is extracted, calculation is as follows:
Wherein, gcRepresent central pixel point (xc,yc) gray value；For p using central pixel point as origin, R is the field of radius
Inside there are P pixel, gpIt is the gray value of p-th of neighborhood territory pixel point；S (x) is two-valued function, 0 is taken when x is less than 0, otherwise
Take 1.
LBP feature space is quantified as m grade, calculates the core histogram probability density of each grade, generation is
Object module.
The information such as target range, the angle that the basis (3.1) is extracted start to carry out path planning, and result are sent
To behaviour control layer, comprising:
(3.2.1) considers that false-alarm feelings may occur for the influence of uncertain factor in sensor detection process, target acquisition
Condition, and the remoter possibility of distance is higher, sets the max-thresholds d an of distancemaxWith minimum threshold dmin, when USV and target away from
From more than dmaxWhen, it can not generally detect target, detect target if still reporting, then it is assumed that be false-alarm, do not change rule at this time
The path drawn；
(3.2.2) is when being in maximal distance threshold d with target rangemaxWith minimum threshold of distance dminBetween when, sensor
It can detect the presence of target, but observation still has uncertainty.If the abscissa of object central point is away from image at this time
Heart abscissa is more than mThA pixel then adjusts USV bow to angle according to the pixel number exceeded, keeps object central point
Abscissa is away from picture centre abscissa mThWithin a pixel；
(3.2.3) when with object distance be less than dminWhen, it is believed that sensor detected is real goal, is delayed at this time
Slowly close-target is leaned on, until certain safe distance DsWhen stop motion, the state of observed object: use Kalman Filter Estimation target
Speed of related movement v, if be less than setting threshold value vTh, then it is considered as static target, otherwise it is assumed that being moving target.For quiet
Only target sends to control layer and instructs, and so that USV is done rotary motion around it in the form of face object, prevents target unexpected
It is mobile；For moving target, then enables and keep certain safe distance D between USV and targetsStablize tracking.
The beneficial effects of the present invention are:
In waterborne target detection with position fixing process, it joined the Smear-eliminated technique of image based on convolutional neural networks, change
It has been apt to objective fuzzy problem caused by seawater fluctuation, object movement and USV are navigated by water, and has been connected entirely based on 24 convolutional layers and 2
The recurrence mode for connecing layer, which detects target, has the advantages that speed is fast, background false detection rate is low, and USV target detection is made to have real-time,
And not vulnerable to interference such as sea sunlight refractions.In crucial interframe object tracking process, with LBP operator extraction image object area
The textural characteristics in domain reduce illumination variation interference caused by target following as object module.In USV path planning and
During behaviour control, so that target is maintained at center to angle using remote adjustment USV bow, closely chased after according to location information
The mode of track target solves the problems, such as that distant object position error is big, target detection false alarm rate is high.
Detailed description of the invention
Fig. 1 is overall flow figure of the invention；
Fig. 2 is step 2 of embodiment of the present invention key interframe target following program flow chart；
Fig. 3 is binocular ranging flow diagram；
Fig. 4 is the simplification relational expression model of sensor detection probability and target relative distance；
Specific embodiment
The present invention is described further with reference to the accompanying drawing.
The present invention discloses a kind of deep learning detection method based on the tracking of USV water surface optical target, includes the following steps:
S1) video camera acquires video in real time, and first selecting key frame, de-fuzzy is handled again, finally detects the water surface with convolutional neural networks
Target, and with binocular ranging localization its with respect to USV position, location information is transmitted to path planning layer；S2) crucial interframe
Target following, according to the similarity iterative search target of object module and target candidate model based on LBP feature in present frame
Position, equally transmission position, return to S1 after tracking；S3) path planning layer planning path tracks target, and setting is maximum
Threshold value and minimum threshold, current goal distance do not change path when being more than max-thresholds；Current goal distance is in max-thresholds
Between minimum threshold, adjustment bow keeps object central point to deviate picture centre axis no more than m to angleThA pixel；Currently
It is slowly close to safe distance D when target range is less than minimum thresholds, Kalman Filter Estimation target relative movement speed v,
If being less than threshold value vThThen it is considered as static, and is turned round around target, be otherwise considered as movement, stablizing under safe distance is kept to track.This
Invention is big to illumination variation, the scene detection effect of camera shake is good, strong real-time, and tracking when enhancing different distance
Reliability.
A kind of deep learning detection method based on the tracking of USV water surface optical target, this method include waterborne target detection
With positioning, three crucial interframe target following, USV path planning and behaviour control systems, waterborne target detection and positioning system
Step are as follows:
(a1) USV loads video camera and acquires video in real time, and it is embedding that vision signal by image pick-up card is transmitted to the inside USV
Enter formula computer, computer first selectes a key frame every n frame, then carries out de-fuzzy processing to it, finally utilizes convolution
Neural network detects waterborne target；
(a2) it is arrived using the central point that the binocular camera binocular distance measuring method on USV calculates the target detected in (a1)
The distance of left lens camera, by this position of the distance as target relative to video camera；Measured by the IMU carried further according to USV
USV installation site on USV of Angle of Trim and roll angle, left lens camera, pass through to calculate and obtain target with respect to USV's
Position and angle information；
(a3) communication protocol comprising object location information is sent to USV path planning with network communication；
Crucial interframe Target Tracking System step are as follows:
(b1) target detected according to (a1) extracts LBP (the Local Binary in key frame target region
Pattern, local binary patterns) feature, object module is established according to the probability density of feature submodel；
(b2) the LBP feature submodel probability density of present frame candidate target is calculated, and with similarity function metric objective
Similarity between model and target candidate model；
(b3) target position of Meanshift vector iterative search present frame is used；
(b4) when the drift distance of target's center position is less than threshold value or the number of iterations is more than threshold value, the frame target is exported
Position；
(b5) communication protocol comprising object location information is sent to USV path planning with network communication；
(b6) repeat (b2)~(b5) until the tracking of crucial interframe terminates, after return to (a1)；
USV path planning and behaviour control system step are as follows:
(c1) path planning layer circulation waits (a3), (b5) to send communication protocol, once receiving, just sums up verification,
By the location information for extracting object after verification；
(c2) according to information such as (c1) target range, angles extracted, start to carry out path planning, and result is sent to
Behaviour control layer；
(c3) USV behaviour control layer adjusts speed of the bow to angle and navigation according to the instruction of path planning layer；
The step (a1) includes the following steps:
(a1.1) principle of blurred picture is generated according to sensor, the M frame image captured during exposure is used after being averaged
Camera response function is mapped as blurred picture, produces clear image and blurred picture pair with this.Pass through training convolutional neural networks
Learn the intrinsic function relationship of image pair, realizes waterborne target image deblurring；
(a1.2) waterborne target data set, 24 layer convolutional Neural net of the end-to-end training based on GoogLeNeT are made in advance
Network and 2 full articulamentums, key frame is input in trained neural network, output object boundary frame coordinate, confidence level and
The prediction of classification；
(a1.3) it first rejects confidence level and is less than the output of threshold value as a result, carrying out non-maximum restraining to remaining result again: to institute
Bounding box sorts from large to small formation sequence by confidence level, selects the highest bounding box of confidence level, if itself and any remaining frame
IOU be greater than threshold value the residue frame is then rejected from sequence；Selected from sequence next bounding box repeat the above process up to
Terminate；
The step (b1) includes the following steps:
(b1.1) target detected according to (a1), extracts the LBP feature of target area, calculation is as follows:
Wherein gcRepresent central pixel point (xc,yc) gray value；Using central pixel point as origin, R is in the field of radius
There are P pixel, gpIt is the gray value of p-th of neighborhood territory pixel point；S (x) is two-valued function, and 0 is taken when x is less than 0, is otherwise taken
1。
(b1.2) it is quantified as m grade in LBP feature space, calculates the core histogram probability density of each grade, generated
Be object module.
The step (c2) includes the following steps:
(c2.1) in view of the influence of uncertain factor in sensor detection process, false-alarm feelings may occur for target acquisition
Condition, and the remoter possibility of distance is higher, sets the max-thresholds d an of distancemaxWith minimum threshold dmin.When USV and target away from
From more than dmaxWhen, it can not generally detect target, detect target if still reporting, then it is assumed that be false-alarm, do not change planning at this time
Path；
(c2.2) when being in maximal distance threshold d with target rangemaxWith minimum threshold of distance dminBetween when, sensor energy
Detect the presence of target, but observation still has uncertainty.If the abscissa of object central point is away from picture centre at this time
Abscissa is more than mThA pixel then adjusts USV bow to angle according to the pixel number exceeded, keeps object central point horizontal
Coordinate is away from picture centre abscissa mThWithin a pixel；
(c2.3) when with object distance be less than dminWhen, it is believed that sensor detected is real goal, at this time slowly
By close-target, until certain safe distance DsWhen stop motion, the state of observed object: with Kalman Filter Estimation target
Speed of related movement v, if being less than the threshold value v of settingTh, then it is considered as static target, otherwise it is assumed that being moving target.For static
Target sends to control layer and instructs, and so that USV is done rotary motion around it in the form of face object, prevents target from moving suddenly
It is dynamic；For moving target, then enables and keep certain safe distance D between USV and targetsStablize tracking.
Unmanned water surface ship (Unmanned Surface Vehicle, abbreviation USV), is increasingly becoming the supplement of Ship platform
Or substitute, have many advantages, such as small in size, high speed, preferable stealthy, intelligent, unmanned injures and deaths, sea can be completed at lower cost
On a large scale, a variety of military and non-military operations also can be performed in scientific investigation and engineering duty for a long time, as sea area searches and rescues, leads
Boat and hydro_geography prospecting；Hydrographic information monitoring, marine meterologal prediction, aquatic organism research, exploration of ocean resources and region
Sea chart is drawn；Coastal waters zone defence；Scouting, search, detection and the removal of mines of specified sea areas；Anti-submarine warfare, anti-special operations and
Hit pirate, anti-terrorism attack etc..
USV needs autonomous navigation in the case where unmanned intervene and completes various tasks, so must have to ocean ring
Border and the entirely autonomous perception and understandability of all kinds of realizations of goal therein, wherein target detection is to can be USV with tracking
The information such as position, posture and the motion profile of waterborne target are provided.Video camera detected by receiving target ontology radiation energy and
Target is tracked, there is anti-reconnaissance capability and the stronger advantage of anti-electronic jamming capability, and the target information observed is abundant, directly
The property seen and reliability are relatively high.Currently, countries in the world, which have had a profound understanding of, develops the object detecting and tracking based on light vision
The importance and urgency of technology.It searched and rescued for sea area, specified sea areas investigation, hit the tasks such as pirate, need closely to detect
Even strike target.Therefore, the target tracking strategy for studying USV also has having very important significance.
Do detection and tracking problems faced using light vision system in USV at present to be mainly reflected in: illumination and weather become
Change will affect picture quality, the movement of object often results in objective fuzzy, this just largely reduces target detection tracking
Accuracy rate；The shake of camera caused by seawater fluctuation and USV high speed operation, not only increases challenge to target detection, to USV
Motion control will also result in very big influence.
Present invention aim to address deficiency in the prior art, provide a kind of based on the tracking of USV water surface optical target
Deep learning detection method solves the problems, such as the high false alarm rate of target detection when distant object tracking, reduces objective fuzzy and light
Interference according to variation to object detecting and tracking.
In order to achieve the above object, the technical solution adopted by the present invention are as follows: it is a kind of based on the USV water surface optical target tracking
Deep learning detection method, this method include waterborne target detection with positioning, crucial interframe target following, USV path planning and
Three systems of behaviour control, waterborne target detection and positioning system step are as follows:
(a1) USV loads video camera and acquires video in real time, and it is embedding that vision signal by image pick-up card is transmitted to the inside USV
Enter formula computer, computer first selectes a key frame every n frame, then carries out de-fuzzy processing to it, finally utilizes convolution
Neural network detects waterborne target；
(a2) central point of the target that (a1) is detected is calculated using the binocular camera binocular distance measuring method on USV to left
The distance of lens camera, by this position of the distance as target relative to video camera；Measured by the IMU carried further according to USV
Installation site of the Angle of Trim and roll angle, left lens camera of USV on USV obtains position of the target with respect to USV by calculating
It sets and angle information；
(a3) communication protocol comprising object location information is sent to USV path planning with network communication；
Crucial interframe target following step are as follows:
(b1) target detected according to (a1) extracts LBP (the Local Binary in key frame target region
Pattern, local binary patterns) feature, object module is established according to the probability density of feature submodel；
(b2) the LBP feature submodel probability density of present frame candidate target is calculated, and with similarity function metric objective
Similarity between model and target candidate model；
(b3) target position of Meanshift vector iterative search present frame is used；
(b4) when the drift distance of target's center position is less than threshold value or the number of iterations is more than threshold value, the frame target is exported
Position；
(b5) communication protocol comprising object location information is sent to USV path planning with network communication；
(b6) repeat (b2)~(b5) until the tracking of crucial interframe terminates, after return to (a1)；
USV path planning and behaviour control system step are as follows:
(c1) path planning layer circulation waits (a3), (b5) to send communication protocol, once receiving, just sums up verification,
By the location information for extracting object after verification；
(c2) according to information such as (c1) target range, angles extracted, start to carry out path planning, and result is sent to
Behaviour control layer；
(c3) USV behaviour control layer adjusts speed of the bow to angle and navigation according to the instruction of path planning layer；
Further, the step (a1) includes the following steps:
(a1.1) principle of blurred picture is generated according to sensor, the M frame image captured during exposure is used after being averaged
Camera response function is mapped as blurred picture, produces clear image and blurred picture pair with this.Pass through training convolutional neural networks
Learn the intrinsic function relationship of image pair, realizes waterborne target image deblurring；
(a1.2) waterborne target data set, 24 layer convolutional Neural net of the end-to-end training based on GoogLeNeT are made in advance
Network and 2 full articulamentums, key frame is input in trained neural network, output object boundary frame coordinate, confidence level and
Class prediction；
(a1.3) it first rejects confidence level and is less than the output of threshold value as a result, carrying out non-maximum restraining to remaining result again: to institute
Bounding box sorts from large to small formation sequence by confidence level, selects the highest bounding box of confidence level, if itself and any remaining frame
IOU be greater than threshold value the residue frame is then rejected from sequence；Selected from sequence next bounding box repeat the above process up to
Terminate；
Further, the step (b1) includes the following steps:
(b6.1) target detected according to (a1), extracts the LBP feature of target area, calculation is as follows:
Wherein gcRepresent central pixel point (xc,yc) gray value；Using central pixel point as origin, R is in the field of radius
There are P pixel, gpIt is the gray value of p-th of neighborhood territory pixel point；S (x) is two-valued function, and 0 is taken when x is less than 0, is otherwise taken
1。
(b6.2) it is quantified as m grade in LBP feature space, calculates the core histogram probability density of each grade, generated
Be object module.
Further, the step (c3) includes the following steps:
(c3.1) in view of the influence of uncertain factor in sensor detection process, false-alarm feelings may occur for target acquisition
Condition, and the remoter possibility of distance is higher, sets the max-thresholds d an of distancemaxWith minimum threshold dmin.When USV and target away from
From more than dmaxWhen, it can not generally detect target, detect target if still reporting, then it is assumed that be false-alarm, do not change planning at this time
Path；
(c3.2) when being in maximal distance threshold d with target rangemaxWith minimum threshold of distance dminBetween when, sensor energy
Detect the presence of target, but observation still has uncertainty.If the abscissa of object central point is away from picture centre at this time
Abscissa is more than mThA pixel then adjusts USV bow to angle according to the pixel number exceeded, keeps object central point horizontal
Coordinate is away from picture centre abscissa mThWithin a pixel；
(c3.3) when with object distance be less than dminWhen, it is believed that sensor detected is real goal, at this time with slow
Slow speed leans on close-target, until certain safe distance is DsWhen stop movement, the state of observed object: use Kalman filtering
The speed of related movement v of target is estimated, if being less than the threshold value v of settingTh, then it is considered as static target, otherwise it is assumed that being movement mesh
Mark.It for static target, sends and instructs to control layer, USV is made to do rotary motion around it in the form of face object, prevent
Only target moves suddenly；For moving target, then enables and keep certain safe distance D between USV and targetsStablize tracking.
A kind of deep learning detection method based on the tracking of USV water surface optical target proposed by the present invention, including walk as follows
It is rapid:
Step 1: the detection of key frame waterborne target and positioning are carried out to the video of input, save target object location and classification
Input of the information as next step, and transmit these information to foundation of the path planning layer as decision.Especially by following
Step is implemented:
1.1, first in the preparation stage, need: 1) the convolutional neural networks model of training image deblurring, make clear figure
The image pair of picture and blurred picture, wherein blurred picture uses for reference the principle that camera generates blurred picture: camera is indirect in exposure period
Light is received, clear image can accumulate stimulation every time to generate blurred picture, be simulated and be generated by following formula:
In above formula, M is the sampling frame number captured during exposure, and s [i] is i-th of articulating frame, and g is to reflect clear image
Penetrate into the camera response function of observation image.The non-linear of blurred picture and clear image is fitted by convolutional neural networks
Mapping relations realize deblurring.
2) the convolutional neural networks model of training objective detection.Production includes passenger boat, sailing boat, yacht, warship, five class of buoy
The data set of target, totally 1000, wherein training set, verifying collection and test set account for 60%, 10%, 30% respectively.Pass through migration
Learning trained convolutional layer and two full articulamentums, loss function expression formula based on GoogLeNeT end-to-endly is
In above formula, s2Indicate that image is divided into s × s lattice, B represents the bounding box number of each prediction, wherein the s of this paper
7, B is taken to take 2.I indicates that the sequence number of grid, j indicate bounding box sequence number,Indicate whether there is object on i-th of grid,WithJ-th of bounding box for respectively indicating i-th of grid is responsible for and not responsible detection object, and x, y, w, h are in object
Heart point coordinate, width and height, CiAnd piIt (c) is the probability of i-th of quadrille object and the probability for belonging to c class.
1.2 input key frame in network, obtain one 7 × 7 × 15 vector, 15 represent the p of each gridi(c) and 2
The coordinate (x, y, w, h) and C of the bounding box of a predictioni, find out pi(c)×CiGreater than the bounding box of threshold value, and with non-very big suppression
System rejects the bounding box for repeating to predict same target, and finally obtained is the target detected.
The center position of the 1.3 object boundary frame baselines obtained according to 1.2 calculate using binocular distance measuring method and be obtained
The central point of object boundary frame baseline is obtained to the distance of left lens camera, by this position of the distance as target relative to video camera
It sets；Installation of the Angle of Trim θ and roll angle Ф, left lens camera of USV measured by the IMU carried further according to USV on USV
Position obtains position of the target with respect to USV and angle information by calculating.Binocular ranging process is as shown in Figure 3.
To obtain target relative position in a manner of agreement by network communication and transmission to path planning layer as decision
Foundation.
Step 2: for crucial interframe target following, color characteristic is converted into textural characteristics, reduces illumination variation to mesh
The influence for marking tracking, implements especially by following steps:
The 2.1 coordinates of targets information obtained according to step 1.2, calculate the LBP feature of target area, calculation is as follows:
In above formula, LBPP,R(xc,yc) and gcIt is central pixel point (xc,yc) LBP characteristic value and gray value, R be middle imago
For vegetarian refreshments at a distance from neighbor pixel, P is neighbor pixel number, gpRepresent the gray value of neighbor pixel p, the value of s (x)
For two-valued function, otherwise it is 1 that when x is less than 0, value, which is 0,.
2.2 for object module and target candidate model, the probability density of u=1,2 ..., m grades of LBP feature submodels
Are as follows:
In above formula,K (x) is kernel function, and h is kernel function bandwidth, and y is mesh
Candidate region central point is marked, n is pixel number in region, xiIt is the normalization position of pixel in region, b (xi) it is xi?
Aspect indexing value in histogram.
2.3 introduce the similarity between Bhattacharyya coefficient metric objective model and target candidate model, use
The target position of Meanshift iterative search present frame makes similarity reach maximum, iteration once after kernel function central point by
y0It is moved to y1:
2.4 set primary iteration number as 0, and every iteration an iteration number adds one, when the drift distance of target's center position
When exceeding threshold value lower than threshold value or the number of iterations, the corresponding target position of the secondary iteration result and road is transferred to according to 1.3 outputs
Diameter planning layer.
Step 3: during USV path planning and behaviour control, according to target detection in step 1 and step 2 with
The target relative position positioned when tracking does different decisions to remote and short distance respectively and tracks target, specific logical
Cross following steps implementation:
3.1 in view of uncertain factor in sensor detection process influence, sensor detection probability it is opposite with target away from
From simplification relational expression it is as follows, model is as shown in Figure 4.
P in above formulaD∈ [0,1] is the detection probability of sensor, PF∈ [0,1] is the false-alarm probability of sensor, p (b (k) |
dk) it is detection probability of the sensor at the k moment, otherwise it is 0, d that b (k), which is two-value vector, and it is then 1 that the k moment, which detects target,kIt is
The relative distance of k moment USV and target.
Set the max-thresholds d an of distancemaxWith minimum threshold dmin.When being more than d with target rangemaxWhen, sensor
Detection probability is false-alarm probability, is difficult accurately to detect dbjective state at this time, detects target if still reporting, then it is assumed that is empty
It is alert, do not change the path of planning at this time.
When being in maximal distance threshold d with target rangemaxWith minimum threshold of distance dminBetween when, sensor can detect
The presence of target, but detection probability is observed with detection range linear decline and still has uncertainty in the section.At this time
If the abscissa of object central point is more than m away from picture centre abscissaThA pixel, then according to the pixel number exceeded
USV bow is adjusted to angle, keeps object central point abscissa away from picture centre abscissa mThWithin a pixel, bow is adjusted
Frequency to angle is excessively high larger to USV loss, therefore threshold value mThWhat should not be arranged is too small.
When with object distance be less than dminWhen, it is believed that sensor detected is real goal, at this time with slowly speed
Degree leans on close-target, until certain safe distance is DsWhen stop motion, with the speed of related movement of Kalman Filter Estimation target
V is with object observing motion state, the tasks such as completion close-ups even strike target.If being less than the threshold value v of settingTh, then regard
For static target, otherwise it is assumed that being moving target.For static target, sends and instruct to control layer, make it with face object
Form do rotary motion around it, prevent target from moving suddenly；For moving target, then USV is enabled to keep certain safe distance
For DsStablize tracking.
Claims (4)
1. a kind of deep learning detection method based on the tracking of USV water surface optical target is detected and positioning, key by waterborne target
Three interframe target following, USV path planning and behaviour control system compositions, which comprises the following steps:
Waterborne target detection and positioning system step are as follows:
(1.1) USV loads video camera and acquires video in real time, and vision signal is transmitted to USV by image pick-up card and is internally embedded
Formula computer, computer first selectes a key frame every n frame, then carries out de-fuzzy processing to it, finally utilizes convolution mind
Waterborne target is detected through network；
(1.2) central point of the target detected in step (1.1) is calculated using the binocular camera binocular distance measuring method on USV
To the distance of left lens camera, by this position of the distance as target relative to video camera；It is surveyed further according to the USV IMU carried
Installation site of the Angle of Trim and roll angle, left lens camera of the USV obtained on USV passes through and calculates acquisition target with respect to USV's
Position and angle information；
(1.3) communication protocol comprising object location information is sent to USV path planning with network communication；
Crucial interframe Target Tracking System step are as follows:
(2.1) target detected according to step (1.1) extracts the LBP feature in key frame target region, according to feature submodel
Probability density establish object module；
(2.2) the LBP feature submodel probability density of present frame candidate target is calculated, and with similarity function metric objective model
Similarity between target candidate model；
(2.3) target position of Meanshift vector iterative search present frame is used；
(2.4) when the drift distance of target's center position is less than threshold value or the number of iterations is more than threshold value, the frame target position is exported
It sets；
(2.5) communication protocol comprising object location information is sent to USV path planning with network communication；
(2.6) step (2.2)-(2.5) are repeated until the tracking of crucial interframe terminates, after return step (1.1)；
USV path planning and behaviour control system step are as follows:
(3.1) path planning layer circulation waiting step (1.3), (2.5) send communication protocol, once receiving, just sum up school
It tests, by the location information for extracting object after verification；
(3.2) information such as target range, angle extracted according to step (3.1) start to carry out path planning, and result are sent
To behaviour control layer；
(3.3) USV behaviour control layer adjusts speed of the bow to angle and navigation according to the instruction of path planning layer.
2. a kind of deep learning detection method based on the tracking of USV water surface optical target according to claim 1, feature
It is, the USV loads video camera and acquires video in real time, and vision signal is transmitted to USV by image pick-up card and is internally embedded
Formula computer, computer first selectes a key frame every n frame, then carries out de-fuzzy processing to it, finally utilizes convolution mind
Waterborne target is detected through network, comprising:
(1.1.1) generates the principle of blurred picture according to sensor, and the M frame image captured during exposure uses phase after being averaged
Machine receptance function is mapped as blurred picture, clear image and blurred picture pair is produced with this, by training convolutional neural networks
The intrinsic function relationship of image pair is practised, realizes waterborne target image deblurring；
(1.1.2) in advance make waterborne target data set, 24 layer convolutional neural networks of the end-to-end training based on GoogLeNeT and
2 full articulamentums, key frame is input in trained neural network, output object boundary frame coordinate, confidence level and classification
Prediction；
(1.1.3) first rejects confidence level and is less than the output of threshold value as a result, carrying out non-maximum restraining to remaining result again: to all sides
Boundary's frame sorts from large to small formation sequence by confidence level, selects the highest bounding box of confidence level, if the IOU of itself and any remaining frame
The residue frame is then rejected from sequence greater than threshold value；Next bounding box is selected from sequence to repeat the above process up to terminating.
3. a kind of deep learning detection method based on the tracking of USV water surface optical target according to claim 1, feature
It is, the target detected according to step (1.1), the LBP feature in key frame target region is extracted, according to feature submodel
Probability density establish object module, comprising:
According to the target that step (1.1) detects, the LBP feature of target area is extracted, calculation is as follows:
Wherein, gcRepresent central pixel point (xc,yc) gray value；Using central pixel point as origin, R be radius field in have P
A pixel, gpIt is the gray value of p-th of neighborhood territory pixel point；S (x) is two-valued function, and 0 is taken when x is less than 0, otherwise takes 1；
LBP feature space is quantified as m grade, calculates the core histogram probability density of each grade, generation is target
Model.
4. a kind of deep learning detection method based on the tracking of USV water surface optical target according to claim 1, feature
It is, the information such as target range, angle that the basis (3.1) is extracted start to carry out path planning, and result is sent to row
For control layer, comprising:
(3.2.1) considers that false alarm condition may occur for the influence of uncertain factor in sensor detection process, target acquisition, and
The remoter possibility of distance is higher, sets the max-thresholds d an of distancemaxWith minimum threshold dmin, when USV and target range are more than
dmaxWhen, it can not generally detect target, detect target if still reporting, then it is assumed that be false-alarm, do not change the road of planning at this time
Diameter；
(3.2.2) is when being in maximal distance threshold d with target rangemaxWith minimum threshold of distance dminBetween when, sensor can be examined
The presence of target is measured, but observation still has uncertainty, if the abscissa of object central point is away from picture centre cross at this time
Coordinate is more than mThA pixel then adjusts USV bow to angle according to the pixel number exceeded, keeps object central point abscissa
Away from picture centre abscissa mThWithin a pixel；
(3.2.3) when with object distance be less than dminWhen, it is believed that sensor detected is real goal, is slowly leaned at this time
Close-target, until certain safe distance DsWhen stop motion, the state of observed object: opposite with Kalman Filter Estimation target
Movement velocity v, if being less than the threshold value v of settingTh, then it is considered as static target, otherwise it is assumed that be moving target, for static target,
It sends and instructs to control layer, so that USV is done rotary motion around it in the form of face object, prevent target from moving suddenly；It is right
In moving target, then enables and keep certain safe distance D between USV and targetsStablize tracking.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910120133.5A CN109859202B (en) | 2019-02-18 | 2019-02-18 | Deep learning detection method based on USV water surface optical target tracking |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910120133.5A CN109859202B (en) | 2019-02-18 | 2019-02-18 | Deep learning detection method based on USV water surface optical target tracking |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109859202A true CN109859202A (en) | 2019-06-07 |
| CN109859202B CN109859202B (en) | 2022-04-12 |
Family
ID=66898273
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910120133.5A Active CN109859202B (en) | 2019-02-18 | 2019-02-18 | Deep learning detection method based on USV water surface optical target tracking |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109859202B (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110393165A (en) * | 2019-07-11 | 2019-11-01 | 浙江大学宁波理工学院 | A kind of off-lying sea cultivation net cage bait-throwing method based on Autoamtic bait putting ship |
| CN110398720A (en) * | 2019-08-21 | 2019-11-01 | 深圳耐杰电子技术有限公司 | A kind of anti-unmanned plane detection tracking interference system and photoelectric follow-up working method |
| CN110796186A (en) * | 2019-10-22 | 2020-02-14 | 华中科技大学无锡研究院 | Dry and wet garbage identification and classification method based on improved YOLOv3 network |
| CN110930433A (en) * | 2019-11-21 | 2020-03-27 | 华南理工大学 | High-speed moving object machine vision target detection method based on image restoration |
| CN111538059A (en) * | 2020-05-11 | 2020-08-14 | 东华大学 | Self-adaptive rapid dynamic positioning system and method based on improved Boltzmann machine |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20140266851A1 (en) * | 2013-03-14 | 2014-09-18 | Counter Echo Solutions LLC | System and Methods for Countering Satellite-Navigated Munitions |
| CN106773741A (en) * | 2017-03-02 | 2017-05-31 | 华南理工大学 | A kind of unmanned boat dynamic positioning system and method |
| CN107818571A (en) * | 2017-12-11 | 2018-03-20 | 珠海大横琴科技发展有限公司 | Ship automatic tracking method and system based on deep learning network and average drifting |
| CN108303078A (en) * | 2018-01-29 | 2018-07-20 | 哈尔滨工程大学 | A kind of early warning of omnidirection shipping anti-collision and navigation system based on stereoscopic vision |
| CN108459602A (en) * | 2018-02-28 | 2018-08-28 | 哈尔滨工程大学 | The autonomous method of docking of drive lacking unmanned boat under more obstacle complex environments |
| CN108803632A (en) * | 2018-09-19 | 2018-11-13 | 哈尔滨工程大学 | Unmanned surface vehicle total state constrained trajectory tracking and controlling method based on saturation compensation technology |
| CN109242019A (en) * | 2018-09-01 | 2019-01-18 | 哈尔滨工程大学 | A kind of water surface optics Small object quickly detects and tracking |
-
2019
- 2019-02-18 CN CN201910120133.5A patent/CN109859202B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20140266851A1 (en) * | 2013-03-14 | 2014-09-18 | Counter Echo Solutions LLC | System and Methods for Countering Satellite-Navigated Munitions |
| CN106773741A (en) * | 2017-03-02 | 2017-05-31 | 华南理工大学 | A kind of unmanned boat dynamic positioning system and method |
| CN107818571A (en) * | 2017-12-11 | 2018-03-20 | 珠海大横琴科技发展有限公司 | Ship automatic tracking method and system based on deep learning network and average drifting |
| CN108303078A (en) * | 2018-01-29 | 2018-07-20 | 哈尔滨工程大学 | A kind of early warning of omnidirection shipping anti-collision and navigation system based on stereoscopic vision |
| CN108459602A (en) * | 2018-02-28 | 2018-08-28 | 哈尔滨工程大学 | The autonomous method of docking of drive lacking unmanned boat under more obstacle complex environments |
| CN109242019A (en) * | 2018-09-01 | 2019-01-18 | 哈尔滨工程大学 | A kind of water surface optics Small object quickly detects and tracking |
| CN108803632A (en) * | 2018-09-19 | 2018-11-13 | 哈尔滨工程大学 | Unmanned surface vehicle total state constrained trajectory tracking and controlling method based on saturation compensation technology |
Non-Patent Citations (2)
| Title |
|---|
| GREGORY P. SCOTT ET.AL: "Autonomous robotic refueling of an unmanned surface vehicle in varying sea states", 《2015 IEEE/RSJ INTERNATIONAL CONFERENCE ON INTELLIGENT ROBOTS AND SYSTEMS (IROS)》 * |
| 周鸿祥: "无人艇的目标跟踪策略研究", 《中国优秀博硕士学位论文全文数据库(硕士)工程科技Ⅱ辑》 * |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110393165A (en) * | 2019-07-11 | 2019-11-01 | 浙江大学宁波理工学院 | A kind of off-lying sea cultivation net cage bait-throwing method based on Autoamtic bait putting ship |
| CN110393165B (en) * | 2019-07-11 | 2021-06-25 | 浙江大学宁波理工学院 | Open sea aquaculture net cage bait feeding method based on automatic bait feeding boat |
| CN110398720A (en) * | 2019-08-21 | 2019-11-01 | 深圳耐杰电子技术有限公司 | A kind of anti-unmanned plane detection tracking interference system and photoelectric follow-up working method |
| CN110398720B (en) * | 2019-08-21 | 2024-05-03 | 深圳耐杰电子技术有限公司 | Anti-unmanned aerial vehicle detection tracking interference system and working method of photoelectric tracking system |
| CN110796186A (en) * | 2019-10-22 | 2020-02-14 | 华中科技大学无锡研究院 | Dry and wet garbage identification and classification method based on improved YOLOv3 network |
| CN110930433A (en) * | 2019-11-21 | 2020-03-27 | 华南理工大学 | High-speed moving object machine vision target detection method based on image restoration |
| CN111538059A (en) * | 2020-05-11 | 2020-08-14 | 东华大学 | Self-adaptive rapid dynamic positioning system and method based on improved Boltzmann machine |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109859202B (en) | 2022-04-12 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109859202A (en) | A kind of deep learning detection method based on the tracking of USV water surface optical target | |
| CN110414396B (en) | Unmanned ship perception fusion algorithm based on deep learning | |
| CN110782481A (en) | Unmanned ship intelligent decision method and system | |
| CN110232350A (en) | A kind of real-time water surface multiple mobile object detecting and tracking method based on on-line study | |
| CN107818326A (en) | A kind of ship detection method and system based on scene multidimensional characteristic | |
| Liu et al. | Detection and pose estimation for short-range vision-based underwater docking | |
| CN110275153A (en) | A kind of waterborne target detection and tracking based on laser radar | |
| CN105184816A (en) | Visual inspection and water surface target tracking system based on USV and detection tracking method thereof | |
| Haghbayan et al. | An efficient multi-sensor fusion approach for object detection in maritime environments | |
| CN109919026B (en) | Surface unmanned ship local path planning method | |
| CN110108704A (en) | A kind of automatic monitoring and pre-alarming method of cyanobacteria and its automatic monitoring and alarming system | |
| Ren et al. | Two AUVs guidance method for self-reconfiguration mission based on monocular vision | |
| CN108830224A (en) | A kind of high-resolution remote sensing image Ship Target Detection method based on deep learning | |
| Liu et al. | Ship recognition and tracking system for intelligent ship based on deep learning framework | |
| CN116486243A (en) | DP-ViT-based sonar image target detection method | |
| Zhang et al. | Nearshore vessel detection based on Scene-mask R-CNN in remote sensing image | |
| CN115080903A (en) | Offshore multi-target multi-modal matching fusion method based on intelligent optimization algorithm | |
| Duan et al. | Unmanned aerial vehicle recognition of maritime small-target based on biological eagle-eye vision adaptation mechanism | |
| Shi et al. | Obstacle type recognition in visual images via dilated convolutional neural network for unmanned surface vehicles | |
| CN114913409A (en) | Camouflage target identification method for marine organisms | |
| Li et al. | Vision-based target detection and positioning approach for underwater robots | |
| CN109887004A (en) | A kind of unmanned boat sea area method for tracking target based on TLD algorithm | |
| Zhang et al. | Object detection and tracking method of AUV based on acoustic vision | |
| CN116246139A (en) | Target identification method based on multi-sensor fusion for unmanned ship navigation environment | |
| CN117218380A (en) | Dynamic target detection tracking method for unmanned ship remote sensing image |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |