CN111062383A - Image-based ship detection depth neural network algorithm - Google Patents

Image-based ship detection depth neural network algorithm Download PDF

Info

Publication number
CN111062383A
CN111062383A CN201911065233.9A CN201911065233A CN111062383A CN 111062383 A CN111062383 A CN 111062383A CN 201911065233 A CN201911065233 A CN 201911065233A CN 111062383 A CN111062383 A CN 111062383A
Authority
CN
China
Prior art keywords
ship
prediction
image
neural network
box
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201911065233.9A
Other languages
Chinese (zh)
Inventor
邵叶秦
丁政年
李志伟
马雪仪
李�杰
向阳
施佺
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Nantong University
Original Assignee
Nantong University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Nantong University filed Critical Nantong University
Priority to CN201911065233.9A priority Critical patent/CN111062383A/en
Publication of CN111062383A publication Critical patent/CN111062383A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/25—Determination of region of interest [ROI] or a volume of interest [VOI]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • Artificial Intelligence (AREA)
  • Biomedical Technology (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Health & Medical Sciences (AREA)
  • Multimedia (AREA)
  • Image Analysis (AREA)

Abstract

The invention discloses an image-based ship detection depth neural network algorithm, which comprises the following steps: firstly, sampling an image to enable the length and the width of the image to be twice of the original length and width; dividing the image into grids of S-S, predicting B boundary predictions for each grid, and giving 6 parameters including ship positions, confidence degrees and classification probabilities for each boundary prediction; thirdly, extracting the characteristics of each grid through a cascade cavity convolution neural network, realizing multi-resolution ship boundary prediction through characteristic fusion, and determining the position of a ship; and fourthly, designing a loss function, and balancing the prediction frame with the hull part and the prediction frame without the hull part by setting different scale factors. Based on the research of the convolutional neural network in the field of computer vision, the invention integrates classification and regression in a deep neural network for multi-target real-time detection on the basis of feature learning, and has strong accuracy and rapidity.

Description

Image-based ship detection depth neural network algorithm
Technical Field
The invention belongs to the field of ship detection, and particularly relates to an image-based ship detection deep neural network algorithm (SD-DCNN).
Background
The automatic detection of ships based on images is one of the basic problems of automatic driving and safe driving of ships and is also an important problem in the field of computer vision. In particular, since the small vessels are not equipped with equipment such as ais (automatic identification system), they cannot actively report information such as their own position, and are not easily sensed by other vessels, and thus, they are likely to cause a collision of the vessels. The traditional ship detection method adopts manual characteristics, the accuracy and robustness of the algorithm are insufficient, and false detection and missing detection can occur. The existing deep learning-based method can realize automatic detection of ships, but detection omission easily occurs to small ships, and driving safety is affected. Therefore, current ship driving mainly depends on manual observation to realize safe running of the ship.
Disclosure of Invention
The purpose of the invention is as follows: in order to solve the defects of the prior art, the invention provides an image-based ship detection deep neural network algorithm (SD-DCNN), which particularly considers small ships on the basis of detecting large and medium ships so as to realize the automatic perception of various ships.
The technical scheme is as follows: an image-based ship detection deep neural network algorithm is characterized in that: the method comprises the following detection processes:
firstly, sampling an image to enable the length and the width of the image to be 2 times of the original length and width, and extracting initial features by convolution operation;
secondly, dividing the image into grids of S X S, predicting B ship boundaries for each grid, wherein each boundary prediction gives 6 parameters, namely X, Y, W, H, SHIPCFIdence and SHIPPPro, wherein (X, Y) is the central horizontal coordinate of a ship body prediction frame and the central vertical coordinate of the ship body prediction frame; (W, H) is the width of the hull prediction box and the height of the hull prediction box; SHIPPORO is the probability that the object in the prediction frame belongs to the ship; the SHIPCFIdence is the credibility of the ship body in the ship body prediction frame;
SHIPConfidence＝Pr(ship)*IOU(pred|truth) (1)
wherein, Pr(ship) represents whether or not a hull exists in the prediction box:
Figure BDA0002259114790000011
IOU (pred | truth) represents the intersection ratio of the prediction box to the real box:
Figure BDA0002259114790000012
bb (pred) is a prediction box based on training data; BB (treth) is a real frame during target detection; area (·) represents area calculation;
thirdly, extracting features through a cascade cavity convolution neural network, realizing multi-resolution ship boundary prediction through feature fusion, and determining the position of a ship; the convolution operation with the length of 2 is adopted each time to realize downsampling;
and fourthly, designing a loss function, and setting different scale factors to make corresponding proportional balance for the prediction frame with the hull part and the prediction frame without the hull part.
As an optimization: and the third step is specifically realized by realizing a cascade cavity convolution neural network on the basis of the original deep convolution neural network to extract specific characteristics of the ship target and fusing the characteristics into a high-level network to extract high-level abstract characteristics.
As an optimization: the third step is specifically realized by giving three prediction results and adjusting the parameters of the network simultaneously in combination with different resolutions instead of giving one prediction result when the ship boundary is predicted.
As an optimization: in the fourth step, the loss function is as follows:
Figure BDA0002259114790000021
wherein the content of the first and second substances,
Figure BDA0002259114790000022
judging whether the jth prediction boundary box in the low i networks is responsible for judging the ship body target;
Figure BDA0002259114790000023
Figure BDA0002259114790000024
predicting the coordinates of the target object;
Figure BDA0002259114790000025
a prediction of confidence for a prediction bounding box containing a hull portion;
Figure BDA0002259114790000026
a prediction of confidence for a prediction bounding box that does not contain a hull portion;
Figure BDA0002259114790000027
to determine if the center of the hull falls within the grid.
Has the advantages that: based on the research of the convolutional neural network in the field of computer vision, the invention integrates classification and regression in a deep neural network for multi-target real-time detection on the basis of feature learning, and has strong accuracy and rapidity.
Different from the traditional method that the features (such as Local Binary Pattern features and Historic of Oriented Gradient features) are extracted firstly in target detection, a classification model (such as Support Vector Machine) is adopted to obtain a target, the SD-DCNN algorithm uses the regression idea, the features are not required to be designed manually, a classifier or a locator is not required to be trained independently, and only the target region prediction and the target category prediction are required to be integrated into a single neural network model. The method adopts a single neural network to unify candidate frame extraction, feature extraction, target classification and target positioning, and realizes end-to-end target detection. The SD-DCNN algorithm is compared with other object detection methods such as RCNN and Fast RCNN in the aspects of detection speed and accuracy, and the SD-DCNN has obvious advantages in the aspect of speed.
Drawings
FIG. 1 is a schematic diagram of a general process for SD-DCNN target detection according to the present invention;
FIG. 2 is a schematic diagram of the SD-DCNN network structure of the present invention; wherein, (a) the ship detects the deep neural network; (b) cascading a hole convolution module;
FIG. 3 is a diagram illustrating the SD-DCNN network results of the present invention; wherein, the rectangular box represents the detection result of the ship.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below so that those skilled in the art can better understand the advantages and features of the present invention, and thus the scope of the present invention will be more clearly defined. The embodiments described herein are only a few embodiments of the present invention, rather than all embodiments, and all other embodiments that can be derived by one of ordinary skill in the art without inventive faculty based on the embodiments described herein are intended to fall within the scope of the present invention.
Examples
Based on the research of the convolutional neural network in the field of computer vision, the classification and regression are fused in a deep neural network on the basis of feature learning to carry out multi-target real-time detection, and specifically, the invention relates to an image-based ship detection deep neural network algorithm (SD-DCNN).
SD-CDNN target detection Process
Since there are a plurality of ship targets in the image, each prediction frame needs to be discriminated. The specific detection process is as follows:
firstly, sampling an image to enable the length and the width of the image to be 2 times of the original length and width, and extracting initial features by convolution operation;
secondly, dividing the image into grids (Grid cells) of S, predicting B Bounding Boxes (Bounding Boxes) for each Grid, wherein each frame gives 6 parameters, namely X, Y, W, H, SHIPCFIdence and SHIPPPro, wherein (X, Y) are the central horizontal coordinate of the ship body prediction box and the central vertical coordinate of the ship body prediction box; (W, H) is the width of the hull prediction box and the height of the hull prediction box; SHIPPORO is the probability that the object in the prediction frame belongs to the ship; the SHIPCFIdence is the credibility of the ship body in the ship body prediction frame;
SHIPConfidence＝Pr(ship)*IOU(pred|truth) (1)
wherein, Pr(ship) represents whether or not a hull exists in the prediction box:
Figure BDA0002259114790000041
IOU (pred | truth) represents the intersection ratio of the prediction box to the real box:
Figure BDA0002259114790000042
bb (pred) is a prediction box based on training data; BB (treth) is a real frame during target detection; area (·) represents the area.
And thirdly, extracting features through a cascade cavity convolution neural network, and realizing ship boundary prediction through feature fusion. The specific implementation is that the cascaded cavity convolution neural network is realized on the basis of the original deep convolution neural network and is used for extracting specific characteristics of a ship target, and the characteristics are fused to predict the boundary of the ship in a multi-resolution mode so as to determine the position of the ship.
And fourthly, designing a loss function, wherein the effect of the loss function is to make corresponding proportional balance for the prediction box with the hull part and the prediction box without the hull part by setting different scale factors. The loss function is of the form:
Figure BDA0002259114790000043
wherein the content of the first and second substances,
Figure BDA0002259114790000044
judging whether the jth prediction boundary box in the low i networks is responsible for judging the ship body target;
Figure BDA0002259114790000045
Figure BDA0002259114790000046
predicting the coordinates of the target object;
Figure BDA0002259114790000047
a prediction of confidence for a prediction bounding box containing a hull portion;
Figure BDA0002259114790000048
a prediction of confidence for a prediction bounding box that does not contain a hull portion;
Figure BDA0002259114790000049
to determine if the center of the hull falls within the grid.
The general flow diagram of the SD-DCNN target detection is shown in FIG. 1 below.
SD-DCNN training network
The images are input into an SD-DCNN training set, the SD-DCNN automatically divides the input images into 13-13 grids, if the center of the ship body is located in the grids, the SD-DCNN considers the grids to be responsible for detecting the ship body part, each grid needs to process two pre-selection frames, 6 parameters in each pre-selection frame need to be determined, and the pre-selection frames are X, Y, W, H, SHIPConfigudence and SHIPPRO.
Two prediction boxes need to predict 12 parameters. Furthermore, as can be seen from equation 5, if it is necessary to predict the class probability Pr (class | ship) under the target condition, the probability of determining the ship hull by a certain prediction box can be obtained by multiplying Pr (ship) and Pr (class | ship). Where Pr (ship) is the probability of the hull type, and Pr (class | ship) is the determination probability of the presence of a ship in the present prediction box.
Pr(class)＝Pr(class|ship)*Pr(ship) (5)
The design structure of the SD-DCNN network is shown in FIG. 2.
3. Testing SD-DCNN algorithm
In the process of predicting the SD-DCNN, the output type of the network needs to be judged to obtain the grid with the target, and the position information of the prediction frame corresponding to the grid is obtained, so that the corresponding target detection can be completed.
And (3) outputting two prediction boxes by each grid, predicting 6 parameters by each prediction box, and finally obtaining the output of 338 prediction box parameters, wherein the probability of the ship body is set to be 0.2 by formula 5, the ship body is not the ship body target when the probability is less than 0.2, and the ship body is the target ship body when the probability is more than 0.2. Arranging the ship positions in the prediction frames according to the descending order of the probability, combining the overlapped prediction frames, obtaining the final ship position through continuous repetition, and marking the ship position by a rectangular frame, as shown in figure 3.
Analysis of detection results of SD-DCNN algorithm
(1) Rapidity of operation
The Fast RCNN and SD-DCNN detectors were tested separately under CPU and GPU, and the results are shown in Table 1.
TABLE 1 detection speed contrast (frames/sec)
Figure BDA0002259114790000051
Under the CPU, the real-time performance of the CPU and the CPU cannot be met; under the GPU, the detection speed of Fast RCNN is far lower than twenty-four frames per second, while the detection performance of SD-DCNN is excellent.
(2) Accuracy of
The error rates of SD-DCNN and Fast RCNN were measured, and the results are shown in Table 2 below.
TABLE 2 detection accuracy comparison (error Rate)
Figure BDA0002259114790000061
As can be seen from the table, SD-DCNN has advantages in detection speed and higher accuracy.
The experimental environment of the invention is as follows: intel (R) core (TM) i5-5200U CPU @2.20GHz 2.20GHz4.00GB, NVIDIA TITAN XP, Ubuntu64 bit operating system. Under the experimental environment, the speed of system detection can meet the requirements of real-time performance and accuracy.
According to the invention, the method for detecting the multiple targets of the ship in real time is successful through actual video analysis, and has strong accuracy and rapidity.

Claims (6)

1. An image-based ship detection deep neural network algorithm is characterized in that: the method comprises the following detection processes:
firstly, sampling an image to enable the length and the width of the image to be 2 times of the original length and width, and extracting initial features by convolution operation;
secondly, dividing the image into grids of S X S, predicting B bounding boxes for each grid, and giving 6 parameters to each frame, namely X, Y, W, H, SHIPCFIdence and SHIPPPro, wherein (X, Y) is the central horizontal coordinate of the ship body prediction box and the central vertical coordinate of the ship body prediction box; (W, H) is the width of the hull prediction box and the height of the hull prediction box; SHIPPORO is the probability that the object in the prediction frame belongs to the ship; the SHIPCFIdence is the credibility of the ship body in the ship body prediction frame;
SHIPConfidence＝Pr(ship)*IOU(pred|truth) (1)
wherein, Pr(ship) represents whether or not a hull exists in the prediction box:
Figure FDA0002259114780000011
IOU (pred | truth) represents the intersection ratio of the prediction box to the real box:
Figure FDA0002259114780000012
bb (pred) is a prediction box based on training data; BB (treth) is a real frame during target detection; area (·) represents area calculation;
thirdly, extracting features through a cascade cavity convolution neural network, realizing multi-resolution ship boundary prediction through feature fusion, and determining the position of a ship; the convolution operation with the length of 2 is adopted each time to realize downsampling;
and fourthly, designing a loss function, and carrying out corresponding proportional balance on the prediction frame with the hull part and the prediction frame without the hull part by setting different scale factors.
2. The image-based ship detection depth neural network algorithm of claim 1, wherein: in the first step, in order to take the small ship which is difficult to detect into account, the image is sampled by linear interpolation in the specific implementation process, the image is used for amplifying the small ship, the initial characteristic is extracted, and the accurate detection of the subsequent ship is facilitated.
3. The image-based ship detection depth neural network algorithm of claim 1, wherein: and in the third step, in order to take small ships which are difficult to detect into account, the cascade cavity convolution neural network is realized on the basis of the original deep convolution neural network for extracting the specific characteristics of the ship target in the concrete implementation process, and the characteristics are fused into the high-level network to extract the high-level abstract characteristics, so that rich characteristic information is obtained.
4. The image-based ship detection depth neural network algorithm of claim 1, wherein: in the third step, in order to take small ships which are difficult to detect into account, the extraction of ship features and the prediction of ship boundaries are realized under multiple resolutions in specific implementation.
5. The image-based ship detection depth neural network algorithm of claim 1, wherein: and in the third step, in order to take the small ships which are difficult to detect into account, the convolution operation with the step length of 2 is adopted to reduce the image size in each down-sampling process, instead of the simple pooling operation, so that the information of the small ships is kept as much as possible.
6. The image-based ship detection depth neural network algorithm of claim 1, wherein: in the fourth step, the loss function is as follows:
Figure FDA0002259114780000021
wherein the content of the first and second substances,
Figure FDA0002259114780000022
judging whether the jth prediction boundary box in the low i networks is responsible for judging the ship body target;
Figure FDA0002259114780000023
Figure FDA0002259114780000024
predicting the coordinates of the target object;
Figure FDA0002259114780000025
a prediction of confidence for a prediction bounding box containing a hull portion;
Figure FDA0002259114780000026
a prediction of confidence for a prediction bounding box that does not contain a hull portion;
Figure FDA0002259114780000027
to determine if the center of the hull falls within the grid.
CN201911065233.9A 2019-11-04 2019-11-04 Image-based ship detection depth neural network algorithm Pending CN111062383A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201911065233.9A CN111062383A (en) 2019-11-04 2019-11-04 Image-based ship detection depth neural network algorithm

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201911065233.9A CN111062383A (en) 2019-11-04 2019-11-04 Image-based ship detection depth neural network algorithm

Publications (1)

Publication Number Publication Date
CN111062383A true CN111062383A (en) 2020-04-24

Family

ID=70297584

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201911065233.9A Pending CN111062383A (en) 2019-11-04 2019-11-04 Image-based ship detection depth neural network algorithm

Country Status (1)

Country Link
CN (1) CN111062383A (en)

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111738295A (en) * 2020-05-22 2020-10-02 南通大学 Image segmentation method and storage medium
CN112348742A (en) * 2020-11-03 2021-02-09 北京信工博特智能科技有限公司 Image nonlinear interpolation obtaining method and system based on deep learning
CN113158787A (en) * 2021-03-11 2021-07-23 上海海事大学 Ship detection classification method under complex marine environment
CN113160253A (en) * 2020-12-29 2021-07-23 南通大学 Three-dimensional medical image segmentation method based on sparse mark and storage medium
CN113763484A (en) * 2021-09-17 2021-12-07 交通运输部水运科学研究所 Ship target positioning and speed estimation method based on video image analysis technology
CN113837086A (en) * 2021-09-24 2021-12-24 南通大学 Reservoir phishing person detection method based on deep convolutional neural network

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108256634A (en) * 2018-02-08 2018-07-06 杭州电子科技大学 A kind of ship target detection method based on lightweight deep neural network
CN110147807A (en) * 2019-01-04 2019-08-20 上海海事大学 A kind of ship intelligent recognition tracking

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108256634A (en) * 2018-02-08 2018-07-06 杭州电子科技大学 A kind of ship target detection method based on lightweight deep neural network
CN110147807A (en) * 2019-01-04 2019-08-20 上海海事大学 A kind of ship intelligent recognition tracking

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
杨名等: "甚高速区域卷积神经网络的船舶视频检测方法" *

Cited By (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111738295A (en) * 2020-05-22 2020-10-02 南通大学 Image segmentation method and storage medium
CN111738295B (en) * 2020-05-22 2024-03-22 南通大学 Image segmentation method and storage medium
CN112348742A (en) * 2020-11-03 2021-02-09 北京信工博特智能科技有限公司 Image nonlinear interpolation obtaining method and system based on deep learning
CN112348742B (en) * 2020-11-03 2024-03-26 北京信工博特智能科技有限公司 Image nonlinear interpolation acquisition method and system based on deep learning
CN113160253A (en) * 2020-12-29 2021-07-23 南通大学 Three-dimensional medical image segmentation method based on sparse mark and storage medium
CN113160253B (en) * 2020-12-29 2024-01-30 南通大学 Three-dimensional medical image segmentation method based on sparse markers and storage medium
CN113158787A (en) * 2021-03-11 2021-07-23 上海海事大学 Ship detection classification method under complex marine environment
CN113158787B (en) * 2021-03-11 2024-04-05 上海海事大学 Ship detection and classification method under complex marine environment
CN113763484A (en) * 2021-09-17 2021-12-07 交通运输部水运科学研究所 Ship target positioning and speed estimation method based on video image analysis technology
CN113837086A (en) * 2021-09-24 2021-12-24 南通大学 Reservoir phishing person detection method based on deep convolutional neural network

Similar Documents

Publication Publication Date Title
CN111062383A (en) Image-based ship detection depth neural network algorithm
CN108961235B (en) Defective insulator identification method based on YOLOv3 network and particle filter algorithm
CN110610166B (en) Text region detection model training method and device, electronic equipment and storage medium
CN113688652B (en) Abnormal driving behavior processing method and device
CN111784685A (en) Power transmission line defect image identification method based on cloud edge cooperative detection
CN112633149B (en) Domain-adaptive foggy-day image target detection method and device
CN106446896A (en) Character segmentation method and device and electronic equipment
CN112836639A (en) Pedestrian multi-target tracking video identification method based on improved YOLOv3 model
CN115205264A (en) High-resolution remote sensing ship detection method based on improved YOLOv4
CN110598698B (en) Natural scene text detection method and system based on adaptive regional suggestion network
CN114155527A (en) Scene text recognition method and device
CN113487600B (en) Feature enhancement scale self-adaptive perception ship detection method
CN111882620A (en) Road drivable area segmentation method based on multi-scale information
CN114332473A (en) Object detection method, object detection device, computer equipment, storage medium and program product
CN115761513A (en) Intelligent remote sensing identification method for mountain large landslide based on semi-supervised deep learning
CN112800934A (en) Behavior identification method and device for multi-class engineering vehicle
CN114581744A (en) Image target detection method, system, equipment and storage medium
CN113361528B (en) Multi-scale target detection method and system
CN112561885B (en) YOLOv 4-tiny-based gate valve opening detection method
CN113344148A (en) Marine ship target identification method based on deep learning
CN111767919A (en) Target detection method for multi-layer bidirectional feature extraction and fusion
CN111476226A (en) Text positioning method and device and model training method
CN112487920B (en) Convolution neural network-based crossing behavior identification method
CN114898290A (en) Real-time detection method and system for marine ship
CN114743045A (en) Small sample target detection method based on double-branch area suggestion network

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination