CN112200163A - Underwater benthos detection method and system - Google Patents
Underwater benthos detection method and system Download PDFInfo
- Publication number
- CN112200163A CN112200163A CN202011393784.0A CN202011393784A CN112200163A CN 112200163 A CN112200163 A CN 112200163A CN 202011393784 A CN202011393784 A CN 202011393784A CN 112200163 A CN112200163 A CN 112200163A
- Authority
- CN
- China
- Prior art keywords
- benthos
- fusion
- anchor frame
- underwater
- detected
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 97
- 230000004927 fusion Effects 0.000 claims abstract description 96
- 238000010586 diagram Methods 0.000 claims abstract description 59
- 238000000034 method Methods 0.000 claims abstract description 31
- 238000012549 training Methods 0.000 claims abstract description 25
- 230000006870 function Effects 0.000 claims description 12
- 238000000605 extraction Methods 0.000 claims description 5
- 238000012935 Averaging Methods 0.000 claims description 3
- 238000010606 normalization Methods 0.000 claims description 3
- 238000005070 sampling Methods 0.000 description 5
- 241000251511 Holothuroidea Species 0.000 description 3
- 238000011176 pooling Methods 0.000 description 3
- 238000011897 real-time detection Methods 0.000 description 3
- 235000020637 scallop Nutrition 0.000 description 3
- 241000237503 Pectinidae Species 0.000 description 2
- 238000006467 substitution reaction Methods 0.000 description 2
- 241000257465 Echinoidea Species 0.000 description 1
- 241000237509 Patinopecten sp. Species 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000004438 eyesight Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
- 230000016776 visual perception Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/213—Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/25—Fusion techniques
- G06F18/253—Fusion techniques of extracted features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/25—Determination of region of interest [ROI] or a volume of interest [VOI]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Physics & Mathematics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Computation (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Engineering & Computer Science (AREA)
- Evolutionary Biology (AREA)
- Bioinformatics & Computational Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Multimedia (AREA)
- Biophysics (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Computing Systems (AREA)
- Molecular Biology (AREA)
- General Health & Medical Sciences (AREA)
- Computational Linguistics (AREA)
- Biomedical Technology (AREA)
- Health & Medical Sciences (AREA)
- Human Computer Interaction (AREA)
- Image Analysis (AREA)
Abstract
The invention relates to a method and a system for detecting underwater benthos, wherein the method comprises the following steps: acquiring a plurality of underwater benthos images and corresponding characteristic information; according to each underwater benthos image and corresponding characteristic information, establishing a benthos detection model, comprising the following steps: extracting a multi-dimensional characteristic diagram for each underwater benthos image; performing feature fusion on the multi-dimensional feature map based on the feature pyramid network to obtain a fusion feature map; generating an anchor frame according to the fusion characteristic diagram; adjusting the fusion characteristic diagram through an anchor frame to obtain an adjusted fusion characteristic diagram; training the RPN based on each fusion characteristic diagram and the corresponding anchor frame to obtain a trained RPN; determining an interested area based on the RPN and each anchor frame; generating a benthos detection model according to each region of interest and corresponding characteristic information; based on the benthos detection model, the to-be-detected characteristic information of the benthos to be detected can be accurately determined, and the detection precision is improved.
Description
Technical Field
The invention relates to the technical field of robot vision, in particular to a method and a system for detecting underwater benthos.
Background
The general problem of target detection has been studied extensively in recent years and researchers have maintained a great deal of interest in its theory, algorithms and applications. However, the development of underwater target detection research is slow. The underwater target detection problem is a basic problem of visual perception of the underwater robot and a key premise of autonomous operation of the underwater robot, and the detection precision and timeliness directly influence whether an autonomous operation task of the robot succeeds or not.
The underwater robot needs to have higher requirements on the real-time performance of detection in practical application due to the actual requirements of autonomous operation, which is a balance between the problem solving precision and the problem solving speed. The existing underwater target detection work is mostly based on the traditional method, namely, shallow features of underwater pictures are manually extracted, and then whether the underwater targets are included is judged through a classifier. Although the underwater target detection method has been greatly developed for decades, many problems still exist, and for the detection task of benthic organisms (sea cucumbers, scallops and sea urchins), the most critical three problems are how to improve the detection precision of deformable organisms (such as the sea cucumbers and the scallops) to the greatest extent, how to alleviate the problem of unbalanced training samples caused by uneven growth distribution of the benthic organisms and how to alleviate the problem of limited underwater visibility on the premise of ensuring the real-time performance of the algorithm.
Disclosure of Invention
In order to solve the above problems in the prior art, i.e. to improve the detection accuracy while ensuring real-time detection, the invention aims to provide a method and a system for detecting underwater benthos.
In order to solve the technical problems, the invention provides the following scheme:
an underwater benthos detection method, comprising:
acquiring a plurality of underwater benthos images and corresponding characteristic information;
according to each underwater benthos image and corresponding characteristic information, a benthos detection model is established, and the method specifically comprises the following steps:
extracting a multi-dimensional feature map from each underwater benthos image;
performing feature fusion on the multi-dimensional feature map based on the feature pyramid network to obtain a fusion feature map with reduced dimensions;
generating an anchor frame according to the fusion characteristic diagram;
adjusting the fusion characteristic diagram through the anchor frame to obtain an adjusted fusion characteristic diagram;
training the RPN based on each fusion characteristic diagram and the corresponding anchor frame to obtain a trained RPN;
determining an interested area based on the trained RPN and each anchor frame;
generating a benthos detection model according to each region of interest and corresponding characteristic information;
and determining the characteristic information to be detected of the benthos to be detected according to the image of the benthos to be detected based on the benthos detection model.
Optionally, the feature map is determined according to the following formula:
wherein,is thatk-thThe modulation factor of the location of the position,which represents the image after the normalization process,representing the feature map calculated by the modulated deformable convolution module,a weight that can be learned is represented,pthe location of the sample is represented by,for the firstkA predetermined offset from the position of the seat,a value representing the amount of shift that can be learned,k-this shown askOne position, K, represents the number of sample positions a convolution kernel has.
Optionally, generating an anchor frame according to the fused feature map specifically includes:
according to the fusion feature map, respectively predicting the shape and the position of an anchor frame through a first convolution branch and a second convolution molecule;
and decoding the two shapes and the two positions to obtain the anchor frame corresponding to the fusion characteristic diagram.
Optionally, the Loss function of the first convolution branch is Focal local; the penalty function for the first convolution branch is BoundedIOU Loss.
Optionally, the training the RPN network based on each fusion feature map and the corresponding anchor frame to obtain a trained RPN network specifically includes:
determining the IOU (input output Unit) ratio of the intersection and the union of the two frames based on each fusion feature graph and the corresponding anchor frame;
classifying the corresponding anchor frame into a positive sample anchor frame or a negative sample anchor frame according to the size of the IOU;
and respectively training the RPN according to the positive sample anchor frame and the negative sample anchor frame to obtain the trained RPN.
Optionally, determining the characteristic information to be detected of the benthos to be detected according to the image of the benthos to be detected based on the benthos detection model, specifically including:
extracting a multi-dimensional characteristic graph to be detected of the benthic organism image to be detected;
determining an interested region from a multi-dimensional characteristic map to be detected based on the benthic organism detection model;
distributing each region of interest to the corresponding feature graph to be tested according to the area of the region of interest, and determining the corresponding coordinate region;
uniformly taking a plurality of reference points from the coordinate area aiming at each coordinate area;
for each reference point, determining four reference points closest to the reference point from the characteristic diagram to be detected;
calculating an output value of the reference point according to the four reference points and the reference point by adopting a bilinear interpolation method;
and averaging the output values of the reference points to obtain the output information of the coordinate area, wherein the output information of the coordinate area is the characteristic information to be detected of the benthic organisms to be detected.
Optionally, the underwater benthos detection method further comprises:
connecting each piece of feature information to be tested to a fully connected network;
calculating an IOU value corresponding to each interested area;
classifying the corresponding region of interest into a positive sample region or a negative sample region according to the size of the IOU value;
determining a loss function of the benthic organism detection model from the positive and negative sample regions;
and correcting the benthos detection model according to the loss function of the benthos detection model.
In order to solve the technical problems, the invention also provides the following scheme:
an underwater benthos detection system, comprising:
the acquisition unit is used for acquiring a plurality of underwater benthos images and corresponding characteristic information;
the modeling unit is used for establishing a benthos detection model according to each underwater benthos image and corresponding characteristic information;
the modeling unit includes:
the extraction module is used for extracting a multi-dimensional feature map from each underwater benthos image;
the fusion module is used for carrying out feature fusion on the multi-dimensional feature map based on the feature pyramid network to obtain a fusion feature map with reduced dimensions;
the generating module is used for generating an anchor frame according to the fusion characteristic diagram;
the adjusting module is used for adjusting the fusion characteristic diagram through the anchor frame to obtain an adjusted fusion characteristic diagram;
the training module is used for training the RPN based on each fusion characteristic diagram and the corresponding anchor frame to obtain a trained RPN;
the determining module is used for determining the region of interest based on the trained RPN and each anchor frame;
the modeling module is used for generating a benthos detection model according to each region of interest and the corresponding characteristic information;
and the detection unit is used for determining the characteristic information to be detected of the benthos to be detected according to the image of the benthos to be detected based on the benthos detection model.
In order to solve the technical problems, the invention also provides the following scheme:
an underwater benthic organism detection system comprising:
a processor; and
a memory arranged to store computer executable instructions that, when executed, cause the processor to:
acquiring a plurality of underwater benthos images and corresponding characteristic information;
according to each underwater benthos image and corresponding characteristic information, a benthos detection model is established, and the method specifically comprises the following steps:
extracting a multi-dimensional feature map from each underwater benthos image;
performing feature fusion on the multi-dimensional feature map based on the feature pyramid network to obtain a fusion feature map with reduced dimensions;
generating an anchor frame according to the fusion characteristic diagram;
adjusting the fusion characteristic diagram through the anchor frame to obtain an adjusted fusion characteristic diagram;
training the RPN based on each fusion characteristic diagram and the corresponding anchor frame to obtain a trained RPN;
determining an interested area based on the trained RPN and each anchor frame;
generating a benthos detection model according to each region of interest and corresponding characteristic information;
and determining the characteristic information to be detected of the benthos to be detected according to the image of the benthos to be detected based on the benthos detection model.
In order to solve the technical problems, the invention also provides the following scheme:
a computer-readable storage medium storing one or more programs that, when executed by an electronic device including a plurality of application programs, cause the electronic device to:
acquiring a plurality of underwater benthos images and corresponding characteristic information;
according to each underwater benthos image and corresponding characteristic information, a benthos detection model is established, and the method specifically comprises the following steps:
extracting a multi-dimensional feature map from each underwater benthos image;
performing feature fusion on the multi-dimensional feature map based on the feature pyramid network to obtain a fusion feature map with reduced dimensions;
generating an anchor frame according to the fusion characteristic diagram;
adjusting the fusion characteristic diagram through the anchor frame to obtain an adjusted fusion characteristic diagram;
training the RPN based on each fusion characteristic diagram and the corresponding anchor frame to obtain a trained RPN;
determining an interested area based on the trained RPN and each anchor frame;
generating a benthos detection model according to each region of interest and corresponding characteristic information;
and determining the characteristic information to be detected of the benthos to be detected according to the image of the benthos to be detected based on the benthos detection model.
According to the embodiment of the invention, the invention discloses the following technical effects:
the benthos detection model is established based on the multiple underwater benthos images and the corresponding characteristic information, detection of benthos can be achieved in real time, the multi-dimensional characteristic diagram is extracted from the underwater benthos images, the dimension reduction characteristic fusion is carried out, the anchor frame is generated, the RPN network is trained, the region of interest is determined, the characteristic information to be detected of the benthos to be detected can be accurately determined, and the detection precision is high.
Drawings
FIG. 1 is a flowchart of the method for detecting underwater benthos according to the present invention;
FIG. 2 is a schematic block diagram of the underwater benthos detection system of the present invention.
Description of the symbols:
the system comprises an acquisition unit-1, a modeling unit-2, an extraction module-21, a fusion module-22, a generation module-23, an adjustment module-24, a training module-25, a determination module-26, a modeling module-27 and a detection unit-3.
Detailed Description
Preferred embodiments of the present invention are described below with reference to the accompanying drawings. It should be understood by those skilled in the art that these embodiments are only for explaining the technical principle of the present invention, and are not intended to limit the scope of the present invention.
The invention aims to provide an underwater benthos detection method, which is characterized in that a benthos detection model is established based on a plurality of underwater benthos images and corresponding characteristic information, real-time detection of benthos can be realized, and the detection precision is high by extracting a multi-dimensional characteristic diagram from the underwater benthos images, reducing the dimension characteristic fusion, generating an anchor frame, training an RPN network and determining an interested region, so that the to-be-detected characteristic information of the benthos to be detected can be accurately determined.
In order to make the aforementioned objects, features and advantages of the present invention comprehensible, embodiments accompanied with figures are described in further detail below.
As shown in FIG. 1, the underwater benthos detection method of the present invention comprises:
step 100: acquiring a plurality of underwater benthos images and corresponding characteristic information;
step 200: according to each underwater benthos image and corresponding characteristic information, a benthos detection model is established, and the method specifically comprises the following steps:
step 210: extracting a multi-dimensional feature map from each underwater benthos image;
step 220: performing feature fusion on the multi-dimensional feature map based on the feature pyramid network to obtain a fusion feature map with reduced dimensions;
step 230: generating an anchor frame according to the fusion characteristic diagram;
step 240: adjusting the fusion characteristic diagram through the anchor frame to obtain an adjusted fusion characteristic diagram;
step 250: training the RPN based on each fusion characteristic diagram and the corresponding anchor frame to obtain a trained RPN;
step 260: determining an interested area based on the trained RPN and each anchor frame;
step 270: generating a benthos detection model according to each region of interest and corresponding characteristic information;
step 300: and determining the characteristic information to be detected of the benthos to be detected according to the image of the benthos to be detected based on the benthos detection model.
In step 200, the acquired image of the underwater benthos is normalized to obtain dimensions of 256 × 512 × 1024 × 2048. In this embodiment, an image is read from a camera fixedly connected to a robot, and the image with the adjusted size passes through a backbone network. In order to relieve the problem of geometric deformation of non-rigid benthos (such as sea cucumber and scallop), the invention replaces the convolution layers in the third, fourth and fifth convolution groups with variability convolution.
Specifically, in step 210, a feature map is determined according to the following formula:
wherein,is thatk-thThe modulation factor of the location of the position,which represents the image after the normalization process,representing the feature map calculated by the modulated deformable convolution module,a weight that can be learned is represented,pthe location of the sample is represented by,for the firstkA predetermined offset from the position of the seat,a value representing the amount of shift that can be learned,k-this shown askEach position, K represents oneThe convolution kernel has a number of sample locations.
In step 220, to extract the multi-scale features, the present invention introduces the feature pyramid FPN method into the baseline method. Inputting feature maps with dimensions of 256 × 512 × 1024 × 2048, and obtaining 5 feature maps after passing through a feature pyramid network.
In step 230, generating an anchor frame according to the fused feature map, which specifically includes:
step 231: and respectively predicting the shape and the position of the anchor frame through the first convolution branch and the second convolution molecule according to the fusion feature map.
Wherein the Loss function of the first convolution branch is Focal local; the penalty function for the first convolution branch is BoundedIOU Loss.
Step 232: and decoding the two shapes and the two positions to obtain the anchor frame corresponding to the fusion characteristic diagram.
In step 240, the shape information of the anchor frame is merged into the merged feature map, and the branch result of the anchor frame shape obtained in step 230 is input to the bias term of the 3 × 3 deformable convolution after being subjected to a 1 × 1 convolution.
In step 250, the training an RPN network based on each fusion feature map and the corresponding anchor frame to obtain a trained RPN network specifically includes:
step 251: and determining the ratio IOU of the intersection and the union of the two frames based on each fusion feature graph and the corresponding anchor frame.
Step 252: and classifying the corresponding anchor frame into a positive sample anchor frame or a negative sample anchor frame according to the size of the IOU.
Matching all Anchor frames with the labels in the characteristic information, and judging whether the Anchor frame Anchor contains a sample according to the size of the IOU:
when the IOU is more than or equal to 0.7, the Anchor frame Anchor contains samples, namely the Anchor frame Anchor is divided into positive sample Anchor frames; otherwise, the Anchor frame Anchor does not contain the sample, namely the Anchor frame Anchor is divided into negative sample Anchor frames.
Step 253: and respectively training the RPN according to the positive sample anchor frame and the negative sample anchor frame to obtain the trained RPN.
In step 260, the information of RPN network prediction and the corresponding Anchor are decoded to generate a set of prediction boxes, i.e. ROI (region of interest). At the moment, the number of the prediction frames is huge, and in order to improve the quality of the prediction frames, the positive sample anchor frame and the negative sample anchor frame are randomly sampled, and 256 sample anchor frames are sampled.
Further, in step 300, in order to ensure that the feature dimensions corresponding to each ROI are consistent and eliminate the error of ROI Pooling during the two approximate roundings as much as possible, the present invention adds ROI Align Pooling to the baseline method.
Preferably, the determining the characteristic information to be detected of the benthos to be detected based on the benthos detection model according to the image of the benthos to be detected specifically includes:
step 310: and extracting a multi-dimensional characteristic diagram to be detected of the benthic organism image to be detected.
Step 320: and determining a region of interest from the multi-dimensional characteristic map to be detected based on the benthic organism detection model.
Step 330: and distributing each region of interest to the corresponding feature map to be measured according to the area of the region of interest, and determining the corresponding coordinate region.
Step 340: and uniformly taking a plurality of reference points from the coordinate area for each coordinate area.
Step 350: and for each reference point, determining four reference points closest to the reference point from the characteristic diagram to be detected.
Step 360: and calculating the output value of the reference point according to the four reference points and the reference point by adopting a bilinear interpolation method.
Step 370: and averaging the output values of the reference points to obtain the output information of the coordinate area, wherein the output information of the coordinate area is the characteristic information to be detected of the benthic organisms to be detected.
In addition, the underwater benthos detection method of the invention further comprises:
step 410: and connecting each piece of feature information to be tested to the full-connection network.
Specifically, 256-dimensional characteristic information to be measured obtained after ROI Align Pooling is connected to a full-connection network.
Step 420: and calculating the IOU value corresponding to each interested area.
Step 410: the corresponding regions of interest are classified as either positive or negative sample regions depending on the size of the IOU value.
Specifically, if the IOU is greater than or equal to 0.5, the region of interest is a positive sample region, otherwise, the region of interest is a negative sample region.
Positive sample area sampling: sampling is carried out according to the proportion of 1:3, and 512 samples are sampled in total. For negative sample regions, the present invention employs an IOU balanced sampling strategy, as opposed to ordinary random sampling.
And dividing the screened negative samples into K buckets according to the corresponding IOU size.
wherein,Nwhich represents the number of selected samples, is,is shown asThe number of candidate samples of the sub-iteration, and the maximum threshold value of the T iteration times. In the present embodiment, it is preferred that,Tthe value is 3.
Step 440: determining a loss function of the benthic organism detection model from the positive and negative sample regions.
Step 450: and correcting the benthos detection model according to the loss function of the benthos detection model.
The method adopts a method based on modularized deformable convolution to relieve the problem that the marine life is easy to deform after being disturbed. The underwater visibility of part of sea areas is limited, so that the robot cannot approach a target in a short distance in order to see clear sea creatures, and an overlong or too wide target appears in a visual field. Aiming at the problem of unbalanced distribution of underwater samples, the invention adopts barrel-dividing random sampling of collected samples according to IOU numerical values. According to the method, the GSDCN algorithm customized for the sea area of the yellow sea is adopted, and the highest detection precision of the URPC2018 data set is obtained on the premise of not adopting a universal skill.
In addition, the invention also provides an underwater benthos detection system which can ensure real-time detection and improve detection precision.
As shown in fig. 2, the underwater benthos detection system of the present invention includes an acquisition unit 1, a modeling unit 2, and a detection unit 3.
The acquisition unit 1 is used for acquiring a plurality of underwater benthos images and corresponding characteristic information;
the modeling unit 2 is used for establishing a benthos detection model according to the images of the underwater benthos and the corresponding characteristic information.
The detection unit 3 is used for determining the characteristic information to be detected of the benthos to be detected according to the image of the benthos to be detected based on the benthos detection model.
Further, the modeling unit 2 includes an extraction module 21, a fusion module 22, a generation module 23, an adjustment module 24, a training module 25, a determination module 26, and a modeling module 27.
Specifically, the extraction module 21 is configured to extract a multi-dimensional feature map from each underwater benthos image;
the fusion module 22 is configured to perform feature fusion on the multi-dimensional feature map based on the feature pyramid network to obtain a fusion feature map with reduced dimensions;
the generating module 23 is configured to generate an anchor frame according to the fusion feature map;
the adjusting module 24 is configured to adjust the fusion feature map through the anchor frame to obtain an adjusted fusion feature map;
the training module 25 is configured to train the RPN network based on each fusion feature map and the corresponding anchor frame to obtain a trained RPN network;
the determining module 26 is configured to determine an area of interest based on the trained RPN network and each anchor frame;
the modeling module 27 is configured to generate a benthic organism detection model according to each region of interest and the corresponding feature information.
In addition, the invention also provides the following scheme:
an underwater benthic organism detection system comprising:
a processor; and
a memory arranged to store computer executable instructions that, when executed, cause the processor to:
acquiring a plurality of underwater benthos images and corresponding characteristic information;
according to each underwater benthos image and corresponding characteristic information, a benthos detection model is established, and the method specifically comprises the following steps:
extracting a multi-dimensional feature map from each underwater benthos image;
performing feature fusion on the multi-dimensional feature map based on the feature pyramid network to obtain a fusion feature map with reduced dimensions;
generating an anchor frame according to the fusion characteristic diagram;
adjusting the fusion characteristic diagram through the anchor frame to obtain an adjusted fusion characteristic diagram;
training the RPN based on each fusion characteristic diagram and the corresponding anchor frame to obtain a trained RPN;
determining an interested area based on the trained RPN and each anchor frame;
generating a benthos detection model according to each region of interest and corresponding characteristic information;
and determining the characteristic information to be detected of the benthos to be detected according to the image of the benthos to be detected based on the benthos detection model.
Further, the invention also provides the following scheme:
a computer-readable storage medium storing one or more programs that, when executed by an electronic device including a plurality of application programs, cause the electronic device to:
acquiring a plurality of underwater benthos images and corresponding characteristic information;
according to each underwater benthos image and corresponding characteristic information, a benthos detection model is established, and the method specifically comprises the following steps:
extracting a multi-dimensional feature map from each underwater benthos image;
performing feature fusion on the multi-dimensional feature map based on the feature pyramid network to obtain a fusion feature map with reduced dimensions;
generating an anchor frame according to the fusion characteristic diagram;
adjusting the fusion characteristic diagram through the anchor frame to obtain an adjusted fusion characteristic diagram;
training the RPN based on each fusion characteristic diagram and the corresponding anchor frame to obtain a trained RPN;
determining an interested area based on the trained RPN and each anchor frame;
generating a benthos detection model according to each region of interest and corresponding characteristic information;
and determining the characteristic information to be detected of the benthos to be detected according to the image of the benthos to be detected based on the benthos detection model.
Compared with the prior art, the underwater benthos detection system and the computer readable storage medium have the same beneficial effects as the underwater benthos detection method, and are not repeated herein.
So far, the technical solutions of the present invention have been described in connection with the preferred embodiments shown in the drawings, but it is easily understood by those skilled in the art that the scope of the present invention is obviously not limited to these specific embodiments. Equivalent changes or substitutions of related technical features can be made by those skilled in the art without departing from the principle of the invention, and the technical scheme after the changes or substitutions can fall into the protection scope of the invention.
Claims (10)
1. An underwater benthos detection method, comprising:
acquiring a plurality of underwater benthos images and corresponding characteristic information;
according to each underwater benthos image and corresponding characteristic information, a benthos detection model is established, and the method specifically comprises the following steps:
extracting a multi-dimensional feature map from each underwater benthos image;
performing feature fusion on the multi-dimensional feature map based on the feature pyramid network to obtain a fusion feature map with reduced dimensions;
generating an anchor frame according to the fusion characteristic diagram;
adjusting the fusion characteristic diagram through the anchor frame to obtain an adjusted fusion characteristic diagram;
training the RPN based on each fusion characteristic diagram and the corresponding anchor frame to obtain a trained RPN;
determining an interested area based on the trained RPN and each anchor frame;
generating a benthos detection model according to each region of interest and corresponding characteristic information;
and determining the characteristic information to be detected of the benthos to be detected according to the image of the benthos to be detected based on the benthos detection model.
2. The underwater benthos detection method according to claim 1, wherein the feature map is determined according to the following formula:
wherein,is thatk-thThe modulation factor of the location of the position,which represents the image after the normalization process,representing the feature map calculated by the modulated deformable convolution module,a weight that can be learned is represented,pthe location of the sample is represented by,for the firstkA predetermined offset from the position of the seat,a value representing the amount of shift that can be learned,k-th: first, thekOne position, K, represents the number of sample positions a convolution kernel has.
3. The method for detecting the underwater benthos according to claim 1, wherein generating an anchor frame according to the fused feature map specifically comprises:
according to the fusion feature map, respectively predicting the shape and the position of an anchor frame through a first convolution branch and a second convolution molecule;
and decoding the two shapes and the two positions to obtain the anchor frame corresponding to the fusion characteristic diagram.
4. The method of detecting benthic organisms according to claim 3 wherein the Loss function of the first convolution branch is Focal local; the penalty function for the first convolution branch is BoundedIOU Loss.
5. The method for detecting underwater benthos according to claim 1, wherein the training of the RPN network based on each fused feature map and the corresponding anchor frame to obtain the trained RPN network specifically comprises:
determining the IOU (input output Unit) ratio of the intersection and the union of the two frames based on each fusion feature graph and the corresponding anchor frame;
classifying the corresponding anchor frame into a positive sample anchor frame or a negative sample anchor frame according to the size of the IOU;
and respectively training the RPN according to the positive sample anchor frame and the negative sample anchor frame to obtain the trained RPN.
6. The method for detecting the underwater benthos according to claim 1, wherein the determining the characteristic information to be detected of the benthos to be detected according to the image of the benthos to be detected based on the benthos detection model specifically includes:
extracting a multi-dimensional characteristic graph to be detected of the benthic organism image to be detected;
determining an interested region from a multi-dimensional characteristic map to be detected based on the benthic organism detection model;
distributing each region of interest to the corresponding feature graph to be tested according to the area of the region of interest, and determining the corresponding coordinate region;
uniformly taking a plurality of reference points from the coordinate area aiming at each coordinate area;
for each reference point, determining four reference points closest to the reference point from the characteristic diagram to be detected;
calculating an output value of the reference point according to the four reference points and the reference point by adopting a bilinear interpolation method;
and averaging the output values of the reference points to obtain the output information of the coordinate area, wherein the output information of the coordinate area is the characteristic information to be detected of the benthic organisms to be detected.
7. The underwater benthos detection method according to claim 6, further comprising:
connecting each piece of feature information to be tested to a fully connected network;
calculating an IOU value corresponding to each interested area;
classifying the corresponding region of interest into a positive sample region or a negative sample region according to the size of the IOU value;
determining a loss function of the benthic organism detection model from the positive and negative sample regions;
and correcting the benthos detection model according to the loss function of the benthos detection model.
8. An underwater benthos detection system, comprising:
the acquisition unit is used for acquiring a plurality of underwater benthos images and corresponding characteristic information;
the modeling unit is used for establishing a benthos detection model according to each underwater benthos image and corresponding characteristic information;
the modeling unit includes:
the extraction module is used for extracting a multi-dimensional feature map from each underwater benthos image;
the fusion module is used for carrying out feature fusion on the multi-dimensional feature map based on the feature pyramid network to obtain a fusion feature map with reduced dimensions;
the generating module is used for generating an anchor frame according to the fusion characteristic diagram;
the adjusting module is used for adjusting the fusion characteristic diagram through the anchor frame to obtain an adjusted fusion characteristic diagram;
the training module is used for training the RPN based on each fusion characteristic diagram and the corresponding anchor frame to obtain a trained RPN;
the determining module is used for determining the region of interest based on the trained RPN and each anchor frame;
the modeling module is used for generating a benthos detection model according to each region of interest and the corresponding characteristic information;
and the detection unit is used for determining the characteristic information to be detected of the benthos to be detected according to the image of the benthos to be detected based on the benthos detection model.
9. An underwater benthic organism detection system comprising:
a processor; and
a memory arranged to store computer executable instructions that, when executed, cause the processor to:
acquiring a plurality of underwater benthos images and corresponding characteristic information;
according to each underwater benthos image and corresponding characteristic information, a benthos detection model is established, and the method specifically comprises the following steps:
extracting a multi-dimensional feature map from each underwater benthos image;
performing feature fusion on the multi-dimensional feature map based on the feature pyramid network to obtain a fusion feature map with reduced dimensions;
generating an anchor frame according to the fusion characteristic diagram;
adjusting the fusion characteristic diagram through the anchor frame to obtain an adjusted fusion characteristic diagram;
training the RPN based on each fusion characteristic diagram and the corresponding anchor frame to obtain a trained RPN;
determining an interested area based on the trained RPN and each anchor frame;
generating a benthos detection model according to each region of interest and corresponding characteristic information;
and determining the characteristic information to be detected of the benthos to be detected according to the image of the benthos to be detected based on the benthos detection model.
10. A computer-readable storage medium storing one or more programs that, when executed by an electronic device including a plurality of application programs, cause the electronic device to:
acquiring a plurality of underwater benthos images and corresponding characteristic information;
according to each underwater benthos image and corresponding characteristic information, a benthos detection model is established, and the method specifically comprises the following steps:
extracting a multi-dimensional feature map from each underwater benthos image;
performing feature fusion on the multi-dimensional feature map based on the feature pyramid network to obtain a fusion feature map with reduced dimensions;
generating an anchor frame according to the fusion characteristic diagram;
adjusting the fusion characteristic diagram through the anchor frame to obtain an adjusted fusion characteristic diagram;
training the RPN based on each fusion characteristic diagram and the corresponding anchor frame to obtain a trained RPN;
determining an interested area based on the trained RPN and each anchor frame;
generating a benthos detection model according to each region of interest and corresponding characteristic information;
and determining the characteristic information to be detected of the benthos to be detected according to the image of the benthos to be detected based on the benthos detection model.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202011393784.0A CN112200163B (en) | 2020-12-03 | 2020-12-03 | Underwater benthos detection method and system |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202011393784.0A CN112200163B (en) | 2020-12-03 | 2020-12-03 | Underwater benthos detection method and system |
Publications (2)
Publication Number | Publication Date |
---|---|
CN112200163A true CN112200163A (en) | 2021-01-08 |
CN112200163B CN112200163B (en) | 2021-06-25 |
Family
ID=74033683
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN202011393784.0A Active CN112200163B (en) | 2020-12-03 | 2020-12-03 | Underwater benthos detection method and system |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN112200163B (en) |
Cited By (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN112766340A (en) * | 2021-01-11 | 2021-05-07 | 中山大学 | Depth capsule network image classification method and system based on adaptive spatial mode |
CN113343964A (en) * | 2021-08-09 | 2021-09-03 | 湖南汇视威智能科技有限公司 | Balanced underwater acoustic image target detection method |
CN113762229A (en) * | 2021-11-10 | 2021-12-07 | 山东天亚达新材料科技有限公司 | Intelligent identification method and system for building equipment in building site |
Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20160300127A1 (en) * | 2014-05-05 | 2016-10-13 | Atomwise Inc. | Systems and methods for applying a convolutional network to spatial data |
CN109034245A (en) * | 2018-07-27 | 2018-12-18 | 燕山大学 | A kind of object detection method merged using characteristic pattern |
CN110751185A (en) * | 2019-09-26 | 2020-02-04 | 高新兴科技集团股份有限公司 | Training method and device of target detection model |
CN111126399A (en) * | 2019-12-28 | 2020-05-08 | 苏州科达科技股份有限公司 | Image detection method, device and equipment and readable storage medium |
CN111242071A (en) * | 2020-01-17 | 2020-06-05 | 陕西师范大学 | Attention remote sensing image target detection method based on anchor frame |
-
2020
- 2020-12-03 CN CN202011393784.0A patent/CN112200163B/en active Active
Patent Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20160300127A1 (en) * | 2014-05-05 | 2016-10-13 | Atomwise Inc. | Systems and methods for applying a convolutional network to spatial data |
CN109034245A (en) * | 2018-07-27 | 2018-12-18 | 燕山大学 | A kind of object detection method merged using characteristic pattern |
CN110751185A (en) * | 2019-09-26 | 2020-02-04 | 高新兴科技集团股份有限公司 | Training method and device of target detection model |
CN111126399A (en) * | 2019-12-28 | 2020-05-08 | 苏州科达科技股份有限公司 | Image detection method, device and equipment and readable storage medium |
CN111242071A (en) * | 2020-01-17 | 2020-06-05 | 陕西师范大学 | Attention remote sensing image target detection method based on anchor frame |
Cited By (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN112766340A (en) * | 2021-01-11 | 2021-05-07 | 中山大学 | Depth capsule network image classification method and system based on adaptive spatial mode |
CN112766340B (en) * | 2021-01-11 | 2024-06-04 | 中山大学 | Depth capsule network image classification method and system based on self-adaptive spatial mode |
CN113343964A (en) * | 2021-08-09 | 2021-09-03 | 湖南汇视威智能科技有限公司 | Balanced underwater acoustic image target detection method |
CN113343964B (en) * | 2021-08-09 | 2022-06-10 | 湖南汇视威智能科技有限公司 | Balanced underwater acoustic image target detection method |
CN113762229A (en) * | 2021-11-10 | 2021-12-07 | 山东天亚达新材料科技有限公司 | Intelligent identification method and system for building equipment in building site |
Also Published As
Publication number | Publication date |
---|---|
CN112200163B (en) | 2021-06-25 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN112200163B (en) | Underwater benthos detection method and system | |
CN109145830B (en) | Intelligent water gauge identification method | |
CN110728658A (en) | High-resolution remote sensing image weak target detection method based on deep learning | |
CN103218787B (en) | Multi-source heterogeneous remote sensing image reference mark automatic acquiring method | |
CN111626993A (en) | Image automatic detection counting method and system based on embedded FEFnet network | |
CN113052109A (en) | 3D target detection system and 3D target detection method thereof | |
CN112800964A (en) | Remote sensing image target detection method and system based on multi-module fusion | |
CN104200495A (en) | Multi-target tracking method in video surveillance | |
Lou et al. | Accurate multi-view stereo 3D reconstruction for cost-effective plant phenotyping | |
CN103198477A (en) | Apple fruitlet bagging robot visual positioning method | |
CN113850865A (en) | Human body posture positioning method and system based on binocular vision and storage medium | |
CN110827312A (en) | Learning method based on cooperative visual attention neural network | |
CN115187803B (en) | Positioning method for picking process of famous tea tender shoots | |
CN109165603B (en) | Ship detection method and device | |
CN110633711A (en) | Computer device and method for training feature point detector and feature point detection method | |
CN116079749B (en) | Robot vision obstacle avoidance method based on cluster separation conditional random field and robot | |
CN113657423A (en) | Target detection method suitable for small-volume parts and stacked parts and application thereof | |
CN108681691A (en) | A kind of marine ships and light boats rapid detection method based on unmanned water surface ship | |
CN117495891B (en) | Point cloud edge detection method and device and electronic equipment | |
Li et al. | Vision-based target detection and positioning approach for underwater robots | |
CN113628170A (en) | Laser line extraction method and system based on deep learning | |
CN117152601A (en) | Underwater target detection method and system based on dynamic perception area routing | |
CN110345919A (en) | Space junk detection method based on three-dimensional space vector and two-dimensional plane coordinate | |
CN114140698A (en) | Water system information extraction algorithm based on FasterR-CNN | |
CN114463628A (en) | Deep learning remote sensing image ship target identification method based on threshold value constraint |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
GR01 | Patent grant | ||
GR01 | Patent grant |