CN109543679A - A kind of dead fish recognition methods and early warning system based on depth convolutional neural networks - Google Patents

A kind of dead fish recognition methods and early warning system based on depth convolutional neural networks Download PDF

Info

Publication number
CN109543679A
CN109543679A CN201811365779.1A CN201811365779A CN109543679A CN 109543679 A CN109543679 A CN 109543679A CN 201811365779 A CN201811365779 A CN 201811365779A CN 109543679 A CN109543679 A CN 109543679A
Authority
CN
China
Prior art keywords
fish
layer
dead fish
image
neural networks
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201811365779.1A
Other languages
Chinese (zh)
Inventor
尹绍武
谢万里
王涛
张红燕
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Nanjing Normal University
Original Assignee
Nanjing Normal University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Nanjing Normal University filed Critical Nanjing Normal University
Priority to CN201811365779.1A priority Critical patent/CN109543679A/en
Publication of CN109543679A publication Critical patent/CN109543679A/en
Pending legal-status Critical Current

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00Scenes; Scene-specific elements
    • G06V20/60Type of objects
    • G06V20/62Text, e.g. of license plates, overlay texts or captions on TV images
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00Pattern recognition
    • G06F18/20Analysing
    • G06F18/24Classification techniques
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00Pattern recognition
    • G06F18/20Analysing
    • G06F18/25Fusion techniques
    • G06F18/253Fusion techniques of extracted features
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/04Architecture, e.g. interconnection topology
    • G06N3/045Combinations of networks
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/08Learning methods

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • General Physics & Mathematics (AREA)
  • Evolutionary Computation (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • General Engineering & Computer Science (AREA)
  • Computing Systems (AREA)
  • Software Systems (AREA)
  • Molecular Biology (AREA)
  • Computational Linguistics (AREA)
  • Biophysics (AREA)
  • Biomedical Technology (AREA)
  • Mathematical Physics (AREA)
  • General Health & Medical Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Evolutionary Biology (AREA)
  • Multimedia (AREA)
  • Image Analysis (AREA)
  • Image Processing (AREA)

Abstract

The invention discloses a kind of dead fish recognition methods and early warning system based on deep neural network, this method comprises: acquisition includes the image of live fish and dead fish, the coordinate position of dead fish and live fish in whole image is labeled, using the image after mark as the input of neural network;Use depth convolutional neural networks as feature extractor, feature extraction is carried out to the image of input;The feature of extraction is sent into RPN network, and determination is background or fish body, and is returned to its frame coordinate;The output of RPN network is sent to the pond ROI layer, then accesses full articulamentum, classified and returned, and judgement is live fish or dead fish and calculates coordinate position.Early warning system based on this method proposition can fish and the dead fish quantity turned in the water surface be faced upward in detection automatically in the case where no supervision, and are sounded an alarm.The present invention is that rescue live fish avoids the general pool of large area from striving for valuable time, the powerful guarantee economic benefit of aquatic products practitioner, and promotes the sound development of culture fishery.

Description

A kind of dead fish recognition methods and early warning system based on depth convolutional neural networks
Technical field
The present invention relates to aquatic products intelligence cultural methods, and in particular to a kind of dead fish knowledge based on depth convolutional neural networks Other method and early warning system.
Background technique
In aquaculture industry, tradition is mainly artificial field observation or by farm for the supervision of aquatic products It lays camera and carries out remote reviewing, due to can not manually ensure the enough supervision time, and bring additional labour branch Out, aquatic products lead to Large Scale Death often due to lacking supervision, every year due to a lack of aquatic products Large Scale Death caused by supervising Example it is countless, bring serious burden to national economy, significantly limit the development of aquatic products industry.
In recent years, although scholar has developed some aquatic products Internet of Things equipment to monitor aquaculture situation, such as automatic aeration Equipment, water quality detection system etc. meet the production needs of aquatic products practitioner to a certain extent.But these equipment mainly focus In the monitoring to factors such as water quality, environment, it is often relatively simple that detection is related to project, such as oxygen content, salt compounds or often See heavy metal ion, cannot accomplish comprehensive detection.Since there is only the deteriorations of common water pollution at present, there are also viruses It invades, Changes in weather, and the reasons such as rare metal caused by the various techniques or material with social production continuous renewal, Existing aquatic products monitoring device is caused increasingly to be unable to satisfy the demand of fisherman.On the one hand can to by increasing detection project Equipment cost is greatly improved, it, can not very in time on the other hand with the continuous renewal of people's production requirement and operation mode It is able to update and realize.Further for much stress stronger fish, if assisting cultivation work by traditional water quality monitoring alarm Make, being also easier to occur alarm opportunity lags behind the fish reaction time and miss rescue good opportunity.Therefore, it is necessary to a kind of higher It imitates, is more convenient, more accurate dead fish early warning mechanism.
Summary of the invention
Goal of the invention: in view of the deficiencies of the prior art, the present invention provides a kind of based on the dead of depth convolutional neural networks Fish recognition methods can realize dead fish quickly, accurately, automatically or turn over the identification on the pool.
Another object of the present invention is to provide a kind of dead fish early warning systems based on depth convolutional neural networks.
Technical solution: a kind of dead fish recognition methods based on depth convolutional neural networks, by constructing Faster rcnn frame The depth convolutional neural networks framework of structure is identified and is classified to dead fish, comprising the following steps:
S10, acquisition include the image of live fish and dead fish, are marked to the coordinate position of dead fish and live fish in whole image Note, using the image after mark as the input of neural network;
S20, use depth convolutional neural networks as feature extractor, feature extraction is carried out to the image of input;
S30, RPN network is sent into the output of depth convolutional neural networks, determines that individual features are background or fish body, and Its frame coordinate is returned;
The output of S40, RPN network is sent to the pond ROI layer, then accesses full articulamentum, while being classified and returning behaviour Make, judgement is live fish or dead fish and calculates coordinate position.
Preferably, depth convolutional neural networks include 5 layers of convolutional layer, 3 pond layers in the step S20, and two connect entirely Layer is connect, specific arrangement is as follows:
First layer is the feature that convolutional layer is used to extract dead fish, and the size of convolution kernel is 7*7, step-length 2*2,96 convolution Core;
The second layer is pond layer for compressing to the characteristic pattern of input, and the size of pond layer is 3*3, and step-length is 2*2;
Third layer is that convolutional layer further extracts feature, is mapped to high dimensional feature, includes 256 convolution kernels, the convolution of 5*5 Core size, step-length 2*2;
4th layer is used for dimensionality reduction for pond layer, and the size of pond layer is 3*3, and step-length is 2*2;
Layer 5 is that convolutional layer further extracts feature, is mapped to high dimensional feature, includes 384 convolution kernels, and convolution kernel is big Small is 3*3;
Layer 6 is that convolutional layer further extracts feature, is mapped to high dimensional feature, includes 384 convolution kernels, and convolution kernel is big Small is 3*3;
Layer 7 is that convolutional layer further extracts feature, is mapped to high dimensional feature, includes 256 convolution kernels, 3*3 size Convolution kernel, step-length are 1*1;
8th layer is pond layer for Feature Dimension Reduction, and the size of pond layer is 3*3, and step-length is 2*2;
9th layer is full articulamentum, first by the characteristic pattern flattening on upper layer, then accesses full articulamentum, is used for Fusion Features;
The tenth layer of full articulamentum in position, for output of classifying.
Preferably, when the depth convolutional neural networks carry out feature extraction, using the depth network in ImageNet collection The weight of upper pre-training is initial weight.
Preferably, RPN network learns prediction by marking with the more similar proposal of real border frame in the step S30 Region, comprising the following steps:
After (31) first convolutional networks generate several features by convolutional layer, each pixel on characteristic pattern, Generate several different size of candidate region frames;
(32) feature is consistent according to the position on the relative position and original image on characteristic pattern, then passes through a volume Product network regards two classification and recurrence as to the candidate region of generation, judges that it is background or fish image;
(33) background image is given up, retains the characteristic pattern containing fish image.
Preferably, classify in the step S40 to fish and classified by softmax function:
Wherein, hθIt is the output of network concealed layer neuron, θ, θ 1, θ 2 is the parameter of model, in Controlling model Value minimizes it error function, x(i)It is i-th of dead fish of acquisition or the image pattern of live fish, y(i)It is i-th of input sample This corresponding classification,It is so that probability distribution and for 1.
A kind of dead fish early warning system based on depth convolutional neural networks, the system comprises image collecting devices, image Processing unit and prior-warning device, wherein image collecting device captures fish image and is transmitted to image processing apparatus, image procossing Device using the method for any of claims 1-5, the depth convolutional neural networks based on Faster rcnn framework into The positioning and identification of row dead fish, dynamic detection dead fish, and be automatically positioned its target position, when detecting that dead fish quantity is more than pre- When the item number first set, image processing apparatus sends to prior-warning device and instructs, and prompts that alarm occurs, staff is reminded to check.
Preferably, the prior-warning device is connected to the mobile end equipment of raiser by mobile network, dead when detecting When fish is greater than specified item number, prior-warning device issues alarm sound signal in cultivation site, while to the mobile end equipment of raiser Dead fish information is transmitted, alarm signal occurs in mobile terminal.
Preferably, described image acquisition device includes camera, and the camera is arranged in cultivation with default spacing isolation In cylinder.
Preferably, the system also includes multiple plane mirrors, the multiple plane mirror is separately positioned on the left and right of culturing jar The opposite of two sides and camera.
The utility model has the advantages that
1, the present invention combines computer vision technique and depth learning technology, realizes aquaculture and is fully automated inspection It surveys, fish Large Scale Death phenomenon can be measured in real time and be given warning in advance, striven for preciousness for rescue live fish resource Time has greatly ensured the economic interests of aquatic products practitioner.
2, the present invention realizes the detection of dead fish by building deep neural network, belongs to deep learning scope, compared to biography The biological character recognition methods of system has higher precision and generalization ability, can sufficiently meet production requirement.
3, the non-destructive testing of the invention by computer vision technique, so that not having to contact water body and fish body realizes to work The detection of life and death state during fish culture, this touchless mode, so that the harm to aquatic products is preferably minimized.
4, the present invention can be simultaneously emitted by two-way alarm when detecting a certain number of dead fishes, and alarm is at the scene all the way It sounds an alarm, another way is sent in the mobile end equipment of aquatic products practitioner by aquatic products Internet of Things equipment, sufficiently conveyed early warning Information.
5, system structure of the invention is simple, and deployment is convenient, constructs the environment at complete or collected works visual angle using plane mirror, is suitble to water Industry is produced to promote the use of.
Detailed description of the invention
Fig. 1 is dead fish recognition methods flow chart according to an embodiment of the present invention;
Fig. 2 is the structural block diagram of dead fish early warning system according to an embodiment of the present invention;
Fig. 3 is the architecture diagram of Faster rcnn network according to an embodiment of the present invention;
Fig. 4 is the recognition effect figure according to an embodiment of the present invention in the lesser situation of fish image;
Fig. 5 is the recognition effect figure according to an embodiment of the present invention in the lesser situation of fish image;
Fig. 6 is the recognition effect figure according to an embodiment of the present invention in the case where fish image is relatively fuzzy;
Fig. 7 is the response diagram of depth convolutional neural networks according to an embodiment of the present invention;
Fig. 8 is the evolution graph of depth convolutional neural networks according to an embodiment of the present invention;
Fig. 9 is the software interface of early warning system according to an embodiment of the present invention.
Specific embodiment
Technical solution of the present invention is described further with reference to the accompanying drawing.
Referring to Fig.1, a kind of dead fish recognition methods based on depth convolutional neural networks, comprising the following steps: S10, acquisition Image including live fish and dead fish is labeled the coordinate position of dead fish and live fish in whole image, by the figure after mark As the input as neural network;S20, use depth convolutional neural networks as feature extractor, the image of input is carried out Feature extraction;S30, RPN network is sent into the output of depth convolutional neural networks, determines that individual features are background or fish body, And its frame coordinate is returned;The output of S40, RPN network is sent to the pond ROI layer, then accesses full articulamentum, while into Row classification and recurrence operation, judgement are live fish or dead fish and calculate coordinate position.
Referring to Fig. 2, the dead fish early warning system proposed based on this method includes image collecting device, image processing apparatus and pre- Alarm device, wherein image collecting device uses camera, to allow camera sufficiently to capture the image of fish body, according to water The fish that production culturing jar size sets certain density is placed in one, and certain area of isolation is arranged in culturing jar, culturing jar The opposite of the left and right sides and video camera is respectively provided with a face plane mirror, for expanding the field range of camera.Due to Faster- RCNN network query function amount is larger, and the image that camera captures need to accelerate to calculate by carrying the server of high performance video cards.Service Device carries out the positioning and identification of dead fish based on the depth convolutional neural networks of Faster rcnn framework, and dynamic detection dead fish is (herein Dead fish also includes facing upward the fish for turning over moribund condition), and it is automatically positioned its target position over the display, when detecting dead fish quantity When reaching preset threshold value item number, image processing apparatus, that is, server sends to prior-warning device and instructs, and prompts that alarm occurs, The situation of possible occurrence of large-area fish death at this time, reminds staff to check.
General frame such as Fig. 3 institute used in image processing apparatus for the depth convolutional neural networks of dead fish identification Show.Its major network uses the Faster rcnn network architecture, mainly includes ZFnet network and RPN (Region Proposal Networks) two submodules of network, wherein ZFnet network is used for feature extraction using depth convolutional neural networks, extracts Feature be sent into RPN network (dotted portion in figure), tentatively corrected for distinguishing background with fish object pixel and prediction block.It After be sequentially ingressed into interest region (ROI) pond layer, a full articulamentum, finally again execute class prediction function and bounding box it is pre- Brake.
The method process for carrying out dead fish early warning according to the network architecture of above-mentioned building is as follows:
Dead fish and live fish authentic specimen are acquired, obtained true picture is marked into fish body (live fish and dead using annotation tool Fish) coordinate position, obtain the coordinate in the upper left corner and the coordinate in the lower right corner.Collected picture and coordinate are incorporated into Faster Rcnn network is trained.The initial weight of network is initial weight of the depth network in ImageNet pre-training, ImageNet It is a computer vision system identification project name, is the current maximum database of image recognition in the world, this is smooth by the U.S. The identifying system that the computer scientist of good fortune simulates the mankind is established.The weight of initialization has been trained on other data set Network weight so that network is having certain recognition capability and generalization ability at the very start.Specific training process is such as Under:
Fish body characteristics of image is extracted by the convolutional neural networks of main body (ZFnet network) first, the feature of extraction is sent into RPN network.Specifically, ZFnet network has following structure:
(1) first layer is that convolutional layer is used to extract the feature of dead fish, and the size of convolution kernel is 7*7, step-length 2*2,96 Convolution kernel is for extracting feature.
(2) second layer is pond layer, and the size of pond layer is 3*3, and step-length is 2*2.Pond layer to the characteristic pattern of input into Row compression, on the one hand makes characteristic pattern become smaller, and simplifies network query function complexity;On the one hand Feature Compression is carried out, main feature is extracted.
(3) it includes 256 convolution kernels, the convolution kernel size of 5*5, step-length 2*2, further with volume that third layer, which is convolutional layer, Lamination extracts feature, is mapped to high dimensional feature.
(4) the 4th layers are pond layer, and the size of pond layer is 3*3, and step-length is 2*2, further dimensionality reduction.
(5) layer 5 is convolutional layer, includes 384 convolution kernels, convolution kernel size is 3*3, is further extracted with convolutional layer Feature is mapped to high dimensional feature.
(6) layer 6 is convolutional layer, includes 384 convolution kernels, convolution kernel size is 3*3, is further extracted with convolutional layer Feature is mapped to high dimensional feature.
(7) it includes 256 convolution kernels that layer 7, which is convolutional layer, and the convolution kernel of 3*3 size, step-length is 1*1, further with volume Lamination extracts feature, is mapped to high dimensional feature.
(8) the 8th layers are pond layers, and the size of pond layer is 3*3, and step-length is 2*2, Feature Dimension Reduction.
(9) the 9th layers are full articulamentum.First by the characteristic pattern flattening on upper layer, then full articulamentum is accessed, melted for feature It closes.
(10) the tenth layers are full articulamentum, for output of classifying
After the characteristic pattern that RPN network is inputted, convolution operation is carried out by characteristic pattern of the sliding window to input, often Characteristic pattern is mapped to lower dimension by a sliding window, is carried out two classification to the output of the vector, that is, is determined the feature of input It is background or fish body, and its frame coordinate is returned.RPN network is made of four parts: convolutional layer, anchor frame, classification Prediction module, bounding box prediction module, convolutional layer effect mainly have two: extracting pixel characteristic and transmit to next layer, and sit Mark returns;Anchor frame refers to the frame corresponding region generated on each pixel to characteristic pattern;Class prediction module, boundary Frame prediction module has each served as judge whether it is background, the effect that frame returns.
Specifically, the training process of RPN network is as follows:
(1) anchor frame (anchor) is generated.Centered on each pixel, multiple sizes anchor frame different with ratio and right is generated The mark answered.Each anchor frame is indicated using corresponding 256 dimensional feature of its center pixel.For the anchor frame of generation, IOU is used (Intersection-over-Union) it selects positive and negative sample set to carry out network training, selects rule is defined as:The class that is positive is assert when working as IOU>0.7, i.e. current pixel block is identified as fish, as IOU<0.3 It waits and assert the class that is negative, is i.e. current pixel block is the background not comprising fish, and other to do discard processing, IOU defines candidate frame and mark Determine the degree of overlapping of frame.The position coordinates and background value of fish manually marked out before training use, as background or fish this The true value of two classification problems is constantly trained, after training, this set of network parameters is just as the true value that frame returns May determine that whether fish and coordinate are at which.Manually the coordinate that mark also maps to come and the coordinate of anchor frame so before It is compared.
(2) as follows using gradient decline and error backpropagation algorithm, process when training:
Step 1: determining the input X (x of network1,x2,...,xv) and output Y (y1,y2,...yg), node in hidden layer. The connection weight w of initialization systemijAnd wjk, threshold value b, learning rate and activation primitive, common activation primitive Sigmoid are as follows:
Step 2: the connection weight w according to input data X, between input layer and hidden layerijAnd the threshold value a of hidden layer Calculate the output H of hidden layer are as follows:
Step 3: according to hidden layer and output layer connection weight wjkWith threshold value b, the output of neural network is calculated are as follows:
Step 4: exporting y according to networkkWith desired output dk, calculate error are as follows:
errk=yk-dkK=1,2,3...m
5th step is to calculate error function err to wjkAsk local derviation, yiIt is the input of output layer.Reference formula are as follows:
6th step is to calculate error function err to wijAsk local derviation, hiIt is hidden layer input.Reference formula are as follows:
Error function is the function of an opening upwards, there is minimum, chain type derivation, and error function differentiates to parameter, Exactly find the parameter at error function minimum moment, the data of input can be mapped to number and actual by such parameter Several errors is minimum.
Step 7: network parameter updates.Wherein, η is factor of momentum.Right value update formula are as follows:
Step 8: calculating the global error of network, reference formula are as follows:
Step 9: whether error in judgement meets the requirements, satisfaction then determines network convergence, is unsatisfactory for returning to second step, carry out Next step iteration.
It is defined, defines its loss function are as follows:
Wherein: i is the p per a batch of all anchor framesiIt is the probability that anchor frame belongs to positive class anchor frame, the class if anchor frame is positive Anchor frame, pi *It is 1, otherwise is 0, tiBe one be prediction block parametrization coordinate,It is the parameter coordinate of target frame.Classification Loss LclsIt is the logarithm loss of two classification.Classification and the output for returning layer are { piAnd { ti, pass through Ncls、Nreg, λ be normalized, Ncls、NregFor treating the normalization of classification data, the number specifically summed it up, λ is the penalty coefficient of regression function, is used Weight distribution when loss is calculated with Classification Loss is returned when network training.Only classify into carry out two in RPN network and carries out Feature is extracted, is judged as fish body or background, without dead fish or the category classification of live fish.After obtaining coordinate, need pair Prediction block carries out recurrence fine tuning, makes it closer to true coordinate frame.
tx=(x-xa)/wa,
ty=(y-ya)/ha,
tw=log (w/wa),
th=log (h/ha),
Wherein: x, y, w and h indicate centre coordinate, width and the height of prediction block, xa, ya, waAnd haIt indicates to generate anchor frame Centre coordinate, width and height, x*, y*, w*And h*Indicate centre coordinate, width and the height of target frame, ti=[tx,ty, tw,th] it is one group of vector, indicate the parametrization coordinate of the coordinate frame of prediction,It is that positive class is true time corresponding Return the parametrization coordinate of frame.
Subsequently enter the pond interest region (ROI) layer.Referring to Fig. 3, the input of the pond ROI layer has two parts: first is that RPN net The output of network, second is that characteristic pattern of the original image after convolution.Such purpose is can be with Fusion Features, since RPN exists The mode of a large amount of derivation training, in order to find error function minimum value, information content can be lost after derivation, such as xnPower is asked It is exactly x after leadingn-1, Fusion Features can make up the decaying of information to a certain extent.Each area is directly set in the floor of the pond ROI N × m is arranged in the output size in domain, then obtaining the output of n × m shape to each region.Specifically, each region is existed N and m block is evenly dividing on height and width respectively, fixed point is to nearest integer if dividing boundary and being not integer.Then for Each divides region, exports its greatest member value.
Full articulamentum is subsequently entered, the classification of dead fish and live fish is carried out, and frame coordinate is returned.Wherein classification is logical It crosses softmax function and executes classification feature, softmax function is described as follows:
Wherein, θ, θ 1, θ 2 is the parameter of model, minimizes it error function, i.e. parameter for the value in Controlling model So that the predicted value of Faster rcnn network is approached to true value, hθIt is the output of network concealed layer neuron, hθIt is network pair Each sample corresponds to the probability output of each classification, and referred to herein as calculating after rejecting background to be the pixel of fish It is the probability of live fish and dead fish in block, and the maximum one kind of output probability value.x(i)It is the figure for acquiring i-th of dead fish or live fish Decent, y(i)It is the corresponding classification of i-th of input sample,It is so that probability distribution and for 1.It is every a kind of general Rate can all export, but it is last classification that output probability value is maximum.
For the evolutionary process of depth convolutional neural networks as shown in table 1, table 2, table 1 is the network for having used initial weight training Error condition, table 2 is the network error situation that initial weight is not used, it can be found that using the network of initial weight at the beginning Just has certain generalization ability, this is because network uses the weight ginseng of the pre-training on ImageNet data set first Number, with the increase of frequency of training, network error converges to smaller range.
Table 1 uses transfer learning experiment effect
Note: network global error when ZF_loss is transfer learning, rpn network class when ZF_rpn_cls is transfer learning Error, ZF_rpn_box be transfer learning when rpn net regression error.ZF2_loss is network when transfer learning is not used Global error, ZF2_rpn_cls are the error of rpn network class when transfer learning is not used, and ZF2_rpn_box is to be not used to move The error of rpn net regression when moving study.
Transfer learning experiment effect is not used in table 2
Note: network global error when ZF_loss is transfer learning, the error of network class when ZF_cls is transfer learning, The error of net regression when ZF_box is transfer learning.ZF2_loss is network global error when transfer learning is not used, ZF2_ Cls is the error of network class when transfer learning is not used, and ZF2_box is the error of net regression when transfer learning is not used.
Fig. 4 to Fig. 6 shows the dead fish situation in Faster-rcnn network detection actual production environment.Respectively in complexity It tests under background, in fuzzy pixel, the lesser situation of individual to model, experiment effect is shown, model realizes precisely It identifies and positions, the case where missing inspection does not occur.Fig. 7 is the characteristic response figure of depth convolutional neural networks, is followed successively by image warp It crosses convolutional layer 1, convolutional layer 2, convolutional layer 3, convolutional layer 4, convolutional layer 5, pond layer 1, pond layer 2, RPN and returns layer, RPN classification layer Network later exports result.In Fig. 8, (a) to (e) is the evolutionary process of the network in transfer learning, by using ZFnet and VGG-16 network compares the global error of network and the error of each sub-network as feature extractor, wherein total in (a) Error, has recorded the overall error of all training stages, and (b)-(e) has recorded the classification or regression error in corresponding stage.As a result it shows When showing ZFnet and VGG-16 as feature extractor, Faster-rcnn overall performance of network maintains an equal level.(f) in Fig. 8 is to use Transfer learning is not used in the comparison of global error during the network evolution of initial weight and unused initial weight as the result is shown When, the global error of network is larger, and convergence is slow.Fig. 9 is the software interface of the dead fish identifying system based on deep neural network, Camera will be opened by clicking start button, detect dead fish, be clicked and stopped to stop detecting, clicking selection video will open with dead The video of fish is tested.Early warning system is connect using Internet of Things module with internet simultaneously, and when prediction occurring, system exists simultaneously The mobile end equipment of culturing chamber scene and raiser are simultaneously emitted by two-way alarm.
The present invention provides a kind of dead fish early warning systems based on depth convolutional neural networks, can be continuous with twenty four hours Operation persistently monitors live fish life and death situation when no supervision, when dead fish quantity is more than preset threshold value, system It sounds an alarm, while warning message being transmitted in the mobile device of fisherman by Internet of Things equipment.The system realizes aquatic products industry Non-destructive testing and dead fish real-time monitoring, avoid large-scale dead fish and happen, effectively ensured aquaculture peace Entirely, the economic interests of raiser be ensure that.

Claims (9)

1. a kind of dead fish recognition methods based on depth convolutional neural networks, which is characterized in that the method passes through building The depth convolutional neural networks framework of Faster rcnn framework is identified and is classified to dead fish, comprising the following steps:
S10, acquisition include the image of live fish and dead fish, are labeled to the coordinate position of dead fish and live fish in whole image, Using the image after mark as the input of neural network;
S20, use depth convolutional neural networks as feature extractor, feature extraction is carried out to the image of input;
S30, RPN network is sent into the output of depth convolutional neural networks, determines that individual features are background or fish body, and to it Frame coordinate is returned;
The output of S40, RPN network is sent to the pond ROI layer, then accesses full articulamentum, while being classified and returning operation, sentences It is disconnected to be live fish or dead fish and calculate coordinate position.
2. the dead fish recognition methods according to claim 1 based on depth convolutional neural networks, which is characterized in that the step Depth convolutional neural networks include 5 layers of convolutional layer, 3 pond layers in rapid S20, and two full articulamentums, specifically arrangement is as follows:
First layer is the feature that convolutional layer is used to extract dead fish, and the size of convolution kernel is 7*7, step-length 2*2,96 convolution kernels;
The second layer is pond layer for compressing to the characteristic pattern of input, and the size of pond layer is 3*3, and step-length is 2*2;
Third layer is that convolutional layer further extracts feature, is mapped to high dimensional feature, includes 256 convolution kernels, the convolution kernel of 5*5 is big It is small, step-length 2*2;
4th layer is used for dimensionality reduction for pond layer, and the size of pond layer is 3*3, and step-length is 2*2;
Layer 5 is that convolutional layer further extracts feature, is mapped to high dimensional feature, includes 384 convolution kernels, and convolution kernel size is 3*3;
Layer 6 is that convolutional layer further extracts feature, is mapped to high dimensional feature, includes 384 convolution kernels, and convolution kernel size is 3*3;
Layer 7 is that convolutional layer further extracts feature, is mapped to high dimensional feature, includes 256 convolution kernels, the convolution of 3*3 size Core, step-length are 1*1;
8th layer is pond layer for Feature Dimension Reduction, and the size of pond layer is 3*3, and step-length is 2*2;
9th layer is full articulamentum, first by the characteristic pattern flattening on upper layer, then accesses full articulamentum, is used for Fusion Features;
The tenth layer of full articulamentum in position, for output of classifying.
3. the dead fish recognition methods according to claim 2 based on depth convolutional neural networks, which is characterized in that the depth When spending convolutional neural networks progress feature extraction, the weight using depth network pre-training on ImageNet collection is initial power Weight.
4. the dead fish recognition methods according to claim 1 based on depth convolutional neural networks, which is characterized in that the step RPN network learns prediction by marking with the more similar proposal region of real border frame in rapid S30, comprising the following steps:
After (31) first convolutional networks generate several features by convolutional layer, each pixel on characteristic pattern is generated Several different size of candidate region frames;
(32) feature is consistent according to the position on the relative position and original image on characteristic pattern, then passes through a convolution net Network regards two classification and recurrence as to the candidate region of generation, judges that it is background or fish image;
(33) background image is given up, retains the characteristic pattern containing fish image.
5. the dead fish recognition methods according to claim 1 based on depth convolutional neural networks, which is characterized in that the step Classify in rapid S40 to fish and classified by softmax function:
Wherein, hθIt is the output of network concealed layer neuron, θ, θ 1, θ 2 is the parameter of model, makes it for the value in Controlling model Minimize error function, x(i)It is i-th of dead fish of acquisition or the image pattern of live fish, y(i)It is that i-th of input sample is corresponding Classification,It is so that probability distribution and for 1.
6. a kind of dead fish early warning system based on depth convolutional neural networks, which is characterized in that the system comprises Image Acquisition Device, image processing apparatus and prior-warning device, wherein image collecting device captures fish image and is transmitted to image processing apparatus, Image processing apparatus is using the method for any of claims 1-5, the depth convolution mind based on Faster rcnn framework Carry out the positioning and identification of dead fish through network, dynamic detection dead fish, and be automatically positioned its target position, when detecting dead fish number When amount is more than preset item number, image processing apparatus sends to prior-warning device and instructs, and prompts that alarm occurs, reminds work people Member checks.
7. the dead fish early warning system according to claim 6 based on depth convolutional neural networks, which is characterized in that described Prior-warning device is connected to the mobile end equipment of raiser by mobile network, when detecting that dead fish is greater than specified item number, early warning Device issues alarm sound signal in cultivation site, while transmitting dead fish information to the mobile end equipment of raiser, in mobile terminal Alarm signal occurs.
8. the dead fish early warning system according to claim 6 based on depth convolutional neural networks, which is characterized in that the figure As acquisition device includes camera, the camera is arranged in culturing jar with default spacing isolation.
9. the dead fish early warning system according to claim 8 based on depth convolutional neural networks, which is characterized in that further include Multiple plane mirrors, the multiple plane mirror are separately positioned on the left and right sides of culturing jar and the opposite of camera.
CN201811365779.1A 2018-11-16 2018-11-16 A kind of dead fish recognition methods and early warning system based on depth convolutional neural networks Pending CN109543679A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201811365779.1A CN109543679A (en) 2018-11-16 2018-11-16 A kind of dead fish recognition methods and early warning system based on depth convolutional neural networks

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201811365779.1A CN109543679A (en) 2018-11-16 2018-11-16 A kind of dead fish recognition methods and early warning system based on depth convolutional neural networks

Publications (1)

Publication Number Publication Date
CN109543679A true CN109543679A (en) 2019-03-29

Family

ID=65847663

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201811365779.1A Pending CN109543679A (en) 2018-11-16 2018-11-16 A kind of dead fish recognition methods and early warning system based on depth convolutional neural networks

Country Status (1)

Country Link
CN (1) CN109543679A (en)

Cited By (17)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110298234A (en) * 2019-05-15 2019-10-01 国网湖南省电力有限公司 Substation's charging zone safe early warning method and system based on human body attitude identification
CN110463650A (en) * 2019-09-16 2019-11-19 福建省水产研究所(福建水产病害防治中心) A kind of parent's shrimp ponds automated cleaning control device
CN110876356A (en) * 2019-11-07 2020-03-13 张彩萍 Fishpond oxygenation equipment
CN110910067A (en) * 2019-11-25 2020-03-24 南京师范大学 Intelligent regulation and control method and system for live fish transportation water quality by combining deep learning and Q-learning
CN111738139A (en) * 2020-06-19 2020-10-02 中国水产科学研究院渔业机械仪器研究所 Cultured fish monitoring method and system based on image recognition
CN111738140A (en) * 2020-06-19 2020-10-02 中国水产科学研究院渔业机械仪器研究所 Image recognition-based cultured shrimp real-time monitoring method
CN112149612A (en) * 2020-10-12 2020-12-29 海略(连云港)科技有限公司 Marine organism recognition system and recognition method based on deep neural network
CN112446872A (en) * 2020-12-07 2021-03-05 吉首大学 Giant salamander abnormal behavior identification method based on image identification
CN112580662A (en) * 2020-12-09 2021-03-30 中国水产科学研究院渔业机械仪器研究所 Method and system for recognizing fish body direction based on image features
CN112669336A (en) * 2020-12-09 2021-04-16 中国水产科学研究院渔业机械仪器研究所 Method and system for planning cutting path based on fish body characteristics
CN112966698A (en) * 2021-03-23 2021-06-15 西安电子科技大学 Freshwater fish image real-time identification method based on lightweight convolutional network
CN113420759A (en) * 2021-06-11 2021-09-21 中国船舶重工集团公司七五0试验场 Anti-occlusion and multi-scale dead fish identification system and method based on deep learning
CN113591672A (en) * 2021-07-28 2021-11-02 常州大学 Detection method for identifying fish state based on Mask-Rcnn
CN113598122A (en) * 2021-08-04 2021-11-05 广东海洋大学 Cage dead fish collecting robot control system and method
CN113762229A (en) * 2021-11-10 2021-12-07 山东天亚达新材料科技有限公司 Intelligent identification method and system for building equipment in building site
CN114609952A (en) * 2022-04-07 2022-06-10 广州大学 Fish culture monitoring method and system based on machine vision
CN118038259A (en) * 2024-04-10 2024-05-14 华南农业大学 Method, system, equipment and medium for identifying death of fish cultured in fishpond

Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1598866A (en) * 2004-09-08 2005-03-23 江苏大学 Fish disease automatic diagnosing method processed by image and its device
CN103605330A (en) * 2013-10-23 2014-02-26 浙江海洋学院 Offshore aquaculture remote monitoring system and monitoring method thereof
CN104966122A (en) * 2015-07-11 2015-10-07 浙江大学宁波理工学院 System for automatically counting live fishes based on machine vision
CN205958008U (en) * 2016-06-01 2017-02-15 湖南理工学院 A fish monitoring device that dies for aquaculture
CN106550223A (en) * 2017-01-13 2017-03-29 湖南理工学院 A kind of dead fish supervising device and monitoring method for aquaculture
CN107423745A (en) * 2017-03-27 2017-12-01 浙江工业大学 Fish activity classification method based on neural network
CN108132099A (en) * 2017-12-20 2018-06-08 佛山市洛克威特科技有限公司 A kind of fish monitoring system
CN207940236U (en) * 2018-01-30 2018-10-09 上海理工大学 Shoal of fish automatic health detects and archive management device
CN108805064A (en) * 2018-05-31 2018-11-13 中国农业大学 A kind of fish detection and localization and recognition methods and system based on deep learning

Patent Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1598866A (en) * 2004-09-08 2005-03-23 江苏大学 Fish disease automatic diagnosing method processed by image and its device
CN103605330A (en) * 2013-10-23 2014-02-26 浙江海洋学院 Offshore aquaculture remote monitoring system and monitoring method thereof
CN104966122A (en) * 2015-07-11 2015-10-07 浙江大学宁波理工学院 System for automatically counting live fishes based on machine vision
CN205958008U (en) * 2016-06-01 2017-02-15 湖南理工学院 A fish monitoring device that dies for aquaculture
CN106550223A (en) * 2017-01-13 2017-03-29 湖南理工学院 A kind of dead fish supervising device and monitoring method for aquaculture
CN107423745A (en) * 2017-03-27 2017-12-01 浙江工业大学 Fish activity classification method based on neural network
CN108132099A (en) * 2017-12-20 2018-06-08 佛山市洛克威特科技有限公司 A kind of fish monitoring system
CN207940236U (en) * 2018-01-30 2018-10-09 上海理工大学 Shoal of fish automatic health detects and archive management device
CN108805064A (en) * 2018-05-31 2018-11-13 中国农业大学 A kind of fish detection and localization and recognition methods and system based on deep learning

Non-Patent Citations (5)

* Cited by examiner, † Cited by third party
Title
RANJU MANDAL 等: "Assessing fish abundance from underwater video using deep neural networks", 《2018 INTERNATIONAL JOINT CONFERENCE ON NEURAL NETWORKS (IJCNN)》 *
XIU LI 等: "Fast Accurate Fish Detection and Recognition ofUnderwater Images with Fast R-CNN", 《OCEANS 2015 - MTS/IEEE WASHINGTON》 *
杨红 等: "渔业水域污染死鱼的ANN识别模型的建立", 《上海水产大学学报》 *
林明旺: "深度学习在鱼类图像识别与分类中的应用", 《数学技术与应用》 *
郭耀华: "深度学习——卷积神经网络 的经典网络(LeNet-5、AlexNet、ZFNet、VGG-16、GoogLeNet、ResNet)", 《HTTPS://WWW.CNBLOGS.COM/GUOYAOHUA/P/8534077.HTML》 *

Cited By (20)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110298234A (en) * 2019-05-15 2019-10-01 国网湖南省电力有限公司 Substation's charging zone safe early warning method and system based on human body attitude identification
CN110463650B (en) * 2019-09-16 2021-10-19 福建省水产研究所(福建水产病害防治中心) Parent shrimp pond self-cleaning controlling means
CN110463650A (en) * 2019-09-16 2019-11-19 福建省水产研究所(福建水产病害防治中心) A kind of parent's shrimp ponds automated cleaning control device
CN110876356A (en) * 2019-11-07 2020-03-13 张彩萍 Fishpond oxygenation equipment
CN110910067A (en) * 2019-11-25 2020-03-24 南京师范大学 Intelligent regulation and control method and system for live fish transportation water quality by combining deep learning and Q-learning
CN111738139A (en) * 2020-06-19 2020-10-02 中国水产科学研究院渔业机械仪器研究所 Cultured fish monitoring method and system based on image recognition
CN111738140A (en) * 2020-06-19 2020-10-02 中国水产科学研究院渔业机械仪器研究所 Image recognition-based cultured shrimp real-time monitoring method
CN112149612A (en) * 2020-10-12 2020-12-29 海略(连云港)科技有限公司 Marine organism recognition system and recognition method based on deep neural network
CN112446872A (en) * 2020-12-07 2021-03-05 吉首大学 Giant salamander abnormal behavior identification method based on image identification
CN112580662A (en) * 2020-12-09 2021-03-30 中国水产科学研究院渔业机械仪器研究所 Method and system for recognizing fish body direction based on image features
CN112669336A (en) * 2020-12-09 2021-04-16 中国水产科学研究院渔业机械仪器研究所 Method and system for planning cutting path based on fish body characteristics
CN112966698A (en) * 2021-03-23 2021-06-15 西安电子科技大学 Freshwater fish image real-time identification method based on lightweight convolutional network
CN113420759A (en) * 2021-06-11 2021-09-21 中国船舶重工集团公司七五0试验场 Anti-occlusion and multi-scale dead fish identification system and method based on deep learning
CN113591672A (en) * 2021-07-28 2021-11-02 常州大学 Detection method for identifying fish state based on Mask-Rcnn
CN113591672B (en) * 2021-07-28 2024-05-03 常州大学 Detection method for identifying fish state based on Mask-Rcnn
CN113598122A (en) * 2021-08-04 2021-11-05 广东海洋大学 Cage dead fish collecting robot control system and method
CN113762229A (en) * 2021-11-10 2021-12-07 山东天亚达新材料科技有限公司 Intelligent identification method and system for building equipment in building site
CN114609952A (en) * 2022-04-07 2022-06-10 广州大学 Fish culture monitoring method and system based on machine vision
CN114609952B (en) * 2022-04-07 2024-04-26 广州大学 Fish culture monitoring method and system based on machine vision
CN118038259A (en) * 2024-04-10 2024-05-14 华南农业大学 Method, system, equipment and medium for identifying death of fish cultured in fishpond

Similar Documents

Publication Publication Date Title
CN109543679A (en) A kind of dead fish recognition methods and early warning system based on depth convolutional neural networks
Zhou et al. Evaluation of fish feeding intensity in aquaculture using a convolutional neural network and machine vision
CN113537106B (en) Fish ingestion behavior identification method based on YOLOv5
CN111178197B (en) Mass R-CNN and Soft-NMS fusion based group-fed adherent pig example segmentation method
CN104992223B (en) Intensive population estimation method based on deep learning
CN110414305A (en) Artificial intelligence convolutional neural networks face identification system
CN109635875A (en) A kind of end-to-end network interface detection method based on deep learning
CN107818302A (en) Non-rigid multiple dimensioned object detecting method based on convolutional neural networks
CN109166094A (en) A kind of insulator breakdown positioning identifying method based on deep learning
CN108805070A (en) A kind of deep learning pedestrian detection method based on built-in terminal
CN108509976A (en) The identification device and method of animal
CN107844797A (en) A kind of method of the milking sow posture automatic identification based on depth image
CN107909008A (en) Video target tracking method based on multichannel convolutive neutral net and particle filter
CN109948501A (en) The detection method of personnel and safety cap in a kind of monitor video
CN111666897A (en) Oplegnathus punctatus individual identification method based on convolutional neural network
CN114255357A (en) Group-breeding pig identity identification and health monitoring method based on computer vision
CN109087305A (en) A kind of crack image partition method based on depth convolutional neural networks
CN114359727A (en) Tea disease identification method and system based on lightweight optimization Yolo v4
CN115527234A (en) Infrared image cage dead chicken identification method based on improved YOLOv5 model
CN114898405B (en) Portable broiler chicken anomaly monitoring system based on edge calculation
CN114463843A (en) Multi-feature fusion fish abnormal behavior detection method based on deep learning
CN116824626A (en) Artificial intelligent identification method for abnormal state of animal
CN117333948A (en) End-to-end multi-target broiler behavior identification method integrating space-time attention mechanism
Sun et al. A recognition method of cattle and sheep based on convolutional neural network
CN115908843A (en) Superheat degree recognition model training method, recognition method, equipment and storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
RJ01 Rejection of invention patent application after publication

Application publication date: 20190329

RJ01 Rejection of invention patent application after publication