CN114742806A - Fish body morphological feature measurement method based on key point coordinate regression - Google Patents

Fish body morphological feature measurement method based on key point coordinate regression Download PDF

Info

Publication number
CN114742806A
CN114742806A CN202210427524.3A CN202210427524A CN114742806A CN 114742806 A CN114742806 A CN 114742806A CN 202210427524 A CN202210427524 A CN 202210427524A CN 114742806 A CN114742806 A CN 114742806A
Authority
CN
China
Prior art keywords
fish body
fish
length
key point
network
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202210427524.3A
Other languages
Chinese (zh)
Inventor
胡祝华
赵瑶池
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Hainan University
Original Assignee
Hainan University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Hainan University filed Critical Hainan University
Priority to CN202210427524.3A priority Critical patent/CN114742806A/en
Publication of CN114742806A publication Critical patent/CN114742806A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/155—Segmentation; Edge detection involving morphological operators
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/60—Analysis of geometric attributes
    • G06T7/62—Analysis of geometric attributes of area, perimeter, diameter or volume
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10004—Still image; Photographic image
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02A—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
    • Y02A40/00—Adaptation technologies in agriculture, forestry, livestock or agroalimentary production
    • Y02A40/80—Adaptation technologies in agriculture, forestry, livestock or agroalimentary production in fisheries management
    • Y02A40/81—Aquaculture, e.g. of fish

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • General Health & Medical Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Computational Linguistics (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • Biomedical Technology (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Biophysics (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Geometry (AREA)
  • Quality & Reliability (AREA)
  • Image Analysis (AREA)

Abstract

The invention provides a fish body morphological feature measurement method based on key point coordinate regression, which comprises the following steps: step 1, obtaining a fish body image by using an acquisition device; step 2, inputting the fish body image to be detected in the step 1 into a trained brand new deep learning model consisting of ResNet-50 (residual error network), FPN (feature pyramid network) and KPCRN (key point coordinate regression network); step 3, obtaining 8 key point coordinates output from the model in the step 2
Figure DDA0003608956420000011
And 4, calculating the distance between the coordinates of the key points to obtain various morphological length parameters of the fish body. The method for measuring the fish body morphological parameters almost does not need post-processing, can simplify data set labeling, facilitate the training and testing of models, reduce the time cost and the computational cost of measurement, and has robustness to the change of scenes and the change of fish body postures, thereby greatly improving the accuracy of fish body morphological length measurement.

Description

Fish body morphological feature measurement method based on key point coordinate regression
Technical Field
The invention relates to the field of deep learning and image processing, in particular to a fish body morphological feature measurement method based on key point coordinate regression.
Background
In aquaculture management, various morphological characteristics of fish, such as body length, body width, caudal peduncle length and caudal peduncle width, reflect the culture environment and growth conditions of individual fish, and are the main information basis for fish feeding, medication, fishing, grading and breeding trait analysis. For example, rapid, mass measurement of breeding morphological trait traits is a key issue for elite breeding of aquatic animals; the rapid and accurate acquisition of morphological characteristic data can provide effective guidance for management and control in the culture/fishing process. Therefore, the intelligent measurement of the morphological characteristics of the fish body is beneficial to improving the production efficiency and increasing the income, and has important significance.
The development process of the fish morphology measurement method can be divided into an artificial method, a traditional vision-based method, a machine learning-based vision method and a deep learning-based vision method. The oldest measurement method is manual. An operator holds a vernier caliper and other tools to carry out contact type measurement, and the problems of high labor intensity, low efficiency, strong subjectivity of detection results and the like exist. Moreover, the fish is easy to dehydrate and die due to the long time consumption of measurement.
The automatic measurement of the morphological characteristics of the fish body can be realized to a certain extent by utilizing the computer vision technology, and the intelligent fish culture system becomes a powerful tool for intelligent culture. These vision-based methods work in two steps: 1. firstly, acquiring a fish image, and 2, calculating by adopting computer vision or an image processing algorithm to obtain the morphological characteristics of the fish body.
For the acquisition of the fish body image in the step 1, non-interference acquisition and interference acquisition can be performed according to whether the normal life of the fish is interfered by the shooting process. The non-interference acquisition can be used for shooting the fish on the spot and dynamically in the actual growth environment of the fish or in an induced concentration mode, and the normal growth of the fish is basically not influenced by the mode; the interference type collection often needs to catch and anaesthetize the fish in advance and is placed on a fixed device for shooting, so that the normal growth of the fish can be interfered to a certain extent. However, the resolution of the fish body obtained by interference type acquisition is fixed, the image is clear, the subsequent requirement on a calculation algorithm can be reduced, and the measurement precision is far better than that of non-interference type acquisition. In addition, the existing non-interference acquisition can only measure the characteristics of body length, body width and the like, but cannot measure the characteristics of the caudal peduncle.
For the algorithm in step 2, there are a traditional vision-based method and a vision method combined with machine learning. The former is the aforementioned research work. In recent years, machine learning methods have become the mainstream in computer vision because they can automatically learn the mapping relationship between input and output. Also, measurement techniques combining computer vision and machine learning have become a trend in the current measurement of fish morphological characteristics. However, the conventional machine learning method needs to manually extract the features of the fish in the image, which depend on the subjective design of human, and the method is likely to fail when the illumination of the scene, the color of the fish body, the size, position, and inclination of the fish body, and the like, change. This sensitivity to scenes and targets limits the application of traditional vision and machine learning methods to fish morphology measurements.
At present, deep learning has strong robustness to changes of scenes and targets due to the fact that stable characteristics can be automatically learned from data, and the method becomes an advanced method in the field of current artificial intelligence. However, the existing methods realize the measurement of the morphological characteristics of the fish body based on the pixel-level classification of the image, and have a plurality of disadvantages. Firstly, before training, the methods need to respectively carry out pixel-level labeling on the fish body and the caudal peduncle, and the labeling difficulty is high; secondly, the pixel-level classification task requires higher calculation cost; moreover, the post-processing method therein makes the whole algorithm sensitive to scene and object changes. For example, the irregular placement of fish during the measurement process can challenge the performance of the post-processing algorithm.
Disclosure of Invention
The invention aims to provide a fish body morphological characteristic measuring method based on Key Point Coordinate Regression, which provides a Key Point Coordinate Regression Network (KPCRN) and a KPCRN-based deep learning model. The model is used for predicting the coordinates of key points of the fish body, and finally, the distance between coordinate points is simply calculated to obtain various morphological length parameters of the fish body. The method has stronger robustness on the placing posture of the fish body in measurement, and improves the precision, efficiency and easy operability of the measurement method.
1. A fish body morphological feature measurement method based on key point coordinate regression comprises the following steps:
step 1, obtaining a fish body image by using an acquisition device;
step 2, inputting the fish body image to be detected in the step 1 into a trained brand new deep learning model consisting of ResNet-50 (residual error network), FPN (feature pyramid network) and KPCRN (key point coordinate regression network);
step 3, obtaining 8 key point coordinates output from the model in the step 2
Figure BDA0003608956400000021
Step 4, calculating the distance between the coordinates of the key points to obtain various morphological length parameters of the fish body:
Figure BDA0003608956400000022
Lfw＝d(P4,P5) (7)
Lcpl＝d(P6,P7) (8)
Lcpw＝d(P7,P8) (9)
wherein
Figure BDA0003608956400000031
The abscissa and ordinate of the ith key point are shown, wherein d represents the Euclidean distance between two points. L isfl，Lfw,LcplAnd LcpwThe length, width, length and width of the tail handle of the fish. Further according to the formulas (1) to (4), the length { L) of the four morphological pixels of the fish body can be calculatedfl,Lfw,Lcpl,Lcpw}；
Let the ratio of the actual length of the acquisition device to the pixel length of the acquisition device be k, taking the body length of a fish body as an example, it can be determined according toObtaining the actual length L of the fish body by the formula (8)rfl:
Lrfl＝k×Lfl (10)
2. Further, the method in step 2 further comprises: KPCRN (key point coordinate regression network) is defined as:
Figure BDA0003608956400000032
wherein
Figure BDA0003608956400000033
Is a fully connected operation, characteristic of FPN output
Figure BDA0003608956400000034
First pass through a 3X 3 convolutional layer conv3Is enhanced. Enhanced features through a 16-channel convolutional layer conv3. And predicting the coordinates of the key points on the current scale through pooling and flattening operations and then through a full connection layer. And finally, predicting the key point coordinates of each scale by utilizing full-link layer fusion to obtain 8 final key point coordinates.
3. Further, the method in step 2 further comprises: the average smoothing L1 loss between the 8 predicted coordinates and their corresponding real coordinates in the deep neural network model is defined as:
Figure BDA0003608956400000035
wherein xp isiAnd xgiRespectively, the predicted value and the real value of the ith coordinate, SmoothL1(. cndot.) is defined as:
Figure BDA0003608956400000036
the invention has the beneficial effects that:
(1) the method of the invention regards the measurement problem as the coordinate regression problem of the key points in the image. This allows the method to simplify data set labeling, facilitate training and testing of models, and reduce the time and effort costs of measurements.
(2) The invention provides a key-point-coordinate regression network (KPCRN), a brand-new deep convolutional neural network model for predicting key point coordinates of a fish body is obtained by training the network, 8 key points of the fish body can be predicted in real time by using the model, and morphological characteristic parameters such as the body length, the body width, the caudal peduncle length, the caudal peduncle width and the like of the fish body are obtained in real time. The method almost does not need post-processing, and has robustness on scene change and fish posture change, thereby greatly improving the accuracy of fish morphological length measurement.
Drawings
FIG. 1 is an abstract attached drawing of a fish morphological feature measurement method based on key point coordinate regression according to the invention;
FIG. 2 is a diagram of a fish body image, a body length, a body width, a caudal peduncle length, a caudal peduncle width, and key points in an embodiment of the invention;
FIG. 3 is a schematic diagram of a network structure based on KPCRN in the embodiment of the present invention;
FIG. 4 is a flow chart of fish morphological feature measurement based on key point coordinate regression in the embodiment of the present invention;
FIG. 5 is a picture of a fish body to be detected under a normal background and a complex background in an embodiment of the present invention;
Detailed Description
The concept, specific steps and technical effects of the present invention will be clearly and completely described below in conjunction with the embodiments and fig. 1 to 5 to fully understand the objects, features and effects of the present invention. It is to be understood that the described embodiments are merely exemplary of the present invention, and that functional, methodological, or structural equivalents or substitutions that are described by those of ordinary skill in the art based on the described embodiments are within the scope of the present invention.
This embodiment trains and tests the model on the server and uses the trained network model to test on the development board. The experimental environment on the server is: intel (R) core (TM) i7-7800X, Windows 1064 bit operating system, GPU GTX1080Ti, Python3 and Tensorflow were used. The experimental environment on the development board was: 6-core NVIDIACar-mel ARM v 8.264-bit CPU, Unbuntu 18.04.6LTS of arrach64 architecture, NVIDIAVolta as GPU, Python3 and Tensorflow.
As shown in fig. 1, the method for measuring morphological characteristics of a fish body based on the regression of coordinates of key points provided in this example includes the following steps:
step 1, obtaining a fish body image by using an acquisition device;
FIG. 1 is a diagram showing the body length, body width, tail stalk length and tail stalk width of the fish according to the national standard (the State administration of quality supervision, inspection and quarantine, China Committee for standardization and management, germplasm inspection of cultured fishes, part 3 of character determination, GBT18654.3-2008 Beijing: China Standard Press, 2008:07.) of the people's republic of China, as line segment L in the diagramfl，Lfw,LcplAnd LcpwAs shown. The intelligent measurement of fish body morphological length based on deep learning aims at training a deep artificial neural network model from fish body original images and labeled data, and the morphological pixel length { L of a fish body can be calculated according to the modelfl,Lfw,Lcpl,LcpwAnd further calculating the morphological actual length { L ] of the fish bodyrfl,Lrfw,Lrcpl,Lrcpw}. Wherein, the morphological pixel length { L of the fish body is calculated from the fish body imagefl,Lfw,Lcpl,LcpwThe key to the whole measurement process. Order to
Figure BDA0003608956400000051
Is a sample data of, among others,
Figure BDA0003608956400000052
for the collected fish body image, it can be expressed as
Figure BDA0003608956400000053
m is the number of pixel points in the image,
Figure BDA0003608956400000054
and labeling the image correspondingly.
Step 2, inputting the fish body image to be detected in the step 1 into a deep learning model consisting of ResNet-50 (residual error network), FPN (feature pyramid network) and KPCRN (key point coordinate regression network);
as shown in fig. 3, the deep artificial neural network proposed by the present invention is composed of ResNet-50 (residual error network), FPN (feature pyramid network) and KPCRN (key point coordinate regression network). ResNet extracts image features, integrates network deep features and shallow features to obtain rich and abstract target features, and solves the problem of gradient disappearance through residual connection, so that richer semantic features can be learned by adopting a deeper network. The FPN module is used for improving the capability of the network to process multi-scale targets, and the characteristic pyramid output C of the FPN to ResNet-50 is equal to { C ═ CiI ═ 1,2.3,4} is treated as follows:
Figure BDA0003608956400000055
Figure BDA0003608956400000056
wherein phi1Is a 1 × 1 convolution layer for reducing channel Ci. pool is a pooling operation, upsamplale denotes a bilinear interpolation upsampling operation with an upsampling factor of 2. For the coarsest feature PiBy passing it to a 3 x 3 convolutional layer conv with one channel 5123Multi-scale features with simplified enhancement and then through FPN
Figure BDA0003608956400000057
In order to fuse the multi-scale features and accurately predict the positions of the key points, the following KPCRN is designed.
Figure BDA0003608956400000058
Wherein
Figure BDA0003608956400000059
Is a fully connected operation, characteristic of FPN output
Figure BDA00036089564000000510
First pass through a 3X 3 convolutional layer conv3Is enhanced. Enhanced features through a 16-channel convolutional layer conv3. And predicting the coordinates of the key points on the current scale through pooling and flattening operations and then through a full connection layer. And finally, predicting the coordinates of key points of each scale by utilizing full-link layer fusion to obtain 8 final key point coordinates.
To train the deep neural network model, the average smoothing L1 loss between 8 predicted coordinates and their corresponding real coordinates is defined as:
Figure BDA0003608956400000061
wherein xp isiAnd xgiRespectively, the predicted value and the real value of the ith coordinate, SmoothL1(. cndot.) is defined as:
Figure BDA0003608956400000062
step 3, obtaining 8 key point coordinates output from the model in the step 2
Figure BDA0003608956400000063
And 4, calculating the distance between the coordinates of the key points to obtain various morphological length parameters of the fish body.
Fig. 4 is a calculation process of morphological characteristics of fish body proposed herein. As can be seen from the figure, the labeling is different from the labeling in the conventional method
Figure BDA0003608956400000064
Representing 8 of the fish body imagesSet of keypoint coordinates, wherein
Figure BDA0003608956400000065
The abscissa and ordinate of the ith key point. The 8 key points are shown as the red marked points in FIG. 1, and the length { L } of the fish body morphologyfl,Lfw,Lcpl,LcpwThe correspondence relationship is shown as the following formula. Where d represents the euclidean distance between two points.
Figure BDA0003608956400000066
Lfw＝d(P4,P5) (7)
Lcpl＝d(P6,P7) (8)
Lcpw＝d(P7,P8) (9)
The method of the embodiment of the invention aims to obtain an artificial neural network model through training, and the model can represent a slave image
Figure BDA0003608956400000067
Mapping relation to 8 key point coordinates
Figure BDA0003608956400000068
And by simply calculating the Euclidean distance between the key points
Figure BDA0003608956400000069
And obtaining the morphological length parameter of the fish body.
Further according to the formulas (1) to (4), the length { L) of the four morphological pixels of the fish body can be calculatedfl,Lfw,Lcpl,Lcpw}. Let the ratio of the actual length of the fish plate of the collecting device to the pixel length of the fish plate of the collecting device be k, and take the body length of the fish body as an example, the body length L of the fish body can be obtained according to the formula (8)rfl。
Lrfl＝k×Lfl (10)
The body width of the fish can be calculated by the same methodActual values of length and width of the tail handlerfw,Lrcpl,Lrcpw。
The experimental objects of the embodiment of the invention are trachinotus ovatus and oval required, and all experimental images (in a color JPG format, 4608 × 3456 pixels) are acquired from a Zhenhai aquiculture base (a research and production base of oceanographic academy of Hainan university, Xincun Ling-Water county, China). In order to verify the effectiveness of the method provided by the embodiment of the invention, the fish body collection work provided by the embodiment of the invention is carried out under two different scenes. In the first case, the background of the fish body image obtained by the embodiment of the present invention is a simple green background, as shown in fig. 5 (a). In the second case, the captured oval liquid is placed on a common wood board to be photographed to obtain a fish image. In this case, the background in the obtained image is complicated, and there is a large amount of interference, as shown in fig. 5 (b). For convenience of expression, the data sets acquired in the two scenarios are respectively referred to as a simple data set and a complex data set in the embodiment of the present invention. Each data set had 600 fish images, 400 of which were randomly selected as training sets and the rest as test sets. In addition, considering that the existing method has low performance when the fish body is placed irregularly (which greatly reduces the easy operability of the method in the practical application process), the embodiment of the invention collects 50 randomly placed fish body images in a simple environment to compare the robustness of different methods for the fish body rotation.
To compare the performance of the different methods, the examples of the invention used the Mean Absolute Percent Error (MAPE) as an evaluation index, defined as
Figure BDA0003608956400000071
Where N represents the number of images tested,
Figure BDA0003608956400000072
indicates the prediction length, ymRepresenting the true length, and m represents the index of the image.
The embodiment of the invention trains different deep learning models by adopting two existing methods and the embodiment of the invention on a simple data set and a complex data set respectively. The former (Yu C, Fan X, Hu Z, et al. segmentation and measurement scheme for mesh morphologic targets based on Mask R-CNN [ J ]. Information Processing in the analysis, 2020,7(4):523-534.) adopts the classical image segmentation network Mask R-CNN (He K, Gkioxari G, Doll's R P, et al. Mask R-CNN [ C ]// Proceedings of the IEEE international conference component vision.2017:2961-2969 ]); the latter (Yu C, Hu Z, Han B, et al. Intelligent Measurement of Morphological Characteristics of Fish Using Improved U-Net [ J ]. Electronics,2021,10(12):1426.) network models were trained Using U-Net (Ronneberger O, Fischer P, Brox T.U-Net: computational networks for biological image segmentation [ C ]// International Conference Medical computing. spring, Cham,2015: 234-. All network models were trained iteratively at 40000. Table 1 lists a detailed description of various deep artificial neural network models, including the data sets used for training, the training methods, and the morphological features targeted by the models. It is worth mentioning that the existing morphological feature measurement method regards the measurement problem as the classification problem of all pixel points in the image, and the essential difference between the method of the present invention and the previous method is: the morphological length measurement problem is considered as a regression problem for the keypoint coordinates. In addition, for the two existing methods involved in comparison, the fish body morphological feature and caudal peduncle morphological feature measurement need 2 neural network models, that is, separate training is needed, and all feature measurements in the method of the embodiment of the present invention can be performed in one model. Therefore, the method of the invention is improved in precision, efficiency and easy operability.
TABLE 1 details of the trained deep neural model
Figure BDA0003608956400000081
Result of accuracy comparison
Table 2 shows the comparison of the measurement accuracy of the body length and body width of the fish. Under a simple background, for body length measurement, the method of the embodiment of the invention is optimal, reaches 1.1 percent and is respectively 3.7 percent and 2.93 percent higher than a method based on a Mask R-CNN network and a method based on a U-Net network. For the measurement of body width, the method of the embodiment of the invention is superior to a method based on a Mask R-CNN network, but inferior to the method based on a U-Net network. In a complex environment, a large amount of interferents in the background and low contrast at the fish body boundary (as shown in fig. 6(b), the color of the boundary at the fish tail is very close to that of the fish carrying plate) challenge the performance of the method, and for the method based on the Mask R-CNN network and the method based on the U-Net network, the performance is obviously lower than the measurement performance in a simple background, and especially the method based on the U-Net network has a poor segmentation effect in a complex background. The MAPE value of the method of the embodiment of the invention is far lower than that of the existing method. It can be preliminarily considered that: for methods such as Mask R-CNN network-based methods and U-Net network-based methods, which consider measurement as a pixel segmentation task, each point is at the same important degree, but a large number of pixel points are not critical to the measurement of fish morphological characteristics. This makes the importance of the key points in the whole training very low, so that the classification performance of the pixels in important areas such as the fishtail is low, and the measurement result is finally influenced. This effect is more pronounced when the background and object discrimination of the image is not great. The method in the embodiment of the invention focuses on the coordinate values of the key points in the predicted image, so that the method is more suitable for the task of measuring the morphological length of the fish body.
TABLE 2 MAPE comparison of different measurement methods (%)
Figure BDA0003608956400000091
Table 3 compares the measurement accuracy of the shank length and shank width. It is worth mentioning that for the segmentation task, the caudal peduncle region is a small-area region, which seriously challenges image segmentation such as a Mask R-CNN network-based method and a U-Net network-based methodModel and fish morphology measurements. As can be seen from Table 3, the method based on the U-Net network fails to measure, because the adopted Mask R-CNN model fails to completely divide in the experiment, so that the tail handle length and the tail handle width cannot be measured finally. While MAPE values measured by a Mask R-CNN network-based method are less than satisfactory. In a simple environment, LcplAnd LcpwThe MAPE of the invention was 42.6% and 31.2%, respectively, which are much higher than 9.9% and 3.8% of the process proposed in the examples of the invention. In a complex environment, LcplAnd LcpwThe MAPE was 56.1% and 36.5%, respectively, which was also much higher than 16.1% and 10.1% of the process proposed in the examples of the present invention.
TABLE 3 MAPE comparison of different methods for measuring tail peduncles, (%) ("-" indicates measurement failure)
Figure BDA0003608956400000092
Robust contrast to fish body tilt
During the measurement of morphological characteristics of fish bodies, the fish may not be placed completely horizontally in the shooting scene. This random inclination will affect the measurement of the morphological characteristics of the fish. In order to test the performance of different methods for measurements at different inclinations of the fish body, the embodiment of the present invention uses the inclined fish body image to test the performance of the previously obtained model, as shown in table 4.
TABLE 4 comparison of MAPE Performance (%) "in different methods for measuring tilted Fish bodies
Figure BDA0003608956400000101
As can be seen from Table 3, the MAPE of the method of the embodiment of the invention is lower than that of the method based on the Mask R-CNN network and the method based on the U-Net network. The method based on the Mask R-CNN network and the method based on the U-Net network are divided into two steps of image segmentation and post-processing. The performance of each step influences the final measurement performance, especially the post-processing link, the excessively complex post-processing algorithm in the comparison method, and the adopted traditional image processing method all result in low performance of the fish body under the condition of tilting.
Comparing table 4 with tables 2 and 3, it can be seen that the measurement error of the method based on Mask R-CNN is significantly higher on the tilted fish body data set than on the simple data set, the MAPE of the body length and width measurement increases by 3.2% and 16.0%, while the MAPE of the caudal stem length and width increases greatly, and the measurement fails completely. MAPE (MaPE) increase of the fish body length and width measured by the method based on the U-Net network is large, and particularly measurement error of the body width is large. In contrast, the MAPE of the method of the present invention was minimal and the performance tested on tilted fish was less amplified than the MAPE tested on the simple data set.
And (3) efficiency comparison:
TABLE 5 comparison of the efficiencies of the different methods
Figure BDA0003608956400000102
Table 5 shows the time consumption of the 3 methods. The training time of the 2 nd to 4 th behaviors on the server is 7.72h and 5.78h respectively, wherein the training time of the 2 nd behavior deep neural network model in training 40000epochs is far higher than that of the model in the method provided by the text, and the training time of the Mask R-CNN method based on the Mask R-CNN network and that of the method based on the U-Net network is 7.72h and 5.78h respectively.
Behavior 3 inference time and post-processing time of the model required to measure a morphological feature during the testing phase. The time for model reasoning of the method based on the Mask R-CNN network is 0.238s, and the post-processing requires 0.287 s. The time of model reasoning of the method based on the U-Net network is 0.245s, and the time of post-processing is 0.07 s. The time consumption of the method herein in the test phase was only 0.074s and 0.003 s. Because the network model based on KPCRN provided by the embodiment of the invention regards the measurement problem as the coordinate regression problem of the key points, the morphological length can be obtained only by predicting 8 key point coordinates, and the training time and the reasoning time are greatly saved. The network model provided by the embodiment of the invention is almost end-to-end, and basically does not need a post-processing part, thereby greatly saving the test time of the model.
And 4, testing the time required by the four morphological characteristics of the fish body length, the body width, the tail handle length and the tail handle width by the behavior 4. The method based on the Mask R-CNN network and the method based on the U-Net network are about 12 times and 6 times of the method of the embodiment of the invention. This is because the existing method needs to use different models to calculate the body length/width and caudal peduncle length/width parameters, so two inference times and post-processing time are needed. The model obtained by the training of the method of the embodiment of the invention can predict all key points for 4 feature calculations, so that only one inference time is needed, thereby further greatly saving the testing time of the method.
Test time on development board for acts 5-6. Where behavior 5 measures the time required for one feature and behavior 6 measures the time required for four features. Comparing the third and fourth rows, the time cost of the method of the embodiment of the invention is far less than that of the existing method.
It will be evident to those skilled in the art that the invention is not limited to the details of the foregoing illustrative embodiments, and that the present invention may be embodied in other specific forms without departing from the spirit or essential attributes thereof. The present embodiments are therefore to be considered in all respects as illustrative and not restrictive, the scope of the invention being indicated by the appended claims rather than by the foregoing description, and all changes which come within the meaning and range of equivalency of the claims are therefore intended to be embraced therein. The details of the embodiments are not to be interpreted as limiting the scope of the invention, and any obvious changes, such as equivalent alterations, simple substitutions and the like, based on the technical solution of the invention, can be interpreted without departing from the spirit and scope of the invention.

Claims (3)

1. A fish body morphological feature measurement method based on key point coordinate regression is characterized by comprising the following steps: the method comprises the following steps:
step 1, obtaining a fish body image by using an acquisition device;
step 2, inputting the fish body image to be detected in the step 1 into a trained brand new deep learning model consisting of ResNet-50 (residual error network), FPN (feature pyramid network) and KPCRN (key point coordinate regression network);
step 3, obtaining 8 key point coordinates output from the model in the step 2
Figure FDA0003608956390000011
Step 4, calculating the distance between the coordinates of the key points to obtain various morphological length parameters of the fish body:
Figure FDA0003608956390000012
Lfw＝d(P4,P5) (7)
Lcpl＝d(P6,P7) (8)
Lcpw＝d(P7,P8) (9)
wherein
Figure FDA0003608956390000013
The abscissa and ordinate of the ith key point are shown, wherein d represents the Euclidean distance between two points. L isfl，Lfw,LcplAnd LcpwThe length, width, length and width of the tail handle of the fish. Further according to the formulas (1) - (4), the lengths { L) of the four morphological pixels of the fish body can be calculatedfl,Lfw,Lcpl,Lcpw}；
Making the actual length of the collecting device and the collecting deviceThe actual length L of the fish body can be obtained according to the formula (8) by taking the length of the fish body as an example and taking the pixel length ratio as krfl:
Lrfl＝k×Lfl (10) 。
2. The method of claim 1, wherein step 2 further comprises: KPCRN (keypoint coordinate regression network) is defined as:
Figure FDA0003608956390000014
wherein
Figure FDA0003608956390000015
Is a fully connected operation, characteristic of FPN output
Figure FDA0003608956390000016
First pass through a 3X 3 convolutional layer conv3Is enhanced. Enhanced features through a 16-channel convolutional layer conv3. And predicting the coordinates of the key points on the current scale through pooling and flattening operations and then through a full connection layer. And finally, predicting the coordinates of key points of each scale by utilizing full-link layer fusion to obtain 8 final key point coordinates.
3. The method of claim 1, wherein step 2 further comprises: the average smoothing L1 loss between the 8 predicted coordinates and their corresponding real coordinates in the deep neural network model is defined as:
Figure FDA0003608956390000021
wherein xp isiAnd xgiRespectively, the predicted value and the true value of the ith coordinate, SmoothL1(. cndot.) is defined as:
Figure FDA0003608956390000022
。
CN202210427524.3A 2022-04-21 2022-04-21 Fish body morphological feature measurement method based on key point coordinate regression Pending CN114742806A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210427524.3A CN114742806A (en) 2022-04-21 2022-04-21 Fish body morphological feature measurement method based on key point coordinate regression

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210427524.3A CN114742806A (en) 2022-04-21 2022-04-21 Fish body morphological feature measurement method based on key point coordinate regression

Publications (1)

Publication Number Publication Date
CN114742806A true CN114742806A (en) 2022-07-12

Family

ID=82284428

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210427524.3A Pending CN114742806A (en) 2022-04-21 2022-04-21 Fish body morphological feature measurement method based on key point coordinate regression

Country Status (1)

Country Link
CN (1) CN114742806A (en)

Citations (15)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20090142348A1 (en) * 2006-07-28 2009-06-04 Genentech, Inc. Novel Gene Disruptions, Compositions and Methods Relating Thereto
EP3316220A1 (en) * 2016-10-26 2018-05-02 Balfegó & Balfegó S.L. Method for determining tuna biomass in a water zone and corresponding system
CN109275609A (en) * 2018-11-14 2019-01-29 常州大学 Common fresh-water fishes type automatic identifying method based on image procossing
WO2020023467A1 (en) * 2018-07-24 2020-01-30 Aquabyte, Inc. Unique identification of freely swimming fish in an aquaculture environment
US20200113158A1 (en) * 2017-06-28 2020-04-16 Observe Technologies Limited Data collection system and method for feeding aquatic animals
CN111127396A (en) * 2019-11-21 2020-05-08 中国农业大学 Fish weight measuring and calculating method and device
WO2020134255A1 (en) * 2018-12-27 2020-07-02 南京芊玥机器人科技有限公司 Method for monitoring growth situations of fishes based on machine vision
CN111553709A (en) * 2020-03-25 2020-08-18 田庆兵 Reservoir ecological fish big data traceability system and method
CN111696150A (en) * 2020-05-19 2020-09-22 杭州飞锐科技有限公司 Method for measuring phenotypic data of channel catfish
CN111862048A (en) * 2020-07-22 2020-10-30 浙大城市学院 Automatic fish posture and length analysis method based on key point detection and deep convolution neural network
US20210018742A1 (en) * 2018-04-12 2021-01-21 Google Llc Augmented reality microscope for pathology with overlay of quantitative biomarker data
CN112669348A (en) * 2020-12-18 2021-04-16 浙江大学 Fish body posture estimation and fish body phenotype data measurement method and device
CN112716650A (en) * 2021-01-25 2021-04-30 南开大学 Robotized zebra fish main vein microinjection method
CN112861872A (en) * 2020-12-31 2021-05-28 浙大城市学院 Penaeus vannamei phenotype data determination method, device, computer equipment and storage medium
CN113591671A (en) * 2021-07-28 2021-11-02 常州大学 Fish growth detection method based on Mask-Rcnn recognition

Patent Citations (15)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20090142348A1 (en) * 2006-07-28 2009-06-04 Genentech, Inc. Novel Gene Disruptions, Compositions and Methods Relating Thereto
EP3316220A1 (en) * 2016-10-26 2018-05-02 Balfegó & Balfegó S.L. Method for determining tuna biomass in a water zone and corresponding system
US20200113158A1 (en) * 2017-06-28 2020-04-16 Observe Technologies Limited Data collection system and method for feeding aquatic animals
US20210018742A1 (en) * 2018-04-12 2021-01-21 Google Llc Augmented reality microscope for pathology with overlay of quantitative biomarker data
WO2020023467A1 (en) * 2018-07-24 2020-01-30 Aquabyte, Inc. Unique identification of freely swimming fish in an aquaculture environment
CN109275609A (en) * 2018-11-14 2019-01-29 常州大学 Common fresh-water fishes type automatic identifying method based on image procossing
WO2020134255A1 (en) * 2018-12-27 2020-07-02 南京芊玥机器人科技有限公司 Method for monitoring growth situations of fishes based on machine vision
CN111127396A (en) * 2019-11-21 2020-05-08 中国农业大学 Fish weight measuring and calculating method and device
CN111553709A (en) * 2020-03-25 2020-08-18 田庆兵 Reservoir ecological fish big data traceability system and method
CN111696150A (en) * 2020-05-19 2020-09-22 杭州飞锐科技有限公司 Method for measuring phenotypic data of channel catfish
CN111862048A (en) * 2020-07-22 2020-10-30 浙大城市学院 Automatic fish posture and length analysis method based on key point detection and deep convolution neural network
CN112669348A (en) * 2020-12-18 2021-04-16 浙江大学 Fish body posture estimation and fish body phenotype data measurement method and device
CN112861872A (en) * 2020-12-31 2021-05-28 浙大城市学院 Penaeus vannamei phenotype data determination method, device, computer equipment and storage medium
CN112716650A (en) * 2021-01-25 2021-04-30 南开大学 Robotized zebra fish main vein microinjection method
CN113591671A (en) * 2021-07-28 2021-11-02 常州大学 Fish growth detection method based on Mask-Rcnn recognition

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
GRAHAM G.MONKMAN等: "Using machine vision to estimate fish length from images using regional convolutional neural network", 《METHODS IN ECOLOGY EVOLUTION》, 16 July 2019 (2019-07-16), pages 2045 - 2055 *
陆旺: "基于深度学习的鳙鱼识别算法研究及应用", 《中国优秀硕士学位论文全文数据库农业科技辑》, no. 04, 15 April 2022 (2022-04-15), pages 052 - 96 *

Similar Documents

Publication Publication Date Title
CN107316307B (en) Automatic segmentation method of traditional Chinese medicine tongue image based on deep convolutional neural network
CN109325942B (en) Fundus image structure segmentation method based on full convolution neural network
Aquino et al. vitisBerry: An Android-smartphone application to early evaluate the number of grapevine berries by means of image analysis
Yu et al. Segmentation and measurement scheme for fish morphological features based on Mask R-CNN
CN109410168B (en) Modeling method of convolutional neural network for determining sub-tile classes in an image
CN109472781B (en) Diabetic retinopathy detection system based on serial structure segmentation
Mohamed et al. Msr-yolo: Method to enhance fish detection and tracking in fish farms
WO2023060777A1 (en) Pig body size and weight estimation method based on deep learning
CN109635875A (en) A kind of end-to-end network interface detection method based on deep learning
CN107844797A (en) A kind of method of the milking sow posture automatic identification based on depth image
WO2022167005A1 (en) Deep neural network-based method for detecting living cell morphology, and related product
CN108062749B (en) Identification method and device for levator ani fissure hole and electronic equipment
CN107835654A (en) Image processing apparatus, image processing method and image processing program
CN108182423A (en) A kind of poultry Activity recognition method based on depth convolutional neural networks
CN110307903B (en) Method for dynamically measuring non-contact temperature of specific part of poultry
CN110827304A (en) Traditional Chinese medicine tongue image positioning method and system based on deep convolutional network and level set method
CN102567734A (en) Specific value based retina thin blood vessel segmentation method
CN113269191A (en) Crop leaf disease identification method and device and storage medium
CN111428655A (en) Scalp detection method based on deep learning
CN112102332A (en) Cancer WSI segmentation method based on local classification neural network
Kaya A novel method for optic disc detection in retinal images using the cuckoo search algorithm and structural similarity index
Yu et al. An intelligent measurement scheme for basic characters of fish in smart aquaculture
CN114898405A (en) Portable broiler chicken abnormity monitoring system based on edge calculation
CN114140437A (en) Fundus hard exudate segmentation method based on deep learning
KR102430946B1 (en) System and method for diagnosing small bowel preparation scale

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination