CN118038498B - Fine granularity-based bee and monkey identity recognition method - Google Patents

Fine granularity-based bee and monkey identity recognition method Download PDF

Info

Publication number
CN118038498B
CN118038498B CN202410426112.7A CN202410426112A CN118038498B CN 118038498 B CN118038498 B CN 118038498B CN 202410426112 A CN202410426112 A CN 202410426112A CN 118038498 B CN118038498 B CN 118038498B
Authority
CN
China
Prior art keywords
monkey
bee
face
image
images
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202410426112.7A
Other languages
Chinese (zh)
Other versions
CN118038498A (en
Inventor
李婷萱
张馨弋
倪庆永
丁绍芸
杨梓淇
邱茜
郭敬杰
管艳
雷宇杰
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Sichuan Agricultural University
Original Assignee
Sichuan Agricultural University
Filing date
Publication date
Application filed by Sichuan Agricultural University filed Critical Sichuan Agricultural University
Priority to CN202410426112.7A priority Critical patent/CN118038498B/en
Publication of CN118038498A publication Critical patent/CN118038498A/en
Application granted granted Critical
Publication of CN118038498B publication Critical patent/CN118038498B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Abstract

The invention discloses a fine granularity-based bee and monkey identity recognition method, which comprises the following steps: s1, shooting a plurality of facial images of a plurality of honeybees and monkeys to be identified under different shooting conditions, and cleaning and data enhancement to obtain a honeybee and monkey facial image set; s2, cutting out face areas of all images in the face image set of the bee and monkey to obtain a face cutting area set of the bee and monkey; s3, obtaining eye contours of all images in the face clipping region set of the bee and monkey; s4, carrying out face alignment on images in the image set of the face cutting area of the bee and monkey, and constructing a sample set; s5, an asymmetric bilinear convolutional neural network is constructed, the asymmetric bilinear convolutional neural network is carried out by utilizing samples in the sample set, and the asymmetric bilinear convolutional neural network is obtained based on training to carry out the recognition of the bee and monkey. The invention improves the recognition accuracy of small differences among different individuals of the honeybee and monkey based on fine-granularity image recognition of the face of the honeybee and monkey.

Description

Fine granularity-based bee and monkey identity recognition method
Technical Field
The invention relates to a bee and monkey identification method, in particular to a fine-granularity-based bee and monkey identification method.
Background
In recent years, the reduction of hunting and habitat has led to a dramatic drop in the number of monkeys. Studies on animals need to be based on accurate identification of individuals. The individual bee and monkey is small and has toxicity, and the injection of the chip into the body of the individual bee and monkey or the sleeving of the necklace on the neck of the individual bee and monkey has great and dangerous damage to the body of the bee and monkey. Therefore, the computer vision-based face detection of the bee and monkey is more suitable for the actual individual detection of the bee and monkey.
With the continuous development of deep learning technology, cases of applying deep learning to wild animal protection and animal face recognition are increasing, but the face recognition research on endangered primates with low attention such as honey monkeys is still less.
However, the difference of facial features of the bee and monkey is small, and the bee and monkey belongs to the category of fine-grained recognition (fine-grained recognition refers to recognition tasks that objects with similar appearance and same structure cannot be well recognized by means of integral features, and only fine differences of local areas need to be carefully observed to accurately judge), and common recognition algorithms are difficult to capture small feature differences of the objects, such as hair color, patterns, eye sizes and the like. In the aspect of deep learning, a convolutional neural network is often used for extracting local features, but as the depth of the network increases, only very weak long-distance feature relations can be captured, and multi-level information can not be captured, so that the recognition accuracy of small differences of faces of different individuals of the honeysuckles is low.
Disclosure of Invention
The invention aims to overcome the defects of the prior art, and provides a method for identifying the identity of a bee and monkey based on fine granularity, which is used for improving the accuracy of identifying tiny differences among different individuals of the bee and monkey based on fine granularity image identification of the face of the bee and monkey.
The aim of the invention is realized by the following technical scheme: a bee and monkey identity recognition method based on fine granularity comprises the following steps:
s1, shooting a plurality of facial images of a plurality of honeybees and monkeys to be identified under different shooting conditions, and cleaning and data enhancement to obtain a honeybee and monkey facial image set;
S2, carrying out face labeling on partial images in the face image set of the bee and monkey, training a YOLOv model by using the labeled images, carrying out face recognition on images which are not subjected to face labeling by using a YOLOv model obtained by training, and then cutting out face areas of all images in the face image set of the bee and monkey to obtain a face cutting area set of the bee and monkey;
S3, labeling the edge contours of the eyes of the honeysuckers in partial images in the face cutting area set of the honeysuckers, training a U-Net model by using the labeled images, and detecting images without the edge contours of the eyes in the face cutting area set of the honeysuckers by using the trained U-Net model to obtain the eye contours of all the images in the face cutting area set of the honeysuckers;
s4, carrying out face alignment on images in the image set of the face cutting area of the bee and monkey, and constructing a sample set;
s5, an asymmetric bilinear convolutional neural network is constructed, the asymmetric bilinear convolutional neural network is carried out by utilizing samples in the sample set, and the asymmetric bilinear convolutional neural network is obtained based on training to carry out the recognition of the bee and monkey.
The beneficial effects of the invention are as follows: the invention improves the recognition accuracy of small differences among different individuals of the honeybee and monkey based on fine-granularity image recognition of the face of the honeybee and monkey.
Drawings
FIG. 1 is a flow chart of the method of the present invention.
Detailed Description
The technical solution of the present invention will be described in further detail with reference to the accompanying drawings, but the scope of the present invention is not limited to the following description.
The invention captures multi-level information, and further solves the technical problem of low accuracy in identifying small differences among different individuals of the bee and monkey. The method for recognizing the fine granularity of the face of the bee and monkey by using the computer vision technology to detect the face of the bee and monkey is provided, and a new thought is provided for solving the recognition problem of the bee and monkey, in particular:
As shown in fig. 1, the method for identifying the identity of the bee and monkey based on fine granularity comprises the following steps:
s1, shooting a plurality of facial images of a plurality of honeybees and monkeys to be identified under different shooting conditions, and cleaning and data enhancement to obtain a honeybee and monkey facial image set;
the step S1 includes:
S101, setting a unique number of each of a plurality of honeybees and monkeys to be identified;
A1, shooting facial images of any one of the honeybees and the monkeys needing to be identified under different shooting conditions;
The shooting conditions comprise illumination, shooting angles and shooting distances, and when the face images of the honeybee and monkey are acquired, various illumination intensities, various shooting angles and various shooting distances are required to be preset; the different shooting conditions are as follows: each different combination of illumination intensity, shooting angle and shooting distance, namely, the facial image of the bee and monkey is required to be shot under each combination of illumination intensity, shooting angle and shooting distance, and the shot image is marked by using the number of the bee and monkey;
A2, cleaning the collected facial image of the bee and monkey:
firstly, manually deleting an unclear image;
then calculating the similarity of the images by using an SSIM algorithm, and deleting one of the images if the similarity between the two images is larger than a set threshold value;
s102, repeatedly executing the step S101 for each bee-monkey needing to be identified, and completing facial image acquisition of all the bee-monkeys needing to be identified;
s103, processing each image obtained in the step S102 by using a random data enhancement method to realize data enhancement, wherein the enhanced images form a bee-monkey face image set;
The random data enhancement method adopts one or more modes of random in the modes of turning, adding noise, erasing, changing brightness and gray scale to process the image so as to realize the enhancement of the image data. Ensuring that each image undergoes at least one process. Therefore, the image data set is enriched, complex and various environments are simulated as much as possible, and effective identification of the bees and the monkeys is realized.
In an embodiment of the present application, a monkey face image dataset is produced by acquiring images of a monkey face. Different conditions of illumination, angles, distances and the like are used for shooting facial images of the bee and monkey so as to increase the diversity and the robustness of the data. We collected facial images of approximately 6000 69 different bee and monkey.
S2, carrying out face labeling on partial images in the face image set of the bee and monkey, training a YOLOv model by using the labeled images, carrying out face recognition on images which are not subjected to face labeling by using a YOLOv model obtained by training, and then cutting out face areas of all images in the face image set of the bee and monkey to obtain a face cutting area set of the bee and monkey;
the face labeling is carried out on part of the images, and then the face areas of the rest images are automatically identified based on YOLOv models, so that the labeling workload can be effectively reduced;
The step S2 includes:
S201, selecting 10% of images from the facial image set of the bee and monkey, labeling facial area bounding boxes of the bee and monkey by adopting label software Labelimg and a rectangular label library, and completing data set construction in a yolo data set format;
s202, constructing a YOLOv target detection model based on YOLOv algorithm, and training the YOLOv target detection model by utilizing a dataset;
S203, carrying out face region prediction on unlabeled images in the facial image set of the bee and monkey by using the trained YOLOv target detection model to obtain a face region boundary box of the unlabeled images;
s204, after the face area detection is completed, cutting each image of the face image set of the bee and monkey, and only preserving the face area part inside the face area boundary box to obtain the face cutting area set of the bee and monkey.
S3, labeling the edge contours of the eyes of the honeysuckers in partial images in the face cutting area set of the honeysuckers, training a U-Net model by using the labeled images, and detecting images without the edge contours of the eyes in the face cutting area set of the honeysuckers by using the trained U-Net model to obtain the eye contours of all the images in the face cutting area set of the honeysuckers;
the method has the advantages that the edge contours of the eyes of the honeybee and monkey in part of the images are marked, and then the edge contours of the eyes of the honeybee and monkey in the rest images are automatically identified based on the U-Net model, so that the marking workload can be effectively reduced;
the step S3 includes:
S301, selecting 10% of images from a face clipping region set of the bee and monkey, marking the outline of the edge of the eye of the bee and monkey by using Labelme image marking tools, obtaining a JSON file after marking, and completing construction of a data set in a VOC data set format;
s302, constructing a U-Net model, and training the model by utilizing a data set constructed in a VOC data set format;
S303, predicting the images without the eye edge contours marked in the face cutting area set of the bee and monkey by using the trained U-Net model to obtain the eye contours of all the images in the face cutting area set of the bee and monkey.
S4, carrying out face alignment on images in the image set of the face cutting area of the bee and monkey, and constructing a sample set;
The step S4 includes:
S401, carrying out face correction on any picture in the image collection of the face clipping region of the bee and monkey:
B1, calculating the average value of pixels in the eye according to the edge contour of the eye, wherein the average value is used as the center point of the eye, and the calculation formula of the geometric center is as follows:
Wherein (x c,yc) is the center point of the eye, N is the number of pixels inside the eye, and (x i,yi) is the coordinates of the ith pixel;
B2, rotating the image: calculating rotation angles according to the center points of the two eyes by using an affine transformation method, and then constructing a rotation matrix to rotate the image to a horizontal position;
the calculation formula of affine transformation is as follows:
calculating the rotation angle:
Constructing a rotation matrix:
For each pixel point (x, y) in the image, a rotation matrix is applied, resulting in rotated points (x ', y'):
Wherein (x 1, y 1) and (x 2, y 2) are the center points of the two eyes, θ is the rotation angle, and R is the rotation matrix;
S402, repeatedly executing the step S401 for each image in the image set of the face clipping region of the bee and monkey, and completing face alignment of all the images;
s403, after face alignment is carried out on each image in the image collection of the face cutting area of the bee and monkey, the aligned image is taken as a characteristic, and the bee and monkey number corresponding to the image is taken as a classification label, so that a sample is constructed, and a sample set is formed.
S5, an asymmetric bilinear convolutional neural network is constructed, the asymmetric bilinear convolutional neural network is carried out by utilizing samples in the sample set, and the asymmetric bilinear convolutional neural network is obtained based on training to carry out the recognition of the bee and monkey.
Fine-grained image classification (Fine-Grained Categorization), also called Sub-category image classification (Sub-Category Recognition), is a very popular research topic in the fields of computer vision, pattern recognition, etc. in recent years. The aim is to carry out finer subclassification on large classes with coarse granularity, but the classification difficulty of fine granularity images is higher than that of common image classification tasks due to fine class-to-class differences and larger class-to-class differences among the subclasses.
The task of classifying fine-grained images requires the resolution of different individual images under the species "monkey". Different individuals of the same class of species often differ only in subtle areas such as hair color, pattern, and eye size. Not only is the difficulty and challenges of fine-grained image tasks certainly greater for computers and for the average population. The asymmetric bilinear CNN model is a fine-grained classification model;
The step S5 includes:
s501, constructing an asymmetric bilinear convolutional neural network, wherein the asymmetric bilinear convolutional neural network comprises a VGG16 network, a ResNet network, a full-connection layer and a softmax layer;
in an asymmetric bilinear convolutional neural network, an input image is respectively sent to a VGG16 network and a ResNet network for feature extraction, then the results after feature extraction are subjected to outer product fusion to form final individual identity features, and then the final individual identity features are sequentially sent to a full-connection layer and a softmax layer for feature classification output;
S502, training the asymmetric bilinear convolutional neural network by taking an image in a sample set as input of the asymmetric bilinear convolutional neural network and taking a classification label of the image as expected output of the asymmetric bilinear convolutional neural network to obtain a trained asymmetric bilinear convolutional neural network for recognizing the bee and monkey.
In the embodiment of the application, the sample set can be divided into a training set and a testing set, the training of the asymmetric bilinear convolutional neural network is finished by using the samples in the training set, and the accuracy of the asymmetric bilinear convolutional neural network is tested by using the data in the testing set.
In the embodiment of the application, individual identification is the basis of animal behavioural and ecological studies, and has important significance for protecting endangered species. It is important to correctly identify the identity of an individual, for which we use a method of strongly supervising fine-grained classification in order to obtain better classification accuracy. This process of identifying different bees and monkeys requires a more accurate and careful process to ensure the accuracy of the identification.
When the trained asymmetric bilinear convolutional neural network is used for recognizing the honey and monkey, firstly, a trained YOLOv target detection model is used for detecting the face image of the honey and monkey to be recognized to obtain a face region of the honey and monkey to be recognized, clipping is carried out to obtain a face clipping region to be recognized, then, a trained U-Net model is used for detecting the face clipping region to be recognized to obtain the outline of the edge of the eyes of the face clipping region to be recognized, after face alignment is carried out according to the step S401, the outline of the eye edge is input into the trained asymmetric bilinear convolutional neural network, and a honey and monkey recognition result is output by the trained asymmetric bilinear convolutional neural network.
The foregoing is a preferred embodiment of the invention, and it is to be understood that the invention is not limited to the form disclosed herein, but is not to be construed as limited to other embodiments, but is capable of other combinations, modifications and environments and is capable of changes or modifications within the scope of the inventive concept, either as a result of the foregoing teachings or as a result of the knowledge or knowledge of the relevant art. And that modifications and variations which do not depart from the spirit and scope of the invention are intended to be within the scope of the appended claims.

Claims (4)

1. A fine granularity-based bee and monkey identity recognition method is characterized by comprising the following steps of: the method comprises the following steps:
s1, shooting a plurality of facial images of a plurality of honeybees and monkeys to be identified under different shooting conditions, and cleaning and data enhancement to obtain a honeybee and monkey facial image set;
S2, carrying out face labeling on partial images in the face image set of the bee and monkey, training a YOLOv model by using the labeled images, carrying out face recognition on images which are not subjected to face labeling by using a YOLOv model obtained by training, and then cutting out face areas of all images in the face image set of the bee and monkey to obtain a face cutting area set of the bee and monkey;
S3, labeling the edge contours of the eyes of the honeysuckers in partial images in the face cutting area set of the honeysuckers, training a U-Net model by using the labeled images, and detecting images without the edge contours of the eyes in the face cutting area set of the honeysuckers by using the trained U-Net model to obtain the eye contours of all the images in the face cutting area set of the honeysuckers;
s4, carrying out face alignment on images in the image set of the face cutting area of the bee and monkey, and constructing a sample set;
The step S4 includes:
S401, carrying out face correction on any picture in the image collection of the face clipping region of the bee and monkey:
B1, calculating the average value of pixels in the eye according to the edge contour of the eye, wherein the average value is used as the center point of the eye, and the calculation formula of the geometric center is as follows:
Wherein (x c,yc)) is the center point of the eye, N is the number of pixels inside the eye, and (x i,yi) is the coordinates of the ith pixel;
B2, rotating the image: calculating rotation angles according to the center points of the two eyes by using an affine transformation method, and then constructing a rotation matrix to rotate the image to a horizontal position;
the calculation formula of affine transformation is as follows:
calculating the rotation angle:
Constructing a rotation matrix:
For each pixel point (x, y) in the image, a rotation matrix is applied, resulting in rotated points (x ', y'):
Wherein (x 1, y 1) and (x 2, y 2) are the center points of the two eyes, θ is the rotation angle, and R is the rotation matrix;
S402, repeatedly executing the step S401 for each image in the image set of the face clipping region of the bee and monkey, and completing face alignment of all the images;
s403, after face alignment is carried out on each image in the image set of the face cutting area of the bee and monkey, the aligned image is taken as a characteristic, and a bee and monkey number corresponding to the image is taken as a classification label, so that a sample is constructed, and a sample set is formed;
s5, an asymmetric bilinear convolutional neural network is constructed, the asymmetric bilinear convolutional neural network is carried out by utilizing samples in a sample set, and the asymmetric bilinear convolutional neural network is obtained based on training to carry out the recognition of the bee and monkey;
The step S5 includes:
s501, constructing an asymmetric bilinear convolutional neural network, wherein the asymmetric bilinear convolutional neural network comprises a VGG16 network, a ResNet network, a full-connection layer and a softmax layer;
in an asymmetric bilinear convolutional neural network, an input image is respectively sent to a VGG16 network and a ResNet network for feature extraction, then the results after feature extraction are subjected to outer product fusion to form final individual identity features, and then the final individual identity features are sequentially sent to a full-connection layer and a softmax layer for feature classification output;
S502, training the asymmetric bilinear convolutional neural network by taking an image in a sample set as input of the asymmetric bilinear convolutional neural network and taking a classification label of the image as expected output of the asymmetric bilinear convolutional neural network to obtain a trained asymmetric bilinear convolutional neural network for recognizing the bee and monkey.
2. The fine-grained monkey-based identification method as claimed in claim 1, wherein the method comprises the steps of: the step S1 includes:
S101, setting a unique number of each of a plurality of honeybees and monkeys to be identified;
A1, shooting facial images of any one of the honeybees and the monkeys needing to be identified under different shooting conditions;
The shooting conditions comprise illumination, shooting angles and shooting distances, and when the face images of the honeybee and monkey are acquired, various illumination intensities, various shooting angles and various shooting distances are required to be preset; the different shooting conditions are as follows: each different combination of illumination intensity, shooting angle and shooting distance, namely, the facial image of the bee and monkey is required to be shot under each combination of illumination intensity, shooting angle and shooting distance, and the shot image is marked by using the number of the bee and monkey;
A2, cleaning the collected facial image of the bee and monkey:
firstly, manually deleting an unclear image;
then calculating the similarity of the images by using an SSIM algorithm, and deleting one of the images if the similarity between the two images is larger than a set threshold value;
s102, repeatedly executing the step S101 for each bee-monkey needing to be identified, and completing facial image acquisition of all the bee-monkeys needing to be identified;
s103, processing each image obtained in the step S102 by using a random data enhancement method to realize data enhancement, wherein the enhanced images form a bee-monkey face image set;
The random data enhancement method adopts one or more modes of random in the modes of turning, adding noise, erasing, changing brightness and gray scale to process the image so as to realize the enhancement of the image data.
3. The fine-grained monkey-based identification method as claimed in claim 1, wherein the method comprises the steps of: the step S2 includes:
S201, selecting 10% of images from the facial image set of the bee and monkey, labeling facial area bounding boxes of the bee and monkey by adopting label software Labelimg and a rectangular label library, and completing data set construction in a yolo data set format;
s202, constructing a YOLOv target detection model based on YOLOv algorithm, and training the YOLOv target detection model by utilizing a dataset;
S203, carrying out face region prediction on unlabeled images in the facial image set of the bee and monkey by using the trained YOLOv target detection model to obtain a face region boundary box of the unlabeled images;
s204, after the face area detection is completed, cutting each image of the face image set of the bee and monkey, and only preserving the face area part inside the face area boundary box to obtain the face cutting area set of the bee and monkey.
4. The fine-grained monkey-based identification method as claimed in claim 1, wherein the method comprises the steps of: the step S3 includes:
S301, selecting 10% of images from a face clipping region set of the bee and monkey, marking the outline of the edge of the eye of the bee and monkey by using Labelme image marking tools, obtaining a JSON file after marking, and completing construction of a data set in a VOC data set format;
s302, constructing a U-Net model, and training the model by utilizing a data set constructed in a VOC data set format;
S303, predicting the images without the eye edge contours marked in the face cutting area set of the bee and monkey by using the trained U-Net model to obtain the eye contours of all the images in the face cutting area set of the bee and monkey.
CN202410426112.7A 2024-04-10 Fine granularity-based bee and monkey identity recognition method Active CN118038498B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202410426112.7A CN118038498B (en) 2024-04-10 Fine granularity-based bee and monkey identity recognition method

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202410426112.7A CN118038498B (en) 2024-04-10 Fine granularity-based bee and monkey identity recognition method

Publications (2)

Publication Number Publication Date
CN118038498A CN118038498A (en) 2024-05-14
CN118038498B true CN118038498B (en) 2024-06-28

Family

ID=

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109086792A (en) * 2018-06-26 2018-12-25 上海理工大学 Based on the fine granularity image classification method for detecting and identifying the network architecture
CN116343284A (en) * 2022-12-19 2023-06-27 四川农业大学 Attention mechanism-based multi-feature outdoor environment emotion recognition method

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109086792A (en) * 2018-06-26 2018-12-25 上海理工大学 Based on the fine granularity image classification method for detecting and identifying the network architecture
CN116343284A (en) * 2022-12-19 2023-06-27 四川农业大学 Attention mechanism-based multi-feature outdoor environment emotion recognition method

Similar Documents

Publication Publication Date Title
Sun et al. Transferring deep knowledge for object recognition in low-quality underwater videos
CN110334706B (en) Image target identification method and device
CN108960245B (en) Tire mold character detection and recognition method, device, equipment and storage medium
CN108520226B (en) Pedestrian re-identification method based on body decomposition and significance detection
Mohamed et al. Msr-yolo: Method to enhance fish detection and tracking in fish farms
Lin Face detection in complicated backgrounds and different illumination conditions by using YCbCr color space and neural network
Mathur et al. Crosspooled FishNet: transfer learning based fish species classification model
CN1975759A (en) Human face identifying method based on structural principal element analysis
Ye et al. A two-stage real-time YOLOv2-based road marking detector with lightweight spatial transformation-invariant classification
CN111274883B (en) Synthetic sketch face recognition method based on multi-scale HOG features and deep features
CN108764159A (en) Animal face recognition methods under condition of small sample and system
WO2013075295A1 (en) Clothing identification method and system for low-resolution video
CN112464850A (en) Image processing method, image processing apparatus, computer device, and medium
CN110188646B (en) Human ear identification method based on fusion of gradient direction histogram and local binary pattern
Naseer et al. Multimodal Objects Categorization by Fusing GMM and Multi-layer Perceptron
CN112825120B (en) Face illumination evaluation method, device, computer readable storage medium and equipment
CN112101293A (en) Facial expression recognition method, device, equipment and storage medium
CN118038498B (en) Fine granularity-based bee and monkey identity recognition method
Curran et al. The use of neural networks in real-time face detection
KR100703528B1 (en) Apparatus and method for recognizing an image
CN118038498A (en) Fine granularity-based bee and monkey identity recognition method
CN113658223B (en) Multi-row person detection and tracking method and system based on deep learning
Yi et al. Face detection method based on skin color segmentation and facial component localization
Jain et al. Effective approach for facial expression recognition using hybrid square-based diagonal pattern geometric model
CN109614877B (en) Method for identifying attribute of pedestrian with shielding in low-resolution monitoring scene

Legal Events

Date Code Title Description
PB01 Publication
SE01 Entry into force of request for substantive examination
GR01 Patent grant