CN115223198A - Pig behavior identification method, pig behavior identification system, computer equipment and storage medium - Google Patents
Pig behavior identification method, pig behavior identification system, computer equipment and storage medium Download PDFInfo
- Publication number
- CN115223198A CN115223198A CN202210785276.XA CN202210785276A CN115223198A CN 115223198 A CN115223198 A CN 115223198A CN 202210785276 A CN202210785276 A CN 202210785276A CN 115223198 A CN115223198 A CN 115223198A
- Authority
- CN
- China
- Prior art keywords
- pig
- convolution
- network
- space
- data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/084—Backpropagation, e.g. using gradient descent
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/46—Descriptors for shape, contour or point-related descriptors, e.g. scale invariant feature transform [SIFT] or bags of words [BoW]; Salient regional features
- G06V10/462—Salient features, e.g. scale invariant feature transforms [SIFT]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/20—Movements or behaviour, e.g. gesture recognition
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Evolutionary Computation (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Software Systems (AREA)
- Computing Systems (AREA)
- Artificial Intelligence (AREA)
- Medical Informatics (AREA)
- Databases & Information Systems (AREA)
- Human Computer Interaction (AREA)
- Biophysics (AREA)
- Biomedical Technology (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Molecular Biology (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Psychiatry (AREA)
- Social Psychology (AREA)
- Life Sciences & Earth Sciences (AREA)
- Image Analysis (AREA)
Abstract
The invention relates to the technical field of behavior recognition, and discloses a pig behavior recognition method, a pig behavior recognition system, a computer device and a storage medium, wherein the pig behavior recognition method comprises the following steps: constructing a pig posture graph according to preset pig posture key points; carrying out attitude estimation on the video information of the pig, and extracting key point sequence data according to the attitude graph of the pig; and inputting the key point sequence data into a trained improved space-time diagram convolution network to identify the pig behavior to obtain a pig behavior classification prediction result, wherein the improved space-time diagram convolution network comprises a first space-time diagram convolution network and a second space-time diagram convolution network. The invention improves the perception capability of the network to the whole posture, relieves the gradient decline problem in the network identification process, improves the utilization rate of the characteristic information of the shallow network, and improves the accuracy of the pig behavior identification.
Description
Technical Field
The invention relates to the technical field of behavior recognition, in particular to a pig behavior recognition method, a pig behavior recognition system, computer equipment and a storage medium.
Background
In recent years, with the development of computational science and technology, the large-scale automatic monitoring pig breeding is the development trend in the future. However, at present, china is still in the traditional manual monitoring stage, and the monitoring mode has the problems of strong subjective factors, large errors, incapability of monitoring all weather and the like; the problems can be well avoided by adopting a computer vision technology. At present, pig behavior recognition methods can be divided into contact sensor and non-contact sensor methods. But behavior recognition based on the contact sensor can influence the living comfort of the pigs to a certain extent, and in long-term use, the problems of sensitivity change and damage of the sensor, increase of breeding cost and the like are caused.
The non-contact sensor based on the computer vision technology identifies the pig behaviors mostly by adopting images or videos as data, and has the defects of large data processing capacity, high requirements on instruments and equipment and the like. Although Yan et al propose an ST-GCN space-time diagram convolution network based on human body posture recognition in 2018 on the basis of a GCN network, the method is mostly applied to detection of pedestrian behaviors, little research is carried out on behavior recognition of pigs, and the existing ST-GCN network cannot be well applied to recognition of the behaviors of the pigs on the basis of difference of human bodies and the pigs only in posture and behavior activities.
Disclosure of Invention
In order to solve the technical problems, the invention provides a pig behavior identification method, a pig behavior identification system, a computer device and a storage medium, wherein a partition strategy with wider adjacent information and stronger association of adjacent nodes is obtained, and the structure of a spatio-temporal graph convolutional network is improved, so that the gradient descent problem is relieved, the utilization of characteristic information by the whole network is improved, and the aim of improving the pig behavior identification accuracy is fulfilled.
In a first aspect, a pig behavior identification method based on an improved space-time graph convolutional network is provided in a first embodiment of the present invention, where the method includes:
constructing a pig posture graph according to preset pig posture key points;
carrying out attitude estimation on the video information of the pig, and extracting key point sequence data according to the attitude graph of the pig;
and inputting the key point sequence data into a trained improved space-time diagram convolution network to identify the pig behaviors to obtain a pig behavior classification prediction result, wherein the improved space-time diagram convolution network comprises a first space-time diagram convolution network and a second space-time diagram convolution network.
Further, the specific steps of performing pose estimation on the pig video information and extracting the key point sequence data according to the pig pose graph include:
carrying out pig target detection on pig video information through a YOLO model to obtain a pig target area;
carrying out pig position extraction on the pig target area through an IOU-Tracker model to obtain a single pig position area;
converting the single pig position area into a single pig position picture, and preprocessing the single pig position picture;
and carrying out attitude estimation on the preprocessed single-pig position picture through an OpenPose model, extracting a pig attitude sequence according to the pig attitude picture, and generating key point sequence data.
Further, the improved space-time graph convolutional network is partitioned according to a space structure, the perception distance between key points is 3, and the neighborhood of the key points is divided into 7 subsets according to the perception distance.
Further, the subset of neighborhoods is calculated using the following formula:
in the formula, v ti Is a key point, r j Is the distance of node j from the center of gravity, r i Distance from root node to center of gravity, D is the perceived distance between key points, l ti (v ti ) Is a key point v ti A neighborhood subset of (a).
Further, the first spatio-temporal graph convolution network comprises three layers of first spatio-temporal graph convolution blocks, and the number of channels of the three layers of first spatio-temporal graph convolution blocks is 64, 128 and 256 in sequence; the second space-time graph convolutional network comprises nine layers of second space-time graph volume blocks, and the number of channels of the nine layers of second space-time graph volume blocks is 64, 128, 256 and 256 in sequence.
Further, the specific step of inputting the key point sequence data into a trained improved space-time diagram convolutional network to identify the pig behavior to obtain the pig behavior classification prediction result includes:
inputting the key point sequence data into a first time-space diagram convolution network to perform three-layer convolution operation, and respectively generating first convolution data, second convolution data and third convolution data after each layer of convolution operation;
taking the first convolution data as the input of a second space-time diagram convolution network, and generating fourth convolution data after three-layer convolution operation;
inputting the first convolution data and the fourth convolution data into a fourth layer of the second space-time graph convolution network, and performing three-layer convolution operation to generate fifth convolution data;
inputting the second convolution data and the fifth convolution data into a seventh layer of the second space-time diagram convolution network, and generating sixth convolution data after three-layer convolution operation;
and splicing the third convolution data and the sixth convolution data, and inputting the spliced data into a classifier to perform classification and prediction of the behaviors of the pigs.
Further, the preset pig posture key points comprise 20 pig key points including a nose, a right eye, a right ear, a left eye, a left ear, a right shoulder, a right front knee, a right front hoof, a right hip, a right back knee, a right back hoof, a left shoulder, a left front knee, a left front hoof, a left hip, a left back knee, a left back hoof, a back, an abdomen and a caudal vertebra, and 28 connected pig key point connecting pairs.
In a second aspect, a second embodiment of the present invention provides a pig behavior recognition system based on an improved space-time graph convolutional network, the system including:
the attitude map construction module is used for constructing a pig attitude map according to preset pig attitude key points;
the key point extraction module is used for carrying out attitude estimation on the pig video information and extracting key point sequence data according to the pig attitude graph;
and the behavior identification prediction module is used for inputting the key point sequence data into a trained improved space-time graph convolution network to identify the behavior of the pig so as to obtain a pig behavior classification prediction result, wherein the improved space-time graph convolution network comprises a first space-time graph convolution network and a second space-time graph convolution network.
In a third aspect, an embodiment of the present invention further provides a computer device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, where the processor implements the steps of the method when executing the computer program.
In a fourth aspect, the present invention further provides a computer-readable storage medium, on which a computer program is stored, where the computer program is executed by a processor to implement the steps of the above method.
According to the pig behavior identification method, the pig behavior identification system, the pig behavior identification computer equipment and the pig behavior identification storage medium, a partition strategy which is wider in adjacent node information range, more compact in motion correlation and capable of improving network perception capability is provided, a new network structure is constructed by fusing a retracted space-time graph convolution network and a space-time graph convolution network, the gradient descent problem of the original space-time graph convolution network is relieved, the utilization rate of the whole network structure to characteristic information is improved, and therefore the pig behavior identification accuracy is improved.
Drawings
FIG. 1 is a schematic flow chart of a pig behavior identification method based on an improved spatiotemporal graph convolutional network according to an embodiment of the present invention;
FIG. 2 is a schematic diagram of a network structure of an ST-GCN network according to an embodiment of the present invention;
FIG. 3 is a pig posture diagram of a pig behavior recognition method based on an improved spatiotemporal graph convolutional network according to an embodiment of the present invention;
FIG. 4 is a schematic flow chart of step S20 in FIG. 1;
fig. 5 is a schematic view of video data processing in step S20 in fig. 1;
FIG. 6 is a schematic diagram of the network structure of the improved space-time graph convolutional network of step S30 in FIG. 1;
FIG. 7 is a schematic flow chart of step S30 in FIG. 1;
fig. 8 is a schematic structural diagram of a pig behavior recognition system based on an improved time-space diagram convolutional network according to an embodiment of the present invention;
fig. 9 is an internal structural diagram of a computer device in the embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, but not all, embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Referring to fig. 1, a first embodiment of the present invention provides a pig behavior identification method based on an improved space-time graph convolutional network, where the method includes:
and S10, constructing a pig posture graph according to preset pig posture key points.
The embodiment of the invention is based on the classification prediction of an improved ST-GCN network, wherein the ST-GCN is a classification network based on a graph convolution neural network, the input of the ST-GCN is time sequence key point data, and the time sequence data comprises the space coordinates and continuous time frame sequence information of the key points, and the ST-GCN network model is simply explained by combining the ST-GCN network structure of figure 2.
The network structure of the ST-GCN mainly comprises an input layer, a convolutional layer and a classifier, wherein input data in the input layer is a key point sequence x (N, C, T and V), in the formula, N is the number of Batch-Size video samples, C is the feature number of key points and comprises the coordinate position and confidence information of bone key points, T is the number of input video frames, V is the number of key points, the convolutional layer comprises nine layers of spatio-temporal map convolutional blocks, the number of channels in the first three layers is 64, the number of channels in the middle three layers is 128, the number of channels in the last three layers is 256, and after the key point sequence passes through the nine layers of spatio-temporal map convolutional blocks, a standard SoftMax function is adopted as the classifier to conduct behavior classification prediction.
The convolution algorithm of the ST-GCN network comprises space convolution and multi-scale time convolution, construction is carried out on the basis of two-dimensional convolution, discrete feature points in a space are processed by adopting graph convolution, in the two-dimensional convolution, a convolution kernel with the size of K multiplied by K is set, the number of input channels is C, and then single-channel output is carried out on a certain node x:
wherein p is a down-sampling function, the sampling region is the neighborhood (h, w) of x, the weight function w provides a weight vector of inner product calculation, and the weight is shared and is not related to the input position x.
Based on the above description, it can be obtained that the ST-GCN network uses the key point sequence as input data, the existing ST-GCN network is mainly applied to the recognition of human body behaviors, the detection of pig behaviors is rarely studied, but the ST-GCN network is applied to the recognition of pig behaviors in the invention, but the key points aiming at the pig postures need to be reset due to the difference between the human body postures and the pig postures.
The embodiment of the invention sets the posture of the pig based on the characteristics of the bone, the appearance and the like of the pig, and the pig has different characteristics of body state hypertrophy, short and small limbs, big and big mouth and tail. Therefore, in order to reflect the change of the body parts and postures of the pigs in different motion states, the invention designs the postures of the pigs with 20 key points based on the skeleton structure, the posture characteristics and the like of the pigs, and the specific key parts are as follows: the joints of the bones of the pig, such as the scapula of the forepaw, the hip of the hindpaw, the joints of the legs and the hooves of the four limbs; parts with biological characteristics of pigs, such as the mouth, eyes and ears of pigs; in order to embody the rich body shape of the pig, the back, abdomen and tail vertebrae of the pig are taken as key points. And finally, connecting key points according to the characteristics of the pigs to form a posture graph of the pigs. A specific posture diagram of the pig is shown in fig. 3, and the corresponding reference numerals of the key point parts are shown in table 1 below, it should be understood that the following key points are only a preferred mode of the embodiment and are not specifically limited, and of course, different positions and numbers may be set as the key points according to actual situations, and are not described in detail herein.
TABLE 1 Key Point location labels
And S20, carrying out attitude estimation on the video information of the pig, and extracting key point sequence data according to the attitude graph of the pig.
After the key points and the posture maps of the pigs are set, corresponding key point sequences can be extracted from the videos of the pigs according to the key points of the pigs, and the specific steps are shown in fig. 4:
step S201, carrying out pig target detection on pig video information through a YOLO model to obtain a pig target area;
step S202, carrying out pig position extraction on the pig target area through an IOU-Tracker model to obtain a single pig position area;
step S203, converting the single pig position area into a single pig position picture, and preprocessing the single pig position picture;
and S204, carrying out attitude estimation on the preprocessed single-pig position picture through an OpenPose model, extracting a pig attitude sequence according to the pig attitude picture, and generating key point sequence data.
Referring to fig. 5, video information of a pig may be obtained by installing a camera device in an ambient environment such as a fence or a ceiling of a pig farm to capture and collect activity information of the pig in real time, and for the collected video information of the pig, the pig needs to be detected from the ambient environment of the video, here, an embodiment of the present invention preferably adopts a YOLO target detection model to detect the position of the pig in the video, so as to obtain a target area of the pig, and simultaneously extracts continuous single-pig position information from the target area of the pig by combining with an IOU-Tracker target tracking model, and converts the obtained position area of the single pig into a position picture of the single pig.
Because the uncertainty of the moving position of the pig may cause the problem that the position of a single pig in the picture is unclear, and the like, data preprocessing needs to be performed on the picture before the key point extraction is performed on the picture, including the modes of denoising the picture, smearing a pig target with serious shielding and the like, so that the interference caused by the unclear part on the identification is avoided, and finally, the attitude sequence of the pig can be extracted from the single pig position picture according to the preset pig key point and the preset attitude image by using an OpenPose attitude estimation model, so that the key point sequence data of the pig is generated. The specific extraction step may refer to the common use method of each model, and of course, the model used in this embodiment is only a preferred method, and models or algorithms with the same or similar functions may be applied to this embodiment, which will not be described in detail herein.
And S30, inputting the key point sequence data into a trained improved space-time diagram convolutional network to identify the pig behaviors to obtain a pig behavior classification prediction result, wherein the improved space-time diagram convolutional network comprises a first space-time diagram convolutional network and a second space-time diagram convolutional network.
After the key point sequence data of the pig is extracted, the improved space-time diagram convolutional network can be input to perform the behavior recognition of the pig, and before the pig behavior recognition is performed, please refer to fig. 6, and we first describe the network structure of the improved space-time diagram convolutional network in detail.
In the existing ST-GCN network, in order to fully utilize neighborhood information of key points and enhance the relation between the key points, the ST-GCN network adopts a strategy of partitioning the key points, and uses the key points v ti The neighboring node, which is the center point, is divided into K neighborhoods B (v) ti ). The ST-GCN algorithm provides three partitioning strategies: 1) Non-labeled zone, with center point v ti Nodes adjacent and within distance D =1 serve as neighborhood B (v) ti ) The partitioning mode is simple, but the priority does not exist among the nodes in the adjacent areas, so that the local difference among the nodes is easily lost; 2) Distance partition strategy, which is to partition two neighborhoods B (v) according to distances D =0 and 1 ti ) Subsets, forming two different weight vectors, strengthen the neighborhood B (v) ti ) Constructing node difference; 3) Spatial structure partitioning strategy that partitions neighborhood B (v) ti ) The division into three subsets: (1) the root node itself; (2) centripetal subset: a subset of nodes closer to the center of gravity than the root node; (3) centrifugation subset: a subset of nodes that are further from the center of gravity than the root node, may be represented by the following equation:
in the formula, r j Is the distance of node j from the center of gravity, r i The distance from the root node to the center of gravity.
Although the node perception range of the spatial structure partitioning strategy is wider than that of the first two partitioning strategies, the problems that the motion relation between a root node and adjacent nodes is lacked and the perception range of the nodes is too small still exist, therefore, the embodiment of the invention combines the posture structure of a pig, and in order to further expand the perception range of the nodes and enhance the relevance between the nodes, the perception distance between key points is expanded based on the characteristics of the spatial structure partitioning strategy, the distance D is increased to 3, and the key point V is constructed according to the distance D ti As shown in the following equation:
the new partitioning strategy for the pig posture not only contains local motion information of the root node, but also strengthens the motion relation between the root node and the adjacent nodes, so that the perception capability of the network on the whole posture is improved, and the accuracy of behavior recognition is increased.
Further, since the number of layers of the existing ST-GCN network is large, the problem of gradient descent or gradient disappearance is easily caused when an error is reversely propagated by using a gradient descent method, and the problem becomes more and more obvious along with the increase of the number of layers, in order to alleviate the problem of gradient descent and improve the utilization rate of network characteristic information of each layer, the embodiment of the present invention improves the existing ST-GCN network structure, designs a reduced ST-GCN network, namely an MST-GCN network, which only uses a three-layer spatio-temporal map convolution block for convolution operation, the number of channels of each layer is respectively 64, 128 and 256, the MST-GCN network can initially extract motion characteristics, then the initially extracted information of the MST-GCN network is fused with the ST-GCN network, and the behavior identification prediction of a pig key point sequence through the network of the structure can alleviate the problem of gradient descent or gradient disappearance and improve the utilization rate of characteristic information of a shallow layer network, and the following describes only a behavior identification process of the embodiment of the present invention by combining fig. 6 and fig. 7.
Step S301, inputting the key point sequence data into a first space-time diagram convolution network to perform three-layer convolution operation, and respectively generating first convolution data, second convolution data and third convolution data after each layer of convolution operation;
step S302, the first convolution data is used as the input of a second space-time diagram convolution network, and after three-layer convolution operation is carried out, fourth convolution data is generated;
step S303, inputting the first convolution data and the fourth convolution data into a fourth layer of the second space-time diagram convolution network, and generating fifth convolution data after performing three-layer convolution operation;
step S304, inputting the second convolution data and the fifth convolution data into a seventh layer of the second space-time diagram convolution network, and generating sixth convolution data after three-layer convolution operation;
and S305, splicing the third convolution data and the sixth convolution data, and inputting the spliced data into a classifier to perform classification and prediction of the pig behaviors.
The embodiment of the invention inputs the key point sequence data into the MST-GCN network, and after three layers of convolution operation, the output result of each layer of convolution operation can be obtained; then, taking the first layer output result of the MST-GCN network as an input layer of the ST-GCN network, and obtaining a third layer output result of the ST-GCN network after three layers of convolution operation in the ST-GCN network; fusing a first layer output result of the MST-GCN network and a third layer output result of the ST-GCN network together to be used as the input of a fourth layer of the ST-GCN network, and obtaining a sixth layer output result of the ST-GCN network after three-layer convolution operation from the fourth layer to the sixth layer in the ST-GCN network; then, the second layer output result of the MST-GCN network and the sixth layer output result of the ST-GCN network are fused and input to the seventh layer of the ST-GCN network, and the ninth layer output result of the ST-GCN network is obtained after the last three layers of convolution operation; and finally, after the third-layer output result of the MST-GCN network and the ninth-layer output result of the ST-GCN network are spliced, the spliced third-layer output result is input into a SoftMax classifier to realize the behavior classification prediction of the pigs.
The behavior classification of the pigs can be generally divided into normal behaviors and abnormal behaviors, the normal behaviors comprise eating, drinking, sleeping, excreting, walking and the like, the abnormal behaviors comprise activities exceeding a normal activity range, such as biting, fighting and the like, the identified abnormal behaviors can be timely notified to managers for processing, the normal behaviors can be identified and classified and counted according to the monitored activities of the pigs and behavior types, the statistical data can provide more detailed and accurate data support for feeding and adjusting the pigs in each growth stage in the feeding process, and the feeding efficiency of the pigs can be improved.
In order to better verify that the embodiment of the invention has better identification effect, the method of the embodiment and the existing pig identification method are tested by taking Top-1 accuracy as a comparison parameter, the specific test process can refer to the method and the use method of the existing ST-GCN model, and detailed description and explanation are not provided herein, and the test result shows that compared with the existing space structure partition strategy, the Top-1 of the partition strategy of the invention is improved by 0.59%, which shows that the partition strategy of the invention can obtain wider adjacent node information and the relevance of the nodes is stronger; compared with the Top-1 of the existing ST-GCN network model, the improved ST-GCN model of the invention is improved by 0.30 percent, namely the network model of the invention can slow down the gradient descent problem and improve the utilization of the network to the whole characteristic information; finally, the invention fuses the new partition strategy and the improved network structure to achieve the Top-1 of 95.00 percent, and improves the Top-1 by 2.06 percent compared with the ST-GCN network, so that the performance of the network model of the invention is obviously superior to that of the ST-GCN network model.
According to the pig behavior recognition method based on the improved space-time diagram convolutional network, pig key points are set according to the bone posture of a pig, a pig posture diagram is generated, the generated key point sequence is more in line with the recognition requirement of the pig, the sensing distance of the key points is expanded, more neighborhood subsets are constructed to improve the sensing capability of the network on the whole posture, the fusion of the reduced space-time diagram convolutional network and the space-time diagram convolutional network is further designed, the gradient reduction problem caused by the existing ST-GCN network is relieved, the utilization rate of feature information of a shallow network is improved, and therefore the accuracy of pig behavior recognition is improved.
Referring to fig. 8, based on the same inventive concept, a second embodiment of the present invention provides a pig behavior recognition system based on an improved space-time graph convolutional network, including:
the attitude map construction module 10 is used for constructing a pig attitude map according to preset pig attitude key points;
the key point extraction module 20 is used for carrying out attitude estimation on the video information of the pigs and extracting key point sequence data according to the attitude graph of the pigs;
and the behavior identification prediction module 30 is configured to input the key point sequence data into a trained improved space-time diagram convolutional network to identify the behavior of the pig, so as to obtain a pig behavior classification prediction result, where the improved space-time diagram convolutional network includes a first space-time diagram convolutional network and a second space-time diagram convolutional network.
Technical features and technical effects of the pig behavior recognition system based on the improved space-time diagram convolutional network provided by the embodiment of the invention are the same as those of the method provided by the embodiment of the invention, and are not repeated herein. The modules in the pig behavior recognition system based on the improved space-time graph convolutional network can be wholly or partially realized by software, hardware and a combination thereof. The modules can be embedded in a hardware form or independent of a processor in the computer device, and can also be stored in a memory in the computer device in a software form, so that the processor can call and execute operations corresponding to the modules.
Referring to fig. 9, in an embodiment, an internal structure of a computer device may specifically be a terminal or a server. The computer apparatus includes a processor, a memory, a network interface, a display, and an input device connected by a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device comprises a nonvolatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of an operating system and computer programs in the non-volatile storage medium. The network interface of the computer device is used for communicating with an external terminal through a network connection. The computer program is executed by a processor to implement a method for pig behavior recognition. The display screen of the computer equipment can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer equipment can be a touch layer covered on the display screen, a key, a track ball or a touch pad arranged on a shell of the computer equipment, an external keyboard, a touch pad or a mouse and the like.
It will be appreciated by those of ordinary skill in the art that the architecture shown in FIG. 9 is a block diagram of only a portion of the architecture associated with the subject application, and is not intended to limit the computing devices to which the subject application may be applied, as a particular computing device may include more or less components than those shown in the figures, or may combine certain components, or have the same arrangement of components.
In addition, an embodiment of the present invention further provides a computer device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, where the processor implements the steps of the method when executing the computer program.
Furthermore, an embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored, and the computer program, when executed by a processor, implements the steps of the above method.
To sum up, the method, the system, the computer device and the storage medium for identifying the pig behavior provided by the embodiment of the invention construct a pig posture graph according to preset pig posture key points; carrying out attitude estimation on the video information of the pig, and extracting key point sequence data according to the attitude graph of the pig; and inputting the key point sequence data into a trained improved space-time diagram convolution network to identify the pig behaviors to obtain a pig behavior classification prediction result, wherein the improved space-time diagram convolution network comprises a first space-time diagram convolution network and a second space-time diagram convolution network. According to the method, the reasonability of input data is improved by designing the posture graph of the pig, a network structure integrating a reduced space-time graph convolution network and a space-time graph convolution network is designed, the sensing distance of key points and neighborhood subsets of the improved space-time graph convolution network are increased, the kinematic relation between root nodes and adjacent nodes is enhanced, the sensing capability of the network on the whole posture is improved, the problem of gradient decrease of network identification in reverse transmission is solved, the utilization rate of shallow layer network characteristic information is improved, and the accuracy of pig behavior identification is integrally improved.
The embodiments in this specification are described in a progressive manner, and all the same or similar parts of the embodiments are directly referred to each other, and each embodiment is described with emphasis on differences from other embodiments. In particular, for the system embodiment, since it is substantially similar to the method embodiment, the description is simple, and for the relevant points, reference may be made to the partial description of the method embodiment. It should be noted that, the technical features of the embodiments may be arbitrarily combined, and for the sake of brevity, all possible combinations of the technical features in the embodiments are not described, but should be considered as the scope of the present specification as long as there is no contradiction between the combinations of the technical features.
The above-mentioned embodiments only express some preferred embodiments of the present application, and the description thereof is more specific and detailed, but not construed as limiting the scope of the invention. It should be noted that, for those skilled in the art, various modifications and substitutions can be made without departing from the technical principle of the present invention, and these should be construed as the protection scope of the present application. Therefore, the protection scope of the present patent shall be subject to the protection scope of the claims.
Claims (10)
1. A pig behavior identification method is characterized by comprising the following steps:
constructing a pig posture graph according to preset pig posture key points;
carrying out attitude estimation on the video information of the pig, and extracting key point sequence data according to the attitude graph of the pig;
and inputting the key point sequence data into a trained improved space-time diagram convolution network to identify the pig behaviors to obtain a pig behavior classification prediction result, wherein the improved space-time diagram convolution network comprises a first space-time diagram convolution network and a second space-time diagram convolution network.
2. The pig behavior recognition method according to claim 1, wherein the posture estimation is performed on pig video information, and the specific step of extracting the key point sequence data according to the pig posture graph comprises:
carrying out pig target detection on pig video information through a YOLO model to obtain a pig target area;
carrying out pig position extraction on the pig target area through an IOU-Tracker model to obtain a single pig position area;
converting the single pig position area into a single pig position picture, and preprocessing the single pig position picture;
and carrying out attitude estimation on the preprocessed single-pig position picture through an OpenPose model, extracting a pig attitude sequence according to the pig attitude picture, and generating key point sequence data.
3. The pig behavior recognition method according to claim 1, wherein the improved space-time graph convolutional network is partitioned according to a spatial structure, the perceptual distance between key points is 3, and the neighborhood of the key points is divided into 7 subsets according to the perceptual distance.
4. The pig behavior recognition method of claim 3, wherein the subset of neighborhoods is calculated using the following formula:
in the formula, v ti Is a key point, r j Is the distance of node j from the center of gravity, r i The distance from the root node to the center of gravity, D is the sensing distance between key points, l ti (v ti ) Is a key point v ti A neighborhood subset of (a).
5. The pig behavior recognition method of claim 1, wherein the first spatio-temporal graph convolutional network comprises three layers of first spatio-temporal graph volume blocks, and the number of channels of the three layers of first spatio-temporal graph volume blocks is 64, 128 and 256 in sequence; the second space-time graph convolutional network comprises nine layers of second space-time graph volume blocks, and the number of channels of the nine layers of second space-time graph volume blocks is 64, 128, 256 and 256 in sequence.
6. The pig behavior recognition method according to claim 5, wherein the specific step of inputting the key point sequence data into a trained improved spatiotemporal graph convolutional network to recognize the pig behavior to obtain the pig behavior classification prediction result comprises:
inputting the key point sequence data into a first time-space diagram convolution network to carry out three-layer convolution operation, and respectively generating first convolution data, second convolution data and third convolution data after each layer of convolution operation;
taking the first convolution data as the input of a second time-space diagram convolution network, and generating fourth convolution data after performing three-layer convolution operation;
inputting the first convolution data and the fourth convolution data into a fourth layer of the second space-time graph convolution network, and performing three-layer convolution operation to generate fifth convolution data;
inputting the second convolution data and the fifth convolution data into a seventh layer of the second space-time diagram convolution network, and generating sixth convolution data after three-layer convolution operation;
and splicing the third convolution data and the sixth convolution data, and inputting the spliced data into a classifier to perform classification and prediction of the behaviors of the pigs.
7. The pig behavior recognition method according to claim 1, wherein the preset pig posture key points comprise 20 pig key points of a nose, a right eye, a right ear, a left eye, a left ear, a right shoulder, a right front knee, a right front hoof, a right hip, a right rear knee, a right rear hoof, a left shoulder, a left front knee, a left front hoof, a left hip, a left rear knee, a left rear hoof, a back, an abdomen and a caudal vertebra, and 28 connected pig key point connection pairs.
8. A pig behavior recognition system, comprising:
the attitude map construction module is used for constructing a pig attitude map according to preset pig attitude key points;
the key point extraction module is used for carrying out attitude estimation on the pig video information and extracting key point sequence data according to the pig attitude graph;
and the behavior identification prediction module is used for inputting the key point sequence data into a trained improved space-time graph convolution network to identify the behavior of the pig so as to obtain a pig behavior classification prediction result, wherein the improved space-time graph convolution network comprises a first space-time graph convolution network and a second space-time graph convolution network.
9. A computer arrangement comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor implements the steps of the method according to any of claims 1 to 7 when executing the computer program.
10. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the method of any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210785276.XA CN115223198A (en) | 2022-07-05 | 2022-07-05 | Pig behavior identification method, pig behavior identification system, computer equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210785276.XA CN115223198A (en) | 2022-07-05 | 2022-07-05 | Pig behavior identification method, pig behavior identification system, computer equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115223198A true CN115223198A (en) | 2022-10-21 |
Family
ID=83610127
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210785276.XA Pending CN115223198A (en) | 2022-07-05 | 2022-07-05 | Pig behavior identification method, pig behavior identification system, computer equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115223198A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117253031A (en) * | 2023-11-16 | 2023-12-19 | 应急管理部天津消防研究所 | Forest fire monitoring method based on multi-element composite deep learning |
-
2022
- 2022-07-05 CN CN202210785276.XA patent/CN115223198A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117253031A (en) * | 2023-11-16 | 2023-12-19 | 应急管理部天津消防研究所 | Forest fire monitoring method based on multi-element composite deep learning |
| CN117253031B (en) * | 2023-11-16 | 2024-01-30 | 应急管理部天津消防研究所 | Forest fire monitoring method based on multi-element composite deep learning |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111382679B (en) | Method, system and equipment for evaluating severity of gait dyskinesia of Parkinson's disease | |
| JP4643766B1 (en) | Moving body detection apparatus and moving body detection method | |
| CN109635783B (en) | Video monitoring method, device, terminal and medium | |
| CN110363817B (en) | Target pose estimation method, electronic device, and medium | |
| CN113963445B (en) | Pedestrian falling action recognition method and equipment based on gesture estimation | |
| CN107624189A (en) | Method and apparatus for generating forecast model | |
| CN111062263B (en) | Method, apparatus, computer apparatus and storage medium for hand gesture estimation | |
| JP2022516398A (en) | Image processing methods and image processing equipment, processors, electronic devices and storage media | |
| CN112528974B (en) | Distance measuring method and device, electronic equipment and readable storage medium | |
| CN113869276B (en) | Lie recognition method and system based on micro-expression | |
| CN111652181B (en) | Target tracking method and device and electronic equipment | |
| CN112419326A (en) | Image segmentation data processing method, device, equipment and storage medium | |
| CN110738650A (en) | infectious disease infection identification method, terminal device and storage medium | |
| Yan et al. | Human-object interaction recognition using multitask neural network | |
| CN110569757B (en) | Multi-posture pedestrian detection method based on deep learning and computer storage medium | |
| CN115223198A (en) | Pig behavior identification method, pig behavior identification system, computer equipment and storage medium | |
| CN111241926A (en) | Attendance checking and learning condition analysis method, system, equipment and readable storage medium | |
| CN111563492B (en) | Fall detection method, fall detection device and storage device | |
| CN113488163A (en) | Hypomyosis identification method, device, equipment and medium based on machine vision | |
| KR20210026664A (en) | Apparatus, method and computer program for categorizing motion of object | |
| CN113792700B (en) | Storage battery car in-box detection method and device, computer equipment and storage medium | |
| CN115424346A (en) | Human body sitting posture detection method and device, computer equipment and system | |
| CN111488476B (en) | Image pushing method, model training method and corresponding devices | |
| CN114387670A (en) | Gait recognition method and device based on space-time feature fusion and storage medium | |
| CN114495272A (en) | Motion recognition method, motion recognition device, storage medium, and computer apparatus |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |