CN111666881B - Giant panda pacing, bamboo eating and estrus behavior tracking analysis method - Google Patents
Giant panda pacing, bamboo eating and estrus behavior tracking analysis method Download PDFInfo
- Publication number
- CN111666881B CN111666881B CN202010510090.4A CN202010510090A CN111666881B CN 111666881 B CN111666881 B CN 111666881B CN 202010510090 A CN202010510090 A CN 202010510090A CN 111666881 B CN111666881 B CN 111666881B
- Authority
- CN
- China
- Prior art keywords
- background
- pixel
- current pixel
- frame
- foreground
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/20—Movements or behaviour, e.g. gesture recognition
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/23—Clustering techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
- G06F18/2411—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on the proximity to a decision surface, e.g. support vector machines
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
- G06V10/267—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion by performing operations on regions, e.g. growing, shrinking or watersheds
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/46—Descriptors for shape, contour or point-related descriptors, e.g. scale invariant feature transform [SIFT] or bags of words [BoW]; Salient regional features
- G06V10/462—Salient features, e.g. scale invariant feature transforms [SIFT]
- G06V10/464—Salient features, e.g. scale invariant feature transforms [SIFT] using a plurality of salient features, e.g. bag-of-words [BoW] representations
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/40—Scenes; Scene-specific elements in video content
- G06V20/41—Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/20—Movements or behaviour, e.g. gesture recognition
- G06V40/23—Recognition of whole body movements, e.g. for sport training
- G06V40/25—Recognition of walking or running movements, e.g. gait recognition
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Data Mining & Analysis (AREA)
- Multimedia (AREA)
- Psychiatry (AREA)
- Life Sciences & Earth Sciences (AREA)
- General Health & Medical Sciences (AREA)
- General Engineering & Computer Science (AREA)
- Social Psychology (AREA)
- Human Computer Interaction (AREA)
- Evolutionary Computation (AREA)
- Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Evolutionary Biology (AREA)
- Computational Linguistics (AREA)
- Software Systems (AREA)
- Image Analysis (AREA)
Abstract
The invention relates to the technical field of information, and provides a panda pacing and eating bamboo and estrus behavior tracking analysis method. The method aims at solving the problems that the panda motion background is complex, and the traditional background extraction algorithm is difficult to achieve the ideal foreground target extraction effect. The method comprises the following steps that 1, a panda video image is input, and a video frame is subjected to foreground target extraction by an improved vibe method; step 2, performing morphological corrosion expansion on the extracted foreground template; step 3, taking the smallest circumscribed rectangle of the outline with the largest area of the communication area as a target area and taking the centroid of the target area as the position of the target; and 4, carrying out the same operation as the step 1-3 on each frame of image, outputting the movement track and movement speed of the pandas, and analyzing the behaviors.
Description
Technical Field
The invention relates to the technical field of information, and provides a panda pacing and eating bamboo and estrus behavior tracking analysis method.
Background
Pandas are special rare or endangered wild animals in China. For many years, panda populations have been under pressure for habitat loss and fragmentation due to human activities such as large-area forest cutting, barren, hunting, construction of large infrastructures such as highway railways, etc. At present, only about 1864 wild pandas exist, and are distributed on the south foot of Minshan, qiong, qinling mountain and Qinling mountain. The site protection, namely the artificial breeding and reproduction, is one of the basic approaches of the protection of endangered species, namely the supplement and expansion of the site protection (namely the habitat protection), and has important effects on increasing the population quantity, maintaining the existing breeding population of the housed panda and maintaining the continuation of the species. However, the current population of the containment panda faces the problems of high morbidity, low birth rate, poor health condition, degraded behaviors and the like.
Protection of endangered wild animals includes two important means of on-site protection and detour protection, and detour protection has made an important breakthrough in recent years as an important supplementary means of on-site protection of pandas. The rearing of pandas in stock was started in 1936, 11-month hakuni (Rush Harkness) obtained one two-month-old male pandas in the Sichuan-Wenchen grass slope in 1936 and named "Su Lin" (pandas international lineage number 1). "Su Lin" is the first living panda to be brought out of the country and developed in chicago zoos in the united states at month 2 in 1937. After the establishment of new China, the panda is raised from the adult zoo for the first time in 1953, and the history of raising pandas in China is started. Although the raising of pandas has been over 70 years old, the raising and breeding history of pandas is abnormal and tortuous. The progress from 1936 to 90 th century is slow, and the breeding of the containment pandas is very difficult. The technologies of raising, breeding and disease prevention and control of the housed pandas have made great progress since the 90 s of the last century, especially since 2000, but there are some directions in which further research is needed.
Most of the behavior recognition technologies based on videos are used for recognizing human behaviors, the behavior recognition of animals is few, and only a small number of researchers are used for researching the behavior recognition of animals such as pigs, chickens and the like. The research on pandas is mainly in the biological fields of genes, genetics and the like, a small amount of detection research on pandas in static images is carried out, the research on pandas behavior recognition in videos is currently in a blank stage, research analysis is carried out on pandas behavior recognition and tracking in videos, the monitoring of physiological, mental health and propagation states of pandas is facilitated, and contribution is made to the promotion of the health state and population number of the population of pandas.
In order to master the physiological health, mental health and oestrus state of the pandas, the actions of eating bamboo, pacing and oestrus of the pandas are detected and recorded, then analyzed, and if the pandas are abnormal, reasonable measures are taken in time to ensure that the pandas are in a healthy state.
Disclosure of Invention
The invention aims to solve the problems that the panda motion background is complex, and the traditional background extraction algorithm is difficult to achieve the ideal foreground target extraction effect.
The invention adopts the following technical scheme to solve the technical problems:
a giant panda pacing behavior tracking analysis method is characterized by comprising the following steps:
step 1, inputting panda video images, and extracting foreground targets of the video frames by using an improved vibe method;
step 2, performing morphological corrosion expansion on the extracted foreground template;
step 3, taking the smallest circumscribed rectangle of the outline with the largest area of the communication area as a target area and taking the centroid of the target area as the position of the target;
and 4, carrying out the same operation as the step 1-3 on each frame of image, outputting the movement track and movement speed of the pandas, and analyzing the behaviors.
In the above technical solution, the improved vibe method includes the following steps:
step 1.1, initializing a background, and selecting the first n frames of a video by using a multi-frame averaging method to construct an initial background B0;
step 1.2, a sample set M (x, y) = { v1, v2, &..vn }, where vi is an 8-neighborhood random sample value of (x, y), i=1, 2, …, N, is created for each pixel (x, y) of the initial background B0;
step 1.3, calculating an i-th frame image fi (i=2, 3..n):
TB i ＝F OTSU (abs(f i -B rd ))
TF i ＝F OTSU (abs(f i -f i-1 ))
R i ＝TF i +(1-a)·TB i
wherein B is rd Representing the background of selecting the rd sample from each sample set, rd is a randomly selected value from {1,2,..N }, F OTSU (. Cndot.) represents the background segmentation threshold, TB, after foreground segmentation calculated using the OTSU method i Segmentation threshold value and Inf of background differential result calculated by OTSU method i (x, y) represents the binarization result of the ith frame image at (x, y), TF i Segmentation threshold representing frame difference result calculated by OTSU method, R i The value of the radius threshold R of the ith frame is represented, and alpha is a weighting coefficient and is generally a zero point;
such as Inf i (x, y) =1, the following process is performed:
step 1.3.1, judging whether the current pixel (x, y) is background, judging whether the current pixel is background by calculating the similarity degree between the current pixel (x, y) and the corresponding sample set, and specifically calculating as follows:
cnt j the method comprises the steps of representing a judging result of the similarity degree of a current pixel (x, y) and a jth background sample pixel in a background sample set, and judging that the current pixel point is a background pixel if the sum of comparison results of the current pixel and all background pixel points in the background sample set is greater than or equal to a threshold value T; otherwise, foreground pixels; f (f) i Showing an ith frame of video frame, which refers to the video frame where the current video frame is located; dis represents the Euclidean distance between two pixels; v j Representing background samplesThe j-th pixel point in the set;
DB i (x, y) represents a judgment result that the pixel point (x, y) in the ith frame image is a foreground or background pixel point, and the current pixel is a foreground pixel, namely DBi (x, y) =1;
the current pixel (x, y) is the background pixel, i.e. DB i When (x, y) =0, background update is performed with a probability of 1/θ, the background update is divided into two parts of current sample set update and neighborhood update, θ is a time sampling factor, and is generally taken as 16, it is not necessary to update the background model in every new video frame, and when a pixel point is classified as a background point, it has a probability of 1/θ to update the background model;
first, the sample set is updated with the pixel value f of the current pixel (x, y) i (x, y) replacing a randomly selected one of the samples v in its corresponding set of background samples M (x, y) i d is v i d＝f i (x，y)；
Secondly, a neighborhood update is carried out, and a current pixel (x) at a position is randomly selected in 8 neighborhood of the current pixel (x, y) 1 ，y 1 ) And then the background sample set M (x 1 ，y 1 ) Medium shorthand selects one sample v 1 The current pixel is used for replacement, namely v i ＝f i (x，y)。
The invention provides a panda bamboo eating and oestrus behavior identification method, which comprises the following steps:
step 1, inputting panda video images, and extracting foreground targets of the video frames by using an improved vibe method to obtain foreground target images;
step 2, constructing a multi-scale space pyramid in the foreground target image, acquiring candidate points of a dense track through dense sampling, and extracting the dense track from different space scales;
step 3, using ut to represent horizontal component in the optical flow field, vt to represent vertical component in the optical flow field, ω= (ut, vt)Then the dense optical flow field between the t frame and the t+1st frame image is represented, and the characteristic point Pt= (xt, yt) on the t frame image is in the optical flow field omega t The above smoothing process is performed by using a median filter M, and the position on the t+1st frame corresponding to the point after smoothing is defined as:
wherein the method comprises the steps of
Is represented by (x) t ，y t ) A circular region of center omega t For the light stream domain, M is median filtering (please supplement), and the motion trail (P) is formed by connecting the characteristic points tracked in the subsequent frames in series t ，P t+1 ，……)；
Step 4, tracking the characteristic points in an optical flow field to form a motion track, restraining the tracking length L to avoid tracking drift phenomenon caused by long-time tracking, constructing a characteristic descriptor along a dense track, collecting HOG and track shapes as shape descriptors, and utilizing HOF and MBH as motion descriptors;
step 5, performing dimension reduction on the obtained feature descriptors by adopting principal component analysis (Principal Component Analysis, PCA), mapping data from a high-dimensional space to a low-latitude space, and simultaneously ensuring that as much main information as possible is reserved during mapping to obtain feature descriptors with feature dimension d after dimension reduction;
step 6, modeling local features by adopting a Gaussian Mixture Model (GMM) based on feature coding and classification of Fisher Vector, taking the number K of Gaussian clusters, and training a local feature set by using an EM algorithm to solve the GMM; then using Fisher Vector to encode the feature descriptors after dimension reduction, wherein the feature dimension obtained after encoding is 2 Xd X K;
and 8, finally, sending the obtained coded feature descriptors into an SVM classifier for classification.
In the above technical solution, the improved vibe method includes the following steps:
step 1.1, initializing a background, and selecting the first n frames of a video by using a multi-frame averaging method to construct an initial background B0;
step 1.2, a sample set M (x, y) = { v1, v2, &..vn }, where vi is an 8-neighborhood random sample value of (x, y), i=1, 2, …, N, is created for each pixel (x, y) of the initial background B0;
step 1.3, calculating an i-th frame image fi (i=2, 3..n):
TB i ＝F OTSU (abs(f i -B rd ))
TF i ＝F OTSU (abs(f i -f i-1 ))
R i ＝TF i +(1-a)·TB i
wherein B is rd Representing the background of selecting the rd sample from each sample set, rd is a randomly selected value from {1,2,..N }, F OTSU (. Cndot.) represents the background segmentation threshold, TB, after foreground segmentation calculated using the OTSU method i Segmentation threshold value and Inf of background differential result calculated by OTSU method i (x, y) represents the binarization result of the ith frame image at (x, y), TF i Segmentation threshold representing frame difference result calculated by OTSU method, R i The value of the radius threshold R of the ith frame is represented, and alpha is a weighting coefficient and is generally a zero point;
such as Inf i (x, y) =1, the following process is performed:
step 1.3.1, judging whether the current pixel (x, y) is background, judging whether the current pixel is background by calculating the similarity degree between the current pixel (x, y) and the corresponding sample set, and specifically calculating as follows:
cnt j representing a current imageJudging the similarity degree of the pixel (x, y) and the jth background sample pixel in the background sample set, and judging the current pixel point as a background pixel if the sum of the comparison results of the current pixel and all the background pixel points in the background sample set is greater than or equal to a threshold value T; otherwise, foreground pixels; f (f) i Showing an ith frame of video frame, which refers to the video frame where the current video frame is located; dis represents the Euclidean distance between two pixels; v j Representing the j-th pixel point in the background sample set;
DB i (x, y) represents a judgment result that the pixel point (x, y) in the ith frame image is a foreground or background pixel point, and the current pixel is a foreground pixel, namely DBi (x, y) =1;
the current pixel (x, y) is the background pixel, i.e. DB i When (x, y) =0, background updating is carried out with the probability of 1/theta, the background updating is divided into two parts of current sample set updating and neighborhood updating, and theta is a time sampling factor;
first, the sample set is updated with the pixel value f of the current pixel (x, y) i (x, y) replacing a randomly selected one of the samples v in its corresponding set of background samples M (x, y) i d is v i d＝f i (x，y)；
Secondly, a neighborhood update is carried out, and a current pixel (x) at a position is randomly selected in 8 neighborhood of the current pixel (x, y) 1 ，y 1 ) And then the background sample set M (x 1 ，y 1 ) Medium shorthand selects one sample v 1 The current pixel is used for replacement, namely v i ＝f i (x，y)。
Because the invention adopts the technical scheme, the invention has the following beneficial effects:
because pandas live in an artificially constructed environment, compared with the wild environment, the pandas have limited activity space and insufficient environment, and some pandas possibly have mental boredom after a period of time, so that mental problems are generated, and the pandas can do repetitive actions, such as repeated walking in a closed route, and the actions are also called as the plate-carving and walking actions. In order to identify the pacing behavior of the panda, the following method is adopted:
1) Because the panda motion background is complex, the traditional background extraction algorithm is difficult to achieve the ideal foreground target extraction effect, and the improved vibe algorithm is used for extracting the panda foreground target. By constructing an initial background by adopting a multi-frame averaging method and then modeling a background model, the problem that the traditional vibe algorithm cannot reflect scene changes in time and the quality of the extracted foreground target is low is solved, and the accuracy of extracting the foreground target is effectively improved.
2) Carrying out morphological corrosion expansion operation and connected domain analysis on the extracted image, taking the outline with the largest area of the connected region as a panda active region, solving the minimum circumscribed rectangle of the region, and taking the centroid of the region as the position of the panda; and carrying out the same tracking operation on each frame of image to obtain the movement track of the pandas. Compared with the traditional method, the method provided by the invention ensures higher tracking accuracy and reduces the computational complexity.
3) By analyzing the repeatability of the movement track, judging whether the movement belongs to the movement behavior of the notch board, judging whether the movement habit of the panda is abnormal, if so, indicating that the mental state is wrong, and timely taking corresponding measures to treat the panda.
Drawings
FIG. 1 is a schematic diagram of the steps in the flow of the present invention.
Detailed Description
The present invention will be described in further detail with reference to the drawings and examples, in order to make the objects, technical solutions and advantages of the present invention more apparent. It should be understood that the particular embodiments described herein are illustrative only and are not intended to limit the invention, i.e., the embodiments described are merely some, but not all, of the embodiments of the invention. The components of the embodiments of the present invention generally described and illustrated in the figures herein may be arranged and designed in a wide variety of different configurations.
Thus, the following detailed description of the embodiments of the invention, as presented in the figures, is not intended to limit the scope of the invention, as claimed, but is merely representative of selected embodiments of the invention. All other embodiments, which can be made by a person skilled in the art without making any inventive effort, are intended to be within the scope of the present invention.
It is noted that relational terms such as "first" and "second", and the like, are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.
In the panda behavior recognition process, the following key problems mainly exist:
the first key problem is how to accurately track the pandas and record the movement track and movement duration of the pandas.
The second key problem is how to accurately identify the behavior of pandas, because the shape, behavior and human being of pandas are very different, most of the existing behavior identification based on video is to identify the behavior of human body, and aiming at the characteristics of irregular behavior and changeable shape of pandas, a reasonable algorithm is required to be designed to realize more accurate identification and record.
Tracking and analyzing the pacing behaviors of pandas. The panda repeatedly walks for more than three times along the same route on the same road section is called as a notch board pacing behavior, and the occurrence of the behavior possibly means that the mental condition of the panda is abnormal to a certain extent, unconscious repeated behavior is generated, and a certain measure is needed to be taken in time for intervention, so that the mental condition of the panda is improved.
Panda feeding behavior recognition analysis, wherein panda feeding is mainly performed by bamboo, feeding time is compared in a peace time through research recognition analysis, and if the time is abnormal, whether the analysis is related to abnormal health conditions of pandas, such as teeth or digestive system, is caused, and treatment means need to be timely adopted.
Panda oestrus behavior recognition, wherein the occurrence frequency of special behaviors of pandas in oestrus period can be increased suddenly, such as inverted rubbing yin, side lifting, leg rubbing yin, tail lifting and the like. The panda estrus can be effectively monitored by identifying and counting the occurrence times of the special behaviors, so that preparation is made for the breeding work of the panda.
The invention provides a panda pacing behavior tracking analysis method, which is characterized by comprising the following steps of:
step 1, inputting panda video images, and extracting foreground targets of the video frames by using an improved vibe method;
step 2, performing morphological corrosion expansion on the extracted foreground template;
step 3, taking the smallest circumscribed rectangle of the outline with the largest area of the communication area as a target area and taking the centroid of the target area as the position of the target;
and 4, carrying out the same operation as the step 1-3 on each frame of image, outputting the movement track and movement speed of the pandas, and analyzing the behaviors.
In the above technical solution, the improved vibe method includes the following steps:
step 1.1, initializing a background, and selecting the first n frames of a video by using a multi-frame averaging method to construct an initial background B0;
step 1.2, a sample set M (x, y) = { v1, v2, &..vn }, where vi is an 8-neighborhood random sample value of (x, y), i=1, 2, …, N, is created for each pixel (x, y) of the initial background B0;
step 1.3, calculating an i-th frame image fi (i=2, 3..n):
TB i ＝F OTSU (abs(f i -B rd ))
TF i ＝F OTSU (abs(f i -f i-1 ))
R i ＝TF i +(1-a)·TB i
wherein B is rd Representing the background of selecting the rd sample from each sample set, rd is a randomly selected value from {1,2,..N }, F OTSU (. Cndot.) represents the background segmentation threshold, TB, after foreground segmentation calculated using the OTSU method i Segmentation threshold value and Inf of background differential result calculated by OTSU method i (x, y) represents the binarization result of the ith frame image at (x, y), TF i Segmentation threshold representing frame difference result calculated by OTSU method, R i The value of the radius threshold R of the ith frame is represented, and alpha is a weighting coefficient and is generally a zero point;
such as Inf i (x, y) =1, the following process is performed:
step 1.3.1, judging whether the current pixel (x, y) is background, judging whether the current pixel is background by calculating the similarity degree between the current pixel (x, y) and the corresponding sample set, and specifically calculating as follows:
cnt j the method comprises the steps of representing a judging result of the similarity degree of a current pixel (x, y) and a jth background sample pixel in a background sample set, and judging that the current pixel point is a background pixel if the sum of comparison results of the current pixel and all background pixel points in the background sample set is greater than or equal to a threshold value T; otherwise, foreground pixels; f (f) i Showing an ith frame of video frame, which refers to the video frame where the current video frame is located; dis represents the Euclidean distance between two pixels; v j Representing the j-th pixel point in the background sample set;
DB i (x, y) represents a judgment result that the pixel point (x, y) in the ith frame image is a foreground or background pixel point, and the current pixel is a foreground pixel, namely DBi (x, y) =1;
the current pixel (x, y) is the background pixel, i.e. DB i When (x, y) =0, background update is performed with a probability of 1/θ, the background update is divided into two parts of current sample set update and neighborhood update, θ is a time sampling factor, and is generally taken as 16, it is not necessary to update the background model in every new video frame, and when a pixel point is classified as a background point, it has a probability of 1/θ to update the background model;
first, the sample set is updated with the pixel value f of the current pixel (x, y) i (x, y) replacing a randomly selected one of the samples v in its corresponding set of background samples M (x, y) i d is v i d＝f i (x，y)；
Secondly, a neighborhood update is carried out, and a current pixel (x) at a position is randomly selected in 8 neighborhood of the current pixel (x, y) 1 ，y 1 ) And then the background sample set M (x 1 ，y 1 ) Medium shorthand selects one sample v 1 The current pixel is used for replacement, namely v i ＝f i (x，y)。
The behavior recognition and analysis of pandas eating bamboo can often identify the health state of pandas, if the eating time suddenly increases or decreases greatly, the situation appears several times continuously, whether the pandas has problems in teeth or digestive system needs to be checked, and the treatment needs to be performed in time. Two steps are needed for realizing the recognition and recording of panda bamboo eating behaviors: firstly, identifying the behavior of eating the bamboo, secondly, recording the time length of eating the bamboo, comparing the time length with historical data, and analyzing whether abnormal conditions exist or not.
The panda estrus behavior recognition and analysis, the propagation of pandas is an important factor influencing the population quantity of pandas, and the pandas has short estrus period, so that the pandas estrus behavior is monitored in advance, the pandas estrus period is accurately mastered, the propagation of pandas can be promoted to a great extent, the pandas quantity is enlarged, and the pandas estrus behavior recognition method has important significance.
The estrus behavior of pandas is marked with smell or rubbed into the vagina, and the pandas moves rapidly, lifts the tail, collides, etc. The panda estrus behavior recognition analysis mainly comprises two steps: firstly, the oestrus behaviors are classified and identified, secondly, the number of oestrus behaviors is recorded, and the occurrence number of each behavior is recorded, so that the subsequent analysis of the oestrus period of the pandas is facilitated.
Panda behavior recognition based on foreground object extraction and dense tracks, wherein panda activities are in an artificially constructed activity area, and the background is complex. If the dense track extraction is directly performed based on the original image, the feature bit number is too high, the calculation amount is large, and a large amount of background redundant information is contained, and a behavior recognition method based on foreground object extraction and dense tracks is provided for solving the problem. Firstly, extracting a target area of a video frame, then extracting a dense track in the target area, constructing a feature descriptor along the track, reducing the feature dimension of the obtained feature descriptor by using principal component analysis PCA, reducing the calculated amount, modeling the local feature by using a Gaussian mixture model, encoding the feature by using a Fisher vector, and finally training and classifying by using SVM.
The invention provides a panda bamboo eating and oestrus behavior identification method, which comprises the following steps:
step 1, inputting panda video images, and extracting foreground targets of the video frames by using an improved vibe method to obtain foreground target images;
step 2, constructing a multi-scale space pyramid in the foreground target image, acquiring candidate points of a dense track through dense sampling, and extracting the dense track from different space scales;
step 3, using ut to represent horizontal component in the optical flow field, vt to represent vertical component in the optical flow field, and ω= (ut, vt) thenShowing dense optical flow field between t frame and t+1st frame, for characteristic point Pt= (xt, yt) on t frame image in optical flow field omega t The above smoothing process is performed by using a median filter M, and the position on the t+1st frame corresponding to the point after smoothing is defined as:
wherein the method comprises the steps of
Is represented by (x) t， ，y t ) A circular region of center omega t For the light stream domain, M is median filtering (please supplement), and the motion trail (P) is formed by connecting the characteristic points tracked in the subsequent frames in series t ，P t+1 ，……)；
Step 4, tracking the characteristic points in an optical flow field to form a motion track, restraining the tracking length L to avoid tracking drift phenomenon caused by long-time tracking, constructing a characteristic descriptor along a dense track, collecting HOG and track shapes as shape descriptors, and utilizing HOF and MBH as motion descriptors;
step 5, performing dimension reduction on the obtained feature descriptors by adopting principal component analysis (Principal Component Analysis, PCA), mapping data from a high-dimensional space to a low-latitude space, and simultaneously ensuring that as much main information as possible is reserved during mapping to obtain feature descriptors with feature dimension d after dimension reduction;
step 6, modeling local features by adopting a Gaussian Mixture Model (GMM) based on feature coding and classification of Fisher Vector, taking the number K of Gaussian clusters, and training a local feature set by using an EM algorithm to solve the GMM; then using Fisher Vector to encode the feature descriptors after dimension reduction, wherein the feature dimension obtained after encoding is 2 Xd X K;
and 8, finally, sending the obtained coded feature descriptors into an SVM classifier for classification.
In the above technical solution, the improved vibe method includes the following steps:
step 1.1, initializing a background, and selecting the first n frames of a video by using a multi-frame averaging method to construct an initial background B0;
step 1.2, a sample set M (x, y) = { v1, v2, &..vn }, where vi is an 8-neighborhood random sample value of (x, y), i=1, 2, …, N, is created for each pixel (x, y) of the initial background B0;
step 1.3, calculating an i-th frame image fi (i=2, 3..n):
TB i ＝F OTSU (abs(f i -B rd ))
TF i ＝F OTSU (abs(f i -f i-1 ))
R i ＝TF i +(1-a)·TB i
wherein B is rd Representing the background of selecting the rd sample from each sample set, rd is a randomly selected value from {1,2,..N }, F OTSU (. Cndot.) represents the background segmentation threshold, TB, after foreground segmentation calculated using the OTSU method i Segmentation threshold value and Inf of background differential result calculated by OTSU method i (x, y) represents the binarization result of the ith frame image at (x, y), TF i Segmentation threshold representing frame difference result calculated by OTSU method, R i The value of the radius threshold R of the ith frame is represented, and alpha is a weighting coefficient and is generally a zero point;
such as Inf i (x, y) =1, the following process is performed:
step 1.3.1, judging whether the current pixel (x, y) is background, judging whether the current pixel is background by calculating the similarity degree between the current pixel (x, y) and the corresponding sample set, and specifically calculating as follows:
cnt j representing a current pixelJudging the similarity degree of the (x, y) and the jth background sample pixel in the background sample set, and judging that the current pixel point is a background pixel if the sum of the comparison results of the current pixel and all the background pixel points in the background sample set is greater than or equal to a threshold value T; otherwise, foreground pixels; f (f) i Showing an ith frame of video frame, which refers to the video frame where the current video frame is located; dis represents the Euclidean distance between two pixels; v j Representing the j-th pixel point in the background sample set;
DB i (x, y) represents a judgment result that the pixel point (x, y) in the ith frame image is a foreground or background pixel point, and the current pixel is a foreground pixel, namely DBi (x, y) =1;
the current pixel (x, y) is the background pixel, i.e. DB i When (x, y) =0, background updating is carried out with the probability of 1/theta, the background updating is divided into two parts of current sample set updating and neighborhood updating, and theta is a time sampling factor;
first, the sample set is updated with the pixel value f of the current pixel (x, y) i (x, y) replacing a randomly selected one of the samples v in its corresponding set of background samples M (x, y) i d is v i d＝f i (x，y)；
Secondly, a neighborhood update is carried out, and a current pixel (x) at a position is randomly selected in 8 neighborhood of the current pixel (x, y) 1 ，y 1 ) And then the background sample set M (x 1 ，y 1 ) Medium shorthand selects one sample v 1 The current pixel is used for replacement, namely v i ＝f i (x，y)。
Claims (2)
1. A giant panda pacing behavior tracking analysis method is characterized by comprising the following steps:
step 1, inputting panda video images, and extracting foreground targets of the video frames by using an improved vibe method;
step 2, performing morphological corrosion expansion on the extracted foreground template;
step 3, taking the smallest circumscribed rectangle of the outline with the largest area of the communication area as a target area and taking the centroid of the target area as the position of the target;
step 4, carrying out the same operation as the step 1-3 on each frame of image, outputting the movement track and movement speed of the pandas, and analyzing the behaviors;
the improved vibe method comprises the following steps:
step 1.1, initializing a background, and selecting the first n frames of a video by using a multi-frame averaging method to construct an initial background B0;
step 1.2, for each pixel (x, y) of the initial background B0, creating a sample set M (x, y) = { v1, v2,..v }, where viIs an 8-neighborhood random sampling value of (x, y),i=1，2，⋯，N；
step 1.3, pair IiFrame imagefi(i=2,3..n) And (3) performing calculation:
wherein the method comprises the steps of
Representing selection of the first sample setrdA background of the composition of the individual samples,rdis a value randomly selected from {1,2,..N }, is }>
Representing the foreground segmentation post-background segmentation threshold value calculated by the OTSU method, < >>
Segmentation threshold value representing the calculation of background differential result by OTSU method,/->
Represent the firstiFrame image is +.>
Binarization results at->
Segmentation threshold value representing the result of calculating the frame difference by OTSU method,/->
Represent the firstiFrame radius thresholdRIs used for the value of (a) and (b),αis a weighting coefficient;
such as
The following process is performed:
step 1.3.1, judging the current pixel
Whether or not it is background by calculating the current pixel +.>
The similarity degree of the current pixel and the corresponding sample set is used for judging whether the current pixel is background, and the specific calculation is as follows:
current pixel
Is a background pixel, i.e.)>
When the method is used, background updating is carried out according to the probability of 1/theta, wherein the background updating is divided into two parts, namely current sample set updating and neighborhood updating, and theta is a time sampling factor;
first, sample set update with current pixel
Pixel value +.>
Replace its corresponding background sample set +.>
Is a randomly selected sample->
Namely +.>
；
Secondly, the neighborhood is updated, the current pixel
Randomly selecting a position current pixel in 8 neighborhoods of (1)>
In the corresponding background sample set +.>
Medium shorthand select a sample +.>
The current pixel is used for replacement, namely
。
2. A panda bamboo eating and oestrus behavior recognition method is characterized by comprising the following steps:
step 1, inputting panda video images, and extracting foreground targets of the video frames by using an improved vibe method to obtain foreground target images;
step 2, constructing a multi-scale space pyramid in the foreground target image, acquiring candidate points of a dense track through dense sampling, and extracting the dense track from different space scales;
step 3, useutRepresenting the horizontal component in the optical flow field,vtrepresenting the vertical component in an optical flow field， 
=(ut，vt) Then the dense optical flow field between the t frame and the t+1st frame image is represented, and the characteristic point Pt= (xt, yt) on the t frame image is in the optical flow field->
The above smoothing process is performed by using a median filter M, and the position on the t+1st frame corresponding to the point after smoothing is defined as:
wherein the method comprises the steps of
A circular area as the center,>
is thatThe light flow field M is median filtering, and the characteristic points tracked in the subsequent frames are connected in series to form a motion track (>
，……）；
Step 4, tracking the characteristic points in an optical flow field to form a motion track, restraining the tracking length L to avoid tracking drift phenomenon caused by long-time tracking, constructing a characteristic descriptor along a dense track, collecting HOG and track shapes as shape descriptors, and utilizing HOF and MBH as motion descriptors;
step 5, performing dimension reduction on the obtained feature descriptors by adopting principal component analysis (Principal Component Analysis, PCA), mapping data from a high-dimensional space to a low-latitude space, and simultaneously ensuring that as much main information as possible is reserved during mapping to obtain feature descriptors with feature dimension d after dimension reduction;
step 6, modeling local features by adopting a Gaussian Mixture Model (GMM) based on feature coding and classification of Fisher Vector, taking the number K of Gaussian clusters, and training a local feature set by using an EM algorithm to solve the GMM; then using Fisher Vector to encode the feature descriptors after dimension reduction, wherein the feature dimension obtained after encoding is 2 Xd X K;
step 8, finally, sending the obtained coded feature descriptors into an SVM classifier for classification;
the improved vibe method comprises the following steps:
step 1.1, initializing a background, and selecting the first n frames of a video by using a multi-frame averaging method to construct an initial background B0;
step 1.2, for each pixel (x, y) of the initial background B0, creating a sample set M (x, y) = { v1, v2,..v }, where viIs an 8-neighborhood random sampling value of (x, y),i=1，2，⋯，N；
step 1.3, pair IiFrame imagefi(i=2,3..n) And (3) performing calculation:
wherein the method comprises the steps of
Representing selection of the first sample setrdA background of the composition of the individual samples,rdis a value randomly selected from {1,2,..N }, is }>
Representing the foreground segmentation post-background segmentation threshold value calculated by the OTSU method, < >>
Segmentation threshold value representing the calculation of background differential result by OTSU method,/->
Represent the firstiFrame image is +.>
Binarization results at->
Segmentation threshold value representing the result of calculating the frame difference by OTSU method,/->
Represent the firstiFrame radius thresholdRIs used for the value of (a) and (b),αis a weighting coefficient;
such as
The following process is performed:
step 1.3.1, judging the current pixel
Whether or not it is background by calculating the current pixel +.>
The similarity degree of the current pixel and the corresponding sample set is used for judging whether the current pixel is background, and the specific calculation is as follows:
current pixel
Is a background pixel, i.e.)>
When the method is used, background updating is carried out according to the probability of 1/theta, wherein the background updating is divided into two parts, namely current sample set updating and neighborhood updating, and theta is a time sampling factor; />
First, sample set update with current pixel
Pixel value +.>
Replace its corresponding background sample set +.>
Is a randomly selected sample->
Namely +.>
；
Secondly, the neighborhood is updated, the current pixel
Randomly selecting a position current pixel in 8 neighborhoods of (1)>
In the corresponding background sample set +.>
Medium shorthand select a sample +.>
The current pixel is used for replacement, namely
。/>
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010510090.4A CN111666881B (en) | 2020-06-08 | 2020-06-08 | Giant panda pacing, bamboo eating and estrus behavior tracking analysis method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010510090.4A CN111666881B (en) | 2020-06-08 | 2020-06-08 | Giant panda pacing, bamboo eating and estrus behavior tracking analysis method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111666881A CN111666881A (en) | 2020-09-15 |
| CN111666881B true CN111666881B (en) | 2023-04-28 |
Family
ID=72386859
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010510090.4A Active CN111666881B (en) | 2020-06-08 | 2020-06-08 | Giant panda pacing, bamboo eating and estrus behavior tracking analysis method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111666881B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112016527B (en) * | 2020-10-19 | 2022-02-01 | 成都大熊猫繁育研究基地 | Panda behavior recognition method, system, terminal and medium based on deep learning |
| CN113963298A (en) * | 2021-10-25 | 2022-01-21 | 东北林业大学 | Wild animal identification tracking and behavior detection system, method, equipment and storage medium based on computer vision |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108230364A (en) * | 2018-01-12 | 2018-06-29 | 东南大学 | A kind of foreground object motion state analysis method based on neural network |
| CN109377517A (en) * | 2018-10-18 | 2019-02-22 | 哈尔滨工程大学 | A kind of animal individual identifying system based on video frequency tracking technology |
| CN110060278A (en) * | 2019-04-22 | 2019-07-26 | 新疆大学 | The detection method and device of moving target based on background subtraction |
Family Cites Families (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP2015252B1 (en) * | 2007-07-08 | 2010-02-17 | Université de Liège | Visual background extractor |
| EP2834722A1 (en) * | 2012-04-02 | 2015-02-11 | Thomson Licensing | Method for calibration free gaze tracking using low cost camera |
| CN103125443B (en) * | 2013-03-06 | 2014-05-07 | 成都大熊猫繁育研究基地 | Method for timely releasing panda pairs to allow natural mating |
| CN104331905A (en) * | 2014-10-31 | 2015-02-04 | 浙江大学 | Surveillance video abstraction extraction method based on moving object detection |
| EP4250144A3 (en) * | 2015-10-14 | 2024-01-17 | President And Fellows Of Harvard College | Automatically classifying animal behavior |
| CN105741319B (en) * | 2016-01-22 | 2018-05-08 | 浙江工业大学 | Improvement visual background extracting method based on blindly more new strategy and foreground model |
| CN106571014A (en) * | 2016-10-24 | 2017-04-19 | 上海伟赛智能科技有限公司 | Method for identifying abnormal motion in video and system thereof |
| US10706558B2 (en) * | 2017-08-04 | 2020-07-07 | Université de Liège | Foreground and background detection method |
| CN108198207A (en) * | 2017-12-22 | 2018-06-22 | 湖南源信光电科技股份有限公司 | Multiple mobile object tracking based on improved Vibe models and BP neural network |
| CN108346160A (en) * | 2017-12-22 | 2018-07-31 | 湖南源信光电科技股份有限公司 | The multiple mobile object tracking combined based on disparity map Background difference and Meanshift |
| CN109614928B (en) * | 2018-12-07 | 2020-03-10 | 成都大熊猫繁育研究基地 | Panda face recognition method based on limited training data |
| CN109670440B (en) * | 2018-12-14 | 2023-08-08 | 央视国际网络无锡有限公司 | Identification method and device for big bear cat face |
| CN111144236B (en) * | 2019-12-10 | 2024-04-26 | 华南师范大学 | Cockroach mating behavior analysis method, system and storage medium |
| CN110931024B (en) * | 2020-02-18 | 2020-05-22 | 成都大熊猫繁育研究基地 | Audio-based prediction method and system for natural mating result of captive pandas |
-
2020
- 2020-06-08 CN CN202010510090.4A patent/CN111666881B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108230364A (en) * | 2018-01-12 | 2018-06-29 | 东南大学 | A kind of foreground object motion state analysis method based on neural network |
| CN109377517A (en) * | 2018-10-18 | 2019-02-22 | 哈尔滨工程大学 | A kind of animal individual identifying system based on video frequency tracking technology |
| CN110060278A (en) * | 2019-04-22 | 2019-07-26 | 新疆大学 | The detection method and device of moving target based on background subtraction |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111666881A (en) | 2020-09-15 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106778902A (en) | Milk cow individual discrimination method based on depth convolutional neural networks | |
| CN111882560B (en) | Lung parenchyma CT image segmentation method based on weighted full convolution neural network | |
| CN111666881B (en) | Giant panda pacing, bamboo eating and estrus behavior tracking analysis method | |
| Yin et al. | Recognition of sick pig cough sounds based on convolutional neural network in field situations | |
| CN103854027A (en) | Crowd behavior identification method | |
| Hu et al. | Dual attention-guided feature pyramid network for instance segmentation of group pigs | |
| CN109902564A (en) | A kind of accident detection method based on the sparse autoencoder network of structural similarity | |
| Chen et al. | A kinetic energy model based on machine vision for recognition of aggressive behaviours among group-housed pigs | |
| Christ et al. | Segmentation of brain tumors using meta heuristic algorithms | |
| El-Henawy et al. | Cattle identification using segmentation-based fractal texture analysis and artificial neural networks | |
| Yu et al. | A recognition method of soybean leaf diseases based on an improved deep learning model | |
| CN115830490A (en) | Multi-target tracking and behavior statistical method for herd health pigs | |
| CN114092450A (en) | Real-time image segmentation method, system and device based on gastroscopy video | |
| CN114822823B (en) | Tumor fine classification system based on cloud computing and artificial intelligence fusion multi-dimensional medical data | |
| Rodriguez et al. | Automated video monitoring of unmarked and marked honey bees at the hive entrance | |
| CN112381818B (en) | Medical image identification enhancement method for subclass diseases | |
| El-Henawy et al. | A new muzzle classification model using decision tree classifier | |
| CN115830078B (en) | Multi-target pig tracking and behavior recognition method, computer equipment and storage medium | |
| CN107194352A (en) | A kind of pedestrian counting method of video monitoring, apparatus and system | |
| CN113449712B (en) | Goat face identification method based on improved Alexnet network | |
| CN116509335A (en) | Gradual evolution automatic sleep stage-dividing method | |
| Li et al. | Recognition of fine-grained sow nursing behavior based on the SlowFast and hidden Markov models | |
| Kim et al. | Automatic identification of a coughing animal using audio and video data | |
| Nasiri et al. | An automated video action recognition-based system for drinking time estimation of individual broilers | |
| Johnston et al. | Robust population single neuronal calcium signal extraction using SCOUT allows for longitudinal analysis of behavior-associated neural ensemble dynamics |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |