CN116266415A - Action evaluation method, system and device based on body building teaching training and medium - Google Patents
Action evaluation method, system and device based on body building teaching training and medium Download PDFInfo
- Publication number
- CN116266415A CN116266415A CN202111523962.1A CN202111523962A CN116266415A CN 116266415 A CN116266415 A CN 116266415A CN 202111523962 A CN202111523962 A CN 202111523962A CN 116266415 A CN116266415 A CN 116266415A
- Authority
- CN
- China
- Prior art keywords
- gesture
- standard
- user
- standard gesture
- gestures
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
- Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
- Y02P90/30—Computing systems specially adapted for manufacturing
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a motion evaluation method, a motion evaluation system, a motion evaluation device and a motion evaluation medium based on body-building teaching training, which relate to the field of body-building, acquire body-building videos, and preset standard postures in the body-building videos; acquiring a time period corresponding to the standard gesture in the body-building video; acquiring frame images of continuous moments corresponding to the exercise video in the time period; identifying a plurality of user gestures corresponding to the frame images at continuous moments in the time period; and comparing the user gestures with the standard gestures, and judging whether the user gestures and the standard gestures are of the same type. When the invention is used for comparison, the model is constructed through the convolutional neural network, and the 16 skeletal key points of the human body and the corresponding two-dimensional position coordinates are respectively identified, so that the accuracy is higher, whether the user gesture is standard or not can be quickly obtained, and the use efficiency is improved.
Description
Technical Field
The invention relates to the field of body building, in particular to a motion evaluation method, a motion evaluation system, a motion evaluation device and a motion evaluation medium based on body building teaching training.
Background
In recent years, the health literacy of the national is continuously improved, the requirement for body building exercises is also continuously increased, and the market of the body building industry is huge. Various intelligent body-building apparatuses are developed rapidly, the existing mirror body-building apparatuses are provided with various apparatuses in a machine body and are displayed and/or mirror-image-displayed by a front screen, and a user can exercise and train against displayed body-building contents, so that when the intelligent body-building apparatus is used, a body-building coach can exercise and teach a plurality of users in a video live broadcast or prerecorded mode, wherein the body-building coach generally performs unified body-building teaching according to the general body-building level of the plurality of users, such as the body-building level of a flow yoga level 2 and the like when performing body-building teaching.
In the teaching process, the actions of the user need to be monitored and evaluated to see whether the actions of the user reach standards, and currently, common human body behavior detection methods comprise myoelectricity detection, air bag sensor information acquisition, a visual image method and the like. The myoelectricity detection method is to identify the motion of a human body by utilizing a biological myoelectricity signal generated by the motion of the human body, but the user is required to wear a sensor when using the myoelectricity detection method, which is mostly used for scientific research in a specific scene and does not meet the requirement of usual body building. The method of the air bag is similar to the myoelectricity detection method, and the related sensor is worn in the movement process of the user to acquire the movement information. Thus, the methods based on the use of wearing sensors, whether myoelectric or balloon, require the user to wear the sensors and are not suitable for posture correction of the user during daily fitness exercises. Methods of visual image, such methods entail estimating gestures and actions of a user using an image captured by a user device implementation including user contour estimation, user skeleton map estimation, and the like. The main application is openpost, which trains a graphic neural network by using a large amount of human activity data and labels, so as to recognize the gestures of a human body. However, in actual use, the human body gesture recognition capability is poor, and whether the user action is standard or not cannot be detected well.
Disclosure of Invention
The invention aims to better acquire the user gesture and judge whether the user gesture meets the standard or not when the user is subjected to body building training through the body building video, so that the body building effect of the body building video is ensured.
In order to achieve the above object, the present invention provides a motion evaluation method based on fitness teaching training, comprising:
acquiring a body-building video, wherein a standard gesture is preset in the body-building video;
acquiring a time period corresponding to the standard gesture in the body-building video;
acquiring frame images of continuous moments corresponding to the exercise video in the time period;
identifying a plurality of user gestures corresponding to the frame images at continuous moments in the time period;
and comparing the user gestures with the standard gestures, and judging whether the user gestures and the standard gestures are of the same type.
For the invention, when evaluating whether the actions of the user are of the same type, not every standard gesture of a coach in the body-building video is compared with the gesture of the user, but only the preset standard gesture is compared, the gesture of the user is obtained for comparison in the time period of the standard gesture in the body-building video, and the standard gesture and the gesture of the user are both static gestures, so that the operation and the use are more convenient during the comparison.
Meanwhile, the standard gesture of the invention appears at a certain time point of the body-building video, but when the gesture of the user is recognized, the invention acquires the gesture of the user in a front-back interval of the time point, namely in a time period, and compares the gesture of the user corresponding to each frame of image in the time period with the standard gesture.
Furthermore, the method for comparing the user gestures and the standard gestures by the twin neural network model specifically comprises the following steps:
training a twin neural network model to obtain a trained standard gesture recognition model;
and inputting the user gesture and the standard gesture into a standard gesture recognition model, and judging whether the user gesture and the standard gesture are of the same type.
When the twin neural network is trained, the model training process is trained through larger samples, for example, gesture samples such as hand lifting and leg lifting are used, so that the model training learns how to extract gesture features, namely, the process of mapping from 32 dimensions to 100 dimensions.
Further, inputting the user gesture and the standard gesture into a standard gesture recognition model, and judging whether the user gesture and the standard gesture are of the same type or not, specifically comprising:
acquiring skeleton key points of standard gestures and user gestures and position coordinates corresponding to each skeleton key point;
inputting the position coordinates corresponding to each skeleton key point of the standard gesture and the user gesture into a trained standard gesture recognition model to respectively obtain an output vector V1 of the standard gesture and an output vector V2 of the user gesture;
calculating Euclidean distance between an output vector V1 of the standard gesture and an output vector V2 of the user gesture;
and judging whether the user gesture and the standard gesture are of the same type or not according to the Euclidean distance of the output vector V1 of the standard gesture and the output vector V2 of the user gesture.
Wherein, in the invention, a human body posture has 16 bone points with two-dimensional coordinates, each bone point has x and y coordinate components, then a human body posture can be abstracted into a 32-dimensional bone point vector, namely [ x1, y1, x2, y2, x3, y3, …, x16, y16]. After passing through the trained gesture recognition model, the 32-dimensional skeleton point vector is mapped into a higher-dimensional vector, and the output vector is 100 dimensions in the invention, namely, the output vector V1 of the standard gesture and the output vector V2 of the user gesture are 100 dimensions, namely [ a1, a2, a3, …, a100]. When the gesture comparison is carried out, the trained models of the standard gesture and the user gesture are respectively mapped into a 100-dimensional vector, namely V1 and V2, and the Euclidean distance between V1 and V2 is calculated.
The present invention uses a deep neural network that accepts a 32-dimensional vector, i.e., a human pose in the present invention, and then through a series of intermediate layer operations, such as nonlinear correction, full join, etc., ultimately outputs a 100-dimensional vector. This 100-dimensional vector is a highly abstract feature; finally, if the two gestures are very similar, the Euclidean distance of the two 100-dimensional vectors output by the network is very small, otherwise, the Euclidean distance is very large.
The number of nodes from input to output of each layer is 32- >64- >128- >100 respectively, namely, a vector of 32 dimensions is input, a vector of 100 dimensions is output, and an Euclidean distance calculation formula of n dimensions is mapped to 100 dimensions, namely, n=100:
and inputting the user gesture and the standard gesture into a standard gesture recognition model, and judging whether the user gesture and the standard gesture are of the same type. The method comprises the steps of inputting a user gesture and a standard gesture into a standard gesture recognition model, outputting the acquaintance scores of the user gesture and the standard gesture, obtaining the user gesture with the highest acquaintance score as a scoring result, and judging whether the user gesture and the standard gesture are of the same type or not according to the scoring result.
Preferably, the Euclidean distance threshold T is obtained based on the standard gesture recognition model, and the threshold T is used for judging whether the user gesture and the standard gesture are of the same type, wherein if the Euclidean distance output by the standard gesture recognition model is greater than or equal to the threshold T, the user gesture and the standard gesture are of the same type, and if the Euclidean distance output by the standard gesture recognition model is less than or equal to the threshold T, the user gesture and the standard gesture are of different types.
Furthermore, after training the twin network model, a threshold T of the euclidean distance is found on the test set: if the Euclidean distance of the two poses exceeds T, the two poses are not considered to be of the same type; otherwise, the same model is considered.
For each threshold T, an ROC curve can be drawn, the area under the ROC curve, called AUC, being a value of 0-1, the larger the AUC, the better the model performance. An optimal threshold T-best is found such that AUC is maximum across the test set. Colloquially, the AUC is the largest, that is, the model will judge as many gestures as possible that belong to the same class, and simultaneously will misjudge as few as possible that belong to the same class. After the optimal distance threshold T-best is obtained, a critical score, for example, 40 scores, is set according to the actual service requirement, which means that at this time, the model considers that the two gestures are just at similar and dissimilar critical points. The mapping relation is as follows: the actual distance t is within the interval of [0, T-best ], and the similarity score s is [100,40]; when the actual distance T is (T-best, infinity), the similarity score s is (40, 0).
And, the standard gesture and the user gesture each comprise 16 skeleton key points, and the 16 skeleton key points respectively correspond to a two-dimensional position coordinate. Wherein the 16 skeletal key points include head top, head bottom, neck, right shoulder, right elbow, right hand, left shoulder, left elbow, left hand, right crotch, right knee, right foot, left crotch, left knee, left foot, and patella.
Corresponding to the method in the invention, the invention also provides a motion evaluation system based on body-building teaching training, comprising:
the acquisition module is used for acquiring the exercise video, acquiring a time period corresponding to a preset standard gesture in the exercise video and acquiring frame images of continuous moments corresponding to the exercise video in the time period;
the identification module is used for identifying a plurality of user gestures corresponding to the frame images at continuous moments in the time period;
the comparison module is used for comparing the user gestures with the standard gestures to obtain a comparison result;
and the judging module is used for judging whether the user gesture and the standard gesture are of the same type according to the comparison result.
Corresponding to the method in the invention, the invention also provides an electronic device which comprises a memory, a processor and a computer program stored in the memory and capable of running on the processor, wherein the processor realizes the steps of the action evaluation method based on the body building teaching training when executing the computer program.
Corresponding to the method in the invention, the invention also provides a storage medium, and the computer readable storage medium stores a computer program which realizes the steps of the action evaluation method based on body building teaching training when being executed by a processor.
The one or more technical schemes provided by the invention have at least the following technical effects or advantages:
according to the invention, when the actions of the user are evaluated to be of the same type, each standard gesture of a coach in the body-building video is not compared with the gesture of the user, but only the preset standard gesture is compared, the gesture of the user is obtained for comparison in a time period when the standard gesture appears in the body-building video, and the standard gesture and the gesture of the user are both static gestures, so that the operation and the use are more convenient during comparison. Secondly, the preset standard gesture can be a fixed-point action, and the exercise training condition of the user can be more accurately mastered when the user action is evaluated.
In comparison, the model is built through the convolutional neural network, and the 16 skeleton key points of the human body and the corresponding two-dimensional position coordinates are respectively identified, so that the accuracy is higher, whether the user gesture is standard or not can be quickly obtained, and the use efficiency is improved.
Drawings
The accompanying drawings, which are included to provide a further understanding of embodiments of the invention and are incorporated in and constitute a part of this application, illustrate embodiments of the invention. In the drawings:
FIG. 1 is a flow chart of a method of action evaluation based on fitness teaching training;
FIG. 2 is a schematic diagram of the composition of an action evaluation system based on fitness teaching training;
FIG. 3 is a schematic diagram when two poses belong to different types;
FIG. 4 is a schematic diagram when two poses are of the same type;
fig. 5 is a ROC curve.
Detailed Description
In order that the above-recited objects, features and advantages of the present invention will be more clearly understood, a more particular description of the invention will be rendered by reference to the appended drawings and appended detailed description. In addition, the embodiments of the present invention and the features in the embodiments may be combined with each other without collision.
In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present invention, however, the present invention may be practiced in other ways than within the scope of the description, and the scope of the invention is therefore not limited to the specific embodiments disclosed below.
It will be appreciated by those skilled in the art that in the present disclosure, the terms "longitudinal," "transverse," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," "outer," etc. refer to an orientation or positional relationship based on that shown in the drawings, which is merely for convenience of description and to simplify the description, and do not indicate or imply that the apparatus or elements referred to must have a particular orientation, be constructed and operated in a particular orientation, and therefore the above terms should not be construed as limiting the present invention.
It will be understood that the terms "a" and "an" should be interpreted as referring to "at least one" or "one or more," i.e., in one embodiment, the number of elements may be one, while in another embodiment, the number of elements may be plural, and the term "a" should not be interpreted as limiting the number.
Referring to fig. 1, fig. 1 is a flow chart of a motion evaluation method based on fitness teaching training, and the invention provides a motion evaluation method based on fitness teaching training, which includes:
acquiring a body-building video, wherein a standard gesture is preset in the body-building video;
acquiring a time period corresponding to the standard gesture in the body-building video;
acquiring frame images of continuous moments corresponding to the exercise video in the time period;
identifying a plurality of user gestures corresponding to the frame images at continuous moments in the time period;
and comparing the user gestures with the standard gestures, and judging whether the user gestures and the standard gestures are of the same type. Inputting the user gesture and the standard gesture into a standard gesture recognition model, outputting the acquaintance scores of the user gesture and the standard gesture, acquiring the user gesture with the highest acquaintance score as a scoring result, and judging whether the user gesture and the standard gesture are of the same type or not through the scoring result.
Wherein, compare a plurality of user gesture and standard gesture, concretely include:
training a twin neural network model to obtain a trained standard gesture recognition model;
and inputting the user gesture and the standard gesture into a standard gesture recognition model, and judging whether the user gesture and the standard gesture are of the same type.
Inputting the user gesture and the standard gesture into a standard gesture recognition model, and judging whether the user gesture and the standard gesture are of the same type or not, wherein the method specifically comprises the following steps of:
acquiring skeleton key points of standard gestures and user gestures and position coordinates corresponding to each skeleton key point;
inputting the position coordinates corresponding to each skeleton key point of the standard gesture and the user gesture into a trained standard gesture recognition model to respectively obtain an output vector V1 of the standard gesture and an output vector V2 of the user gesture;
calculating Euclidean distance between an output vector V1 of the standard gesture and an output vector V2 of the user gesture;
and judging whether the user gesture and the standard gesture are of the same type or not according to the Euclidean distance of the output vector V1 of the standard gesture and the output vector V2 of the user gesture.
Preferably, the Euclidean distance threshold T is obtained based on the standard gesture recognition model, and the threshold T is used for judging whether the user gesture and the standard gesture are of the same type, wherein if the Euclidean distance output by the standard gesture recognition model is greater than or equal to the threshold T, the user gesture and the standard gesture are of the same type, and if the Euclidean distance output by the standard gesture recognition model is less than or equal to the threshold T, the user gesture and the standard gesture are of different types.
The standard gesture and the user gesture comprise 16 skeleton key points, and the 16 skeleton key points respectively correspond to a two-dimensional position coordinate. The 16 skeletal key points include the top of the head, the bottom of the head, the neck, the right shoulder, the right elbow, the right hand, the left shoulder, the left elbow, the left hand, the right crotch, the right knee, the right foot, the left crotch, the left knee, the left foot, and the patella.
The action evaluation method based on body building teaching training in the invention is described below with reference to specific examples:
step 1, acquiring a body-building video, wherein a standard gesture is preset in the body-building video; in the embodiment, the body-building device is a body-building mirror, and the body-building video is played on the mirror surface of the body-building mirror;
step 2, acquiring a time period corresponding to the standard gesture in the exercise video, and acquiring frame images of continuous moments corresponding to the exercise video in the time period;
step 3, identifying a plurality of user gestures corresponding to the frame images at continuous moments in the time period;
step 3.1, the body-building mirror identifies the user action in the target body-building area, and extracts the characteristics of the target body-building area to obtain the user gesture;
step 4, comparing a plurality of user gestures with the standard gestures, and judging whether the user gestures are of the same type or not;
training a twin neural network model to obtain a trained standard gesture recognition model;
step 4.2, acquiring skeleton key points of a standard gesture and a user gesture and position coordinates corresponding to each skeleton key point, wherein the standard gesture and the user gesture both comprise 16 skeleton key points, and the 16 skeleton key points respectively correspond to a two-dimensional position coordinate; the 16 bone key points comprise head top, head bottom, neck, right shoulder, right elbow, right hand, left shoulder, left elbow, left hand, right crotch, right knee, right foot, left crotch, left knee, left foot and patella;
step 4.3, inputting the position coordinates corresponding to each skeleton key point of the standard gesture and the user gesture into a trained standard gesture recognition model to respectively obtain an output vector V1 of the standard gesture and an output vector V2 of the user gesture;
calculating Euclidean distance between an output vector V1 of the standard gesture and an output vector V2 of the user gesture;
and judging whether the user gesture and the standard gesture are of the same type or not according to the Euclidean distance of the output vector V1 of the standard gesture and the output vector V2 of the user gesture.
Wherein: the neural network has 4 layers, the node number from input to output of each layer is 32- >64- >128- >100 respectively, namely, a vector of 32 dimensions is input, a vector of 100 dimensions is output, and an Euclidean distance calculation formula of n dimensions space is mapped to 100 dimensions, namely, n=100:
the standard gesture recognition model outputs two high-dimensional vectors, in this embodiment 100-dimensional vectors, and if the two gestures are of different types, as shown in fig. 3, the euclidean distance of the points where the two gestures map to the high-dimensional space will be far; conversely, if the two poses are of the same type as shown in FIG. 4, the Euclidean distance of the points where the two poses map to the high dimensional space will be very close.
Step 4.4, acquiring a Euclidean distance threshold T based on a standard gesture recognition model, wherein the threshold T is used for judging whether the gesture of the user is of the same type with the standard gesture;
if the Euclidean distance of the scoring result output by the standard gesture recognition model is greater than or equal to a threshold value T, the user gesture and the standard gesture are of the same type, and if the Euclidean distance of the scoring result output by the standard gesture recognition model is less than or equal to the threshold value T, the user gesture and the standard gesture are of different types;
and 4.5, converting the Euclidean distance into the acquaintance score of the user gesture and the standard gesture, and acquiring the user gesture with the highest acquaintance score as a scoring result.
Specifically, if the euclidean distance of the two poses exceeds a threshold T, it is not considered to be of the same type; otherwise, the same model is considered. For each threshold T, an ROC curve can be plotted, as shown in fig. 5, with the area under the ROC curve, called AUC, being a value of 0-1, the greater the AUC, the better the model performance. An optimal threshold T-best is found such that AUC is maximum across the test set. If the AUC is maximum, the model can judge as many gestures originally belonging to the same class as possible, and simultaneously, can misjudge two gestures not belonging to the same class as few as possible. After obtaining the optimal distance threshold T-best, we set a critical score, such as 40 scores, according to the actual business requirements, which means that at this time, the model considers that the two poses are just at similar and dissimilar critical points. The mapping relationship is then as follows: the actual distance t is within the interval of [0, T-best ], and the similarity score s is [100,40]; when the actual distance T is (T-best, infinity), the similarity score s is (40, 0). The threshold T is 40 in this embodiment.
Example two
Based on the embodiment 1, the action evaluation method based on fitness teaching training in the present invention is described below with reference to specific examples: in the practical use process, the user of the action evaluation method carries out exercise training action demonstration, action teaching and normal follow-up on the user through three stages, in the process, the actions of the user are identified, whether the user generates the actions in the first stage, whether the user carries out the follow-up in the second stage, whether the actions of the user reach the standard in the third stage, and the grading result is fed back to the user. When the user actions are identified and compared, the method and the device for identifying and comparing the user actions according to the first time period or the second time period corresponding to the preset first standard gesture or the second standard gesture do not need to identify and compare the whole video, the comparison effect is better, the result obtaining speed is faster, the body-building effect of the user is effectively ensured, the body-building condition of the user can be obtained at each stage, the use efficiency of body-building is improved, and the method and the device for identifying and comparing the user actions are more convenient to use for a long time. The method specifically comprises the following steps:
step 1, acquiring a first exercise video of exercise, wherein in the embodiment, the exercise device is an exercise mirror, and the first exercise video is played on the mirror surface of the exercise mirror;
step 2, acquiring a first user gesture according to a first body-building video, and judging whether the user performs body-building training or not;
step 2.1, the user performs follow-up in a target exercise area of the exercise mirror according to the first exercise video;
step 2.2, presetting a first standard posture in a first body building video;
step 2.3, obtaining a second time period when the first standard gesture appears in the first body-building video;
step 2.4, in a second time period, the body-building mirror recognizes the actions of the user in the target body-building area, and performs feature extraction on the target body-building area, wherein the feature extraction comprises 16 skeleton key points, and the 16 skeleton key points respectively correspond to a two-dimensional position coordinate; the 16 skeletal key points comprise a top of head, a bottom of head, a neck, a right shoulder, a right elbow, a right hand, a left shoulder, a left elbow, a left hand, a right crotch, a right knee, a right foot, a left crotch, a left knee, a left foot and a patella, so as to obtain a first user gesture;
step 2.5, if the first user gesture obtained in the second time period has different actions, indicating that the user is doing the following, namely, the user is doing the body-building training; if the first user gesture is not acquired or is not generated in the second time period, the user does not follow, namely the user does not perform body building training. In step 2, the user's actions are recognized during a second period of time, but no scoring is performed in the process.
Step 3, acquiring a second user gesture according to the second exercise video, scoring the second user gesture, and judging whether the user follows the second exercise video to perform exercise training according to the scoring result;
step 3.1, playing a second exercise video on the mirror surface of the exercise mirror;
step 3.2, presetting a second standard posture according to a second body-building video;
step 3.3, obtaining a first time period when the second standard gesture appears in the second body-building video;
step 3.4, in the first time period, obtaining a video clip corresponding to the second body-building video in the first time period, and carrying out framing treatment on the video clip to obtain frame images of a plurality of continuous moments corresponding to the video clip;
step 3.5, in a first time period, the body-building mirror recognizes the actions of a user in a target body-building area, and performs feature extraction on the target body-building area, wherein the feature extraction comprises 16 skeleton key points, and the 16 skeleton key points respectively correspond to a two-dimensional position coordinate; the 16 skeletal key points comprise a head top, a head bottom, a neck, a right shoulder, a right elbow, a right hand, a left shoulder, a left elbow, a left hand, a right crotch, a right knee, a right foot, a left crotch, a left knee, a left foot and a patella, so as to obtain a plurality of second user gestures corresponding to the frame images one by one;
step 3.6, comparing a plurality of corresponding frame images with a plurality of second user gestures to obtain an acquaintance score of each second user gesture based on the corresponding frame images;
step 3.61 training the twin neural network model to obtain a trained standard gesture recognition model
Step 3.62, acquiring skeleton key points of a second standard gesture and a second user gesture and position coordinates corresponding to each skeleton key point, wherein the second standard gesture and the second user gesture both comprise 16 skeleton key points, and the 16 skeleton key points respectively correspond to a two-dimensional position coordinate; the 16 bone key points comprise head top, head bottom, neck, right shoulder, right elbow, right hand, left shoulder, left elbow, left hand, right crotch, right knee, right foot, left crotch, left knee, left foot and patella; inputting the position coordinates corresponding to each skeleton key point of the second standard gesture and the second user gesture into a trained standard gesture recognition model to respectively obtain an output vector V1 of the second standard gesture and an output vector V2 of the second user gesture;
calculating Euclidean distance between the output vector V1 of the second standard gesture and the output vector V2 of the second user gesture;
and judging whether the second user gesture and the second standard gesture are of the same type or not according to the Euclidean distance of the output vector V1 of the second standard gesture and the output vector V2 of the second user gesture.
Wherein: the neural network has 4 layers, the node number from input to output of each layer is 32- >64- >128- >100 respectively, namely, a vector of 32 dimensions is input, a vector of 100 dimensions is output, and an Euclidean distance calculation formula of n dimensions space is mapped to 100 dimensions, namely, n=100:
step 3.7, converting the Euclidean distance into a degree of acquaintance score of the user gesture and the standard gesture, and acquiring the user gesture with the highest degree of acquaintance score as a scoring result;
step 3.8, obtaining the second user gesture with the highest acquaintance score as a scoring result; if the scoring result is greater than or equal to the scoring threshold, the second standard gesture and the second user gesture are of the same type, and the user performs body building training; if the scoring result is less than or equal to the scoring threshold, the second standard gesture and the second user gesture are not of the same type, and the user does not perform body building training. Acquiring a Euclidean distance threshold T based on a standard gesture recognition model, wherein the threshold T is used for judging whether the gesture of a user is of the same type with the standard gesture; if the Euclidean distance of the scoring result output by the standard gesture recognition model is greater than or equal to a threshold value T, the user gesture and the standard gesture are of the same type, and if the Euclidean distance of the scoring result output by the standard gesture recognition model is less than or equal to the threshold value T, the user gesture and the standard gesture are of different types.
Step 4, acquiring a first user gesture according to the first body-building video, scoring the first user gesture, and feeding back a scoring result to the user;
step 4.1, the user performs follow-up in a target exercise area of the exercise mirror according to the first exercise video;
step 4.2, presetting a first standard posture in a first body building video;
step 4.3, obtaining a second time period when the first standard gesture appears in the first body-building video;
step 4.4, in a second time period, the body-building mirror recognizes the actions of the user in the target body-building area, and performs feature extraction on the target body-building area, wherein the feature extraction comprises 16 skeleton key points, and the 16 skeleton key points respectively correspond to a two-dimensional position coordinate; the 16 skeletal key points comprise a top of head, a bottom of head, a neck, a right shoulder, a right elbow, a right hand, a left shoulder, a left elbow, a left hand, a right crotch, a right knee, a right foot, a left crotch, a left knee, a left foot and a patella, so as to obtain a first user gesture;
step 4.5, comparing the first standard gesture with a plurality of first user gestures to obtain an acquaintance score of each first user gesture based on the first standard gesture;
step 4.51; inputting the first standard gesture, the skeleton key points of the first user gesture and the position coordinates corresponding to each skeleton key point into a standard gesture recognition model, and outputting the Euclidean distance between the first standard gesture and the first user gesture;
step 4.52, converting the Euclidean distance into a score of the degree of acquaintance;
step 4.53, obtaining the first user gesture with the highest acquaintance score as a scoring result; if the scoring result is greater than or equal to the scoring threshold, the first standard gesture and the first user gesture are of the same type, and the user action reaches the standard; if the scoring result is smaller than or equal to the scoring threshold, the first standard gesture and the first user gesture are not of the same type, and the user action does not reach the standard. Acquiring a Euclidean distance threshold T based on a standard gesture recognition model, wherein the threshold T is used for judging whether the gesture of a user is of the same type with the standard gesture; if the Euclidean distance of the scoring result output by the standard gesture recognition model is greater than or equal to a threshold value T, the user gesture and the standard gesture are of the same type, and if the Euclidean distance of the scoring result output by the standard gesture recognition model is less than or equal to the threshold value T, the user gesture and the standard gesture are of different types.
Step 4.54, acquiring the first user gesture with the highest acquaintance score as a scoring result, and feeding back the scoring result to the user;
if the scoring result is greater than or equal to the scoring threshold, the first standard gesture and the first user gesture are of the same type, and the user action reaches the standard; if the scoring result is smaller than or equal to the scoring threshold, the first standard gesture and the first user gesture are not of the same type, and the user action does not reach the standard. The scoring threshold is 40 in this embodiment.
In this embodiment, if the first standard gesture occurs in the exercise video at 10000 ms. Since the user follows the video exercise and his action is more than the course, we will set intervals near 10000ms, such as the first 800ms and the last 200ms, that is, the time intervals [10000-800,10000+200] are in the interval with the total duration of 1 second, each frame calculates the similarity between the first standard gesture and the first user gesture, and then outputs the score with high similarity in the interval as the final score.
In this embodiment, if the second standard pose appears in the exercise video at 10000 ms. Since the user follows the video exercise and his action is more than the course, we will set intervals near 10000ms, such as the first 800ms and the last 200ms, that is, the time intervals [10000-800,10000+200] are in the interval with the total duration of 1 second, calculate the similarity between each frame and the gesture of the first user, and then output the score with high similarity pair in the interval as the final score.
In a specific calculation, the first standard posture and the second standard posture are both static postures, and are not one continuous action. The specific method of comparison is to train a twin network structure model based on a convolutional neural network, which accepts two poses and maps the two poses to one point in a high-dimensional space respectively. In this embodiment, the second exercise video may be slow motion playing of the first exercise video, or may be motion decomposition of the first exercise video.
Example III
Referring to fig. 2, fig. 2 is a schematic diagram of a motion evaluation system based on fitness teaching training, and in a second embodiment of the present invention, a motion evaluation system based on fitness teaching training is provided, and on the basis of embodiment 2, the system includes:
the acquisition module is used for acquiring the exercise video, acquiring a time period corresponding to a preset standard gesture in the exercise video and acquiring frame images of continuous moments corresponding to the exercise video in the time period;
the identification module is used for identifying a plurality of user gestures corresponding to the frame images at continuous moments in the time period;
the comparison module is used for comparing the user gestures with the standard gestures to obtain a comparison result;
and the judging module is used for judging whether the user gesture and the standard gesture are of the same type according to the comparison result.
Example IV
An electronic device according to a fourth embodiment of the present invention includes a memory, a processor, and a computer program stored in the memory and executable on the processor, where the processor implements the steps of the exercise teaching training-based action evaluation method when executing the computer program.
The processor may be a central processing unit, or may be other general purpose processors, digital signal processors, application specific integrated circuits, off-the-shelf programmable gate arrays or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or the like. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
The memory may be used to store the computer program and/or the module, and the processor may implement various functions of the exercise teaching training based motion assessment device of the invention by running or executing the data stored in the memory. The memory may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating system, application programs required for at least one function (such as a sound playing function, an image playing function, etc.), and the like. In addition, the memory may include high-speed random access memory, and may also include non-volatile memory, such as a hard disk, memory, plug-in hard disk, smart memory card, secure digital card, flash memory card, at least one magnetic disk storage device, flash memory device, or other volatile solid state storage device.
Example five
A fifth embodiment of the present invention provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the exercise teaching training-based action evaluation method.
The computer storage media of embodiments of the invention may take the form of any combination of one or more computer-readable media. The computer readable medium may be a computer readable signal medium or a computer readable storage medium. The computer readable storage medium can be, for example, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples (a non-exhaustive list) of the computer-readable storage medium would include the following: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ReadOnlyMemory, ROM), an erasable programmable read-only memory ((ErasableProgrammableReadOnlyMemory, EPROM) or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In this document, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.
While preferred embodiments of the present invention have been described, additional variations and modifications in those embodiments may occur to those skilled in the art once they learn of the basic inventive concepts. It is therefore intended that the following claims be interpreted as including the preferred embodiments and all such alterations and modifications as fall within the scope of the invention.
It will be apparent to those skilled in the art that various modifications and variations can be made to the present invention without departing from the spirit or scope of the invention. Thus, it is intended that the present invention also include such modifications and alterations insofar as they come within the scope of the appended claims or the equivalents thereof.
Claims (10)
1. The action evaluation method based on body building teaching training is characterized by comprising the following steps:
acquiring a body-building video, wherein a standard gesture is preset in the body-building video;
acquiring a time period corresponding to the standard gesture in the body-building video;
acquiring frame images of continuous moments corresponding to the exercise video in the time period;
identifying a plurality of user gestures corresponding to the frame images at continuous moments in the time period;
and comparing the user gestures with the standard gestures, and judging whether the user gestures and the standard gestures are of the same type.
2. The exercise teaching training-based action evaluation method according to claim 1, wherein comparing a plurality of user gestures with a standard gesture specifically comprises:
training a twin neural network model to obtain a trained standard gesture recognition model;
and inputting the user gesture and the standard gesture into a standard gesture recognition model, and judging whether the user gesture and the standard gesture are of the same type.
3. The exercise teaching training-based action evaluation method according to claim 2, wherein inputting the user gesture and the standard gesture into the standard gesture recognition model, judging whether the user gesture and the standard gesture are of the same type, specifically comprises:
acquiring skeleton key points of standard gestures and user gestures and position coordinates corresponding to each skeleton key point;
inputting the position coordinates corresponding to each skeleton key point of the standard gesture and the user gesture into a trained standard gesture recognition model to respectively obtain an output vector V1 of the standard gesture and an output vector V2 of the user gesture;
calculating Euclidean distance between an output vector V1 of the standard gesture and an output vector V2 of the user gesture;
and judging whether the user gesture and the standard gesture are of the same type or not according to the Euclidean distance of the output vector V1 of the standard gesture and the output vector V2 of the user gesture.
4. The exercise teaching training-based action evaluation method according to claim 3, wherein the euclidean distance threshold T is obtained based on a standard gesture recognition model, the threshold T is used for judging whether the user gesture and the standard gesture are of the same type, wherein if the euclidean distance output by the standard gesture recognition model is less than or equal to the threshold T, the user gesture and the standard gesture are of the same type, and if the euclidean distance output by the standard gesture recognition model is greater than the threshold T, the user gesture and the standard gesture are of different types.
5. A method of motion assessment based on fitness teaching training according to claim 3, wherein the standard pose and the user pose each comprise 16 skeletal keypoints, the 16 skeletal keypoints corresponding to a two-dimensional position coordinate.
6. The exercise teaching training based motion evaluation method according to claim 5, wherein the 16 skeletal keypoints comprise a top of head, a bottom of head, a neck, a right shoulder, a right elbow, a right hand, a left shoulder, a left elbow, a left hand, a right crotch, a right knee, a right foot, a left crotch, a left knee, a left foot, a patella.
7. The exercise teaching training-based action evaluation method according to claim 3, wherein the user gesture and the standard gesture are input into the standard gesture recognition model, the degree of acquaintance score of the user gesture and the standard gesture is output, the user gesture with the highest degree of acquaintance score is obtained as a scoring result, and whether the user gesture is of the same type as the standard gesture is judged through the scoring result.
8. Action evaluation system based on body-building teaching training, its characterized in that includes:
the acquisition module is used for acquiring the exercise video, acquiring a time period corresponding to a preset standard gesture in the exercise video and acquiring frame images of continuous moments corresponding to the exercise video in the time period;
the identification module is used for identifying a plurality of user gestures corresponding to the frame images at continuous moments in the time period;
the comparison module is used for comparing the user gestures with the standard gestures to obtain a comparison result;
and the judging module is used for judging whether the user gesture and the standard gesture are of the same type according to the comparison result.
9. An electronic device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that the processor, when executing the computer program, implements the steps of the exercise teaching training based action evaluation method according to any of claims 1-7.
10. A storage medium storing a computer program, characterized in that the computer program when executed by a processor implements the steps of the exercise teaching training based motion assessment method according to any one of claims 1-7.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111523962.1A CN116266415A (en) | 2021-12-14 | 2021-12-14 | Action evaluation method, system and device based on body building teaching training and medium |
| PCT/CN2022/070026 WO2023108842A1 (en) | 2021-12-14 | 2022-01-04 | Motion evaluation method and system based on fitness teaching training |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111523962.1A CN116266415A (en) | 2021-12-14 | 2021-12-14 | Action evaluation method, system and device based on body building teaching training and medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116266415A true CN116266415A (en) | 2023-06-20 |
Family
ID=86742862
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111523962.1A Pending CN116266415A (en) | 2021-12-14 | 2021-12-14 | Action evaluation method, system and device based on body building teaching training and medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116266415A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117198529A (en) * | 2023-08-31 | 2023-12-08 | 深圳市恒安特斯网络科技有限公司 | Moving image processing method, moving image processing device and readable storage medium |
| CN117438040A (en) * | 2023-12-22 | 2024-01-23 | 亿慧云智能科技（深圳）股份有限公司 | Exercise course self-adaptive configuration method, device, equipment and storage medium |
-
2021
- 2021-12-14 CN CN202111523962.1A patent/CN116266415A/en active Pending
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117198529A (en) * | 2023-08-31 | 2023-12-08 | 深圳市恒安特斯网络科技有限公司 | Moving image processing method, moving image processing device and readable storage medium |
| CN117438040A (en) * | 2023-12-22 | 2024-01-23 | 亿慧云智能科技（深圳）股份有限公司 | Exercise course self-adaptive configuration method, device, equipment and storage medium |
| CN117438040B (en) * | 2023-12-22 | 2024-04-05 | 亿慧云智能科技（深圳）股份有限公司 | Exercise course self-adaptive configuration method, device, equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108256433B (en) | Motion attitude assessment method and system | |
| WO2021051579A1 (en) | Body pose recognition method, system, and apparatus, and storage medium | |
| CN112926423B (en) | Pinch gesture detection and recognition method, device and system | |
| CN106648078B (en) | Multi-mode interaction method and system applied to intelligent robot | |
| CN106874826A (en) | Face key point-tracking method and device | |
| Min et al. | Support vector machine approach to fall recognition based on simplified expression of human skeleton action and fast detection of start key frame using torso angle | |
| CN116266415A (en) | Action evaluation method, system and device based on body building teaching training and medium | |
| CN108875586B (en) | Functional limb rehabilitation training detection method based on depth image and skeleton data multi-feature fusion | |
| WO2017161734A1 (en) | Correction of human body movements via television and motion-sensing accessory and system | |
| WO2023108842A1 (en) | Motion evaluation method and system based on fitness teaching training | |
| CN104794446A (en) | Human body action recognition method and system based on synthetic descriptors | |
| CN113269013A (en) | Object behavior analysis method, information display method and electronic equipment | |
| Hachaj et al. | Human actions recognition on multimedia hardware using angle-based and coordinate-based features and multivariate continuous hidden Markov model classifier | |
| US20220148296A1 (en) | Method and system for symmetric recognition of handed activities | |
| CN111274854B (en) | Human body action recognition method and vision enhancement processing system | |
| CN113392744A (en) | Dance motion aesthetic feeling confirmation method and device, electronic equipment and storage medium | |
| CN116262171A (en) | Body-building training method, system and device based on body-building device and medium | |
| Ye et al. | Human motion recognition using three-dimensional skeleton model based on RGBD vision system | |
| Haider et al. | A Hand Gesture Recognition based Communication System for Mute people | |
| CN111507555A (en) | Human body state detection method, classroom teaching quality evaluation method and related device | |
| CN112580526A (en) | Student classroom behavior identification system based on video monitoring | |
| Xie | Intelligent Analysis Method of Sports Training Posture Based on Artificial Intelligence | |
| Padmanand et al. | Malaysian Sign Language Recognition Using 3D Hand Pose Estimation | |
| Patil et al. | Intensity Based distinctive feature extraction and matching using Scale invariant feature transform for Indian Sign Language | |
| Ascenso | Development of a non-invasive motion capture system for swimming biomechanics |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |