CN116469174A - Deep learning sitting posture measurement and detection method based on monocular camera - Google Patents

Deep learning sitting posture measurement and detection method based on monocular camera Download PDF

Info

Publication number
CN116469174A
CN116469174A CN202310598869.XA CN202310598869A CN116469174A CN 116469174 A CN116469174 A CN 116469174A CN 202310598869 A CN202310598869 A CN 202310598869A CN 116469174 A CN116469174 A CN 116469174A
Authority
CN
China
Prior art keywords
sitting posture
posture
deep learning
person
measured
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202310598869.XA
Other languages
Chinese (zh)
Inventor
薛昭
郭玉石
周晓军
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Nanjing Xingchen Micro Vision Optoelectronic Technology Co ltd
Original Assignee
Nanjing Xingchen Micro Vision Optoelectronic Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Nanjing Xingchen Micro Vision Optoelectronic Technology Co ltd filed Critical Nanjing Xingchen Micro Vision Optoelectronic Technology Co ltd
Priority to CN202310598869.XA priority Critical patent/CN116469174A/en
Publication of CN116469174A publication Critical patent/CN116469174A/en
Pending legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/20—Movements or behaviour, e.g. gesture recognition
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0464—Convolutional networks [CNN, ConvNet]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/084—Backpropagation, e.g. using gradient descent
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/22—Image preprocessing by selection of a specific region containing or referencing a pattern; Locating or processing of specific regions to guide the detection or recognition
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
    • G06V10/75—Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/161—Detection; Localisation; Normalisation
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
    • Y02T10/00—Road transport of goods or passengers
    • Y02T10/10—Internal combustion engine [ICE] based vehicles
    • Y02T10/40—Engine management systems

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Multimedia (AREA)
  • Evolutionary Computation (AREA)
  • Software Systems (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Artificial Intelligence (AREA)
  • Computing Systems (AREA)
  • Medical Informatics (AREA)
  • Databases & Information Systems (AREA)
  • Human Computer Interaction (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Data Mining & Analysis (AREA)
  • Molecular Biology (AREA)
  • General Engineering & Computer Science (AREA)
  • Mathematical Physics (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • Social Psychology (AREA)
  • Psychiatry (AREA)
  • Measurement Of The Respiration, Hearing Ability, Form, And Blood Characteristics Of Living Organisms (AREA)

Abstract

The invention discloses a deep learning sitting posture measurement and detection method based on a monocular camera, which relates to the technical field of sitting posture detection and is characterized by comprising the following steps of: a monocular camera is used for acquiring a sitting posture RGB image of the whole body of a measured person; converting the sitting posture RGB image into a sitting posture depth image, and extracting position information of key points in the sitting posture depth image, wherein the position information of the key points comprises human body position information and human face position information; the human body position information and the human face position information are constructed to form a point cloud data set to be matched, and the point cloud data set to be matched and the reference point cloud data set are matched to calculate the posture difference of the measured person.

Description

Deep learning sitting posture measurement and detection method based on monocular camera
Technical Field
The invention relates to the technical field of sitting posture detection, in particular to a deep learning sitting posture measurement and detection method based on a monocular camera.
Background
Now, as learning and working sit up for longer and longer periods of time. It is counted that people sit up for more than 8 hours on average every day today, and in the case of sitting up for a long time, poor sitting postures can seriously affect the physical health of the human body, such as vision attenuation, spinal development, cervical spondylosis and the like, because it is difficult for people to keep good sitting postures all the time. Therefore, in order to promote people to correct bad sitting postures, develop good sitting postures, reduce the probability of suffering from diseases such as myopia and cervical vertebra diseases, etc., it is very necessary to perform sitting posture detection.
However, the existing deep learning sitting posture measurement and detection method based on the monocular camera only measures the sitting posture of the measured person and cannot evaluate the sitting posture of the measured person, so that whether the sitting posture of the measured person is correct or not cannot be well reminded.
Disclosure of Invention
Aiming at the defects existing in the prior art, the invention aims to provide a deep learning sitting posture measurement and detection method based on a monocular camera.
In order to achieve the above purpose, the present invention provides the following technical solutions:
a deep learning sitting posture measurement and detection method based on a monocular camera comprises the following steps:
and acquiring a sitting posture RGB image of the whole body of the measured person through a monocular camera.
Converting the sitting posture RGB image into a sitting posture depth image, and extracting position information of key points in the sitting posture depth image, wherein the position information of the key points comprises human body position information and human face position information.
Constructing human body position information and human face position information to form a point cloud data set to be matched, and carrying out matching calculation on the point cloud data set to be matched and a reference point cloud data set to obtain actual posture information; the actual pose information comprises human body actual pose information and human face actual pose information.
And evaluating the sitting posture of the measured person in an evaluation model according to the actual posture information.
Preferably, the sitting posture RGB image of the subject includes the head, shoulders, chest, abdomen, knees and feet of the subject.
The invention can completely see the head, shoulders, chest, abdomen, knees and feet of the measured person through the camera. In practical application, the camera shoots the sitting posture RGB image of the whole body of the measured person with 640x480 resolution ratio, and converts the sitting posture RGB image into a sitting posture depth image. The present invention predicts depth information (i.e., distance from camera) from a single RGB image using a method based on a monocular depth estimation technique. Specifically, the invention trains sitting posture RGB images by using ResNet-50 network, thereby predicting the depth value of each pixel point, and the depth value is used for accurately measuring the human body posture. The invention uses the NYUv2 data set for training, and the data set comprises more than 14 ten thousand depth images and RGB images corresponding to the depth images, thereby improving the accuracy of depth estimation.
Preferably, the face position information includes eye position information, nose position information, and mouth position information.
According to the invention, the position information of 25 human body key points can be extracted by using an attitude estimation algorithm based on OpenPose. In the training stage, the invention uses the COCO data set for training, and the data set comprises more than 20 tens of thousands of images of human body gestures, thereby improving the accuracy of gesture estimation. Specifically, the invention uses a stacked Hourgass network to estimate the gesture, and the network comprises 8 Hourgass modules, so that each key point of the human body can be extracted rapidly and accurately.
The invention also comprises the detection of the key points of the human face. In the aspect of face key point detection, the invention adopts a PFLD (Parallel Facial Landmark Detection) network. The PFLD network adopts a parallel feature extraction strategy and a hierarchical regression method, 68 key points on the face including eyes, nose, mouth and other parts can be detected at the same time, so that the detection precision and speed are further improved. The invention can rapidly and accurately extract the position information of key points of human body and human face, and provides powerful support for analyzing and analyzing sitting postures of human body.
According to the invention, the ICP algorithm in the PCL (Point Cloud Library) library is used for carrying out point cloud matching on the extracted key point information, so that accurate measurement on the human body and human face gestures is realized. The ICP algorithm is an iterative point cloud matching algorithm, which can match one point cloud data set to another point cloud data set, and calculate a transformation relationship between the two, including posture parameters such as a rotation matrix and a translation vector. In the invention, RGB image is converted into depth image, thus obtaining three-dimensional position information of key points of human body and human face. The location information of these key points constitutes a point cloud data set, called the point cloud to be matched. Meanwhile, in order to obtain real pose information of the human body and the human face, a reference point cloud data set needs to be provided, and point clouds in the point cloud data set represent three-dimensional position information of the human body and the human face under standard pose. By matching the point cloud to be matched with the reference point cloud, the gesture difference between the point cloud to be matched and the reference point cloud can be calculated, and further the actual gesture information of the human body and the human face is obtained. Therefore, in the invention, the ICP algorithm is used for matching the point cloud to be matched with the reference point cloud to obtain three-dimensional posture information of the human body and the human face, including angle, position, posture and other information, thereby providing accurate basic data for subsequent posture analysis, motion control and other applications.
Preferably, the specific steps of establishing the evaluation model are as follows:
collecting sitting posture data: and collecting sitting posture data under different sitting postures, wherein the sitting posture data comprise position data of key points and corresponding point sitting posture labels.
And (3) data processing: and performing data cleaning, denoising and alignment processing on the position data of the key points to obtain key point position processing data, and dividing the key point position processing data into a training set, a verification set and a test set for training and evaluating the deep learning model.
Establishing a deep learning model: and constructing a deep learning model according to the key point position processing data.
Model training: training the deep learning model by using the training set, and iteratively optimizing the deep learning model by using a back propagation algorithm and an optimizer.
Model evaluation: and evaluating the deep learning model by using the verification set to obtain an evaluation result, and adjusting the super-parameters and the structure of the deep learning model according to the evaluation result.
Model application: and testing the trained deep learning model by using the test set, and evaluating the sitting posture of the measured person in the evaluation model according to the actual posture information.
Preferably, the sitting posture of the measured person is evaluated in an evaluation model according to the actual posture information, specifically:
the sitting posture of the measured person is evaluated as a correct posture or an incorrect posture.
Preferably, the specific types of the error gestures are:
the head of the person to be measured is inclined downwards and the eyes look downwards.
The head of the person to be measured is inclined to one side.
The face is held, and the measured person holds the chin or face by hand.
Neck forward tilting: the neck of the person to be measured is inclined forward.
Body forward tilting: the body of the measured person is tilted forward.
Preferably, when the sitting posture of the subject is evaluated as the wrong posture, the wrong posture is fed back to the subject.
Preferably, the actual posture information of the measured person is recorded and analyzed to generate a chart.
Preferably, the error posture of the measured person is analyzed to generate a corresponding chart, and the type and the frequency of the error posture of the measured person are judged.
Real-time feedback will be provided for the wrong posture and an alarm will be given to the measured person according to the classification of the wrong posture. Specifically, we will use sound, vibration, etc. to give a reminder to the measured person to draw the attention of the measured person. At the same time, we will also provide correction advice to the measurands, helping them improve sitting posture. Correction advice will be based on the type of missitting of the person being measured, including body adjustments, muscle relaxation, stretching movements, etc.
The sitting posture data analysis function will include the following:
sitting habit analysis: the historical sitting posture records of the measured person are analyzed, and corresponding charts and statistical data are generated to help the measured person know the sitting posture habit of the measured person. For example, we will analyze the sitting time, sitting angle, sitting level, etc. of the person being measured and visualize these data to help the person being measured understand his or her sitting condition better.
Missitting posture analysis: we will analyze the type of missitting of the person being measured and generate corresponding charts and statistics to help the person being measured understand his own missitting type and frequency. For example, we will analyze the frequency of occurrence of the types of missitting such as low head, askew head, face, excessive neck forward tilt, excessive body forward tilt, etc. of the person being measured and visualize these data to help the person being measured to better understand his/her missitting condition.
Improvement advice: based on the historical sitting posture record and the missitting posture analysis of the measured person, we will provide personalized improvement suggestions. For example, for a person to be measured who frequently has a low head, we will suggest that they adjust the height of the computer or use the headrest; for the measurands who frequently have a askew head, we will suggest that they adjust the seat height or use side supports; for the measured person who frequently has excessive anteversion, we will suggest that they use lumbar support or do lumbar stretching exercises, etc.
In summary, our sitting posture detector will not just be a simple posture monitoring device, but an all-around sitting posture assessment and feedback system. By performing deep learning analysis on the posture and key point information of the measured person, we will be able to assess the sitting posture of the measured person in real time and provide personalized real-time feedback and improvement advice. By recording and analyzing the historical sitting posture data of the measured person, the measured person is helped to know the sitting posture habit and improvement condition of the measured person, and a more scientific and effective sitting posture improvement scheme is provided for the measured person.
Compared with the prior art, the invention has the following beneficial effects:
in the invention, the sitting posture of the measured person is estimated in an estimation model according to the actual posture information, real-time feedback is provided for the wrong posture, and an alarm is sent to the measured person according to the classification of the wrong sitting posture. Specifically, we will use sound, vibration, etc. to give a reminder to the measured person to draw the attention of the measured person. At the same time, we will also provide correction advice to the measurands, helping them improve sitting posture. Correction advice will be based on the type of missitting of the person being measured, including body adjustments, muscle relaxation, stretching movements, etc.
Drawings
Fig. 1 is a schematic flow chart of a method for measuring and detecting a deep learning sitting posture based on a monocular camera;
fig. 2 is a schematic flow chart of an evaluation model establishment in a monocular camera-based deep learning sitting posture measurement and detection method;
fig. 3 is a schematic diagram of error gesture types in a method for measuring and detecting a deep learning sitting gesture based on a monocular camera.
Detailed Description
Referring to fig. 1 to 3.
Embodiment one further describes a deep learning sitting posture measurement and detection method based on a monocular camera.
A deep learning sitting posture measurement and detection method based on a monocular camera comprises the following steps:
and acquiring a sitting posture RGB image of the whole body of the measured person through a monocular camera.
Converting the sitting posture RGB image into a sitting posture depth image, and extracting position information of key points in the sitting posture depth image, wherein the position information of the key points comprises human body position information and human face position information.
Constructing human body position information and human face position information to form a point cloud data set to be matched, and carrying out matching calculation on the point cloud data set to be matched and a reference point cloud data set to obtain actual posture information; the actual pose information includes human body actual pose information and human face actual pose information.
And evaluating the sitting posture of the measured person in an evaluation model according to the actual posture information.
The sitting posture RGB image of the person to be measured includes the head, shoulders, chest, abdomen, knees and feet of the person to be measured.
The invention can completely see the head, shoulders, chest, abdomen, knees and feet of the measured person through the camera. In practical application, the camera shoots the sitting posture RGB image of the whole body of the measured person with 640x480 resolution ratio, and converts the sitting posture RGB image into a sitting posture depth image. The present invention predicts depth information (i.e., distance from camera) from a single RGB image using a method based on a monocular depth estimation technique. Specifically, the invention trains sitting posture RGB images by using ResNet-50 network, thereby predicting the depth value of each pixel point, and the depth value is used for accurately measuring the human body posture. The invention uses the NYUv2 data set for training, and the data set comprises more than 14 ten thousand depth images and RGB images corresponding to the depth images, thereby improving the accuracy of depth estimation.
The face position information includes eye position information, nose position information, and mouth position information.
According to the invention, the position information of 25 human body key points can be extracted by using an attitude estimation algorithm based on OpenPose. In the training stage, the invention uses the COCO data set for training, and the data set comprises more than 20 tens of thousands of images of human body gestures, thereby improving the accuracy of gesture estimation. Specifically, the invention uses a stacked Hourgass network to estimate the gesture, and the network comprises 8 Hourgass modules, so that each key point of the human body can be extracted rapidly and accurately.
The invention also comprises the detection of the key points of the human face. In the aspect of face key point detection, the invention adopts a PFLD (Parallel Facial Landmark Detection) network. The PFLD network adopts a parallel feature extraction strategy and a hierarchical regression method, 68 key points on the face including eyes, nose, mouth and other parts can be detected at the same time, so that the detection precision and speed are further improved. The invention can rapidly and accurately extract the position information of key points of human body and human face, and provides powerful support for analyzing and analyzing sitting postures of human body.
According to the invention, the ICP algorithm in the PCL (Point Cloud Library) library is used for carrying out point cloud matching on the extracted key point information, so that accurate measurement on the human body and human face gestures is realized. The ICP algorithm is an iterative point cloud matching algorithm, which can match one point cloud data set to another point cloud data set, and calculate a transformation relationship between the two, including posture parameters such as a rotation matrix and a translation vector. In the invention, RGB image is converted into depth image, thus obtaining three-dimensional position information of key points of human body and human face. The location information of these key points constitutes a point cloud data set, called the point cloud to be matched. Meanwhile, in order to obtain real pose information of the human body and the human face, a reference point cloud data set needs to be provided, and point clouds in the point cloud data set represent three-dimensional position information of the human body and the human face under standard pose. By matching the point cloud to be matched with the reference point cloud, the gesture difference between the point cloud to be matched and the reference point cloud can be calculated, and further the actual gesture information of the human body and the human face is obtained. Therefore, in the invention, the ICP algorithm is used for matching the point cloud to be matched with the reference point cloud to obtain three-dimensional posture information of the human body and the human face, including angle, position, posture and other information, thereby providing accurate basic data for subsequent posture analysis, motion control and other applications.
The specific steps of the establishment of the evaluation model are as follows:
collecting sitting posture data: and collecting sitting posture data under different sitting postures, wherein the sitting posture data comprise position data of key points and corresponding point sitting posture labels.
And (3) data processing: and performing data cleaning, denoising and alignment processing on the position data of the key points to obtain key point position processing data, and dividing the key point position processing data into a training set, a verification set and a test set for training and evaluating the deep learning model.
Establishing a deep learning model: and constructing a deep learning model according to the key point position processing data.
Model training: training the deep learning model by using the training set, and iteratively optimizing the deep learning model by using a back propagation algorithm and an optimizer.
Model evaluation: and evaluating the deep learning model by using the verification set to obtain an evaluation result, and adjusting the super-parameters and the structure of the deep learning model according to the evaluation result.
Model application: and testing the trained deep learning model by using the test set, and evaluating the sitting posture of the measured person in the evaluation model according to the actual posture information.
The sitting posture of the measured person is evaluated in an evaluation model according to the actual posture information, specifically:
the sitting posture of the measured person is evaluated as a correct posture or an incorrect posture.
The specific types of error gestures are:
the head of the person to be measured is tilted downward with the eyes looking downward, causing the neck and back to be subjected to excessive load.
The head of the person to be measured is tilted to one side, resulting in uneven load on the neck and shoulders.
The face is held by the hand of the person to be measured, which causes the neck and shoulders to be subjected to excessive load.
Neck forward tilting: the neck of the person to be measured is tilted forward, resulting in overload of the neck and back.
Body forward tilting: the body of the person to be measured is tilted forward, causing the waist and the back to be subjected to excessive load.
Example two
The following technical features are added on the basis of the first embodiment:
a deep learning sitting posture measurement and detection method based on a monocular camera comprises the following steps:
and acquiring a sitting posture RGB image of the whole body of the measured person through a monocular camera.
Converting the sitting posture RGB image into a sitting posture depth image, and extracting position information of key points in the sitting posture depth image, wherein the position information of the key points comprises human body position information and human face position information.
Constructing human body position information and human face position information to form a point cloud data set to be matched, and carrying out matching calculation on the point cloud data set to be matched and a reference point cloud data set to obtain actual posture information; the actual pose information comprises human body actual pose information and human face actual pose information.
And evaluating the sitting posture of the measured person in an evaluation model according to the actual posture information.
The sitting posture of the measured person is evaluated as a correct posture or an incorrect posture.
When the sitting posture of the measured person is the correct posture, the deep learning sitting posture measurement detection method based on the monocular camera informs the measured person of the result, so that the measured person can understand that the sitting posture is the correct posture, and the sitting posture can be maintained later.
When the sitting posture of the measured person is wrong, real-time feedback is provided for the wrong posture, and an alarm is given to the measured person according to the classification of the wrong sitting posture. Specifically, we will use sound, vibration, etc. to give a reminder to the measured person to draw the attention of the measured person. At the same time, we will also provide correction advice to the measurands, helping them improve sitting posture. Correction advice will be based on the type of missitting of the person being measured, including body adjustments, muscle relaxation, stretching movements, etc.
When the sitting posture of the subject is evaluated as an erroneous posture, the erroneous posture is fed back to the subject.
Recording and analyzing the actual posture information of the measured person to generate a chart.
And analyzing the error posture of the measured person to generate a corresponding chart, and judging the type and the frequency of the error posture of the measured person.
Real-time feedback will be provided for the wrong posture and an alarm will be given to the measured person according to the classification of the wrong posture. Specifically, we will use sound, vibration, etc. to give a reminder to the measured person to draw the attention of the measured person. At the same time, we will also provide correction advice to the measurands, helping them improve sitting posture. Correction advice will be based on the type of missitting of the person being measured, including body adjustments, muscle relaxation, stretching movements, etc.
The sitting posture data analysis function will include the following:
sitting habit analysis: the historical sitting posture records of the measured person are analyzed, and corresponding charts and statistical data are generated to help the measured person know the sitting posture habit of the measured person. For example, we will analyze the sitting time, sitting angle, sitting level, etc. of the person being measured and visualize these data to help the person being measured understand his or her sitting condition better.
Missitting posture analysis: we will analyze the type of missitting of the person being measured and generate corresponding charts and statistics to help the person being measured understand his own missitting type and frequency. For example, we will analyze the frequency of occurrence of the types of missitting such as low head, askew head, face, excessive neck forward tilt, excessive body forward tilt, etc. of the person being measured and visualize these data to help the person being measured to better understand his/her missitting condition.
Improvement advice: based on the historical sitting posture record and the missitting posture analysis of the measured person, we will provide personalized improvement suggestions. For example, for a person to be measured who frequently has a low head, we will suggest that they adjust the height of the computer or use the headrest; for the measurands who frequently have a askew head, we will suggest that they adjust the seat height or use side supports; for the measured person who frequently has excessive anteversion, we will suggest that they use lumbar support or do lumbar stretching exercises, etc.
In summary, our sitting posture detector will not just be a simple posture monitoring device, but an all-around sitting posture assessment and feedback system. By performing deep learning analysis on the posture and key point information of the measured person, we will be able to assess the sitting posture of the measured person in real time and provide personalized real-time feedback and improvement advice. By recording and analyzing the historical sitting posture data of the measured person, the measured person is helped to know the sitting posture habit and improvement condition of the measured person, and a more scientific and effective sitting posture improvement scheme is provided for the measured person.
The above description is only a preferred embodiment of the present invention, and the protection scope of the present invention is not limited to the above examples, and all technical solutions belonging to the concept of the present invention belong to the protection scope of the present invention. It should be noted that modifications and adaptations to the present invention may occur to one skilled in the art without departing from the principles of the present invention and are intended to be within the scope of the present invention.

Claims (9)

1. The deep learning sitting posture measurement and detection method based on the monocular camera is characterized by comprising the following steps of:
a monocular camera is used for acquiring a sitting posture RGB image of the whole body of a measured person;
converting the sitting posture RGB image into a sitting posture depth image, and extracting position information of key points in the sitting posture depth image, wherein the position information of the key points comprises human body position information and human face position information;
constructing human body position information and human face position information to form a point cloud data set to be matched, and carrying out matching calculation on the point cloud data set to be matched and a reference point cloud data set to obtain actual posture information; the actual gesture information comprises human body actual gesture information and human face actual gesture information;
and evaluating the sitting posture of the measured person in an evaluation model according to the actual posture information.
2. A monocular camera-based deep learning sitting posture measurement and detection method according to claim 1, wherein the sitting posture RGB image of the person to be measured includes the head, shoulder, chest, abdomen, knees and feet of the person to be measured.
3. A monocular camera-based deep learning sitting position measurement and detection method according to claim 2, wherein the face position information includes eye position information, nose position information and mouth position information.
4. A monocular camera-based deep learning sitting posture measurement and detection method according to claim 3, wherein the specific steps of establishing the evaluation model are as follows:
collecting sitting posture data: collecting sitting posture data under different sitting postures, including position data of key points and corresponding point sitting posture labels;
and (3) data processing: performing data cleaning, denoising and alignment processing on the position data of the key points to obtain key point position processing data, and dividing the key point position processing data into a training set, a verification set and a test set for training and evaluating a deep learning model;
establishing a deep learning model: constructing a deep learning model according to the key point position processing data;
model training: training the deep learning model by using a training set, and performing iterative optimization on the deep learning model by using a back propagation algorithm and an optimizer;
model evaluation: evaluating the deep learning model by using the verification set to obtain an evaluation result, and adjusting super parameters and structures of the deep learning model according to the evaluation result;
model application: and testing the trained deep learning model by using the test set, and evaluating the sitting posture of the measured person in the evaluation model according to the actual posture information.
5. The method for measuring and detecting the sitting posture of the person to be measured based on the monocular camera according to claim 4, wherein the sitting posture of the person to be measured is estimated in an estimation model according to the actual posture information, specifically:
the sitting posture of the measured person is evaluated as a correct posture or an incorrect posture.
6. The monocular camera-based deep learning sitting posture measurement and detection method according to claim 5, wherein the specific types of the erroneous posture are:
a lower head, the head of the measured person tilts downwards, and eyes look downwards;
a head of a person to be measured is tilted to one side;
a face support, wherein a measured person holds the chin or the face by hands;
neck forward tilting: the neck of the measured person leans forward;
body forward tilting: the body of the measured person is tilted forward.
7. The method for measuring and detecting a deep learning sitting posture based on a monocular camera according to claim 6, wherein when the sitting posture of the person to be measured is evaluated as an erroneous posture, the erroneous posture is fed back to the person to be measured.
8. The monocular camera-based deep learning sitting posture measurement and detection method according to claim 7, wherein the actual posture information of the measured person is recorded and analyzed to generate a chart.
9. The method for measuring and detecting the deep learning sitting posture based on the monocular camera according to claim 8, wherein the error posture of the measured person is analyzed to generate a corresponding chart, and the type and the frequency of the error posture of the measured person are judged.
CN202310598869.XA 2023-05-23 2023-05-23 Deep learning sitting posture measurement and detection method based on monocular camera Pending CN116469174A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202310598869.XA CN116469174A (en) 2023-05-23 2023-05-23 Deep learning sitting posture measurement and detection method based on monocular camera

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202310598869.XA CN116469174A (en) 2023-05-23 2023-05-23 Deep learning sitting posture measurement and detection method based on monocular camera

Publications (1)

Publication Number Publication Date
CN116469174A true CN116469174A (en) 2023-07-21

Family

ID=87175563

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202310598869.XA Pending CN116469174A (en) 2023-05-23 2023-05-23 Deep learning sitting posture measurement and detection method based on monocular camera

Country Status (1)

Country Link
CN (1) CN116469174A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117746505A (en) * 2023-12-21 2024-03-22 武汉星巡智能科技有限公司 Learning accompanying method and device combined with abnormal sitting posture dynamic detection and robot

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117746505A (en) * 2023-12-21 2024-03-22 武汉星巡智能科技有限公司 Learning accompanying method and device combined with abnormal sitting posture dynamic detection and robot

Similar Documents

Publication Publication Date Title
CN107169453B (en) Sitting posture detection method based on depth sensor
CN113496216B (en) Multi-angle falling high-risk identification method and system based on skeleton key points
CN112069933A (en) Skeletal muscle stress estimation method based on posture recognition and human body biomechanics
AU2017202152B2 (en) Method and system for determining postural balance of a person
CN107506706A (en) A kind of tumble detection method for human body based on three-dimensional camera
CN105740780A (en) Method and device for human face in-vivo detection
JP2022517769A (en) 3D target detection and model training methods, equipment, equipment, storage media and computer programs
CN109512188B (en) Sitting posture detection method and device and seat
US11547324B2 (en) System and method for human motion detection and tracking
WO2021068781A1 (en) Fatigue state identification method, apparatus and device
CN116469174A (en) Deep learning sitting posture measurement and detection method based on monocular camera
CN109785396A (en) Writing posture monitoring method based on binocular camera, system, device
US11771327B2 (en) System and method for human motion detection and tracking
CN114120357B (en) Neural network-based myopia prevention method and device
CN112232128B (en) Eye tracking based method for identifying care needs of old disabled people
CN112464793A (en) Method, system and storage medium for detecting cheating behaviors in online examination
US20230237694A1 (en) Method and system for detecting children's sitting posture based on face recognition of children
Prueksanusak et al. An ergonomic chair with Internet of thing technology using SVM
CN114419676A (en) Sitting posture analysis method and device based on artificial intelligence, computer equipment and medium
CN111627526B (en) Eye movement and attention feature vector determination method for children ADHD screening and evaluation system
CN112541897A (en) Facial paralysis degree evaluation system based on artificial intelligence
CN116543455A (en) Method, equipment and medium for establishing parkinsonism gait damage assessment model and using same
CN106327484B (en) A method of it is assessed for dentist's operation posture
CN115998288A (en) Sitting posture detection method based on multiple sensors
CN115578789A (en) Scoliosis detection apparatus, system, and computer-readable storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination