CN116831560A - Human height detection method based on skeleton key point recognition - Google Patents

Human height detection method based on skeleton key point recognition Download PDF

Info

Publication number
CN116831560A
CN116831560A CN202310538428.0A CN202310538428A CN116831560A CN 116831560 A CN116831560 A CN 116831560A CN 202310538428 A CN202310538428 A CN 202310538428A CN 116831560 A CN116831560 A CN 116831560A
Authority
CN
China
Prior art keywords
image
height
nose
ankle
depth camera
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202310538428.0A
Other languages
Chinese (zh)
Inventor
郑煜涵
蒋婉玥
刘晓瑞
葛树志
刘银华
张中浩
张瑞
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Qingdao University
Original Assignee
Qingdao University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Qingdao University filed Critical Qingdao University
Priority to CN202310538428.0A priority Critical patent/CN116831560A/en
Publication of CN116831560A publication Critical patent/CN116831560A/en
Pending legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/103—Static body considered as a whole, e.g. static pedestrian or occupant recognition
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/103—Detecting, measuring or recording devices for testing the shape, pattern, colour, size or movement of the body or parts thereof, for diagnostic purposes
    • A61B5/107—Measuring physical dimensions, e.g. size of the entire body or parts thereof
    • A61B5/1072—Measuring physical dimensions, e.g. size of the entire body or parts thereof measuring distances on the body, e.g. measuring length, height or thickness
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/103—Detecting, measuring or recording devices for testing the shape, pattern, colour, size or movement of the body or parts thereof, for diagnostic purposes
    • A61B5/107—Measuring physical dimensions, e.g. size of the entire body or parts thereof
    • A61B5/1075—Measuring physical dimensions, e.g. size of the entire body or parts thereof for measuring dimensions by non-invasive methods, e.g. for determining thickness of tissue layer
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/103—Detecting, measuring or recording devices for testing the shape, pattern, colour, size or movement of the body or parts thereof, for diagnostic purposes
    • A61B5/107—Measuring physical dimensions, e.g. size of the entire body or parts thereof
    • A61B5/1079—Measuring physical dimensions, e.g. size of the entire body or parts thereof using optical or photographic means
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/10—Image acquisition
    • G06V10/12—Details of acquisition arrangements; Constructional details thereof
    • G06V10/14—Optical characteristics of the device performing the acquisition or on the illumination arrangements
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/10—Image acquisition
    • G06V10/12—Details of acquisition arrangements; Constructional details thereof
    • G06V10/14—Optical characteristics of the device performing the acquisition or on the illumination arrangements
    • G06V10/145—Illumination specially adapted for pattern recognition, e.g. using gratings
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/34—Smoothing or thinning of the pattern; Morphological operations; Skeletonisation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks

Landscapes

  • Engineering & Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Medical Informatics (AREA)
  • General Health & Medical Sciences (AREA)
  • Surgery (AREA)
  • Biomedical Technology (AREA)
  • Molecular Biology (AREA)
  • Animal Behavior & Ethology (AREA)
  • Public Health (AREA)
  • Heart & Thoracic Surgery (AREA)
  • Dentistry (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • Veterinary Medicine (AREA)
  • Biophysics (AREA)
  • Pathology (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Evolutionary Computation (AREA)
  • Artificial Intelligence (AREA)
  • Software Systems (AREA)
  • Databases & Information Systems (AREA)
  • Computing Systems (AREA)
  • Human Computer Interaction (AREA)
  • Measurement Of The Respiration, Hearing Ability, Form, And Blood Characteristics Of Living Organisms (AREA)
  • Length Measuring Devices By Optical Means (AREA)

Abstract

The invention discloses a human height detection method based on skeleton key point identification, which comprises the steps of firstly acquiring an image containing a target to be detected in real time through a depth camera, acquiring pixel coordinates of skeleton key points of the target to be detected in the image by adopting a human skeleton detector, calculating the relative height from a nose to an ankle in the image based on the pixel coordinates of the skeleton key points, then bringing the pixel coordinates of any skeleton key point into the depth camera, calculating the distance between the skeleton key points and the depth camera, then calculating the visible longitudinal distance of the depth camera, and finally calculating the real height of the target to be detected. The depth camera is accurately utilized to measure the height of the human body in cooperation with bone detection, and the height detection can be performed when the human body is in a non-standing posture. The invention only needs one depth image and then outputs reliable results, thereby saving a great deal of manpower and time.

Description

Human height detection method based on skeleton key point recognition
Technical field:
the invention belongs to the technical field of human body height detection, and particularly relates to a human body height detection method based on skeleton key point recognition.
The background technology is as follows:
in daily life, height is a major concern. However, the judgment of the height by naked eyes is always irregular, the judgment of the height by the depth camera not only can improve the accuracy, but also can automatically acquire the height information of the person by a computer for further processing, and in recent years, with the development of deep learning and image processing technology, the height detection technology is also continuously developed and perfected. The existing height detection technology is very lacking, the technology for detecting the height through a neural network is few, and the existing height detection needs to be compared by using a reference object, so that the accuracy and the practicability are deficient. The trend in the field of height detection technology is constantly towards more accurate, diversified and personalized. The current height detection and amplification mostly needs to calibrate a reference object, and the height of a person is obtained by converting the scale proportion of the person and the reference object, so that the reference object is difficult to determine, and the algorithm is inconvenient to realize under a fixed scene.
Recently, researchers in san diego and Adobe, university of california, have proposed a monocular vision-based measurement method that can recover the absolute dimensions of a scene and a target by measuring the height of the target in a photograph, the height of a camera, and the viewing angle orientation parameters, which can be accurately implemented in an unrestricted environment using monocular vision. Fig. 1 is a diagram of detection results, and it can be found that the detection principle is to scale the white stool, so that when the size of the white stool is changed, the height of the human body is changed, when no stool appears in the environment, the height of the human body is not detected, and when the human body is in different postures, the detected height is also changed, and the real-time performance is poor. In the method and the device for estimating the absolute depth of the image by using the patent CN 115797432A, the Marshall trunk index of the target is obtained through calculation according to skeleton key points, and then the absolute height of the target is obtained from a corresponding table of the Marshall trunk index and the height in a table look-up mode according to the Marshall trunk index.
In the fields of three-dimensional reconstruction, medical treatment, clothing size, etc., human height data are indispensable. In most cases, we will require the tester to stand upright and then measure the height with a meter or other tool, which will consume a lot of time and manpower. In particular, in practice, if we have no measuring tool, or if the person to be measured is a child, or if the injury is not standing straight, it is very difficult to measure the height.
The invention comprises the following steps:
the invention aims to seek to design a human body height detection method based on skeleton key point recognition, and solves the technical problems that the existing height measurement technology needs a reference object and the height measurement of a human body in a non-standing posture cannot be performed. According to the method, the height result of the target to be detected is obtained according to the target image to be detected obtained by the depth camera, and the technical blank of height identification by deep learning at present is filled.
In order to achieve the above purpose, the invention relates to a human height detection method based on skeleton key point recognition, which specifically comprises the following steps:
the method comprises the steps of (1) obtaining an image containing a target to be detected in real time through a depth camera, wherein the image size is M pixels multiplied by N pixels, when the target to be detected stands, performing the step (2), and when the target to be detected does not stand, performing the step (3);
(2) Acquiring pixel coordinates of skeleton key points of an object to be detected in an image by using a human skeleton detector, wherein the pixel coordinates comprise a nose (X 0 ,Y 0 ) And right ankle (X) 4 ,Y 4 ) Or (b)Nose (X) 0 ,Y 0 ) And left ankle (X) 7 ,Y 7 )，
Through nose (X) 0 ,Y 0 ) And right ankle (X) 4 ,Y 4 ) Or nose (X) 0 ,Y 0 ) And left ankle (X) 7 ,Y 7 ) The relative height Y of the nose to the ankle in the image is calculated:
or->
(3) Acquiring pixel coordinates of skeleton key points of an object to be detected in an image by using a human skeleton detector, wherein the pixel coordinates comprise a nose (X 0 ,Y 0 ) Neck (X) 1 ,Y 1 ) Right buttocks (X) 2 ,Y 2 ) And right knee (X) 3 ,Y 3 ) Right ankle (X) 4 ,Y 4 ) Or nose (X) 0 ,Y 0 ) Neck (X) 1 ,Y 1 ) Left buttocks (X) 5 ,Y 5 ) Left knee (X) 6 ,Y 6 ) And left ankle (X) 7 ,Y 7 )，
The relative neck-to-hip distance a in the image is calculated,
or->
The relative distance B between buttocks and knees in the image is calculated,
or->
The relative distance C from knee to ankle in the image is calculated,
or->
Then: the relative height Y of nose to ankle in the image:
Y＝Y 1 -Y 0 +A+B+C；
(4) Bringing the pixel coordinates of any bone key point in the step (2) or (3) into a depth camera, calculating the distance Z between the bone key point and the depth camera, calculating the visible longitudinal distance H of the depth camera according to the following formula,
wherein Z is the distance between the object to be measured and the camera, and θ is the longitudinal angle of the image shot by the depth camera;
(5) Finally, the true height S of the target to be measured is calculated according to the following formula:
where L is the nose-to-ankle true height, μ is the proportionality constant between the true height S of the target to be measured and the nose-to-ankle height L.
Specifically, the depth camera is RealSense, and the human skeleton detector is openpost.
Specifically, according to a function aligned_depth_frame_distance (x, y) in the depth camera Realsense, an actual distance Z between the object to be measured and the camera is calculated.
Compared with the prior art, the invention accurately utilizes the depth camera to measure the height of the human body in combination with bone detection, and can also detect the height when the human body is in a non-standing posture. The invention only needs one depth image and then outputs reliable results, thereby saving a great deal of manpower and time.
Description of the drawings:
FIG. 1 is a diagram showing a method for measuring height of a human body based on monocular vision in the prior art.
FIG. 2 is a flow chart of a method for detecting height of a human body based on skeletal key point recognition according to the present invention.
Fig. 3 is a photograph of the RealSense camera referred to in example 1.
Fig. 4 is a network configuration diagram of openpost according to embodiment 1.
Fig. 5 is a schematic diagram of coordinates of key points of bones of a human body in a standing posture.
Fig. 6 is a schematic diagram of coordinates of key points of bones of a human body in a non-standing posture.
FIG. 7 is a RealSense RGB image size diagram in a standing position.
Fig. 8 is a dimensional diagram between the subject and the camera at the time of photographing of fig. 7.
The specific embodiment is as follows:
the invention is further described below by way of examples.
Example 1:
according to the human height detection method based on skeleton key point recognition, openPose is used as a human skeleton detector, a distance between a depth camera and a person is obtained by matching with a RealSense depth camera, and the human height is judged by combining distance information, wherein a specific flow is shown in FIG. 2.
According to the method, the two-dimensional position information of each pixel point in the image can be represented by pixel coordinates of an image of a target to be detected, which is obtained by the depth camera, and the depth camera can also obtain the distance between each pixel point and the depth camera by the pixel coordinates, wherein the target to be detected is a person. The depth camera used in this embodiment is RealSense, and the manufacturer is intel. The implementation principle of the current depth camera is mainly divided into three types: structured light, tof, binocular imaging, respectively. The RealSense uses a structured light scheme. Fig. 3 shows the RealSense camera, four cameras on the front, left infrared camera from left to right, infrared spot projector, right infrared camera and RGB camera. .
In the embodiment, the human skeleton detector detects the joint points of the human body in the target image to be detected as skeleton key points, and the human skeleton information is described through the skeleton key points. openPose is an open source library based on convolutional neural network and supervised learning and written by taking caffe as a framework, can track facial expression, trunk, limbs and even fingers of people, is suitable for a single person and multiple people, and has good robustness. The method can be called as the first real-time multi-person two-dimensional attitude estimation based on deep learning in the world, is a milestone in human-computer interaction, and provides a high-quality information dimension for robot understanding.
The embodiment relates to a human height detection method based on skeleton key point recognition, which specifically comprises the following steps:
the method comprises the steps of (1) acquiring an image containing a target to be measured in real time through a depth camera (RealSense), wherein the image size is M pixels multiplied by N pixels, when the target to be measured stands, performing the step (2), and when the target to be measured does not stand, performing the step (3);
in this embodiment, the image is a RealSense RGB image, the size of the RealSense RGB image is M pixels×n pixels, the pixel coordinate of the upper right corner of the image is (0, 0), and the pixel coordinate of the lower left corner of the image is (M, N);
(2) Acquiring pixel coordinates of skeleton key points of an object to be detected in an image by using a human skeleton detector (such as openpost), including a nose (X) 0 ,Y 0 ) And right ankle (X) 4 ,Y 4 ) Or nose (X) 0 ,Y 0 ) And left ankle (X) 7 ,Y 7 )，
Through nose (X) 0 ,Y 0 ) And right ankle (X) 4 ,Y 4 ) Or nose (X) 0 ,Y 0 ) And left ankle (X) 7 ,Y 7 ) The relative height Y of the nose to the ankle in the image is calculated:
or->
(3) Acquiring pixel coordinates of skeleton key points of an object to be detected in an image by using a human skeleton detector (such as openpost), including a nose (X) 0 ,Y 0 ) Neck (X) 1 ,Y 1 ) Right buttocks (X) 2 ,Y 2 ) And right knee (X) 3 ,Y 3 ) Right ankle (X) 4 ,Y 4 ) Or nose (X) 0 ,Y 0 ) Neck (X) 1 ,Y 1 ) Left buttocks (X) 5 ,Y 5 ) Left knee (X) 6 ,Y 6 ) And left ankle (X) 7 ,Y 7 )，
The relative neck-to-hip distance a in the image is calculated,
or->
The relative distance B between buttocks and knees in the image is calculated,
or->
The relative distance C from knee to ankle in the image is calculated,
or->
Then: the relative height Y of nose to ankle in the image:
Y＝Y 1 -Y 0 +A+B+C；
(4) Bringing the pixel coordinates of any bone key point in the step (2) or the step (3) into a depth camera, calculating the distance Z between the bone key point and the depth camera, and calculating the visible longitudinal distance H of the depth camera according to the following formula, wherein the longitudinal distance H corresponds to N in the target image to be detected:
wherein Z is the distance between the object to be measured and the camera, and θ is the longitudinal angle of the image shot by the depth camera;
since the object to be measured is basically parallel to the depth camera during shooting, namely the distance between any bone key point and the depth camera is the same.
In this embodiment, according to the function aligned_depth_frame_distance (x, y) in the depth camera readsense, the actual distance Z between the object to be measured and the camera is calculated, and then the visible longitudinal distance H of the readsense camera is calculated according to the following formula:
wherein Z is the distance between the object to be measured and the camera, and θ is the longitudinal angle of the RealSense RGB image shot by the RealSense camera;
(5) Finally, the true height S of the target to be measured is calculated according to the following formula:
where L is the nose-to-ankle true height, L corresponds to Y, μ is a proportionality constant between the true height S of the object to be measured and the nose-to-ankle height L, and μ=1.07 is verified based on a large amount of experimental data.

Claims (3)

1. The human height detection method based on skeleton key point recognition is characterized by comprising the following steps of:
the method comprises the steps of (1) obtaining an image containing a target to be detected in real time through a depth camera, wherein the image size is M pixels multiplied by N pixels, when the target to be detected stands, performing the step (2), and when the target to be detected does not stand, performing the step (3);
(2) Acquiring pixel coordinates of skeleton key points of an object to be detected in an image by using a human skeleton detector, wherein the pixel coordinates comprise a nose (X 0 ,Y 0 ) And right ankle (X) 4 ,Y 4 ) Or nose (X) 0 ,Y 0 ) And left ankle (X) 7 ,Y 7 )，
Through nose (X) 0 ,Y 0 ) And right ankle (X) 4 ,Y 4 ) Or nose (X) 0 ,Y 0 ) And left ankle (X) 7 ,Y 7 ) The relative height Y of the nose to the ankle in the image is calculated:
or->
(3) Acquiring pixel coordinates of skeleton key points of an object to be detected in an image by using a human skeleton detector, wherein the pixel coordinates comprise a nose (X 0 ,Y 0 ) Neck (X) 1 ,Y 1 ) Right buttocks (X) 2 ,Y 2 ) And right knee (X) 3 ,Y 3 ) Right ankle (X) 4 ,Y 4 ) Or nose (X) 0 ,Y 0 ) Neck (X) 1 ,Y 1 ) Left buttocks (X) 5 ,Y 5 ) Left knee (X) 6 ,Y 6 ) And left ankle (X) 7 ,Y 7 )，
The relative neck-to-hip distance a in the image is calculated,
or->
The relative distance B between buttocks and knees in the image is calculated,
or->
The relative distance C from knee to ankle in the image is calculated,
or->
Then: the relative height Y of nose to ankle in the image:
Y＝Y 1 -Y 0 +A+B+C；
(4) Bringing the pixel coordinates of any bone key point in the step (2) or (3) into a depth camera, calculating the distance Z between the bone key point and the depth camera, calculating the visible longitudinal distance H of the depth camera according to the following formula,
wherein Z is the distance between the object to be measured and the camera, and θ is the longitudinal angle of the image shot by the depth camera;
(5) Finally, the true height S of the target to be measured is calculated according to the following formula:
where L is the nose-to-ankle true height, μ is the proportionality constant between the true height S of the target to be measured and the nose-to-ankle height L.
2. The method for detecting human height based on bone key point recognition according to claim 1, wherein the depth camera is RealSense and the human bone detector is openpost.
3. The human height detection method based on bone key point recognition according to claim 1, wherein the actual distance Z between the object to be detected and the camera is calculated according to a function aligned_depth_frame_get_distance (x, y) in the depth camera Realsense.
CN202310538428.0A 2023-05-12 2023-05-12 Human height detection method based on skeleton key point recognition Pending CN116831560A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202310538428.0A CN116831560A (en) 2023-05-12 2023-05-12 Human height detection method based on skeleton key point recognition

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202310538428.0A CN116831560A (en) 2023-05-12 2023-05-12 Human height detection method based on skeleton key point recognition

Publications (1)

Publication Number Publication Date
CN116831560A true CN116831560A (en) 2023-10-03

Family

ID=88164187

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202310538428.0A Pending CN116831560A (en) 2023-05-12 2023-05-12 Human height detection method based on skeleton key point recognition

Country Status (1)

Country Link
CN (1) CN116831560A (en)

Similar Documents

Publication Publication Date Title
Viswakumar et al. Human gait analysis using OpenPose
US10417775B2 (en) Method for implementing human skeleton tracking system based on depth data
US9898651B2 (en) Upper-body skeleton extraction from depth maps
US9235753B2 (en) Extraction of skeletons from 3D maps
CA3042819A1 (en) Method and system for postural analysis and measuring anatomical dimensions from a digital image using machine learning
WO2012077286A1 (en) Object detection device and object detection method
CN110074788B (en) Body data acquisition method and device based on machine learning
Wang et al. Facial feature extraction in an infrared image by proxy with a visible face image
Li et al. Posture recognition technology based on kinect
Liu et al. Simple method integrating OpenPose and RGB-D camera for identifying 3D body landmark locations in various postures
JP3144400B2 (en) Gesture recognition device and method
Wang et al. Pose determination of human faces by using vanishing points
CN115035546A (en) Three-dimensional human body posture detection method and device and electronic equipment
Yamauchi et al. Recognition of walking humans in 3D: Initial results
CN109801326A (en) It is a kind of for obtaining the image measuring method of human somatotype data
CN109740458B (en) Method and system for measuring physical characteristics based on video processing
Swadzba et al. Tracking objects in 6D for reconstructing static scenes
CN116831560A (en) Human height detection method based on skeleton key point recognition
CN108694348B (en) Tracking registration method and device based on natural features
Yüksel et al. Analyses of body measurement with depth image data using motion capture sensor
Wang et al. Registration of infrared and visible-spectrum imagery for face recognition
Jiménez et al. Face tracking and pose estimation with automatic three-dimensional model construction
JP7136344B2 (en) Camera calibration method, camera and program
CN114463663A (en) Method and device for calculating height of person, electronic equipment and storage medium
Gruendig et al. 3d head pose estimation with symmetry based illumination model in low resolution video

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination