Summary of the invention
The object of the present invention is to provide a kind of based on human body attitude recognition technology automatic batch according to the method and apparatus of Image Acquisition Human Height measurement data.
The invention discloses a kind of height measurement method based on human body attitude identification, comprising:
Obtain the scene image that comprises human body attitude by image acquiring device;
Remove scene background, cut apart the human body image obtaining in scene image;
Human body image is carried out to human body attitude identification, judge that whether human body attitude is upright, if so, obtains Human Height by the space actual range calculating between human body image highs and lows;
If judge that human body attitude is not erectility, human body image is carried out to human body attitude identification, judge that whether human body attitude is for opening one's arms, if so, obtain Human Height by the space actual range calculating between the most left point of human body image and the rightest point; If not, identify and cut apart the different body parts that form height or arm exhibition in human body image by human body attitude, and calculate respectively the space physical length of the different body parts of described formation height or arm exhibition, height or the arm exhibition of the space physical length summation estimation human body of the different body parts to described formation height or arm exhibition obtain Human Height;
Wherein, by the pixel coordinate of the pixel of human body image being converted to the space actual coordinate of corresponding human body, and space actual coordinate based on being converted to calculates the space actual range between the human body that human body image pixel is corresponding.
Preferably, described image acquiring device is depth transducer, and described scene image is depth image, and the Pixel Information of described depth image comprises the space actual coordinate of pixel coordinate and corresponding scene point;
The space actual coordinate that the described pixel coordinate by the pixel of human body image is converted to corresponding human body comprises, according to the Pixel Information of described depth image, pixel coordinate is converted to the space actual coordinate of corresponding human body.
Preferably, described image acquiring device is two dimensional image filming apparatus, and described scene image is two dimensional image, and described method by taking to obtain image in the described two dimensional image filming apparatus preset distance situation of human body distance;
The space actual coordinate that the described pixel coordinate by the pixel of human body image is converted to corresponding human body comprises, solves the space actual coordinate of imaging object point by following formula based on pixel coordinate:
Wherein, (X
w, Y
w, Z
w) be scene point space actual coordinate, the initial point of space actual coordinates is got the camera lens photocentre position of described two dimensional image filming apparatus, the pixel coordinate that (u, v) is respective pixel, (u
o, v
o) be the pixel coordinate of picture centre, f
u, f
vbe respectively the equivalent focal length of X and Y-direction, f
u, f
v, u
o, v
ofor the inner parameter of two dimensional image filming apparatus, Z
wequal described preset distance.
Preferably, described image acquiring device is the consecutive image filming apparatus that obtains image sequence, described method is calculated respectively at least two human body height values according at least two corresponding images of moment in described image sequence, using the meansigma methods of described at least two human body height values as height measurements, or using the Human Height value that occurrence number is maximum in calculated Human Height value as height measurements, or using the Human Height value that occurrence number is maximum in the described Human Height value calculating with the weighted mean of the Human Height value that obtains according to consecutive frame image calculation as height measurements.
Preferably, the different body parts of described formation height are head, trunk, thigh and shank;
The different body parts of described formation arm exhibition are shoulder breadth, left upper arm, lower-left arm, the left hand palm, right upper arm, bottom right arm, the right hand palm.
The invention also discloses a kind of body height measuring device based on human body attitude identification, comprising:
Scene image acquisition module, for obtaining the scene image that comprises human body attitude by image acquiring device;
Human body image acquisition module, for removing scene background, cut apart the human body image obtaining in scene image;
The first height computing module, for human body image is carried out to human body attitude identification, judges that whether human body attitude is upright, if so, obtains Human Height by the space actual range calculating between human body image highs and lows;
The second height computing module, for in the time judging that human body attitude is not erectility, human body image is carried out to human body attitude identification, judge that whether human body attitude is for opening one's arms, if so, obtain Human Height by the actual range calculating between the most left point of human body image and the rightest point; If not, identify and cut apart the different body parts that form height or arm exhibition in human body image by human body attitude, and calculate respectively the space physical length of the different body parts of described formation height or arm exhibition, and height or the arm exhibition of the space physical length summation estimation human body of different body parts to described formation height or arm exhibition obtain Human Height;
Wherein, by the pixel coordinate of the pixel of human body image being converted to the space actual coordinate of corresponding human body, and space actual coordinate based on being converted to calculates the space actual range between the human body that human body image pixel is corresponding.
Preferably, described image acquiring device is depth transducer, described scene image is depth image, the Pixel Information of described depth image comprises the space actual coordinate of pixel coordinate and corresponding scene point, and described the first height computing module and the second height computing module are converted to pixel coordinate according to the Pixel Information of described depth image the space actual coordinate of corresponding human body.
Preferably, described image acquiring device is two dimensional image filming apparatus, described scene image is two dimensional image, described two dimensional image filming apparatus by taking to obtain image in the described two dimensional image filming apparatus preset distance situation of human body distance, and described the first height computing module and the second height computing module solve the space actual coordinate of imaging object point based on pixel coordinate by following formula:
Wherein, (X
w, Y
w, Z
w) be imaging object point space actual coordinate, the initial point of space actual coordinates is got the camera lens photocentre position of described two dimensional image filming apparatus, the pixel coordinate that (u, v) is pixel, (u
o, v
o) be the pixel coordinate of picture centre, f
u, f
vbe respectively the equivalent focal length of X and Y-direction, f
u, f
v, u
o, v
ofor the inner parameter of two dimensional image filming apparatus, Z
wfor described preset distance.
The present invention is by the automatic identification to human body attitude, to upright, open one's arms and the human body image of other attitude adopts different height account forms, human body in image is in the time being difficult to directly to estimate the attitude of its height, utilize its length that forms the different parts of height or arm exhibition to sue for peace to estimate Human Height, automaticity is high, and treatment effeciency is much higher than prior art.
Detailed description of the invention
Further illustrate technical scheme of the present invention below in conjunction with accompanying drawing and by detailed description of the invention.
Fig. 1 a and Fig. 1 b are that the embodiment of the present invention is carried out height summation measuring principle figure.
Human body is upright, that is, human body is standing state, now the head of human body, trunk, thigh and shank roughly linearly and this straight line perpendicular to the ground.In the time that human body is upright, Human Height can calculate by the difference in height of human body peak and minimum point.
, find through statistics, length and Height Ratio when human body opens one's arms are more approaching meanwhile, approximate the trying to achieve of arm length degree when height can open one's arms with human body.Open one's arms, refer to that human body both arms launch, the arm of human body and trunk plane are positioned at same plane, and with roughly state linearly of the shoulder of human body.In the time that human body opens one's arms, Human Height can obtain by the space actual range calculating between the most left point of human body and the rightest point.
When human body is in non-erectility, also when the non-state opening one's arms, Human Height can be by suing for peace and be similar to acquisition human body different piece, as shown in Figure 1a, and can be by the different body part length that form Human Height be sued for peace to ask for height:
Height+trunk height+thigh length+shank length (1) of height=head
Or, as shown in Figure 1 b, open up to obtain approximate height by read group total human body arm:
Height=shoulder breadth+left upper arm length+lower-left arm lengths+left hand length+right upper arm length+bottom right arm lengths+right hand length (2)
Wherein, gage beam exhibition and summation estimation arm are opened up to measure height and are applicable to not show in image all occasions of part of human body.
Fig. 2 is the method flow diagram that the embodiment of the present invention is carried out height measurement.Described method comprises the steps:
Step 100, obtain the scene image that comprises human body attitude by image acquiring device.
Step 200, process described scene image, cut apart the human body image obtaining in image.
In this step, can detect or image detection is obtained the Pixel Information of human body in image by connected domain, after other background information beyond human body pixel in scene image is removed, remaining image just only has human body, and the numerical value that makes the pixel except human body image in image is all zero.
Step 300, identify to judge that by human body image being carried out to human body attitude whether human body attitude is upright, if so, perform step 400, if not, perform step 500.
Chinese patent application CN101576953A discloses a kind of sorting technique of human body attitude, by the human region in tracking image, human region is carried out to the extraction of body shape, obtain human body outline, and according to number of people positional information, image length and width are recently classified to the human body attitude in image, this application also discloses the training module that is input to grader according to the characteristic vector of the external profile of lineup of artificial demarcation as training sample, set up the grader of human body attitude by training, and further realize the classification for the human body attitude in image according to grader.The grader of this human body attitude can be Adaboost (self-adaptive enhancement algorithm) grader, SVM (support vector machine) grader or neutral net etc.
Chinese patent application CN101989326A discloses a kind of recognition methods of human body attitude, mates to search for the swooping template action attitude of mating most by the characteristic information extracting in depth image with the template database of reservation.
Can realize the human body attitude identification for depth image or ordinary two-dimensional image by above method, thereby judge that human body attitude is uprightly or in other attitudes.Certainly, the present invention is not limited to the disclosed method of above document for the judgement of human body attitude.In any prior art, can all can be used to carry out for any method of utilizing the human body attitude identification of this step.
Step 400, pass through to calculate the space actual range acquisition Human Height between pixel highs and lows in human body image, process ends.
In this step, can obtain by directly searching for the minimum and maximum non-zero pixels of vertical coordinate value in image pixel the highs and lows of human body image, can obtain the space actual coordinate of the human body that described highs and lows is corresponding by pixel coordinate-space coordinate transformation, thereby thereby can calculate its space actual range according to the space actual coordinate of the highs and lows of upright human body obtains Human Height.Here, pixel coordinate refers to the coordinate that represents pixel present position in image, pixel coordinate system is a two-dimentional coordinate system, refer to the coordinate that represents object present position in real world with time space actual coordinate, space actual coordinates (also referred to as world coordinate system) is a three-dimensional system of coordinate.Space actual range refers in reality scene, the air line distance between 2, and it can calculate by space actual coordinate.
Step 500, judge that by human body image being carried out to human body attitude identification human body attitude, whether for opening one's arms, if so, performs step 600, if otherwise perform step 700.
Step 600, the space actual range that passes through to calculate between the most left point of human body image and the rightest point obtain Human Height, process ends.
In this step, can obtain by directly search for the minimum and maximum non-zero pixels of horizontal coordinate value in image pixel the most left point and the rightest point of human body image, the space actual coordinate of left point and human body corresponding to the rightest point described in can obtaining by image coordinate-space coordinate transformation, thereby can calculate its space actual range according to the space actual coordinate of the most left point of the human body that opens one's arms and the rightest point and obtain its arm length degree, estimate to obtain Human Height thereby obtain.
Step 700, identify and cut apart the different body parts that form height or arm exhibition in human body image by human body attitude, represent corresponding human body part with line segment.For example, head, trunk, thigh, shank or shoulder, arm etc. or represent the joint of corresponding human body parts with point, as elbow joint, shoulder joint etc.
Document " Articulated Human Pose Estimation and Search in (Almost) Unconstrained Still Images " Eichner, M.and Marin-Jimenez, M.and Zisserman, A.and Ferrari, V., ETH Zurich, D-ITET, BIWI, Technical Report No.272, September2010 and corresponding website disclose a kind of human body recognizer and system (http://www.vision.ee.ethz.ch/~calvin/articulated_human_pose_estima tion_code/), described algorithm and system can be cut apart by processing two-dimension human body image recognition the each several part of the human body that obtains the different attitudes in image, the head of for example human body, trunk, large arm, forearm, hands and thigh and shank etc.
Document " Real-Time Human Pose Recognition in Parts from Single Depth Images " .Jamie Shotton, Andrew Fitzgibbon, Mat Cook, Toby Sharp, Mark Finocchio, Richard Moore, Alex Kipman, and Andrew Blake.CVPR, 2011 disclose the recognition methods of the real-time human body attitude part of a kind of depth image, by depth image is processed, difficult attitude estimation problem is converted into simply and has realized and from depth image, separated the object of obtaining human synovial location information by pixel classification problem.
Can realize for the human body key point identification of depth image or ordinary two-dimensional image and cutting apart by above method.Certainly, the present invention is not limited to the disclosed method of above document for the judgement of human body attitude.
Step 800, calculate respectively the space physical length of described different body parts, and height or the arm exhibition of the space physical length summation estimation human body of different body parts to described formation height or arm exhibition obtain Human Height.
The human body key point that identification is obtained, for example, the pixel coordinate that forms the shoulder joint of upper arm and the pixel of elbow joint extracts and obtains, then each several part image coordinate-space coordinate transformation identification being obtained can obtain the space actual coordinate of the scene point that pixel is corresponding (being also above-mentioned human body key point), calculate its space physical length according to space actual coordinate, for example, obtain the physical length of human body upper arm by calculating the space actual range of shoulder joint and elbow joint.
Wherein, in above-mentioned steps 400, in the time that human body is upright, require image acquiring device can take the image of Whole Body.Obtain peak pixel coordinate and the minimum point pixel coordinate in human body image by search.By the conversion of pixel coordinate-space actual coordinate, according to the pixel coordinate (u of upright human body peak in image
max, v
max) and the pixel coordinate (u of minimum point
min, v
min), calculate the human body peak space actual coordinate (x acquiring
max, y
max, z
max) and the space actual coordinate (x of minimum point
min, y
min, z
min).Calculate upright Human Height according to following formula:
H=y
max-y
min (3)
In above-mentioned steps 600, when human body in image is when opening one's arms, now require image acquiring device can take the depth image of upper half of human body.Obtain the rightest some pixel coordinate and the most left some pixel coordinate in human body image by search.Change by pixel coordinate-space actual coordinate, according to the pixel coordinate (u of the upright the rightest point of human body in image
right, v
right) and the pixel coordinate (u of the most left point
left, v
left), calculate the rightest space of points actual coordinate of the human body (x acquiring
right,
yright, z
right), the most left point (x
left, y
left, z
left).Calculate upright Human Height according to following formula:
H=x
right-x
left (4)
Depth transducer is in the time obtaining scene depth image, and each point (scene point) in scene is projected to imaging plane, forms the pixel of image, and therefore, each pixel of depth image exists corresponding with it scene point.For instance, in Fig. 1 b the pixel at the finger tip place of human body image corresponding to the fingertip location of human body in reality scene.Depth transducer is forming the space actual coordinate that can calculate when depth image between pixel coordinate and the corresponding scene point of obtaining depth image.The information of the each pixel in depth image comprises { (u, v) (x, y, z) }, wherein (u, v) for representing the position of this pixel at image, (x, y, z) is for representing the position in space of scene point corresponding to this pixel,, the space actual coordinate of part corresponding to this pixel.For the depth image of the present embodiment, according to the difference of depth transducer institute operation technique, the mode of choosing of the initial point of its space actual coordinate can have difference, for example, adopt the depth transducer of structure light measurement, the zero of its space actual coordinates (also can be described as world coordinate system) is set in focal length of camera place conventionally, as shown in Figure 3.Thus, by the one-to-one relationship of image pixel coordinate and space actual coordinate, can obtain according to image pixel coordinate the space actual coordinate of pixel.
What need to further illustrate is, in the present embodiment, during due to subsequent calculations Human Height, what adopt is the mode of the space actual range between the corresponding scene point of the crucial pixel of computed image, its relative distance only and between scene point is relevant, location-independent with coordinate origin, therefore, in fact the mode of choosing of pixel coordinate and space coordinates initial point does not affect for subsequent calculations Human Height, as long as depth transducer comprises corresponding pixel coordinate and scene point space actual coordinate in the Pixel Information of the depth image forming.
Fig. 4 is the model schematic diagram of two dimensional image imaging.As shown in Figure 4, in another embodiment of the present invention, adopt two dimensional image filming apparatus to obtain the scene image that comprises human body as described image acquiring device, now, when image is obtained in requirement, human body is a known preset distance apart from the distance of two dimensional image filming apparatus, for example 2.5 meters.In the time that the described two dimensional image filming apparatus distance of human body distance is known, can pixel coordinate be converted to by changes in coordinates to the space actual coordinate of human body.The image that two dimensional image filming apparatus obtains can be divided into gray level image and coloured image, and in black white image, the information of each pixel comprises { (u, v), I (u, v) }, in coloured image, the information of each pixel comprises { (u, v) (r, g, b) }, wherein, (u, v) be the pixel coordinate of this pixel, the gray value that I (u, v) is this pixel, (r, g, b) is the color value of this pixel.
In the present embodiment, carry out the conversion of pixel coordinate-space actual coordinate with pinhole imaging system model.As shown in Figure 4, the imaging on the imaging plane of image acquiring device of the scene point in scene forms corresponding image slices vegetarian refreshments, (X
w, Y
w, Z
w) be the space actual coordinate of scene point, (u, v) is the pixel coordinate of corresponding imaging point pixel.Wherein, taking the camera lens optical center of lens place of image acquiring device as the initial point of space actual coordinates, according to pinhole imaging system principle, the described space of imaging plane distance actual coordinates initial point distance is focal distance f.According to Similar Principle of Triangle, there is following relation in the space actual coordinate at pixel coordinate and corresponding fields sight spot:
Wherein, (u
o, v
o) be picture centre (intersection point of optical axis and the plane of delineation) coordinate, in fact picture centre coordinate has explained the position of pixel coordinate system with respect to described picture centre, as long as provide picture centre coordinate, the initial point of pixel coordinate system can be chosen the optional position of imaging plane as required.F
u=f/d
x, f
v=f/d
ybe defined as respectively the equivalent focal length of X and Y-direction, in above formula, f is camera lens focal length, d
x, d
yfor camera review pixel is in x axle and the axial size of y.F
u, f
v, u
o, v
othese 4 parameters are only relevant with video camera internal structure, are therefore called the inner parameter of video camera, and these parameters can directly obtain from video camera.
Distance Z at known human body apart from video camera
wand in the situation of the inner parameter of above-mentioned video camera, can calculate corresponding space actual coordinate (X according to pixel coordinate by formula (5)
w, Y
w, Z
w).
In another embodiment of the present invention, can be from video, be also to add up height in continuous image sequence, to obtain higher precision.In this embodiment, can select and not distinguish in the same time corresponding multiple image from image sequence, calculate respectively multiple Human Height values according to the multiple image of choosing, then the Human Height value calculating is averaging, using this meansigma methods as Human Height measured value.For example, the t of selecting video
1, t
2... t
nn the image that moment is corresponding, calculates respectively n human body height values according to the n a choosing image, this n human body height values is asked on average, using meansigma methods as Human Height measured value.
Except calculating mean value is from multiple image obtains more accurate Human Height measured value, can also calculate multiple Human Height values according to the corresponding multiple image of not distinguishing in the same time of video, set up the rectangular histogram of these multiple height values that calculate, by height numerical value corresponding cylinders maximum occurrence number in rectangular histogram (be also in the plurality of Human Height value appearance/number of repetition at most/height values that frequency is the highest) be human body height measurements.For example, the t of selecting video
1, t
2... t
nn the image that moment is corresponding, calculates respectively n human body height values according to the n a choosing image, and wherein a certain height values x has occurred m time in n estimation, is the Human Height value that occurrence number is maximum, chooses so height values x as Human Height measured value.
In addition, can also ask the weighted average of the corresponding height numerical value of multiple cylinders that cylinder that in rectangular histogram, occurrence number is the highest is adjacent as Human Height measured value, wherein the weighting weight of each height numerical value is the occurrence number of this cylinder numerical value in whole rectangular histogram.For example, in the n calculating a human body height values, wherein a certain height values x has occurred m time in n estimation, is the Human Height value that occurrence number is maximum, chooses the adjacent height values x in moment both sides that this height values x occurs
1, x
2... x
2m, these height values are weighted on average according to the number of times occurring in n estimation, using result as Human Height measured value.
Fig. 5 is the block diagram of body height measuring device of the present invention.As shown in Figure 5, described body height measuring device comprises that scene image acquisition module, human body image acquisition module, the first attitude judge module, the second attitude judge module, upright height computing module, arm are opened up height computing module, human body image is cut apart module and height summation module.
Wherein, scene image acquisition module is for obtaining by image acquiring device the scene image that comprises human body attitude;
Human body image acquisition module, for the treatment of described scene image, is cut apart the human body image obtaining in image;
The first attitude judge module is used for carrying out human body attitude identification and judges whether human body attitude is upright;
The second attitude judge module is used for carrying out human body attitude identification and judges that whether human body attitude is for opening one's arms;
Upright height computing module, in the time that the first attitude judge module judges that human body attitude is upright, obtains Human Height by the space actual range calculating between human body pixel highs and lows;
Arm exhibition height computing module is for judge that at the second attitude judge module human body attitude is when opening one's arms, by the space actual range acquisition Human Height between the calculating the most left point of human body image and the rightest point;
Human body image is cut apart module for judging that at the first attitude judge module human body attitude is not for upright, and the second attitude judge module judges that human body attitude is not when opening one's arms, identify and cut apart the different body parts that form height or arm exhibition in human body image by human body attitude, represent corresponding human body part or with key point mark human synovial position with line segment.
Height summation module is for calculating respectively the space physical length of described different body parts, and height or the arm exhibition of the length summation estimation human body of different body parts to described formation height or arm exhibition obtain Human Height.
Wherein, the first attitude judge module and upright height computing module have formed the first height computing module jointly; The second attitude judge module, arm exhibition height computing module, human body image are cut apart module and the common formation of height summation module the second height computing module.
The present invention is by the automatic identification to human body attitude, to upright, open one's arms and the human body image of other attitude adopts different height account forms, human body in image is in the time being difficult to directly to estimate the attitude of its height, utilize its length that forms the different parts of height or arm exhibition to sue for peace to estimate Human Height, automaticity is high, and treatment effeciency is much higher than prior art.
Obviously, those skilled in the art should be understood that, above-mentioned of the present invention each module or each step can realize with general accountant, they can concentrate on single accountant, or be distributed on the network that multiple accountants form, alternatively, they can realize with the executable program code of computer installation, thereby they can be stored in storage device and be carried out by accountant, or they are made into respectively to each integrated circuit modules, or the multiple modules in them or step are made into single integrated circuit module to be realized.Like this, the present invention is not restricted to the combination of any specific hardware and software.
The foregoing is only the preferred embodiments of the present invention, be not limited to the present invention, to those skilled in the art, the present invention can have various changes and variation.All any amendments of doing, be equal to replacement, improvement etc., within protection scope of the present invention all should be included within spirit of the present invention and principle.