WO2016021152A1 - 姿勢推定方法および姿勢推定装置 - Google Patents
姿勢推定方法および姿勢推定装置 Download PDFInfo
- Publication number
- WO2016021152A1 WO2016021152A1 PCT/JP2015/003803 JP2015003803W WO2016021152A1 WO 2016021152 A1 WO2016021152 A1 WO 2016021152A1 JP 2015003803 W JP2015003803 W JP 2015003803W WO 2016021152 A1 WO2016021152 A1 WO 2016021152A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- posture
- candidate
- analysis target
- target image
- region
- Prior art date
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
- G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
- G06T7/74—Determining position or orientation of objects or cameras using feature-based methods involving reference images or patches
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
- G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
- G06T7/248—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments involving reference images or patches
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
- G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
- G06T7/251—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments involving models
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
- G06T7/277—Analysis of motion involving stochastic approaches, e.g. using Kalman filters
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/60—Analysis of geometric attributes
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
- G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
- G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
- G06T7/75—Determining position or orientation of objects or cameras using feature-based methods involving models
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
- G06T7/77—Determining position or orientation of objects or cameras using statistical methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
- G06V10/457—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components by analysing connectivity, e.g. edge linking, connected component analysis or slices
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/103—Static body considered as a whole, e.g. static pedestrian or occupant recognition
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/20—Movements or behaviour, e.g. gesture recognition
- G06V40/23—Recognition of whole body movements, e.g. for sport training
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10016—Video; Image sequence
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20092—Interactive image processing based on input by user
- G06T2207/20101—Interactive definition of point of interest, landmark or seed
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30196—Human being; Person
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30221—Sports video; Sports image
Definitions
- the present disclosure relates to a posture estimation method and a posture estimation device for estimating a posture of a person included in an image from the image.
- Non-Patent Document 1 there is a technique for estimating the posture of a person (hereinafter referred to as “subject”) included in an image from the image (see, for example, Non-Patent Document 1).
- Non-Patent Document 1 (hereinafter referred to as “conventional technique”) first estimates the head position by extracting the contour shape of the head from the image, and uses the estimated head position as a reference to determine the posture of the person.
- the backbone link model that defines
- the backbone link model in the prior art means that a person's posture is determined by the position, width, height, and angle of each of the five parts of the head, upper torso, lower torso, upper thigh, and lower thigh. It is a model to be defined.
- the conventional technology a large number of particles representing a plurality of postures are set, and the likelihood indicating the certainty that each part of each particle exists in the set region is calculated from the image feature of the part. Then, the conventional technique estimates that the posture with the highest weighted average value of the likelihood of all parts is the posture taken by the subject.
- ABS Athlete Behavior Analysis System
- Athletes take a variety of postures including the above-mentioned unusual postures. Therefore, a technique that can estimate the posture of a person included in an image with higher accuracy is desired.
- An object of the present disclosure is to provide a posture estimation method and a posture estimation apparatus that can estimate a posture of a person included in an image with higher accuracy.
- the posture estimation method of the present disclosure is a method by which the processor estimates the posture of a person in the analysis target image.
- This processor inputs an analysis target image, and sets a plurality of reference positions including a human head position and a waist position for the input analysis target image. Then, based on the joint base link model that defines the posture of the person by the arrangement of a plurality of point positions including the head position and the waist position and the plurality of part regions, and a part in the analysis target image A candidate area is determined. Further, based on the part image feature that is the image feature of the part region in the image obtained by photographing the person and the image feature of the determined candidate region, it is determined whether or not the person included in the analysis target image is taking the posture. to decide.
- the posture estimation device of the present disclosure is a posture estimation device having a processor.
- This processor inputs an analysis target image, and sets a plurality of reference positions including a human head position and a waist position for the input analysis target image. Then, based on the joint base link model that defines the posture of the person by the arrangement of a plurality of point positions including the head position and the waist position and the plurality of part regions, and a part in the analysis target image A candidate area is determined. Further, based on the part image feature that is the image feature of the part region in the image obtained by photographing the person and the image feature of the determined candidate region, it is determined whether or not the person included in the analysis target image is taking the posture. to decide.
- the posture of a person included in an image can be estimated with higher accuracy.
- FIG. 1 is a block diagram showing an example of the configuration of the posture estimation apparatus according to the present embodiment.
- FIG. 2 is a diagram for explaining an example of the joint base link model in the present embodiment.
- FIG. 3 is a diagram for explaining an example of the learning of the part image feature in the present embodiment.
- FIG. 4 is a diagram illustrating an example of the operation of the posture estimation apparatus according to the present embodiment.
- FIG. 5 is a diagram illustrating an example of an input video in the present embodiment.
- FIG. 6 is a diagram illustrating an example of how the reference position is set in the present embodiment.
- FIG. 7 is a diagram illustrating an example of a particle group generated in the present embodiment.
- FIG. 8 is a diagram illustrating an example of a particle group generated from only one reference position as a reference.
- FIG. 1 is a block diagram showing an example of the configuration of the posture estimation apparatus according to the present embodiment.
- FIG. 2 is a diagram for explaining an example of the joint base link model in the present embodiment.
- FIG. 9 is a diagram illustrating an example of the candidate posture estimated from the initial particles in the present embodiment.
- FIG. 10 is a diagram illustrating an example of a candidate posture estimated from the additional particles in the present embodiment.
- FIG. 11 is a diagram illustrating an example of an experimental result in the present embodiment.
- FIG. 1 is a block diagram showing an example of the configuration of the posture estimation apparatus according to the present embodiment.
- the posture estimation apparatus 100 shown in FIG. 1 includes, for example, a CPU (Central Processing Unit) as a processor, a storage medium such as a ROM (Read Only Memory) storing a control program, and a RAM (Random Access). Memory) and the like. In this case, the function of each unit described above is realized by the CPU executing the control program.
- the posture estimation apparatus 100 includes, for example, a communication circuit, and performs data input / output with another apparatus through communication using the communication circuit.
- the posture estimation apparatus 100 includes a user interface such as a liquid crystal display with a touch panel, for example, and performs information display and operation reception using the user interface.
- the posture estimation apparatus 100 includes a model information storage unit 110, an image input unit 120, a reference position setting unit 130, a candidate area determination unit 140, a posture determination unit 150, and a determination result output unit 160.
- the model information storage unit 110 stores in advance a joint base link model which is a kind of human body model and a part image feature which is an image feature of each part of a human body.
- the human body model is a constraint condition on the arrangement and size of each part of the person in the image, and is information indicating the posture of the person (characteristic of the human body).
- the joint base link model used in this embodiment is a human body model suitable for accurately estimating an extraordinary posture such as a posture in sports, and a posture state with a plurality of state variables as axes. Defined using space. More specifically, the joint base link model is a human body model that defines the posture of a person by arranging a plurality of point positions including a head position and a waist position and a plurality of region regions. Details of the joint base link model will be described later.
- the part image feature is an image feature of a region of a body part (hereinafter referred to as “part region”) such as a trunk or a left upper thigh in an image obtained by photographing a person. Details of the part image feature will be described later.
- the image input unit 120 inputs a video that is a target of human extraction or human posture estimation. Then, the image input unit 120 sequentially outputs a plurality of time-series image frames constituting the video (hereinafter referred to as “analysis target image”) to the reference position setting unit 130 and the candidate area determination unit 140.
- the image input unit 120 accesses a server on the Internet and acquires a video stored in the server.
- the analysis target image is, for example, a wide area still image obtained by photographing the entire field of American football.
- an XY coordinate system based on the position of the lower left corner of the image is set.
- the reference position setting unit 130 sets a plurality of reference positions including a head position and a waist position of a person (hereinafter referred to as “subject”) included in the analysis target image with respect to the input analysis target image. In the present embodiment, it is assumed that there are two reference positions, a head position and a waist position. Then, the reference position setting unit 130 outputs reference position information indicating the set reference position to the candidate area determination unit 140.
- the reference position setting unit 130 displays, for example, the analysis target image of the first frame of the video, and sets the reference position based on the user's operation. Details of the setting of the reference position will be described later.
- Candidate area determination unit 140 is based on the joint base link model stored in model information storage unit 110 and the plurality of reference positions indicated by the input reference position information, and the region of the part area in the input analysis target image. A candidate area is determined.
- the candidate area determination unit 140 for example, for the analysis target image of the first frame of the video, based on the joint base link model, a plurality of posture samples (arrangement of a plurality of point positions and a plurality of part areas). ) Is generated. Then, for each of the plurality of generated samples, the candidate area determination unit 140 aligns the sample with the analysis target image with reference to the reference position, thereby arranging the plurality of point positions and the plurality of part areas in the analysis target image. (Hereinafter referred to as “mapping sample”).
- the candidate area determination unit 140 generates a sample in the form of arranging a large number of candidate areas in the vicinity of each part based on the position and orientation of the subject in the immediately preceding frame, Determine the mapping sample.
- the candidate area determination unit 140 outputs mapping sample information indicating the mapping sample (that is, indicating the determined candidate area) and the input analysis target image to the posture determination unit 150. Details of the determination of the candidate area (mapped sample) will be described later.
- the posture determination unit 150 applies the input analysis target image to the input analysis target image based on the part image feature of each part stored in the model information storage unit 110 and the image feature of each candidate area indicated by the input mapping sample information. It is determined whether or not the included person has an attitude corresponding to any mapping sample. That is, the posture determination unit 150 determines whether or not the analysis target image includes a person who takes the posture of the mapping sample indicated by the mapping sample information.
- posture determination unit 150 calculates part-specific likelihood indicating the likelihood that the candidate area is a corresponding part area for each of a plurality of candidate areas included in the plurality of mapping samples.
- the posture determination unit 150 includes, for each of the plurality of mapping samples, a person who takes the posture of the mapping sample based on a part or all of the calculated plurality of region-specific likelihoods. The overall likelihood indicating the certainty is calculated. Then, the posture determination unit 150 determines that the posture of the mapping sample having the highest overall likelihood is the posture taken by the person included in the analysis target image.
- mapping sample corresponds to a particle in the particle filter
- posture determination process realized by the candidate area determination unit 140 and the posture determination unit 150 corresponds to a particle filter process
- the particle filter is a method of sampling the state space to be estimated with a large number of particles generated according to the system model, calculating the likelihood for each particle, and estimating the state with the weighted average.
- the details of the particle filter processing are described in, for example, Non-Patent Document 2, and will not be described here.
- posture determination unit 150 outputs posture estimation information indicating a posture determined to be taken by a person included in the analysis target image and the input analysis target image to determination result output unit 160. .
- the posture determination unit 150 also feeds back the mapping sample information indicating the mapping sample with the highest overall likelihood to the candidate region determination unit 140 as information indicating the position and posture of the subject in the immediately preceding frame. Details of posture estimation will be described later.
- the candidate area determination unit 140 and the posture determination unit 150 perform particle generation and likelihood calculation using a low-dimensional posture state space in which the dimensions of the posture state space are reduced. Details of the dimension reduction of the posture state space and details of generation of particles using the low-dimensional posture state space will be described later.
- the candidate area determination unit 140 and the posture determination unit 150 efficiently search the state space and estimate the state by repeating the processing of the state space sampling, likelihood calculation, and state estimation. Details of such posture estimation repetition will be described later.
- the determination result output unit 160 outputs the input posture estimation information. Such output includes display of posture estimation information, recording on a recording medium, transmission to another device, and the like.
- the posture estimation information is information indicating a mapping sample of the estimated posture
- the determination result output unit 160 generates an image indicating the mapping sample and displays the image superimposed on the analysis target image.
- the posture estimation apparatus 100 having such a configuration generates particles using a posture state space with reduced dimensions of a human body model corresponding to a variety of postures, and arranges each part by likelihood determination based on image features. Is estimated. Thereby, the posture estimation apparatus 100 can estimate the posture of the person included in the image with higher accuracy and higher speed.
- FIG. 2 is a diagram for explaining an example of a joint base link model.
- the joint base link model (or sports backbone link model) 210 is a two-dimensional skeletal model composed of left and right legs, a torso and a head.
- the joint base link model 210 includes a human head position 220, a waist position (lumbar joint position) 221, a left knee position 222, a right knee position 223, a left ankle position 224, and a right ankle position 225 in an image obtained by photographing a person. , Including an arrangement of six point locations.
- the left and right are not limited to the left and right of a person, but refer to the left and right for distinguishing in FIG.
- the coordinate value of the head position 220 in the XY coordinate system is expressed as (x 0 , y 0 ).
- the coordinate value of the waist position 221 in the XY coordinate system is represented as (x 1 , y 1 ).
- the line segment l 1 connects the head position 220 and the waist position 221, and the line segment l 2 connects the waist position 221 and the left knee position 222.
- Line segment l 3 connects waist position 221 and right knee position 223, and line segment l 4 connects left knee position 222 and left ankle position 224.
- Line segment l 5 connects right knee position 223 and right ankle position 225.
- the length of the line segment l 1 is represented by the symbol s.
- the length of the line segments l 2 to l 5 is given by the ratio l 2 to l 5 with respect to s. That is, the symbols l 2 to l 5 can be used in two ways: as a part name and as a part length.
- the line segments l 1 to l 5 correspond to the head and torso axes, the left upper thigh axis, the upper right thigh axis, the left lower thigh axis, and the right lower thigh axis, respectively.
- the angle (upper body absolute angle) of the line segment l 1 with respect to the reference direction 230 such as the vertical direction is represented by the symbol ⁇ 1 .
- the angles (leg relative angle, relative angle around the hip joint) of the line segments l 2 and l 3 with respect to the line segment l 1 are represented by symbols ⁇ 2 and ⁇ 3 in this order.
- the angle of the line segment 4 with respect to the line segment l 2 (leg relative angle, relative angle around the left knee joint) is represented by the symbol ⁇ 4 .
- Angle of the line segment l 5 for the line segment l 3 is represented by the symbol theta 5.
- angles ⁇ 1 to ⁇ 5 correspond to the tilt of the head and torso, the tilt of the upper right thigh, the tilt of the right lower leg, the tilt of the left upper leg, and the tilt of the left lower leg, respectively.
- the joint base link model 210 includes two sets of coordinate values (x 0 , y 0 ), (x 1 , y 1 ), one distance s, four distance ratios l 2 to l 5 , and five angles ⁇ 1 to It consists of a 14-dimensional state variable (parameter) of ⁇ 5 . That is, a plurality of postures can be defined by changing the value of each state variable of the joint base link model 210. Note that the range of change of the value of each state variable and the step size (hereinafter referred to as “sample condition”) are predetermined for each state variable, and constitute the joint base link model 210.
- the coordinate value (x 0 , y 0 ) of the head position 220 is uniquely determined by the coordinate value (x 1 , y 1 ), the distance s, and the angle ⁇ 1 of the waist position 221. Therefore, the coordinate value (x 0 , y 0 ) of the head position 220 can be omitted.
- the coordinate value (x 1 , y 1 ) of the waist position 221 is represented by the symbol u
- the coordinate value (x 0 , y 0 ) of the head position 220 is represented by the symbol u ′.
- the joint base link model 210 includes a human head region 240, a torso region 241, a left upper thigh region 242, an upper right thigh region 243, a left lower thigh region 244, and a right lower thigh region 245 (hereinafter referred to as “parts”).
- Region is defined as a relative region with respect to the positions 221 to 225. Therefore, by changing the value of each state variable of the joint base link model 210, it is possible to define the relative position of each part in each of a plurality of postures. Further, by fitting the joint base link model 210 to the image, it is possible to define an area occupied by each part in each of a plurality of postures in the image.
- the joint base link model and the part image feature of each part are determined in advance based on a plurality of learning images (template images) obtained by photographing a person and stored in the model information storage unit 110.
- the joint base link model and the part image feature are hereinafter collectively referred to as “model information” as appropriate.
- FIG. 3 is a diagram for explaining an example of the learning of the part image feature.
- a model information generation device (not shown, which may be the posture estimation device 100) that generates model information displays a learning image 250 including a subject 251.
- the operator confirms the displayed learning image 250 while using the pointing device to the learning image 250, the head position 260, the waist position 261, the left knee position 262, the right knee position 263, and the left ankle position 264.
- a plurality of point positions including right ankle position 265.
- positions 260 to 265 correspond to the positions 220 to 225 (see FIG. 2) of the joint base link model 210.
- the designation of the positions 260 to 265 with respect to the learning image 250 is the designation of the positions 220 to 225 of the joint base link model 210, and corresponds to the designation of the state variable of the joint base link model 210.
- the operator diagonally displays the head region 270, the torso region 271, the left upper thigh region 272, the upper right thigh region 273, the left lower thigh region 274, and the right lower thigh region 275 with respect to the learning image 250. It is specified by a rectangle generated by operation.
- the horizontal width of each area is determined.
- the method of designating each area is not limited to the method of designating with a rectangle.
- each region may be automatically designated for each region based on a ratio determined for each length. That is, the areas 270 to 275 may be set based on a predetermined relative position (area range) with respect to the positions 220 to 225.
- the model information generation apparatus extracts (samples) image features such as a color histogram and the number of foreground pixels (for example, the number of pixels other than green as the field color) from each of the set regions 270 to 275. . Then, the model information generating apparatus records the extracted image features and the relative positions (region ranges) with respect to the plurality of positions 220 to 225 of the region in association with the corresponding part identification information.
- samples image features such as a color histogram and the number of foreground pixels (for example, the number of pixels other than green as the field color) from each of the set regions 270 to 275.
- the model information generating apparatus records the extracted image features and the relative positions (region ranges) with respect to the plurality of positions 220 to 225 of the region in association with the corresponding part identification information.
- the model information generation apparatus performs such processing on a plurality of learning images, and accumulates a plurality of image features (and relative positions) for each part. Then, the model information generation apparatus sets the average value for each part of the stored image features (and relative positions) as the part image feature (and relative position) of each part. Then, the image feature (and relative position) of each part is stored in the model information storage unit 110.
- the plurality of learning images are a large number of images taken for various scenes, times, and subjects.
- a part image feature is obtained from a learning image obtained by photographing the person wearing the uniform. It is desirable to learn.
- the state variable vector (posture parameter) x of the joint base link model 210 (see FIG. 2) is expressed by the following equation (1), for example.
- the symbol p j is obtained by principal component analysis (PCA) on learning data of lengths l 2 to l 5 and angles ⁇ 1 to ⁇ 5 obtained from a plurality of (for example, 300) learning images.
- PCA principal component analysis
- the coefficient of the j-th principal component vector P j .
- the top five principal component vectors of the contribution rate are used as the basis vectors of the posture state space.
- the principal component vector P j is a vector in which deviations of lengths l 2 to l 5 and angles ⁇ 1 to ⁇ 5 are arranged, and is represented by, for example, the following expression (3).
- the state variable vector x ' has 12 dimensions, whereas the state variable vector x' has 8 dimensions.
- the symbol theta - 1 represents the average value of the angle theta 1 in the training data.
- the symbol Q is a set of j that satisfies ⁇ j 1 ⁇ 0.
- ⁇ 2 there are an infinite number of solutions of the coefficient p j that satisfies j ⁇ Q in Equation (4). For this reason, it is difficult to uniquely determine the coefficient p j (j ⁇ Q) of each particle.
- the posture estimation apparatus 100 performs a reverse operation on a hyperplane (a plane of an arbitrary dimension) in which a solution exists from two reference positions in a low-dimensional posture state space reduced in dimension by principal component analysis, and is unique on the hyperplane. Generate particles in
- the candidate area determination unit 140 sets initial particles in the low-dimensional posture state section.
- the initial particle is a candidate region of each part for a plurality of predetermined postures for roughly estimating the posture.
- the candidate area determination unit 140 maps the set initial particles on the hyperplane calculated backward from the two reference positions for each posture.
- the hyperplane is expressed by, for example, the following formula (5).
- the symbol c is a constant
- the first expression of Expression (5) represents a hyperplane in the
- the candidate area determination unit 140 obtains a coefficient p j satisfying Equation (5) from the coefficient p ⁇ j of the principal component vector satisfying j ⁇ Q of the sample to be mapped. Then, the candidate area determination unit 140 realizes mapping of the sample to the hyperplane by replacing the coefficient p ⁇ j with the calculated p j .
- the coefficient p j becomes an unstable value as the value of c ⁇ in the denominator on the right side approaches zero.
- the candidate area determination unit 140 excludes the sample from the search target. Further, the candidate area decision unit 140, for each sample, the coordinate values u ⁇ , to calculate the coefficients p j from equation (8) in terms of plus Gaussian noise u ⁇ '. That is, the candidate area determination unit 140 allows a change (error) in two reference positions according to a Gaussian distribution to a particle. Thereby, convergence to the local solution may be avoided and a more reliable arrival to the global optimum solution may be achieved.
- FIG. 4 is a diagram illustrating an example of the operation of the posture estimation apparatus 100.
- step S1010 the image input unit 120 starts video input.
- FIG. 5 is a diagram showing an example of the input video.
- a panoramic video 310 of an American football field is input to the image input unit 120.
- the panoramic image 310 includes a plurality of players 311.
- the reference position setting part 130 displays the image (analysis object image) of the start frame of the input image
- FIG. 6 is a diagram illustrating an example of how two reference positions are set by a drag-and-drop operation.
- the analysis target image 320 shown in FIG. 6 is, for example, an enlarged part of the panoramic video 310 (see FIG. 5).
- the user confirms the head position 322 and the waist position 323 of the player 311 included in the displayed analysis target image 320, and performs a drag and drop (D & D) operation on the analysis target image 320 as indicated by an arrow 324. . That is, the pressing state is started with the head position 322 specified, the specified position is moved to the waist position 323 while maintaining the pressing state, and the pressing state is released at the waist position 323.
- D & D drag and drop
- ⁇ Two reference positions can be easily set by such a drag and drop operation.
- the user sequentially performs a drag-and-drop operation on all of the posture estimation targets, that is, each of the players 311 of the panoramic video 310.
- the reference position setting unit 130 acquires, for each player 311, two reference positions (position 322 and waist position 323) of the set players 311.
- two reference position setting methods for example, a method of simply clicking two points, a method of sliding two points on the touch panel or the like, a method of simultaneously touching two points on the touch panel or the like, two points by gesture
- Various other methods such as a method for designating the image may be adopted.
- step S1030 the candidate area determination unit 140 selects one video frame in order from the start frame.
- step S1040 the candidate area determination unit 140 generates a plurality of samples by randomly changing the state variable based on the joint base link model.
- a sample generated first for a certain frame is appropriately referred to as an “initial sample”.
- each region of the initial sample is appropriately referred to as “initial particle”.
- step S1050 the candidate area determination unit 140 maps the particles of the initial sample on the hyperplane calculated backward from the two set reference positions (head position and waist position).
- FIG. 7 is a diagram showing an example of a particle group when the head position 322 and the waist position 323 are set.
- FIG. 8 is a diagram showing an example of a particle group when only the waist position 323 is set as a reference.
- position judgment part 150 calculates the likelihood as each part area
- step S1070 posture determination unit 150 determines whether the candidate posture satisfies a predetermined end condition.
- the predetermined condition is a condition corresponding to the accuracy of the posture estimation result of the candidate posture being equal to or higher than a predetermined level, or that the accuracy has reached a limit level.
- Posture determination unit 150 proceeds to step S1080 if the candidate posture does not satisfy the end condition (S1070: NO).
- FIG. 9 is a diagram showing an example of a candidate posture estimated from the initial particles.
- posture estimation apparatus 100 determines whether or not such a deviation has occurred using the above-described end condition. Then, when there is a deviation, posture estimation apparatus 100 performs posture estimation again based on the candidate posture.
- step S1080 of FIG. 4 the candidate area determination unit 140 sets particles again on the hyperplane based on the candidate posture, and returns the process to step S1060.
- the particles set in step S1080 are appropriately referred to as “additional particles”.
- steps S1060 and S1070 posture determination unit 150 again performs likelihood calculation, candidate posture determination, and end condition determination for the additional particles.
- Posture estimation apparatus 100 repeats steps S1060 to S1080 until a candidate posture satisfying the end condition is obtained. Then, posture determination unit 150 advances the process to step S1090 when the candidate posture satisfies the end condition (S1070: YES).
- FIG. 10 is a diagram illustrating an example of a candidate posture estimated from the additional particles.
- the position of each particle 330 in the candidate posture is determined by repeating the processing in steps S1060 to S1080 in FIG. 4 to the position of each part in the actual posture of the player 311 included in the analysis target image 320 ( Closer to the region).
- step S1090 the determination result output unit 160 outputs the posture having the highest overall likelihood, that is, the last determined candidate posture as a solution of the posture of the person included in the analysis target image.
- step S1100 the candidate area determination unit 140 determines whether or not the next frame exists.
- the candidate area determination unit 140 returns the process to step S1030.
- posture estimation apparatus 100 performs processing for estimating the posture with respect to a new frame based on the posture estimation result in the immediately preceding frame.
- the position and orientation of each subject in subsequent frames after the start frame are probabilistically estimated based on the image characteristics with reference to the position and orientation of the subject in the immediately preceding frame.
- the candidate area determination unit 140 applies a constant velocity linear motion model to a position space on a person's image under the assumption that the center of gravity of the person moves at a constant speed.
- region determination part 140 employ
- the candidate area determination unit 140 can efficiently generate particles of each subsequent frame.
- the candidate area determination unit 140 ends the series of processes.
- the posture estimation apparatus 100 can estimate the posture (position) of each person at each time on a video including a large number of people, such as a video shot of an American football game. it can. In addition, the posture estimation apparatus 100 can perform the posture estimation with high accuracy based on a simple operation by a user.
- the candidate area determination unit 140 calculates only the total likelihood based on the total value of the part-specific likelihoods of the top four parts having the highest part-specific likelihoods. Based on the above, the candidate posture may be determined.
- the player's body may block part of the other player's body.
- there are many intense contacts such as tackles and blocks, and such shielding often occurs.
- the posture estimation apparatus 100 performs not only forward tracking of video but also reverse tracking of video, compares or integrates both tracking results (posture estimation results), and outputs a final estimation result. Also good.
- the reference position setting unit 130 displays the last frame of the video, for example, and accepts the reference position setting.
- the image size of the video to be analyzed is 5120 ⁇ 720 pixels.
- the size of the player in the video is about 20 ⁇ 60 pixels.
- the forward tracking results and the reverse tracking results of all players were output for the actual video of one play.
- the initial position setting of the backbone link model in the prior art method is the area where the rectangular area of the backbone link model and the player's silhouette overlap by manually manipulating the principal component and size after clicking the head position of the player Was adjusted by maximizing.
- the initial position setting of the joint base link model in the method of the present embodiment is performed by dragging and dropping from the head position to the waist position.
- the upper body of the joint base link model is automatically set to match the silhouette of the player by such setting.
- FIG. 11 is a diagram showing an example of experimental results.
- the horizontal axis represents percentage.
- tracking success rate indicates the percentage of frames that are determined to be tracking success in each frame as a result of performing forward tracking and reverse tracking of all 22 players on the target video.
- Matching success rate indicates the percentage of frames that have been successfully tracked by either forward tracking or reverse tracking.
- Matching semi-success rate indicates the percentage of frames in which either forward tracking or reverse tracking is successful.
- the “matching failure rate” indicates the number of frames that have failed to be tracked in both forward tracking and reverse tracking.
- the method according to the present embodiment using the joint-based link model improves the tracking success rate by 5% and the matching success rate is 9% compared to the conventional method using the backbone link model. % Accuracy has been improved. Moreover, it turned out that the method of this Embodiment becomes lower both in a matching semi-success rate and a matching failure rate. As described above, it has been experimentally found that the posture estimation by the method of the present embodiment can estimate the posture with high accuracy in a state where the load on the operator is further reduced.
- the inventor quantitatively evaluated the accuracy of posture estimation by the posture estimation method of the present embodiment (hereinafter referred to as “proposed method”) using wide area still images of American football.
- a method hereinafter referred to as 1 RPM
- 1 RPM a method that estimates a posture semi-automatically from one reference point (reference position) was used, and the estimation accuracy was compared with the proposed method.
- the only difference from the 1 RPM proposed method is the particle mapping method, and the other 1 RPM attitude estimation procedure is basically the same as the proposed method.
- the ratio S of the area where the rectangle of each part overlaps the part of the target player on the image is visually measured.
- the posture in which S of all parts is 1/3 or more is determined as the correct answer. Of all the parts, the posture in which one or more rectangles (particles) with S of 1/10 or less exist is determined to be an error.
- players who are difficult to make a correct visual inspection in steps (2) and (3) are evaluated. Except for the ambiguous evaluation results, a new player to be evaluated was added.
- the threshold value for S in procedures (2) and (3) has been obtained by another experiment as the minimum value that can be stably analyzed by the behavior analysis system (ABAS).
- the particles generated by the proposed method are as shown in FIG. 7, and the particles generated at 1 RPM are as shown in FIG.
- the proposed method particles are mapped onto the hyperplane in the posture state space, and the search range is reduced. Therefore, compared to the conventional method, the area where the player model is drawn is smaller and particles that are efficient for searching are generated. It was done.
- the correct answer rate for 30 players was only 32.1% with 1PRM, while it was 82.1% with the proposed method. As described above, it was found from experiments that the proposed method can estimate the posture with higher accuracy than 1 RPM.
- the proposed method shows that the player position can be tracked more accurately. I understood. As a result, it was confirmed that the proposed method is effective as an initial posture setting method in the behavior analysis system, and can simplify the manual input work of the user in the behavior analysis system.
- posture estimation apparatus 100 can flexibly express the position and shape of each part even when the posture greatly fluctuates, and a human body model corresponding to more various postures Posture estimation is performed using the joint base link model. Thereby, the posture estimation apparatus 100 can estimate the posture of the person included in the image with higher accuracy.
- the posture estimation apparatus 100 generates particles using the posture state space with reduced dimensions, and estimates the arrangement of each part by likelihood determination based on image features. Thereby, the posture estimation apparatus 100 can estimate the posture of the person included in the image at higher speed (with a low processing load).
- the posture estimation apparatus 100 calculates the overall likelihood after calculating the region-specific likelihood, and performs posture estimation. Thereby, the posture estimation apparatus 100 can perform stable posture estimation even when the human image has partial occlusion.
- posture estimation apparatus 100 accepts the setting of two reference positions by a simple operation called drag and drop, and generates particles on the hyperplane based on the set reference positions. Thereby, the posture estimation apparatus 100 can realize the high-precision posture estimation with a small work load.
- the posture estimation apparatus 100 repeats the process of generating and evaluating particles until the end condition is satisfied. Thereby, the posture estimation apparatus 100 can estimate the posture of the person included in the image with higher accuracy.
- the posture estimation apparatus 100 enables robust posture estimation and human tracking even in a sports image in which a person's posture varies greatly.
- region used for a joint base link model are not limited to the above-mentioned example.
- the point positions used in the joint base link model may not include the left and right ankle positions, or may include the left and right elbow positions and the wrist positions.
- the region of the region may not include the left and right lower leg regions, or may include the left and right upper arm portions and forearm portions, for example.
- a part of the configuration of the posture estimation apparatus 100 may be separated from other parts by being arranged in an external device such as a server on the network.
- posture estimation apparatus 100 needs to have a communication unit for communicating with such an external device.
- the present disclosure can be applied not only to an image of American football but also to an image or image taken of a person such as an image of other sports.
- the posture estimation method of the present disclosure includes an image input step, a reference position setting step, a candidate area determination step, and a posture determination step.
- an analysis target image is input.
- the reference position setting step a plurality of reference positions including a human head position and a waist position are set for the input analysis target image.
- the candidate region determination step the posture of the person is defined by a plurality of point positions (positions) including a head position and a waist position, and a joint base link model defined by arrangement of a plurality of part regions, and a plurality of set reference positions. Based on this, the candidate region candidate region in the analysis target image is determined.
- whether the person included in the analysis target image has the posture based on the part image feature that is the image feature of the part region in the image obtained by photographing the person and the image feature of the determined candidate region Judge whether or not.
- the posture estimation method may include an image display step for displaying the analysis target image and an operation reception step for receiving a drag-and-drop operation on the displayed analysis target image.
- the start point and end point of the drag-and-drop operation are set for the analysis target image as the head position and the waist position, respectively.
- the candidate region determination step may determine a candidate region for each of the plurality of region regions.
- the posture determination step may include a part-specific likelihood calculation step and an overall likelihood evaluation step.
- region-specific likelihood calculation step for each of the plurality of candidate regions, a region-specific likelihood indicating the probability that the candidate region is a corresponding region is calculated.
- overall likelihood evaluation step it is determined whether or not a person included in the analysis target image has the posture described above based on some or all of the plurality of calculated part-specific likelihoods.
- the joint base link model may include a combination of a plurality of state variables that define the arrangement.
- the candidate area determination step includes an initial sample generation step and an initial particle mapping step.
- the initial sample generation step the value of the state variable is changed to determine the relative positional relationship between the plurality of point positions and the plurality of part regions for each of the plurality of postures.
- the initial particle mapping step a plurality of candidate regions are determined based on the determined relative positional relationship and the set reference positions for each of the plurality of postures.
- the posture determination step includes an initial posture estimation step.
- a plurality of postures are obtained by performing a region-specific likelihood calculation step and an overall likelihood evaluation step on the plurality of candidate regions determined in the initial particle mapping step.
- a candidate posture that is a posture that is likely to be taken by a person included in the analysis target image is determined.
- the initial particle mapping step is a super-constraint that is constrained by a plurality of reference positions in a low-dimensional posture state space obtained by reducing the dimensions of the posture state space around a plurality of state variables by principal component analysis. You may determine a candidate area
- the posture estimation method may include an additional candidate region determination step including an additional sample generation step and an additional particle mapping step, and an additional posture estimation step.
- the additional sample generation step the value of the state variable is changed using the candidate posture determined in the initial posture estimation step as a reference, and the relative positional relationship of the additional candidate posture that approximates the candidate posture is determined.
- the additional particle mapping step each additional candidate region of the plurality of region regions in the analysis target image is determined based on the relative positional relationship between the additional candidate postures and the set reference positions.
- a posture that is likely to be taken by a person included in the analysis target image is determined by performing processing of the region-specific likelihood calculation step and the overall likelihood evaluation step on the additional candidate posture.
- the overall likelihood evaluation step in the additional posture estimation step may include a process repetition step, a posture determination step, and a determination result output step.
- the process repetition step it is determined whether or not the plurality of region-specific likelihood values satisfy a predetermined end condition. If the predetermined end condition is not satisfied, an additional candidate region determination step and an additional posture estimation step are performed. The process performed with the additional candidate posture determined immediately before as a reference is repeated.
- the posture determination step when a predetermined predetermined condition is satisfied, it is determined that the finally determined additional candidate posture is a posture taken by a person included in the analysis target image.
- the determination result output step information indicating the determined posture is output.
- the posture estimation device includes a model information storage unit, an image input unit, a reference position setting unit, a candidate area determination unit, and a posture determination unit.
- the model information storage unit defines a posture of a person based on a plurality of point positions (positions) including a head position and a waist position in an image obtained by photographing the person, and a plurality of region regions, and a joint base link model, A part image feature that is an image feature of a part region in an image is stored.
- the image input unit inputs an analysis target image.
- the reference position setting unit sets a plurality of reference positions including a human head position and a waist position for the input analysis target image.
- the candidate area determination unit determines a candidate area of the part area in the analysis target image based on the acquired joint base link model and the set reference positions.
- the posture determination unit determines whether or not a person included in the analysis target image has the posture based on the determined image feature of the candidate region and the acquired partial image feature of the corresponding partial region. .
- the present disclosure is useful as a posture estimation method and a posture estimation device that can estimate a posture of a person included in an image with higher accuracy.
- posture estimation device 110 model information storage unit 120 image input unit 130 reference position setting unit 140 candidate region determination unit 150 posture determination unit 160 determination result output unit 210 joint base link model 220, 260, 322 head position (position) 221,261,323 Waist position (position) 222,262 Left knee position (position) 223,263 Right knee position (position) 224,264 Left ankle position (position) 225,265 Right ankle position (position) 230 Reference directions 240, 270 such as vertical direction Head region (region) 241,271 trunk region (region) 242,272 Left upper thigh region (region) 243,273 Upper right thigh region (region) 244,274 Left lower leg region (region) 245,275 Right lower leg region (region) 250 learning image 251 subject 310 panoramic video 311 player 320 analysis target image 324 arrow 330 particle l 1 , l 2 , l 3 , l 4 , l 5 line segment ⁇ 1 , ⁇ 2 , ⁇ 3 , ⁇ 4 , ⁇ 5 angle
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Multimedia (AREA)
- Human Computer Interaction (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Evolutionary Biology (AREA)
- Probability & Statistics with Applications (AREA)
- Life Sciences & Earth Sciences (AREA)
- Geometry (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Psychiatry (AREA)
- Social Psychology (AREA)
- Image Analysis (AREA)
Abstract
Description
図1は、本実施の形態に係る姿勢推定装置の構成の一例を示すブロック図である。
図2は、関節ベースリンクモデルの一例を説明するための図である。
関節ベースリンクモデル、および、各部位の部位画像特徴は、人を撮影した複数の学習用画像(テンプレート画像)に基づいて予め決定され、モデル情報格納部110に格納されている。関節ベースリンクモデルおよび部位画像特徴は、以下、適宜、「モデル情報」と総称する。
関節ベースリンクモデル210(図2参照)の状態変数ベクトル(姿勢パラメータ)xは、例えば、以下の式(1)で表される。
候補領域決定部140は、低次元姿勢状態区間に、初期パーティクルを設定する。ここで、初期パーティクルとは、姿勢を大まかに推定するための、予め定められた複数の姿勢についての各部位の候補領域である。候補領域決定部140は、姿勢毎に、設定された初期パーティクルを、2つの参照位置から逆算される超平面上に写像する。
次に、姿勢推定装置100の動作について説明する。
次に、姿勢推定装置100を用いて行った実験について説明する。
本開示者は、アメリカンフットボール1試合分の全選手の軌跡データ出力を想定して、実験を行った。アメリカンフットボールは、各チーム11人の合計22人で試合を行う。試合では、両チームが向かい合った静止状態からプレーが始まり、タックル等によりボールの前進が止まることによりプレーが終了する。1プレーの平均時間は、5秒程度であり、1プレーの最大時間は、10数秒程度である。アメリカンフットボールの試合は、このような短いプレーの積み重ねにより進行する。試合時間は60分だが、作戦会議の時間等も含まれるため、実際のプレー時間の合計は11分程度である(非特許文献3参照)。
発明者は、アメリカンフットボールの広域静止画像を用いて、本実施の形態の姿勢推定の手法(以下「提案手法」という)による姿勢推定の精度を定量的に評価した。また、従来手法として、1つの参照点(参照位置)から半自動的に姿勢を推定する手法(以降、1RPMと呼ぶ)を用い、提案手法との推定精度を比較した。1RPMの提案手法と異なる点は、パーティクルの写像方法のみであり、その他の1RPMの姿勢推定の手順は、基本的に、提案手法と同一である。
(2)全部位のSが1/3以上となる姿勢を、正解と判定
(3)全部位のうち、Sが1/10以下となる矩形(パーティクル)が1つ以上存在する姿勢を、誤りと判定
なお、手順(2)、(3)において正誤の目視判定が難しい選手は評価から除き、新たに評価対象選手を追加することにより、曖昧な評価結果を除くようにした。手順(2)、(3)におけるSに対する閾値は、行動解析システム(ABAS)で安定に解析を開始することが可能な最低限の値として、別の実験により求めておいた。
以上のように、本実施の形態に係る姿勢推定装置100は、姿勢が大きく変動している場合でも柔軟に各部位を位置や形状を表現することができ、より多様な姿勢に対応した人体モデルである関節ベースリンクモデルを用いて、姿勢推定を行う。これにより、姿勢推定装置100は、画像に含まれる人の姿勢を、より高精度に推定することができる。
なお、関節ベースリンクモデルに用いられる点位置および部位領域は、上述の例に限定されない。例えば、関節ベースリンクモデルに用いられる点位置は、左右の足首位置を含まなくてもよいし、左右の肘位置や手首位置を含んでいてもよい。また、部位領域は、例えば、左右の下腿部領域を含まなくてもよいし、左右の上腕部や前腕部を含んでもよい。
本開示の姿勢推定方法は、画像入力ステップと、参照位置設定ステップと、候補領域決定ステップと、姿勢判断ステップとを有する。画像入力ステップでは、解析対象画像を入力する。参照位置設定ステップでは、入力された解析対象画像に対して、人の頭位置および腰位置を含む複数の参照位置を設定する。候補領域決定ステップでは、人の姿勢を、頭位置および腰位置を含む複数の点位置(位置)並びに複数の部位領域の配置により定義する関節ベースリンクモデルと、設定された複数の参照位置とに基づいて、解析対象画像における部位領域の候補領域を決定する。姿勢判断ステップでは、人を撮影した画像における部位領域の画像特徴である部位画像特徴と、決定された候補領域の画像特徴とに基づいて、解析対象画像に含まれる人が上記姿勢を取っているか否かを判断する。
110 モデル情報格納部
120 画像入力部
130 参照位置設定部
140 候補領域決定部
150 姿勢判断部
160 判定結果出力部
210 関節ベースリンクモデル
220,260,322 頭位置(位置)
221,261,323 腰位置(位置)
222,262 左膝位置(位置)
223,263 右膝位置(位置)
224,264 左足首位置(位置)
225,265 右足首位置(位置)
230 鉛直方向等の基準方向
240,270 頭部領域(領域)
241,271 胴部領域(領域)
242,272 左上腿部領域(領域)
243,273 右上腿部領域(領域)
244,274 左下腿部領域(領域)
245,275 右下腿部領域(領域)
250 学習用画像
251 被写体
310 パノラマ映像
311 選手
320 解析対象画像
324 矢印
330 パーティクル
l1,l2,l3,l4,l5 線分
θ1,θ2,θ3,θ4,θ5 角度
Claims (14)
- プロセッサが解析対象画像内の人の姿勢を推定する方法であって、
前記プロセッサは、
前記解析対象画像を入力し、
入力された前記解析対象画像に対して、人の頭位置および腰位置を含む複数の参照位置を設定し、
人の姿勢を、頭位置および腰位置を含む複数の点位置並びに複数の部位領域、の配置により定義する関節ベースリンクモデルと、設定された前記複数の参照位置と、に基づいて、前記解析対象画像における前記部位領域の候補領域を決定し、
人を撮影した画像における部位領域の画像特徴である部位画像特徴と、決定された前記候補領域の画像特徴と、に基づいて、前記解析対象画像に含まれる前記人が前記姿勢を取っているか否かを判断する、
姿勢推定方法。 - 前記プロセッサは、
前記解析対象画像を表示し、
表示された前記解析対象画像に対するドラッグアンドドロップ操作を受け付け、
前記ドラッグアンドドロップ操作の開始点および終了点を、それぞれ、前記頭位置および前記腰位置として前記解析対象画像に対して設定することで前記参照位置を設定する、
請求項1に記載の姿勢推定方法。 - 前記プロセッサは、
前記複数の部位領域のそれぞれについて、前記候補領域を決定し、
複数の前記候補領域のそれぞれについて、前記候補領域が対応する前記部位領域であることの確からしさを示す部位別尤度を算出し、
算出された複数の前記部位別尤度の一部または全部に基づいて、前記解析対象画像に含まれる前記人が前記姿勢を取っているか否かを判断する、
請求項1に記載の姿勢推定方法。 - 前記関節ベースリンクモデルは、前記配置を定義する複数の状態変数の組み合わせを含み、
前記プロセッサは、
前記状態変数の値を変化させて、複数の前記姿勢のそれぞれについて、前記複数の点位置および前記複数の部位領域の相対位置関係を決定し、
前記複数の姿勢のそれぞれについて、決定された前記相対位置関係と、設定された前記複数の参照位置と、に基づいて、前記複数の候補領域を決定し、
前記複数の姿勢のそれぞれについて、決定された前記複数の候補領域に対して前記解析対象画像に含まれる人が取っている可能性が高い姿勢である候補姿勢を決定する、
請求項1に記載の姿勢推定方法。 - 前記プロセッサは、
前記複数の状態変数を軸とする姿勢状態空間の次元を主成分分析により削減した低次元姿勢状態空間の、前記複数の参照位置により拘束される超平面を用いて、前記候補領域を決定する、
請求項4に記載の姿勢推定方法。 - 前記プロセッサは、
前記候補姿勢を基準として前記状態変数の値を変化させて、前記候補姿勢に近似する追加候補姿勢の前記相対位置関係を決定し、
前記追加候補姿勢の前記相対位置関係と、設定された前記複数の参照位置と、に基づいて、前記解析対象画像における前記複数の部位領域のそれぞれの追加候補領域を決定し、
前記追加候補領域に対して、前記解析対象画像に含まれる人が取っている可能性が高い姿勢を決定する、
請求項4に記載の姿勢推定方法。 - 前記プロセッサは、
前記複数の部位別尤度の値が、所定の終了条件を満たすか否かを判断し、
前記所定の終了条件が満たされていない場合、前記追加候補領域の決定および前記追加姿勢の決定を、直前に決定された前記追加候補姿勢を基準として行う処理を繰り返し、
前記所定の所定条件が満たされた場合、最後に決定された前記追加候補姿勢を、前記解析対象画像に含まれる人が取っている姿勢であると判定し、
判定された前記姿勢を示す情報を出力する、
請求項6に記載の姿勢推定方法。 - プロセッサを有する姿勢推定装置であって、
前記プロセッサは、
前記解析対象画像を入力し、
入力された前記解析対象画像に対して、人の頭位置および腰位置を含む複数の参照位置を設定し、
人の姿勢を、頭位置および腰位置を含む複数の点位置並びに複数の部位領域、の配置により定義する関節ベースリンクモデルと、設定された前記複数の参照位置と、に基づいて、前記解析対象画像における前記部位領域の候補領域を決定し、
人を撮影した画像における部位領域の画像特徴である部位画像特徴と、決定された前記候補領域の画像特徴と、に基づいて、前記解析対象画像に含まれる前記人が前記姿勢を取っているか否かを判断する、
姿勢推定装置。 - 前記プロセッサは、
前記解析対象画像を表示し、
表示された前記解析対象画像に対するドラッグアンドドロップ操作を受け付け、
前記ドラッグアンドドロップ操作の開始点および終了点を、それぞれ、前記頭位置および前記腰位置として前記解析対象画像に対して設定することで前記参照位置を設定する、
請求項8に記載の姿勢推定装置。 - 前記プロセッサは、
前記複数の部位領域のそれぞれについて、前記候補領域を決定し、
複数の前記候補領域のそれぞれについて、前記候補領域が対応する前記部位領域であることの確からしさを示す部位別尤度を算出し、
算出された複数の前記部位別尤度の一部または全部に基づいて、前記解析対象画像に含まれる前記人が前記姿勢を取っているか否かを判断する、
請求項8に記載の姿勢推定装置。 - 前記関節ベースリンクモデルは、前記配置を定義する複数の状態変数の組み合わせを含み、
前記プロセッサは、
前記状態変数の値を変化させて、複数の前記姿勢のそれぞれについて、前記複数の点位置および前記複数の部位領域の相対位置関係を決定し、
前記複数の姿勢のそれぞれについて、決定された前記相対位置関係と、設定された前記複数の参照位置と、に基づいて、前記複数の候補領域を決定し、
前記複数の姿勢のそれぞれについて、決定された前記複数の候補領域に対して前記解析対象画像に含まれる人が取っている可能性が高い姿勢である候補姿勢を決定する、
請求項8に記載の姿勢推定装置。 - 前記プロセッサは、
前記複数の状態変数を軸とする姿勢状態空間の次元を主成分分析により削減した低次元姿勢状態空間の、前記複数の参照位置により拘束される超平面を用いて、前記候補領域を決定する、
請求項11に記載の姿勢推定装置。 - 前記プロセッサは、
前記候補姿勢を基準として前記状態変数の値を変化させて、前記候補姿勢に近似する追加候補姿勢の前記相対位置関係を決定し、
前記追加候補姿勢の前記相対位置関係と、設定された前記複数の参照位置と、に基づいて、前記解析対象画像における前記複数の部位領域のそれぞれの追加候補領域を決定し、
前記追加候補領域に対して、前記解析対象画像に含まれる人が取っている可能性が高い姿勢を決定する、
請求項11に記載の姿勢推定装置。 - 前記プロセッサは、
前記複数の部位別尤度の値が、所定の終了条件を満たすか否かを判断し、
前記所定の終了条件が満たされていない場合、前記追加候補領域の決定および前記追加姿勢の決定を、直前に決定された前記追加候補姿勢を基準として行う処理を繰り返し、
前記所定の所定条件が満たされた場合、最後に決定された前記追加候補姿勢を、前記解析対象画像に含まれる人が取っている姿勢であると判定し、
判定された前記姿勢を示す情報を出力する、
請求項13に記載の姿勢推定装置。
Priority Applications (3)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
EP15828976.9A EP3179446A4 (en) | 2014-08-06 | 2015-07-29 | Orientation estimation method, and orientation estimation device |
US15/322,911 US20170154441A1 (en) | 2014-08-06 | 2015-07-29 | Orientation estimation method, and orientation estimation device |
JP2016539833A JPWO2016021152A1 (ja) | 2014-08-06 | 2015-07-29 | 姿勢推定方法および姿勢推定装置 |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
JP2014160366 | 2014-08-06 | ||
JP2014-160366 | 2014-08-06 |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2016021152A1 true WO2016021152A1 (ja) | 2016-02-11 |
Family
ID=55263448
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/JP2015/003803 WO2016021152A1 (ja) | 2014-08-06 | 2015-07-29 | 姿勢推定方法および姿勢推定装置 |
Country Status (4)
Country | Link |
---|---|
US (1) | US20170154441A1 (ja) |
EP (1) | EP3179446A4 (ja) |
JP (1) | JPWO2016021152A1 (ja) |
WO (1) | WO2016021152A1 (ja) |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP7067709B1 (ja) | 2022-02-28 | 2022-05-16 | 株式会社ワコール | スキンモデルから骨格に基づく体長を統計的に分析するプログラム、装置及び方法 |
Families Citing this family (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US10540778B2 (en) * | 2017-06-30 | 2020-01-21 | Intel Corporation | System for determining anatomical feature orientation |
JP6424309B1 (ja) * | 2018-08-15 | 2018-11-14 | 裕樹 有光 | 採寸値に基づく3次元モデルを生成するプログラム及び装置 |
CN110102050B (zh) * | 2019-04-30 | 2022-02-18 | 腾讯科技(深圳)有限公司 | 虚拟对象显示方法、装置、电子设备及存储介质 |
Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2005349176A (ja) * | 2004-05-14 | 2005-12-22 | Rise Corp | 顎運動解析方法及び顎運動解析システム |
WO2007010893A1 (ja) * | 2005-07-19 | 2007-01-25 | Nec Corporation | 関節物***置姿勢推定装置及びその方法ならびにプログラム |
JP2012120647A (ja) * | 2010-12-07 | 2012-06-28 | Alpha Co | 姿勢検出装置 |
JP2012190159A (ja) * | 2011-03-09 | 2012-10-04 | Canon Inc | 情報処理装置、情報処理方法及びプログラム |
Family Cites Families (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US7212665B2 (en) * | 2004-11-05 | 2007-05-01 | Honda Motor Co. | Human pose estimation with data driven belief propagation |
-
2015
- 2015-07-29 WO PCT/JP2015/003803 patent/WO2016021152A1/ja active Application Filing
- 2015-07-29 JP JP2016539833A patent/JPWO2016021152A1/ja active Pending
- 2015-07-29 US US15/322,911 patent/US20170154441A1/en not_active Abandoned
- 2015-07-29 EP EP15828976.9A patent/EP3179446A4/en not_active Withdrawn
Patent Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2005349176A (ja) * | 2004-05-14 | 2005-12-22 | Rise Corp | 顎運動解析方法及び顎運動解析システム |
WO2007010893A1 (ja) * | 2005-07-19 | 2007-01-25 | Nec Corporation | 関節物***置姿勢推定装置及びその方法ならびにプログラム |
JP2012120647A (ja) * | 2010-12-07 | 2012-06-28 | Alpha Co | 姿勢検出装置 |
JP2012190159A (ja) * | 2011-03-09 | 2012-10-04 | Canon Inc | 情報処理装置、情報処理方法及びプログラム |
Non-Patent Citations (1)
Title |
---|
See also references of EP3179446A4 * |
Cited By (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP7067709B1 (ja) | 2022-02-28 | 2022-05-16 | 株式会社ワコール | スキンモデルから骨格に基づく体長を統計的に分析するプログラム、装置及び方法 |
JP2023125703A (ja) * | 2022-02-28 | 2023-09-07 | 株式会社ワコール | スキンモデルから骨格に基づく体長を統計的に分析するプログラム、装置及び方法 |
Also Published As
Publication number | Publication date |
---|---|
US20170154441A1 (en) | 2017-06-01 |
JPWO2016021152A1 (ja) | 2017-05-18 |
EP3179446A1 (en) | 2017-06-14 |
EP3179446A4 (en) | 2017-08-09 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
WO2021129064A1 (zh) | 姿态获取方法、关键点坐标定位模型的训练方法和装置 | |
JP5635736B2 (ja) | 情報処理装置および情報処理方法 | |
US8824802B2 (en) | Method and system for gesture recognition | |
JP5483899B2 (ja) | 情報処理装置および情報処理方法 | |
CN110544301A (zh) | 一种三维人体动作重建***、方法和动作训练*** | |
WO2021099778A1 (en) | Real-time system for generating 4d spatio-temporal model of a real world environment | |
CN108647663B (zh) | 基于深度学习和多层次图结构模型的人体姿态估计方法 | |
CN108875586B (zh) | 一种基于深度图像与骨骼数据多特征融合的功能性肢体康复训练检测方法 | |
WO2016021152A1 (ja) | 姿勢推定方法および姿勢推定装置 | |
CN110544302A (zh) | 基于多目视觉的人体动作重建***、方法和动作训练*** | |
WO2017161734A1 (zh) | 通过电视和体感配件矫正人体动作及*** | |
CN114333046A (zh) | 舞蹈动作评分方法、装置、设备和存储介质 | |
CN108416800A (zh) | 目标跟踪方法及装置、终端、计算机可读存储介质 | |
CN105096343B (zh) | 一种移动目标跟踪方法及装置 | |
AU2020436768B2 (en) | Joint rotation inferences based on inverse kinematics | |
WO2021064960A1 (ja) | 運動認識方法、運動認識プログラムおよび情報処理装置 | |
WO2022137450A1 (ja) | 情報処理装置、情報処理方法およびプログラム | |
CN114067953A (zh) | 康复训练方法、***及计算机可读存储介质 | |
Nguyen et al. | Vision-based global localization of points of gaze in sport climbing | |
CN115994944A (zh) | 三维关键点预测方法、训练方法及相关设备 | |
CN115294623B (zh) | 一种人体全身动作捕捉方法、装置、存储介质及终端 | |
Varia et al. | A refined 3d dataset for the analysis of player actions in exertion games | |
Persson | 3D Estimation of Joints for Motion Analysis in Sports Medicine: A study examining the possibility for monocular 3D estimation to be used as motion analysis for applications within sports with the goal to prevent injury and improve sport specific motion | |
CN117351565A (zh) | 一种基于计算机视觉的儿童精细动作评估方法与*** | |
CN115205744A (zh) | 一种针对花样滑冰运动的智能运动辅助方法及装置 |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 15828976 Country of ref document: EP Kind code of ref document: A1 |
|
ENP | Entry into the national phase |
Ref document number: 2016539833 Country of ref document: JP Kind code of ref document: A |
|
REEP | Request for entry into the european phase |
Ref document number: 2015828976 Country of ref document: EP |
|
WWE | Wipo information: entry into national phase |
Ref document number: 15322911 Country of ref document: US |
|
NENP | Non-entry into the national phase |
Ref country code: DE |