WO2024116936A1 - Information processing device and method of estimating neonate's body weight - Google Patents

Information processing device and method of estimating neonate's body weight Download PDF

Info

Publication number
WO2024116936A1
WO2024116936A1 PCT/JP2023/041637 JP2023041637W WO2024116936A1 WO 2024116936 A1 WO2024116936 A1 WO 2024116936A1 JP 2023041637 W JP2023041637 W JP 2023041637W WO 2024116936 A1 WO2024116936 A1 WO 2024116936A1
Authority
WO
WIPO (PCT)
Prior art keywords
model
image
target
subject
head
Prior art date
Application number
PCT/JP2023/041637
Other languages
French (fr)
Japanese (ja)
Inventor
紀功仁 川末
政時 金子
Original Assignee
国立大学法人宮崎大学
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 国立大学法人宮崎大学 filed Critical 国立大学法人宮崎大学
Publication of WO2024116936A1 publication Critical patent/WO2024116936A1/en

Links

Images

Classifications

    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/103—Detecting, measuring or recording devices for testing the shape, pattern, colour, size or movement of the body or parts thereof, for diagnostic purposes
    • A61B5/107—Measuring physical dimensions, e.g. size of the entire body or parts thereof
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/60—Analysis of geometric attributes
    • G06T7/62—Analysis of geometric attributes of area, perimeter, diameter or volume

Definitions

  • the present invention relates to an information processing device and a method for estimating the weight of a newborn baby.
  • Patent Document 1 discloses a technology for measuring the heart rate of a newborn inside an incubator. For some newborns (particularly premature babies), the act of touching to obtain biometric information may be considered an invasive procedure. The technology of Patent Document 1 has the advantage that the heart rate can be measured without touching the newborn, thereby reducing the number of invasive procedures.
  • Patent Document 1 depending on the type of biometric information, it is necessary to touch the newborn to measure it. For example, when measuring weight, it is necessary to touch the newborn.
  • the present invention aims to make it possible to measure biometric information without touching the object, which in conventional technology requires touching the object to measure it.
  • the information processing device of the present invention comprises an acquisition unit that acquires image information showing a three-dimensional image of an object lying on a flat surface captured from above, a recognition unit that recognizes an object image representing the object among the objects displayed in the three-dimensional image, and an estimation unit that uses the object image to estimate the shape of the object, including the missing lower part in the object image, assuming that the object is in contact with the flat surface at its bottom, and calculates the object's physical information using the estimated object shape.
  • biometric information that previously required touching the subject to measure can now be measured without touching the subject.
  • FIG. 2 is a diagram for explaining each component of the information processing system.
  • FIG. 2 is a diagram illustrating the hardware configuration of a computer.
  • FIG. 2 is a functional block diagram of the information processing system.
  • FIG. 11 is a diagram for explaining a specific example of a target image.
  • FIG. 11 is another diagram for explaining a specific example of the target image.
  • FIG. 2 is a diagram for explaining a configuration for generating an object model.
  • FIG. 11 is a diagram for explaining head size approximation processing, etc.
  • 11A and 11B are diagrams for explaining a head model deformation process.
  • 11A and 11B are diagrams for explaining a torso model transformation process.
  • FIG. 11 is a diagram for explaining a target model according to the second embodiment.
  • 13A and 13B are diagrams for explaining ellipse app
  • FIG. 1 is a diagram for explaining each component of the information processing system 1.
  • the information processing system 1 of this embodiment includes a computer 100 and a depth camera 200. The above components are connected so that they can communicate with each other.
  • an incubator is used as the computer 100.
  • a mat is provided inside the incubator.
  • a newborn baby hereinafter "subject S” lies on the upper surface of the mat (hereinafter "flat surface A").
  • Flat surface A is a plane with an approximately rectangular outer edge.
  • the depth camera 200 generates a three-dimensional image (distance image) that includes depth information indicating the distance to the subject.
  • the three-dimensional image may be a point cloud image captured using LIDAR (Light Detection and Ranging, Laser Imaging Detection and Ranging) technology.
  • LIDAR Light Detection and Ranging, Laser Imaging Detection and Ranging
  • the weight or height of the subject S is estimated by the computer 100 by capturing an image of the subject S with the depth camera 200.
  • image information Dg of a three-dimensional image showing the target S is transmitted to the computer 100.
  • the computer 100 uses the image information Dg received from the depth camera 200 to generate a target model M showing the shape (size) of the photographed target S (see FIG. 6 described below).
  • the volume of the target model M described above is estimated as the volume of the photographed target S, and the weight of the target S is estimated by multiplying the estimated result by the density.
  • the height of the target model M is estimated as the height of the photographed target S.
  • an incubator is used as the computer 100, but the computer 100 may be configured separately from the incubator.
  • the depth camera 200 may also have the functions of the computer 100.
  • the incubator may also be equipped with the depth camera 200.
  • the image information Dg is transmitted from the depth camera 200 to the computer 100 via wireless communication
  • the image information D may be transmitted via wired communication.
  • FIG. 2 is a diagram showing the hardware configuration of this embodiment.
  • the computer 100 includes a CPU (Central Processing Unit) 101, a ROM (Read Only Memory) 102, a RAM (Random access memory) 103, a monitor 104, a memory 105, and a communication unit 106.
  • CPU Central Processing Unit
  • ROM Read Only Memory
  • RAM Random access memory
  • the ROM 102 of the computer 100 stores various data including programs in a non-volatile manner.
  • the CPU 101 executes the programs to realize various functions (such as the estimation unit 13) described below.
  • the RAM 103 temporarily stores, for example, various information referenced by the CPU 101 when executing the programs.
  • the monitor 104 displays various information. For example, the monitor 104 displays the biological information of the subject S in an incubator.
  • the memory 105 stores various types of information in a non-volatile manner.
  • a flash memory can be used as the memory 105.
  • the memory 105 stores information necessary for estimating the weight and height of the target S. Specifically, information indicating a head model Mh, a torso model Mb, an arm model Ma, and a leg model Ml for generating a target model M described below is stored in the memory 105.
  • the communication unit 106 receives various types of information from an external device. For example, image information Dg transmitted from the depth camera 200 is received by the communication unit 106.
  • FIG. 3 is a functional block diagram of the information processing system 1 in this embodiment.
  • the information processing system 1 includes an information processing device 10 and an image capturing device 20.
  • the above-mentioned computer 100 functions as the information processing device 10 by executing a program
  • the depth camera 200 functions as the image capturing device 20.
  • image information Dg indicating the captured three-dimensional image G is transmitted from the image capturing device 20 to the information processing device 10.
  • the information processing device 10 includes an acquisition unit 11, a recognition unit 12, an estimation unit 13, a memory unit 14, a calculation unit 15, and a measurement unit 16.
  • the acquisition unit 11 acquires image information Dg indicating a three-dimensional image G obtained by photographing an object S lying on a flat surface A from above. Specifically, the acquisition unit 11 acquires the image information Dg from the photographing device 20.
  • the three-dimensional image G obtained by photographing an object S lying on a flat surface A from above includes a flat image Ga representing the flat surface A in addition to an object image Gs representing the object S (see FIG. 5).
  • the recognition unit 12 recognizes a target image Gs representing a target S among the objects displayed in the three-dimensional image. Specifically, the recognition unit 12 separately recognizes the target image Gs representing the target S and the flat image Ga representing the flat surface A.
  • a technique for recognizing each object in the three-dimensional image G for example, a segmentation technique is preferably adopted. AI (Artificial Intelligence) technology may be adopted for the above segmentation.
  • each object in the three-dimensional image G is segmented using a trained FCN (Fully Convolutional Network) (for example, the technology described in JP 2022-29169 A may be adopted).
  • FCN Fast Convolutional Network
  • the estimation unit 13 estimates the volume of the target S using the target image Gs. Let us assume that a target image Gs that represents the entire target S (both the upper and lower parts) has been captured. In this case, the volume of the target S approximately matches the volume of the target image Gs. Therefore, the volume of the target image Gs that represents the entire target S can be estimated as the volume of the target S.
  • this embodiment employs a configuration that can estimate the volume of the target S with high accuracy even from a target image Gs in which the lower portion of the target S is missing.
  • the estimation unit 13 of this embodiment assumes that the bottom side of the target S is in contact with the flat surface A, and estimates the volume of the target S, including the bottom portion missing in the target image Gs, using the target image Gs.
  • the storage unit 14 stores model information indicating the shape of each model (such as head model Mh) representing each part (such as the head) of the target S (see FIG. 6 described below).
  • the model information in this embodiment is image information indicating a head model Mh that is a model of the head of the target S, image information indicating a torso model Mb that is a model of the torso of the target S, image information indicating an arm model Ma that is a model of the arms of the target S, and image information indicating a leg model Ml that is a model of the legs of the target S.
  • the estimation unit 13 uses the target image Gs to deform the shape of each model (Mh, Mb, Ma, Ml) so that the shape approximates the shape of the part of the target S to which the model corresponds.
  • the estimation unit 13 also uses each model after deformation to generate a target model M that indicates the overall shape of the target S.
  • the estimation unit 13 estimates the volume of the target model M as the volume of the target S.
  • the estimation unit 13 deforms the shapes of the head model Mh and torso model Mb of the models, the estimation unit 13 deforms the shapes of the models assuming that the target S is in contact with the flat surface A on the underside. Specifically, the estimation unit 13 identifies a reference plane Fa that passes through the flat image Ga (see FIG. 5). The estimation unit 13 also deforms the models (Mh, Mb) based on the distance from the target image Gs to the reference plane Fa (Lhz in FIG. 8(b)) (see FIGS. 7(a)(b), 8(a)-(c), and 9(a)(b)), and estimates the volume of the target S using the deformed models. The above configuration will be described in detail later.
  • FIG. 4 is a diagram for explaining a specific example of the configuration up to the generation of the target image Gs.
  • a mat is provided inside the incubator so that the flat surface A is approximately parallel to the horizontal direction.
  • the vertical direction may be referred to as the "z-axis direction” below.
  • the longitudinal direction of the flat surface A may be referred to as the "x-axis direction” and the width direction of the flat surface A may be referred to as the "y-axis direction”.
  • the x-y plane is approximately parallel to the horizontal direction. Note that in this embodiment, it is assumed that the target S lies on the flat surface A with the top of the head facing the x-axis direction, as shown in FIG. 4.
  • a newborn baby lying on a flat surface A is photographed as a target S by the photographing device 20 (depth camera 200).
  • the target S is photographed from above (in the direction of arrow d).
  • the flat surface A is photographed in addition to the target S.
  • image information Dg indicating the three-dimensional image G is transmitted to the information processing device 10 (Sa1 in Figure 4).
  • the center of Figure 4 shows a simulated three-dimensional image G captured by the imaging device 20.
  • the direction in the three-dimensional image space corresponding to the z-axis direction (vertical direction) may be referred to as the "Z-axis direction”.
  • the direction corresponding to the x-axis direction may be referred to as the "X-axis direction”
  • the direction corresponding to the y-axis direction may be referred to as the "Y-axis direction”.
  • the three-dimensional image G indicated by the image information Dg includes a target image Gs and a flat image Ga.
  • the target image Gs represents the target S
  • the flat image Ga represents the flat surface A. Note that, for the sake of explanation, the target image Gs and the flat image Ga are shown in different colors in Figure 4 (the same applies to Figure 5 described below).
  • the information processing device 10 recognizes each image of the three-dimensional image G, including the target image Gs and the flat image Ga, using segmentation technology (Sa2 in FIG. 4). For example, the target image Gs and the flat image Ga are recognized as images showing separate objects.
  • the lower part of Figure 4 shows an excerpt of the target image Gs from the three-dimensional image G.
  • the target image Gs includes a head image Gsh, a chest image Gsc, a waist image Gsw, an arm image Gsa, and a leg image Gsl.
  • the head image Gsh represents the head of the target S.
  • the chest image Gsc represents the chest of the torso of the target S from the neck to the navel
  • the waist image Gsw represents the torso of the target S from the navel to the legs
  • the arm image Gsa represents the arms of the target S
  • the leg image Gsl represents the legs of the target S.
  • Each of the above images is recognized as an image showing a separate object using segmentation techniques.
  • FIG. 5 is a diagram for explaining the reference plane Fa.
  • the reference plane Fa is a plane that passes through the flat surface A, and is used when estimating the volume of the target S.
  • FIG. 5 shows a simulated diagram of a three-dimensional image G (target image Gs, flat image Ga). In the specific example of FIG. 5, it is assumed that the three-dimensional image G is viewed in the X-axis direction. Note that in FIG. 5, the head image Gsh and chest image Gsc are selected from the target image Gs and the other images (such as the arm image Gsa) are omitted.
  • the specific example in FIG. 5 like the specific example in FIG. 4 described above, assumes that subject S lying on flat surface A is photographed from above.
  • the subject image Gs includes an image showing the upper part of subject S, but does not include (is missing) the lower part of subject S.
  • the specific example in FIG. 5 assumes that subject S (newborn baby) is lying on his back on flat surface A.
  • the head image Gsh of subject image Gs represents the face side (eyes, nose, mouth) of subject S, but does not include the back side of subject S's head.
  • the chest image Gsc of subject image Gs represents the abdominal side (chest, navel) of subject S, but does not include the back side of subject S.
  • the information processing device 10 of this embodiment identifies a reference plane Fa from a flat image Ga (Sb in FIG. 5).
  • the flat image Ga is a point cloud image.
  • the reference plane Fa is calculated by plane approximation using the least squares method. That is, the plane that minimizes the sum of the squares of the distances from each point that constitutes the flat image Ga is identified as the reference plane Fa.
  • Figure 5 shows a conceptual diagram of the reference plane Fa.
  • the flat image Ga is perpendicular to the Z-axis direction (vertical direction). Therefore, the reference plane Fa is also perpendicular to the Z-axis direction.
  • the flat image Ga is also parallel to the X-Y plane (horizontal direction). Therefore, the reference plane Fa is also parallel to the X-Y plane.
  • the lower portion of the subject image Gs is missing.
  • the lower portion (Sh, Sb) of the subject S missing from the subject image Gs is indicated by dashed lines.
  • the back of the head Sh missing from the head image Gsh and the back Sb missing from the chest image Gsc of the subject image Gs are indicated by dashed lines.
  • the lower end Pah of the back of the head Sh of subject S typically touches the flat surface A. That is, the lower end Pah of the back of the head Sh of subject S is missing from the target image Gs, but the point in the three-dimensional image space corresponding to the lower end Pah is located on the reference plane Fa.
  • the lower end Pac of the back Sb of subject S typically touches the flat surface A. That is, the lower end Pac of the back Sb of subject S is missing from the target image Gs, but the point in the three-dimensional image space corresponding to the lower end Pac is located on the reference plane Fa.
  • FIG. 6 is a diagram for explaining a specific example of each process for generating the target model M.
  • the target model M represents the overall shape of the target S.
  • the information processing device 10 generates the target model M from the captured target image Gs, and estimates the volume of the target model M as the volume of the target S.
  • model information indicating a head model Mh, a torso model Mb, an arm model Ma, and a leg model Ml is stored in advance in the storage unit 14 of the information processing device 10.
  • the torso model Mb is configured to include a chest model Mc and a waist model Mw.
  • the head model Mh (before deformation) stored in the memory unit 14 represents the average shape of the head of the subject S.
  • the torso model Mb stored in the memory unit 14 represents the average shape of the torso of the subject S
  • the waist model Mw stored in the memory unit 14 represents the average shape of the waist of the subject S
  • the arm model Ma stored in the memory unit 14 represents the average shape of the arms of the subject S
  • the leg model Ml stored in the memory unit 14 represents the average shape of the legs of the subject S.
  • the information processing device 10 When the information processing device 10 acquires a target image Gs from the photographing device 20, it extracts a head image Gsh from the target image Gs. The information processing device 10 then executes a head size approximation process (S1 in FIG. 6) using the head image Gs and the head model Mh. In the head size approximation process, the rough shape (size) of the head model Mh is determined (see FIG. 7(a) described below). After executing the head size approximation process, the information processing device 10 then executes a head model rotation process (S2 in FIG. 6). In the head model rotation process, the orientation of the head model Mh is rotated (adjusted) in accordance with the orientation of the head image Gh (see FIG. 7(b)).
  • the information processing device 10 executes a head model deformation process (S3 in FIG. 6).
  • the shape of the head model Mh is deformed so as to approximate the shape of the head of the target S (to the size of the head of the target S) assuming that the target S is in contact with the flat surface A on the underside (see FIGS. 8(a) to (c) described later).
  • the information processing device 10 executes a torso image rotation process (S4 in FIG. 7).
  • the torso image rotation process the orientation of the torso model Mb and the orientation of the torso image Gb are aligned.
  • the torso model Mb is oriented in the X-axis direction beforehand.
  • the torso image rotation process the torso image Gb is rotated so that it faces the X-axis direction.
  • the torso image Gb is a point cloud image.
  • the information processing device 10 uses principal component analysis to identify the orientation of the torso image Gb, and rotates the torso image Gb so that it is parallel to the X-axis direction.
  • a configuration for adjusting the orientation of a point cloud image for example, the configuration described in JP 2014-44078 A can be adopted.
  • the information processing device 10 executes a torso model deformation process (S5 in FIG. 6).
  • a torso model deformation process As will be described in detail later, in the torso model deformation process, the shape of the torso model Mb is deformed so as to approximate the shape of the torso of the target S, assuming that the target S is in contact with the flat surface A on the underside (see FIGS. 9(a) and (b) described later).
  • the information processing device 10 performs arm model deformation processing (S6 in FIG. 6) to deform the shape of the arm model Ma so as to approximate the shape of the arm of the subject S.
  • the above-mentioned arm image Gsa includes an image representing the area from the shoulder to the elbow of the subject S (hereinafter referred to as the "first arm image Gsa1”) and an image representing the area from the elbow to the fingertips (hereinafter referred to as the "second arm image Gsa2").
  • Each of the above images is distinguished by segmentation technology.
  • the arm model Ma includes a first arm model Ma1 representing the area from the shoulder to the elbow and a second arm model Ma2 representing the area from the elbow to the fingertips.
  • the information processing device 10 deforms the first arm model Ma1 using the first arm image Gsa1. Similarly, in the arm model deformation process, the information processing device 10 deforms the second arm model Ma2 using the second arm image Gsa2.
  • the target image Gs includes an arm image Ga representing the right arm of the target S and an arm image Ga representing the left arm.
  • the arm model Ma includes an arm model Ma representing the right arm of the target S and an arm model Ma representing the left arm.
  • the arm model Ma representing the right arm is deformed using the arm image Ga representing the right arm.
  • the arm model Ma representing the left arm is deformed using the arm image Ga representing the left arm.
  • a specific example of the arm model deformation process will be described later as a modified example.
  • the information processing device 10 performs leg model deformation processing (S7 in FIG. 6) to deform the shape of the leg model Ml so that it approximates the shape of the leg of the target S.
  • the leg image Gsl described above includes an image representing the leg of the target S from the base to the knee (hereinafter referred to as the "first leg image Gsl1") and an image representing the leg from the knee to the toes (hereinafter referred to as the "second leg image Gsl2").
  • Each of the above images is distinguished by segmentation technology.
  • the leg model Ml includes the first leg model Ml1 representing the thigh of the target S and the second leg model Ml2 representing the part from the knee to the toes.
  • the information processing device 10 deforms the first leg model Ml1 using the first leg image Gsl1. Similarly, in the leg model deformation process, the information processing device 10 deforms the second leg model Ml2 using the second leg image Gsl2.
  • the target image Gs includes a leg image Gsl representing the right leg of the target S and a leg image Gsl representing the left leg.
  • the leg model Ml includes a leg model Ml representing the right leg of the target S and a leg model Ml representing the left leg.
  • the leg model Ml representing the right leg is deformed using the leg image Gsl representing the right leg.
  • the leg model Ml representing the left leg is deformed using the leg image Gsl representing the left leg.
  • a specific example of the leg model deformation process will be described later as a modified example.
  • the head model Mh, torso model Mb (chest model Mc, waist model Mw), arm model Ma and arm model Ml stored in the memory unit 14 are deformed according to the captured target image Gs.
  • the information processing device 10 generates the target model M by appropriately combining each model after deformation.
  • the above target model M approximates the actual shape of the target S.
  • FIG. 7(a) is a diagram for explaining the details of the head size approximation process (S1 in FIG. 6).
  • the rough size of the head model Mh is determined.
  • the head image Gsh is approximated as a sphere by the least squares method using the point cloud that constitutes the head image Gsh.
  • the diameter d1 of the spherically approximated head image Gsh is determined.
  • the head model Mh before deformation is approximated as a sphere, and the diameter d2 of the spherically approximated head model Mh is determined.
  • the information processing device 10 also uses the ratio Rd to change the size of the head model Mh. Specifically, the head model Mh before deformation is multiplied by the ratio Rd. According to the above head size approximation process, the size of the head model Mh becomes closer to the actual head size of the target S. However, in this embodiment, by executing a head model deformation process (see Figure 8) in addition to the head size approximation process, the shape of the head model Mh is deformed to closely approximate the shape of the head of the target S with high accuracy.
  • FIG. 7(b) is a diagram for explaining the details of the head model rotation process (S2 in FIG. 6).
  • the orientation of the head model Mh is rotated (adjusted) according to the orientation of the head image Gh. Specifically, if the coordinates of an arbitrary point in the head image Gh (hereinafter “target point Ps") are (xi, yi, zi), and the coordinates of a point in the head model Mh that corresponds to the target point Ps (hereinafter "corresponding point Pm”) (representing the same part of the target S) are (xi', yi', zi'), the information processing device 10 calculates the following rotational movement transformation formula (r, t).
  • the information processing device 10 substitutes a combination of a specific target point Ps and a corresponding point Pm corresponding to the target point Ps.
  • a specific target point Ps and a corresponding point Pm corresponding to the target point Ps have been conventionally known.
  • the technique described in JP 2022-128652 A can be adopted.
  • the information processing device 10 detects the right eye Ps1, left eye Ps2, nose Ps3, mouth Ps4, and left ear Ps5 of the target S in the head image Gs as target points Ps.
  • the information processing device 10 detects the right eye Pm1, left eye Pm2, nose Pm3, mouth Pm4, and left ear Pm5 in the head model Mh as target points Pm.
  • the information processing device 10 calculates elements r and t by substituting multiple sets of combinations of the target point Ps and the corresponding point Pm, and determines the rotational movement transformation equation. Specifically, the total number of elements r to be calculated (9) and the number of elements t (3) is 12. Furthermore, by substituting one set of the target point Ps and the corresponding point Pm, three linear equations containing the elements r and t as coefficients are obtained. In this embodiment, 15 linear equations are obtained by substituting the above-mentioned five sets of combinations of the target point Ps and the corresponding point Pm, and elements r and t are determined using some of the linear equations.
  • the information processing device 10 uses a rotational movement transformation formula to move the corresponding point Pm of the head model Mh to the target point Ps of the target image Gs. In other words, the information processing device 10 rotates the orientation of the head model Mh to the orientation of the head image Gsh.
  • FIGS. 8(a) to 8(c) are diagrams for explaining the details of the head model deformation process.
  • the head model deformation process deforms the shape of the head model Mh so that it closely approximates the shape of the head of the subject S with high accuracy.
  • FIG. 8(a) is a diagram for explaining the reference box Ch and the adjustment box Ci.
  • the reference box Ch is generated according to the size of the target image Gs.
  • the adjustment box Ci is generated according to the size of the head model Mh immediately before the head model deformation process is executed.
  • the ratio Rhi between the size of the reference box Ch and the size of the adjustment box Ci is calculated, and the head model Mh is deformed (enlarged or reduced) according to the ratio Rhi.
  • the reference box body Ch is a rectangular parallelepiped and is composed of a bottom surface Fh1, side surfaces Fh2 to Fh5, and a top surface Fh6.
  • each surface Fh of the reference box body Ch is parallel to either the X-Y plane, the X-Z plane, or the Y-Z plane.
  • the bottom surface Fh1 and top surface Fh6 are parallel to the X-Y plane
  • side surfaces Fh2 and Fh5 are parallel to the Y-Z plane
  • side surfaces Fh3 and Fh4 are parallel to the X-Z plane.
  • the adjustment box Ci is a rectangular parallelepiped and is composed of a bottom surface Fi1, side surfaces Fi2 to Fi5, and a top surface Fi6.
  • each surface Fi is parallel to either the X-Y plane, the X-Z plane, or the Y-Z plane.
  • the bottom surface Fi1 and the top surface Fi6 are parallel to the X-Y plane
  • the side surfaces Fi2 and Fi5 are parallel to the Y-Z plane
  • the side surfaces Fi3 and Fi4 are parallel to the X-Z plane.
  • FIG. 8(b) is a diagram for explaining the configuration for generating the reference box body Ch.
  • the left part of FIG. 8(b) shows a conceptual diagram of the reference box body Ch as viewed from the Z-axis direction (top).
  • the reference box body Ch is generated so that it can store the head image Gsh.
  • FIG. 8(b) shows the head image Gsh stored in the reference box body Ch.
  • the side Fh2 of the reference box body Ch passes through the point Ph2 where the X coordinate of the head image Gsh is maximum.
  • the side Fh5 of the reference box body Ch passes through the point Ph5 where the X coordinate of the head image Gsh is minimum.
  • the side Fh3 of the reference box body Ch passes through the point Ph3 where the Y coordinate of the head image Gsh is maximum
  • the side Fh4 passes through the point Ph4 where the Y coordinate of the head image Gsh is minimum.
  • each side surface Fh (2 to 5) of the reference box body Ch is generated so as to surround the head image Gsh on all four sides when viewed from the Z-axis direction.
  • the right side of Figure 8(b) shows a conceptual diagram of the head image Gsh when viewed from the X-axis direction.
  • the top surface Fh6 of the reference box body Ch passes through point Ph6 of the head image Gsh where the Z coordinate is maximum.
  • each of the side surfaces Fh (2 to 5) and top surface Fh6 of the reference box body Ch is generated so as to be in contact with the target image Gh.
  • the head image Gsh does not include the lower part of the target S
  • the point (bottom end) where the Z coordinate of the head image Gsh is the minimum is different from the actual bottom end (Pah) of the target S. Therefore, if the bottom surface Fh1 of the reference box body Ch were generated at the point in the target image Gsh where the Z coordinate is the minimum, it may not be possible to deform the shape of the head model Mh to closely approximate the shape of the target S with high accuracy. In these cases, there is the inconvenience that the volume and height of the target S cannot be estimated with high accuracy.
  • the applicant has focused on the fact that the lower end Pah of the subject S lying on the flat surface A is located on the flat surface A.
  • FIG. 8(b) the lower part Sh of the head of the subject S that is missing from the head image Gs is shown by a dashed line.
  • the reference plane Fa is generated at a position corresponding to the flat surface A. Therefore, even if the lower part Sh of the subject S is not included in the head image Gs, it can be assumed that the point corresponding to the lower end Pah of the lower part Sh in the three-dimensional image space is on the reference plane Fa.
  • this embodiment employs a configuration in which the bottom surface Fh1 of the reference box body Ch is located on the reference surface Fa.
  • the above configuration is a configuration in which the lower part Sh of the target S is assumed to be in contact with the flat surface A at its lower end Pah, and the volume of the target S including the lower part Sh that is missing in the target image Gs is estimated using the target image Gs.
  • FIG. 8(c) is a diagram for explaining the configuration for generating the adjustment box Ci.
  • FIG. 8(c) shows a conceptual diagram of the adjustment box Ci as viewed from the Z-axis direction (top).
  • the adjustment box Ci is generated so that it can accommodate the head model Mh.
  • the top surface Fi6 of the adjustment box Ci passes through the point Qi6 of the head model Mh where the Z coordinate is maximum.
  • the side surface Fi2 of the adjustment box Ci passes through the point Qi2 of the head model Mh where the X coordinate is maximum, and the side surface Fi5 of the adjustment box Ci passes through the point Qi5 of the head model Mh where the X coordinate is minimum.
  • the side surface Fi3 of the adjustment box Ci passes through the point Pi3 of the head model Mh where the Y coordinate is maximum, and the side surface Fi4 passes through the point Pi4 of the head model Mh where the Y coordinate is minimum.
  • the head model Mh represents the entire target S including the lower part.
  • the bottom surface Fi1 of the adjustment box Ci passes through the point Qi1 of the head model Mh where the Z coordinate is minimum.
  • the information processing device 10 After determining the ratios Rhix, Rhiy, and Rhiz (sometimes collectively referred to as "ratio Rhi"), the information processing device 10 deforms (enlarges or reduces) the head model Mh using the ratio Rhi. Specifically, the information processing device 10 multiplies the head model Mh by Rhix in the X-axis direction, by Rhiy in the Y-axis direction, and by Rhiz in the Z-axis direction. According to the above head model deformation process, even if the lower part of the subject S is not photographed, the reference box body Ch is generated assuming that the lower end Pah of the subject S is in contact with the flat surface A, making it easier to approximate the shape of the head model Mh to the shape of the head of the subject S with high accuracy.
  • FIGS. 9(a) and 9(b) are diagrams for explaining the details of the torso model deformation process.
  • the shape of the torso model Mb is deformed using the reference box Cb and the adjustment box Cc, similar to the head model deformation process described above.
  • the subject S in this embodiment is assumed to be a newborn, and newborns often wear disposable diapers.
  • the waist image Gw in the torso image Gb will have a large area that represents the disposable diaper covering the waist of the newborn.
  • the reference box body Cb in this embodiment is generated based on the chest image Gc in the torso image Gb.
  • FIG. 9(a) is a diagram for explaining the reference box Cb.
  • the reference box Cb is a rectangular parallelepiped and is generated so that it can store the chest image Gc.
  • the reference box Cb is composed of a bottom surface Fb1, side surfaces Fb2 to Fb5, and a top surface Fb6.
  • each of the above surfaces Fb (1 to 6) is parallel to either the X-Y plane, the X-Z plane, or the Y-Z plane.
  • the bottom surface Fb1 and the top surface Fb6 are parallel to the X-Y plane
  • the side surfaces Fb2 and Fb5 are parallel to the Y-Z plane
  • the side surfaces Fb3 and Fb4 are parallel to the X-Z plane.
  • the top surface Fb6 is in contact with point Pb6 in the chest image Gc, where the Z coordinate is maximum.
  • the side surface Fb2 is in contact with point Pb2 in the chest image Gc, where the X coordinate is minimum
  • the side surface Fb5 is in contact with point Pb5 in the chest image Gc, where the X coordinate is maximum.
  • the side surface Fb4 is in contact with point Pb4 in the chest image Gc, where the Y coordinate is minimum
  • the side surface Fb3 is in contact with point Pb3 in the chest image Gc, where the Y coordinate is maximum.
  • a reference box body Cb is generated assuming that the lower end Pac of the subject S's chest is located on flat surface A (similar to the reference box body Ch shown in Figure 8(b) above).
  • the lower portion Sc of the chest of the subject S is missing from the chest image Gc.
  • the bottom surface Fb1 of the reference box body Cb is generated on the reference plane Fa (flat surface A) where the actual lower end portion Pac of the chest is located.
  • FIG. 9(b) is a diagram for explaining the adjustment box Cc.
  • the adjustment box Cc is a rectangular parallelepiped and is generated so that it can house the chest model Mc.
  • the adjustment box Cc is composed of a bottom surface Fc1, side surfaces Fc2 to Fc5, and a top surface Fc6.
  • each of the above surfaces Fc (1 to 6) is parallel to either the X-Y plane, the X-Z plane, or the Y-Z plane.
  • the top surface Fc6 and the bottom surface Fc1 are parallel to the X-Y plane
  • the side surfaces Fc2 and Fc5 are parallel to the Y-Z plane
  • the side surfaces Fc3 and Fc4 are parallel to the X-Z plane.
  • the top face Fc6 is in contact with the point Qc6 of the chest model Mc where the Z coordinate is maximum.
  • the side face Fc2 is in contact with the point Qc2 of the chest model Mc where the X coordinate is minimum
  • the side face Fc5 is in contact with the point Qc5 of the chest model Mc where the X coordinate is maximum.
  • the side face Fc3 is in contact with the point Qc3 of the chest model Mc where the Y coordinate is maximum
  • the side face Fc4 is in contact with the point Qc4 of the chest model Mc where the Y coordinate is minimum.
  • the chest model Mc represents the overall shape of the target S including the lower part. Taking the above into consideration, the bottom face Fc1 of the adjustment box Cc passes through the lower end Qc1 of the chest model Mc where the Z coordinate is minimum.
  • the information processing device 10 After determining the ratios Rbcx, Rbcy, and Rbcz (sometimes collectively referred to as "ratio Rbc"), the information processing device 10 deforms (enlarges and reduces) the torso model Mb (chest model Mc+waist model Mw) based on the ratio Rbc. Specifically, the information processing device 10 multiplies the torso model Mb by Rbcx in the X-axis direction, by Rbcy in the Y-axis direction, and by Rbcz in the Z-axis direction.
  • the shape of the torso model Mb can be deformed to closely approximate the shape of the torso of the subject S with high accuracy by generating a reference box body Cb assuming that the lower end Pac of the chest of the subject S is in contact with the flat surface A.
  • FIG. 10(a) is a diagram for explaining the volume estimation process.
  • the volume estimation process the volume of the target S is estimated.
  • the head model deformation process see FIGS. 8(a)-(c)
  • torso model deformation process see FIGS. 9(a) and (b)
  • arm model deformation process and leg model deformation process are used to deform the head model Mh, torso model Mb, arm model Ma, and leg model Ml so as to approximate the shapes of each part of the target S.
  • the target model M is generated by combining each model after the above deformations.
  • FIG. 10(a) shows a mock-up of the target model M.
  • each cross section Mp shown in FIG. 10(a) is obtained by cutting the target model M parallel to the Y-Z plane while shifting the position in the X-axis direction.
  • FIG. 10(a) shows two adjacent cross sections Mp. As shown in FIG. 10(a), adjacent cross sections Mp are separated by a "distance ⁇ x" in the X-axis direction. The distance ⁇ x is sufficiently small compared to the size of the target image Gs. When the distance ⁇ x is sufficiently small, the volume V of the target model M is calculated (approximated) by the following equation 2. Note that "An” in equation 2 means the area of the n-th cross section Mp from the tip of the target model M in the X-axis direction. Also, "N” in equation 2 means the total number of cross sections Mp.
  • the shape (size) of the target model M is similar to the shape of the actual target S. Therefore, the volume V of the target model M calculated in the volume estimation process can be estimated as the volume of the actual target S.
  • the information processing device 10 multiplies the volume V of the target model M by the average density D to calculate the weight W of the target S.
  • FIG. 10(b) is a diagram for explaining the height estimation process.
  • the height of the target model M is calculated, and the calculation result is estimated as the height of the target S.
  • the information processing device 10 detects predetermined feature points in the target model M in the height estimation process.
  • point Pa corresponding to the top of the head of the target S point Pb corresponding to the shoulder joint, point Pc corresponding to the hip joint, point Pd corresponding to the knee joint, and point Pe corresponding to the ankle are detected in the target model M.
  • the technology described in Japanese Patent No. 6868875 can be adopted.
  • the information processing device 10 When the information processing device 10 detects feature points in the target model M, it calculates the horizontal distance Lab from point Pa corresponding to the top of the head to point Pb corresponding to the shoulder joint. In addition, the information processing device 10 calculates the horizontal distance Lbc from point Pb corresponding to the shoulder joint to point Pc corresponding to the hip joint, the distance Lcd from point Pc corresponding to the hip joint to point Pd corresponding to the knee joint, and the distance Lde from point Pd corresponding to the knee joint to point Pe corresponding to the ankle.
  • the information processing device 10 calculates the sum of the distances Lab, Lbc, Lcd, and Lde as the height H of the target model M.
  • the height H of the target model M described above is estimated as the height of the target S represented by the target model M.
  • the height estimation process can be modified as appropriate.
  • the feature points detected from the target model M are not limited to the above examples, as long as they can measure the height of the target model M.
  • FIG. 11 is a flowchart of the image capture processing executed by the information processing device 10.
  • the image capture device 20 transmits image information Dg indicating the three-dimensional image G (including the target image Gs) to the information processing device 10.
  • the information processing device 10 executes the image capture processing, for example, when it acquires (receives) the image information Dg.
  • the trigger for executing the image capture processing can be changed as appropriate.
  • the information processing device 10 executes an image recognition process (S0).
  • image recognition process each object (including the target S) depicted in the three-dimensional image G indicated by the image information Dg is recognized.
  • the information processing device 10 executes a head size approximation process (S1).
  • head size approximation process the rough shape (size) of the head model Mh is determined (see FIG. 7(a) above).
  • the information processing device 10 also executes a head model rotation process. In the head model rotation process, the orientation of the head model Mh is rotated (adjusted) in accordance with the orientation of the head image Gh (see FIG. 7(b) above).
  • the information processing device 10 executes the head model deformation process (S3).
  • the shape of the head model Mh is deformed so as to closely approximate the shape of the head of the subject S with high accuracy (see Figures 8(a) to (c) above).
  • the information processing device 10 also executes a torso image rotation process (S4).
  • the torso image rotation process the torso image Gsb is rotated so as to be parallel to the X-axis direction.
  • the information processing device 10 executes a torso model deformation process (S5).
  • the shape of the torso model Mb is deformed so as to closely approximate the shape of the torso of the target S with high accuracy (see Figures 9(a) and (b) above).
  • the information processing device 10 executes an arm model deformation process (S6) to deform the shape of the arm model Ma so that it approximates the shape of the arm of the subject S.
  • the information processing device 10 also executes a leg model deformation process (S7) to deform the shape of the leg model Ml so that it approximates the shape of the leg of the subject S.
  • a target model M is generated by combining a head model Mh deformed in the most recent head model deformation process, a torso model Mb deformed in the torso model deformation process, an arm model Ma deformed in the arm model deformation process, and a leg model Ml deformed in the leg model deformation process.
  • the information processing device 10 executes a volume estimation process (S9).
  • the volume estimation process the volume of the object model M generated in the most recent object model generation process is calculated (see FIG. 10(a) above).
  • the information processing device 10 executes a weight estimation process (S10).
  • the weight estimation process the information processing device 10 multiplies the volume of the object model M calculated in the most recent volume estimation process by the average density, and stores the calculation result as the weight W of the object S.
  • the information processing device 10 executes a height estimation process (S11).
  • the height estimation process the height of the target model M generated in the most recent target model generation process is measured, and the measurement result is stored as the height H of the target S (see FIG. 10(b) above).
  • the information processing device 10 executes a biometric information display process (S12).
  • the weight W of the target S calculated in the most recent weight estimation process and the height H of the target S calculated in the height estimation process are displayed, for example, on the monitor 104 of the incubator.
  • the information processing device 10 ends the image capture process.
  • Medical procedures in which a medical tube is inserted into a subject have been known for some time.
  • one such medical procedure is the insertion of a gastric tube into a subject (including patients and newborns) who cannot orally ingest food.
  • the gastric tube is inserted through the nostril, passes through the esophagus, and reaches the inside of the stomach.
  • the appropriate length of the gastric tube varies for each subject. Therefore, traditionally, the appropriate length of the gastric tube has been estimated as the sum of the length from the ear hole to the space between the eyebrows on the subject's surface and the length from the navel to the middle of the xiphoid process.
  • the second embodiment aims to make it possible to estimate the appropriate length of a medical tube (e.g., a gastric tube) to be inserted into the object without touching the object.
  • a medical tube e.g., a gastric tube
  • FIG. 12 is a conceptual diagram of the head model Mh and torso model Mb of the second embodiment. For the sake of explanation, the head model Mh and torso model Mb are shown in combination in FIG. 12. FIG. 12 is also a cross-sectional view of the head model Mh and torso model Mb cut horizontally on the X-Z plane.
  • a target image Gs (three-dimensional image) representing the target S is captured by the imaging device 20.
  • the head model Mh, torso model Mb, arm model Ma and leg model Ml are deformed based on the target image Gs to generate the target model M.
  • the volume of the target model M is estimated as the volume of the target S.
  • the weight of the target S is calculated from the estimated volume.
  • the height of the target model M is estimated as the height of the target S.
  • the target model M in the second embodiment is configured to include a tube model Mt (Mth, Mtb).
  • the tube model Mt represents a medical tube of an appropriate length to be inserted into the target S represented by the combination of the head model Mh and the torso model Mb.
  • the tube model Mt represents the shape of a gastric tube (an example of a medical tube) when inserted into the target S.
  • the gastric tube when inserted into the target S bends in the path from the nostril through the esophagus to the stomach.
  • the tube model Mt bends in the same way as the gastric tube when inserted into the target S.
  • the tube model Mt is provided in a region of the combination of the head model Mh and the torso model Mb that corresponds to the medical tube inserted into the subject S.
  • the tube model Mt representing a gastric tube is located in a region that corresponds to the path from the nostril of the subject S through the esophagus to the inside of the stomach.
  • Figure 12 shows the region St that corresponds to the stomach of the subject S.
  • the end of the tube model Mt on the torso model Mb side is located inside the region St.
  • the tube model Mt includes a first tube model Mth provided inside the head model Mh and a second tube model Mtb provided inside the torso model Mb.
  • a head model deformation process (see FIGS. 8(a) to (c)) is executed, and the head model Mh is deformed to the size of the head of the target S.
  • the first tube model Mth is deformed as an image integrated with the head model Mh.
  • a torso model deformation process see FIGS.
  • the torso model Mb is deformed to the size of the torso of the target S.
  • the second tube model Mtb is deformed as an image integrated with the torso model Mb.
  • a target model M that approximates the shape of the target S is generated.
  • the target model M also includes a tube model Mt (a combination of a first tube model Mth and a second tube model Mtb) that is deformed (stretched) to the length to be inserted into the target S. Therefore, by calculating the length of the tube model Mt in the target model M, the length of the medical tube to be inserted into the target S can be determined.
  • the weight of the target S is estimated by generating the target model M and multiplying the volume of the target model M by the average density.
  • the configuration for estimating the weight of the target S is not limited to the above example.
  • a target model M is generated in the same manner as in the first embodiment described above. Furthermore, in the third embodiment, the length of a specific portion of the target model M is identified as an explanatory variable X for estimating the weight of the target S.
  • the explanatory variable X is the length of a specific portion of the target model M that has a causal relationship with the weight of the target S.
  • n+1 (n is a positive integer) explanatory variables (X0, X1, X2, X3...Xn) are identified. For example, the body length, chest circumference, chest width, chest length, leg length, hand length, etc. of the target model M are identified as explanatory variables X.
  • the information processing device 10 after identifying each explanatory variable X, calculates (estimates) the weight W of the subject S by substituting the explanatory variable X into the following equation 3.
  • the coefficients k0 to kn in equation 3 are determined, for example, by multiple regression analysis.
  • the method of determining the coefficients k0 to kn is not limited to multiple regression analysis.
  • the target model M may include a tube model Mt.
  • the configuration for estimating the weight W of the target S using explanatory variables is not limited to the above example.
  • a machine learning model trained by machine learning using teacher data of the explanatory variables X and the weight W may be used.
  • a machine learning model multiple decision tree models
  • the weight W of the target S is determined (estimated).
  • the arm model deformation process can be modified as appropriate.
  • the arm image Gsa includes a first arm image Gsa1 representing the area from the shoulder to the elbow of the subject S, and a second arm image Gsa2 representing the area from the elbow to the fingertips.
  • the arm model Ma includes a first arm model Ma1 representing the area from the shoulder to the elbow of the subject S, and a second arm model Ma2 representing the area from the elbow to the fingertips.
  • the information processing device 10 approximates the first arm image Gsa1 to a cylinder. Specifically, the information processing device 10 identifies the orientation of the first arm image Gsa1 (hereinafter, "direction vector V") by principal component analysis. The information processing device 10 uses the least squares method to generate a cylinder (hereinafter, "cylinder Cg") whose center line is parallel to the direction vector V and whose side surface approximately overlaps with the first arm image Gsa1.
  • cylinder Cg cylinder whose center line is parallel to the direction vector V and whose side surface approximately overlaps with the first arm image Gsa1.
  • the information processing device 10 generates a cylinder (hereinafter "cylinder Cm") that is similar to the first arm model Ma1 in a similar manner to the method of generating the cylinder Cg from the first arm image Gsa1.
  • the information processing device 10 calculates the ratio "Rg/Rm” by dividing the diameter Rg (thickness) of the bottom surface of the cylinder Cg by the diameter Rm of the bottom surface of the cylinder Cm, and enlarges or reduces the thickness of the first arm model Ma1 according to the ratio "Rg/Rm".
  • the information processing device 10 also calculates the ratio "Tg/Tm" by dividing the height Tg (length) of the cylinder Cg by the height Tm of the cylinder Cm, and enlarges or reduces the length of the first arm model Ma1 according to the ratio "Tg/Tm".
  • the shape of the first arm model Ma1 is deformed so as to approximate the shape of the upper arm of the subject S.
  • the shape of the second arm model Ma2 is deformed using the second arm image Gsa2.
  • the leg model deformation process can be modified as appropriate.
  • the leg image Gsl includes a first leg image Gsl1 representing the leg from the base to the knee, and a second leg image Gsl2 representing the leg from the knee to the toes.
  • Each of the above images is distinguished by segmentation technology.
  • the leg model Ml includes a first leg model Ml1 representing the thigh of the subject S, and a second leg model Ml2 representing the part from the knee to the toes.
  • the information processing device 10 uses the first leg image Gsl1 to deform the shape of the first leg model Ml1 in a manner similar to the manner in which the shape of the first arm model Ma1 in the above-mentioned modified example (1) was deformed. With the above configuration, the shape of the first leg model Ml1 is deformed so as to approximate the shape of the thigh of the target S. Furthermore, the information processing device 10 uses the second leg image Gsl2 to deform the shape of the second leg model Ml2 in a manner similar to the manner in which the shape of the first arm model Ma1 was deformed. With the above configuration, the shape of the second leg model Ml2 is deformed so as to approximate the shape of the leg from the knee to the toes of the target S.
  • the method of determining the reference plane Fa can be changed as appropriate.
  • a configuration in which the imaging device 20 is fixed to an incubator is conceivable.
  • the distance from the imaging device 20 to the flat surface A is constant, and the reference plane Fa through which the flat image Ga passes is common regardless of the three-dimensional image G.
  • the reference plane Fa can be stored in advance in the information processing device 10. Therefore, there is an advantage in that the process for identifying the reference plane Fa from the flat image Ga can be omitted.
  • the head size approximation process estimates the shape (size) of the head of the target S by approximating the head image Gsh as a sphere.
  • the head image Gsh is approximated as an elliptical sphere.
  • FIGS. 13(a) and 13(b) are diagrams for explaining a modified example in which the head image Gsh can be approximated by an ellipse.
  • each cross section of the head image Gsh cut parallel to the Z-X plane and at equal intervals in the Y-axis direction is approximated by an ellipse.
  • the model Mh is then deformed (enlarged or reduced) to the size of the image generated by superimposing the approximated ellipses.
  • FIG. 13(a) shows the head image Gsh before it is approximated by an ellipse.
  • the reference plane Fa flat plane A
  • the shape of the head of the subject S missing from the head image Gsh is shown by a dashed line.
  • subject S newborn baby
  • subject S inside an incubator may have medical tubes attached.
  • part of the head is occluded by subject S's hand (arm) or tube, and the occluded part is missing from the head image Gsh.
  • the head image Gsh below the arm image Gsa is missing.
  • the arm image Gsa may be recognized as part of the head image Gsh.
  • the present modified example has a configuration that suppresses the above inconveniences. The above configuration will be described in detail below.
  • the ellipse with a larger evaluation value V i can be estimated to be more similar to the actual shape of the head of the target S.
  • a case is assumed in which an equation E i of an ellipse is obtained by combining points Pa1 to Pa6 in the head image Gsh, and an equation E i of another ellipse is obtained by combining points Pb1 to Pb6.
  • FIG. 13(a) a case is assumed in which an equation E i of an ellipse is obtained by combining points Pa1 to Pa6 in the head image Gsh, and an equation E i of another ellipse is obtained by combining points Pb1 to Pb6.
  • the evaluation value Vi of n ellipses is calculated, and the ellipse with the largest evaluation value Vi is estimated to be the ellipse that represents the actual shape of the head of the target S.
  • FIG. 13B is a flowchart of the ellipse approximation process.
  • the ellipse approximation process determines an equation E i of an ellipse that approximates the shape of the head of the target S. Specifically, when the ellipse approximation process is started, the information processing device 10 randomly determines six points P in the target image Gs (S101), and calculates an equation E i of an ellipse that passes through each of the points P (S102). The information processing device 10 also obtains the number of points on the target image Gs i that are located on the ellipse of the equation E i as an evaluation value V i (S103).
  • the information processing device 10 determines whether or not the evaluation values V i of n ellipses have been obtained (S104). If the number of ellipses for which the evaluation values V i have been obtained is less than n (S104: No), the information processing device 10 repeatedly executes the above-mentioned steps S101 to S104. On the other hand, if the number of ellipses for which the evaluation value V i has been calculated reaches n (S104: Yes), the information processing device 10 stores the equation E i of the ellipse having the maximum evaluation value V i among the n ellipses (S105) and terminates the ellipse approximation process.
  • the length of the radius of the ellipse obtained by the ellipse approximation process should be half the length from the reference plane Fa to the vertex of the target image Gs (Lc in FIG. 13(a)). Therefore, the difference (error) between the length of the radius of the ellipse obtained by the ellipse approximation process and the length Lc may be calculated, and if the difference exceeds a predetermined threshold, an error may be determined. If an error is determined, the ellipse approximation process may be executed again. Also, the inclination of the ellipse may be calculated from the equation Ei , and the orientation of the ellipse may be adjusted according to the calculation result.
  • the estimated information of the target S is not limited to weight and height.
  • the information processing device 10 (measurement unit 16) may be configured to estimate the head circumference, chest circumference, waist circumference, arm thickness, and leg thickness of the target S.
  • the information processing device 10 may be configured to measure the head circumference, chest circumference, waist circumference, arm thickness, and leg thickness of the generated target model M, and estimate each measurement result as the head circumference, chest circumference, waist circumference, arm thickness, and leg thickness of the target S.
  • the information processing device (10) of this aspect includes an acquisition unit (11) that acquires image information (Dg) showing a three-dimensional image (G) of a target (S) lying on a flat surface (A) taken from above, a recognition unit (12) that recognizes a target image (Gs) that represents the target among the objects displayed in the three-dimensional image, and an estimation unit (13) that estimates the shape of the target, including the lower part missing in the target image, by using the target image, assuming that the lower part of the target is in contact with the flat surface, and calculates the physical information (volume, weight, height, etc.) of the target using the estimated shape of the target.
  • Dg image information showing a three-dimensional image (G) of a target (S) lying on a flat surface (A) taken from above
  • a recognition unit (12) that recognizes a target image (Gs) that represents the target among the objects displayed in the three-dimensional image
  • an estimation unit (13) that estimates the shape of the target, including the lower part missing in the target image, by
  • the physical information of the target can be estimated without touching the target.
  • the physical information of the target can be estimated with high accuracy. Note that "estimating the shape of the target" in the present invention is sufficient as long as the physical information of the target S can be calculated, and is not limited to estimating the shape of the target S in detail.
  • the information processing device of this aspect includes a storage unit (14) that stores model information indicating the shape of a target model (such as a head model), a recognition unit recognizes a flat image (Ga) that represents a flat surface (A) in a three-dimensional image, and an estimation unit identifies a reference surface (Fa) that passes through the flat image, and deforms the model according to the ratio of the distance (Lhz, Lbz) from a target point (Ph6 in FIG. 8(b), Pb6 in FIG.
  • a storage unit (14) that stores model information indicating the shape of a target model (such as a head model), a recognition unit recognizes a flat image (Ga) that represents a flat surface (A) in a three-dimensional image, and an estimation unit identifies a reference surface (Fa) that passes through the flat image, and deforms the model according to the ratio of the distance (Lhz, Lbz) from a target point (Ph6 in FIG. 8(b), Pb
  • the information processing device calculates the weight of the subject as physical information, and can estimate the weight of the subject without touching the subject.
  • the information processing device calculates the height of the target as physical information, and can estimate the height of the target without touching the target.
  • the model includes a tube model (Mt) representing a medical tube (e.g., a gastric tube) inserted into a target, and the estimation unit is capable of deforming the model and the tube model together when deforming models other than the tube model (head model Mh, chest model Mb). According to this aspect, it is possible to estimate the optimal length of the medical tube without touching the target.
  • Mt tube model representing a medical tube
  • the estimation unit is capable of deforming the model and the tube model together when deforming models other than the tube model (head model Mh, chest model Mb).
  • the information processing device (10) of this embodiment includes an acquisition unit (11) for acquiring image information (Dg) showing a three-dimensional image (G) of a subject (S) lying on a flat surface (A) taken from above, a recognition unit (12) for recognizing a subject image (Gs) showing the subject and a flat image (Ga) showing the flat surface among the images showing each object displayed in the three-dimensional image, a storage unit (14) for storing model information showing the shape of the subject model, and an estimation unit (13) for estimating the subject's weight, and the estimation unit specifies a reference plane passing through the flat image, deforms the model according to the ratio of the distance in the vertical axis direction from a subject point in the subject image to the reference plane and the distance in the vertical axis direction from a corresponding point in the model corresponding to the subject point to the lower end of the model, specifies the length of a predetermined part of the deformed model as an explanatory variable for estimating the subject's weight, and estimates
  • the method for estimating the weight of a newborn includes the steps of taking a three-dimensional image of the newborn, estimating the shape of the newborn from the three-dimensional image by a computer (S1 to S9 in FIG. 11), and estimating the weight of the newborn from the estimated shape by a computer (S10 in FIG. 11).
  • a computer S1 to S9 in FIG. 11
  • a computer S10 in FIG. 11
  • the act of touching the newborn to measure physical information with a weighing scale can be considered an invasive act.
  • the weight of the newborn is estimated without touching the newborn, which has the advantage of reducing the number of invasive acts.
  • 10 information processing device
  • 11 acquisition unit
  • 12 recognition unit
  • 13 estimation unit
  • 14 storage unit
  • 15 calculation unit
  • 16 measurement unit
  • 20 imaging device.

Landscapes

  • Health & Medical Sciences (AREA)
  • Physics & Mathematics (AREA)
  • Engineering & Computer Science (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Biophysics (AREA)
  • Heart & Thoracic Surgery (AREA)
  • General Physics & Mathematics (AREA)
  • Dentistry (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Geometry (AREA)
  • Pathology (AREA)
  • Biomedical Technology (AREA)
  • Theoretical Computer Science (AREA)
  • Medical Informatics (AREA)
  • Molecular Biology (AREA)
  • Surgery (AREA)
  • Animal Behavior & Ethology (AREA)
  • General Health & Medical Sciences (AREA)
  • Public Health (AREA)
  • Veterinary Medicine (AREA)
  • Image Processing (AREA)

Abstract

This invention makes it possible to measure a subject's body volume without touching the subject. This invention comprises: an acquisition unit for acquiring image information indicating a three-dimensional image in which a subject lying down on a flat surface has been imaged from above; a recognition unit for recognizing, from among objects displayed in the three-dimensional image, a subject image representing the subject; and an estimation unit for using the subject image to estimate the body volume of the subject, inclusive of a lower portion of the subject that is missing in the subject image, assuming that the lower portion is in contact with the flat surface. The invention also comprises: a step for capturing a three-dimensional image of a neonate; a step in which a computer estimates the neonate's body volume from the three-dimensional image; and a step in which the computer estimates the neonate's body weight from the estimated body volume.

Description

情報処理装置及び新生児の体重の推定方法Information processing device and method for estimating newborn baby's weight
　本発明は、情報処理装置及び新生児の体重の推定方法に関する。 The present invention relates to an information processing device and a method for estimating the weight of a newborn baby.
　従来から、平坦面（例えば、マットの上面）に横たわる対象（例えば、新生児）から各種の生体情報を取得する技術が提案される。例えば、特許文献１には、保育器内部の新生児の心拍数を計測する技術が開示される。新生児（特に早産児）によっては、生体情報を取得するために触れる行為が侵襲的な行為に相当し得る。特許文献１の技術によれば、新生児に触れることなく心拍数を計測できるため、侵襲的な行為の回数を抑制できるという利点がある。 　Technologies have been proposed for obtaining various types of biometric information from a subject (e.g., a newborn) lying on a flat surface (e.g., the top surface of a mat). For example, Patent Document 1 discloses a technology for measuring the heart rate of a newborn inside an incubator. For some newborns (particularly premature babies), the act of touching to obtain biometric information may be considered an invasive procedure. The technology of Patent Document 1 has the advantage that the heart rate can be measured without touching the newborn, thereby reducing the number of invasive procedures.
特開２０２２－７８７０３公報JP 2022-78703 A
　ただし、特許文献１の技術では、生体情報の種類によっては、新生児に触れて計測する必要がある。例えば、体重を計測する場合、新生児に触れる必要がある。以上の事情を考慮して、本発明は、従来技術では対象に触れて計測する必要がある生体情報を、対象に触れずに計測可能にすることを目的とする。 However, with the technology of Patent Document 1, depending on the type of biometric information, it is necessary to touch the newborn to measure it. For example, when measuring weight, it is necessary to touch the newborn. In consideration of the above circumstances, the present invention aims to make it possible to measure biometric information without touching the object, which in conventional technology requires touching the object to measure it.
　以上の課題を解決するために、本発明の情報処理装置は、平坦面に横たわる対象を上方から撮影した３次元画像を示す画像情報を取得する取得部と、３次元画像に表示される各物体のうち対象を表す対象画像を認識する認識部と、対象が下側で平坦面に接しているものとして、対象画像においては欠落した下側部分を含む対象の形状を、当該対象画像を用いて推定する推定部とを具備し、推定した対象の形状を用いて、対象の身体情報を算出する。 In order to solve the above problems, the information processing device of the present invention comprises an acquisition unit that acquires image information showing a three-dimensional image of an object lying on a flat surface captured from above, a recognition unit that recognizes an object image representing the object among the objects displayed in the three-dimensional image, and an estimation unit that uses the object image to estimate the shape of the object, including the missing lower part in the object image, assuming that the object is in contact with the flat surface at its bottom, and calculates the object's physical information using the estimated object shape.
　本発明によれば、従来技術では対象に触れて計測する必要があった生体情報が対象に触れずに計測可能になる。 　With the present invention, biometric information that previously required touching the subject to measure can now be measured without touching the subject.
情報処理システムの各構成を説明するための図である。FIG. 2 is a diagram for explaining each component of the information processing system. コンピュータのハードウェア構成図である。FIG. 2 is a diagram illustrating the hardware configuration of a computer. 情報処理システムの機能ブロック図である。FIG. 2 is a functional block diagram of the information processing system. 対象画像の具体例を説明するための図である。FIG. 11 is a diagram for explaining a specific example of a target image. 対象画像の具体例を説明するための他の図である。FIG. 11 is another diagram for explaining a specific example of the target image. 対象モデルを生成するための構成を説明するための図である。FIG. 2 is a diagram for explaining a configuration for generating an object model. 頭部サイズ概算処理等を説明するための図である。FIG. 11 is a diagram for explaining head size approximation processing, etc. 頭部モデル変形処理を説明するための図である。11A and 11B are diagrams for explaining a head model deformation process. 胴体モデル変形処理を説明するための図である。11A and 11B are diagrams for explaining a torso model transformation process. 体重推定処理および身長推定処理を説明するための図である。FIG. 11 is a diagram for explaining a weight estimation process and a height estimation process. 画像撮影時処理のフローチャートである。4 is a flowchart of a process during image capture. 第２実施形態の対象モデルを説明するための図である。FIG. 11 is a diagram for explaining a target model according to the second embodiment. 変形例における楕円近似処理を説明するための図である。13A and 13B are diagrams for explaining ellipse approximation processing in the modified example.
　図１は、情報処理システム１の各構成を説明するための図である。本実施形態の情報処理システム１は、コンピュータ１００およびデプスカメラ２００を含む。以上の各構成は、通信可能に接続される。本実施形態では、コンピュータ１００として保育器が採用される。図１に示す通り、保育器の内部にはマットが設けられる。マットの上面（以下「平坦面Ａ」）には新生児（以下「対象Ｓ」）が横たわる。平坦面Ａは、外縁が略矩形の平面である。 FIG. 1 is a diagram for explaining each component of the information processing system 1. The information processing system 1 of this embodiment includes a computer 100 and a depth camera 200. The above components are connected so that they can communicate with each other. In this embodiment, an incubator is used as the computer 100. As shown in FIG. 1, a mat is provided inside the incubator. A newborn baby (hereinafter "subject S") lies on the upper surface of the mat (hereinafter "flat surface A"). Flat surface A is a plane with an approximately rectangular outer edge.
　デプスカメラ２００は、被写体までの距離を示す深度情報を含む３次元画像（距離画像）を生成する。例えば、３次元画像としては、ＬＩＤＡＲ（Light Detection and Ranging、Laser Imaging Detection and Ranging）技術により撮影された点群画像が想定される。本実施形態では、デプスカメラ２００で対象Ｓを撮影することにより、当該対象Ｓの体重または身長がコンピュータ１００により推定される。 The depth camera 200 generates a three-dimensional image (distance image) that includes depth information indicating the distance to the subject. For example, the three-dimensional image may be a point cloud image captured using LIDAR (Light Detection and Ranging, Laser Imaging Detection and Ranging) technology. In this embodiment, the weight or height of the subject S is estimated by the computer 100 by capturing an image of the subject S with the depth camera 200.
　具体的には、デプスカメラ２００で対象Ｓを撮影すると、当該対象Ｓを示す３次元画像の画像情報Ｄｇがコンピュータ１００へ送信される。コンピュータ１００は、デプスカメラ２００から受信した画像情報Ｄｇを用いて、撮影された対象Ｓの形状（大きさ）を示す対象モデルＭを生成する（後述の図６参照）。以上の対象モデルＭの体積は、撮影された対象Ｓの体積として推定され、推定結果に密度を掛けることで当該対象Ｓの体重が推定される。また、対象モデルＭの身長は、撮影された対象Ｓの身長として推定される。以上の構成については詳細に後述する。 Specifically, when the target S is photographed by the depth camera 200, image information Dg of a three-dimensional image showing the target S is transmitted to the computer 100. The computer 100 uses the image information Dg received from the depth camera 200 to generate a target model M showing the shape (size) of the photographed target S (see FIG. 6 described below). The volume of the target model M described above is estimated as the volume of the photographed target S, and the weight of the target S is estimated by multiplying the estimated result by the density. In addition, the height of the target model M is estimated as the height of the photographed target S. The above configuration will be described in detail later.
　なお、本実施形態では、コンピュータ１００として保育器を用いたが、コンピュータ１００が保育器とは別の構成としてもよい。また、デプスカメラ２００がコンピュータ１００の機能を兼備する構成としてもよい。また、保育器がデプスカメラ２００を具備する構成としてもよい。画像情報Ｄｇがデプスカメラ２００からコンピュータ１００へ無線通信により送信される構成に替えて、有線通信により画像情報Ｄが送信される構成としてもよい。 In this embodiment, an incubator is used as the computer 100, but the computer 100 may be configured separately from the incubator. The depth camera 200 may also have the functions of the computer 100. The incubator may also be equipped with the depth camera 200. Instead of a configuration in which the image information Dg is transmitted from the depth camera 200 to the computer 100 via wireless communication, the image information D may be transmitted via wired communication.
　図２は、本実施形態のハードウェア構成図である。コンピュータ１００は、ＣＰＵ（Central Processing Unit）１０１、ＲＯＭ（Read Only Memory）１０２、ＲＡＭ（Random access memory）１０３、モニタ１０４、メモリ１０５および通信部１０６を含んで構成される。 Figure 2 is a diagram showing the hardware configuration of this embodiment. The computer 100 includes a CPU (Central Processing Unit) 101, a ROM (Read Only Memory) 102, a RAM (Random access memory) 103, a monitor 104, a memory 105, and a communication unit 106.
　コンピュータ１００のＲＯＭ１０２は、プログラムを含む各種のデータを不揮発的に記憶する。ＣＰＵ１０１は、プログラムを実行することで、後述の各種の機能（推定部１３など）を実現する。ＲＡＭ１０３は、例えばＣＰＵ１０１がプログラムを実行する際に参照する各種の情報を一時的に記憶する。モニタ１０４は、各種の情報を表示する。例えば、保育器における対象Ｓの生体情報をモニタ１０４は表示する。 The ROM 102 of the computer 100 stores various data including programs in a non-volatile manner. The CPU 101 executes the programs to realize various functions (such as the estimation unit 13) described below. The RAM 103 temporarily stores, for example, various information referenced by the CPU 101 when executing the programs. The monitor 104 displays various information. For example, the monitor 104 displays the biological information of the subject S in an incubator.
　メモリ１０５は、各種の情報を不揮発的に記憶する。メモリ１０５としては、例えば、フラッシュメモリが採用され得る。メモリ１０５は、対象Ｓの体重および身長を推定するために必要な情報を記憶する。具体的には、後述する対象モデルＭを生成するための頭部モデルＭｈ、胴体モデルＭｂ、腕部モデルＭａおよび脚部モデルＭｌを示す情報がメモリ１０５に記憶される。通信部１０６は、外部装置から各種の情報を受信する。例えば、デプスカメラ２００から送信された画像情報Ｄｇが通信部１０６により受信される。 The memory 105 stores various types of information in a non-volatile manner. For example, a flash memory can be used as the memory 105. The memory 105 stores information necessary for estimating the weight and height of the target S. Specifically, information indicating a head model Mh, a torso model Mb, an arm model Ma, and a leg model Ml for generating a target model M described below is stored in the memory 105. The communication unit 106 receives various types of information from an external device. For example, image information Dg transmitted from the depth camera 200 is received by the communication unit 106.
　図３は、本実施形態における情報処理システム１の機能ブロック図である。図３に示す通り、情報処理システム１は、情報処理装置１０および撮影装置２０を含む。例えば、上述のコンピュータ１００がプログラムを実行することで情報処理装置１０として機能し、デプスカメラ２００が撮影装置２０として機能する。上述した通り、撮影された３次元画像Ｇを示す画像情報Ｄｇが撮影装置２０から情報処理装置１０へ送信される。 FIG. 3 is a functional block diagram of the information processing system 1 in this embodiment. As shown in FIG. 3, the information processing system 1 includes an information processing device 10 and an image capturing device 20. For example, the above-mentioned computer 100 functions as the information processing device 10 by executing a program, and the depth camera 200 functions as the image capturing device 20. As described above, image information Dg indicating the captured three-dimensional image G is transmitted from the image capturing device 20 to the information processing device 10.
　図３に示す通り、情報処理装置１０は、取得部１１、認識部１２、推定部１３、記憶部１４、算出部１５および計測部１６を含む。取得部１１は、平坦面Ａに横たわる対象Ｓを上方から撮影した３次元画像Ｇを示す画像情報Ｄｇを取得する。具体的には、取得部１１は、撮影装置２０から画像情報Ｄｇを取得する。詳細には後述するが、平坦面Ａに横たわる対象Ｓを上方から撮影した３次元画像Ｇには、当該対象Ｓを表す対象画像Ｇｓに加え、平坦面Ａを表す平坦画像Ｇａが含まれる（図５参照）。 As shown in FIG. 3, the information processing device 10 includes an acquisition unit 11, a recognition unit 12, an estimation unit 13, a memory unit 14, a calculation unit 15, and a measurement unit 16. The acquisition unit 11 acquires image information Dg indicating a three-dimensional image G obtained by photographing an object S lying on a flat surface A from above. Specifically, the acquisition unit 11 acquires the image information Dg from the photographing device 20. As will be described in detail later, the three-dimensional image G obtained by photographing an object S lying on a flat surface A from above includes a flat image Ga representing the flat surface A in addition to an object image Gs representing the object S (see FIG. 5).
　認識部１２は、３次元画像に表示される各物体のうち対象Ｓを表す対象画像Ｇｓを認識する。具体的には、対象Ｓを表す対象画像Ｇｓと平坦面Ａを表す平坦画像Ｇａとを認識部１２は別々に認識する。３次元画像Ｇの各物体を認識するための技術としては、例えば、セグメンテーションの技術が好適に採用される。以上のセグメンテーションにはＡＩ（Artificial Intelligence）技術が採用され得る。例えば、学習済みのＦＣＮ（Fully Convolutional Networks：全層畳み込みネットワーク）を用いて、３次元画像Ｇにおける各物体がセグメンテーションされる（例えば、特開２０２２－２９１６９号公報に記載の技術が採用され得る）。 The recognition unit 12 recognizes a target image Gs representing a target S among the objects displayed in the three-dimensional image. Specifically, the recognition unit 12 separately recognizes the target image Gs representing the target S and the flat image Ga representing the flat surface A. As a technique for recognizing each object in the three-dimensional image G, for example, a segmentation technique is preferably adopted. AI (Artificial Intelligence) technology may be adopted for the above segmentation. For example, each object in the three-dimensional image G is segmented using a trained FCN (Fully Convolutional Network) (for example, the technology described in JP 2022-29169 A may be adopted).
　推定部１３は、対象画像Ｇｓを用いて対象Ｓの体積を推定する。仮に、対象Ｓの全体（上側部分と下側部分との双方）を表す対象画像Ｇｓが撮影された場合を想定する。以上の場合、当該対象Ｓの体積は当該対象画像Ｇｓの体積と略一致する。したがって、対象Ｓの全体を表す対象画像Ｇｓの体積を当該対象Ｓの体積として推定できる。 The estimation unit 13 estimates the volume of the target S using the target image Gs. Let us assume that a target image Gs that represents the entire target S (both the upper and lower parts) has been captured. In this case, the volume of the target S approximately matches the volume of the target image Gs. Therefore, the volume of the target image Gs that represents the entire target S can be estimated as the volume of the target S.
　ただし、実際は、平坦面Ａに横たわる対象Ｓを上方から撮影した場合、当該対象の上側部分のみが撮影され下側部分（平坦面Ａ側）は撮影されないのが通常である（後述の図５参照）。したがって、対象画像Ｇｓにおいて対象Ｓの下側部分が欠損する。以上の対象画像Ｇｓの体積は、対象Ｓの実際の体積として用いることができない。 However, in reality, when an image of a subject S lying on a flat surface A is taken from above, it is normal that only the upper part of the subject is photographed and the lower part (on the flat surface A side) is not photographed (see FIG. 5 described below). Therefore, the lower part of the subject S is missing from the subject image Gs. The volume of the subject image Gs described above cannot be used as the actual volume of the subject S.
　以上の事情を考慮して、本実施形態では、対象Ｓの下側部分が欠損した対象画像Ｇｓからであっても、当該対象Ｓの体積を高精度に推定できる構成を採用した。具体的には、本実施形態の推定部１３は、対象Ｓが下側で平坦面Ａに接しているものとして、対象画像Ｇｓにおいては欠落した下側部分を含む対象Ｓの体積を、当該対象画像Ｇｓを用いて推定する。 In consideration of the above circumstances, this embodiment employs a configuration that can estimate the volume of the target S with high accuracy even from a target image Gs in which the lower portion of the target S is missing. Specifically, the estimation unit 13 of this embodiment assumes that the bottom side of the target S is in contact with the flat surface A, and estimates the volume of the target S, including the bottom portion missing in the target image Gs, using the target image Gs.
　より具体的には、記憶部１４は、対象Ｓの各部分（頭部など）を表す各モデル（頭部モデルＭｈなど）の形状を示すモデル情報を記憶する（後述の図６参照）。本実施形態のモデル情報は、対象Ｓの頭部のモデルである頭部モデルＭｈを示す画像情報、対象Ｓの胴体のモデルである胴体モデルＭｂを示す画像情報、対象Ｓの腕部のモデルである腕部モデルＭａを示す画像情報および対象Ｓの脚部のモデルである脚部モデルＭｌを示す画像情報である。 More specifically, the storage unit 14 stores model information indicating the shape of each model (such as head model Mh) representing each part (such as the head) of the target S (see FIG. 6 described below). The model information in this embodiment is image information indicating a head model Mh that is a model of the head of the target S, image information indicating a torso model Mb that is a model of the torso of the target S, image information indicating an arm model Ma that is a model of the arms of the target S, and image information indicating a leg model Ml that is a model of the legs of the target S.
　推定部１３は、対象画像Ｇｓを用いて、各モデル（Ｍｈ、Ｍｂ、Ｍａ、Ｍｌ）の形状を、当該モデルが対応する対象Ｓの部分の形状に近似する様に変形する。また、推定部１３は、変形後の各モデルを用いて、対象Ｓの全体の形状を示す対象モデルＭを生成する。推定部１３は、当該対象モデルＭの体積を対象Ｓの体積と推定する。 The estimation unit 13 uses the target image Gs to deform the shape of each model (Mh, Mb, Ma, Ml) so that the shape approximates the shape of the part of the target S to which the model corresponds. The estimation unit 13 also uses each model after deformation to generate a target model M that indicates the overall shape of the target S. The estimation unit 13 estimates the volume of the target model M as the volume of the target S.
　また、推定部１３は、各モデルのうち頭部モデルＭｈおよび胴体モデルＭｂの形状を変形する際に、対象Ｓが下側で平坦面Ａに接しているものとして、当該モデルの形状を変形する。具体的には、推定部１３は、平坦画像Ｇａを通る基準面Ｆａを特定する（図５参照）。また、推定部１３は、対象画像Ｇｓから基準面Ｆａまでの距離（図８（ｂ）のＬｈｚ）に基づいて、モデル（Ｍｈ、Ｍｂ）を変形し（図７（ａ）（ｂ）、図８（ａ）～（ｃ）、図９（ａ）（ｂ）参照）、変形後のモデルを用いて、対象Ｓの体積を推定する。以上の構成については、詳細に後述する。 When the estimation unit 13 deforms the shapes of the head model Mh and torso model Mb of the models, the estimation unit 13 deforms the shapes of the models assuming that the target S is in contact with the flat surface A on the underside. Specifically, the estimation unit 13 identifies a reference plane Fa that passes through the flat image Ga (see FIG. 5). The estimation unit 13 also deforms the models (Mh, Mb) based on the distance from the target image Gs to the reference plane Fa (Lhz in FIG. 8(b)) (see FIGS. 7(a)(b), 8(a)-(c), and 9(a)(b)), and estimates the volume of the target S using the deformed models. The above configuration will be described in detail later.
　算出部１５は、推定部１３が推定した対象Ｓの体積を用いて、当該対象Ｓの体重を算出する。具体的には、情報処理装置１０は、平均的な対象Ｓの密度（以下「平均密度Ｄ」）を予め記憶する。算出部１５は、推定部１３が推定した対象Ｓの体積Ｖに平均密度Ｄを掛けて当該対象Ｓの体重Ｗを算出する（Ｗ＝Ｖ×Ｄ）。計測部１６は、推定部１３が変形したモデルを用いて、対象Ｓの身長を計測する。具体的には、計測部１６は、対象モデルＭにおける所定箇所の長さを対象Ｓの身長として推定する（図１０参照）。 The calculation unit 15 calculates the weight of the target S using the volume of the target S estimated by the estimation unit 13. Specifically, the information processing device 10 pre-stores the average density of the target S (hereinafter, "average density D"). The calculation unit 15 multiplies the volume V of the target S estimated by the estimation unit 13 by the average density D to calculate the weight W of the target S (W = V x D). The measurement unit 16 measures the height of the target S using the model deformed by the estimation unit 13. Specifically, the measurement unit 16 estimates the length of a specified part of the target model M as the height of the target S (see Figure 10).
　図４は、対象画像Ｇｓが生成されるまでの構成の具体例を説明するための図である。本実施形態では、平坦面Ａが水平方向と略水平になる様に、保育器の内部にマットが設けられる。以下、説明のため、鉛直方向を「ｚ軸方向」と記載する場合がある。また、平坦面Ａの長手方向を「ｘ軸方向」と記載し、平坦面Ａの幅方向を「ｙ軸方向」と記載する場合がある。以上の場合、ｘ－ｙ平面は水平方向と略平行になる。なお、本実施形態では、図４に示す通り、頭頂部がｘ軸方向を向くように対象Ｓが平坦面Ａに横たわる場合を想定する。 FIG. 4 is a diagram for explaining a specific example of the configuration up to the generation of the target image Gs. In this embodiment, a mat is provided inside the incubator so that the flat surface A is approximately parallel to the horizontal direction. For the sake of explanation, the vertical direction may be referred to as the "z-axis direction" below. The longitudinal direction of the flat surface A may be referred to as the "x-axis direction" and the width direction of the flat surface A may be referred to as the "y-axis direction". In the above cases, the x-y plane is approximately parallel to the horizontal direction. Note that in this embodiment, it is assumed that the target S lies on the flat surface A with the top of the head facing the x-axis direction, as shown in FIG. 4.
　本実施形態では、図４の上側部分に示す通り、平坦面Ａに横たわる新生児が対象Ｓとして撮影装置２０（デプスカメラ２００）により撮影される。図４の具体例では、対象Ｓの上側から（矢印ｄの方向へ）対象Ｓを撮影した場合を想定する。以上の場合、対象Ｓに加え平坦面Ａが撮影される。撮影装置２０により３次元画像Ｇが撮影されると、当該３次元画像Ｇを示す画像情報Ｄｇが情報処理装置１０へ送信される（図４のＳａ１）。 In this embodiment, as shown in the upper part of Figure 4, a newborn baby lying on a flat surface A is photographed as a target S by the photographing device 20 (depth camera 200). In the specific example of Figure 4, it is assumed that the target S is photographed from above (in the direction of arrow d). In the above case, the flat surface A is photographed in addition to the target S. When a three-dimensional image G is photographed by the photographing device 20, image information Dg indicating the three-dimensional image G is transmitted to the information processing device 10 (Sa1 in Figure 4).
　図４の中央部分には、撮影装置２０により撮影された３次元画像Ｇの模擬図が示される。以下、説明のため、ｚ軸方向（鉛直方向）に対応する３次元画像空間における方向を「Ｚ軸方向」と記載する場合がある。同様に、ｘ軸方向に対応する方向を「Ｘ軸方向」と記載し、ｙ軸方向に対応する方向を「Ｙ軸方向」と記載する場合がある。図４に示す通り、画像情報Ｄｇが示す３次元画像Ｇには、対象画像Ｇｓおよび平坦画像Ｇａが含まれる。対象画像Ｇｓは、対象Ｓを表し平坦画像Ｇａは平坦面Ａを表す。なお、図４において、説明のため、対象画像Ｇｓの色彩および平坦画像Ｇａの色彩を相違させて示す（後述の図５においても同様）。 The center of Figure 4 shows a simulated three-dimensional image G captured by the imaging device 20. Hereinafter, for the sake of explanation, the direction in the three-dimensional image space corresponding to the z-axis direction (vertical direction) may be referred to as the "Z-axis direction". Similarly, the direction corresponding to the x-axis direction may be referred to as the "X-axis direction", and the direction corresponding to the y-axis direction may be referred to as the "Y-axis direction". As shown in Figure 4, the three-dimensional image G indicated by the image information Dg includes a target image Gs and a flat image Ga. The target image Gs represents the target S, and the flat image Ga represents the flat surface A. Note that, for the sake of explanation, the target image Gs and the flat image Ga are shown in different colors in Figure 4 (the same applies to Figure 5 described below).
　情報処理装置１０は、対象画像Ｇｓおよび平坦画像Ｇａを含む３次元画像Ｇの各画像をセグメンテーションの技術により認識する（図４のＳａ２）。例えば、対象画像Ｇｓと平坦画像Ｇａとは別々の物体を示す画像として認識される。 The information processing device 10 recognizes each image of the three-dimensional image G, including the target image Gs and the flat image Ga, using segmentation technology (Sa2 in FIG. 4). For example, the target image Gs and the flat image Ga are recognized as images showing separate objects.
　図４の下側部分には、３次元画像Ｇのうち対象画像Ｇｓが抜粋して示される。図４に示す通り、対象画像Ｇｓは、頭部画像Ｇｓｈ、胸部画像Ｇｓｃ、腰部画像Ｇｓｗ、腕部画像Ｇｓａおよび脚部画像Ｇｓｌを含む。頭部画像Ｇｓｈは対象Ｓの頭部を表す。同様に、胸部画像Ｇｓｃは対象Ｓの胴体のうち首から臍までの胸部を表し、腰部画像Ｇｓｗは対象Ｓの胴体のうち臍から脚側を表し、腕部画像Ｇｓａは対象Ｓの腕部を表し、脚部画像Ｇｓｌは対象Ｓの脚部を表す。以上の各画像は、セグメンテーションの技術により別々の物体を示す画像として認識される。 The lower part of Figure 4 shows an excerpt of the target image Gs from the three-dimensional image G. As shown in Figure 4, the target image Gs includes a head image Gsh, a chest image Gsc, a waist image Gsw, an arm image Gsa, and a leg image Gsl. The head image Gsh represents the head of the target S. Similarly, the chest image Gsc represents the chest of the torso of the target S from the neck to the navel, the waist image Gsw represents the torso of the target S from the navel to the legs, the arm image Gsa represents the arms of the target S, and the leg image Gsl represents the legs of the target S. Each of the above images is recognized as an image showing a separate object using segmentation techniques.
　図５は、基準面Ｆａを説明するための図である。詳細には後述するが基準面Ｆａは、平坦面Ａを通る平面であり、対象Ｓの体積を推定する際に用いられる。図５には、３次元画像Ｇ（対象画像Ｇｓ、平坦画像Ｇａ）の模擬図が示される。図５の具体例では、３次元画像ＧをＸ軸方向へ見た場合を想定する。なお、図５において、対象画像Ｇｓのうち頭部画像Ｇｓｈおよび胸部画像Ｇｓｃを抜粋して示し、他の画像（腕部画像Ｇｓａなど）は省略して示す。 FIG. 5 is a diagram for explaining the reference plane Fa. Although details will be described later, the reference plane Fa is a plane that passes through the flat surface A, and is used when estimating the volume of the target S. FIG. 5 shows a simulated diagram of a three-dimensional image G (target image Gs, flat image Ga). In the specific example of FIG. 5, it is assumed that the three-dimensional image G is viewed in the X-axis direction. Note that in FIG. 5, the head image Gsh and chest image Gsc are selected from the target image Gs and the other images (such as the arm image Gsa) are omitted.
　図５の具体例は、上述の図４の具体例と同様に、平坦面Ａに横たわる対象Ｓを上方から撮影した場合を想定する。以上の場合、対象画像Ｇｓには、対象Ｓのうち上側部分を表す画像が含まれる一方、対象Ｓの下側部分が含まれない（欠落する）。例えば、図５の具体例では、対象Ｓ（新生児）が平坦面Ａに仰向けに横たわる場合を想定する。以上の場合、対象画像Ｇｓの頭部画像Ｇｓｈは、対象Ｓの顔側（目、鼻、口）を表す一方、対象Ｓの後頭部側が含まれない。また、対象画像Ｇｓの胸部画像Ｇｓｃは、対象Ｓの腹部側（胸、臍）を表す一方、対象Ｓの背中側が含まれない。 The specific example in FIG. 5, like the specific example in FIG. 4 described above, assumes that subject S lying on flat surface A is photographed from above. In the above case, the subject image Gs includes an image showing the upper part of subject S, but does not include (is missing) the lower part of subject S. For example, the specific example in FIG. 5 assumes that subject S (newborn baby) is lying on his back on flat surface A. In the above case, the head image Gsh of subject image Gs represents the face side (eyes, nose, mouth) of subject S, but does not include the back side of subject S's head. In addition, the chest image Gsc of subject image Gs represents the abdominal side (chest, navel) of subject S, but does not include the back side of subject S.
　本実施形態の情報処理装置１０は、平坦画像Ｇａから基準面Ｆａを特定する（図５のＳｂ）。具体的には、平坦画像Ｇａは点群画像である。本実施形態では、最小二乗法を用いた平面近似により基準面Ｆａを算出する。すなわち、平坦画像Ｇａを構成する各点からの距離の二乗の総和が最小になる平面が基準面Ｆａとして特定される。 The information processing device 10 of this embodiment identifies a reference plane Fa from a flat image Ga (Sb in FIG. 5). Specifically, the flat image Ga is a point cloud image. In this embodiment, the reference plane Fa is calculated by plane approximation using the least squares method. That is, the plane that minimizes the sum of the squares of the distances from each point that constitutes the flat image Ga is identified as the reference plane Fa.
　図５には、基準面Ｆａの概念図が示される。なお、上述した通り、平坦画像ＧａはＺ軸方向（鉛直方向）に対して垂直である。したがって、基準面ＦａもＺ軸方向に対して垂直になる。また、平坦画像ＧａはＸ－Ｙ平面（水平方向）と平行になる。したがって、基準面ＦａもＸ－Ｙ平面と平行になる。 Figure 5 shows a conceptual diagram of the reference plane Fa. As mentioned above, the flat image Ga is perpendicular to the Z-axis direction (vertical direction). Therefore, the reference plane Fa is also perpendicular to the Z-axis direction. The flat image Ga is also parallel to the X-Y plane (horizontal direction). Therefore, the reference plane Fa is also parallel to the X-Y plane.
　上述した通り、対象Ｓを上側から撮影した場合、対象画像Ｇｓの下側部分が欠落する。図５には、対象画像Ｇｓから欠落した対象Ｓの下側部分（Ｓｈ、Ｓｂ）が破線で示される。具体的には、対象画像Ｇｓのうち頭部画像Ｇｓｈから欠落した後頭部Ｓｈ、および、胸部画像Ｇｓｃから欠落した背中部Ｓｂが破線で示される。 As mentioned above, when the subject S is photographed from above, the lower portion of the subject image Gs is missing. In FIG. 5, the lower portion (Sh, Sb) of the subject S missing from the subject image Gs is indicated by dashed lines. Specifically, the back of the head Sh missing from the head image Gsh and the back Sb missing from the chest image Gsc of the subject image Gs are indicated by dashed lines.
　平坦面Ａに対象Ｓ（新生児）が横たわる場合、当該対象Ｓの後頭部Ｓｈの下端部Ｐａｈが平坦面Ａに接するのが通常である。すなわち、対象Ｓの後頭部Ｓｈの下端部Ｐａｈは対象画像Ｇｓからは欠落するが、下端部Ｐａｈに対応する３次元画像空間における点は基準面Ｆａに位置する。同様に、平坦面Ａに対象Ｓが横たわる場合、当該対象Ｓの背中部Ｓｂの下端部Ｐａｃが平坦面Ａに接するのが通常である。すなわち、対象Ｓの背中部Ｓｂの下端部Ｐａｃは対象画像Ｇｓからは欠落するが、下端部Ｐａｃに対応する３次元画像空間における点は基準面Ｆａに位置する。 When subject S (newborn baby) lies on flat surface A, the lower end Pah of the back of the head Sh of subject S typically touches the flat surface A. That is, the lower end Pah of the back of the head Sh of subject S is missing from the target image Gs, but the point in the three-dimensional image space corresponding to the lower end Pah is located on the reference plane Fa. Similarly, when subject S lies on flat surface A, the lower end Pac of the back Sb of subject S typically touches the flat surface A. That is, the lower end Pac of the back Sb of subject S is missing from the target image Gs, but the point in the three-dimensional image space corresponding to the lower end Pac is located on the reference plane Fa.
　図６は、対象モデルＭを生成するための各処理の具体例を説明するための図である。上述した通り、対象モデルＭは、対象Ｓの全体の形状を示す。情報処理装置１０は、撮影した対象画像Ｇｓから対象モデルＭを生成し、当該対象モデルＭの体積を対象Ｓの体積として推定する。 FIG. 6 is a diagram for explaining a specific example of each process for generating the target model M. As described above, the target model M represents the overall shape of the target S. The information processing device 10 generates the target model M from the captured target image Gs, and estimates the volume of the target model M as the volume of the target S.
　図６に示す通り、情報処理装置１０の記憶部１４には頭部モデルＭｈ、胴体モデルＭｂ、腕部モデルＭａおよび脚部モデルＭｌを示すモデル情報が予め記憶される。胴体モデルＭｂは、胸部モデルＭｃおよび腰部モデルＭｗを含んで構成される。 As shown in FIG. 6, model information indicating a head model Mh, a torso model Mb, an arm model Ma, and a leg model Ml is stored in advance in the storage unit 14 of the information processing device 10. The torso model Mb is configured to include a chest model Mc and a waist model Mw.
　記憶部１４が記憶する（変形前の）頭部モデルＭｈは、対象Ｓの頭部の平均的な形状を表す。同様に、記憶部１４が記憶する胴体モデルＭｂは、対象Ｓの胴体の平均的な形状を表し、記憶部１４が記憶する腰部モデルＭｗは、対象Ｓの腰部の平均的な形状を表し、記憶部１４が記憶する腕部モデルＭａは、対象Ｓの腕部の平均的な形状を表し、記憶部１４が記憶する脚部モデルＭｌは、対象Ｓの脚部の平均的な形状を表す。 The head model Mh (before deformation) stored in the memory unit 14 represents the average shape of the head of the subject S. Similarly, the torso model Mb stored in the memory unit 14 represents the average shape of the torso of the subject S, the waist model Mw stored in the memory unit 14 represents the average shape of the waist of the subject S, the arm model Ma stored in the memory unit 14 represents the average shape of the arms of the subject S, and the leg model Ml stored in the memory unit 14 represents the average shape of the legs of the subject S.
　情報処理装置１０は、撮影装置２０から対象画像Ｇｓを取得すると、当該対象画像Ｇｓから頭部画像Ｇｓｈを抽出する。また、情報処理装置１０は、頭部画像Ｇｓおよび頭部モデルＭｈを用いて頭部サイズ概算処理（図６のＳ１）を実行する。頭部サイズ概算処理では、頭部モデルＭｈの大まかな形状（大きさ）が決定される（後述の図７（ａ）参照）。また、情報処理装置１０は、頭部サイズ概算処理を実行した後に、頭部モデル回転処理（図６のＳ２）を実行する。頭部モデル回転処理では、頭部モデルＭｈの向きが頭部画像Ｇｈの向きに応じて回転（調整）される（図７（ｂ）参照）。 When the information processing device 10 acquires a target image Gs from the photographing device 20, it extracts a head image Gsh from the target image Gs. The information processing device 10 then executes a head size approximation process (S1 in FIG. 6) using the head image Gs and the head model Mh. In the head size approximation process, the rough shape (size) of the head model Mh is determined (see FIG. 7(a) described below). After executing the head size approximation process, the information processing device 10 then executes a head model rotation process (S2 in FIG. 6). In the head model rotation process, the orientation of the head model Mh is rotated (adjusted) in accordance with the orientation of the head image Gh (see FIG. 7(b)).
　その後、情報処理装置１０は、頭部モデル変形処理（図６のＳ３）を実行する。詳細には後述するが、頭部モデル変形処理において、対象Ｓが下側で平坦面Ａに接しているものとして、対象Ｓの頭部の形状と近似する様に（対象Ｓの頭部の大きさに）頭部モデルＭｈの形状が変形される（後述の図８（ａ）～（ｃ）参照）。 Then, the information processing device 10 executes a head model deformation process (S3 in FIG. 6). As will be described in detail later, in the head model deformation process, the shape of the head model Mh is deformed so as to approximate the shape of the head of the target S (to the size of the head of the target S) assuming that the target S is in contact with the flat surface A on the underside (see FIGS. 8(a) to (c) described later).
　情報処理装置１０は、胴体画像回転処理（図７のＳ４）を実行する。胴体画像回転処理では、胴体モデルＭｂの向きと胴体画像Ｇｂの向きとが揃えられる。本実施形態の胴体モデルＭｂは予めＸ軸方向を向いている。胴体画像回転処理では、胴体画像ＧｂがＸ軸方向を向くように回転される。具体的には、胴体画像Ｇｂは点群画像である。情報処理装置１０は、主成分分析を用いて、胴体画像Ｇｂの方向を特定し、Ｘ軸方向に平行となる様に当該胴体画像Ｇｂを回転させる。点群画像の向きを調整する構成としては、例えば、特開２０１４－４４０７８号公報に記載の構成が採用され得る。 The information processing device 10 executes a torso image rotation process (S4 in FIG. 7). In the torso image rotation process, the orientation of the torso model Mb and the orientation of the torso image Gb are aligned. In this embodiment, the torso model Mb is oriented in the X-axis direction beforehand. In the torso image rotation process, the torso image Gb is rotated so that it faces the X-axis direction. Specifically, the torso image Gb is a point cloud image. The information processing device 10 uses principal component analysis to identify the orientation of the torso image Gb, and rotates the torso image Gb so that it is parallel to the X-axis direction. As a configuration for adjusting the orientation of a point cloud image, for example, the configuration described in JP 2014-44078 A can be adopted.
　胴体画像回転処理を実行した後に、情報処理装置１０は、胴体モデル変形処理（図６のＳ５）を実行する。詳細には後述するが、胴体モデル変形処理において、対象Ｓが下側で平坦面Ａに接しているものとして、対象Ｓの胴体の形状と近似する様に胴体モデルＭｂの形状が変形される（後述の図９（ａ）（ｂ）参照）。 After executing the torso image rotation process, the information processing device 10 executes a torso model deformation process (S5 in FIG. 6). As will be described in detail later, in the torso model deformation process, the shape of the torso model Mb is deformed so as to approximate the shape of the torso of the target S, assuming that the target S is in contact with the flat surface A on the underside (see FIGS. 9(a) and (b) described later).
　情報処理装置１０は、腕部モデル変形処理（図６のＳ６）により、対象Ｓの腕部の形状に近似する様に腕部モデルＭａの形状を変形させる。上述の腕部画像Ｇｓａは、対象Ｓの肩から肘までを表す画像（以下「第１腕部画像Ｇｓａ１」）と肘から指先までを表す画像（以下「第２腕部画像Ｇｓａ２」）とが含まれる。以上の各画像はセグメンテーションの技術により区別される。また、腕部モデルＭａは、図６に示す通り、肩から肘までを表す第１腕部モデルＭａ１と肘から指先までを表す第２腕部モデルＭａ２とが含まれる。 The information processing device 10 performs arm model deformation processing (S6 in FIG. 6) to deform the shape of the arm model Ma so as to approximate the shape of the arm of the subject S. The above-mentioned arm image Gsa includes an image representing the area from the shoulder to the elbow of the subject S (hereinafter referred to as the "first arm image Gsa1") and an image representing the area from the elbow to the fingertips (hereinafter referred to as the "second arm image Gsa2"). Each of the above images is distinguished by segmentation technology. Furthermore, as shown in FIG. 6, the arm model Ma includes a first arm model Ma1 representing the area from the shoulder to the elbow and a second arm model Ma2 representing the area from the elbow to the fingertips.
　情報処理装置１０は、腕部モデル変形処理において、第１腕部画像Ｇｓａ１を用いて第１腕部モデルＭａ１を変形する。同様に、情報処理装置１０は、腕部モデル変形処理において、第２腕部画像Ｇｓａ２を用いて第２腕部モデルＭａ２を変形する。なお、対象画像Ｇｓは、対象Ｓの右腕を表す腕部画像Ｇａと左腕を表す腕部画像Ｇａとを含む。また、腕部モデルＭａは、対象Ｓの右腕を表す腕部モデルＭａと左腕を表す腕部モデルＭａとを含む。腕部モデル変形処理では、右腕を表す腕部画像Ｇａを用いて右腕を表す腕部モデルＭａが変形される。同様に、左腕を表す腕部画像Ｇａを用いて左腕を表す腕部モデルＭａが変形される。なお、腕部モデル変形処理の具体例については、変形例として後述する。 In the arm model deformation process, the information processing device 10 deforms the first arm model Ma1 using the first arm image Gsa1. Similarly, in the arm model deformation process, the information processing device 10 deforms the second arm model Ma2 using the second arm image Gsa2. The target image Gs includes an arm image Ga representing the right arm of the target S and an arm image Ga representing the left arm. The arm model Ma includes an arm model Ma representing the right arm of the target S and an arm model Ma representing the left arm. In the arm model deformation process, the arm model Ma representing the right arm is deformed using the arm image Ga representing the right arm. Similarly, the arm model Ma representing the left arm is deformed using the arm image Ga representing the left arm. A specific example of the arm model deformation process will be described later as a modified example.
　情報処理装置１０は、脚部モデル変形処理（図６のＳ７）により、対象Ｓの脚部の形状に近似する様に脚部モデルＭｌの形状を変形させる。上述の脚部画像Ｇｓｌは、対象Ｓの脚のつけ根から膝までを表す画像（以下「第１脚部画像Ｇｓｌ１」）と膝から爪先までを表す画像（以下「第２脚部画像Ｇｓｌ２」）とが含まれる。以上の各画像はセグメンテーションの技術により区別される。また、図６に示す通り、脚部モデルＭｌは、対象Ｓの太腿を表す第１脚部モデルＭｌ１と膝より爪先側を表す第２脚部モデルＭｌ２とを含む。 The information processing device 10 performs leg model deformation processing (S7 in FIG. 6) to deform the shape of the leg model Ml so that it approximates the shape of the leg of the target S. The leg image Gsl described above includes an image representing the leg of the target S from the base to the knee (hereinafter referred to as the "first leg image Gsl1") and an image representing the leg from the knee to the toes (hereinafter referred to as the "second leg image Gsl2"). Each of the above images is distinguished by segmentation technology. Furthermore, as shown in FIG. 6, the leg model Ml includes the first leg model Ml1 representing the thigh of the target S and the second leg model Ml2 representing the part from the knee to the toes.
　情報処理装置１０は、脚部モデル変形処理において、第１脚部画像Ｇｓｌ１を用いて第１脚部モデルＭｌ１を変形する。同様に、情報処理装置１０は、脚部モデル変形処理において、第２脚部画像Ｇｓｌ２を用いて第２脚部モデルＭｌ２を変形する。なお、対象画像Ｇｓは、対象Ｓの右脚を表す脚部画像Ｇｓｌと左脚を表す脚部画像Ｇｓｌとを含む。また、脚部モデルＭｌは、対象Ｓの右脚を表す脚部モデルＭｌと左脚を表す脚部モデルＭｌとを含む。脚部モデル変形処理では、右脚を表す脚部画像Ｇｓｌを用いて右脚を表す脚部モデルＭｌが変形される。同様に、左脚を表す脚部画像Ｇｓｌを用いて左脚を表す脚部モデルＭｌが変形される。なお、脚部モデル変形処理の具体例については、変形例として後述する。 In the leg model deformation process, the information processing device 10 deforms the first leg model Ml1 using the first leg image Gsl1. Similarly, in the leg model deformation process, the information processing device 10 deforms the second leg model Ml2 using the second leg image Gsl2. The target image Gs includes a leg image Gsl representing the right leg of the target S and a leg image Gsl representing the left leg. The leg model Ml includes a leg model Ml representing the right leg of the target S and a leg model Ml representing the left leg. In the leg model deformation process, the leg model Ml representing the right leg is deformed using the leg image Gsl representing the right leg. Similarly, the leg model Ml representing the left leg is deformed using the leg image Gsl representing the left leg. A specific example of the leg model deformation process will be described later as a modified example.
　以上の説明から理解される通り、記憶部１４に記憶される頭部モデルＭｈ、胴体モデルＭｂ（胸部モデルＭｃ、腰部モデルＭｗ）、腕部モデルＭａおよび腕部モデルＭｌは、撮影された対象画像Ｇｓに応じて変形される。情報処理装置１０は、変形後の各モデルを適宜に組合せて対象モデルＭを生成する。以上の対象モデルＭは、対象Ｓの実際の形状に近似する。以下、上述の頭部モデルＭｈを変形するための構成（図７（ａ）（ｂ）および図８（ａ）～（ｃ）参照）、および、胴体モデルＭｂを変形するための構成（図９（ａ）（ｂ）参照）を詳細に説明する。 As can be understood from the above explanation, the head model Mh, torso model Mb (chest model Mc, waist model Mw), arm model Ma and arm model Ml stored in the memory unit 14 are deformed according to the captured target image Gs. The information processing device 10 generates the target model M by appropriately combining each model after deformation. The above target model M approximates the actual shape of the target S. Below, the configuration for deforming the above-mentioned head model Mh (see Figures 7(a)(b) and Figures 8(a)-(c)) and the configuration for deforming the torso model Mb (see Figures 9(a)(b)) will be described in detail.
　図７（ａ）は、頭部サイズ概算処理（図６のＳ１）の詳細を説明するための図である。上述した通り、頭部サイズ概算処理では、頭部モデルＭｈの大まかな大きさが決定される。具体的には、図７に示す通り、頭部画像Ｇｓｈを構成する点群を用いて最小二乗法により当該頭部画像Ｇｓｈを球近似する。また、球近似した頭部画像Ｇｓｈの直径ｄ１を求める。同様に、変形前の頭部モデルＭｈを球近似し、球近似した頭部モデルＭｈの直径ｄ２を求める。 FIG. 7(a) is a diagram for explaining the details of the head size approximation process (S1 in FIG. 6). As described above, in the head size approximation process, the rough size of the head model Mh is determined. Specifically, as shown in FIG. 7, the head image Gsh is approximated as a sphere by the least squares method using the point cloud that constitutes the head image Gsh. In addition, the diameter d1 of the spherically approximated head image Gsh is determined. Similarly, the head model Mh before deformation is approximated as a sphere, and the diameter d2 of the spherically approximated head model Mh is determined.
　情報処理装置１０は、球近似した頭部画像Ｇｓｈの直径ｄ１と球近似した頭部モデルＭｈの直径ｄ２との比率Ｒｄ（Ｒｄ＝ｄ１／ｄ２）を求める。また、情報処理装置１０は、比率Ｒｄを用いて頭部モデルＭｈの大きさを変更する。具体的には、変形前の頭部モデルＭｈを比率Ｒｄ倍する。以上の頭部サイズ概算処理によれば、頭部モデルＭｈの大きさが実際の対象Ｓの頭部の大きさに近くなる。ただし、本実施形態では、頭部サイズ概算処理に加え頭部モデル変形処理（図８参照）を実行することで、対象Ｓの頭部の形状に高精度に近似する様に頭部モデルＭｈの形状が変形される。 The information processing device 10 obtains the ratio Rd (Rd = d1/d2) between the diameter d1 of the spherically approximated head image Gsh and the diameter d2 of the spherically approximated head model Mh. The information processing device 10 also uses the ratio Rd to change the size of the head model Mh. Specifically, the head model Mh before deformation is multiplied by the ratio Rd. According to the above head size approximation process, the size of the head model Mh becomes closer to the actual head size of the target S. However, in this embodiment, by executing a head model deformation process (see Figure 8) in addition to the head size approximation process, the shape of the head model Mh is deformed to closely approximate the shape of the head of the target S with high accuracy.
　図７（ｂ）は、頭部モデル回転処理（図６のＳ２）の詳細を説明するための図である。上述した通り、頭部モデル回転処理では、頭部画像Ｇｈの向きに応じて頭部モデルＭｈの向きが回転（調整）される。具体的には、頭部画像Ｇｈにおける任意の点（以下「対象点Ｐｓ」）の座標を（ｘｉ，ｙｉ，ｚｉ）、当該対象点Ｐｓに対応する（対象Ｓにおける同じ部分を表す）頭部モデルＭｈの点（以下「対応点Ｐｍ」）の座標を（ｘｉ´，ｙｉ´，ｚｉ´）とした場合、情報処理装置１０は以下の回転移動変換式（ｒ、ｔ）を求める。 FIG. 7(b) is a diagram for explaining the details of the head model rotation process (S2 in FIG. 6). As described above, in the head model rotation process, the orientation of the head model Mh is rotated (adjusted) according to the orientation of the head image Gh. Specifically, if the coordinates of an arbitrary point in the head image Gh (hereinafter "target point Ps") are (xi, yi, zi), and the coordinates of a point in the head model Mh that corresponds to the target point Ps (hereinafter "corresponding point Pm") (representing the same part of the target S) are (xi', yi', zi'), the information processing device 10 calculates the following rotational movement transformation formula (r, t).
（数１）
Figure JPOXMLDOC01-appb-I000001
 (Equation 1)
Figure JPOXMLDOC01-appb-I000001
　情報処理装置１０は、回転移動変換式（ｒ、ｔ）を求めるために、具体的な対象点Ｐｓと当該対象点Ｐｓに対応する対応点Ｐｍとの組合せを代入する。具体的には、顔における各特徴点（瞳など）を顔画像から検出する技術が従来から知られている。以上の技術としては、例えば、特開２０２２－１２８６５２号公報に記載の技術が採用され得る。情報処理装置１０は、頭部画像Ｇｓにおける対象Ｓの右目Ｐｓ１、左目Ｐｓ２、鼻Ｐｓ３、口Ｐｓ４および左耳Ｐｓ５を対象点Ｐｓとして検出する。同様に、情報処理装置１０は、頭部モデルＭｈにおける右目Ｐｍ１、左目Ｐｍ２、鼻Ｐｍ３、口Ｐｍ４および左耳Ｐｍ５を対象点Ｐｍとして検出する。 In order to obtain the rotational movement transformation formula (r, t), the information processing device 10 substitutes a combination of a specific target point Ps and a corresponding point Pm corresponding to the target point Ps. Specifically, techniques for detecting each feature point (such as pupils) in a face from a face image have been conventionally known. As the above technique, for example, the technique described in JP 2022-128652 A can be adopted. The information processing device 10 detects the right eye Ps1, left eye Ps2, nose Ps3, mouth Ps4, and left ear Ps5 of the target S in the head image Gs as target points Ps. Similarly, the information processing device 10 detects the right eye Pm1, left eye Pm2, nose Pm3, mouth Pm4, and left ear Pm5 in the head model Mh as target points Pm.
　情報処理装置１０は、対象点Ｐｓと対応点Ｐｍとの組合せを複数組代入することで、要素ｒおよび要素ｔを算出し、回転移動変換式を求める。具体的には、算出すべき要素ｒの個数（９個）と要素ｔの個数（３個）との合計は１２個である。また、対象点Ｐｓと対応点Ｐｍとを１組代入することで、要素ｒと要素ｔとを係数として含む３個の一次方程式が得られる。本実施形態では、上述の５組の対象点Ｐｓと対応点Ｐｍとの組合せを代入して１５個の一次方程式を得て、当該一次方程式の一部を用いて要素ｒと要素ｔとを求める。 The information processing device 10 calculates elements r and t by substituting multiple sets of combinations of the target point Ps and the corresponding point Pm, and determines the rotational movement transformation equation. Specifically, the total number of elements r to be calculated (9) and the number of elements t (3) is 12. Furthermore, by substituting one set of the target point Ps and the corresponding point Pm, three linear equations containing the elements r and t as coefficients are obtained. In this embodiment, 15 linear equations are obtained by substituting the above-mentioned five sets of combinations of the target point Ps and the corresponding point Pm, and elements r and t are determined using some of the linear equations.
　情報処理装置１０は、回転移動変換式を用いて、頭部モデルＭｈの対応点Ｐｍを対象画像Ｇｓの対象点Ｐｓに移動させる。すなわち、情報処理装置１０は、頭部モデルＭｈの向きを頭部画像Ｇｓｈの向きに回転させる。 The information processing device 10 uses a rotational movement transformation formula to move the corresponding point Pm of the head model Mh to the target point Ps of the target image Gs. In other words, the information processing device 10 rotates the orientation of the head model Mh to the orientation of the head image Gsh.
　図８（ａ）から図８（ｃ）は、頭部モデル変形処理の詳細を説明するための図である。上述した通り、頭部モデル変形処理により、対象Ｓの頭部の形状に高精度に近似する様に頭部モデルＭｈの形状が変形される。 FIGS. 8(a) to 8(c) are diagrams for explaining the details of the head model deformation process. As described above, the head model deformation process deforms the shape of the head model Mh so that it closely approximates the shape of the head of the subject S with high accuracy.
　図８（ａ）は、基準箱体Ｃｈおよび調整用箱体Ｃｉを説明するための図である。詳細には後述するが、対象画像Ｇｓの大きさに応じて基準箱体Ｃｈが生成される。また、頭部モデル変形処理が実行される直前の頭部モデルＭｈの大きさに応じて調整用箱体Ｃｉが生成される。頭部モデル変形処理では、基準箱体Ｃｈの大きさと調整用箱体Ｃｉの大きさとの比率Ｒｈｉが求められ、比率Ｒｈｉに応じて頭部モデルＭｈが変形（拡大、縮小）される。 FIG. 8(a) is a diagram for explaining the reference box Ch and the adjustment box Ci. As will be described in detail later, the reference box Ch is generated according to the size of the target image Gs. Furthermore, the adjustment box Ci is generated according to the size of the head model Mh immediately before the head model deformation process is executed. In the head model deformation process, the ratio Rhi between the size of the reference box Ch and the size of the adjustment box Ci is calculated, and the head model Mh is deformed (enlarged or reduced) according to the ratio Rhi.
　図８（ａ）に示す通り、基準箱体Ｃｈは直方体であり底面Ｆｈ１、側面Ｆｈ２～Ｆｈ５および上面Ｆｈ６で構成される。また、図８（ａ）に示す通り、Ｘ－Ｙ平面、Ｘ－Ｚ平面およびＹ－Ｚ平面の何れかに基準箱体Ｃｈの各面Ｆｈは平行である。具体的には、各面Ｆｈ（１～６）のうち底面Ｆｈ１および上面Ｆｈ６はＸ－Ｙ平面に平行であり、側面Ｆｈ２および側面Ｆｈ５はＹ－Ｚ平面に平行であり、側面Ｆｈ３および側面Ｆｈ４はＸ－Ｚ平面に平行である。 As shown in FIG. 8(a), the reference box body Ch is a rectangular parallelepiped and is composed of a bottom surface Fh1, side surfaces Fh2 to Fh5, and a top surface Fh6. Also, as shown in FIG. 8(a), each surface Fh of the reference box body Ch is parallel to either the X-Y plane, the X-Z plane, or the Y-Z plane. Specifically, of each surface Fh (1 to 6), the bottom surface Fh1 and top surface Fh6 are parallel to the X-Y plane, side surfaces Fh2 and Fh5 are parallel to the Y-Z plane, and side surfaces Fh3 and Fh4 are parallel to the X-Z plane.
　図８（ａ）に示す通り、調整用箱体Ｃｉは直方体であり底面Ｆｉ１、側面Ｆｉ２～Ｆｉ５および上面Ｆｉ６で構成される。また、図８（ａ）に示す通り、Ｘ－Ｙ平面、Ｘ－Ｚ平面およびＹ－Ｚ平面の何れかに各面Ｆｉは平行である。具体的には、各面Ｆｉ（１～６）のうち底面Ｆｉ１および上面Ｆｉ６はＸ－Ｙ平面に平行であり、側面Ｆｉ２および側面Ｆｉ５はＹ－Ｚ平面に平行であり、側面Ｆｉ３および側面Ｆｉ４はＸ－Ｚ平面に平行である。 As shown in FIG. 8(a), the adjustment box Ci is a rectangular parallelepiped and is composed of a bottom surface Fi1, side surfaces Fi2 to Fi5, and a top surface Fi6. Also, as shown in FIG. 8(a), each surface Fi is parallel to either the X-Y plane, the X-Z plane, or the Y-Z plane. Specifically, of the surfaces Fi (1 to 6), the bottom surface Fi1 and the top surface Fi6 are parallel to the X-Y plane, the side surfaces Fi2 and Fi5 are parallel to the Y-Z plane, and the side surfaces Fi3 and Fi4 are parallel to the X-Z plane.
　図８（ｂ）は、基準箱体Ｃｈを生成するための構成を説明するための図である。図８（ｂ）の左側部分には、Ｚ軸方向側（上側）から見た基準箱体Ｃｈの概念図が示される。本実施形態の基準箱体Ｃｈは、頭部画像Ｇｓｈを収納可能に生成される。図８（ｂ）には、基準箱体Ｃｈに収納された頭部画像Ｇｓｈが示される。図８（ｂ）に示す通り、基準箱体Ｃｈの側面Ｆｈ２は、頭部画像ＧｓｈのうちＸ座標が最大となる点Ｐｈ２を通る。また、基準箱体Ｃｈの側面Ｆｈ５は、頭部画像ＧｓｈのうちＸ座標が最少となる点Ｐｈ５を通る。基準箱体Ｃｈの側面Ｆｈ３は、頭部画像ＧｓｈのうちＹ座標が最大となる点Ｐｈ３を通り、側面Ｆｈ４は、頭部画像ＧｓｈのうちＹ座標が最少となる点Ｐｈ４を通る。 FIG. 8(b) is a diagram for explaining the configuration for generating the reference box body Ch. The left part of FIG. 8(b) shows a conceptual diagram of the reference box body Ch as viewed from the Z-axis direction (top). In this embodiment, the reference box body Ch is generated so that it can store the head image Gsh. FIG. 8(b) shows the head image Gsh stored in the reference box body Ch. As shown in FIG. 8(b), the side Fh2 of the reference box body Ch passes through the point Ph2 where the X coordinate of the head image Gsh is maximum. In addition, the side Fh5 of the reference box body Ch passes through the point Ph5 where the X coordinate of the head image Gsh is minimum. The side Fh3 of the reference box body Ch passes through the point Ph3 where the Y coordinate of the head image Gsh is maximum, and the side Fh4 passes through the point Ph4 where the Y coordinate of the head image Gsh is minimum.
　以上の説明から理解される通り、基準箱体Ｃｈの各側面Ｆｈ（２～５）は、Ｚ軸方向側から見て頭部画像Ｇｓｈの四方を囲む様に生成される。また、図８（ｂ）の右側部分には、Ｘ軸方向側から見た頭部画像Ｇｓｈの概念図が示される。図８（ｂ）に示す通り、基準箱体Ｃｈの上面Ｆｈ６は、頭部画像ＧｓｈのうちＺ座標が最大となる点Ｐｈ６を通る。以上の通り、基準箱体Ｃｈのうち各側面Ｆｈ（２～５）および上面Ｆｈ６の各々は、対象画像Ｇｈに接するように生成される。 As can be understood from the above explanation, each side surface Fh (2 to 5) of the reference box body Ch is generated so as to surround the head image Gsh on all four sides when viewed from the Z-axis direction. Also, the right side of Figure 8(b) shows a conceptual diagram of the head image Gsh when viewed from the X-axis direction. As shown in Figure 8(b), the top surface Fh6 of the reference box body Ch passes through point Ph6 of the head image Gsh where the Z coordinate is maximum. As described above, each of the side surfaces Fh (2 to 5) and top surface Fh6 of the reference box body Ch is generated so as to be in contact with the target image Gh.
　ところで、対象Ｓを上側（Ｚ軸側）から撮影した場合、対象Ｓの下側部分は撮影されないという事情がある。以上の事情から、対象Ｓの下側部分は頭部画像Ｇｓｈから欠落する。ただし、本実施形態では、対象Ｓの形状に高精度に近似する様に頭部モデルＭｈの形状を変形させるために、対象Ｓの頭部の下端部Ｐａｈに対応する位置に基準箱体Ｃｈの底面Ｆｈ１を生成する必要がある。 However, when the subject S is photographed from above (Z-axis side), the lower part of the subject S is not photographed. For these reasons, the lower part of the subject S is missing from the head image Gsh. However, in this embodiment, in order to deform the shape of the head model Mh so that it closely approximates the shape of the subject S with high accuracy, it is necessary to generate the bottom surface Fh1 of the reference box body Ch at a position corresponding to the lower end Pah of the head of the subject S.
　しかし、頭部画像Ｇｓｈは対象Ｓの下側部分を含まないため、頭部画像ＧｓｈのＺ座標が最少となる点（下端）は、対象Ｓの実際の下端部（Ｐａｈ）とは異なる。したがって、仮に、対象画像ＧｓｈのうちＺ座標が最少となる点に、基準箱体Ｃｈの底面Ｆｈ１を生成した場合、対象Ｓの形状に高精度に近似する様に頭部モデルＭｈの形状を変形できない場合がある。以上の場合、対象Ｓの体積および身長を高精度に推定できない不都合がある。 However, because the head image Gsh does not include the lower part of the target S, the point (bottom end) where the Z coordinate of the head image Gsh is the minimum is different from the actual bottom end (Pah) of the target S. Therefore, if the bottom surface Fh1 of the reference box body Ch were generated at the point in the target image Gsh where the Z coordinate is the minimum, it may not be possible to deform the shape of the head model Mh to closely approximate the shape of the target S with high accuracy. In these cases, there is the inconvenience that the volume and height of the target S cannot be estimated with high accuracy.
　以上の不都合を抑制するため、出願人は、平坦面Ａに横たわる対象Ｓの下端部Ｐａｈが当該平坦面Ａ上に位置する事に着眼した。図８（ｂ）には、対象Ｓの頭部のうち頭部画像Ｇｓから欠落した下側部分Ｓｈが破線で示される。上述した通り、基準面Ｆａは平坦面Ａに対応する位置に生成される。したがって、対象Ｓの下側部分Ｓｈが頭部画像Ｇｓには含まれない場合であっても、３次元画像空間における当該下側部分Ｓｈの下端部Ｐａｈに対応する点は、基準面Ｆａ上にあると仮定できる。 In order to prevent the above inconveniences, the applicant has focused on the fact that the lower end Pah of the subject S lying on the flat surface A is located on the flat surface A. In FIG. 8(b), the lower part Sh of the head of the subject S that is missing from the head image Gs is shown by a dashed line. As described above, the reference plane Fa is generated at a position corresponding to the flat surface A. Therefore, even if the lower part Sh of the subject S is not included in the head image Gs, it can be assumed that the point corresponding to the lower end Pah of the lower part Sh in the three-dimensional image space is on the reference plane Fa.
　以上の事情を考慮して、本実施形態では、基準箱体Ｃｈの底面Ｆｈ１が基準面Ｆａに位置する構成を採用した。以上の構成は、対象Ｓの下側部分Ｓｈが下端部Ｐａｈで平坦面Ａに接しているものとして、対象画像Ｇｓにおいては欠落した下側部分Ｓｈを含む対象Ｓの体積を、当該対象画像Ｇｓを用いて推定する構成であるとも換言される。 In consideration of the above circumstances, this embodiment employs a configuration in which the bottom surface Fh1 of the reference box body Ch is located on the reference surface Fa. In other words, the above configuration is a configuration in which the lower part Sh of the target S is assumed to be in contact with the flat surface A at its lower end Pah, and the volume of the target S including the lower part Sh that is missing in the target image Gs is estimated using the target image Gs.
　図８（ｃ）は、調整用箱体Ｃｉを生成するための構成を説明するための図である。図８（ｃ）には、Ｚ軸方向側（上側）から見た調整用箱体Ｃｉの概念図が示される。調整用箱体Ｃｉは、頭部モデルＭｈが収納可能に生成される。 FIG. 8(c) is a diagram for explaining the configuration for generating the adjustment box Ci. FIG. 8(c) shows a conceptual diagram of the adjustment box Ci as viewed from the Z-axis direction (top). The adjustment box Ci is generated so that it can accommodate the head model Mh.
　図８（ｃ）の右側部分に示す通り、調整用箱体Ｃｉの上面Ｆｉ６は、頭部モデルＭｈのうちＺ座標が最大となる点Ｑｉ６を通る。また、図８（ｃ）の左側部分に示す通り、調整用箱体Ｃｉの側面Ｆｉ２は、頭部モデルＭｈのうちＸ座標が最大となる点Ｑｉ２を通り、調整用箱体Ｃｉの側面Ｆｉ５は、頭部モデルＭｈのうちＸ座標が最少となる点Ｑｉ５を通る。調整用箱体Ｃｉの側面Ｆｉ３は、頭部モデルＭｈのうちＹ座標が最大となる点Ｐｉ３を通り、側面Ｆｉ４は、頭部モデルＭｈのうちＹ座標が最少となる点Ｐｉ４を通る。頭部モデルＭｈは、下側部分を含む対象Ｓの全体を表す。図８（ｃ）に示す通り、調整用箱体Ｃｉの底面Ｆｉ１は、頭部モデルＭｈのうちＺ座標が最少となる点Ｑｉ１を通る。 As shown in the right part of FIG. 8(c), the top surface Fi6 of the adjustment box Ci passes through the point Qi6 of the head model Mh where the Z coordinate is maximum. Also, as shown in the left part of FIG. 8(c), the side surface Fi2 of the adjustment box Ci passes through the point Qi2 of the head model Mh where the X coordinate is maximum, and the side surface Fi5 of the adjustment box Ci passes through the point Qi5 of the head model Mh where the X coordinate is minimum. The side surface Fi3 of the adjustment box Ci passes through the point Pi3 of the head model Mh where the Y coordinate is maximum, and the side surface Fi4 passes through the point Pi4 of the head model Mh where the Y coordinate is minimum. The head model Mh represents the entire target S including the lower part. As shown in FIG. 8(c), the bottom surface Fi1 of the adjustment box Ci passes through the point Qi1 of the head model Mh where the Z coordinate is minimum.
　情報処理装置１０は、頭部モデル変形処理において、基準箱体Ｃｈの大きさ（形状）に応じて頭部モデルＭｈを変形する。具体的には、情報処理装置１０は、上述の図８（ｂ）に示す基準箱体ＣｈのＸ軸方向への長さＬｈｘと図８（ｃ）に示す調整用箱体ＣｉのＸ軸方向への長さＬｉｘとの比率Ｒｈｉｘ（＝Ｌｈｘ／Ｌｉｘ）を求める。また、情報処理装置１０は、上述の図８（ｂ）に示す基準箱体ＣｈのＹ軸方向への長さＬｈｙと図８（ｃ）に示す調整用箱体ＣｉのＹ軸方向への長さＬｉｙとの比率Ｒｈｉｙ（＝Ｌｈｙ／Ｌｉｙ）を求める。同様に、情報処理装置１０は、上述の図８（ｂ）に示す基準箱体ＣｈのＺ軸方向への長さＬｈｚと図８（ｃ）に示す調整用箱体ＣｉのＺ軸方向への長さＬｉｚとの比率Ｒｈｉｚ（＝Ｌｈｚ／Ｌｉｚ）を求める。 In the head model deformation process, the information processing device 10 deforms the head model Mh according to the size (shape) of the reference box body Ch. Specifically, the information processing device 10 obtains the ratio Rhix (=Lhx/Lix) between the length Lhx in the X-axis direction of the reference box body Ch shown in FIG. 8(b) and the length Lix in the X-axis direction of the adjustment box body Ci shown in FIG. 8(c). The information processing device 10 also obtains the ratio Rhiy (=Lhy/Liy) between the length Lhy in the Y-axis direction of the reference box body Ch shown in FIG. 8(b) and the length Liy in the Y-axis direction of the adjustment box body Ci shown in FIG. 8(c). Similarly, the information processing device 10 obtains the ratio Rhiz (=Lhz/Liz) between the length Lhz in the Z-axis direction of the reference box body Ch shown in FIG. 8(b) and the length Liz in the Z-axis direction of the adjustment box body Ci shown in FIG. 8(c).
　比率Ｒｈｉｘ、比率Ｒｈｉｙおよび比率Ｒｈｉｚ（「比率Ｒｈｉ」と総称する場合がある）を求めた後に、情報処理装置１０は、比率Ｒｈｉを用いて頭部モデルＭｈを変形（拡大、縮小）する。具体的には、情報処理装置１０は、頭部モデルＭｈをＸ軸方向へＲｈｉｘ倍し、Ｙ軸方向へＲｈｉｙ倍し、Ｚ軸方向へＲｈｉｚ倍する。以上の頭部モデル変形処理によれば、対象Ｓの下側部分が撮影されない場合であっても、対象Ｓの下端部Ｐａｈが平坦面Ａに接していると仮定して基準箱体Ｃｈを生成することにより、頭部モデルＭｈの形状が対象Ｓの頭部の形状に高精度に近似し易くなる。 After determining the ratios Rhix, Rhiy, and Rhiz (sometimes collectively referred to as "ratio Rhi"), the information processing device 10 deforms (enlarges or reduces) the head model Mh using the ratio Rhi. Specifically, the information processing device 10 multiplies the head model Mh by Rhix in the X-axis direction, by Rhiy in the Y-axis direction, and by Rhiz in the Z-axis direction. According to the above head model deformation process, even if the lower part of the subject S is not photographed, the reference box body Ch is generated assuming that the lower end Pah of the subject S is in contact with the flat surface A, making it easier to approximate the shape of the head model Mh to the shape of the head of the subject S with high accuracy.
　図９（ａ）および図９（ｂ）は、胴体モデル変形処理の詳細を説明するための図である。胴体モデル変形処理では、上述の頭部モデル変形処理と同様に、基準箱体Ｃｂおよび調整用箱体Ｃｃを用いて胴体モデルＭｂの形状が変形される。 FIGS. 9(a) and 9(b) are diagrams for explaining the details of the torso model deformation process. In the torso model deformation process, the shape of the torso model Mb is deformed using the reference box Cb and the adjustment box Cc, similar to the head model deformation process described above.
　ただし、本実施形態の対象Ｓは新生児を想定し、新生児は紙オムツを履いている場合が多い。以上の場合、胴体画像Ｇｂのうち腰部画像Ｇｗは、新生児の腰部を覆う紙オムツを表す領域が多くなる。以上の事情を考慮して、本実施形態の基準箱体Ｃｂは、胴体画像Ｇｂのうち胸部画像Ｇｃに基づいて生成される。 However, the subject S in this embodiment is assumed to be a newborn, and newborns often wear disposable diapers. In this case, the waist image Gw in the torso image Gb will have a large area that represents the disposable diaper covering the waist of the newborn. Taking the above into consideration, the reference box body Cb in this embodiment is generated based on the chest image Gc in the torso image Gb.
　図９（ａ）は、基準箱体Ｃｂを説明するための図である。基準箱体Ｃｂは、直方体であり胸部画像Ｇｃを収納可能に生成される。具体的には、基準箱体Ｃｂは、底面Ｆｂ１、側面Ｆｂ２～Ｆｂ５および上面Ｆｂ６で構成される。以上の各面Ｆｂ（１～６）は、図９（ａ）に示す通りＸ－Ｙ平面、Ｘ－Ｚ平面およびＹ－Ｚ平面の何れかに平行である。具体的には、各面Ｆｂ（１～６）のうち底面Ｆｂ１および上面Ｆｂ６はＸ－Ｙ平面に平行であり、側面Ｆｂ２および側面Ｆｂ５はＹ－Ｚ平面に平行であり、側面Ｆｂ３および側面Ｆｂ４はＸ－Ｚ平面に平行である。 FIG. 9(a) is a diagram for explaining the reference box Cb. The reference box Cb is a rectangular parallelepiped and is generated so that it can store the chest image Gc. Specifically, the reference box Cb is composed of a bottom surface Fb1, side surfaces Fb2 to Fb5, and a top surface Fb6. As shown in FIG. 9(a), each of the above surfaces Fb (1 to 6) is parallel to either the X-Y plane, the X-Z plane, or the Y-Z plane. Specifically, of the surfaces Fb (1 to 6), the bottom surface Fb1 and the top surface Fb6 are parallel to the X-Y plane, the side surfaces Fb2 and Fb5 are parallel to the Y-Z plane, and the side surfaces Fb3 and Fb4 are parallel to the X-Z plane.
　図９（ａ）に示す通り、上面Ｆｂ６は、胸部画像ＧｃのうちＺ座標が最大となる点Ｐｂ６と接する。また、側面Ｆｂ２は、胸部画像ＧｃのうちＸ座標が最小となる点Ｐｂ２と接し、側面Ｆｂ５は胸部画像ＧｃのうちＸ座標が最大となる点Ｐｂ５と接する。同様に、側面Ｆｂ４は、胸部画像ＧｃのうちＹ座標が最小となる点Ｐｂ４と接し、側面Ｆｂ３は胸部画像ＧｃのうちＹ座標が最大となる点Ｐｂ３と接する。 As shown in FIG. 9(a), the top surface Fb6 is in contact with point Pb6 in the chest image Gc, where the Z coordinate is maximum. In addition, the side surface Fb2 is in contact with point Pb2 in the chest image Gc, where the X coordinate is minimum, and the side surface Fb5 is in contact with point Pb5 in the chest image Gc, where the X coordinate is maximum. Similarly, the side surface Fb4 is in contact with point Pb4 in the chest image Gc, where the Y coordinate is minimum, and the side surface Fb3 is in contact with point Pb3 in the chest image Gc, where the Y coordinate is maximum.
　上述した通り、対象Ｓを上側（Ｚ軸側）から撮影した場合、対象Ｓの下側部分は撮影されないという事情がある。以上の事情から、対象Ｓの下側部分は胸部画像Ｇｃから欠落する。図９（ａ）には胸部画像Ｇｃから欠落した対象Ｓの胸部の下側部分Ｓｂが破線で示される。 As mentioned above, when the subject S is photographed from above (Z-axis side), the lower part of the subject S is not photographed. For these reasons, the lower part of the subject S is missing from the chest image Gc. In Figure 9(a), the lower part Sb of the subject S's chest that is missing from the chest image Gc is shown by a dashed line.
　ただし、平坦面Ａに対象Ｓが横たわる場合、対象Ｓの胸部の下側部分は平坦面Ａに接するのが通常であるという事情がある。以上の事情を考慮して、本実施形態では、対象Ｓの胸部の下端部Ｐａｃが平坦面Ａに位置するものとして基準箱体Ｃｂを生成する（上述の図８（ｂ）で示した基準箱体Ｃｈと同様）。 However, when subject S lies on flat surface A, the lower part of the subject S's chest is usually in contact with flat surface A. Taking the above into consideration, in this embodiment, a reference box body Cb is generated assuming that the lower end Pac of the subject S's chest is located on flat surface A (similar to the reference box body Ch shown in Figure 8(b) above).
　具体的には、図９（ａ）に示す通り、対象Ｓの胸部のうち下側部分Ｓｃが胸部画像Ｇｃから欠落する。以上の事情を考慮して、本実施形態では、胸部画像Ｇｃの下端部に替えて、実際の胸部の下端部Ｐａｃが位置する基準面Ｆａ（平坦面Ａ）上に基準箱体Ｃｂの底面Ｆｂ１を生成する。以上の構成によれば、例えば、基準箱体Ｃｂの底面が胸部画像Ｇｃの下端部を通る構成と比較して、対象Ｓの体重または身長を高精度に推定できる。 Specifically, as shown in FIG. 9(a), the lower portion Sc of the chest of the subject S is missing from the chest image Gc. Taking the above into consideration, in this embodiment, instead of the lower end portion of the chest image Gc, the bottom surface Fb1 of the reference box body Cb is generated on the reference plane Fa (flat surface A) where the actual lower end portion Pac of the chest is located. With the above configuration, for example, the weight or height of the subject S can be estimated with high accuracy compared to a configuration in which the bottom surface of the reference box body Cb passes through the lower end portion of the chest image Gc.
　図９（ｂ）は、調整用箱体Ｃｃを説明するための図である。調整用箱体Ｃｃは、直方体であり胸部モデルＭｃを収納可能に生成される。具体的には、調整用箱体Ｃｃは、底面Ｆｃ１、側面Ｆｃ２～Ｆｃ５および上面Ｆｃ６で構成される。以上の各面Ｆｃ（１～６）は、図９（ｂ）に示す通りＸ－Ｙ平面、Ｘ－Ｚ平面およびＹ－Ｚ平面の何れかに平行である。具体的には、各面Ｆｃ（１～６）のうち上面Ｆｃ６および底面Ｆｃ１はＸ－Ｙ平面に平行であり、側面Ｆｃ２および側面Ｆｃ５はＹ－Ｚ平面に平行であり、側面Ｆｃ３および側面Ｆｃ４はＸ－Ｚ平面に平行である。 FIG. 9(b) is a diagram for explaining the adjustment box Cc. The adjustment box Cc is a rectangular parallelepiped and is generated so that it can house the chest model Mc. Specifically, the adjustment box Cc is composed of a bottom surface Fc1, side surfaces Fc2 to Fc5, and a top surface Fc6. As shown in FIG. 9(b), each of the above surfaces Fc (1 to 6) is parallel to either the X-Y plane, the X-Z plane, or the Y-Z plane. Specifically, of the surfaces Fc (1 to 6), the top surface Fc6 and the bottom surface Fc1 are parallel to the X-Y plane, the side surfaces Fc2 and Fc5 are parallel to the Y-Z plane, and the side surfaces Fc3 and Fc4 are parallel to the X-Z plane.
　図９（ｂ）に示す通り、調整用箱体Ｃｃの各面Ｆｃ（１～６）のうち上面Ｆｃ６は、胸部モデルＭｃのうちＺ座標が最大となる点Ｑｃ６と接する。また、側面Ｆｃ２は、胸部モデルＭｃのうちＸ座標が最小となる点Ｑｃ２と接し、側面Ｆｃ５は胸部モデルＭｃのうちＸ座標が最大となる点Ｑｃ５と接する。同様に、側面Ｆｃ３は、胸部モデルＭｃのうちＹ座標が最大となる点Ｑｃ３と接し、側面Ｆｃ４は、胸部モデルＭｃのうちＹ座標が最少となる点Ｑｃ４と接する。胸部モデルＭｃは、下側部分を含む対象Ｓの全体の形状を表す。以上の事情を考慮して、調整用箱体Ｃｃの底面Ｆｃ１は、胸部モデルＭｃにおいてＺ座標が最少となる下端部Ｑｃ１を通る。 As shown in FIG. 9(b), of the faces Fc (1-6) of the adjustment box Cc, the top face Fc6 is in contact with the point Qc6 of the chest model Mc where the Z coordinate is maximum. In addition, the side face Fc2 is in contact with the point Qc2 of the chest model Mc where the X coordinate is minimum, and the side face Fc5 is in contact with the point Qc5 of the chest model Mc where the X coordinate is maximum. Similarly, the side face Fc3 is in contact with the point Qc3 of the chest model Mc where the Y coordinate is maximum, and the side face Fc4 is in contact with the point Qc4 of the chest model Mc where the Y coordinate is minimum. The chest model Mc represents the overall shape of the target S including the lower part. Taking the above into consideration, the bottom face Fc1 of the adjustment box Cc passes through the lower end Qc1 of the chest model Mc where the Z coordinate is minimum.
　情報処理装置１０は、胴体モデル変形処理において、基準箱体Ｃｂの大きさ（形状）に応じて胸部モデルＭｃを変形する。具体的には、情報処理装置１０は、上述の図９（ａ）に示す基準箱体ＣｂのＸ軸方向への長さＬｂｘと、図９（ｂ）に示す調整用箱体ＣｃのＸ軸方向への長さＬｃｘとの比率Ｒｂｃｘ（＝Ｌｂｘ／Ｌｃｘ）を求める。また、情報処理装置１０は、上述の図９（ａ）に示す基準箱体ＣｂのＹ軸方向への長さＬｂｙと、図９（ｂ）に示す調整用箱体ＣｃのＹ軸方向への長さＬｃｙとの比率Ｒｂｃｙ（＝Ｌｂｙ／Ｌｃｙ）を求める。同様に、情報処理装置１０は、上述の図９（ａ）に示す基準箱体ＣｂのＺ軸方向への長さＬｂｚと、図９（ｂ）に示す調整用箱体ＣｃのＺ軸方向への長さＬｃｚとの比率Ｒｂｃｚ（＝Ｌｂｚ／Ｌｃｚ）を求める。 In the torso model deformation process, the information processing device 10 deforms the chest model Mc according to the size (shape) of the reference box body Cb. Specifically, the information processing device 10 obtains the ratio Rbcx (=Lbx/Lcx) between the length Lbx in the X-axis direction of the reference box body Cb shown in FIG. 9(a) and the length Lcx in the X-axis direction of the adjustment box body Cc shown in FIG. 9(b). The information processing device 10 also obtains the ratio Rbcy (=Lby/Lcy) between the length Lby in the Y-axis direction of the reference box body Cb shown in FIG. 9(a) and the length Lcy in the Y-axis direction of the adjustment box body Cc shown in FIG. 9(b). Similarly, the information processing device 10 obtains the ratio Rbcz (=Lbz/Lcz) between the length Lbz in the Z-axis direction of the reference box body Cb shown in FIG. 9(a) and the length Lcz in the Z-axis direction of the adjustment box body Cc shown in FIG. 9(b).
　比率Ｒｂｃｘ、比率Ｒｂｃｙおよび比率Ｒｂｃｚ（「比率Ｒｂｃ」と総称する場合がある）を求めた後に、情報処理装置１０は、比率Ｒｂｃに基づいて胴体モデルＭｂ（胸部モデルＭｃ＋腰部モデルＭｗ）を変形（拡大、縮小）する。具体的には、情報処理装置１０は、胴体モデルＭｂをＸ軸方向へＲｂｃｘ倍し、Ｙ軸方向へＲｂｃｙ倍し、Ｚ軸方向へＲｂｃｚ倍する。以上の胴体モデル変形処理によれば、対象Ｓの下側部分が撮影されない場合であっても、対象Ｓの胸部の下端部Ｐａｃが平坦面Ａに接していると仮定して基準箱体Ｃｂを生成することにより、対象Ｓの胴体の形状に高精度に近似する様に胴体モデルＭｂの形状を変形できる。 After determining the ratios Rbcx, Rbcy, and Rbcz (sometimes collectively referred to as "ratio Rbc"), the information processing device 10 deforms (enlarges and reduces) the torso model Mb (chest model Mc+waist model Mw) based on the ratio Rbc. Specifically, the information processing device 10 multiplies the torso model Mb by Rbcx in the X-axis direction, by Rbcy in the Y-axis direction, and by Rbcz in the Z-axis direction. According to the above torso model deformation process, even if the lower part of the subject S is not photographed, the shape of the torso model Mb can be deformed to closely approximate the shape of the torso of the subject S with high accuracy by generating a reference box body Cb assuming that the lower end Pac of the chest of the subject S is in contact with the flat surface A.
　図１０（ａ）は、体積推定処理を説明するための図である。体積推定処理では、対象Ｓの体積が推定される。上述した通り、本実施形態では、頭部モデル変形処理（図８（ａ）～（ｃ）参照）、胴体モデル変形処理（図９（ａ）、（ｂ）参照）、腕部モデル変形処理および脚部モデル変形処理により、頭部モデルＭｈ、胴体モデルＭｂ、腕部モデルＭａおよび脚部モデルＭｌが、対象Ｓの各部分の形状に近似する様に変形される。対象モデルＭは、以上の変形後の各モデルを組合せて生成される。図１０（ａ）には、対象モデルＭの模擬図が示される。 FIG. 10(a) is a diagram for explaining the volume estimation process. In the volume estimation process, the volume of the target S is estimated. As described above, in this embodiment, the head model deformation process (see FIGS. 8(a)-(c)), torso model deformation process (see FIGS. 9(a) and (b)), arm model deformation process, and leg model deformation process are used to deform the head model Mh, torso model Mb, arm model Ma, and leg model Ml so as to approximate the shapes of each part of the target S. The target model M is generated by combining each model after the above deformations. FIG. 10(a) shows a mock-up of the target model M.
　体積推定処理において、情報処理装置１０は、図１０（ａ）に示す各断面Ｍｐを生成する。各断面Ｍｐは、Ｘ軸方向の位置をずらしながらＹ－Ｚ平面に平行に対象モデルＭを切断したものである。図１０（ａ）には、各断面Ｍｐのうち隣合う２個を抜粋して示す。図１０（ａ）に示す通り、隣合う各断面ＭｐはＸ軸方向に「距離Δｘ」だけ離れている。距離Δｘは、対象画像Ｇｓの大きさに対して十分に小さい。距離Δｘが十分に小さい場合、対象モデルＭの体積Ｖは、以下の数２により算出（近似）される。なお、数２における「Ａｎ」は、対象モデルＭのＸ軸方向の先端からｎ個目の断面Ｍｐの面積を意味する。また、数２における「Ｎ」は、断面Ｍｐの総数を意味する。 In the volume estimation process, the information processing device 10 generates each cross section Mp shown in FIG. 10(a). Each cross section Mp is obtained by cutting the target model M parallel to the Y-Z plane while shifting the position in the X-axis direction. FIG. 10(a) shows two adjacent cross sections Mp. As shown in FIG. 10(a), adjacent cross sections Mp are separated by a "distance Δx" in the X-axis direction. The distance Δx is sufficiently small compared to the size of the target image Gs. When the distance Δx is sufficiently small, the volume V of the target model M is calculated (approximated) by the following equation 2. Note that "An" in equation 2 means the area of the n-th cross section Mp from the tip of the target model M in the X-axis direction. Also, "N" in equation 2 means the total number of cross sections Mp.
（数２）
 (Equation 2)
　上述した通り、対象モデルＭの形状（大きさ）は実際の対象Ｓの形状に近似する。したがって、体積推定処理で算出した対象モデルＭの体積Ｖは、実際の対象Ｓの体積として推定できる。また、情報処理装置１０は、対象モデルＭの体積Ｖに平均密度Ｄを掛けて当該対象Ｓの体重Ｗを算出する。 As described above, the shape (size) of the target model M is similar to the shape of the actual target S. Therefore, the volume V of the target model M calculated in the volume estimation process can be estimated as the volume of the actual target S. In addition, the information processing device 10 multiplies the volume V of the target model M by the average density D to calculate the weight W of the target S.
　図１０（ｂ）は、身長推定処理を説明するための図である。身長推定処理では、対象モデルＭの身長が算出され、当該算出結果が対象Ｓの身長として推定される。具体的には、情報処理装置１０は、身長推定処理において、対象モデルＭにおける予め定められた特徴点を検出する。本実施形態の身長推定処理では、図１０（ｂ）に示す通り、対象モデルＭのうち対象Ｓの頭頂部に対応する点Ｐａ、肩関節に対応する点Ｐｂ、股関節に対応する点Ｐｃ、膝関節に対応する点Ｐｄおよび足首に対応する点Ｐｅを検出する。なお、対象モデルＭにおける特徴点を検出する技術としては、例えば、特許第６８６８８７５号公報に記載の技術が採用され得る。 FIG. 10(b) is a diagram for explaining the height estimation process. In the height estimation process, the height of the target model M is calculated, and the calculation result is estimated as the height of the target S. Specifically, the information processing device 10 detects predetermined feature points in the target model M in the height estimation process. In the height estimation process of this embodiment, as shown in FIG. 10(b), point Pa corresponding to the top of the head of the target S, point Pb corresponding to the shoulder joint, point Pc corresponding to the hip joint, point Pd corresponding to the knee joint, and point Pe corresponding to the ankle are detected in the target model M. Note that, as a technology for detecting feature points in the target model M, for example, the technology described in Japanese Patent No. 6868875 can be adopted.
　情報処理装置１０は、対象モデルＭにおける特徴点を検出すると、頭頂部に対応する点Ｐａから肩関節に対応する点Ｐｂまでの水平距離Ｌａｂを算出する。また、情報処理装置１０は、肩関節に対応する点Ｐｂから股関節に対応する点Ｐｃまでの水平距離Ｌｂｃ、股関節に対応する点Ｐｃから膝関節に対応する点Ｐｄまでの距離Ｌｃｄ、および、膝関節に対応する点Ｐｄから足首に対応する点Ｐｅまでの距離Ｌｄｅを算出する。 When the information processing device 10 detects feature points in the target model M, it calculates the horizontal distance Lab from point Pa corresponding to the top of the head to point Pb corresponding to the shoulder joint. In addition, the information processing device 10 calculates the horizontal distance Lbc from point Pb corresponding to the shoulder joint to point Pc corresponding to the hip joint, the distance Lcd from point Pc corresponding to the hip joint to point Pd corresponding to the knee joint, and the distance Lde from point Pd corresponding to the knee joint to point Pe corresponding to the ankle.
　また、情報処理装置１０は、距離Ｌａｂと距離Ｌｂｃと距離Ｌｃｄと距離Ｌｄｅとの和を対象モデルＭの身長Ｈとして算出する。以上の対象モデルＭの身長Ｈは、当該対象モデルＭが表す対象Ｓの身長として推定される。なお、身長推定処理は適宜に変更できる。例えば、対象モデルＭから検出される特徴点は、当該対象モデルＭの身長が計測できれば足り、以上の例に限定されない。 In addition, the information processing device 10 calculates the sum of the distances Lab, Lbc, Lcd, and Lde as the height H of the target model M. The height H of the target model M described above is estimated as the height of the target S represented by the target model M. The height estimation process can be modified as appropriate. For example, the feature points detected from the target model M are not limited to the above examples, as long as they can measure the height of the target model M.
　図１１は、情報処理装置１０が実行する画像撮影時処理のフローチャートである。上述した通り、撮影装置２０は、３次元画像Ｇ（対象画像Ｇｓを含む）を示す画像情報Ｄｇを情報処理装置１０へ送信する。情報処理装置１０は、例えば、画像情報Ｄｇを取得（受信）した契機で画像撮影時処理を実行する。ただし、画像撮影時処理が実行される契機は適宜に変更できる。 FIG. 11 is a flowchart of the image capture processing executed by the information processing device 10. As described above, the image capture device 20 transmits image information Dg indicating the three-dimensional image G (including the target image Gs) to the information processing device 10. The information processing device 10 executes the image capture processing, for example, when it acquires (receives) the image information Dg. However, the trigger for executing the image capture processing can be changed as appropriate.
　画像撮影時処理を開始すると、情報処理装置１０は、画像認識処理（Ｓ０）を実行する。画像認識処理では、画像情報Ｄｇが示す３次元画像Ｇに表される各物体（対象Ｓを含む）が認識される。画像認識処理を実行すると、情報処理装置１０は、頭部サイズ概算処理（Ｓ１）を実行する。頭部サイズ概算処理では、頭部モデルＭｈの大まかな形状（大きさ）が決定される（上述の図７（ａ）参照）。また、情報処理装置１０は、頭部サイズ概算処理を実行した後に、頭部モデル回転処理を実行する。頭部モデル回転処理では、頭部モデルＭｈの向きが、頭部画像Ｇｈの向きに応じて回転（調整）される（上述の図７（ｂ）参照）。 When the image capture process starts, the information processing device 10 executes an image recognition process (S0). In the image recognition process, each object (including the target S) depicted in the three-dimensional image G indicated by the image information Dg is recognized. After executing the image recognition process, the information processing device 10 executes a head size approximation process (S1). In the head size approximation process, the rough shape (size) of the head model Mh is determined (see FIG. 7(a) above). After executing the head size approximation process, the information processing device 10 also executes a head model rotation process. In the head model rotation process, the orientation of the head model Mh is rotated (adjusted) in accordance with the orientation of the head image Gh (see FIG. 7(b) above).
　頭部モデル回転処理を実行した後に、情報処理装置１０は、頭部モデル変形処理（Ｓ３）を実行する。上述した通り、頭部モデル変形処理では、対象Ｓの頭部の形状に高精度に近似する様に頭部モデルＭｈの形状が変形される（上述の図８（ａ）～（ｃ）参照）。 After executing the head model rotation process, the information processing device 10 executes the head model deformation process (S3). As described above, in the head model deformation process, the shape of the head model Mh is deformed so as to closely approximate the shape of the head of the subject S with high accuracy (see Figures 8(a) to (c) above).
　また、情報処理装置１０は、胴体画像回転処理（Ｓ４）を実行する。胴体画像回転処理では、Ｘ軸方向に対して平行になる様に胴体画像Ｇｓｂが回転される。その後、情報処理装置１０は、胴体モデル変形処理（Ｓ５）を実行する。上述した通り、胴体モデル変形処理では、対象Ｓの胴体の形状に高精度に近似する様に胴体モデルＭｂの形状が変形される（上述の図９（ａ）（ｂ）参照）。 The information processing device 10 also executes a torso image rotation process (S4). In the torso image rotation process, the torso image Gsb is rotated so as to be parallel to the X-axis direction. After that, the information processing device 10 executes a torso model deformation process (S5). As described above, in the torso model deformation process, the shape of the torso model Mb is deformed so as to closely approximate the shape of the torso of the target S with high accuracy (see Figures 9(a) and (b) above).
　情報処理装置１０は、腕部モデル変形処理（Ｓ６）を実行して、対象Ｓの腕部の形状に近似する様に腕部モデルＭａの形状を変形する。また、情報処理装置１０は、脚部モデル変形処理（Ｓ７）を実行して、対象Ｓの脚部の形状に近似する様に脚部モデルＭｌの形状を変形する。 The information processing device 10 executes an arm model deformation process (S6) to deform the shape of the arm model Ma so that it approximates the shape of the arm of the subject S. The information processing device 10 also executes a leg model deformation process (S7) to deform the shape of the leg model Ml so that it approximates the shape of the leg of the subject S.
　その後、情報処理装置１０は、対象モデル生成処理（Ｓ８）を実行する。対象モデル生成処理では、直近の頭部モデル変形処理で変形された頭部モデルＭｈ、胴体モデル変形処理で変形された胴体モデルＭｂ、腕部モデル変形処理で変形された腕部モデルＭａ、および、脚部モデル変形処理で変形された脚部モデルＭｌを組合せて対象モデルＭが生成される。 Then, the information processing device 10 executes a target model generation process (S8). In the target model generation process, a target model M is generated by combining a head model Mh deformed in the most recent head model deformation process, a torso model Mb deformed in the torso model deformation process, an arm model Ma deformed in the arm model deformation process, and a leg model Ml deformed in the leg model deformation process.
　対象モデル生成処理を実行した後に、情報処理装置１０は、体積推定処理（Ｓ９）を実行する。体積推定処理では、直近の対象モデル生成処理で生成された対象モデルＭの体積が算出される（上述の図１０（ａ）参照）。体積推定処理を実行した後に、情報処理装置１０は、体重推定処理（Ｓ１０）を実行する。体重推定処理において、情報処理装置１０は、直近の体積推定処理で算出された対象モデルＭの体積に平均密度を掛けて、計算結果を対象Ｓの体重Ｗとして記憶する。 After executing the object model generation process, the information processing device 10 executes a volume estimation process (S9). In the volume estimation process, the volume of the object model M generated in the most recent object model generation process is calculated (see FIG. 10(a) above). After executing the volume estimation process, the information processing device 10 executes a weight estimation process (S10). In the weight estimation process, the information processing device 10 multiplies the volume of the object model M calculated in the most recent volume estimation process by the average density, and stores the calculation result as the weight W of the object S.
　体重推定処理を実行した後に、情報処理装置１０は、身長推定処理（Ｓ１１）を実行する。身長推定処理では、直近の対象モデル生成処理で生成された対象モデルＭの身長が計測され、計測結果が対象Ｓの身長Ｈとして記憶される（上述の図１０（ｂ）参照）。身長推定処理を実行した後に、情報処理装置１０は、生体情報表示処理（Ｓ１２）を実行する。生体情報表示処理において、直近の体重推定処理で算出した対象Ｓの体重Ｗ、および、身長推定処理で算出した対象Ｓの身長Ｈが例えば保育器のモニタ１０４に表示される。生体情報表示処理を実行すると、情報処理装置１０は、画像撮影時処理を終了する。 After executing the weight estimation process, the information processing device 10 executes a height estimation process (S11). In the height estimation process, the height of the target model M generated in the most recent target model generation process is measured, and the measurement result is stored as the height H of the target S (see FIG. 10(b) above). After executing the height estimation process, the information processing device 10 executes a biometric information display process (S12). In the biometric information display process, the weight W of the target S calculated in the most recent weight estimation process and the height H of the target S calculated in the height estimation process are displayed, for example, on the monitor 104 of the incubator. After executing the biometric information display process, the information processing device 10 ends the image capture process.
＜第２実施形態＞
　本発明の他の実施形態を以下に説明する。なお、以下に例示する各形態において作用や機能が第１実施形態と同等である要素については、第１実施形態の説明で参照した符号を流用して各々の詳細な説明を適宜に省略する。 Second Embodiment
Other embodiments of the present invention will be described below. In each of the following exemplary embodiments, the elements having the same actions and functions as those of the first embodiment will be designated by the same reference numerals as those in the first embodiment, and detailed descriptions thereof will be omitted as appropriate.
　従来から、医療用チューブを対象に挿入する医療行為が知られている。以上の医療行為としては、例えば、食事を経口摂取出来ない対象（患者、新生児を含む）に対して、胃管チューブを挿入する医療行為がある。以上の医療行為において、胃管チューブは、鼻孔から挿入され、食道を通り、胃の内部に到達する。また、以上の医療行為においては、胃の内部に先端（チューブの出口）が位置する適切な長さの胃管チューブを用いる必要がある。 　Medical procedures in which a medical tube is inserted into a subject have been known for some time. For example, one such medical procedure is the insertion of a gastric tube into a subject (including patients and newborns) who cannot orally ingest food. In such medical procedures, the gastric tube is inserted through the nostril, passes through the esophagus, and reaches the inside of the stomach. Furthermore, in such medical procedures, it is necessary to use a gastric tube of an appropriate length so that its tip (the exit point of the tube) is located inside the stomach.
　ただし、鼻孔から胃の内部までの長さは、対象により相違するという事情がある。したがって、対象毎に適切な胃管チューブの長さが変化する。そこで、従来から、対象の表面における耳孔から眉間までの長さと臍から剣状突起の中間までの長さとの合計を、胃管チューブの適切な長さと推定していた。 However, the length from the nostril to the inside of the stomach differs depending on the subject. Therefore, the appropriate length of the gastric tube varies for each subject. Therefore, traditionally, the appropriate length of the gastric tube has been estimated as the sum of the length from the ear hole to the space between the eyebrows on the subject's surface and the length from the navel to the middle of the xiphoid process.
　しかし、対象の表面における長さを測定する際に、当該対象に触れる必要がある。また、対象によっては、以上の行為が侵襲的な行為に相当し得る。以上の事情を考慮して、第２実施形態では、対象に触れることなく当該対象に挿入する医療用チューブ（例えば、胃管チューブ）の適切な長さを推定可能にすることを目的とする。 However, when measuring the length on the surface of an object, it is necessary to touch the object. Furthermore, depending on the object, the above action may be considered invasive. Taking the above into consideration, the second embodiment aims to make it possible to estimate the appropriate length of a medical tube (e.g., a gastric tube) to be inserted into the object without touching the object.
　図１２は、第２実施形態の頭部モデルＭｈおよび胴体モデルＭｂの概念図である。図１２においては、説明のため、頭部モデルＭｈと胴体モデルＭｂとを組合せて示す。また、図１２は、頭部モデルＭｈと胴体モデルＭｂとをＸ－Ｚ平面に水平に切断した断面図である。 FIG. 12 is a conceptual diagram of the head model Mh and torso model Mb of the second embodiment. For the sake of explanation, the head model Mh and torso model Mb are shown in combination in FIG. 12. FIG. 12 is also a cross-sectional view of the head model Mh and torso model Mb cut horizontally on the X-Z plane.
　第２実施形態では、第１実施形態と同様に、対象Ｓを表す対象画像Ｇｓ（３次元画像）が撮影装置２０により撮影される。また、第２実施形態では、第１実施形態と同様に、頭部モデルＭｈ、胴体モデルＭｂ、腕部モデルＭａおよび脚部モデルＭｌが対象画像Ｇｓに基づいて変形され対象モデルＭが生成される。また、第２実施形態では、第１実施形態と同様に、対象モデルＭの体積が対象Ｓの体積として推定される。また、推定した体積から対象Ｓの体重が算出される。第２実施形態では、第１実施形態と同様に、対象モデルＭの身長が対象Ｓの身長として推定される。 In the second embodiment, as in the first embodiment, a target image Gs (three-dimensional image) representing the target S is captured by the imaging device 20. Also, in the second embodiment, as in the first embodiment, the head model Mh, torso model Mb, arm model Ma and leg model Ml are deformed based on the target image Gs to generate the target model M. Also, in the second embodiment, as in the first embodiment, the volume of the target model M is estimated as the volume of the target S. Also, the weight of the target S is calculated from the estimated volume. In the second embodiment, as in the first embodiment, the height of the target model M is estimated as the height of the target S.
　図１２に示す通り、第２実施形態の対象モデルＭは、チューブモデルＭｔ（Ｍｔｈ、Ｍｔｂ）を含んで構成される。チューブモデルＭｔは、頭部モデルＭｈと胴体モデルＭｂとの組合せが示す対象Ｓに挿入すべき適切な長さの医療用チューブを表す。具体的には、チューブモデルＭｔは、対象Ｓに挿入された状態の胃管チューブ（医療用チューブの一例）の形状を示す。対象Ｓに挿入された状態の胃管チューブは、鼻孔から食道を介して胃に至るまでの経路において屈曲する。チューブモデルＭｔは、対象Ｓに挿入された状態の胃管チューブと同様に屈曲する。 As shown in FIG. 12, the target model M in the second embodiment is configured to include a tube model Mt (Mth, Mtb). The tube model Mt represents a medical tube of an appropriate length to be inserted into the target S represented by the combination of the head model Mh and the torso model Mb. Specifically, the tube model Mt represents the shape of a gastric tube (an example of a medical tube) when inserted into the target S. The gastric tube when inserted into the target S bends in the path from the nostril through the esophagus to the stomach. The tube model Mt bends in the same way as the gastric tube when inserted into the target S.
　チューブモデルＭｔは、頭部モデルＭｈと胴体モデルＭｂとの組合せのうち、対象Ｓに挿入された医療用チューブに対応する領域に設けられる。例えば、胃管チューブを表すチューブモデルＭｔは、対象Ｓの鼻孔から食道を介して胃の内部に至るまでの経路に対応する領域に位置する。なお、図１２には、対象Ｓの胃に対応する領域Ｓｔが示される。図１２に示す通り、チューブモデルＭｔの胴体モデルＭｂ側の端部は領域Ｓｔの内側に位置する。 The tube model Mt is provided in a region of the combination of the head model Mh and the torso model Mb that corresponds to the medical tube inserted into the subject S. For example, the tube model Mt representing a gastric tube is located in a region that corresponds to the path from the nostril of the subject S through the esophagus to the inside of the stomach. Note that Figure 12 shows the region St that corresponds to the stomach of the subject S. As shown in Figure 12, the end of the tube model Mt on the torso model Mb side is located inside the region St.
　図１２に示す通り、チューブモデルＭｔは、頭部モデルＭｈの内部に設けられる第１チューブモデルＭｔｈと胴体モデルＭｂの内部に設けられる第２チューブモデルＭｔｂとを含む。第２実施形態では、上述の第１実施形態と同様に、頭部モデル変形処理（図８（ａ）～（ｃ）参照）が実行され、頭部モデルＭｈが対象Ｓの頭部の大きさに変形される。また、頭部モデル変形処理において、頭部モデルＭｈと一体の画像として第１チューブモデルＭｔｈが変形される。また、第２実施形態では、上述の第１実施形態と同様に、胴体モデル変形処理（図９（ａ）（ｂ）参照）が実行され、胴体モデルＭｂが対象Ｓの胴体の大きさに変形される。また、胴体モデル変形処理において、胴体モデルＭｂと一体の画像として第２チューブモデルＭｔｂが変形される。 As shown in FIG. 12, the tube model Mt includes a first tube model Mth provided inside the head model Mh and a second tube model Mtb provided inside the torso model Mb. In the second embodiment, as in the first embodiment described above, a head model deformation process (see FIGS. 8(a) to (c)) is executed, and the head model Mh is deformed to the size of the head of the target S. In the head model deformation process, the first tube model Mth is deformed as an image integrated with the head model Mh. In the second embodiment, as in the first embodiment described above, a torso model deformation process (see FIGS. 9(a) and (b)) is executed, and the torso model Mb is deformed to the size of the torso of the target S. In the torso model deformation process, the second tube model Mtb is deformed as an image integrated with the torso model Mb.
　以上の第２実施形態では、対象Ｓの形状に近似する対象モデルＭが生成される。また、当該対象モデルＭは、当該対象Ｓに挿入すべき長さに変形（伸縮）されたチューブモデルＭｔ（第１チューブモデルＭｔｈと第２チューブモデルＭｔｂとの組合せ）を含む。したがって、対象モデルＭにおけるチューブモデルＭｔの長さを算出することで、対象Ｓに挿入すべき医療用チューブの長さが把握できる。 In the second embodiment described above, a target model M that approximates the shape of the target S is generated. The target model M also includes a tube model Mt (a combination of a first tube model Mth and a second tube model Mtb) that is deformed (stretched) to the length to be inserted into the target S. Therefore, by calculating the length of the tube model Mt in the target model M, the length of the medical tube to be inserted into the target S can be determined.
＜第３実施形態＞
　上述の第1実施形態では、対象モデルＭを生成し、当該対象モデルＭの体積に平均密度を掛けることで、対象Ｓの体重を推定した。しかし、対象Ｓの体重を推定する構成は、以上の例に限定されない。 Third Embodiment
In the above-described first embodiment, the weight of the target S is estimated by generating the target model M and multiplying the volume of the target model M by the average density. However, the configuration for estimating the weight of the target S is not limited to the above example.
　第３実施形態では、上述の第１実施形態と同様に対象モデルＭが生成される。また、第３実施形態では、対象モデルＭの所定部分の長さが、対象Ｓの体重を推定するための説明変数Ｘとして特定される。説明変数Ｘは、対象モデルＭのうち対象Ｓの体重と因果関係を有する所定部分の長さである。第３実施形態では、ｎ＋１個（ｎは正の整数）の説明変数（Ｘ０、Ｘ１、Ｘ２、Ｘ３…Ｘｎ）が特定される。例えば、対象モデルＭの体長、胸囲、胸幅、胸長、足の長さ、手の長さなどが説明変数Ｘとして特定される。 In the third embodiment, a target model M is generated in the same manner as in the first embodiment described above. Furthermore, in the third embodiment, the length of a specific portion of the target model M is identified as an explanatory variable X for estimating the weight of the target S. The explanatory variable X is the length of a specific portion of the target model M that has a causal relationship with the weight of the target S. In the third embodiment, n+1 (n is a positive integer) explanatory variables (X0, X1, X2, X3...Xn) are identified. For example, the body length, chest circumference, chest width, chest length, leg length, hand length, etc. of the target model M are identified as explanatory variables X.
　第３実施形態の情報処理装置１０は、各説明変数Ｘを特定すると、当該説明変数Ｘを以下の数３に代入して対象Ｓの体重Ｗを算出（推定）する。なお、数３における係数ｋ０～ｋｎは、例えば、重回帰分析により決定される。ただし、係数ｋ０～ｋｎの決定方法は重回帰分析に限定されない。 In the third embodiment, the information processing device 10, after identifying each explanatory variable X, calculates (estimates) the weight W of the subject S by substituting the explanatory variable X into the following equation 3. Note that the coefficients k0 to kn in equation 3 are determined, for example, by multiple regression analysis. However, the method of determining the coefficients k0 to kn is not limited to multiple regression analysis.
（数３）
Ｗ＝ｋ０×Ｘ０＋ｋ１×Ｘ１＋ｋ２×Ｘ２＋ｋ３×Ｘ３＋…＋ｋｎ×Ｘｎ (Equation 3)
W = k0 x X0 + k1 x X1 + k2 x X2 + k3 x X3 + ... + kn x Xn
　以上の第３実施形態によれば、上述の第１実施形態と同様な効果が奏せられる。なお、第３実施形態においても、第２実施形態と同様に、対象モデルＭにチューブモデルＭｔが含まれる構成としてもよい。また、説明変数（Ｘ０、Ｘ１、Ｘ２、Ｘ３…Ｘｎ）により対象Ｓの体重Ｗを推定する構成は以上の例に限定されない。例えば、説明変数Ｘと体重Ｗとの教師データにより機械学習された機械学習モデルを用いる構成としてもよい。具体的には、ランダムフォレストのアリゴリズムを用いて機械学習モデル（複数の決定木モデル）を生成する。以上の構成では、対象Ｓの対象モデルＭが生成されると、当該対象モデルＭから説明変数Ｘが特定され、当該説明変数Ｘが機械学習モデルに入力されると、当該対象Ｓの体重Ｗが決定（推定）される。 According to the third embodiment, the same effect as in the first embodiment can be achieved. In the third embodiment, as in the second embodiment, the target model M may include a tube model Mt. The configuration for estimating the weight W of the target S using explanatory variables (X0, X1, X2, X3...Xn) is not limited to the above example. For example, a machine learning model trained by machine learning using teacher data of the explanatory variables X and the weight W may be used. Specifically, a machine learning model (multiple decision tree models) is generated using a random forest algorithm. In the above configuration, when the target model M of the target S is generated, the explanatory variable X is identified from the target model M, and when the explanatory variable X is input to the machine learning model, the weight W of the target S is determined (estimated).
＜変形例＞
　以上の各形態は多様に変形される。具体的な変形の態様を以下に例示する。以下の例示から任意に選択された２以上の態様は適宜に併合され得る。 <Modification>
The above embodiments can be modified in various ways. Specific modified embodiments are exemplified below. Two or more embodiments selected from the following examples can be appropriately combined.
（１）上述の各形態において、腕部モデル変形処理は適宜に変更できる。例えば、腕部画像Ｇｓａは、図６を用いて上述した通り、対象Ｓの肩から肘までを表す第１腕部画像Ｇｓａ１と肘から指先までを表す第２腕部画像Ｇｓａ２とを含む。また、図６に示す通り、腕部モデルＭａは、対象Ｓの肩から肘までを表す第１腕部モデルＭａ１と肘から指先までを表す第２腕部モデルＭａ２とを含む。 (1) In each of the above-described embodiments, the arm model deformation process can be modified as appropriate. For example, as described above with reference to FIG. 6, the arm image Gsa includes a first arm image Gsa1 representing the area from the shoulder to the elbow of the subject S, and a second arm image Gsa2 representing the area from the elbow to the fingertips. Also, as shown in FIG. 6, the arm model Ma includes a first arm model Ma1 representing the area from the shoulder to the elbow of the subject S, and a second arm model Ma2 representing the area from the elbow to the fingertips.
　情報処理装置１０は、腕部モデル生成処理において、第１腕部画像Ｇｓａ１を円柱近似する。具体的には、情報処理装置１０は、第１腕部画像Ｇｓａ１の向き（以下「方向ベクトルＶ」）を主成分分析により特定する。情報処理装置１０は、中心線が方向ベクトルＶと平行な円柱のうち、側面が第１腕部画像Ｇｓａ１と略重なる円柱（以下「円柱Ｃｇ」）を、最小二乗法を用いて生成する。 In the arm model generation process, the information processing device 10 approximates the first arm image Gsa1 to a cylinder. Specifically, the information processing device 10 identifies the orientation of the first arm image Gsa1 (hereinafter, "direction vector V") by principal component analysis. The information processing device 10 uses the least squares method to generate a cylinder (hereinafter, "cylinder Cg") whose center line is parallel to the direction vector V and whose side surface approximately overlaps with the first arm image Gsa1.
　情報処理装置１０は、第１腕部画像Ｇｓａ１から円柱Ｃｇを生成したのと同様な方法で、第１腕部モデルＭａ１と近似する円柱（以下「円柱Ｃｍ」）を生成する。情報処理装置１０は、円柱Ｃｇの底面の直径Ｒｇ（太さ）を円柱Ｃｍの底面の直径Ｒｍで除した比率「Ｒｇ／Ｒｍ」を算出し、第１腕部モデルＭａ１の太さを比率「Ｒｇ／Ｒｍ」に応じて拡大または縮小する。また、情報処理装置１０は、円柱Ｃｇの高さＴｇ（長さ）を円柱Ｃｍの高さＴｍで除した比率「Ｔｇ／Ｔｍ」を算出し、第１腕部モデルＭａ１の長さを比率「Ｔｇ／Ｔｍ」に応じて拡大または縮小する。以上の構成によれば、対象Ｓの上腕の形状に近似する様に第１腕部モデルＭａ１の形状が変形される。同様に、第２腕部画像Ｇｓａ２を用いて第２腕部モデルＭａ２の形状を変形する。 The information processing device 10 generates a cylinder (hereinafter "cylinder Cm") that is similar to the first arm model Ma1 in a similar manner to the method of generating the cylinder Cg from the first arm image Gsa1. The information processing device 10 calculates the ratio "Rg/Rm" by dividing the diameter Rg (thickness) of the bottom surface of the cylinder Cg by the diameter Rm of the bottom surface of the cylinder Cm, and enlarges or reduces the thickness of the first arm model Ma1 according to the ratio "Rg/Rm". The information processing device 10 also calculates the ratio "Tg/Tm" by dividing the height Tg (length) of the cylinder Cg by the height Tm of the cylinder Cm, and enlarges or reduces the length of the first arm model Ma1 according to the ratio "Tg/Tm". According to the above configuration, the shape of the first arm model Ma1 is deformed so as to approximate the shape of the upper arm of the subject S. Similarly, the shape of the second arm model Ma2 is deformed using the second arm image Gsa2.
（２）上述の各形態において、脚部モデル変形処理は適宜に変更できる。例えば、脚部画像Ｇｓｌは、図６を用いて上述した通り、脚のつけ根から膝までを表す第１脚部画像Ｇｓｌ１と膝から爪先までを表す第２脚部画像Ｇｓｌ２とが含まれる。以上の各画像はセグメンテーションの技術により区別される。また、図６に示す通り、脚部モデルＭｌは、対象Ｓの太腿を表す第１脚部モデルＭｌ１と膝より爪先側を表す第２脚部モデルＭｌ２とを含む。 (2) In each of the above embodiments, the leg model deformation process can be modified as appropriate. For example, as described above with reference to FIG. 6, the leg image Gsl includes a first leg image Gsl1 representing the leg from the base to the knee, and a second leg image Gsl2 representing the leg from the knee to the toes. Each of the above images is distinguished by segmentation technology. Also, as shown in FIG. 6, the leg model Ml includes a first leg model Ml1 representing the thigh of the subject S, and a second leg model Ml2 representing the part from the knee to the toes.
　情報処理装置１０は、上述の変形例（１）における第１腕部モデルＭａ１の形状を変形した方法と同様な方法で、第１脚部画像Ｇｓｌ１を用いて第１脚部モデルＭｌ１の形状を変形する。以上の構成によれば、第１脚部モデルＭｌ１の形状が対象Ｓの太腿の形状に近似する様に変形される。また、情報処理装置１０は、第１腕部モデルＭａ１の形状を変形した方法と同様な方法で、第２脚部画像Ｇｓｌ２を用いて第２脚部モデルＭｌ２の形状を変形する。以上の構成によれば、第２脚部モデルＭｌ２の形状が対象Ｓの膝から爪先までの脚部の形状に近似する様に変形される。 The information processing device 10 uses the first leg image Gsl1 to deform the shape of the first leg model Ml1 in a manner similar to the manner in which the shape of the first arm model Ma1 in the above-mentioned modified example (1) was deformed. With the above configuration, the shape of the first leg model Ml1 is deformed so as to approximate the shape of the thigh of the target S. Furthermore, the information processing device 10 uses the second leg image Gsl2 to deform the shape of the second leg model Ml2 in a manner similar to the manner in which the shape of the first arm model Ma1 was deformed. With the above configuration, the shape of the second leg model Ml2 is deformed so as to approximate the shape of the leg from the knee to the toes of the target S.
（３）上述の各形態において、基準面Ｆａの決定方法は適宜に変更できる。例えば、撮影装置２０と平坦面Ａとの位置関係が固定される構成を想定する。以上の構成としては、撮影装置２０が保育器に固定される構成が考えられる。以上の構成では、撮影装置２０から平坦面Ａまでの距離が一定となり、平坦画像Ｇａが通る基準面Ｆａは３次元画像Ｇによらず共通になる。以上の構成では、情報処理装置１０に基準面Ｆａを予め記憶できる。したがって、平坦画像Ｇａから基準面Ｆａを特定するための処理が省略できるという利点がある。 (3) In each of the above embodiments, the method of determining the reference plane Fa can be changed as appropriate. For example, assume a configuration in which the positional relationship between the imaging device 20 and the flat surface A is fixed. As an example of the above configuration, a configuration in which the imaging device 20 is fixed to an incubator is conceivable. In the above configuration, the distance from the imaging device 20 to the flat surface A is constant, and the reference plane Fa through which the flat image Ga passes is common regardless of the three-dimensional image G. In the above configuration, the reference plane Fa can be stored in advance in the information processing device 10. Therefore, there is an advantage in that the process for identifying the reference plane Fa from the flat image Ga can be omitted.
（４）上述の第１実施形態では、頭部サイズ概算処理において、頭部画像Ｇｓｈを球近似することで、対象Ｓの頭部の形状（大きさ）を推定した。以上の構成に替えて、頭部画像Ｇｓｈを楕円球近似する構成を採用してもよい。 (4) In the first embodiment described above, the head size approximation process estimates the shape (size) of the head of the target S by approximating the head image Gsh as a sphere. Instead of the above configuration, a configuration may be adopted in which the head image Gsh is approximated as an elliptical sphere.
　図１３（ａ）および図１３（ｂ）は、頭部画像Ｇｓｈを楕円球近似可能な変形例を説明するための図である。当該変形例では、Ｚ－Ｘ平面に平行に、且つ、Ｙ軸方向に等間隔に頭部画像Ｇｓｈを切断した場合の各断面を楕円近似する。また、近似された各楕円を重ね合わせて生成された画像の大きさに、モデルＭｈを変形（拡大・縮小）する。図１３（ａ）には、楕円近似される前の頭部画像Ｇｓｈが示される。上述した通り、対象Ｓの頭部のうち基準面Ｆａ（平坦面Ａ）側は撮影されないため頭部画像Ｇｓｈから欠損する。なお、図１３（ａ）には、頭部画像Ｇｓｈから欠損した対象Ｓの頭部の形状が破線で示される。 FIGS. 13(a) and 13(b) are diagrams for explaining a modified example in which the head image Gsh can be approximated by an ellipse. In this modified example, each cross section of the head image Gsh cut parallel to the Z-X plane and at equal intervals in the Y-axis direction is approximated by an ellipse. The model Mh is then deformed (enlarged or reduced) to the size of the image generated by superimposing the approximated ellipses. FIG. 13(a) shows the head image Gsh before it is approximated by an ellipse. As described above, the reference plane Fa (flat plane A) side of the head of the subject S is not photographed and is therefore missing from the head image Gsh. Note that in FIG. 13(a), the shape of the head of the subject S missing from the head image Gsh is shown by a dashed line.
　ところで、対象Ｓ（新生児）は、自身の頭部を手で触っている場合がある。また、保育器内部の対象Ｓには医療用のチューブが取付けられている場合がある。以上の場合、対象Ｓの手（腕）やチューブで頭部の一部分が遮蔽され、遮蔽された部分が頭部画像Ｇｓｈから欠損する。図１３（ａ）の具体例では、対象Ｓの腕により頭部の一部分が遮蔽された場合を想定する。以上の場合、図１３（ａ）に示す通り、腕部画像Ｇｓａの下側（Ｚ軸方向逆側）の頭部画像Ｇｓｈは欠損する。また、腕部画像Ｇｓａが頭部画像Ｇｓｈの一部分として認識され得る。 Incidentally, subject S (newborn baby) may be touching his/her own head with his/her hand. Also, subject S inside an incubator may have medical tubes attached. In such cases, part of the head is occluded by subject S's hand (arm) or tube, and the occluded part is missing from the head image Gsh. In the specific example of Figure 13(a), it is assumed that part of the head is occluded by subject S's arm. In such cases, as shown in Figure 13(a), the head image Gsh below the arm image Gsa (opposite side in the Z-axis direction) is missing. Also, the arm image Gsa may be recognized as part of the head image Gsh.
　仮に、腕部画像Ｇｓａおよび頭部画像Ｇｓｈを含む画像を楕円近似した場合、近似された楕円は、実際の対象Ｓの頭部の形状と一致しない可能性が高くなる。したがって、以上の場合、実際の対象Ｓの頭部の形状が高精度に生成されないという事情がある。以上の事情を考慮して、当該変形例は、以上の不都合が抑制される構成を具備する。以上の構成について、以下で詳細に説明する。 If an image including an arm image Gsa and a head image Gsh is approximated by an ellipse, there is a high possibility that the approximated ellipse will not match the actual shape of the head of the subject S. Therefore, in the above case, the actual shape of the head of the subject S will not be generated with high accuracy. Taking the above into consideration, the present modified example has a configuration that suppresses the above inconveniences. The above configuration will be described in detail below.
　楕円は５個の未知数Ａ～Ｅを用いて「Ｘｉ ２＋ＡＸｉＹｉ＋ＢＹｉ ２＋ＣＸｉ＋ＤＹｉ＋Ｅ」（以下「方程式Ｅｉ」）で表される。したがって、頭部画像Ｇｓｈにおける５個以上の点の座標（Ｘｉ，Ｙｉ）の組合せを方程式Ｅｉに代入することで、頭部画像Ｇｓｈを通る楕円が求められる（未知数である５個の係数Ａ～Ｅが求められる）。当該変形例では、頭部画像Ｇｓｈにおける６個の点の座標（Ｘｉ，Ｙｉ）を用いて方程式Ｅｉを求める。また、当該変形例では、頭部画像Ｇｓｈにおける各点の組合せをランダムに変化させながら、ｎ個（例えばｎ＝１０００）の楕円の方程式Ｅｉを求める。 The ellipse is expressed by " Xi2 + AXi Yi + BYi2 + CXi + DYi + E" (hereinafter "equation Ei ") using five unknowns A to E. Therefore, by substituting a combination of coordinates ( Xi , Yi ) of five or more points in the head image Gsh into the equation Ei , an ellipse passing through the head image Gsh is obtained (five unknown coefficients A to E are obtained). In this modification, the equation Ei is obtained using the coordinates ( Xi , Yi ) of six points in the head image Gsh. In this modification, the equation Ei of n ellipses (for example, n=1000) is obtained while randomly changing the combination of each point in the head image Gsh .
　また、楕円の方程式Ｅｉを求めた後に、当該楕円に位置する頭部画像Ｇｓｈ上の点の個数（以下「評価値Ｖｉ」）を計数する。以上の変形例では、評価値Ｖｉが大きい楕円ほど、実際の対象Ｓの頭部の形状と近似すると推定できるという事情がある。例えば、図１３（ａ）の具体例では、頭部画像Ｇｓｈにおける点Ｐａ１～Ｐａ６の組合せにより楕円の方程式Ｅｉが求められ、点Ｐｂ１～Ｐｂ６の組合せにより他の楕円の方程式Ｅｉが求められた場合を想定する。また、図１３（ａ）の具体例では、点Ｐａ１～Ｐａ６は全て頭部画像Ｇｓｈに位置する一方、点Ｐｂ１～Ｐｂ６のうち点Ｐｂ５は、頭部画像Ｇｓｈではなく腕部画像Ｇｓａに位置する場合を想定する。以上の場合、点Ｐａ１～Ｐａ６の組合せにより求められた楕円は、実際の対象Ｓの頭部の形状と近似し易い。すなわち、点Ｐａ１～Ｐａ６の組合せにより求められた楕円は、頭部画像Ｇｓｈと重なる領域が多く、評価値Ｖｉが大きくなり易い。一方、点Ｐｂ１～Ｐｂ６の組合せにより求められた楕円は、実際の対象Ｓの頭部の形状と近似し難い。すなわち、点Ｐｂ１～Ｐｂ６の組合せにより求められた楕円は、頭部画像Ｇｓｈと重なる領域が少なく、評価値Ｖｉが小さくなり易い。 After the equation E i of the ellipse is obtained, the number of points on the head image Gsh located on the ellipse (hereinafter, "evaluation value V i ") is counted. In the above modified example, the ellipse with a larger evaluation value V i can be estimated to be more similar to the actual shape of the head of the target S. For example, in the specific example of FIG. 13(a), a case is assumed in which an equation E i of an ellipse is obtained by combining points Pa1 to Pa6 in the head image Gsh, and an equation E i of another ellipse is obtained by combining points Pb1 to Pb6. In addition, in the specific example of FIG. 13(a), a case is assumed in which points Pa1 to Pa6 are all located on the head image Gsh, while point Pb5 of points Pb1 to Pb6 is located on the arm image Gsa, not on the head image Gsh. In the above cases, the ellipse obtained by combining points Pa1 to Pa6 is more likely to approximate the actual shape of the head of the target S. That is, an ellipse obtained by combining points Pa1 to Pa6 has a large area that overlaps with the head image Gsh, and the evaluation value Vi is likely to be large. On the other hand, an ellipse obtained by combining points Pb1 to Pb6 is unlikely to approximate the actual shape of the head of target S. That is, an ellipse obtained by combining points Pb1 to Pb6 has a small area that overlaps with the head image Gsh, and the evaluation value Vi is likely to be small.
　以上の説明から理解される通り、評価値Ｖｉが大きい楕円ほど、実際の対象Ｓの頭部の形状と近似するという事情がある。以上の事情を考慮して、当該変形例では、ｎ個の楕円の評価値Ｖｉを求め、評価値Ｖｉが最大の楕円を、実際の対象Ｓの頭部の形状を表す楕円であると推定する。 As can be understood from the above explanation, the larger the evaluation value Vi of an ellipse, the more closely it resembles the actual shape of the head of the target S. Taking the above into consideration, in this modified example, the evaluation value Vi of n ellipses is calculated, and the ellipse with the largest evaluation value Vi is estimated to be the ellipse that represents the actual shape of the head of the target S.
　図１３（ｂ）は、楕円近似処理のフローチャートである。楕円近似処理により、対象Ｓの頭部の形状に近似する楕円の方程式Ｅｉが決定される。具体的には、楕円近似処理を開始すると、情報処理装置１０は、対象画像Gｓにおける６個の点Ｐをランダムに決定し（Ｓ１０１）、当該各点Ｐを通る楕円の方程式Ｅｉを算出する（Ｓ１０２）。また、情報処理装置１０は、方程式Ｅｉの楕円上に位置する対象画像Ｇｓｉ上の点の個数を評価値Ｖｉとして求める（Ｓ１０３）。その後、情報処理装置１０は、ｎ個の楕円の評価値Ｖｉを求めたか否かを判定する（Ｓ１０４）。評価値Ｖｉを求めた楕円の個数がｎ個未満の場合（Ｓ１０４：Ｎｏ）、情報処理装置１０は、上述のステップＳ１０１～Ｓ１０４を繰返し実行する。一方、評価値Ｖｉを求めた楕円の個数がｎ個に達した場合（Ｓ１０４：Ｙｅｓ）、情報処理装置１０は、ｎ個の楕円のうち評価値Ｖｉが最大の楕円の方程式Ｅｉを保存して（Ｓ１０５）楕円近似処理を終了する。 FIG. 13B is a flowchart of the ellipse approximation process. The ellipse approximation process determines an equation E i of an ellipse that approximates the shape of the head of the target S. Specifically, when the ellipse approximation process is started, the information processing device 10 randomly determines six points P in the target image Gs (S101), and calculates an equation E i of an ellipse that passes through each of the points P (S102). The information processing device 10 also obtains the number of points on the target image Gs i that are located on the ellipse of the equation E i as an evaluation value V i (S103). Thereafter, the information processing device 10 determines whether or not the evaluation values V i of n ellipses have been obtained (S104). If the number of ellipses for which the evaluation values V i have been obtained is less than n (S104: No), the information processing device 10 repeatedly executes the above-mentioned steps S101 to S104. On the other hand, if the number of ellipses for which the evaluation value V i has been calculated reaches n (S104: Yes), the information processing device 10 stores the equation E i of the ellipse having the maximum evaluation value V i among the n ellipses (S105) and terminates the ellipse approximation process.
　なお、楕円近似処理で求めた楕円の半径の長さは、基準面Ｆａから対象画像Ｇｓの頂点までの長さ（図１３（ａ）におけるＬｃ）の半分になる筈である。そこで、楕円近似処理で求めた楕円の半径の長さと長さＬｃとの差（誤差）を求め、当該差が所定の閾値を超えた場合、エラー判定される構成としてもよい。エラー判定された場合、再度、楕円近似処理が実行される構成としてもよい。また、方程式Ｅｉから楕円の傾きを算出し、算出結果に応じて楕円の向きを調整する構成としてもよい。 The length of the radius of the ellipse obtained by the ellipse approximation process should be half the length from the reference plane Fa to the vertex of the target image Gs (Lc in FIG. 13(a)). Therefore, the difference (error) between the length of the radius of the ellipse obtained by the ellipse approximation process and the length Lc may be calculated, and if the difference exceeds a predetermined threshold, an error may be determined. If an error is determined, the ellipse approximation process may be executed again. Also, the inclination of the ellipse may be calculated from the equation Ei , and the orientation of the ellipse may be adjusted according to the calculation result.
（５）上述の各形態において、推定される対象Ｓの情報は、体重および身長に限定されない。例えば、情報処理装置１０（計測部１６）が対象Ｓの頭囲、胸囲、腹囲、腕の太さ、脚の太さを推定する構成としてもよい。具体的には、情報処理装置１０は、生成した対象モデルＭの頭囲、胸囲、腹囲、腕の太さ、脚の太さを計測し、各計測結果を対象Ｓの頭囲、胸囲、腹囲、腕の太さ、脚の太さとして推定する構成としてもよい。 (5) In each of the above-mentioned embodiments, the estimated information of the target S is not limited to weight and height. For example, the information processing device 10 (measurement unit 16) may be configured to estimate the head circumference, chest circumference, waist circumference, arm thickness, and leg thickness of the target S. Specifically, the information processing device 10 may be configured to measure the head circumference, chest circumference, waist circumference, arm thickness, and leg thickness of the generated target model M, and estimate each measurement result as the head circumference, chest circumference, waist circumference, arm thickness, and leg thickness of the target S.
＜本実施形態の態様例の作用、効果のまとめ＞
＜第１態様＞
　本態様の情報処理装置（１０）は、平坦面（Ａ）に横たわる対象（Ｓ）を上方から撮影した３次元画像（Ｇ）を示す画像情報（Ｄｇ）を取得する取得部（１１）と、３次元画像に表示される各物体のうち対象を表す対象画像（Ｇｓ）を認識する認識部（１２）と、対象の下側部分が平坦面に接しているものとして、対象画像においては欠落した下側部分を含む対象の形状を、当該対象画像を用いて推定する推定部（１３）とを具備し、推定した対象の形状を用いて、対象の身体情報（体積、体重、身長など）を算出する。本態様によれば、対象に触れることなく当該対象の身体情報を推定できる。また、態様によれば、対象の下側部分が欠落した対象画像からであっても、当該対象の身体情報を高精度に推定可能になる。なお、本発明における「対象の形状を…推定する」とは、当該対象Ｓの身体情報を算出可能であれば足り、対象Ｓの形状を細部にわたり推定することに限定されない。 <Summary of the functions and effects of the exemplary embodiment>
<First aspect>
The information processing device (10) of this aspect includes an acquisition unit (11) that acquires image information (Dg) showing a three-dimensional image (G) of a target (S) lying on a flat surface (A) taken from above, a recognition unit (12) that recognizes a target image (Gs) that represents the target among the objects displayed in the three-dimensional image, and an estimation unit (13) that estimates the shape of the target, including the lower part missing in the target image, by using the target image, assuming that the lower part of the target is in contact with the flat surface, and calculates the physical information (volume, weight, height, etc.) of the target using the estimated shape of the target. According to this aspect, the physical information of the target can be estimated without touching the target. In addition, according to the aspect, even if the lower part of the target is missing from the target image, the physical information of the target can be estimated with high accuracy. Note that "estimating the shape of the target" in the present invention is sufficient as long as the physical information of the target S can be calculated, and is not limited to estimating the shape of the target S in detail.
＜第２態様＞
　本態様の情報処理装置は、対象のモデル（頭部モデルなど）の形状を示すモデル情報を記憶する記憶部（１４）を具備し、認識部は、３次元画像のうち平坦面（Ａ）を表す平坦画像（Ｇａ）を認識し、推定部は、平坦画像を通る基準面（Ｆａ）を特定し、対象画像における対象点（図８（ｂ）のＰｈ６、図９（ａ）のＰｂ６）から基準面までの上下軸方向（Ｚ軸方向）における距離（Ｌｈｚ、Ｌｂｚ）と当該対象点に対応するモデルにおける対応点（図８（ｃ）のＱｉ６、図９（ｂ）のＱｃ６）から当該モデルの下端部までの上下軸方向における距離（Ｌｉｚ、Ｌｃｚ）との比率に応じてモデルを変形し、変形後のモデルの形状を、前記対象の形状として推定する。本態様によれば、例えば、例えば下側部分が欠落した対象画像のみを用いて対象の身体情報を推定する構成と比較して、当該対象の身体情報を高精度に推定可能になる。 <Second aspect>
The information processing device of this aspect includes a storage unit (14) that stores model information indicating the shape of a target model (such as a head model), a recognition unit recognizes a flat image (Ga) that represents a flat surface (A) in a three-dimensional image, and an estimation unit identifies a reference surface (Fa) that passes through the flat image, and deforms the model according to the ratio of the distance (Lhz, Lbz) from a target point (Ph6 in FIG. 8(b), Pb6 in FIG. 9(a)) in the target image to the reference surface in the up-down axis direction (Z axis direction) to the distance (Liz, Lcz) from a corresponding point (Qi6 in FIG. 8(c), Qc6 in FIG. 9(b)) in the model corresponding to the target point to the lower end of the model in the up-down axis direction, and estimates the shape of the deformed model as the shape of the target. According to this aspect, for example, it is possible to estimate the physical information of the target with high accuracy, compared to a configuration in which the physical information of the target is estimated using only a target image with a missing lower portion.
＜第３態様＞
　本態様の情報処理装置は、身体情報として対象の体重を算出する。本態様によれば、対象に触れることなく当該対象の体重を推定できる。 <Third aspect>
According to the present aspect, the information processing device calculates the weight of the subject as physical information, and can estimate the weight of the subject without touching the subject.
＜第４態様＞
　本態様の情報処理装置は、身体情報として対象の身長を算出する。本態様によれば、対象に触れることなく当該対象の身長を推定できる。 <Fourth aspect>
According to the present aspect, the information processing device calculates the height of the target as physical information, and can estimate the height of the target without touching the target.
＜第５態様＞
　本態様の情報処理装置１０は、モデルは、対象に挿入された状態の医療用チューブ（例えば、胃管チューブ）を表すチューブモデル（Ｍｔ）を含み、推定部は、チューブモデル以外のモデル（頭部モデルＭｈ、胸部モデルＭｂ）を変形する際に、当該モデルとチューブモデルとを一体的に変形可能である。本態様によれば、最適な医療用チューブの長さを、対象に触れることなく推定可能になる。 <Fifth aspect>
In the information processing device 10 of this aspect, the model includes a tube model (Mt) representing a medical tube (e.g., a gastric tube) inserted into a target, and the estimation unit is capable of deforming the model and the tube model together when deforming models other than the tube model (head model Mh, chest model Mb). According to this aspect, it is possible to estimate the optimal length of the medical tube without touching the target.
＜第６態様＞
　本態様の情報処理装置（１０）は、平坦面（Ａ）に横たわる対象（Ｓ）を上方から撮影した３次元画像（Ｇ）を示す画像情報（Ｄｇ）を取得する取得部（１１）と、３次元画像に表示される各物体を表す各画像うち対象を表す対象画像（Ｇｓ）と平坦面を表す平坦画像（Ｇａ）とを認識する認識部（１２）と、対象のモデルの形状を示すモデル情報を記憶する記憶部（１４）と、対象の体重を推定する推定部（１３）とを具備し、推定部は、平坦画像を通る基準面を特定し、対象画像における対象点から基準面までの上下軸方向における距離と当該対象点に対応するモデルにおける対応点から当該モデルの下端部までの上下軸方向における距離との比率に応じてモデルを変形し、変形後のモデルにおける所定部分の長さを対象の体重を推定するための説明変数として特定し、特定した説明変数を用いて、対象の体重を推定する。以上の本態様によれば、上述の第3態様と同様な効果が奏せられる。 <Sixth aspect>
The information processing device (10) of this embodiment includes an acquisition unit (11) for acquiring image information (Dg) showing a three-dimensional image (G) of a subject (S) lying on a flat surface (A) taken from above, a recognition unit (12) for recognizing a subject image (Gs) showing the subject and a flat image (Ga) showing the flat surface among the images showing each object displayed in the three-dimensional image, a storage unit (14) for storing model information showing the shape of the subject model, and an estimation unit (13) for estimating the subject's weight, and the estimation unit specifies a reference plane passing through the flat image, deforms the model according to the ratio of the distance in the vertical axis direction from a subject point in the subject image to the reference plane and the distance in the vertical axis direction from a corresponding point in the model corresponding to the subject point to the lower end of the model, specifies the length of a predetermined part of the deformed model as an explanatory variable for estimating the subject's weight, and estimates the subject's weight using the specified explanatory variable. According to the above-mentioned embodiment, the same effect as that of the third embodiment can be achieved.
＜第７態様＞
　本態様の新生児の体重の推定方法は、新生児の３次元画像を撮影するステップと、３次元画像から新生児の形状をコンピュータが推定するステップ（図１１のＳ１～Ｓ９）と、推定された形状から新生児の体重をコンピュータが推定するステップ（同図のＳ１０）とを具備する。新生児にとって、身体情報を体重計により計測するために触れられる行為は、侵襲的な行為に相当し得る。本態様によれば、新生児に触れることなく当該新生児の体重が推定されるため、侵襲的な行為の回数を抑制できるという利点がある。 <Seventh aspect>
The method for estimating the weight of a newborn according to this embodiment includes the steps of taking a three-dimensional image of the newborn, estimating the shape of the newborn from the three-dimensional image by a computer (S1 to S9 in FIG. 11), and estimating the weight of the newborn from the estimated shape by a computer (S10 in FIG. 11). For a newborn, the act of touching the newborn to measure physical information with a weighing scale can be considered an invasive act. According to this embodiment, the weight of the newborn is estimated without touching the newborn, which has the advantage of reducing the number of invasive acts.
１０…情報処理装置、１１…取得部、１２…認識部、１３…推定部、１４…記憶部、１５…算出部、１６…計測部、２０…撮影装置。 10: information processing device, 11: acquisition unit, 12: recognition unit, 13: estimation unit, 14: storage unit, 15: calculation unit, 16: measurement unit, 20: imaging device.

Claims (7)

  1. 　平坦面に横たわる対象を上方から撮影した３次元画像を示す画像情報を取得する取得部と、
    　前記３次元画像に表示される各物体のうち前記対象を表す対象画像を認識する認識部と、
    　前記対象の下側部分が前記平坦面に接しているものとして、前記対象画像においては欠落した下側部分を含む前記対象の形状を、当該対象画像を用いて推定する推定部とを具備し、
    　推定した前記対象の形状を用いて、前記対象の身体情報を算出する
    　情報処理装置。     an acquisition unit that acquires image information indicating a three-dimensional image of an object lying on a flat surface captured from above;
    a recognition unit that recognizes an object image representing the target among the objects displayed in the three-dimensional image;
    an estimation unit that estimates a shape of the object including a lower portion missing in the target image by assuming that a lower portion of the object is in contact with the flat surface, using the target image;
    An information processing device that calculates physical information of the target using the estimated shape of the target.
  2. 　前記対象のモデルの形状を示すモデル情報を記憶する記憶部を具備し、
    　前記認識部は、前記３次元画像のうち前記平坦面を表す平坦画像を認識し、
    　前記推定部は、
    　前記平坦画像を通る基準面を特定し、
    　前記対象画像における対象点から前記基準面までの上下軸方向における距離と当該対象点に対応する前記モデルにおける対応点から当該モデルの下端部までの上下軸方向における距離との比率に応じて前記モデルを変形し、
    　変形後の前記モデルの形状を、前記対象の形状として推定する
    　請求項１に記載の情報処理装置。     A storage unit is provided for storing model information indicating a shape of a model of the object,
    The recognition unit recognizes a flat image representing the flat surface from the three-dimensional image,
    The estimation unit is
    identifying a reference plane passing through the flat image;
    deforming the model according to a ratio of a distance in an up-down axis direction from a target point in the target image to the reference plane and a distance in an up-down axis direction from a corresponding point in the model corresponding to the target point to a lower end of the model;
    The information processing apparatus according to claim 1 , wherein a shape of the model after deformation is estimated as a shape of the target.
  3. 　前記身体情報として前記対象の体重を算出する
    　請求項１または請求項２に記載の情報処理装置。     The information processing device according to claim 1 , further comprising: a weight of the subject being calculated as the physical information.
  4. 　前記身体情報として前記対象の身長を算出する
    　請求項１または請求項２に記載の情報処理装置。     The information processing device according to claim 1 , further comprising: a height of the subject calculated as the physical information.
  5. 　前記モデルは、前記対象に挿入された状態の医療用チューブを表すチューブモデルを含み、
    　前記推定部は、前記チューブモデル以外の前記モデルを変形する際に、当該モデルと前記チューブモデルとを一体的に変形可能である
    　請求項２に記載の情報処理装置。     the model includes a tube model representing a medical tube inserted into the subject;
    The information processing device according to claim 2 , wherein when the estimation unit deforms the model other than the tube model, the estimation unit is capable of deforming the model and the tube model together.
  6. 　平坦面に横たわる対象を上方から撮影した３次元画像を示す画像情報を取得する取得部と、
    　前記３次元画像に表示される各物体を表す各画像うち前記対象を表す対象画像と前記平坦面を表す平坦画像とを認識する認識部と、
    　前記対象のモデルの形状を示すモデル情報を記憶する記憶部と、
    　前記対象の体重を推定する推定部とを具備し、
    　前記推定部は、
    　前記平坦画像を通る基準面を特定し、
    　前記対象画像における対象点から前記基準面までの上下軸方向における距離と当該対象点に対応する前記モデルにおける対応点から当該モデルの下端部までの上下軸方向における距離との比率に応じて前記モデルを変形し、
    　変形後の前記モデルにおける所定部分の長さを前記対象の体重を推定するための説明変数として特定し、
    　特定した前記説明変数を用いて、前記対象の体重を推定する
    　情報処理装置。     an acquisition unit that acquires image information indicating a three-dimensional image of an object lying on a flat surface captured from above;
    a recognition unit that recognizes an object image representing the target and a flat image representing the flat surface among the images representing the objects displayed in the three-dimensional image;
    A storage unit that stores model information indicating a shape of a model of the object;
    and an estimation unit for estimating a weight of the subject,
    The estimation unit is
    identifying a reference plane passing through the flat image;
    deforming the model according to a ratio of a distance in an up-down axis direction from a target point in the target image to the reference plane and a distance in an up-down axis direction from a corresponding point in the model corresponding to the target point to a lower end of the model;
    Identifying a length of a predetermined portion of the model after deformation as an explanatory variable for estimating the body weight of the subject;
    The information processing device estimates a weight of the subject by using the identified explanatory variables.
  7. 　新生児の３次元画像を撮影するステップと、
    　前記３次元画像から前記新生児の形状をコンピュータが推定するステップと、
    　推定された前記形状から前記新生児の身体情報をコンピュータが推定するステップと
    　を具備する新生児の体重の推定方法。     taking a three-dimensional image of the newborn;
    A step of estimating the shape of the newborn baby from the three-dimensional image by a computer;
    and a step of estimating physical information of the newborn from the estimated shape by a computer.
PCT/JP2023/041637 2022-12-01 2023-11-20 Information processing device and method of estimating neonate's body weight WO2024116936A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2022192854 2022-12-01
JP2022-192854 2022-12-01

Publications (1)

Publication Number Publication Date
WO2024116936A1 true WO2024116936A1 (en) 2024-06-06

Family

ID=91323730

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2023/041637 WO2024116936A1 (en) 2022-12-01 2023-11-20 Information processing device and method of estimating neonate's body weight

Country Status (1)

Country Link
WO (1) WO2024116936A1 (en)

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2020111269A1 (en) * 2018-11-30 2020-06-04 Ａｒｉｔｈｍｅｒ株式会社 Dimensional data calculation device, product manufacturing device, information processing device, silhouette image generating device, and terminal device
JP2020181400A (en) * 2019-04-25 2020-11-05 キヤノン株式会社 Image processing system, imaging device, control method, program and recording medium
WO2020260631A1 (en) * 2019-06-26 2020-12-30 Viking Genetics Fmba Weight determination of an animal based on 3d imaging
WO2021084677A1 (en) * 2019-10-31 2021-05-06 日本電気株式会社 Image processing device, image processing method, and non-transitory computer-readable medium having image processing program stored thereon
JP2022059493A (en) * 2020-10-01 2022-04-13 株式会社Preferred Networks Model generation method, model generation device, image processing method, and image processing device

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2020111269A1 (en) * 2018-11-30 2020-06-04 Ａｒｉｔｈｍｅｒ株式会社 Dimensional data calculation device, product manufacturing device, information processing device, silhouette image generating device, and terminal device
JP2020181400A (en) * 2019-04-25 2020-11-05 キヤノン株式会社 Image processing system, imaging device, control method, program and recording medium
WO2020260631A1 (en) * 2019-06-26 2020-12-30 Viking Genetics Fmba Weight determination of an animal based on 3d imaging
WO2021084677A1 (en) * 2019-10-31 2021-05-06 日本電気株式会社 Image processing device, image processing method, and non-transitory computer-readable medium having image processing program stored thereon
JP2022059493A (en) * 2020-10-01 2022-04-13 株式会社Preferred Networks Model generation method, model generation device, image processing method, and image processing device

Similar Documents

Publication Publication Date Title
CN109008944B (en) Sight line measuring device, ROM, and sight line measuring method
KR101930652B1 (en) Gait analysis system and computer program recorded on recording medium
US7862172B2 (en) Gaze direction measuring method and gaze direction measuring device
Gaber et al. Quantifying facial paralysis using the kinect v2
JP6973258B2 (en) Image analyzers, methods and programs
US20120310075A1 (en) Medical measurement system and method
JP2008136789A (en) Eyeball parameter estimating instrument and method
US20220207921A1 (en) Motion recognition method, storage medium, and information processing device
JP2000251078A (en) Method and device for estimating three-dimensional posture of person, and method and device for estimating position of elbow of person
CN110059670B (en) Non-contact measuring method and equipment for head and face, limb movement angle and body posture of human body
KR20140003265A (en) Method for measuring movement angle of joint
KR102357001B1 (en) Scoliosis Diagnosis Method and System using 3D Depth Camera
KR102553141B1 (en) Method and device for providing alopecia information
JP4527471B2 (en) 3D fundus image construction and display device
JP7283841B2 (en) Gaze measurement device
US11179096B2 (en) Scoliosis diagnosis assistance device, scoliosis diagnosis assistance method, and program
WO2024116936A1 (en) Information processing device and method of estimating neonate&#39;s body weight
US20230240594A1 (en) Posture assessment program, posture assessment apparatus, posture assessment method, and posture assessment system
JP6288770B2 (en) Face detection method, face detection system, and face detection program
KR101788960B1 (en) Automatic Measuring System for Range of Motion and Automatic Measuring Method for range of Motion Using the Same
KR102310964B1 (en) Electronic Device, Method, and System for Diagnosing Musculoskeletal Symptoms
JP6652263B2 (en) Mouth region detection device and mouth region detection method
WO2020261403A1 (en) Height estimation device, height estimation method, and non-transitory computer-readable medium having program stored thereon
JP6525180B1 (en) Target number identification device
KR102456055B1 (en) Apparatus and method for quantitatively analyzing and evaluating posture to train exercise by repetitive motion