CN114220149A - Method, device, equipment and storage medium for acquiring true value of head posture - Google Patents
Method, device, equipment and storage medium for acquiring true value of head posture Download PDFInfo
- Publication number
- CN114220149A CN114220149A CN202111501742.9A CN202111501742A CN114220149A CN 114220149 A CN114220149 A CN 114220149A CN 202111501742 A CN202111501742 A CN 202111501742A CN 114220149 A CN114220149 A CN 114220149A
- Authority
- CN
- China
- Prior art keywords
- image acquisition
- face
- acquisition equipment
- key point
- point information
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/22—Image preprocessing by selection of a specific region containing or referencing a pattern; Locating or processing of specific regions to guide the detection or recognition
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/20—Scenes; Scene-specific elements in augmented reality scenes
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/59—Context or environment of the image inside of a vehicle, e.g. relating to seat occupancy, driver state or inner lighting conditions
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Theoretical Computer Science (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Oral & Maxillofacial Surgery (AREA)
- Human Computer Interaction (AREA)
- Length Measuring Devices By Optical Means (AREA)
- Image Analysis (AREA)
Abstract
The application discloses a method, a device, equipment and a storage medium for acquiring a true value of a head posture. The method comprises the steps that a plurality of image acquisition devices are used for acquiring images of a target object at the same moment, two-dimensional face key points marked in a plurality of acquired images are used as data bases of three-dimensional reconstruction target object face key points, and then a head posture true value of the target object corresponding to the images is acquired. Because this scheme need not acquire head posture truth with the help of wearing equipment, consequently not influenced by wearing angle. The precision of the acquired true value of the head posture is ensured by simultaneously using a plurality of image acquisition devices and three-dimensional reconstruction of key points of the human face. Further, when the head posture truth value is applied to the fields of driver fatigue degree analysis, virtual reality body sensing games, commodity purchase desire analysis, face verification and the like, the analysis result of the data can be more accurate.
Description
Technical Field
The present application relates to the field of image processing technologies, and in particular, to a method, an apparatus, a device, and a storage medium for acquiring a true value of a head pose.
Background
The head pose truth values include Yaw angle Yaw, Pitch, and Roll angles Roll. Wherein the yaw angle, the pitch angle and the roll angle are respectively the angles of rotation in the euler angle vector coordinate system with respect to the y-axis, the x-axis and the z-axis. The head posture truth value is applied to various fields, such as the fields of analysis of the fatigue degree of a driver, virtual reality motion sensing games, commodity purchase desire analysis, face verification and the like.
The head pose true value can be obtained through a sensor in the wearable device at present. However, the wearing angle of the wearable device easily affects the accuracy of the acquired true value of the head posture. In combination with the above application fields, if the accuracy of the true value of the head posture is not good, the accuracy of the result of data analysis is easily affected. For example, the accuracy of the acquired true value of the head posture of the driver is poor, so that the driver with high fatigue degree is judged as a non-fatigue state by mistake, and the voice prompt is not made accurately in time. Therefore, the improvement of the acquisition precision of the true value of the head posture is an urgent technical problem to be solved.
Disclosure of Invention
Based on the above problems, the present application provides a method, an apparatus, a device, and a storage medium for acquiring a true value of a head pose.
The embodiment of the application discloses the following technical scheme:
in a first aspect, the present application provides a method for acquiring a true value of a head pose, including:
acquiring images acquired by a plurality of image acquisition devices on a target object at the same moment;
labeling the face key points in the images acquired by each image acquisition device in the plurality of image acquisition devices to obtain face key point information corresponding to each image acquisition device;
reconstructing human face three-dimensional key point information corresponding to each image acquisition device based on human face key point information corresponding to each image acquisition device and parameters of each image acquisition device;
establishing a face coordinate system based on face three-dimensional key point information corresponding to target image acquisition equipment, wherein the target image acquisition equipment is one of the image acquisition equipment;
and acquiring a head posture true value of the target object corresponding to the target image according to the coordinate system of the target image acquisition equipment and the face coordinate system, wherein the target image is an image acquired by the target image acquisition equipment on the target object at the moment.
In an optional implementation manner, the obtaining a true value of a head pose of the target object corresponding to the target image according to the coordinate system of the target image capturing device and the face coordinate system includes:
according to the face coordinate system and the coordinate system of the target image acquisition equipment, obtaining a rotation matrix of the face coordinate system relative to the coordinate system of the target image acquisition equipment;
and acquiring a head posture true value of the target object corresponding to the target image according to the rotation matrix.
In an optional implementation manner, the establishing a face coordinate system based on the face three-dimensional key point information corresponding to the target image acquisition device includes:
determining a face plane based on the face three-dimensional key point information corresponding to the target image acquisition equipment;
determining a normal vector of the face plane according to the face plane;
and establishing a face coordinate system of the target object based on the face plane and the normal vector of the face plane.
In an optional implementation manner, the reconstructing three-dimensional face keypoint information corresponding to each image acquisition device based on face keypoint information corresponding to each image acquisition device and parameters of each image acquisition device includes:
reconstructing human face three-dimensional key point information corresponding to the target image acquisition equipment by a triangulation reconstruction method according to human face key point information corresponding to the target image acquisition equipment, human face key point information corresponding to reference image acquisition equipment, internal parameters of the target image acquisition equipment, internal parameters of the reference image acquisition equipment and external parameters between the target image acquisition equipment and the reference image acquisition equipment; the reference image acquisition device is any image acquisition device except the target image acquisition device in the plurality of image acquisition devices;
and obtaining the face three-dimensional key point information corresponding to the reference image acquisition equipment according to the face three-dimensional key point information corresponding to the target image acquisition equipment and the external parameters.
In an optional implementation manner, the method for acquiring the head pose truth value further includes:
and storing the image and the head pose true value which correspond to each other as an image true value pair.
In an optional implementation manner, the acquiring images acquired by a plurality of image acquisition devices on a target object at the same time includes:
providing a plurality of different lighting conditions to a space in which the target object is located;
and acquiring images acquired by the plurality of image acquisition devices on the target object at the same moment under the same illumination condition.
In an alternative implementation, when the plurality of image acquisition devices acquire the image of the target object, the target object is seated on a seat, and the seat is used for simulating a seat in a real vehicle environment; the setting parameters of the image acquisition equipment relative to the seat are determined according to the setting parameters of a simulation object in the real vehicle environment relative to the seat;
the simulated object includes at least one of:
a pillar a, a pillar B, a dashboard, a front windshield, or a left side vehicle glass.
In a second aspect, the present application provides an apparatus for acquiring a true value of a head pose, including:
the image acquisition module is used for acquiring images acquired by the plurality of image acquisition devices on the target object at the same moment;
the key point labeling module is used for labeling the face key points in the images acquired by the image acquisition equipment in the image acquisition equipment to obtain the face key point information corresponding to the image acquisition equipment;
the three-dimensional key point reconstruction module is used for reconstructing human face three-dimensional key point information corresponding to each image acquisition device based on human face key point information corresponding to each image acquisition device and parameters of each image acquisition device;
the coordinate system establishing module is used for establishing a face coordinate system based on face three-dimensional key point information corresponding to target image acquisition equipment, and the target image acquisition equipment is one of the image acquisition equipment;
and the truth value acquisition module is used for acquiring a head posture truth value of the target object corresponding to the target image according to the coordinate system of the target image acquisition equipment and the face coordinate system, wherein the target image is an image acquired by the target image acquisition equipment on the target object at the moment.
In a third aspect, the present application provides a device for acquiring a true value of a head posture, including a processor and a memory; the memory is used for storing a computer program; the processor is configured to perform the method for acquiring a head pose truth value as provided by the first aspect according to the computer program.
In a fourth aspect, the present application provides a computer-readable storage medium for storing a computer program which, when executed by a processor, performs the method for obtaining a true value of a head pose as provided in the first aspect.
Compared with the prior art, the method has the following beneficial effects:
the application provides a method for acquiring a true value of a head posture. The method comprises the steps that a plurality of image acquisition devices are used for acquiring images of a target object at the same moment, two-dimensional face key points marked in a plurality of acquired images are used as data bases of three-dimensional reconstruction target object face key points, and then a head posture true value of the target object corresponding to the images is acquired. Because this scheme need not acquire head posture truth with the help of wearing equipment, consequently not influenced by wearing angle. The precision of the acquired true value of the head posture is ensured by simultaneously using a plurality of image acquisition devices and three-dimensional reconstruction of key points of the human face. Further, when the head posture truth value is applied to the fields of driver fatigue degree analysis, virtual reality body sensing games, commodity purchase desire analysis, face verification and the like, the analysis result of the data can be more accurate.
Drawings
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings needed to be used in the description of the embodiments or the prior art will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art that other drawings can be obtained according to the drawings without inventive exercise.
Fig. 1 is a flowchart of a method for acquiring a true value of a head pose according to an embodiment of the present disclosure;
fig. 2 is a schematic view of a scene where images of a target object are acquired by a plurality of image acquisition devices according to an embodiment of the present application;
fig. 3 is a schematic structural diagram of an apparatus for acquiring a true value of a head posture according to an embodiment of the present application.
Detailed Description
As described previously, obtaining true values of head pose currently typically requires relying on sensors of the wearable device. However, the angle of the device worn by a person may affect the accuracy of the acquired true value of the head posture, so that when the true value of the head posture is applied to data analysis, the accuracy of the analysis result is also affected. The inventor researches and proposes a technical scheme that a plurality of image acquisition devices are used for acquiring images of a person at the same time and acquiring a true value of the head posture by the images. According to the scheme, a wearing device does not need to be worn by a person, and a high-precision head posture truth value can be obtained only by means of the image. Furthermore, the accuracy of the data analysis result obtained when the head posture truth value is applied to data analysis is ensured.
In order to make the technical solutions of the present application better understood, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
Referring to fig. 1, this figure is a flowchart of a method for acquiring a true value of a head pose according to an embodiment of the present application. The method as shown in fig. 1 comprises the following steps:
s101: images acquired by a plurality of image acquisition devices on a target object at the same time are acquired.
In the embodiment of the application, the method includes that a plurality of image acquisition devices acquire images of a target object at the same time. The target object refers to a subject that needs to acquire a true value of the head pose. For example, when a true value of the head posture of mr a needs to be acquired, mr a is taken as a target object, and images of mr a at the same time are acquired by a plurality of image acquisition devices.
The image capture device may be any device capable of capturing and forming an image, and may be, for example, an RGB camera. The type and model of the image capturing device are not limited herein. The plurality of image capturing devices are specifically two or more image capturing devices. I.e. requiring simultaneous image acquisition of the target object, at least two image acquisition devices are used. The setting parameters of the plurality of image acquisition devices are different or not completely the same. The setting parameters include position, height, angle, and the like.
S102: and labeling the face key points in the images acquired by each image acquisition device in the plurality of image acquisition devices to obtain the face key point information corresponding to each image acquisition device.
In the step, the face key points are labeled on the images acquired by the image acquisition equipment. Annotated face keypoints may include, but are not limited to: eyebrow, eyebrow peak, tail, inner canthus, outer canthus, pupil center, nose wing, nostril, corner of mouth, etc. As an example, 70 key points of a human face in an image are annotated. The number of labels for the key points is not limited herein.
Identifying and marking the key points of the face in the image containing the face belongs to a mature technology in the field, so the specific implementation mode of the step is not limited here. The face key point information may include: the pixel coordinates of the key point in the image. Because the image is a two-dimensional image, the face key point information obtained by labeling also refers to the information of the face key points in the two-dimensional image.
S103: and reconstructing the three-dimensional key point information of the face corresponding to each image acquisition device based on the key point information of the face corresponding to each image acquisition device and the parameters of each image acquisition device.
The execution purpose of the step is to construct the information of the three-dimensional face key points through the two-dimensional face key points marked in the images acquired by a plurality of image acquisition devices at the same time and different image acquisition devices. In specific implementation, for each image, a group of corresponding human face three-dimensional key point information is acquired. Because the image and the image acquisition equipment have the correspondence, the image acquisition equipment can be understood to have the correspondence with the obtained human face three-dimensional key point information. Taking a certain acquisition time as an example, the three-dimensional face key point information corresponding to each image acquisition device needs to be reconstructed in the step. For example, there are 14 image capturing devices in total, and for the same capturing time, it is necessary to obtain the three-dimensional key point information of the face corresponding to each of the 14 image capturing devices.
Of the 14 image capturing devices, different image capturing devices have unique labels, such as device No. 1, device No. 2, and device No. … 14. The order of the reference numerals is not limited herein. The numbering may be done, for example, by sequentially incrementing the ordinal number in one direction.
An exemplary implementation of S103 is described below:
and reconstructing the three-dimensional key point information of the face corresponding to the target image acquisition equipment by a triangulation reconstruction method according to the key point information of the face corresponding to the target image acquisition equipment, the key point information of the face corresponding to the reference image acquisition equipment, the internal parameters of the target image acquisition equipment, the internal parameters of the reference image acquisition equipment and the external parameters between the target image acquisition equipment and the reference image acquisition equipment. The reference image capturing apparatus is any one of the plurality of image capturing apparatuses other than the target image capturing apparatus. And obtaining the face three-dimensional key point information corresponding to the reference image acquisition equipment according to the face three-dimensional key point information corresponding to the target image acquisition equipment and the external parameters. The face three-dimensional key point information may include three-dimensional coordinates of the face key point in the image acquisition device coordinate system where the face key point is located.
Assume that device No. 4 is a target image capturing device and device No. 5 is a reference image capturing device. When the step is executed, the face key point information corresponding to the No. 4 device, the face key point information corresponding to the No. 5 device, the internal parameters of the No. 4 device, the internal parameters of the No. 5 device and the external parameters between the No. 4 device and the No. 5 device are reconstructed to obtain the face three-dimensional key point information corresponding to the No. 4 device through a triangularization reconstruction method. When the three-dimensional key point information of the human face is triangularly reconstructed, the three-dimensional key point information can be realized through a triangularization reconstruction function, and the triangularization reconstruction belongs to a mature technology in the field, so the specific implementation process is not repeated. After the face three-dimensional key point information corresponding to the number 4 device is obtained, because external parameters between different image acquisition devices are calibrated in advance, the face three-dimensional key point information corresponding to the number 4 device can be converted into coordinate systems of other image acquisition devices by means of the external parameters between the other image acquisition devices and the number 4 device, and the face three-dimensional key point information corresponding to the other image acquisition devices is obtained. Therefore, the three-dimensional key point information of the human face corresponding to each of the 14 image acquisition devices can be obtained.
In the following, only the three-dimensional key point information of the face corresponding to one target image capturing device is taken as an example, and the manner of acquiring the true value of the head pose is described. The three-dimensional key point information of the human face corresponding to other image acquisition devices can also execute corresponding operations in the same implementation mode. For convenience of explanation, in the embodiment of the present application, the image of the target object acquired by the target image acquisition device at the above time is referred to as a target image.
S104: and establishing a face coordinate system based on the face three-dimensional key point information corresponding to the target image acquisition equipment, wherein the target image acquisition equipment is one of the image acquisition equipment.
When the step is specifically implemented, the face plane can be determined based on the face three-dimensional key point information corresponding to the target image acquisition equipment. As an example, a face plane may be constructed based on coordinates of 3 to 4 key points of the face in the coordinate system of the target image capturing device. The specific selected face key points are not limited.
And then, determining a normal vector of the face plane according to the face plane. While the coordinates of 3 to 4 face key points are used in determining the face plane, it is understood that two non-parallel spatial vectors can be obtained from the coordinates of the 3 to 4 face key points. In a specific implementation, a normal vector of the face plane can be obtained by cross-multiplying the two vectors. Alternatively, the unit vector of the normal vector can be determined by the cross function of NumPy (the full name: numerical Python, an open source numerical calculation extension of Python).
In conjunction with the foregoing, the face plane and the normal vector of the face plane have been acquired. Further, a face coordinate system of the target object may be established based on the plane and a normal vector of the plane. It can be understood that, since the face three-dimensional key point information is obtained in the coordinate system of the target image capturing device, the face coordinate system established based on the face three-dimensional key point information is also based on the coordinate system of the target image capturing device.
S105: and acquiring a head posture true value of a target object corresponding to the target image according to a coordinate system and a face coordinate system of the target image acquisition equipment, wherein the target image is an image acquired by the target image acquisition equipment to the target object at any moment.
On the basis of the known coordinate system of the target image acquisition equipment and the known face coordinate system, the rotation matrix of the face coordinate system relative to the coordinate system of the target image acquisition equipment can be obtained according to the face coordinate system and the coordinate system of the target image acquisition equipment. Since the rotation matrix has an association relationship with three angles (Yaw angle Yaw, Pitch angle Pitch, and Roll angle) in the head pose true value, the head pose true value of the target object can be obtained based on the association relationship according to the rotation matrix. The head pose true value is obtained on the basis of the coordinate system of the target image acquisition equipment and the face coordinate system, the face coordinate system is also on the basis of the coordinate system of the target image acquisition equipment, and the target image acquisition equipment has a correspondence with the target image shot at the moment, so the head pose true value obtained in the step can be associated with the target image.
For example, the image and the derived head pose truth value may also be stored as an image true value pair. Thus, the one-to-one correspondence relationship between the images and the truth values is established. Storing image true value pairs facilitates the use of head pose true values in subsequent applications, such as training models. As an example, the model may be a model that determines a true value of the head posture through an image, or further, the model may be a model that determines a safety factor of driving through an image or a model that analyzes a degree of driving fatigue, or the like. The specific function of the trained model is not limited herein.
The above is the method for acquiring the true value of the head posture provided by the embodiment of the application. In the method, a plurality of image acquisition devices are used for acquiring images of a target object at the same moment, two-dimensional face key points marked in a plurality of acquired images are used as data bases of three-dimensional reconstruction target object face key points, and then a head pose true value of the target object corresponding to the images is acquired. Because this scheme need not acquire head posture truth with the help of wearing equipment, consequently not influenced by wearing angle. The precision of the acquired true value of the head posture is ensured by simultaneously using a plurality of image acquisition devices and three-dimensional reconstruction of key points of the human face. Further, when the head posture truth value is applied to the fields of driver fatigue degree analysis, virtual reality body sensing games, commodity purchase desire analysis, face verification and the like, the analysis result of the data can be more accurate.
In a specific application, in order to provide more diversified and abundant data for a usage scenario of a subsequent head pose truth value, it is proposed in this embodiment that, when S101 is implemented, a plurality of different lighting conditions may be provided for a space where a target object is located. And acquiring images acquired by a plurality of image acquisition devices on the target object at the same moment under the same illumination condition. As an example, the images acquired by all the image acquisition devices at the same time are acquired under the primary illumination condition, the images acquired by all the image acquisition devices at the same time are acquired under the secondary illumination condition, and the images acquired by all the image acquisition devices at the same time are acquired under the tertiary illumination condition. In practical applications, the true values of the head pose obtained by performing the method under different lighting conditions may be different even though the same image capturing device and the target object maintain the head pose still. By collecting images under different illumination and respectively acquiring the true values of the head posture, the accuracy of data analysis can be improved. For example, when data analysis needs to be performed under a first-stage lighting condition, data analysis under the first-stage lighting condition is more accurate in the data analysis result because the image is acquired and the true value of the head posture is acquired in the first-stage lighting condition, and not only the image is acquired and the true value of the head posture is acquired in other lighting conditions.
The first, second, and third illumination conditions are described as examples of different illumination conditions, and the levels of different illumination conditions are not limited herein. For example, it may be that 4-level lighting conditions are involved. Wherein the higher the number of stages, the weaker the intensity of illumination; the lower the number of stages, the higher the intensity of the illumination. In another example, the lighting condition is classified into a daylight lighting condition, an infrared light lighting condition, an ultraviolet light lighting condition, and the like. The above illumination condition can be realized by natural environment, and can also be realized by a lighting device. For example, the selection of the light source type, the on/off of the light, the intensity and the irradiation angle in the lighting device are adjusted or controlled to realize different illumination conditions.
When the real value of the head posture needs to be applied to a data analysis scene in the driving field, the embodiment of the application can take adaptive measures in an image acquisition stage. For example, when a plurality of image capturing devices capture an image of a target object, the target object is seated on a seat that is used to simulate a seat in a real vehicle environment. For example, the target object represents a driver, and the seat in which it is located represents a driver seat. The setting parameters of the image acquisition equipment relative to the seat are determined according to the setting parameters of the simulation object relative to the seat in the real vehicle environment. Wherein setting the parameters includes: position, height, angle, etc. The simulated object includes at least one of: a pillar a, a pillar B, a dashboard, a front windshield, or a left side vehicle glass. As an example, No. 1 device is provided by simulating an a pillar, No. 2 device is provided by simulating a B pillar, No. 3 device is provided by simulating a dashboard, No. 4 device is provided by simulating a front windshield, and No. 5 device is provided by simulating a left side windshield.
Fig. 2 is a scene schematic diagram of acquiring an image of a target object by a plurality of image acquisition devices according to an embodiment of the present disclosure. Fig. 2 illustrates the image acquisition of a target object on a seat by 14 cameras, and the lighting device surrounding the target object in the scene for changing the lighting conditions.
Besides the advantages mentioned above, there are also many advantages in the technical solution of the embodiment of the present application:
in the embodiment of the application, when a plurality of image acquisition devices are used for acquiring, videos of the same person can be recorded at the same time, and continuous head posture truth values are calculated according to the human face three-dimensional key point information of the video frame numbers. Therefore, the technical scheme of the application has continuity in the aspect of acquiring the true value of the head posture.
The method has the image acquisition equipment for shooting the side face image of the person, so the scheme has the capability of acquiring a true value of the wide-angle head posture.
The method adopts a non-contact head posture acquisition method, and the acquisition method is simple and convenient to implement.
Based on the method for acquiring the true value of the head posture provided by the foregoing embodiment, correspondingly, the present application also provides an apparatus for acquiring the true value of the head posture. The following describes a specific implementation of the apparatus with reference to an embodiment. Fig. 3 is a schematic structural diagram of a device for acquiring a true value of a head posture. The apparatus 300 shown in fig. 3 comprises:
an image acquisition module 301, configured to acquire images acquired by multiple image acquisition devices at the same time for a target object;
a key point labeling module 302, configured to label a face key point in an image acquired by each of the image acquisition devices in the plurality of image acquisition devices, so as to obtain face key point information corresponding to each of the image acquisition devices;
a three-dimensional key point reconstruction module 303, configured to reconstruct face three-dimensional key point information corresponding to each image acquisition device based on face key point information corresponding to each image acquisition device and parameters of each image acquisition device;
a coordinate system establishing module 304, configured to establish a face coordinate system based on face three-dimensional key point information corresponding to a target image acquisition device, where the target image acquisition device is one of the image acquisition devices;
a true value obtaining module 305, configured to obtain a true value of a head pose of the target object corresponding to a target image according to the coordinate system of the target image acquisition device and the face coordinate system, where the target image is an image acquired by the target image acquisition device at the time point for the target object.
The method comprises the steps that a plurality of image acquisition devices are used for acquiring images of a target object at the same moment, two-dimensional face key points marked in a plurality of acquired images are used as data bases of three-dimensional reconstruction target object face key points, and then a head posture true value of the target object corresponding to the images is acquired. Because this scheme need not acquire head posture truth with the help of wearing equipment, consequently not influenced by wearing angle. The precision of the acquired true value of the head posture is ensured by simultaneously using a plurality of image acquisition devices and three-dimensional reconstruction of key points of the human face. Further, when the head posture truth value is applied to the fields of driver fatigue degree analysis, virtual reality body sensing games, commodity purchase desire analysis, face verification and the like, the analysis result of the data can be more accurate.
Optionally, the truth value obtaining module 305 includes:
the rotation matrix obtaining unit is used for obtaining a rotation matrix of the face coordinate system relative to the coordinate system of the target image acquisition equipment according to the face coordinate system and the coordinate system of the target image acquisition equipment;
and the true value acquisition unit is used for acquiring a true value of the head posture of the target object corresponding to the target image according to the rotation matrix.
Optionally, the coordinate system establishing module 304 includes:
the face plane determining unit is used for determining a face plane based on the face three-dimensional key point information corresponding to the target image acquisition equipment;
the normal vector determining unit is used for determining a normal vector of the face plane according to the face plane;
and the coordinate system establishing unit is used for establishing a face coordinate system of the target object based on the face plane and the normal vector of the face plane.
Optionally, the three-dimensional keypoint reconstruction module 303 includes:
reconstructing human face three-dimensional key point information corresponding to the target image acquisition equipment by a triangulation reconstruction method according to human face key point information corresponding to the target image acquisition equipment, human face key point information corresponding to reference image acquisition equipment, internal parameters of the target image acquisition equipment, internal parameters of the reference image acquisition equipment and external parameters between the target image acquisition equipment and the reference image acquisition equipment; the reference image acquisition device is any image acquisition device except the target image acquisition device in the plurality of image acquisition devices;
and obtaining the face three-dimensional key point information corresponding to the reference image acquisition equipment according to the face three-dimensional key point information corresponding to the target image acquisition equipment and the external parameters.
Optionally, the apparatus 300 for acquiring a true value of a head posture further includes:
and the storage module is used for storing the mutually corresponding image and head posture truth values as image true value pairs.
Optionally, the image acquisition module 301 comprises:
the illumination unit is used for providing a plurality of different illumination conditions for the space where the target object is located;
the image acquisition unit is used for acquiring images acquired by the plurality of image acquisition devices on the target object at the same moment under the same illumination condition.
Based on the method and the device for acquiring the true value of the head posture provided by the foregoing embodiments, correspondingly, the present application further provides a device for acquiring the true value of the head posture, which includes a processor and a memory; the memory is used for storing a computer program; the processor is configured to execute the method for acquiring a true value of a head pose provided by the foregoing method embodiments according to the computer program. In addition, the processor in the device may be used to control the lighting means to provide switchable lighting conditions.
Based on the method, the apparatus and the device for acquiring a true value of a head pose provided by the foregoing embodiments, accordingly, the present application also provides a computer readable storage medium for storing a computer program, which is executed by a processor to perform the method for acquiring a true value of a head pose as provided by the foregoing method embodiments.
The above description is only one specific embodiment of the present application, but the scope of the present application is not limited thereto, and any changes or substitutions that can be easily conceived by those skilled in the art within the technical scope of the present application should be covered by the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.
Claims (10)
1. A method for acquiring a true value of a head posture is characterized by comprising the following steps:
acquiring images acquired by a plurality of image acquisition devices on a target object at the same moment;
labeling the face key points in the images acquired by each image acquisition device in the plurality of image acquisition devices to obtain face key point information corresponding to each image acquisition device;
reconstructing human face three-dimensional key point information corresponding to each image acquisition device based on human face key point information corresponding to each image acquisition device and parameters of each image acquisition device;
establishing a face coordinate system based on face three-dimensional key point information corresponding to target image acquisition equipment, wherein the target image acquisition equipment is one of the image acquisition equipment;
and acquiring a head posture true value of the target object corresponding to the target image according to the coordinate system of the target image acquisition equipment and the face coordinate system, wherein the target image is an image acquired by the target image acquisition equipment on the target object at the moment.
2. The method according to claim 1, wherein the obtaining a true value of the head pose of the target object corresponding to the target image according to the coordinate system of the target image capturing device and the face coordinate system comprises:
according to the face coordinate system and the coordinate system of the target image acquisition equipment, obtaining a rotation matrix of the face coordinate system relative to the coordinate system of the target image acquisition equipment;
and acquiring a head posture true value of the target object corresponding to the target image according to the rotation matrix.
3. The method according to claim 1, wherein the establishing of the face coordinate system based on the face three-dimensional key point information corresponding to the target image acquisition device comprises:
determining a face plane based on the face three-dimensional key point information corresponding to the target image acquisition equipment;
determining a normal vector of the face plane according to the face plane;
and establishing a face coordinate system of the target object based on the face plane and the normal vector of the face plane.
4. The method according to claim 1, wherein reconstructing three-dimensional key point information of a face corresponding to each of the image capturing devices based on the key point information of the face corresponding to each of the image capturing devices and parameters of each of the image capturing devices comprises:
reconstructing human face three-dimensional key point information corresponding to the target image acquisition equipment by a triangulation reconstruction method according to human face key point information corresponding to the target image acquisition equipment, human face key point information corresponding to reference image acquisition equipment, internal parameters of the target image acquisition equipment, internal parameters of the reference image acquisition equipment and external parameters between the target image acquisition equipment and the reference image acquisition equipment; the reference image acquisition device is any image acquisition device except the target image acquisition device in the plurality of image acquisition devices;
and obtaining the face three-dimensional key point information corresponding to the reference image acquisition equipment according to the face three-dimensional key point information corresponding to the target image acquisition equipment and the external parameters.
5. The method according to any one of claims 1-4, further comprising:
and storing the image and the head pose true value which correspond to each other as an image true value pair.
6. The method according to any one of claims 1-4, wherein said acquiring images acquired by a plurality of image acquisition devices on a target object at the same time comprises:
providing a plurality of different lighting conditions to a space in which the target object is located;
and acquiring images acquired by the plurality of image acquisition devices on the target object at the same moment under the same illumination condition.
7. The method of any one of claims 1-4, wherein the plurality of image capture devices capture images of the target object while the target object is seated on a seat configured to simulate a seat in a real-world vehicle environment; the setting parameters of the image acquisition equipment relative to the seat are determined according to the setting parameters of a simulation object in the real vehicle environment relative to the seat;
the simulated object includes at least one of:
a pillar a, a pillar B, a dashboard, a front windshield, or a left side vehicle glass.
8. An apparatus for obtaining a true value of a head posture, comprising:
the image acquisition module is used for acquiring images acquired by the plurality of image acquisition devices on the target object at the same moment;
the key point labeling module is used for labeling the face key points in the images acquired by the image acquisition equipment in the image acquisition equipment to obtain the face key point information corresponding to the image acquisition equipment;
the three-dimensional key point reconstruction module is used for reconstructing human face three-dimensional key point information corresponding to each image acquisition device based on human face key point information corresponding to each image acquisition device and parameters of each image acquisition device;
the coordinate system establishing module is used for establishing a face coordinate system based on face three-dimensional key point information corresponding to target image acquisition equipment, and the target image acquisition equipment is one of the image acquisition equipment;
and the truth value acquisition module is used for acquiring a head posture truth value of the target object corresponding to the target image according to the coordinate system of the target image acquisition equipment and the face coordinate system, wherein the target image is an image acquired by the target image acquisition equipment on the target object at the moment.
9. The device for acquiring the true value of the head posture is characterized by comprising a processor and a memory; the memory is used for storing a computer program; the processor is configured to perform a method for acquiring a head pose truth value according to any one of claims 1 to 7 according to the computer program.
10. A computer-readable storage medium for storing a computer program which, when executed by a processor, performs the method for acquiring a true value of a head pose according to any one of claims 1 to 7.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111501742.9A CN114220149A (en) | 2021-12-09 | 2021-12-09 | Method, device, equipment and storage medium for acquiring true value of head posture |
| PCT/CN2022/071709 WO2023103145A1 (en) | 2021-12-09 | 2022-01-13 | Head pose truth value acquisition method, apparatus and device, and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111501742.9A CN114220149A (en) | 2021-12-09 | 2021-12-09 | Method, device, equipment and storage medium for acquiring true value of head posture |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114220149A true CN114220149A (en) | 2022-03-22 |
Family
ID=80700651
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111501742.9A Pending CN114220149A (en) | 2021-12-09 | 2021-12-09 | Method, device, equipment and storage medium for acquiring true value of head posture |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN114220149A (en) |
| WO (1) | WO2023103145A1 (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117351540B (en) * | 2023-09-27 | 2024-04-02 | 东莞莱姆森科技建材有限公司 | Bathroom mirror integrated with LED and head action recognition |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109960986A (en) * | 2017-12-25 | 2019-07-02 | 北京市商汤科技开发有限公司 | Human face posture analysis method, device, equipment, storage medium and program |
| CN111414798B (en) * | 2019-02-03 | 2022-12-06 | 沈阳工业大学 | Head posture detection method and system based on RGB-D image |
| CN111832373A (en) * | 2019-05-28 | 2020-10-27 | 北京伟景智能科技有限公司 | Automobile driving posture detection method based on multi-view vision |
| WO2022246605A1 (en) * | 2021-05-24 | 2022-12-01 | 华为技术有限公司 | Key point calibration method and apparatus |
| CN113689503B (en) * | 2021-10-25 | 2022-02-25 | 北京市商汤科技开发有限公司 | Target object posture detection method, device, equipment and storage medium |
-
2021
- 2021-12-09 CN CN202111501742.9A patent/CN114220149A/en active Pending
-
2022
- 2022-01-13 WO PCT/CN2022/071709 patent/WO2023103145A1/en unknown
Also Published As
| Publication number | Publication date |
|---|---|
| WO2023103145A1 (en) | 2023-06-15 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111325823B (en) | Method, device and equipment for acquiring face texture image and storage medium | |
| US11379996B2 (en) | Deformable object tracking | |
| CN108875524B (en) | Sight estimation method, device, system and storage medium | |
| CN109559332B (en) | Sight tracking method combining bidirectional LSTM and Itracker | |
| CN109407547A (en) | Multi-camera in-loop simulation test method and system for panoramic visual perception | |
| CN111835984B (en) | Intelligent light supplementing method and device, electronic equipment and storage medium | |
| CN108171673A (en) | Image processing method, device, vehicle-mounted head-up-display system and vehicle | |
| Selim et al. | AutoPOSE: Large-scale Automotive Driver Head Pose and Gaze Dataset with Deep Head Orientation Baseline. | |
| CN112639846A (en) | Method and device for training deep learning model | |
| CN105159452B (en) | A kind of control method and system based on human face modeling | |
| CN103356163A (en) | Fixation point measurement device and method based on video images and artificial neural network | |
| CN107145224B (en) | Human eye sight tracking and device based on three-dimensional sphere Taylor expansion | |
| CN113808160A (en) | Sight direction tracking method and device | |
| CN110163113B (en) | Human behavior similarity calculation method and device | |
| CN108629248A (en) | A kind of method and apparatus for realizing augmented reality | |
| CN109407828A (en) | One kind staring the point estimation method and system, storage medium and terminal | |
| CN110222651A (en) | A kind of human face posture detection method, device, terminal device and readable storage medium storing program for executing | |
| CN113419623A (en) | Non-calibration eye movement interaction method and device | |
| CN114356072A (en) | System and method for detecting spatial orientation of wearable device | |
| CN116030519A (en) | Learning attention detection and assessment method for live broadcast teaching platform | |
| CN114220149A (en) | Method, device, equipment and storage medium for acquiring true value of head posture | |
| CN110841266A (en) | Auxiliary training system and method | |
| JP2021527888A (en) | Methods and systems for performing eye tracking using off-axis cameras | |
| Martin et al. | An evaluation of different methods for 3d-driver-body-pose estimation | |
| CN112099330B (en) | Holographic human body reconstruction method based on external camera and wearable display control equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |