CN109242031B - Training method, using method, device and processing equipment of posture optimization model - Google Patents
Training method, using method, device and processing equipment of posture optimization model Download PDFInfo
- Publication number
- CN109242031B CN109242031B CN201811106206.7A CN201811106206A CN109242031B CN 109242031 B CN109242031 B CN 109242031B CN 201811106206 A CN201811106206 A CN 201811106206A CN 109242031 B CN109242031 B CN 109242031B
- Authority
- CN
- China
- Prior art keywords
- posture
- pose
- information
- loss function
- optimization model
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
Landscapes
- Engineering & Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Image Analysis (AREA)
Abstract
The invention provides a training method, a using method, a device and processing equipment of a posture optimization model, and relates to the technical field of posture detection, wherein the training method comprises the following steps: obtaining a training sample; determining a posture evaluation loss function and a posture change loss function of the posture optimization model; determining a total loss function according to the posture evaluation loss function and the posture change loss function; and training the posture optimization model by using the initial posture information, the image and the posture evaluation information until the total loss function is converged, and taking the corresponding parameter when the total loss function is converged as the parameter of the posture optimization model. The training method, the using method, the device and the processing equipment of the posture optimization model provided by the embodiment of the invention can output the posture change quantity of the object, namely output beautiful posture recommendation, thereby improving the shooting effect.
Description
Technical Field
The invention relates to the technical field of posture detection, in particular to a training method, a using method, a device and processing equipment of a posture optimization model.
Background
In the photographing process, the subject and the photographer can be divided according to the positions, and the subject takes an expected posture in front of the lens to perform the photographing operation by the photographer. The photographer cannot observe the posture of the photographer in the lens, and needs to make suggestions, but is limited by the shooting level of the photographer and the fact that the photographer cannot completely understand the shooting intention, so that the photographer cannot provide proper posture recommendation, and the shooting effect presented finally is always unsatisfactory.
In order to solve the problem that the above-mentioned method of manually shooting the recommended posture is poor in effect, no effective solution has been proposed yet.
Disclosure of Invention
In view of the above, the present invention provides a training method, a using method, an apparatus and a processing device for a pose optimization model, which can recommend a beautiful pose to a subject and improve a shooting effect.
In order to achieve the above purpose, the embodiment of the present invention adopts the following technical solutions:
in a first aspect, an embodiment of the present invention provides a method for training a posture optimization model, including: acquiring a training sample, wherein the training sample comprises an image of an object, initial posture information and posture evaluation information; determining a posture evaluation loss function and a posture change loss function of the posture optimization model; determining a total loss function from the pose evaluation loss function and the pose change loss function; and training the posture optimization model by using the initial posture information, the image and the posture evaluation information until the total loss function is converged, and taking a corresponding parameter when the total loss function is converged as a parameter of the posture optimization model.
Further, the posture evaluation loss function represents a degree of posture beauty of the subject; the pose change loss function represents a pose change amount of the object.
Further, the step of obtaining training samples includes: performing key point detection on the image of the object through a key point detection model to acquire the initial posture information; posture evaluation information input by a user is received.
Further, the pose optimization model comprises: a posture evaluation model and a posture change model; the step of determining a pose evaluation loss function and a pose change loss function of the pose optimization model comprises: training the pose change model using the initial pose information and the image, taking a norm of an output of the pose change model as a pose change loss function; training the pose evaluation model using the initial pose information, optimized pose information, the image, and the pose evaluation information, the output of the pose evaluation model being a pose evaluation loss function; the optimized pose information is a sum of the initial pose information and the corresponding pose change model output.
Further, the step of determining a total loss function from the pose evaluation loss function and the pose change loss function comprises: and summing the posture evaluation loss function and the posture change loss function according to a preset weight coefficient to obtain a total loss function.
In a second aspect, an embodiment of the present invention provides a method for using a pose optimization model, which is applied to a terminal, where the pose optimization model obtained by the method for training the pose optimization model provided in any one of the first aspect of the terminal includes: acquiring a real-time image and initial posture information of an object; inputting the real-time image and the initial pose information into the pose optimization model to cause the pose optimization model to output pose optimization information; determining posture change prompt information according to the posture optimization information; and outputting the gesture change prompt information.
Further, the step of determining posture change prompting information according to the posture optimization information includes: determining key point change information of the object according to the posture optimization information; and using the key point change information as the posture change prompt information.
In a third aspect, an embodiment of the present invention provides a training apparatus for a posture optimization model, including: the system comprises a first acquisition module, a second acquisition module and a third acquisition module, wherein the first acquisition module is used for acquiring a training sample, and the training sample comprises an image of an object, initial posture information and posture evaluation information; the first determination module is used for determining a posture evaluation loss function and a posture change loss function of the posture optimization model; a second determination module for determining a total loss function from the pose evaluation loss function and the pose change loss function; and the model training module is used for training the posture optimization model by using the initial posture information, the image and the posture evaluation information until the total loss function is converged, and taking a corresponding parameter when the total loss function is converged as a parameter of the posture optimization model.
In a fourth aspect, an embodiment of the present invention provides a device for using a pose optimization model, which is applied to a terminal, where the terminal includes a pose optimization model obtained by the method for training the pose optimization model provided in any one of the first aspects and a key point detection model, and the device includes: the second acquisition module is used for acquiring a real-time image and initial posture information of the object; an optimization module for inputting the real-time image and the initial pose information into the pose optimization model to cause the pose optimization model to output pose optimization information; the prompt determining module is used for determining posture change prompt information according to the posture optimization information; and the output module is used for outputting the posture change prompt information.
In a fifth aspect, an embodiment of the present invention provides a processing device, which includes a memory, a processor, and a computer program stored on the memory and executable on the processor, and when the processor executes the computer program, the processor implements the steps of the method according to any one of the first aspects.
In a sixth aspect, the present invention provides a computer readable medium having a program code executable by a processor, the program code causing the processor to execute the steps of the method according to any one of the first aspect.
According to the training method, the using method, the device and the processing equipment of the posture optimization model provided by the embodiment of the invention, the posture optimization model is trained through a total loss function determined by the posture evaluation loss function and the posture change loss function according to the initial posture information, the image and the posture evaluation information of the object until the total loss function is converged, corresponding parameters when the total loss function is converged are used as parameters of the posture optimization model, and after the initial posture information and the image are input, the posture change amount of the object can be output, namely, graceful posture recommendation is output, so that the shooting effect is improved.
Additional features and advantages of the disclosure will be set forth in the description which follows, or in part may be learned by the practice of the above-described techniques of the disclosure, or may be learned by practice of the disclosure.
In order to make the aforementioned objects, features and advantages of the present disclosure more comprehensible, preferred embodiments accompanied with figures are described in detail below.
Drawings
In order to more clearly illustrate the embodiments of the present disclosure or the technical solutions in the prior art, the drawings needed to be used in the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are some embodiments of the present disclosure, and other drawings can be obtained by those skilled in the art without creative efforts.
Fig. 1 is a schematic structural diagram of an electronic device according to an embodiment of the present invention;
FIG. 2 is a flow chart illustrating a method for training a gesture optimization model according to an embodiment of the present invention;
FIG. 3 is a flow chart illustrating a method for using a gesture optimization model provided by an embodiment of the invention;
FIG. 4 is a block diagram illustrating an exemplary training apparatus for a pose optimization model according to an embodiment of the present invention;
FIG. 5 is a block diagram illustrating an apparatus for using a gesture optimization model according to an embodiment of the present invention.
Detailed Description
To make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings, and it is apparent that the described embodiments are some, but not all embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
In view of the problems that in the existing shooting process, a photographer proposes a posture suggestion for a shot subject, the posture is not attractive, and the shooting effect is poor, in order to improve the problems, the embodiment of the invention provides a training method, a using method, a device and processing equipment of a posture optimization model, and the following detailed description is given through the embodiment of the invention.
The first embodiment is as follows:
first, an example electronic device 100 for implementing a training method, a using method, an apparatus, and a processing device of a pose optimization model according to an embodiment of the present invention will be described with reference to fig. 1.
As shown in fig. 1, an electronic device 100 includes one or more processing devices 102 and one or more storage devices 104. Optionally, the electronic device 100 shown in FIG. 1 may also include an input device 106, an output device 108, and a data acquisition device 110, which are interconnected via a bus system 112 and/or other form of connection mechanism (not shown). It should be noted that the components and structure of the electronic device 100 shown in fig. 1 are exemplary only, and not limiting, and the electronic device may have other components and structures as desired.
The processing device 102 may be a gateway, or may be an intelligent terminal, or may be a device including a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), or other form of processing unit having data processing capability and/or instruction execution capability, and may process data of other components in the electronic device 100, and may control other components in the electronic device 100 to perform desired functions.
The storage 104 may include one or more computer program products that may include various forms of computer-readable storage media, such as volatile memory and/or non-volatile memory. The volatile memory may include, for example, Random Access Memory (RAM), cache memory (cache), and/or the like. The non-volatile memory may include, for example, Read Only Memory (ROM), hard disk, flash memory, etc. On which one or more computer program instructions may be stored that may be executed by processing device 102 to implement client functionality (implemented by the processing device) and/or other desired functionality in embodiments of the present invention described below. Various applications and various data, such as various data used and/or generated by the applications, may also be stored in the computer-readable storage medium.
The input device 106 may be a device used by a user to input instructions and may include one or more of a keyboard, a mouse, a microphone, a touch screen, and the like.
The output device 108 may output various information (e.g., images or sounds) to the outside (e.g., a user), and may include one or more of a display, a speaker, and the like.
The data acquisition device 110 may acquire an image of a subject and store the acquired image in the storage device 104 for use by other components. Illustratively, the data acquisition device 110 may be a camera.
For example, the components of the electronic device for implementing the training method, the using method, the apparatus, and the processing device of the gesture optimization model according to the embodiment of the present invention may be integrated, or may be distributed, such as integrating the processing device 102, the storage device 104, the input device 106, and the output device 108, and separately arranging the data acquisition device 110.
Exemplary electronic devices for implementing the training method, the using method, the apparatus and the processing device of the gesture optimization model according to the embodiments of the present invention may be implemented as smart terminals such as smart phones, tablet computers, smart watches, cameras, etc. which can perform shooting.
Example two:
in accordance with an embodiment of the present invention, there is provided an embodiment of an action recognition method, it should be noted that the steps illustrated in the flowchart of the drawings may be performed in a computer system such as a set of computer-executable instructions, and that while a logical order is illustrated in the flowchart, in some cases, the steps illustrated or described may be performed in an order different than here.
FIG. 2 is a flowchart of a method for training a pose optimization model according to an embodiment of the present invention, as shown in FIG. 2, the method comprising the steps of:
step S202, training samples are obtained. Wherein the training sample comprises an image of the object, initial pose information, and pose evaluation information. The object may be a person or other animate body, as may the object that may receive the gesture optimization suggestion.
The training samples can be obtained through images of the object, and the images can be obtained through shooting by a shooting-capable intelligent terminal. After the image of the object is obtained, the key point detection can be performed on the image through the key point detection model, and initial posture information is obtained. The key point detection model can adopt a 3D key point detection model, the positions of key points in an image can be detected and output, and the positions of the key points can be influenced by the action, the angle and the like of an object facing the intelligent terminal. The initial pose information identifies an initial pose of the object, i.e., a pose that has not been pose optimized.
The image generally shows the height, weight, length, and scale of the subject. The pose evaluation information is user evaluation information received in advance, the user can label the beauty degree of the image according to the pose of the object in the image, and modes such as but not limited to two-classification, scoring and the like can be used, for example, a beauty pose graph and a ugly pose graph are distinguished, or a beauty degree score is given.
In step S204, a pose evaluation loss function and a pose change loss function of the pose optimization model are determined.
Wherein, the pose optimization model can adopt, but is not limited to, a deep neural network and the like. The above-described posture-evaluation loss function represents a degree of beauty of the posture of the subject, and the posture-change loss function represents a posture-change amount of the subject. During the training process, the value of the posture evaluation loss function is expected to be as large as possible, and the optimized posture is better to be represented; it is desirable that the absolute value of the posture change loss function is as small as possible, and the smaller the range of motion change that the object needs to perform. The specific form of the posture evaluation loss function and the posture change loss function may be determined according to a network model actually used by the posture optimization model, which is not limited in this embodiment.
The process of training by the above-described loss function is explained, for example, by taking an object as an example. Firstly, posture information of a human body is calculated through a 3D key point detection model. The posture of a person is represented by a parameter vector A, each dimension represents the position of a 3D key point of the human body, an original picture is marked as I, a posture evaluation model is a function F (A, I), the input is A and I, and the output is a real number, namely the good-looking degree of the posture. The posture optimization model is marked as G (A, I), the input is A and I, the output is a parameter vector, the actual meaning is the change amount of the posture, and A + G (A, I) is the optimized posture. There are two loss functions to optimize in the training: one is a posture evaluation loss function F (A + G (A, I), I), namely the optimized posture grace degree, which needs to be as large as possible; one is that the posture change loss function G (a, I) | |, i.e., the norm of the posture vector change magnitude, is as small as possible.
In step S206, a total loss function is determined from the posture evaluation loss function and the posture change loss function.
Two loss functions need to be considered simultaneously during training, so that a good posture recommendation effect is achieved, and the two loss functions can be summed according to preset weight coefficients to obtain a total loss function. If training is performed according to each loss function in sequence, the corresponding parameters are changed when the previous loss function is better when training is performed according to the next loss function, and the training needs to be performed repeatedly, so that the efficiency is low. Since the value of the posture evaluation loss function F (a + G (a, I) needs to be as large as possible, and the value of the posture change loss function G (a, I) | needs to be as small as possible, the optimization trends of the two are opposite, and one of them needs to take a positive value and the other needs to take a negative value during summation.
And step S208, training the posture optimization model by using the initial posture information, the image and the posture evaluation information until the total loss function is converged, and taking the corresponding parameters when the total loss function is converged as the parameters of the posture optimization model.
The posture optimization model aims at reducing the total loss function through a large amount of sample data, each parameter in the model can be obtained by using optimization algorithms such as random gradient descent, gradient return, back propagation and the like, when the total loss function is converged, the corresponding parameter when the total loss function is converged is used as the parameter of the posture optimization model, and at the moment, the model of the parameter is set to be the trained posture optimization model. The posture optimization model can optimize and recommend the input initial posture information and images, and the output of the model is used as the change amount of the posture of the object, so that beautiful posture suggestions can be obtained.
According to the training method of the posture optimization model provided by the embodiment of the invention, the posture optimization model is trained through the total loss function determined by the posture evaluation loss function and the posture change loss function according to the initial posture information, the image and the posture evaluation information of the object until the total loss function is converged, the corresponding parameter when the total loss function is converged is used as the parameter of the posture optimization model, and after the initial posture information and the image are input, the posture change amount of the object can be output, namely, the beautiful posture recommendation is output, so that the shooting effect is improved.
The posture optimization model may include a posture evaluation model and a posture change model, and the posture evaluation model may be trained first in the process of training the posture optimization model. The training process may be performed using the initial pose information obtained by detecting the captured image of the object by the key point detection model, the image obtained by directly using the captured image, and pose evaluation information obtained by labeling each captured image by the user. And fitting and training the model through the information, and learning the difference between the good posture and the bad posture to obtain a posture evaluation model. The model may use methods including, but not limited to, a deep neural network, a support vector machine, a random decision forest, etc., and the fitting training may use methods including, but not limited to, stochastic gradient descent, gradient backtransmission, etc.
In the training of the posture evaluation model, a posture change model is trained, the input values of the posture change model are initial posture information and an image, the posture change model is trained using the initial posture information and the image, and the norm of the output of the posture evaluation model is used as a posture evaluation loss function. Meanwhile, the posture evaluation model is retrained using the initial posture information, the optimized posture information, the image and the posture evaluation information, and the output of the posture evaluation model is used as a posture evaluation loss function. Wherein the optimized pose information is a sum of the initial pose information and a corresponding pose change model output.
After the pose optimization model is obtained through training, the embodiment further provides a using method of the pose optimization model, and the using method is applied to a terminal, and the terminal comprises the pose optimization model obtained through the training method of the pose optimization model and a key point detection model. The terminal may be the electronic device provided in embodiment 1. Taking the electronic device as an example of a mobile phone, the electronic device generally comprises an image acquisition part (a rear camera of the mobile phone), a core calculation part (a mobile phone calculation unit), a storage medium (mobile phone storage) and a voice part (a mobile phone loudspeaker).
FIG. 3 is a flow chart of a method of using a pose optimization model according to an embodiment of the present invention, as shown in FIG. 3, the method comprising the steps of:
step S302, a real-time image and initial pose information of the object are acquired.
The terminal can acquire a real-time image of the object through the camera device of the terminal. And inputting the real-time image into the key point detection model to obtain initial posture information. Inputting the obtained real-time image into the key point detection model to obtain the initial posture information of the object, which is described in detail in the foregoing embodiment and is not described again.
Step S304, inputting the real-time image and the initial posture information into a posture optimization model, so that the posture optimization model outputs posture optimization information.
Step S306, determining posture change prompting information according to the posture optimization information.
The posture optimization information output by the posture optimization model can be a parameter vector, the actual meaning of the posture optimization information is the change amount of the posture, the key point change information of the object can be determined through the posture optimization information, and then the key point change information is used as the posture change prompt information. For example, the key point is the left hand of a person, and the change information may be 10 cm down the left hand and 10 cm to the right. The posture optimization model can comprise a posture evaluation model and a posture change model, the output posture optimization information can also comprise information such as the beauty degree of the initial posture, the beauty degree of the optimized posture and the like, and a prompt of the beauty degree can be provided for the object.
In step S308, posture change prompt information is output.
The terminal outputs the posture change prompt message, and the posture change prompt message can be performed through voice or displayed through a display screen to display characters or videos. The shooting person can be transferred to the shot person through the prompt of the display screen, and the function of prompting the shot person can also be achieved.
According to the using method of the posture optimization model provided by the embodiment of the invention, the real-time image of the object is input into the key point detection model to obtain the initial posture information, the posture optimization information is output through the posture optimization model, and the posture change prompt information and the posture change output are determined, so that the posture recommendation suggestion can be output based on the current real-time image of the object, and the shooting effect is improved.
Example three:
corresponding to the training method of the pose optimization model provided in the second embodiment, an embodiment of the present invention provides a training apparatus of the pose optimization model, and referring to a structural block diagram of the training apparatus of the pose optimization model shown in fig. 4, the training apparatus includes:
a first obtaining module 402, configured to obtain a training sample, where the training sample includes an image of an object, initial pose information, and pose evaluation information;
a first determining module 404, configured to determine a pose evaluation loss function and a pose change loss function of the pose optimization model;
a second determining module 406 for determining a total loss function from the pose evaluation loss function and the pose change loss function;
and the model training module 408 is configured to train the pose optimization model by using the initial pose information, the image, and the pose evaluation information until the total loss function converges, and use a corresponding parameter when the total loss function converges as a parameter of the pose optimization model.
According to the training device of the posture optimization model provided by the embodiment of the invention, the posture optimization model is trained according to the initial posture information, the image and the posture evaluation information of the object through the total loss function determined by the posture evaluation loss function and the posture change loss function until the total loss function is converged, the corresponding parameter when the total loss function is converged is used as the parameter of the posture optimization model, and after the initial posture information and the image are input, the posture change amount of the object can be output, namely, the beautiful posture recommendation is output, so that the shooting effect is improved.
In one embodiment, the pose evaluation loss function represents a degree of pose elegance of the object; the posture change loss function represents a posture change amount of the object.
In another embodiment, the first obtaining module 402 is further configured to: performing key point detection on the image of the object through a key point detection model to obtain initial posture information; posture evaluation information input by a user is received.
In another embodiment, the above pose optimization model comprises: a posture evaluation model and a posture change model; the first determining module 404 is further configured to: training a pose change model using the initial pose information and the image, taking a norm of an output of the pose change model as a pose change loss function; training a pose evaluation model using the initial pose information, the optimized pose information, the image, and the pose evaluation information, the output of the pose evaluation model being a pose evaluation loss function; the optimized pose information is the sum of the initial pose information and the corresponding pose change model output.
In another embodiment, the second determining module 406 is further configured to: and summing the posture evaluation loss function and the posture change loss function according to a preset weight coefficient to obtain a total loss function.
The embodiment of the present invention provides a device for using a posture optimization model, which is applied to a terminal, where the terminal obtains the posture optimization model by using the aforementioned training method for the posture optimization model, and referring to a structural block diagram of the device for using a posture optimization model shown in fig. 5, the device includes:
a second obtaining module 502, configured to obtain a real-time image and initial pose information of an object;
an optimization module 504 for inputting the real-time image and the initial pose information into the pose optimization model, such that the pose optimization model outputs pose optimization information;
a prompt determining module 506, configured to determine posture change prompt information according to the posture optimization information;
and an output module 508 for outputting the gesture change prompt information.
According to the using device of the posture optimization model, the real-time image of the object is input into the key point detection model, the initial posture information is obtained, the posture optimization information is output through the posture optimization model, the posture change prompt information and the posture change output are determined, the posture recommendation suggestion can be output based on the current real-time image of the object, and therefore the shooting effect is improved.
In one embodiment, the prompt determining module 506 is further configured to: determining key point change information of the object according to the posture optimization information; the key point change information is used as posture change prompt information.
The device provided by the embodiment has the same implementation principle and technical effect as the foregoing embodiment, and for the sake of brief description, reference may be made to the corresponding contents in the foregoing method embodiment for the portion of the embodiment of the device that is not mentioned.
The embodiment of the present invention further provides a processing device, which includes a memory and a processor, where the memory stores a computer program that can be executed on the processor, and the processor implements the steps of the method provided in the foregoing embodiment when executing the computer program. Optionally, the electronic device may further comprise an image capture device or a fingerprint sensor.
It is clear to those skilled in the art that, for convenience and brevity of description, the specific working process of the apparatus described above may refer to the corresponding process in the foregoing method embodiment, and is not described herein again.
Further, the present embodiment also provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the computer program performs the steps of the method provided by the foregoing method embodiment.
The training method, the using method, the apparatus and the computer program product of the processing device for the gesture optimization model provided in the embodiments of the present invention include a computer-readable storage medium storing program codes, instructions included in the program codes may be used to execute the methods provided in the foregoing method embodiments, and specific implementations may refer to the method embodiments and are not described herein again.
The embodiment also provides a computer program, and the computer program can be stored on a storage medium in the cloud or the local. When being executed by a computer or processor, for performing the methods provided in the previous method embodiments and for implementing the respective modules in the apparatus according to embodiments of the invention. For specific implementation, reference may be made to the method embodiment, which is not described herein again.
Those of ordinary skill in the art will appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.
In the several embodiments provided in the present application, it should be understood that the disclosed apparatus and method may be implemented in other ways. For example, the above-described apparatus embodiments are merely illustrative, and for example, the division of the units is only one logical functional division, and there may be other divisions when actually implementing, for example, a plurality of units or components may be combined or integrated into another device, or some features may be omitted, or not implemented.
The various apparatus embodiments of the present application may be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof. Those skilled in the art will appreciate that a microprocessor or Digital Signal Processor (DSP) may be used in practice to implement some or all of the functionality of some of the blocks in an apparatus according to embodiments of the present invention. The present application may also be embodied as apparatus programs (e.g., computer programs and computer program products) for performing a portion or all of the methods described herein. For example, the programs of the present application may be stored on a computer readable medium or may be in the form of one or more signals. Such a signal may be downloaded from an internet website or provided on a carrier signal or in any other form.
The above-described functions of the present application, if implemented in the form of software functional units and sold or used as a separate product, may be stored in a computer-readable storage medium. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.
Finally, it should be noted that: although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art will understand that: any person skilled in the art can modify or easily conceive the technical solutions described in the foregoing embodiments or equivalent substitutes for some technical features within the technical scope of the present disclosure; such modifications, changes or substitutions do not depart from the spirit and scope of the embodiments of the present invention, and they should be construed as being included therein. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.
Claims (9)
1. A method for training a pose optimization model, comprising:
acquiring a training sample, wherein the training sample comprises an image of an object, initial posture information and posture evaluation information;
determining a posture evaluation loss function and a posture change loss function of the posture optimization model; wherein the pose evaluation loss function represents a degree of pose elegance of the subject; the pose change loss function represents a pose change amount of the object;
summing the posture evaluation loss function and the posture change loss function according to a preset weight coefficient to obtain a total loss function;
and training the posture optimization model by using the initial posture information, the image and the posture evaluation information until the total loss function is converged, and taking a corresponding parameter when the total loss function is converged as a parameter of the posture optimization model.
2. The method of claim 1, wherein the step of obtaining training samples comprises:
performing key point detection on the image of the object through a key point detection model to acquire the initial posture information; posture evaluation information input by a user is received.
3. The method of claim 1, wherein the pose optimization model comprises: a posture evaluation model and a posture change model; the step of determining a pose evaluation loss function and a pose change loss function of the pose optimization model comprises:
training the pose change model using the initial pose information and the image, taking a norm of an output of the pose change model as a pose change loss function;
training the pose evaluation model using the initial pose information, optimized pose information, the image, and the pose evaluation information, the output of the pose evaluation model being a pose evaluation loss function; the optimized pose information is a sum of the initial pose information and the corresponding pose change model output.
4. A method for using a pose optimization model, the method being applied to a terminal, the terminal comprising the pose optimization model obtained by the method for training the pose optimization model according to any one of claims 1-3, the method comprising:
acquiring a real-time image and initial posture information of an object;
inputting the real-time image and the initial pose information into the pose optimization model to cause the pose optimization model to output pose optimization information;
determining posture change prompt information according to the posture optimization information;
and outputting the gesture change prompt information.
5. The method of claim 4, wherein the step of determining gesture change prompt information based on the gesture optimization information comprises:
determining key point change information of the object according to the posture optimization information;
and using the key point change information as the posture change prompt information.
6. A training device for a posture optimization model, comprising:
the system comprises a first acquisition module, a second acquisition module and a third acquisition module, wherein the first acquisition module is used for acquiring a training sample, and the training sample comprises an image of an object, initial posture information and posture evaluation information;
the first determination module is used for determining a posture evaluation loss function and a posture change loss function of the posture optimization model; wherein the pose evaluation loss function represents a degree of pose elegance of the subject; the pose change loss function represents a pose change amount of the object;
the second determining module is used for summing the posture evaluation loss function and the posture change loss function according to a preset weight coefficient to obtain a total loss function;
and the model training module is used for training the posture optimization model by using the initial posture information, the image and the posture evaluation information until the total loss function is converged, and taking a corresponding parameter when the total loss function is converged as a parameter of the posture optimization model.
7. An apparatus for using a pose optimization model, wherein the apparatus is applied to a terminal, and the terminal comprises a pose optimization model obtained by the training method of the pose optimization model according to any one of claims 1-3, and the apparatus comprises:
the second acquisition module is used for acquiring a real-time image and initial posture information of the object;
an optimization module for inputting the real-time image and the initial pose information into the pose optimization model to cause the pose optimization model to output pose optimization information;
the prompt determining module is used for determining posture change prompt information according to the posture optimization information;
and the output module is used for outputting the posture change prompt information.
8. A processing device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor implements the steps of the method of any of the preceding claims 1 to 5 when executing the computer program.
9. A computer-readable medium having program code executable by a processor, the program code causing the processor to perform the method of any of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811106206.7A CN109242031B (en) | 2018-09-20 | 2018-09-20 | Training method, using method, device and processing equipment of posture optimization model |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811106206.7A CN109242031B (en) | 2018-09-20 | 2018-09-20 | Training method, using method, device and processing equipment of posture optimization model |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109242031A CN109242031A (en) | 2019-01-18 |
| CN109242031B true CN109242031B (en) | 2021-01-12 |
Family
ID=65056086
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811106206.7A Active CN109242031B (en) | 2018-09-20 | 2018-09-20 | Training method, using method, device and processing equipment of posture optimization model |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109242031B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112489036A (en) * | 2020-12-14 | 2021-03-12 | Oppo（重庆）智能科技有限公司 | Image evaluation method, image evaluation device, storage medium, and electronic apparatus |
| CN113221681B (en) * | 2021-04-26 | 2023-09-26 | 平安科技（深圳）有限公司 | Model training method, action gesture generating method, device, equipment and medium |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1933553A (en) * | 2005-09-16 | 2007-03-21 | 英华达(上海)电子有限公司 | Method for position regulation prompting based on human face identification combined with TTS in digital camera |
| JP2007153035A (en) * | 2005-12-01 | 2007-06-21 | Auto Network Gijutsu Kenkyusho:Kk | Occupant sitting judgement system |
| CN101216883B (en) * | 2007-12-29 | 2010-11-10 | 北京中星微电子有限公司 | A photography method and device |
| CN104717413A (en) * | 2013-12-12 | 2015-06-17 | 北京三星通信技术研究有限公司 | Shooting assistance method and equipment |
-
2018
- 2018-09-20 CN CN201811106206.7A patent/CN109242031B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN109242031A (en) | 2019-01-18 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109816009B (en) | Multi-label image classification method, device and equipment based on graph convolution | |
| CN108710847B (en) | Scene recognition method and device and electronic equipment | |
| CN110532984B (en) | Key point detection method, gesture recognition method, device and system | |
| CN107704838B (en) | Target object attribute identification method and device | |
| CN108961303B (en) | Image processing method and device, electronic equipment and computer readable medium | |
| CN107920257B (en) | Video key point real-time processing method and device and computing equipment | |
| WO2017185630A1 (en) | Emotion recognition-based information recommendation method and apparatus, and electronic device | |
| CN107944420B (en) | Illumination processing method and device for face image | |
| JP2021526698A (en) | Image generation methods and devices, electronic devices, and storage media | |
| CN108876804B (en) | Matting model training and image matting method, device and system and storage medium | |
| US9838616B2 (en) | Image processing method and electronic apparatus | |
| WO2017206400A1 (en) | Image processing method, apparatus, and electronic device | |
| CN111008935B (en) | Face image enhancement method, device, system and storage medium | |
| CN111369428B (en) | Virtual head portrait generation method and device | |
| CN108875539B (en) | Expression matching method, device and system and storage medium | |
| CN109785264B (en) | Image enhancement method and device and electronic equipment | |
| CN110728319B (en) | Image generation method and device and computer storage medium | |
| CN109242031B (en) | Training method, using method, device and processing equipment of posture optimization model | |
| KR20230015430A (en) | Method and apparatus for processing face information, electronic device and storage medium | |
| CN111639667B (en) | Image recognition method, device, electronic equipment and computer readable storage medium | |
| CN110765843B (en) | Face verification method, device, computer equipment and storage medium | |
| CN106250813B (en) | Facial expression migration method and equipment | |
| CN112906571A (en) | Living body identification method and device and electronic equipment | |
| CN110197459B (en) | Image stylization generation method and device and electronic equipment | |
| CN109685015B (en) | Image processing method and device, electronic equipment and computer storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |