CN113674053A - Virtual intelligent display system of try-on mask - Google Patents

Abstract

A virtual intelligent display system of a try-on mask relates to a virtual intelligent display system of a try-on mask, and the intelligent virtual mask try-on system and method are systems which are generated in order to make up for the fact that the try-on mask cannot be carried out in the mask consumption process. The decorative mask is a novel consumer product, and the decoration is the function of non-wear-out. The system can utilize the face recognition model to carry out face positioning, carry out the laminating of gauze mask picture, update the position in real time to carry out the zooming to the gauze mask picture. The gesture posture is monitored, the system timely reacts to carry out gesture switching mask style instructions, and switching can also be carried out through specific keys. Whether the users like the electronic book is judged by using the user expressions in the fitting process, and the data washed by the users are formed, so that a big data model is formed and is used for carrying out personalized recommendation on the users of different types, and the consumption process is more intelligent, reasonable and rapid.

Description

Virtual intelligent display system of try-on mask

Technical Field

The invention relates to a virtual intelligent showing system for a try-on mask, in particular to a virtual intelligent showing system for a try-on mask.

Background

The mask is one of the important means for epidemic prevention. With the occurrence of epidemic situations, more masks appear in each household. In the past, the mask only has a single blue style, and the printed mask with the same protection function gradually emerges, so that the decoration effect of the mask begins to appear. As with clothes, masks of different colors, patterns and styles can exhibit different effects on different consumers. The traditional mask consumer industry cannot try on the mask due to the sterile requirement of the mask. How to try on easily and select a mask style more suitable for the user is still a difficult problem in the mask consumption industry.

The mask has the epidemic prevention function, and gradually shows the decoration function, so that the mask is used as a small ornament by more and more people. The fastest method suitable for the user is selected from the masks in different shapes and colors, the mask is tried on once, and finally the selection is completed through comparison. However, the contactless sale of the mask is an insurmountable barrier, and all types of masks are bought and tried on, which causes financial waste. Based on the above current situation, the inventors of the present application have made extensive studies and have proposed a virtual fitting system for an intelligent mask according to the present invention.

Disclosure of Invention

The invention aims to provide a virtual intelligent showing system for a try-on mask. The gesture posture is monitored, the system timely reacts to carry out gesture switching mask style instructions, and switching can also be carried out through specific keys. Whether the users like the electronic book is judged by using the user expressions in the fitting process, and the data washed by the users are formed, so that a big data model is formed and is used for carrying out personalized recommendation on the users of different types, and the consumption process is more intelligent, reasonable and rapid.

The purpose of the invention is realized by the following technical scheme:

a virtual intelligent showing system of a try-on mask comprises a big data recommendation model, a big data recommendation model and a big data recommendation model, wherein the big data recommendation model is used for performing personalized recommendation on different types of users; the gesture recognition module is used for capturing gestures of a user, switching the types of the masks in the fitting process and increasing intelligent experience; the expression recognition module is used for recognizing the expression of the user in the fitting process, knowing the preference of the user and training a big data recommendation model; the big data recommendation model updates the model in real time according to the expression recognition module and the selection of the user; the big data recommendation model reorders the try-on sequence of the masks according to the user information and tops a plurality of masks with high hobby probability; the gesture recognition model adjusts the threshold value to prevent interference of other factors in the try-on picture.

The virtual intelligent display system of the try-on mask has the following operation mode:

providing a mask type database, recording owned mask style picture files in advance, and reserving an increased file path; acquiring, positioning and transmitting face information into a try-on system by using a face recognition module; synthesizing the mask picture and the face image, and returning to a real-time image; updating the face information in real time, positioning the face of each frame, and acquiring a positioning point and a mask size required by synthesis; and (4) continuously zooming and rotating the virtual mask by an angle for the user who continuously moves left and right or back and forth, and selecting the mask type suitable for the user.

The invention has the advantages and effects that:

the invention aims to make up for a system which is generated when the mask cannot be tried on in the consumption process. The decorative mask is a novel consumer product, and the decoration is the function of non-wear-out. The system can utilize the face recognition model to carry out face positioning, carry out the laminating of gauze mask picture, update the position in real time to carry out the zooming to the gauze mask picture. The gesture posture is monitored, the system timely reacts to carry out gesture switching mask style instructions, and switching can also be carried out through specific keys. Whether the users like the electronic book is judged by using the user expressions in the fitting process, and the data washed by the users are formed, so that a big data model is formed and is used for carrying out personalized recommendation on the users of different types, and the consumption process is more intelligent, reasonable and rapid.

Drawings

FIG. 1 is a flow chart of a fitting system of the present invention;

FIG. 2 is a schematic diagram of the facial recognition step of the present invention;

FIG. 3 is a schematic view of a try-on procedure of the mask of the present invention;

FIG. 4 is a distribution diagram of facial feature points of the present inventors;

FIG. 5 is a diagram of expression recognition classification according to the present invention;

fig. 6 is a hand recognition feature point diagram according to the present invention.

Detailed Description

The present invention will be described in detail with reference to the embodiments shown in the drawings.

The invention relates to a virtual mask try-on system, which comprises a virtual mask try-on device, a mask type database and a user preference database, wherein the system operation method comprises the following steps:

the mask virtual try-on device acquires a video stream containing the whole face of a user;

extracting required facial feature points from the video stream by using a face recognition model to obtain feature point coordinates;

taking out the mask image recommended by the user preference database from the mask type database;

zooming the mask image according to the face information obtained by the face recognition model, so that the zoomed mask image is matched with the face image;

positioning the mask image on a face image and displaying the face image by using an image processing technology;

the mask virtual try-on device judges whether a user likes or not by using an expression recognition model, and sends face information and preference data acquired by the device to the user preference database for model training;

the mask virtual try-on device can utilize a gesture recognition model or man-machine interaction to switch mask types until the try-on is completed, and all data are stored in the user preference database.

The method comprises the following specific steps:

(1) the user selects to start trying on and starts the camera module;

(2) acquiring a video stream containing a face image and a hand image by using a camera, wherein the hand image is not necessary;

in step S101, the face image may be acquired by a camera with a higher pixel, and then the acquired video information is transmitted. The camera can be installed with on the display, when watching the display promptly, the positive information of people's face is shot to the camera. In order to smoothly perform image recognition and processing, the proportion of the human face acquired by the camera in the displayed image should be not less than 1/25. Preferably, the human face is located in the middle of the displayed image, has a ratio of not less than 1/16, and is oriented in the forward direction.

The camera can also be a remote camera, and the camera uploads the obtained video stream to the virtual try-on device at the cloud, so that the user can process the recorded video. So that the user can not use the device or perform virtual try-on of the mask for other people not around.

(3) The face recognition module analyzes the video stream to obtain coordinate values of the required face characteristic points in a coordinate system taking the camera picture as a reference, and meanwhile, the coordinates of the face characteristic points in each frame of picture are continuously updated. Generally, there are characteristic points in facial features, cheekbones and mandible positions;

and step S102, the face recognition module carries out frame-by-frame processing on the video stream, carries out face positioning by using the trained model, extracts face characteristic information, establishes a coordinate system by taking the display image as a reference, and positions the coordinates of the characteristic points. All coordinates are stored as a python dictionary in which feature points correspond to the relative coordinate positions of the feature points, and the face feature point information is imported in the next module at step S103.

(4) Rearranging the try-on sequence of the mask according to the data given by the big data recommendation model, and advancing the style of the mask with high love probability;

numbering each mask of the mask type database in advance, reserving a file path for a new mask style, inputting human characteristic point information as data, training the model according to the pre-training model and the data acquired in the later try-on process to output favorite mask records of the user, and performing personalized sequencing on each user according to predicted data to improve the try-on efficiency.

(5) Step S201, according to the face feature information obtained by the face recognition module, selecting two pupil positions as two feature points, describing whether the face of a user inclines or not according to the difference of Y values (vertical direction) of relative coordinates of the two points, calculating the angle between the face of the user and the horizontal direction by using the coordinates of the two points, and outputting angle information;

step S202, redefining the sizes of the mask styles in the mask type database to meet the face sizes of users, continuously zooming the masks according to continuously updated feature point coordinates,

due to the fact that the sizes of the faces of users are different, the distances between the users and the cameras are changed, the angles of the faces of the users are different, and the fitting system updates and replaces the sizes of the masks in real time according to face size information identified by the face identification module. Even if the user continuously shakes left and right and moves back and forth, the virtual mask can be ensured to be worn on the face of the user.

And step S203, attaching the mask picture which is properly zoomed to a human, positioning by using a coordinate dictionary in the data dictionary, and finely adjusting at a calculated angle to ensure attachment.

(6) Capturing gestures by using a gesture recognition module or sending out a command for switching the mask style through a man-machine interaction interface button;

the system is simultaneously provided with a gesture recognition module and a button for a human-computer interaction interface of a user, so that the joint points of the hand are positioned, and the coordinate information of each characteristic point of the hand is utilized. Wherein when the gesture recognition module recognizes the next or previous gesture, an instruction is given to the try-on system. Meanwhile, in order to prevent the unintentional influence of the gestures of other people in the background picture, the threshold value of the gesture recognition system is corrected, and the influence of other factors is avoided.

(7) Judging the likes and dislikes of the user according to the reaction after the user tries to wear the mask by using an expression recognition module, storing data into a user preference database module, training an intelligent recommendation model, and increasing the scale of a training set of the model;

the expression recognition function utilizes an expression recognition model to distinguish, captures the expression of a user after trying on a mask, analyzes each frame of picture in a video stream, totally divides the expression recognition into 7 labels, considers the surprised and neutral label values as 0, considers the happiness as 1 and considers the sadness, the anger, the fear and the disgust as-1, and stores the face data and the mask type of the frame as a training set into a user preference database module. Each face data was evaluated for that mask type and a big data model was trained. The user can obtain the fondness of different users to different masks, and the higher the predicted label value is, the more likely the user likes the mask. The part of training set is the key for improving the big data recommendation model, and the intelligent model has higher accuracy by using continuously updated data.

(8) The user selects to complete the try-on;

and finishing the try-on.

Claims (2)

1. The virtual intelligent showing system for the try-on mask is characterized by comprising a big data recommendation model, a big data recommendation model and a big data recommendation model, wherein the big data recommendation model is used for performing personalized recommendation on different types of users; the gesture recognition module is used for capturing gestures of a user, switching the types of the masks in the fitting process and increasing intelligent experience; the expression recognition module is used for recognizing the expression of the user in the fitting process, knowing the preference of the user and training a big data recommendation model; the big data recommendation model updates the model in real time according to the expression recognition module and the selection of the user; the big data recommendation model reorders the try-on sequence of the masks according to the user information and tops a plurality of masks with high hobby probability; the gesture recognition model adjusts the threshold value to prevent interference of other factors in the try-on picture.

2. The virtual intelligent appearance system of a try-on mask of claim 1, wherein the system operates as follows:

providing a mask type database, recording owned mask style picture files in advance, and reserving an increased file path; acquiring, positioning and transmitting face information into a try-on system by using a face recognition module; synthesizing the mask picture and the face image, and returning to a real-time image; updating the face information in real time, positioning the face of each frame, and acquiring a positioning point and a mask size required by synthesis; and (4) continuously zooming and rotating the virtual mask by an angle for the user who continuously moves left and right or back and forth, and selecting the mask type suitable for the user.

Images