WO2023194466A1 - Method for recommending cosmetic products using a knn algorithm - Google Patents
Method for recommending cosmetic products using a knn algorithm Download PDFInfo
- Publication number
- WO2023194466A1 WO2023194466A1 PCT/EP2023/058988 EP2023058988W WO2023194466A1 WO 2023194466 A1 WO2023194466 A1 WO 2023194466A1 EP 2023058988 W EP2023058988 W EP 2023058988W WO 2023194466 A1 WO2023194466 A1 WO 2023194466A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- product
- cosmetic product
- source
- source image
- characterizing parameters
- Prior art date
Links
- 239000002537 cosmetic Substances 0.000 title claims abstract description 93
- 238000000034 method Methods 0.000 title claims abstract description 37
- 238000013528 artificial neural network Methods 0.000 claims abstract description 29
- 239000013598 vector Substances 0.000 claims description 22
- 238000004590 computer program Methods 0.000 claims description 6
- 238000010801 machine learning Methods 0.000 claims description 5
- 238000012545 processing Methods 0.000 claims description 5
- 238000012549 training Methods 0.000 claims description 3
- 230000003796 beauty Effects 0.000 description 6
- 238000009877 rendering Methods 0.000 description 3
- 238000004891 communication Methods 0.000 description 2
- 238000013527 convolutional neural network Methods 0.000 description 2
- 230000002194 synthesizing effect Effects 0.000 description 2
- 241001237961 Amanita rubescens Species 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 230000003190 augmentative effect Effects 0.000 description 1
- 238000007796 conventional method Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 239000002966 varnish Substances 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/168—Feature extraction; Face representation
- G06V40/171—Local features and components; Facial parts ; Occluding parts, e.g. glasses; Geometrical relationships
-
- A—HUMAN NECESSITIES
- A45—HAND OR TRAVELLING ARTICLES
- A45D—HAIRDRESSING OR SHAVING EQUIPMENT; EQUIPMENT FOR COSMETICS OR COSMETIC TREATMENTS, e.g. FOR MANICURING OR PEDICURING
- A45D44/00—Other cosmetic or toiletry articles, e.g. for hairdressers' rooms
- A45D44/005—Other cosmetic or toiletry articles, e.g. for hairdressers' rooms for selecting or displaying personal cosmetic colours or hairstyle
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
- G06F18/2413—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on distances to training or reference patterns
- G06F18/24147—Distances to closest patterns, e.g. nearest neighbour classification
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
Definitions
- the present invention relates to a method for recommending a cosmetic product, which method comprises implementing an encoding artificial neural network.
- the choice of beauty products often depends on the personal characteristics of a wearer. These characteristics may include lifestyle, colour preferences, body chemistry, style preferences and/or physical attributes of the wearer. Thus, one shade of lipstick that works well on a young woman with blond hair and fair skin may not work well on a more mature woman with dark hair and dark skin. Beauty professionals, whether they be in beauty establishments, retail establishments or other places, are therefore often asked for assistance in selecting beauty products. Of course, the ability to obtain useful advice depends both on the level of personal attention available and on the level of competence of the beauty professional who is giving the advice. For example, if a beauty counter in a retail establishment does not have sufficient staff to manage the customer traffic or if the staff are poorly trained, the quality of the advice may suffer as a result.
- This task consists in extracting a make-up style from a reference portrait image and in synthesizing it on the target image of a different person.
- the make-up attributes are extracted with the aid of a neural network and rendered on the source image with the aid of a generative model, such as a GAN or VAE.
- the faces into face groups based on the skin tone value of each of the faces, the face groups each comprising faces of a similar skin tone which are grouped from the input image and grouped from the source image based on an average skin tone value difference less than or equal to a grouping threshold of the faces with a similar skin tone;
- the colour characteristics comprising the skin tones of the respective faces in the face group pair as part of the colour characteristics applied to modify the input image.
- WO2014168678A1 discloses a system comprising:
- a user information module wherein the user information module captures an image of the user’s skin
- a graphical user interface wherein the graphical user interface allows the selection of a skin-related application from a plurality of skin-related applications
- processor coupled to the user information module, to the graphical user interface and to the database of skin images, wherein the processor is configured to:
- an output display wherein the output display displays the simulated image of the user’s skin.
- US2021209427A1 discloses an apparatus comprising:
- a processing circuit for accepting at least one image representative of the skin of the face of a user
- a communication circuit for transmitting the accepted image to machine learning models and for receiving a regimen recommendation from the machine learning models
- a user interface circuit for displaying the regimen recommendation to the user
- the user interface circuit is further configured to display images of human faces to the user
- the processing circuit being further configured to accept an input of the user which classifies the characteristics of the skin of the face based on images which are supplied to it via the user interface circuit;
- the communication interface circuit being further configured to transmit the input of the user to machine learning models as training data.
- the problem posed by the invention is:
- the product recommendations may thus be used to assist makeup consumers in making an aesthetic choice, or even potentially be used to obtain a product having this colour.
- One subject of the invention is a method for extracting information about a source cosmetic product from a source image and recommending a target cosmetic product close to the source cosmetic product from a database.
- the method implements a convolutional neural network trained to retrieve the characteristics of the source image and offering two approaches for using the k-nearest neighbours method, and to find the target product in the database of reference products that is closest to the source cosmetic product.
- a method for recommending a cosmetic product comprising: obtaining a source image of a source cosmetic product applied to a first person, implementing an encoding artificial neural network configured so as to determine characterizing parameters of the source cosmetic product from the source image,
- the recommendation comprising the implementation of a k-nearest neighbours algorithm (k-NN).
- the method may be used to recommend cosmetic products of various types.
- the method may be used for make-up products or haircare products.
- the make-up products may be blusher, foundation, mascara, lipstick, lip gloss, nail varnish or eyeliner.
- the haircare product may be a dye.
- the cosmetic product is a lipstick.
- the invention has the advantage of implementing the nearest neighbours algorithm which obtains reliable and precise results.
- the method thus implements an artificial neural network trained to automatically extract parameters of the source cosmetic product from the source image. These parameters make it possible to search the database for a target cosmetic product defined by close parameters.
- the artificial neural network that is used has the advantage of needing few computing resources to be executed, in comparison in particular with conventional methods using inverse graphics networks in which a gradient descent is performed.
- the proposed method may therefore be implemented by devices with low computational capabilities, in particular by portable devices, for example smartphones, tablets, for example.
- said at least one source image is obtained using a photography device.
- the recommended target product may be displayed on a screen.
- the cosmetic product is a make-up product for the lips.
- the parameters of the vector may then be an opacity, a colour, a reflection intensity, an amount and a texture of the make-up product for the lips.
- a cosmetic product recommendation engine comprising a k-nearest neighbours algorithm (k-NN), the recommendation engine receiving, as input, the vector of characterizing parameters that was determined using the encoding artificial neural network and generating, as output, a recommendation of a target cosmetic product from the database.
- k-NN k-nearest neighbours algorithm
- a computer program product comprising instructions that, when the program is executed by a computer, prompt said computer to:
- a cosmetic product recommendation engine comprising a k-nearest neighbours algorithm (k-NN)
- the recommendation engine receiving, as input, the vector of characterizing parameters that was determined using the encoding artificial neural network and generating, as output, a recommendation of a target cosmetic product from the database.
- a system comprising: - a memory which stores the computer program product defined above, the source image and the database,
- a processing unit configured so as to implement the computer program product
- a photography device configured so as to acquire the source image
- a screen configured so as to display the target cosmetic product.
- the device according to the invention has one or more of the following features, taken alone or in combination:
- Said at least one source image is obtained using a photography device, and the recommendation of the cosmetic product is displayed on a screen.
- the cosmetic product is a make-up product for the lips, and the parameters of the vector are an opacity, a colour, a reflection intensity, an amount and a texture of the make-up product for the lips.
- the source cosmetic product is entered into the database.
- the k-nearest neighbours algorithm (k-NN) is implemented after extracting values using a machine learning model.
- the characterizing parameters of the source cosmetic product are stored in the database.
- the implementation of the encoding artificial neural network comprises detecting shimmering on the source image; classifying the source image depending on the detected shimmering; differentiating the algorithm for recommending the target cosmetic product depending on the classification of the source image.
- the source image is preprocessed to crop the body area and the characterizing parameters of the source cosmetic product are shown in an E-type vector (product R, product G, product B, gloss, gloss detail, moisture, intensity), so as to supply the input of the recommendation algorithm.
- E-type vector product R, product G, product B, gloss, gloss detail, moisture, intensity
- Shimmering is detected on the source image and the characterizing parameters of the source cosmetic product are shown in an E-type vector (product R, product G, product B, gloss, gloss detail, moisture, intensity, shimmering R, shimmering G, shimmering B), so as to supply the input of the recommendation algorithm.
- FIG. 1 illustrates one mode of implementation of a method for recommending a cosmetic product.
- the rendering generation method comprises obtaining a source image .
- the source image is an image of a real cosmetic product PC applied to a person.
- the cosmetic product is a make-up product for the face or a haircare product
- the source image may be a portrait of a person to whom the make-up product or the haircare product is applied.
- the cosmetic product is a lipstick.
- the method may comprise defining an area of interest of the source image , the area of interest corresponding to the area of the source image in which the cosmetic product PC is located, in this case the mouth of the person to whom the lipstick is applied.
- the method according to the invention comprises implementing an encoding artificial neural network.
- This encoding artificial neural network is trained to extract characterizing parameters of a cosmetic product from an image received as input of this encoding artificial neural network.
- the characterizing parameters make it possible to define the characteristics of the cosmetic product present in the image received as input.
- the cosmetic product is a make-up product for the lips, in particular a lipstick or a lip gloss
- the characterizing parameters may be chosen from a gloss, a gloss detail, a moisture, an intensity, a shimmering R, a shimmering G, a shimmering B of the make-up product for the lips.
- the encoding artificial neural network E is for example a convolutional neural network.
- the encoding artificial neural network is implemented by taking the source image or the defined area of interest of the source image as input.
- the encoding artificial neural network E then makes it possible to determine characterizing parameters associated with the cosmetic product PC shown in the source image or in the area of interest of the source image. These characterizing parameters are shown in a vector .
- the rendering generation method then comprises implementing a product recommendation engine comprising the implementation of a k -nearest neighbours algorithm ( k -NN).
- the recommendation engine is configured to receive, as input, the vector of the characterizing parameters of the source cosmetic product.
- the recommendation engine is also configured to generate, as output, a recommendation of a cosmetic product based on the characterizing parameters provided as input.
- the cosmetic product generated by the recommendation then has the characteristics defined by the characterizing parameters provided as input of the recommendation engine (KNN).
- the recommendation engine is implemented by receiving, as input, at least said vector or of characterizing parameters that was determined by the encoding artificial neural network.
- the recommendation engine then makes it possible to generate at least one recommendation of a cosmetic product.
- Such a recommendation method may be used by a user to choose a cosmetic product virtually, from an image of a person to whom is applied the cosmetic product that the user wants to try on virtually.
- the recommendation method may also be used to generate, in a simple manner, vectors of characterizing parameters of a cosmetic product from a source image . These vectors of characterizing parameters may then be used as input of a recommendation engine when a user wishes to try on the cosmetic product virtually.
- the source image ( ) is classified depending on the shimmering detected on the source image.
- the shimmering may for example be detected with the aid of an artificial intelligence algorithm.
- an encoding artificial neural network E1 makes it possible to determine characterizing parameters associated with the cosmetic product. These characterizing parameters are shown in a vector , namely ⁇ colour (R, G, B), gloss, gloss detail, moisture, intensity>.
- the vector is then processed by a supervised learning algorithm implementing the k-nearest neighbours KNN method which generates a recommendation of one or more cosmetic products (brand a, brand b, brand c) wherein the recommendation engine compares the characterizing parameters of the source image to those of the reference products of a database of products with no shimmering using a KNN algorithm.
- the database of products with no shimmering comprises N products with no shimmering, each product ranging from 1 to N being defined by a brand with which a colour, a gloss, a gloss detail, a moisture and an intensity are associated.
- an encoding artificial neural network E2 makes it possible to determine characterizing parameters associated with the cosmetic product. These characterizing parameters are shown in a vector , namely ⁇ shimmering (R, G, B), colour (R, G, B), gloss, gloss detail, moisture, intensity>.
- the vector is then processed by a supervised learning algorithm implementing the k-nearest neighbours KNN method which generates a recommendation of one or more cosmetic products (brand i, brand j, brand k) wherein the recommendation engine compares the characterizing parameters of the source image to those of the reference products of a database of products with shimmering using a KNN algorithm.
- the database of products with shimmering comprises M products with shimmering, each product ranging from 1 to M being defined by a brand with which a shimmering, a colour, a gloss, a gloss detail, a moisture and an intensity are associated.
- the generated cosmetic product then has the characteristics defined by the characterizing parameters provided as input of the recommendation artificial neural network.
- the invention is not limited to the embodiment of the example shown.
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Oral & Maxillofacial Surgery (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Evolutionary Computation (AREA)
- Data Mining & Analysis (AREA)
- Multimedia (AREA)
- Artificial Intelligence (AREA)
- Computing Systems (AREA)
- Databases & Information Systems (AREA)
- Medical Informatics (AREA)
- Software Systems (AREA)
- Human Computer Interaction (AREA)
- Life Sciences & Earth Sciences (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Evolutionary Biology (AREA)
- General Engineering & Computer Science (AREA)
- Image Analysis (AREA)
- Image Processing (AREA)
Abstract
The invention relates to a method for recommending a cosmetic product, comprising: obtaining a source image (Xref) of a source cosmetic product (PC) applied to a body area of a person, implementing an encoding artificial neural network (E) configured so as to determine characterizing parameters of the source cosmetic product (PC) from the source image (Xref), identifying, amongst reference cosmetic products of a database, a target cosmetic product having the characterizing parameters closest to those of the source cosmetic product (PC); and recommending the target cosmetic product.
Description
The present invention relates to a method for recommending a cosmetic product, which method comprises implementing an encoding artificial neural network.
Prior art
The choice of beauty products often depends on the personal characteristics of a wearer. These characteristics may include lifestyle, colour preferences, body chemistry, style preferences and/or physical attributes of the wearer. Thus, one shade of lipstick that works well on a young woman with blond hair and fair skin may not work well on a more mature woman with dark hair and dark skin. Beauty professionals, whether they be in beauty establishments, retail establishments or other places, are therefore often asked for assistance in selecting beauty products. Of course, the ability to obtain useful advice depends both on the level of personal attention available and on the level of competence of the beauty professional who is giving the advice. For example, if a beauty counter in a retail establishment does not have sufficient staff to manage the customer traffic or if the staff are poorly trained, the quality of the advice may suffer as a result.
Nowadays, virtual try-on technologies are widespread on retail platforms online and on social media. In particular, for makeup, consumers have the opportunity to try on cosmetic products virtually in augmented reality before buying. Although it is easy to create virtual makeup for entertainment purposes, the parametrization of a rendering engine for synthesizing realistic images of a given cosmetic product remains a tedious task and requires in-depth knowledge in computer graphics. Furthermore, consumers are often invited to choose from a selection of predefined shades of makeup but are not able to try on makeup from inspiring reference images on social networks.
In recent years, the computer vision sector has tried to provide a solution to this problem thanks to the progress made in the task of transferring make-up styles. This task consists in extracting a make-up style from a reference portrait image and in synthesizing it on the target image of a different person. The make-up attributes are extracted with the aid of a neural network and rendered on the source image with the aid of a generative model, such as a GAN or VAE.
Document US10936853B1 discloses a method implemented using a computer device, the method comprising:
receiving an input image to be modified based on the colour characteristics of a source image;
detecting faces in the input image and in the source image;
determining a skin tone value reflective of a skin tone of each of the faces;
grouping the faces into face groups based on the skin tone value of each of the faces, the face groups each comprising faces of a similar skin tone which are grouped from the input image and grouped from the source image based on an average skin tone value difference less than or equal to a grouping threshold of the faces with a similar skin tone;
matching a face group pair as input image face group paired with a source image face group; and
generating a modified image from the input image based on the colour characteristics of the source image, the colour characteristics comprising the skin tones of the respective faces in the face group pair as part of the colour characteristics applied to modify the input image.
WO2014168678A1 discloses a system comprising:
a user information module, wherein the user information module captures an image of the user’s skin;
a graphical user interface, wherein the graphical user interface allows the selection of a skin-related application from a plurality of skin-related applications;
a database of skin images;
a processor coupled to the user information module, to the graphical user interface and to the database of skin images, wherein the processor is configured to:
determine data about the image of the user’s skin from the image of the user’s skin;
identify one or more sets of skin image data in the database of skin images which correspond to the data about the image of the user’s skin based on one or more parameters specified by the skin-related application; and
apply at least one image processing filter, which corresponds to the identified sets of skin image data from the database of skin images, to the image of the user’s skin in order to generate a simulated image of the user’s skin; and
an output display, wherein the output display displays the simulated image of the user’s skin.
US2021209427A1 discloses an apparatus comprising:
a processing circuit for accepting at least one image representative of the skin of the face of a user;
a communication circuit for transmitting the accepted image to machine learning models and for receiving a regimen recommendation from the machine learning models;
a user interface circuit for displaying the regimen recommendation to the user,
wherein the user interface circuit is further configured to display images of human faces to the user;
the processing circuit being further configured to accept an input of the user which classifies the characteristics of the skin of the face based on images which are supplied to it via the user interface circuit; and
the communication interface circuit being further configured to transmit the input of the user to machine learning models as training data.
These documents do not disclose any recommendation of cosmetic products.
Disclosure of the invention
There is a need to provide product recommendations to makeup consumers based on a reference portrait image, found for example on social networks, directly, that is to say without consulting, by appointment, a person qualified in the art.
The problem posed by the invention is:
Extracting the values of the characteristics of a source cosmetic product (source), in particular of a lipstick, from an image containing the product applied to a body surface, in particular the lips.
Recommending a cosmetic product from a database of cosmetic products, in particular a lipstick, which has predicted characteristic values based on the source lipstick.
The product recommendations may thus be used to assist makeup consumers in making an aesthetic choice, or even potentially be used to obtain a product having this colour.
One subject of the invention is a method for extracting information about a source cosmetic product from a source image and recommending a target cosmetic product close to the source cosmetic product from a database. The method implements a convolutional neural network trained to retrieve the characteristics of the source image and offering two approaches for using the k-nearest neighbours method, and to find the target product in the database of reference products that is closest to the source cosmetic product.
According to one aspect, what is proposed is a method for recommending a cosmetic product, comprising:
obtaining a source image of a source cosmetic product applied to a first person,
implementing an encoding artificial neural network configured so as to determine characterizing parameters of the source cosmetic product from the source image,
obtaining a source image of a source cosmetic product applied to a first person,
implementing an encoding artificial neural network configured so as to determine characterizing parameters of the source cosmetic product from the source image,
identifying, amongst reference cosmetic products of a database, a target cosmetic product having the characterizing parameters closest to those of the source cosmetic product; and
recommending the target cosmetic product,
the recommendation comprising the implementation of a k-nearest neighbours algorithm (k-NN).
The method may be used to recommend cosmetic products of various types. For example, the method may be used for make-up products or haircare products. In particular, the make-up products may be blusher, foundation, mascara, lipstick, lip gloss, nail varnish or eyeliner. The haircare product may be a dye. In particular, the cosmetic product is a lipstick.
The invention has the advantage of implementing the nearest neighbours algorithm which obtains reliable and precise results.
The method thus implements an artificial neural network trained to automatically extract parameters of the source cosmetic product from the source image. These parameters make it possible to search the database for a target cosmetic product defined by close parameters.
The artificial neural network that is used has the advantage of needing few computing resources to be executed, in comparison in particular with conventional methods using inverse graphics networks in which a gradient descent is performed.
The proposed method may therefore be implemented by devices with low computational capabilities, in particular by portable devices, for example smartphones, tablets, for example.
Advantageously, said at least one source image is obtained using a photography device. Moreover, the recommended target product may be displayed on a screen.
In one advantageous mode of implementation, the cosmetic product is a make-up product for the lips. The parameters of the vector may then be an opacity, a colour, a reflection intensity, an amount and a texture of the make-up product for the lips.
According to another aspect, what is proposed is a method for training an encoding artificial neural network as described above, comprising:
extracting, from a database, predefined characterizing parameters associated with reference cosmetic products,
implementing an encoding artificial neural network, so as to determine the characterizing parameters of a source cosmetic product, taking a source image as input,
implementing a cosmetic product recommendation engine comprising a k-nearest neighbours algorithm (k-NN), the recommendation engine receiving, as input, the vector of characterizing parameters that was determined using the encoding artificial neural network and generating, as output, a recommendation of a target cosmetic product from the database.
According to another aspect, what is proposed is a computer program product comprising instructions that, when the program is executed by a computer, prompt said computer to:
- extract, from a database, predefined characterizing parameters associated with reference cosmetic products,
- implementing an encoding artificial neural network, so as to determine the characterizing parameters of a source cosmetic product, taking a source image as input,
- implementing a cosmetic product recommendation engine comprising a k-nearest neighbours algorithm (k-NN), the recommendation engine receiving, as input, the vector of characterizing parameters that was determined using the encoding artificial neural network and generating, as output, a recommendation of a target cosmetic product from the database.
According to another aspect, what is proposed is a system comprising:
- a memory which stores the computer program product defined above, the source image and the database,
- a memory which stores the computer program product defined above, the source image and the database,
- a processing unit configured so as to implement the computer program product,
- a photography device configured so as to acquire the source image,
- a screen configured so as to display the target cosmetic product.
- a photography device configured so as to acquire the source image,
- a screen configured so as to display the target cosmetic product.
Preferred embodiments
Preferably, the device according to the invention has one or more of the following features, taken alone or in combination:
Said at least one source image is obtained using a photography device, and the recommendation of the cosmetic product is displayed on a screen.
The cosmetic product is a make-up product for the lips, and the parameters of the vector are an opacity, a colour, a reflection intensity, an amount and a texture of the make-up product for the lips.
The source cosmetic product is entered into the database.
The k-nearest neighbours algorithm (k-NN) is implemented after extracting values using a machine learning model.
The characterizing parameters of the source cosmetic product are stored in the database.
The implementation of the encoding artificial neural network comprises detecting shimmering on the source image; classifying the source image depending on the detected shimmering; differentiating the algorithm for recommending the target cosmetic product depending on the classification of the source image.
No shimmering is detected on the source image, the source image is preprocessed to crop the body area and the characterizing parameters of the source cosmetic product are shown in an E-type vector (product R, product G, product B, gloss, gloss detail, moisture, intensity), so as to supply the input of the recommendation algorithm.
Shimmering is detected on the source image and the characterizing parameters of the source cosmetic product are shown in an E-type vector (product R, product G, product B, gloss, gloss detail, moisture, intensity, shimmering R, shimmering G, shimmering B), so as to supply the input of the recommendation algorithm.
The invention may be better understood from reading the following detailed description of a nonlimiting exemplary implementation thereof, and from studying the appended schematic and partial drawing, in which:
The rendering generation method comprises obtaining a source image . The source image is an image of a real cosmetic product PC applied to a person. For example, when the cosmetic product is a make-up product for the face or a haircare product, the source image may be a portrait of a person to whom the make-up product or the haircare product is applied. In this case, the cosmetic product is a lipstick.
The method may comprise defining an area of interest of the source image , the area of interest corresponding to the area of the source image in which the cosmetic product PC is located, in this case the mouth of the person to whom the lipstick is applied.
In particular, the method according to the invention comprises implementing an encoding artificial neural network. This encoding artificial neural network is trained to extract characterizing parameters of a cosmetic product from an image received as input of this encoding artificial neural network.
The characterizing parameters make it possible to define the characteristics of the cosmetic product present in the image received as input. For example, when the cosmetic product is a make-up product for the lips, in particular a lipstick or a lip gloss, the characterizing parameters may be chosen from a gloss, a gloss detail, a moisture, an intensity, a shimmering R, a shimmering G, a shimmering B of the make-up product for the lips.
The encoding artificial neural network E is for example a convolutional neural network.
More particularly, in the method according to the invention, the encoding artificial neural network is implemented by taking the source image or the defined area of interest of the source image as input. The encoding artificial neural network E then makes it possible to determine characterizing parameters associated with the cosmetic product PC shown in the source image or in the area of interest of the source image. These characterizing parameters are shown in a vector .
The rendering generation method then comprises implementing a product recommendation engine comprising the implementation of a k-nearest neighbours algorithm (k-NN).
The recommendation engine is configured to receive, as input, the vector of the characterizing parameters of the source cosmetic product. The recommendation engine is also configured to generate, as output, a recommendation of a cosmetic product based on the characterizing parameters provided as input.
The cosmetic product generated by the recommendation then has the characteristics defined by the characterizing parameters provided as input of the recommendation engine (KNN).
In particular, in the recommendation method, the recommendation engine is implemented by receiving, as input, at least said vector or of characterizing parameters that was determined by the encoding artificial neural network. The recommendation engine then makes it possible to generate at least one recommendation of a cosmetic product.
Such a recommendation method may be used by a user to choose a cosmetic product virtually, from an image of a person to whom is applied the cosmetic product that the user wants to try on virtually.
The recommendation method may also be used to generate, in a simple manner, vectors of characterizing parameters of a cosmetic product from a source image . These vectors of characterizing parameters may then be used as input of a recommendation engine when a user wishes to try on the cosmetic product virtually.
According to the invention, the source image ( ) is classified depending on the shimmering detected on the source image. The shimmering may for example be detected with the aid of an artificial intelligence algorithm.
If no shimmering is detected, an encoding artificial neural network E1 makes it possible to determine characterizing parameters associated with the cosmetic product. These characterizing parameters are shown in a vector , namely <colour (R, G, B), gloss, gloss detail, moisture, intensity>.
The vector is then processed by a supervised learning algorithm implementing the k-nearest neighbours KNN method which generates a recommendation of one or more cosmetic products (brand a, brand b, brand c) wherein the recommendation engine compares the characterizing parameters of the source image to those of the reference products of a database of products with no shimmering using a KNN algorithm.
The implementation of the KNN recommendation engine makes it possible to obtain a product recommendation based on the characterizing parameters shown in the vector
In the example shown, the database of products with no shimmering comprises N products with no shimmering, each product ranging from 1 to N being defined by a brand with which a colour, a gloss, a gloss detail, a moisture and an intensity are associated.
If shimmering is detected, an encoding artificial neural network E2 makes it possible to determine characterizing parameters associated with the cosmetic product. These characterizing parameters are shown in a vector , namely <shimmering (R, G, B), colour (R, G, B), gloss, gloss detail, moisture, intensity>.
The vector is then processed by a supervised learning algorithm implementing the k-nearest neighbours KNN method which generates a recommendation of one or more cosmetic products (brand i, brand j, brand k) wherein the recommendation engine compares the characterizing parameters of the source image to those of the reference products of a database of products with shimmering using a KNN algorithm.
The implementation of the KNN recommendation engine makes it possible to obtain a product recommendation based on the characterizing parameters shown in the vector
In the example shown, the database of products with shimmering comprises M products with shimmering, each product ranging from 1 to M being defined by a brand with which a shimmering, a colour, a gloss, a gloss detail, a moisture and an intensity are associated.
The generated cosmetic product then has the characteristics defined by the characterizing parameters provided as input of the recommendation artificial neural network.
The invention is not limited to the embodiment of the example shown.
Claims (12)
- Method for recommending a cosmetic product, comprising:
obtaining a source image (Xref) of a source cosmetic product (PC) applied to a body area of a person,
implementing an encoding artificial neural network configured so as to determine characterizing parameters of the source cosmetic product (PC) from the source image (Xref),
identifying, amongst reference cosmetic products of a database, a target cosmetic product having the characterizing parameters closest to those of the source cosmetic product (PC); and
recommending the target cosmetic product,
the recommendation comprising the implementation of a k-nearest neighbours algorithm (k-NN). - Method according to Claim 1, wherein said at least one source image is obtained using a photography device, and wherein the recommendation of the cosmetic product is displayed on a screen.
- Method according to either one of the preceding claims, wherein the cosmetic product (PC) is a make-up product for the lips, and wherein the parameters of the vector are an opacity, a colour, a reflection intensity, an amount and a texture of the make-up product for the lips.
- Method according to any one of the preceding claims, wherein the source cosmetic product (PC) is entered into the database.
- Method according to any one of the preceding claims, wherein the k-nearest neighbours algorithm (k-NN) is implemented after extracting values using a machine learning model.
- Method according to any one of the preceding claims, characterized in that the characterizing parameters of the source cosmetic product (PC) are stored in the database.
- Method according to any one of the preceding claims, wherein the implementation of the encoding artificial neural network (E1, E2) comprises detecting shimmering on the source image (Xref); classifying the source image (Xref) depending on the detected shimmering; differentiating the algorithm for recommending the target cosmetic product depending on the classification of the source image (Xref).
- Method according to the preceding claim, wherein no shimmering is detected on the source image, the source image is preprocessed to crop the body area and the characterizing parameters of the source cosmetic product are shown in an E-type vector (product R, product G, product B, gloss, gloss detail, moisture, intensity), so as to supply the input of the recommendation algorithm.
- Method according to Claim 8, wherein shimmering is detected on the source image and the characterizing parameters of the source cosmetic product are shown in an E-type vector (product R, product G, product B, gloss, gloss detail, moisture, intensity, shimmering R, shimmering G, shimmering B), so as to supply the input of the recommendation algorithm.
- Method for training an encoding artificial neural network of one of the preceding claims, comprising:
extracting, from a database, predefined characterizing parameters associated with reference cosmetic products,
implementing an encoding artificial neural network (E1, E2), so as to determine the characterizing parameters of a source cosmetic product (PC), taking a source image as input,
implementing a cosmetic product recommendation engine comprising a k-nearest neighbours algorithm (k-NN), the recommendation engine receiving, as input, the vector (E1(Xref), E2(Xref)) of characterizing parameters that was determined using the encoding artificial neural network (E1, E2) and generating, as output, a recommendation of a target cosmetic product from the database. - Computer program product comprising instructions that, when the program is executed by a computer, prompt said computer to:
extract, from a database, predefined characterizing parameters associated with reference cosmetic products,
implement an encoding artificial neural network (E1, E2), so as to determine the characterizing parameters of a source cosmetic product (PC), taking a source image as input,
implement a cosmetic product recommendation engine comprising a k-nearest neighbours algorithm (k-NN), the recommendation engine receiving, as input, the vector (E1(Xref), E2(Xref)) of characterizing parameters that was determined using the encoding artificial neural network (E1, E2) and generating, as output, a recommendation of a target cosmetic product from the database. - System comprising:
a memory which stores the computer program product according to the preceding claim, the source image (Xref) and the database,
a processing unit configured so as to implement the computer program product, a photography device configured so as to acquire the source image,
a screen configured so as to display the target cosmetic product.
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
FRFR2203201 | 2022-04-07 | ||
FR2203201A FR3134469A1 (en) | 2022-04-07 | 2022-04-07 | Cosmetic product recommendation method using a kNN algorithm |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2023194466A1 true WO2023194466A1 (en) | 2023-10-12 |
Family
ID=82196774
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/EP2023/058988 WO2023194466A1 (en) | 2022-04-07 | 2023-04-05 | Method for recommending cosmetic products using a knn algorithm |
Country Status (2)
Country | Link |
---|---|
FR (1) | FR3134469A1 (en) |
WO (1) | WO2023194466A1 (en) |
Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2014168678A1 (en) | 2013-04-09 | 2014-10-16 | Elc Management Llc | Skin diagnostic and image processing systems, apparatus and articles |
US20190272659A1 (en) * | 2014-02-23 | 2019-09-05 | Northeastern University | System for Beauty, Cosmetic, and Fashion Analysis |
US10936853B1 (en) | 2019-10-04 | 2021-03-02 | Adobe Inc. | Skin tone assisted digital image color matching |
US20210209427A1 (en) | 2018-01-05 | 2021-07-08 | L'oreal | Machine-implemented facial health and beauty assistant |
-
2022
- 2022-04-07 FR FR2203201A patent/FR3134469A1/en active Pending
-
2023
- 2023-04-05 WO PCT/EP2023/058988 patent/WO2023194466A1/en unknown
Patent Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2014168678A1 (en) | 2013-04-09 | 2014-10-16 | Elc Management Llc | Skin diagnostic and image processing systems, apparatus and articles |
US20190272659A1 (en) * | 2014-02-23 | 2019-09-05 | Northeastern University | System for Beauty, Cosmetic, and Fashion Analysis |
US20210209427A1 (en) | 2018-01-05 | 2021-07-08 | L'oreal | Machine-implemented facial health and beauty assistant |
US10936853B1 (en) | 2019-10-04 | 2021-03-02 | Adobe Inc. | Skin tone assisted digital image color matching |
Also Published As
Publication number | Publication date |
---|---|
FR3134469A1 (en) | 2023-10-13 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US10799010B2 (en) | Makeup application assist device and makeup application assist method | |
KR101140533B1 (en) | Method and system for recommending a product based upon skin color estimated from an image | |
JP4435809B2 (en) | Virtual makeup apparatus and method | |
CN109840825A (en) | The recommender system of physical features based on user | |
US11978242B2 (en) | Systems and methods for improved facial attribute classification and use thereof | |
CN108694736B (en) | Image processing method, image processing device, server and computer storage medium | |
US11010894B1 (en) | Deriving a skin profile from an image | |
KR102253750B1 (en) | Makeup recommendation and selling cosmetics platform service using deep learning | |
JP2002224049A (en) | Portable terminal unit, advice system, skin diagnosing evaluating method, skin diagnosing evaluating program, makeup advice providing method and makeup advice providing program | |
WO2018029963A1 (en) | Make-up assistance apparatus and make-up assistance method | |
JP2020526756A (en) | A system for managing hair condition information and how to optimize beauty consultation | |
KR20210017287A (en) | System and method for recommending color of cosmetic product by sharing information with influencer | |
WO2023194466A1 (en) | Method for recommending cosmetic products using a knn algorithm | |
KR102425873B1 (en) | Personal color diagnostic method and system based on machine learning and augmented reality | |
KR102465453B1 (en) | A virtual makeup composition A.I. processing apparatus and a method using it | |
KR20220022433A (en) | System for buying service of cosmetic object and applying selective makeup effect | |
JPWO2022002961A5 (en) | ||
Yan et al. | Exploring the Facial Color Representative Regions Using the Humanae Images | |
Lee et al. | A cognitive knowledge-based system for hair and makeup recommendation based on facial features classification | |
KR20200107486A (en) | Virtual makeup composition processing apparatus | |
Tzou et al. | How AI and AR can help beauty industry | |
US20240135424A1 (en) | Information processing apparatus, information processing method, and program | |
KR102532561B1 (en) | Method for providing consulting data for personal style | |
US20240065420A1 (en) | Interreality cosmetics for a smart cosmetic device system | |
Kamble | Foundation Makeup Shade Recommendation using Computer Vision Based on Skin Tone Recognition |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 23717507 Country of ref document: EP Kind code of ref document: A1 |