EP-4738150-A2 - TECHNIQUES FOR IDENTIFYING SKIN COLOR IN IMAGES HAVING UNCONTROLLED LIGHTING CONDITIONS

EP4738150A2EP 4738150 A2EP4738150 A2EP 4738150A2EP-4738150-A2

Abstract

The present invention concerns a method of training a machine learning model (106) to estimate a skin color (108) of a face in an image. The method comprises: receiving, by a computing device, at least one training image that includes a face of a training subject; receiving tagging information for the image, wherein the tagging information includes ground-truth skin color information; normalizing the training image to create a normalized training image by detecting, centering and zooming the face to a predetermined size; adding the normalized image and tagging information to a training data store (216); training the machine learning model (106) with gradient-descent technique using stored normalized image and tagging information; wherein receiving tagging information further comprises detecting a color reference chart in the image, determining an illuminant color from the chart, adjusting the image color based on the illuminant, and determining the skin color (108) from the adjusted image.

Inventors

ELFAKHRI, CHRISTINE
VALCESHINI, Florent
TRAN, Loic
PERROT, Matthieu
KIPS, Robin
MALHERBE, EMMANUEL

Assignees

L'OREAL
Elfakhri, Christine

Dates

Publication Date: 20260506
Application Date: 20200709

Claims (3)

A method (300) of training a machine learning model (106) to estimate a skin color (108) of a face in an image, the method comprising: receiving, by a computing device, at least one training image that includes a face of a training subject; receiving, by the computing device, tagging information for the at least one training image, wherein the tagging information includes ground truth skin color information for the training subject; normalizing, by the computing device, the at least one training image to create at least one normalized training image, wherein normalizing the at least one training image to create at least one normalized training image includes: detecting, by the computing device, a face in the at least one training image; centering, by the computing device, the face in the at least one training image; and zooming, by the computing device, the at least one training image such that the face is a predetermined size; adding, by the computing device, the at least one normalized training image and the tagging information to a training data store (216); and training, by the computing device using a gradient descent technique, the machine learning model (106) to determine skin colors (108) of faces using at least the at least one normalized training image and the tagging information stored in the training data store (216); wherein receiving tagging information for the at least one training image includes: detecting, by the computing device, a color reference chart in the training image; determining, by the computing device, an illuminant color based on the color reference chart; adjusting, by the computing device, a color of the at least one training image based on the determined illuminant color; and determining, by the computing device, the skin color (108) information from the adjusted color of the at least one training image.
The method (300) of claim 1, wherein receiving at least one training image includes: receiving, by the computing device, a video; and extracting, by the computing device, at least one training image from the video.
The method (300) of any of claim 1 to claim 2, wherein receiving tagging information for the at least one training image includes receiving, by the computing device, skin color (108) information collected from the training subject by a spectrophotometer.

Description

CROSS-REFERENCE TO RELATED APPLICATION This application claims the benefit of U.S. Application No. 16/516,080, filed July 18, 2019; the contents of which are hereby incorporated by reference in their entirety for all purposes. SUMMARY This summary is provided to introduce a selection of concepts in a simplified form that are further described below in the Detailed Description. This summary is not intended to identify key features of the claimed subject matter, nor is it intended to be used as an aid in determining the scope of the claimed subject matter. In some embodiments, a method of training a machine learning model to estimate a skin color of a face in an image is provided. A computing device receives at least one training image that includes a face of a training subject. The computing device receives tagging information for the at least one training image. The computing device adds the at least one training image and the tagging information to a training data store. The computing device trains the machine learning model to determine skin colors of faces using information stored in the training data store. In some embodiments, a method of using one or more machine learning models to estimate a skin color of a face is provided. A computing device receives at least one image that includes a face of a live subject. The computing device processes the at least one image using at least one machine learning model to obtain a determination of a skin color. The computing device presents the skin color. In some embodiments, a system is provided. The system comprises a skin color prediction unit including computational circuitry configured to receive at least one training image that includes a face of a training subject; receive tagging information for the at least one training image; add the at least one training image and the tagging information to a training data store; and train the machine learning model to determine skin colors of faces using the training data set. In some embodiments, a system is provided. The system comprises a skin color prediction unit and a predicted skin color unit. The skin color prediction unit includes computational circuitry configured to generate a pixel-wise prediction score for skin color of a face in an image using one or more convolutional neural network image classifiers. The predicted skin color unit includes computational circuitry configured to generate a virtual display of a predicted skin color of a user responsive to one or more inputs based on the prediction scores for the skin color of the face. DESCRIPTION OF THE DRAWINGS The foregoing aspects and many of the attendant advantages of this invention will become more readily appreciated as the same become better understood by reference to the following detailed description, when taken in conjunction with the accompanying drawings, wherein: FIGURE 1 is a schematic illustration of a non-limiting example embodiment of a system that uses at least one machine learning model to generate an automatic estimation of a skin color according to various aspects of the present disclosure;FIGURE 2 is a block diagram that illustrates non-limiting example embodiments of a mobile computing device and a skin color determination device according to various aspects of the present disclosure;FIGURE 3 is a flowchart that illustrates a non-limiting example embodiment of a method of training a machine learning model to estimate a skin color of a face in an image according to various aspects of the present disclosure;FIGURE 4 is a schematic drawing that illustrates a non-limiting example embodiment of normalization of an image that includes a face according to various aspects of the present disclosure;FIGURE 5 is a flowchart that illustrates a non-limiting example embodiment of a method of using a machine learning model to estimate a skin color of a face in an image according to various aspects of the present disclosure; andFIGURE 6 is a block diagram that illustrates aspects of an exemplary computing device appropriate for use as a computing device of the present disclosure. DETAILED DESCRIPTION Shopping for products online, including via mobile computing devices, is a convenient way for consumers to browse and obtain products. In addition to having large catalogs of items available at the click of a mouse or a tap of a finger, various technologies exist to provide online shoppers with a wide range of recommendations for products that they may be interested in based on other products they are interested in, their purchase history, their reviews of other products, the purchase history and reviews of other shoppers, and so on. However, there are certain types of products for which an in-person experience has been difficult to replace with an online interaction. For example, beauty products such as foundation or other makeup are difficult to browse online, and are difficult to recommend in an automated manner. This is primarily because the charac