CN-121983222-A - Skin care suggestion intelligent recommendation method, system and equipment based on skin state detection

CN121983222ACN 121983222 ACN121983222 ACN 121983222ACN-121983222-A

Abstract

The invention provides an intelligent skin care suggestion recommending method, system and equipment based on skin state detection, belonging to the field of intelligent computation, wherein the method comprises the steps of obtaining a target skin image; the skin care method comprises the steps of detecting the skin state in a target skin image by using a pre-trained skin state detection model to obtain skin state data, determining a skin state conclusion according to the skin state data, obtaining a nursing problem input by a user, determining an answer to the nursing problem by using a question-answering model based on a pre-constructed skin care knowledge base according to the nursing problem, the skin state conclusion and the target skin image, and displaying the skin state conclusion and the answer to the nursing problem, wherein the question-answering model is obtained by training BEiTV by using a training sample set in advance.

Inventors

LI HUANYU
WEI ZIKUN
ZHU LING
LV JUNWEI

Assignees

云南云科特色植物提取实验室有限公司
上海贝芙汀科技有限公司
李寰宇

Dates

Publication Date: 20260505
Application Date: 20231024

Claims (7)

1. The intelligent skin care suggestion recommending method based on skin state detection is characterized by comprising the following steps of: acquiring a target skin image; Detecting the skin state in the target skin image by adopting a pre-trained skin state detection model to obtain skin state data; Determining a skin state conclusion from the skin state data; Acquiring a nursing problem input by a user; Based on a pre-constructed skin care knowledge base, determining answers to the care questions by adopting a question-answer model according to the care questions, the skin state junction and the target skin images, and displaying the answers to the skin state junction and the care questions, wherein the question-answer model is obtained by training BEiT V by adopting a training sample set in advance, and the training sample set comprises a plurality of sample images, skin state conclusions corresponding to the sample images and sample care questions.
2. The intelligent skin care recommendation method based on skin condition detection according to claim 1, wherein the step of acquiring the target skin image comprises the following steps: acquiring a preliminary skin image; detecting the face position in the preliminary skin image; And cutting the face area in the preliminary skin image based on the face position to obtain a target skin image.
3. The intelligent skin care advice recommendation method according to claim 1, wherein the skin state detection model comprises an acne detection model, a acne mark segmentation model, an oil extraction classification model, a skin color classification model, a sensitivity classification model, a pore classification model, a blackhead classification model and a black eye segmentation model; The acne detection model is obtained by training an end-to-end target detector DINO by adopting an acne sample set in advance, wherein the acne sample set comprises a plurality of acne sample images, acne detection frames in the acne sample images and corresponding acne types, and the acne types comprise white heads, papules, pustules, nodules and cysts; The acne mark segmentation model is obtained by training a UNet network in advance by adopting an acne mark sample set, wherein the acne mark sample set comprises a plurality of acne mark sample images and acne mark outlines in each acne mark sample image; The oil outlet classification model is obtained by training a residual neural network ResNet by adopting an oil outlet sample set in advance, wherein the oil outlet sample set comprises a plurality of oil outlet sample images and oil outlet degrees of the oil outlet sample images; the skin color classification model is obtained by training a residual neural network ResNet by adopting a skin color sample set in advance, wherein the skin color sample set comprises skin color sample images and skin color types of each skin color sample image; the sensitive classification model is obtained by training a residual neural network ResNet by adopting a sensitive sample set in advance, wherein the sensitive sample set comprises a plurality of sensitive sample images and sensitive types of the sensitive sample images; the pore classification model is obtained by training a residual neural network ResNet by adopting a pore sample set in advance, wherein the pore sample set comprises a plurality of pore sample images and pore types of the pore sample images; the black head classification model is obtained by training a residual neural network ResNet by adopting a black head sample set in advance, wherein the black head sample set comprises a plurality of black head sample images and black head types of the black head sample images; The black eye segmentation model is obtained by training a Swin Transformer by adopting a black eye sample set in advance, wherein the black eye sample set comprises a plurality of black eye sample images, black eye detection frames in each black eye sample image and corresponding black eye types; The skin state data comprise an acne detection frame, an acne type, a acne mark segmentation result, an oil outlet degree, a skin color type, a sensitivity type, a pore type, a blackhead type, a blackeye segmentation result and a blackeye type.
4. The intelligent skin care advice recommendation method according to claim 3, wherein the skin condition conclusion comprises white head number, pimple number, pustule number, nodule number, cyst number, acne mark ratio, oil extraction degree, skin color type, sensitivity type, pore type and blackhead type; determining a skin state conclusion according to the skin state data, which specifically comprises the following steps: determining the number of white heads, the number of papules, the number of pustules, the number of nodules and the number of cysts in the target skin image according to the acne detection frames in the target skin image and the acne types of the acne detection frames; and determining the acne mark duty ratio in the target skin image according to the area of the acne mark detection frame in the target skin image and the area of the target skin image.
5. The intelligent skin care advice recommendation method based on skin condition detection according to claim 1, wherein based on a pre-constructed skin care knowledge base, according to the care questions, the skin condition conclusions and the target skin images, an answer to the care questions is determined by adopting a question-answer model, and specifically comprising: encoding the nursing problem, the skin state conclusion and the target skin image through a question and answer model respectively to obtain a problem feature vector, a state feature vector and a graphic feature vector; searching a pre-constructed skin care knowledge base for knowledge with similarity of the question feature vector, the state feature vector and the graphic feature vector being larger than a set threshold value through a question-answering model, and taking the knowledge as an answer to the care question.
6. The intelligent skin care advice recommending system based on skin state detection is characterized by comprising: The image acquisition module is used for acquiring a target skin image; The state detection module is connected with the image acquisition module and is used for detecting the skin state in the target skin image by adopting a pre-trained skin state detection model to obtain skin state data; the conclusion determining module is connected with the state detecting module and is used for determining a skin state conclusion according to the skin state data; The problem acquisition module is used for acquiring nursing problems input by a user; The answer determining module is respectively connected with the image obtaining module, the conclusion determining module and the problem obtaining module and is used for determining answers of the nursing problems by adopting a question-answer model based on a pre-built skin nursing knowledge base according to the nursing problems, the skin state conclusions and the target skin images and displaying the answers of the skin state conclusions and the nursing problems, wherein the question-answer model is obtained by training BEiT V through a training sample set in advance, and the training sample set comprises a plurality of sample images, skin state conclusions corresponding to the sample images and sample nursing problems.
7. An electronic device comprising a memory for storing a computer program and a processor that runs the computer program to cause the electronic device to perform the skin care advice intelligent recommendation method based on skin condition detection of any one of claims 1 to 5.

Description

Skin care suggestion intelligent recommendation method, system and equipment based on skin state detection Technical Field The invention relates to the field of intelligent computation, in particular to an intelligent skin care suggestion recommendation method, system and equipment based on skin state detection. Background Various skin problems of the face are easy to influence and puzzles the daily life of people, but the skin detection of a professional medical institution is time-consuming and labor-consuming, and the price is high. The relevant answers on the web page are confetti and eight doors, are difficult to distinguish, and the current skin care products are numerous in name, so that the user is difficult to select the skin care scheme which is most suitable for the user. Disclosure of Invention The invention aims to provide an intelligent recommending method, system and equipment for skin care advice based on skin state detection, which can improve the accuracy of skin state detection and can personally recommend the skin care advice. In order to achieve the above object, the present invention provides the following solutions: An intelligent skin care advice recommendation method based on skin state detection, comprising: acquiring a target skin image; Detecting the skin state in the target skin image by adopting a pre-trained skin state detection model to obtain skin state data; Determining a skin state conclusion from the skin state data; Acquiring a nursing problem input by a user; Based on a pre-constructed skin care knowledge base, determining answers to the care questions by adopting a question-answer model according to the care questions, the skin state junction and the target skin images, and displaying the answers to the skin state junction and the care questions, wherein the question-answer model is obtained by training BEiT V by adopting a training sample set in advance, and the training sample set comprises a plurality of sample images, skin state conclusions corresponding to the sample images and sample care questions. Optionally, the skin state detection model comprises an acne detection model, a acne mark segmentation model, an oil extraction classification model, a skin color classification model, a sensitivity classification model, a pore classification model, a blackhead classification model and a black eye segmentation model; The acne detection model is obtained by training an end-to-end target detector DINO by adopting an acne sample set in advance, wherein the acne sample set comprises a plurality of acne sample images, acne detection frames in the acne sample images and corresponding acne types, and the acne types comprise white heads, papules, pustules, nodules and cysts; The acne mark segmentation model is obtained by training a UNet network in advance by adopting an acne mark sample set, wherein the acne mark sample set comprises a plurality of acne mark sample images and acne mark outlines in each acne mark sample image; The oil outlet classification model is obtained by training a residual neural network ResNet by adopting an oil outlet sample set in advance, wherein the oil outlet sample set comprises a plurality of oil outlet sample images and oil outlet degrees of the oil outlet sample images; the skin color classification model is obtained by training a residual neural network ResNet by adopting a skin color sample set in advance, wherein the skin color sample set comprises skin color sample images and skin color types of each skin color sample image; the sensitive classification model is obtained by training a residual neural network ResNet by adopting a sensitive sample set in advance, wherein the sensitive sample set comprises a plurality of sensitive sample images and sensitive types of the sensitive sample images; the pore classification model is obtained by training a residual neural network ResNet by adopting a pore sample set in advance, wherein the pore sample set comprises a plurality of pore sample images and pore types of the pore sample images; the black head classification model is obtained by training a residual neural network ResNet by adopting a black head sample set in advance, wherein the black head sample set comprises a plurality of black head sample images and black head types of the black head sample images; The black eye segmentation model is obtained by training a Swin Transformer by adopting a black eye sample set in advance, wherein the black eye sample set comprises a plurality of black eye sample images, black eye detection frames in each black eye sample image and corresponding black eye types; The skin state data comprise an acne detection frame, an acne type, a acne mark segmentation result, an oil outlet degree, a skin color type, a sensitivity type, a pore type, a blackhead type, a blackeye segmentation result and a blackeye type. In order to achieve the above purpose, the present invention also provides the following solutions: An