EP-4742152-A2 - SYSTEM AND METHODS FOR USING ARTIFICIAL INTELLIGENCE FOR SKIN CONDITION DIAGNOSIS AND TREATMENT OPTIONS

EP4742152A2EP 4742152 A2EP4742152 A2EP 4742152A2EP-4742152-A2

Abstract

Methods, systems, and storage media for determining a numerical classification of human skin color, determining one or more characteristics of skin based on images, and determining a personalized treatment plan for one or more skin conditions or issues are disclosed. The system can receive images of skin and access user profile information, such as biometric information, medical record information, and other clinically relevant information. The system can determine a classification of a skin color of the user using the image and the biometric information by providing the image and the biometric information as input to a skin color classifier. Using the skin color, the system can determine one or more characteristics of the skin in the image and, if needed, determine and provide at least one personalized treatment plan to a computing device of a user.

Inventors

MACLELLAN, Reid
ROOSSIN, Paul

Assignees

Cortina Health, Inc.

Dates

Publication Date: 20260513
Application Date: 20210810

Claims (15)

A system for analyzing skin images to determine a personalized treatment plan, comprising: a data processing system comprising one or more processors and a memory, the data processing system configured to: receive an image captured by a camera, the image depicting a diseased portion of skin of a user; access a medical record associated with the user to identify a skin characteristics of the diseased portion of skin represented in the image; determine a plurality of treatment plans for the diseased portion of skin based on the skin characteristics and the image; select a first treatment plan of the plurality of treatment plans based on the medical record; and provide the first treatment plan to a computing device of the user.
The system of claim 1, wherein to access the medical record, the data processing system is further configured to: receive, from a medical database, a set of treatment preferences; and select the first treatment plan further based on the set of treatment preferences.
The system of claim 1, wherein to provide the first treatment plan, the data processing system is further configured to store, in a medical database, the first treatment plan in association with the medical record of the user.
The system of claim 1, wherein the data processing system is further configured to: receive a skin diagnosis from the medical record; and determine the plurality of treatment plans further based on at least the skin diagnosis.
The system of claim 1, wherein the data processing system is further configured to select the first treatment plan further based on at least one of: a frequency of prescription of the first treatment plan, outcome data for the first treatment plan, or a historical diagnosis of the user.
The system of claim 1, wherein the data processing system is further configured to: receive a second image of the diseased portion of skin of the user; determine a treatment progress of the diseased portion of skin of the user based on the image and the second image; and select a second treatment of the plurality of treatment plans based on the medical record and the treatment progress.
The system of claim 6, wherein to determine the treatment progress of the diseased portion of skin of the user, the data processing system is further configured to determine a change in characteristics of the diseased portion of skin between the image and the second image.
The system of claim 6, wherein the data processing system is further configured to transmit the treatment progress of the diseased portion of skin to the computing device of the user.
The system of claim 6, wherein the data processing system is further configured to store the treatment progress of the diseased portion of skin in the medical record of the user.
The system of claim 1, wherein the data processing system is further configured to: provide an application to the computing device that, when executed, presents a graphical user interface to the user on the computing device; and transmit the first treatment plan to the application executing on the computing device.
A system for identifying one or more characteristics of skin, comprising: a data processing system comprising one or more processors and a memory, the data processing system configured to: receive an image captured by a camera, the image depicting a portion of skin of a user; determine, based on the image, a plurality of probability scores that each correspond to a respective one of a plurality of skin characteristics; select a skin characteristic of the plurality of skin characteristics based on the plurality of probability scores; and provide the skin characteristic to a computing device of the user.
The system of claim 11, wherein to provide the skin characteristic, the data processing system is further configured to store the skin characteristic in a medical data record associated with the user.
The system of claim 11, wherein the data processing system is further configured to: identify, based on the image and the skin characteristic, a treatment option to treat a skin disease corresponding to the skin characteristic; and transmit the treatment option to the computing device of the user.
The system of claim 11, wherein the data processing system is further configured to: access, from a medical database, a medical record associated with the user; and determine the plurality of probability scores further based on the medical record.
The system of claim 11, wherein the data processing system is further configured to: receive a second image of the portion of skin at a later time; determine second skin characteristics using the second image; and compare the second skin characteristics with the skin characteristic to determine a change in characteristics of the portion of skin.

Description

CROSS-REFERENCES TO RELATED APPLICATIONS This application claims the benefit of and priority to United States Provisional Patent Application No. 63/064,061, filed August 11, 2020, the contents of which is incorporated herein by reference in its entirety. BACKGROUND Specialists, such as dermatologists, specialize in analyzing, diagnosing, and providing treatment for diseases of the skin. However, in certain areas there may be limited access to specialists that can provide consistent treatment to common skin ailments. Instead, individuals often only have access to a primary care physician who may not be qualified to accurately diagnose and treat various skin ailments. It is particularly critical to diagnose certain skin conditions, such as cancer, as early as possible to provide the greatest probability of positive treatment outcomes. SUMMARY It would be advantageous for a system to automatically analyze, detect, and diagnose issues or diseases of the skin without the immediate need for a specialist. The systems and methods of this technical disclosure provide an artificial-intelligence (AI) based solution to analyze, diagnose, and provide treatment plans for various skin conditions. This technical solution can provide a cloud-based service for the analysis of skin images of a user or patient to determine the color, characteristics, and other information about the skin of the user. This technical solution can access or retrieve the medical records or other user records through encrypted and secure channels, and use this information to provide more accurate analysis, diagnoses, and individualized treatment plans. At least one aspect of this technical solution is directed to determining a skin color classification for a user. Other implementations rely on manual processes (e.g., human analysis and classification) to determine the skin color or tone of a user. This technical solution utilizes specially trained artificial intelligence models to accurately and automatically determine the skin color of a user. A user can provide images of healthy regions of their skin, for example by taking pictures using a camera of a smartphone. The user can also provide additional information, in the form of medical records, biometric information, or other information relating the color or tone of the skin of the user. This information can be collected by an application executing on a computing device of the user. Using the images and provided information, this technical solution can utilize a trained skin color classification model to output a value, such as a Fitzpatrick skin score, that is representative of the skin color of the user. The technical solution can subsequently store the skin score with the information provided by the user, and provide the skin score to one or more computing devices associated with healthcare professionals or a computing device (e.g., the computing device) of the user. At least one other aspect of this technical solution is directed to analyzing images of diseased skin and automatically determining an appropriate diagnosis. Using a computing device or other computing device, a user can provide one or more images of a portion of skin that may be diseased. The user may also provide additional information related to medical history, biometric information, or information about the portion of skin that is to be analyzed (e.g., levels of pain, etc.). This information can be provided, for example, via a computing device such as a smartphone equipped with a camera. This technical solution can receive the user-provided images and information, and access one or more medical records of the user to gather additional information to make an accurate diagnosis. Other skin characteristics, such as a skin color value or classification (e.g., Fitzpatrick skin score, etc.) may be used for the identification and diagnosis of one or more skin diseases. This technical solution can provide the information received from the user and the medical records as an input to skin characteristics classification model, which can provide one or more skin characteristic probability values as an output. The technical solution can subsequently select one or more skin characteristics associated with the probability values (e.g., when one or more of the probability values satisfy or exceed a threshold, etc.). Upon selecting the one or more skin characteristics, the technical solution can store the skin characteristics with data records of the user and provide the characteristics to a computing device of a healthcare professional or the computing device of the user (e.g., a computing device of the user, etc.). At least one other aspect of this technical solution is directed to analyzing images of diseased skin to automatically determine and update a treatment plan for the skin issue. Other implementations rely on trained specialists to manually identify and select appropriate treatment plans for each user. This technical solution offers