US-20260127744-A1 - SYSTEMS AND METHODS FOR EYELID LOCALIZATION

US20260127744A1US 20260127744 A1US20260127744 A1US 20260127744A1US-20260127744-A1

Abstract

Systems and methods for localizing an upper eyelid in an image of a subject are provided. An image of an eye of the subject is obtained in electronic format. The image is inputted into a trained neural network comprising at least 10,000 parameters, thereby obtaining a set of coordinates for an upper eyelid in the image. This obtaining and inputting can be repeated over the course of a non-zero duration thereby obtaining a corresponding set of coordinates for the upper eyelid in each image in a plurality of images. Each corresponding set of coordinates for the upper eyelid from each image in the plurality of images can be used to determine whether the subject is afflicted with a neurological condition.

Inventors

Logan Sean Teder
Eric Fouad Abboud
Ryan Nicholas Fiorini

Assignees

Blinktbi, Inc.

Dates

Publication Date: 20260507
Application Date: 20251230

Claims (20)

1 . A method for localizing an upper eyelid in an image of a subject, the method comprising: (a) obtaining the image of an eye of the subject in electronic format, wherein the image of the eye of the subject comprises a corresponding plurality of pixels and one or more pixel values for each pixel in the corresponding plurality of pixels; and (b) inputting the image into a trained neural network comprising at least 10,000 parameters, wherein: the trained neural network comprises an initial convolutional neural network layer that receives a grey-scaled pixel value for each pixel in the corresponding plurality of pixels as input into the neural network, the initial convolutional neural network layer includes a first activation function, and the initial convolutional neural network layer convolves the corresponding plurality of pixels into more than 10 separate parameters for each pixel in the plurality of pixels, thereby obtaining a set of coordinates for an upper eyelid in the image.
2 . The method of claim 1 , the method further comprising: repeating the obtaining (a) and inputting (b) over the course of a non-zero duration thereby obtaining a corresponding set of coordinates for the upper eyelid in each image in a plurality of images, and using each corresponding set of coordinates for the upper eyelid from each image in the plurality of images to determine whether the subject is afflicted with a neurological condition.
3 . The method of claim 2 , wherein the neurological condition is a result of a traumatic event, a head impact, or a mild traumatic brain injury.
4 . The method of claim 2 , wherein the plurality of images is taken upon stimulation of at least one facial region of the subject using at least one stimulator so as to cause an involuntary blink response in the subject and wherein each image in the plurality of images is an image of the involuntary blink response.
5 . The method of claim 4 , wherein the at least one facial region is selected from the group consisting of the temple, the outer canthus, and the eye.
6 . The method of claim 4 , wherein the stimulator is provided in proximity of the left eye, the right eye, or both.
7 . The method of claim 2 , wherein the plurality of images is taken for the left eye of the subject, the right eye of the subject, or both.
8 . The method of claim 4 , wherein the at least one stimulator is selected from the group consisting of a puff of fluid, a mechanical contact, one or more flashes of light, an electrical current, and a sound.
9 . The method of claim 4 , further comprising: using each corresponding set of coordinates for the upper eyelid in the plurality of images to obtain a first measurement of a characteristic of the involuntary blink response in the subject, wherein the characteristic of the involuntary blink response is selected from the group consisting of individual latency, differential latency, number of oscillations, change in tonic lid position, horizontal lid velocity, vertical lid velocity, time to close, time to open, total blink time, and time under threshold; using each corresponding set of coordinates for the upper eyelid in the plurality of images to obtain a second measurement of the characteristic of the involuntary blink response for the subject, wherein the second measurement represents a baseline condition for the subject; comparing the first measurement and the second measurement; and when the difference between the first measurement and the second measurement exceeds a predetermined threshold value, determining that the subject is afflicted with a neurological condition.
10 - 13 . (canceled)
14 . The method of claim 2 , further comprising: obtaining a corresponding set of coordinates for a lower eyelid in each image in the plurality of images, thereby localizing a lower eyelid in each image in the plurality of images; and using each corresponding set of coordinates for both the upper eyelid and the lower eyelid in the plurality of images to determine whether the subject is afflicted with the neurological condition.
15 - 17 . (canceled)
18 . The method of claim 1 , wherein the trained neural network further comprises a pooling layer that pools the more than 10 separate parameters for each pixel in the plurality of pixels outputted by the initial convolutional neural network layer.
19 . The method of claim 1 , wherein the initial convolutional neural network layer has a stride of two or more.
20 . The method of claim 18 , wherein the trained neural network further comprises a plurality of intermediate blocks including a first intermediate block and a final intermediate block, wherein the first intermediate block takes as input the output of the pooling layer; each intermediate block in the plurality of intermediate blocks other than the first intermediate block and the final intermediate block takes, as input, an output of another intermediate block in the plurality of intermediate blocks and has an output that serves as input to another intermediate block in the plurality of intermediate blocks, and wherein each intermediate block comprises a respective first convolutional layer comprising more than 1000 parameters, wherein the respective convolutional layer has a corresponding activation function.
21 . The method of claim 20 , wherein each intermediate block in the plurality of intermediate blocks comprises: a corresponding second convolutional layer that takes, as input, an output of the respective first convolutional layer, and a merge layer that merges (i) an output of the respective second convolutional layer and (ii) an output of a preceding intermediate block in the plurality of intermediate blocks, and wherein: each intermediate block in the plurality of intermediate blocks has a corresponding input size and a corresponding output size, and, when the corresponding input size of a respective intermediate block differs from the corresponding output size, the respective intermediate block further comprises a corresponding third convolutional layer that receives, as input, the (ii) output of the preceding intermediate block, wherein the corresponding third convolutional layer convolves the (ii) output of the preceding intermediate block prior to the merging (i) and (ii) by the merge layer.
22 - 23 . (canceled)
24 . The method of claim 20 , wherein the final intermediate block takes, as input, an output of another intermediate block in the plurality of intermediate blocks and produces, as output, a flattened data structure comprising a predetermined plurality of values.
25 . The method of claim 24 , wherein the neural network further comprises a regressor block including a first dropout layer, a first linear layer, and a corresponding activation function, wherein the regressor block takes, as input, the flattened data structure comprising the predetermined plurality of values, wherein the first dropout layer removes a first subset of values from the plurality of values in the flattened data structure, based on a first dropout rate, and the first linear layer applies a first linear transformation to the plurality of values in the flattened data structure.
26 - 27 . (canceled)

Description

CROSS REFERENCE TO RELATED APPLICATIONS This application is a continuation of U.S. patent application Ser. No. 17/827,528, filed May 27, 2022, which claims priority to U.S. Provisional Ser. No. 63/194,554, entitled “Systems and Methods for Eyelid Localization,” filed May 28, 2021, and U.S. Provisional Ser. No. 63/275,749, entitled “Systems and Methods for Eyelid Localization,” filed Nov. 4, 2021, the content of each of which is hereby incorporated by reference, in its entirety, for all purposes. TECHNICAL FIELD This specification describes using a neural network to locate an eyelid in an image. BACKGROUND Blinking is an activity that serves physiologic maintenance, protective, and brain restorative functions. With every blink, an ocular lubricant is swept across the eyeball providing moisture and anti-microbial protection to the orbit. Blinking also provides rest and reset for retinal cells that are activated and fade when fixed on an object. Blinks can be classified as voluntary, acquired, passive, spontaneous and reflexive; each with a distinct cause, innervation pathway and temporal profile. Reflexive blinks, also called the blink reflexes, can be elicited by tactile, light, and sound stimulation, and serve to provide a first line of defense to the globe and the brain behind it. Numerous studies have examined the blink reflex in normal populations, as well as the changes that occur in a variety of neurological conditions, such as Parkinson's disease, Huntington's disease, schizophrenia and severe traumatic brain injury. Indeed, the sensitivity of the blink reflex to a variety of insults indicates that it may be a useful tool for assessing neurological function. This is particularly relevant given the current public interest in developing methods to detect concussions, offering a more encompassing field sobriety test, and monitoring patients for early onset of Alzheimer's disease. The blink reflex is a primitive brainstem response to an external stimulus, such as air, visual cues or electrical signals, which can be affected by multiple neurological disorders, including those that affect the dopaminergic circuit that controls the eyelid. Previous studies using electromyography have shown that diffuse axonal injury and exercise result in measurable changes in the blink reflex. As described above, previous studies have also examined the changes that occur to the blink reflex in a variety of neurological conditions, such as Parkinson's disease, Huntington's disease, schizophrenia and severe traumatic brain injury. See, for example, Garner et al., 2018, “Blink reflex parameters in baseline, active, and head-impact Division I athletes,” Cogent Engineering, 5:1429110; doi: 10.1080/23311916.2018.1429110. However, despite the correlation between altered blink reflex parameters and neurological health, quantitative approaches for examining the blink reflex have not been adopted in clinical practice, and qualitative assessments remain standard. In recent decades, considerable technological advancements have been made. For instance, digital image capture technologies have improved significantly over the past two decades and have significantly decreased in cost. The high frame rates currently available allow for measurements in the millisecond range, where image capture occurs at speeds that facilitate the quantification of blink reflexes. Concurrently, processing power has increased, making analysis of the thousands of image frames recorded at high frame rates feasible. High-speed image capture has been successfully employed to record and measure eyelid location during a single, voluntary blink in both healthy subjects and in patients with blepharoptosis. However, no study has used high-speed image capture to study reflexive blinks. Furthermore, methods that use non-invasive measurements and machine learning-based image analysis to measure eyelid location, or to diagnose a variety of neurological conditions, are lacking. See, for example, Tsai et al., 2017, “Development of a Non-Invasive Blink Reflexometer,” IEEE J Transl Eng Health Med; 5:3800204; doi: 10.1109/JTEHM.2017.2782669, which is hereby incorporated by reference. Given the above background, there is a need in the art for systems and methods of processing and analyzing images for eyelid location. SUMMARY Advantageously, the present disclosure provides robust techniques for localizing an eyelid in an image. Moreover, the combination of linear algebra, statistics, model architecture and image analysis used in some embodiments of the present disclosure provides additional model training and image localization power beyond previous model architectures used for image localization. Technical solutions (e.g., computing systems, methods, and non-transitory computer-readable storage mediums) for addressing the above-identified problems with analyzing datasets are provided in the present disclosure. The following presents a summary of the invention in order to provide