Search

EP-4150504-B1 - RETINAL IMAGE PROCESSING

EP4150504B1EP 4150504 B1EP4150504 B1EP 4150504B1EP-4150504-B1

Inventors

  • Wakeford, Peter Robert
  • PELLEGRINI, ENRICO

Dates

Publication Date
20260506
Application Date
20200514

Claims (15)

  1. An ocular imaging system (10) for acquiring a retinal image (510) of at least a portion of a retina (20) of an eye (30), comprising: an image acquisition module (40) configured to acquire the retinal image (510); a landmark location prediction module (50) configured to use a machine learning algorithm to predict, as predicted locations of landmark features in the retinal image (510), a first location (L 1 ) of a first landmark feature in the retinal image (510) and a second location (L 2 ) of a second landmark feature in the retinal image (510), wherein the first landmark feature is a fovea of the eye (30) and the second landmark feature is an optic disc of the eye (30); and an apparatus (60) for alerting a user of the ocular imaging system (10) to an unreliability in at least one of the predicted locations of the landmark features, the apparatus (60) comprising: a receiver module (61) configured to receive the predicted locations (L 1 , L 2 ) of the landmark features; a distance metric evaluation module (62) configured to use the predicted locations (L 1 , L 2 ) of the landmark features to evaluate a distance metric which is indicative of a distance between the first landmark feature and the second landmark feature; an outlier detector module (63) configured to determine, using data indicative of a probability distribution of a distance ( d Fov-OD ) between the first landmark feature and the second landmark feature obtained from measurements of the distance in a set of retinal images different from the retinal image (510), an indication of whether the evaluated distance metric lies outside a predetermined interval ( I ) of the probability distribution which includes a peak of the probability distribution; and an alert generator module (64) configured to generate an alert indicating that at least one of the predicted locations of the landmark features in the retinal image (510) is unreliable in a case where the determined indication indicates that the evaluated distance metric lies outside the predetermined interval ( I ) of the probability distribution.
  2. The ocular imaging system (10) according to claim 1, wherein the distance metric comprises one of a Euclidean distance and a Manhattan distance between the first landmark feature and the second landmark feature, and the outlier detector module (63) is configured to use the data indicative of a probability distribution of the one of the Euclidean distance and the Manhattan distance between the first landmark feature and the second landmark feature obtained from measurements of the one of the Euclidean distance and the Manhattan distance in a set of retinal images different from the retinal image (510) to determine the indication of whether the evaluated distance metric lies outside the predetermined interval ( I ) of the probability distribution.
  3. The ocular imaging system (10) according to claim 1 or claim 2, wherein the distance metric comprises a weighted distance between the first landmark feature and the second landmark feature, the weighted distance being a product of the distance between the first landmark feature and the second landmark feature and a weighting factor ( w 1 ) whose value increases with increasing absolute values of a difference between the distance and a value of the distance corresponding to the peak of the probability distribution.
  4. The ocular imaging system (10) according to any of claims 1 to 3, wherein the distance metric evaluation module (62) is further configured to use the predicted first location (L 1 ) to evaluate a second distance metric which is indicative of a second distance being a distance between the predicted first location (L 1 ) and a reference location (L ref ), the outlier detector module (63) is further configured to determine, using data indicative of a second probability distribution of a distance ( d fov-ref ) between the first landmark feature and the reference location (L ref ), which second probability distribution is based on measurements of the distance ( d fov-ref ) between the first landmark feature and the reference location (L ref ) in a set of retinal images different from the retinal image (510), a second indication being an indication of whether the evaluated second distance metric lies outside a predetermined interval ( I ') of the second probability distribution which includes a peak of the second probability distribution, and the alert generator module (64) is configured to generate, as the alert, an alert indicating that the predicted first location in the retinal image (510) is unreliable in a case where the determined second indication indicates that the evaluated second distance metric does not lie within the predetermined interval ( I ) of the second probability distribution.
  5. The ocular imaging system (10) according to any preceding claim, wherein the landmark location prediction module (50) is further configured to use the machine learning algorithm to classify the retinal image (510) as belonging to one of a plurality of predetermined different classes, based on the predicted locations of the landmark features in the retinal image (510), and the alert generator module (64) is further configured to generate, as the alert, an alert further indicating that the classification of the retinal image (510) by the machine learning algorithm is unreliable.
  6. The ocular imaging system (10) according to claim 5, wherein the landmark location prediction module (50) is configured to classify the retinal image (510) as belonging the one of the plurality of predetermined different classes by classifying the retinal image (510) as being a retinal image (510) of either a left eye or a right eye of a subject.
  7. An apparatus (60) for processing, as predicted locations of landmark features in a retinal image (510) of at least a portion of a retina (20) of an eye (30), a first location (L 1 ) of a first landmark feature in the retinal image (510) and a second location (L 2 ) of a second landmark feature in the retinal image (510) that have been predicted by a machine learning algorithm, to alert a user of the apparatus (60) to an unreliability in at least one of the predicted locations of the landmark features, wherein the first landmark feature is a fovea of the eye (30) and the second landmark feature is an optic disc of the eye (30), the apparatus (60) comprising: a receiver module (61) configured to receive the predicted locations (L 1 , L 2 ) of the landmark features; a distance metric evaluation module (62) configured to use the predicted locations (L 1 , L 2 ) of the landmark features to evaluate a distance metric which is indicative of a distance between the first landmark feature and the second landmark feature; an outlier detector module (63) configured to determine, using data indicative of a probability distribution of a distance between the first landmark feature and the second landmark feature obtained from measurements of the distance in a set of retinal images different from the retinal image (510), an indication of whether the evaluated distance metric lies outside a predetermined interval ( I ) of the probability distribution which includes a peak of the probability distribution; and an alert generator module (64) configured to generate an alert indicating that at least one of the predicted locations of the landmark features in the retinal image (510) is unreliable in a case where the determined indication indicates that the evaluated distance metric lies outside the predetermined interval ( I ) of the probability distribution.
  8. A method of processing, as predicted locations of landmark features in a retinal image (510) of at least a portion of a retina (20) of an eye (30), a first location (L 1 ) of a first landmark feature in the retinal image (510) and a second location (L 2 ) of a second landmark feature in the retinal image (510) that have been predicted by a machine learning algorithm, to alert a user to an unreliability in at least one of the predicted locations of the landmark features, wherein the first landmark feature is a fovea of the eye (30) and the second landmark feature is an optic disc of the eye (30), the method comprising: receiving (S10) the predicted locations (L 1 , L 2 ) of the landmark features; using (S20) the predicted locations (L 1 , L 2 ) of the landmark features to evaluate a distance metric which is indicative of a distance between the first landmark feature and the second landmark feature; determining (S30), using data indicative of a probability distribution of a distance ( d Fov-OD ) between the first landmark feature and the second landmark feature obtained from measurements of the distance ( d Fov-OD ) between the first landmark feature and the second landmark feature in retinal images different from the retinal image (510), an indication of whether the evaluated distance metric lies outside a predetermined interval ( I ) of the probability distribution which includes a peak of the probability distribution; and generating (S40) an alert indicating that the at least one of the predicted locations of the landmark features in the retinal image (510) is unreliable, in a case where the determined indication indicates that the evaluated distance metric lies outside the predetermined interval ( I ) of the probability distribution.
  9. The method according to claim 8, wherein the distance metric comprises one of a Euclidean distance and a Manhattan distance between the first landmark feature and the second landmark feature, and data indicative of a probability distribution of the one of the Euclidean distance and the Manhattan distance the between the first landmark feature and the second landmark feature obtained from measurements of the one of the Euclidean distance and the Manhattan distance in a set of retinal images different from the retinal image (510), is used to determine the indication of whether the evaluated distance metric lies outside the predetermined interval ( I ) of the probability distribution.
  10. The method according to claim 8 or claim 9, wherein the distance metric comprises a weighted distance between the first landmark feature and the second landmark feature, the weighted distance being a product of the distance between the first landmark feature and the second landmark feature and a weighting factor ( w 1 ) whose value increases with increasing absolute values of a difference between the distance between the first landmark feature and the second landmark feature and a value of the distance corresponding to the peak of the probability distribution.
  11. The method according to any of claims 8 to 10, further comprising: using the predicted first location (L 1 ) to evaluate a second distance metric which is indicative of a second distance between the predicted first location (L 1 ) and a reference location (L ref ) of the first landmark feature; determining, using data indicative of a second probability distribution of a distance ( d fov-ref ) between the first landmark feature and the reference location (L ref ), which second probability distribution is based on measurements of the distance d ( fov-ref ) between the first landmark feature and the reference location (L ref ) in retinal images different from the retinal image (510), a second indication being an indication of whether the evaluated second distance metric lies outside a predetermined interval ( I ') of the second probability distribution which includes a peak of the second probability distribution; and generating, as the alert, an alert indicating that the first predicted location in the retinal image (510) is unreliable in a case where the determined second indication indicates that the evaluated second distance metric lies outside the predetermined interval ( I ') of the second probability distribution.
  12. The method according to any of claims 8 to 11, further comprising: using the machine learning algorithm to predict the first location (L 1 ) of the first landmark feature in the retinal image (510) and the second location (L 2 ) of the second landmark feature in the retinal image (510).
  13. The method according to claim 12, wherein the machine learning algorithm further classifies the retinal image (510) as belonging to one of a plurality of predetermined different classes, based on the predicted locations of the landmark features in the retinal image (510), and the method comprises generating, as the alert, an alert further indicating that the classification of the retinal image (510) by the machine learning algorithm is unreliable.
  14. The method according to claim 13, wherein classifying the retinal image (510) as belonging the one of the plurality of predetermined different classes comprises classifying the retinal image (510) as being a retinal image (510) of either a left eye or a right eye of a subject.
  15. A computer program (290) comprising computer program instructions which, when executed by a computer, cause the computer to perform a method according to at least one of claims 8 to 14.

Description

[Technical Field] Example aspects herein generally relate to the field of ocular image data processing systems and, more particularly, to techniques for processing retinal images using machine learning algorithms. [Background] A variety of ocular imaging systems, such as scanning laser ophthalmoscopes and fundus cameras, are commonly used to acquire images of the retina of a subject's eye. The acquired retinal images may be inspected by an ophthalmologist or other medical professional to assess the health of the retina. The acquired retinal images may also be processed automatically by image processing software for a variety of purposes. For example, machine learning algorithms, such as convolutional neural networks (CNN), can be used to carry out ocular image classification (for example, to classify an image as belonging to a first class comprising images of a left eye, or to a second class comprising images of a right eye), using locations of retinal landmarks (e.g. optic disc or fovea) that have been predicted by the machine learning algorithm on the basis of the information in the retinal image. Such machine learning algorithms are typically trained on a dataset of ocular images and then used to perform predictions on new images. Further background is provided in the following documents. The article titled "Fast detection of the optic disc and fovea in color fundus photographs" by M. Niemeijer et al., Medical Image Analysis (Oxford University Press), Vol. 13, No. 6, pages 859-870 (1 December 2009), discloses a fully automated, fast method to detect the fovea and the optic disc in digital color photographs of the retina. The method makes few assumptions about the location of both structures in the image. The problem of localizing structures in a retinal image is defined as a regression problem. A kNN regressor is utilized to predict the distance in pixels in the image to the object of interest at any given location in the image based on a set of features measured at that location. The method combines cues measured directly in the image with cues derived from a segmentation of the retinal vasculature. A distance prediction is made for a limited number of image locations and the point with the lowest predicted distance to the optic disc is selected as the optic disc center. Based on this location the search area for the fovea is defined. The location with the lowest predicted distance to the fovea within the foveal search area is selected as the fovea location. The method is trained with 500 images for which the optic disc and fovea locations are known. An extensive evaluation was done on 500 images from a diabetic retinopathy screening program and 100 specially selected images containing gross abnormalities. The method found the optic disc in 99.4% and the fovea in 96.8% of regular screening images and for the images with abnormalities these numbers were 93.0% and 89.0% respectively. The article titled "Optic Disc - Fovea Distance, Axial Length and Parapapillary Zones. The Beijing Eye Study 2011" by R. A. Jonas et al., PLOS ONE, Vol. 10, No. 9 page e0138701 (21 September 2015), presents measurements of the distance between the optic disc center and the fovea (DFD) based on fundus photographs obtained in the population-based cross-sectional Beijing Eye Study 2011, which included 3468 individuals aged 50+ years. The DFD (mean: 4.76mm) was found to increase with longer axial length, larger parapapillary alpha zone and parapapillary beta/gamma zone, and larger disc area. [Summary] The present invention provides an ocular imaging system according to Claim 1, an apparatus according to Claim 7, a method of processing according to Claim 8, and a computer program according to Claim 15. [Brief Description of the Drawings] Example embodiments of the disclosure will now be explained in detail, by way of non-limiting example only, with reference to the accompanying figures described below. Like reference numerals appearing in different ones of the figures can denote identical or functionally similar elements, unless indicated otherwise. Fig. 1 is a schematic illustration of an ocular imaging system comprising an apparatus for alerting a user to an unreliability in at least one predicted location of a landmark feature in a retinal image according to a first example embodiment herein.Fig. 2 is a block diagram illustrating an example implementation of the apparatus of the first example embodiment in programmable signal processing hardware.Fig. 3 is a flow diagram illustrating a computer-implemented method of alerting a user to an unreliability in at least one predicted location of a landmark feature in a retinal image in accordance with the first example embodiment.Fig. 4 illustrates a processing by a machine learning algorithm of an acquired retinal image, which shows a fovea as a first landmark feature of the retina and an optic disc as a second landmark feature of the retina, to predict the location of the first landm