Search

EP-3518245-B1 - IMAGE GENERATION FROM A MEDICAL TEXT REPORT

EP3518245B1EP 3518245 B1EP3518245 B1EP 3518245B1EP-3518245-B1

Inventors

  • SHARMA, PUNEET
  • NEUMANN, Dominik
  • SOENIUS, Ruth, J.
  • HARTANG, Ulrich

Dates

Publication Date
20260506
Application Date
20180129

Claims (11)

  1. A method of generating a first image (318, 418, 518, 618) from a medical text report (314, 514), the method comprising, by an apparatus (616): acquiring a medical text report (314, 514, 614) comprising one or more natural language statements; analysing the medical text report (314, 514, 614) using a computer-implemented analysis process, to: determine for each natural language statement whether the statement satisfies a predetermined criterion with respect to a first medical finding; and determine, for each natural language statement determined as satisfying the predetermined criterion with respect to the first medical finding, one or more first parameters associated with the first medical finding, wherein the computer-implemented analysis process comprises a text classification based algorithm, and wherein the predetermined criterion comprises that the statement is classified by the text classification algorithm as positively indicating the presence of the first medical finding; and responsive to a determination that a said statement satisfies the predetermined criterion: outputting structured data from the text classification algorithm to an image generation algorithm, the structured data comprising information indicating that the text classification algorithm determined that the medical text report (314, 514, 614) included one or more natural language statements satisfying the predetermined criterion with respect to the first medical finding, and further comprising information indicating one or more of the first parameters associated with the one or more natural language statements satisfying the predetermined criterion; and generating, by the image generation algorithm and based on the structured data, the first image (318, 418, 518, 618), wherein generating the first image (318, 418, 518, 618) comprises: determining, from the structured data, a relevant image template (320, 420, 520); retrieving the relevant image template (320, 420, 520) from a storage; and using the information indicating one or more of the first parameters to overlay an image (324, 326, 328, 424, 426, 428, 524, 526, 522, 528) representing the first medical finding onto a section of an image of the relevant image template (320, 420, 520) so as to indicate the first medical finding on the relevant image template, thereby to generate the first image (318, 418, 518, 618), wherein one or more of the first parameters comprises a location parameter relating to a location of the first medical finding, wherein the information indicating the one or more of the first parameters comprises information indicating the location parameter, and wherein generating the first image comprises: determining, based on the information indicating the location parameter, the section of the image of the relevant image template (320, 420, 520) onto which the image (324, 326, 328, 424, 426, 428, 524, 526, 522, 528) representing the first medical finding is to be overlaid; and overlaying the image (324, 326, 328, 424, 426, 428, 524, 526, 522, 528) representing the medical finding onto the determined section of the image of the relevant image template (320, 420, 520).
  2. The method according to claim 1, wherein the first parameters comprise one or both of a type parameter relating to a type of the first medical finding and an extent parameter relating to an extent of the first medical finding; and the overlaying the image representing the first medical finding onto the image of the image template is based at least in part on one or both of the type parameter and the extent parameter.
  3. The method according to claim 1 or claim 2, wherein the medical text report (314, 514, 614) is a radiology report (314, 514, 614).
  4. The method according to any one of claim 1 to claim 3, wherein the image template (320, 420) comprises a pictogram (320, 420) representing a patient anatomy to which the medical text report is directed; and wherein the image (324, 326, 328, 424, 426, 428,) representing the first medical finding is added to the pictogram (320, 420) so as to indicate the first medical finding on the pictogram (320, 420).
  5. The method according to any one of claim 1 to claim 4, wherein the computer-implemented analysis process comprises a machine learning based algorithm.
  6. The method according to claim 5, wherein the method comprises training, by the apparatus (616), the machine learning based algorithm based on a database of textual statements (252), each statement being annotated with a ground-truth label (254, 256, 258) corresponding to the first medical finding.
  7. The method according to claim 6, wherein the training comprises: generating, by the apparatus (616), based on a plurality of medical text reports, one or more word-embeddings for one or more words of the plurality of medical text reports; and determining, by the apparatus (616), for each of the textual statements, one or more said word embeddings representative of the textual statement (252); and wherein the training is based on the word-embeddings representative of each textual statement (252).
  8. The method according to claim 6 or claim 7, wherein the training is implemented using a Long-Short-Term-Memory based Recurrent Neural Network architecture.
  9. The method according to any one of claim 1 to claim 8, wherein the method comprises: generating, by the apparatus (616), a medical report (628, 630, 632), the medical report (628, 630, 632) comprising the first image (318, 418, 518, 618).
  10. Apparatus (616) for generating a first image from a medical text report (314, 514, 614), the apparatus (616) comprising a processor (616a) and a memory (616b), the memory (616b) storing a program which when executed by the processor (616a) causes the apparatus (616) to perform the method according to any one of claim 1 to claim 9.
  11. A computer program comprising instructions which when executed on a computer (616) cause the computer (616) to perform the method according to any one of claim 1 to claim 9.

Description

Technical Field The present invention relates to generating images, and more specifically to generating an image from a medical text report, such as a radiology report. Background Medical text reports, for example radiology reports, are used in the medical field to communicate information between medical professionals. For example, a patient's physician may refer a patient for certain medical imaging, for example Computed Tomography Angiography (CTA). A radiographer then carries out the medical imaging of the patient and a radiologist then reads or interprets the resulting image to produce a text-based radiology report of their findings. A medical finding may be defined as an observation made of the patient, for example by a doctor, for example as a result of examination, imaging, or other tests of the patient. The radiology report is then provided to the referring physician. The referring physician may then make a diagnosis of a condition of the patient based on the radiology report. The radiology report is often the main form of communication between the radiologist and a referring physician. The reports include narrative text dictated by the radiologist, and, in some cases, are structured according to a predefined format. While preparing the report, the radiologist balances two opposing aspects: to be as comprehensive as possible, and still ensure that all the key findings are communicated succinctly to the referring physician. This can result in reports that are too verbose, with some key findings (positive or negative) often hidden or difficult to appreciate among the narrative text. This can have severe consequences such as misdiagnosis or missed findings. Increasingly, medical text reports, such as radiology reports, have another audience in addition to the referring physician, namely the patient. For efficient communication with the patient, it is important that the reports are easily understandable to a non-medical professional (e.g. the patient). However, radiology reports often cannot fulfil these competing demands and constraints. An existing attempt to address this is for the radiologist to adopt guidelines, which can result in structured reports having a predefined format. However, the structured reports still suffer from the drawbacks that it is difficult to extract the most relevant information both for medical professionals and non-medical professionals, and potentially highly important secondary findings may still be difficult to appreciate amongst the narrative text, possibly resulting in misdiagnosis. Presenting medical reports using images, illustrating relevant medical information, for example, would mitigate the above issues. It is therefore desirable to provide an efficient method of generating such images. US 2012/035963 A1 relates to a system that automatically retrieves report templates based on diagnostic information. US 2007/237377 A1 relates to a method for supporting creation of a medical report. Summary According to a first aspect of the present invention, there is provided a method of generating a first image from a medical text report, the method comprising, by an apparatus: acquiring a medical text report comprising one or more natural language statements; analysing the medical text report using a computer-implemented analysis process, to: determine for each natural language statement whether the statement satisfies a predetermined criterion with respect to a first medical finding; and determine, for each natural language statement determined as satisfying the predetermined criterion with respect to the first medical finding, one or more first parameters associated with the first medical finding, wherein the computer-implemented analysis process comprises a text classification based algorithm, and wherein the predetermined criterion comprises that the statement is classified by the text classification algorithm as positively indicating the presence of the first medical finding; and responsive to a determination that a said statement satisfies the predetermined criterion: outputting structured data from the text classification algorithm to an image generation algorithm, the structured data comprising information indicating that the text classification algorithm determined that the medical text report included one or more natural language statements satisfying the predetermined criterion with respect to the first medical finding, and further comprising information indicating one or more of the first parameters associated with the one or more natural language statements satisfying the predetermined criterion; and generating, by the image generation algorithm and based on the structured data, the first image, wherein generating the first image comprises: determining, from the structured data, a relevant image template; retrieving the relevant image template from a storage; and using the information indicating one or more of the first parameters to overlay an image representing