EP-4736180-A1 - IMPROVED PICTURE ARCHIVING COMMUNICATION SYSTEM FOR VISUALIZING MEDICAL IMAGE DATA WITH EXPECTED FINDINGS AND ATTENTION POINTS

Abstract

An improved PACS system includes a configmed processor that executes computer readable instructions stored in memory which cause the processor to load images for assisting a user with reading medical images, obtain an expected finding for the images, display an image of the images, including the expected finding at a region of the image at which the expected finding was identified, sense an eye gaze of the user reading the images, generate a 3-D eye gaze cloud based on the sensed eye gaze, and generate a visualization that includes the image, first graphical indicia representing the expected medical finding overlaid over the image at the region of the image at which the expected finding was identified, and second graphical indicia representing the 3-D eye gaze cloud over a region of the image data at which the eye gaze was sensed for a predetermined period of time.

Inventors

• WIJN, Victor
• SEVENSTER, MERLIJN
• KNOESTER, Jaap
• TUINHOUT, Jelle Jeroen
• VOSBERGEN, Sandra
• KUHLMANN, Daan
• SHAMDASANI, Vijay Thakur
• GLICKBERG, Yan

Assignees

• Koninklijke Philips N.V.

Dates

Publication Date

20260506

Application Date

20240627

Claims (15)

1. 1. A computer-implemented method, the method comprising the steps of: uploading a set of medical images on an apparatus configured for assisting a user with reading the set of medical images; obtaining an expected finding from the set of medical images; displaying at least one imageof the set of medical images, including the expected finding at a region of the image, at which the expected finding was identified; sensing eye gaze behavior of the user in reading the at least one image; generating a 3-D eye gaze cloud based on the sensed eye gaze behavior, wherein the 3-D eye gaze cloud is derived from the set of medical images; and generating a visualization that includes the image, a first graphical indicia representing the expected medical finding overlaid over the image at the region of the image at which the expected finding was identified, and a second graphical indicia representing the 3-D eye gaze cloud over a region of the image data at which the eye gaze behavior was sensed for a predetermined period of time.
2. 2. The method of claim 1, further comprising: generating an imaging report, wherein the imaging report includes at least a part of the visualization.
3. 3. The method of any one of claims 1 to 2, further comprising: accumulating a total amount of eye gaze time at a plurality of regions of the plurality of images of the set of medical images; generating a map of a region in at least one image of the plurality of images in the set of medical images at which the total amount of eye gaze time satisfies a predetermined threshold, for the plurality of the images; and combining the map of the region from each of the at least one image of the plurality of images to generate the 3-D eye gaze cloud.
4. 4. The method of any one of claims 1 to 3, further comprising: adjusting a transparency and/oropaqueness of at least the second graphical indicia representing the 3-D eye gaze cloud.
5. 5. The method of claim 4, further comprising: adjusting the transparency and/or opaqueness of at least the first graphical indicia representing the expected medical finding.
6. 6. The method of any one of claims 1 to 5, further comprising: sensing at least one reading characteristic of the user reading of the images; and generating at least one characteristic cloud representing the at least one reading characteristic of the user reading the images based on the sensed at least one reading characteristic, wherein the visualization further includes third graphical indicia representing the at least one characteristic cloud in conjunction with the second graphical indicia representing the 3-D eye gaze cloud.
7. 7. The method of claim 6, wherein the at least one reading characteristic of the user includes at least one of: stress level measured by a stress reading device, heart rate measured by a hearth rate reading device, fatigue measured by a fatigue reading device, facial expression measured by a fatigue reading device, skin conductivity measured by a fatigue reading device.
8. 8. The method of any one of claims 1 to 7, wherein the expected finding includes a finding identified through anartificial intelligence algorithm, or a finding identified by a user from a previous imaging examination(s).
9. 9. The method of claim 8, further comprising: evaluating a human-artificial intelligence algorithm collaboration or interaction; and generating a report summarizing a reading behavior of the user based on the humanartificial intelligence algorithm collaboration or interaction.
10. 10. The method of any one of claims 1 to 9, further comprising the steps of: evaluating the visualization, wherein the expected finding is displayed based on a viewing preset; and modifying the viewing preset for a display of a subsequent expected finding for the user based on a result of the evaluation.
11. 11. The method of any one of claims 1 to 10, further comprising the steps of: Analyzing the review during reviewing of the visualization by a user; Identifying that a pause have been taken by the user in the review, and identifying a current visualization; and during continuation of the review, continuing from the current visualization.
12. 12. The method of any one of claims 1 to 11, wherein the expected finding includes a confidence level, and further comprising: determining a trust level of the user on the expected finding based on the confidence level of the expected finding and the 3-D eye gaze cloud, wherein the trust level is determined by determining eye gaze time for the expected finding, wherein low eye gaze time is being indicative of high trust level, and high eye gaze time being indicative of low trust level.
13. 13. The method of any one of claims 1 to 12, further comprising: generating a video from at least a portion of the set of medical images, the at least portion of the set of medical images comprising revealing at least one of how the user gazed on a particular image and scrolled through the set of medical images.
14. 14. A picture archiving and communication system comprising: a processor (106) configured to execute computer-executable instructions to: load a set of medical images on an apparatus configured for assisting a user with reading a set of medical images; obtain an expected finding for the set of medical images; display at least one image of the set of medical images, including the expected finding at a region of the at least one image at which the expected finding was identified; sense an eye gaze of the user reading the at least one image; generate a 3-D eye gaze cloud based on the sensed eye gaze behavior, wherein the 3-D eye gaze cloud is derived from the set of medical images; and generate a visualization that includes the at least one image, first graphical indicia representing the expected medical finding overlaid over the image at the region of the at least one image at which the expected finding was identified, and second graphical indicia representing the 3-D eye gaze cloud over a region of the at least one image data at which the eye gaze was sensed for a predetermined period of time.
15. 15. A computer program product comprising instructions which, when the program is executed by a computer, cause the computer to carry out the steps of the method of any one of claims 1 to

Description

IMPROVED PICTURE ARCHIVING COMMUNICATION SYSTEM FOR VISUALIZING MEDICAL IMAGE DATA WITH EXPECTED FINDINGS AND ATTENTION POINTS FIELD OF THE INVENTION The following generally relates to a Picture Archiving Communication System (PACS) and more particularly to an improved PACS system for visualizing medical image data with expected findings and attention points. BACKGROUND OF THE INVENTION According to Wikipedia, a Picture Archiving Communication System (PACS) “is a medical imaging technology which provides economical storage and convenient access to images from multiple modalities (source machine types).” More specifically, a PACS is a specialized apparatus for medical imaging that provides support for a radiologist in reading (e.g., accessing, processing, and/or analyzing) images created by different imaging modalities such as, but not limited to, Digital Radiography (DR), Computed Tomography (CT), Magnetic Resonance (MR), and Nuclear Medicine (NM). A PACS can be designed in many different ways. An exemplary PACS technology infrastructure may include imaging device interfaces, storage devices, host computers, communication networks, and display systems often integrated by a flexible software package for supporting a radiologist in reading a patient case through a diagnostic workflow. Such a patient case is often referred to as an “image study.” Common specialized hardware components may include for example, patient data servers, data/modality interfaces, PACS controllers with database and archive, and display workstations connected by communication networks for handling and managing efficient data/image flow. A PACS is a synergy of specialized hardware and flexible software. The flexible software also includes various functions such as for example, but not limited to, measurement, segmentation, tumor or lesion identification, landmark detection, visualization, and reporting on findings. Medical image studies are received / retrieved and/or transmitted digitally, e.g., over a secure network, and/or portable physical storage medium. Medical images are communicated to and/or from a PACS electronically/digitally via the Digital Imaging and Communications in Medicine (DICOM) protocol, which includes a file format definition and a network communications protocol and uses Transmission Control Protocol (TCP) / Internet Protocol (IP) TCP/IP to communicate between systems. Non-image data, such as a scanned document, may be incorporated, e.g., using formats such as Portable Document Format (PDF). Patient data is typically handled as follows. In general, an examination order is transmitted from a Hospital Information System (HIS) to a Radiology Information System (RIS) via HL.7 and an imaging modality via DICOM. The patient is scanned with the imaging modality. The images are transmitted to a PACS Server. The images are assigned to a radiologist(s) and displayed in a worklist for a radiologist of the group who will read the images at a PACS. The radiologist loads and reads the images at the PACS and creates a report, which is saved (e.g., to the RIS). The report is transmitted to the HIS and/or Electronic Medical Record (EMR) via HL.7 and the images are transmitted to the HIS via DICOM. In some instances, images are read more than once, partially, or entirely. For instance, a radiologist may need to temporarily leave the PACS, e.g., to attend to another matter. In such an instance, when the radiologist returns to the PACS to resume their reading of the image study, the radiologist may need to re-read part of the image(s) to recall where they left off. In another instance, a patient may request a second opinion by another radiologist. In yet another instance, a radiologist in training reads the images and dictates a preliminary report, and then an attending radiologist reviews the images and the preliminary report with the radiologist in training and finalizes the report. In another instance, a radiology department may need to review the reading of an image study for quality assurance. Current technological developments in radiology include improving reading efficiency. However, this may increase pressure and affect the overall reading experience of radiologists. This, in combination with the inconsistency and decrease in active staff within the radiology department in hospitals, can cause a significant increase in workload and stress on the radiologists. This is reflected in the increase in burnout rates of radiologists indicated in the “Statement of Support National Academy of Medicine Collaborative on Clinician Well-Being and Resilience,” American College of Radiology, 5 January 2018. In view of at least the above, there is an unresolved need for an improved technological development(s) for reading medical image data, and more specifically a need to improve current PACS technology or systems. SUMMARY OF THE INVENTION In one aspect, a computer implemented method is disclosed, the method comprising the steps of: uplo