Search

EP-4738272-A1 - AUTOMATIC IDENTIFICATION OF REVIEWED IMAGE DATA

EP4738272A1EP 4738272 A1EP4738272 A1EP 4738272A1EP-4738272-A1

Abstract

A computer-implemented method performed within an image viewing and/or processing environment comprising the following steps: - a recognition program (7) obtaining screen capture content of at least part of an image display of a first processing application (11) based on image data loaded into the first processing application; - the recognition program (7) automatically attempting to retrieve identification information from the screen capture content based on at least one screen location of identification information from a library of screen locations (9); - classifying the screen capture content as recognizable if identification information can be retrieved and, based on the retrieved identification information, the image data can be identified in a data storage of image data (5) the second application identification has access to or as non-recognizable if the library of screen locations (9) currently does not contain any screen locations and/or if no identification information can be retrieved; - in case the screen capture content is classified as non-recognizable: the recognition program (7) performing a learning process to determine at least one screen location of identification information on the image display based on changes happening over time on the image display and/or based on user input and storing the at least one screen location of identification information in the library of screen locations (9); - in case the screen capture content is classified as recognizable: providing the identification information to the second processing application (12).

Inventors

  • GOTMAN, SHLOMO

Assignees

  • Koninklijke Philips N.V.

Dates

Publication Date
20260506
Application Date
20241031

Claims (15)

  1. A computer-implemented method for performing within an image viewing and/or processing environment, the environment comprising at least a first processing application (11) being configured to at least partially display image data, and a second processing application (12), and a recognition program (7) having access to a library of screen locations (9) of identification information and comprising or having access to a recognition algorithm (8) that is configured to recognize identification information at a particular screen location, wherein the identification information is information suitable to identify the image data, the method comprising the following steps: - the recognition program (7) obtaining screen capture content of at least part of an image display of the first processing application (11), wherein the image display is based on image data at least partially loaded into the first processing application (11); - the recognition program (7) automatically attempting to retrieve identification information from the screen capture content based on at least one screen location of identification information from the library of screen locations (9) and by applying the recognition algorithm (8); - classifying the screen capture content - as recognizable if identification information can be retrieved and, based on the retrieved identification information, the image data can be identified in a data storage of image data (5) the second application identification has access to or - as non-recognizable if the library of screen locations (9) currently does not contain any screen locations and/or if no identification information can be retrieved; - in case the screen capture content is classified as non-recognizable: the recognition program (7) performing a learning process to determine at least one screen location of identification information on the image display based on changes happening over time on the image display and/or based on user input and storing the at least one screen location of identification information in the library of screen locations (9); - in case the screen capture content is classified as recognizable: providing the identification information to the second processing application (12).
  2. The method according to claim 1, wherein the learning process is performed while the first processing application (11) is running.
  3. The method according to claim 2, wherein the learning process comprises: - tracking changes on the image display that are due to user input and/or registering user input.
  4. The method according to any one of the preceding claims, wherein the learning process comprises: - tracking a user input concerning a selection of image data, - retrieving identification information from the selected image data, and - searching for the identification information on the screen capture content via the recognition algorithm (8) to locate the screen location of the corresponding identification information on the screen capture content.
  5. The method according to any one of the preceding claims, wherein obtaining screen capture content comprises obtaining the screen capture content multiple times and/or continuously, wherein the learning process comprises analysing one or multiple sections of the screen capture content that are changing over time and determining that or whether the at least one screen location of identification information is within a changing section of the screen capture content.
  6. The method according to claim 4 and 5, wherein the learning process comprises searching for the identification information via the recognition algorithm (8) in the one or multiple changing sections of the screen capture content.
  7. The method according to any one of the preceding claims, wherein, at least during the learning process, the recognition program (7) provides a user interface that allows a user the option to select identification information on the image display of the first processing application (11) to input at least one identification information selection; wherein the identification information selection is received by the recognition program (7) and, based on the identification information selection, the at least one screen location of identification information is determined by the recognition program (7).
  8. The method according to any one of the preceding claims, wherein the identification information comprises at least one or more of the following: - a study number, an accession number (21), a series number (22), an image number (23), a slice location (24), a patient name, a patient identification number, a study date, a series label and/or description; and/or - wherein the image data comprises medical image data (30), in particular comprising at least one or more of the following: computed tomography image data, photon counting image data, spectral computed tomography image data, magnetic resonance image data, nuclear imaging data, positron emission tomography data, single photon emission computed tomography imaging data, ultrasound image data, digital pathology data and/or digital X-ray radiogrammetry data.
  9. The method according to any one of the preceding claims, wherein the recognition program (7) is further configured to recognize application identification information on the screen capture content for identifying the first processing application (11), wherein the screen locations in the library of screen locations (9) are stored together with the corresponding application identification information.
  10. The method according to any one of the preceding claims, wherein classifying the screen capture content further comprises: classifying the screen capture content as non-findable if identification information can be retrieved and, based on the retrieved identification information, no image data can be identified in a data storage of image data (5) the second application identification has access to; wherein, in case the screen capture content is classified as non-findable, a user prompt is output notifying a user that the image data is missing and/or asking the user to verify the identification information.
  11. The method according to claim 10, wherein in case in case the screen capture content is classified as non-findable and/or in case a user input is received that verifies the identification information, a further data storage (4) of image data the first processing application (11) is retrieving its image data from is accessed, and image data corresponding to the identification information is located in the further data storage (4) and, if located, transferred to the data storage (5) the second application has access to and/or to the second processing application (12).
  12. The method according to any one of the preceding claims, wherein, after storing the at least one screen location of identification information in the library of screen locations (9) during the learning process, the method is applied again starting with the step of obtaining screen capture content or starting with the step of attempting to retrieve identification information.
  13. A computer program (6) comprising a recognition program (7) and instructions which, when the program is executed by a computer having installed thereon and/or having access to a first processing application (11) and a second processing application (12), at least the first processing application (11) being configured to at least partially display image data, and the computer having access to a library of screen locations (9), cause the computer to carry out the method according to any one of the preceding claims.
  14. An image visualisation and/or processing computing system comprising a processing unit (1) and a computer-readable storage medium (2), in particular non-transient storage medium (2), having stored thereon a computer program (6) according to claim 13 and a first processing application (11) and a second processing application (12), at least the first processing application (11) being configured to at least partially display image data.
  15. The computing system according to claim 14, comprising a data storage, in particular non-transient data storage, with a library of screen locations (9).

Description

FIELD OF THE INVENTION The invention relates to a computer-implemented method performed within an image viewing and/or processing environment, a corresponding computer program, and a corresponding image visualisation and/or processing computing system. BACKGROUND OF THE INVENTION Clinical diagnostic reading of medical images is typically performed by a user on a visualization system such as a PACS (Picture Archiving and Communication System) or a similar system (for example an advanced visualization workstation). While providing basic common viewing capabilities, general visualization systems, such as PACS, often lack the ability for advanced visualization and processing techniques. In addition, they may not able to leverage proprietary, modality-vendor dependent information stored with the images, or proprietary image formats, like CT spectral bases images (SBI). Corresponding functionalities and/or further applications may, however, be provided by different, specific programs, e.g. programs provided by different vendors. To provide these additional capabilities, a possible solution is to provide an add-on application, e.g. a processing application, for example in form of a software application that is running in the PACS environment (e.g., on a PACS workstation), as a client in the client-server architecture, or as a web page. Accordingly, there may be two different applications running in the same environment but as two different applications. However, the two different applications may in many cases not be capable of sharing any information, in particular in case the two applications are from two different vendors. It would, however, be desirable to enable such sharing of information. For example, sharing information between applications may allow a user to select a patient, an examination, and/or an image only once, as part of one application, and this information could be seamlessly used by another application, e.g. a processing application, within the same environment. This could help the user, such as a radiologist, to save time and improve workflow efficiency. Information about a patient, an examination, an image, etc. may also be referred to as "context" or "context information". Without context sharing, the user typically has to work separately with two applications, meaning to select the data separately. One solution could be to provide a non-standard-interface, such as API or command URL, to provide the context to a processing application. However, since there is no widely accepted standard, i.e. the interface is non-standard, this typically requires integration of a processing application with each other application, in particular of different vendors, within the visualization environment separately. This context may be referred to as "tight integration" between applications. However, this approach is often not possible or very difficult to implement. It may take a lot of development time and typically requires adaptions of the integration each time any of the two applications is updated to a new version. Another approach could be to universally use only one uniform standard for all the applications, such as CCOW (Clinical Context Object Workgroup standard). However, a uniform standard that is used by all applications has not been established successfully so far and may be difficult or even impossible to establish in the future for various reasons. In WO 2014/174461 A2, a method is proposed for sharing viewing context between visualization applications by capturing visual context displayed via a visualization application and identify a study based on the captured visual context. This method may provide good results. However, to provide the best results, the method typically involves complex calibration procedures for various screen layouts that have to be repeated with every change, such as an upgrade in the software. It would therefore be desirable to find a more simple and practicable solution for the above-mentioned problems, in particular enabling a functional and easier to implement solution for context sharing in in diagnostic reading systems and third-party applications running thereon. OBJECT OF THE INVENTION It is, therefore, an object of the invention to provide a method to better address the above-mentioned problems, in particular to enable an easier to implement and/or to use solution for sharing context information in medical image reading environments and/or between different applications. SUMMARY OF THE INVENTION To better address this object, a method according to claim 1, a computer program according to claim 13, and a system according to claim 14 is provided. Advantageous embodiments are set out in the dependent claims. Any features, advantages or alternative embodiments described herein in relation to the claimed method are also applicable to the other claim categories and vice versa. According to a first aspect, a computer-implemented method for performing within an image v