DE-102024132853-A1 - Generating training data for a machine learning model and using a machine learning model

DE102024132853A1DE 102024132853 A1DE102024132853 A1DE 102024132853A1DE-102024132853-A1

Abstract

In a computer-implemented method for generating training data for a machine learning model for use in a clinical setting, sensor data from at least one sensor present in the clinical setting is received. Situational information is received, which depends on a change in the state of a device in the clinical setting and/or an action by a person in the clinical setting. At least a portion of the sensor data is automatically annotated using this situational information to generate the training data.

Inventors

Klaus Brüderle
Tim Heinzmann

Assignees

KARL STORZ SE & CO. KG

Dates

Publication Date: 20260513
Application Date: 20241111

Claims (15)

A computer-implemented method for generating training data for a machine learning model for use in a clinical setting, comprising the steps of: Receiving (S101) sensor data from at least one sensor present in the clinical setting; Receiving (S102) situational information, which depends on a change in the state of a facility in the clinical setting and/or an action of a person in the clinical setting; and Generating (S103) the training data by automatically annotating at least some of the sensor data, using the situational information.
Procedure according to Claim 1 , where the machine learning model is trained using the annotated sensor data.
Procedure according to Claim 1 or 2 , whereby, depending on the situation information, it is determined whether the sensor data is automatically annotated or discarded.
Method according to one of the preceding claims, further comprising the step of: Determining the situation information based on the sensor data and/or on further sensor data.
Method according to any of the preceding claims, wherein the action of a person in the clinical situation comprises at least one of activating or deactivating a device used in the clinical situation by the person, of a certain movement of a device used in the clinical situation by the person, and of uttering a certain word or phrase by the person.
Method according to one of the preceding claims, wherein the clinical situation comprises video endoscopy, wherein the sensor data comprises image data and/or video data, and wherein the machine learning model is trained for use in smoke gas detection, progress detection of the video endoscopy and/or energy monitoring of a high-frequency device used in the video endoscopy.
Method for using a machine learning model in a clinical situation, comprising the steps of: Receiving (S201) sensor data from at least one sensor present in the clinical situation; Using (S202) a machine learning model to generate a recommendation for action and/or to automatically perform an action, using the sensor data as input data to the machine learning model; and Determining (S203) response information based on a person's response to the recommendation for action and/or to the action performed.
Procedure according to Claim 7 , whereby the machine learning model is adapted depending on the response information.
Procedure according to Claim 7 or 8 , where the response information includes information on whether the person acted in accordance with the recommended course of action.
Procedure according to one of the Claims 7 until 9 , where the response information includes information on whether the person canceled or reversed the automatic execution of the action.
Procedure according to one of the Claims 7 until 10 , where an algorithm is used to further calculate a reliability measure which indicates how reliable the recommended action and/or the automatic execution of the action is, and where the algorithm is adapted depending on the response information.
Procedure according to one of the Claims 7 until 11 , whereby the machine learning model is adapted depending on a user profile of the responding person.
Device (100) for generating training data for a machine learning model for use in a clinical situation, comprising: an interface (101) configured to: receive sensor data from at least one sensor present in the clinical situation, and receive situation information which depends on a change in the state of a device in the clinical situation and/or an action of a person in the clinical situation; and a computing device (102) configured to automatically annotate at least part of the sensor data using the situation information in order to generate training data.
Device (200) for using a machine learning model in a clinical situation, comprising: an interface (201) configured to receive sensor data from at least one sensor present in the clinical situation; a computing unit (202) configured to use a machine learning model to generate a recommendation for action and/or to automatically perform an action, wherein the sensor data are used as input data for the machine learning model; and a response information determination unit (204) configured to determine response information depending on a person's response to the recommendation for action and/or to the action performed.
System (300) for use in a clinical situation, comprising: a device (100) for generating training data for a machine learning model according to Claim 13 ; and a device (200) for using the machine learning model according to Claim 14 .

Description

The present invention relates to a method and a device for generating training data for a machine learning model in a clinical setting. The present invention further relates to a method and a device for using a machine learning model in a clinical setting. Finally, the present invention relates to a system comprising a device for generating training data for a machine learning model and a device for using the machine learning model in a clinical setting. In the medical field, systems are known that can provide support in clinical situations, such as during certain procedures. For example, from the WO 2024/008854 A1 a system-assisted system for minimally invasive surgery is known. The application of machine learning or artificial intelligence (AI) methods in clinical situations has the potential to improve support during interventions, diagnostic accuracy, efficiency and patient care. In endoscopy, for example, AI algorithms can be used to analyze endoscopic images in real time and detect anomalies. AI systems can also support the user in making a diagnosis, as well as provide real-time feedback and recommendations to highlight or confirm potential pathological findings. Machine learning models are trained on large amounts of data to recognize patterns and features that are characteristic of certain disease states. In the current stage of machine learning, generating training data requires considerable effort. Data must first be carefully sorted, evaluated, and weighted manually before it can be integrated into the training process. These manual steps are not only time-consuming but also carry the risk of errors and inconsistencies. Summary of the invention It is therefore an object of the present invention to provide devices and methods for generating training data for a machine learning model in a clinical setting, in order to simplify the generation of training data. Furthermore, methods and devices for using a machine learning model in a clinical setting are to be provided. Finally, a system that combines the two aforementioned devices is to be provided. This problem is solved by the subject matter of the independent claims of the present invention. Advantageous embodiments are the subject matter of the dependent claims. According to a first aspect, a computer-implemented method for generating training data for a machine learning model for use in a clinical setting is provided. Sensor data is received from at least one sensor present in the clinical setting. Situational information is received, which depends on a change in the state of a device in the clinical setting and/or an action by a person in the clinical setting. At least a portion of the sensor data is automatically annotated using the situational information to generate the training data. A fundamental idea of the present invention is that the sensor data can be at least partially annotated (i.e., labeled) automatically. This eliminates the need for time-consuming manual annotation. Situational information is taken into account for this purpose. Within the scope of this invention, situational information can be understood as information that depends on or describes states, changes in state, actions, events, or the like in the clinical situation. For example, a specific action by a person (such as a treating physician or assistant) can indicate the circumstances or events present in the clinical situation, and the sensor data can be labeled accordingly. Similarly, a change in the state of a device used in the clinical setting can indicate the occurrence of an event or the presence of a specific situation, allowing the sensor data to be labeled accordingly. This enables more efficient generation of training data for the machine learning model. Within the scope of this invention, a clinical situation can be understood to mean an operation, a surgical procedure, the treatment of a patient, an external examination of a patient, or the like. In particular, medical or surgical instruments may be used. A change in the state of a device can refer, in particular, to switching it on, off, or changing its operating parameters. The device could be, for example, a medical instrument or system. In an endoscopy, for instance, a smoke extraction system could be activated or deactivated. According to one embodiment of the method for generating training data for the machine learning model, the training data can be generated autonomously, locally and anonymously, which is particularly advantageous with regard to data protection considerations. According to one embodiment, the method is further provided for generating the machine learning model. For this purpose, the machine learning model is trained using the annotated sensor data. The trained machine learning model can then be output and used in the clinical setting. According to one embodiment of the method for generating training data for the machine learning model, the system determines, based on situational i