CN-121986380-A - Medical device voice and gesture control method, apparatus and system

Abstract

Methods, apparatuses, and systems for voice and gesture control of medical devices are disclosed herein. Example devices include, for example, an audio interface that converts received audio to text. The audio interface uses the current operating state of the medical device to determine a command from the text and whether the command is enabled. When the command is enabled, the audio interface sends a command message to a response manager, which converts the command to a medical device command signal. The response manager then sends a medical device command signal to the control engine of the medical device to change how the medical device operates based on the received audio. The same medical device command signals are used for similar touch and/or gesture inputs such that the control engine need not be modified based on which input modalities are implemented on the medical device.

Inventors

• D. Neyman

Assignees

• 万益特健康有限责任公司
• 万益特美国医疗保健有限责任公司

Dates

Publication Date

20260505

Application Date

20240930

Priority Date

20231005

Claims (20)

1. 1. A dialysis machine comprising: A memory device storing audio rules specifying: (i) Which audio commands are enabled based on the current state of the dialysis machine, and (Ii) Selecting which dialysis machine command signal based on the audio command and the current state of the dialysis machine; A display screen including buttons for inputting commands; a control interface configured to present a graphical user interface for display by the display screen and to output a touch command indicating a specified dialysis machine operation to be performed; A microphone; an audio interface configured to convert audio signals from the microphone into audio commands indicative of a specified dialysis machine operation to be performed; a control engine configured to perform dialysis machine operations based on the received dialysis machine command signals, and A processor communicatively coupled to the memory device, the display screen, the control interface, the control engine, and the audio interface, the processor comprising: A response manager configured to: receiving one of (a) the touch command from the control interface or (b) the audio command from the audio interface, Communicate with the control engine to determine a current state of the dialyzer, When the audio command is received and the (i) audio rule indicates that the audio command is enabled, determining the dialysis machine command signal corresponding to the audio command using the (ii) rule and a current state of the dialysis machine, and The dialysis machine command signal is sent to the control engine.
2. 2. The dialysis machine of claim 1, wherein the current state of the dialysis machine comprises at least one of a programmed setting, a fluid line setting, disinfection, fluid line pre-priming, recirculation, blood return, treatment management, and service mode.
3. 3. The dialysis machine of claim 1, wherein the current state comprises an indication of the graphical user interface displayed on the display screen.
4. 4. The dialysis machine of claim 1, wherein the control interface comprises at least one of a button, knob, switch, control surface, dial, or touch screen integrated with the display screen.
5. 5. The dialysis machine of claim 1, wherein the dialysis machine command signal specified by the audio rule indicates at least one of setting or changing dialysis machine treatment parameters, starting dialysis treatment, pausing dialysis treatment, stopping dialysis treatment, deasserting or muting an alarm or warning, starting a pre-priming operation, stopping a pre-priming operation, confirming completion of a treatment setting step, starting a disinfection operation, initiating a call to a clinician, changing a language of an audio command, changing a trigger word, selecting features or options provided within a user interface shown on the display screen, navigating between different user interfaces, closing the display screen, or starting recording audio.
6. 6. The dialysis machine of claim 1, wherein the audio command comprises at least one of a command word, a command phrase, or a command word and a parameter value.
7. 7. The dialysis machine of claim 1, wherein the audio interface is configured to convert the audio signals from the microphone into the audio commands by: converting the audio signal into text, a unique message or a signal; comparing the text, unique message or signal with the audio command provided in the audio rule, and When there is a match or a better comparison between the text, unique message or signal and one of the audio commands, the matching or better compared audio command is sent to the response manager.
8. 8. The dialysis machine of claim 7, wherein the audio rule further comprises at least one trigger word, and Wherein the audio interface is further configured to: comparing the text, unique message or signal with the at least one trigger word, and When there is a match between the text, unique message or signal and the at least one trigger word, the text, unique message or signal or additional text, unique message or signal is compared to an audio command provided in the audio rule to convert an audio signal from the microphone to the audio command.
9. 9. The dialysis machine of claim 8, wherein the audio interface is further configured to: Starting a timer when the at least one trigger word is detected within the text, and The text, unique message or signal or the additional text, unique message or signal is compared to the audio command until the timer reaches a predefined time limit.
10. 10. The dialysis machine of claim 8, wherein the response manager is further configured to, upon receipt of the audio command: determining which button or option of the control interface corresponds to the audio command, and Buttons or options of the control interface are illuminated or otherwise highlighted for a specified duration.
11. 11. The dialysis machine of claim 1, wherein the dialysis machine comprises at least one of a hemodialysis machine, a hemodiafiltration machine, a peritoneal dialysis machine, or a continuous kidney replacement therapy ("CRRT") machine.
12. 12. The dialysis machine of claim 1, wherein, when the response manager receives the touch command from the control interface, the response manager is configured to: determining the dialysis machine command signal corresponding to the touch command using the (ii) rule, and The dialysis machine command signal is sent to the control engine.
13. 13. The dialysis machine of claim 1, wherein, when the audio command is received and the (i) audio rule indicates that the audio command is disabled, the response manager is configured to at least one of: Discarding the audio command, or A warning is generated indicating that the audio command is discarded.
14. 14. The dialysis machine of claim 1, wherein at least one of the control interface and the audio interface is included within the processor.
15. 15. The dialysis machine of claim 1, further comprising: gesture sensor, and A gesture interface communicatively coupled to the gesture sensor, the gesture interface configured to convert video or data indicative of arm/hand movements into gesture commands indicative of specified dialysis machine operations to be performed.
16. 16. The dialysis machine of claim 15, wherein the response manager is further configured to: receiving the gesture command from the gesture interface; When the gesture command is received and a gesture rule indicates that the gesture command is enabled, determining the dialysis machine command signal corresponding to the gesture command using the gesture rule and a current state of the dialysis machine, and The dialysis machine command signal is sent to the control engine.
17. 17. The dialysis machine of claim 1, further comprising at least one pump, pinch valve, or solenoid configured to control the flow of fluid to perform at least some of the dialysis machine operations, Wherein the control engine is configured to control the at least one pump, pinch valve or solenoid based on the dialysis device command signal.
18. 18. A method, comprising: Receiving an audio input from a microphone of a dialysis machine in an audio interface of the dialysis machine; Converting the audio input to text, a unique message, or a signal via the audio interface; Determining via the audio interface whether the received text, unique message or signal includes a specified trigger word; Determining, via the audio interface, whether a command word is included in the received text, unique message, or signal within a period of time from when the trigger word was received when the received text, unique message, or signal includes the specified trigger word; Upon receipt of the command word within the time period, determining, via the audio interface, whether an audio command corresponding to the command word is enabled using a data structure of audio rules and a current operating state of the dialysis machine; When the command word is enabled, sending an audio command message to a response manager of the dialysis machine indicating the audio command; Determining dialysis machine command signals corresponding to the audio commands using the same data structure or different data structures via the response manager, and The dialysis machine command signal is sent from the response manager to a control engine of the dialysis machine.
19. 19. The method of claim 18, further comprising discarding, via the audio interface, additional received text, unique message or signal until the trigger word is received again when no command word is received during the period of time.
20. 20. The method of claim 18, further comprising causing at least one button or graphical button of the dialysis machine to illuminate or flash when the command word is enabled.

Description

Medical device voice and gesture control method, apparatus and system Background Renal replacement therapy ("RRT") is a therapy that replaces the normal hemofiltration function of the kidney. It is used when the kidneys fail to function properly, which is known as renal failure and includes acute kidney injury ("AKI") and chronic kidney disease ("CKD"). RRT involves the removal of water from patients with renal failure and exchange of solutes with the body. One example of RRT is extracorporeal blood treatment, in which blood is circulated outside the patient and is in interfacial contact with one or more medical fluids. Modes of extracorporeal blood treatment include hemodialysis ("HD"), hemofiltration ("HF") and hemodiafiltration ("HDF"). Another example of RRT is peritoneal dialysis ("PD") in which a medical fluid is infused into the peritoneal cavity of a patient to form an interface with the patient's blood through the peritoneum. RRT is performed by a dialysis system formed by arranging one or more disposable components on a dialysis machine. Medical fluids used in HD and PD are commonly referred to as dialysis fluids. In HF, the medical fluid is referred to as a replacement fluid because it is infused into the patient's blood to replace the fluid removed during treatment. In HDF, both a dialysis fluid and a substitution fluid are used. By using a different type of dialysis machine, the extracorporeal blood treatment performed by HD, HF or HDF is different for the treatment of patients with AKI than for patients with CKD. Generally, AKI patients are continuously treated for a longer period of time and at a lower fluid flow rate than CKD patients. This continuous treatment is commonly referred to as CRRT ("continuous renal replacement therapy"). To ensure accurate and consistent monitoring and control of fluid removal (known as ultrafiltration), AKI machines are typically provided with scales for measuring the weight of fresh therapeutic fluid and the weight of therapeutic fluid used during treatment. CKD machines in turn use flow meters or volumetric pumps to control ultrafiltration. The PD machine (also referred to as a cycler) may include at least one scale to measure the weight of fresh therapeutic fluid infused into the peritoneal cavity and the weight of used therapeutic fluid withdrawn from the peritoneal cavity. Alternatively, the cycler may use a volumetric pump to control ultrafiltration. In any of the above modes, the automated dialysis machine is located in a medical center or in the patient's home. To program or change the treatment, the patient or clinician must currently physically press a control interface, such as a button or touch screen. However, many patients receiving dialysis treatment may have reduced flexibility such that pressing the control interface may be difficult or error-prone. In addition, some patients may lie down during dialysis treatment and may not want to sit up or otherwise reposition themselves to have to interact with the dialysis machine. In addition to the patient, the clinician may prefer not to physically touch the control interface of the dialysis machine. For example, a clinician may have to disinfect their hands each time they contact the dialysis machine. In other cases, the clinician may set up a line set or care for the patient between physical programming of the dialysis machine. There is therefore a need for a dialysis system that provides control of the hands-off. Disclosure of Invention Example systems, methods, and apparatus for audio and/or gesture control of medical devices are disclosed herein. An example device includes an audio interface that converts received audio into text, unique messages, or signals. The audio interface uses the current operating state of the medical device to determine a command from a text, unique message or signal and whether the command is enabled. When the command is enabled, the audio interface sends a command message to a response manager, which converts the command to a medical device command signal. The response manager then sends a medical device command signal to the control engine of the medical device to change how the medical device operates based on the received audio. The same medical device command signals are used for similar touch and/or gesture inputs such that the control engine need not be modified based on which input modalities are implemented on the medical device. In view of the disclosure herein and without limiting the disclosure in any way, in a first aspect of the disclosure, which may be combined with any other aspect listed herein, the dialysis machine comprises a memory device that stores audio rules. The audio rules specify (i) which audio commands are enabled based on the current state of the dialysis machine, and (ii) which dialysis machine command signal is selected based on the audio commands and the current state of the dialysis machine. The dialysis machine also includes a display s