Search

EP-4738289-A1 - VIDEO ANALYSIS METHOD AND SYSTEM DETERMINING OR PREDICTING PATIENT MOVEMENTS IN A MEDICAL OBSERVATION SETTING

EP4738289A1EP 4738289 A1EP4738289 A1EP 4738289A1EP-4738289-A1

Abstract

The present disclosure relates to a computer-implemented method (300) performing video analysis of a video-feed (111), where the method comprises the steps of: obtaining or extracting a plurality of frames (112) from at least a segment of the video-feed (111), generating pose data (112') for each or a least a predetermined number of the plurality of frames (112), the generated pose data (112') representing at least a part of a person or patient (101) comprised by at least one, typically a plurality, of the plurality of frames (112), processing (304), by a first patient movement detector or predictor (200), the generated pose data (112'), and/or first additional data derived on the basis of the generated pose data (112'), to detect or predict at least a first movement of the patient (101) resulting in first data representing first movement detection output data (150).

Inventors

  • AOUN, Toni Elias
  • LANGE, MICHAEL
  • MADSEN, KARSTEN FRANK RYE
  • NIELSEN, Jacob Eric
  • JANSEN, Tejs
  • HAMMER, Niels Risør
  • PEDERSEN, Lasse Helmer
  • INGDAM-LINDGREN, Gustaf Mark Hasse

Assignees

  • M2Call ApS

Dates

Publication Date
20260506
Application Date
20241029

Claims (15)

  1. A computer-implemented method (300) performing video analysis of a video-feed (111), where the method comprises the steps of: - obtaining or extracting a plurality of frames (112) from at least a segment of the video-feed (111), - generating pose data (112') for each or a least a predetermined number of the plurality of frames (112), the generated pose data (112') representing at least a part of a patient (101) comprised by at least one, typically a plurality, of the (first) plurality of frames (112), - processing (304), by a first patient movement detector or predictor (200), the generated pose data (112'), and/or first additional data derived on the basis of the generated pose data (112'), to detect or predict at least a first movement of the patient (101) resulting in first movement data representing first movement detection output data (150).
  2. The computer-implemented method according to claim 1, wherein the computer-implemented method (300) further comprises the step of: - processing (304), by at least a second patient movement detector or predictor (200), the generated pose data (112'), and/or at least second additional data derived on the basis of the generated pose data (112'), to detect or predict at least a second movement of the patient (101) resulting in second movement data representing at least second movement detection output data (150).
  3. The computer-implemented method according to claim 2, wherein the at least one second patient movement detector or predictor (200) is configured to execute one or more of: - detect or predict a hand to face or arm to face movement of the patient (101) and generate the second movement data in response thereto, - detect or predict a cough-related movement of the patient (101) and generate the second movement data in response thereto, - detect or predict an arm movement of the patient (101) and generate the second movement data in response thereto, - detect or predict conditional arm movement of the patient (101) and generate the second movement data in response thereto, - detect or predict eye movement of the patient (101) and generate data the second movement data in response thereto, - detect or predict mouth movement patient (101) and generate data the second movement data in response thereto, - detect or predict a waking and/or sleeping state of the patient (101) over a predetermined period of time and generate data the second movement data in response thereto, - detect or predict leaving a bed, chair, or resting device and generate data the second movement data in response thereto, and - detect or predict whether the patient (101) is located in a bed, a chair, or a resting device or not and generate the second movement data in response thereto.
  4. The computer-implemented method according to any one of claims 1 - 3, wherein the first patient movement detector or predictor (200) is configured to: - determine or predict whether there is any movement, or any movement in accordance with one or more predetermined criteria, within a particular frame of the plurality of frames (112) compared to at least another adjacent frame of the plurality of frames (112) and generate the first movement data in response thereto.
  5. The computer-implemented method according to claim 4 as dependent on claim 2 or 3, wherein the functionality of the first patient movement detector or predictor (200) is carried out, resulting in the first movement data, before the functionality of the at least one second patient movement detector or predictor (200), and where whether the functionality of the at least one second patient movement detector or predictor (200) is carried out or not is dependent on the first movement data so that the functionality of the at least one second patient movement detector or predictor (200), or a part thereof, is not carried out if the first patient movement detector or predictor (200) does not detect or predict any movement in accordance with the one or more predetermined criteria.
  6. The computer-implemented method according to any one of claims 1 - 5, wherein - the method further comprises deriving data representing a contour or boundary (601) for a frame (112) in response to the generated pose data (112') of that frame (112), the contour or boundary (601) defining an area around at least a part of the patient (101) being present in that frame (112), and - wherein the first movement data representing first movement detection output data (150) and/or the second movement data representing second movement detection output data (150) is/are derived processing data of the frame (112) within the derived contour or boundary (601).
  7. The computer-implemented method according to any one of claims 1 - 6, wherein - the method further comprises determining or designating the pixels or parts of a particular frame (112) that constitutes a foreground of the particular frame (112), and - the first movement data representing first movement detection output data (150) and/or the second movement data representing second movement detection output data (150) is/are derived processing data of the particular frame (112) determined or designated to constitute the foreground of the particular frame (112).
  8. The computer-implemented method according to any one of claims 1-7, wherein the method further comprises automatically turning off the video-feed (111), and thereby the processing of the frames (112) of the video-feed (111), when it is detected that another person than the patient (101) is present in at least one frame of the plurality of frames (112).
  9. The computer-implemented method according to any one of claims 1 - 8, wherein the wherein the method further comprises logging and storing detected or predicted movements of the patient (101) in response to the first movement data representing first movement detection output data (150) and/or the second movement data representing second movement detection output data (150).
  10. The computer-implemented method according to any one of claims 1 - 9, wherein the computer-implemented method (300) further comprises the step of: - automatically triggering a clinical alarm on one or more user devices (120), each user device (120) associated with a respective predetermined medical professional, in response to the generated first and/or second movement data (150).
  11. The computer-implemented method according to claim 10, wherein the clinical alarm is triggered firstly on a first group of the user devices (120) and secondly is escalated and triggered on a second group of the user devices (120) if no acknowledgement is received from at least one of the first group of user devices (120) within a predetermined period of time from triggering the clinical alarm on the first group of user devices (120).
  12. The computer-implemented method according to any one of claims 10 - 11, wherein the method comprises providing a live-feed or a substantial live-feed to a first user device (120) the clinical alarm was triggered on.
  13. The computer-implemented method according to any one of claims 1 - 12, wherein the video-feed (111) is obtained by a video camera (110) where the video camera (110) is arranged to capture video of a predetermined observation or monitoring area or space (105), the observation or monitoring area or space (105) e.g. or preferably comprising a bed, chair, or other resting device assigned to the patient (101) in a health care facility.
  14. The computer-implemented method according to any one of claims 1 - 13, wherein the computer-implemented method (300) performing video analysis of a video-feed (111) comprises: - obtaining or receiving data representing additional sensor data from one or more additional sensors (125), - providing at least a part of the data representing additional sensor data to one or more of the first patient movement detector or predictor (200) and the at least one second patient movement detector or predictor (200) and generate the first and/or second movement data also in response thereto.
  15. An electronic data processing system (100), comprising one or more processing units (501, 502) connected to an electronic memory (503), and one or more signal transmitter and receiver communications elements (504) for communicating via a computer network, wherein the one or more processing units (502) are programmed and configured to execute the computer-implemented method (300) according to any one of claims 1 - 14.

Description

Field of the invention The present invention relates generally to a computer-implemented video analysis method and system. More particularly, the present invention relates to a computer-implemented method and a system configured to perform video analysis of a video-feed to detect or predict one or more pre-determined patient movements patterns and potentially generating a clinical alarm in response thereto and/or providing other functionality in response thereto. Background In a medical context or setting of for example a healthcare facility such as a hospital or similar, certain patients requires close attention, observation, and monitoring by medical professionals, e.g. due to their medical state or that there is a higher than normal risk that severe and adverse events could happen to a large detriment of the patient. In such cases, quick and prompt medical attention is critical to avoid or at least reduce detrimental issues to the patient. Such patients could e.g. be patients recovering from post-surgery, sedated or comatose patients, seizure prone patients, patients with spinal cord injuries or muscular dystrophy such as amyotrophic lateral sclerosis (ALS), etc. Some patients require more or less 24h a day observation or monitoring by a medical professional or at least for extended time periods of a day. The crucial observation and monitoring take up resources and much time of the medical staff, even if assisted by certain tools and systems, time that could otherwise be spent for other patients or other tasks. When monitoring patients for example using video surveillance there are, at least in some countries and regions, strict rules and regulations (e.g. The General Data Protection Regulation (GDPR) for European Union and the European Economic Area and other data privacy regulations e.g. in the United States and elsewhere) to adhere to in order to safeguard the patient's privacy, dignity, etc. There are also rules governing what patient data can be obtained, stored, etc., how, and for how long. It is not a given everywhere that video surveillance of a patient can be obtained or stored, even in a hospital setting. Practically speaking, in a medical context or setting it is not possible or feasible to monitor or check up on patients at all times - both night and day. Additionally, certain patients will typically not be capable of alerting a medical professional or staff on their own. Furthermore, the time it takes from an incident until receiving medical attention may be critical and it can potentially have very detrimental effects for a patient if no or too late medical attention is provided. Patent specification US 8,743,200 B2 discloses an activity monitor where a camera obtains a number of images. A Motion Level, representing the amount of change between images, is ongoingly derived by subtracting a current image from a previous image and summing the values of pixels in the resulting image to arrive at a numeric representation of the Motion Level. A derived Motion Level is compared against a Motion Threshold and if the derived Motion Level is greater than the Motion Threshold, a (numeric) Motion Alert Level is increased by a Motion Alert Integrator and otherwise decreased. This will - in case of ongoing motion with no or less non-motion than motion - build-up the Motion Alert Level over time where the Motion Alert Level ongoingly is compared to a Motion Alert Threshold. If there is less motion than non-motion, the Motion Alert Level will decrease over time. In case the build-up Motion Alert Level surpasses the Motion Alert Threshold, an alarm may be triggered. Basically, an alarm is triggered based on all movement (as derived by subtracting images and summing the pixel values in the resulting image). This is fairly noise prone especially in low level lighting. Additionally, the gradual build-up and gradual decrease of levels, respectively, involves an eminent risk of generating false negative alarms (missing a situation where medical attention should have been made) but also false positive alarms (too many alarms). Too many alarms have a high risk of inducing alarm fatigue with the patient and caregiver. Alarm fatigue is announced by the US organisation ECRI and numerous scientific papers as one of the most important predictors for morbidity and mortality within patient monitoring systems. Furthermore, only subtracting images and summing pixel values (even in predefined areas or zones) is generally a very coarse (and too coarse) estimation of movement; in particular for fine(er) movements of a person. Additionally, only overall movement for an entire video frame is estimated. It would be an advantage to be able to reliably detect or estimate movement data, in particular data representing one or more specific movements, for a person or patient, in particular in a medical professional context. It would also be an advantage to provide such using video analysis of a video-feed (or frames therefrom). Fina