CN-122025042-A - Method and system for anonymizing raw surgical procedure video

Abstract

The present disclosure provides various embodiments for anonymizing raw surgical procedure video recorded by a recording device, such as an endoscopic camera, during a surgical procedure performed on a patient within an Operating Room (OR). In one aspect, a method for anonymizing an original surgical procedure video recorded by a recording device within an OR is disclosed. The method can begin by receiving a set of raw surgical videos corresponding to a surgical procedure performed within the OR. The method next merges the set of raw surgical videos to generate a surgical procedure video corresponding to the surgical procedure. Next, the method detects image-based personally identifiable information embedded in a set of original video images of the surgical procedure video. When image-based personally identifiable information is detected, the method automatically de-identifies the detected image-based personally identifiable information in the surgical procedure video.

Inventors

• J, Venkataraman
• P. Garcia kiroy

Assignees

• 威博外科公司

Dates

Publication Date

20260512

Application Date

20190524

Priority Date

20190521

Claims (20)

1. 1.A method, comprising: Receiving surgical video captured by an endoscope inserted into a patient during a surgical procedure performed in an operating room; An in vitro event in a video segment of the surgical video is detected by, the event occurs while the endoscope is still in an open state but is removed from the patient: detection using image processing algorithms 1) in a first set of video frames of a surgical video, the field of view of the endoscope transitions from within the patient to outside the patient as a starting phase of the in vitro event, 2) in a second set of video frames of the surgical video, the subsequent field of view of the endoscope transitions from outside the patient to within the patient as an ending phase of the in vitro event, and Determining that the time interval between the start phase and the end phase is within a threshold time range, and In response to detecting the in vitro event, personally identifiable information PII embedded in the video clip is de-identified.
2. 2. The method of claim 1, wherein the video clip comprises a plurality of video frames comprising the in vitro event, wherein the PII that is de-identified is present in a portion, but not all, of the plurality of video frames.
3. 3. The method of claim 1, wherein the PII comprises text information or facial images related to a patient or a surgical personnel of the surgical procedure, wherein de-identifying the PII comprises blurring or editing the text information or the facial images from the video clip.
4. 4. The method of claim 1, wherein the surgical video comprises a plurality of video frames including the first and second sets of video frames, and the video clip comprises a sequence of one or more video frames of the plurality of video frames, wherein de-identifying the PII comprises black-screen processing each video frame of the sequence of one or more video frames between the first and second sets of video frames.
5. 5. The method of claim 1, further comprising identifying the video segment of the surgical video based on the surgical procedure, wherein the detecting the in vitro event is performed in response to identifying the video segment.
6. 6. The method of claim 1, wherein the image processing algorithm is a Machine Learning (ML) based in vitro event detection model trained to detect in vitro events based on labeled video segments of a set of in vitro events extracted from actual surgical procedure video.
7. 7. The method of claim 5, wherein the surgical video comprises one or more de-identified video segments, wherein identifying the video segments comprises: determining that the video clip has a high probability of the in vitro event based on the surgical video, and It is determined that the video clip is not one of the one or more de-identified video clips.
8. 8. An apparatus, comprising: at least one processor, and A memory having instructions stored therein that, when executed by the at least one processor, cause the apparatus to: Receiving surgical video captured by an endoscope inserted into a patient during a surgical procedure performed in an operating room; An in vitro event in a video segment of the surgical video is detected by, the in vitro event occurs while the endoscope is still in an open state but is removed from the patient: detection using image processing algorithms 1) in a first set of video frames of the surgical video, the field of view of the endoscope transitions from within the patient to outside the patient as a starting phase of the in vitro event, 2) in a second set of video frames of the surgical video, the subsequent field of view of the endoscope transitions from outside the patient to within the patient as an ending phase of the in vitro event, and Determining that the time interval between the start phase and the end phase is within a threshold time range, and In response to detecting the in vitro event, personally identifiable information PII embedded in the video clip is de-identified.
9. 9. The apparatus of claim 8, wherein the surgical video comprises a data file containing raw video data captured by the endoscope during a surgical procedure previously performed in the operating room.
10. 10. The apparatus of claim 8, wherein the PII comprises text information or a face related to a patient or a surgical personnel of the surgical procedure, wherein the instructions to de-identify the PII comprise instructions to blur or edit the text information or the face from the video clip.
11. 11. The apparatus of claim 8, wherein the surgical video comprises a plurality of video frames including the first and second sets of video frames and the video clip comprises a sequence of one or more video frames of the plurality of video frames, wherein the instructions to de-identify the PII comprise instructions to black screen each video frame in the sequence of one or more video frames between the first and second sets of video frames.
12. 12. The apparatus of claim 8, wherein the memory further comprises instructions to identify the video segment of the surgical video based on the surgical procedure, wherein the detecting of the in vitro event is performed in response to the identification of the video segment.
13. 13. The apparatus of claim 8, wherein the image processing algorithm is a Machine Learning (ML) based in vitro event detection model trained to detect in vitro events based on a set of labeled in vitro event video clips extracted from actual surgical procedure video.
14. 14. The apparatus of claim 12, wherein the surgical video comprises one or more de-identified video segments, wherein the instructions to identify the video segments comprise instructions to: determining that the video clip has a high probability of the in vitro event based on the surgical video, and It is determined that the video clip is not one of the one or more de-identified video clips.
15. 15. A non-transitory machine-readable medium having instructions that, when executed by at least one processor, cause the at least one processor to: Receiving a surgical video captured by an endoscope inserted into a patient in a surgical procedure performed in an operating room; an in vitro event in a video segment of the surgical video is detected by, the in vitro event occurs while the endoscope is still in an open state but is removed from the patient: Detection using an image processing algorithm 1) in a first set of video frames of the surgical video, a field of view of the endoscope transitions from within the patient to outside the patient as a beginning phase of the in vitro event, 2) in a second set of video frames of the surgical video, a subsequent field of view of the endoscope transitions from outside the patient to within the patient as an ending phase of the in vitro event, and Determining that the time interval between the start phase and the end phase is within a threshold time range, and In response to detecting the in vitro event, personally identifiable information PII embedded in the video clip is de-identified.
16. 16. The non-transitory machine-readable medium of claim 15, wherein the PII comprises text information or facial information related to a patient or a surgical personnel of the surgical procedure, wherein the instructions to de-identify PII comprise instructions to blur or edit the text information or the facial information from the video clip.
17. 17. The non-transitory machine-readable medium of claim 15, wherein the surgical video comprises a plurality of video frames including the first and second sets of video frames and the video clip comprises a sequence of one or more of the plurality of video frames, wherein the instructions to de-identify the PII comprise instructions to black screen each video frame in the sequence of one or more video frames between the first and second sets of video frames.
18. 18. The non-transitory machine-readable medium of claim 15, further comprising instructions to identify the video segment of the surgical video based on the surgical procedure, wherein the detecting the in vitro event is performed in response to the identification of the video segment.
19. 19. The non-transitory machine-readable medium of claim 15, wherein the image processing algorithm is a Machine Learning (ML) based in vitro event detection model trained to detect in vitro events based on a set of labeled in vitro event video clips extracted from an actual surgical procedure video.
20. 20. The non-transitory machine-readable medium of claim 18, wherein the surgical video comprises one or more de-identified video segments, wherein the instructions to identify the video segments comprise instructions to: determining that the video clip has a high probability of the in vitro event based on the surgical video, and It is determined that the video clip is not one of the one or more de-identified video clips.

Description

Method and system for anonymizing raw surgical procedure video Technical Field The present disclosure relates generally to building surgical procedure video analysis tools, and more particularly, to systems, devices, and techniques for anonymizing raw surgical procedure video to de-identify personally identifiable information and provide anonymized surgical procedure video for various research purposes. Background Recorded video of medical procedures, such as surgery, contains very valuable and rich information for medical education and training, assessing and analyzing the quality of the surgery and the skill of the surgeon, as well as for improving the outcome of the surgery and the skill of the surgeon. There are many surgical procedures that involve displaying and capturing video images of the surgical procedure. For example, almost all minimally invasive procedures (MIS) such as endoscopy, laparoscopy, and arthroscopy involve the use of cameras and video images to assist the surgeon. In addition, prior art robotic-assisted surgery requires capturing intraoperative video images and displaying to the surgeon on a monitor. Thus, for many of the above surgical procedures, such as gastric lavage or cholecystectomy, a large amount of surgical video already exists and continues to be created as a result of a large number of surgical cases being performed by many different surgeons from different hospitals. The simple fact that a large (and increasing) number of surgical videos of a particular surgical procedure exist makes processing and analyzing the surgical videos of a given procedure a potential machine learning problem. However, the original surgical video from the record in the Operating Room (OR) may contain all kinds of patient information in the form of text-based identifiers, including the patient's name, medical record number, age, gender, demographics, date and time of surgery, and so forth. In addition, some surgical procedure videos may also contain sensitive and private information captured inside the OR, such as information written on a whiteboard in the OR and the face of the surgical staff. Thus, before the original surgical procedure video can be used for various research purposes such as for building machine learning tools, the original surgical procedure video needs to be anonymized so as to be free of personally identifiable information and to comply with HIPAA regulations and procedures. There are several automated anonymizing tools available for removing text identifiers from a file, and for detecting sensitive information from a medical image file, such as a CT scan of a patient, X-rays, etc., and removing it from the medical image file. However, existing techniques for anonymizing buried sensitive information in raw protocol video are typically based on manual, which requires a human operator to view individual videos to identify sensitive information in video frames, and then anonymize the sensitive information manually (e.g., by removal or removal). The manual-based video anonymization process is laborious and time-consuming. In particular, building a machine learning tool requires that a large number of raw surgical procedure videos be anonymized first, which makes manual-based video anonymization impractical for machine learning purposes. Unfortunately, there are no existing automated anonymizing tools for anonymizing sensitive information buried in the original surgical video. Disclosure of Invention The present disclosure provides various embodiments for anonymizing raw surgical procedure video recorded by a recording device, such as an endoscopic camera, during a surgical procedure performed on a patient within an Operating Room (OR). In one aspect, a method for anonymizing an original surgical procedure video recorded by a recording device within an OR is disclosed. The method can begin by receiving a set of raw surgical videos corresponding to a surgical procedure performed within the OR. The method next merges the set of raw surgical videos to generate a surgical procedure video corresponding to the surgical procedure. Next, the method detects image-based personally identifiable information embedded in a set of original video images of the surgical procedure video. When image-based personally identifiable information is detected, the method automatically de-identifies the detected image-based personally identifiable information in the surgical procedure video. In some embodiments, the method merges the set of raw surgical videos to generate the surgical procedure video by analyzing a set of file names associated with the set of raw surgical videos to determine a correct order relative to the surgical procedure, and then stitching the set of raw surgical videos together based on the determined order. In some embodiments, the method detects image-based personally identifiable information embedded in one or more video images of the surgical procedure video by detecting persona