US-12620223-B2 - Identifying variation in surgical approaches

US12620223B2US 12620223 B2US12620223 B2US 12620223B2US-12620223-B2

Abstract

An aspect includes a computer-implemented method that identifies variations in surgical approaches to medical procedures. Surgical videos documenting multiple cases of a medical procedure are analyzed to identify different surgical approaches used by service providers when performing the medical procedure. According to some aspects surgical phases are identified in each surgical video and groups of similar surgical phase sequences are grouped into surgical approaches.

Inventors

Carole RJ Addis
Sheldon K. Hall
Pinja ME Haikka
George Bruce Murgatroyd

Assignees

DIGITAL SURGERY LIMITED

Dates

Publication Date: 20260505
Application Date: 20220414

Claims (18)

1 . A system comprising: a machine learning training system comprising one or more machine learning models that are trained to identify a plurality of surgical phases in a video of a surgical procedure; and a data analysis system configured to identify different surgical approaches in a plurality of videos capturing a same type of surgical procedure, wherein the identifying different surgical approaches comprises: receiving a plurality of surgical videos, each of the plurality of surgical videos capturing a workflow of the same type of surgical procedure; segmenting each of the plurality of surgical videos into a plurality of surgical phases based on surgical phases identified by the machine learning training system; generating a plurality of symbols representing each segment in each of the plurality of surgical videos to form symbolic representations of the plurality of surgical videos as workflows comprising ordered sequences of the surgical phases; clustering the symbolic representations of the plurality of surgical videos to form clusters of surgical videos having similar phase sequences; and outputting the clusters to a display device for display, wherein each of the clusters represents a different surgical approach for the same type of surgical procedure.
2 . The system of claim 1 , wherein the identifying different surgical approaches further comprises outputting an indication of a surgical service provider associated with one or more of the surgical videos or clusters to the display device for display.
3 . The system of claim 2 , wherein the surgical service provider is a group of physicians at a hospital.
4 . The system of claim 2 , wherein the surgical service provider is a physician.
5 . The system of claim 2 , wherein the identifying different surgical approaches further comprises outputting, to the display device, metrics related to surgical approaches taken by different surgical service providers.
6 . The system of claim 1 , wherein the clustering comprises: calculating a distance metric between each pair of the plurality of surgical videos; storing the calculated distance metrics in a distance matrix; and clustering similar workflows based at least in part on the distance matrix.
7 . The system of claim 1 , wherein the identifying different surgical approaches further comprises adding a label to each of the different surgical approaches and outputting the labels to the display device.
8 . A computer-implemented method comprising: receiving, by a processor, a plurality of surgical videos, each of the plurality of surgical videos capturing a workflow of a same type of surgical procedure; segmenting each of the plurality of surgical videos into surgical phases; generating a plurality of symbols representing each segment in each of the plurality of surgical videos to form symbolic representations of the plurality of surgical videos as workflows comprising ordered sequences of the surgical phases; clustering, by the processor, the plurality of surgical videos into clusters of similar workflows, the clustering based at least in part on the ordered sequences of the surgical phases of each of the surgical videos; and identifying, by the processor, surgical approaches that are unique to each of the clusters, the identifying based at least in part on the surgical phases of each of the surgical videos.
9 . The computer-implemented method of claim 8 , further comprising labeling each of the unique surgical approaches.
10 . The computer-implemented method of claim 8 , wherein the applying clustering techniques comprises: calculating a distance metric between each pair of the plurality of surgical videos; storing the calculated distance metrics in a distance matrix; and clustering similar workflows based at least in part on the distance matrix.
11 . The computer-implemented method of claim 10 , further comprising determining optimal clustering hyperparameters for the plurality of surgical videos.
12 . The computer-implemented method of claim 8 , further comprising outputting, by the processor to a display device, a graphical representation of the workflows in the clusters of similar workflows and the identified surgical approaches.
13 . The computer-implemented method of claim 12 , wherein the graphical representation identifies a surgical service provider associated with each of the workflows.
14 . A computer program product comprising a memory device having computer-executable instructions stored thereon, which when executed by one or more processors cause the one or more processors to perform operations comprising: visualizing different surgical approaches used by service providers when performing a surgical procedure, the visualizing comprising: receiving a plurality of surgical videos, each of the plurality of surgical videos capturing a workflow of a service provider performing the surgical procedure; segmenting each of the plurality of surgical videos into surgical phases; generating a plurality of symbols representing each segment in each of the plurality of surgical videos to form symbolic representations of the plurality of surgical videos as workflows comprising ordered sequences of the surgical phases; clustering the plurality of surgical videos into clusters of similar workflows, the clustering based at least in part on the ordered sequences of the surgical phases of each of the surgical videos; identifying surgical approaches that are unique to each of the clusters, the identifying based at least in part on the surgical phases of each of the surgical videos; and outputting, to a display device, a graphical representation of the identified surgical approaches.
15 . The computer program product of claim 14 , wherein the visualizing further comprises: receiving user input via a user interface of the graphical representation; and in response to the user input, outputting to the display device, a second graphical representation that includes providers that use the identified surgical approaches.
16 . The computer program product of claim 14 , wherein the visualizing further comprises: receiving user input via a user interface of the graphical representation; and in response to the user input, outputting to the display device, a second graphical representation that includes a label describing each of the identified surgical approaches.
17 . The computer program product of claim 14 , wherein the visualizing further comprises: receiving user input via a user interface of the graphical representation; and in response to the user input, outputting to the display device, a second graphical representation that includes additional information describing characteristics of one or both of a service provider or a patient.
18 . The computer program product of claim 14 , wherein the clustering comprises: calculating a distance metric between each pair of the plurality of surgical videos; storing the calculated distance metrics in a distance matrix; and clustering similar workflows based at least in part on the distance matrix.

Description

CROSS REFERENCE TO RELATED APPLICATIONS This application is a National Stage application of PCT/EP2022/060030, filed Apr. 14, 2022, which claims the benefit of U.S. Provisional Patent Application No. 63/175,209, filed Apr. 15, 2021 and U.S. Provisional Patent Application No. 63/208, 171, filed Jun. 8, 2021, all of which are incorporated by reference in their entirety herein. BACKGROUND The present invention relates in general to computing technology and relates more particularly to computing technology for identifying variations in surgical approaches. Computer-assisted systems, particularly computer-assisted surgery systems (CASs), rely on video data digitally captured during a surgery. Such video data can be stored and/or streamed. In some cases, the video data can be used to augment a person's physical sensing, perception, and reaction capabilities. For example, such systems can effectively provide the information corresponding to an expanded field of vision, both temporal and spatial, that enables a person to adjust current and future actions based on the part of an environment not included in his or her physical field of view. Alternatively, or in addition, the video data can be stored and/or transmitted for several purposes such as archival, training, post-surgery analysis, and/or patient consultation. The process of analyzing and comparing a large amount of video data from multiple surgical procedures to identify commonalities can be highly subjective and error-prone due, for example, to the volume of data and the numerous factors (e.g., patient condition, physician preferences, etc.) that impact the workflow of each individual surgical procedure that is being analyzed. SUMMARY According to one or more aspects, a system includes a machine learning training system that includes one or more machine learning models that are trained to identify a plurality of surgical phases in a video of a surgical procedure. The system also includes a data analysis system configured to identify different surgical approaches in a plurality of videos capturing a same type of surgical procedure. Identifying the different surgical approaches includes receiving a plurality of surgical videos, each of the plurality of surgical videos capturing a workflow of the same type of surgical procedure, segmenting each of the plurality of surgical videos into a plurality of surgical phases based on surgical phases identified by the machine learning training system, and representing each segment in each of the plurality of surgical videos as a symbol. The symbolic representations of the plurality of surgical videos are clustered to form groups of surgical videos having similar workflows and the groups are output to a display device for display. Each group represents a different surgical approach. In one or more aspects, the identifying different surgical approaches further includes outputting an indication of a surgical service provider associated with one or more of the surgical videos or groups to the display device for display. In one or more aspects, the surgical service provider is a hospital. In one or more aspects, the surgical service provider is a physician. In one or more aspects, the identifying different surgical approaches further includes outputting a comparison of surgical approaches taken by different surgical service providers to the display device for display. In one or more aspects, the clustering includes calculating a distance metric between each pair of the plurality of surgical videos, storing the calculated distance metrics in a distance matrix, and grouping similar workflows based at least in part on the distance matrix In one or more aspects, the identifying different surgical approaches further includes adding a label to each of the different surgical approaches and outputting the labels to the display device. According to one or more aspects, a computer-implemented method includes receiving, by a processor, a plurality of surgical videos, each of the plurality of surgical videos capturing a workflow of a same type of surgical procedure and each of the plurality of surgical videos segmented into surgical phases. The method further includes grouping, by the processor, the plurality of surgical videos into groups of similar workflows, the grouping based at least in part on the surgical phases of each of the surgical video. The method further includes identifying, by the processor, surgical approaches that are unique to each of the groups, the identifying based at least in part on the surgical phases of each of the surgical videos. In one or more aspects, the method further includes labeling each of the unique surgical approaches. In one or more aspects, the grouping includes applying clustering techniques In one or more aspects, the applying clustering techniques includes calculating a distance metric between each pair of the plurality of surgical videos, storing the calculated distance metrics in a distance ma