Search

CN-122024982-A - Surgical procedure recording and instruction interaction system based on multi-mode biological feature verification

CN122024982ACN 122024982 ACN122024982 ACN 122024982ACN-122024982-A

Abstract

The application discloses a surgical procedure recording and instruction interaction system based on multi-mode biological feature verification. The system includes a multi-channel microphone array for collecting audio signals within an operating room environment, a biometric sensor for collecting biometric information of a physician, and a surgical interaction and verification processor. The processor is configured to process the audio signal to generate a physician voice stream of high signal-to-noise ratio, convert the voice stream into a text stream in real-time and perform natural language understanding to process and form one or more structured data items, and upon receipt of a commit trigger instruction or activation of a biometric sensor, perform non-repudiation verification and encryption signing of the one or more structured data items based on the biometric information to generate a signed data block. The application solves the technical problems of operation record delay, information distortion and weak evidence chain in the prior art.

Inventors

  • SHAO YAJUAN
  • LIN XIANPIN
  • ZHANG YIJIN

Assignees

  • 浙江大学

Dates

Publication Date
20260512
Application Date
20260129

Claims (10)

  1. 1. A surgical procedure recording and instruction interaction system based on multi-modal biometric verification, comprising: A multi-channel microphone array configured to collect multiple raw audio streams within an operating room environment; a biometric sensor configured to collect biometric data of a primary doctor; And, a surgical interaction and verification processor configured to: Performing spatial filtering processing on the multiple paths of original audio streams, so as to generate a path of doctor voice stream with high signal-to-noise ratio; performing real-time voice-to-text processing and natural language understanding processing on the high signal-to-noise ratio physician voice stream to divide the physician voice stream into a text containing process description and a text containing executable instructions, and processing the text to form one or more data items to be signed; and in response to a commit triggering event, associating the one or more data items to be signed with the biometric data collected by the biometric sensor and performing a cryptographic signing operation to generate a non-tamperable signed data block containing a timestamp.
  2. 2. The system of claim 1, wherein the surgical interaction and verification processor is a heterogeneous system-on-chip physically integrated with: A digital signal processor core configured to perform the spatial filtering process; a central processor core cluster configured to perform the real-time speech-to-text processing and the natural language understanding processing; And a security co-processor configured to perform the cryptographic signing operation in a physically isolated environment.
  3. 3. The system of claim 2, wherein the heterogeneous system on a chip further integrates a state-gated asynchronous data bus configured to connect the central processor core cluster with the security co-processor.
  4. 4. The system of claim 3, wherein the state-gated asynchronous data bus is configured to operate in two mutually exclusive operating states: a record state in which the central processor core cluster is allowed to write the one or more data entries to be signed into a shared temporary cache; and a commit state in which write permissions to the temporary cache are locked by hardware logic and a one-time, uninterrupted transfer of data blocks is performed to transfer the entire contents of the temporary cache as one data snapshot to the security co-processor.
  5. 5. The system of claim 1, wherein the commit triggering event includes at least one of: The natural language understanding process of the surgical interaction and verification processor recognizes a preset submitting trigger voice command; Or, the biometric sensor is activated by a specific physical action of the primary physician.
  6. 6. The system of claim 1, wherein the biometric sensor is an iris scanner or a voiceprint acquisition microphone.
  7. 7. The system of claim 2, wherein the security co-processor is further configured to: Before the encryption signature operation is executed, comparing and verifying the biological characteristic data acquired in real time with a prestored biological characteristic template; and continuing to execute the encryption signature operation only if the comparison verification is successful.
  8. 8. The system of claim 1, wherein the signed data block comprises a data content portion and a cryptographically signed portion, wherein the data content portion comprises the one or more data items to be signed, the timestamp, and a verifier identity, and wherein the cryptographically signed portion is generated by applying a cryptographic hash algorithm to the data content portion.
  9. 9. The system of claim 1, wherein the system further comprises: a display unit configured to present text converted by the physician voice stream and the one or more data items to be signed in real-time for immediate confirmation by the physician of the primary knife, prior to execution of the cryptographic signing operation.
  10. 10. A surgical procedure recording and instruction interaction system based on multi-modal biometric verification, comprising: A spatial filtering module configured to generate a high signal-to-noise ratio physician voice stream based on multiple original audio streams acquired from a multi-channel microphone array; a semantic understanding and structuring module configured to convert the high signal-to-noise ratio physician voice stream into structured data items, the structured data items being distinguished into process descriptive text and executable instruction text; And a secure signing and curing module configured to perform a cryptographic signing operation on the structured data item based on biometric data of the primary doctor collected from a biometric sensor in response to a commit trigger event to generate a tamper-resistant signed data block containing a time stamp.

Description

Surgical procedure recording and instruction interaction system based on multi-mode biological feature verification Technical Field The application relates to the technical field of medical information, in particular to a surgical procedure recording and instruction interaction system based on multi-mode biological feature verification. Background In modern surgical practice, an exhaustive and accurate surgical record is critical to ensure medical quality, meet legal and regulatory requirements, and conduct subsequent clinical studies. Currently, the generation of surgical records relies primarily on the physician of the primary doctor writing with memory after surgery, or by the patrolling nurse assisting in the recording during surgery. The conventional mode has a plurality of inherent technical problems that firstly, post-operation recall type records are easy to distort information or miss key details caused by memory deviation, secondly, the manual records in operation can be distracted by auxiliary personnel, each oral command and key decision issued by a doctor are difficult to capture completely in real time, and finally, confirmation and signing of the records are usually completed after a few hours or even longer after operation, so that evidence links have a remarkable delay in time. Disclosure of Invention The application aims to provide a surgical procedure recording and instruction interaction system based on multi-mode biological feature verification, which aims to solve the technical problems of delay, high risk of information distortion and weak legal evidence chain in the surgical procedure recording in the prior art. In a first aspect, the application provides a surgical procedure recording and instruction interaction system based on multi-modal biometric verification, comprising a multi-channel microphone array configured to collect multiple paths of raw audio streams within an operating room environment, a biometric sensor configured to collect biometric data of a primary medical practitioner, and a surgical interaction and verification processor configured to spatially filter the multiple paths of raw audio streams to generate a high signal-to-noise ratio physician voice stream, to perform real-time voice-to-text processing and natural language understanding processing on the high signal-to-noise ratio physician voice stream to distinguish the physician voice stream into text containing a procedure description and text containing executable instructions and to form one or more data entries to be signed after the foregoing text processing, and to correlate the one or more data entries to be signed with the biometric data collected by the biometric sensor in response to a commit trigger event and to perform an encrypted signature operation to generate a tamper-proof signed data block containing a timestamp. Optionally, the surgical interactive and verification processor is a heterogeneous system-on-chip physically integrated with a digital signal processor core configured to perform the spatial filtering process, a central processor core cluster configured to perform the real-time voice-to-text process and the natural language understanding process, and a security co-processor configured to perform the cryptographic signing operation in a physically isolated environment. Optionally, the heterogeneous system-on-chip further integrates a state-gated asynchronous data bus configured to connect the central processor core cluster with the security co-processor. Optionally, the state-gated asynchronous data bus is configured to operate in two mutually exclusive operating states, a transcription state in which the central processor core cluster is allowed to write the one or more data entries to be signed into a shared temporary cache, and a commit state in which write permissions to the temporary cache are locked by hardware logic and a one-time, uninterrupted transfer of data blocks is performed to transfer the entire contents of the temporary cache as one data snapshot to the security co-processor. Optionally, the submission triggering event includes at least one of the natural language understanding process of the surgical interaction and verification processor recognizing a preset submission triggering voice command or the biometric sensor being activated by a specific physical action of the primary physician. Optionally, the biometric sensor is an iris scanner or a voiceprint acquisition microphone. Optionally, the security co-processor is further configured to compare the biometric data collected in real-time with a pre-stored biometric template prior to performing the cryptographic signature operation, and to continue performing the cryptographic signature operation only if the comparison verification is successful. Optionally, the signed data block comprises a data content part and a cryptographic signature part, wherein the data content part comprises the one or more data items to be