Search

CN-121998495-A - Method and system for comprehensive clinical competence assessment of medical students

CN121998495ACN 121998495 ACN121998495 ACN 121998495ACN-121998495-A

Abstract

The application belongs to the technical field of medical education, and discloses a method and a system for evaluating comprehensive clinical competence of medical students, which are characterized in that video and audio data in the interaction process of the medical students and virtual patients are collected, a response text is obtained by combining a voice recognition technology, and the medical student response text is subjected to semantic alignment with an authoritative medical knowledge base by a search enhancement generation technology; the method comprises the steps of evaluating language definition, co-emotion performance, attitude performance, eye contact rate and expression liveness of a medical student based on multi-modal data, finally obtaining comprehensive scores through weighting and fusing various quantitative indexes, generating a feedback report, and encrypting and storing original data, so that the problems of mutual cutting of expertise and clinical communication ability evaluation, strong evaluation subjectivity, lack of objective quantitative indexes and insufficient non-language communication evaluation existing in the medical student evaluation in the current medical education system are solved.

Inventors

  • YAN JIE
  • ZHANG LU

Assignees

  • 中国人民解放军陆军军医大学

Dates

Publication Date
20260508
Application Date
20260121

Claims (10)

  1. 1. A method for comprehensive clinical competence assessment of a medical student, the method comprising the steps of: A1. Collecting video data and audio data in the interaction process of medical students and virtual patients, and carrying out voice recognition on the audio data to obtain a medical student response text; A2. based on a search enhancement generation technology, carrying out semantic alignment on the medical response text and an authoritative medical knowledge base, and calculating knowledge accuracy according to an alignment result; A3. based on the video data, the audio data and the medical student response text, evaluating the communication performance of the medical students to obtain communication performance indexes, wherein the communication performance indexes comprise language definition, co-emotion performance indexes, attitude performance indexes, eye contact rate and expression liveness; A4. weighting and fusing all the quantization indexes to obtain comprehensive scores, wherein the quantization indexes comprise the knowledge accuracy and all the communication performance indexes; A5. And generating a feedback report containing dimension scores, time sequence positioning and learning suggestions according to the comprehensive scores and the quantization indexes, and encrypting and storing original data, wherein the original data comprises the video data and the audio data.
  2. 2. The method for comprehensive clinical competence assessment of medical students according to claim 1, wherein step A2 comprises: A201. The dialogue scene information comprises question type information of a virtual patient and department type information of the medical student; A202. Carrying out named entity identification and key point extraction on the medical response text to obtain medical entities and key points in the medical response text; A203. based on the dialogue scene information, the medical entity and the key points, searching Top-K authoritative documents with highest relativity from the authoritative medical knowledge base by utilizing a search enhancement generation technology, and extracting knowledge points in the searched authoritative documents, wherein the Top-K is a preset positive integer; A204. comparing the key points with the knowledge points, and respectively calculating the coverage degree of the medical response text on the knowledge points and the matching degree of the key points to obtain semantic coverage rate and F1 score; A205. and carrying out weighted fusion on the semantic coverage rate and the F1 score to obtain knowledge accuracy.
  3. 3. The method for comprehensive clinical competence assessment of a medical student according to claim 2, wherein step a204 comprises: Acquiring semantic vectors of the key points and semantic vectors of the knowledge points; Calculating the semantic similarity between the semantic vector of each key point and the semantic vector of each knowledge point; For each key point, determining the matching state of the key point according to the semantic similarity between the semantic vector of the key point and the semantic vector of each knowledge point and a preset similarity threshold; The matching states of the key points are synthesized, and the F1 score is calculated; For each knowledge point, judging whether the semantic similarity between the knowledge points and the medical response text is not smaller than the key point of the preset similarity threshold value, and determining the coverage state of the knowledge point according to the judging result; and calculating the semantic coverage rate according to the coverage state of each knowledge key point.
  4. 4. The method for comprehensive clinical competence assessment of medical students according to claim 1, wherein step A3 comprises: A301. calculating language definition according to the medical response text; A302. extracting acoustic features according to the audio data, and calculating a co-emotion expression index and an attitude expression index based on the acoustic features; A303. and extracting non-linguistic visual features according to the video data, and calculating eye contact rate and expression liveness based on the non-linguistic visual features.
  5. 5. The method for comprehensive clinical competence assessment of a medical student according to claim 4, wherein step a301 comprises: respectively carrying out term identification, syntax analysis and syntax structure analysis on the medical response text to obtain term ratio, average sentence length and clause nesting ratio; Carrying out standardization processing on the term ratio, the average sentence length and the clause nesting ratio to obtain a standardized term ratio, a standardized average sentence length and a standardized clause nesting ratio; carrying out weighted fusion on the normalized term ratio, the normalized average sentence length and the normalized clause nesting ratio to obtain language definition; And generating a language expression improvement prompt when the term ratio exceeds a preset ratio threshold or the average sentence length exceeds a preset length threshold.
  6. 6. The method for integrated clinical competence assessment of a medical student of claim 4, wherein step a302 comprises: Extracting acoustic features according to the audio data, wherein the acoustic features comprise at least one of fundamental frequency, energy, speech speed, pause ratio, breaking rate and response time delay; and calculating the co-situation expression index and the attitude expression index by utilizing a pre-trained classification model or regression model based on the acoustic characteristics.
  7. 7. The method for integrated clinical competence assessment of a medical student of claim 4, wherein step a303 comprises: Extracting non-linguistic visual features from the video data, wherein the non-linguistic visual features comprise face key points, head gestures and sight line directions of the medical students; Acquiring camera calibration parameters; Determining whether a line-of-sight contact state is present between the medical student and the virtual patient based on the camera calibration parameters, the head pose, and the line-of-sight direction; Counting the duration of time that the medical student is in line-of-sight contact with the virtual patient; Acquiring the total interaction time of the interaction process of the medical student and the virtual patient, and calculating the ratio of the duration to the total interaction time to obtain the eye contact rate; And calculating the expression change intensity of the medical student based on the motion amplitude of the face key points to obtain the expression liveness.
  8. 8. The method for comprehensive clinical competence assessment of medical students according to claim 2, further comprising the step of, prior to step A4: A6. Triggering low confidence evaluation processing when the semantic coverage rate, the confidence level of voice recognition or the quality of the original data is lower than a corresponding preset threshold value, wherein the low confidence evaluation processing comprises index degradation processing or adjustment of virtual patient speech, and the index degradation processing is used for reducing quantization indexes used for calculating comprehensive scores.
  9. 9. The method for comprehensive clinical competence assessment of medical students according to claim 1, wherein the comprehensive score is calculated in step A4 using a regression model trained based on gold standard data cross-validation; The regression model is: tcs=a kas+b lci+c EPI LCI +. C EPI; Wherein TCS is the comprehensive score, KAS is the knowledge accuracy, LCI is the language definition, EPI is the co-emotion expression index, API is the attitude expression index, ECR is the eye contact rate, FEI is the expression liveness, a, b, c, d, e, f is a weight factor.
  10. 10. The system for evaluating comprehensive clinical competence of medical students is characterized by comprising a student end, a teacher end and a server; the student end is used for providing an interaction interface for interaction with the virtual patient for the medical student, collecting video data and audio data in the interaction process of the medical student and the virtual patient, and uploading the video data and the audio data to the server; the server is configured with: the voice recognition module is used for carrying out voice recognition on the audio data to obtain a medical response text; The knowledge accuracy evaluation module is used for carrying out semantic alignment on the medical response text and the authoritative medical knowledge base based on the search enhancement generation technology, and calculating knowledge accuracy according to an alignment result; The communication performance evaluation module is used for evaluating the communication performance of the medical students based on the video data, the audio data and the medical response text to obtain communication performance indexes, wherein the communication performance indexes comprise language definition, co-emotion performance indexes, attitude performance indexes, eye contact rate and expression liveness; The comprehensive scoring module is used for carrying out weighted fusion on each quantization index to obtain a comprehensive score, wherein the quantization index comprises the knowledge accuracy and each communication performance index; the first feedback module is used for generating a feedback report containing dimension scores, time sequence positioning and learning suggestions according to the comprehensive scores and the various quantization indexes, encrypting and storing original data and sending the feedback report to the student end, wherein the original data comprises the video data and the audio data; And the second feedback module is used for generating feedback information containing the weak point information of the medical student and teaching strategy suggestions according to the comprehensive scores and the quantitative indexes and sending the feedback information to the teacher end.

Description

Method and system for comprehensive clinical competence assessment of medical students Technical Field The application relates to the technical field of medical education, in particular to a method and a system for comprehensive clinical competence assessment of medical students. Background In the current medical education system, the problem of mutual cleavage between professional knowledge and clinical communication capability assessment is common in an assessment system of medical students. The examination of the expertise is mainly performed through traditional modes of pen test, oral test and the like, and focuses on the memory and understanding of 'hard knowledge' of medical theory, disease diagnosis, treatment scheme and the like. Assessment of communication capabilities is often performed independently of expertise assessment, typically by specific stations in Objective Structured Clinical Exams (OSCE). This split assessment model results in a fundamental disadvantage that it does not truly reflect the comprehensive capabilities that a physician needs in clinical practice. In a real medical scenario, each interaction of a doctor with a patient is a high fusion of expertise and communication skills. The existing separation type assessment method cannot effectively measure the comprehensive capacity, so that deviation exists between the assessment result and the actual clinical competence of students. In addition, the existing medical student assessment method has the problems of remarkable subjectivity and lack of objective quantitative indexes, especially in the aspect of communication capability assessment. In conventional assessment modes, whether teacher observation, standardized Patient (SP) feedback, or peer assessment, the assessment results depend largely on the personal experience, expertise level, and subjective judgment of the evaluator. Different evaluators may have great difference in evaluation of the same communication behavior, so that reliability and effectiveness of evaluation results are questioned. The subjective evaluation mode can not only provide accurate and operable improvement suggestions for students, but also hardly establish uniform and comparable teaching quality standards among teachers and institutions. Meanwhile, non-language communication, including facial expression, eye contact, body posture, gestures, intonation, and the like, plays a vital role in doctor-patient communication. However, in existing medical student assessment systems, assessment of non-verbal communication capabilities is often ignored or streamed. Traditional evaluation methods rely primarily on visual inspection by the evaluator, which is not only inefficient, but also difficult to capture fine, transient, non-verbal signals. Because of the lack of objective quantitative criteria, it is difficult for an evaluator to accurately describe and evaluate the non-verbal communication performance of a student, and often only general feedback can be given, and specific and operable improved guidance cannot be provided for the student. In view of the above, there is a need in the art for improvements. Disclosure of Invention The application aims to provide a method and a system for comprehensive clinical competence assessment of medical students, and aims to solve the problems of mutual cleavage of professional knowledge and clinical communication ability assessment, strong subjectivity of assessment, lack of objective quantitative indexes and insufficient non-language communication assessment in the current medical education system. In a first aspect, the present application provides a method for comprehensive clinical competence assessment for medical students, the method comprising the steps of: A1. Collecting video data and audio data in the interaction process of medical students and virtual patients, and carrying out voice recognition on the audio data to obtain a medical student response text; A2. based on a search enhancement generation technology, carrying out semantic alignment on the medical response text and an authoritative medical knowledge base, and calculating knowledge accuracy according to an alignment result; A3. based on the video data, the audio data and the medical student response text, evaluating the communication performance of the medical students to obtain communication performance indexes, wherein the communication performance indexes comprise language definition, co-emotion performance indexes, attitude performance indexes, eye contact rate and expression liveness; A4. weighting and fusing all the quantization indexes to obtain comprehensive scores, wherein the quantization indexes comprise the knowledge accuracy and all the communication performance indexes; A5. And generating a feedback report containing dimension scores, time sequence positioning and learning suggestions according to the comprehensive scores and the quantization indexes, and encrypting and storing original d