CN-122025118-A - Depression risk assessment method based on modal perception low rank adaptation

CN122025118ACN 122025118 ACN122025118 ACN 122025118ACN-122025118-A

Abstract

The invention discloses a depression risk assessment method based on modal perception low-rank adaptation, which comprises the following steps of firstly collecting multi-modal data of a patient and preprocessing the multi-modal data, wherein the multi-modal data comprises clinical data and MRI medical image data, secondly distributing virtual input to a missing mode to keep consistency of input dimensions, thirdly inputting the preprocessed data into a pre-trained multi-modal model and a low-rank adaptation module to splice, and injecting a modal adaptation prompt word to enhance modal perception, and fourthly outputting a depression risk assessment result through a classification head. The method can effectively solve the problem of multi-modal data missing, enhance the modal sensing capability and improve the accuracy and the robustness of depression risk assessment.

Inventors

HUANG MANLI
LIN BO
ZHOU ZHENYU
ZHAO HAOYANG

Assignees

浙江大学医学院附属第一医院(浙江省第一医院)
浙江大学

Dates

Publication Date: 20260512
Application Date: 20251229

Claims (9)

1. A depression risk assessment method based on modal perception low-rank adaptation is characterized by comprising the following steps: step one, collecting and preprocessing multi-modal data of a patient, wherein the multi-modal data comprises clinical data and MRI medical image data; step two, virtual input is distributed to the missing mode, and consistency of input dimensions is maintained; Step three, inputting the preprocessed data into a pre-trained multi-mode model and a low-rank adaptation module for splicing, and injecting a mode adaptation prompt word to enhance mode perception; And step four, outputting a depression risk assessment result through the classification head.
2. The depression risk assessment method based on modal awareness low-rank adaptation according to claim 1, wherein the specific steps of preprocessing the multi-modal data in the step one are as follows: For clinical data, performing Z-score normalization on the numerical value type features to enable the mean value to be 0 and the standard deviation to be 1, performing single-heat coding on the category type features, and extracting sentence embedded vectors of the text type features by using a pre-trained language model BERT-base model to obtain semantic features of the text type features; For MRI medical image data, the original T1 weighted MRI image is preprocessed by FreeSurfer and other tools, including intensity nonunion correction, skull peeling, gray matter/white matter/cerebrospinal fluid segmentation and brain region division based on atlas, and then the processed image is linearly registered to a standard MNI space and finally cut or interpolated to a uniform size.
3. The depression risk assessment method based on the modal awareness low-rank adaptation according to claim 1 or 2, wherein the specific steps of assigning virtual input to the missing modality in the second step and maintaining the consistency of input dimensions are as follows: If the clinical data mode is missing, setting the text input of the clinical data mode as an empty character string "", and mapping the empty character string into a specific embedded vector after the empty character string passes through the BERT embedded layer; if the MRI image mode is missing, a zero matrix with the same standard size is created as input, and after the zero matrix passes through the image embedding layer, a zero vector with the same real image embedding dimension is obtained.
4. The depression risk assessment method based on modal perception low-rank adaptation according to claim 1 or 2, wherein the specific mode of inputting the preprocessed data into the pre-trained multi-modal model and the low-rank adaptation module in the third step is as follows, firstly inputting the model, splicing the numerical characteristics of the clinical form with the text embedding vector through multi-layer perceptron processing to form an input representation text token of the clinical modality, then converting the MRI image into an image token sequence through an embedding layer, splicing the image token sequence and the image token sequence, finally adding a position code position embedding to retain sequence position information, and representing the following steps: Where [ sep ] is represented as a modality segmenter to help the model distinguish between different modality representations, and the finally obtained E is represented as a complete input sequence.
5. The depression risk assessment method based on modal awareness low-rank adaptation according to claim 4, wherein the specific mode of injecting the modal adaptation prompt word in the third step is as follows, firstly, the data after the splicing is completed is respectively input into a pre-training visual language model and a low-rank adaptation module, wherein the low-rank adaptation module firstly uses a shared lower projection matrix to extract potential representation of multi-modal information, and for each modality, a dedicated upper projection layer is provided, and the corresponding matrix is selectively activated according to the availability of the input modality, specifically: Wherein, the Represented as a modality specific upper projection layer, Represented as a shared lower projection matrix, Expressed as a modality type; Then, the complete sequence is processed by the low-rank adaptation module and then is inserted into the weight of a first layer transducer model block of the pre-training multi-mode model, and the weight is expressed as follows: Wherein, the Represented as original pre-trained model first layer tansformer weights, Represented as low-rank adaptation module weights, Expressed as a complete input sequence, finally obtained The output of the original pre-training model is fused with the modal specificity increment information learned by the low-rank adaptation module; And finally, injecting a mode adaptation prompt word into each layer of transducer module, wherein the mode adaptation prompt word is also provided with a corresponding mode exclusive prompt, namely a text prompt only, an image prompt only and a text image prompt according to the mode data owned by the patient.
6. The depression risk assessment method based on the mode-aware low-rank adaptation of claim 5, wherein the specific method for selectively activating the corresponding matrix according to the availability of the input mode in the third step is as follows: if only clinical data exists in the patient, the clinical mode adaptation matrix is obtained If the patient only has MRI data, then an MRI modality adaptation matrix is obtained If both patient text and MRI data are present, the result is the sum of the projected adaptation matrix of the two modalities 。
7. The depression risk assessment method based on the modal awareness low-rank adaptation of claim 6, wherein the specific mode of the modal adaptation prompt word injected into each layer of transducer module in the third step is as follows: Wherein X is the final output characteristic of the implanted transducer layer, D is the output characteristic of the low-rank adaptation module, and M is the modal adaptation prompt word characteristic vector.
8. The method for evaluating risk of depression based on modal awareness low-rank adaptation according to claim 1 or 2, wherein the specific steps of outputting the result of evaluating risk of depression through the classifying head in the fourth step are as follows: And inputting the representation obtained by fusing the output of the pre-training model and the modal specificity increment information learned by the low-rank adaptation module into a subsequent transducer block for feature extraction, taking the feature vector corresponding to the output [ CLS ] token as the comprehensive representation fused with the multi-modal information, and inputting the feature vector into a classifier MLP consisting of a series of full-connection layers and nonlinear activation functions to realize depression diagnosis.
9. The classification process is expressed as: 。

Description

Depression risk assessment method based on modal perception low rank adaptation Technical Field The invention relates to a depression risk assessment, in particular to a depression risk assessment method based on modal perception low-rank adaptation. Background Depression (also known as major depressive disorder) is a common mental disorder with significant social, psychological and physiological effects. According to World Health Organization (WHO) data, depression is one of the leading causes of disability worldwide and is also one of the important causes of suicide. The pathogenesis of the Chinese medicinal composition is complex, and relates to various aspects of genetics, environment, social psychology and the like. Depression is often manifested by symptoms of long-term depressed mood, reduced interest, loss of energy, sleep disorders, etc., and in severe cases, may lead to a patient losing the ability to self-care in daily life. The incidence of depression increases year by year and has a high recurrence, so early diagnosis and risk assessment are important for the prevention and treatment of depression. Existing risk assessment of depression is mainly dependent on a variety of data sources such as clinical data, psychological assessment, brain images (e.g., MRI, fMRI), genetic data, and biomarkers. By using mathematical models and algorithms, researchers can extract features of these data and predict the risk of developing depression in individuals. However, while ideal models for multimodal data fusion are expected to improve prediction accuracy, they pose significant challenges in practical applications. The most prominent limitation is that when an individual lacks one or more types of modal data (e.g., cannot acquire brain image data, or the patient does not want to fill out a scale), the generalization ability and risk assessment accuracy of a model trained based on complete data can be significantly reduced. Such "modality absence" scenarios are very common in primary medical institutions or resource limited environments, and therefore, existing risk assessment methods have certain shortcomings. Disclosure of Invention Aiming at the defects existing in the prior art, the invention aims to provide a depression risk assessment method based on modal perception low-rank adaptation, and a low-rank adaptation module of the method realizes depression diagnosis under partial modal deficiency. In order to achieve the purpose, the invention provides a depression risk assessment method based on modal perception low-rank adaptation, which comprises the following steps: step one, collecting and preprocessing multi-modal data of a patient, wherein the multi-modal data comprises clinical data and MRI medical image data; step two, virtual input is distributed to the missing mode, and consistency of input dimensions is maintained; Step three, inputting the preprocessed data into a pre-trained multi-mode model and a low-rank adaptation module for splicing, and injecting a mode adaptation prompt word to enhance mode perception; And step four, outputting a depression risk assessment result through the classification head. As a further improvement of the present invention, the specific steps of preprocessing the multimodal data in the step one of the modality-aware low-rank adaptation-based depression risk assessment method are as follows: For clinical data, performing Z-score normalization on the numerical value type features to enable the mean value to be 0 and the standard deviation to be 1, performing single-heat coding on the category type features, and extracting sentence embedded vectors of the text type features by using a pre-trained language model BERT-base model to obtain semantic features of the text type features; For MRI medical image data, the original T1 weighted MRI image is preprocessed by FreeSurfer and other tools, including intensity nonunion correction, skull peeling, gray matter/white matter/cerebrospinal fluid segmentation and brain region division based on atlas, and then the processed image is linearly registered to a standard MNI space and finally cut or interpolated to a uniform size. As a further improvement of the present invention, in the step two of the depression risk assessment method based on the mode perception low-rank adaptation, virtual input is allocated to the missing mode, and the specific steps of maintaining the consistency of input dimensions are as follows: If the clinical data mode is missing, setting the text input of the clinical data mode as an empty character string "", and mapping the empty character string into a specific embedded vector after the empty character string passes through the BERT embedded layer; if the MRI image mode is missing, a zero matrix with the same standard size is created as input, and after the zero matrix passes through the image embedding layer, a zero vector with the same real image embedding dimension is obtained. In the step three of