CN-122004763-A - System and method for evaluating cognitive function by combining traditional Chinese medicine and western medicine

CN122004763ACN 122004763 ACN122004763 ACN 122004763ACN-122004763-A

Abstract

The application provides a cognitive function evaluation system and a cognitive function evaluation method for Chinese and Western medicine fusion. The method comprises the steps of collecting tongue image signals, pulse image signals, facial images, eye movement signals and brain computer oxygen signals of a tested person through a signal collecting unit, extracting features of the tongue image signals, the pulse image signals, the facial images, the eye movement signals and the brain computer oxygen signals through a signal processing unit, carrying out feature fusion on the extracted features by adopting a preset feature fusion algorithm to obtain target multi-modal fusion features, inputting the target multi-modal fusion features into a pre-trained traditional Chinese and Western medicine fusion function evaluation model, obtaining multi-dimensional cognition evaluation results output by the traditional Chinese and Western medicine fusion function evaluation model, and displaying the multi-dimensional cognition evaluation results to the tested person. Thus, the cognitive function of the tested person can be comprehensively, multi-dimensionally and quantitatively evaluated by means of artificial intelligence technology, and the cognitive disorder can be discovered early.

Inventors

ZHANG TENGYU
LI ZENGYONG
ZHANG JINGSHA
WANG RUIHAO

Assignees

国家康复辅具研究中心

Dates

Publication Date: 20260512
Application Date: 20260130

Claims (10)

1. The system for evaluating the cognitive function of the Chinese and Western medicine fusion is characterized by comprising a signal acquisition unit, a signal processing unit and an interaction unit, wherein: The signal acquisition unit is used for acquiring tongue image signals, pulse image signals, facial images, eye movement signals and brain computer oxygen signals of the tested person; The signal processing unit is used for extracting characteristics of the tongue image signal, the pulse image signal, the facial image, the eye movement signal and the brain computer oxygen signal, carrying out characteristic fusion on the extracted characteristics by adopting a preset characteristic fusion algorithm to obtain target multi-modal fusion characteristics, inputting the target multi-modal fusion characteristics into a pre-trained traditional Chinese and western medicine fusion function evaluation model, and obtaining a multi-dimensional cognition evaluation result output by the traditional Chinese and western medicine fusion function evaluation model; And the interaction unit is used for displaying the multidimensional cognitive assessment result to the tested person.
2. The cognitive function assessment system according to claim 1, wherein the signal acquisition unit comprises a tongue image acquisition module, the tongue image acquisition module comprises an illumination unit, an image acquisition unit and an acquisition barrel, the illumination unit is a plurality of LED lamps distributed on the inner wall of the acquisition barrel, and the image acquisition unit is fixed at the bottom of the acquisition barrel and is used for continuously acquiring tongue images of the tested person to obtain the tongue image.
3. The cognitive function assessment system according to claim 1, wherein the signal acquisition unit comprises a pulse condition acquisition module, the pulse condition acquisition module comprises an inflatable wristband, array piezoelectric sensors are uniformly distributed in the inflatable wristband, each piezoelectric sensor covers the on-size, off-size and under-size parts, and the pulse condition acquisition module is used for acquiring a fluctuation signal of pulse waves of each part as the pulse condition.
4. The cognitive function assessment system according to claim 3, wherein the inflatable wristband includes an inflation/deflation device inside for continuously inflating/deflating in accordance with a preset air pressure range in response to a measured pulse condition instruction, so that the array piezoelectric sensor acquires the fluctuation signals of the pulse waves at different air pressures.
5. The cognitive function assessment system according to claim 1, wherein the signal acquisition unit comprises a desktop image acquisition device, the desktop image acquisition device is connected with the interaction unit, a position calibration module is preset in the interaction unit, and the position calibration module is used for generating and displaying position prompt information according to the face image of the person to be tested acquired by the desktop image acquisition device.
6. The cognitive function assessment system according to claim 1, wherein the signal acquisition unit comprises an eye movement acquisition module, the eye movement acquisition module comprises an infrared light source, a high-speed camera and an eye movement track analysis module, the eye movement track analysis module is used for acquiring an eye pupil image acquired by the high-speed camera in an infrared light source on state, and determining an eye movement track as the eye movement signal based on the eye pupil image.
7. The cognitive function assessment system according to claim 1, wherein the signal acquisition unit comprises an electroencephalogram signal acquisition electrode, an electroencephalogram signal acquisition probe, a near infrared light source and a channel positioning device, wherein each electroencephalogram signal acquisition electrode is detachably fixed on a reserved fixed hole site of the channel positioning device through a buckle, one channel positioning device corresponds to one electroencephalogram signal acquisition channel, and one electroencephalogram signal acquisition channel comprises one electroencephalogram signal acquisition electrode, one electroencephalogram signal acquisition probe and one near infrared light source; the electroencephalogram signal acquisition electrode is used for acquiring electroencephalogram signals of the head of the tested person; the brain oxygen signal acquisition probe is used for acquiring an optical signal of the head of the person to be tested in the state that the near infrared light source is started, and converting the optical signal into a hemoglobin concentration change signal.
8. The cognitive function assessment system of claim 1, wherein the signal processing module is specifically configured to: Extracting the characteristics of the tongue picture signals according to a preset tongue picture characteristic extraction algorithm to obtain tongue picture characteristics ; Extracting the characteristics of the pulse signals according to a preset pulse characteristic extraction algorithm to obtain pulse characteristics ; Extracting features of the facial image according to a preset facial feature extraction algorithm to obtain facial features ; Extracting features of the facial image according to a preset facial expression feature extraction algorithm to obtain facial expression features ; Extracting the characteristics of the eye movement signals according to a preset eye movement characteristic extraction algorithm to obtain eye movement characteristics ; Extracting the characteristics of the brain computer oxygen signal according to a preset brain function characteristic extraction algorithm to obtain brain function characteristics 。
9. The cognitive function assessment system of claim 8, wherein a cross-modal attention network fusion model is preset in the signal processing unit, the cross-modal attention network fusion model comprises a linear projection layer or a full connection layer, and the signal processing unit is specifically configured to: Characterizing the tongue by the linear projection layer or the full connection layer Pulse condition characteristics Characteristic of facial image Facial expression features Eye movement characteristics Brain function characterization Mapping to the same feature dimension to construct a joint feature representation J; Using the joint characteristic representation J to respectively and uniformly match the tongue characteristic Pulse condition characteristics Characteristic of facial image Facial expression features Eye movement characteristics Brain function characterization Calculating a tongue image joint cross-correlation matrix, a pulse image joint cross-correlation matrix, a surface image joint cross-correlation matrix, a facial expression joint cross-correlation matrix, an eye movement joint cross-correlation matrix and a brain function joint cross-correlation matrix; Feature extraction is carried out on each joint cross-correlation matrix through attention force diagram and feature enhancement, and fusion tongue image features are generated Fusing pulse characteristics Fusion of facial image features Fusion of facial expression features Fusion of eye movement features Fusion of brain functional features ; Respectively for the characteristics of the fusion tongue Fusing pulse characteristics Fusion of facial image features Fusion of facial expression features Fusion of eye movement features Fusion of brain functional features Performing N rounds of recursion iteration to obtain enhanced tongue image features, enhanced pulse image features, enhanced facial expression features, enhanced eye movement features and enhanced brain function features; And splicing the tongue image enhancing feature, the pulse image enhancing feature, the facial expression enhancing feature, the eye movement enhancing feature and the brain function enhancing feature to obtain the target multi-mode fusion feature.
10. A method for evaluating cognitive function by combining traditional Chinese medicine and western medicine, wherein the method is applied to the cognitive function evaluation system as claimed in any one of claims 1 to 9, and the method comprises: collecting tongue image signals, pulse image signals, facial images, eye movement signals and brain computer oxygen signals of a tested person; Extracting features of the tongue image signals, the pulse image signals, the facial image, the eye movement signals and the brain computer oxygen signals, carrying out feature fusion on the extracted features by adopting a preset feature fusion algorithm to obtain target multi-modal fusion features, and inputting the target multi-modal fusion features into a pre-trained Chinese and Western medicine fusion function evaluation model; and acquiring a multidimensional cognitive assessment result output by the Chinese and Western medicine fusion function assessment model, and displaying the multidimensional cognitive assessment result to the tested person.

Description

System and method for evaluating cognitive function by combining traditional Chinese medicine and western medicine Technical Field The application relates to the technical field of medical diagnosis, in particular to a cognitive function evaluation system and method for Chinese and Western medicine fusion. Background The course of senile dementia is irreversible, but cognitive function is often reduced for a long time before dementia occurs, if cognitive function can be evaluated early, cognitive dysfunction can be found early, so that the development of cognitive dysfunction can be prevented and intervened early, the development of cognitive dysfunction can be delayed and even reversed, and the incidence of dementia is effectively reduced. Meanwhile, the number of cognitive dysfunction people caused by various diseases such as cerebral apoplexy, cerebral palsy, autism and the like is large, and the comprehensive quantitative evaluation of the cognitive function is also a necessary means for knowing the functional level of a patient, making a rehabilitation training scheme and evaluating the rehabilitation effect. At present, the cognitive function evaluation is mainly based on a scale, the evaluation is long in time consumption and high in subjectivity, and depends on professional evaluation staff, so that early screening of cognitive impairment is very inconvenient for some penmanship institutions and communities lacking professional evaluation staff. Therefore, there is a great need for a simple, easy to operate, objective and quantitative evaluation system for evaluation screening. Most of the existing cognitive function evaluation systems are products for simply electronizing various scales and performing cognitive evaluation with a small amount of assistance of eye movement indexes, but the indexes are relatively single, and the function evaluation is not comprehensive enough. Disclosure of Invention In view of the above, the embodiment of the application provides a system and a method for evaluating cognitive functions by combining traditional Chinese medicine and western medicine, which are used for evaluating the cognitive functions of a tested person in a multi-index manner by combining traditional Chinese medicine and western medicine. In a first aspect, an embodiment of the present application provides a system for evaluating cognitive functions by integrating traditional Chinese medicine and western medicine, where the system includes a signal acquisition unit, a signal processing unit, and an interaction unit, where: The signal acquisition unit is used for acquiring tongue image signals, pulse image signals, facial images, eye movement signals and brain computer oxygen signals of the tested person; The signal processing unit is used for extracting characteristics of the tongue image signal, the pulse image signal, the facial image, the eye movement signal and the brain computer oxygen signal, carrying out characteristic fusion on the extracted characteristics by adopting a preset characteristic fusion algorithm to obtain target multi-modal fusion characteristics, inputting the target multi-modal fusion characteristics into a pre-trained traditional Chinese and western medicine fusion function evaluation model, and obtaining a multi-dimensional cognition evaluation result output by the traditional Chinese and western medicine fusion function evaluation model; And the interaction unit is used for displaying the multidimensional cognitive assessment result to the tested person. In a second aspect, an embodiment of the present application provides a method for evaluating cognitive function by combining a traditional Chinese medicine and a western medicine, where the method is applied to the cognitive function evaluating system in the first aspect, and the method includes: collecting tongue image signals, pulse image signals, facial images, eye movement signals and brain computer oxygen signals of a tested person; and extracting features of the tongue image signals, the pulse image signals, the facial image, the eye movement signals and the brain computer oxygen signals, carrying out feature fusion on the extracted features by adopting a preset feature fusion algorithm to obtain target multi-modal fusion features, inputting the target multi-modal fusion features into a Chinese and Western medicine fusion function evaluation model obtained by training in advance, obtaining a multi-dimensional cognition evaluation result output by the Chinese and Western medicine fusion function evaluation model, and displaying the multi-dimensional cognition evaluation result to a tested person. In a third aspect, an embodiment of the present application provides an electronic device, where the electronic device includes a processor and a memory storing a program, where the program includes instructions that when executed by the processor cause the processor to perform the method for evaluating cognitive functions o