CN-121996908-A - Man-machine interaction dynamic control method based on multi-mode data and edge computing equipment

CN121996908ACN 121996908 ACN121996908 ACN 121996908ACN-121996908-A

Abstract

The human-computer interaction dynamic control method and the edge computing device based on the multi-mode data synchronously acquire the multi-mode data of a user in the process of using the electronic device with the human-computer interaction interface, wherein the multi-mode data comprise physiological data and behavior data, the user experience level is determined based on the physiological data and the behavior data, adjustment information of the human-computer interaction interface of the electronic device is determined based on the user experience level, and elements or interaction modes of the human-computer interaction interface of the electronic device are adjusted according to the adjustment information, so that the user interacts with the electronic device based on the adjusted human-computer interaction interface or interaction modes. According to the application, the interaction mode is dynamically adjusted based on the user experience level, so that the accurate adaptation of the interaction mode and the user use requirement is realized, and the user experience and the use efficiency of man-machine interaction are remarkably improved.

Inventors

Request for anonymity
Request for anonymity
Request for anonymity
Request for anonymity

Assignees

北京津发科技股份有限公司

Dates

Publication Date: 20260508
Application Date: 20251225

Claims (10)

1. A man-machine interaction dynamic control method based on multi-mode data is characterized by comprising the following steps: synchronously collecting multi-mode data of a user in the process of using the electronic equipment with the man-machine interaction interface, wherein the multi-mode data comprises physiological data and behavior data; determining a current user experience level of the electronic device based on the physiological data and the behavioral data; determining adjustment information of a human-computer interaction interface of the electronic equipment based on the user experience level; And adjusting elements or interaction modes of the man-machine interaction interface of the electronic equipment according to the adjustment information, so that the user interacts with the electronic equipment based on the adjusted man-machine interaction interface or interaction modes.
2. The method of claim 1, wherein the step of determining the position of the substrate comprises, Before synchronously acquiring multi-mode data of a user in the process of using the electronic equipment with the man-machine interaction interface, determining evaluation indexes of user experience levels corresponding to user attributes according to a pre-established user attribute and user experience level evaluation association relation, wherein the evaluation indexes of the user experience levels corresponding to different user attributes are different; The determining the user experience level of the electronic equipment based on the physiological data and the behavior data comprises determining the current user experience level of the electronic equipment based on the physiological data, the behavior data and an evaluation index of the user experience level corresponding to the user attribute.
3. The method of claim 2, wherein determining the current user experience level of the electronic device based on the physiological data, the behavioral data, and an evaluation indicator of the user experience level corresponding to the user attribute comprises: When the user attribute of the user is a medical staff, acquiring first physiological data and first behavior data related to cognitive load, fatigue state and concentration state, and determining the user experience level of the electronic equipment based on the first physiological data and the first behavior data; And when the user attribute of the user is a patient, acquiring second physiological data and second behavior data related to an anxiety state and an operation adaptation state, and determining the user experience level of the electronic device based on the second physiological data and the second behavior data.
4. The method of claim 1, wherein the synchronously collecting multi-modal data of the user during use of the electronic device with the human-machine interface comprises: collecting brain electricity data, electrocardio data, skin electricity activity data, eye movement data or myoelectricity data of a user in the process of using the electronic equipment by the user; and collecting the operation time or the operation error probability of the user in the operation interaction process of the user and the electronic equipment.
5. The method of claim 1, wherein the synchronously collecting the multimodal data of the user during use of the electronic device with the human-machine interface, the method further comprises: in the process of using the electronic equipment by a tester, acquiring multi-mode test data and subjective scoring data of the tester; Determining a user experience grade label based on the subjective scoring data; building training data according to the multi-mode test data and the user experience grade label; and performing model training based on the training data to obtain an experience level determining model, wherein the experience level determining model is used for determining the corresponding user experience level based on the physiological data and the behavior data.
6. The method of claim 1, wherein determining adjustment information for a human-machine interaction interface of the electronic device based on the user experience level comprises: When the attribute of the user is a medical staff, generating guide intensity adjustment information of the electronic equipment according to the user experience level; When the attribute of the user is a patient, generating simplification degree adjustment information of the electronic equipment according to the user experience level; the lower the user experience level is, the higher the guiding strength of the electronic equipment for the patient is after adjustment, and the higher the simplification degree of the electronic equipment for medical staff is.
7. The method of claim 1, wherein adjusting the elements or interactions of the human-machine interaction interface of the electronic device according to the adjustment information comprises one or more of the following combinations: the simplified adjustment of the operation interface comprises highlighting or amplifying key controls and hiding or folding non-core function menus; simplifying and adjusting the operation flow, including skipping unnecessary confirmation steps and merging multi-step operation into a single shortcut instruction; the guiding strength of the electronic equipment is improved, and the guiding strength comprises the steps of outputting guiding voice prompts, displaying dynamic visual guidance and giving out tactile warning.
8. A man-machine interaction dynamic control device based on multi-mode data is characterized by comprising: The acquisition module is used for synchronously acquiring multi-modal data of a user in the process of using the electronic equipment with the man-machine interaction interface, wherein the multi-modal data comprises physiological data and behavior data; The user experience level determining module is used for determining the current user experience level of the electronic equipment based on the physiological data and the behavior data; The adjustment information determining module is used for determining adjustment information of a man-machine interaction interface of the electronic equipment based on the user experience level; And the dynamic interaction control module is used for adjusting elements or interaction modes of the man-machine interaction interface of the electronic equipment according to the adjustment information so that the user interacts with the electronic equipment based on the adjusted man-machine interaction interface or interaction modes.
9. An edge computing device comprising a memory for storing computer program instructions and a processor for executing the program instructions, wherein the computer program instructions, when executed by the processor, cause the edge computing device to perform the method of any of claims 1 to 7.
10. A computer readable storage medium, characterized in that the computer readable storage medium comprises a stored program, wherein the program, when run, controls a device in which the computer readable storage medium is located to perform the method of any one of claims 1 to 7.

Description

Man-machine interaction dynamic control method based on multi-mode data and edge computing equipment Technical Field The application relates to the field of man-machine interaction and user experience, in particular to a man-machine interaction dynamic control method based on multi-mode data and edge computing equipment. Background With the increasing prominence of the core function of the electronic equipment in diagnosis and treatment, the man-machine interaction quality of the electronic equipment directly influences the working efficiency of medical staff, the cognitive load and the use experience of patients. However, in the prior art, single operation behavior data or independent risk management is focused, integrated collection of state data of medical staff and patients is lacking, a real-time closed-loop mechanism of 'monitoring-early warning-intervention' is not constructed, so that an interaction mode of electronic equipment configuration is disjointed from real experience of staff, and core requirements of medical scenes on safety, timeliness and full-flow man-machine interaction are difficult to meet. Disclosure of Invention In view of the above, the application provides a man-machine interaction dynamic control method, a device and an edge computing device based on multi-mode data, which are beneficial to solving the problems that the interaction mode of the electronic device in the prior art is difficult to accurately match with the user requirement and influence the user experience. In a first aspect, an embodiment of the present application provides a method for dynamically controlling human-computer interaction based on multimodal data, including: synchronously collecting multi-mode data of a user in the process of using the electronic equipment with the man-machine interaction interface, wherein the multi-mode data comprises physiological data and behavior data; determining a current user experience level of the electronic device based on the physiological data and the behavioral data; determining adjustment information of a human-computer interaction interface of the electronic equipment based on the user experience level; And adjusting elements or interaction modes of the man-machine interaction interface of the electronic equipment according to the adjustment information, so that the user interacts with the electronic equipment based on the adjusted man-machine interaction interface or interaction modes. In an optional embodiment, before the synchronous acquisition of the multi-mode data of the user in the process of using the electronic device with the man-machine interaction interface, determining an evaluation index of the user experience level corresponding to the user attribute according to a pre-established association relation between the user attribute and the user experience level evaluation, wherein the evaluation indexes of the user experience levels corresponding to different user attributes are different; The determining the user experience level of the electronic equipment based on the physiological data and the behavior data comprises determining the current user experience level of the electronic equipment based on the physiological data, the behavior data and an evaluation index of the user experience level corresponding to the user attribute. In an alternative embodiment, determining the current user experience level of the electronic device based on the physiological data, the behavior data, and the evaluation index of the user experience level corresponding to the user attribute includes: When the user attribute of the user is a medical staff, acquiring first physiological data and first behavior data related to cognitive load, fatigue state and concentration state, and determining the user experience level of the electronic equipment based on the first physiological data and the first behavior data; And when the user attribute of the user is a patient, acquiring second physiological data and second behavior data related to an anxiety state and an operation adaptation state, and determining the user experience level of the electronic device based on the second physiological data and the second behavior data. In an alternative embodiment, the step of synchronously collecting multi-mode data of a user in the process of using the electronic device with the man-machine interaction interface includes: collecting brain electricity data, electrocardio data, skin electricity activity data, eye movement data or myoelectricity data of a user in the process of using the electronic equipment by the user; and collecting the operation time or the operation error probability of the user in the operation interaction process of the user and the electronic equipment. In an alternative embodiment, the method further includes, before the synchronously collecting the multi-modal data of the user during the use of the electronic device with the man-machine interaction interface: in the p