CN-121971109-A - Action instruction generation method and system, electronic device and storage medium

CN121971109ACN 121971109 ACN121971109 ACN 121971109ACN-121971109-A

Abstract

The embodiment of the invention discloses a method and a system for generating action instructions, electronic equipment and a storage medium. The method comprises the steps of generating a first target instruction for controlling the peripheral auxiliary equipment to execute a first target action based on an electroencephalogram signal acquired by the electroencephalogram signal acquisition equipment aiming at an auxiliary object when a first target visual stimulus identifier is observed, identifying a specific action based on an electromyogram signal acquired by the electromyogram signal acquisition equipment when the auxiliary object executes the specific action, generating a second target instruction for controlling the peripheral auxiliary equipment to execute a second target action according to a corresponding relation between a plurality of pre-configured specific actions and the plurality of target actions, and controlling the movement of the peripheral auxiliary equipment to execute a combined action according to the first target instruction and the second target instruction. The proposal can stimulate the auxiliary object to generate the brain electrical signal by means of visual stimulation identification, and can determine the action required to be executed by the peripheral auxiliary equipment in a short time by utilizing the electromyographic signal of the auxiliary object.

Inventors

ZHANG RUIZHONG

Assignees

北京镁伽机器人科技有限公司

Dates

Publication Date: 20260505
Application Date: 20251224

Claims (11)

1. The motion instruction generation method is characterized by being applied to a control device of peripheral auxiliary equipment, wherein the control device is used for being connected with an electroencephalogram signal acquisition device and an electromyogram signal acquisition device respectively, the electroencephalogram signal acquisition device is used for acquiring electroencephalogram signals of an auxiliary object, the electromyogram signal acquisition device is used for acquiring electromyogram signals of the auxiliary object, the control device is connected with a display device, the display device is used for displaying a plurality of visual stimulus identifiers corresponding to target motions one by one, and flicker frequencies of the visual stimulus identifiers corresponding to different target motions are different; The method comprises the following steps: Determining a flicker frequency of a first target visual stimulus identifier of the electroencephalogram signal generated by visual stimulus for an auxiliary object based on the electroencephalogram signal acquired by the electroencephalogram signal acquisition device when the first target visual stimulus identifier is observed, so as to determine a first target action corresponding to the first target visual stimulus identifier, and generating a first target instruction for controlling the peripheral auxiliary device to execute the first target action; Identifying a specific action based on the electromyographic signals acquired by the electromyographic signal acquisition device when the auxiliary object executes the specific action, determining a second target action corresponding to the identified specific action according to the corresponding relation between a plurality of preset specific actions and a plurality of target actions, and generating a second target instruction for controlling the peripheral auxiliary device to execute the second target action; and controlling the motion of the peripheral auxiliary equipment to execute the combined action of the first target action and the second target action according to the first target instruction and the second target instruction.
2. The method of claim 1, wherein identifying the particular action comprises: identifying an action type and an action parameter of the specific action; And determining a second target action corresponding to the identified specific action according to the corresponding relation between the preset specific actions and the target actions, wherein the second target action comprises the following steps: Determining a target action set corresponding to the identified action type according to the corresponding relation between a plurality of preset action types and a plurality of target action sets; And selecting the action in the target action set as the second target action according to the identified action parameter.
3. The method of claim 2, wherein the first target action and the specific action each belong to the plurality of action sets; For each target action set of the plurality of target action sets, the target action set can be associated to other target action sets except the target action set through the plurality of action types, and the plurality of action types and the other target action sets have a one-to-one correspondence; According to the corresponding relation between a plurality of pre-configured action types and a plurality of target action sets, determining the target action set corresponding to the identified action type comprises the following steps: Determining a first target action set where the first target action is located, displaying a plurality of target action sets associated with the first target action set and action types respectively corresponding to each target action set, and determining a second target action set associated with the first target action set according to the identified action types, wherein the second target action set is a target action set for determining the second target action subsequently.
4. The method of claim 2, wherein the actions within each target action set are preconfigured with a correspondence to action parameters, the action parameters comprising at least one of: The number of actions, the amplitude of actions, and the frequency of actions.
5. The method of claim 1, wherein the electromyographic signals are those corresponding to a plurality of muscle groups, And identifying a specific action based on the electromyographic signals acquired by the electromyographic signal acquisition device when the auxiliary object performs the specific action, comprising: And determining the second target action according to the energy of the electromyographic signals of each muscle group and a preset mapping relation between preconfigured electromyographic signal energy and a plurality of action types, wherein the preset mapping relation comprises a plurality of possible comparison results between the energy of at least two electromyographic signals and a one-to-one correspondence between the plurality of action types.
6. The method of claim 1, wherein the controlling the motion of the peripheral auxiliary device according to the first and second target instructions to perform the combined action of the first and second target actions comprises: Generating control information according to an execution strategy corresponding to an action combination formed by the first target action and the second target action, the first target instruction and the second target instruction, so that a processor of peripheral auxiliary equipment controls the peripheral auxiliary equipment to move according to the control instruction corresponding to the control information; the control device is a processor of the peripheral auxiliary equipment or is connected with the processor of the peripheral auxiliary equipment.
7. The method according to claim 1, wherein the method further comprises: And marking the first target visual stimulus identifier corresponding to the first target action and the second target visual stimulus identifier corresponding to the second target action, wherein the marking mode of the first target visual stimulus identifier is different from the marking mode of the second target visual stimulus identifier.
8. The method of claim 2, wherein the step of determining the position of the substrate comprises, The visual stimulus identifiers are divided into a plurality of groups of visual stimulus identifiers according to the set dividing mode of the plurality of specific actions, and target action sets corresponding to the groups of visual stimulus identifiers can be associated to target action sets corresponding to other groups of visual stimulus identifiers except the group of visual stimulus identifiers through the plurality of action types according to any one of the groups of visual stimulus identifiers, and the plurality of types and the target action sets corresponding to the other groups of visual stimulus identifiers have a one-to-one correspondence; Displaying prompt information of action types adopted by the target action sets corresponding to one group of visual stimulus identifiers of the multiple groups of visual stimulus identifiers to other target action sets corresponding to other groups of visual stimulus identifiers respectively on the display device, wherein the prompt information is used for prompting action categories corresponding to the second target instructions of the auxiliary object; The action parameters comprise action times, and any target action corresponding to any one of the plurality of groups of visual stimulus identifiers can be respectively associated to each target action corresponding to any one of the other groups of visual stimulus identifiers through the plurality of action times; The multiple sets of visual stimulus identifiers each have a corresponding display area on the display device, and each visual stimulus identifier in each group of the plurality of groups of visual stimulus identifiers is displayed in a preset sequence in the display area according to the corresponding action times.
9. The action instruction generation system is characterized by being applied to a control device of peripheral auxiliary equipment, wherein the control device is used for being connected with an electroencephalogram signal acquisition device and an electromyogram signal acquisition device respectively, the electroencephalogram signal acquisition device is used for acquiring electroencephalogram signals of an auxiliary object, the electromyogram signal acquisition device is used for acquiring electromyogram signals of the auxiliary object, the control device is connected with a display device, the display device is used for displaying a plurality of visual stimulus identifiers corresponding to target actions one by one, and flicker frequencies of the visual stimulus identifiers corresponding to different target actions are different; the action instruction generation system is configured to perform the action instruction generation method according to any one of claims 1 to 8.
10. An electronic device comprising a processor and a memory, wherein the memory has stored therein computer program instructions which, when executed by the processor, are adapted to carry out the action instruction generating method of any of claims 1-8.
11. A storage medium having stored thereon program instructions for performing the action instruction generating method according to any of claims 1-8 when run.

Description

Action instruction generation method and system, electronic device and storage medium Technical Field The invention relates to the field of medical rehabilitation, in particular to a method and a system for generating action instructions, electronic equipment and a storage medium. Background In recent years, with the development of robotics and communication technologies, a control method for controlling peripheral auxiliary devices based on Brain-computer interface (BCI) -Computer Interface-is increasingly applied to the field of medical rehabilitation. For some patients losing limb movement function due to limb injury, the control system based on the brain-computer interface can help the patients to control actions of peripheral auxiliary equipment such as wheelchairs, mechanical arms, exoskeletons and the like, and the patients can perform rehabilitation training independently under the assistance of the peripheral auxiliary equipment, so that the time and energy of accompanying personnel can be greatly reduced. However, some peripheral auxiliary devices on the market today have a high degree of freedom of movement, and often there are multiple moveable joints or parts (e.g., a multi-axis robotic arm having multiple axes that achieve a combined motion by rotation). However, for the patient to be assisted who cannot freely move or even has a high paralysis degree, because there is a great limit to the movement of the limbs of the body, the compound control instruction for simultaneously controlling a plurality of joints or parts cannot be sent through the brain-computer interface, the instruction can often only be sent through a single mode, the movement of a plurality of joints or parts of the assisting device cannot be simultaneously controlled, and more time is taken for sending the instruction. Although the present situation that the display device is used for carrying out visual stimulation on a patient so as to enable the patient to generate the electroencephalogram signals required by the control command appears, the command input mode of the visual stimulation can reduce the difficulty of outputting the command to a certain extent and is more friendly for the patient with high limb paralysis degree, the scheme can not solve the problem of outputting complex commands for executing compound actions, when the control command quantity is large, the patient needs to watch a screen for a long time, the patient is easy to feel tired, the attention of the patient is easy to be reduced, and when the attention of the patient is lower, the recognition accuracy of the electroencephalogram signals can be correspondingly reduced, so that the control capability of the patient on peripheral auxiliary equipment is reduced. Currently, no effective solution has been proposed for the above problems. Disclosure of Invention The present invention has been made in view of the above-described problems. The invention provides a method and a system for generating action instructions, electronic equipment and a storage medium. The scheme can stimulate the auxiliary object to generate the electroencephalogram signal meeting the requirements by means of visual stimulation identification, and can determine more complex combined actions which need to be executed by peripheral auxiliary equipment in a short time by utilizing the electromyographic signal generated when the auxiliary object executes specific actions, so that time is saved and efficiency is improved. According to one aspect of the invention, a motion instruction generating method is provided, which is applied to a control device of peripheral auxiliary equipment, wherein the control device is used for being respectively connected with an electroencephalogram signal acquisition device and an electromyogram signal acquisition device, the electroencephalogram signal acquisition device is used for acquiring an electroencephalogram signal of an auxiliary object, the electromyogram signal acquisition device is used for acquiring an electromyogram signal of the auxiliary object, the control device is connected with a display device, the display device is used for displaying a plurality of visual stimulus identifiers corresponding to target motions one by one, and the flicker frequencies of the visual stimulus identifiers corresponding to different target motions are different; the method comprises the steps of determining flicker frequency of a first target visual stimulus identifier of an electroencephalogram signal generated by visual stimulus aiming at an electroencephalogram signal acquired by an electroencephalogram signal acquisition device when a first target visual stimulus identifier is observed on an auxiliary object, determining a first target action corresponding to the first target visual stimulus identifier and generating a first target instruction for controlling a peripheral auxiliary device to execute the first target action, identifying the specific acti