Search

CN-121578894-B - Signal processing method based on brain control system, device and equipment

CN121578894BCN 121578894 BCN121578894 BCN 121578894BCN-121578894-B

Abstract

The embodiment of the disclosure provides a signal processing method based on a brain control system, the brain control system, a device and equipment, and relates to the technical field of brain-computer interfaces, wherein the method comprises the steps of acquiring a brain electrical signal of a user under the condition that a target interface is displayed on a stimulation end; the method comprises the steps of sending an instruction to be executed, which is determined based on a user electroencephalogram signal, to a controlled device, wherein the instruction to be executed is used for the controlled device to acquire a second image in real time and send the second image to a stimulating end in the process of executing an operation corresponding to the instruction to be executed, receiving the second image sent by the controlled device, determining an execution result of the operation based on the second image, updating the second image to be a first image when the execution result does not meet a preset condition, and returning to execute the step of acquiring the user electroencephalogram signal when the execution result meets the preset condition and displaying a target interface on the stimulating end. Therefore, the visual requirement of the user in the brain control related equipment scene can be effectively met.

Inventors

  • CHEN XIAOGANG
  • CUI HONGYAN
  • LI CHAOHUI
  • XIE XINYING
  • LIU SHUAI

Assignees

  • 中国医学科学院生物医学工程研究所

Dates

Publication Date
20260512
Application Date
20260127

Claims (8)

  1. 1. A signal processing method based on a brain control system, wherein the method is applied to a stimulation end in the brain control system, and comprises the following steps: Acquiring a user electroencephalogram signal under the condition that a target interface is displayed on the stimulation end, wherein the central area of the target interface is used for displaying a first image acquired by controlled equipment in real time, and the edge area of the target interface is used for displaying a plurality of instruction targets and a flicker area associated with each instruction target; Transmitting an instruction to be executed, which is determined based on the electroencephalogram signal of the user, to the controlled equipment, wherein the instruction to be executed is used for acquiring a second image in real time and transmitting the second image to the stimulation end in the process of executing the operation corresponding to the instruction to be executed by the controlled equipment; receiving a second image sent by the controlled device; determining an execution result of the operation based on the second image, updating the second image to be the first image under the condition that the execution result does not meet the preset condition, and returning to execute the step of acquiring the brain electrical signal of the user under the condition that the target interface is displayed on the stimulation end until the execution result meets the preset condition; the method further comprises the steps of: Acquiring interface layout information of the target interface; Determining a central area and an edge area in the target interface based on the interface layout information, and determining a display area and a plurality of flashing areas respectively corresponding to a plurality of instruction targets in the edge area, so that the central area, the display area respectively corresponding to the plurality of instruction targets and the plurality of flashing areas are not overlapped with each other; the interface layout information comprises the display size of a display area corresponding to the instruction targets, the display size of the flashing area and the relative position between the display area of each instruction target and the flashing area associated with the instruction target; the obtaining the interface layout information of the target interface includes: determining a target distance from the eye position point of the user to the target interface; Calculating a relative position between each of the command targets and a blinking region associated with the command target based on the target distance, the relative position including a relative angle and a relative distance; And calculating the display size of the display area corresponding to the instruction target and the display size of the flicker area associated with the instruction target based on the relative position between each instruction target and the flicker area associated with the instruction target.
  2. 2. The method according to claim 1, wherein before the sending the instruction to be executed, which is determined based on the user electroencephalogram signal, to the controlled apparatus, the method further comprises: determining the instruction to be executed based on the electroencephalogram signal of the user; calculating the executable probability of the instruction to be executed based on the first image and the instruction to be executed; And under the condition that the executable probability of the to-be-executed instruction is smaller than the preset execution probability, determining that the to-be-executed instruction is an error instruction, and returning to execute the step of acquiring the user electroencephalogram signal under the condition that the target interface is displayed on the stimulation end until the executable probability of the to-be-executed instruction is larger than or equal to the preset execution probability.
  3. 3. The method according to claim 1 or 2, wherein different blinking regions blink at different blinking frequencies, the method further comprising, before the sending of the instruction to be executed, determined based on the user electroencephalogram signal, to the controlled device: Determining a template signal corresponding to the instruction target based on the flicker frequency of a flicker area associated with the instruction target aiming at each instruction target in the plurality of instruction targets to obtain a plurality of template signals; Calculating the similarity between the electroencephalogram signal of the user and each template signal to obtain a plurality of similarities; And determining a template signal with the maximum similarity with the electroencephalogram signal of the user in the plurality of template signals as a target template signal based on the plurality of similarities, and determining an instruction indicated by the first instruction target corresponding to the target template signal as the instruction to be executed.
  4. 4. A signal processing method based on a brain control system, which is applied to a controlled device in the brain control system, the method comprising: receiving an instruction to be executed, which is sent by a stimulation end and is determined based on a user electroencephalogram signal, wherein the central area of a target interface displayed by the stimulation end is used for displaying a first image acquired by controlled equipment in real time, and the edge area of the target interface is used for displaying a plurality of instruction targets and a flicker area associated with each instruction target; During the process of executing the operation corresponding to the instruction to be executed, acquiring a second image in real time; the second image is sent to the stimulation end, so that the stimulation end can determine an execution result of the controlled device to execute the operation based on the second image, update the second image to be the first image under the condition that the execution result does not meet the preset condition, and return to execute the step of acquiring the electroencephalogram signal of the user under the condition that the stimulation end displays a target interface until the execution result meets the preset condition; the display areas and the flashing areas corresponding to the plurality of instruction targets in the central area, the edge area and the edge area in the target interface are determined by the stimulating end based on the interface layout information of the target interface; the interface layout information comprises the display size of a display area corresponding to the instruction targets, the display size of the flashing area and the relative position between the display area of each instruction target and the flashing area associated with the instruction target; The stimulation end calculates the relative position between each instruction target and the scintillation area associated with the instruction target by determining the target distance between the eye position point of the user and the target interface and based on the target distance, wherein the relative position comprises a relative angle and a relative distance, and calculates the display size of the display area corresponding to the instruction target and the display size of the scintillation area associated with the instruction target based on the relative position between each instruction target and the scintillation area associated with the instruction target.
  5. 5. The brain control system is characterized by comprising a collection end, a stimulation end and controlled equipment, wherein the collection end is in communication connection with the stimulation end, and the stimulation end is in communication connection with the controlled equipment; The system comprises a target interface, an acquisition end, a stimulation end, a control end and a control end, wherein the acquisition end is used for acquiring a user brain electrical signal under the condition that the target interface is displayed by the stimulation end and feeding back the user brain electrical signal to the stimulation end, the central area of the target interface is used for displaying a first image acquired by controlled equipment in real time, and the edge area of the target interface is used for displaying a plurality of instruction targets and a flicker area associated with each instruction target; the stimulation end is used for sending an instruction to be executed, which is determined based on the electroencephalogram signal of the user, to the controlled equipment; The controlled device is configured to collect a second image in real time during the process of executing the operation corresponding to the instruction to be executed, and send the second image to the stimulation end; The method comprises the steps of receiving a first image sent by a controlled device, receiving a second image sent by the controlled device, determining an execution result of the operation based on the first image, updating the first image to be the second image when the execution result does not meet a preset condition, and returning to execute the step of acquiring the brain wave signal of the user when the target interface is displayed by the stimulation device until the execution result meets the preset condition; The stimulation end is also used for acquiring interface layout information of the target interface; determining a central area and an edge area in the target interface based on the interface layout information, and determining a display area and a plurality of flashing areas corresponding to a plurality of instruction targets in the edge area respectively so that the central area, the display area corresponding to the plurality of instruction targets and the plurality of flashing areas do not overlap with each other; The stimulation end is further used for determining the target distance between the eye position point of the user and the target interface, calculating the relative position between each instruction target and the flicker region associated with the instruction target based on the target distance, wherein the relative position comprises a relative angle and a relative distance, and calculating the display size of the display region corresponding to the instruction target and the display size of the flicker region associated with the instruction target based on the relative position between each instruction target and the flicker region associated with the instruction target.
  6. 6. A signal processing device based on a brain control system, characterized in that it is applied to a stimulation end in the brain control system, said device comprising: the device comprises an acquisition module, a target interface and a control module, wherein the acquisition module is used for acquiring a user brain electrical signal under the condition that the target interface is displayed on the stimulation end, the central area of the target interface is used for displaying a first image acquired by controlled equipment in real time, and the edge area of the target interface is used for displaying a plurality of instruction targets and a flicker area associated with each instruction target; the sending module is used for sending an instruction to be executed, which is determined based on the electroencephalogram signal of the user, to the controlled equipment, and acquiring a second image in real time and sending the second image to the stimulation end in the process of executing the operation corresponding to the instruction to be executed by the controlled equipment; the receiving module is used for receiving the second image sent by the controlled equipment; The determining module is used for determining an execution result of the operation based on the second image, updating the second image into the first image under the condition that the execution result does not meet the preset condition, and returning to execute the step of acquiring the electroencephalogram signal of the user under the condition that the target interface is displayed on the stimulating end until the execution result meets the preset condition; The acquisition module is further used for acquiring interface layout information of the target interface; The determining module is further configured to determine a central area and an edge area in the target interface based on the interface layout information, and determine a display area and a plurality of scintillation areas respectively corresponding to a plurality of instruction targets in the edge area, so that the central area, the display area respectively corresponding to the plurality of instruction targets, and the plurality of scintillation areas do not overlap with each other; the interface layout information comprises the display size of a display area corresponding to the instruction targets, the display size of the flashing area and the relative position between the display area of each instruction target and the flashing area associated with the instruction target; The determining module is further used for determining a target distance between the eye position point of the user and the target interface; A calculation module for calculating a relative position between each of the command targets and a blinking region associated with the command target based on the target distance, the relative position including a relative angle and a relative distance; the calculation module is further used for calculating the display size of the display area corresponding to the instruction target and the display size of the flicker area associated with the instruction target based on the relative position between each instruction target and the flicker area associated with the instruction target.
  7. 7. A signal processing apparatus based on a brain control system, for application to a controlled device in the brain control system, the apparatus comprising: The device comprises a receiving module, a display module and a control module, wherein the receiving module is used for receiving an instruction to be executed, the instruction is sent by a stimulation end and is determined based on a user brain electrical signal, a central area of a target interface displayed by the stimulation end is used for displaying a first image acquired by controlled equipment in real time, and an edge area of the target interface is used for displaying a plurality of instruction targets and a flicker area associated with each instruction target; The execution module is used for acquiring a second image in real time in the process of executing the operation corresponding to the instruction to be executed; the sending module is used for sending the second image to the stimulation end, determining an execution result of the controlled device to execute the operation based on the second image by the stimulation end, updating the second image to be the first image under the condition that the execution result does not meet a preset condition, and returning to execute the step of acquiring the electroencephalogram signal of the user under the condition that the target interface is displayed by the stimulation end until the execution result meets the preset condition; the display areas and the flashing areas corresponding to the plurality of instruction targets in the central area, the edge area and the edge area in the target interface are determined by the stimulating end based on the interface layout information of the target interface; the interface layout information comprises the display size of a display area corresponding to the instruction targets, the display size of the flashing area and the relative position between the display area of each instruction target and the flashing area associated with the instruction target; The stimulation end calculates the relative position between each instruction target and the scintillation area associated with the instruction target by determining the target distance between the eye position point of the user and the target interface and based on the target distance, wherein the relative position comprises a relative angle and a relative distance, and calculates the display size of the display area corresponding to the instruction target and the display size of the scintillation area associated with the instruction target based on the relative position between each instruction target and the scintillation area associated with the instruction target.
  8. 8. An electronic device comprising at least one processor, and a memory communicatively coupled to the at least one processor, wherein, The memory stores instructions executable by the at least one processor to enable the at least one processor to perform the brain-controlled system-based signal processing method of any one of claims 1 to 4.

Description

Signal processing method based on brain control system, device and equipment Technical Field The disclosure relates to the technical field of brain-computer interfaces, and in particular relates to a signal processing method based on a brain control system, a device and equipment. Background The brain-computer interface technology can realize the direct communication between the brain and external equipment under the condition of not depending on muscles and peripheral nerves. While the currently prevailing paradigm includes a steady-state visual evoked potential (SSVEP) paradigm that enables control of external devices by identifying responses of the brain visual cortex to visual stimuli of specific frequencies, such as blinking lights or moving images. However, most SSVEP-based brain-computer interface systems employ medium-low frequency flicker stimuli that are prone to visual fatigue, since SSVEP signals are most responsive in the medium-low frequency range and fovea has visual advantages. Also, existing SSVEP-based brain-computer interface systems typically present stimuli in the central field of view, which both exacerbates visual discomfort and crowds visual space for displaying critical feedback information. In this way, the existing brain-computer interface system based on the SSVEP is difficult to meet the visual requirement of the user, so that the user experience is affected. Disclosure of Invention The present disclosure provides a signal processing method based on a brain control system, a device and equipment, so as to at least solve the above technical problems in the prior art. In a first aspect, an embodiment of the present disclosure provides a signal processing method based on a brain control system, applied to a stimulation end in the brain control system, where the method includes: The method comprises the steps of acquiring a user brain electrical signal under the condition that a target interface is displayed on a stimulation end, wherein the central area of the target interface is used for displaying a first image acquired by controlled equipment in real time, and the edge area of the target interface is used for displaying a plurality of instruction targets and a flicker area associated with each instruction target; Transmitting an instruction to be executed, which is determined based on the electroencephalogram signal of the user, to the controlled equipment, wherein the instruction to be executed is used for acquiring a second image in real time and transmitting the second image to the stimulation end in the process of executing the operation corresponding to the instruction to be executed by the controlled equipment; receiving a second image sent by the controlled device; and determining an execution result of the operation based on the second image, updating the second image to be the first image under the condition that the execution result does not meet the preset condition, and returning to execute the step of acquiring the electroencephalogram signal of the user under the condition that the target interface is displayed on the stimulation end until the execution result meets the preset condition. In a second aspect, an embodiment of the present disclosure provides a signal processing method based on a brain control system, which is applied to a controlled device in the brain control system, where the method includes: Receiving an instruction to be executed, which is sent by a stimulation end and is determined based on a user electroencephalogram signal, wherein the central area of a target interface displayed by the stimulation end is used for displaying a first image acquired by controlled equipment in real time, and the edge area of the target interface is used for displaying a plurality of instruction targets and a flicker area associated with each instruction target; In the process of executing the operation corresponding to the instruction to be executed, acquiring a second image in real time; And sending a second image to the stimulation end, wherein the second image is used for determining an execution result of the controlled device execution operation based on the second image by the stimulation end, updating the second image into the first image under the condition that the execution result does not meet the preset condition, and returning to execute the step of acquiring the user brain electrical signal under the condition that the target interface is displayed by the stimulation end until the execution result meets the preset condition. In a third aspect, an embodiment of the present disclosure provides a brain control system, where the system includes a collecting end, a stimulating end, and a controlled device, where the collecting end is communicatively connected to the stimulating end, and the stimulating end is communicatively connected to the controlled device; The system comprises a target interface, an acquisition end, a control end and a control end, wh