US-12622630-B2 - Apparatus for motor imagery training combined with somatosensory stimuli and operation method thereof

US12622630B2US 12622630 B2US12622630 B2US 12622630B2US-12622630-B2

Abstract

Disclosed is an apparatus for a motor imagery training, which includes a measuring unit that measures a brain signal while a user performs a motor imagery training, a preprocessing unit that performs preprocessing with respect to the brain signal, a feature extraction unit that selects a time period including information related to a motor imagery from the preprocessed brain signal and calculates feature data corresponding to the brain signal of the selected time period, and a classification unit that classifies the brain signal into one of a plurality of classes based on the feature data, and the motor imagery training is any one of a first training in which the user imagines moving a body part and a second training in which the user imagines feeling a somatosensory stimuli of a tangible object using the body part.

Inventors

Laehyun Kim
Jihyeon Ha
Sangin Park
Da-hye KIM

Assignees

KOREA INSTITUTE OF SCIENCE AND TECHNOLOGY

Dates

Publication Date: 20260512
Application Date: 20230531
Priority Date: 20221101

Claims (20)

1 . An apparatus comprising: a measuring unit configured to measure a brain signal while a user performs a motor imagery training; a preprocessing unit configured to perform preprocessing with respect to the brain signal; a feature extraction unit configured to select a time period including information related to a motor imagery from the preprocessed brain signal and to calculate feature data corresponding to the brain signal of the selected time period; and a classification unit configured to classify the brain signal into one of a plurality of classes based on the feature data, and wherein the motor imagery training is: any one of a first training in which the user imagines moving a body part and a second training in which the user imagines feeling a somatosensory stimuli of a tangible object using the body part.
2 . The apparatus of claim 1 , wherein the body part includes a left hand, a right hand, and a right foot of the user, and wherein the plurality of classes include the motor imagery of the left hand, the motor imagery of the right hand, and the motor imagery of the right foot.
3 . The apparatus of claim 1 , wherein the somatosensory stimuli is any one of a somatosensory sensation generated when holding the tangible object and a somatosensory sensation generated when feeling a vibration of the tangible object.
4 . The apparatus of claim 1 , wherein the first training includes: a first step in which the user moves the body part; and a second step of imagining that the user moves the body part.
5 . The apparatus of claim 4 , wherein the first step includes a period in which the brain signal of the user corresponds to a resting state, a period in which the user moves the body part, and a period in which a neurofeedback related to a movement of the body part is provided to the user, and the second step includes a period in which the brain signal of the user corresponds to a resting state, a period in which the user imagines moving the body part, and a period in which a neurofeedback related to the imagery is provided to the user.
6 . The apparatus of claim 1 , wherein the second training includes: a first step in which the user feels the somatosensory stimuli of the tangible object using the body part; and a second step in which the user imagines feeling the somatosensory stimuli of the tangible object using the body part.
7 . The apparatus of claim 6 , wherein the first step includes a period in which the brain signal of the user corresponds to a resting state, a period in which the user feels the somatosensory stimuli of the tangible object using the body part, and a period in which a neurofeedback related to the somatosensory stimuli is provided to the user, and the second step includes a period in which the brain signal of the user corresponds to a resting state, a period in which the user imagines feeling the somatosensory stimuli of the tangible object using the body part, and a period in which a neurofeedback related to the imagery is provided to the user.
8 . The apparatus of claim 1 , wherein the measuring unit measures both a first brain signal corresponding to the first training of the user and a second brain signal corresponding to the second training of the user, and wherein, when a first accuracy with which the classification unit classifies the first brain signal of the user into the plurality of classes is less than a predetermined threshold value, a second accuracy with which the classification unit classifies the second brain signal of the user into the plurality of classes is higher than the first accuracy.
9 . The apparatus of claim 1 , wherein the preprocessing unit performs digital filtering on the brain signal, applies a re-standard method to the brain signal, performs downsampling on the brain signal, and removes artifacts of the brain signal.
10 . A method of operating an apparatus for a motor imagery training, the method comprising; measuring a brain signal while a user performs a motor imagery training; performing preprocessing with respect to the brain signal; selecting a time period including information related to a motor imagery from the preprocessed brain signal; calculating feature data corresponding to the brain signal of the selected time period; and classifying the brain signal into one of a plurality of classes based on the feature data, and wherein the measuring of the brain signal while the motor imagery training is performed includes: measuring the brain signal during a first training in which the user imagines moving a body part; and measuring the brain signal during a second training in which the user imagines feeling a somatosensory stimuli of a tangible object using the body part.
11 . The method of claim 10 , wherein the body part includes a left hand, a right hand, and a right foot of the user, and wherein the plurality of classes include the motor imagery of the left hand, the motor imagery of the right hand, and the motor imagery of the right foot.
12 . The method of claim 10 , wherein the somatosensory stimuli is any one of a somatosensory sensation generated when holding the tangible object and a somatosensory sensation generated when feeling a vibration of the tangible object.
13 . The method of claim 10 , wherein the measuring of the brain signal while the first training is performed includes: measuring the brain signal while the user moves the body part; and measuring the brain signal while the user imagines moving the body part.
14 . The method of claim 13 , wherein the measuring of the brain signal while the user moves the body part includes: adjusting the brain signal of the user to correspond to a resting state; moving the body part by the user; and providing a neurofeedback related to the movement of the body part to the user, and wherein the measuring of the brain signal while the user imagines moving the body part includes: adjusting the brain signal of the user to correspond to a resting state; imagining moving the body part by the user; and providing a neurofeedback related to the imagery to the user.
15 . The method of claim 10 , wherein the measuring of the brain signal while the second training is performed includes: measuring the brain signal while the user feels the somatosensory stimuli of the tangible object using the body part; and measuring the brain signal while the user imagines feeling the somatosensory stimuli of the tangible object using the body part.
16 . The method of claim 15 , wherein the measuring of the brain signal while the user feels the somatosensory stimuli of the tangible object using the body part includes: adjusting the brain signal of the user to correspond to a resting state; feeling the somatosensory stimuli of the tangible object using the body part by the user; and providing a neurofeedback related to the imagery to the user, and wherein the measuring of the brain signal while the user imagines feeling the somatosensory stimuli of the tangible object using the body part includes: adjusting the brain signal of the user to correspond to a resting state; imagining that the user feels the somatosensory stimuli of the tangible object using the body part; and providing a neurofeedback related to the imagery to the user.
17 . The method of claim 10 , wherein, when the first accuracy in which the first brain signal of the user is classified into the plurality of classes is less than a predetermined threshold value, a second accuracy in which the second brain signal of the user is classified into the plurality of classes is higher than the first accuracy.
18 . The method of claim 10 , wherein the performing of the preprocessing includes: performing a digital filtering on the brain signal; applying a re-standard method to the brain signal; performing a downsampling on the brain signal; and removing artifacts of the brain signal.
19 . A non-transitory computer-readable medium comprising a program code that, when executed by a processor, causes the processor to execute operations of: measuring a brain signal while a user performs a motor imagery training; performing preprocessing with respect to the brain signal; selecting a time period including information related to a motor imagery from the preprocessed brain signal; calculating feature data corresponding to the brain signal of the selected time period; and classifying the brain signal into one of a plurality of classes based on the feature data, and wherein the measuring of the brain signal while the motor imagery training is performed includes: measuring the brain signal during a first training in which the user imagines moving a body part; and measuring the brain signal during a second training in which the user imagines feeling a somatosensory stimuli of a tangible object using the body part.
20 . The non-transitory computer-readable medium of claim 19 , wherein, when a first accuracy in which the first brain signal of the user is classified into the plurality of classes is less than a predetermined threshold value, a second accuracy in which the second brain signal of the user is classified into the plurality of classes is higher than the first accuracy.

Description

CROSS-REFERENCE TO RELATED APPLICATIONS This application claims priority under 35 U.S.C. § 119 to Korean Patent Application No. 10-2022-0144020, filed on Nov. 1, 2022 in the Korean Intellectual Property Office, the disclosures of which are incorporated by reference herein in their entireties. BACKGROUND Embodiments of the present disclosure described herein relate to an apparatus for measuring and analyzing brain signals, and more particularly, relate to an apparatus for motor imagery training combined with a somatosensory stimuli, and an operating method thereof. In the field of neuroengineering, research has been conducted on a technology that enables quadriplegic patients who cannot move to control an external driving device such as a robot arm with only thoughts by using a brain-computer interface (BCI). The brain-computer interface generally refers to acquiring and analyzing human brain signals to control computers and external devices connected thereto. Acquisition of brain signals may be performed mainly through an electroencephalogram (EEG) measurement, which is a non-invasive measurement method. In order for patients to be able to perform meaningful motions by controlling the driving device using the brain-computer interface, the driving device should be controlled in a desired position or direction. The most important thing in this research is signal processing technology that analyzes a motor imagery of users converts the analyzed result into control signals. When the signal processing is done well, the movement intended by the user is transferred to the driving device, and the movement can be controlled to move according to the user's thoughts. Therefore, to implement various movements intended by the user, a system capable of accurately analyzing motor imagery brain signals when the user imagines the movement is essential. Existing motor image-based brain-computer interfaces show not high accuracy, and the resulting low reliability is also a problem. Recently, studies are being conducted to improve accuracy and reliability using visual, tactile, electrical, and auditory feedbacks, and studies to improve performance of a motor imagery through hybrid training combining the motor imagery and a somatosensory stimuli feedback are also being conducted. SUMMARY Embodiments of the present disclosure provide an apparatus for a motor imagery training combined with a somatosensory stimuli of a tangible object and an operation method thereof. According to an embodiment of the present disclosure, an apparatus for a motor imagery training includes a measuring unit that measures a brain signal while a user performs a motor imagery training, a preprocessing unit that performs preprocessing with respect to the brain signal, a feature extraction unit that selects a time period including information related to a motor imagery from the preprocessed brain signal and calculates feature data corresponding to the brain signal of the selected time period, and a classification unit that classifies the brain signal into one of a plurality of classes based on the feature data, and the motor imagery training is any one of a first training in which the user imagines moving a body part and a second training in which the user imagines feeling a somatosensory stimuli of a tangible object using the body part. According to an embodiment of the present disclosure, a method of operating an apparatus for a motor imagery training includes measuring a brain signal while a user performs a motor imagery training, performing preprocessing with respect to the brain signal, selecting a time period including information related to a motor imagery from the preprocessed brain signal, calculating feature data corresponding to the brain signal of the selected time period, and classifying the brain signal into one of a plurality of classes based on the feature data, and the measuring of the brain signal while the motor imagery training is performed includes measuring the brain signal during a first training in which the user imagines moving a body part, and measuring the brain signal during a second training in which the user imagines feeling a somatosensory stimuli of a tangible object using the body part. According to an embodiment of the present disclosure, a non-transitory computer-readable medium including a program code that, when executed by a processor, causes the processor to execute operations of measuring a brain signal while a user performs a motor imagery training, performing preprocessing with respect to the brain signal, selecting a time period including information related to a motor imagery from the preprocessed brain signal, calculating feature data corresponding to the brain signal of the selected time period, and classifying the brain signal into one of a plurality of classes based on the feature data, and the measuring of the brain signal while the motor imagery training is performed includes measuring the brain signal duri