KR-20260063487-A - HAND PRESSURE ESTIMATION SYSTEM, HAND PRESSURE ESTIMATION DEVICE, METHOD FOR LEARNING ARTIFICIAL NEURAL NETWORK TO ESTIMATE HAND PRESSURE

KR20260063487AKR 20260063487 AKR20260063487 AKR 20260063487AKR-20260063487-A

Abstract

The present invention relates to a hand pressure estimation system comprising: a tracking sensor that generates tracking data by sensing posture and pressure by a hand-wearable sensor mounted on a user's hand or by sensing posture by a vision sensor; an electromyography sensor mounted on a user's hand or arm that generates electromyography data by detecting electrical activity of muscles; and a hand pressure estimation device that outputs a result of estimating hand pressure from the tracking data and the electromyography data using a pre-trained artificial neural network, wherein the hand pressure estimation device trains the artificial neural network to output a result of estimating hand pressure by integrating the tracking data sensing posture and pressure by the hand-wearable sensor and the electromyography data by the electromyography sensor during the training process of the artificial neural network, and can output a result of estimating hand pressure through the artificial neural network by integrating the tracking data sensing posture by the vision sensor and the electromyography data by the electromyography sensor during the testing process of the artificial neural network.

Inventors

윤상호
서경진
서정훈
정한석

Assignees

한국과학기술원

Dates

Publication Date: 20260507
Application Date: 20241030

Claims (9)

An acquisition unit that acquires tracking data measured by a tracking sensor and electromyography (EMG) data measured by an electromyography sensor; A storage unit including instructions for estimating hand pressure from the tracking data and the electromyography data using a pre-trained artificial neural network; and A processing unit that processes to output the result of estimating the hand pressure by executing the above command; comprising Hand pressure estimation device.
In Article 1, The above processing unit is, A feature extraction unit that extracts a first feature of the tracking data and a second feature of the electromyography data; A data integration unit that integrates the first feature and the second feature; and A pressure estimation unit that estimates the hand pressure based on the result of the above integration; Hand pressure estimation device.
In Article 1, The above processing unit sets pressure and posture data measured by a hand-worn sensor as the tracking data, and The artificial neural network described above is trained to estimate the hand pressure by inputting pressure and posture data measured by the hand-wearable sensor and the electromyography data as integrated training data. Hand pressure estimation device.
In Article 1, The above processing unit sets the attitude data measured by a vision sensor or an inertial sensor as the tracking data, and The artificial neural network above estimates the hand pressure by inputting posture data measured by the vision sensor or inertial sensor and the electromyography data as integrated training data. Hand pressure estimation device.
In Paragraph 3, The hand-wearable sensor described above includes at least one of a data glove, a wired glove, and a cyber glove. Hand pressure estimation device.
In Article 1, The above electromyography data includes surface electromyography data. Hand pressure estimation device.
A tracking sensor that generates tracking data by sensing posture and pressure using a hand-wearable sensor mounted on a user's hand, or by sensing posture using a vision sensor; An electromyography sensor mounted on the user's hand or arm that detects electrical activity of muscles and generates electromyography data; and A hand pressure estimation device that outputs a result of estimating hand pressure from the tracking data and the electromyography data using a pre-trained artificial neural network; wherein The above hand pressure estimation device is, In the training process of the artificial neural network, the artificial neural network is trained to output a result of estimating the hand pressure by integrating the tracking data sensing posture and pressure by the hand-wearable sensor and the electromyography data by the electromyography sensor. In the testing process of the above artificial neural network, the tracking data sensing the posture by the vision sensor and the electromyography data by the electromyography sensor are integrated to output the result of estimating the hand pressure through the artificial neural network. Hand pressure estimation system.
As a computer-readable recording medium storing a computer program, The above computer program includes instructions for a processor to perform a method of training an artificial neural network to estimate hand pressure in a hand pressure estimation system, and The above method is, A step of acquiring tracking data from a tracking sensor and acquiring electromyography data from an electromyography sensor; A step of extracting a first feature of the tracking data and a second feature of the electromyography data; A step of integrating the first feature and the second feature; and The method includes the step of training an artificial neural network to output a result of estimating the hand pressure using the integrated data for learning obtained from the above integration as input; The first feature above includes features for pressure and posture data measured by a hand-wearable sensor mounted on the user's hand. Computer-readable recording medium.
As a computer program stored on a computer-readable recording medium, The above computer program includes instructions for a processor to perform a method of training an artificial neural network to estimate hand pressure in a hand pressure estimation system, and The above method is, A step of acquiring tracking data from a tracking sensor and acquiring electromyography data from an electromyography sensor; A step of extracting a first feature of the tracking data and a second feature of the electromyography data; A step of integrating the first feature and the second feature; and The method includes the step of training an artificial neural network to output a result of estimating the hand pressure using the integrated data for learning obtained from the above integration as input; The first feature above includes features for pressure and posture data measured by a hand-wearable sensor mounted on the user's hand. A computer program stored on a computer-readable recording medium.

Description

Hand pressure estimation system, hand pressure estimation device, method for learning an artificial neural network to estimate hand pressure The present invention relates to a technique for estimating hand pressure. The advancement of digital technology is accelerating the emergence of new technologies that provide users with realistic experiences, such as virtual reality (VR), augmented reality (AR), and mixed reality (MR). These technologies are being utilized in various fields, including education, entertainment, architecture, and medicine, by enabling new ways of interaction that allow for exploring virtual worlds or integrating virtual objects into real environments. In particular, as the hand is one of the most natural tools for human-computer interaction, active research is being conducted on technologies that enable the manipulation and interaction of virtual objects through sophisticated hand movements and gestures. Nevertheless, estimating the actual pressure felt by a user when grasping or touching an object without wearing separate equipment or gloves remains a significant challenge in the interaction between a hand and a virtual object. Previous studies have primarily inferred these interactions by utilizing visual information, hand posture, and electromyography (EMG) signal analysis; however, there have been limitations in accurately measuring or predicting the pressure actually applied by the hand. In particular, performance often degrades significantly when involving various hand postures and interactions with objects, which has prevented this technology from reaching practical application. The aforementioned background technology is technical information that the inventor possessed for the derivation of the present invention or acquired during the process of deriving the present invention, and it cannot be considered as publicly known technology disclosed to the general public prior to the filing of the present invention. FIG. 1 is a block diagram of a hand pressure estimation system according to an embodiment of the present invention. FIG. 2 is a block diagram illustrating the function of a hand pressure estimation device included in a hand pressure estimation system according to an embodiment of the present invention. Figure 3 is a block diagram illustrating the specific functions of the processing unit of Figure 2. FIG. 4 is a block diagram of an artificial neural network included in the storage unit of FIG. 2, and is a block diagram for exemplarily explaining the process of training an artificial neural network to estimate hand pressure. FIG. 5 is a block diagram of an artificial neural network included in the storage unit of FIG. 2, and is a block diagram for exemplarily explaining the process of estimating hand pressure using a previously trained artificial neural network. FIG. 6 is a perspective view illustrating an example of use of a hand pressure estimation system according to an embodiment of the present invention. The advantages and features of the present invention and the methods for achieving them will become clear by referring to the embodiments described below in detail together with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below but can be implemented in various forms. These embodiments are provided merely to ensure that the disclosure of the present invention is complete and to fully inform those skilled in the art of the scope of the invention, and the scope of the present invention is defined only by the claims. In describing the embodiments of the present invention, specific descriptions of known functions or configurations will be omitted unless actually necessary for describing the embodiments of the present invention. Furthermore, the terms described below are defined in consideration of the functions in the embodiments of the present invention, and these may vary depending on the intentions or practices of the user or operator. Therefore, such definitions should be based on the content throughout this specification. Existing hand-based interaction technology in augmented reality has two major limitations. First, since most technologies focus solely on tracking hand position and movement, they fall short of accurately reproducing or mimicking the pressure and force users actually feel when manipulating objects within a virtual environment. Consequently, users find it difficult to experience intuitive and realistic feedback while manipulating virtual objects, which acts as a factor hindering immersion and the naturalness of interaction in augmented reality. Second, existing pressure estimation methods suffer from performance degradation when interacting with various hand postures and objects. Since hand movements and gestures within augmented reality are highly diverse and hand size, shape, and movement patterns can vary from user to user, it is difficult in reality to achieve consistent and accurate pressu