KR-20260064030-A - METHOD FOR TRANSLATION RELATED TO SIGN LANGUAGE AND ELECTRONIC APPARATUS THEREFOR

KR20260064030AKR 20260064030 AKR20260064030 AKR 20260064030AKR-20260064030-A

Abstract

A method for translation related to sign language and an electronic device for the same are disclosed. A method for translation related to sign language according to the present disclosure may include: a step of extracting at least a portion of voice data from input data including voice data; a step of converting the extracted voice data into text data corresponding thereto; a step of extracting at least a portion of words included in the text data; a step of identifying a sign language animation corresponding to each of the extracted words; and a step of providing three-dimensional modeled sign language data corresponding to the voice data using the identified sign language animations.

Inventors

김명호
이지민
민병호
박상욱
변지현
조윤식

Assignees

숭실대학교산학협력단

Dates

Publication Date: 20260507
Application Date: 20241031

Claims (10)

In a method performed by an electronic device, A step of extracting at least some voice data from input data including voice data; A step of converting the extracted voice data into corresponding text data; A step of extracting at least some of the words included in the above text data; A step of identifying a sign language animation corresponding to each of the above-mentioned extracted words; and A step comprising providing three-dimensional modeled sign language data corresponding to the voice data using the identified sign language animations, Sign language translation methods.
In Article 1, The step of extracting the above voice data is, A step of capturing voice data along the time axis from the above input data; A step of stopping the capture if a sound exceeding a threshold is not identified for a certain period of time or longer; A step of extracting voice data from the time the above capture started until the time the above capture stopped; and A step comprising resuming the capture of the input data after the point in time when the capture was stopped, Sign language translation methods.
In Article 1, The step of extracting at least some of the words included in the above text data is, A step of identifying at least some of the words included in the above text data; A step of converting at least some of the identified words into their base forms using a natural language processing model library; and A step comprising generating words converted to the above basic form in the form of a list, Sign language translation methods.
In Article 1, The step of identifying the above sign language animation is, A step comprising transmitting information for each of the above-mentioned extracted words to a script in which a Unity-based 3D sign language module operates using a TCP (Transmission Control Protocol) socket, Sign language translation methods.
In Article 1, The step of providing the above sign language data is, A method comprising the step of sequentially playing the identified sign language animations through a Unity-based 3D sign language module. Sign language translation methods.
In a method performed by an electronic device, A step of extracting multiple feature points from each of the multiple sign language image data included in the sign language video; A step of inputting feature point data including coordinate information of the plurality of feature points into a recurrent neural network-based artificial intelligence model to output a word corresponding to each sign language image data; and A step comprising combining words corresponding to the plurality of sign language image data to generate text data corresponding to the sign language image, Sign language translation methods.
In Article 6, The step of extracting the above plurality of feature points is, The method comprises the step of extracting one or more joint coordinates, one or more left-hand feature point coordinates, and one or more right-hand feature point coordinates from each of the plurality of sign language image data. Sign language translation methods.
In Article 6, The above artificial intelligence model is, An architecture comprising one or more LSTM (Long Short-Term Memory) layers, a dropout layer, and one or more fully connected layers sequentially connected, Sign language translation methods.
In Article 8, The fully connected layer at the end of the above artificial intelligence model is, A node including the same number of sign language videos used for training the above artificial intelligence model, Sign language translation methods.
Includes memory and processor containing instructions, The above processor is connected to the above memory, An electronic device that performs either the method of claim 1 or claim 6.

Description

Method for translation related to sign language and electronic apparatus therefor The present disclosure relates to a technology for performing bidirectional translation between a normal language and a sign language using an electronic device, and more specifically, to a technology for translating images, audio, text, etc. in which a normal language is used into sign language, or for translating sign language into text. Existing technologies in the field of sign language translation focus on the recognition of English Sign Language. Consequently, there is a lack of translation research regarding various sign languages, including Korean Sign Language. Furthermore, models for bidirectional translation between sign language and text do not exist, and research has been conducted solely for unidirectional communication. As a result, real-time communication between the deaf and hearing populations was difficult, and the deaf faced challenges in enjoying various content, such as the now-popularized OTT (Over The Top) services. Furthermore, the lack of communication support for the deaf in educational environments sometimes infringed upon their right to learn. Furthermore, despite the necessity of smooth communication for performing tasks in the workplace, the lack of sign language interpretation infrastructure for the Deaf has limited the expansion of job opportunities for them, leading to inefficiency in the job preparation process. FIG. 1 is a block diagram illustrating a system for performing sign language-related translation according to one embodiment. FIG. 2 is a flowchart illustrating a sign language-related translation method according to one embodiment. FIG. 3 is an exemplary drawing of an architecture that performs sign language-related translation according to one embodiment. Figure 4 is an exemplary diagram showing the flow of data according to the architecture of Figure 3. FIG. 5 is a flowchart illustrating a sign language-related translation method according to another embodiment. FIG. 6 is an exemplary drawing of an architecture for performing sign language-related translation according to another embodiment. Figure 7 is an exemplary diagram showing the flow of data according to the architecture of Figure 6. FIG. 8 is a flowchart illustrating a sign language-related translation method according to another embodiment. FIG. 9a is an exemplary drawing showing the result of recognizing a fingerprint according to the embodiment of FIG. 8. FIG. 9b is an exemplary drawing showing a three-dimensional modeled fingerprint generated according to an embodiment of FIG. 8. FIG. 10 shows a block diagram of another electronic device in one embodiment. Hereinafter, exemplary embodiments according to the present invention will be described in detail with reference to the contents described in the attached drawings. However, the present invention is not limited or restricted by exemplary embodiments. Unless otherwise defined, all terms used in this specification (including technical and scientific terms) shall be used in a meaning that is commonly understood by those skilled in the art to which this disclosure belongs, but this may vary depending on the intent of those skilled in the art, case law, the emergence of new technology, etc. Furthermore, terms defined in commonly used dictionaries are not to be interpreted ideally or excessively unless explicitly and specifically defined otherwise. In certain cases, terms have been selected at the applicant's discretion, and in such cases, their meanings will be described in detail in the relevant explanatory section. Accordingly, terms used in this disclosure should be defined not merely by their names, but based on their meanings and the content throughout this disclosure. Throughout this specification, when a part is described as "comprising" a certain component, this means that, unless specifically stated otherwise, it does not exclude other components but may include additional components. Furthermore, the singular form used in this specification includes the plural form unless specifically stated otherwise. Additionally, the expression "at least one of a, b, and/or c" as used throughout this specification may encompass 'a alone', 'b alone', 'c alone', 'a and b', 'a and c', 'b and c', or 'a, b, and c all'. Meanwhile, terms such as "first and/or second" used in this specification may be used to describe various components, but they are used solely for the purpose of distinguishing one component from another and are not intended to limit the description to the component referred to by such terms. For example, without departing from the scope of the present invention, the first component may be named the second component, and the second component may also be named the first component. Additionally, terms such as “…part,” “…module,” etc., as described in this specification refer to a unit that processes at least one function or operation, which may be implemented in hardwa