Search

EP-4736141-A1 - METHOD AND SYSTEM FOR THE AUTOMATIC TRANSLATION OF SIGN LANGUAGE INTO A DIFFERENT IDIOM

EP4736141A1EP 4736141 A1EP4736141 A1EP 4736141A1EP-4736141-A1

Abstract

A method for the automatic translation of sign language into a different idiom, comprising a step a) provision of means for images and/or video acquisition (2); a step b) provision of first digital data (Di); a step c) provision of second digital data (D2); a step d) provision of third digital data (Ds) into metadata of a predetermined idiom; a step e) provision of storage means (3); a step f) provision of processing means (4) operatively connected to the means for image and/or video acquisition (2) and to said storage means (3); a step g) processing of the first digital data (D 1 ) and said second digital data (D 2 ) to obtain digital reference data (DREF) associated with the meaning of sign language; a step (h) of saving the reference data in the storage means (3); a step (i) activation of said means for images and/or videos acquisition (2); a step (j) provision of fourth digital data (D 4 ) associated with the sequence of images and/or videos; a step k) comparison between the fourth digital data (D 4 ) and the reference data (DREF) to determine the meaning of the body movement followed by the user in the sequence of images and/or videos; a step I) association of the meaning with the third data (D 3 ); a step m) generation of fifth digital data (D 5 ) in the idiom associated with the third data (D 3 ) corresponding to the meaning determined during the execution of step k).

Inventors

  • MERAFINA, Marcantonio Angelo
  • LA CAMERA, Marco

Assignees

  • Cloudia Research S.p.A.

Dates

Publication Date
20260506
Application Date
20240628

Claims (9)

  1. 1. A method for the automatic translation of sign language into a different idiom, wherein the sign language involves the adoption of signs reproduced by a user and consisting of static parameters and dynamic parameters associated with Configuration, Location, Orientation, Expression, and Movement, wherein such method comprises the following steps: a) provision of means for images and/or video acquisition (2); b) provision of first digital data (Di) associated with the first statistic parameters of sign language; c) provision of second digital data (D2) associated with the dynamic parameters of sign language; d) provision of third digital data (D3) associated with metadata of a predetermined idiom and different from sign language; e) provision of storage means (3) suited to save said first digital parameters and said second digital parameters; f) provision of processing means (4) operatively connected to said means for images and/or video acquisition (2) and said storage means (3); g) processing by said processing means (4) of said first digital data (Di) and said second digital data (D2) to obtain digital reference data (D REF ) associated with the meaning of sign language; h) saving of the reference data in said storage means (3); i) activation of said means for images and/or video acquisition (2) so as to capture a sequence of images and/or video associated with the user's body movement reproducing sign language; j) provision of fourth digital data (D 4 ) associated with the sequence of images and/or video acquired in said activation step i); k) comparison between said fourth digital data (D 4 ) provided in said step j) and the reference data (D REF ) obtained in said processing step g) and saved in said storage means (3) during the execution of said step h), said comparison being suited to determine the meaning of the body movement performed by the user in the sequence of images and/or video acquired in sign language during step i); l) association of the meaning determined in said step j) with the third digital data (Da) defined in said step d); m) generation of fifth digital data (Ds) in the idiom associated with said third digital data (Da), the meaning of said fifth digital data (Ds) corresponding to the meaning determined during the execution of said step k).
  2. 2. Method according to claim 1 , characterized in that it comprises a step I) of provision of display means (5), said display means (5) being operatively connected to said processing means (3).
  3. 3. Method according to claim 2, characterized in that said display means (5) are suited to promote the display of the sequence of fifth digital data (Ds) generated during the execution of said step m).
  4. 4. Method according to one or more of the preceding claims, characterized in that said first digital data (Di) provided in said step b) comprise information relative to the Location, Configuration and Orientation in the manner defined in sign language.
  5. 5. Method according to one or more of the preceding claims, characterized in that said second digital data (D2) provided in said step c) comprise information related to the Movement in the manner defined by sign language.
  6. 6. Method according to one or more of the preceding claims, characterized in that it includes a step o) of providing and training an Artificial Intelligence system (6).
  7. 7. Method according to claim 6, characterized in that said step o) of training of said Artificial Intelligence system (6) is performed by using said first digital data (Di) and said second digital data (D2).
  8. 8. Method according to claim 6 or 7, characterized in that said first digital data (Di), said second digital data (D2) and said third digital data (D3) are suited to define respective vector-type parameters.
  9. 9. A system for the automatic translation of sign language into a different idiom, including: - an electronic device provided with: - means for images and/or video acquisition (2); - storage means (3); - processing means (4) operatively connected to said storage means (3); - display means (5); - a computer program; characterize in that said computer program comprises a plurality of instructions storable in said storage mean (3) and executable by said storage means (3), said plurality of Instructions being suited to implement the method for the automatic translation of sign language into a different idiom according to one or more of the preceding claims.

Description

METHOD AND SYSTEM FOR THE AUTOMATIC TRANSLATION OF SIGN LANGUAGE INTO A DIFFERENT IDIOM DESCRIPTION Field of application of the invention [001] The present invention concerns the technical field of rapid communication via electronic devices (e.g. smartphones, tablets, laptops, personal computers, etc.) and the subject of the invention is a method for automatic translation of sign language into a different idiom. [002] The invention also includes a system for the automatic translation of sign language into a different idiom based on this method. State of the art [003] As is known, in recent years there has been an increasing will to implement the use of sign language as an inclusive form of communication for hearing-impaired people. [004] Thanks to the advances in technology, electronic devices have opened up new lines of communication to facilitate interaction between people who use sign language and those who do not. [005] Sign language is a visual-gestural language that is used worldwide in communities of deaf or severely hearing-impaired people [006] Previously, communication between these people was 'local', i.e. carried out mainly in person, whereas the development of new technologies has made it possible to use this form of communication also at a distance, through the use of common electronic devices. [007] Information technology and new communication platforms have provided new opportunities for hearing-impaired people to connect more easily with the rest of the world, helping to break down language barriers and foster inclusion. [008] One of the main applications of the use of sign language in electronic devices is through video calls and real-time messaging platforms. [009] Thanks to smartphones, tablets, personal computers and other devices equipped with webcams and an Internet connection, people can communicate using sign language, allowing them to establish real-time conversations with friends, family, colleagues, etc. [0010] In addition, applications and software have been developed for helping hearing- impaired people, to offer help in interpreting texts or voices in order to allow them to interact with electronic devices designed to act as virtual assistants or voice recognition systems. [0011] These technologies are making digital services more accessible to all hearing- impaired people, and therefore their use is rapidly spreading among users of all age groups in order to promote the inclusion of a part of the population that cannot currently take full advantage of the resources and benefits offered by digital communications. [0012] It is evident, therefore, how the diffusion of new digital communication systems capable of integrating the sign language allows a greater appreciation and integration of the community of hearing-impaired people by the rest of society. [0013] Despite the progress achieved, the integration of sign language in digital devices still has some drawbacks. [0014] Firstly, current electronic systems available to a hearing-impaired user only allow the automatic generation of subtitles, for the language spoken by an able-bodied user. These systems are an aid for hearing-impaired people that help them to interact with an able-bodied interlocutor. [0015] However, such systems are slow and imprecise, thus preventing smooth, realtime communication between a hearing-impaired person and one or more able-bodied individuals. [0016] Moreover, current electronic systems do not provide a translation from sign language to a different idiom (and vice versa). [0017] Furthermore, current digital systems for converting sign language to other idioms have a low degree of integration and interoperability with the common messaging/video platforms used by the majority of the population. [0018] This problem makes it rather difficult for a hearing-impaired person to fully exploit the potential of social media, with the consequence of relegating them to a more marginal role within society. Presentation of the invention [0019] The present invention intends to overcome the above-mentioned technical drawbacks by providing a particularly effective and efficient method for the automatic translation of sign language into a different idiom. [0020] In particular, the main purpose of the present invention is to provide a method capable of providing an automatic translation of sign language into a different idiom in a smooth, essentially seamless manner. [0021] A further purpose of the present invention is to provide a method for the automatic real-time translation of sign language into a different idiom, i.e. without any particular delay from the moment the action is performed by the person to when the output provided to a listener is generated. [0022] Another purpose of the present invention is to provide a method for the automatic translation of sign language into a different idiom that can be easily reproduced by all devices that are able to interface with all current software/platforms. [0023] A