US-20260126859-A1 - METHOD FOR GESTURE-BASED CONTROL OF ELECTRONIC DEVICES

US20260126859A1US 20260126859 A1US20260126859 A1US 20260126859A1US-20260126859-A1

Abstract

Embodiments of the present disclosure are directed to systems and methods for gesture-based command control of electronic devices. The invention integrates multiple sensors and cameras with machine learning algorithms to accurately detect and interpret user gestures and verbal commands. Sensors embedded in wearable devices, cameras positioned in the user's environment, and user equipment (UE) capture hand and finger movements and visual data. These inputs are processed using neural networks to recognize specific gestures and interpret verbal commands. The recognized gestures and commands are then translated into actions for connected devices, facilitating intuitive and efficient control.

Inventors

Kirthi Arjun PUCHA
Jhon Nelson Sihotang

Assignees

T-MOBILE INNOVATIONS LLC

Dates

Publication Date: 20260507
Application Date: 20241107

Claims (20)

1 . A method for controlling electronic devices using gestures comprising: detecting one or more movements of a user; determining, using a machine learning algorithm, one or more user specific gestures based on the one or more movements of the user; generating, based on the one or more user specific gestures, a hypothesis regarding a meaning of the one or more user specific gestures; prompting the user to confirm or correct the hypothesis; receiving, from the user, a response indicating whether the hypothesis is correct; updating a personalized gesture library based on the response from the user; generating one or more commands for a connected electronic device based on the one or more user specific gestures; and causing the one or more commands to be executed on the connected electronic device.
2 . The method of claim 1 , the detecting the one or more movements of the user being done by one or more sensors integrated into a wearable device worn by the user.
3 . The method of claim 1 , further comprising capturing one or more gestures using one or more cameras, the gestures being used to determine the one or more user specific gestures.
4 . The method of claim 3 , wherein the one or more cameras are positioned in a user's environment to provide multiple angles of view.
5 . The method of claim 4 , wherein the one or more cameras are positioned on the user.
6 . The method of claim 1 , further comprising storing the one or more user specific gestures and the one or more commands in a gesture library.
7 . The method of claim 6 , wherein the gesture library is updated based on a user feedback.
8 . The method of claim 6 , wherein the generating the one or more commands is performed using a neural network trained on a large dataset of gesture-command pairs.
9 . The method of claim 1 , wherein the generating the one or more commands comprises mapping the gestures to predefined commands stored in a command database.
10 . The method of claim 9 , wherein the command database is customizable by the user or a service provider.
11 . The method of claim 1 , wherein the generating the one or more commands comprises a reinforcement learning model that adjusts the machine learning algorithm based on a behavior of the user.
12 . A system for controlling electronic devices, the system comprising: a wearable device worn by a user, the wearable device including one or more sensors configured to detect movements of the user; one or more cameras configured to capture gestures of the user; a processing module configured to recognize one or more user specific gestures from the movements of the user and the gestures of the user, wherein the processing module is further configured to generate a hypothesis regarding a meaning of the one or more user specific gestures and prompt the user to confirm or correct the hypothesis; a personalized gesture library configured to store the one or more user specific gestures, wherein the personalized gesture library is updated based on a response from the user indicating whether the hypothesis is correct; a machine learning module configured to translate the one or more user specific gestures into one or more commands for a connected electronic device; and a communication module configured to transmit the one or more commands to the connected electronic device for execution.
13 . The system of claim 12 , wherein the processing module uses a gesture machine learning model to recognize the one or more user specific gestures.
14 . The system of claim 13 , wherein the gesture machine learning model uses reinforcement learning to learn to determine the one or more user specific gestures.
15 . The system of claim 12 , wherein the processing module determines one or more user-specific gesture patterns.
16 . The system of claim 15 , wherein the one or more user-specific gesture patterns are stored in a personalized gesture library.
17 . A non-transitory computer-readable medium storing instructions that, when executed by one or more processors, cause a system to perform a method for controlling electronic devices using gestures, the method comprising: detecting one or more hand or finger movements of a user, the one or more hand or finger movements of the user being detected by one or more sensors integrated into a wearable device worn by the user; capturing one or more gestures of the user, the one or more gestures of the user being captured using one or more cameras; determining one or more user specific gestures based on the one or more hand or finger movements of the user and the one or more gestures; generating, based on the one or more user specific gestures, a hypothesis regarding a meaning of the one or more user specific gestures; prompting the user to confirm or correct the hypothesis; receiving, from the user, a response indicating whether the hypothesis is correct; updating a personalized gesture library based on the response from the user; translating, using a machine learning algorithm, the one or more user specific gestures into one or more commands for a connected electronic device; and causing the one or more commands to be executed on the connected electronic device.
18 . The non-transitory computer-readable medium of claim 17 , wherein the machine learning algorithm is trained based on a user's unique gesture patterns through feedback mechanisms and reinforcement learning.
19 . The non-transitory computer-readable medium of claim 18 , wherein the feedback mechanisms comprises a user confirmation or a correction of the one or more gestures.
20 . The non-transitory computer-readable medium of claim 17 , wherein the method further comprises storing the one or more user specific gestures and the one or more commands in a gesture library.

Description

SUMMARY The present disclosure is directed, in part, to methods and systems for detecting and interpreting user gestures to control connected electronic devices, substantially as shown and/or described in connection with the figures. This disclosure provides innovative mechanisms for integrating multiple data sources and employing advanced machine learning techniques to enable seamless and intuitive user interactions with connected devices. According to various aspects of the technology, the disclosed methods introduce solutions to the problem of accurately interpreting user inputs in a connected environment. By implementing a system capable of detecting user gestures through sensors embedded in wearable devices and capturing gestures via multiple cameras or other motion-capturing devices (e.g., RADAR devices, LIDAR devices, etc.), the disclosed methods and systems ensure that user intents can be precisely understood and executed. These outcomes are achieved through a method where sensors monitor hand and finger movements while cameras capture visual data. Machine learning algorithms are used to recognize specific gestures. The recognized gestures are then translated into commands for connected devices, enabling efficient and accurate execution of user intentions. This summary is provided to introduce a selection of concepts in a simplified form that are further described below in the detailed description. This summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used in isolation as an aid in determining the scope of the claimed subject matter. BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 illustrates an exemplary computing device for use with the present disclosure; FIG. 2 illustrates a diagram of an exemplary network environment in which implementations of the present disclosure may be employed; FIG. 3 illustrates an exemplary network environment in which implementations of the present disclosure may be employed; FIG. 4 illustrates an exemplary network environment in which implementations of the present disclosure may be employed; and FIG. 5 illustrates a flow diagram of an exemplary method for communicating with connected electronic devices. DETAILED DESCRIPTION The subject matter of embodiments of the invention is described with specificity herein to meet statutory requirements. However, the description itself is not intended to limit the scope of this patent. Rather, the inventors have contemplated that the claimed subject matter might be embodied in other ways, to include different steps or combinations of steps similar to the ones described in this document, in conjunction with other present or future technologies. Moreover, although the terms “step” and/or “block” may be used herein to connote different elements of methods employed, the terms should not be interpreted as implying any particular order among or between various steps herein disclosed unless and except when the order of individual steps is explicitly described. Various technical terms, acronyms, and shorthand notations are employed to describe, refer to, and/or aid the understanding of certain concepts pertaining to the present disclosure. Unless otherwise noted, said terms should be understood in the manner they would be used by one with ordinary skill in the telecommunication arts. An illustrative resource that defines these terms can be found in Newton's Telecom Dictionary, (e.g., 32d Edition, 2022). As used herein, the term “base station” refers to a centralized component or system of components that is configured to wirelessly communicate (receive and/or transmit signals) with a plurality of stations (i.e., wireless communication devices, also referred to herein as user equipment (UE(s))) in a particular geographic area. As used herein, the term “network access technology (NAT)” is synonymous with wireless communication protocol and is an umbrella term used to refer to the particular technological standard/protocol that governs the communication between a UE and a base station; examples of network access technologies include 3G, 4G, 5G, 6G, 802.11x, and the like. Embodiments of the technology described herein may be embodied as, among other things, a method, system, or computer-program product. Accordingly, the embodiments may take the form of a hardware embodiment, or an embodiment combining software and hardware. An embodiment takes the form of a computer-program product that includes computer-useable instructions embodied on one or more computer-readable media that may cause one or more computer processing components to perform particular operations or functions. Computer-readable media include both volatile and nonvolatile media, removable and nonremovable media, and contemplate media readable by a database, a switch, and various other network devices. Network switches, routers, and related components are conventional in nature, as are means of communicating