CN-117203641-B - Training a machine learning module for radar-based gesture detection in a surrounding computing environment

Abstract

Techniques and apparatuses to train a machine learning module to perform radar-based gesture detection in a surrounding computing environment are described. A smart device (104) with a radar system (102) may support ambient computing by providing a gesture-based user interface that is free of eye interactions and less cognitive requirements than other smart devices that rely on a physical user interface. The radar system (102) may be designed to address various challenges associated with ambient computing, including power consumption, environmental changes, background noise, size, and user privacy. The radar system (102) uses a surrounding computing machine learning module (222) to quickly recognize gestures performed by users up to at least two meters away. A surrounding computing machine learning module (222) is trained using, at least in part, a two-stage evaluation process, which includes a segmented classification task and an unsegmented recognition task.

Inventors

• LIN RUIZHI
• LIAN JIMEI
• Nicholas Edward Gillian
• Andrew C. Ferch
• SHAN ZHONGREN
• Blake Charles Yako

Assignees

• 谷歌有限责任公司

Dates

Publication Date

20260512

Application Date

20220408

Priority Date

20210409

Claims (16)

1. 1. A method for training a machine learning module, the method comprising: evaluating the machine learning module using a two-stage evaluation process, the evaluating comprising: Performing a segmented classification task using the machine learning module using pre-segmented data to evaluate errors associated with classification of a plurality of gestures, the pre-segmented data comprising complex radar data having a plurality of gesture segments, each gesture segment of the plurality of gesture segments comprising a gesture motion, a center of the gesture motion across the plurality of gesture segments having a same relative timing alignment within each gesture segment; performing an unsegmented recognition task using the machine learning module using continuous time series data to evaluate a false positive rate, the continuous time series data including other complex radar data, and One or more elements of the machine learning module are adjusted to reduce the error and the false positive rate, wherein the one or more elements include an overall architecture, training data, and/or super parameters of the machine learning module.
2. 2. The method of claim 1, wherein the continuous time series data is not segmented in time.
3. 3. The method of claim 1, wherein the continuous time series data comprises a negative record associated with at least one user naturally moving or performing a repetitive motion similar to the plurality of gestures.
4. 4. The method of claim 1, further comprising: Training internal parameters of the machine learning module using the second pre-segmented data prior to evaluating the machine learning module, and The machine learning module is optimized for super parameters using the third pre-segmented data prior to evaluating the machine learning module.
5. 5. The method of claim 1, further comprising: a random offset is applied to the pre-segmented data and the continuous time series data prior to evaluating the machine learning module.
6. 6. The method of claim 5, wherein applying the random offset comprises at least one of: applying a phase rotation to the complex radar data, and Amplitude scaling is applied to the complex radar data.
7. 7. The method of claim 1, further comprising: Generating the pre-segmented data prior to performing the segmented classification task, the generating of the pre-segmented data comprising: Detecting a center of gesture motion within each gesture segment that is positively registered; Aligning a timing window based on the detected center of the gesture motion, and The gesture segment is resized based on the timing window to generate the pre-segmented data.
8. 8. The method of claim 7, wherein detecting the center of the gesture motion comprises detecting a zero-doppler crossing within each gesture segment of the positive record.
9. 9. The method of claim 1, wherein the pre-segmented data comprises: A positive record associated with at least one user performing the plurality of gestures, and Negative recordings associated with the at least one user naturally moving or performing repetitive motions similar to the plurality of gestures.
10. 10. The method of claim 9, wherein the positive record and the negative record comprise the complex radar data recorded by a radar system.
11. 11. The method of claim 10, wherein the complex radar data represents a complex range-doppler plot associated with a plurality of receive channels.
12. 12. The method of claim 10, wherein the positive record is associated with the at least one user performing each of the plurality of gestures multiple times at different distances from the radar system.
13. 13. The method of claim 10, wherein the positive record is associated with the at least one user performing each of the plurality of gestures multiple times at different angles relative to the radar system.
14. 14. The method of claim 1, wherein the error represents a situation in which the machine learning module misclassifies a gesture performed by a user or misclassifies a background motion performed by the user as a gesture.
15. 15. The method of any one of claims 1 to 14, further comprising: Performing another unsegmented recognition task to evaluate the detection rate, and The one or more elements are adjusted to increase the detection rate.
16. 16. A system comprising a radar system and a processor configured to process complex radar data generated by the radar system according to a machine learning module that has been trained in accordance with the method of any one of claims 1 to 15.

Description

Training a machine learning module for radar-based gesture detection in a surrounding computing environment Background As smart devices become more common, users incorporate them into everyday life. For example, a user may use one or more smart devices to obtain daily weather and traffic information, control the temperature of a household, answer a doorbell, turn a light on or off, and/or play background music. However, interacting with some smart devices can be cumbersome and inefficient. For example, the smart device can have a physical user interface that may require a user to navigate through one or more cues by physically touching the smart device. In this case, the user must divert attention away from other primary tasks to interact with the smart device, which can be inconvenient and damaging. Disclosure of Invention Techniques and apparatuses for training a machine learning module to perform radar-based gesture detection in a surrounding computing (ambient computing) environment are described. Smart devices with radar systems are able to support ambient computing by providing gesture-based user interfaces that are free of eye interactions and less cognitive requirements than other smart devices that rely on physical user interfaces. Radar systems can be designed to address various challenges associated with ambient computing, including power consumption, environmental changes, background noise, size, and user privacy. The radar system uses a surrounding computing machine learning module to quickly recognize gestures up to at least two meters away performed by a user. Training of the surrounding computing machine learning module involves, in part, a two-stage evaluation process. The two-stage evaluation process includes a first stage that performs a segmented classification task using pre-segmented data. The second stage uses the unsegmented or continuous time series data and the gesture de-jitterer to perform the unsegmented recognition task. An unsegmented recognition task may be significantly more challenging than a segmented classification task because it is unknown when a gesture occurs in continuous time series data. By performing segmented classification tasks and non-segmented recognition tasks, the surrounding computing machine learning module may be trained to filter background noise and have a sufficiently low false positive rate to enhance the user experience. Aspects described below include a method for training a machine learning module to perform a surrounding calculation. The method includes evaluating the machine learning module using a two-stage evaluation process. The evaluation includes performing a segmented classification task using the pre-segmented data using a machine learning module to evaluate errors associated with classification of the plurality of gestures. The pre-segmented data includes complex radar data having multiple gesture segments. Each gesture segment of the plurality of gesture segments includes a gesture motion. The centers of gesture motion across multiple gesture segments have the same relative timing alignment within each gesture segment. The method also includes performing an unsegmented recognition task using the continuous time series data using the machine learning module to evaluate the false positive rate. The continuous time series data includes other complex radar data. The method additionally includes adjusting one or more elements of the machine learning module to reduce the rate of errors and false positives. Aspects described below also include a system that includes a radar system and a processor. The processor is configured to process complex radar data generated by the radar system according to a machine learning module that has been trained according to any of the described methods. Aspects described below include a computer-readable storage medium comprising computer-executable instructions that, in response to execution by a processor, cause a system to perform any of the described methods. Aspects described below also include a smart device that includes a radar system and a processor. The processor is configured to process complex radar data generated by the radar system according to a machine learning module that has been trained according to any of the described methods. Aspects described below also include systems having means for training a machine learning module of a radar system to perform surrounding calculations. Drawings Apparatus and techniques for training a machine learning module to perform radar-based gesture detection in a surrounding computing environment are described with reference to the following figures. The same numbers are used throughout the drawings to reference like features and components: 1-1 illustrate an example environment in which ambient computing using a radar system can be implemented; 1-2 illustrate example swipe gestures associated with surrounding computing; 1-3 illustrate example tap gestures asso