US-12619307-B2 - Distributed system-on-a-chip for human activity recognition

Abstract

In certain aspects, a system-on-a-chip (SoC) for human activity recognition includes a plurality of integrated low-noise amplifiers configured to sense electromyogram (EMG) signals. The SoC includes a mixed-signal circuitry configured to receive the EMG signals from the plurality of integrated low-noise amplifiers, wherein the mixed-signal circuitry is configured to digitalize and extract time-domain features from the EMG signals. The SoC includes an artificial intelligence (AI) core comprising a reconfigurable neural network (NN) configured to receive, from the mixed-signal circuitry, the time-domain features that were extracted, wherein the reconfigurable NN is configured to recognize a local gesture based on time-domain features that is extracted. The SoC includes an analog data path circuitry configured to sense distance measurements and to transmit the distance measurements and the local gesture that is recognized.

Inventors

• Jie Gu
• Yijie Wei

Assignees

• NORTHWESTERN UNIVERSITY

Dates

Publication Date

20260505

Application Date

20240605

Claims (17)

1. 1 . A system-on-a-chip for human activity recognition, comprising: a plurality of integrated low-noise amplifiers configured to sense electromyogram (EMG) signals; a mixed-signal circuitry configured to receive the EMG signals from the plurality of integrated low-noise amplifiers, wherein the mixed-signal circuitry is configured to digitalize and extract time-domain features from the EMG signals; an artificial intelligence (AI) core comprising a reconfigurable neural network (NN) configured to receive, from the mixed-signal circuitry, the time-domain features that were extracted, wherein the reconfigurable NN is configured to recognize a local gesture based on time-domain features that is extracted; and an analog data path circuitry configured to sense distance measurements and to transmit the distance measurements and the local gesture that is recognized, wherein the analog data path circuitry comprises an IR LED transceiver configured to transmit the distance measurements and the local gesture that is recognized.
2. 2 . The system-on-a-chip of claim 1 , wherein the AI core comprises a long short-term memory (LSTM) configured to receive the distance measurements and the local gesture that is recognized, wherein the LSTM is configured to derive final classification results based on the distance measurements and the local gesture.
3. 3 . The system-on-a-chip of claim 2 , wherein the final classification results classify human activities comprising one of waving, shooting, arching, and punching.
4. 4 . The system-on-a-chip of claim 1 , wherein the plurality of integrated low-noise amplifiers comprise 6 channels with a tunable gain in a range of 35 dB to 55 dB.
5. 5 . The system-on-a-chip of claim 1 , wherein the reconfigurable NN is a 3-layer fully connected neural network (FCNN).
6. 6 . The system-on-a-chip of claim 1 , wherein the analog data path circuitry comprises a receiver configured to receive another distance measurement and another local gesture from another system-on-a-chip.
7. 7 . A system for human activity recognition, comprising: a plurality of distributed system-on-a-chips, wherein each system-on-a-chip comprises: a plurality of integrated low-noise amplifiers configured to sense electromyogram (EMG) signals; a mixed-signal circuitry configured to receive the EMG signals from the plurality of integrated low-noise amplifiers, wherein the mixed-signal circuitry is configured to digitalize and extract time-domain features from the EMG signals; an artificial intelligence (AI) core comprising a reconfigurable neural network (NN) configured to receive, from the mixed-signal circuitry, the time-domain features that were extracted, wherein the reconfigurable NN is configured to recognize a local gesture based on time-domain features that is extracted; and an analog data path circuitry configured to sense distance measurements to a preceding system-on-a-chip and configured to transmit the distance measurements and the local gesture that is recognized, wherein the analog data path circuitry comprises an IR LED transceiver configured to transmit both the distance measurements and the local gesture that is recognized to a next system-on-a-chip.
8. 8 . The system of claim 7 , wherein the AI core comprises a long short-term memory (LSTM) configured to receive the distance measurements and the local gesture that is recognized.
9. 9 . The system of claim 7 , wherein the plurality of integrated low-noise amplifiers comprise 6 channels with a tunable gain in a range of 35 dB to 55 dB.
10. 10 . The system of claim 7 , wherein the reconfigurable NN is a 3-layer fully connected neural network (FCNN).
11. 11 . The system of claim 7 , wherein each system-on-a-chip further comprises a chip ID representing communication order between each system-on-a-chip of the plurality of distributed system-on-a-chips.
12. 12 . The system of claim 11 , wherein the analog data path circuitry comprises a receiver configured to receive the distance measurements and the local gesture from the preceding system-on-a-chip.
13. 13 . The system of claim 12 , wherein the LSTM of a system-on-a-chip comprising a final communication order status is configured to derive final classification results based on the distance measurements and the local gestures, collectively.
14. 14 . The system of claim 13 , wherein the final classification results classify human activities comprising one of waving, shooting, arching, and punching.
15. 15 . A method of human activity recognition, comprising: sensing electromyogram (EMG) signals via a plurality of integrated low-noise amplifiers; receiving, at a mixed-signal circuitry, the EMG signals from the plurality of integrated low-noise amplifiers, wherein the mixed-signal circuitry is configured to digitalize and extract time-domain features from the EMG signals; receiving, at artificial intelligence (AI) core comprising a reconfigurable neural network (NN) from the mixed-signal circuitry, the time-domain features that were extracted, wherein the reconfigurable NN is configured to recognize a local gesture based on time-domain features that is extracted; and transmitting, by an analog data path circuitry, the local gesture that is recognized, wherein the analog data path circuitry is configured to sense distance measurements and transmit the distance measurements, wherein the analog data path circuitry comprises an IR LED transceiver configured to transmit the distance measurements and the local gesture that is recognized.
16. 16 . The method of claim 15 , further comprising: receiving, at a long short-term memory (LSTM) of the AI core, the distance measurements and the local gesture that is recognized, wherein the LSTM is configured to derive final classification results based on the distance measurements and the local gesture.
17. 17 . The method of claim 15 , wherein the analog data path circuitry comprises a receiver configured to receive another distance measurement and another local gesture from another system-on-a-chip.

Description

CROSS-REFERENCE TO RELATED APPLICATION The present application claims the benefit of priority under 35 U.S.C. § 119 from U.S. Provisional Patent Application Ser. No. 63/506,537 entitled “Distributed System-on-a-Chip for Human Activity Recognition,” filed on Jun. 6, 2023, the disclosure of which is hereby incorporated by reference in its entirety for all purposes. STATEMENT OF FEDERALLY FUNDED RESEARCH OR SPONSORSHIP This invention was made with government support under grant numbers CNS1816870 and CCF2208573 awarded by the National Science Foundation. The government has certain rights in the inventions. TECHNICAL FIELD The present disclosure generally relates to system-on-a-chip, and more specifically relates to distributed system-on-a-chip for human activity recognition. BACKGROUND Virtual Reality (VR) and Augmented Reality (AR) applications have recently experienced significant growth driven by gaming, workplace assistance, and social networking, to name a few. VR/AR offers a new level of virtual immersion to users by seamlessly blending the real and digital worlds. However, current VR/AR systems primarily rely on conventional techniques such as joysticks and IMU gloves along with external cameras for motion tracking. These conventional methods suffer from low resolution for sophisticated gestures from users and use of cameras which often have limited view-of-sight and face challenges in a non-stationary environment. The description provided in the background section should not be assumed to be prior art merely because it is mentioned in or associated with the background section. The background section may include information that describes one or more aspects of the subject technology. SUMMARY According to certain aspects of the present disclosure, a system-on-a-chip for human activity recognition is provided. The system-on-a-chip includes a plurality of integrated low-noise amplifiers configured to sense electromyogram (EMG) signals. The SoC includes a mixed-signal circuitry configured to receive the EMG signals from the plurality of integrated low-noise amplifiers, wherein the mixed-signal circuitry is configured to digitalize and extract time-domain features from the EMG signals. The SoC includes an artificial intelligence (AI) core comprising a reconfigurable neural network (NN) configured to receive, from the mixed-signal circuitry, the time-domain features that were extracted, wherein the reconfigurable NN is configured to recognize a local gesture based on time-domain features that is extracted. The SoC includes an analog data path circuitry configured to sense distance measurements and to transmit the distance measurements and the local gesture that is recognized. According to other aspects of the present disclosure, a system is provided. The system includes a plurality of system-on-a-chips. Each system-on-a-chip of the plurality of system-on-a-chips includes a plurality of integrated low-noise amplifiers configured to sense electromyogram (EMG) signals. Each system-on-a-chip of the plurality of system-on-a-chips includes a mixed-signal circuitry configured to receive the EMG signals from the plurality of integrated low-noise amplifiers, wherein the mixed-signal circuitry is configured to digitalize and extract time-domain features from the EMG signals. Each system-on-a-chip of the plurality of system-on-a-chips includes an artificial intelligence (AI) core comprising a reconfigurable neural network (NN) configured to receive, from the mixed-signal circuitry, the time-domain features that were extracted, wherein the reconfigurable NN is configured to recognize a local gesture based on time-domain features that is extracted. Each system-on-a-chip of the plurality of system-on-a-chips includes an analog data path circuitry configured to sense distance measurements to a preceding system-on-a-chip and configured to transmit the distance measurements and the local gesture that is recognized. According to other aspects of the present disclosure, a method is provided. The method includes sensing electromyogram (EMG) signals via a plurality of integrated low-noise amplifiers. The method includes receiving, at a mixed-signal circuitry, the EMG signals from the plurality of integrated low-noise amplifiers, wherein the mixed-signal circuitry is configured to digitalize and extract time-domain features from the EMG signals. The method includes receiving, at artificial intelligence (AI) core comprising a reconfigurable neural network (NN) from the mixed-signal circuitry, the time-domain features that were extracted, wherein the reconfigurable NN is configured to recognize a local gesture based on time-domain features that is extracted. The method includes transmitting, by an analog data path circuitry, the local gesture that is recognized, wherein the analog data path circuitry is configured to sense distance measurements and transmit the distance measurements. It is understood that other configurations of th