CN-115943388-B - Wearable electronic device with built-in intelligent monitoring implemented using deep learning accelerator and random access memory

Abstract

Systems, apparatuses, and methods related to deep learning accelerators and memory are described. For example, a wearable electronic device may be configured to execute instructions having matrix operands and configured with a housing to be worn on a person, a sensor having one or more sensor elements to generate measurements associated with the person, a random access memory to store instructions executable by the deep learning accelerator and to store a matrix of an artificial neural network, a transceiver to monitor an output of the artificial neural network generated using the measurements as input to the artificial neural network. Based on the output, the controller may control selective storage of measurement data from the sensor and/or selective transfer of data from the wearable electronic device to a separate computer system.

Inventors

• P. Kali

Assignees

• 美光科技公司

Dates

Publication Date

20260512

Application Date

20210610

Priority Date

20200619

Claims (20)

1. 1. A wearable device, comprising: a housing adapted to be worn on a person; At least one processing unit disposed in the housing and configured to execute instructions having matrix operands; a random access memory configured in the housing and configured to store first data representing weights of an artificial neural network and store second data representing the instructions executable by the at least one processing unit to perform matrix calculations of the artificial neural network using the first data representing the weights of the artificial neural network; a transceiver configured in the housing and configured to communicate with a computer system separate from the wearable device; A sensor having one or more sensor elements configured to generate third data representing a measurement value related to the person, and A controller disposed in the housing and coupled with the transceiver, the sensor, and the random access memory, wherein the controller is configured to write the third data representing the measurement value into the random access memory as an input to the artificial neural network; wherein the at least one processing unit is further configured to execute the instructions to generate an output of the artificial neural network based at least in part on the first data and the third data stored in the random access memory, and Wherein the controller is further configured to monitor a condition associated with the person based on the output of the artificial neural network and to control the transceiver to communicate with the computer system in response to identifying the condition from the output of the artificial neural network.
2. 2. The wearable device of claim 1, wherein the housing includes a strap adapted to be worn on the wrist of the person, the arm of the person, or the head of the person.
3. 3. The wearable device of claim 1, wherein the housing comprises a watch, a pair of glasses, a head-mounted display, a glove, a ring of fingers, or an adhesive patch.
4. 4. The wearable device of claim 1, wherein the one or more sensor elements are configured to measure temperature, bio-current, bio-voltage, light intensity, pressure, mechanical stress, mechanical strain, touch, acceleration, rotation, infrared radiation, or vibration, or any combination thereof.
5. 5. The wearable device of claim 4, wherein the output includes an identification of an event, a feature, an object, a classification, a pattern, or a diagnosis, or any combination thereof.
6. 6. The wearable device of claim 5, wherein the controller is configured to selectively store data for transmission to the computer system based on the output of the artificial neural network.
7. 7. The wearable device of claim 5, wherein the controller is configured to generate an alert to the computer system based on the output of the artificial neural network.
8. 8. The wearable device of claim 6, further comprising: An integrated circuit die of a field programmable gate array FPGA or an application specific integrated circuit ASIC implements a deep learning accelerator comprising the at least one processing unit and a control unit configured to load the instructions from the random access memory for execution.
9. 9. The wearable device of claim 8, wherein the control unit includes the controller.
10. 10. The wearable device of claim 8, further comprising: An integrated circuit package configured to enclose at least the integrated circuit die of an FPGA or ASIC and one or more integrated circuit die of the random access memory.
11. 11. The wearable device of claim 10, wherein the at least one processing unit includes a matrix-matrix unit configured to operate on two matrix operands of an instruction; Wherein the matrix-matrix unit comprises a plurality of matrix-vector units configured to operate in parallel; Wherein each of the plurality of matrix-vector units comprises a plurality of vector-vector units configured to operate in parallel; Wherein each of the plurality of vector-vector units includes a plurality of multiply-accumulate units configured to operate in parallel, an Wherein each of the plurality of multiply-accumulate units includes a neuromorphic memory configured to perform multiply-accumulate operations via analog circuitry.
12. 12. The wearable device of claim 11, wherein the random access memory and the deep learning accelerator are formed on separate integrated circuit die and connected by through silicon vias, TSVs.
13. 13. The wearable device of claim 12, wherein the transceiver is configured to communicate according to a communication protocol of a wireless personal area network or a wireless local area network.
14. 14. A method implemented in a wearable electronic device, comprising: Receiving, via a transceiver of the wearable electronic device, first data and second data representing weights of an artificial neural network, the second data representing instructions having matrix operands and executable by at least one processing unit enclosed within the wearable electronic device to implement matrix calculations of the artificial neural network using the first data representing the weights of the artificial neural network; Storing, in a random access memory of the wearable electronic device, the first data representing the weights and the second data representing the instructions; Generating, by a sensor of the wearable electronic device having one or more sensor elements, third data representing a measurement value related to a person wearing the wearable electronic device; Executing, by the at least one processing unit, the instructions represented by the second data stored in the random access memory, generating an output from the artificial neural network based at least in part on the first data and the third data stored in the random access memory; monitoring, by the wearable electronic device, a condition based on the output from the artificial neural network, and Communication with a computer system via the transceiver in response to the identification of the condition.
15. 15. The method as recited in claim 14, further comprising: discarding measurements from the sensor based on the output from the artificial neural network.
16. 16. The method as recited in claim 14, further comprising: An output from the artificial neural network is transmitted from the wearable electronic device to the computer system without transmitting a measurement from which the output was generated to the computer system.
17. 17. The method as recited in claim 14, further comprising: Storing the third data representing the measurement in the wearable electronic device for a period of time after communicating with the computer system; Receiving a request from the computer system for the third data representing the measured value during the time period, and In response to the request, the third data representing the measurement value is transmitted from the wearable electronic device to the computer system.
18. 18. The method as recited in claim 14, further comprising: Storing the third data representing the measurement in the wearable electronic device for a predetermined period of time after communicating with the computer system, and In response to determining that a request for the third data representing the measurement value has not been received from the computer system within the predetermined period of time, the third data representing the measurement value is deleted from the wearable electronic device.
19. 19. A wearable device, comprising: A housing adapted to be attached to a portion of a person; a sensor having one or more sensor elements disposed on or in the housing and configured to generate a measurement of the person; A random access memory configured to store a model of an artificial neural network; a field programmable gate array FPGA or an application specific integrated circuit ASIC having: A memory interface to access the random access memory; a control unit, and At least one processing unit configured to execute instructions having matrix operands to perform calculations of the artificial neural network according to the model, and A transceiver configured to communicate with a computer system using a wireless communication connection; wherein the sensor is configured to store the measurement value into the random access memory as an input to the artificial neural network; Wherein the FPGA or ASIC is configured to perform the computation of the artificial neural network according to the model to convert the input into an output from the artificial neural network, and Wherein the wearable device is configured to monitor the output from the artificial neural network to control the storage of the measurement values in the wearable device and the transmission of data to the computer system.
20. 20. The wearable apparatus of claim 19, wherein the random access memory comprises a non-volatile memory configured to store the model of the artificial neural network, the model comprises instructions executable by the FPGA or ASIC, and the at least one processing unit comprises a matrix-matrix unit configured to operate on two matrix operands of instructions.

Description

Wearable electronic device with built-in intelligent monitoring implemented using deep learning accelerator and random access memory RELATED APPLICATIONS The present application claims priority from U.S. patent application Ser. No. 16/906,241, entitled "wearable electronic device (WEARABLE ELECTRONIC DEVICE WITH BUILT-IN INTELLIGENT MONITORING IMPLEMENTED USING DEEP LEARNING ACCELERATOR AND RANDOM ACCESS MEMORY)" with built-in intelligent monitoring implemented using deep learning accelerator and random Access memory," filed on even 19, 6/2020, the entire disclosure of which is incorporated herein by reference. Technical Field At least some embodiments disclosed herein relate generally to wearable electronic devices, and more particularly, but not limited to, wearable devices with integrated accelerators for Artificial Neural Networks (ANNs), such as ANNs configured via machine learning and/or deep learning. Background An Artificial Neural Network (ANN) uses a neural network to process inputs to the network and generate outputs from the network. For example, each neuron in the network receives a set of inputs. Some inputs to neurons may be outputs of some neurons in a network, and some inputs to neurons may be inputs provided to a neural network. The input/output relationship between neurons in the network represents the connectivity of neurons in the network. For example, each neuron may have a set of bias, activation functions, and synaptic weights for its inputs, respectively. The activation function may take the form of a step function, a linear function, a logarithmic sigmoid (log-sigmoid) function, or the like. Different neurons in a network may have different activation functions. For example, each neuron may generate a weighted sum of its input and its bias, and then generate an output as a function of the weighted sum, which is calculated using the activation function of the neuron. The relationship between the inputs and outputs of an ANN is generally defined by an ANN model that includes data representing connectivity of neurons in a network, as well as bias, activation functions, and synaptic weights for each neuron. Based on a given ANN model, the computing device may be configured to compute an output of the network from a given set of inputs to the network. For example, input to the ANN network may be generated based on camera input, and output from the ANN network may be identification of an item such as an event or object. In general, an ANN may be trained using a supervised approach, in which parameters in the ANN are adjusted to minimize or reduce errors between known outputs associated with or generated by respective inputs and calculated outputs generated via application of the inputs to the ANN. Examples of supervised learning/training methods include reinforcement learning and learning with error correction. Alternatively or in combination, an unsupervised approach may be used to train an ANN in which the exact output produced by a given set of inputs is not known until the training is completed. ANNs may be trained to classify items into multiple categories, or to classify data points into clusters. Multiple training algorithms may be employed for complex machine learning/training paradigms. Deep learning uses multiple layers of machine learning to progressively extract features from input data. For example, a lower layer may be configured to identify edges in an image, and a higher layer may be configured to identify items, such as faces, objects, events, etc., captured in the image based on using the edges detected by the lower layer. Deep learning may be implemented via an Artificial Neural Network (ANN), such as a deep neural network, a deep belief network, a recurrent neural network, and/or a convolutional neural network. Deep learning has been applied to many application fields such as computer vision, speech/audio recognition, natural language processing, machine translation, bioinformatics, drug design, medical image processing, games, and the like. Drawings Embodiments are illustrated by way of example, and not by way of limitation, in the figures of the accompanying drawings in which like reference numerals refer to similar elements. FIG. 1 shows an integrated circuit device with a configured deep learning accelerator and random access memory, according to one embodiment. Figure 2 shows a processing unit configured to perform a matrix-matrix operation according to one embodiment. FIG. 3 shows a processing unit configured to perform matrix-vector operations according to one embodiment. FIG. 4 shows a processing unit configured to perform vector-vector operations according to one embodiment. FIG. 5 shows a deep learning accelerator and random access memory configured to autonomously apply input to a trained artificial neural network, according to one embodiment. Fig. 6 shows a wearable electronic device configured with intelligent monitoring using an integrated circuit device