CN-121986345-A - Headset device with parallel neural network

Abstract

An apparatus (e.g., an ear-worn device such as a hearing aid) may include a neural network circuit and a control circuit. The neural network circuit may be configured to implement a neural network system including at least a first neural network and a second neural network that operate in parallel. The control circuit may be configured to control the neural network system to receive a first input signal, process the first input signal using the first neural network to produce a first output, and process the first input signal using the second neural network to produce a second output, combine the first output and the second output, reset one or more states of the first neural network, and reset one or more states of the second neural network at a different time than the one or more states of the first neural network.

Inventors

• Igor lovczynski
• Philip Meyers Four generations
• Jonathan Markowski
• Israel Malkin
• Andrew Casper
• NICHOLAS MORRIS

Assignees

• 弗泰研科股份有限公司

Dates

Publication Date

20260505

Application Date

20240828

Priority Date

20230829

Claims (20)

1. 1. An ear-worn device comprising: A neural network circuit configured to implement a neural network system including at least a first neural network and a second neural network configured to operate in parallel, and A control circuit configured to control the neural network system to: resetting one or more states of the first neural network; Receiving a first audio signal after resetting one or more states of the first neural network; Processing the first audio signal using the first neural network to produce a first output and processing the first audio signal using the second neural network to produce a second output, and The first output and the second output are combined using a first weight for the first output and a second weight for the second output, wherein the second weight is greater than the first weight.
2. 2. The ear-worn device of claim 1, wherein the control circuit is configured to: Resetting one or more states of the first neural network at one or more first reset times, and Resetting one or more states of the second neural network at one or more second reset times; wherein the one or more second reset times are different from the one or more first reset times.
3. 3. The ear-worn device of claim 1, wherein: The neural network is further configured to: receiving a second audio signal after combining the first output and the second output; processing the second audio signal using the first neural network to produce a third output and processing the second audio signal using the second neural network to produce a fourth output; combining the third output and the fourth output using a third weight for the third output and a fourth weight for the fourth output, wherein the third weight is greater than the first weight, and One or more states of the second neural network are reset after receiving the second audio signal.
4. 4. The ear-worn device of claim 3, wherein the control circuit is further configured to control the neural network system to: Receiving a third audio signal after resetting one or more states of the second neural network; Processing the third audio signal using the first neural network to produce a fifth output and processing the third audio signal using the second neural network to produce a sixth output; combining the fifth output and the sixth output using a fifth weight for the fifth output and a sixth weight for the sixth output, wherein the fifth weight is greater than the sixth weight; receiving a fourth audio signal after receiving the third audio signal; processing the fourth audio signal using the first neural network to produce a seventh output and processing the fourth audio signal using the second neural network to produce an eighth output, and The seventh output and the eighth output are combined using a seventh weight for the seventh output and an eighth weight for the eighth output, wherein the eighth weight is greater than the sixth weight.
5. 5. The ear-mounted device of claim 1, wherein all layers of the first neural network are configured to process the first audio signal and fewer than all layers of the second neural network are configured to process the first audio signal.
6. 6. The ear-mounted device of claim 5, wherein the neural network system is configured to reset one or more states of a particular layer of the first neural network when resetting one or more states of the first neural network.
7. 7. The ear-mounted device of claim 6, wherein the neural network system is further configured to reset one or more states of a different layer of the first neural network after resetting one or more states of a particular layer of the first neural network.
8. 8. The ear-mounted device of claim 7, wherein a time between resetting one or more states of a particular layer of the first neural network and resetting one or more states of a different layer of the first neural network is approximately equal to 1 second, approximately equal to 60 seconds, or between 1 second and 60 seconds.
9. 9. The ear-mounted device of claim 6, wherein the neural network system is configured to, when processing the first audio signal using the first neural network to produce the first output and processing the first audio signal using the second neural network to produce the second output, process the first audio signal using a layer of the first neural network to produce the first output and process the first audio signal using a layer of the second neural network to produce the second output.
10. 10. The ear-mounted device of claim 9, wherein the neural network system is further configured to feed the combined first and second outputs to a subsequent layer of the first neural network.
11. 11. The ear-mounted device of claim 1, wherein the neural network system is configured to operate the first neural network for a warm-up period after resetting one or more states of the first neural network and weight an output of the first neural network to zero for the warm-up period.
12. 12. The ear-mounted device of claim 1, wherein the neural network system is configured to shut down the first neural network for a shutdown period prior to resetting one or more states of the first neural network.
13. 13. The ear-mounted device of claim 1, wherein the weight applied to the output of the first neural network depends at least in part on an amount of time that has elapsed since the one or more states of the first neural network were reset.
14. 14. The ear-mounted device of claim 13, wherein the weight applied to the output of the first neural network transitions from low to high after resetting one or more states of the first neural network and then transitions from high to low before the next resetting of one or more states of the first neural network.
15. 15. The ear-worn device of claim 1, wherein the neural network circuit is implemented on-chip.
16. 16. The ear-mounted device of claim 1, wherein the first output from the first neural network comprises a combination of a plurality of outputs from the first neural network, and the neural network system is configured to wait until the first neural network produces the plurality of outputs before determining the first output.
17. 17. The ear-mounted device of claim 1, wherein the first weight and the second weight are determined according to a weighting scheme comprising a linear piecewise function or a smoothing function.
18. 18. The ear-mounted device of claim 1, wherein the first neural network and the second neural network are trained to reduce noise in an audio signal.
19. 19. The ear-mounted device of claim 1, wherein the first neural network and the second neural network comprise the same architecture and the same weights.
20. 20. The ear-worn device of claim 1, wherein the first neural network and the second neural network comprise a recurrent neural network.

Description

Headset device with parallel neural network Technical Field The present disclosure also relates to an apparatus, such as an ear-worn device, employing a neural network to process signals. Background Hearing aids and the like are used to help hearing impaired persons to better listen. Typically, hearing aids amplify received sound. Some hearing aids attempt to remove ambient noise from incoming sound. Disclosure of Invention A Recurrent Neural Network (RNN) is a neural network where the result of processing of one time step may affect the processing of a subsequent time step. Thus, the RNN has a "state" that can last from one time step to another, which represents context information derived from analysis of previous inputs. It should be understood that other types of neural networks may also be stateful (i.e., have and use states), and that the methods described herein may also be used. The RNN may include an input layer, one or more RNN layers, and an output layer. Each RNN layer may include input nodes, output nodes, and states. In some embodiments, there may be a state, which may be referred to as a "hidden state". In some embodiments, such as a Long Short Term Memory (LSTM) type RNN, there may be two states, which may be referred to as a "hidden state" and a "cell state. At each time step, the state of the previous time step may be concatenated with the input of the current time step. Recurrent neural networks and other stateful neural networks may have drawbacks. Over time, the state in the neural network may drift and obtain a set of values that were never obtained during the training process. Thus, the performance of the neural network may decrease over time. Long term performance degradation may be avoided by resetting certain states (i.e., one or more states) of the neural network. However, doing so may have the disadvantage that the performance of the neural network may drop immediately after the reset state, as the neural network operates without the context information obtained from processing the previous input. Thus, while resetting the state may solve the problem of state drift to a value that has never been reached during training, another problem arises. The inventors have found that to solve these two problems of stateful neural networks, multiple neural networks may operate in parallel on the same input, but their states may reset at mutually offset times. In this way, both neural networks can avoid the problem of long-term degradation, and at any given point in time, at least one of the neural networks may have established state information that facilitates calculation of its output. The outputs of the neural networks may be combined so that the combined outputs of the parallel neural networks more fully exploit the predictions of the neural network whose state is at the optimal processing point. Thus, the neural network system can reduce its dependence on the neural network whose state is at a non-optimal point. In other words, the output of one neural network may be weighted more than the output of another neural network. For example, the weight of the newly reset neural network may be lower than the weight of another neural network. The parallel neural networks may have different architectures and different weights, or have the same architecture and weights. Due to the reset time interleaving of states, parallel neural networks may have different states even though their architecture and weights are the same. In general, one or more states in the neural network may be reset. In some embodiments, all states in the neural network may be reset. In some embodiments, only certain types of states in the neural network may be reset. For example, in an LSTM neural network, in some embodiments, only the cell state may be reset, and the hidden state is not reset. In some embodiments, only certain states of one or more layers of the neural network may be reset, and states from different layers or groups of layers may be reset at different times. As a specific example, all states of one layer (e.g., layer 1) may be reset at one time, then all states of a different layer (e.g., layer 2) may be reset at another time, and so on. Thus, as described herein, one or more states of a reset neural network may refer to all states in the reset neural network, only certain states (e.g., certain types or more of states) on the reset neural network, all states of one or more layers (e.g., one layer) in the reset neural network, and/or certain states (e.g., certain types or more of states) of one or more layers (e.g., one layer) of the reset neural network. As described herein, resetting a state may refer to actively changing a value in the state to 0, or actively changing a value of the state to other values than zero. Further, as described herein, resetting a state may refer to actively changing a value in a state immediately or for a limited time. In the latter case, the reset may be smooth, such that the va