US-12625677-B2 - Circuit for efficiently performing operations on input data to compute an interpretable and differentiable function

Abstract

A circuit for efficiently performing operations on input data to compute an interpretable and differentiable function, comprising a first level processing unit that obtains one or more first level inputs and a second level processing unit. The first level processing unit comprises (i) a first level MA unit that is configured to compute a first level weighted sum of the one or more first level inputs, and (ii) a logarithmic unit that is configured to compute a first level output. The second level processing unit obtains one or more second level inputs. The second level processing unit comprising (i) a second level MA unit that is configured to compute a second level weighted sum of the one or more second level inputs, and adding the computed second level product, and (ii) an antilog unit that is configured to compute a second level output.

Inventors

• JAYADEVA

Assignees

• SPARSEMIND TECHNOLOGY LABS PRIVATE LIMITED

Dates

Publication Date

20260512

Application Date

20250609

Priority Date

20240610

Claims (20)

1. 1 . A circuit for efficiently performing operations on input data to compute an interpretable and differentiable function, comprising: a memory that comprises a first set of instructions and a second set of instructions; and at least one first level processing unit that executes the first set of instructions and is configured to: obtain one or more first level inputs, wherein each first level input is associated with a first level weight, the first level processing unit comprising: (i) a first level multiply-add (MA) unit that is configured to compute a first level weighted sum of the one or more first level inputs by multiplying each first level input with the associated first level weight to compute a first level product and adding the computed first level products, wherein the first level weighted sum is the weighted sum of the first-level inputs; and (ii) a log circuit that is configured to compute a first level output by performing a logarithmic operation on the first level weighted sum, wherein the first level output is the logarithm of the first-level weighted sum of the first level inputs; and at least one second level processing unit that executes the second set of instructions and is configured to: obtain one or more second level inputs comprising the first level output of each first level processing unit or at least one of (i) the one or more first level inputs of each first level processing unit (ii) the first level weighted sum of each first level processing unit and (iii) the first level output of each first level processing unit, wherein each second level input is associated with a second level weight, the second level processing unit comprising: (i) a second level multiply-add (MA) unit that is configured to compute a second level weighted sum of the one or more second level inputs by multiplying each second level input with the associated second level weight to compute a second level product, and adding the computed second level product, wherein the second level weighted sum is a weighted sum of the second level inputs; and (ii) an antilog circuit that is configured to compute a second level output by performing an antilog operation on the second level weighted sum, wherein the second level output is a product of weighted polynomials of the first level inputs or a product of (i) weighted polynomials of the one or more first level inputs and (ii) weighted exponentials of the one or more first level inputs and (iii) weighted exponentials of the one or more first level weighted sum of the first level processing units, wherein the second level output is the interpretable and differentiable function, wherein the memory is configured to store (i) the first level inputs, the first level weights, the first level weighted sums, the first level output of each first level processing unit, and (ii) the second level inputs, the second level weights, the second level weighted sums, and the second level output of each second level processing unit.
2. 2 . The circuit of claim 1 , wherein the first set of instructions and the second set of instructions are configured to initialize the weights of the one or more first level inputs and the one or more second level inputs to compute the second level output corresponding to the one or more first level inputs, wherein the first set of instructions and the second set of instructions are configured to update the weights of the one or more first level inputs and the one or more second level inputs using a training algorithm that is selected from one or more training algorithms based on the first level inputs and the second level inputs, to learn an input-output mapping function when the circuit dynamically receives input data, wherein the first set of instructions and the second set of instructions are further configured to update the weights of the one or more first level inputs and the one or more second level inputs by (i) producing an expression corresponding to the second level output; and (ii) computing derivatives or partial derivatives of desired orders of the second level output by using the expression corresponding to the second level output, wherein the first set of instructions and the second set of instructions are further configured to determine an importance of the one or more first level inputs using the updated weights associated with the one or more first level inputs and the one or more second level inputs, removing near-zero weights assigned to at least one of the one or more first level inputs to improve efficiency and reduce overfitting, thereby the circuit automatically learns which first level inputs are more relevant during training; and wherein the circuit comprises a control unit that is configured to orchestrate the first set of instructions and the second set of instructions of the circuit.
3. 3 . The circuit of claim 1 , the circuit comprising, at least one third level processing unit that executes a third set of instructions and is configured to: obtain one or more third level inputs comprising (a) the second level output of the at least one second level processing unit or (b) the one or more first level inputs of each first level processing unit, the first level weighted sum of each first level processing unit, the first level output of each first level processing unit, the second level weighted sum of each second level processing unit, and the second level output of each second level processing unit, wherein each third level input is associated with a third level weight, the third level processing unit comprising, a third level multiply-add (MA) unit that is configured to compute a third level weighted sum of the one or more third level inputs by multiplying each third level input with the associated third level weight to compute a third level product, and adding the computed third level products, wherein the third level weighted sum is the weighted sum of the third level inputs of third level processing units, and compute a third level output by performing an output function operation on the third level weighted sum, wherein the third level output is the output function operated on the third level weighted sum.
4. 4 . The circuit of claim 3 , the circuit comprising, the memory that is configured to store the third level inputs, the third level weights, the third level weighted sums, and the third level output of each third level processing unit; the first set of instructions, the second set of instructions, and the third set of instructions that are configured to: initialize the weights of the one or more first level inputs, the one or more second level inputs, and the one or more third level inputs to compute the third level output, wherein the first set of instructions, the second set of instructions, and the third set of instructions are further configured to update the weights of the one or more first level inputs, the one or more second level inputs, and the one or more third level inputs by (i) producing an expression corresponding to the third level output; and (ii) computing derivatives or partial derivatives of desired orders of the third level output by using the expression corresponding to the third level output; and a control unit configured to orchestrate the first set of instructions, the second set of instructions, and the third set of instructions of the circuit.
5. 5 . The circuit of claim 4 , wherein the circuit is configured to represent shapes and functions by using the weights of the first level, second level, and third level processing units, wherein the circuit is configured to identify similar shapes and functions by matching the weights corresponding to stored shapes and functions with the weights corresponding to shapes and functions in the input.
6. 6 . The circuit of claim 3 , wherein the second level weights associated with the second level inputs of the processing unit are all set to one to further optimize computational efficiency.
7. 7 . The circuit of claim 3 , wherein weights of a first circuit are initialized and modified by configuring the first set of instructions, the second set of instructions, and the third set of instructions that operate on the weighted sums, the inputs, and the outputs of the first circuit and one or more second circuits.
8. 8 . The circuit of claim 3 , wherein the first circuit is configured to produce an encrypted signal of the input, wherein the second circuit decrypts of the encrypted signals by a second three layer circuit, with the weights associated with the first, second, and third level inputs of the second circuit having condition with the weights associated with the third, second, and first layer inputs of the first circuit, respectively, wherein the weights of the connections are represented by matrices, wherein the matrices satisfy the conditions to enable the decryption, wherein the conditions are if all matrices contain real-valued elements or if the matrices contain complex-valued elements, wherein a number of inputs, and a number of processing units at the first, second, and third levels are all equal.
9. 9 . The circuit of claim 3 , wherein the third level outputs of the first circuit are fed to the a second circuit as input, wherein the first circuit encodes the first level inputs of the first circuit to produce the third level outputs to change a dimensionality of the input that is suitable for the second circuit; and wherein the second circuit is configured to decode the third level outputs of the first circuit to produce the first level inputs of the first circuit.
10. 10 . The circuit of claim 1 , wherein the second level output is a product of degree one polynomials of the one or more first level inputs, wherein the second level output is the interpretable and differentiable function composed of a product of linear polynomials, wherein different differentiable functions are computed by changing the weights associated with first level inputs.
11. 11 . The circuit of claim 1 , wherein the circuit is embodied to implement a two-layer neural network, comprising: (a) at least one of first layer neuron circuit that obtains one or more first layer inputs, each first layer input associated with a first layer weight, wherein each first layer neuron circuit is configured to (i) compute a first layer weighted sum by multiplying each first layer input with the associated first layer weight to compute a first layer product and adding the computed first layer products, and (ii) pass the first layer weighted sum through a logarithm activation function to produce a first layer neuron output, wherein the first layer weighted sum is a weighted sum of the first level inputs, and wherein the first layer output is the logarithm of the first layer weighted sum of the first layer inputs; and (b) at least one second layer neuron circuit that obtains at least one of second layer inputs comprising the one or more first layer inputs of each first layer neuron circuit, the first layer weighted sum of each first layer neuron circuit, and the first layer output of each first layer neuron circuit, wherein each second layer input is associated with a second layer weight; wherein each second layer neuron circuit is configured to (i) compute a second layer weighted sum by multiplying each second layer input by the associated second layer weight to compute a second layer product, and adding the computed second layer products, and (ii) pass the second layer weighted sum through an antilog activation function to produce a second layer neuron output; wherein the second layer output comprises a product of (i) weighted polynomials of the one or more first layer inputs, (ii) the weighted exponentials of the first layer inputs, and (iii) weighted exponentials of the first layer weighted sums.
12. 12 . The circuit of claim 11 , wherein the circuit is embodied to implement the two-layer neural network capable of inferencing and learning the weights of the one or more first layer inputs of each first layer neuron circuit, the weights of the one or more second layer inputs of each second layer neuron circuit, and the second layer output of each second layer neuron circuit, wherein the circuit: selects a training algorithm to compute the weights of the circuit used in the two-layer neural network, using the first layer inputs and the second layer inputs obtained for training the neural network; executes the selected training algorithm; uploads the weights associated with the one or more first layer inputs and the one or more second layer inputs into the circuit using the first set of instructions, and the second set of instructions, wherein the first set of instructions, and the second set of instructions are defined for inferencing; and generates, by the two-layer neural network, the second level output using the updated weights learned by the selected training algorithm.
13. 13 . The circuit of claim 11 , wherein the circuit is embodied to implement a three-layer neural network comprising: at least one of third layer neuron circuit that obtains at least one of third layer input, comprising the one or more first layer inputs of each first layer neuron circuit, the first layer weighted sum of each first layer neuron circuit, the first layer output of each first layer neuron circuit, the second layer output of each second layer processing unit, the second layer weighted sum of each second layer neuron circuit, and the second layer output of each second layer neuron circuit, wherein each third layer neuron circuit is configured to, (i) compute a third layer weighted sum by multiplying each third layer input by an associated third layer weight to compute a third layer product and add the computed third layer products; and (ii) produce a third layer output by passing the third layer weighted sum through an output activation function, wherein the third layer output is the output activation function applied to the third layer weighted sum.
14. 14 . The circuit of claim 13 , wherein the circuit is embodied to implement the three-layer neural network capable of inferencing and learning the associated weights of the one or more first layer inputs of each first layer neuron circuit, the associated weights of the one or more second layer inputs of each second layer neuron circuit, the second layer output of each second layer neuron circuit, the associated weights of the one or more third layer inputs of each first third layer neuron circuit, and the third layer output of each third layer neuron circuit, wherein the circuit, selects a training algorithm to compute the weights of the circuit used in the three-layer neural network, using the first layer inputs, the second layer inputs, the third layer inputs, the first layer outputs, the second layer outputs, and the third layer outputs obtained for training the neural network; executes the selected training algorithm; uploads the weights associated with the one or more first layer inputs, the weights associated with one or more second layer inputs, and the weights associated with one or more third layer inputs into the circuit using the first set of instructions, the second set of instructions, and the third set of instructions, wherein the first set of instructions, the second set of instructions, and the third set of instructions are defined for inferencing; and generates, by the three-layer neural network, the third layer outputs using the updated weights learned by the selected training algorithm.
15. 15 . A method for efficiently performing operations on input data to compute an interpretable and differentiable function implemented by a circuit, the method comprising: obtaining, by at least one first level processing unit of the circuit, one or more first level inputs, wherein each first level input is associated with a first level weight; computing, by the at least one first level processing unit, a first level weighted sum of the one or more first level inputs by multiplying each first level input by the associated first level weight to compute a first level product and adding the computed first level products, wherein the first level weighted sum is the weighted sum of the first-level inputs; computing, by the at least one first level processing unit, a first level output by performing a logarithmic operation on the first level weighted sum, wherein the first level output is the logarithm of the first-level weighted sum of the first level inputs; performing a skip connection to obtain, by at least one second level processing unit of the circuit, one or more second level inputs by selecting from among at least one of (i) the one or more first level inputs of each first level processing unit (ii) the first level weighted sum of each first level processing unit and (iii) the first level output of each first level processing unit, wherein each second level input is associated with a second level weight; computing, by the at least one second level processing unit, a second level weighted sum of the one or more second level inputs by multiplying each second level input with the associated second level weight to compute a second level product, and adding the computed second level product, wherein the second level weighted sum is a weighted sum of the second level inputs; and computing, by the at least one second level processing unit, a second level output by performing an antilog operation on the second level weighted sum, wherein the second level output is a product of weighted polynomials of the first level inputs or a product of (i) weighted polynomials of the one or more first level inputs and (ii) weighted exponentials of the one or more first level inputs and (iii) weighted exponentials of the one or more first level weighted sum of the first level processing units, wherein the second level output is the interpretable and differentiable function, wherein circuit comprises a memory that is configured to store (i) the first level inputs, the first level weights, the first level weighted sums, the first level output of each first level processing unit, and (ii) the second level inputs, the second level weights, the second level weighted sums, and the second level output of each second level processing unit; and initializing, by a first set of instructions and a second set of instructions in the memory, the weights of the one or more first level inputs and the one or more second level inputs to compute the second-level output corresponding to the one or more first level inputs; and orchestrating, by a control unit of the circuit, the first set of instructions and the second set of instructions of the circuit.
16. 16 . The method of claim 15 , wherein the first set of instructions and the second set of instructions are configured to update the weights of the one or more first level inputs and the one or more second level inputs using a training algorithm that is selected from one or more training algorithms based on the first level inputs and the second level inputs, to learn an input-output mapping function when the circuit dynamically receives input data, wherein the first set of instructions and the second set of instructions are further configured to update the weights of the one or more first level inputs and the one or more second level inputs by (i) producing an expression corresponding to the second level output; and (ii) computing derivatives or partial derivatives of desired orders of the second level output by using the expression corresponding to the second level output, wherein the first set of instructions and the second set of instructions are further configured to determine an importance of the one or more first level inputs using the updated weights associated with the one or more first level inputs and the one or more second level inputs, removing near-zero weights assigned to at least one of the one or more first level inputs to improve efficiency and reduce overfitting, thereby the circuit automatically learns which first level inputs are more relevant during training.
17. 17 . The method of claim 15 , the method comprising, obtaining, by at least one third level processing unit of the circuit, one or more third level inputs comprising (a) the second level output of the at least one second level processing unit or (b) the one or more first level inputs of each first level processing unit, the first level weighted sum of each first level processing unit, the first level output of each first level processing unit, the second level weighted sum of each second level processing unit, and the second level output of each second level processing unit, wherein each third level input is associated with a third level weight; computing, by the at least one third level processing unit, a third level weighted sum of the one or more third level inputs by multiplying each third level input with the associated third level weight to compute a third level product, and adding the computed third level products, wherein the third level weighted sum is the weighted sum of the third level inputs of third level processing units; and computing, by the at least one third level processing unit, a third level output by performing an output function operation on the third level weighted sum, wherein the third level output is the output function operated on the third level weighted sum.
18. 18 . The method of claim 17 , the method comprising, storing the third level inputs, the third level weights, the third level weighted sums, and the third level output of each third level processing unit in the memory; initializing, by the first set of instructions, the second set of instructions, and the third set of instructions stored in the memory, the weights of the one or more first level inputs, the one or more second level inputs, and the one or more third level inputs to compute the third level output, wherein the first set of instructions, the second set of instructions, and the third set of instructions are further configured to update the weights of the one or more first level inputs, the one or more second level inputs, and the one or more third level inputs by (i) producing an expression corresponding to the third level output; and (ii) computing derivatives or partial derivatives of desired orders of the third level output by using the expression corresponding to the third level output; and orchestrating, by the control unit of the circuit, the first set of instructions, the second set of instructions, and the third set of instructions of the circuit.
19. 19 . The method of claim 15 , wherein the second level output is a product of degree one polynomials of the one or more first level inputs, wherein the second level output is the interpretable and differentiable function composed of a product of linear polynomials, wherein different differentiable functions are computed by changing the weights associated with first level inputs.
20. 20 . The method of claim 15 , wherein the second level weights associated with the second level inputs of the processing unit are all set to one to further optimize computational efficiency.

Description

BACKGROUND Technical Field The embodiments herein generally relate to an interpretable neural network and, more particularly, a circuit for efficiently performing operations on input data to compute an interpretable and differentiable function. The disclosure also relates to a processor-implemented method for efficiently performing operations on input data to compute an interpretable and differentiable function. Description of the Related Art In recent years, there has been a growing interest in developing neural network architectures that can effectively learn and represent complex input-output mappings in a transparent and interpretable manner. One significant challenge in this area is balancing model complexity with interpretability, as traditional neural networks often lack transparency due to their black-box nature. Another key challenge is attaining differentiable computation that is both stable and meaningful across all layers of the network. In many traditional architectures, differentiation can become numerically unstable or lose significance in very deep models, leading to gradient vanishing or exploding issues. This hampers effective learning and limits the ability to precisely control or interpret how parameter changes affect the overall function being learned. Overfitting is also a persistent problem in standard neural networks, particularly those with a large number of parameters. Models tend to memorize training data instead of learning generalized patterns, which leads to poor performance on new, unseen data. Deep architectures, due to their high capacity, are especially prone to this behavior when trained on limited or noisy datasets. Although it has been shown that a 3-layer neural network can learn any input-output map, a constructive method for realizing such a network has not been established in the literature. Most current AI/ML methods use deep neural network architectures with a large number of layers. Deep neural networks often comprise hundreds or even thousands of layers. The performance of Large Language Models (LLMs) and image recognition systems stems from these deep architectures. However, such networks contain billions of parameters, demanding immense computational resources for both training and inference. The resulting high-power consumption poses a major challenge to their deployment, particularly on resource-constrained platforms. Recently, there has been a renewed focus on neural networks with a small number of layers that can learn from a given dataset. Traditional transformer neural networks face challenges in learning long-range dependencies in sequential data efficiently. The computational cost of capturing dependencies that rely on pairwise correlations between tokens or features increases quadratically or even exponentially with the number of tokens used. This exponential growth in computational cost poses significant challenges in training transformer models effectively and limits their applicability to real-world tasks. Moreover, a limitation of many AI and machine learning methods is their lack of interpretability. Neural network models often fail to provide insight into how they make predictions or how recognition systems operate, functioning instead as black boxes. This becomes especially problematic in domains like experimental data analysis, where quantitative predictions alone may be insufficient, and a clear understanding of the underlying reasoning is essential for scientific interpretation. These challenges lack of interpretability, unstable differentiability, and overfitting, are compounded by the software-based, abstract nature of typical neural networks. As an alternative, focusing on circuit-level implementations of neural networks offers a structured and physical framework that can help address these issues. However, implementing such models directly in circuit form introduces its complexities, such as maintaining smooth differentiability and functional clarity while minimizing hardware resource usage. The challenge lies in designing circuits that not only compute correctly, but also inherently support traceable, generalizable, and well-conditioned learning behavior. Accordingly, there remains a need for a more efficient method for mitigating and/or overcoming drawbacks associated with current methods. SUMMARY In view of the foregoing, embodiments herein provide a circuit for efficiently performing operations on input data to compute an interpretable and differentiable function. The circuit includes at least one first level processing unit and at least one second level processing unit. The at least one first level processing unit obtains one or more first level inputs. Each first level input is associated with a first level weight, the first level processing unit. The at least one first level processing unit includes a first level multiply-add (MA) unit and a logarithmic unit. The first level multiply-add (MA) unit configured to compute a first leve