CN-121981170-A - Neural network generator

Abstract

The invention discloses a neural network generator. A neural network is a collection of interconnected nodes, where each node processes input data and outputs the processing results to any other node connected thereto. Neural networks are typically composed of various layers that perform different processing tasks at their inputs. Systems, methods, and computer program products are provided for generating one or more functional layers in a neural network.

Inventors

• M. Drodorsky
• M. Vilmo

Assignees

• 辉达公司

Dates

Publication Date

20260505

Application Date

20191231

Priority Date

20181231

Claims (20)

1. 1. A processor, comprising: One or more circuits to adjust a number of inputs and/or a number of outputs of one or more nodes corresponding to one or more portions of the generated neural network portion based at least in part on whether the one or more nodes are likely to generate valid results.
2. 2. The processor of claim 1, wherein the one or more circuits are to generate one or more linear equations from a matrix comprising values of one or more parameters.
3. 3. The processor of claim 1, wherein the one or more circuits are to receive a matrix generated based on constraints defined on: the size of the matrix, and A range of one or more values in a matrix of values.
4. 4. The processor of claim 1, wherein the one or more circuits are to receive a matrix defining a number of anchor nodes for the one or more portions of the generated neural network portion and connections between the anchor nodes.
5. 5. The processor of claim 1, wherein the one or more portions of the generated neural network portion are generated using one or more linear equations by: determining a volume of each of a plurality of anchor nodes in the one or more portions of the generated neural network portion, and The dimensions of each anchor node are determined based on the volumes of the anchor nodes.
6. 6. The processor of claim 5, wherein the volume of each of the plurality of anchor nodes is determined by: Simplifying the one or more linear equations such that each linear equation represents a volume of a respective anchor node as a function of one or more volumes of input anchor nodes in the anchor node, Determining an integer scale factor for the anchor node from the one or more reduced linear equations, Determining a least significant body of the one or more input anchor nodes using the integer scale factor, Selecting a volume of each of the one or more input anchor nodes based on the determined minimum valid volume of each of the one or more input anchor nodes, and Based on the selected volumes of each of the one or more input anchor nodes and the simplified one or more linear equations, volumes of each of the remaining anchor nodes are calculated.
7. 7. The processor of claim 1, wherein the one or more circuits are to randomly generate the neural network portion.
8. 8. The processor of claim 1, wherein the one or more circuits are to solve a set of linear equations to indicate an effective dimension of the one or more nodes in the generated neural network portion.
9. 9. The processor of claim 1, wherein the one or more circuits are to calculate a number of rows and a number of columns of a matrix using the matrix as input.
10. 10. A non-transitory computer-readable medium having stored thereon a set of instructions that, when executed by one or more processors, cause the one or more processors to: The number of inputs and/or the number of outputs of one or more nodes corresponding to one or more portions of the generated neural network portion is adjusted based at least in part on whether the one or more nodes are likely to generate valid results.
11. 11. The non-transitory computer-readable medium of claim 10, wherein the set of instructions, when executed by the one or more processors, cause the one or more processors to receive as input a matrix containing values of one or more parameters, wherein the matrix is generated based on constraints defined on: the size of the matrix, and A range of one or more values in a matrix of values.
12. 12. The non-transitory computer-readable medium of claim 10, wherein the set of instructions, when executed by the one or more processors, cause the one or more processors to receive as input a matrix containing values of one or more parameters, wherein the matrix defines a number of anchor nodes for one or more portions of the generated neural network portion and connections between the anchor nodes.
13. 13. The non-transitory computer-readable medium of claim 10, wherein one or more portions of the generated neural network portion are generated using one or more linear equations by: Determining a volume of each of a plurality of anchor nodes in the one or more portions of the generated neural network, and The dimensions of each of the anchor nodes are determined based on the volumes of the anchor nodes.
14. 14. The non-transitory computer readable medium of claim 13, wherein the body of each of the plurality of anchor nodes is determined by: Simplifying the one or more linear equations such that each linear equation represents a volume of a respective anchor node as a function of one or more volumes of input anchor nodes in the anchor node, Determining an integer scale factor for the anchor node from the one or more reduced linear equations, Determining a least significant body of the one or more input anchor nodes using the integer scale factor, Selecting a volume of each of the one or more input anchor nodes based on the determined minimum valid volume of each of the one or more input anchor nodes, and Based on the selected volumes of each of the one or more input anchor nodes and the simplified linear equation, volumes of each of the remaining anchor nodes are calculated.
15. 15. The non-transitory computer-readable medium of claim 10, wherein the set of instructions, when executed by the one or more processors, cause the one or more processors to solve a set of linear equations to randomly generate a neural network portion.
16. 16. The non-transitory computer-readable medium of claim 10, wherein one or more portions of the generated neural network portion are further generated by: a function of each of a plurality of connections between anchor nodes is determined based on the dimensions of the anchor nodes to which the connections between the anchor nodes are linked.
17. 17. The non-transitory computer-readable medium of claim 16, wherein functions for linking respective ones of the connections of a first one of the anchor nodes and a second one of the anchor nodes comprise one or more operations that convert dimensions of the first anchor node to dimensions of the second anchor node.
18. 18. A method, comprising: The number of inputs and/or the number of outputs of one or more nodes corresponding to one or more portions of the generated neural network portion is adjusted based at least in part on whether the one or more nodes are likely to generate valid results.
19. 19. The method of claim 18, wherein one or more portions of the generated neural network portion are generated by: One or more portions of the generated neural network portion are randomly generated using one or more linear equations, wherein a solution of the one or more linear equations indicates that the one or more nodes are valid.
20. 20. The method of claim 18, further comprising: receiving the matrix as input, and The number of rows and the number of columns of the matrix are calculated, the number of rows and the number of columns corresponding to the dimensions of the inputs and outputs of the one or more nodes of the generated neural network portion.

Description

Neural network generator The patent application of the invention is a divisional application of the patent application of the invention with the application date of 2019, 12 months and 31 days, the application number of 201911412065.6 and the invention name of a neural network generator. Technical Field One embodiment relates to generating a neural network. Background In general, a neural network is a collection of interconnected nodes, where each node processes input data and outputs the processing results to any other node connected thereto. Neural networks are typically composed of various layers that perform different processing tasks on their inputs. Unfortunately, current techniques for generating neural networks typically use cumbersome processes that may not guarantee efficient working results. There is a need to address these problems and/or other problems associated with the prior art. Drawings FIG. 1 illustrates a flow chart of a method for generating at least a portion of a neural network, according to one embodiment. Fig. 2A shows an illustration of a directed graph representation of a portion of a neural network constructed by a matrix, according to one embodiment. FIG. 2B illustrates the directed graph representation of FIG. 2A with input nodes and output nodes connected to the portion of the neural network, according to one embodiment. FIG. 2C illustrates the directed graph representation of FIG. 2B with computed indications of node volumes, according to one embodiment. FIG. 2D illustrates the directed graph representation of FIG. 2C with an indication of the determined node dimensions, according to one embodiment. FIG. 2E illustrates a function of a connection between node X1 and node X2 in the directed graph representation of FIG. 2D, according to one embodiment. FIG. 2F illustrates a function of a connection between node X2 and node X3 in the directed graph representation of FIG. 2D, according to one embodiment. FIG. 2G illustrates an exemplary neural network portion resulting in part from the functions of FIG. 2E and the functions of FIG. 2F, according to one embodiment. FIG. 2H illustrates a directed graph representation of a subgraph constructed from a matrix of portions of the neural network of FIG. 2G, according to one embodiment. FIG. 2I illustrates the directed graph representation of FIG. 2H with an indication of the calculated node volumes and the determined node dimensions, according to one embodiment. FIG. 3 illustrates a parallel processing unit according to one embodiment. FIG. 4A illustrates a general processing cluster within the parallel processing unit of FIG. 3, according to one embodiment. FIG. 4B illustrates a memory partition unit of the parallel processing unit of FIG. 3, according to one embodiment. FIG. 5A illustrates the streaming multiprocessor of FIG. 4A, according to an embodiment. Fig. 5B is a conceptual diagram of a processing system implemented using the PPU of fig. 3, according to one embodiment. Fig. 5C illustrates an exemplary system in which the various architectures and/or functions of the various previous embodiments may be implemented. FIG. 6 is a conceptual diagram of a graphics processing pipeline implemented by the PPU of FIG. 3, according to one embodiment. Detailed Description FIG. 1 illustrates a flow chart of a method 100 for generating at least a portion of a neural network, according to one embodiment. The "portion" of the neural network described below may refer to a sub-portion of the neural network that is connected to one or more other sub-portions of the neural network, or to the entire neural network. In one embodiment, the method 100 may be performed by one or more processing units, and/or may also be performed by a program, custom circuitry, or a combination thereof. In one embodiment, the method 100 may be performed by a GPU (graphics processing unit), a CPU (central processing unit), or any processor capable of performing arithmetic computations, such as those described below with reference to one or more of the remaining figures. Further, those of ordinary skill in the art will appreciate that any system that performs the method 100 is within the scope and spirit of the embodiments described herein. In operation 102, a set of linear equations is received as input. In one embodiment, the system of linear equations includes integer linear equations. In one embodiment, the number of linear equations in the system of linear equations is equal to the number of anchor nodes (anchor nodes) to be included in a portion of the neural network generated using the system of linear equations. In one embodiment, each linear equation in the set of linear equations includes a plurality of variables, each variable representing a corresponding anchor node to be included in the portion of the neural network. In one embodiment, the coefficients for each variable are defined based on a predefined range of values (e.g., in a pseudo-random manner). In one