JP-2026075385-A - Neural network implementation device, method and program thereof, and neural network device

Abstract

[Problem] The present invention provides a neural network implementation device, a neural network implementation method, and a neural network implementation program for implementing a subset network of a neural network that can improve the accuracy of the correct answer in an arithmetic circuit, as well as a neural network device equipped with the neural network implementation device. [Solution] The neural network implementation device of the present invention is a device that implements a subset network of a neural network, which is equipped with pseudorandom numbers generated by a pseudorandom number generator as edge weights, into an arithmetic circuit AC, wherein the weights of some of the multiple edges in the subset network are weights generated by machine learning. [Selection Diagram] Figure 1

Inventors

• 能勢 陽平
• 石橋 真人
• 的野 賢一

Assignees

• マツダ株式会社

Dates

Publication Date

20260508

Application Date

20241022

Claims (7)

1. A neural network implementation device that implements a subset network of a neural network, which uses pseudorandom numbers generated by a pseudorandom number generator as edge weights, into an arithmetic circuit, The weights of some of the edges among the multiple edges in the aforementioned subset network are weights generated by machine learning. A neural network implementation device.
2. An external memory circuit, separate from the arithmetic circuit, stores, for each edge in the neural network, supermask information, which is information representing a supermask indicating whether or not the edge is valid, and trained weight information, which is information representing the weights generated by machine learning among the weights of each edge in the subset network. A pseudo-random number generator that generates the aforementioned pseudo-random numbers, For each edge in the neural network, the system includes a selector that, based on the supermask information, selects either the weight represented by the learned weight information or a pseudorandom number generated by the pseudorandom number generator, and outputs the selected one as the weight of the edge to the arithmetic circuit, if the edge is valid. The neural network implementation device according to claim 1.
3. The weights generated by the aforementioned machine learning are arranged at equal intervals when the weights of each edge in the subset network are sequentially lined up in a row from the first weight of the first layer to the last weight of the final layer. The neural network implementation device according to claim 1.
4. A neural network implementation method that implements a subset network of a neural network in an arithmetic circuit, which uses pseudorandom numbers generated by a pseudorandom number generator as edge weights, The weights of some of the edges among the multiple edges in the aforementioned subset network are weights generated by machine learning. Methods for implementing neural networks.
5. A neural network implementation program for causing a computer to function as a neural network implementation device according to any one of claims 1 to 3.
6. A neural network implementation device according to any one of claims 1 to 3, The arithmetic circuit is provided as described above. A neural network device.
7. A neural network implementation device according to claim 2, The arithmetic circuit is provided, The calculation circuit comprises the pseudo-random number generator and the selector. A neural network device.

Description

This invention relates to a neural network implementation device for implementing a neural network in an arithmetic circuit, a neural network implementation method, a neural network implementation program, and a neural network device equipped with the neural network implementation device. In recent years, so-called artificial intelligence has been researched, developed, and is progressing. One such theory is the Hidden Neural Network Theory (Strong Lottery Ticket Hypothesis), which is disclosed, for example, in Patent Document 1. In this Hidden Neural Network Theory, each weight assigned to each edge (connection) between each node in the neural network is generated by a pseudo-random number generator, and the importance (score) of the edges is machine-learned by fixing each weight. Edges with scores above a predetermined threshold are extracted, and the network formed by the extracted edges and their nodes (subnetwork, subneural network) is selected (extracted) from the neural network as the neural network of the inference model (prediction model) actually used for inference (prediction). The training dataset used for machine learning the scores is the same training dataset used when machine learning each weight of each edge in a neural network in normal machine learning. The aforementioned score (importance) increases as the edge contributes (influences) more to reducing the error between the correct information (training data) and the output result of the neural network for multiple inputs. In a typical neural network, each weight of each edge is acquired (determined) through machine learning. However, in a hidden neural network, each weight of each edge is fixed, and machine learning is not performed on each weight. Instead, as described above, the edge scores are learned through machine learning. When implementing the neural network of the inference model in an computational circuit, such as an inference LSI (Large Scale Integration), it is necessary to transfer the weights of each edge in the neural network of the inference model, which are stored in an external memory circuit, to the computational circuit. In Hidden Neural Network Theory, by equipping the computational circuit with a pseudo-random number generator to generate the weights of each edge, it is sufficient to transfer a supermask indicating whether each edge is valid or not from the external memory circuit to the computational circuit. Therefore, the amount of data transferred from the external memory circuit to the computational circuit can be reduced compared to a normal neural network using the entire network. Japanese Patent Publication No. 2023-078975 This is a block diagram showing the configuration of a neural network device that includes a neural network implementation device in an embodiment.This figure illustrates the distribution of trained weights and the generation of selection control signals in a neural network device equipped with the aforementioned neural network implementation device, where the pseudo-random number weight fixing rate α is 0.5 or less.This figure illustrates the distribution of trained weights and the generation of selection control signals in a neural network device equipped with the aforementioned neural network implementation device, when the pseudo-random number weight fixing rate α is greater than 0.5.This is a flowchart illustrating the operation of generating a selection control signal in a neural network device equipped with the aforementioned neural network implementation device.As an example, this graph shows the relationship between the number of machine learning steps (epochs) in the score and the number of correct answers in the resulting subset network.As an example, this graph shows the relationship between the pseudo-random number weighting rate, the number of correct answers, and the amount of data transferred. Hereinafter, one or more embodiments of the present invention will be described with reference to the drawings. However, the scope of the invention is not limited to the disclosed embodiments. Components denoted by the same reference numerals in each figure indicate the same component, and their descriptions are omitted where appropriate. In this specification, general reference numerals are used without subscripts, while individual components are indicated with subscripts. The neural network device in this embodiment comprises a neural network implementation device and an arithmetic circuit. This neural network implementation device is a device that implements a subset network of a neural network, which uses pseudorandom numbers generated by a pseudorandom number generator as edge weights, into the arithmetic circuit. In this embodiment, the weights of some of the multiple edges in the subset network of this neural network implementation device are weights generated by machine learning. The arithmetic circuit is a circuit that performs calculations, and is the circuit on which the