US-12626111-B2 - Calibrating peripheral variability

US12626111B2US 12626111 B2US12626111 B2US 12626111B2US-12626111-B2

Abstract

Embodiments herein disclose computer-implemented methods, computer program products and computer systems for balancing neural network weight asymmetries. The computer-implemented method may include providing a neural network with weights comprising one or more major conductance pairs and one or more minor conductance pairs. The method may further include programming the one or more major conductance pairs to force an inference output to an expected duration value, determining a positive weight coefficient based on the one or more major conductance pairs and a negative weight coefficient based on the one or more minor conductance pairs, determining one or more target weights based on one or more of the positive weight coefficient and the negative weight coefficient, programming the one or more minor conductance pairs to force the inference output to the expected duration value, and programming the one or more major conductance pairs with the one or more target weights.

Inventors

STEFANO AMBROGIO
Geoffrey Burr
CHARLES MACKIN
Pritish Narayanan
Hsinyu Tsai

Assignees

INTERNATIONAL BUSINESS MACHINES CORPORATION

Dates

Publication Date: 20260512
Application Date: 20210617

Claims (20)

1 . A computer-implemented method for balancing neural network weight asymmetries, comprising: evaluating, by one or more processors, edge mirror variability of complementary metal oxide semicondutor (CMOS) circuitry located peripheral to a memory array, wherein evaluating the edge mirror variability of the CMOS circuitry includes: resetting, by the one or more processors, a conductance of each memory device of one or more major conductance pairs of memory devices and one or more minor conductance pairs of memory devices included in the memory array; programming, by one or more processors, the conductance of each memory device of the one or more major conductance pairs of memory devices to force an inference output to an expected duration value; and determining, by the one or more processors, based on the programed conductances of the one more major conductance pairs of memory devices, a current mirror mismatch between positive and negative weight branches of the memory array; determining, by the one or more processors, one or more target weights to compensate for the current mirror mismatch between the positive and negative weight branches of the memory array; and programming, by the one or more processors, the conductance of one or more memory devices of the one or more major conductance pairs of memory devices to the one or more target weights.
2 . The computer-implemented method of claim 1 , wherein the one or more major conductance pairs of memory devices are phase change material (PCM) conductances corresponding to a major positive conductance G+ and a major negative conductance G− and the one or more minor conductance pairs of memory devices are PCM conductances corresponding to a minor positive conductance g+ and a minor negative conductance g−.
3 . The computer-implemented method of claim 1 , further comprising: prior to programming the conductance of the one or more memory devices of the one or more major conductance pairs of memory devices to the one or more target weights: resetting, by the one or more processors, the conductance of each memory device of the one or more major conductance pairs of memory devices; and programming, by the one or more processors, the conductance of each memory device of the one or more minor conductance pairs of memory devices to force the inference output to the expected duration value.
4 . The computer-implemented method of claim 2 , further comprising providing one or more weights to a neural network, wherein the one or more weights comprise positive columns of memory devices in the memory array for a positive weight branch having the major positive conductance G+ and the minor positive conductance g+ and negative columns of memory devices in the memory array for a negative weight branch having the major negative conductance G− and the minor negative conductance g−.
5 . The computer-implemented method of claim 4 , further comprising: extracting, by the one or more processors, a positive weight coefficient as a sum of conductances in the positive columns for the positive weight branch; and extracting, by the one or more processors, a negative weight coefficient as a sum of conductances in the negative columns for the negative weight branch, wherein the one or more target weights are further based on one or more of the positive weight coefficient and the negative weight coefficient.
6 . The computer-implemented method of claim 1 , wherein programming the conductance of each memory device of the one or more major conductance pairs of memory devices and programming the conductance of each memory device of the one or more minor conductance pairs of memory devices further comprise: applying, by the one or more processors, a voltage input comprising a constant amplitude and increasing pulse widths to one or more weights, wherein each weight is formed from a major conductance pair of memory devices and a minor conductance pair of memory devices, and wherein the increasing pulse widths increase by a predetermined amount from a first weight of the one or more weights to a last weight of the one or more weights.
7 . The computer-implemented method of claim 1 , wherein the inference output is forced to the expected duration value when a sum of currents between the one or more major conductance pairs of memory devices or the one or more minor conductance pairs of memory devices is 0.
8 . The computer-implemented method of claim 3 , further comprising: incrementally programming, by the one or more processors, the one or more minor conductance pairs of memory devices to correct write errors incurred during the programming of the conductance of the one or more memory devices of the one or more major conductance pairs of memory devices to the one or more target weights.
9 . A computer program product for balancing neural network weight asymmetries, the computer program product comprising: one or more computer readable storage media and program instructions stored on the one or more computer readable storage media, the program instructions comprising: program instructions to evaluate edge mirror variability of complementary metal oxide semicondutor (CMOS) circuitry located peripheral to a memory array, wherein the program instructions to evaluate the edge mirror variability of the CMOS circuitry include instructions to: reset a conductance of each memory device of one or more major conductance pairs of memory devices and one or more minor conductance pairs of memory devices included in the memory array; program instructions to program the conductance of each memory device of one or more major conductance pairs of memory devices to force an inference output to an expected duration value; and program instructions to determine, based on the programed conductances of the one more major conductance pairs of memory devices, a current mirror mismatch between positive and negative weight branches of the memory array; program instructions to determine one or more target weights to compensate for the current mirror mismatch between the positive and negative weight branches of the memory array; and program instructions to program the conductance of one or more memory devices of the one or more major conductance pairs of memory devices to the one or more target weights.
10 . The computer program product of claim 9 , wherein the one or more major conductance pairs of memory devices are phase change material (PCM) conductances corresponding to a major positive conductance G+ and a major negative conductance G− and the one or more minor conductance pairs of memory devices are PCM conductances corresponding to a minor positive conductance g+ and a minor negative conductance g−.
11 . The computer program product of claim 9 , further comprising: prior to the program instructions to program the conductance of the one or more memory devices of the one or more major conductance pairs of memory devices to the one or more target weights: program instructions to reset the conductance of each memory device of the one or more major conductance pairs of memory devices; and program instructions to program the conductance of each memory device of the one or more minor conductance pairs of memory devices to force the inference output to the expected duration value.
12 . The computer program product of claim 10 , further comprising program instructions to provide one or more weights to a neural network, wherein the one or more weights comprise positive columns of memory devices in the memory array for a positive weight branch having the major positive conductance G+ and the minor positive conductance g+ and negative columns of memory devices in the memory array for a negative weight branch having the major negative conductance G− and the minor negative conductance g−.
13 . The computer program product of claim 12 , further comprising: program instructions to extract a positive weight coefficient as a sum of conductances in the positive columns for the positive weight branch; and program instructions to extract a negative weight coefficient as a sum of conductances in the negative columns for the negative weight branch, wherein the one or more target weights are further based on one or more of the positive weight coefficient and the negative weight coefficient.
14 . The computer program product of claim 9 , wherein the program instructions to program the conductance of each memory device of the one or more major conductance pairs memory devices and program the conductance of each memory device of the one or more minor conductance pairs of memory devices further comprise: program instructions to apply a voltage input comprising a constant amplitude and increasing pulse to one or more weights, wherein each weight is formed from a major conductance pair of memory devices and a minor conductance pair of memory devices, and wherein the increasing pulse widths increase by a predetermined amount from a first weight of the one or more weights to a last weight of the one or more weights.
15 . The computer program product of claim 9 , wherein the inference output is forced to the expected duration value when a sum of currents between the one or more major conductance pairs of memory devices or the one or more minor conductance pairs of memory devices is 0.
16 . The computer program product of claim 11 , further comprising: program instructions to incrementally program the one or more minor conductance pairs of memory devices to correct write errors incurred during the programming of the conductance of the one or more memory devices of the one or more major conductance pairs of memory devices to the one or more target weights.
17 . A computer system for balancing neural network weight asymmetries, the computer system comprising: one or more computer processors; one or more computer readable storage media; program instructions stored on the one or more computer readable storage media for execution by at least one of the one or more processors, the program instructions comprising: program instructions to evaluate edge mirror variability of complementary metal oxide semicondutor (CMOS) circuitry located peripheral to a memory array, wherein the program instructions to evaluate the edge mirror variability of the CMOS circuitry include instructions to: reset a conductance of each memory device of one or more major conductance pairs of memory devices and one or more minor conductance pairs of memory devices included in the memory array; program instructions to program the conductance of each memory device of one or more major conductance pairs of memory devices to force an inference output to an expected duration value; and program instructions to determine, based on the programed conductances of the one more major conductance pairs of memory devices, a current mirror mismatch between positive and negative weight branches of the memory array; program instructions to determine one or more target weights to compensate for the current mirror mismatch between the positive and negative weight branches of the memory array; and program instructions to program the conductance of one or more memory devices of the one or more major conductance pairs of memory devices to the one or more target weights.
18 . The computer system of claim 17 , wherein the one or more major conductance pairs of memory devices are phase change material (PCM) conductances corresponding to a major positive conductance G+ and a major negative conductance G− and the one or more minor conductance pairs of memory devices are PCM conductances corresponding to a minor positive conductance g+ and a minor negative conductance g−.
19 . The computer system of claim 17 , further comprising: prior to the program instructions to program the conductance of the one or more memory devices of the one or more major conductance pairs of memory devices to the one or more target weights: program instructions to reset the conductance of each memory device of the one or more major conductance pairs of memory devices; and program instructions to program the conductance of each memory device of the one or more minor conductance pairs of memory devices to force the inference output to the expected duration value.
20 . The computer system of claim 18 , further comprising: program instructions to extract a positive weight coefficient as a sum of conductances in the positive columns for the positive weight branch; and program instructions to extract a negative weight coefficient as a sum of conductances in the negative columns for the negative weight branch, wherein the one or more target weights are further based on one or more of the positive weight coefficient and the negative weight coefficient.

Description

BACKGROUND The present invention relates generally to the field of neural networks, and more particularly to balancing neural network weight asymmetries. Neural Networks (NNs) can be encoded in hardware chips that perform NN inference at high and low power. Such inference chips leverage Non-Volatile Memories (NVMs), such as Phase Change Material (PCM), and Resistive Memory (RRAM)) to encode weights in a NN. In certain designs, multiple devices are assigned to each weight, with a variable distribution of the programmed weight W (i.e., W=G+−G−+(g+−g−)/F) over various devices, where each G represents the conductance of each device and F is a scaling factor. However, Complementary Metal Oxide Semiconductor (CMOS) hardware periphery is significantly affected by variability, leading to a degraded evaluation of the ΣWx Multiply-Accumulate operation. There are some methodologies to calibrate CMOS peripheral variability, for example ADC column-to-column variation, but such calibration cannot eliminate all sources of variability, such as current mirror mismatch. SUMMARY The present invention is described in various embodiments disclosing computer-implemented methods, computer program products, and computer systems for balancing neural network weight asymmetries. One embodiment of the present disclosure is a computer-implemented method for balancing neural network weight asymmetries comprising one or more processors configured for providing a neural network with one or more weights, wherein the weights comprise one or more major conductance pairs and one or more minor conductance pairs; programming the one or more major conductance pairs to force an inference output to an expected duration value; determining a positive weight coefficient based on the one or more major conductance pairs and a negative weight coefficient based on the one or more minor conductance pairs; determining one or more target weights based on one or more of the positive weight coefficient and the negative weight coefficient; programming the one or more minor conductance pairs to force the inference output to the expected duration value; and programming the one or more major conductance pairs with the one or more target weights. In an embodiment, the one or more major conductance pairs are phase change material (PCM) conductances corresponding to a major positive conductance G+ and a major negative conductance G− and the minor conductance pairs are PCM conductances corresponding to a minor positive conductance g+ and a minor negative conductance g−. The computer-implemented method may further include prior to programming the one or more major conductance pairs, resetting the one or more major conductance pairs; and prior to programming the one or more minor conductance pairs, resetting the one or more major conductance pairs. In an embodiment, the one or more weights may include positive columns for a positive weight branch having the major positive conductance G+ and the minor positive conductance g+ and negative columns for a negative weight branch having the major negative conductance G− and the minor negative conductance g−. In an embodiment, the computer-implemented method may further include extracting the positive weight coefficient as a sum of conductances in the positive columns for the positive weight branch; and extracting the negative weight coefficient as a sum of conductances in the negative columns for the negative weight branch. In an embodiment, programming the one or more major conductance pairs and the one or more minor conductance pairs may further include applying a voltage input comprising a constant amplitude and increasing pulse widths to the one or more weights, wherein the increasing pulse widths increase by a predetermined amount from a first weight of the one or more weights to a last weight of the one or more weights. In an embodiment, the inference output is forced to the expected duration value when a sum of currents between the one or more major conductance pairs or the one or more minor conductance pairs is 0. In an embodiment, the computer-implemented method may further include determining mismatch values between the positive columns and the negative columns to identify fixed peripheral asymmetries. Further, the computer-implemented method may apply the mismatch values to the one or more major conductance pairs to counteract the fixed peripheral asymmetries. In an embodiment, the computer-implemented method may further include incrementally programming the one or more minor conductance pairs to correct write errors incurred during the programming the one or more major conductance pairs with the one or more target weights. BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 depicts a block diagram of a distributed data processing environment for balancing neural network weight asymmetries, in accordance with an embodiment of the present invention; FIG. 2 depicts a neural network for balancing neural network weight asymmetrie