KR-20260063969-A - Network router for performing collective operations and accelerator system including the network router

Abstract

A network router comprises: a receiver configured to receive a first input packet along a first direction from a first network router and a second input packet along a second direction from a second network router, and to output either the first input packet or the second input packet as a set packet; a network controller configured to receive a set packet output from the receiver and to output the set packet through a first path or a second path based on the packet type of the set packet; a buffer circuit configured to store a set packet transmitted from the network controller through a second path so as to be mutually distinguished according to the packet type of the set packet; and a reduce operation circuit that receives the set packet stored in the buffer circuit and performs a reduce operation using the received set packet.

Inventors

• 김구현
• 김창현
• 신경철

Assignees

• 에스케이하이닉스 주식회사

Dates

Publication Date

20260507

Application Date

20241031

Claims (20)

1. A receiver configured to receive a first input packet along a first direction from a first network router, receive a second input packet along a second direction from a second network router, and output either of the first input packet and the second input packet as a set packet; A network controller configured to receive the set packet output from the receiver and output the set packet through a first path or a second path based on the packet type of the set packet; A buffer circuit configured to store a set packet transmitted from the network controller through the second path in a manner that distinguishes it from one another according to the packet type of the set packet; and A network router comprising a reduce operation circuit that receives the set packet stored in the buffer circuit and performs a reduce operation using the received set packet.
2. In paragraph 1, the receiver is, A first receiving buffer storing the first input packet; and A network router including a second receiving buffer that stores the second input packet.
3. In paragraph 2, The above receiver is a network router configured to output first the packet with the higher output priority order among the first input packet stored in the first receiving buffer and the second input packet stored in the second receiving buffer.
4. In paragraph 1, A network router further comprising a transmitter configured to transmit a first output packet along the first direction to the second network router and to transmit a second output packet along the second direction to the first network router.
5. In paragraph 4, the transmitter is, A first transmission buffer that stores a set packet, in which the output transmission direction is set in the first direction, as the first output packet; and A network router comprising a second transmission buffer that stores a set packet, in which the output transmission direction is set in the second direction, as the second output packet.
6. In paragraph 1, A network router in which the packet type of the above-mentioned set packet is set to any one of a transmission packet, an all-gather packet, and a reduce packet.
7. In paragraph 6, The packet used in the send operation, the packet used in the broadcast operation, the packet used in the gather operation, the packet used in the scatter operation, the reduce result packet generated by the first reduce operation in the reduce operation, and the reduce-scatter result packet generated by the second reduce operation in the reduce-scatter operation are configured to be processed as the transmission packet. The packet used in the all-gather operation and the all-reduce result packet generated by the third reduce operation in the all-reduce operation are configured to be processed as the all-gather packet, and A network router configured such that a set packet used as an operand for the first reduce operation, the second reduce operation, and the third reduce operation, and a partial sum packet generated in the first reduce operation, the second reduce operation, and the third reduce operation are processed as the reduce packet.
8. In paragraph 6, The above network router further includes a transmitter configured to transmit a first output packet along the first direction to the second network router and to transmit a second output packet along the second direction to the first network router, wherein A network router configured such that, if the set packet received from the receiver corresponds to the transmission packet, the transmission packet is transmitted to the transmitter or the buffer circuit based on the destination of the transmission packet.
9. In paragraph 8, the above network controller is, If the above transmission packet corresponds to a transmission path packet having a destination other than the network router, the above transmission path packet is transmitted to the transmitter through the first path, and, A network router configured to transmit the transmission target packet to the buffer circuit via the second path when the transmission packet corresponds to a transmission target packet having the network router as a destination.
10. In paragraph 8, A network router configured such that, if the aggregation packet received from the receiver corresponds to the all-gather packet and the reduce packet, the network controller transmits the all-gather packet and the reduce packet to the buffer circuit.
11. In paragraph 6, The above network router further includes a transmitter configured to transmit a first output packet along the first direction to the second network router and to transmit a second output packet along the second direction to the first network router, wherein The above network controller is a network router comprising a first packet transmission circuit, a second packet transmission circuit, a third packet transmission circuit, and a fourth packet transmission circuit sequentially arranged between the receiver and the transmitter.
12. In Paragraph 11, The first packet transmission circuit, the second packet transmission circuit, the third packet transmission circuit, and the fourth packet transmission circuit each have one input terminal, a first output terminal, and a second output terminal, wherein The input terminal, the first output terminal, and the second output terminal of the first packet transmission circuit are respectively coupled to the receiver, the input terminal of the second packet transmission circuit, and the buffer circuit, and The first output terminal and the second output terminal of the second packet transmission circuit are respectively coupled to the input terminal and the buffer circuit of the third packet transmission circuit, and The first output terminal and the second output terminal of the third packet transmission circuit are respectively coupled to the input terminal and the buffer circuit of the fourth packet transmission circuit, and The first output terminal and the second output terminal of the fourth packet transmission circuit are a network router that is respectively coupled to the first transmission buffer and the second transmission buffer of the transmitter.
13. In Paragraph 12, The input terminal of the above-mentioned fourth packet transmission circuit is a network router coupled to the above-mentioned buffer circuit.
14. In Paragraph 13, The above-mentioned first packet transmission circuit is, If the set packet input through the input terminal corresponds to the transmission packet and the all-gather packet, the transmission packet and the all-gather packet are output through the first output terminal, and, A network router configured to output the reduce packet through the second output terminal when the set packet input through the input terminal corresponds to the reduce packet.
15. In Paragraph 13, The above second packet transmission circuit is, If the set packet input through the above input terminal corresponds to the transmission packet, the transmission packet is output through the first output terminal, and, A network router configured to output the all-gather packet through the second output terminal when the set packet input through the input terminal corresponds to the all-gather packet.
16. In Paragraph 13, The above third packet transmission circuit is, If the set packet input through the above input terminal corresponds to the transmission packet, and the transmission packet corresponds to a transmission path packet having a network router other than the network router as its destination, the transmission path packet is output through the first output terminal, and A network router configured to output the transmission target packet through the second output terminal when the set packet input through the input terminal corresponds to the transmission packet, and the transmission packet corresponds to a transmission target packet having the network router as a destination.
17. In Paragraph 16, The above-mentioned fourth packet transmission circuit is, If the output transmission direction of the transmission path packet input from the third packet transmission circuit through the input terminal is the first direction, the transmission path packet is output through the first output terminal, and, A network router configured to output the transmission path packet through the second output terminal if the output transmission direction of the transmission path packet input from the third packet transmission circuit through the input terminal is the second direction.
18. In Paragraph 16, The above-mentioned fourth packet transmission circuit is, If the output transmission direction of the set packet input from the buffer circuit through the input terminal is the first direction, the set packet is output through the first output terminal, and, A network router configured to output the set packet through the second output terminal if the output transmission direction of the set packet input from the buffer circuit through the input terminal is the second direction.
19. In Paragraph 13, The above buffer circuit is, A send buffer that stores a set of packets transmitted from a scratch pad coupled to the above network router; A receive buffer storing the above all-gather packet, a set packet transmitted from another network router and having the above network router as a destination, and a set packet output from the above reduce operation circuit and having the above network router as a destination; A partial buffer storing a set packet used as the first operand of the above reduce operation; and A network router comprising a reduce buffer that stores a set packet used as a second operand of the above reduce operation.
20. In Paragraph 19, The above send buffer is, The transmission packet, the all-gather packet, and the reduce packet to be transmitted from a scratch pad coupled to the network router to another network router are received and stored, and the stored transmission packet, the all-gather packet, and the reduce packet are transmitted to the input terminal of the fourth packet transmission circuit. If an all-gather packet received from another network router corresponds to an all-gather path packet having a destination other than the network router, the all-gather path packet is stored, and the stored all-gather packet is transmitted to the input terminal of the fourth packet transmission circuit, and A network router configured to transmit the reduce pass packet to the input terminal of the fourth packet transmission circuit when the reduce packet generated by the reduce operation in the above reduce operation circuit corresponds to a reduce pass packet having a destination other than the above network router.

Description

Network router for performing collective operations and accelerator system including the network router Various embodiments of the present disclosure relate to a network router and an accelerator system including the same, and in particular to a network router and an accelerator system including the same that performs collective operations. Large Language Model (LLM) systems are complex Artificial Intelligence (AI) models designed to understand and generate human-like text based on vast amounts of training data. These models utilize deep learning techniques, particularly neural networks, to analyze language patterns and generate consistent and contextually appropriate text. A key characteristic of LLMs is their scale, which allows them to capture complex linguistic structures and nuances as they are trained on massive datasets containing billions of words. The architecture of LLMs typically consists of multiple layers of artificial neural network units, and transformer architectures, in particular, are gaining prominence due to their ability to handle long-range dependencies within text. Recently, there have been attempts to perform AI computations based on LLMs in accelerator systems where AI accelerators communicate via network routers. Accordingly, in order for artificial intelligence computation based on LLM to be performed efficiently, it is necessary to improve the network communication capabilities between the artificial intelligence accelerators constituting the accelerator system. FIG. 1 is a block diagram showing an accelerator system according to one example of the present disclosure. Figure 2 is a block diagram showing an example of an accelerator included in the accelerator system of Figure 1. FIG. 3 is a drawing showing a network router according to one example of the present disclosure. FIGS. 4A and FIGS. 4B are drawings shown to explain the send operation in the accelerator system of FIG. 1, including the network router of FIG. 3. Figure 5 is a diagram shown to explain the operation of the second network router in the second step of the send operation of Figure 4A. Figure 6 is a diagram shown to explain the operation of the first network router in the second step of the send operation of Figure 4A. Figure 7 is a diagram shown to explain the operation of the fourth network router in the third step of the send operation of Figure 4B. FIGS. 8A and FIGS. 8B are drawings shown to illustrate the broadcast operation in the accelerator system of FIG. 1, including the network router of FIG. 3. Figure 9 is a diagram shown to explain the operation of the second network router in the second step of the broadcast operation of Figure 8A. FIG. 10 is a diagram shown to explain the operation of the third network router in the second stage of the broadcast operation of FIG. 8A. FIG. 11 is a diagram shown to explain the operation of the third network router in the third step of the broadcast operation of FIG. 8B. FIGS. 12A and FIGS. 12B are drawings shown to illustrate the gather operation in the accelerator system of FIG. 1, including the network router of FIG. 3. FIGS. 13A and FIGS. 13B are drawings shown to explain the operation of the third network router in the second stage of the gather operation of FIG. 12A. FIGS. 14A to 14C are drawings shown to explain the operation of the second network router in the second stage of the gather operation of FIG. 12A. FIGS. 15A and FIGS. 15B are drawings shown to illustrate the all-gather operation in the accelerator system of FIG. 1, including the network router of FIG. 3. FIGS. 16A and FIGS. 16B are diagrams shown to explain the operation of the second network router in the second stage of the all-gather operation of FIG. 15A. FIGS. 17A and FIGS. 17B are diagrams shown to explain the operation of the second network router in the third step of the all-gather operation of FIG. 15B. FIG. 18 is a diagram shown to explain the operation of the second network router in the fourth step of the all-gather operation of FIG. 15B. FIGS. 19A and FIGS. 19B are drawings shown to explain the scatter operation in the accelerator system of FIG. 1, including the network router of FIG. 3. FIG. 20 is a diagram shown to explain the operation of the second network router in the second stage of the scatter operation of FIG. 19A. FIGS. 21A and FIGS. 21B are drawings shown to illustrate an example of a reduce operation in the accelerator system of FIG. 1, including the network router of FIG. 3. FIGS. 22A and FIGS. 22B are drawings shown to explain the operation of the second network router in the second step of the reduce operation of FIG. 21A. FIGS. 23A and FIGS. 23B are drawings shown to explain the operation of the third network router in the second step of the reduce operation of FIG. 21A. FIGS. 24A and FIGS. 24B are drawings shown to explain the operation of the second network router in the third step of the reduce operation of FIG. 21B. FIGS. 25A to 25C are drawings shown to illustrate o