Search

EP-4495792-B1 - SYSTEMS FOR HIGH-SPEED COMPUTING USING AN OPTICAL INTERCHANGE

EP4495792B1EP 4495792 B1EP4495792 B1EP 4495792B1EP-4495792-B1

Inventors

  • TOMA, Horia Alexandru
  • SHEN, ZUOWEI
  • EDWARDS, WILLIAM F. JR.
  • RAJAMANI, GURUSHANKAR
  • LIU, HONG
  • MOHAMMED, ILYAS

Dates

Publication Date
20260513
Application Date
20240718

Claims (11)

  1. A system (101, 102, 103) for computing and memory communication comprising: a plurality of high-bandwidth memory units, HBMs (104), wherein each HBM (104) has a memory optical interface (110); a plurality of computing units (102), wherein each of the computing units (102) has a computing optical interface (110); an optical interchange (106) that is optically connected to the memory optical interfaces (110) of the plurality of HBMs (104) and to the computing optical interfaces (110) of the plurality of computing units (102), wherein the optical interchange (106) includes a demultiplexer (107) configured to transmit a read-related signal received from any one of the plurality of HBMs (104) to one or more of the computing units (102) from the plurality of computing units (102), and wherein the optical interchange (106) includes a switch (108) configured to transmit a write-related signal received from any one of the plurality of computing units (102) to any one of the plurality of HBMs (104), and wherein the memory optical interfaces (110) of the plurality of HBMs (104) further comprises a first memory optical interface(112) configured to send read-related signals to the demultiplexer (107) and a second memory optical interface (114) configured to receive write-related signals from the switch (108); and wherein the computing optical interface (110) of the plurality of computing units (102) further comprises a first computing optical interface (114) configured to receive read-related signals from the demultiplexer (107) and a second computing optical interface (112) configured to send write-related signals to the switch (108).
  2. The system (101, 102, 103) of claim 1, wherein the demultiplexer (107) of the optical interchange (106) further comprises an input optical interface (115) for each of the plurality of HBMs (104) and an output optical interface (116) for each of the computing units (102); and/or wherein the demultiplexer (107) is further configured to simultaneously transmit the read-related signal to more than one computing unit (102) from the plurality of computing units (102); and/or wherein the demultiplexer (107) is further configured to transmit the read-related signal while the switch (108) transmits the write-related signal.
  3. The system (101, 102, 103) of one of claims 1 to 2, wherein the plurality of HBMs (104) includes active HBMs and at least one spare HBM, and wherein the switch (108) is configured to transmit write-related signals to only active HBMs.
  4. The system (201, 301) of one of claims 1 to 3, wherein the plurality of HBMs (104) are located on a first substrate (230, 334) and the plurality of computing (102) units are located on a second substrate (232, 332).
  5. The system (201, 301) of claim 4, wherein the first substrate (230, 334) is located on a first device and the second substrate is located on a second device (232, 332).
  6. The system (101, 201, 301) of one of claims 1 to 5, wherein the memory optical interfaces (110) and the computing optical interfaces (110) are electrically connected via one or more interposers (310, 320).
  7. The system (201, 301) of one of claims 1 to 6, further comprising: a first package having the plurality of high-bandwidth memory units HBMs (104); and a second package having the plurality of computing units (102).
  8. The system (101, 201, 301) of claim 7, wherein the optical interchange (106) further comprises an optical chiplet optically connected to a plurality of optical fibers (122), and wherein each of the computing units (102) are optically connected to one or more of the plurality of optical fibers (122).
  9. The system (101, 201, 301) of one of claims 7 or 8, wherein the plurality of HBMs (104) are configured as part of a 3D memory stack.
  10. The system (201, 301) of claim 9, wherein the first package is located on a first device and the second package is located on a second device.
  11. The system (301) of one of claims 7 to 10, wherein the optical interchange (106) is positioned on the first package (334).

Description

BACKGROUND High-speed computing can be performed using computing packages that have a plurality of high bandwidth memory ("HBM") and a plurality of computing dies. However, it is becoming increasingly difficult to fit additional memory and computing dies onto packages in a manner that allows for efficient transmission of data using electrical transmissions. In addition, known systems of optical transmission provide for inefficient utilization of memory and computing dies. US2017279555A1 discloses adding and dropping signals in a node of an optical network, wherein the node includes a reconfigurable optical add/drop multiplexer (ROADM). US2019214365A1 discloses an apparatus which may include a memory circuit die configured to store a lookup table that converts first data to second data. US2021257021A1 discloses a computer memory system including an electro-optical chip, an electrical fanout chip electrically connected to an electrical interface of the electro-optical chip, and at least one dual in-line memory module (DIMM) slot electrically connected to the electrical fanout chip. BRIEF SUMMARY The claimed subject matter is defined in the independent claims. Dependent claims specify embodiments thereof. The present application relates to systems for providing high bandwidth connections between memory and computing units. Aspects of the disclosure allow for the high-speed memory and computing units to be disaggregated while maintaining a high bandwidth at relatively low power and cost. Communication between the memory and computing units is achieved via an optical interchange that allows for optical signals to be transmitted between the memory and computing units in an efficient manner. The optical interchange may be configured so that any computing unit may write data to any memory unit. In addition, the optical interchange may be configured so that a memory unit can simultaneously broadcast data to a plurality of computing units. In accordance with aspects of the disclosure, a system for computing and memory communication may include: a plurality of high-bandwidth memory units (HBMs), wherein each HBM has a memory optical interface; a plurality of computing units, wherein each of the computing units has a computing optical interface; and an optical cable that is optically connected to the memory optical interfaces of the plurality of HBMs and to the computing optical interfaces of the plurality of computing units, wherein the optical I/O includes a demultiplexer configured to transmit a read-related signal received from any one of the plurality of HBMs to one or more of the computing units from the plurality of computing units, and wherein the optical interchange includes a switch configured to transmit a write-related signal received from any one of the plurality of computing units to any one of the plurality of HBMs. In accordance with other aspects of the disclosure, the memory optical interfaces of the plurality of HBMs may further include a first memory optical interface configured to send read-related signals and a second memory optical interface configured to receive write-related signals. The computing optical interface of the plurality of computing units may further include a first computing optical interface configured to receive read-related signals and a second computing optical interface configured to send write-related signals. In accordance with still other aspects of the disclosure, the demultiplexer of the optical interchange may further include an input optical interface for each of the plurality of HBMs and an output optical interface for each of the computing units. The demultiplexer may be further configured to simultaneously transmit the read-related signal to more than one computing unit. The demultiplexer may be further configured to transmit the read-related signal while the switch transmits the write-related signal. In accordance with other aspects of the disclosure, the plurality of HBMs may include active HBMs and at least one spare HBM, and wherein the switch is configured to transmit write-related signals to only active HBMs. In accordance with yet other aspects of the disclosure, plurality of HBMs may be located on a first substrate and the plurality of computing units may be located on a second substrate. In addition, the first substrate may be located on a first device and the second substrate may be located on a second device. In accordance with still other aspects of the disclosure, the memory optical interfaces and the computing optical interfaces may be electrically connected via one or more interposers. In accordance with other aspects of the disclosure, a system for computing and memory communication may include: a first package having a plurality of high-bandwidth memory units (HBMs); a second package having a plurality of computing units; and an optical interchange that is configured to optically connect the HBMs and the computing units, wherein the optical interchan