US-20260126923-A1 - APPARATUS INCLUDING AN ARRAY OF PRE-CONFIGURABLE MEMORY AND STORAGE

US20260126923A1US 20260126923 A1US20260126923 A1US 20260126923A1US-20260126923-A1

Abstract

An apparatus including a high bandwidth memory circuit and associated systems and methods are disclosed herein. The high bandwidth memory circuit can include two or more physical layer circuits to communicate with neighboring devices. The high bandwidth memory circuit can broadcast a status to the neighboring devices. The neighboring devices can be configured according to the operating demands of the high bandwidth memory circuit.

Inventors

Lingming Yang
Raghukiran Sreeramaneni
Nevil N. Gajera

Assignees

MICRON TECHNOLOGY, INC.

Dates

Publication Date: 20260507
Application Date: 20260105

Claims (19)

1 . An apparatus, comprising: an interposer having at least a first input and output (IO) circuit and a second IO circuit; at least one memory cube including first local memory and at least a first physical layer circuit and a second physical layer circuit, the at least one memory cube mounted on the interposer with the first physical layer circuit connected to the first IO circuit and the second physical layer circuit connected to the second IO circuit; a first processing unit mounted on the interposer and connected to the at least one memory cube by the first IO circuit; and a second processing unit mounted on the interposer and connected to the at least one memory cube by the second IO circuit, wherein the at least one memory cube is configured to: receive, from the first processing unit, a first command with a first priority indication through the first IO circuit and the first physical layer circuit, receive, from the second processing unit, a second command with a second priority indication through the second IO circuit and the second physical layer circuit, and execute the first command and the second command in an order based upon comparing the first priority indication and the second priority indication.
2 . The apparatus of claim 1 , wherein the at least one first memory cube is further configured to: identify that an address of a storage location identified by the first command is in the at least one memory cube; and access the storage location to perform a read operation or a write operation.
3 . The apparatus of claim 1 , wherein the first priority indication and the second priority indication each indicate a time at which the corresponding one of the first and second commands was received by the at least one memory cube.
4 . The apparatus of claim 1 , wherein the first priority indication and the second priority indication each indicate a handshake status for the corresponding one of the first and second processing units.
5 . The apparatus of claim 1 , wherein the first priority indication and the second priority indication each indicate whether the corresponding one of the first and second processing units directly generated the corresponding command or received the corresponding command from a remotely connected third processing unit.
6 . The apparatus of claim 1 , wherein the first physical layer circuit and the second physical layer circuit are connected with one or more through silicon vias.
7 . The apparatus of claim 1 , wherein one of the first physical layer circuit and the second physical layer circuit indicates an inactive status while the at least one memory cube is communicating over the other of the first physical layer circuit and the second physical layer circuit.
8 . A memory device, comprising: two or more physical layer circuits to communicate with two or more external devices; the memory device configured to: receive, from a first one of the two or more external devices, a first command with a first priority indication through a first physical layer circuit of the two or more physical layer circuits, receive, from a second one of the two or more external devices, a second command with a second priority indication through a second physical layer circuit of the two or more physical layer circuits, execute the first command and the second command in an order based upon comparing the first priority indication and the second priority indication.
9 . The memory device of claim 8 , wherein the memory device is further configured to: identify that an address of a storage location identified by the first command is in the memory device; and access the storage location to perform a read operation or a write operation.
10 . The memory device of claim 8 , wherein the first priority indication and the second priority indication each indicate a time at which the corresponding one of the first and second commands was received by the memory device.
11 . The memory device of claim 8 , wherein the first priority indication and the second priority indication each indicate a handshake status for the corresponding one of the first and second external devices.
12 . The memory device of claim 8 , wherein the first priority indication and the second priority indication each indicate whether the corresponding one of the first and second external devices directly generated the corresponding command or received the corresponding command from a remotely connected third external device.
13 . The memory device of claim 8 , wherein the two or more physical layer circuits are connected with one or more through silicon vias.
14 . The memory device of claim 8 , wherein one of the first physical layer circuit and the second physical layer circuit indicates an inactive status while the at least one memory cube is communicating over the other of the first physical layer circuit and the second physical layer circuit.
15 . A method comprising: receive, from a first external device, a first command with a first priority indication through a first physical layer circuit of a memory cube; receive, from a second external device, a second command with a second priority indication through a second physical layer circuit of the memory cube; determine an execution order for the first command and the second command by comparing the first priority indication and the second priority indication; and executing the first command and the second command in the determined execution order.
16 . The method of claim 15 , wherein executing the first command comprises: identify that an address of a storage location identified by the first command is in the memory cube; and access the storage location to perform a read operation or a write operation.
17 . The method of claim 15 , wherein the first priority indication and the second priority indication each indicate a time at which the corresponding one of the first and second commands was received by the memory cube.
18 . The method of claim 15 , wherein the first priority indication and the second priority indication each indicate a handshake status for the corresponding one of the first and second external devices.
19 . The method of claim 15 , wherein the first priority indication and the second priority indication each indicate whether the corresponding one of the first and second external devices directly generated the corresponding command or received the corresponding command from a remotely connected third external device.

Description

CROSS-REFERENCE TO RELATED APPLICATIONS The present application is a continuation of U.S. Patent Application No. 18/789,660, filed July 30, 2024, which claims priority to U.S. Provisional Patent Application No. 63/543,516, filed October 11, 2023, the disclosure of which is incorporated herein by reference in its entirety. TECHNICAL FIELD The present technology is directed to apparatuses, such as semiconductor devices including memory and processors, and several embodiments are directed to semiconductor devices that include a configuration of processing units, high bandwidth memory, and high bandwidth storage. BACKGROUND An apparatus (e.g., a processor, a memory device, a memory system, or a combination thereof) can include one or more semiconductor circuits configured to store and/or process information. For example, the apparatus can include a memory device, such as a volatile memory device, a non-volatile memory device, or a combination device. Memory devices, such as dynamic random-access memory (DRAM) and/or high bandwidth memory (HBM), can utilize electrical energy to store and access data. With technological advancements in embedded systems and increasing applications, the market is continuously looking for faster, more efficient, and smaller devices. To meet the market demands, the semiconductor devices are being pushed to the limit with various improvements. Improving devices, generally, may include increasing circuit density, increasing circuit capacity, increasing operating speeds or otherwise reducing operational latency, increasing reliability, increasing data retention, reducing power consumption, or reducing manufacturing costs, among other metrics. However, attempts to meet the market demands, such as by reducing the overall device footprint, can often introduce challenges in other aspects, such as for maintaining circuit robustness and/or failure detectability. BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 is a cross-sectional view of a system-in-package device in accordance with embodiments of the technology. FIG. 2 is a block diagram of a memory device in accordance with embodiments of the technology. FIG. 3 is a block diagram of a high bandwidth memory system with multiple high bandwidth memory cubes and processing units connected in series in accordance with an embodiment of the present technology. FIG. 4 is a block diagram of a first example configuration of a high bandwidth memory system with multiple graphics processing units, high bandwidth memory cubes, and high bandwidth memory storage cubes connected in a pre-configurable arrangement in accordance with an embodiment of the present technology. FIG. 5 is a block diagram of a second example configuration of a high bandwidth memory system with multiple graphics processing units, high bandwidth memory cubes, and high bandwidth memory storage cubes connected in a pre-configurable arrangement in accordance with an embodiment of the present technology. FIG. 6A is a flow diagram illustrating an example method of operating an apparatus in accordance with an embodiment of the present technology. FIG. 6B is a flow diagram illustrating an example method of operating an apparatus in accordance with an embodiment of the present technology. FIG. 7 is a block diagram of an apparatus in accordance with an embodiment of the present technology. FIG. 8 is a block diagram of a system that includes an apparatus configured in accordance with embodiments of the present technology. DETAILED DESCRIPTION As described in greater detail below, the technology disclosed herein relates to an apparatus, such as for memory systems, systems with memory devices, related methods, etc., with multiple high bandwidth memory circuits, high bandwidth storage circuits, and processing units together. In some embodiments, an apparatus (e.g., a memory circuit or device, such as a high bandwidth memory (HBM) and/or a RAM, and/or a corresponding system) can be coupled to a processor, such as a graphics processing unit (GPU), via an interposer. Additionally, the apparatus can include an array of HBM cubes and high bandwidth storage (HBS) cubes connected to one or more GPUs. For context, advances in computing have increased the demand for multiple processor configurations. For example, improvements for graphics (e.g., in gaming applications), high-bandwidth multi-process or multi-thread computations (e.g., in machine learning or artificial intelligence applications) have increased the need for additional processors (e.g., GPUs) and corresponding memory in addition to more traditional or central processor and memory. For example, a local processor and one or more separate memory devices can be grouped as a unit by being included in a semiconductor package or by being mounted on an intermediate substrate (e.g., silicon interposer or a printed circuit board (PCB)). The combined unit of local processor and the memory devices can be coupled to and operate with other similar units and/or a c