US-12619558-B2 - Memory controller reservation of resources for cache hit

US12619558B2US 12619558 B2US12619558 B2US 12619558B2US-12619558-B2

Abstract

A memory controller circuit manages access to a memory cache circuit and storage circuits. The memory controller receives a memory access request, and attempts to reserve entries in a first set of storage circuits that are needed to process a cache hit prior to determining whether the memory access request hits in the memory cache circuit. This reservation attempt is performed without attempting to reserve other sets of storage circuits that are needed to process other possible outcomes of the memory access request, including a cache miss. If the memory controller circuit has successfully reserved entries in all of the first set of storage circuits, processing of the memory access request may be initiated. Conversely, if the memory controller is unable to reserve an entry in at least one of the first set of storage circuits, processing of the memory access request is inhibited.

Inventors

Ilya Granovsky
Jurgen M. Schulz
Tom Greenshtein
Elli Bagelman

Assignees

APPLE INC.

Dates

Publication Date: 20260505
Application Date: 20240829

Claims (20)

1 . An apparatus, comprising: a memory controller circuit of a computer system that includes, on one or more co-packaged integrated circuits (ICs): a memory cache circuit; a plurality of storage circuits; and control circuitry configured to manage access to the memory cache circuit and a portion of a system memory by reserving ones of the plurality of storage circuits, wherein the control circuitry, in response to receiving a memory access request, is configured to attempt to reserve, prior to a determination of whether the memory access request hits in the memory cache circuit, entries in a first set of the plurality of storage circuits that are needed to process a cache hit, without reserving entries in other sets of storage circuits of the plurality of storage circuits that are needed to process other possible outcomes of the memory access request, including a cache miss; and wherein the control circuitry is configured to initiate processing of the memory access request based on successfully reserving entries in all of the first set of storage circuits, and to inhibit processing of the memory access request based on being unable to reserve an entry in at least one of the first set of storage circuits; and wherein the other sets of storage circuits of the plurality of storage circuits include a different set of storage circuits used to process a cache miss, and wherein the control circuitry is configured to reserve entries in the different set of storage circuits after initiation of the memory access request and in response to a determination of the cache miss.
2 . The apparatus of claim 1 , wherein the control circuitry includes: a tag pipeline circuit configured to determine if the memory access request hits in the memory cache circuit.
3 . The apparatus of claim 2 , wherein the first set of the plurality of storage circuits includes a retry queue circuit having a plurality of entries configured to store information for those memory access requests that are determined, during a given pass through the tag pipeline circuit, to require another pass through the tag pipeline circuit.
4 . The apparatus of claim 3 , wherein the control circuitry, in response to the tag pipeline circuit detecting a cache hit to the memory cache circuit for the memory access request, is configured to: complete, using entries in the first set of storage circuits that have been reserved, the memory access request; and release a reserved entry in the retry queue circuit in response to the tag pipeline circuit detecting the cache hit for the memory access request.
5 . The apparatus of claim 3 , wherein the control circuitry, in response to the tag pipeline circuit detecting a cache miss for the memory access request and making a determination of availability of storage circuits other than those in the first set that are needed to complete the memory access request, is configured to: release a reserved entry in the retry queue circuit; and release any other entries in the first set of storage circuits that are not needed to complete the memory access request.
6 . The apparatus of claim 3 , wherein the control circuitry, in response to the tag pipeline circuit detecting a cache miss for the memory access request and making a determination of lack of availability of storage circuits other than those in the first set that are needed to complete the memory access request, is configured to store information relating to the memory access request to a reserved entry in the retry queue circuit, wherein the control circuitry is configured to cause the memory access request to be retried from the retry queue circuit at a later point in time.
7 . The apparatus of claim 2 , wherein the first set of the plurality of storage circuits also includes a cache hit output buffer configured to store information resulting from a hit in the memory cache circuit.
8 . The apparatus of claim 2 , wherein the first set of the plurality of storage circuits includes: a data pipe queue circuit configured to store information being sent from the tag pipeline circuit to a data pipe circuit in response to a determination of a cache hit, the data pipe circuit being configured to store write data to be sent to the memory cache circuit and to store read data to be retrieved from the memory cache circuit; and a read data buffer circuit configured to store information relating to a cache hit for a read-modify-write instruction.
9 . The apparatus of claim 1 , wherein the other sets of storage circuits of the plurality of storage circuits include a second set of storage circuits that includes a snoop queue circuit configured to store information relating to snoops generated by memory access requests, and wherein the control circuitry is configured to reserve entries in the second set of storage circuits after determining that a particular snoop has been generated for a particular memory access request.
10 . A method, comprising: receiving, at a memory controller circuit of a computer system with one or more co-packaged integrated circuits (ICs), a memory access request, wherein the memory controller circuit includes a memory cache circuit; attempting to reserve, by the memory controller circuit, entries in a first set of a plurality of storage circuits within the memory controller circuit that are needed to process a cache hit in the memory cache circuit, without attempting to reserve other ones of the plurality of storage circuits that would be needed to be reserved in response to the memory access request generating a snoop or a cache miss, wherein the first set of storage circuits includes a retry queue circuit configured to store information for memory access requests that are determined, during a given pass through the memory controller circuit, to require another pass through the memory controller circuit; in response to successfully reserving entries in the first set of storage circuits, initiating, by the memory controller circuit, the memory access request; and determining, by the memory controller circuit, whether the memory access request results in a cache hit in the memory cache circuit; and wherein the other ones of the plurality of storage circuits include a different set of storage circuits used to process a cache miss, and wherein the control circuitry is configured to reserve entries in the different set of storage circuits after initiation of the memory access request and in response to a determination of the cache miss.
11 . The method of claim 10 , further comprising: detecting, by the memory controller circuit, a cache hit to the memory cache circuit for the memory access request; in response to the detecting the cache hit and determining that the memory access request does not require a snoop to complete: completing the memory access request using entries in the first set of storage circuits that have been reserved; and releasing a reserved entry in the retry queue circuit for the memory access request.
12 . The method of claim 10 , further comprising: detecting, by the memory controller circuit, a cache hit to the memory cache circuit for the memory access request; in response to the detecting the cache hit and determining that the memory access request requires a snoop to complete: reserving an entry in a snoop queue circuit, wherein the snoop queue circuit is not one of the first set of storage circuits; and completing the memory access request using the reserved entry in the snoop queue circuit.
13 . The method of claim 10 , further comprising: detecting, by the memory controller circuit, a cache miss to the memory cache circuit for the memory access request; making, by the memory controller circuit, a determination of availability of entries in a second set of the plurality of storage circuits that are needed to complete the memory access request, the second set differing from the first set; and in response to detecting the cache miss and the determination of availability: releasing a reserved entry in the retry queue circuit; and releasing any other entries in the first set of storage circuits not needed to complete the memory access request.
14 . The method of claim 10 , further comprising: detecting, by the memory controller circuit, a cache miss to the memory cache circuit for the memory access request; making, by the memory controller circuit, a determination of unavailability of one or more entries in a second set of the plurality of storage circuits that are needed to complete the memory access request, the second set differing from the first set; and in response to detecting the cache miss and the determination of unavailability: storing information relating to the memory access request to a reserved entry in the retry queue circuit, wherein the memory controller circuit is configured to cause the memory access request to be retried from the retry queue circuit at a later point in time.
15 . An apparatus, comprising: a computer system that includes, on one or more co-packaged integrated circuits (ICs): a plurality of memory plane controller circuits, wherein a given one of the plurality of memory plane controller circuits includes: a memory cache circuit; a plurality of storage circuits; and control circuitry configured to: receive a memory access request from a network of the computer system; attempt to reserve a first set of resources in the plurality of storage circuits that are needed to complete a cache hit, without attempting to reserve other sets of resources needed in case of a cache miss or a snoop, the first set of resources including an entry in a retry queue circuit that is configured to store information for memory access requests that are determined, during a given pass through the given memory plane controller circuit, to require another pass through the given memory plane controller circuit; initiate processing of the memory access request based on successfully reserving the first set of resources; inhibit processing of the memory access request based on not successfully reserving at least one of the first set of resources; and in response to detecting a cache hit for the memory access request, complete the memory access request and release a reservation for the retry queue circuit wherein the other sets of resources of the plurality of storage circuits include a different set of storage circuits used to process a cache miss, and wherein the control circuitry is configured to reserve entries in the different set of storage circuits after initiation of the memory access request and in response to a determination of the cache miss.
16 . The apparatus of claim 15 , wherein the first set of resources includes an entry in a read buffer circuit configured to store data associated with a read-modify-write command that is a hit in the memory cache circuit.
17 . The apparatus of claim 15 , wherein the control circuitry is configured to: detect a cache miss to the memory cache circuit for the memory access request; make a determination of unavailability of entries in storage circuits other than those in the first set that are needed to complete the memory access request; and in response to detecting the cache miss and the determination of unavailability: store information relating to the memory access request in a reserved entry in the retry queue circuit, wherein the given memory plane controller circuit is configured to cause the memory access request to be retried from the retry queue circuit at a later point in time.
18 . The apparatus of claim 15 , wherein the control circuitry includes an arbitration circuit configured to: reserve an entry in the retry queue circuit for the memory access request; and select the memory access request for issuance for processing by a pipeline circuit of the given memory plane controller circuit as a result of an arbitration process and based on availability of the entry in the retry queue circuit for reservation, thus ensuring that unavailability of the retry queue circuit will not cause the memory access request to be returned to the network for resubmission to the given memory plane controller circuit.
19 . The apparatus of claim 18 , wherein the network is one of a plurality of networks within the computer system, and wherein the given memory plane controller circuit is configured to receive the memory access request from a particular one of a plurality of network interface (NI) circuits coupled to the given memory plane controller circuit, a given one of plurality of NI circuits being coupled to a respective one of the plurality of networks, and the particular NI circuit being coupled to the network.
20 . An apparatus, comprising: a memory controller circuit of a computer system that includes, on one or more co-packaged integrated circuits (ICs): a memory cache circuit; a plurality of storage circuits; and control circuitry configured to manage access to the memory cache circuit and a portion of a system memory by reserving ones of the plurality of storage circuits, wherein the control circuitry, in response to receiving a memory access request, is configured to attempt to reserve, prior to a determination of whether the memory access request hits in the memory cache circuit, entries in a first set of the plurality of storage circuits that are needed to process a cache hit, without reserving entries in other sets of storage circuits of the plurality of storage circuits that are needed to process other possible outcomes of the memory access request, including a cache miss; and wherein the control circuitry is configured to initiate processing of the memory access request based on successfully reserving entries in all of the first set of storage circuits, and to inhibit processing of the memory access request based on being unable to reserve an entry in at least one of the first set of storage circuits; and wherein the other sets of storage circuits of the plurality of storage circuits include a second set of storage circuits that includes a snoop queue circuit configured to store information relating to snoops generated by memory access requests, and wherein the control circuitry is configured to reserve entries in the second set of storage circuits after determining that a particular snoop has been generated for a particular memory access request.

Description

CROSS REFERENCE TO RELATED APPLICATIONS This application claims priority to U.S. Provisional Application Nos. 63/584,716 entitled “Memory Controller Reservation of Retry Queue” and filed on Sep. 22, 2023; and 63/584,727 entitled “Memory Controller Reservation of Resources for Cache Hit,” also filed on Sep. 22, 2023. Both of these provisional applications are incorporated by reference herein in their entireties. The present application is also related to U.S. application Ser. No. 18/819,755, entitled “Memory Controller Reservation of Retry Queue,” filed on the same day as the present application. BACKGROUND Technical Field This application relates generally to computer systems, and more specifically to processing memory transactions within such systems. Description of the Related Art Computers are comprised of diverse hardware components, each responsible for specific tasks within the system. These components typically include on-chip central processing units (CPUs), graphics processors units (GPUs) input/output devices, input/output interfaces, often alongside other components such as radio modems. Furthermore, these components may communicate with each other and with memory components to store and modify information relating to their operation. As such, computer components may access various types of memory components. Memory components include Random Access Memory (RAM), which provides fast, temporary storage for actively used data. One example of RAM is Dynamic RAM (DRAM), a type of volatile computer memory that stores data in a digital format as electrical charges in capacitors and is commonly used as the main memory in computers. Facilitating communication between computer components and memory is the memory controller circuit, which is hardware responsible for managing data transactions between the computer components and various memory modules. Memory controller circuits exhibit diverse technical designs based on their context. For instance, memory controller circuits for high-performance computing clusters may emphasize parallelism between multiple components, while those for embedded systems may instead prioritize power efficiency. In a computer system with many components, a memory controller circuits acts as an interface between various components and memory modules, overseeing memory management tasks. BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1A is a block diagram of one embodiment of a memory controller circuit configured to reserve an entry in a retry queue circuit. FIG. 1B is a block diagram of one embodiment of a memory plane controller circuit configured to reserve an entry in a retry queue circuit. FIG. 2 is a block diagram of one embodiment of an arbitration circuit within a memory controller circuit. FIG. 3A is a block diagram of one embodiment of a memory controller circuit. FIG. 3B is a block diagram of one embodiment of a retry queue circuit within a memory controller circuit. FIG. 3C is a block diagram of one embodiment of a snoop queue circuit within a memory controller circuit. FIG. 3D is a block diagram illustrating support for dirty bits of half cache lines in one embodiment of a computer system with a hierarchy of caches. FIG. 4A is a block diagram illustrating one embodiment of a network interface circuit coupled between a network and a memory controller circuit. FIG. 4B is a block diagram illustrating one embodiment of a computer system in which a memory controller circuit is coupled to multiple networks via respective network interface circuits. FIG. 4C is a block diagram illustrating one embodiment of a computer system in which a network interface circuit is coupled to multiple memory plane controller circuits. FIG. 5A is a block diagram of one embodiment of an upstream path of a network interface circuit. FIG. 5B is a block diagram of one embodiment of a downstream path of a network interface circuit. FIG. 6A is a flow diagram of one embodiment of a method for reserving an entry in a retry queue circuit of a memory controller circuit of a computer system. FIG. 6B is a flow diagram of one embodiment of a method for interfacing with a memory controller circuit of a computer system by a network interface circuit. FIG. 7 is a block diagram of one embodiment of a memory controller circuit within different sets of resources needed for different outcomes of a memory access request. FIG. 8 is a block diagram of one embodiment of a memory controller circuit that includes a memory cache circuit. FIG. 9 is a block diagram that illustrates storage circuit resources within one embodiment of a memory controller circuit. FIG. 10 is a block diagram that illustrates storage circuit resources within one embodiment of an interface to system memory within a memory controller circuit. FIG. 11 is a flow diagram illustrating various possible resource reservation outcomes for different outcomes of a memory access request. FIG. 12 is a flow diagram of one embodiment of a method for reserving resources