KR-102964796-B1 - Memory tiering techniques in computing systems

Abstract

A technique for memory tiering in a computing device is disclosed. One exemplary technique includes, upon receiving a request to read data corresponding to a system memory section, retrieving metadata from a metadata portion of the first tier and data from a data portion of the first tier from a first tier in the first memory. The method may include analyzing data location information to determine whether the first tier currently contains data corresponding to the system memory section in the request. In response to the determination that the first tier currently contains data corresponding to the system memory section in the request, the data retrieved from the data portion of the first memory may be transmitted to a processor in response to the received request. The method may include identifying a memory location in memory that contains data corresponding to the system memory section and retrieving data from the identified memory location.

Inventors

• 아가르왈 이슈와르
• 크리소스 조지 자카리아스
• 로셀 마르티네즈 오스카

Assignees

• 마이크로소프트 테크놀로지 라이센싱, 엘엘씨

Dates

Publication Date

20260513

Application Date

20220602

Priority Date

20210709

Claims (20)

1. A memory management method in a computing device having a processor, a first memory located near the processor and configured as a cache for the processor, a second memory separated from the processor and interfaced with the processor, and a memory controller configured to manage the operation of the first and second memories, wherein In the memory controller above, receiving a request from the processor to read data corresponding to a system memory section from the processor's cache; and In response to receiving the above request to read, using the memory controller, A step of retrieving data from a data portion and metadata from a metadata portion within a first section of the first memory, from a first section within the first memory - the metadata from the metadata portion encodes data location information of a plurality of system memory sections within a first section of the first memory, a second section of the first memory, and one or more additional sections within the second memory - ; A step of analyzing the data location information within the retrieved metadata from the metadata portion of the first section in the first memory to determine whether the first section in the first memory currently contains data corresponding to the system memory section in the received request; and In response to determining that the first section in the first memory currently contains data corresponding to the system memory section in the received request, the step of transmitting the retrieved data from the data portion of the first section in the first memory to the processor in response to the received request. A memory management method including
2. In paragraph 1, In response to determining that the first section in the first memory does not currently contain data corresponding to the system memory section in the received request, A step of further analyzing the data location information within the retrieved metadata to identify whether the second section within the first memory or the memory location within the second memory includes data corresponding to the system memory section in the received request; and A step of retrieving data from the second section in the first memory or the memory location in the second memory, and providing the retrieved data to the processor in response to the received request. A memory management method that further includes
3. In paragraph 1, In response to determining that the first section in the first memory does not currently contain data corresponding to the system memory section in the received request, A step of further analyzing the data location information within the retrieved metadata to determine whether the second section within the first memory or the memory location within the second memory includes the data corresponding to the system memory section in the received request; If it is determined that the second section within the first memory contains data corresponding to the system memory section in the received request, A step of retrieving the data from the second section within the first memory and providing the retrieved data to the processor in response to the received request; and A step of recording the retrieved data from the second section in the first memory into the first section in the first memory. A memory management method that further includes
4. In paragraph 1, In response to determining that the first section in the first memory does not currently contain data corresponding to the system memory section in the received request, A step of further analyzing the data location information within the retrieved metadata to determine whether the second section within the first memory or the memory location within the second memory includes the data corresponding to the system memory section in the received request; If it is determined that the second section within the first memory contains data corresponding to the system memory section in the received request, A step of retrieving the data from a second section within the first memory and providing the retrieved data to the processor in response to the received request; A step of recording the retrieved data from the second section in the first memory into the first section in the first memory; and A step of modifying the metadata within the metadata portion of the first memory to indicate that the data corresponding to the system memory section in the received request is now within the first section of the first memory. A memory management method that further includes
5. In paragraph 1, In response to determining that the first section in the first memory does not currently contain data corresponding to the system memory section in the received request, A step of further analyzing the data location information within the retrieved metadata to determine whether the second section within the first memory or the memory location within the second memory includes the data corresponding to the system memory section in the received request; If it is determined that the second section within the first memory contains data corresponding to the system memory section in the received request, A step of retrieving the data from the second section within the first memory and providing the retrieved data to the processor in response to the received request; A step of recording the retrieved data from the second section in the first memory into the first section in the first memory; and A step of recording the retrieved data from the first section of the first memory into the second section of the first memory. A memory management method that further includes
6. In paragraph 1, In response to determining that the first section in the first memory does not currently contain data corresponding to the system memory section in the received request, A step of further analyzing the data location information within the retrieved metadata to determine whether the second section within the first memory or the memory location within the second memory includes the data corresponding to the system memory section in the received request; If it is determined that the second section within the first memory contains data corresponding to the system memory section in the received request, A step of retrieving the data from the second section within the first memory and providing the retrieved data to the processor in response to the received request; A step of recording the retrieved data from the second section in the first memory into the first section in the first memory; A step of recording the retrieved data from the first section of the first memory into the second section of the first memory; and A step of modifying the metadata within the metadata portion within the first section of the first memory: The fact that the data corresponding to the system memory section in the received request is now within the first section of the first memory, and The fact that the data previously held in the first section of the first memory is now in the second section of the first memory A step of modifying the above metadata to indicate A memory management method that further includes
7. In paragraph 1, The metadata from the above metadata portion includes one or more bits, and The above data location information includes a combination of one or more bits that individually correspond to one of the plurality of system memory sections, and The step of analyzing the above data location information is: A step of identifying a combination of one or more bits; and A step of determining whether the above identified combination corresponds to the system memory section in the received request A memory management method that includes
8. In paragraph 1, The metadata from the above metadata portion includes one or more Error Checking and Correction (ECC) bits, and The above data location information includes a combination of one or more ECC bits that individually correspond to one of the plurality of system memory sections, and The step of analyzing the above data location information is: A step of identifying a combination of one or more of the above ECC bits; and A step of determining whether the combination of the identified ECC bits corresponds to the system memory section in the received request. A memory management method that includes
9. In computing devices, processor; A first memory located near the processor and configured as a cache for the processor; A second memory separated from the processor and interfacing with the processor; and A memory controller configured to manage the operation of the first and second memories. Includes, The above memory controller is: When the processor receives a request to read data corresponding to a system memory section from the processor's cache, To retrieve data from a first section of the first memory, from metadata from a metadata portion of the first section of the first memory and from a data portion, wherein the metadata from the metadata portion encodes data location information of a plurality of system memory sections within the first section of the first memory, the second section of the first memory, and one or more additional sections of the second memory. To determine whether the first section in the first memory currently contains data corresponding to the system memory section in the received request, to analyze the data location information within the retrieved metadata from the metadata portion of the first section in the first memory; and In response to determining that the first section in the first memory currently contains data corresponding to the system memory section in the received request, the retrieved data from the data portion of the first section in the first memory is transmitted to the processor in response to the received request. A computing device having instructions executable by the above memory controller.
10. In Paragraph 9, The above memory controller is: In response to determining that the first section in the first memory does not currently contain data corresponding to the system memory section in the received request, To further analyze the data location information within the retrieved metadata to determine whether the second section within the first memory or the memory location within the second memory includes data corresponding to the system memory section in the received request; and Retrieves the data from the second section within the first memory or the memory location within the second memory, and provides the retrieved data to the processor in response to the received request. A computing device that includes additional executable commands.
11. In Paragraph 9, The above memory controller is: In response to determining that the first section in the first memory does not currently contain data corresponding to the system memory section in the received request, Further analyzing the data location information within the retrieved metadata to determine whether the second section within the first memory or the memory location within the second memory includes the data corresponding to the system memory section in the received request; If it is determined that the second section within the first memory contains data corresponding to the system memory section in the received request, Retrieving the data from the second section within the first memory and providing the retrieved data to the processor in response to the received request; and To record the retrieved data from the second section within the first memory in the first section within the first memory. A computing device that includes additional executable commands.
12. In Paragraph 9, The above memory controller is: In response to determining that the first section in the first memory does not currently contain data corresponding to the system memory section in the received request, To further analyze the data location information within the retrieved metadata to determine whether the second section within the first memory or the memory location within the second memory includes the data corresponding to the system memory section in the received request; To record the retrieved data from the second section in the first memory in the first section in the first memory; and To modify the metadata within the metadata portion of the first memory to indicate that the data corresponding to the system memory section in the received request is now within the first section of the first memory. A computing device that includes additional executable commands.
13. In Paragraph 9, The above memory controller is: In response to determining that the first section in the first memory does not currently contain data corresponding to the system memory section in the received request, To further analyze the data location information within the retrieved metadata to determine whether the second section within the first memory or the memory location within the second memory includes the data corresponding to the system memory section in the received request; To retrieve the data from the second section within the first memory and to provide the retrieved data to the processor in response to the received request; To record the retrieved data from the second section in the first memory in the first section in the first memory; and To record the retrieved data from the first section of the first memory in the second section of the first memory. A computing device that includes additional executable commands.
14. In Paragraph 9, The above memory controller is: In response to determining that the first section in the first memory does not currently contain data corresponding to the system memory section in the received request, Further analyzing the data location information within the retrieved metadata to determine whether the second section within the first memory or the memory location within the second memory includes the data corresponding to the system memory section in the received request; If it is determined that the second section within the first memory contains data corresponding to the system memory section in the received request, Retrieving the data from the second section within the first memory and providing the retrieved data to the processor in response to the received request; To record the retrieved data from the second section within the first memory in the first section within the first memory; To record the retrieved data from the first section of the first memory in the second section within the first memory; and To modify the metadata within the metadata portion within the first section of the first memory in order to indicate that the data corresponding to the system memory section in the received request is now within the first section of the first memory, and that the data previously held within the first section of the first memory is now within the second section of the first memory. A computing device that includes additional executable commands.
15. In Paragraph 9, The metadata from the above metadata portion includes one or more bits, and The above data location information includes a combination of one or more bits that individually correspond to one of the plurality of system memory sections, and Analyzing the above data location information is: Identifying a combination of one or more of the above bits; Determining whether the above identified combination corresponds to the system memory section in the above received request A computing device that includes
16. A memory management method in a computing device having a processor, a first memory located near the processor and configured as a cache for the processor, a second memory separated from the processor and interfaced with the processor, and a memory controller configured to manage the operation of the first and second memories, wherein In the memory controller above, receiving a request from the processor to write a block of data corresponding to a system memory section to the processor's cache; and In response to receiving a request to write a block of the above data, using the memory controller, A step of retrieving metadata from a metadata portion of a first section within the first memory from a first section of the first memory—the metadata from the metadata portion encodes data location information of a plurality of system memory sections within a first section of the first memory, a second section of the first memory, and one or more additional sections within the second memory— ; A step of analyzing the data location information within the retrieved metadata from the metadata portion of the first section in the first memory to determine whether the first section in the first memory currently contains data corresponding to the system memory section in the received request; and In response to determining that the first section within the first memory currently contains data corresponding to the system memory section in the received request, A step of writing a block of data corresponding to the system memory section to the first section within the first memory; and A step indicating completion of writing the block of data to the processor above. A memory management method including
17. In Paragraph 16, In response to determining that the first section in the first memory does not currently contain data corresponding to the system memory section in the received request, A step of further analyzing the data location information within the retrieved metadata to determine whether the second section within the first memory or the memory location within the second memory includes data corresponding to the system memory section in the received request; A step of writing a block of data to the second section in the first memory or to the memory location in the second memory; and A step indicating completion of writing the block of data to the processor above. A memory management method that further includes
18. In Paragraph 16, In response to determining that the first section in the first memory does not currently contain data corresponding to the system memory section in the received request, A step of further analyzing the data location information within the retrieved metadata to determine whether the second section within the first memory or the memory location within the second memory includes data corresponding to the system memory section in the received request; A step of maintaining current data in the first section within the first memory; A step of writing a block of data to the second section in the first memory or to the memory location in the second memory; and A step indicating completion of writing the block of data to the processor above. A memory management method that further includes
19. In Paragraph 17, The metadata from the above metadata portion includes one or more bits, and The above data location information includes a combination of one or more bits that individually correspond to one of the plurality of system memory sections, and The step of analyzing the above data location information is: A step of identifying a combination of one or more bits; and A step of determining whether the above identified combination corresponds to the system memory section in the above received request A memory management method that includes
20. In Paragraph 17, The metadata from the above metadata portion includes one or more error checking and correction (ECC) bits, and The above data location information includes a combination of one or more ECC bits that individually correspond to one of the plurality of system memory sections, and The step of analyzing the above data location information is: A step of identifying a combination of one or more of the above ECC bits; and A step of determining whether the combination of the identified ECC bits corresponds to the system memory section in the received request. A memory management method that includes

Description

Memory tiering techniques in computing systems In computing, memory typically refers to a computing component used to store data for immediate access by a central processing unit (CPU) in a computer or other type of computing device. In addition to memory, a computer may also include one or more computer storage devices (e.g., hard disk drives or HDDs) that permanently store data in the computer. During operation, data such as application instructions may first be loaded from the computer storage devices into memory. The CPU can then execute the application instructions loaded into memory to provide computing services such as word processing, online meetings, etc. This overview is provided to introduce, in a simplified form, selected concepts among those further explained in the detailed description below. This overview is not intended to identify principal or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter. Certain computing devices may include a CPU configured to access different types of memory. For example, a computing device may include a first type of memory that is high-speed and a second type of memory that is slower. An exemplary first type of memory may be Double Data Rate (DDR) Synchronous Dynamic Random-Access Memory (SDRAM) packaged with the CPU. Because this first type of memory is physically close to the CPU, it is sometimes referred to as "near memory." Examples of the second type of memory may include those that the CPU can interface with via Compute Express Link (CXL) or other suitable protocols. Because this second type of memory is sometimes located further away from the CPU than near memory, it may be referred to as "far memory." Using high-speed memory as near memory for the CPU may have certain disadvantages. For example, DDR SDRAM is typically more expensive than that used for far memory. Near memory may also not be accessible or even visible to the operating system (OS) on the computing device. Instead, the CPU has exclusive control over near-field memory. In some data center servers, the cost of DDR SDRAM used as near-field memory can account for up to about 50% of the total cost of the data center server. Therefore, if near-field memory is visible to the OS and accessible by the OS, the capital investment in the data center server and the associated costs for providing various computing services from the data center server can be significantly reduced. Some embodiments of the disclosed technology relate to implementing memory multi-tiering depending on which near memory can be used as a swap buffer for far memory, instead of a dedicated cache memory for the CPU within the computing device. Thus, the CPU can continue to cache data in near memory while the near memory and far memory are exposed to the OS as addressable and allocatable system memory. In certain implementations, a hardware memory controller (e.g., a DRAM controller) may be configured to manage swapping operations with cacheline granularity (e.g., 64 bytes). Thus, the computing device may not require any software intervention or affect the software. In other implementations, a memory controller having both hardware and software components may be used to control these swapping operations. In certain implementations, near memory may have the same or even more units of storage space as far memory. For example, a range of system memory addresses may be covered by a combination of near memory and far memory in a ratio of 2 to 1, 2 to 2, 3 to 1, 3 to 2, 4 to 1, 4 to 3, or other suitable ratios of integers greater than or equal to 1. In one exemplary example, a range of system memory addresses (e.g., 512 GB) may be covered by a combination of near memory and far memory in a ratio of 2 to 2, namely two 128 GB near memory units and two 128 GB far memory units. As such, the range of system memory may be divided into four sections (e.g., A, B, C, and D), each section corresponding to a section of storage space in either near memory or far memory. In a specific embodiment, when using near memory as a swap buffer for far memory, multiple sections of near memory may be configured into individual look-through tiers. For example, a first section of near memory may be configured as Tier 1, and a second section (151B) of near memory may be configured as Tier 2. Far memory may be configured as Tier 3, which may include one or more additional sections. Thus, during an operation, when performing a read of data such as a cache line, the memory controller may be configured to initially determine whether Tier 1 of near memory contains the cache line. When Tier 1 contains the cache line, the memory controller retrieves the cache line from Tier 1, provides the cache line to the requesting entity, and terminates the read operation. When Tier 1 does not contain the cache line, the memory controller may determine whether Tier 2 contains the cache line. This operation