US-12619462-B2 - Transaction performance by parallel WAL IO and parallel waking up transaction commit waiters

Abstract

A method for performing logging of modifications of a database includes, for each backend process of a plurality of backend processes simultaneously, writing a respective log entry to a write-ahead log buffer, submitting a respective commit request requesting the respective log entry be committed to a write-ahead log, and sleeping the respective backend process. The method also includes writing, using a dedicated writing process and direct asynchronous input/output, one or more of the respective log entries in the write-ahead log buffer to the write-ahead log. The dedicated writing process is different from each respective backend process of the plurality of backend processes. The method also includes updating a log sequence number pointer based on the respective log sequence numbers of the one or more of the respective log entries and waking, based on the log sequence number pointer, one or more of the respective backend processes.

Inventors

• Yingjie He
• Yi Ding

Assignees

• GOOGLE LLC

Dates

Publication Date

20260505

Application Date

20230428

Claims (20)

1. 1 . A computer-implemented method executed by data processing hardware that causes the data processing hardware to perform operations comprising: for each respective backend process of a plurality of backend processes simultaneously: writing a respective log entry to a write-ahead log buffer, the respective log entry associated with a respective log sequence number, the respective log sequence number representing a location within the write-ahead log buffer, the write-ahead log buffer comprising volatile memory; submitting a respective commit request, the respective commit request requesting the respective log entry be committed to a write-ahead log; and sleeping the respective backend process; writing, using a dedicated writing process and direct asynchronous input/output, one or more of the respective log entries in the write-ahead log buffer to the write-ahead log, the write-ahead log comprising distributed non-volatile memory, the dedicated writing process different from each respective backend process of the plurality of backend processes; updating a log sequence number pointer based on the respective log sequence numbers of the one or more of the respective log entries; and waking, based on the log sequence number pointer, one or more of the respective backend processes.
2. 2 . The method of claim 1 , wherein waking the one or more of the respective backend processes comprises, for each respective backend process of the one or more of the respective backend processes: selecting, from a thread pool, a respective thread; and waking, using the selected respective thread, the respective backend process.
3. 3 . The method of claim 1 , wherein sleeping the respective backend process comprises a futex operation.
4. 4 . The method of claim 3 , wherein each respective backend process is associated with a separate futex operation.
5. 5 . The method of claim 1 , wherein updating the log sequence number pointer comprises determining a contiguous portion of the write-ahead log occupied by the one or more of the respective log entries.
6. 6 . The method of claim 1 , wherein the write-ahead log buffer comprises a plurality of blocks, each block representing a fixed amount of the write-ahead log buffer.
7. 7 . The method of claim 6 , wherein: each respective block of the plurality of blocks is associated with a respective block pointer; and updating the log sequence number pointer based on the respective log sequence numbers of the one or more of the respective log entries is based on the respective block pointers.
8. 8 . The method of claim 7 , wherein the write-ahead log buffer is associated with a block counter representing a quantity of blocks that are full.
9. 9 . The method of claim 8 , wherein updating the log sequence number pointer based on the respective log sequence numbers of the one or more of the respective log entries is further based on the block counter.
10. 10 . The method of claim 1 , where submitting the respective commit request comprises queuing the respective commit request in a commit queue.
11. 11 . A system comprising: data processing hardware; and memory hardware in communication with the data processing hardware, the memory hardware storing instructions that when executed on the data processing hardware cause the data processing hardware to perform operations comprising: for each respective backend process of a plurality of backend processes simultaneously: writing a respective log entry to a write-ahead log buffer, the respective log entry associated with a respective log sequence number, the respective log sequence number representing a location within the write-ahead log buffer, the write-ahead log buffer comprising volatile memory; submitting a respective commit request, the respective commit request requesting the respective log entry be committed to a write-ahead log; and sleeping the respective backend process; writing, using a dedicated writing process and direct asynchronous input/output, one or more of the respective log entries in the write-ahead log buffer to the write-ahead log, the write-ahead log comprising distributed non-volatile memory, the dedicated writing process different from each respective backend process of the plurality of backend processes; updating a log sequence number pointer based on the respective log sequence numbers of the one or more of the respective log entries; and waking, based on the log sequence number pointer, one or more of the respective backend processes.
12. 12 . The system of claim 11 , wherein waking the one or more of the respective backend processes comprises, for each respective backend process of the one or more of the respective backend processes: selecting, from a thread pool, a respective thread; and waking, using the selected respective thread, the respective backend process.
13. 13 . The system of claim 11 , wherein sleeping the respective backend process comprises a futex operation.
14. 14 . The system of claim 13 , wherein each respective backend process is associated with a separate futex operation.
15. 15 . The system of claim 11 , wherein updating the log sequence number pointer comprises determining a contiguous portion of the write-ahead log occupied by the one or more of the respective log entries.
16. 16 . The system of claim 11 , wherein the write-ahead log buffer comprises a plurality of blocks, each block representing a fixed amount of the write-ahead log buffer.
17. 17 . The system of claim 16 , wherein: each respective block of the plurality of blocks is associated with a respective block pointer; and updating the log sequence number pointer based on the respective log sequence numbers of the one or more of the respective log entries is based on the respective block pointers.
18. 18 . The system of claim 17 , wherein the write-ahead log buffer is associated with a block counter representing a quantity of blocks that are full.
19. 19 . The system of claim 18 , wherein updating the log sequence number pointer based on the respective log sequence numbers of the one or more of the respective log entries is further based on the block counter.
20. 20 . The system of claim 11 , where submitting the respective commit request comprises queuing the respective commit request in a commit queue.

Description

CROSS REFERENCE TO RELATED APPLICATIONS Technical Field This disclosure relates to transaction performance by parallel write-ahead logging (WAL) input/output (IO) and parallel waking up transaction commit waiters. BACKGROUND Transaction performance is critical for online transaction processing (OLTP) applications. In most database management systems (DBMS), before a transaction is committed, a write-ahead log must be flushed to disk. Conventionally, write-ahead logging (WAL) is a serialized operation, where a semaphore or mutex is used to wake transaction committers waiting for transactions to flush to the log. This leads to significant overhead costs when there are large numbers of waiters and does not fully utilize modern processors or disk capacity. SUMMARY One aspect of the disclosure provides a computer-implemented method for performing logging of modifications of a database. The method, when executed by data processing hardware, causes the data processing hardware to perform operations. For each respective backend process of a plurality of backend processes simultaneously, the operations include writing a respective log entry to a write-ahead log buffer. The respective log entry is associated with a respective log sequence number and the respective log sequence number represents a location within the write-ahead log buffer. The write-ahead log buffer includes volatile memory. The operations also include submitting a respective commit request requesting the respective log entry be committed to a write-ahead log and sleeping the respective backend process. The operations also include writing, using a dedicated writing process and direct asynchronous input/output, one or more of the respective log entries in the write-ahead log buffer to the write-ahead log. The write-ahead log includes distributed non-volatile memory and the dedicated writing process is different from each respective backend process of the plurality of backend processes. The operations include updating a log sequence number pointer based on the respective log sequence numbers of the one or more of the respective log entries and waking, based on the log sequence number pointer, one or more of the respective backend processes. Implementations of the disclosure may include one or more of the following optional features. In some implementations, waking the one or more of the respective backend processes includes, for each respective backend process of the one or more of the respective backend processes, selecting, from a thread pool, a respective thread and waking, using the selected respective thread, the respective backend process. Optionally, sleeping the respective backend process includes a futex operation. Each respective backend process may be associated with a separate futex operation. In some implementations, updating the log sequence number pointer includes determining a contiguous portion of the write-ahead log occupied by the one or more of the respective log entries. In some examples, the write-ahead log buffer includes a plurality of blocks and each block represents a fixed amount of the write-ahead log buffer. In some of these examples, each respective block of the plurality of blocks is associated with a respective block pointer and updating the log sequence number pointer based on the respective log sequence numbers of the one or more of the respective log entries is based on the respective block pointers. The write-ahead log buffer may be associated with a block counter representing a quantity of blocks that are full. In some of these examples, updating the log sequence number pointer based on the respective log sequence numbers of the one or more of the respective log entries is further based on the block counter. Optionally, submitting the respective commit request includes queuing the respective commit request in a commit queue. Another aspect of the disclosure provides a system for logging of modifications of a database. The system includes data processing hardware and memory hardware in communication with the data processing hardware. The memory hardware stores instructions that when executed on the data processing hardware cause the data processing hardware to perform operations. For each respective backend process of a plurality of backend processes simultaneously, the operations include writing a respective log entry to a write-ahead log buffer. The respective log entry is associated with a respective log sequence number and the respective log sequence number represents a location within the write-ahead log buffer. The write-ahead log buffer includes volatile memory. The operations also include submitting a respective commit request requesting the respective log entry be committed to a write-ahead log and sleeping the respective backend process. The operations also include writing, using a dedicated writing process and direct asynchronous input/output, one or more of the respective log entries in the write-ahead lo