EP-4736020-A1 - TRANSACTIONAL ACCESS TO RESOURCE REPOSITORIES

Abstract

Resources, like source code, can be stored in a repository and managed through a resource repository system. The resource repository system includes a processor and a memory for storing a plurality of data structures, each data structure comprising a current version of a corresponding resource and at least one past version of the corresponding resource. The processor is communicatively coupled to the memory and configured to: request a transaction on a resource of a repository; create a snapshot of the repository; perform reads and/or writes of the transaction against the snapshot; evaluate potential conflicts between concurrently committed transactions; if no conflict is found, commit the transaction by persisting it to a write-ahead log; and update the repository based on the write-ahead log.

Inventors

• HILTUNEN, Sami

Assignees

• GitLab Inc.

Dates

Publication Date

20260506

Application Date

20240509

Claims (20)

1. 1. A resource repository system comprising: a memory for storing a plurality of data structures, each data structure comprising a current version of a corresponding resource and at least one past version of the corresponding resource; a processor communicatively coupled to the memory and configured to: receive a request for a transaction on a resource of a repository; create a snapshot of the repository; perform reads and/or writes of the transaction against the snapshot; evaluate potential conflicts between the reads and/or the writes of the transaction and those of other concurrently committed transactions; if no conflict is found, commit the transaction to a write-ahead log; and update the repository based on the changes recorded in the write-ahead log.
2. 2. The system of claim 1, wherein the processor is configured to commit the transaction to the write-ahead log by: creating a log entry containing the changes to be committed with the transaction; and persisting the log entry into the write-ahead log.
3. 3. The system of claim 2, wherein the processor is configured to update the repository by: reading a transaction’s changes from the write-ahead log; and based on the changes of the transaction, applying the transaction to the repository by performing the recorded changes.
4. 4. The system of claim 1, wherein the processor is configured to delete the snapshot after committing the transaction to the write-ahead log.
5. 5. The system of claim 1, wherein evaluating the potential conflicts comprises checking transactions logged in the write-ahead log to determine whether the resource is changed by a concurrent transaction that committed concurrently against the repository.
6. 6. The system of claim 1, wherein the processor is configured to block writes into the repository while one or more snapshots are created to maintain consistency.
7. 7. The system of claim 1 , wherein if a crash occurs while one or more transactions have been committed to the write-ahead log but not yet folly applied to the repository, the processor is configured to recover commited transactions from the write-ahead log and update the repository accordingly.
8. 8. The system of claim 1 , wherein if a conflict is found, the processor is configured to discard the transaction and delete the snapshot associated with the transaction.
9. 9. The system of claim 1, wherein the processor is configured to provide an alert associated with the conflict if found.
10. 10. The system of claim 1, wherein the processor is configured to create the snapshot of the repository by: copying a directory structure of the repository into a temporary directory and hard linking resources of the repository to the temporary directory; or using a copy-on-write functionality of a filesystem to create a clone of the repository into a temporary directory.
11. 11. A method for performing a transaction on a resource on a database, the method comprising: receiving a request for a transaction on a resource of a repository; creating a snapshot of the repository; performing reads and/or writes of the transaction against the snapshot; evaluating potential conflicts between the reads and/or the writes of the transaction and those of other concurrently committed transactions; if no conflict is found, commiting the transaction to a write-ahead log; and updating the repository based on the changes recorded in the write-ahead log.
12. 12. The method of claim 11 , wherein committing the transaction to the write-ahead log comprises: creating a log entry containing the changes to be committed with the transaction; and persisting the log entry into the write-ahead log.
13. 13. The method of claim 12, wherein updating the repository comprises: reading a transaction’s changes from the write-ahead log; and based on the changes of the transaction, applying the transaction to the repository by performing the recorded changes.
14. 14. The method of claim 11 further comprises deleting the snapshot after committing the transaction to the write-ahead log.
15. 15. The method of claim 1 1 wherein evaluating the potential conflicts comprises checking transactions logged in the write-ahead log to determine whether the resource is changed by a concurrent transaction that committed concurrently against the repository.
16. 16. The method of claim 11 further comprising blocking writes into the repository while one or more snapshots are created to maintain consistency.
17. 17. The method of claim 11 further comprising, if a crash occurs while one or more transactions have been commited to the write-ahead log but not yet folly applied to the reposi tory, the processor is configured to recover committed transactions from the write- ahead log and update the repository accordingly.
18. 18. The method of claim 11 further comprising, if a conflict is found, discarding the transaction and delete the snapshot associated with the transaction.
19. 19. The method of claim 11 wherein creating the snapshot of the repository comprises: copying a directory structure of the repository into a temporary directory and hard linking resources of the repository to the temporary directory; or using a filesystem’s copy-on-write functionality to create a clone of the repository into a temporary directory.
20. 20. A computer program product comprising a non-transitory computer readable storage medium having instructions encoded thereon that, when executed by a computing system, cause the computing system to perform operations including: creating a snapshot of the repository; performing reads and/or writes of the transaction against the snapshot; evaluating potential conflicts between the reads and/or the writes of the transaction and those of other concurrently committed transactions; if no conflict is found, committing the transaction to a write-ahead log; and updating the repository based on the changes recorded in the write-ahead log.

Description

TRANSACTIONAL ACCESS TO RESOURCE REPOSITORIES Inventor: Sami Hiltunen BACKGROUND CROSS REFERENCE TO RELATED APPLICATIONS [0001] This application claims the benefit of U.S. Application No. 18/368,378, entitled “Transactional Access to Resource Repositories” and filed September 14, 2023, which is incorporated by reference. TECHNICAL FIELD [0002] The subject matter described relates to providing transactional access to resource repositories. BACKGROUND INFORMATION [0003] Resources, like source code, can be managed through a resource repository system. However, the resource repository system may not guarantee Atomicity, Consistency, Isolation, and Durability (ACID) properties. Not adhering to ACID properties can lead to various issues with data integrity and correctness. The properties guarantee: Atomicity: a transaction’s changes are performed as a single unit. Either all changes of a transaction are successfully performed or none of them are. Consistency: the data is always in consistent state when the transaction begins and ends. Isolation: transactions are isolated from the changes of other concurrent transactions. Transactions behave as if they were executed serially, one after one sequentially even if they were actually executed concurrently. Durability: the changes committed by a transaction are persisted even in face of system failures. SUMMARY [OOM] The above and other problems may be addressed by systems and methods for providing transactional access to resource repositories. In one embodiment, the resource repository system includes a processor and a memory for storing data structures. Each data structure can store a current version of a corresponding resource and at least one past version of the corresponding resource. The processor is communicatively coupled to the memory and configured to: request a transaction on a resource; create a snapshot of the repository; perform the transaction’s reads and/or writes against the snapshot; evaluate potential conflicts between the transaction and other concurrently commited transactions; if no conflict is found, commit the transaction to a write-ahead log; and update the repository based on the write-ahead log. BRIEF DESCRIPTION OF THE DRAWINGS [0005] Figure (FIG.) 1 is a block diagram of a networked computing environment suitable for managing the code of a software project, according to one embodiment. FIG. 2 illustrates a block diagram of the computing server of FIG. 1, according to one embodiment. [0006] FIG. 3 is a flowchart depicting a process for providing transactional access to a resource repository, according to one embodiment. [0007] FIG. 4 illustrate a sequence diagram describing how transaction flows, according to one embodiment. [0008] FIG. 5 is a block diagram illustrating an example computer suitable for use in the networked computing environment of FIG. 1, according to one embodiment. DETAILED DESCRIPTION [0009] The figures and the following description describe certain embodiments by way of illustration only. One skilled in the art will readily recognize from the following description that alternative embodiments of the structures and methods may be employed without departing from the principles described. Wherever practicable, similar or like reference numbers are used in the figures to indicate similar or like functionality. Where elements share a common numeral followed by a different letter, this indicates the elements are similar or identical. A reference to the numeral alone generally refers to any one or any combination of such elements unless the context indicates otherwise. EXAMPLE SYSTEMS [0010] FIG. 1 illustrates one embodiment of a networked computing environment 100 suitable for managing the code of a software project. In the embodiment shown, the networked computing environment includes a source code repository 110, one or more client devices 120, one or more code scanners 130, and a computing server 190, all connected via a network 170. In oilier embodiments, the networked computing environment 100 includes different or additional elements. In addition, the functions may be distributed among the elements in a different manner than described. For example, although the code scanners 130 and the computing server 190 are shown as distinct entities, in some embodiments the corresponding functionality is provided by a single computing system (e.g., a server). [0011] The source code repository 110 includes one or more computer-readable storage media that store the code for one or more software projects. In one embodiment, the source code repository 110 stores the code for a source code management (SCM) system. The SCM system may store multiple versions of the code for the software project to enable updates to be rolled back and developers to review the historical status of the code, etc. [0012] The client devices 120 are computing devices with which software engineers may edit code and connnit updates to the code repository 110. F