US-12625635-B1 - Maintaining virtual address range assignments for a vast storage system

Abstract

A method for execution by a processing module of a computing device of a storage network includes tracking virtual address range assignments of a virtual address range for a plurality of memories of the storage network, where data objects are dispersed storage error encoded into pluralities of sets of encoded data slices, and encoded data slices of the pluralities of sets of encoded data slices are stored in the plurality of memories in accordance with the virtual address range assignments. The method further includes determining an updated plurality of memories based on detecting a change to the plurality of memories. The method further includes establishing updated virtual address range assignments for the updated plurality of memories based on the change.

Inventors

• Manish Motwani

Assignees

• PURE STORAGE, INC.

Dates

Publication Date

20260512

Application Date

20210212

Claims (20)

1. 1 . A method for execution, the method comprising: storing a plurality of data via a set of storage units, wherein a first set of data of the plurality of data are stored via a first plurality of memory locations of a first storage unit of the set of storage units, wherein a proper subset of the first set of data are stored in a proper subset of the first plurality of memory locations; migrating only the proper subset of the first set of data from the first storage unit to a second storage unit of the set of storage units; while migrating the proper subset of the first set of data, processing a plurality of data access requests by: based on a first subset of the plurality of data access requests for the first set of data and not the proper subset of the set of data, fully processing each of the first subset of the plurality of data access requests via accessing the first storage unit; and based on a second subset of the plurality of data access requests for the proper subset of the first set of data, performing at least some processing of each of the second subset of the plurality of data access requests; and after the migrating the proper subset of the first set of data is complete: completing processing of the each of the second subset of the plurality of data access requests via accessing the second storage unit to process the data pertaining to the each of the second subset of the plurality of data access requests.
2. 2 . The method of claim 1 , wherein completing processing of the each of the second subset of the plurality of data access requests includes updating storage of the proper subset of the first set of data in the second storage unit.
3. 3 . The method of claim 2 , wherein the second subset of the plurality of data access requests include at least one of: a write request or an edit request.
4. 4 . The method of claim 1 , wherein the first plurality of memory locations of the first storage unit are associated with a first addressing set, and wherein the plurality of data access requests indicate addresses of the first addressing set.
5. 5 . The method of claim 1 , wherein the processing of the each of the second subset of the plurality of data access requests is completed based on a notification that migration of the proper subset of the first set of data is complete.
6. 6 . The method of claim 5 , wherein completing processing of the each of the second subset of the plurality of data access requests is based on coordination of at least one corresponding update via the at least a portion of at least one other storage unit and the second storage unit.
7. 7 . The method of claim 1 , wherein a second proper subset of the first set of data are stored in a second proper subset of the first plurality of memory locations, further comprising: migrating only the second proper subset of the first set of data from the first storage unit to a third storage unit of the set of storage units; wherein processing the plurality of data access requests is further based on, while migrating the migrating the second proper subset of the first set of data: based on a third subset of the plurality of data access requests for the second proper subset of the first set of data, performing at least some processing of each of the third subset of the plurality of data access requests.
8. 8 . The method of claim 7 , wherein fully processing the each of the first subset of the plurality of data access requests via accessing the first storage unit is further based on the first subset of the plurality of data access requests each indicating corresponding data not included in the second proper subset of the first set of data.
9. 9 . The method of claim 1 , wherein the second storage unit is offline during migrating of the proper subset of the first set of data, and wherein the processing of the each of the second subset of the plurality of data access requests is completed based on the second storage unit coming back online after migrating of the proper subset of the first set of data.
10. 10 . The method of claim 1 , wherein the second storage unit stores a second set of data prior to migration of the proper subset of the first set of data, and wherein processing the plurality of data access requests while migrating the first set of data is further based on: based on a third subset of the plurality of data access requests the second set of data, performing at least some processing of each of the third subset of the plurality of data access requests.
11. 11 . The method of claim 10 , wherein the method further comprises: after the migrating the proper subset of the first set of data is complete and after the second storage unit is back online: completing processing of the each of the third subset of the plurality of data access requests via accessing the second storage unit to process the data pertaining to the each of the third subset of the plurality of data access requests.
12. 12 . The method of claim 10 , wherein the first plurality of memory locations of the first storage unit are associated with a first addressing set, wherein a second plurality of memory locations of the second storage unit are associated with a second addressing set, wherein the first subset of the plurality of data access requests and the second subset of the plurality of data access requests indicate addresses of the first addressing set, and wherein the third subset of the plurality of data access requests indicate addresses of the second addressing set.
13. 13 . The method of claim 1 , wherein migrating the proper subset of the first set of data is based on performing a listing operation.
14. 14 . The method of claim 1 , wherein the plurality of data includes a plurality of error encoded data slices generated via error encoding a plurality of data objects.
15. 15 . The method of claim 1 , wherein the plurality of data access requests are processed via at least a portion of at least one other storage unit of the set of storage units allocated to process the plurality of data access requests during migration of the proper subset of the first set of data.
16. 16 . The method of claim 15 , wherein the at least the portion of the at least one other storage unit is established as a virtual storage unit for the migration of the proper subset of the first set of data, and wherein completing processing of the each of the second subset of the plurality of data access requests is based on coordination of the completing processing of the each of the second subset of the plurality of data access requests between the virtual storage unit and the second storage unit.
17. 17 . The method of claim 16 , wherein the coordination includes: sending, from the virtual storage unit to the second storage unit, the data pertaining to the each of the second subset of the plurality of data access requests.
18. 18 . The method of claim 17 , wherein the coordination further includes: sending, from the virtual storage unit to the second storage unit, the second subset of the plurality of data access requests.
19. 19 . The method of claim 1 , further comprising: processing a second plurality of data access requests regarding data of the proper subset of the first set of data after the migrating the proper subset of the first set of data is complete based on accessing the second storage unit.
20. 20 . A computing device comprising: memory; an interface; and a processing module operably coupled to the memory and the interface, wherein the processing module is operable to: store a plurality of data via a set of storage units, wherein a first set of data of the plurality of data are stored via a first plurality of memory locations of a first storage unit of the set of storage units, wherein a proper subset of the first set of data are stored in a proper subset of the first plurality of memory locations; migrate only the proper subset of the first set of data from the first storage unit to a second storage unit of the set of storage units; while migrating the proper subset of the first set of data, process a plurality of data access requests by: based on a first subset of the plurality of data access requests for the first set of data and not included in the proper subset of the set of data, fully processing each of the first subset of the plurality of data access requests via accessing the first storage unit; and based on a second subset of the plurality of data access requests for the proper subset of the first set of data, performing at least some processing of each of the second subset of the plurality of data access requests; and after the migrating the proper subset of the first set of data is complete: complete processing of the each of the second subset of the plurality of data access requests via accessing the second storage unit to process the data pertaining to the each of the second subset of the plurality of data access requests.

Description

CROSS-REFERENCE TO RELATED APPLICATIONS This application claims priority pursuant to 35 U.S.C. § 120 as a continuation of U.S. patent application Ser. No. 16/115,235, filed Aug. 28, 2018, entitled, “SUPPORTING LIVE MIGRATIONS AND RE-BALANCING WITH A VIRTUAL STORAGE UNIT,” which is a continuation-in-part of U.S. patent application Ser. No. 13/289,200, filed Nov. 4, 2011, entitled, “BALANCING MEMORY UTILIZATION IN A DISPERSED STORAGE NETWORK,” issued as U.S. Pat. No. 10,084,770 on Sep. 25, 2018, which claims priority pursuant to 35 U.S.C. § 119(e) to U.S. Provisional Patent Application Ser. No. 61/411,478, filed Nov. 9, 2010, entitled “BALANCING MEMORY UTILIZATION IN A DISPERSED STORAGE NETWORK,” all of which are hereby incorporated herein by reference in their entirety and made part of the present application for all purposes. STATEMENT REGARDING FEDERALLY SPONSORED RESEARCH OR DEVELOPMENT Not Applicable. INCORPORATION-BY-REFERENCE OF MATERIAL SUBMITTED ON A COMPACT DISC Not Applicable. BACKGROUND OF THE INVENTION Technical Field of the Invention This invention relates generally to computer networks and more particularly to dispersing error encoded data. Description of Related Art Computing devices are known to communicate data, process data, and/or store data. Such computing devices range from wireless smart phones, laptops, tablets, personal computers (PC), work stations, and video game devices, to data centers that support millions of web searches, stock trades, or on-line purchases every day. In general, a computing device includes a central processing unit (CPU), a memory system, user input/output interfaces, peripheral device interfaces, and an interconnecting bus structure. As is further known, a computer may effectively extend its CPU by using “cloud computing” to perform one or more computing functions (e.g., a service, an application, an algorithm, an arithmetic logic function, etc.) on behalf of the computer. Further, for large services, applications, and/or functions, cloud computing may be performed by multiple cloud computing resources in a distributed manner to improve the response time for completion of the service, application, and/or function. For example, Hadoop is an open source software framework that supports distributed applications enabling application execution by thousands of computers. In addition to cloud computing, a computer may use “cloud storage” as part of its memory system. As is known, cloud storage enables a user, via its computer, to store files, applications, etc. on an Internet storage system. The Internet storage system may include a RAID (redundant array of independent disks) system and/or a dispersed storage system that uses an error correction scheme to encode data for storage. It is further known, that during data migration, the availability of disks involved in the migration for normal IO is affected. BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWING(S) FIG. 1 is a schematic block diagram of an embodiment of a dispersed or distributed storage network (DSN) in accordance with the present invention; FIG. 2 is a schematic block diagram of an embodiment of a computing core in accordance with the present invention; FIG. 3 is a schematic block diagram of an example of dispersed storage error encoding of data in accordance with the present invention; FIG. 4 is a schematic block diagram of a generic example of an error encoding function in accordance with the present invention; FIG. 5 is a schematic block diagram of a specific example of an error encoding function in accordance with the present invention; FIG. 6 is a schematic block diagram of an example of a slice name of an encoded data slice (EDS) in accordance with the present invention; FIG. 7 is a schematic block diagram of an example of dispersed storage error decoding of data in accordance with the present invention; FIG. 8 is a schematic block diagram of a generic example of an error decoding function in accordance with the present invention; FIG. 9A is a schematic block diagram of an embodiment of a storage unit in accordance with the present invention; FIG. 9B is a diagram illustrating an example of free memory availability for a plurality of memories in accordance with the present invention; FIG. 9C is a diagram illustrating another example of free memory availability for a plurality of memories in accordance with the present invention; FIG. 9D is a diagram illustrating another example of free memory availability for a plurality of memories in accordance with the present invention; FIG. 9E is a schematic block diagram of another embodiment of a storage unit in accordance with the present invention; FIG. 9F is a flowchart illustrating an example of migrating encoded data slices in accordance with the present invention; FIG. 10A is a flowchart illustrating an example of re-assigning memories in accordance with the present invention; FIG. 10B is a flowchart illustrating an example of assigning a new memory in