CN-121807239-B - Error correction storage method and system based on polynomial interpolation mixed verification

CN121807239BCN 121807239 BCN121807239 BCN 121807239BCN-121807239-B

Abstract

The application relates to the technical field of distributed storage, and discloses an error correction storage method and system based on polynomial interpolation hybrid verification. On the basis of carrying out block storage on original data, the method constructs a unified polynomial expression relation based on the number relation among data blocks and the generation relation of check data, and calculates and forms a state quantity used for representing the data recovery supporting capacity according to the unified polynomial expression relation. And judging the value interval of the state quantity, distinguishing different data failure processing modes, and selecting a corresponding data recovery strategy under the corresponding mode to execute recovery operation. The method ensures that the recovery process after the data failure has clear processing basis and stable execution path, is beneficial to reducing the uncertainty of the data recovery to the system resource occupation state in the distributed storage environment with continuously changing node scale, fault mode and operation load, and is beneficial to maintaining the long-term stable operation of the system.

Inventors

CAI HONGLIANG
He Daoxian
TANG DAN
YAO JUNLONG
HE LEI

Assignees

成都信息工程大学

Dates

Publication Date: 20260512
Application Date: 20260309

Claims (10)

1. An error correction storage method based on polynomial interpolation mixed verification is characterized by comprising the following steps: Acquiring original data to be stored in a distributed storage system, and dividing the original data into a plurality of data blocks; Calculating and forming a first data recovery support state quantity based on the data blocks, wherein the first data recovery support state quantity is a composite technical quantity and is used for representing an association consistency state and a redundancy coverage state of the data blocks under a polynomial expression relationship, the first data recovery support state quantity is obtained by constructing a unified polynomial expression relationship based on a number relationship of the data blocks and a generation relationship of corresponding check data, and calculating the corresponding integrity of the data blocks and the check data blocks based on the polynomial expression relationship; Determining a data failure processing mode based on the value interval of the first data recovery support state quantity, wherein the data recovery strategy based on the local association relation is determined when the first data recovery support state quantity is in the first value interval, and the data recovery strategy based on the global association relation is determined when the first data recovery support state quantity is in the second value interval; Selecting a corresponding data recovery strategy according to the determined data failure processing mode, and executing data recovery processing based on the polynomial expression relationship; and outputting a data recovery result.
2. The method of claim 1, wherein the first data recovery support state quantity is formed based on historical data block storage state information and current data block distribution state information, wherein the historical data block storage state information includes historical data block integrity states and historical verification data availability states, and wherein the current data block distribution state information includes a distribution relationship of data blocks and verification data blocks in a storage node.
3. The method of claim 2, wherein the first data recovery support state quantity is comprised of at least two sub-state quantities including a first associated coherency sub-state quantity for characterizing the integrity of a polynomial representation relationship between data blocks and a first redundant overlay sub-state quantity for characterizing the coverage of a data block by a check data block.
4. A method according to claim 3, wherein the first data recovery support state quantity is calculated by weighted combination of the first associated consistency sub-state quantity and a first redundancy covering sub-state quantity, and is output in the form of a numerical interval for representing a data recovery support capability level.
5. The method of claim 4, wherein the interval of values of the first data recovery support state quantity is partitioned based on historical operating state statistics and includes at least one upper interval and at least one lower interval for differentiating between different data recovery support capacity levels.
6. The method of claim 1, wherein different data failure handling modes are determined based on the first data recovery support state quantity being in different value intervals, respectively, the different value intervals corresponding to different number ranges of data block failure conditions.
7. The method according to claim 1, wherein, before performing the operation of determining the data failure processing mode based on the first data recovery support state quantity, consistency judgment is made on the first data recovery support state quantity formed in a plurality of consecutive time periods, and the corresponding data failure processing mode is determined only when the consistency satisfies a preset condition.
8. The method of claim 1, wherein the data window length or historical data reference range that participates in the formation of the first data recovery support state quantity is dynamically adjusted in different data failure handling modes.
9. The method of claim 1, wherein the data recovery result includes a data block group identification to which the failed data block belongs, for characterizing a logical extent of the failed location in the distributed storage system.
10. An error correction storage system based on polynomial interpolation hybrid verification, comprising: The data acquisition and blocking module is used for acquiring original data to be stored in the distributed storage system and dividing the original data into a plurality of data blocks; The state quantity forming module is connected with the data acquisition and blocking module and is used for calculating and forming a first data recovery support state quantity based on the data blocks, wherein the first data recovery support state quantity is a composite technical quantity and is used for representing an association consistency state and a redundancy covering state of the data blocks under a polynomial representation relation, the state quantity forming module is used for constructing a unified polynomial representation relation based on the number relation of the data blocks and a generation relation of corresponding check data, and calculating the corresponding integrity of the data blocks and the check data blocks based on the polynomial representation relation; The processing mode determining module is connected with the state quantity forming module and is used for determining a data failure processing mode based on the value interval of the first data recovery supporting state quantity, wherein the data recovery strategy based on the local association relation is determined when the first data recovery supporting state quantity is in the first value interval, and the data recovery strategy based on the global association relation is determined when the first data recovery supporting state quantity is in the second value interval; the data recovery executing module is connected with the processing mode determining module and is used for selecting a corresponding data recovery strategy according to the determined data failure processing mode and executing data recovery processing based on the polynomial expression relation; And the result output module is connected with the data recovery execution module and is used for outputting a data recovery result.

Description

Error correction storage method and system based on polynomial interpolation mixed verification Technical Field The embodiment of the invention relates to the technical field of distributed storage, in particular to an error correction storage method and system based on polynomial interpolation hybrid verification. Background During the engineering operation of a distributed storage system, data is typically stored in a plurality of storage nodes in a decentralized manner to meet capacity expansion and reliability requirements. As the system scale increases and the operating time increases, the number of storage nodes, the operating load, and the failure mode exhibit constantly changing characteristics. In practical applications, when a storage node fails or a data block fails, the system needs to perform a corresponding data recovery operation to maintain the integrity and availability of the data. However, in a complex and dynamically changing operation environment, the recovery process after the occurrence of data failure often has a large occupation on system resources, and the resource consumption level has a large uncertainty, so that the system is difficult to keep stable and controllable under different operation conditions. Such an uncertain resource occupancy state can adversely affect the proper operation of the system in engineering practice, such as causing network bandwidth contention, increased storage read-write pressure, and processing delay fluctuations, thereby limiting the continued service capability of the system in the event of high load or multi-node failure. Particularly, under the condition that the node scale, the fault mode and the operation load are continuously changed, the influence of the data recovery process on the overall stability of the system is more obvious, and the data recovery process is easy to become an important factor for restricting the long-term stable operation of the distributed storage system. Therefore, how to effectively support the recovery process after the data failure under complex and changeable operation conditions, and avoid the unpredictable influence on the system operation state, becomes a technical problem which objectively exists in the existing distributed storage technology and needs attention. Disclosure of Invention Aiming at the technical problems, an error correction storage method and system based on polynomial interpolation mixed verification are provided to effectively support a recovery process after data failure in a distributed storage system, so that uncertain influence of data recovery on the system operation state is reduced under complex operation conditions. To achieve the above object, according to a first aspect of the present invention, there is provided an error correction storage method based on polynomial interpolation hybrid verification, including the steps of: Acquiring original data to be stored in a distributed storage system, and dividing the original data into a plurality of data blocks; Calculating and forming a first data recovery support state quantity based on the data blocks, wherein the first data recovery support state quantity is a composite technical quantity and is used for representing the association consistency state and the redundancy covering state of the data blocks under the polynomial representation relation, the first data recovery support state quantity is obtained by constructing a unified polynomial representation relation based on the number relation of the data blocks and the generation relation of corresponding check data, and calculating the corresponding integrity of the data blocks and the check data blocks based on the polynomial representation relation; Determining a data failure processing mode based on a value interval of a first data recovery support state quantity, wherein the data failure processing mode is determined to be a data recovery strategy based on a local association relationship when the first data recovery support state quantity is in the first value interval, and is determined to be a data recovery strategy based on a global association relationship when the first data recovery support state quantity is in a second value interval; Selecting a corresponding data recovery strategy according to the determined data failure processing mode, and executing data recovery processing based on the polynomial expression relation; and outputting a data recovery result. In a second aspect of the present invention, there is provided an error correction storage system based on polynomial interpolation hybrid verification, comprising: the data acquisition and blocking module is used for acquiring original data to be stored in the distributed storage system and dividing the original data into a plurality of data blocks; The state quantity forming module is connected with the data acquisition and blocking module and is used for calculating and forming a first data recovery supporting s