CN-121996160-A - Data migration method, device, equipment and storage medium thereof

CN121996160ACN 121996160 ACN121996160 ACN 121996160ACN-121996160-A

Abstract

The application belongs to the technical field of big data processing, and relates to a data migration method, a device, equipment and a storage medium thereof, wherein a target data migration instruction is analyzed; obtaining migration direction information of target data based on an analysis result, determining new and old storage system information before and after migration of the target data by analyzing the migration direction information of the target data, obtaining the target data according to the old storage system information, and migrating the target data to a new storage system by combining the migration direction information and preset different data migration strategies. The data migration method provided by the application realizes that the target data is migrated to the new storage system by combining different data migration strategies when the data is migrated, can selectively migrate the migration strategies according to different migration actual conditions, and especially can realize high-efficiency data migration under the condition that the data storage system is more stable in a large data migration scene of financial service or medical service.

Inventors

LIU TANYI
ZHANG GUANCHENG
WAN SHUWU

Assignees

平安科技（深圳）有限公司

Dates

Publication Date: 20260508
Application Date: 20260106

Claims (10)

1. A method of data migration, comprising the steps of: Analyzing a target data migration instruction; obtaining migration pointing information of the target data based on the analysis result; Determining new and old storage system information before and after the migration of the target data by analyzing migration pointing information of the target data; acquiring the target data according to the old storage system information; And combining a preset data migration strategy and the migration pointing information to migrate the target data to a new storage system, wherein the preset data migration strategy comprises a first data migration strategy and a second data migration strategy.
2. The data migration method of claim 1, wherein prior to performing the step of obtaining the target data from old storage system information, the method further comprises: identifying service importance levels corresponding to different data fields in the target data respectively; And according to the service importance level, respectively executing remote mirror image data backup operation on different data fields in the target data in a mode of the service importance level from high to low.
3. The data migration method of claim 1, wherein prior to the step of migrating the target data to a new storage system by performing the step of combining a preset data migration policy with the migration targeting information, the method further comprises: Identifying security levels respectively corresponding to different data fields in the target data; and according to the security level and the preset security level, encrypting the data field with the security level reaching the preset security level to obtain the target data to be migrated after the encryption.
4. A data migration method according to any one of claims 1 or 3, wherein the step of migrating the target data to a new storage system by combining a preset data migration policy and the migration direction information specifically includes: Analyzing the new and old storage system information in the migration pointing information respectively, wherein the analyzed new and old storage system information both contain compatibility information of corresponding data storage structure formats; Based on the compatibility information, identifying whether the new storage system is completely compatible with the data structure format of the target data in the old storage system; If the new storage system is completely compatible with the data structure format of the target data in the old storage system, migrating the target data to the new storage system by utilizing the first data migration strategy; and if the new storage system is not fully compatible with the data structure format of the target data in the old storage system, migrating the target data to the new storage system by utilizing the second data migration strategy.
5. The data migration method of claim 4, wherein the step of migrating the target data to a new storage system using the first data migration policy specifically comprises: identifying a data management node, a data storage node and a management relationship between the data management node and the data storage node contained in the old storage system; constructing corresponding data management nodes and data storage nodes in a target storage space of the new storage system according to the data management nodes, the data storage nodes and the management relations between the data management nodes and the data storage nodes contained in the old storage system; And storing the different data fields in the target data into the corresponding data management nodes and data storage nodes in the new storage system one by one according to the corresponding data management nodes and data storage nodes in the old storage system of the different data fields in the target data.
6. The data migration method of claim 4, wherein the step of migrating the target data to a new storage system using the second data migration policy specifically comprises: And migrating the target data to the new storage system by adopting a preset data migration method, specifically, loading the converted data to the new storage system by utilizing a preset data integration and management assembly through data extraction, data cleaning, de-duplication and data structure format conversion.
7. The data migration method of claim 4, wherein after the step of migrating the target data to a new storage system by combining the preset data migration policy and the migration direction information is performed, the method further comprises: Triggering and executing a target setting instruction, and setting an old storage system as a data standby system, wherein the target setting instruction comprises a previously preset setting instruction after the target data migration is completed; setting the data storage priority of the new storage system as the highest priority; And verifying the data migration result of the new storage system by utilizing the service logic execution verification data prepared in advance to obtain a service logic execution verification result corresponding to the new storage system, wherein if the service logic execution verification result is passed, the target data migration is successful, and if the service logic execution verification result is not passed, the target data is not migrated successfully.
8. A data migration apparatus, comprising: The data migration instruction analysis module is used for analyzing the target data migration instruction; The migration pointing information acquisition module is used for acquiring migration pointing information of the target data based on the analysis result; The new and old storage system information determining module is used for determining new and old storage system information before and after the migration of the target data by analyzing the migration pointing information of the target data; The target data acquisition module is used for acquiring the target data according to the old storage system information; And the target data migration module is used for migrating the target data to a new storage system by combining a preset data migration strategy and the migration pointing information, wherein the preset data migration strategy comprises a first data migration strategy and a second data migration strategy.
9. A computer device comprising a memory having stored therein computer readable instructions which when executed by a processor implement the steps of the data migration method of any one of claims 1 to 7.
10. A computer readable storage medium having stored thereon computer readable instructions which when executed by a processor implement the steps of the data migration method of any one of claims 1 to 7.

Description

Data migration method, device, equipment and storage medium thereof Technical Field The application relates to the technical field of big data processing, which is applied to data migration in an old storage system to a new storage system, and relates to a data migration method, a device, equipment and a storage medium thereof. Background In the big data processing scenario, because of the limitation of the data storage capacity of the old storage system, the aging of the infrastructure, etc., the requirement that the data needs to be migrated to the new storage system can be generated, especially in the financial service or medical service scenario, because the number of clients involved in the two types of services is large, the generated service data can be large, and the long-term use of one old storage system or processing system can cause the data storage and processing pressure, so that the financial service system or medical service system is often required to be upgraded or improved, and the migration processing of the data can be involved. Currently, in order to ensure storage system stability, data migration is performed. However, the existing migration scheme has the problems that after a data storage node is disconnected by adopting a scalable data migration strategy for migration, storage pressure can be shared by the rest data storage nodes, so that the rest data storage disks in an old storage system are occupied, and further, abnormal data storage is caused, and meanwhile, the storage and migration speed of data to a new storage system is slow. Disclosure of Invention The embodiment of the application aims to provide a data migration method, a device, equipment and a storage medium thereof, so that data migration can be performed relatively efficiently under the condition of ensuring the stability of a storage system. In a first aspect, an embodiment of the present application provides a data migration method, which adopts the following technical scheme: a data migration method comprising the steps of: Analyzing a target data migration instruction; obtaining migration pointing information of the target data based on the analysis result; Determining new and old storage system information before and after the migration of the target data by analyzing migration pointing information of the target data; acquiring the target data according to the old storage system information; And combining a preset data migration strategy and the migration pointing information to migrate the target data to a new storage system, wherein the preset data migration strategy comprises a first data migration strategy and a second data migration strategy. In a second aspect, an embodiment of the present application further provides a data migration apparatus, which adopts the following technical scheme: a data migration apparatus comprising: The data migration instruction analysis module is used for analyzing the target data migration instruction; The migration pointing information acquisition module is used for acquiring migration pointing information of the target data based on the analysis result; The new and old storage system information determining module is used for determining new and old storage system information before and after the migration of the target data by analyzing the migration pointing information of the target data; The target data acquisition module is used for acquiring the target data according to the old storage system information; And the target data migration module is used for migrating the target data to a new storage system by combining a preset data migration strategy and the migration pointing information, wherein the preset data migration strategy comprises a first data migration strategy and a second data migration strategy. In a third aspect, an embodiment of the present application further provides a computer device, which adopts the following technical scheme: A computer device comprising a memory having stored therein computer readable instructions which when executed by a processor implement the steps of the data migration method described above. In a fourth aspect, an embodiment of the present application further provides a computer readable storage medium, which adopts the following technical solutions: A computer readable storage medium having stored thereon computer readable instructions which when executed by a processor perform the steps of a data migration method as described above. Compared with the prior art, the embodiment of the application has the following main beneficial effects: The data migration method includes the steps of analyzing a target data migration instruction, obtaining migration direction information of target data based on an analysis result, determining new and old storage system information before and after migration of the target data through analysis of the migration direction information of the target data, obtaining the target data according t