CN-122019509-A - Heterogeneous database migration strategy generation method, equipment and medium

CN122019509ACN 122019509 ACN122019509 ACN 122019509ACN-122019509-A

Abstract

The application discloses a heterogeneous database migration strategy generation method, equipment and medium, wherein the method comprises the steps of respectively obtaining structures and constraint conditions of a source database and a target database, carrying out structural conversion on the structures and the constraint conditions, and generating a structural parameter set; the method comprises the steps of performing risk analysis on constraint conditions of a target database to generate a migration verification data strategy, obtaining a migration verification data set according to the migration verification data strategy, performing data migration operation on the migration verification data set in a preset isolation environment, recording behavior parameters generated in a data migration process, performing risk analysis on the behavior parameters to generate a risk assessment parameter, and matching corresponding migration strategy types from a preset migration processing rule set according to the risk assessment parameter to generate migration strategy parameters which can be directly applied to formal migration. Overcomes the defects of the prior art that the artificial experience and risk discovery are dependent, and obviously improves the certainty and the overall efficiency of the migration process.

Inventors

MIAO JIAN
LI JINXU
ZHANG YUANCHAO
ZHONG WEIFENG
LV XINJIE

Assignees

瀚高基础软件(杭州)有限公司

Dates

Publication Date: 20260512
Application Date: 20260211

Claims (10)

1. A heterogeneous database migration policy generation method, the method comprising: Respectively obtaining the structure and constraint conditions of a source database and a target database, and carrying out structural conversion on the structure and constraint conditions to generate a structural parameter set; Performing risk analysis on the constraint conditions of the target database in the structured parameter set to generate a migration verification data strategy for verifying the suitability of the migration strategy; Obtaining a migration verification data set according to the migration verification data strategy, wherein the migration verification data set comprises test data constructed based on constraint conditions of a target database and real data which are extracted from a source database and meet preset risk characteristics; in a preset isolation environment, executing data migration operation on the migration verification data set, and recording behavior parameters generated in the data migration process, wherein the behavior parameters at least comprise data processing behaviors, type conversion information, constraint triggering information and execution state information; performing risk analysis on the behavior parameters to generate risk assessment parameters, wherein the risk assessment parameters are used for representing migration abnormality, precision change and constraint triggering risk degree of different fields and data types; And matching corresponding migration strategy types from a preset migration processing rule set according to the risk assessment parameters to generate migration strategy parameters which can be directly applied to formal migration.
2. The method according to claim 1, wherein the risk analysis is performed on the target database constraint condition in the structured parameter set to generate a migration verification data policy for verifying suitability of the migration policy, specifically comprising: Analyzing constraint conditions of each field to be migrated in a target database according to the structured parameter set, and extracting risk features which cause migration semantic difference and execution failure, wherein the risk features at least comprise precision and length risks, null constraint risks, character set compatibility risks, data type mapping risks and check constraint risks; aiming at each extracted risk feature, generating a corresponding verification data construction instruction and a data extraction instruction; and integrating the verification data construction instruction and the data extraction instruction which are generated aiming at different risk characteristics to form a migration verification data strategy.
3. The method according to claim 1, wherein the obtaining a migration verification data set according to the migration verification data policy specifically comprises: determining verification items to be extracted from a source database according to data extraction instructions in the migration verification data strategy; generating corresponding data query sentences according to the extracted verification items so as to extract real data meeting preset risk characteristics from a corresponding table of a source database, wherein the generation of the query sentences is based on field metadata and risk judging conditions in the structured parameter set; Determining a verification item of test data to be constructed according to a verification data construction instruction in the migration verification data strategy; constructing test data for test constraint through a preset data generation algorithm according to verification items of the test data and constraint boundaries of a target database in the structured parameter set; combining the real data with the test data to form an initial migration verification data set; And performing duplicate removal and consistency verification on the initial migration verification data set, and adding metadata labels for the initial migration verification data set subjected to duplicate removal and consistency verification to obtain the migration verification data set, wherein the metadata labels at least comprise data sources and specific risk characteristics.
4. The method according to claim 1, wherein the performing a data migration operation on the migration verification data set in a preset isolation environment and recording a behavior parameter generated in a data migration process specifically includes: constructing a test database environment which is independent of a production system and isomorphic to a target production environment, and implementing network isolation and resource limitation configuration on the test database environment; Loading the migration verification data set into the test database environment, and starting a test migration task according to an initial general migration configuration; Writing data into a target database in a preset batch mode, checking the execution state after the data of each batch is written, and if a predictable constraint and blocking error is encountered, rolling back the current batch data and then continuing to execute the subsequent batch; synchronously deploying monitoring probes on all links of data reading, type conversion, transmission and writing so as to capture all abnormal and warning information generated in the migration process; The method comprises the steps of synchronously recording original behavior parameters corresponding to data reading, type conversion, transmission and writing links, wherein the original behavior parameters comprise the state and time consumption of each data processing link, the actual storage value of a source data value in a target database after conversion, the type and specific data value of a triggered constraint, and the execution state and resource occupation index of the whole migration task; associating the original behavior parameters with source data records and target fields of the trigger behaviors; After the test migration is completed, collecting all the associated original behavior parameters, aligning and sorting according to time sequence, and packaging the sorted original behavior parameters by taking the data record in the migration verification data set as an index to obtain a migration behavior log so as to record the behavior parameters generated in the data migration process.
5. The method according to claim 1, wherein the risk analysis is performed on the behavior parameters to generate risk assessment parameters, specifically comprising: Extracting a behavior parameter subset corresponding to each verification data record from the migration behavior log; Calculating risk values of each behavior parameter subset according to a preset risk quantization model, and determining risk values corresponding to each behavior parameter subset; And aggregating and attributing the risk values corresponding to each behavior parameter subset according to the mode of the source database field to generate a set of risk assessment parameters taking the field as a unit, wherein the risk assessment parameters at least comprise risk types, risk grades and risk quantification values.
6. The method of claim 5, wherein the calculating risk values for each subset of the behavioral parameters according to the preset risk quantization model, and determining the risk value corresponding to each subset of the behavioral parameters, specifically includes: performing weighted evaluation according to the error level, the error frequency and the error type triggered in the test migration process to calculate an abnormal risk value in the test migration process; quantifying by comparing the difference degree of the source data value and the actual storage value in the target database and combining the upper precision limit of the target field so as to calculate the precision loss risk value in the test migration process; And according to the boundary distance of the data value distance triggering target database constraint conditions, evaluating so as to calculate a constraint approximation risk value in the test process.
7. The method according to claim 1, wherein the matching, according to the risk assessment parameter, a corresponding migration policy type from a preset migration processing rule set to generate a migration policy parameter that can be directly applied to formal migration specifically includes: Performing multistage matching in a preset migration processing rule base according to the risk type and the risk level in the risk assessment parameter set; generating at least one candidate migration processing rule for each risk field according to the matching result; determining a final candidate rule according to the risk value, the rule execution cost estimation and the historical success rate of each candidate rule for the field with a plurality of candidate rules by a preset decision algorithm; And integrating the final candidate rules selected by all the fields to generate migration strategy parameters which can be directly applied to formal migration, wherein the parameters at least comprise preprocessing cleaning rules aiming at high-risk fields, batch sizes and concurrent thread numbers of different types of data conversion, and processing and alarming strategies when specific errors are encountered in the migration process.
8. The method according to claim 1, wherein the steps of obtaining the structure and the constraint condition of the source database and the target database, respectively, and performing structural transformation on the structure and the constraint condition to generate the structural parameter set include: The metadata access interfaces are respectively connected to the source database and the target database through metadata access interfaces provided by the database system; The method comprises the steps of obtaining structural information of a table to be migrated from a source database, wherein the structural information comprises a table name, a field name of each field, a field data type, a field length, numerical precision, whether null is allowed or not and a default value; The method comprises the steps of obtaining structure information and constraint conditions of a corresponding target table from a target database, wherein the constraint conditions comprise field data types, length and precision limitations, non-null constraints, unique constraints, primary key constraints, foreign key constraints and checking constraints; converting and mapping the format of the obtained constraint conditions of the source database structure and the target database according to a predefined unified data model, and establishing a corresponding relation between a source field and a target field; and integrating the converted information according to the corresponding relation to generate a unified structured parameter set, wherein each record of the structured parameter set completely describes the type mapping relation, constraint difference and data compatibility boundary of one source-target field pair.
9. A heterogeneous database migration policy generation apparatus, characterized by comprising: at least one processor, and A memory communicatively coupled to the at least one processor, wherein, The memory stores instructions executable by the at least one processor to enable the at least one processor to perform a heterogeneous database migration policy generation method according to any of the preceding claims 1-8.
10. A heterogeneous database migration policy generation storage medium storing computer executable instructions, wherein the computer executable instructions are capable of performing a heterogeneous database migration policy generation method according to any of the preceding claims 1-8.

Description

Heterogeneous database migration strategy generation method, equipment and medium Technical Field The present application relates to the field of computer software technologies, and in particular, to a method, an apparatus, and a medium for generating a migration policy of a heterogeneous database. Background With the continuous evolution of information systems and the acceleration of the localization substitution process of databases, the original system constructed based on a specific commercial database generally faces the migration requirement of heterogeneous databases when being upgraded, reconstructed or deployed in a cross-platform manner. Different database management systems have significant differences in data type definition, storage structure, constraint mechanism, transaction processing semantics and the like, so that heterogeneous database migration becomes an unavoidable key link in system transformation. The existing heterogeneous database migration scheme is mostly dependent on manual experience or static template configuration parameters, and lacks an effective verification and evaluation means for the difference between the actual data characteristics of the source database and the constraint conditions of the target database before migration is executed. Especially in domestic substitution scenes, the difference of source and target databases is more prominent, and static configuration is difficult to adapt to the requirements of different systems, business data and migration stages. Moreover, due to the lack of a pre-evaluation mechanism, risks such as data truncation, constraint conflict, semantic inconsistency and the like are often discovered during migration or after completion, the final migration failure and reworking risks are increased, and the efficiency is greatly reduced. Meanwhile, the configuration difference of migration personnel is large, so that the migration process lacks unified standards, the consistency and repeatability of results are difficult to guarantee, and the popularization and application of the heterogeneous database migration technology in a large-scale domestic substitution scene are severely restricted. Disclosure of Invention In order to solve the technical problems in the background art, the embodiment of the application provides a heterogeneous database migration policy generation method, equipment and medium, which comprise the steps of respectively obtaining structures and constraint conditions of a source database and a target database, carrying out structural conversion on the structures and constraint conditions to generate a structured parameter set, carrying out risk analysis on the constraint conditions of the target database in the structured parameter set to generate a migration verification data policy for verifying suitability of the migration policy, obtaining the migration verification data set according to the migration verification data policy, wherein the migration verification data set comprises test data constructed based on the constraint conditions of the target database and real data which are extracted from the source database and meet preset risk characteristics, executing data migration operation on the migration verification data set in a preset isolation environment, recording behavior parameters generated in a data migration process, wherein the behavior parameters at least comprise data processing behaviors, type conversion information, constraint triggering information and execution state information, carrying out risk analysis on the behavior parameters to generate risk assessment parameters, wherein the risk assessment parameters are used for representing different migration trigger fields, types and the degree of constraint triggering the migration rules, and the risk assessment parameters can be directly matched with the migration policy according to the preset migration rules. In one example, risk analysis is performed on constraint conditions of a target database in the structured parameter set to generate a migration verification data policy for verifying suitability of the migration policy, and the method specifically comprises the steps of analyzing constraint conditions of each field to be migrated in the target database according to the structured parameter set, extracting risk features which cause migration semantic differences and execution failures, wherein the risk features at least comprise precision and length risks, null constraint risks, character set compatibility risks, data type mapping risks and verification constraint risks, generating corresponding verification data construction instructions and data extraction instructions for each extracted risk feature, and integrating the verification data construction instructions and the data extraction instructions generated for different risk features to form the migration verification data policy. In one example, a migration verification data set is obt