CN-122019491-A - Control method, device and storage medium for data fusion compression

CN122019491ACN 122019491 ACN122019491 ACN 122019491ACN-122019491-A

Abstract

The application discloses a control method, equipment and storage medium for data fusion compression, and relates to the technical field of data processing, wherein the method comprises the steps of responding to a data compression instruction, determining a candidate compression path matched with inherent attribute characteristics of data to be compressed and calculation force state parameters of a current system; and in the compression process, executing a resource scheduling action based on a resource scheduling rule corresponding to the target execution path, and executing the compression action based on the target execution path to obtain a compressed file. According to the method, the technical problem of low data compression efficiency is solved by dynamic path matching and decision function selection of the optimal compression path and self-adaptive rule iteration, the compression efficiency and the calculation force utilization rate are improved, and the decision rule self-optimization is realized.

Inventors

LI FULIN
ZHANG YUCHEN

Assignees

深圳市石犀科技有限公司

Dates

Publication Date: 20260512
Application Date: 20260415

Claims (10)

1. A method for controlling data fusion compression, the method comprising: Responding to a data compression instruction, and determining candidate compression paths matched with inherent attribute characteristics of data to be compressed and calculation force state parameters of a current system; Determining benefit scores corresponding to the candidate compression paths by taking the inherent attribute characteristics and the calculation force state parameters as independent variables of decision functions, wherein the decision functions correspond to the candidate compression paths one by one; Selecting the candidate compression path with the highest profit score as a target execution path; And in the compression process, executing a resource scheduling action based on a resource scheduling rule corresponding to the target execution path, and executing a compression action based on the target execution path to obtain a compressed file.
2. The method of claim 1, wherein the step of determining candidate compression paths matching the intrinsic attribute characteristics of the data to be compressed and the computational power state parameters of the current system in response to the data compression instruction comprises: responding to the data compression instruction, and determining an initial path meeting attribute requirements and load requirements according to the inherent attribute characteristics and the calculation force state parameters; Analyzing the block continuity characteristics of the data to be compressed, and determining the block partition adaptive parameters of the data to be compressed; And selecting the initial path matched with the blocking adaptive parameter as the candidate compression path.
3. The method for controlling data fusion compression according to claim 2, wherein the step of analyzing the block continuity feature of the data to be compressed and determining the blocking adaptation parameter of the data to be compressed includes: Analyzing the block continuity characteristics of each piece of segmented content in the data to be compressed to obtain a block-level characteristic sequence; calculating local information entropy of all block level features in the block level feature sequence, and arranging the local information entropy according to the sequence of the block level features to obtain a local entropy value sequence; Based on the local entropy value sequence, comparing entropy value differences of adjacent blocks, counting total duty ratio of blocks meeting continuous requirements, and calculating to obtain the blocking adaptive parameters of the data to be compressed.
4. The method of claim 1, wherein determining the profit margin for each candidate compression path using the intrinsic property and the computational power state parameter as arguments of a decision function comprises: normalizing the inherent attribute characteristics and the calculation force state parameters according to an input characteristic rule to generate characteristic input parameters; Configuring gain weight parameters corresponding to the candidate compression paths according to a weight library for each candidate compression path; Inputting the characteristic input parameters into the decision function, and calculating data adaptation benefits, calculation load benefits and transmission cost benefits of the candidate compression paths by combining the benefit sub-item parameters of each candidate compression path; And carrying out weighted summation on the data adaptation benefits, the computational load benefits and the transmission cost benefits of the candidate compression paths according to the benefit weights to obtain the benefit scores corresponding to the candidate compression paths.
5. The method of claim 1, wherein the step of selecting the candidate compression path with the highest profit score as a target execution path comprises: binding and pairing each candidate compression path with the corresponding profit part to obtain a candidate profit list; Correcting the transmission cost of the corresponding candidate path according to the refinement decision mark in the candidate revenue list to obtain a corrected candidate revenue list; sorting the corrected candidate profit list from high to low according to the profit score to obtain an ordered candidate sequence; and screening out the highest benefit score of the ordered candidate sequence, and determining a candidate path corresponding to the highest benefit score as the target execution path of the current compression task.
6. The method for controlling data fusion compression according to claim 1, wherein in the compression process, the step of executing a resource scheduling action based on a resource scheduling rule corresponding to the target execution path and executing a compression action based on the target execution path to obtain the compressed file comprises: In the compression process, matching the resource scheduling rule corresponding to the target execution path, and matching the adaptive compression algorithm through the redundancy of the data to be compressed; preprocessing the data to be compressed according to the resource scheduling rule, and if the data to be compressed is a heterogeneous path, asynchronously transmitting the data to be compressed to a heterogeneous video memory; Invoking a computing power unit corresponding to the target execution path, loading a compression algorithm matched with the data to be compressed, and compressing the preprocessed data to be compressed to obtain segmented compressed data; And carrying out standardization processing on the segmented compressed data, and adding a file check head to obtain the compressed file.
7. The method for controlling data fusion compression according to claim 1, wherein in the compression process, the method for controlling data fusion compression further comprises, after the step of obtaining the compressed file, executing a resource scheduling action based on a resource scheduling rule corresponding to the target execution path and executing a compression action based on the target execution path: correlating the compression speed and the compression ratio of the current compression process with the integrity check result of the compressed file to obtain performance data of the current compression process; Correspondingly storing the performance data, the characteristic parameters and the path selection structure in a historical task library, and screening out the latest effective historical task data; And based on the effective historical task data, carrying out self-adaptive optimization on decision parameters, and synchronously updating the updated decision parameters to a decision rule base.
8. The method for controlling data fusion compression according to claim 7, wherein the step of performing adaptive optimization of decision parameters based on the valid historical task data and synchronously updating the updated decision parameters to a decision rule base comprises: Counting the performance comparison of the two paths and the failure rate of the heterogeneous paths from the effective historical task data, and adjusting boundary constraint according to a threshold optimization rule to obtain a target decision threshold; According to the effective historical task data and the weight optimization rule, iteratively calculating a target weight parameter, and updating a triggering rule of a refinement decision marker; And updating the target decision threshold, the target weight parameter and the updated trigger rule of the refinement decision marker into a rule base so as to optimize the next compression task.
9. A data fusion compression device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the computer program being configured to implement the steps of the method of controlling data fusion compression according to any one of claims 1 to 8.
10. A storage medium, characterized in that the storage medium is a computer-readable storage medium, on which a computer program is stored, which computer program, when being executed by a processor, carries out the steps of the control method of data fusion compression according to any one of claims 1 to 8.

Description

Control method, device and storage medium for data fusion compression Technical Field The present application relates to the field of data processing technologies, and in particular, to a method, an apparatus, and a storage medium for controlling data fusion compression. Background In a mass data storage transmission scene, the hardware adaptation capability and the resource scheduling efficiency of file compression are directly related to the storage utilization rate and the data transmission speed. In the related art, by providing a fixed mode of pure central processing unit compression or pure graphics processing unit compression, data compression processing is completed by relying on calculation power of corresponding hardware, and the mode performs data processing according to a preset fixed processing path, so that resource scheduling conflict is easily caused, and further, the data compression efficiency is low. The foregoing is provided merely for the purpose of facilitating understanding of the technical solutions of the present application and is not intended to represent an admission that the foregoing is prior art. Disclosure of Invention The application mainly aims to provide a control method, equipment and storage medium for data fusion compression, and aims to solve the technical problem of low data compression efficiency. In order to achieve the above object, the present application provides a control method for data fusion compression, the method comprising: Responding to a data compression instruction, and determining candidate compression paths matched with inherent attribute characteristics of data to be compressed and calculation force state parameters of a current system; Determining benefit scores corresponding to the candidate compression paths by taking the inherent attribute characteristics and the calculation force state parameters as independent variables of decision functions, wherein the decision functions correspond to the candidate compression paths one by one; Selecting the candidate compression path with the highest profit score as a target execution path; And in the compression process, executing a resource scheduling action based on a resource scheduling rule corresponding to the target execution path, and executing a compression action based on the target execution path to obtain a compressed file. In one embodiment, in response to the data compression instruction, determining an initial path that meets attribute requirements and load requirements based on the inherent attribute characteristics and the computational power state parameters; Analyzing the block continuity characteristics of the data to be compressed, and determining the block partition adaptive parameters of the data to be compressed; And selecting the initial path matched with the blocking adaptive parameter as the candidate compression path. In an embodiment, analyzing the block continuity characteristics of each piece of segmented content in the data to be compressed to obtain a block-level characteristic sequence; calculating local information entropy of all block level features in the block level feature sequence, and arranging the local information entropy according to the sequence of the block level features to obtain a local entropy value sequence; Based on the local entropy value sequence, comparing entropy value differences of adjacent blocks, counting total duty ratio of blocks meeting continuous requirements, and calculating to obtain the blocking adaptive parameters of the data to be compressed. In an embodiment, normalizing the inherent attribute feature and the calculation force state parameter according to an input feature rule to generate a feature input parameter; Configuring gain weight parameters corresponding to the candidate compression paths according to a weight library for each candidate compression path; Inputting the characteristic input parameters into the decision function, and calculating data adaptation benefits, calculation load benefits and transmission cost benefits of the candidate compression paths by combining the benefit sub-item parameters of each candidate compression path; And carrying out weighted summation on the data adaptation benefits, the computational load benefits and the transmission cost benefits of the candidate compression paths according to the benefit weights to obtain the benefit scores corresponding to the candidate compression paths. In an embodiment, binding and pairing each candidate compression path with the corresponding profit sharing to obtain a candidate profit list; Correcting the transmission cost of the corresponding candidate path according to the refinement decision mark in the candidate revenue list to obtain a corrected candidate revenue list; sorting the corrected candidate profit list from high to low according to the profit score to obtain an ordered candidate sequence; and screening out the highest benefit score of the ordered cand