CN-122001382-A - Data compression method, device, electronic equipment and storage medium

CN122001382ACN 122001382 ACN122001382 ACN 122001382ACN-122001382-A

Abstract

The embodiment of the application provides a data compression method, a data compression device, electronic equipment and a storage medium. The data compression method comprises the steps of decomposing a feature matrix to obtain a first feature dimension matrix and a related feature matrix, wherein the number of feature values in the first feature dimension matrix is larger than that of the related feature matrix, and the space occupied by the first feature dimension matrix is smaller than that of the related feature matrix. The method comprises the steps of mapping the characteristic values in the first characteristic dimension matrix to a compression space to obtain a compression matrix, reducing the occupied space of the first characteristic dimension matrix to be larger, avoiding the problem of precision reduction caused by directly compressing the characteristic matrix, reducing the occupied space of the related characteristic matrix and the compression matrix compared with the characteristic matrix, ensuring that the related characteristic matrix is not compressed, and improving the accuracy of data recovery based on the related characteristic matrix and the compression matrix.

Inventors

WANG NAN
YANG XIN
XING JUNNA
ZHOU PENG

Assignees

阿里云计算有限公司

Dates

Publication Date: 20260508
Application Date: 20241105

Claims (11)

1. A method of data compression, the method comprising: responding to the acquired feature matrix of the data to be stored, decomposing the feature matrix to obtain a first feature dimension matrix and a related feature matrix of the first feature dimension matrix, wherein the number of feature values in the first feature dimension matrix is larger than that of the related feature matrix; compressing each characteristic value of the first characteristic dimension matrix according to a compression parameter to obtain a compression matrix, wherein the number of bits of any characteristic value in the compression matrix is smaller than the number of bits of the corresponding characteristic value in the first characteristic dimension matrix; and correspondingly storing the relevant characteristic matrix and the compression matrix.
2. The method of claim 1, wherein the correlation feature matrix comprises a feature vector matrix and a second feature dimension matrix, and wherein the decomposing the feature matrix to obtain a first feature dimension matrix and a correlation feature matrix of the first feature dimension matrix comprises: multiplying the transpose matrix of the feature matrix by the feature matrix to obtain a first correlation matrix; multiplying the feature matrix by a transpose matrix of the feature matrix to obtain a second correlation matrix; Performing feature decomposition on the first correlation matrix to obtain a first feature value and a first feature vector, and performing feature decomposition on the second correlation matrix to obtain a second feature value and a second feature vector; constructing the second feature dimension matrix according to the first feature vector, and calculating the square root of the first feature value and the square root of the second feature value to obtain the feature vector matrix; and constructing the first characteristic dimension matrix according to the second characteristic vector.
3. The method of claim 1, wherein the compression parameters include a compression ratio and a reference value, and compressing the respective eigenvalues of the first eigenvalue matrix by the compression parameters to obtain a compression matrix comprises: Calculating the difference value between any one of the characteristic values in the first characteristic dimension matrix and the reference value; taking the ratio of the difference value to the compression ratio as a numerical value obtained by reducing the number of bits of the characteristic value; And mapping each numerical value of the first characteristic dimension matrix after the number of reduced digits into the corresponding position in the compression matrix one by one to obtain the compression matrix.
4. A method according to claim 3, further comprising, prior to said compressing the respective eigenvalues of said first eigenvalue matrix according to compression parameters to obtain a compressed matrix: Calculating to obtain a second maximum characteristic value in the compression matrix according to the pre-acquired compression bit number and a first maximum characteristic value in the first characteristic dimension matrix, and calculating to obtain a second minimum characteristic value in the compression matrix according to the pre-acquired compression bit number and a first minimum characteristic value in the first characteristic dimension matrix; and calculating the compression ratio and the reference value of the compression matrix based on the first maximum characteristic value and the second maximum characteristic value and the first minimum characteristic value and the second minimum characteristic value.
5. The method of claim 4, wherein the calculating the compression ratio and the reference value of the compression matrix based on the first maximum eigenvalue, the second maximum eigenvalue, and the first minimum eigenvalue, the second minimum eigenvalue, comprises: Calculating to obtain a first compression range according to the difference value between the first maximum characteristic value and the first minimum characteristic value, and calculating to obtain a second compression range according to the difference value between the second maximum characteristic value and the second minimum characteristic value; Taking the ratio of the first compression range to the second compression range as the compression ratio; And taking the ratio of a target difference value to the compression ratio as the reference value, wherein the target difference value is the difference value between the first minimum characteristic value and the second minimum characteristic value.
6. The method according to claim 1, wherein the method further comprises: responding to a data recovery instruction, and extracting the relevant feature matrix and the compression matrix; Filling the bit number of each characteristic value in the compression matrix by utilizing the compression parameters to obtain a filling characteristic matrix related to the first characteristic dimension matrix; And fusing the relevant feature matrix and the filling feature matrix to obtain a recovery feature matrix of the data to be stored.
7. The method of claim 6, wherein the compression parameters include a compression ratio and a reference value, wherein the filling the number of bits of each eigenvalue in the compression matrix with the compression parameters results in a filled eigenvalue matrix related to the first eigenvalue matrix, comprising: calculating the sum of any characteristic value in the compression matrix and the reference value to obtain an initial value corresponding to the characteristic value; Filling the digits of the initial value according to the compression ratio to obtain a filling numerical value corresponding to the characteristic value; And mapping the calculated filling numerical value into the corresponding position in the filling characteristic matrix by taking the position of the characteristic value in the compression matrix as a reference to obtain the filling characteristic matrix.
8. A data compression apparatus, the apparatus comprising: The system comprises a decomposition module, a first feature dimension matrix, a second feature dimension matrix, a first feature dimension matrix and a second feature dimension matrix, wherein the decomposition module is used for decomposing the feature matrix to obtain a first feature dimension matrix and a related feature matrix of the first feature dimension matrix in response to the acquired feature matrix of the data to be stored; The compression module is used for compressing each characteristic value of the first characteristic dimension matrix according to preset compression parameters to obtain a compression matrix, and the number of bits of any characteristic value in the compression matrix is smaller than that of the corresponding characteristic value in the first characteristic dimension matrix; and the storage module is used for correspondingly storing the relevant characteristic matrix and the compression matrix.
9. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the computer program is executed by the processor to cause the electronic device to perform the method of any one of claims 1-7.
10. A computer readable storage medium having stored thereon a computer program, wherein the program is executed by a processor to implement the method of any of claims 1-7.
11. A computer program product comprising instructions which, when run on a computer, cause the computer to perform the method of any of claims 1-7.

Description

Data compression method, device, electronic equipment and storage medium Technical Field The embodiment of the application relates to the field of data processing, in particular to a data compression method, a data compression device, electronic equipment and a storage medium. Background A large language model (Large Language Models, LLMs) is an artificial intelligence model trained with large numbers of input text, intended to understand and generate natural language text. And when each input text input by the user is received, the large language model generates corresponding output according to the input text, and if each input text is processed, the processing time is long and the waste of calculation resources is caused. Therefore, in order to reduce the related information of repeatedly calculating similar input texts, a Key Value Cache (KV Cache) is generally used to store the K Value and the V Value of the input text in the large language model, and when receiving the input text corresponding to the similar K Value input by the user again, the corresponding output can be obtained according to the V Value, thereby improving the reasoning efficiency. Under the condition that the data volume of the input text is large, large space is occupied for storing the K value and the V value, and in order to reduce the occupation of the K value and the V value of the KV-Cache on the storage space, the K value and the V value of the KV-Cache are generally compressed. However, the existing compression method reduces the occupation of storage space, but reduces the accuracy of data, thereby reducing the accuracy of generating natural language text by a large language model. Disclosure of Invention In order to overcome the problems in the related art, the embodiment of the application provides a data compression method, a data compression device, electronic equipment and a storage medium. According to a first aspect of an embodiment of the present application, there is provided a data compression method, the method including: responding to the acquired feature matrix of the data to be stored, decomposing the feature matrix to obtain a first feature dimension matrix and a related feature matrix of the first feature dimension matrix, wherein the number of feature values in the first feature dimension matrix is larger than that of the related feature matrix; Compressing each characteristic value of the first characteristic dimension matrix according to a preset compression parameter to obtain a compression matrix, wherein the number of bits of any characteristic value in the compression matrix is smaller than that of the corresponding characteristic value in the first characteristic dimension matrix; and correspondingly storing the relevant characteristic matrix and the compression matrix. According to a second aspect of an embodiment of the present application, there is provided a data compression apparatus, the apparatus comprising: The system comprises a decomposition module, a first feature dimension matrix, a second feature dimension matrix, a first feature dimension matrix and a second feature dimension matrix, wherein the decomposition module is used for decomposing the feature matrix to obtain a first feature dimension matrix and a related feature matrix of the first feature dimension matrix in response to the acquired feature matrix of the data to be stored; The compression module is used for compressing each characteristic value of the first characteristic dimension matrix according to preset compression parameters to obtain a compression matrix, and the number of bits of any characteristic value in the compression matrix is smaller than that of the corresponding characteristic value in the first characteristic dimension matrix; and the storage module is used for correspondingly storing the relevant characteristic matrix and the compression matrix. According to a third aspect of embodiments of the present application, there is provided an electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, the computer program being executable by the processor to cause the electronic device to perform the method as described in the first aspect. According to a fourth aspect of embodiments of the present application, there is provided a computer readable storage medium having stored thereon a computer program for execution by a processor to perform the method according to the first aspect. According to a fifth aspect of embodiments of the present application, there is provided a computer program product comprising instructions which, when run on a computer, cause the computer to perform the method according to the first aspect. The technical scheme provided by the embodiment of the application can have the following beneficial effects: In the data compression method in the embodiment of the application, after the feature matrix of the data to be stored