CN-115687352-B - Storage method and device

CN115687352BCN 115687352 BCN115687352 BCN 115687352BCN-115687352-B

Abstract

The embodiment of the invention provides a storage method and a storage device, which are applied to the technical field of big data and comprise the steps of obtaining a plurality of target query sentences, determining query sentence proportions corresponding to N service types respectively based on a matching relation between each target query sentence and the N service types, determining execution efficiency corresponding to M storage models respectively based on the query sentence proportions of the N service types and execution costs corresponding to the N service types when the N service types are stored under the M storage models respectively, and determining at least one target storage model for storing the service data of the N service types from the M storage models based on the execution efficiency corresponding to the M storage models respectively and occupied storage space corresponding to the M storage models respectively. According to the execution efficiency corresponding to each of the M storage models and the occupied storage space corresponding to each of the M storage models, the target storage model is stored in the limited storage space, and the reasonable storage space use is realized.

Inventors

ZHANG BEIBEI
LU KAI
LI YONGPAN
ZHA JUN

Assignees

中国银联股份有限公司

Dates

Publication Date: 20260512
Application Date: 20221103

Claims (11)

1. A method of storing, comprising: Acquiring a plurality of target query sentences; Determining the query statement proportion corresponding to each of N service types based on the matching relation between each target query statement and the N service types; Determining the execution efficiency corresponding to each of M storage models based on the query statement proportion of the N service types and the execution cost corresponding to each of the N service types when the N service types are stored under the M storage models, wherein N and M are positive integers; determining the execution efficiency corresponding to each of M storage models based on the query statement proportion of the N service types and the product of the execution cost corresponding to each of the N service types when the N service types are stored under the M storage models; And determining at least one target storage model for storing the service data of the N service types from the M storage models based on the execution efficiency corresponding to each of the M storage models and the occupied storage space corresponding to each of the M storage models.
2. The method of claim 1, wherein the determining at least one target storage model for storing the traffic data of the N traffic types from the M storage models based on the respective execution efficiencies of the M storage models and the respective occupied storage spaces of the M storage models comprises: determining the storage cost performance of each storage model based on the corresponding execution efficiency and occupied storage space of each storage model; and determining at least one target storage model for storing the service data of the N service types from the M storage models based on the storage cost performance corresponding to each of the M storage models.
3. The method of claim 2, wherein the determining at least one target storage model for storing the traffic data of the N traffic types from the M storage models based on the respective storage cost performance of the M storage models comprises: sorting the obtained multiple storage cost performance to obtain a target sorting result; Selecting target storage models from the M storage models in sequence according to the target sorting result, and storing the target storage models into a preset storage area, and stopping until the residual storage space of the preset storage area is smaller than the occupied storage space of the next added target storage model; and selecting at least one target storage model added to the preset storage area to store the service data of the N service types.
4. The method of claim 1, wherein the obtaining a plurality of target query statements comprises: Acquiring a plurality of historical query sentences; converting each historical query statement into a corresponding candidate query statement by adopting a target template; And screening a plurality of target query sentences from the plurality of candidate query sentences based on the access frequencies corresponding to the plurality of candidate query sentences.
5. The method of claim 4, wherein the screening the plurality of target query terms from the plurality of candidate query terms based on the respective access frequencies of the obtained plurality of candidate query terms comprises: Clustering the candidate query sentences once based on the access frequencies corresponding to the candidate query sentences to obtain a plurality of query sentence clusters; And screening a plurality of target query sentences from the plurality of query sentence clusters.
6. The method of claim 4, wherein the screening the plurality of target query terms from the plurality of candidate query terms based on the respective access frequencies of the obtained plurality of candidate query terms comprises: inputting each candidate query sentence into a classification model, and determining the access frequency type of each candidate query sentence, wherein a training sample of the classification model comprises a plurality of query sentence sets obtained based on historical access frequency clusters; and screening a plurality of target query sentences from the plurality of candidate query sentences based on the access frequency types of the plurality of candidate query sentences.
7. The method of claim 1, wherein the determining the query term proportions for each of the plurality of business types based on the matching relationship between each target query term and the plurality of business types comprises: determining a target service type of each target query statement from a plurality of service types according to the analysis tree characteristics of each target query statement; and determining the query statement proportion corresponding to each of the plurality of service types based on the target service type corresponding to each of the plurality of target query statements.
8. The method of claim 7, wherein determining the target business type for each target query statement from a plurality of business types based on the parse tree characteristics for each target query statement, comprises: And performing secondary clustering on the target query sentences based on the analysis tree characteristics of the target query sentences, and determining the target service type of each target query sentence from the service types based on a clustering result.
9. A storage device, comprising: an acquisition unit configured to acquire a plurality of target query sentences; The processing unit is used for determining the respective corresponding query statement proportion of the N service types based on the matching relation between each target query statement and the N service types, determining the respective corresponding execution efficiency of the M storage models based on the respective corresponding execution cost of the N service types when the N service types are stored under the M storage models, wherein N and M are positive integers, determining the respective corresponding execution efficiency of the M storage models based on the respective corresponding execution efficiency of the M storage models and the respective corresponding occupied storage space of the M storage models, and determining at least one target storage model for storing the service data of the N service types from the M storage models based on the respective corresponding execution efficiency of the M storage models and the respective corresponding occupied storage space of the M storage models.
10. An electronic device comprising at least one processor and at least one memory, wherein the memory stores a computer program that, when executed by the processor, causes the processor to perform the method of any of claims 1 to 8.
11. A computer-readable storage medium, characterized in that the storage medium stores a program which, when run on a computer, causes the computer to implement performing any one of claims 1 to 8.

Description

Storage method and device Technical Field The present invention relates to the field of big data technologies, and in particular, to a method and an apparatus for storing big data. Background With the continuous development of technology, more and more business data are generated for different businesses. When the business data are stored in different databases, the corresponding data models are also different, for example, the data sources are stored in the relational databases, the corresponding data models are two-dimensional table models, the data sources are stored in the key-value type databases, the corresponding data models are KV models, the data sources are stored in the document-oriented databases, and the corresponding data models are full-text retrieval models. Currently, business data is stored in each data model simultaneously, so that the business data can have the capability of processing multiple businesses simultaneously. But because storage resources are limited, there is no reasonable use of storage space. In summary, how to reasonably use the storage space is a technical problem to be solved currently. Disclosure of Invention The embodiment of the invention provides a storage method and a storage device, which are used for solving the problem that a storage space is not reasonably used in the prior art. In a first aspect, an embodiment of the present invention provides a storage method, including obtaining a plurality of target query statements, determining respective query statement proportions of N service types based on a matching relationship between each target query statement and the N service types, determining respective execution efficiencies of the M storage models based on the respective query statement proportions of the N service types and respective execution costs of the N service types when stored under the M storage models, where N and M are positive integers, and determining at least one target storage model for storing service data of the N service types from the M storage models based on the respective execution efficiencies of the M storage models and respective occupied storage spaces of the M storage models. According to the embodiment of the invention, the execution efficiency corresponding to each of M storage models is determined more accurately according to the target query statement, and the occupied storage space of the storage models is considered, so that the at least one target storage model for storing the service data of N service types can be determined more accurately from the M storage models according to the execution efficiency corresponding to each of the M storage models and the occupied storage space corresponding to each of the M storage models, and at least the target storage model is selected from the limited storage space, thereby realizing reasonable use of the storage space. Optionally, the determining at least one target storage model for storing the service data of the N service types from the M storage models based on the execution efficiency and the occupied storage space corresponding to each of the M storage models, and the occupation storage space corresponding to each of the M storage models includes determining a storage cost performance of each storage model based on the execution efficiency and the occupied storage space corresponding to each storage model, and determining at least one target storage model for storing the service data of the N service types from the M storage models based on the storage cost performance corresponding to each of the M storage models. In the embodiment of the invention, by determining the storage cost performance corresponding to each of the M storage models, at least one target storage model for storing the service data of the N service types can be accurately determined from the M storage models, so that the relationship between storage and performance is balanced in a limited storage space, and the reasonable use of the storage space is realized. Optionally, determining at least one target storage model for storing the service data of the N service types from the M storage models based on the storage cost performance corresponding to each of the M storage models includes sorting the obtained storage cost performance to obtain a target sorting result, sequentially selecting the target storage models from the M storage models according to the target sorting result, storing the target storage models in a preset storage area until the remaining storage space of the preset storage area is smaller than the occupied storage space of the next added target storage model, and selecting at least one target storage model added to the preset storage area to store the service data of the N service types. In the embodiment of the invention, since the storage space of the preset storage area is limited, the obtained plurality of storage cost performance needs to be sequenced, and then the target storage model