CN-121996311-A - Data loading method and device, electronic equipment and medium

CN121996311ACN 121996311 ACN121996311 ACN 121996311ACN-121996311-A

Abstract

The embodiment of the application provides a data loading method, a data loading device, electronic equipment and a medium, and belongs to the field of data processing. The data loading method comprises the steps of obtaining a data identifier and a data type of data to be loaded when the electronic equipment performs cache loading, obtaining the data to be loaded from different databases according to the data type and the data identifier, and storing the obtained data to be loaded into a local cache of the electronic equipment. Therefore, the data to be loaded can be quickly loaded to each application interface in the electronic equipment, and the loading efficiency of the data during loading is improved.

Inventors

WANG JIANBIN
TAN XIAOHAN
ZHANG YAO
LI WENZHI

Assignees

顺丰科技有限公司

Dates

Publication Date: 20260508
Application Date: 20241105

Claims (10)

1. A data loading method, applied to an electronic device, the method comprising: Under the condition that the electronic equipment carries out cache loading, acquiring a data identifier and a data type of data to be loaded, wherein the data type is used for determining that the data to be loaded is hot spot data or pre-loaded data; if the data to be loaded is hot spot data, acquiring the data to be loaded from a first database according to the data identifier of the data to be loaded; if the data to be loaded is preloaded data, acquiring the data to be loaded from a second database according to a data identifier of the data to be loaded, wherein the use frequency of the hot spot data is greater than a preset threshold value and less than that of the preloaded data, the data volume of the hot spot data is greater than that of the preloaded data, the second database is a backup database of the first database, and the read-write speed of the first database is greater than that of the second database; and storing the data to be loaded into a local cache of the electronic equipment.
2. The method according to claim 1, wherein the method further comprises: acquiring incremental data, wherein the incremental data comprises at least one of newly added data and changed data; Storing the incremental data into the first database and the second database; And adding the data identification of the incremental data into a cache configuration table, wherein the cache configuration table is used for recording data information of the data which needs to be loaded into the local cache of the electronic equipment.
3. The method of claim 2, wherein the storing the delta data into the first database and the second database comprises: Acquiring a data identifier of the incremental data; Determining a first storage position of the incremental data in the first database according to the data identification of the incremental data, and storing the incremental data in the first database according to the first storage position of the incremental data; and determining a second storage position of the incremental data in the second database according to the data identification of the incremental data, and storing the incremental data in the second database according to the second storage position of the incremental data.
4. The method of claim 3, wherein the determining a first storage location of the delta data in the first database based on the data identification of the delta data and storing the delta data in the first database based on the first storage location comprises: according to the data identification of the incremental data and a preset first calculation rule, calculating to obtain a first hash value of the incremental data; Determining a third storage position of the incremental data according to the first hash value of the incremental data; Determining a first storage position of the incremental data stored in the first database according to a third storage position of the incremental data; And storing the data identification of the incremental data in a third storage location of the incremental data, and storing the incremental data in a first storage location of the incremental data.
5. The method of claim 4, wherein if the data to be loaded is hot spot data, obtaining the data to be loaded from the first database according to the data identifier of the data to be loaded comprises: If the data to be loaded is hot spot data, calculating to obtain a first hash value of the data to be loaded according to the data identification of the data to be loaded and the first calculation rule; Determining a third storage position of the data to be loaded according to the first hash value of the data to be loaded; determining a first storage position of the data to be loaded according to the third storage position of the data to be loaded; And acquiring the data to be loaded from the first database according to the first storage position of the data to be loaded.
6. A method according to claim 3, wherein said determining a second storage location of said delta data in said second database based on a data identification of said delta data, and storing said delta data in said second database based on said second storage location, comprises: Matching the data identification of the incremental data with the data identification in a preloading cache data table, and determining a second storage position of the incremental data stored in a cloud database, wherein the preloading cache data table comprises a plurality of data identifications and storage positions corresponding to the data identifications; and storing the incremental data in the second storage location.
7. The method according to claim 1, wherein the obtaining, in the case of cache loading by the electronic device, a data identifier and a data type of data to be loaded includes: And under the condition that the electronic equipment carries out cache loading, determining the data identification and the data type of the data to be loaded according to the cache configuration table.
8. A data loading apparatus for application to an electronic device, the apparatus comprising: The data acquisition module is used for acquiring a data identifier and a data type of data to be loaded under the condition that the electronic equipment carries out cache loading, and the data type is used for determining that the data to be loaded is hot spot data or pre-loaded data; The first data module is used for acquiring the data to be loaded from a first database according to the data identifier of the data to be loaded if the data to be loaded is hot spot data; The second data module is used for acquiring the data to be loaded from a second database according to the data identifier of the data to be loaded if the data to be loaded is the pre-loaded data, wherein the use frequency of the hot spot data is greater than a preset threshold value and less than the use frequency of the pre-loaded data, the data volume of the hot spot data is greater than the data volume of the pre-loaded data, and the second database is a backup database of the first database; and the data storage module is used for storing the data to be loaded into a local cache of the electronic equipment.
9. An electronic device comprising a memory storing a computer program and a processor implementing the data loading method of any of claims 1 to 7 when the computer program is executed by the processor.
10. A computer readable storage medium storing a computer program, characterized in that the computer program, when executed by a processor, implements the data loading method of any one of claims 1 to 7.

Description

Data loading method and device, electronic equipment and medium Technical Field The present application relates to the field of data processing, and in particular, to a data loading method, apparatus, electronic device, and medium. Background With the rapid development of technologies such as the internet, the internet of things and mobile equipment, the generation speed and the scale of data have reached unprecedented levels. In the prior art, in order to cope with the rapid increase of service data and the improvement of interface performance requirements, a cache technology is generally adopted to improve the system performance, however, when processing data with hundreds of millions of scales, the existing cache technology has the problem of low loading efficiency when loading large-scale data. Disclosure of Invention The embodiment of the application mainly aims to provide a data loading method, a data loading device, electronic equipment and a medium, and aims to solve the problem that the loading efficiency is low when data is loaded due to the defect of a data caching technology. To achieve the above object, a first aspect of an embodiment of the present application provides a data loading method, including: Under the condition that the electronic equipment carries out cache loading, acquiring a data identifier and a data type of data to be loaded, wherein the data type is used for determining that the data to be loaded is hot spot data or pre-loaded data; if the data to be loaded is hot spot data, acquiring the data to be loaded from a first database according to the data identifier of the data to be loaded; if the data to be loaded is preloaded data, acquiring the data to be loaded from a second database according to a data identifier of the data to be loaded, wherein the use frequency of the hot spot data is greater than a preset threshold value and less than that of the preloaded data, the data volume of the hot spot data is greater than that of the preloaded data, the second database is a backup database of the first database, and the read-write speed of the first database is greater than that of the second database; and storing the data to be loaded into a local cache of the electronic equipment. In some embodiments, the method further comprises: acquiring incremental data, wherein the incremental data comprises at least one of newly added data and changed data; Storing the incremental data into the first database and the second database; And adding the data identification of the incremental data into a cache configuration table, wherein the cache configuration table is used for recording data information of the data which needs to be loaded into the local cache of the electronic equipment. In some embodiments, the storing the delta data into the first database and the second database comprises: Acquiring a data identifier of the incremental data; Determining a first storage position of the incremental data in the first database according to the data identification of the incremental data, and storing the incremental data in the first database according to the first storage position of the incremental data; and determining a second storage position of the incremental data in the second database according to the data identification of the incremental data, and storing the incremental data in the second database according to the second storage position of the incremental data. In some embodiments, the determining a first storage location of the incremental data in the first database according to the data identification of the incremental data, and storing the incremental data in the first database according to the first storage location includes: according to the data identification of the incremental data and a preset first calculation rule, calculating to obtain a first hash value of the incremental data; Determining a third storage position of the incremental data according to the first hash value of the incremental data; Determining a first storage position of the incremental data stored in the first database according to a third storage position of the incremental data; And storing the data identification of the incremental data in a third storage location of the incremental data, and storing the incremental data in a first storage location of the incremental data. In some embodiments, if the data to be loaded is hot spot data, the obtaining the data to be loaded from the first database according to the data identifier of the data to be loaded includes: If the data to be loaded is hot spot data, calculating to obtain a first hash value of the data to be loaded according to the data identification of the data to be loaded and the first calculation rule; Determining a third storage position of the data to be loaded according to the first hash value of the data to be loaded; determining a first storage position of the data to be loaded according to the third storage position of the data to be