CN-121998530-A - Intelligent logistics cloud platform data management system and method

CN121998530ACN 121998530 ACN121998530 ACN 121998530ACN-121998530-A

Abstract

The invention relates to the field of data management, in particular to a data management system and method for an intelligent logistics cloud platform. Firstly, preprocessing the acquired original data and bound time stamps and position coordinates to obtain standardized data, introducing a space-time grid exclusive-or disturbance coding algorithm based on the standardized data to obtain space-time characteristic values, then introducing a linear consistency hash variant algorithm based on space-time disturbance based on the standardized data and the space-time characteristic values to obtain a main hash value and an auxiliary virtual hash value, and copying and storing the standardized data to corresponding physical data storage nodes. The technical problems of unbalanced data load, overload and performance bottleneck of hot nodes, single-point fault risk and insufficient fault tolerance are solved.

Inventors

CHEN ZHANGJIE
LIU YIFENG
HUANG WEN
YANG WEIYUE

Assignees

广州一链通互联网科技有限公司

Dates

Publication Date: 20260508
Application Date: 20260128

Claims (9)

1. The intelligent logistics cloud platform data management method is characterized by comprising the following steps of: S1, preprocessing the collected original data, the bound time stamp and the bound position coordinate to obtain standardized data, and introducing a space-time grid exclusive-or disturbance coding algorithm to obtain a space-time characteristic value based on the standardized data; S2, based on the standardized data and the space-time characteristic values, introducing a linear consistency hash variant algorithm based on space-time disturbance to obtain a main hash value and an auxiliary virtual hash value, and copying and storing the standardized data to a corresponding physical data storage node.
2. The intelligent logistics cloud platform data management method of claim 1, wherein the step S1 specifically comprises: And carrying out outlier detection and data standardization based on the acquired original data, the bound time stamp and the bound position coordinates to obtain standardized data.
3. The intelligent logistics cloud platform data management method of claim 2, wherein the step S1 specifically comprises: In the space-time grid exclusive or disturbance coding algorithm, the time attribute of the standardized data is subjected to periodic discretization, and continuous time stamps are mapped into a preset time barrel dividing period through modulo operation, so that the time barrel numbers are obtained.
4. The intelligent logistics cloud platform data management method of claim 2, wherein the step S1 specifically comprises: and carrying out two-dimensional grid division processing on the spatial attribute of the standardized data to obtain a one-dimensional spatial code.
5. The intelligent logistics cloud platform data management method of claim 4, wherein the step S1 specifically comprises: and performing bit exclusive OR operation on the time bucket number and the one-dimensional space code to obtain a space-time characteristic value.
6. The intelligent logistics cloud platform data management method of claim 1, wherein the step S2 specifically comprises: In a linear consistent hash variant algorithm based on space-time disturbance, a main hash value is calculated by adopting a linear congruence form based on standardized data.
7. The intelligent logistics cloud platform data management method of claim 6, wherein the step S2 specifically comprises: and obtaining an auxiliary virtual hash value based on the disturbance offset and the main hash value.
8. The intelligent logistics cloud platform data management method of claim 7, wherein the step S2 specifically comprises: For each piece of standardized data, mapping the main hash value and the auxiliary virtual hash value to different virtual node positions on the consistent hash ring respectively, fixing a corresponding physical data storage node by each virtual node, and copying and storing the standardized data to the physical data storage nodes corresponding to the main hash value and the auxiliary virtual hash value.
9. The intelligent logistics cloud platform data management system applied to the intelligent logistics cloud platform data management method disclosed in claim 1 is characterized by comprising the following parts: the system comprises a data acquisition module, a data preprocessing module, a space-time feature extraction module and a space-time disturbance consistency hash storage module; The data acquisition module is used for deploying sensor equipment and acquiring original data, binding a time stamp accurate to a second or millisecond level and position coordinates in a longitude and latitude form of each piece of original data, and outputting the acquired original data together with the bound time stamp and position coordinates to the data preprocessing module; the data preprocessing module is used for preprocessing the acquired original data, the bound time stamp and the bound position coordinate to generate standardized data, and outputting the standardized data to the space-time feature extraction module and the space-time disturbance consistency hash storage module; The space-time characteristic extraction module is used for generating a space-time characteristic value by introducing a space-time grid exclusive-or disturbance coding algorithm based on standardized data and outputting the space-time characteristic value to the space-time disturbance consistency hash storage module; and the space-time disturbance consistency hash storage module is used for introducing a linear consistency hash variant algorithm based on space-time disturbance based on the standardized data and the space-time characteristic value, generating a main hash value and an auxiliary virtual hash value, mapping the main hash value and the auxiliary virtual hash value to different virtual node positions on a consistency hash ring respectively, fixedly corresponding each virtual node to one physical data storage node, and copying and storing the standardized data to the corresponding physical data storage node.

Description

Intelligent logistics cloud platform data management system and method Technical Field The invention relates to the field of data management, in particular to a data management system and method for an intelligent logistics cloud platform. Background With the rapid development of globalization of electronic commerce and supply chains, modern logistics industry is facing unprecedented explosive growth of data, and during peak or hot business hours, data often appears to be generated centrally in bursts, resulting in serious challenges in storage and querying of traditional data management systems. Therefore, the data management system of the intelligent logistics cloud platform has important theoretical value and remarkable industrial application prospect, and is suitable for various scenes such as a modern intelligent logistics cloud platform, an urban distribution platform, a supply chain collaboration platform, a cold chain logistics monitoring platform and the like. However, the data management system of the existing intelligent logistics cloud platform mostly adopts a centralized or simple distributed storage architecture, such as a distributed system based on a traditional relational database or basic hash map. However, the space-time clustering characteristic of the logistics data makes it easy for data with similar space-time attributes to be mapped to the same or adjacent nodes when the consistent hash algorithm is directly applied, so that overload of hot nodes, unbalanced load, increased query delay and even single-point fault risk are caused. Meanwhile, the requirement of multi-copy redundancy of mass data further amplifies the problems, and the traditional method is difficult to realize effective data dispersion and fault-tolerant protection under the condition of not introducing a complex auxiliary structure. In addition, existing data preprocessing and storage strategies tend to ignore the periodicity and proximity of the spatio-temporal properties, resulting in a non-uniform spatial distribution of hashes, which makes it difficult to accommodate dynamic fluctuations in logistics traffic, such as daily peak hours or regional bursts. In addition, in the traditional distributed storage system, although the consistent hash algorithm supports dynamic node expansion, an active disturbance mechanism aiming at space-time hot spot data is insufficient, and fine load balancing and efficient multi-copy placement are difficult to achieve. Therefore, it is desirable to provide a system and a method for managing data of an intelligent logistics cloud platform to solve the above-mentioned problems. Disclosure of Invention The invention provides an intelligent logistics cloud platform data management system and method, which are used for solving the problems that when a traditional method is used for processing massive data with strong space-time clustering characteristics, serious load imbalance phenomenon is easy to occur, partial nodes are continuously overloaded, the utilization rate of other nodes is low, the utilization efficiency of the whole system resources is low, high concurrency and sudden hot spot data generated by a logistics full link are difficult to effectively disperse, the overload of the hot spot nodes is easy to be caused, the inquiry response delay is obviously increased, even single-point fault risks occur, the stability and the service quality of the system are influenced, and the traditional method is difficult to simultaneously consider the high throughput, the low delay and the fault tolerance capability, and particularly, the technical problems that the data migration cost is high and the system usability is difficult to guarantee under the node dynamic expansion or fault scene are solved. The invention relates to a data management system and method of an intelligent logistics cloud platform, which specifically comprise the following technical scheme: a data management method of an intelligent logistics cloud platform comprises the following steps: S1, preprocessing the collected original data, the bound time stamp and the bound position coordinate to obtain standardized data, and introducing a space-time grid exclusive-or disturbance coding algorithm to obtain a space-time characteristic value based on the standardized data; S2, based on the standardized data and the space-time characteristic values, introducing a linear consistency hash variant algorithm based on space-time disturbance to obtain a main hash value and an auxiliary virtual hash value, and copying and storing the standardized data to a corresponding physical data storage node. Preferably, the S1 specifically includes: And carrying out outlier detection and data standardization based on the acquired original data, the bound time stamp and the bound position coordinates to obtain standardized data. Preferably, the S1 specifically includes: In the space-time grid exclusive or disturbance coding algorithm, the time att