CN-121979459-A - Method and device for dynamically adjusting fragments

CN121979459ACN 121979459 ACN121979459 ACN 121979459ACN-121979459-A

Abstract

The application relates to a method and a device for dynamically adjusting fragments, wherein the method comprises the steps of obtaining a configuration data fragment set corresponding to an area node, wherein the configuration data fragment set comprises at least one configuration data fragment, determining configuration data stored in the configuration data fragment aiming at any configuration data fragment in the configuration data fragment set, searching service priority of the configuration data, and adjusting the configuration data fragment according to the query rate and the service priority of the configuration data fragment per second. By acquiring the configuration data slicing set of the regional node and combining the query rate per second of each configuration data slicing with the service priority of the corresponding configuration data, the slicing granularity is dynamically adjusted, the dynamic change of the data access heat can be accurately adapted, the problem of loading overload of the slicing nodes caused by centralized access of hot spot data is effectively avoided, the hot spot data processing delay is remarkably reduced, and the response efficiency of the system is improved.

Inventors

WU XIAOYANG
CEN DAWEI
LIANG YUYI
HUANG BORONG
LI XIAOYUN

Assignees

珠海金山办公软件有限公司
北京金山办公软件股份有限公司
武汉金山办公软件有限公司

Dates

Publication Date: 20260505
Application Date: 20260210

Claims (14)

1. A method for dynamically adjusting a slice, which is applied to any regional node in a distributed system, the method comprising: acquiring a configuration data slicing set corresponding to the area node, wherein the configuration data slicing set comprises at least one configuration data slicing; determining configuration data stored in the configuration data fragments aiming at any configuration data fragment in the configuration data fragment set, and searching service priority of the configuration data; And adjusting the configuration data fragments according to the query rate per second of the configuration data fragments and the service priority.
2. The method of claim 1, wherein said adjusting said configuration data slices according to said traffic priority and a query rate per second of said configuration data slices comprises: And determining the target slicing granularity according to the query rate per second of the configuration data slicing and the service priority, and adjusting the configuration data slicing according to the target slicing granularity.
3. The method of claim 2, wherein the determining the target fragmentation granularity based on the query rate per second of the configuration data fragments and the traffic priority comprises: Acquiring the granularity of the basic fragments and the resource adaptation coefficient of the distributed system; Determining a dynamic demand weight factor according to the query rate per second of the configuration data fragments and the service priority; And determining the target fragment granularity according to the basic fragment granularity, the dynamic demand weight factor and the distributed system resource adaptation coefficient.
4. The method of claim 3, wherein said determining a dynamic demand weight factor based on the query rate per second of the configuration data slices and the traffic priority comprises: determining a preset service priority coefficient corresponding to the service priority, and determining the logarithm of the query rate per second of the configuration data fragments; and adding the preset service priority coefficient and the logarithm of the query rate per second of the configuration data fragments to obtain a dynamic demand weight factor.
5. The method of claim 1, wherein for any one of the set of configuration data slices, the following processing is further performed: judging whether the query rate per second is greater than a preset hot spot threshold value or not and whether the duration time of the query rate per second greater than the preset hot spot threshold value reaches a preset first time length or not; Under the condition that the query rate per second is larger than a preset hot spot threshold value and the duration time of the query rate per second larger than the preset hot spot threshold value reaches a preset first time length, horizontally splitting the configuration data fragments to obtain a plurality of sub configuration data fragments; And establishing a mapping relation between the configuration data fragments and a plurality of sub-configuration data fragments.
6. The method of claim 1, wherein for any one of the set of configuration data slices, the following processing is further performed: Judging whether the query rate per second is smaller than a preset cold data threshold value or not and whether the duration time of the query rate per second smaller than the preset cold data threshold value reaches a preset second duration time or not; Vertically merging the configuration data fragments under the condition that the query rate per second is smaller than a preset cold data threshold value and the duration time of the query rate per second smaller than the preset cold data threshold value reaches a preset second duration time, so as to obtain merged configuration data fragments; and establishing a mapping relation between the configuration data fragments and the combined configuration data fragments.
7. The method according to claim 1, wherein the method further comprises: After the configuration data is monitored to be changed, determining data content hash, operation track hash and space-time hash of the changed configuration data; And generating the holographic fingerprint of the changed configuration data according to the data content hash, the operation track hash and the space-time hash.
8. The method of claim 7, wherein the determining the data content hash, the operation trace hash, and the spatiotemporal hash of the altered configuration data comprises: Carrying out hash processing on the changed configuration data to obtain data content hash; determining a data change operation sequence corresponding to the changed configuration data, and carrying out hash processing on the data change operation sequence to obtain an operation track hash; Acquiring a fragmentation identification of the configuration data fragmentation, a region identification of the region node and a current time stamp; and carrying out hash processing on the fragment identifier, the region identifier and the current timestamp to obtain space-time hash.
9. The method according to claim 7 or 8, characterized in that the method further comprises: receiving a request for the changed configuration data sent by a client, and responding to the request to acquire the holographic fingerprint and the latest N historical holographic fingerprints, wherein N is a positive integer; returning the holographic fingerprint and the last N times of historical holographic fingerprints to the client so that the client can execute the following steps: checking space-time rationality and continuity of the holographic fingerprint and the last N times of historical holographic fingerprints; and loading the changed configuration data under the condition that space-time rationality and continuity check of the holographic fingerprint and the latest N times of historical holographic fingerprints pass.
10. The method according to claim 1, wherein the method further comprises: after the configuration data is monitored to be changed, sending a configuration data change message to a central area node, so that the central area node executes the following steps: Receiving the configuration data change message and broadcasting the configuration data change message to other regional nodes in the distributed system; The voting results of the other regional nodes are obtained, wherein the other regional nodes vote in response to the configuration data change message, and the voting results are generated; and synchronizing the configuration data change message to other regional nodes according to the voting result.
11. The method of claim 10, wherein synchronizing the configuration data change message to the remaining area nodes based on the voting results comprises: under the condition that the voting result represents the consent result, adding voting weights of nodes in other areas to obtain the consent weight; And under the condition that the agreeing weight exceeds the preset weight, synchronizing the configuration data change message to the other area nodes.
12. The method according to claim 1, wherein the method further comprises: acquiring a special event mark, a service period label and a historical access sequence of the configuration data; Predicting the access probability and the estimated query rate per second of the configuration data in a future preset third duration according to the special event mark, the service period label and the historical access sequence; And determining a preheating strategy of the configuration data according to the access probability and the estimated query rate per second, and preheating the configuration data according to the preheating strategy.
13. The method of claim 12, wherein determining the warm-up policy for the configuration data based on the access probability and the estimated query rate per second comprises: Under the condition that the access probability is in a first probability interval and the estimated query rate per second is in a first query rate interval, determining the warming-up to local cache as a warming-up strategy of the configuration data; determining a preheating strategy for preheating the to-region shared cache as the configuration data under the condition that the access probability is in a second probability interval and the estimated query rate per second is in a second query rate interval; And determining the preheating metadata as the preheating strategy of the configuration data under the condition that the access probability is in a third probability interval and the estimated query rate per second is in a third query rate interval.
14. A dynamic adjustment device for slicing, which is applied to any regional node in a distributed system, the device comprising: The system comprises a segmented set acquisition module, a segmentation module and a segmentation module, wherein the segmented set acquisition module is used for acquiring a configuration data segmented set corresponding to the regional node, and the configuration data segmented set comprises at least one configuration data segmented; the priority determining module is used for determining the configuration data stored in the configuration data fragments aiming at any configuration data fragment in the configuration data fragment set and searching the service priority of the configuration data; And the dynamic adjustment module for the fragments is used for adjusting the configuration data fragments according to the query rate per second of the configuration data fragments and the service priority.

Description

Method and device for dynamically adjusting fragments Technical Field The present application relates to the field of computer processing technologies, and in particular, to a method and an apparatus for dynamically adjusting a slice. Background With the popularization of distributed technology, large-scale distributed systems involving multi-region deployment are widely applied to the key fields of finance, energy sources and the like, and the real-time synchronization, safety and controllability and continuous availability of the system on configuration data have extremely high dependence, so that the efficient partition management of the configuration data becomes a core link for guaranteeing the overall performance of the system. Currently, the mainstream data slicing scheme in the industry mainly adopts a static hash slicing mode or a slicing mode divided according to regions, and both schemes show the characteristic of slicing granularity solidification, namely the slicing rule is fixed after the system deployment and cannot be dynamically adjusted according to the actual running state. The solidification slicing mechanism has the remarkable limitation that because the solidification slicing mechanism cannot adapt to the dynamic change of the data access heat, when a hot spot data centralized access scene appears, the loading overload of the corresponding slicing nodes is easy to cause, the problems of hot spot data processing delay, system response efficiency reduction and the like are caused, and the requirements of a large-scale distributed system on the high real-time performance and the high availability of configuration data are difficult to meet. Disclosure of Invention In order to solve the problems that the load overload of corresponding slicing nodes is easy to cause, the processing delay of hot spot data, the response efficiency of a system is reduced and the like due to the fact that the dynamic change of data access heat cannot be adapted, when a hot spot data centralized access scene occurs, the problems that the requirements of a large-scale distributed system on the high real-time performance and the high availability of configuration data are difficult to meet are solved, the embodiment of the application provides a method and a device for dynamically adjusting the slicing, electronic equipment and a storage medium. The specific technical scheme is as follows: In a first aspect of the embodiment of the present application, there is first provided a method for dynamically adjusting a slice, which is applied to any area node in a distributed system, where the method includes: acquiring a configuration data slicing set corresponding to the area node, wherein the configuration data slicing set comprises at least one configuration data slicing; determining configuration data stored in the configuration data fragments aiming at any configuration data fragment in the configuration data fragment set, and searching service priority of the configuration data; And adjusting the configuration data fragments according to the query rate per second of the configuration data fragments and the service priority. In an optional embodiment, the adjusting the configuration data slice according to the query rate per second of the configuration data slice and the service priority includes: And determining the target slicing granularity according to the query rate per second of the configuration data slicing and the service priority, and adjusting the configuration data slicing according to the target slicing granularity. In an optional embodiment, the determining the target fragmentation granularity according to the query rate per second of the configuration data fragmentation and the service priority includes: Acquiring the granularity of the basic fragments and the resource adaptation coefficient of the distributed system; Determining a dynamic demand weight factor according to the query rate per second of the configuration data fragments and the service priority; And determining the target fragment granularity according to the basic fragment granularity, the dynamic demand weight factor and the distributed system resource adaptation coefficient. In an alternative embodiment, the determining the dynamic demand weight factor according to the query rate per second of the configuration data slice and the service priority includes: determining a preset service priority coefficient corresponding to the service priority, and determining the logarithm of the query rate per second of the configuration data fragments; and adding the preset service priority coefficient and the logarithm of the query rate per second of the configuration data fragments to obtain a dynamic demand weight factor. In an alternative embodiment, the determining the target tile granularity according to the base tile granularity, the dynamic demand weight factor, and the distributed system resource adaptation coefficient includes: And obtain