Search

CN-121585666-B - Server cluster load balancing method based on improved consistency hash and Redis

CN121585666BCN 121585666 BCN121585666 BCN 121585666BCN-121585666-B

Abstract

The invention relates to a server cluster load balancing method based on improved consistency hash and Redis, which comprises the steps of firstly constructing a modularized cluster framework comprising a user management layer, a load balancing layer and the like, collecting node hardware performance parameters and real-time load data through a cluster state monitoring layer, dynamically calculating the number of virtual nodes by the load balancing layer based on parameters such as node performance ratio and the like, mapping a hash ring by adopting an improved consistency hash algorithm, positioning a target physical node, distributing requests by combining dynamic scheduling strategies, realizing active and standby efficient switching by a cluster high availability layer through an election queue mechanism, and completing redundant backup and synchronous sharing of Session data by means of the Redis cluster. The method solves the problems of data inclination, low efficiency of active-standby switching, inconsistent Session sharing and the like in the traditional technology, improves cluster load balancing efficiency and high availability, and is suitable for high concurrence scenes such as medical treatment, finance, electronic commerce and the like.

Inventors

  • YAN JIAHUI

Assignees

  • 四川华鲲振宇智能科技有限责任公司

Dates

Publication Date
20260508
Application Date
20260129

Claims (5)

  1. 1. The server cluster load balancing method based on the improved consistency hash and Redis is characterized by comprising the following steps of: s1, constructing a server cluster architecture, wherein the server cluster architecture comprises a user management layer, a version control layer, a load balancing layer, a cluster high-availability layer and a cluster state monitoring layer, and communication connection is established between the layers through a preset API interface; s2, periodically collecting hardware performance parameters and real-time load data of each node in a server cluster by a cluster state monitoring layer, wherein the hardware performance parameters comprise CPU (Central processing Unit) utilization rate, memory occupancy rate and network bandwidth throughput, and the real-time load data comprise current connection number, task processing queue length and data transmission delay; s3, calculating performance values of all nodes by a load balancing layer, and determining the number of virtual nodes corresponding to each node by taking a node performance ratio and a maximum linear value of the virtual nodes as core parameters; S4, the load balancing layer adopts an improved consistency hash algorithm, maps virtual nodes of each node to a hash ring, calculates user request data through a hash function to obtain a corresponding hash value, positions a target virtual node on the hash ring according to the hash value, and further determines a target physical node; S5, the load balancing layer combines the real-time load data and the positioning result, distributes the user request to the target physical node through a dynamic scheduling strategy, and relocates the optimal physical node based on the node performance value and the residual load capacity if the real-time load of the target physical node exceeds a preset threshold value; s6, the cluster high availability layer monitors the running state of the main node in real time through an election queue mechanism, and when the main node is down, a new main node is elected from the standby nodes to complete the main-standby switching; S7, storing the Session data by using the Redis cluster, and synchronously acquiring the Session data of the original master node from the Redis cluster to a new master node after the cluster high availability layer completes the master-slave switching to realize Session sharing; the specific process of step S3 is as follows: S31, calculating the comprehensive performance value of each node by adopting a weighted summation method based on the preprocessed standardized parameters, wherein the comprehensive performance value is specifically as follows: presetting weight coefficients of all hardware performance parameters; calculating a node comprehensive performance value according to the following formula, wherein the node performance value P=P 1 ×W 1 +P 2 ×W 2 +P 3 ×W 3 ; Wherein P 1 、P 2 、P 3 is a CPU standardized value, a memory standardized value and a network bandwidth standardized value respectively, and the corresponding weights are W 1 、W 2 、W 3 respectively; Traversing all nodes in the server cluster, and respectively calculating to obtain a performance value P n of each node, wherein n is a node sequence number; s32, calculating the performance ratio of a single node according to the performance values of all the nodes: S321, calculating the sum of the performance values of all the nodes of the cluster, wherein the formula is a cluster performance total value P total =ΣP n , and n=1 to N; s322, calculating the performance ratio of each node according to the following formula: node performance ratio r=single node performance value P n /cluster performance total value P total ; s33, calculating the number of virtual nodes corresponding to each node through a preset model based on the node performance ratio, the maximum linear value of the virtual nodes and the total number of cluster nodes; s331, configuring a maximum linear value L of a virtual node, and presetting according to a cluster scale; S332, configuring a preset reference node number M, and configuring the standard node number of a server cluster; s333, solving a calculation model according to the following virtual node quantity: virtual node number v=node performance ratio r×virtual node maximum linear value lx (total number of cluster nodes N/preset reference node number M); S334, rounding the calculation result V; s335, outputting the number of virtual nodes corresponding to each node as an input parameter for improving hash ring mapping in a consistent hash algorithm; The specific process of step S4 is as follows: S41, generating unique identifiers of virtual nodes, namely based on the calculated number of virtual nodes corresponding to each physical node, marking as V n , n as a physical node serial number, and distributing the unique identifiers for all the virtual nodes of each physical node, wherein the specific process is as follows: s411, extracting the core network identifier of each physical node, adopting the IP address of the physical node as a basic identifier, and ensuring the uniqueness and stability of the identifier; s412, sequentially numbering virtual nodes of each physical node, wherein the numbering rule is from 1 to V n ; s413 generating a virtual node unique identification using the IP address-virtual node number concatenation format, S42, calculating hash values of the virtual nodes and the user requests, namely carrying out hash calculation on the virtual node identifiers and the user request data by adopting a unified hash function, and ensuring uniform distribution and calculation efficiency of the hash values, wherein the specific process is as follows: s421, selecting an MD5 hash algorithm as a core hash function, wherein the output length of a hash value is 128 bits; S422, performing MD5 hash calculation on the generated unique identifier of each virtual node, namely converting an identifier character string into a byte array, inputting an MD5 algorithm for operation to obtain a 128-bit hash value, and mapping the 128-bit hash value into an integer hash value in a 0-232-1 interval through Base64 decoding or hexadecimal conversion, and marking the integer hash value as H v ; S423, carrying out hash calculation on user request data, namely extracting core feature data of a user request, including a request URL, a request parameter abstract and a user unique identification Session ID, splicing the core feature data into a request feature character string, adopting an MD5 hash algorithm which is the same as a virtual node to calculate the request feature character string to obtain a 128-bit hash value, and mapping the 128-bit hash value into an integer hash value in a 0-232-1 interval, and marking the integer hash value as H r ; s43, mapping the virtual nodes to the hash ring, namely mapping the hash values H v of all the virtual nodes to a preset hash ring according to the numerical value, and constructing a complete hash ring mapping relation, wherein the specific process is as follows: S431, defining the value range of the hash ring as 0-232-1, and regarding the interval as a closed ring structure; S432, traversing hash values H v of all virtual nodes, taking each H v as a mapping point on a hash ring, and positioning on the hash ring according to the value of Hv; S433, establishing a mapping relation table of virtual node hash values and physical nodes, and storing original physical node information corresponding to each H v , wherein the original physical node information comprises an IP address, a performance value and a current load state; S44, positioning a target virtual node based on the hash value, namely searching a corresponding target virtual node on the hash ring according to the hash value H r of the user request, wherein the specific process is as follows: S441, traversing H v of all virtual nodes on the hash ring, and constructing an ordered hash value list according to the sequence from small to large in value; s442, locating a target virtual node by adopting a clockwise searching rule, namely searching a first H v which is more than or equal to H r in the ordered hash value list, wherein the virtual node corresponding to H v is the target virtual node; S443, if H r is larger than the maximum value of all H v on the hash ring, default positioning is performed on the virtual node corresponding to the H v with the smallest value on the hash ring; S45, determining a target physical node through the target virtual node, namely finally determining the target physical node requested by the user based on the association relation between the target virtual node and the physical node; S451, searching a virtual node hash value-physical node mapping relation table established in the step S43, and acquiring the complete information of the corresponding original physical node, including an IP address, a performance value and a current load state, through H v of the target virtual node; S452, pre-checking the current load state of the target physical node, namely directly determining the real-time load of the physical node as a final target physical node if the real-time load of the physical node does not exceed a preset threshold value; And S453, if the real-time load of the target physical node exceeds a preset threshold, triggering clockwise suboptimal node searching, namely continuously searching the next virtual node on the hash ring in the clockwise direction, repeating the verification process of the steps S451-S452 until the physical node with the load meeting the requirement is found, and determining the physical node as the target physical node.
  2. 2. The server cluster load balancing method based on improved consistent hashing and Redis according to claim 1, wherein step S2 further includes a data preprocessing process, specifically including the following steps: S21, defining reference thresholds of all hardware performance parameters, wherein the reference thresholds comprise a CPU (Central processing Unit) utilization reference threshold, a memory occupancy reference threshold and a network bandwidth throughput reference threshold; S22, carrying out normalization conversion on each parameter, wherein a calculation formula is that a parameter standardization value= (actual parameter value of a node/parameter reference threshold value) multiplied by 100; s23, outputting a CPU standardized value, a memory standardized value and a network bandwidth standardized value of each node, and marking the standardized values as P 1 、P 2 、P 3 .
  3. 3. The server cluster load balancing method based on improved consistent hashing and Redis according to claim 1, wherein the specific process of step S5 is as follows: s51, presetting a load threshold interval and core parameters, namely completing basic parameter configuration before the dynamic scheduling strategy is executed; S52, acquiring an initial positioning result and real-time load data; s53, judging the state of a target node based on a load threshold; s54, request distribution in a normal state or a low load state, wherein in the normal load state, a user request is directly routed to the target physical node, and a request distribution log is recorded; The low load state is that on the basis of completing request distribution, a virtual node dynamic adjustment mechanism is triggered, the number of virtual nodes corresponding to the nodes is increased, the virtual node mapping relation of the hash ring is synchronously updated, and the distribution range of subsequent requests is enlarged; and S55, performing optimal physical node reselection under a high load state, namely starting a node reselection flow when the load rate of the target physical node exceeds a preset threshold value, and screening the optimal node based on the node performance value and the residual load capacity.
  4. 4. The server cluster load balancing method based on improved consistent hashing and Redis according to claim 1, wherein the specific process of step S6 is as follows: S61, initializing a standby node election queue, namely completing construction and configuration of the election queue when a server cluster is started, and providing a candidate node basis for subsequent active-standby switching; S62, monitoring the running state of the main node in real time, wherein the cluster high-availability layer grasps the running state of the main node in real time through a multi-dimensional monitoring mechanism; S63, screening candidate master nodes from the election queue, namely rapidly screening candidate master nodes meeting the conditions based on the initialized election queue; S64, verifying and confirming the candidate master node, namely performing multidimensional verification on the screened candidate master node to ensure that the candidate master node has the capacity of taking over the service of the master node; s65, executing a master-slave switching operation, namely completing role switching, resource allocation and service takeover after determining a new master node, and ensuring seamless connection of services; and S66, confirming the switching result and synchronizing the state, namely after the main and standby switching is completed, verifying the switching effect and synchronizing the state to each layer of the cluster, so as to ensure the consistency of the system.
  5. 5. The server cluster load balancing method based on improved consistent hashing and Redis according to claim 1, wherein the specific process of step S7 is as follows: s71, initializing, generating and storing Session data, namely completing creation of the Session data and storage of a Redis cluster when a user initiates a request to the server cluster for the first time; s72, performing Redis cluster Session data redundancy backup, wherein the Redis cluster realizes real-time data synchronization through a master-slave replication mechanism, and builds redundancy backup; S73, after the master-slave switching, the Session data synchronization triggering, namely after the cluster high availability layer completes the master-slave switching, automatically triggering the Session data synchronization flow of the new master node and the Redis cluster; S74, batch synchronization and verification of the Session data, wherein a new master node acquires the Session data from a designated Redis slave node to complete batch synchronization and integrity verification; S75, session sharing is effective and real-time synchronous maintenance is carried out, namely after initial data synchronization is completed, a new master node enters a Session data real-time synchronous state, and consistency of the Session data in subsequent service interaction is ensured; And S76, after synchronization result feedback and exception handling, after synchronization of the Session data is completed, feeding back the synchronization result and handling possible exception conditions.

Description

Server cluster load balancing method based on improved consistency hash and Redis Technical Field The invention belongs to the technical field of server cluster load balancing, and particularly relates to a server cluster load balancing method based on improved consistency hash and Redis. Background With the rapid development of the information technology, the business scale of key industries such as medical treatment, finance, electronic commerce and the like is continuously expanded, the user access quantity is increased in a explosive manner, and unprecedented stringent requirements are put on the load bearing capacity and high availability of the server cluster. The traditional single-point server architecture has limited hardware resources, so that the problems of resource exhaustion, response delay rapid increase and even system breakdown easily occur when facing to a high-concurrency request scene, and the stable operation requirement of modern business cannot be met. Therefore, the server cluster architecture becomes the mainstream choice, and the load balancing technology and the high-availability design become the core keys for guaranteeing the efficient and stable operation of the cluster. Currently, various load balancing algorithms and high-availability architecture schemes are developed in the industry, but a plurality of technical pain points to be solved still exist in practical application. In the aspect of load balancing, the traditional consistent hash algorithm can solve the problem of data migration when nodes are increased or decreased to a certain extent, but does not consider the hardware performance difference of each node, hash ring mapping is carried out by only adopting a fixed number of virtual nodes, so that the node with strong performance cannot fully exert computing power, the node with weaker performance faces overload risk, data inclination is finally caused, the overall load balancing efficiency of the cluster is reduced, meanwhile, the existing dynamic scheduling strategy is mainly dependent on a single load index to make decisions, the comprehensive consideration of the performance value and the residual load capacity of the node is lacked, and the accurate and balanced allocation of the request flow is difficult to realize. In the aspect of high availability of clusters, a traditional main-standby switching mechanism mainly adopts a simple priority ordering or random election mode to select standby nodes, key factors such as real-time performance states, data synchronization progress and the like of uncombined nodes are adopted, so that the problems of insufficient resources, incomplete data and the like of a new main node possibly exist after switching, service continuity is affected, in addition, session data sharing among the main-standby nodes is an important link for guaranteeing seamless connection of services, a mode of locally storing Session data is partially adopted in the existing scheme, the standby nodes cannot acquire complete Session information after the main node fails, the login state of a user is lost, service flow is interrupted, and even if the Session sharing is realized by adopting distributed storage in the partial scheme, the problems of high data synchronization delay, difficult guarantee of consistency and the like exist. Moreover, the existing server cluster architecture has the problem of high module coupling degree, interaction logic among functional modules is complex, and when service requirements are changed or cluster scale is adjusted, the expansion and adaptation are difficult to quickly perform, so that the maintenance cost and the upgrading difficulty of the system are increased. In summary, how to solve the problems of data inclination, low efficiency of active-standby switching, inconsistent Session sharing, poor system expansibility and the like in the prior art, and realize high-efficiency load balancing and high-reliability operation of a server cluster in a high concurrency scene becomes a technical problem to be overcome in the art. Therefore, the invention aims to provide a server cluster load balancing method based on improved consistent hashing and Redis, so as to solve the technical problems in the prior art. Disclosure of Invention The invention aims to provide a server cluster load balancing method based on improved consistent hashing and Redis, which is used for solving the technical problems in the prior art. In order to solve the technical problems, the invention adopts the following technical scheme: a server cluster load balancing method based on improved consistency hash and Redis comprises the following steps: s1, constructing a server cluster architecture, wherein the server cluster architecture comprises a user management layer, a version control layer, a load balancing layer, a cluster high-availability layer and a cluster state monitoring layer, and communication connection is established between the layers