Search

CN-121411971-B - Method and system for scheduling information creation cloud resources based on multi-core architecture

CN121411971BCN 121411971 BCN121411971 BCN 121411971BCN-121411971-B

Abstract

The application relates to the technical field of distributed computing resource scheduling, and discloses a method and a system for scheduling information creation cloud resources based on a multi-core architecture. The method comprises the steps of collecting load data and chip integration information to generate a real-time monitoring result, carrying out node grouping by adopting a load balancing algorithm according to the monitoring result to determine resource allocation requirements, triggering a resource virtualization technology to execute resource remapping when the node load exceeds a preset threshold to obtain node resource configuration, constructing a cluster structure based on new configuration and determining a resource reconstruction scheme, executing data migration according to the reconstruction scheme, generating a migration path by adopting a compatibility interface algorithm and verifying state synchronization integrity, and activating a dynamic management strategy to optimize a resource allocation model when synchronization is complete. The method can improve the resource utilization rate and the parallel processing capability of the multi-core cluster, solve the problems of low efficiency and stability caused by uneven resources and rapid increase of loads in the created cloud environment, and realize the efficient and autonomous controllable utilization of the resources.

Inventors

  • XIE JIANGTAO
  • HU DAPENG
  • WANG GUIZHENG
  • GONG HAO
  • LIAO ZELONG
  • LIU RUI

Assignees

  • 广东粤数网络建设有限公司

Dates

Publication Date
20260512
Application Date
20251030

Claims (9)

  1. 1. The method for scheduling the created cloud resources based on the multi-core architecture is characterized by comprising the following steps: Step 1, collecting load data and chip integration information of each node through an agent module deployed on a computing node, generating a real-time monitoring result, and determining resource distribution conditions among the nodes; step 2, according to the real-time monitoring result, a load balancing algorithm is applied to group the computing nodes, a node classification result is obtained, and the resource allocation requirement of each node is judged; step 3, if the node classification result shows that the load value of a certain class of nodes exceeds a preset threshold, triggering a resource adjustment mechanism, and remapping the node resources through a resource virtualization technology to obtain adjusted node resource configuration; step 4, constructing a new cluster structure based on the adjusted node resource allocation, extracting reconstruction parameters, and determining a resource reconstruction scheme based on the reconstruction parameters; Step 5, executing a data migration flow according to the resource reconstruction scheme, generating a migration path by adopting a compatibility interface algorithm, and judging the state synchronization integrity in the migration process according to the migration path; step 6, if the state synchronization integrity is confirmed, activating a dynamic management strategy and generating a resource allocation model so as to realize the stable allocation of system resources and the stability of integral operation; the node classification result includes load surge nodes and idle nodes, and the step 3 includes: Aiming at the load surge node, determining a load value based on a real-time monitoring result, judging whether the load value exceeds a preset threshold value, and if so, confirming that the load surge node is in an overload state; after an overload node is detected, triggering an in-situ resource adjustment mechanism, entering a resource virtualization layer to execute resource remapping operation under the condition that node operation is kept uninterrupted, wherein the resource remapping operation is used for identifying the current resource allocation state of each computing node, mapping partial load from the overload node to idle nodes based on a load balancing algorithm, and updating a virtual resource pool; generating an adjusted node resource configuration according to the execution result of the remapping operation; encoding the adjusted node resource configuration into a remapping signal through an interconnection bus, and transmitting the remapping signal through a bus protocol; triggering a security encryption module to carry out integrity verification on the adjusted node resource configuration based on the remapping signal; And if the verification is passed, confirming that the adjusted node resource allocation is effective, and generating a resource allocation report containing the details of the adjusted resource allocation and the verification result.
  2. 2. The method for scheduling information-based cloud resources based on multi-core architecture according to claim 1, wherein the step 1 includes: periodically collecting load data and chip integration information of the computing nodes by a polling mechanism through agent modules deployed on the computing nodes; calculating the resource utilization rate of each computing node by adopting a weighted average method based on the load data; Analyzing and extracting parallel processing capacity data of each computing node based on the chip integration information; Fusing the resource utilization rate and the parallel processing capacity data to generate a structured real-time monitoring result; According to the real-time monitoring result, judging whether resource distribution imbalance exists among the computing nodes by calculating the variance or standard deviation of the resource utilization rate among the nodes; if the resource distribution imbalance is judged to exist, determining an expansibility configuration requirement based on the imbalance degree and the historical trend, wherein the expansibility configuration requirement comprises the number of newly added nodes, a virtual resource mapping strategy and/or parallel capacity lifting parameters; And processing the multi-path monitoring data flow by adopting a multi-thread parallel algorithm based on the parallel processing capacity data, and summarizing to generate an analysis report containing resource distribution statistics and optimization suggestions.
  3. 3. The method for scheduling information-based cloud resources based on multi-core architecture according to claim 2, wherein the step 2 includes: Extracting load parameters of each computing node based on real-time monitoring results, and respectively carrying out weighted summation on the load parameters of each computing node to obtain a load value of each computing node; Comparing the load value with a preset load threshold value, and classifying the computing nodes into load surge nodes and idle nodes according to the comparison result; aiming at the load surge node, calculating the resource allocation demand according to the current load value and the parallel processing capacity data of the node; determining, for an idle node, a resource availability based on its remaining computing resources and available bandwidth; Maintaining synchronization of packet data between the load shock node and the idle node through a cache consistency protocol, including writing the packet data into a shared cache, broadcasting an update message, and adopting a write failure strategy to ensure data consistency; Judging whether the activation condition of a thread scheduling mechanism is met or not based on the consistency rate of packet data synchronization; When the activation condition is met, calculating the thread migration priority according to the resource allocation demand and the resource availability, and executing thread scheduling; a resource allocation demand report is generated based on the node classification, the resource allocation demand, the resource availability, and the thread scheduling result.
  4. 4. The method for scheduling information-based cloud resources based on the multi-core architecture according to claim 1, wherein the step 4 includes: Determining cluster structure parameters including the number of nodes, load distribution and connection topology based on the adjusted node resource configuration; classifying cluster structure parameters through a dynamic scheduling engine, and preferentially processing high-load parameters by adopting a priority queue algorithm; Generating a high-availability cluster structure with redundant node configuration according to the processed parameter set; Extracting key performance indexes from a high-availability cluster structure as reconstruction parameters, wherein the reconstruction parameters comprise node utilization rate and data flow speed; when the node utilization rate exceeds a preset utilization threshold, a resource reconstruction scheme is determined by applying a load balancing algorithm based on the reconstruction parameters, wherein the resource reconstruction scheme comprises a task migration target; optimizing an execution path of a resource reconstruction scheme by adopting a shortest path algorithm through a dynamic scheduling engine, and identifying and avoiding a network delay bottleneck; Based on the optimized execution path, generating a data localization storage path, and verifying the availability of the cluster structure through a simulated data access test; And generating a configuration report containing the cluster structure, the resource reconstruction scheme, the optimized execution path and the verification result.
  5. 5. The method for scheduling information-based cloud resources based on the multi-core architecture according to claim 1, wherein the step 5 includes: analyzing node load data and available resource indexes based on a resource reconstruction scheme, and determining a source node and a target node of data migration; The compatibility of an interface protocol and a data format between a source node and a target node is evaluated through a compatibility interface algorithm, and a shortest path algorithm is adopted to generate a data migration path between the source node and the target node based on an evaluation result; executing the segmented transmission and confirmation flow of the data block according to the data migration path; in the migration process, monitoring energy consumption data of each node in real time through a power consumption management strategy, collecting power consumption values and recording an energy consumption sequence; Based on the energy consumption data, identifying abnormal energy consumption peaks by adopting a time sequence analysis method, and determining intervention points of a fault isolation technology; When the intervention point is triggered, switching a standby path or isolating abnormal nodes through a fault isolation technology, and recovering a normal migration flow; according to the data migration path, comparing the data integrity of the source node and the target node by adopting a hash algorithm, verifying whether the state synchronization is complete, if not, triggering a retransmission mechanism, and re-executing the data migration; And generating a data migration execution report containing the migration duration, the total energy consumption and the synchronous verification result.
  6. 6. The method for scheduling information-based cloud resources based on the multi-core architecture according to claim 1, wherein the step 6 includes: when the verification result of the state synchronization integrity is complete, activating a dynamic management strategy; Performing iterative optimization on execution parameters of the dynamic management strategy by adopting genetic algorithm through autonomous controllable software, wherein the execution parameters comprise a load threshold value, a resource allocation proportion and a synchronization interval time; constructing a resource allocation model describing a resource mapping relation between nodes based on the optimized execution parameters; Integrating the optimization result of the resource allocation model by using a priority queue algorithm through a thread scheduling mechanism to form a unified optimization data set; Based on the optimized data set, extracting node grouping data, calculating parallel processing capacity indexes of each computing node, analyzing the node grouping data and the parallel processing capacity indexes by adopting a k-means clustering algorithm, and determining an equilibrium degree index of the parallel processing capacity; based on the balance index, simulating an operation scene of the resource allocation model, calculating response time and error rate of the resource allocation model under different load conditions, and verifying whether the resource allocation model meets a preset stability threshold; If the stability verification is not passed, adjusting parameters of the resource allocation model and regenerating the model until the verification is passed; And generating a resource allocation model report containing a final resource allocation model, an equilibrium index and a stability verification result.
  7. 7. The method for scheduling information-based cloud resources based on a multi-core architecture according to claim 2, wherein the processing the multi-path monitoring data stream by using a multi-thread parallel algorithm based on the parallel processing capability data, and summarizing the analysis report including the resource distribution statistics and optimization suggestions comprises: For multi-channel monitoring data flows, calculating the processing priority of each data flow based on the data flow size and the arrival time, and forming a priority queue; taking out the data streams from the priority queue, inquiring available computing core resources, and uniformly distributing the core resources to each data stream; According to the resource allocation result, executing parallel processing of multiple paths of data streams, recording core use duration and memory occupation data of each data stream, and generating a resource distribution data table of each node based on the recorded execution result; Calculating the standard deviation of the core use duration and the memory occupation among the nodes according to the resource distribution data table, and obtaining the comprehensive balance score based on weighted average; If the comprehensive balance score exceeds a preset balance threshold, generating a resource distribution optimization suggestion, wherein the optimization suggestion comprises a migration source node, a migration target node and a migration data flow identifier; according to the optimization suggestion, the processing strategy of the multipath data flow is adjusted, and the priority queue and the data flow distribution are updated; Summarizing the adjusted resource distribution data, the comprehensive balance score, the optimization suggestion and the execution result of the optimization suggestion, and generating an analysis report.
  8. 8. The method for scheduling information-based cloud resources based on a multi-core architecture according to claim 1, wherein triggering the security encryption module to perform integrity verification on the adjusted node resource configuration based on the remapped signal comprises: Determining a verification requirement for the adjusted node resource configuration based on the remapping signal from the interconnect bus; generating, by the secure encryption module, an encryption verification key based on the node identification and the resource parameter in the remapped signal; Performing encryption hash calculation on the packet data of the adjusted node resource configuration by using the encryption verification key, comparing a calculation result with a pre-stored reference, and determining the integrity state of the resource configuration according to the comparison result; If the integrity state is abnormal, triggering retransmission of the remapping signals, and sending new remapping signals through a standby path; updating the resource allocation of the computing node by means of the retransmitted remapping signals; generating an integrity verification report according to the updated resource configuration and verification process; Based on the abnormal frequency and the transmission delay in the verification report, adjusting optimization parameters of the security encryption module, wherein the optimization parameters comprise a key generation period and a key length.
  9. 9. A multi-core architecture-based credit cloud resource scheduling system for implementing the multi-core architecture-based credit cloud resource scheduling method as claimed in any one of claims 1 to 8, characterized in that the system comprises: The data acquisition module is used for acquiring load data and chip integration information of each node through an agent module deployed on the computing node, generating a real-time monitoring result and determining resource distribution conditions among the nodes; the grouping evaluation module is used for grouping the computing nodes by applying a load balancing algorithm according to the real-time monitoring result to obtain a node classification result and judging the resource allocation requirement of each node; the resource adjustment module is used for triggering a resource adjustment mechanism when the node classification result shows that the load value of a certain class of nodes exceeds a preset threshold value, and remapping the node resources through a resource virtualization technology to obtain adjusted node resource configuration; The structure construction module is used for constructing a new cluster structure according to the adjusted node resource configuration, extracting reconstruction parameters and determining a resource reconstruction scheme based on the reconstruction parameters; The migration execution module is used for executing a data migration flow according to the resource reconstruction scheme, generating a migration path by adopting a compatibility interface algorithm, and judging the state synchronization integrity in the migration process according to the migration path; And the strategy activation module is used for activating the dynamic management strategy and generating a resource allocation model when the state synchronization integrity is confirmed, so that the stable allocation of system resources and the stability of the whole operation are realized.

Description

Method and system for scheduling information creation cloud resources based on multi-core architecture Technical Field The application relates to the technical field of distributed computing resource scheduling, in particular to a method and a system for scheduling information creation cloud resources based on a multi-core architecture. Background With the rapid development of cloud computing technology, resource scheduling and management become core problems for ensuring efficient operation of a cloud platform. The traditional cloud computing resource scheduling method mainly depends on a single or few core computing nodes, and cannot fully exert the computing capacity and the processing efficiency of the multi-core architecture. Particularly in a cloud computing system integrated by a domestic chip, due to the specificity and technical limitation of a hardware architecture, how to reasonably allocate computing resources, balance loads and ensure high availability and safety of the system becomes a great challenge facing the current technology. With the development of the credit industry, more and more cloud computing platforms begin to adopt domestic chips based on a multi-core architecture, but the existing cloud resource scheduling method is not completely suitable for the novel hardware environment. Most of traditional scheduling methods rely on static load balancing, and lack of dynamic adjustment and real-time optimization mechanisms for multipath data streams and node resources causes unbalanced computing resource allocation, even causes system bottlenecks of overloaded nodes, and affects the processing capacity and stability of a cloud platform. In order to adapt to the characteristics of the multi-core architecture, the current technical research focuses on how to intelligently schedule cloud computing resources through dynamic monitoring and real-time data analysis. The prior art breaks through in part, but the problems of insufficient resource utilization, insufficient dynamic load balancing, imperfect security verification and the like of the multi-core architecture still exist. Therefore, developing a method for scheduling information-created cloud resources in a multi-core architecture environment solves the problems of uneven resource allocation, overload loading and integrity in data migration, improves the stability and expansibility of clusters, and becomes a key problem to be solved in the technical field at present. Disclosure of Invention The invention aims to solve the problems of low resource utilization rate, poor load balance, incomplete data migration, insufficient system stability and the like of the traditional cloud computing resource scheduling method in a multi-core architecture environment, and provides a created cloud resource scheduling method and system based on the multi-core architecture, so as to realize efficient scheduling and dynamic management of cloud resources in a domestic chip environment. In a first aspect, the present application provides a method for scheduling information creation cloud resources based on a multi-core architecture, the method comprising: Step 1, collecting load data and chip integration information of each node through an agent module deployed on a computing node, generating a real-time monitoring result, and determining resource distribution conditions among the nodes; step 2, according to the real-time monitoring result, a load balancing algorithm is applied to group the computing nodes, a node classification result is obtained, and the resource allocation requirement of each node is judged; step 3, if the node classification result shows that the load value of a certain class of nodes exceeds a preset threshold, triggering a resource adjustment mechanism, and remapping the node resources through a resource virtualization technology to obtain adjusted node resource configuration; step 4, constructing a new cluster structure based on the adjusted node resource allocation, extracting reconstruction parameters, and determining a resource reconstruction scheme based on the reconstruction parameters; Step 5, executing a data migration flow according to the resource reconstruction scheme, generating a migration path by adopting a compatibility interface algorithm, and judging the state synchronization integrity in the migration process according to the migration path; And step 6, if the state synchronization integrity is confirmed, activating a dynamic management strategy and generating a resource allocation model so as to realize the stable allocation of system resources and the stability of overall operation. In a second aspect, the present application provides a system for scheduling information-created cloud resources based on a multi-core architecture, the system comprising: The data acquisition module is used for acquiring load data and chip integration information of each node through an agent module deployed on the computing node, gener