Search

CN-122019182-A - Idle time computing power resource dynamic scheduling method based on node allocation

CN122019182ACN 122019182 ACN122019182 ACN 122019182ACN-122019182-A

Abstract

The invention discloses a dynamic scheduling method of idle time computing power resources based on node allocation, which relates to the technical field of node allocation, and sequentially executes light simulation execution of a computing task to be scheduled to generate a task feature vector, collects idle time node hardware architecture characteristic data to generate a node hardware feature label, calculates matching degree based on a bidirectional suitability evaluation matrix, allocates a main execution node, combines a task key grade and a node pool integral credit score to determine redundancy and allocates standby redundant nodes for parallel execution, and the closed-loop scheduling flow of the consensus verification window period and the arbitration mechanism is synchronously set, compared with the prior art, the precision of idle time calculation power scheduling and the reliability of an execution result can be improved, so that the problems of low resource utilization rate and easy error of a calculation result caused by the fact that tasks and nodes are adapted and disjointed and the execution process lacks effective verification in the existing idle time calculation power scheduling can be solved.

Inventors

  • LIU YIMING
  • JIN LE

Assignees

  • 中科信控(北京)科技有限公司

Dates

Publication Date
20260512
Application Date
20260211

Claims (10)

  1. 1. The idle time computing power resource dynamic scheduling method based on node allocation is characterized in that the method is comprehensively executed by a resource scheduling center, and the idle time computing power resource dynamic scheduling method comprises the following steps: S1, performing light simulation execution on a calculation task to be scheduled, collecting hardware performance counter data generated in the simulation execution process, and generating a task feature vector based on the hardware performance counter data; step S2, acquiring hardware architecture characteristic data of idle time nodes participating in distribution through a node agent program, and generating a node hardware characteristic tag based on the hardware architecture characteristic data; s3, inputting the task feature vector and the node hardware feature label into a pre-constructed bidirectional suitability evaluation matrix to obtain a matching degree calculation result of each idle node and a target task; step S4, based on the matching degree calculation result, distributing the calculation task to the idle node with the highest matching degree as a main execution node; Step S5, aiming at the calculation task distributed by the main execution node, determining the node redundancy through a redundancy calculation formula based on the task key level and the overall credit score of the node pool, and distributing standby redundant nodes according to the node redundancy to execute in parallel with the main execution node; after the main execution node and the standby redundant node start to execute, a consensus verification window period is synchronously set for carrying out result consistency verification, and an arbitration mechanism is triggered when consensus is not achieved.
  2. 2. The method for dynamic scheduling of idle time computing resources based on node allocation according to claim 1, wherein in step S1, the hardware performance counter data includes a cache hit rate, a branch prediction failure rate, and a memory access delay, and is collected by a time-series sampling method.
  3. 3. The method for dynamically scheduling idle time computing power resources based on node allocation according to claim 1 is characterized in that in step S1, the task feature vector is generated by adopting a Z-score standardization method to carry out standardization processing on hardware performance counter data so as to eliminate dimension differences, analyzing and screening principal components with accumulated variance contribution rate meeting a preset threshold through principal components, and reducing dimensions to preset feature dimensions; the preset threshold and the preset feature dimension are determined by cross-validation during training of the principal component analysis model.
  4. 4. The dynamic scheduling method of idle time computing resources based on node allocation according to claim 1, wherein in step S2, the node agent is a lightweight program deployed at idle time nodes, the hardware architecture characteristic data is collected according to a period, and the collection period is configured when the agent is deployed; The hardware architecture characteristic data comprises an instruction set architecture supported by a central processing unit, a cache level capacity, the number of computing cores of a graphic processor, a memory bandwidth and a storage device interface type, and is encrypted and reported to a resource scheduling center through an AES symmetric encryption algorithm after acquisition.
  5. 5. The method for dynamically scheduling idle time computing resources based on node allocation according to claim 1, wherein in step S2, the generating process of the node hardware feature tag comprises the steps of classifying and encoding the CPU instruction set architecture and the storage device interface type by adopting single-hot encoding; Hierarchical coding is carried out on the capacity of the cache level according to the cache level and the memory bandwidth according to the bandwidth interval by adopting equidistant coding; Adopting interval coding to code the number of cores calculated by the graphic processor according to the interval of the number of cores; the various codes are spliced into 64-bit binary labels according to the sequence of CPU instruction set architecture coding, cache hierarchy capacity coding, GPU computing core number coding, memory bandwidth coding and storage device interface type coding, and the sequence and the bit number are defined in label generation rules.
  6. 6. The method for dynamically scheduling idle time computing power resources based on node allocation according to claim 1, wherein in step S3, a bidirectional suitability evaluation matrix is constructed in a manner that task feature vector dimensions are used as rows, node hardware feature tag dimensions are used as columns, matrix elements are suitable weights of all dimensions, and the weights are obtained through training of a gradient descent algorithm.
  7. 7. The node allocation-based idle time computing power resource dynamic scheduling method according to claim 1, wherein in step S5, task key classes are divided based on task attributes; the node pool is a set of all idle nodes registered in the resource scheduling center; The overall reputation score of the node pool is calculated based on node historical performance data, wherein the historical performance data comprises task completion rate, calculation result verification passing rate and average response delay, and the weight of each data is determined by adopting a hierarchical analysis method and then obtained through weighted summation.
  8. 8. The method for dynamically scheduling idle time computing resources based on node allocation according to claim 1, wherein in step S5, the arbitration mechanism is implemented by selecting nodes with the credit scores ranked in the preset number from the node pool to form an arbitration node group, and the preset number is defined in an arbitration rule; each arbitration node independently re-executes the target task and adopts a plurality of voting values as a final result; and updating reputation records of the main execution node and the redundant node based on the final result, wherein the node reputation value with consistent result is up-regulated, the node reputation value with inconsistent result is down-regulated, and the up-regulated value and the down-regulated value are defined in a reputation updating rule.
  9. 9. The method for dynamically scheduling idle time computing resources based on node allocation according to claim 1, wherein in step S5, the duration of the common verification window period is associated with the estimated execution duration of the task, and the association rule is window period duration=task estimated execution duration× (5% -10%); the result verification method comprises the steps of calculating hash values of returned results of all nodes, and judging that consensus is achieved when the proportion of the number of nodes with the same hash value to the total execution nodes reaches a preset threshold value, wherein the threshold value is defined in a consensus verification rule.
  10. 10. The method for dynamic scheduling of idle computing power resources based on node allocation according to claim 1, wherein in step S5, a redundancy calculation formula is: wherein R represents node redundancy, C represents task key level, P represents overall reputation score of the node pool, and alpha and beta are adjustment parameters.

Description

Idle time computing power resource dynamic scheduling method based on node allocation Technical Field The invention relates to the technical field of node allocation, in particular to a dynamic scheduling method of idle time computing power resources based on node allocation. Background The idle time computing power resource dynamic scheduling method can dynamically allocate idle time computing power by sensing node load change in real time and turning on idle computing power resources, so that resource waste is avoided, low-priority tasks or sudden computing power demands can be supported, the computing power resource utilization rate is improved, the operation cost is reduced, the overall operation stability of the system is ensured, and single-point overload is avoided. At present, the idle time computing power resource dynamic scheduling method has the problems that the task and node hardware characteristic adaptation lacks fine quantitative evaluation and depends on experience judgment to cause the adaptation to be disjointed, the effective result checking and dynamic consensus mechanism is lacking in the execution process, meanwhile, the redundant configuration is mostly in a fixed mode, the dynamic requirements of different task key grades and node reputation conditions are difficult to match, the node malicious behaviors or abnormal behaviors are also lacking in effective constraint, and finally, the resource utilization rate is low, the calculation result is easy to make mistakes, and the system suitability and the execution reliability are insufficient. Therefore, a dynamic scheduling method of idle computing power resources based on node allocation is now proposed to solve the above problems. Disclosure of Invention The invention mainly aims to provide a node allocation-based idle time computing power resource dynamic scheduling method so as to solve the problems in the background. In order to achieve the purpose, the invention adopts the technical scheme that the method for dynamically scheduling the idle time computing power resources based on node allocation is comprehensively executed by a resource scheduling center, and the method for dynamically scheduling the idle time computing power resources comprises the following steps: S1, performing light simulation execution on a calculation task to be scheduled, collecting hardware performance counter data generated in the simulation execution process, and generating a task feature vector based on the hardware performance counter data; step S2, acquiring hardware architecture characteristic data of idle time nodes participating in distribution through a node agent program, and generating a node hardware characteristic tag based on the hardware architecture characteristic data; s3, inputting the task feature vector and the node hardware feature label into a pre-constructed bidirectional suitability evaluation matrix to obtain a matching degree calculation result of each idle node and a target task; step S4, based on the matching degree calculation result, distributing the calculation task to the idle node with the highest matching degree as a main execution node; Step S5, aiming at the calculation task distributed by the main execution node, determining the node redundancy through a redundancy calculation formula based on the task key level and the overall credit score of the node pool, and distributing standby redundant nodes according to the node redundancy to execute in parallel with the main execution node; after the main execution node and the standby redundant node start to execute, a consensus verification window period is synchronously set for carrying out result consistency verification, and an arbitration mechanism is triggered when consensus is not achieved. Preferably, in step S1, the hardware performance counter data includes a cache hit rate, a branch prediction failure rate, and a memory access delay, and is collected by a time-series sampling method. Preferably, in step S1, the task feature vector is generated by performing standardization processing on hardware performance counter data by adopting a Z-score standardization method to eliminate dimension differences, analyzing and screening principal components with accumulated variance contribution rate meeting a preset threshold value by principal components, and reducing the dimension to a preset feature dimension; the preset threshold and the preset feature dimension are determined by cross-validation during training of the principal component analysis model. Preferably, in step S2, the node agent is a lightweight program deployed at idle node, and the hardware architecture characteristic data is collected according to a period, and the collection period is configured when the agent is deployed; The hardware architecture characteristic data comprises an instruction set architecture supported by a central processing unit, a cache level capacity, the number of computing cores of a