CN-121636199-B - Heterogeneous calculation force cooperative adaptation method based on distributed pooling calculation force

Abstract

The invention discloses a heterogeneous computing force cooperative adaptation method based on distributed pooling computing force, and relates to the technical field of heterogeneous computing force resource scheduling; the method comprises the steps of collecting a basic data set from a computational heterogeneous platform, obtaining a time sequence feature set and a scene feature set through standardized processing, accurately predicting computational power demand trend of each heterogeneous cluster by means of a computational power prediction model formed by a basic time sequence prediction layer, a scene feature enhancement layer and an output fusion layer, setting demand priority by combining a computational power growth rate threshold and a scene label, generating a scheduling decision by means of a coordinated scheduling model, and optimizing resource allocation by means of a dynamic adaptation adjustment mechanism.

Inventors

• WU HAILIN

Assignees

• 泉州砾鹰石科技有限公司

Dates

Publication Date

20260508

Application Date

20260204

Claims (4)

1. 1. The heterogeneous calculation force cooperative adaptation method based on the distributed pooling calculation force is characterized by comprising the following steps of: collecting a basic data set from the heterogeneous computing platform, and carrying out standardization processing on the basic data set to obtain a characteristic data set, wherein the basic data set is used for analyzing the performance of the heterogeneous computing platform, and the characteristic data set comprises a time sequence characteristic set and a scene characteristic set; Inputting the characteristic data set into a pre-trained computational power prediction model, and predicting to obtain a computational power demand trend; generating a scheduling decision based on a computational power demand trend, a basic data set, a demand priority and a pre-trained coordination scheduling model; Generating a scheduling decision based on the calculated demand trend, the base dataset, the demand priority, and a pre-trained coordinated scheduling model, comprising: calling a coordination scheduling model, wherein the coordination scheduling model is constructed based on DDPG algorithm; constructing a state space based on a calculation force demand trend, a basic data set and a demand priority, and inputting the state space into the coordination scheduling model to obtain an action combination; Generating a scheduling decision after constraint verification of the action combination, wherein the scheduling decision comprises a distribution path, node selection and task segmentation proportion; the platform architecture of the computational power heterogeneous platform comprises a computational power resource pool; The computing power resource pool is composed of a plurality of heterogeneous clusters, wherein the heterogeneous clusters are composed of computing power nodes of the same type, and the computing power nodes comprise CPU nodes, GPU nodes and NPU nodes; Predicting the resulting power demand trend, comprising: invoking a computational power prediction model, wherein the computational power prediction model is constructed based on an LSTM model and a gradient lifting tree model; Inputting the characteristic data set into the computational power prediction model to predict and obtain the computational power demand trend of the computational power heterogeneous platform, wherein the computational power demand trend comprises the computational power demands of various heterogeneous clusters; the computational power prediction model comprises a basic time sequence prediction layer, a scene characteristic enhancement layer and an output fusion layer; the scene characteristic enhancement layer is constructed based on a gradient lifting tree model and is used for correcting the calculation force demand trend; the output fusion layer is used for fusing output results of the basic time sequence prediction layer and the scene characteristic enhancement layer according to the fusion weight coefficient to obtain a calculation force demand trend; setting demand priorities according to the calculated force demand trend and the scene label, including: setting a calculation force increase rate threshold; when the demand priority is inconsistent with the scene label, setting the demand priority by utilizing the scene label; the coordination scheduling model comprises an input layer, a state coding layer, a core decision layer and a decision output layer, wherein the core decision layer realizes decisions through MDP and DDPG algorithms.
2. 2. The distributed pooled computing force-based heterogeneous computing force co-adaptation method of claim 1, wherein collecting a base dataset from the computing force heterogeneous platform comprises: setting sampling frequency, wherein the sampling frequency can be set according to different data types; and acquiring a basic data set from the heterogeneous computing platform according to the sampling frequency, wherein the basic data set comprises resource state data, task state data and network link data.
3. 3. The distributed pooling computing force-based heterogeneous computing force collaborative adaptation method according to claim 1, wherein the normalization process comprises a normalization process, a single thermal encoding process, and sliding window statistics.
4. 4. The distributed pooled computing force-based heterogeneous computing force collaborative adaptation method according to claim 1, wherein scheduling computing force resources of the computing force heterogeneous platform according to the scheduling decision comprises: sending the scheduling decision to the power heterogeneous platform; the power calculation heterogeneous platform generates a node task instruction according to a scheduling decision, and selects a power calculation node as a target node according to a task allocation strategy, wherein the task allocation strategy comprises a proximity principle or load balancing; And sending the node task instruction to the target node, wherein the target node starts task execution.

Description

Heterogeneous calculation force cooperative adaptation method based on distributed pooling calculation force Technical Field The invention belongs to the technical field of heterogeneous computing power resource scheduling, and particularly relates to a heterogeneous computing power collaborative adaptation method based on distributed pooling computing power. Background Under the digital economic development background of cloud primordia and AI primordia double-wheel drive, the distributed pooling computing power has become a core technical mode for supporting high computing power demands of complex scenes, heterogeneous computing power hardware such as CPU, GPU, NPU distributed at different physical positions and different attribution subjects is aggregated through network technology, a logically unified computing power resource pool is formed, and the on-demand allocation and cooperative utilization of computing power resources are realized. The existing heterogeneous computing power adaptation scheme mainly comprises three types of open source adaptation scheme based on container arrangement, a cross-platform computing power integration frame and a distributed parallel computing frame, wherein the distributed parallel computing frame is used for dividing complex tasks into sub-tasks and distributing the sub-tasks to different computing power heterogeneous platforms for parallel execution, and the primary integration and utilization of computing power resources are realized in scenes such as big data analysis and AI computing by combining a unified task scheduling and data synchronization mechanism, so that the heterogeneous computing power adaptation scheme is one of core technical schemes in the field of distributed pool computing power. The existing distributed parallel computing framework is limited by technical architecture design and adaptation logic, and has the obvious technical defects that on one hand, a resource description model focuses on basic parameters of a traditional computing unit, the fine characterization capability of heterogeneous hardware performance characteristics is lacked, a resource allocation strategy and concurrency control logic are in static configuration, a dynamic adjustment mechanism is lacked, heterogeneous computing cooperation adaptation capability is lacked, accurate matching of task characteristics and hardware capability cannot be achieved, and resource utilization rate is difficult to improve, on the other hand, a scheduling system depends on a static topology model and a periodic load sampling mechanism, a sampling period is long, dynamic fluctuation of computing force requirements and instantaneous change of a resource pool state cannot be captured in real time, a scheduling model lacks self-adaptive adaptation capability of task types and service scenes, cross-node and cross-cluster load balancing adjustment capability is weak, dynamic scheduling response is lagged, sudden computing force requirements and cross-domain cooperation scenes are difficult to adapt, and task execution efficiency and service operation stability are seriously affected. In order to solve the technical problems, the invention provides a heterogeneous calculation force cooperative adaptation method based on distributed pooling calculation force. Disclosure of Invention The invention aims to at least solve one of the technical problems in the prior art, and therefore, the invention provides a heterogeneous calculation force cooperative adaptation method based on distributed pool calculation force. To achieve the above object, a first aspect of the present invention provides a heterogeneous computing force collaborative adaptation method based on a distributed pooled computing force, including: Collecting a basic data set from the heterogeneous computing platform, and carrying out standardization processing on the basic data set to obtain a characteristic data set, wherein the basic data set is used for analyzing the performance of the heterogeneous computing platform, and the characteristic data set comprises a time sequence characteristic set and a scene characteristic set; Inputting the characteristic data set into a pre-trained computational power prediction model, and predicting to obtain a computational power demand trend; Generating a scheduling decision based on the calculated force demand trend, the basic data set, the demand priority and a pre-trained coordination scheduling model; and scheduling the computational power resources of the computational power heterogeneous platform according to the scheduling decision. In one possible implementation, collecting a base dataset from a computing heterogeneous platform includes: setting sampling frequency, wherein the sampling frequency can be set according to different data types; and acquiring a basic data set from the heterogeneous computing platform according to the sampling frequency, wherein the basic data set comprises resource s