Search

CN-122019184-A - Task processing method and related product

CN122019184ACN 122019184 ACN122019184 ACN 122019184ACN-122019184-A

Abstract

The application provides a task processing method, a Switch chip, a distributed computing network, an electronic device and electronic equipment, which relate to the technical field of the distributed computing network and are characterized in that by sharing an aggregation operation task to a master PM and a slave PM, the master PM and the slave PM respectively bear the aggregation operation of a part of task data, and the master PM obtains the final target aggregation operation result, so that the problems of concentrated calculation hot spots and unbalanced resource utilization are relieved to a certain extent, the calculation efficiency is improved, and the queuing delay is reduced.

Inventors

  • LAN KEJIA
  • HOU SONG

Assignees

  • 海光信息技术股份有限公司

Dates

Publication Date
20260512
Application Date
20260210

Claims (20)

  1. 1. A task processing method is characterized by being applied to a master proxy module PM in a Switch chip, wherein the Switch chip comprises a plurality of tasks PM, the master PM is any one of the tasks PM, and the method comprises the following steps: The method comprises the steps of responding to a request of receiving an aggregation operation task, acquiring first task data required by the aggregation operation task from a processor connected with a main PM, wherein the task PM comprises operation PM and non-operation PM, the operation PM is the task PM for executing the aggregation operation in the task PM, the operation PM comprises the main PM and at least one sub PM, and the non-operation PM is the task PM for not executing the aggregation operation in the task PM; Receiving second task data transmitted by a non-operation PM corresponding to the main PM under the condition that the main PM has the corresponding non-operation PM, and performing aggregation operation on the first task data and the second task data to obtain a first aggregation operation result; Receiving the second aggregation operation result transmitted from PM, and performing aggregation operation on the first aggregation operation result and the second aggregation operation result to obtain a target aggregation operation result; and sending the target aggregation operation result to a target processor, wherein the target processor is a processor using the operation result of the aggregation operation task.
  2. 2. The task processing method according to claim 1, wherein before receiving second task data transmitted from a non-operation PM corresponding to the master PM, and receiving a second polymerization operation result transmitted from a PM, the method further comprises: Sending a reading message to each target PM in the task PM; the target PM is the task PM except the main PM in the task PMs, and the reading message is used for indicating the target PM receiving the reading message to acquire task data required by the aggregation operation task from the connected processor and indicating the operation PM corresponding to the target PM receiving the reading message.
  3. 3. The task processing method according to claim 2, wherein before sending the read message to each target PM in the task PMs, the method further comprises: Determining each target PM according to the processor related to the aggregation operation task; Determining the slave PM and the non-operation PM from the target PM, and determining the corresponding relation between the non-operation PM and the operation PM; and constructing a reading message of each target PM according to the corresponding relation between each non-operation PM and the operation PM.
  4. 4. The task processing method according to claim 3, wherein determining the slave PM and the non-operational PM from among the target PMs, and determining the correspondence between each of the non-operational PMs and the operational PM, comprises: dividing all the task PM into a plurality of PM groups, wherein each PM group comprises at least two task PM; Selecting one task PM from all task PMs in each target PM group as a slave PM of the target PM group according to each target PM group, wherein the target PM group is a PM group which does not contain a master PM, and the other task PMs except the slave PM in the target PM group are non-operation PMs; The method comprises the steps that operation PM corresponding to each non-operation PM in each target PM group is a slave PM in the target PM group, and other task PM except for a master PM in a PM group containing the master PM is the non-operation PM, and operation PM corresponding to each non-operation PM is the master PM.
  5. 5. The task processing method according to claim 4, wherein dividing all task PMs into a plurality of PM groups includes: Calculating a rounded value n after k/m, wherein k is the total number of tasks PM, and m is a preset positive integer greater than 1; Dividing the task PM into n PM groups, wherein the number of the task PM in the n-1 PM groups is m, and the number of the task PM in the 1 PM groups is k- (n-1) m。
  6. 6. The task processing method according to claim 4, wherein for each target PM group, selecting one task PM from among the task PMs in the target PM group as a slave PM of the target PM group, comprises: For each target PM group, one task PM is randomly selected from the task PMs in the target PM group as a slave PM of the target PM group.
  7. 7. The task processing method according to claim 4, wherein for each target PM group, selecting one task PM from among the task PMs in the target PM group as a slave PM of the target PM group, comprises: for each target PM group, the task PM with the optimal performance in the target PM group is taken as the slave PM of the target PM group.
  8. 8. The task processing method according to claim 4, wherein after selecting, for each target PM group, one task PM from among the task PMs in the target PM group as a slave PM of the target PM group, the method further comprises: And adjusting the number of the task PM in each PM group according to the performance condition of the operation PM in each PM group, wherein each adjusted PM group comprises at least 2 task PM, and the better the performance condition of the operation PM in the PM group is, the more the number of the task PM in the adjusted PM group is.
  9. 9. The task processing method according to claim 3, wherein determining the slave PM and the non-operational PM from among the target PMs, and determining the correspondence between each of the non-operational PMs and the operational PM, comprises: acquiring performance conditions of target PM (physical quantity) which are task PM except the main PM; Determining the slave PM and the non-operational PM from each target PM according to the performance condition of each target PM; All tasks PM are divided into n PM groups, wherein n is equal to the number of PM plus 1, and each PM group comprises at least one non-operation PM and one operation PM.
  10. 10. The task processing method according to claim 9, wherein determining the slave PM and the non-operational PM from each of the target PMs according to performance conditions of each of the target PMs includes: And determining that the first n-1 target PM is the slave PM and the rest target PM is the non-operation PM when the performance is ordered from good to bad, wherein n is a preset positive integer greater than or equal to 2.
  11. 11. The task processing method according to claim 9, wherein determining the slave PM and the non-operational PM from each of the target PMs according to performance conditions of each of the target PMs includes: and determining each target PM of which the performance condition meets a preset performance threshold as a slave PM, and determining each target PM of which the performance condition does not meet the preset performance threshold as a non-operation PM.
  12. 12. A task processing method according to any one of claims 9-11, characterized in that for any one task PM the performance situation of the task PM is determined on the basis of the computational load of the task PM and/or the network congestion situation.
  13. 13. The task processing method according to claim 2, wherein before sending the read message to each target PM in the task PMs, the method further comprises: Receiving the identification information of each slave PM transmitted by a master-slave dividing module and the corresponding relation between each non-operation PM and operation PM; and constructing a reading message of each target PM according to the corresponding relation between each non-operation PM and the operation PM.
  14. 14. The method of claim 13, wherein the master-slave partition module is an intermediate module coupled between an originating processor and an originating PM, the originating processor is a processor that initiates the aggregate operation task, and the originating PM is a PM coupled to the originating processor.
  15. 15. The method of claim 13, wherein the master-slave partition module is an initiating PM coupled to an initiating processor, and wherein the initiating processor is a processor that initiates the aggregate operation task.
  16. 16. The method of any one of claims 1-11, 13-15, wherein the master PM is an initiating PM connected to an initiating processor, and wherein the initiating processor is a processor that initiates the aggregate operation task.
  17. 17. The task processing method according to claim 2 to 11, 13 to 15, characterized in that, The reading message sent to the slave PM comprises identification information of the master PM, first indication information used for indicating that the PM is the slave PM, and identification information of each target PM corresponding to the slave PM; the reading message sent to the non-operation PM comprises identification information of the operation PM corresponding to the non-operation PM.
  18. 18. The task processing method according to claim 17, wherein the first instruction information is identification information of the slave PM.
  19. 19. The task processing method according to any one of claims 1 to 11 and 13 to 15, wherein at least part of the tasks PM are located in different Switch chips, wherein the different Switch chips are communicatively connected to each other.
  20. 20. A task processing method is characterized by being applied to a target proxy module PM in a Switch chip, wherein the Switch chip comprises a plurality of tasks PM, the target PM is any task PM except a main PM in the plurality of tasks PM, and the method comprises the following steps: In response to receiving a read message, determining whether the read message is a slave PM according to the read message; in the case of the slave PM, acquiring third task data required by the aggregation operation task from the connected processor; Receiving second task data transmitted by a non-operation PM corresponding to the second task data, and performing aggregation operation on the third task data and the second task data to obtain a second aggregation operation result; and sending the second polymerization operation result to the main PM appointed in the reading message.

Description

Task processing method and related product Technical Field The application relates to the technical field of distributed computing networks, in particular to a task processing method, a Switch chip, a distributed computing network, an electronic device and electronic equipment. Background To meet the increasing demand for computing power, there are distributed computing networks currently in existence in which multiple processors are connected through Switch chips, and greater computing power is provided through coordination of the multiple processors. In order to improve the efficiency of the distributed computing network and release the potential computing power of the processor cluster in the distributed computing network to the maximum extent, there is a paradigm that the network computing power is integrated in a Switch chip, the data packets in transmission are aggregated, and the result is directly fed back to the processor, so that the transmission times are reduced, the computing power of the processor is released, and the overall efficiency of the distributed computing network is prompted. However, this paradigm also introduces new system-level challenges, with the foremost being the contention and scheduling of computing resources. When a processor initiates a large-scale task in a short time, massive aggregate operation data is flushed to a PM (Proxy Module) connected with the processor in the Switch chip, so that the aggregate operation request of the Switch port is proxied and the aggregate operation of the data is completed, and the data participating in the aggregate operation has to be queued up at the PM to be processed, and unpredictable delay jitter is introduced. While other PMs may have spare computing resources. That is, the problems of concentrated computation hot spots, unbalanced resource utilization and serious queuing delay easily occur in the prior paradigm Switch chip. Disclosure of Invention The embodiment of the application aims to provide a task processing method, a Switch chip, a distributed computing network, an electronic device and electronic equipment, which are used for solving the problems of concentrated computing hot spots, unbalanced resource utilization and serious queuing delay which are easy to occur in the Switch chip to a certain extent. The embodiment of the application provides a task processing method which is applied to a main proxy module PM in a Switch chip, wherein the Switch chip comprises a plurality of tasks PM, the main PM is any one of the tasks PM, and the method comprises the following steps: The method comprises the steps of responding to a request of receiving an aggregation operation task, acquiring first task data required by the aggregation operation task from a processor connected with a main PM, wherein the task PM comprises operation PM and non-operation PM, the operation PM is the task PM for executing the aggregation operation in the task PM, the operation PM comprises the main PM and at least one sub PM, the non-operation PM is the task PM for not executing the aggregation operation in the task PM, receiving second task data transmitted by the non-operation PM corresponding to the main PM under the condition that the main PM has the corresponding non-operation PM, and conducting aggregation operation on the first task data and the second task data to obtain a first aggregation operation result, receiving the second aggregation operation result transmitted by the sub PM, conducting aggregation operation on the first aggregation operation result and the second aggregation operation result to obtain a target aggregation operation result, and sending the target aggregation operation result to a target processor, wherein the target processor is a processor using the aggregation operation result of the aggregation operation. In the above-described implementation, by selecting each operation PM (master PM and slave PM) from among the tasks PM related to the aggregate operation task, each operation PM is responsible for the aggregate operation of a part of the task data, and the master PM obtains the final target aggregate operation result. Therefore, the whole aggregate operation task is distributed to a plurality of operation PM, and the problems of concentrated calculation hot spots and unbalanced resource utilization are relieved to a certain extent. Meanwhile, the whole aggregate operation task is distributed to a plurality of operation PMs, so that the calculated amount of each PM participating in calculation is smaller than that of a PM which performs calculation of the whole aggregate operation task, the calculation efficiency on a single PM is higher, and even if queuing occurs, the queuing delay can be shorter. Optionally, before receiving the second task data transmitted from the non-operation PM corresponding to the main PM and receiving the second aggregation operation result transmitted from the PM, the method further comprises s