CN-122019068-A - Operator execution method, device, equipment and medium

Abstract

The invention discloses an operator execution method, an operator execution device, operator execution equipment and an operator execution medium. The method comprises the steps of determining an operator queue and a file cache matched with a new operator stream when a new operator stream is created in a deep learning processor, storing each operator in the operator queue, storing an operator binary file in the file cache, detecting whether a target operator stream of an operator stream calling request meets a packing execution condition when an operator stream calling request is acquired, packing operators in the operator queue matched with the target operator stream after the condition is determined to be met, obtaining an operator group, and sending the operator group to a driving module. According to the embodiment of the invention, after the operator flow is determined to be in accordance with the packing execution condition, each operator in the operator flow is packed into one operator group, each operator in the operator flow is sent to the driving module at one time, the interaction times of the driving module are effectively reduced, and the time consumption of the operator executing process is reduced.

Inventors

• Zheng Hanxun
• ZHAO FENG
• LI YANG
• YANG TIANQI
• ZHENG WENTAO

Assignees

• 中昊芯英(杭州)科技有限公司

Dates

Publication Date

20260512

Application Date

20251024

Claims (10)

1. 1. An operator execution method, comprising: When a new operator flow is created in the deep learning processor, determining an operator queue and a file cache which are matched with the new operator flow, storing each operator in the new operator flow into the operator queue, and storing an operator binary file of the new operator flow into the file cache; When an operator flow calling request is obtained, detecting whether a target operator flow of the operator flow calling request accords with a packing execution condition, packing operators in an operator queue matched with the target operator flow after determining that the target operator flow accords with the packing execution condition to obtain an operator group, and sending the operator group to a driving module so that the driving module executes each operator in the target operator flow.
2. 2. The operator execution method according to claim 1, wherein determining an operator queue and a file cache that match the new operator flow, storing each operator in the new operator flow into the operator queue, storing an operator binary file of the new operator flow into the file cache, comprises: Creating an operator queue matched with the new operator flow according to the length of the dynamic queue, and storing each operator in the new operator flow into the operator queue; And selecting one available buffer from all available buffers as a file buffer matched with the new operator flow, and storing an operator binary file of the new operator flow into the file buffer.
3. 3. The operator execution method according to claim 1, wherein detecting whether the target operator flow of the operator flow call request meets a packing execution condition comprises: Inquiring whether an operator binary file of the target operator flow of the operator flow calling request is cached in a file cache matched with the target operator flow; If the operator binary file of the target operator flow is determined to be cached in a file cache matched with the target operator flow, checking whether an operator queue matched with the target operator flow is fully loaded; and if the operator queue matched with the target operator flow is determined to be fully loaded, determining that the target operator flow accords with a packaging execution condition.
4. 4. The operator execution method according to claim 3, wherein detecting whether the target operator flow of the operator flow call request meets a packing execution condition, further comprises: If the operator binary file of the target operator flow is determined not to be cached in the file cache matched with the target operator flow, determining that the target operator flow does not accord with the packaging execution condition; And if the operator queue matched with the target operator flow is determined to be not fully loaded, determining that the target operator flow does not accord with the packaging execution condition.
5. 5. The operator execution method according to claim 1, further comprising, after transmitting the operator group to a driving module: Generating call record information according to the configuration information of the target operator flow and the operator parameters in the operator flow call request, and storing the call record information into an operator queue matched with the target operator flow.
6. 6. The operator performing method according to claim 1, further comprising: When a plurality of operator flow calling requests are acquired, determining the calling sequence of each operator flow calling request, sequentially detecting whether the target operator flow of each operator flow calling request accords with a packing execution condition according to the calling sequence of each target operator flow, packing operators in an operator queue matched with the target operator flow after determining that the target operator flow accords with the packing execution condition to obtain an operator group, and sending the operator group to a driving module so that the driving module executes each operator in the target operator flow.
7. 7. The operator performing method according to claim 1, further comprising: And adjusting the length of the dynamic queue in the running process of the target deep learning model.
8. 8. An operator execution apparatus, comprising: An operator storage unit, configured to determine an operator queue and a file cache that are matched with a new operator stream when the new operator stream is created in the deep learning processor is detected, store each operator in the new operator stream into the operator queue, and store an operator binary file of the new operator stream into the file cache; the operator execution unit is used for detecting whether a target operator flow of an operator flow call request accords with a packing execution condition or not when the operator flow call request is acquired, packing operators in an operator queue matched with the target operator flow after determining that the target operator flow accords with the packing execution condition to obtain an operator group, and sending the operator group to the driving module so that the driving module executes each operator in the target operator flow.
9. 9. An electronic device, the electronic device comprising: At least one processor; and a memory communicatively coupled to the at least one processor; Wherein the memory stores a computer program for execution by the at least one processor to enable the at least one processor to perform the operator performing method of any one of claims 1-7.
10. 10. A computer readable storage medium storing computer instructions for causing a processor to implement the operator execution method of any one of claims 1-7 when executed.

Description

Operator execution method, device, equipment and medium Technical Field The present invention relates to the field of computer technologies, and in particular, to a method, an apparatus, a device, and a medium for executing an operator. Background With the development of artificial intelligence technology, more and more enterprises begin to perform image processing, text processing or voice processing through deep learning models. When a specified computing task needs to be completed in the running process of the deep learning model, a deep learning processor is generally called to execute each operator in an operator stream corresponding to the specified computing task, so that the specified computing task is completed. In the related technology, the common operator execution scheme is that each operator in the operator flow corresponding to the appointed calculation task is respectively sent to a driving module of the deep learning processor, and each operator in the operator flow corresponding to the appointed calculation task is executed through the driving module. The operator execution scheme in the related technology needs to transmit a plurality of operators respectively, and the interaction times of the driving module are more, so that the time consumption of the operator execution process is longer, and the performance of the deep learning model is reduced. Disclosure of Invention The invention provides an operator execution method, an operator execution device, operator execution equipment and an operator execution medium, which are used for solving the problems that the operator execution scheme in the related technology needs to transmit a plurality of operators respectively, the interaction times of a driving module are more, the time consumption of the operator execution process is longer, and the performance of a deep learning model is reduced. According to an aspect of the present invention, there is provided an operator execution method including: When a new operator flow is created in the deep learning processor, determining an operator queue and a file cache which are matched with the new operator flow, storing each operator in the new operator flow into the operator queue, and storing an operator binary file of the new operator flow into the file cache; When an operator flow calling request is obtained, detecting whether a target operator flow of the operator flow calling request accords with a packing execution condition, packing operators in an operator queue matched with the target operator flow after determining that the target operator flow accords with the packing execution condition to obtain an operator group, and sending the operator group to a driving module so that the driving module executes each operator in the target operator flow. According to another aspect of the present invention, there is provided an operator performing apparatus including: An operator storage unit, configured to determine an operator queue and a file cache that are matched with a new operator stream when the new operator stream is created in the deep learning processor is detected, store each operator in the new operator stream into the operator queue, and store an operator binary file of the new operator stream into the file cache; the operator execution unit is used for detecting whether a target operator flow of an operator flow call request accords with a packing execution condition or not when the operator flow call request is acquired, packing operators in an operator queue matched with the target operator flow after determining that the target operator flow accords with the packing execution condition to obtain an operator group, and sending the operator group to the driving module so that the driving module executes each operator in the target operator flow. According to another aspect of the present invention, there is provided an electronic apparatus including: At least one processor; and a memory communicatively coupled to the at least one processor; Wherein the memory stores a computer program to be executed by the at least one processor, so that the at least one processor can execute the operator execution method according to any embodiment of the present invention. According to another aspect of the present invention, there is provided a computer readable storage medium storing computer instructions for causing a processor to implement the operator execution method according to any one of the embodiments of the present invention. According to another aspect of the invention there is provided a computer program product comprising a computer program which, when executed by a processor, implements the operator execution method according to any of the embodiments of the invention. According to the technical scheme, when a deep learning processor is detected to create a new operator flow, an operator queue and a file cache matched with the new operator flow are determined, each operator in t