CN-115907013-B - Management system, method, equipment and medium for operator of artificial intelligent model

CN115907013BCN 115907013 BCN115907013 BCN 115907013BCN-115907013-B

Abstract

Provided are a management system for operators of an artificial intelligence model, a management method for operators of an artificial intelligence model, an electronic device, and a non-transitory storage medium. An operator manager module for an operator of an artificial intelligence model includes an operator container for storing a plurality of operators, each operator of the plurality of operators having a parameter, wherein the parameter includes at least a type of operation supported by the operator on the operator, a back-end platform type supported by the operator, an operator identification of the operator, and a front-end operator definition plug-in configured to be invoked in response to loading an operator having the parameter to provide the operator having the parameter from the operator container.

Inventors

Request for anonymity
Request for anonymity
Request for anonymity
Request for anonymity
Request for anonymity

Assignees

上海壁仞智能科技有限公司

Dates

Publication Date: 20260508
Application Date: 20220930

Claims (14)

1. A system for managing operators of an artificial intelligence model, comprising: an operator manager module, comprising an operator container for storing a plurality of operators, each operator of the plurality of operators having a parameter, wherein the parameter comprises at least a type of operation supported by the operator on the operator, a back-end platform type supported by the operator, an operator identification of the operator; a front-end operator definition plug-in configured to be invoked in response to loading an operator having the parameter to provide the operator having the parameter from the operator container; a front-end operator optimization definition plug-in configured to be invoked in response to front-end optimization of an operator having the parameters to provide functionality for front-end optimization of an operator having the parameters, Wherein the front-end operator optimization definition plug-in is inserted by: responsive to an operator having the parameter being registered into the operator container, determining whether a front-end optimization strategy needs to be added for the operator having the parameter; Generating a front-end operator optimization definition plug-in if it is determined that a front-end optimization strategy needs to be added for an operator with the parameters; The front-end operator optimization definition plug-in is added to the system.
2. The system of claim 1, wherein the operator manager module further has at least one of a lookup interface, a registration interface, and a loading interface, Wherein the lookup interface is configured to lookup an operator having the parameter in the operator container based on the parameter; the registration interface is configured to register an operator and its parameters into the operator container; The loading interface is configured to load the front-end operator definition plug-in from a configurable path based on the path and plug-in matching rules.
3. The system of claim 1, further comprising an operator transformation plug-in configured to be invoked in response to transforming an operator having the parameters to provide functionality to transform the operator having the parameters to perform graph optimization of a computational graph including the operator, wherein the transforming includes one or more of operator fusion, operator splitting, operator replacement, operator elimination, Wherein the operator conversion plug-in is inserted by: determining, in response to an operator having the parameter being registered in the operator container, whether a conversion policy needs to be added for the operator having the parameter; Generating the operator conversion plugin if it is determined that a conversion strategy needs to be added for the operator with the parameters; The operator conversion plug-in is added to the system.
4. The system of claim 3, further comprising an operator compilation plug-in configured to be invoked in response to compiling a new conversion operator in the operator or optimized computational graph having the parameter to provide a function of compiling a new conversion operator in the operator or optimized computational graph having the parameter, Wherein the operator compilation plug-in is inserted by: in response to an operator having the parameter being registered in the operator container, determining whether a compilation plug-in set is capable of compiling the operator having the parameter or a new conversion operator in the optimized computational graph; generating the operator compilation plug-in if it is determined that the compilation plug-in set is not capable of compiling the operator with the parameters or a new conversion operator in the optimized computational graph; the operator compilation plug-in is added to the system.
5. The system of claim 4, further comprising an operator linking plug-in configured to be invoked in response to linking a new conversion operator in the operator or optimized computational graph having the parameter to provide a function of linking a new conversion operator in the operator or optimized computational graph having the parameter, Wherein the operator linking plug-in is inserted by: In response to an operator having the parameter being registered in the operator container, determining whether a set of link plug-ins is capable of linking the operator having the parameter or a new conversion operator in the optimized computational graph; generating an operator link plug-in if it is determined that the set of link plug-ins is not capable of linking the operator with the parameter or a new conversion operator in the optimized computational graph; The operator link plug-in is added to the system.
6. The system of claim 5, further comprising an operator execution plug-in configured to be invoked in response to execution of an operator having the parameter to provide a function of executing the operator having the parameter, Wherein the operator execution plug-in is inserted by: In response to an operator having the parameter being registered in the operator container, determining whether an operator execution set is capable of executing the operator having the parameter or a new conversion operator in the optimized computational graph; Determining that the operator execution set cannot execute the operator with the parameters or the new conversion operator in the optimized calculation graph, and generating an operator execution plug-in; The operator execution plug-in is added to the system.
7. A method of managing operators of an artificial intelligence model, comprising: Providing an operator manager module, which comprises an operator container, wherein the operator container is used for storing a plurality of operators, each operator of the plurality of operators has parameters, and the parameters at least comprise the type of operation supported by the operator and performed on the operator, the type of a back-end platform supported by the operator and the operator identifier of the operator; Providing a front-end operator definition plug-in configured to be invoked in response to loading an operator having the parameter to provide the operator having the parameter from the operator container; Providing a front-end operator optimization definition plug-in configured to be invoked in response to front-end optimization of an operator having the parameters to provide functionality for front-end optimization of an operator having the parameters, Wherein the providing a front end operator optimization definition plug-in includes: responsive to an operator having the parameter being registered into the operator container, determining whether a front-end optimization strategy needs to be added for the operator having the parameter; Generating a front-end operator optimization definition plug-in if it is determined that a front-end optimization strategy needs to be added for an operator with the parameters; And adding the front-end operator optimization definition plug-in into a management system of operators of the artificial intelligent model.
8. The method of claim 7, further comprising providing at least one of a lookup interface, a registration interface, and a loading interface of the operator manager module, Wherein the lookup interface is configured to lookup an operator having the parameter in the operator container based on the parameter; the registration interface is configured to register an operator and its parameters into the operator container; The loading interface is configured to load the front-end operator definition plug-in from a configurable path based on the path and plug-in matching rules.
9. The method of claim 7, further comprising providing an operator conversion plug-in configured to be invoked in response to converting an operator having the parameter to provide functionality to convert the operator having the parameter to perform graph optimization of a computational graph including the operator, wherein the converting includes one or more of operator fusion, operator splitting, operator replacement, operator elimination, Wherein the providing an operator conversion plug-in includes: determining, in response to an operator having the parameter being registered in the operator container, whether a conversion policy needs to be added for the operator having the parameter; Generating the operator conversion plugin if it is determined that a conversion strategy needs to be added for the operator with the parameters; The operator conversion plug-in is added to the system.
10. The method of claim 9, further comprising providing an operator compilation plug-in configured to be invoked in response to compiling a new conversion operator in the operator or optimized computational graph having the parameter to provide a function of compiling a new conversion operator in the operator or optimized computational graph having the parameter, Wherein the providing an operator compilation plug-in comprises: in response to an operator having the parameter being registered in the operator container, determining whether a compilation plug-in set is capable of compiling the operator having the parameter or a new conversion operator in the optimized computational graph; generating the operator compilation plug-in if it is determined that the compilation plug-in set is not capable of compiling the operator with the parameters or a new conversion operator in the optimized computational graph; the operator compilation plug-in is added to the system.
11. The method of claim 10, further comprising providing an operator linking plug-in configured to be invoked in response to linking a new conversion operator in the operator or optimized computational graph having the parameter to provide a function of linking a new conversion operator in the operator or optimized computational graph having the parameter, Wherein the providing an operator link plug-in comprises: In response to an operator having the parameter being registered in the operator container, determining whether a set of link plug-ins is capable of linking the operator having the parameter or a new conversion operator in the optimized computational graph; generating an operator link plug-in if it is determined that the set of link plug-ins is not capable of linking the operator with the parameter or a new conversion operator in the optimized computational graph; The operator link plug-in is added to the system.
12. The method of claim 11, further comprising providing an operator execution plug-in configured to be invoked in response to execution of an operator having the parameter to provide a function of executing the operator having the parameter, Wherein the providing an operator execution plug-in includes: In response to an operator having the parameter being registered in the operator container, determining whether an operator execution set is capable of executing the operator having the parameter or a new conversion operator in the optimized computational graph; Determining that the operator execution set cannot execute the operator with the parameters or the new conversion operator in the optimized calculation graph, and generating an operator execution plug-in; The operator execution plug-in is added to the system.
13. An electronic device, comprising: A memory for storing instructions; a processor for reading instructions in said memory and performing the method of any of claims 7-12.
14. A non-transitory storage medium having instructions stored thereon, Wherein the instructions, when read by a processor, cause the processor to perform the method of any of claims 7-12.

Description

Management system, method, equipment and medium for operator of artificial intelligent model Technical Field The present application relates to the field of artificial intelligence, and more particularly to a management system of operators of an artificial intelligence model, a management method of operators of an artificial intelligence model, an electronic device, and a non-transitory storage medium. Background Artificial intelligence models are typically neural network models used to make inferential predictions, such as image inference models, speech inference models, etc. An image with animals is input, and a label is directly output through reasoning of an artificial intelligence model to indicate what the object in the image is, such as a dog or a cat. The neural network model may be a convolutional neural network model or the like. The operations of the artificial intelligence model may be implemented by operators in a computational graph (computation graph). The operation of the artificial intelligent model can be converted into the calculation graph corresponding to the artificial intelligent model through a preset calculation graph conversion rule. A computational graph is a multi-graph structure used to represent the computational tasks and data flow processes of an artificial intelligence model. The operator refers to various operations performed on tensors of all layers in the artificial intelligent model, for example, convolution operations performed on input data of the artificial intelligent model by a convolution layer of the artificial intelligent model are convolution operators. Tensors are understood to be multidimensional arrays, which may have any number of dimensions, and different tensors may be represented by different data types and shapes. An operator is understood to be a computational function whose input and output are tensors. The computational graph of the artificial intelligence model may include operators that perform a wide variety of operations on tensors, such as convolution operators, full join operators, pooling operators, transposition operators, sobel operators, reshape operators, transpose operators, and so forth. An artificial intelligence inference engine is a system component that applies logical rules to a knowledge base to infer new information. This process will be repeated as every new fact in the knowledge base may trigger other rules in the inference engine. The inference engine works mainly in one of two modes, special rules or facts, forward link and backward link. The forward link starts with a known fact and then asserts a new fact. The backward link starts from the target and works backward to determine which facts must be asserted to achieve the target. The inference engine loops through three sequential steps, matching rules, selecting rules, and executing rules. This looping may continue until no new rules can be matched. Along with the continuous improvement of the computing power of the mobile phone and the rapid development of deep learning, particularly the continuous maturation of a small network model, the reasoning prediction originally executed at the cloud can be transferred to the terminal for doing. The artificial intelligent AI algorithm is operated at the side of the terminal, so that the method has the advantages of low delay, data privacy consideration, cloud resource saving and the like. Various inference engine frameworks exist, such as ONNX Runtime, mobile artificial intelligence (Mobile Neural Network, MNN), tensorRT, etc. For example, the MNN is a lightweight deep artificial intelligence reasoning engine, and the deep artificial intelligence model is loaded on the end side to make reasoning prediction, so that the system is suitable for servers/personal computers/mobile phones/embedded various devices. The various inference engine frameworks described above all support a degree of custom operators. Custom operators refer to operators written to run specific operations according to the actual needs of the user. There remains a need for an operator management framework to efficiently manage and support custom operators. Disclosure of Invention According to one aspect of the application, an operator management system of an artificial intelligence model is provided, comprising an operator manager module comprising an operator container for storing a plurality of operators, each operator of the plurality of operators having a parameter, wherein the parameter comprises at least a type of operation supported by the operator on the operator, a back-end platform type supported by the operator, an operator identification of the operator, and a front-end operator definition plug-in configured to be invoked in response to loading an operator having the parameter to provide the operator having the parameter from the operator container. According to another aspect of the application, there is provided a method of operator management of an art