CN-114090363-B - Test method of reasoning engine, electronic equipment and storage medium

CN114090363BCN 114090363 BCN114090363 BCN 114090363BCN-114090363-B

Abstract

The application discloses a test method of an inference engine, electronic equipment and a storage medium, which can test the performance of the inference engine on a target hardware accelerator. The test method of the inference engine comprises the steps that an electronic device receives input test resources and identification of a target hardware accelerator, when the fact that the target hardware accelerator is in an available state is determined based on the identification of the target hardware accelerator, a processor of the electronic device creates a test task for the inference engine and distributes the test task and the test resources to the target hardware accelerator, and when the fact that the target hardware accelerator executes the test task based on the test resources is determined to be completed, the processor of the electronic device outputs test results for the inference engine, wherein the test results at least comprise actual total duration of execution of the test task by the target hardware accelerator.

Inventors

LIANG XIUJUAN

Assignees

展讯通信（上海）有限公司
展讯通信（上海）有限公司

Dates

Publication Date: 20260421
Application Date: 20211201
Priority Date: 20211201

Claims (20)

1. A method of testing an inference engine, characterized by applying an electronic device provided with a processor and at least one hardware accelerator, the processor running the inference engine, the method comprising: the electronic equipment receives input test resources and identification of a target hardware accelerator, wherein the test resources comprise test data, a designated test model type, a designated test network and a designated model application type; When it is determined that the electronic device is configured with the target hardware accelerator based on the identification of the target hardware accelerator and the target hardware accelerator is not currently executing other tasks, the processor of the electronic device creates a test task for the inference engine according to the test resource, the test task containing instructions that cause the target hardware accelerator to load and execute the test resource, and allocates the test task and the test resource to the target hardware accelerator; and after determining that the target hardware accelerator executes the test task based on the test resource is completed, the processor of the electronic device outputs a test result aiming at the reasoning engine, wherein the test result at least comprises the actual total duration of the target hardware accelerator executing the test task based on the reasoning engine.
2. The method of claim 1, wherein the test resources include at least a test data type and shape information of the test data, the method further comprising, prior to assigning the test resources to the target hardware accelerator: When the test data type is determined to be an image, the processor of the electronic device randomly generates the test data based on the shape information.
3. The method of claim 2, wherein the shape information includes a target length, a target width, and a target number of channels, wherein the processor of the electronic device randomly generating the test data based on the shape information comprises: The processor of the electronic device randomly generates a test image based on the target length, the target width, and the target channel number.
4. The method of claim 1, wherein the test resources include at least a test data type and shape information of the test data, the method further comprising, prior to assigning the test resources to the target hardware accelerator: when the test data type is determined to be audio, the processor of the electronic device randomly generates the test data based on shape information of the test data.
5. The method of claim 4, wherein the shape information comprises a target time length, a target amplitude, and a target frequency, wherein the processor of the electronic device randomly generating the test data based on the shape information of the test data comprises: the processor of the electronic device generates test audio based on the target duration, the target amplitude, and the target frequency.
6. The method of claim 1, wherein the test results further comprise a first duration for loading the test resources, a second duration for compiling the test resources, and a third duration for reasoning about task results for the test tasks based on the reasoning engine executing the test tasks.
7. The method of claim 1, wherein the test results further comprise task results of the target hardware accelerator performing the test task based on the test resource.
8. The method of claim 1, wherein the test resource comprises a target total length of time the target hardware accelerator performs the test task based on the test resource, and wherein upon determining that the target hardware accelerator performs the test task based on the test resource is complete, the method further comprises: And if the actual total time length is greater than the target total time length, outputting prompt information by a processor of the electronic equipment, wherein the prompt information is used for prompting the optimization of the target hardware accelerator and/or the reasoning engine.
9. The method according to claim 1, wherein the method further comprises: the electronic equipment receives the input test resources and the identifications of at least two target hardware accelerators; When the electronic equipment is configured with the at least two target hardware accelerators based on the identification of the at least two target hardware accelerators, and the at least two target hardware accelerators do not currently execute other tasks, the processor of the electronic equipment creates at least two test tasks aiming at the reasoning engine based on the test resources, wherein each test task comprises an instruction for loading and executing the test resources corresponding to the target hardware accelerator, and the test resources and each test task in the at least two test tasks are respectively distributed to each target hardware accelerator of the at least two target hardware accelerators; After determining that each target hardware accelerator executes a corresponding test task based on the test resource, the processor of the electronic device outputs at least two test results aiming at the reasoning engine, wherein each test result in the at least two test results at least comprises the actual total duration of each target hardware accelerator executing the corresponding test task based on the reasoning engine.
10. An electronic device provided with at least one hardware accelerator, the electronic device running an inference engine, the electronic device comprising: The system comprises a receiving unit, a processing unit and a processing unit, wherein the receiving unit is used for receiving input test resources and identification of a target hardware accelerator, and the test resources comprise test data, specified test model types, specified test networks and specified model application types; A processing unit, configured to create a test task for the inference engine according to the test resource when it is determined that the electronic device is configured with the target hardware accelerator based on the identification of the target hardware accelerator and the target hardware accelerator is not currently executing other tasks, the test task including an instruction to cause the target hardware accelerator to load and execute the test resource, and allocate the test task and the test resource to the target hardware accelerator; And the output unit is used for outputting a test result aiming at the reasoning engine after determining that the target hardware accelerator executes the test task based on the test resource, wherein the test result at least comprises the actual total duration of the target hardware accelerator executing the test task based on the reasoning engine.
11. The electronic device of claim 10, wherein the test resources include at least a test data type and shape information of the test data, the electronic device further comprising: and the first generation unit is used for randomly generating the test data based on the shape information when the test data type is determined to be an image.
12. The electronic device of claim 11, wherein the shape information includes a target length, a target width, and a target number of channels, and wherein the first generating unit is specifically configured to: And randomly generating a test image based on the target length, the target width and the target channel number.
13. The electronic device of claim 10, wherein the test resources include at least a test data type and shape information of the test data, the electronic device further comprising: And the second generation unit is used for randomly generating the test data based on the shape information of the test data when the test data type is determined to be audio.
14. The electronic device of claim 13, wherein the shape information includes a target duration, a target amplitude, and a target frequency, and wherein the second generating unit is specifically configured to: generating test audio based on the target duration, the target amplitude and the target frequency.
15. The electronic device of claim 10, wherein the test results further comprise a first duration for loading the test resources, a second duration for compiling the test resources, and a third duration for reasoning about task results for the test tasks based on the reasoning engine executing the test tasks.
16. The electronic device of claim 10, wherein the test results further comprise task results of the target hardware accelerator performing the test task based on the test resource.
17. The electronic device of claim 10, wherein the test resource comprises a target total duration for the target hardware accelerator to perform the test task based on the test resource, the output unit further to: and if the actual total time length is greater than the target total time length, outputting prompt information, wherein the prompt information is used for prompting the optimization of the target hardware accelerator and/or the inference engine.
18. The electronic device of claim 10, wherein the receiving unit is further configured to: receiving input identifiers of the test resources and at least two target hardware accelerators; the processing unit is further configured to: When it is determined that the electronic device is configured with the at least two target hardware accelerators based on the identification of the at least two target hardware accelerators and no other tasks are currently executed by the at least two target hardware accelerators, creating at least two test tasks for the inference engine based on the test resources, each test task including instructions for loading and executing the test resources corresponding to the target hardware accelerator, and respectively allocating the test resources and each of the at least two test tasks to each of the at least two target hardware accelerators; The output unit is further configured to: And after determining that each target hardware accelerator executes the corresponding test task based on the test resource, outputting at least two test results aiming at the reasoning engine, wherein each test result in the at least two test results at least comprises the actual total duration of each target hardware accelerator executing the corresponding test task based on the reasoning engine.
19. An electronic device is characterized by comprising a data receiving interface, a processor and at least one hardware accelerator, wherein the processor is respectively and electrically connected with the data receiving interface and the at least one hardware accelerator, and an inference engine is operated in the processor, The data receiving interface is used for receiving input test resources and identification of a target hardware accelerator, wherein the test resources comprise test data, specified test model types, specified test networks and specified model application types; the processor is used for creating a test task for the reasoning engine according to the test resource when the electronic device is configured with the target hardware accelerator based on the identification of the target hardware accelerator and the target hardware accelerator does not currently execute other tasks, wherein the test task comprises instructions for enabling the target hardware accelerator to load and execute the test resource, and the test task and the test resource are distributed to the target hardware accelerator; The processor is further configured to: and after determining that the target hardware accelerator executes the test task based on the test resource is completed, the processor of the electronic device outputs a test result aiming at the reasoning engine, wherein the test result at least comprises the actual total duration of the target hardware accelerator executing the test task based on the reasoning engine.
20. A computer readable storage medium, on which a computer program is stored, characterized in that the computer program, when being executed by a processor, implements the steps of the method according to any of claims 1-9.

Description

Test method of reasoning engine, electronic equipment and storage medium [ Field of technology ] The embodiment of the application relates to the technical field of testing, in particular to a test method of an inference engine, electronic equipment and a storage medium. [ Background Art ] Along with the continuous improvement of the computing power of the terminal and the rapid development of deep learning, particularly the continuous maturation of a small network model, the reasoning prediction originally executed at the cloud can be transferred to the terminal for doing. The terminal intelligence is arranged at the side of the terminal to run an artificial intelligence (ARTIFICIAL INTELLIGENCE, AI) algorithm, and compared with the server-side intelligence, the terminal intelligence has the advantages of low delay, data privacy consideration, cloud resource saving and the like. The main functions of the inference engine cover loading, compiling optimization and reasoning of the terminal side deep neural network model, while in the prior art, the inference engine is only run in a processor by default, so that performance test of the inference engine cannot be performed on other hardware accelerators except the processor. [ Invention ] The embodiment of the application provides a test method of an inference engine, electronic equipment and a storage medium, which can test the performance of the inference engine on a hardware accelerator except a processor. In a first aspect, an embodiment of the present application provides a method for testing an inference engine, where the method is applied to an electronic device, where the electronic device is provided with a processor and at least one hardware accelerator, and the processor runs the inference engine, and the method includes: the electronic equipment receives input test resources and identification of a target hardware accelerator; when the target hardware accelerator is determined to be in an available state based on the identification of the target hardware accelerator, the processor of the electronic device creates a test task for the inference engine and allocates the test task and the test resource to the target hardware accelerator; and after determining that the target hardware accelerator executes the test task based on the test resource is completed, the processor of the electronic device outputs a test result aiming at the reasoning engine, wherein the test result at least comprises the actual total duration of the target hardware accelerator executing the test task based on the reasoning engine. In the embodiment of the application, the processor running the reasoning engine can receive the input test resource and the identification of the target hardware accelerator, and then determine whether the target hardware accelerator is in an available state according to the identification of the target hardware accelerator. If the target hardware accelerator is in an available state, a test task for the inference engine is created, and the created test task and the test resource are distributed to the target hardware accelerator. After the target hardware accelerator completes the test task based on the test resource, the processor may output a test result for the inference engine, where the test result includes at least an actual total duration for the target hardware accelerator to execute the test task based on the inference engine. In the method, the processor can distribute the test task and the test resource aiming at the inference engine to the target hardware accelerator according to the target hardware accelerator appointed by the user, so that the test result of the inference engine by the target hardware accelerator can be output, and the targeted optimization of the performance of the target hardware accelerator and/or the inference engine is facilitated. Optionally, the test resource includes at least a test data type and shape information of the test data, and before the test resource is allocated to the target hardware accelerator, the method further includes: When the test data type is determined to be an image, the processor of the electronic device randomly generates the test data based on the shape information. In the embodiment of the application, when the test data type is the image, the test data meeting the test requirement can be randomly generated based on the shape information of the test data, so that a large amount of pre-processing of the test data is not needed manually in advance. Optionally, the shape information includes a target length, a target width, and a target channel number, and the processor of the electronic device randomly generating the test data based on the shape information includes: The processor of the electronic device randomly generates a test image based on the target length, the target width, and the target channel number. According to the embodiment of the application, the test image required by