CN-122019051-A - Automatic evaluation method and device for online programming examination, electronic equipment and storage medium

CN122019051ACN 122019051 ACN122019051 ACN 122019051ACN-122019051-A

Abstract

The application discloses an automatic evaluation method, device, electronic equipment and storage medium for online programming examination, which comprises the steps of pre-configuring a Docker mirror image and a language template rule respectively, associating the Docker mirror image corresponding to a target programming language with the corresponding language template rule, obtaining the corresponding language template rule, generating an execution template based on the corresponding language template rule and grammar components of an executable method set by a question-setting person, binding the execution template with a test case to form a programming test question unit, embedding codes into the execution template to generate a source program, and executing the source program in an isolation sandbox container started by the associated Docker mirror image based on the programming test question unit to generate an evaluation result. Therefore, a multi-language execution frame is automatically generated through the preset language template and the Docker mirror image, the test cases are bound to form a standardized test question unit, and the execution and performance monitoring are unified, so that accurate, fair and reusable automatic evaluation is realized.

Inventors

Yang Huankun
QIN SHA
LI TENG
HONG BINBIN
LIN ZHENGJIE
HUANG WU
Shi Bisong
GUO LIANGCAI
YANG TING
Wen Tingying

Assignees

深圳市海云天科技股份有限公司

Dates

Publication Date: 20260512
Application Date: 20260123

Claims (10)

1. An automatic evaluation method for online programming exams, which is characterized by comprising the following steps: pre-configuring corresponding Docker images and language template rules for each supported programming language respectively; Responding to a target programming language selected by a question provider, associating the Docker mirror image corresponding to the target programming language, acquiring a corresponding language template rule, and generating an execution template based on the corresponding language template rule and grammar components of an executable method set by the question provider; binding the execution template with at least one test case to form a programming test question unit capable of being automatically evaluated; embedding codes submitted by an examinee into the execution template to generate a source program; And executing the source program under the corresponding test case in the isolation sandbox container started by the associated Docker mirror image based on the programming test question unit to generate an evaluation result.
2. The automatic evaluation method of online programming exam according to claim 1, wherein the associating the Docker image corresponding to the target programming language and obtaining the corresponding language template rule in response to the target programming language selected by the question, and generating the execution template based on the corresponding language template rule and the syntax element of the executable method set by the question, comprises: Calling a dock mirror image corresponding to the target programming language, and loading a language processing tool and a runtime environment of the target programming language preloaded in the dock mirror image; retrieving language template rules matched with the target programming language, and extracting key configuration information based on the matched language template rules; taking the key configuration information and the main method name, the return type, the input parameter type and the parameter name set by the themes as inputs, and calling a method initialization algorithm corresponding to the target programming language; and dynamically constructing initial core codes conforming to the grammar rules of the target programming language according to the method initialization algorithm and the key configuration information, and storing the initial core codes as an execution template.
3. The automatic evaluation method of on-line programming exam of claim 2, wherein the key configuration information includes determining an extension of a source code file, constructing a compile command, constructing an execute command, and parsing syntax structure information for generating a method frame.
4. The automatic evaluation method of online programming exams according to claim 1, wherein the step of binding the execution template with at least one test case to form an automatically-evaluating programming test question unit comprises: receiving input parameters and expected output parameters of each test case in the form of character strings; receiving resource limiting parameters set by each test case; The input parameters, the expected output parameters and the resource limitation parameters are associated and packaged into test case data objects corresponding to each test case respectively; And binding the test case data object with the execution template to form the programming test question unit.
5. The method for automatically evaluating an online programming test according to claim 1, wherein the generating an evaluation result based on the programming test question unit by executing the source program under a corresponding test case in an isolated sandbox container started by an associated dock mirror image includes: starting a Docker mirror image associated with the programming test question unit, and creating an isolation sandbox container; Loading at least one bound test case from the programming test question unit, and acquiring input parameters, expected output parameters and resource limitation parameters of the bound test case; deploying the source program into the isolation sandbox container, and injecting the input parameters of the bound test cases into the source program; executing the source program under the constraint of the resource limiting parameters of the bound test case, monitoring the running process of the source program in real time, and capturing standard output content, actual CPU execution time and actual memory consumption; and generating the evaluation result based on the standard output content, the actual CPU execution time and the actual memory consumption.
6. The method for automatically evaluating an online programming test according to claim 5, wherein the generating the evaluation result based on the standard output content, the actual CPU execution time, and the actual memory consumption comprises: Judging whether the actual CPU execution time exceeds the maximum CPU execution time set in the bound test case or whether the actual memory consumption exceeds the maximum memory consumption set in the bound test case; if the actual CPU execution time exceeds the maximum CPU execution time or the actual memory consumption exceeds the maximum memory consumption, generating an evaluation result representing the resource overrun error; If the actual CPU execution time does not exceed the maximum CPU execution time and the actual memory consumption does not exceed the maximum memory consumption, comparing the standard output content with the expected output parameters in the bound test case character by character; And if the standard output content is completely consistent with the expected output parameter, generating an evaluation result which indicates correct, or/and if the standard output content is inconsistent with the expected output parameter, generating an evaluation result which indicates incorrect output.
7. The method for automatically evaluating an online programming test according to claim 1, wherein the generating an evaluation result based on the programming test question unit by executing the source program under a corresponding test case in an isolated sandbox container started by an associated dock mirror image includes: submitting an evaluation task to be executed to a task waiting queue, wherein the evaluation task comprises the programming test question unit and the source program; scheduling a work process by a plurality of sandboxes and concurrently acquiring an evaluation task from the task waiting queue; each sandbox dispatching work process respectively starts a Docker mirror image associated with a corresponding evaluation task, creates an isolated sandbox container, and dynamically allocates limited system resources based on the runtime requirements of a target programming language; and executing the source programs in parallel in the separately created isolated sandboxed containers, and generating corresponding evaluation results.
8. An on-line programming test automatic evaluation device, the device comprising: the configuration module is used for respectively pre-configuring corresponding Docker images and language template rules for each supported programming language; the generation module is used for responding to a target programming language selected by a question-setting person, associating the Docker mirror image corresponding to the target programming language, acquiring the corresponding language template rule, and generating an execution template based on the corresponding language template rule and grammar components of an executable method set by the question-setting person; The forming module is used for binding the execution template with at least one test case to form a programming test question unit capable of being automatically evaluated; the embedding module is used for embedding codes submitted by the examinee into the execution template to generate a source program; and the execution module is used for executing the source program under the corresponding test case in the isolation sandbox container started by the associated Docker mirror image based on the programming test question unit to generate an evaluation result.
9. An electronic device, comprising: one or more processors; a memory; One or more application programs, wherein the one or more application programs are stored in the memory and configured to be executed by the one or more processors, the one or more application programs configured to perform the on-line programming test automatic evaluation method of any one of claims 1-7.
10. A computer readable storage medium having stored therein program code which is callable by a processor to perform the on-line programming test automatic evaluation method according to any one of claims 1-7.

Description

Automatic evaluation method and device for online programming examination, electronic equipment and storage medium Technical Field The application relates to the technical field of automatic evaluation, in particular to an automatic evaluation method, an automatic evaluation device, electronic equipment and a storage medium for online programming examination. Background The current mainstream programming capability evaluation covers multiple dimensions of solving problems, algorithm design, coding implementation and the like, but still faces multiple technical bottlenecks in practical system implementation. Firstly, different programming languages have obvious differences in grammar structures, inlet specifications and input and output modes, so that a large number of language-related initial code frames are manually written when questions are raised, the input is complex and easy to be misplaced, secondly, test cases are usually tightly coupled with specific languages, cross-question or cross-language multiplexing is difficult, and the maintenance cost is high. And the existing platform commonly adopts a mode of 'unified manual appraising after submission'. The problems limit further improvement of accuracy and fairness of programming evaluation. Disclosure of Invention In view of the above problems, the application provides an automatic evaluation method and device for online programming examination, electronic equipment and storage medium, which can solve the above problems. The embodiment of the application provides an automatic evaluation method for online programming examination, which comprises the steps of pre-configuring corresponding Docker images and language template rules for each supported programming language respectively, responding to target programming languages selected by a question producer, associating the Docker images corresponding to the target programming languages, acquiring corresponding language template rules, generating an execution template based on the corresponding language template rules and grammar components of an executable method set by the question producer, binding the execution template with at least one test case to form an automatically-evaluated programming test question unit, embedding codes submitted by an examinee into the execution template to generate a source program, executing the source program in an isolated sandbox container started by the associated Docker images based on the programming test question unit, and generating an evaluation result. The embodiment of the application also provides an on-line programming examination automatic evaluation device which comprises a configuration module, a generation module, a forming module, an embedding module and an execution module, wherein the configuration module is used for respectively pre-configuring corresponding Docker images and language template rules for each supported programming language, the generation module is used for responding to a target programming language selected by a question producer, associating the Docker images corresponding to the target programming language, acquiring corresponding language template rules, generating an execution template based on the corresponding language template rules and grammar components of an executable method set by the question producer, the forming module is used for binding the execution template with at least one test case to form an automatically-evaluating programming test question unit, the embedding module is used for embedding codes submitted by an examinee into the execution template to generate a source program, and the execution module is used for executing the source program under the corresponding test case in an isolated sandbox container started by the associated Docker images based on the programming test question unit to generate an evaluation result. In a third aspect, the embodiment of the application further provides an electronic device, which comprises a processor, a memory and one or more application programs, wherein the one or more application programs are stored in the memory and are configured to be executed by the processor to realize the automatic evaluation method of the online programming test. In a fourth aspect, an embodiment of the present application further provides a computer readable storage medium, where a program code is stored in the computer readable storage medium, where the above-mentioned automatic evaluation method for online programming test is executed when the program code is executed by a processor. The technical scheme includes that corresponding Docker images and language template rules are pre-configured for each supported programming language respectively, a target programming language selected by a question producer is responded, the corresponding Docker images of the target programming language are associated, corresponding language template rules are obtained, an execution template is generated base