CN-121996537-A - FPGA test case automatic generation system and method based on large language model

CN121996537ACN 121996537 ACN121996537 ACN 121996537ACN-121996537-A

Abstract

The invention discloses an automatic generation system and method of an FPGA test case based on a large language model, wherein the automatic generation system and method comprises a requirement input module, a requirement structuring module, a large language model case generation module, a case confirmation module, a verification execution module and a closed loop optimization module. The large language model is guided to disassemble the complex requirements into unrewritable test points through prompt word engineering (Prompt Engineering), so that the test is ensured to cover all logic branches. The large language model can dynamically generate various test cases based on the test points, including normal flow, abnormal flow and edge scene. By combining with the FPGA design specification, the history error cases and other externally hung knowledge bases, the large language model can generate test cases which are closer to actual hardware behaviors.

Inventors

SUN YIDAN
WANG HONGWEI
LI YOU
TANG LIU
ZHANG ZHENYUAN
ZHAO MIN
ZHOU YUANYUAN
SUN YUMING

Assignees

北京轩宇信息技术有限公司

Dates

Publication Date: 20260508
Application Date: 20251205

Claims (9)

1. The FPGA test case automatic generation system based on the large language model is characterized by comprising a requirement input module, a requirement structuring module, a large language model case generation module, a case confirmation module, a verification execution module and a closed loop optimization module; the demand input module is used for receiving the FPGA design related basic information and the marking information provided by the engineer; The demand structuring module is used for disassembling input demands and constructing a scene library to form an atomization test key point; the large language model case generation module is used for receiving the atomization test key points and the scene library and generating executable test cases meeting the requirements; the use case confirmation module is used for an engineer to check the test use case and feed back the use cases which do not pass the check and the modification suggestions to the large language model use case generation module; The verification execution module is used for integrating the confirmed test cases into an FPGA verification environment, executing the cases and analyzing results; The closed loop optimization module is used for feeding back a test result to the large language model use case generation module, and the use case iterative optimization is realized through reinforcement learning optimization prompting words.
2. The automatic FPGA test case generating system based on the large language model is characterized in that the requirement structuring module comprises a prompt word designing unit and a basic scene library building unit, wherein the prompt word designing unit is used for designing prompt words specially aiming at the large language model, a hierarchical prompt strategy is adopted, the content comprises a basic prompt template, a test key point disassembling rule and an output format requirement, the whole design requirement is disassembled into an atomization test key point, and the basic scene library building unit is used for building a basic scene library covering common test modes, including a functional test scene, a boundary test scene and an abnormal condition test scene.
3. The automatic generation system of the FPGA test cases based on the large language model of claim 1, wherein the verification execution module can carry out adaptation integration on the test cases and the FPGA verification environment, ensure that the test cases can be executed in a normal interaction mode, and meanwhile judge the Pass or Fail state of an execution result, and the closed-loop optimization module drives the large language model to carry out prompt word optimization by receiving the test result of the verification execution module.
4. The method for generating the FPGA test case automatic generation system based on the large language model according to any one of claims 1-3 is characterized by comprising the following steps: Step 1, demand input, namely receiving related basic information of the FPGA design provided by an engineer through a demand input module, and simultaneously receiving key information marked by the engineer; Step 2, demand structuring, namely designing prompt words through a prompt word design unit of a demand structuring module, disassembling the FPGA design demand into atomization test points, and building a basic scene library through a basic scene library building unit; Step 3, a large language model generation case, namely a large language model case generation module receives an atomization test key point and a basic scene library, generates a test case which accords with grammar and covers functions, boundaries and abnormal conditions, and forms a test file capable of directly running after self-checking and correction; Step 4, checking the generated test cases by an engineer through a case checking module, and feeding back modification suggestions attached to the cases which do not pass the checking to a large language model case generating module to trigger iterative optimization; And step 5, verification execution, wherein the verification execution module integrates the confirmed test cases into an FPGA verification environment, executes the cases and analyzes the results, records the Pass or Fail states of the test results, and the closed-loop optimization module feeds the test results back to the large language model case generation module, optimizes the prompt word through reinforcement learning, and iteratively generates more accurate test cases based on the optimized prompt word by the large language model.
5. The automatic generation system of FPGA test cases based on a large language model of claim 4, wherein in step 1, the FPGA design related basic information comprises an FPGA design requirement document, an interface definition and a verification target, and the key information marked by the engineer comprises an input-output relationship of a key module, a defect type to be eliminated, a key module identifier, an interface signal identifier and an expected behavior identifier.
6. The automatic FPGA test case generation system based on the large language model is characterized in that in the step 2, the hierarchical prompt strategy of the prompt word design unit designs corresponding prompt words according to different test types, for functional test, each functional module is split into independent test sub-functions, the test direction is defined according to each sub-function, for performance test, the prompt words comprise specific requirements of performance indexes and test methods, for abnormal condition test, the prompt words describe various possible abnormal scenes in detail, the functional test scenes of the basic scene library cover various input and output combination conditions under normal functional operation, the boundary test scenes mainly consider boundary value conditions of input signals, and the abnormal condition test scenes cover various conditions which can cause abnormal operation of the FPGA.
7. The automatic FPGA test case generating system based on the large language model according to claim 4 is characterized in that in step 3, when the large language model is generated, an FPGA design specification, a history error case and other plug-in knowledge base are combined, so that the generated test case is guaranteed to be close to actual hardware behaviors, in the aspect of functional test, each function of the FPGA is guaranteed to be completely covered by the generated case, each function module is guaranteed to be correctly verified under various normal and abnormal input conditions, in the aspect of performance test, the performance index of the FPGA can be accurately measured by the case, in the aspect of abnormal scene test, various possible abnormal conditions can be simulated by the case, and whether the response of the FPGA under the conditions meets expectations or not is detected.
8. The FPGA test case automatic generation system based on the large language model of claim 4, wherein in step 4, the engineer checks whether the test case includes covering a design critical path, whether obvious boundary conditions are omitted, whether logic contradictions exist or not, whether the FPGA grammar specification is met, the modification advice includes an uncovered critical path part, the omitted boundary condition type and the existing logic contradiction concrete description, the large language model case generation module regenerates the test case according to the modification advice until the requirement of the engineer is met, the engineer periodically gathers and reviews feedback, analyzes the large language model generation problem, regenerates or updates the design description after the modification, and forms iterative optimization.
9. The automatic FPGA test case generating system based on the large language model of claim 4, wherein in step 5, when the verification execution module integrates the test cases into the FPGA verification environment, the verification execution module ensures that the test cases can interact with the FPGA design correctly, when the test results are analyzed, whether the execution results of each case accord with expectations or not is judged one by one, the states of Pass or Fail are recorded accurately, for the failed test cases, failure reasons are analyzed in detail and synchronized to the closed-loop optimization module, the closed-loop optimization module feeds back the test results and the failure reasons to the large language model case generating module, the large language model carries out targeted optimization on prompt words through a reinforcement learning mechanism, the optimized prompt words are used for guiding the generation of the next round of test cases, and the precision of the generated test cases is gradually improved through continuous iteration, so that a continuously optimized closed-loop system is formed.

Description

FPGA test case automatic generation system and method based on large language model Technical Field The invention relates to the technical field of FPGA (field programmable gate array) test, in particular to an automatic FPGA test case generation system and method based on a large language model. Background In the development and application process of FPGA design, testing is a key link for ensuring that functions and performances meet design requirements. The traditional FPGA test method generally relies on manually writing test cases, which not only consumes a great deal of manpower and time, but also is easy to cause incomplete test coverage due to human negligence, and can not effectively find hidden problems in design. Along with the continuous improvement of the design complexity of the FPGA, the limitation of the traditional testing method is more obvious, the traditional FPGA verification is highly dependent on engineers to manually write the Testbench and the test vector, when the large-scale design or the frequent iteration is faced, the workload is huge, boundaries or abnormal scenes are easy to miss, and the requirements of a modern electronic system on the high quality and the high reliability of the FPGA are difficult to meet. In recent years, artificial intelligence technology has made remarkable progress in various fields, and the advantages of the artificial intelligence technology in aspects of data processing, pattern recognition, automatic decision making and the like provide a new idea for solving the FPGA test problem. By introducing a large language model, the automatic generation of the FPGA test cases can be realized, so that the test efficiency and the test accuracy are improved, but a complete end-to-end automatic generation scheme of the FPGA test cases based on the large language model is lacked in the prior art, the advantages of the large language model cannot be fully exerted, and various pain points existing in the traditional test method are difficult to solve. Disclosure of Invention The invention aims to provide an automatic FPGA test case generation system and method based on a large language model, so as to solve the problems in the prior art. The FPGA test case automatic generation system based on the large language model comprises a requirement input module, a requirement structuring module, a large language model case generation module, a case confirmation module, a verification execution module and a closed loop optimization module; the demand input module is used for receiving the FPGA design related basic information and the marking information provided by the engineer; The demand structuring module is used for disassembling input demands and constructing a scene library to form an atomization test key point; the large language model case generation module is used for receiving the atomization test key points and the scene library and generating executable test cases meeting the requirements; the use case confirmation module is used for an engineer to check the test use case and feed back the use cases which do not pass the check and the modification suggestions to the large language model use case generation module; The verification execution module is used for integrating the confirmed test cases into an FPGA verification environment, executing the cases and analyzing results; The closed loop optimization module is used for feeding back a test result to the large language model use case generation module, and the use case iterative optimization is realized through reinforcement learning optimization prompting words. The demand structuring module comprises a prompting word design unit and a basic scene library establishment unit, wherein the prompting word design unit is used for designing prompting words specially aiming at a large language model, adopts a hierarchical prompting strategy, comprises basic prompting templates, test key point disassembling rules and output format requirements, disassembles the whole design demand into atomization test key points, and is used for establishing a basic scene library covering common test modes, comprising a functional test scene, a boundary test scene and an abnormal condition test scene. Preferably, the verification execution module can carry out adaptation integration on the test case and the FPGA verification environment, ensure that the test case can be normally and interactively executed, and judge the Pass or Fail state of the execution result at the same time, and the closed-loop optimization module drives the large language model to optimize the prompt word by receiving the test result of the verification execution module. The method for generating the FPGA test case automatic generation system based on the large language model comprises the following steps: Step 1, demand input, namely receiving related basic information of the FPGA design provided by an engineer through a demand input module, and simultaneous