CN-121255270-B - Software bill of materials generation tool evaluation method and device

CN121255270BCN 121255270 BCN121255270 BCN 121255270BCN-121255270-B

Abstract

The invention provides a software bill of materials generating tool evaluation method and a device, which relate to the technical field of computers, and the method comprises the steps of inputting a preselected target software item into an SBOM generating tool to be tested to obtain an original SBOM file output by the SBOM generating tool to be tested; the method comprises the steps of carrying out standardized analysis on an original SBOM file to obtain a corresponding standard SBOM file, carrying out multidimensional evaluation on the standard SBOM file to obtain an evaluation score, wherein the evaluation score comprises a field compliance score, a package level consistency score and a field level consistency score, and obtaining an evaluation result of the SBOM generating tool to be tested based on the evaluation score. According to the method for evaluating the software bill of materials generating tool, provided by the invention, the final evaluation result is generated by integrating the multidimensional scores, so that accurate and reliable evaluation of the SBOM generating tool is realized.

Inventors

LING XIANG
WANG CHENGJIE
Li Jiaoyun
WU JINGZHENG
Luo tianyue
ZHAO CHEN

Assignees

中国科学院软件研究所

Dates

Publication Date: 20260505
Application Date: 20251203

Claims (7)

1. A software bill of materials generation tool evaluation method, comprising: inputting a preselected target software item into an SBOM generating tool to be detected, and obtaining an original SBOM file output by the SBOM generating tool to be detected; carrying out standardized analysis on the original SBOM file to obtain a corresponding standard SBOM file; Performing multidimensional evaluation on the standard SBOM file to obtain an evaluation score, wherein the evaluation score comprises a field compliance score, a package level consistency score and a field level consistency score; based on the evaluation score, an evaluation result of the SBOM generating tool to be tested is obtained; The multi-dimensional evaluation is carried out on the standard SBOM file to obtain an evaluation score, which comprises the following steps: Determining a field to be tested based on user requirements; calculating the occurrence rate of the field to be detected in all fields of the standard SBOM file; Taking the occurrence rate as the field compliance score; Matching software packages in a plurality of different standard SBOM files based on the names of the software packages to obtain a first matched software package and a matched success rate; taking the pairing success rate as the packet level consistency score; And calculating the consistency among the standard data fields in the first pairing software package, and obtaining the field level consistency score.
2. The method for evaluating a software bill of materials generation tool according to claim 1, wherein said performing standardized parsing on said original SBOM file to obtain a corresponding standard SBOM file comprises: Extracting key data fields in each software package in the original SBOM file based on the data exchange standard of the original SBOM file, wherein the key data fields comprise more than two of a software package name, a version number, a license, copyright information and a unique identifier; Converting the key data fields into a preset standard format to obtain standard data fields; And generating the standard SBOM file based on the standard data field.
3. The software bill of materials generation tool evaluation method according to claim 2, further comprising, before said converting said key data fields into a preset standard format: in case of a miss in any of the critical data fields, the missing critical data fields are filled by predefined placeholders.
4. The software bill of materials generation tool evaluation method according to claim 1 wherein in the event that the target software item has a corresponding software benchmark truth information data set, the evaluation score further comprises a package level accuracy score and a field level accuracy score; The software reference truth value information data set is a data set of accurate data corresponding to the target software item; the package level accuracy score is used for representing a pairing success rate between the software package in the standard SBOM file and the software package in the software reference true value information dataset; the field level accuracy score is used for representing consistency among standard data fields in the second pairing software package; and the second pairing software package is a pairing software package which is successfully matched with the software package in the standard SBOM file and the software package in the software reference true value information data set.
5. A software bill of materials generation tool evaluation device, comprising: the generating module is used for inputting a preselected target software item into the SBOM generating tool to be detected, and obtaining an original SBOM file output by the SBOM generating tool to be detected; the analysis module is used for carrying out standardized analysis on the original SBOM file to obtain a corresponding standard SBOM file; The computing module is used for carrying out multi-dimensional evaluation on the standard SBOM file to obtain evaluation scores, wherein the evaluation scores comprise field compliance scores, package level consistency scores and field level consistency scores; the evaluation module is used for obtaining an evaluation result of the SBOM generating tool to be tested based on the evaluation score; The multi-dimensional evaluation is carried out on the standard SBOM file to obtain an evaluation score, which comprises the following steps: Determining a field to be tested based on user requirements; calculating the occurrence rate of the field to be detected in all fields of the standard SBOM file; Taking the occurrence rate as the field compliance score; Matching software packages in a plurality of different standard SBOM files based on the names of the software packages to obtain a first matched software package and a matched success rate; taking the pairing success rate as the packet level consistency score; And calculating the consistency among the standard data fields in the first pairing software package, and obtaining the field level consistency score.
6. An electronic device comprising a memory, a processor and a computer program stored on the memory and running on the processor, wherein the processor implements the software bill of materials generating tool evaluation method according to any one of claims 1 to 4 when executing the computer program.
7. A non-transitory computer readable storage medium having stored thereon a computer program, wherein the computer program when executed by a processor implements the software bill of materials generation tool evaluation method according to any one of claims 1 to 4.

Description

Software bill of materials generation tool evaluation method and device Technical Field The present invention relates to the field of computer technologies, and in particular, to a method and an apparatus for evaluating a software bill of materials generating tool. Background Any item of software (e.g., source code repository) can be automatically analyzed by a software bill of materials (Software Bill of Materials, SBOM) generation tool, direct and indirect dependencies therein identified, and a corresponding SBOM file generated according to the selected SBOM standard. To improve the quality of the generated SBOM file, it is necessary to evaluate the performance of the SBOM generation tool before using the SBOM generation tool. In the prior art, the method for evaluating the quality of the SBOM generating tool mainly comprises single SBOM quality evaluation, package level information accuracy evaluation and evaluation based on artificial synthesis projects. However, single SBOM quality evaluation lacks cross-tool transverse comparison, the capability of an SBOM generating tool cannot be effectively obtained through comparison, the capability of the tool facing a unified evaluation index cannot be obtained, the evaluation result is inaccurate, packet level information accuracy evaluation cannot evaluate the generating capability of the SBOM generating tool on finer granularity information (such as open source license information) of a dependent packet contained in the SBOM, the evaluation is incomplete, the evaluation result accuracy is low, evaluation analysis under a real scene is lacking in evaluation based on artificial synthetic projects, and the capability of the SBOM generating tool facing real software analysis cannot be effectively evaluated, so that the evaluation result is unreliable. Disclosure of Invention The invention provides a software bill of materials generating tool evaluation method and device, which are used for solving the technical problems of inaccurate and unreliable evaluation of an SBOM generating tool in the prior art. The invention provides a software bill of materials generation tool evaluation method, which comprises the following steps: inputting a preselected target software item into an SBOM generating tool to be detected, and obtaining an original SBOM file output by the SBOM generating tool to be detected; carrying out standardized analysis on the original SBOM file to obtain a corresponding standard SBOM file; Performing multidimensional evaluation on the standard SBOM file to obtain an evaluation score, wherein the evaluation score comprises a field compliance score, a package level consistency score and a field level consistency score; and based on the evaluation score, obtaining an evaluation result of the SBOM generating tool to be tested. According to the method for evaluating the software bill of materials generation tool provided by the invention, the standardized analysis is carried out on the original SBOM file to obtain the corresponding standard SBOM file, and the method comprises the following steps: Extracting key data fields in each software package in the original SBOM file based on the data exchange standard of the original SBOM file, wherein the key data fields comprise more than two of a software package name, a version number, a license, copyright information and a unique identifier; Converting the key data fields into a preset standard format to obtain standard data fields; And generating the standard SBOM file based on the standard data field. According to the method for evaluating the software bill of materials generating tool provided by the invention, before the key data fields are converted into the preset standard format, the method further comprises the following steps: in case of a miss in any of the critical data fields, the missing critical data fields are filled by predefined placeholders. According to the method for evaluating the software bill of materials generation tool provided by the invention, the standard SBOM file is subjected to multidimensional evaluation to obtain evaluation scores, and the method comprises the following steps: Determining a field to be tested based on user requirements; calculating the occurrence rate of the field to be detected in all fields of the standard SBOM file; The occurrence rate is used as the field compliance score. According to the method for evaluating the software bill of materials generation tool provided by the invention, the standard SBOM file is subjected to multidimensional evaluation to obtain evaluation scores, and the method comprises the following steps: matching the software packages in the SBOM files with different standards based on the software package names to obtain a first matched software package and a matched success rate; And taking the pairing success rate as the packet level consistency score. According to the method for evaluating the software bill of materials generation to