CN-115905463-B - Target detection algorithm evaluation method, device, equipment and storage medium

Abstract

The application discloses a target detection algorithm evaluation method, a device, equipment and a storage medium, which relate to the technical field of computers and comprise the steps of constructing an evaluation data set of an evaluation data item, wherein the evaluation data set consists of a detection index item and a pre-constructed description keyword item; the method comprises the steps of carrying out synonym merging on all description key terms in an evaluation data set, reconstructing a new evaluation data set based on the coded key terms obtained by coding the merged key terms and the coded index terms obtained by coding the discretized detection index terms, reconstructing a corresponding target key term set based on the currently determined target detection index terms and effective key terms in a target evaluation data set obtained by eliminating description key terms insensitive to the detection index terms in the new evaluation data set, and evaluating an algorithm under a target detection scene corresponding to the target key term set, so that the false detection rate and the detection rate under the use scene can be improved pertinently after the algorithm, and blind optimization is avoided.

Inventors

• QI FAN

Assignees

• 济南博观智能科技有限公司

Dates

Publication Date

20260508

Application Date

20220927

Claims (8)

1. 1. A method for evaluating an object detection algorithm, comprising: The method comprises the steps of constructing an evaluation data set, wherein evaluation data items in the evaluation data set consist of detection index items and description key items constructed in advance; carrying out synonym combination on all the description keyword items in the evaluation data set, and encoding the combined keyword items to obtain encoded keyword items; Discretizing the detection index items in the evaluation data set, and encoding the discretized detection index items to obtain encoded index items; Reconstructing a new evaluation data set based on the encoded key terms and the encoded index terms, and removing description key terms insensitive to the detection index terms in the new evaluation data set to obtain a target evaluation data set; Reconstructing a corresponding target keyword item set based on the currently determined target detection index item and the effective keyword item in the target evaluation data set, and analyzing and evaluating the target detection algorithm under a target detection scene corresponding to the target keyword item set to obtain a corresponding evaluation result; For single-target detection, the detection index item comprises whether an image to be detected and a video to be detected are detected and whether the image to be detected is detected by mistake; The method further comprises the steps of constructing a description category, a scene and the description keyword terms describing the mutual states of the category and the scene, wherein the description keyword terms comprise a structured primary keyword and an unstructured secondary keyword, and determining the influence degree of the corresponding description keyword terms on the target detection index term by determining the conditional probability of each description keyword term; the reconstructing a corresponding target keyword term set based on the currently determined target detection index term and the valid keyword term in the target evaluation data set includes: If the main key word set corresponding to the effective key word item in the target evaluation data set is an empty set, a target main key word with the highest probability of query conditions of the auxiliary key words in the effective key word item is inquired, and a corresponding target key word item set is reconstructed based on a predetermined target auxiliary key word set and a target main key word set formed by the target main key words; and if the primary keyword set is not the empty set, determining a target primary keyword with the maximum conditional probability in the target evaluation data set, and reconstructing a corresponding target keyword item set based on a predetermined target secondary keyword set and a target primary keyword set formed by the target primary keyword.
2. 2. The method according to claim 1, wherein the performing synonym combination on all the descriptive terms in the evaluation dataset to encode the combined terms to obtain encoded terms comprises: Performing duplicate removal on all secondary keywords in the description keyword terms extracted from the evaluation data set to obtain a duplicate-removed secondary keyword set, and encoding all duplicate-removed secondary keywords in the duplicate-removed secondary keyword set by adopting a word vector file to obtain a corresponding secondary keyword vector set; calculating cosine similarity between the secondary keyword vectors in the secondary keyword vector set; Merging the duplicate-removed auxiliary keywords in the duplicate-removed auxiliary keyword set based on the cosine similarity, and encoding the auxiliary keywords in the merged auxiliary keyword set by using a OneHot method to obtain an encoded auxiliary keyword set; Extracting the primary key words in the description key terms from the evaluation data set to obtain a primary key word set, and merging the primary key words in the primary key word set to obtain a merged primary key word set; And encoding the primary key words in the combined primary key word set by adopting a discretization encoding mode to obtain an encoded primary key word set.
3. 3. The method according to claim 2, wherein the merging the duplicate-removed secondary keywords in the duplicate-removed secondary keyword set based on the cosine similarity includes: Judging whether the cosine similarity is not smaller than a first preset threshold value or not; And if the cosine similarity is not smaller than the first preset threshold, merging the duplicate-removed auxiliary keywords in the duplicate-removed auxiliary keyword set.
4. 4. The method for evaluating an object detection algorithm according to claim 2, wherein the removing the description key term in the new evaluation dataset that is insensitive to the detection index term to obtain an object evaluation dataset includes: Calculating the information gain of all the encoded key terms in the new evaluation data set to the detection index terms to obtain gain values corresponding to the encoded key terms; Judging whether the gain value corresponding to the encoded keyword term is not smaller than a preset threshold value or not; if the gain value corresponding to the encoded keyword is not smaller than the preset threshold value, reserving the encoded keyword; And if the gain value corresponding to the encoded keyword is smaller than the preset threshold value, eliminating the encoded keyword from the new evaluation data set to obtain a target evaluation data set.
5. 5. The method according to claim 1, wherein after reconstructing the corresponding target keyword term set based on the currently determined target detection indicator term and the valid keyword terms in the target evaluation dataset, the method further comprises: calculating scene fitness of all auxiliary keywords in the target keyword term set; And sequencing the secondary keywords in the target keyword term set according to the scene fitness.
6. 6. An object detection algorithm evaluation device, characterized by comprising: The data set construction module is used for constructing an evaluation data set, wherein an evaluation data item in the evaluation data set consists of a detection index item and a pre-constructed description keyword item; the keyword merging module is used for carrying out synonym merging on all the description keyword items in the evaluation data set; the keyword coding module is used for coding the combined keyword to obtain a coded keyword; The index discretization module is used for discretizing the detection index items in the evaluation data set; The index coding module is used for coding the discretized detection index items to obtain coded index items; a data set reconstruction module for reconstructing a new evaluation data set based on the encoded key term and the encoded index term; The keyword eliminating module is used for eliminating descriptive keyword items insensitive to the detection index items in the new evaluation data set to obtain a target evaluation data set; The keyword reconstruction module is used for reconstructing a corresponding target keyword item set based on the currently determined target detection index item and the effective keyword items in the target evaluation data set; the detection algorithm evaluation module is used for analyzing and evaluating the target detection algorithm under the target detection scene corresponding to the target keyword item set to obtain a corresponding evaluation result; the target detection algorithm evaluation device is used for detecting a single target, wherein the detection index item comprises whether an image to be detected and a video to be detected are detected and whether false detection is carried out or not; the target detection algorithm evaluation device is used for constructing description categories, scenes and description keyword items describing the mutual states of the categories and the scenes, wherein the description keyword items comprise structured primary keywords and unstructured secondary keywords, and the influence degree of the corresponding description keyword items on the target detection index items is determined by determining the conditional probability of each description keyword item; wherein, the keyword reconstruction module includes: The target primary key term query module is used for querying a target primary key term with the highest conditional probability based on the secondary key term in the effective key term if the primary key term set corresponding to the effective key term in the target evaluation data set is an empty set, and reconstructing a corresponding target key term set based on a predetermined target secondary key term set and a target primary key term set formed by the target primary key term set; And the target primary key word determining module is used for determining a target primary key word with the highest conditional probability in the target evaluation data set if the primary key word set is not the empty set, and reconstructing a corresponding target key word item set based on a predetermined target secondary key word set and a target primary key word set formed by the target primary key words.
7. 7. An electronic device, comprising: A memory for storing a computer program; A processor for executing the computer program to implement the steps of the object detection algorithm evaluation method according to any one of claims 1 to 5.
8. 8. A computer readable storage medium for storing a computer program, wherein the computer program when executed by a processor implements the steps of the object detection algorithm evaluation method according to any one of claims 1 to 5.

Description

Target detection algorithm evaluation method, device, equipment and storage medium Technical Field The present invention relates to the field of computer technologies, and in particular, to a method, an apparatus, a device, and a storage medium for evaluating a target detection algorithm. Background At present, target detection is often the first step of a plurality of intelligent algorithms, the precision and recall rate of the test detection algorithm tend to have great influence on subsequent tasks, but most of the current mainstream methods for testing the detection algorithm are concentrated on classification of detection targets, and no test and evaluation methods aiming at specific intelligent tasks and actual deployment environments are provided, so that the difference between the test results of the detection tasks and the effects of using actual scenes is great, and the intelligent algorithm generates index distortion, on the other hand, the difficulty in index refinement of the algorithm under the scenes is increased due to the omission of specific subdivision scenes of the deployment environments, and a great amount of supplement and improvement of algorithm training and test data are often required, so that huge cost is generated. That is, there is a considerable degree of unmatched algorithm index and actual demand in the current detection algorithm landing area, and an effective algorithm evaluation method is urgently needed, which can accurately evaluate the accuracy of the detection algorithm under the use of an actual scene and guide the optimization direction of later training. Disclosure of Invention Accordingly, the present invention aims to provide a method, apparatus, device and storage medium for evaluating a target detection algorithm, which can accurately evaluate the detection accuracy of the target detection algorithm in different scenes, and facilitate the subsequent targeted improvement of the false detection rate and the detection rate of the target detection algorithm in the use scene, thereby avoiding blind optimization. The specific scheme is as follows: in a first aspect, the application discloses a target detection algorithm evaluation method, which comprises the following steps: The method comprises the steps of constructing an evaluation data set, wherein evaluation data items in the evaluation data set consist of detection index items and description key items constructed in advance; carrying out synonym combination on all the description keyword items in the evaluation data set, and encoding the combined keyword items to obtain encoded keyword items; Discretizing the detection index items in the evaluation data set, and encoding the discretized detection index items to obtain encoded index items; Reconstructing a new evaluation data set based on the encoded key terms and the encoded index terms, and removing description key terms insensitive to the detection index terms in the new evaluation data set to obtain a target evaluation data set; Reconstructing a corresponding target keyword term set based on the currently determined target detection index term and the effective keyword term in the target evaluation data set, and analyzing and evaluating the target detection algorithm under a target detection scene corresponding to the target keyword term set to obtain a corresponding evaluation result. Optionally, before the constructing the evaluation dataset, the method further comprises: And constructing description categories, scenes and description keyword terms for describing the mutual states of the categories and the scenes, wherein the description keyword terms consist of structured primary keywords and unstructured secondary keywords. Optionally, the performing synonym merging on all the description keyword in the evaluation data set, and encoding the merged keyword to obtain an encoded keyword includes: Performing duplicate removal on all secondary keywords in the description keyword extracted from the evaluation data set to obtain a duplicate-removed secondary keyword set, and encoding all duplicate-removed secondary keywords in the duplicate-removed secondary keyword set by using a word vector file to obtain a corresponding secondary keyword vector set; calculating cosine similarity between the secondary keyword vectors in the secondary keyword vector set; Merging the duplicate-removed auxiliary keywords in the duplicate-removed auxiliary keyword set based on the cosine similarity, and encoding the auxiliary keywords in the merged auxiliary keyword set by using a OneHot method to obtain an encoded auxiliary keyword set; Extracting the primary key words in the description key terms from the evaluation data set to obtain a primary key word set, and merging the primary key words in the primary key word set to obtain a merged primary key word set; And encoding the primary key words in the combined primary key word set by adopting a discretization encoding mode to o