CN-122020399-A - Model confidence correction method and system

CN122020399ACN 122020399 ACN122020399 ACN 122020399ACN-122020399-A

Abstract

The application relates to a model confidence coefficient correction method and system, wherein the method comprises the steps of collecting output data and corresponding reference data of a tested digital prototype, obtaining error feature vectors of the output data and the corresponding reference data, inputting the error feature vectors into a composite diagnosis model to obtain fusion diagnosis vectors, determining whether to trigger a trigger decision for correcting the confidence coefficient based on the fusion diagnosis vectors, generating a comprehensive correction factor based on the fusion diagnosis vectors under the condition of triggering the trigger decision for correcting the confidence coefficient, correcting the original confidence coefficient output by the tested digital prototype by utilizing the comprehensive correction factor, obtaining data in a correction process, and carrying out iterative optimization on the model of the tested digital prototype by taking the optimal confidence coefficient calibration as a target. And constructing a multidimensional error feature vector comprising working condition sparsity, error statistics, dynamic response and boundary proximity, so as to realize comprehensive description of test errors.

Inventors

REN CHAOXU
HAN HUIJIE
WEN QIANG
LI ZHIHUI
LU YAO
ZHANG JUNXIA

Assignees

北京航天测控技术有限公司

Dates

Publication Date: 20260512
Application Date: 20251230

Claims (10)

1. A method for model confidence correction, the method comprising: Acquiring output data and corresponding reference data of a digital prototype to be tested, and acquiring error feature vectors of the output data and the corresponding reference data; Inputting the error feature vector into a composite diagnosis model to obtain a fusion diagnosis vector, and determining whether to trigger a trigger decision for correcting the confidence coefficient based on the fusion diagnosis vector; under the condition of triggering a trigger decision for correcting the confidence coefficient, generating a comprehensive correction factor based on the fusion diagnosis vector, and correcting the original confidence coefficient output by the tested digital prototype by utilizing the comprehensive correction factor; And acquiring data in the correction process, and performing iterative optimization on the model of the tested digital prototype by taking the confidence degree calibration degree as a target.
2. The method of claim 1, wherein the obtaining the error feature vector of the output data and corresponding reference data comprises: acquiring a data error between the output data and corresponding reference data; Extracting multidimensional error characteristics based on the data errors and current test working condition parameter information, wherein the multidimensional error characteristics comprise working condition sparsity characteristics, error statistics characteristics, dynamic response characteristics and boundary proximity characteristics; And integrating and splicing the extracted multidimensional error features according to a preset feature ordering rule and a data format to form a complete error feature vector.
3. The method of claim 2, wherein extracting the multi-dimensional error feature based on the data error and current test condition parameter information comprises: Based on the data error and the current test working condition parameter information, obtaining a working condition sparsity characteristic by calculating a weighted mahalanobis distance of the current test input in a characteristic space formed by model historical experience data; acquiring statistical properties of the data errors, wherein the statistical properties at least comprise mean values, variances and extremums of the errors; obtaining error statistical characteristics representing the overall error distribution characteristics by carrying out quantization processing on the statistical attributes; Analyzing a dynamic matching relation between an error signal and a test excitation signal, wherein the dynamic matching relation at least comprises an error energy spectrum and wavelet coherence of the excitation signal in a key frequency band; Obtaining dynamic response characteristics based on the performance mismatch degree of the dynamic matching relation quantization model under a specific dynamic mode; And calculating the standardized distance between the current test condition and the validated effective domain boundary of the model, and quantifying the capability boundary closeness degree of the model under the current working condition according to the standardized distance to obtain the boundary closeness characteristic.
4. A method according to claim 3, wherein the obtaining the sparsity feature by calculating a weighted mahalanobis distance of the current test input in a feature space formed by model historical empirical data includes: Acquiring a history test data set of a model, wherein the history test data set comprises a plurality of history test points, and each history test point corresponds to a history test input parameter, history output data and history errors of the history output data and corresponding reference data; Carrying out standardization processing on the history errors corresponding to each history test point to obtain standardized history errors; mapping the standardized historical error into performance weight of each historical test point through an exponential function based on a preset sensitivity factor, wherein the performance weight and the historical error are positively correlated; Calculating a weighted average vector of the input parameters of the history test in the feature space based on the performance weight of each history test point; calculating a weighted covariance matrix of the historical test input parameters in the feature space based on the performance weights and the weighted mean vectors; Based on the current test input parameters, the weighted mean vector and the weighted covariance matrix, calculating the original weighted Markov distance of the current test input parameters relative to the distribution of the historical test input parameters through a Markov distance formula; acquiring original weighted mahalanobis distances corresponding to all the historical test points in the historical test data set, and calculating the standard deviation of the original weighted mahalanobis distances; And acquiring the standardized weighted Markov distance as a working condition sparsity characteristic according to the standard deviation.
5. The method of claim 1, wherein inputting the error feature vector into a composite diagnostic model to obtain a fused diagnostic vector, and determining whether to trigger a trigger decision to correct the confidence level based on the fused diagnostic vector comprises: Performing similarity matching on the error feature vector and a pre-constructed historical diagnosis case library, and determining basic deviation probability and matching confidence coefficient according to a matching result; acquiring a trend deterioration index according to the matching confidence coefficient; carrying out standardized processing on the basic deviation probability, the trend degradation index and the matching confidence coefficient, and carrying out fusion splicing according to a preset rule to obtain a fusion diagnosis vector; and inputting the fusion diagnosis vector into a preset trigger decision model, so that the trigger decision model outputs a trigger decision of whether to trigger confidence correction and a correction strategy preference weight vector for indicating recommendation weights of different basic correction strategies in correction based on threshold judgment logic of each dimension characteristic in the fusion diagnosis vector.
6. The method of claim 5, wherein said obtaining a trend degradation index based on said and matching confidence comprises: judging whether the matching confidence is higher than a preset reliability threshold; Activating a time sequence prediction model under the condition that the matching confidence is lower than the reliability threshold, inputting a characteristic sequence consisting of a plurality of error characteristic vectors in a current preset period into the time sequence prediction model, and outputting a trend deterioration index; And setting a trend deterioration index as a preset reference value in the case that the matching confidence is higher than or equal to the reliability threshold.
7. The method of claim 1, wherein generating a comprehensive correction factor based on the fused diagnostic vector and correcting the original confidence of the digital prototype output under test using the comprehensive correction factor comprises: Based on the correction strategy preference weight vector output by the trigger decision, selecting at least two basic correction strategies with different principles from a preset correction strategy pool; Aiming at each selected basic correction strategy, calculating a preliminary correction factor corresponding to each basic correction strategy based on the corresponding feature dimension in the fusion diagnosis vector; introducing a strategy effectiveness short-term backtracking mechanism, acquiring effectiveness evaluation results of each basic correction strategy in the correction process within a history preset period, and adjusting fusion weights of each basic correction strategy in fusion according to the evaluation results; Based on the adjusted fusion weight of each basic correction strategy, carrying out weighted summation calculation on the corresponding primary correction factors to generate comprehensive correction factors; and carrying out product operation on the original confidence coefficient output by the tested digital prototype and the comprehensive correction factor to obtain corrected confidence coefficient so as to finish dynamic correction of the original confidence coefficient.
8. The method according to claim 1, wherein the obtaining data in the correction process, targeting optimization of confidence calibrations, iteratively optimizes the model of the digital prototype under test, comprises: Acquiring related data in the original confidence coefficient correction process, wherein the related data at least comprises an error feature vector, a fusion diagnosis vector, a trigger decision result, preliminary correction factors of basic correction strategies, fusion weights, comprehensive correction factors, original confidence coefficient, corrected confidence coefficient and reference verification data for verifying correction effects; taking the minimized expected calibration error as a core, quantifying a confidence coefficient calibration optimization target, and calculating the fit degree of the corrected confidence coefficient and the verification result so as to evaluate the calibration effect of the current correction system; updating the verified data pairs to a pre-constructed historical diagnosis case library by adopting an incremental learning mode, wherein the data pairs comprise error feature vectors, trigger decisions, correction results and verification conclusions; Adopting a reinforcement learning algorithm, taking the minimized expected calibration error as a reward signal, and carrying out iterative adjustment on model parameters of the composite diagnosis model and the multi-strategy fusion correction model so as to improve the diagnosis precision and correction suitability of the model; based on the updated historical diagnosis case library and the adjusted model, selecting typical test working conditions for verification to obtain a verification result; under the condition that the verification result meets a preset optimization threshold, completing the iteration; And returning to the step of acquiring the related data in the original confidence coefficient correction process under the condition that the verification result does not meet the preset optimization threshold value so as to incorporate new correction flow data and verification result to form a closed loop iteration mechanism.
9. A model confidence correction system, the system comprising: The error sensing and feature extraction module is used for acquiring output data and corresponding reference data of the tested digital prototype and acquiring error feature vectors of the output data and the corresponding reference data; the composite diagnosis and intelligent triggering module is used for inputting the error feature vector into a composite diagnosis model to obtain a fusion diagnosis vector, and determining whether to trigger a triggering decision for correcting the confidence coefficient based on the fusion diagnosis vector; The multi-strategy fusion correction execution module is used for generating a comprehensive correction factor based on the fusion diagnosis vector under the condition of triggering a trigger decision for correcting the confidence coefficient, and correcting the original confidence coefficient output by the tested digital prototype by utilizing the comprehensive correction factor; And the self-evolution optimization management module is used for acquiring data in the correction process, and carrying out iterative optimization on the model of the tested digital prototype by taking the optimization confidence degree calibration as a target.
10. The system of claim 1, wherein the system further comprises a dynamic adaptation interface layer; The dynamic adaptation interface layer is used for automatically analyzing the data modes and the confidence label formats of different host digital test platforms, dynamically configuring the data processing pipeline of the error perception and feature extraction module according to analysis results, and simultaneously configuring a data communication interface in front of the corresponding host digital test platform.

Description

Model confidence correction method and system Technical Field The present application relates to the field of data processing technologies, and in particular, to a method and a system for correcting model confidence coefficient. Background With the deep advancement of digital transformation, digital testing based on digital prototypes has been developed as a key link in the product development process. The system replaces or substantially reduces expensive physical testing by applying test stimulus to the digital sample machine and evaluating its response in a virtual environment. The confidence level of the output result of the digital prototype is used as a core element index for measuring the prediction reliability of the output result, and the validity of the test conclusion and the scientificity of the subsequent decision are directly related. However, the current digital test system has obvious defects in the aspect of model confidence degree processing, namely firstly, the static calibration problem is prominent, the confidence degree is usually determined at one time according to limited reference data in the model integration stage, and the dynamic adjustment is difficult along with the deep test, the iterative update of the model or the expansion of working conditions. Second, the error data is underutilized and multi-dimensional digitized error data (e.g., differences from high-fidelity models, historical data, or sparse physical test points) that is continuously generated during the test process cannot be systematically used for online calibration of confidence. Furthermore, risk assessment lacks transparency, and static high confidence may mislead the tester under the model capability boundary or new conditions, resulting in misjudgment of the reliability of the model output, thereby causing design risk. In addition, the prior art mainly focuses on parameter correction or verification of a model, lacks a light-weight, online and self-adaptive mechanism, can comprehensively utilize the multidimensional characteristic of real-time errors, and performs dynamic management by combining intelligent diagnosis and multi-strategy fusion correction and specifically aiming at an evaluation index of confidence. Accordingly, there is a need to develop a model confidence correction method and system that addresses one or more of the problems set forth above. Disclosure of Invention In view of this, in order to solve the above technical problems or part of the technical problems, the embodiments of the present invention provide a method and a system for correcting a model confidence coefficient. In a first aspect, the present application provides a method for correcting confidence coefficient of a model, the method comprising: Acquiring output data and corresponding reference data of a digital prototype to be tested, and acquiring error feature vectors of the output data and the corresponding reference data; Inputting the error feature vector into a composite diagnosis model to obtain a fusion diagnosis vector, and determining whether to trigger a trigger decision for correcting the confidence coefficient based on the fusion diagnosis vector; under the condition of triggering a trigger decision for correcting the confidence coefficient, generating a comprehensive correction factor based on the fusion diagnosis vector, and correcting the original confidence coefficient output by the tested digital prototype by utilizing the comprehensive correction factor; And acquiring data in the correction process, and performing iterative optimization on the model of the tested digital prototype by taking the confidence degree calibration degree as a target. In one possible implementation manner, the obtaining the error feature vector of the output data and the corresponding reference data includes: acquiring a data error between the output data and corresponding reference data; Extracting multidimensional error characteristics based on the data errors and current test working condition parameter information, wherein the multidimensional error characteristics comprise working condition sparsity characteristics, error statistics characteristics, dynamic response characteristics and boundary proximity characteristics; And integrating and splicing the extracted multidimensional error features according to a preset feature ordering rule and a data format to form a complete error feature vector. In one possible implementation manner, the extracting the multi-dimensional error feature based on the data error and the current test condition parameter information includes: Based on the data error and the current test working condition parameter information, obtaining a working condition sparsity characteristic by calculating a weighted mahalanobis distance of the current test input in a characteristic space formed by model historical experience data; acquiring statistical properties of the data errors, wherein the statistical prope