CN-120850736-B - Complex system performance evaluation method and device considering missing data

CN120850736BCN 120850736 BCN120850736 BCN 120850736BCN-120850736-B

Abstract

The application discloses a complex system performance evaluation method and device considering missing data, and relates to the technical field of performance evaluation, wherein the method comprises the steps of calculating an estimated value, data integrity, target weight and reliability of a test index according to observation data of the test index of a complex system; and calculating the sensitivity of the complex system according to the output utility and the data integrity, and evaluating the output utility of the complex system by utilizing the sensitivity when the data integrity changes. According to the application, comprehensive indexes such as data integrity, target weight, reliability and the like are introduced, the method can adapt to non-equidistant characteristics caused by high-frequency and non-uniform interval tests, reduce interference of key feature loss on system performance evaluation, and dynamically capture the variation trend of system output utility through correlation analysis of data integrity and sensitivity.

Inventors

WANG JIE
Zang Xiaoyao
HAN YUE
LI HAIBO
LI HAI

Assignees

中国人民解放军国防科技大学信息通信学院

Dates

Publication Date: 20260508
Application Date: 20250625

Claims (8)

1. A complex system performance evaluation method considering missing data, comprising: calculating an estimated value, data integrity, target weight and reliability of the test index according to the observation data of the test index of the complex system; Constructing a reference value of an evaluation level space, and calculating the output utility of the complex system when the missing data is contained by combining the estimated value, the target weight and the reliability; Calculating the sensitivity of the complex system according to the output utility and the data integrity, and evaluating the output utility of the complex system by using the sensitivity when the data integrity changes; The method for calculating the estimated value, the data integrity, the target weight and the reliability of the test index according to the observed data of the test index of the complex system further comprises the following steps: judging whether the observed data at each moment is valid or not according to the state variable defined by the missing condition of the test index; Calculating an estimated value of a test index according to the observation time point, the state variable and the observation data; defining data integrity, target weight and reliability through the observed data, the state variables and the total observed time number; wherein, the calculating the estimated value of the test index according to the observation time point, the state variable and the observation data further comprises: construction of deletion interval feature quantity The time span of the current missing period and the latest effective observation is characterized, and the calculation formula is as follows: ; Calculating an estimated value of the test index The method comprises the following steps: ; ; Wherein alpha represents a test index, Indicating index Is determined by the time point of the kth observation, Is that Testing the observed value of the index at the moment; Is a defined state variable; Is a valid observation; is the historical average of the observed values; the ability of the recent observation value to provide information for the current missing data is reflected for the dynamic attenuation coefficient; is an exponential decay constant.
2. The complex system performance evaluation method considering missing data as claimed in claim 1, wherein said defining data integrity, target weight and reliability by said state variables, said state variables and total observation time count further comprises: Defining data integrity The data integrity ratio of the characterization test index alpha is calculated as follows: ; Recording the initial weight as Then the target weight of the test index alpha is evaluated under the condition of data missing And reliability degree Can be defined as: ; ; wherein T is the total observation time number; a number representing index unreliable data points; Is a binary sign, if Then Otherwise The method is used for judging whether the observed value is in a trusted threshold range or not; And Respectively representing the mean value and variance of the observed data, and representing the standard value and fluctuation level of the observed data; Is the observed value of the test index; the trusted interval range is adjusted.
3. The method for evaluating performance of a complex system taking into account missing data according to claim 1, wherein said constructing a reference value of an evaluation level space, and calculating an output utility of the complex system when missing data is contained in combination with said estimated value, said target weight, and said reliability, further comprises: Calculating the evaluation confidence coefficient of the corresponding evaluation level space according to the evaluation value and the reference value of the evaluation level space; Based on ER rules, the output utility of the complex system when missing data is contained is calculated according to the evaluation confidence, the target weight and the reliability.
4. The complex system performance evaluation method considering missing data according to claim 3, wherein said calculating an evaluation confidence of a corresponding evaluation level space from said evaluation value and a reference value of said evaluation level space further comprises: evaluating confidence The following can be calculated: ; Wherein, the For evaluating the evaluation value of the test index, the rank space is evaluated Reference value of (2) is , Is given by The quantized utility value of (2) satisfies monotonicity: , representing the inter-level span.
5. The complex system performance evaluation method considering missing data as in claim 1, wherein said calculating the sensitivity of the complex system based on said output utility and said data integrity, evaluating the output utility of the complex system using said sensitivity when said data integrity changes, further comprises: When data integrity Generating small increments The output utility change amount is The definition of the sensitivity function is: ; Can be expressed as: ; Wherein, the In order to fuse the amount of variation in the confidence, Is a utility reference value for fusion confidence.
6. A complex system performance evaluation apparatus that considers missing data, comprising: A parameter calculation module configured to calculate an estimated value, data integrity, target weight, and reliability of a test index of the complex system from observed data of the test index; An output utility module configured to construct a reference value of an evaluation level space and calculate an output utility of the complex system when the missing data is contained in combination with the estimated value, the target weight, and the reliability; A performance evaluation module configured to calculate a sensitivity of the complex system from the output utility and the data integrity, the sensitivity being utilized to evaluate the output utility of the complex system as the data integrity changes; The method for calculating the estimated value, the data integrity, the target weight and the reliability of the test index according to the observed data of the test index of the complex system further comprises the following steps: judging whether the observed data at each moment is valid or not according to the state variable defined by the missing condition of the test index; Calculating an estimated value of a test index according to the observation time point, the state variable and the observation data; defining data integrity, target weight and reliability through the observed data, the state variables and the total observed time number; wherein, the calculating the estimated value of the test index according to the observation time point, the state variable and the observation data further comprises: construction of deletion interval feature quantity The time span of the current missing period and the latest effective observation is characterized, and the calculation formula is as follows: ; Calculating an estimated value of the test index The method comprises the following steps: ; ; Wherein alpha represents a test index, Indicating index Is determined by the time point of the kth observation, Is that Testing the observed value of the index at the moment; Is a defined state variable; Is a valid observation; is the historical average of the observed values; the ability of the recent observation value to provide information for the current missing data is reflected for the dynamic attenuation coefficient; is an exponential decay constant.
7. A complex system performance evaluation device taking into account missing data, characterized by comprising at least one processing unit and at least one storage unit, wherein the storage unit stores a computer program, which when executed by the processing unit, causes the processing unit to perform the steps of the complex system performance evaluation method taking into account missing data according to any one of claims 1-5.
8. A storage medium storing a computer program executable by a complex system performance evaluation device taking into account missing data, the computer program causing the complex system performance evaluation device taking into account missing data to carry out the steps of the complex system performance evaluation method taking into account missing data as claimed in any one of claims 1 to 5 when the computer program is run on the complex system performance evaluation device taking into account missing data.

Description

Complex system performance evaluation method and device considering missing data Technical Field The application relates to the technical field of performance evaluation, in particular to a complex system performance evaluation method and device considering missing data. Background Along with the progress of the complex system to high speed, light weight and intelligent evolution, the reliability of the operational data integrity and performance evaluation of the complex system faces serious challenges, if the performance state of the complex system is evaluated improperly, multiple hazards may be caused, firstly, the erroneous judgment of key indexes may cause cascading faults, such as power grid linkage breakdown, large-scale downtime of a production line and the like, secondly, the evaluation model under the condition of missing data is not considered to deviate from the actual working condition easily, so that maintenance strategy deviation is caused, thirdly, the energy efficiency evaluation of misalignment may trigger invalid scheduling instructions, and industrial cost is increased. Therefore, the construction of a scientific and comprehensive evaluation system is important to ensuring the safe and efficient operation of the complex system. The performance evaluation of the complex system is essentially a systematic analysis method based on multidimensional data fusion, and the working efficiency and the running quality of the system are quantitatively evaluated by analyzing the dynamic association degree among various performance indexes, so that the accurate description of the performance state of the complex system is realized. The hybrid evaluation method fused with the multi-source information is one of the currently mainstream evaluation methods, the evidence reasoning (EVIDENTIAL REASONING, ER) rule is used as a typical paradigm of the hybrid evaluation method, the test indexes are uniformly characterized as evidence bodies by constructing a distributed confidence structure, and the multi-source evidence fusion is realized based on an orthogonal synthesis rule, so that a theoretical framework is provided for revealing the global performance of the complex system. However, in practical engineering application, two major core challenges exist, namely, firstly, a system needs to execute continuous tests with high frequency and non-uniform intervals (such as intermittent working conditions in spacecraft attitude adjustment) under a complex task scene, so that monitoring data presents obvious non-uniform interval characteristics, secondly, the monitoring data is easily subjected to non-random deletion (such as data loss caused by communication interruption of a certain test task) due to factors such as disturbance of a test environment, failure of acquisition equipment and abnormal storage, and the like, and the uncertainty of small sample evaluation is aggravated by a traditional data deletion strategy. Especially when the missing data contains key features, the direct elimination can cause the performance evaluation result to deviate from the real state seriously, so that the accuracy of judging the system state and the reliability of maintenance decision are affected. Disclosure of Invention Aiming at least one defect or improvement requirement of the prior art, the invention provides a complex system performance evaluation method and device considering missing data, which are used for solving the problems that the monitored data is obvious in non-uniform interval characteristic and easy to cause non-random missing when continuous test with high frequency and non-uniform interval is executed in complex task scene in the prior art, and the uncertainty of small sample evaluation is aggravated by the traditional data deletion strategy, particularly when the missing data contains key characteristics, the performance evaluation result is unreal, and the accuracy of judging the system state and the reliability of maintenance decision are influenced. To achieve the above object, according to a first aspect of the present invention, there is provided a complex system performance evaluation method considering missing data, comprising: calculating an estimated value, data integrity, target weight and reliability of the test index according to the observation data of the test index of the complex system; Constructing a reference value of an evaluation level space, and calculating the output utility of the complex system when the missing data is contained by combining the estimated value, the target weight and the reliability; and calculating the sensitivity of the complex system according to the output utility and the data integrity, and evaluating the output utility of the complex system by using the sensitivity when the data integrity changes. In one possible implementation manner, the method calculates an estimated value, data integrity, target weight and reliability of the test index according to the o