Search

EP-4740154-A1 - METHOD FOR EVALUATING THE COMPLIANCE OF A TRACKING SYSTEM USING A RESTRICTED NUMBER OF MEASUREMENTS AND ASSOCIATED DEVICES

EP4740154A1EP 4740154 A1EP4740154 A1EP 4740154A1EP-4740154-A1

Abstract

The present invention relates to a method for evaluating the compliance of a tracking system (10) with a set of requirements, the method comprising at least the following steps: - acquiring a set of values, - for each requirement, calculating a robustness value, the robustness being calculated as the value p of a statistical test indicating whether or not the requirement considered is met, - for each requirement, comparing each robustness value with a predefined threshold according to a respective comparison criterion, and - determining the requirements with which the tracking system (10) complies, with the requirements determined as being compliant being the requirements for which it was determined that the comparison criterion is met in the comparison step.

Inventors

  • BRESSON, Roman
  • LABREUCHE, CHRISTOPHE
  • Honore, Nicolas
  • AHCENE, Azdine

Assignees

  • THALES

Dates

Publication Date
20260513
Application Date
20240705

Claims (14)

  1. CLAIMS 1. A method for assessing the conformity of a tracking system (10) to a set of requirements, at least one requirement being a mandatory requirement and at least one requirement being a recommended requirement, each requirement requiring that a physical quantity be greater than or equal to a threshold or that a physical quantity be less than or equal to a threshold, the assessment method being implemented by computer, the assessment method comprising at least the following steps: - acquiring a set of values, the set of values comprising the values accessible to the tracking system (10) over a predefined time interval, - for each requirement, calculating a robustness value, the robustness being calculated as the p-value of a statistical test indicating whether or not the requirement in question is met, - for each requirement, comparing each robustness value to a predefined threshold according to a respective comparison criterion, and - determining the requirements to which the tracking system (10) complies, the requirements determined as compliant being the requirements for which, at the comparison step, it has been determined that the comparison criterion is met.
  2. 2. Method according to claim 1, in which the method further comprises, for each robustness value not meeting the comparison criterion, a step of determining the cause of non-compliance with the comparison criterion, the cause being either that the requirement is not met or that the number of values is too low to guarantee a predefined reliability threshold for assessing compliance with the requirement in question.
  3. 3. Method according to claim 2, in which, when the determined cause is a number of values that is too low, the method comprises a step of deducing a number of additional values to be acquired in order to obtain an evaluation reliability greater than the predefined reliability threshold.
  4. 4. Method according to claim 3, in which the method comprises a reiteration of the acquisition, calculation and comparison steps until the number of values allows obtaining an evaluation reliability greater than the predefined reliability threshold or reaches a predefined maximum number with an evaluation reliability lower than the predefined threshold.
  5. 5. Method according to claim 4, in which the method further comprises a step of issuing an alert for requirements not met or for requirements for which the number of values of the acquisition, calculation and comparison steps has reached the predefined maximum number with an evaluation reliability lower than the predefined threshold.
  6. 6. A method according to claim 4 or 5, wherein the maximum number is between 40,000 and 60,000.
  7. 7. Method according to any one of claims 1 to 6, in which, during the calculation step, an integral of a regularized incomplete beta function is calculated.
  8. 8. The method of claim 7, wherein, in the calculating step, one p-value is equal to the result of the integral and another p-value is a linear function of the result of the integral.
  9. 9. Method according to any one of claims 1 to 8, in which during the calculation step, a distribution function of the reduced centered normal distribution is calculated.
  10. 10. Method according to any one of claims 1 to 9, in which the method further comprises: - a step of applying an evaluation function specific to each requirement to the values p to obtain evaluation values, and - a step of using the evaluation values to obtain an overall score evaluating the conformity of the tracking system (10) to all of the requirements.
  11. 11. A method according to any one of claims 1 to 10, wherein the number of requirements is greater than or equal to 20.
  12. 12. A method according to any one of claims 1 to 11, wherein the tracking system (10) provides measurement estimates from past data from a plurality of sensors (12), the set of values comprising the measurement estimates.
  13. 13. Evaluation module (22) of the conformity of a tracking system (10) to a set of requirements, at least one requirement being a mandatory requirement and at least a requirement being a recommended requirement, each requirement requiring that a physical quantity be greater than or equal to a threshold or that a physical quantity be less than or equal to a threshold, the evaluation module (22) being capable of: - acquiring a set of values, the set of values comprising the values accessible to the tracking system (10) over a predefined time interval, - for each requirement, calculating a robustness value, the robustness being calculated as the p-value of a statistical test indicating whether or not the requirement considered is met, - for each requirement, comparing each robustness value to a predefined threshold according to a respective comparison criterion, and - determining requirements with which the tracking system (10) complies, the requirements determined as compliant being the requirements for which, in the comparison step, it has been determined that the comparison criterion is met.
  14. 14. Tracking system (10) comprising an evaluation module (22) according to claim 13.

Description

Method for assessing the conformity of a tracking system using a limited number of measurements and associated devices FIELD OF THE INVENTION The present invention relates to a method for assessing the conformity of a tracking system to a set of requirements. The present invention also relates to an assessment module allowing the implementation of the method and a tracking system comprising such an assessment module. TECHNOLOGICAL BACKGROUND OF THE INVENTION The proposed invention is in the field of air traffic control. Airspace control is ensured by tracking all aircraft. This involves supervising air traffic to prevent collisions between aircraft and controlling traffic, both in cruising flight and around airports – takeoff and landing. To implement such control, air traffic controllers use an air traffic monitoring system based on tracking aircraft which is called a tracking system. The tracking system is capable in real time of estimating from data coming from sensors the best possible estimate of the position, heading and speed of each aircraft in a given flight information region. The flight information region is often designated by the acronym FIR referring to the corresponding English name of "Flight Information Region". All of the estimated data form a track. The quality of the track estimation (tracking) varies according to the configuration of the tracking system, the quality of the data acquired by the sensors or even environmental data such as the topology of the terrain or the weather. As such, it is desirable for an air traffic controller to know the quality of the estimation of the tracks reconstructed by the tracking system. For this, it is known to use the requirements of the ESASSP standard which is detailed in the document entitled "EUROCONTROL Specification for ATM Surveillance System Performance" (Volume 1) whose ISBN is 978-2-87497-022-1 and which was published in March 2012. More precisely, a set of requirements defined by this ESASSP standard is evaluated by comparison between the outputs of the tracking system and an estimate of the ideal trajectory of the aircraft. However, calculating the requirements of the standard requires obtaining a large quantity of measurement points, typically between 50,000 and 100,000. This poses a problem in practice for obtaining an on-the-fly evaluation and in particular detecting possible failures during operation. SUMMARY OF THE INVENTION There is therefore a need for a method for evaluating the quality of the tracking carried out by a tracking system with a reduced number of measurements. For this purpose, the description describes a method for assessing the conformity of a tracking system to a set of requirements, at least one requirement being a mandatory requirement and at least one requirement being a recommended requirement, each requirement requiring that a physical quantity be greater than or equal to a threshold or that a physical quantity be less than or equal to a threshold, the assessment method being implemented by computer, the assessment method comprising at least the following steps: - acquiring a set of values, the set of values comprising the values accessible to the tracking system over a predefined time interval, - for each requirement, calculating a robustness value, the robustness being calculated as the p-value of a statistical test indicating whether or not the requirement in question is met, - for each requirement, comparing each robustness value to a predefined threshold according to a respective comparison criterion, and - determining the requirements to which the tracking system complies, the requirements determined as compliant being the requirements for which, at the comparison step, it has been determined that the comparison criterion is met. According to particular embodiments, the evaluation method has one or more of the following characteristics, taken in isolation or in all technically possible combinations: - the method further comprises, for each robustness value not meeting the comparison criterion, a step of determining the cause of the non-compliance with the comparison criterion, the cause being either that the requirement is not met or that the number of values is too low to guarantee a predefined reliability threshold for assessing compliance with the requirement in question. - when the determined cause is too low a number of values, the method comprises a step of deducing a number of additional values to be acquired in order to obtain an assessment reliability greater than the predefined reliability threshold. - the method comprises a reiteration of the acquisition, calculation and comparison steps until the number of values is such that an evaluation reliability greater than the predefined reliability threshold can be obtained or reaches a predefined maximum number with an evaluation reliability lower than the predefined threshold. - the method further comprises a step of issuing an alert for requireme