Search

EP-4093643-B1 - METHOD FOR OPERATING AN AT LEAST PARTIALLY SELF-DRIVING VEHICLE, AND VEHICLE

EP4093643B1EP 4093643 B1EP4093643 B1EP 4093643B1EP-4093643-B1

Inventors

  • KAPOOR, Nikhil
  • SCHLICHT, PETER
  • VARGHESE, Serin

Dates

Publication Date
20260513
Application Date
20210324

Claims (9)

  1. Method for operating an at least partially automatedly driving vehicle (50), wherein sensor data (10) are captured by means of at least one sensor (51), wherein the captured sensor data (10) are reconstructed by means of a reconstruction method (30), wherein the captured sensor data (10) and the reconstructed sensor data (11) are both supplied to a machine-learning-based perception function (2), wherein, by means of a distance measure (31), a distance (32) is determined between outputs (20, 21) which are each generated by means of the perception function (2), wherein the determined distance (32) is compared with at least one predefined threshold value (25), and wherein at least one safety measure (15) is implemented when the determined distance (32) exceeds the at least one predefined threshold value (25), wherein a plurality of different threshold values (25) are predefined, and wherein safety measures (15) are selected depending on the different threshold values (25).
  2. Method according to claim 1, characterized in that the reconstruction method (30) comprises quilting and/or total variance minimization and/or smoothing and/or denoising and/or compression of the captured sensor data (10).
  3. Method according to either of the preceding claims, characterized in that, as a safety measure (15), the perception function (2) is deactivated.
  4. Method according to any of the preceding claims, characterized in that, as a safety measure (15), confidence relating to an output (20) of the perception function (2) is changed.
  5. Method according to any of the preceding claims, characterized in that, as a safety measure (15), at least one confidence value relating to captured sensor data (10) of at least one sensor (51) is changed and/or, as a safety measure (15), sensor data (10) of at least one sensor (51) are no longer taken into account by the perception function (2).
  6. Method according to any of the preceding claims, characterized in that, as a safety measure (15), at least one sensor configuration is modified.
  7. Method according to any of the preceding claims, characterized in that, as a safety measure (15), at least one fallback strategy for the automatedly driving vehicle (50) is activated.
  8. Method according to any of the preceding claims, characterized in that, as a safety measure (15), the automatedly driving vehicle (50) is transitioned into a safe state.
  9. Vehicle (50), wherein the vehicle (50) is driven at least partially automatedly, comprising: at least one sensor (51), wherein the at least one sensor (51) is configured to capture sensor data (10), and a control device (1), wherein the control device (1) is configured to provide a machine-learning-based perception function (2), to reconstruct the captured sensor data (10) by means of a reconstruction method (30), to supply both the captured sensor data (10) and the reconstructed sensor data (11) to the perception function (2), to determine, by means of a distance measure (31), a distance (32) between outputs (20, 21) which are each generated by means of the perception function (2), to compare the determined distance (32) with at least one predefined threshold value (25), and to implement at least one safety measure (15) when the determined distance (32) exceeds the at least one predefined threshold value (25), wherein a plurality of different threshold values (25) are predefined, and wherein safety measures (15) are selected depending on the different threshold values (25).

Description

The invention relates to a method for operating a vehicle that is at least partially automated and to a vehicle. Machine learning, for example based on neural networks, has great potential for application in modern driver assistance systems and automated vehicles. Functions based on deep neural networks process sensor data (for example, from cameras, radar, or lidar sensors) to derive relevant information. This information includes, for example, the type and position of objects in the vehicle's environment, the behavior of the objects, or the road geometry or topology. A key feature in the development of deep neural networks (the training) lies in purely data-driven parameter fitting without expert intervention: Here, the deviation of an output (for a given parameterization) of a neural network from a ground truth is determined (the so-called loss). The loss function used is chosen in such a way that the parameters of the neural network depend on it in a differentiable manner. Within the framework of the gradient descent method, the parameters of the neural network are adjusted in each training step as a function of the derivative of the deviation (determined from several examples). These training steps are repeated very often until the loss no longer decreases. In this approach, the parameters of the neural network are determined without expert assessment or semantically motivated modeling. However, neural networks also have disadvantages. For example, attacks based on adversarial interference in the sensor data/input data can lead to misclassification or incorrect semantic segmentation despite the semantically unchanged content of the captured sensor data. Furthermore, the performance of a neural network is only consistently high if the input data originates from the data domain on which the neural network was trained (i.e., in-sample data). If, on the other hand, the input data comes from... If a different data domain (out-of-sample data) is used, the quality of the output of the neural network may decrease. Particularly in the field of automated driving, where high safety requirements are demanded, out-of-sample data must be able to be detected before further processing takes place. From Chuan Guo et al., Countering Adversarial Images Using Input Transformations, arXiv:1711.00117v3 [cs.CV], Jan. 25, 2018, https://arxiv.org/pdf/1711.00117.pdf , a quilting method and a total variance minimization method for eliminating adversarial disturbances in image data are known. From Y. Bakhti et al., DDSA: A Defense Against Adversarial Attacks Using Deep Denoising Sparse Autoencoder, IEEE Access, Vol. 7, pp. 160397-160407, 2019, doi: 10.1109/ACCESS.2019.2951526 , is a known method for defending against adversarial attacks. From D. Meng and H. Chen, MagNet: a Two-Pronged Defense against Aversarial Examples, Proc. of the 2017 ACM SIGSAC Conference on Computer and Communication Security, CCS '17, October 30, 2017, pp. 135-147, New York, USA, DOI: 10.1145/3133956.3134057 , is a known method for defending against adversarial attacks. From M. Zhang et al., DeepRoad: GAN-Based Metamorphic Testing and Input Validation Framework for Autonomous Driving Systems, Proc. of the 33rd ACM/IEEE International Conference on Automated Software Engineering, ASE 2018, ACM Press, New York, USA, September 3, 2018, pp. 132-142, DOI: 10.1145/3238147.3238187 , is a known method for checking input data of an autonomous driving system. From the US 2019/0135300 A1 A method and a device for unsupervised multimodal anomaly detection in autonomous vehicles are known. An example involves obtaining first sensor data from a first sensor and second sensor data from a second sensor, where the first sensor of a first sensor type is different from the second sensor type of the second sensor; generating first coded sensor data based on the first sensor data and second coded sensor data based on the second sensor data; and generating a context-aware fused sensor data representation of the first and second sensor data based on the first and second sensor data. second coded sensor data; generating first and second reconstructed sensor data based on the contextual fused sensor data representation; determining a deviation estimate based on the first and second reconstructed sensor data, wherein the deviation estimate is representative of a deviation between: the first reconstructed sensor data and the first sensor data; and detecting an anomaly in the deviation estimate, wherein the anomaly indicates an error associated with the first sensor. The invention is based on the objective of creating a method for operating a vehicle that is at least partially automated and a vehicle in which out-of-sample data, in particular caused by adversarial interference, can be detected and a reaction can be made after detection. The problem is solved according to the invention by a method with the features of claim 1 and a vehicle with the features of claim 9. Advantageous e