Search

US-20260126797-A1 - VEHICLE CONTROL DEVICE, VEHICLE, FEATURE OPTIMIZATION DEVICE, SYSTEM, METHOD AND COMPUTER PROGRAM PRODUCT

US20260126797A1US 20260126797 A1US20260126797 A1US 20260126797A1US-20260126797-A1

Abstract

The present subject matter relates to a control device, a feature optimization device, a system, a vehicle, a method and a computer program product for controlling a movement of a vehicle, in particular for controlling a movement of an autonomous driving vehicle. Based on a current image of the vehicle's environment a map is selected from a plurality of maps stored in a database, a global pose of the vehicle is determined based on a comparison of the current image and the selected map and the movement of the vehicle is controlled based on the determined global pose thereof.

Inventors

  • Quan Nguyen
  • Anthony Emeka OHAZULIKE
  • Alireza Ahrabian
  • Ioannis Souflas

Assignees

  • ASTEMO, LTD.

Dates

Publication Date
20260507
Application Date
20251219
Priority Date
20230621

Claims (18)

  1. 1 . A vehicle control device for controlling a movement of a vehicle being equipped with one or more sensors comprising: a localisation unit configured to determine a global pose of the vehicle; and a control unit configured to control the movement of the vehicle based on the determined global pose, wherein the localisation unit includes a visual position localisation unit configured to obtain a current image of an environment of the vehicle from the one or more sensors, to send the current image to a feature optimisation device storing one or more sets of optimised features of the environment of the vehicle in a database, to receive, from the optimisation device, at least one optimised feature from the sets of optimised features based on the current image, and to determine the global pose of the vehicle based on a comparison of the current image and the at least one received optimised feature (M1).
  2. 2 . The vehicle control device according to claim 1 , wherein the at least one optimised feature received by the localisation unit is selected from a plurality of sets of optimised features, each of which is corresponding to one of a plurality of runs, stored in one or more databases based on the current image.
  3. 3 . The vehicle control device according to claim 1 , wherein the visual position localisation unit is configured to perform the comparison by determining one or more features in the current image and the at least one optimised feature received from the optimisation device and computing a relative pose between corresponding features, and to determine the global pose of the vehicle by combining the computed relative pose with a global camera pose derived from the optimised feature (M1, M2).
  4. 4 . The vehicle control device according to claim 1 , wherein the localisation unit further includes a GNSS receiver configured to receive a signal from a global satellite navigation system, and if a signal strength of a received GNSS signal is equal to or greater than a predetermined threshold, the localisation unit is configured to determine the global pose of the vehicle based on the GNSS signal, and if the signal strength of the GNSS signal is lower than the predetermined threshold, the visual position localisation unit is configured to determine the global pose of the vehicle.
  5. 5 . The vehicle control device according to claim 4 , further comprising: an image processing unit configured to obtain an image sequence (I1, I2) from the one or more sensors including a predetermined number of images (I 11 -I 14 , I 21 -I 24 ) of the environment of the vehicle when the vehicle passes through a region where the received GNSS signal is lower than the predetermined threshold, wherein at least a first image (I 11 , I 21 ) and a last image (I 14 , I 24 ) of the image sequence (I1, I2) include a global pose of the vehicle determined based on the GNSS signal, and to send the obtained image sequence (I1, I2) to the feature optimisation device.
  6. 6 . A feature optimisation device providing at least one optimised feature for controlling a movement of a vehicle being equipped with one or more sensors, the feature optimisation device comprising: an input unit configured to receive at least two image sequences (I1, I2) of an environment of the vehicle from a vehicle control device, the image sequences (I1, I2) obtained along different trajectories in the same environment, and to determine one or more features from each image (I 11 -I 14 , I 21 -I 24 ) of the received image sequences (I1, I2); an optimization unit configured to compute, based on the determined one or more features, a first constraint (C 1 , C 2 ) between two successive images (I 11 -I 14 , I 21 -I 24 ) of the at least two image sequences (I1, I2) and a second constraint (C12) between an image (I 11 -I 14 ) of a first image sequence (I1) and an image (I 21 -I 24 ) of a second image sequence (I2), to optimize at least one of the determined features of each image (I 11 -I 14 , I 21 -I 24 ) of the at least two image sequences (I1, I2) based on the first and second constraints (C 1 , C 2 , C 12 ), and to store each of the image sequences (I1, I2) and/or the optimised features in at least two sets of optimised features (M1, M2) in one or more databases; a selection unit configured to receive a current image of the vehicle's environment from the vehicle control device, to select at least one optimised feature from the at least two sets of optimised features (M1, M2) stored in the database based on the current image, and to transmit the selected at least one optimised feature to the vehicle control device.
  7. 7 . The feature optimization device according to claim 6 , wherein the selection unit is configured to determine one or more features from the received current image, and to select one or more optimised features by comparing the determined features of the current image with the optimized features of the sets of optimised features (M1, M2) stored in the database.
  8. 8 . The feature optimization device according to claim 6 , wherein the selection unit is configured to use a probabilistic model and/or an AI model to select optimised features from the sets of optimised features (M1, M2) stored in the database.
  9. 9 . The feature optimization device according to claim 6 , wherein the input unit is configured to determine one or more reference points in each image (I 11 -I 14 , I 21 -I 24 ) of the at least two image sequences (I1, I2) as the one or more features, and the optimization unit is configured to compute the first constraint (C 1 , C 2 ) by determining a first relative pose between two corresponding reference points in the successive images (I 11 -I 14 , I 21 -I 24 ) of the at least two image sequences (I1, I2), and to compute the second constraint (C 12 ) by determining a second relative pose between two corresponding reference points in the image of the first image sequence (I 11 -I 14 ) and the image of the second image sequence (I 21 -I 24 ).
  10. 10 . The feature optimization device according to claim 6 , wherein the input unit is configured to determine an estimated camera pose of each image (I 11 -I 14 , I 21 -I 24 ) of the at least two image sequences (I1, I2) for the at least one feature to be optimized, and the optimization unit is configured to optimize the estimated camera pose based on the first and second constraints (C 1 , C 2 , C 12 ), to generate a global camera pose for each image (I 11 -I 14 , I 21 -I 24 ) of the sets of optimised features (M1, M2).
  11. 11 . The feature optimization device according to claim 9 , wherein the input unit is configured to determine a global pose of the vehicle in a first and a last image (I 11 , I 21 , I 14 , I 24 ) of the at least two image sequences (I1, I2) based on a GNSS signal received by the one or more sensors, and the optimization unit is configured to optimize the estimated camera pose of each image (I 11 -I 14 , I 21 -I 24 ) of the at least two image sequences (I1, I2) based on the first and second constraints (C 1 , C 2 , C 12 ) and the global camera pose in the first and last image (I 11 , I 21 , I 14 , I 24 ).
  12. 12 . The feature optimization device according to claim 6 , wherein if the input unit receives a new image sequence from the vehicle control device, the input unit is configured to determine one or more features from each image of the received new image sequence; and the optimization unit is configured to compute, based on the determined features, a first constraint between two successive images of the new image sequence and a second constraint between an image of the new image sequence and an image of the at least two maps already stored in the database, to optimize at least one of the determined features of each image of the new image sequence based on the first and second constraints, and to store the new image sequence with the set of optimized features in the database.
  13. 13 . The feature optimization device according to claim 12 , wherein the optimization unit is configured to further optimise the at least one of the features of the at least two sets of optimised features (M1, M2) already stored in the database based on the second constraint between the image of the new image sequence and the image of the at least two sets of optimised features (M1, M2).
  14. 14 . A system for controlling a movement of a vehicle, comprising: a vehicle control device for controlling a movement of a vehicle being equipped with one or more sensors comprising: a localisation unit configured to determine a global pose of the vehicle; and a control unit configured to control the movement of the vehicle based on the determined global pose, wherein the localisation unit includes a visual position localisation unit configured to obtain a current image of an environment of the vehicle from the one or more sensors, to send the current image to a feature optimisation device storing one or more sets of optimised features of the environment of the vehicle in a database, to receive, from the optimisation device, at least one optimised feature from the sets of optimised features based on the current image, and to determine the global pose of the vehicle based on a comparison of the current image and the at least one received optimised feature (M1); a feature optimization device according to claim 6 ; and one or more sensors attached to the vehicle and configured to transmit a plurality of sensor signals to the control device and/or the feature optimization device.
  15. 15 . A vehicle comprising: a vehicle control device according to claim 1 , and one or more sensors attached to the vehicle and configured to transmit a plurality of sensor signals to the control device and/or the feature optimization device.
  16. 16 . The vehicle according to claim 15 , further comprising: a feature optimization device providing at least one optimised feature for controlling a movement of a vehicle being equipped with one or more sensors, the feature optimisation device comprising: an input unit configured to receive at least two image sequences (I1, I2) of an environment of the vehicle from a vehicle control device, the image sequences (I1, I2) obtained along different trajectories in the same environment, and to determine one or more features from each image (I 11 -I 14 , I 21 -I 24 ) of the received image sequences (I1, I2); an optimization unit configured to compute, based on the determined one or more features, a first constraint (C 1 , C 2 ) between two successive images (I 11 -I 14 , I 21 -I 24 ) of the at least two image sequences (I1, I2) and a second constraint (C12) between an image (I 11 -I 14 ) of a first image sequence (I1) and an image (I 21 -I 24 ) of a second image sequence (I2), to optimize at least one of the determined features of each image (I 11 -I 14 , I 21 -I 24 ) of the at least two image sequences (I1, I2) based on the first and second constraints (C 1 , C 2 , C 12 ), and to store each of the image sequences (I1, I2) and/or the optimised features in at least two sets of optimised features (M1, M2) in one or more databases; a selection unit configured to receive a current image of the vehicle's environment from the vehicle control device, to select at least one optimised feature from the at least two sets of optimised features (M1, M2) stored in the database based on the current image, and to transmit the selected at least one optimised feature to the vehicle control device.
  17. 17 . A method for controlling a movement of a vehicle equipped with one or more sensors, comprising the steps: receiving at least two image sequences (I1, I2) of an environment of the vehicle from a vehicle control device, the image sequences obtained along different trajectories in the same environment, and determining one or more features from each image (I 11 -I 14 , I 21 -I 24 ) of the received image sequences (I1, I2); computing, based on the determined features, a first constraint (C 1 , C 2 ) between two successive images (I 11 -I 14 , I 21 -I 24 ) of the at least two image sequences (I1, I2) and a second constraint (C12) between an image (I 11 -I 14 ) of a first image sequence (I1) and an image (I 21 -I 24 ) of a second image sequence (I2); optimizing at least one feature of each image (I 11 -I 14 , I 21 -I 24 ) of the at least two image sequences (I1, I2), based on the first and second constraints (C1, C2, C12), to generate a global camera pose for each image (I 11 -I 14 , I 21 -I 24 ), and storing each of the image sequences (I1, I2) in a database, receiving a current image of the vehicle's environment from the vehicle control device; selecting at least one optimised features from at least two sets of optimised features (M1, M2) stored in the database based on the current image; determining the global pose of the vehicle based on a comparison of the feature in the current image and the selected optimised feature of the set of optimised features (M1, M2); and controlling the movement of the vehicle based on the determined global pose.
  18. 18 . A computer program product storable in a memory comprising instructions which, when carried out by a computer, cause the computer to perform the method according to claim 17 .

Description

CROSS-REFERENCE TO RELATED APPLICATION(S) This application is a continuation of International Application No. PCT/JP2024/021764, filed Jun. 14, 2024, which claims priority to and the benefit of German Patent Application No. 10 2023 205 806.5, filed Jun. 21, 2023, the entire contents of both of which are incorporated herein by reference. This application is also a continuation-in-part of International Application No. PCT/JP2024/024291, filed Jul. 4, 2024, which claims priority to and the benefit of German Patent Application 10 2023 206 626.2, filed Jul. 12, 2023, the entire contents of both of which are incorporated herein by reference. TECHNICAL FIELD The present subject matter relates to a vehicle control device, a feature optimization device, a system, a vehicle, a method and a computer program product for controlling a movement of a vehicle. The present subject matter relates to a vehicle control device, a vehicle including a vehicle control device, a prediction device, a system, a method, and a computer program product for controlling a first vehicle when a second vehicle is detected in a vicinity of the first vehicle. BACKGROUND ART Vehicle localisation is one of the most important tasks related to automated driving. Although satellite navigation systems can provide a high localisation accuracy, they are limited to applications where there is sufficient satellite signal availability. However, when driving through an urban area with a high density of towering buildings, signals from navigational satellites may be severely affected. To overcome the limitations of satellite navigation systems, information from multiple sensors such as GNSS, LiDAR, camera, accelerometer, and wheel encoder can be fused for creating a full 3D map of a vehicle environment. In this context, Patent Literature 1 relates to systems and methods for updating an HD map. The system includes a communication interface to receive sensor data acquired of a target region by at least one sensor equipped on a vehicle as the vehicle travels along a trajectory via a network. The system further includes a storage to store the HD map, and a processor that constructs the HD map from a plurality of local HD maps. In order to increase active safety, current vehicles are equipped with a variety of advanced driver assistance systems (ADAS) warning a driver in critical situations or independently supply vehicle guidance tasks. In particular, road intersections are notorious accident blackspots where many accidents occur when turning onto another road. In order to deal with such critical situations, an intersection assistant can support the driver when entering into an intersection. To further increase safety in this regard, knowledge of future trajectories of vehicles (target vehicles) in the vicinity of one's own vehicle (ego vehicle) is essential. In this context, Patent Literature 1 describes a predicting apparatus including an acquiring unit that acquires a movement history of another vehicle from the other vehicle; a predicting unit that predicts a behaviour of the other vehicle based on the movement history; and an output unit that outputs driving support information based on the prediction. The acquiring unit acquires the movement history from the other vehicle in a communication range of inter-vehicle communication, and the predicting unit predicts the behaviour of the other vehicle in the communication range of the inter-vehicle communication. CITATION LIST Patent Literature Patent Literature 1: US 2020/341150 A1Patent Literature 1: US 2018/0370530 A1 SUMMARY OF INVENTION Technical Problem The use of a full 3D map of the vehicle environment, however, has limitations in terms of data efficiency and scalability. To overcome these limitations a solution is proposed which allows accurate and reliable vehicle localisation while reducing computationally intensive operations. However, performing prediction of the future trajectory of a target vehicle still requires improvement and, for example, intersections still may pose problems for known systems which should be overcome for increasing comfort of a driver of an ego vehicle and safety. Solution to Problem The herein described subject matter addresses the technical object of providing data-efficient and accurate vehicle localisation in regions with limited GNSS availability. This object is achieved by the subject matter of the independent claims. Further preferred developments are described in the dependent claims. According to the subject matter set forth in the appended claims, there is proposed a vehicle control device, a feature optimization device, a system, a vehicle, a method and a computer program product for controlling a movement of a vehicle, in particular for controlling a movement of an autonomous driving vehicle. It may also be possible that the vehicle is a non-autonomous driving vehicle having a plurality of advanced driver assistance systems (ADAS). Thus, the meaning of “controlli