Search

EP-4409527-B1 - METHOD AND SYSTEM FOR LANE TRACKING FOR AN AUTONOMOUS VEHICLE

EP4409527B1EP 4409527 B1EP4409527 B1EP 4409527B1EP-4409527-B1

Inventors

  • Chavali, Pothuraju
  • HARIBHAKTA, Sri Hari Bhupala
  • SRIDHAR, MURALIKRISHNA

Dates

Publication Date
20260506
Application Date
20220927

Claims (15)

  1. A computer-implemented method of training a lane tracking system (103) for an autonomous vehicle (101), the method comprising: receiving, by a lane tracking system (103), ground truth values corresponding to a plurality of lane boundary detection points and measured values corresponding to the plurality of lane boundary detection points, from a lane boundary detecting system (105) associated with the lane tracking system (103); determining, for ground truth clothoid point and measured clothoid point formed using a ground truth set and a measured set, respectively, by the lane tracking system (103), co-efficient values of clothoid parameters, to model lane boundaries of a lane, wherein the ground truth set comprises a subset of continuous lane boundary detection points and the corresponding ground truth values, and the measured set comprises a subset of continuous lane boundary detection points and the corresponding measured values; determining, by the lane tracking system (103), Kalman filter parameters for the co-efficient values of the clothoid parameters determined for the measured clothoid points, to track the lane boundaries of the lane, wherein the Kalman filter parameters are determined using at least one neural network; updating, by the lane tracking system (103), the co-efficient values of the clothoid parameters determined for the measured clothoid point, using the corresponding Kalman filter parameters; reconstructing, by the lane tracking system (103), the measured clothoid point, using the corresponding updated co-efficient values of the clothoid parameters, wherein each reconstructed measured clothoid point enables the lane tracking system (103) to track the lane boundaries for the autonomous vehicle (101); and minimizing, by the lane tracking system (103), a training error based on a difference between the reconstructed measured clothoid point and the corresponding ground truth set, in each cycle, until the training error is below a predefined threshold.
  2. The method as claimed in claim 1, wherein the plurality of lane boundary detection points correspond to a left lane boundary and a right lane boundary of the lane.
  3. The method as claimed in any preceding claim comprises dynamically selecting, by the lane tracking system (103), the ground truth set and the measured set required for generating the ground truth clothoid point and the measured clothoid point, respectively.
  4. The method as claimed in any preceding claim comprises determining the plurality of lane boundary detection points using at least one of lane data received from one or more sensors (113) configured in the vehicle and an image frame of the lane received in real-time, wherein the image frame of the lane is received from an image capturing device (115) associated with the autonomous vehicle (101).
  5. The method as claimed in claim 4 comprises retrieving the at least one of the lane data and the image frame of the lane from a database configured to store the lane data and the image frame captured in real-time.
  6. The method as claimed in any preceding claim, wherein the clothoid parameters comprise initial curvature of lane boundary c 0 curvature rate c 1 of the lane boundary, and heading angle β with respect to vehicle's driving direction.
  7. The method as claimed in any preceding claim, wherein the at least one neural network comprises at least one of a neural network with temporal memory,
  8. The method as claimed in any preceding claim, wherein the at least one neural network comprises a recurrent neural network.
  9. The method as claimed in any preceding claim, wherein the at least one neural network comprises a long short-term memory neural network.
  10. The method as claimed in any preceding claim, wherein determining the Kalman filter parameters using the at least one neural networks comprises: providing, by the lane tracking system (103), the co-efficient values of the clothoid parameters determined for the measured clothoid point as an input to a first neural network, wherein the first neural network is trained based on historical co-efficient values of the clothoid parameters determined for clothoid points formed using historical measured and ground truth sets; determining, by the lane tracking system (103), a measurement noise covariance matrix R using the co-efficient values of the clothoid parameters determined for the measured clothoid point, using the first neural network; predicting, by the lane tracking system (103), a state transition Y p of the co-efficient values of the clothoid parameters determined for the measured clothoid point, from one image frame to another image frame, based on velocity of the autonomous vehicle (101) moving along the lane and time difference between consecutive image frames; determining, by the lane tracking system (103), a process noise covariance matrix Q using the predicted state transition as an input to a second neural network, wherein the second neural network is trained using historical autonomous vehicle (101) velocity values and time difference values; predicting, by the lane tracking system (103), an error covariance P p of the predicted state transition using the determined process noise covariance matrix Q; and determining, by the lane tracking system (103), the Kalman filter parameters for the co-efficient values of the clothoid parameters determined for the measured clothoid point, based on the predicted state transition Y p and covariance P p , the determined measurement noise covariance matrix R and the co-efficient values of the clothoid parameters determined for the measured clothoid point.
  11. The method as claimed in any preceding claim further comprises: adding, by the lane tracking system (103), an initial lateral offset between the lane boundaries and the autonomous vehicle (101), to the reconstructed measured clothoid point.
  12. A computer-implemented method of lane tracking for an autonomous vehicle (101), the method comprising: receiving, by a lane tracking system (103), measured values corresponding to a plurality of lane boundary detection points, from a lane boundary detecting system (105) associated with the lane tracking system (103); determining, for a measured clothoid point formed using a measured set, by the lane tracking system (103), co-efficient values of clothoid parameters, to model lane boundaries of a lane, wherein the measured set comprises a subset of continuous lane boundary detection points and the corresponding measured values; determining, by the lane tracking system (103), Kalman filter parameters for the co-efficient values of the clothoid parameters determined for the measured clothoid points, to track the lane boundaries of the lane, wherein the Kalman filter parameters are determined using at least one neural network; updating, by the lane tracking system (103), the co-efficient values of the clothoid parameters determined for the measured clothoid point, using the corresponding Kalman filter parameters; and reconstructing, by the lane tracking system (103), the measured clothoid point, using the corresponding updated co-efficient values of the clothoid parameters, wherein each reconstructed measured clothoid point enables the lane tracking system (103) to track the lane boundaries for the autonomous vehicle (101).
  13. The method as claimed in claim 12, wherein the at least one neural network comprises at least one of a neural network with temporal memory, a recurrent neural network, and a long short-term memory neural network.
  14. A lane tracking system (103) for an autonomous vehicle (101), the lane tracking system (103) comprising: a processor (107); and a memory (111) communicatively coupled to the processor (107), wherein the memory (111) stores the processor (107) instructions, which, on execution, causes the processor (107) to train the lane tracking system (103), wherein for training, the processor (107) is configured to perform the method of any one of claims 1 to 11.
  15. A lane tracking system (103) for an autonomous vehicle (101), the lane tracking system (103) comprising: a processor (107); and a memory (111) communicatively coupled to the processor (107), wherein the memory (111) stores the processor (107) instructions, which, on execution, causes the processor (107) to perform the method of any one of claims 12 to 13.

Description

Technical Field The present subject matter relates generally to the field of autonomous vehicles, and more particularly, but not exclusively to a method and a system for lane tracking for an autonomous vehicle. Background Nowadays, automotive industries have started to move towards autonomous vehicles. Autonomous vehicles, as used in this description and claims, are vehicles that are capable of sensing environment around them in order to move on the roads with or without human intervention. The autonomous vehicles sense the environment with the help of sensors configured in the autonomous vehicles such as Laser, Light Detection and Ranging (LIDAR), Global Positioning System (GPS), computer vision and the like. The autonomous vehicles highly rely on lane detection and tracking on the road for navigating smoothly. Existing lane detection and tracking techniques may use Kalman filters for tracking the lane boundaries. Especially, Kalman filters may be used to predict lane parameters and to smooth the output of a lane tracker which tracks the lane boundaries. Generally, Kalman filters are opted while tracking lane boundaries as the Kalman filter has the capability of estimating dynamics of state vectors, even in the presence of noisy measurements or noisy processes. Major parameters that help in determining the Kalman filter are process noise covariance matrix (Q) and measurement noise covariance matrix (R). The existing lane detection and tracking techniques that rely on Kalman filters for tracking lane boundaries, use predefined or fixed Q and R values for determining Kalman filters. In reality, Q and R are dynamically varying parameters based on scenarios, detectors used for measurement, kind of process used for measurement and tracking, and the like. However, the existing techniques fail to incorporate dynamic nature of Q and R, and instead use fixed or predefined values for Q and R, which affects the accuracy of prediction performed based on the Kalman filters for lane tracking. Inaccurate lane tracking may generate incorrect steering commands and warning signals to the autonomous vehicle, which may jeopardize vehicle safety. Additionally, since Q and R values are fixed in the existing techniques, existing techniques lack the flexibility to incorporate changes occurring in state over time, thus restricting the predictions to only few types or small range of lane structures. Therefore, there is a need for a method that can perform lane tracking using Kalman filters, with enhanced accuracy and flexibility. US 2020/216076 Al discloses a method for determining a current state vector describing location and heading of an ego-vehicle with respect to a lane boundary of a road comprises a step of obtaining road sensor data from at least one road sensor of the ego-vehicle detecting the lane boundaries of the road. The method includes another step, where a measured state vector of the ego-vehicle is calculated from the road sensor data. Furthermore, motion state data related to current heading and velocity of the ego-vehicle is obtained and a predicted state vector of the ego-vehicle is calculated based on the motion state data of the ego-vehicle and a previous state vector of the ego-vehicle. Finally a current state vector is determined by calculating a weighted average of the measured state vector and the predicted state vector of the ego-vehicle. The weights are determined based on characteristics of an upcoming section of the road. JP 2016 057750 A discloses a lane boundary detection means that includes a lane boundary detection part that detects a boundary of a lane where an own vehicle is traveling, on the basis of an image created by an imaging device. An own vehicle travel lane estimation part estimates a lane parameter of the travel lane where the own vehicle is traveling, on the basis of a vehicle travel locus corresponding to an absolute location of the own vehicle detected by the location detection part and the detected lane boundary, and creates data indicating a virtual travel lane where the own vehicle is traveling, on the basis of the estimated lane parameter. "Long Short-Term Memory Kalman Filters:Recurrent Neural Estimators for Pose Regularization", by Huseyin Coskun et.al., Arxiv.Org, Cornell University Library, 201 Olin Library Cornell University Ithaca, Ny 14853, 6 August 2017 (2017-08-06), Xp080951672, herein referred to as Huseyin et. al., discloses the use of Kalman filters. Huseyin et. al discloses that one-shot pose estimation for tasks such as body joint localization, camera pose estimation, and object tracking are generally noisy, and temporal filters have been extensively used for regularization. One of the most widely-used methods is the Kalman filter, which is both extremely simple and general. However, Kalman filters require a motion model and measurement model to be specified a priori, which burdens the modeler and simultaneously demands that the use of explicit models that are often