US-12625495-B2 - Jointly learnable behavior and trajectory planning for autonomous vehicles

Abstract

Systems and methods for generating motion plans for autonomous vehicles are provided. An autonomous vehicle can include a machine-learned motion planning system including one or more machine-learned models configured to generate target trajectories for the autonomous vehicle. The model(s) include a behavioral planning stage configured to receive situational data based at least in part on the one or more outputs of the set of sensors and to generate behavioral planning data based at least in part on the situational data and a unified cost function. The model(s) includes a trajectory planning stage configured to receive the behavioral planning data from the behavioral planning stage and to generate target trajectory data for the autonomous vehicle based at least in part on the behavioral planning data and the unified cost function.

Inventors

• Raquel Urtasun
• Yen-Chen Lin
• Andrei Pokrovsky
• Mengye Ren
• Abbas Sadat
• Ersin Yumer

Assignees

• AURORA OPERATIONS, INC.

Dates

Publication Date

20260512

Application Date

20230719

Claims (20)

1. 1 . An autonomous vehicle, comprising: one or more processors; and one or more non-transitory computer-readable media that store instructions that are executable by the one or more processors to cause the one or more processors to perform operations, the operations comprising: obtaining, from a plurality of sensors of the autonomous vehicle, situational data associated with an environment external to the autonomous vehicle; generating, by a machine-learned motion planning system and using a unified cost function, a plurality of possible trajectories based at least in part on the situational data; and controlling a motion of the autonomous vehicle in the environment according to a trajectory of the plurality of possible trajectories selected, by the machine-learned motion planning system and using the unified cost function.
2. 2 . The autonomous vehicle of claim 1 , wherein the plurality of possible trajectories are generated by sampling one or more values for the plurality of possible trajectories based at least in part on the situational data.
3. 3 . The autonomous vehicle of claim 2 , wherein the one or more values comprise one or more lateral offset values determined with respect to a path.
4. 4 . The autonomous vehicle of claim 1 , wherein a respective possible trajectory comprises a sampled longitudinal trajectory and a sampled lateral trajectory.
5. 5 . The autonomous vehicle of claim 4 , wherein the sampled longitudinal trajectory is parameterized in terms of time, and wherein the sampled lateral trajectory is parameterized in terms of distance along the sampled longitudinal trajectory.
6. 6 . The autonomous vehicle of claim 1 , wherein the unified cost function is optimized by training the machine-learned motion planning system end-to-end.
7. 7 . The autonomous vehicle of claim 6 , wherein training machine-learned motion planning system end-to-end comprises, for a training input comprising training situational data and a ground truth output comprising a reference trajectory: generating, by the machine-learned motion planning system using the unified cost function, a plurality of possible training trajectories based at least in part on the unified cost function and the training input; selecting, by the machine-learned motion planning system using the unified cost function, a particular training trajectory of the plurality of possible training trajectories; determining a loss based at least in part on the reference trajectory and the particular training trajectory; and updating the unified cost function based at least in part on the loss.
8. 8 . The autonomous vehicle of claim 7 , wherein training machine-learned motion planning system end-to-end comprises updating one or more parameters of the machine-learned motion planning system based at least in part on the loss.
9. 9 . The autonomous vehicle of claim 1 , wherein the operations comprise: determining, using the unified cost function, one or more control parameters for executing the particular trajectory.
10. 10 . A computer-implemented method, comprising: obtaining, from a plurality of sensors of an autonomous vehicle, situational data associated with an environment external to the autonomous vehicle; generating, by a machine-learned motion planning system using a unified cost function, a plurality of possible trajectories based at least in part on the situational data; and controlling a motion of the autonomous vehicle in the environment according to a trajectory of the plurality of possible trajectories selected by the machine-learned motion planning system and using the unified cost function.
11. 11 . The computer-implemented method of claim 10 , wherein the plurality of possible trajectories are generated by sampling one or more values for the plurality of possible trajectories based at least in part on the situational data.
12. 12 . The computer-implemented method of claim 11 , wherein the one or more values comprise one or more lateral offset values determined with respect to a path.
13. 13 . The computer-implemented method of claim 10 , wherein a respective possible trajectory comprises a sampled longitudinal trajectory and a sampled lateral trajectory.
14. 14 . The computer-implemented method of claim 13 , wherein the sampled longitudinal trajectory is parameterized in terms of time, and wherein the sampled lateral trajectory is parameterized in terms of distance along the sampled longitudinal trajectory.
15. 15 . The computer-implemented method of claim 14 , wherein the unified cost function is optimized by training the machine-learned motion planning system end-to-end.
16. 16 . The computer-implemented method of claim 15 , wherein training machine-learned motion planning system end-to-end comprises, for a training input comprising training situational data and a ground truth output comprising a reference trajectory: generating, by the machine-learned motion planning system using the unified cost function, a plurality of possible training trajectories based at least in part on the unified cost function and the training input; selecting, by the machine-learned motion planning system using the unified cost function, a particular training trajectory of the plurality of possible training trajectories; determining a loss based at least in part on the reference trajectory and the particular training trajectory; and updating the unified cost function based at least in part on the loss.
17. 17 . The computer-implemented method of claim 16 , wherein training machine-learned motion planning system end-to-end comprises updating one or more parameters of the machine-learned motion planning system based at least in part on the loss.
18. 18 . The computer-implemented method of claim 10 , comprising: determining, using the unified cost function, one or more control parameters for executing the particular trajectory.
19. 19 . One or more non-transitory computer-readable media that store instructions that are executable by one or more processors to cause a computing system to perform operations, the operations comprising: obtaining, from a plurality of sensors of the autonomous vehicle, situational data associated with an environment external to the autonomous vehicle; generating, by a machine-learned motion planning system and using a unified cost function, a plurality of possible trajectories based at least in part on the situational data; and controlling a motion of the autonomous vehicle in the environment according to a trajectory of the plurality of possible trajectories selected, by the machine-learned motion planning system and using the unified cost function.
20. 20 . The one or more non-transitory computer-readable media of claim 19 , wherein the plurality of possible trajectories are generated by sampling one or more values for the plurality of possible trajectories based at least in part on the situational data.

Description

PRIORITY CLAIM The present application is a continuation of U.S. Non-provisional Patent Application No. 16/825,049 having a filing date of Mar. 20, 2020, which claims priority to and the benefit of U.S. Provisional Patent Application No. 62/955,708, titled “Jointly Learnable Behavior and Trajectory Planning for Autonomous Vehicles,” and filed on Dec. 31, 2019. Applicant claims priority to and the benefit of each of such applications and incorporates all such applications herein by reference in its entirety. FIELD The present disclosure relates generally to improving the ability of computing devices to plan motion paths for autonomous vehicles. BACKGROUND An autonomous vehicle is a vehicle that is capable of sensing its environment and navigating without human input. In particular, an autonomous vehicle can observe its surrounding environment using a variety of sensors and can attempt to comprehend the environment by performing various processing techniques on data collected by the sensors. Given knowledge of its surrounding environment, the autonomous vehicle can identify an appropriate motion path for navigating through such surrounding environment. SUMMARY Aspects and advantages of embodiments of the present disclosure will be set forth in part in the following description, or may be learned from the description, or may be learned through practice of the embodiments. One example aspect of the present disclosure is directed to an autonomous vehicle including a set of sensors configured to generate one or more outputs based at least in part on an environment external to the autonomous vehicle one or more processors, and one or more non-transitory computer-readable media that collectively store a machine-learned motion planning system including one or more machine-learned models configured to generate target trajectories for the autonomous vehicle. The machine-learned motion planning system includes a behavioral planning stage configured to receive situational data based at least in part on the one or more outputs of the set of sensors and to generate behavioral planning data based at least in part on the situational data and a unified cost function. The machine-learned motion planning system includes a trajectory planning stage configured to receive the behavioral planning data from the behavioral planning stage and to generate target trajectory data for the autonomous vehicle based at least in part on the behavioral planning data and the unified cost function. The one or more non-transitory computer-readable media collectively store instructions that, when executed by the one or more processors, cause the one or more processors to perform operations. The operations include obtaining the situational data associated with the environment external to the autonomous vehicle, generating, using the behavioral planning stage of the machine-learned motion planning system, behavioral planning data indicative of at least one behavioral planning decision based at least in part on the situational data and the unified cost function, and generating, using the trajectory planning stage of the machine-learned motion planning system, target trajectory data indicative of at least one target trajectory based at least in part on the behavioral planning data and the unified cost function. Another example aspect of the present disclosure is directed to a computer-implemented method of motion planning for an autonomous vehicle. The method includes obtaining, by a computing system comprising one or more computing devices, situational data associated with an environment detected by one or more sensors of the autonomous vehicle, generating, by the computing system using a behavioral planning stage and a unified cost function of a machine-learned motion planning system, data indicative of at least one behavioral decision for the autonomous vehicle based at least in part on the situational data, generating, by the computing system using a trajectory planning stage and the unified cost function of the machine-learned motion planning system, target trajectory data indicative of a target trajectory for the autonomous vehicle based at least in part on the data indicative of at least one behavioral decision for the autonomous vehicle, and generating, by the computing system, one or more motion plans based on the target trajectory. Yet another example aspect of the present disclosure is directed to a computing system, including a machine-learned motion planning system configured to obtain situational data based at least in part on one or more outputs of a set of sensors of an autonomous vehicle and based at least in part on the situational data, generate a behavioral decision using a behavioral planning stage and output a target trajectory for the autonomous vehicle using a trajectory planning stage. The computing system includes one or more processors and one or more non-transitory computer-readable media that store instructions, that when