US-12617428-B2 - Action reference generation pipeline

US12617428B2US 12617428 B2US12617428 B2US 12617428B2US-12617428-B2

Abstract

Techniques for accurately predicting and avoiding collisions with objects detected in an environment of a vehicle are discussed herein. A vehicle computing device can implement a model to generate action references usable in a tree search to control the vehicle. For example, the model may represent one or more trajectory generators that modify a baseline trajectory in different ways to output a set of actions trajectories that are usable to define a tree structure. The set of action trajectories can be used by the vehicle computing device for predicting vehicle actions by the vehicle computing device to control the vehicle.

Inventors

Liam Gallagher
Matthew Van Heukelom

Assignees

Zoox, Inc.

Dates

Publication Date: 20260505
Application Date: 20230531

Claims (20)

1 . A system comprising: one or more processors; and one or more non-transitory computer-readable media storing instructions executable by the one or more processors, wherein the instructions, when executed, cause the system to perform operations comprising: receiving a baseline trajectory associated with a vehicle in an environment, the baseline trajectory including a direction, a velocity, and an acceleration; inputting the baseline trajectory into a generator to modify one or more points along the baseline trajectory; determining, based at least in part on an output from the generator, a set of action trajectories associated with the vehicle; defining a portion of branches of a tree structure based at least in part on the set of action trajectories, the tree structure comprising a same start point for the set of action trajectories; determining, based at least in part on the portion of branches of the tree structure, a candidate trajectory; and controlling the vehicle based at least in part on the candidate trajectory.
2 . The system of claim 1 , the operations further comprising: modifying, based at least in part on a control policy, a shape of at least one action trajectory in the set of action trajectories to output a modified set of action trajectories, wherein the tree structure is further defined based at least in part on the modified set of action trajectories.
3 . The system of claim 1 , wherein the output is a first output, and the operations further comprising: determining, by the generator, presence of a static object in the environment; and receiving, as a second output from the generator, an action trajectory in the set of action trajectories to avoid the static object.
4 . The system of claim 1 , wherein an action trajectory in the set of action trajectories is defined by multiple points, the generator is a first generator, the output is a first output, and the operations further comprising: inputting the action trajectory into a second generator configured to one or more of: relocating one of the multiple points along the action trajectory from a first location to a second location along the action trajectory, modifying a vehicle state associated with one of the multiple points of the action trajectory, modifying a number of the multiple points of the action trajectory, increasing a length of the action trajectory, or determining a stopping action associated with the action trajectory; and receiving a second output from the second generator, wherein the set of action trajectories comprises the second output.
5 . The system of claim 1 , wherein at least one of the set of action trajectories defines a node of the tree structure.
6 . One or more non-transitory computer readable media storing instructions executable by one or more processors, wherein the instructions, when executed, cause the one or more processors to perform operations comprising: receiving a trajectory associated with a vehicle; inputting the trajectory into a component; receiving, from the component, a set of action trajectories comprising a first action trajectory that includes a first modification of the trajectory and a second action trajectory that includes a second modification of the trajectory; inputting the set of action trajectories and the trajectory into a tree search algorithm that searches the first action trajectory, the second action trajectory, and the trajectory in a tree structure, wherein the first action trajectory and the second action trajectory start at a same root node of the tree structure; receiving, as an output of the tree search algorithm, a candidate trajectory; and controlling the vehicle based at least in part on the candidate trajectory.
7 . The one or more non-transitory computer readable media of claim 6 , wherein the component determines a first trajectory of the set of action trajectories based at least in part on modifying, based at least in part on a control policy, a shape of at least the trajectory.
8 . The one or more non-transitory computer readable media of claim 6 , wherein the component is a first component, and the operations further comprise: inputting the trajectory to a second component; determining, by the second component, presence of a static object in an environment; and receiving, from the second component, a second trajectory of the set of action trajectories configured to control the vehicle to avoid the static object.
9 . The one or more non-transitory computer readable media of claim 6 , wherein the component is a first component, the output is a first output, and the operations further comprising: inputting the set of action trajectories to a second component; performing, by the second component, one or more of: relocating one of multiple points along an action trajectory from a first location to a second location between a first end and a second end of the action trajectory, modifying a number of the multiple points of the action trajectory, extending at least one of the multiple points from the second end of the action trajectory to a position in an environment, or determining a stopping action associated with the action trajectory; and receiving a second output from the second component, wherein the set of action trajectories is determined based at least in part on the second output from the second component.
10 . The one or more non-transitory computer readable media of claim 6 , wherein the output is a first output and a second output from the component is an input to another component.
11 . The one or more non-transitory computer readable media of claim 6 , wherein inputting the set of action trajectories into the tree search algorithm comprises: defining the tree structure to include a branch for at least one action trajectory in the set of action trajectories.
12 . The one or more non-transitory computer readable media of claim 6 , wherein: determining the set of action trajectories comprises the component performing a first transformation using a first model and a second transformation using a second model different from the first model.
13 . The one or more non-transitory computer readable media of claim 6 , the operations further comprising: transmitting the set of action trajectories associated with the tree search algorithm to a vehicle computing device; determining, based at least in part on a search of the tree structure, the candidate trajectory for the vehicle to use at a future time; and controlling the vehicle in an environment using the candidate trajectory.
14 . The one or more non-transitory computer readable media of claim 6 , the operations further comprising: determining an identifier associated with the trajectory; determining information describing a data structure of the trajectory; and storing the identifier and the information as profile data in a storage device.
15 . The one or more non-transitory computer readable media of claim 6 , the operations further comprising: determining metadata describing an action trajectory in the set of action trajectories, the metadata including a source of the action trajectory and information of a segment of the action trajectory; and associating the metadata with the tree search algorithm.
16 . The one or more non-transitory computer readable media of claim 6 , where the output is a first output and the operations further comprising: determining the set of action trajectories based at least in part on a second output by a parallel processing unit.
17 . A method comprising: receiving a trajectory associated with a vehicle; inputting the trajectory into a component; receiving, from the component, a set of action trajectories comprising a first action trajectory that includes a first modification of the trajectory and a second action trajectory that includes a second modification of the trajectory; inputting the set of action trajectories and the trajectory into a tree search algorithm that searches the first action trajectory, the second action trajectory, and the trajectory in a tree structure, wherein the first action trajectory and the second action trajectory comprise a same start point in the tree structure; receiving, as an output of the tree search algorithm, a candidate trajectory; and controlling the vehicle based at least in part on the candidate trajectory.
18 . The method of claim 17 , wherein the component determines the first action trajectory of the set of action trajectories based at least in part on modifying, based at least in part on a control policy, a shape of at least the trajectory.
19 . The method of claim 17 , wherein the component is a first component, and further comprising: inputting the trajectory to a second component; determining, by the second component, presence of a static object in an environment; and receiving, from the second component, a second trajectory of the set of action trajectories configured to control the vehicle to avoid the static object.
20 . The method of claim 17 , wherein the output is a first output and a second output from the component is an input to another component.

Description

BACKGROUND Planning systems in autonomous and semi-autonomous vehicles determine actions for a vehicle to take in an operating environment. Actions for a vehicle may be determined based in part on avoiding objects present in the environment. For example, an action may be generated by a planning system to yield to a pedestrian, to change a lane to avoid another vehicle in the road, or the like. Accurately predicting future actions may be necessary to safely operate the vehicle in the vicinity of the object. The actions can be based on an output from a tree search. BRIEF DESCRIPTION OF THE DRAWINGS The detailed description is described with reference to the accompanying figures. In the figures, the left-most digit(s) of a reference number identifies the figure in which the reference number first appears. The use of the same reference numbers in different figures indicates similar or identical components or features. FIG. 1 is an illustration of an example environment, in which an example vehicle predicts trajectories usable to avoid a collision with one or more example objects. FIG. 2 is a block diagram of an example computing device implementing techniques to determine action trajectories for associating with a tree structure as described herein. FIG. 3 is another block diagram of an example computing device implementing techniques to determine action trajectories for associating with a tree structure as described herein. FIG. 4 is a block diagram of an example system for implementing the techniques described herein. FIG. 5 is a flowchart depicting an example process for determining action trajectories for associating with a tree structure using one or more example components. DETAILED DESCRIPTION Techniques for generating trajectories for controlling a vehicle are discussed herein. The techniques can include a vehicle computing device that implements a model to generate action references (e.g., reference trajectories) usable, for example, in a tree search to control the vehicle (though any other use of such trajectories is contemplated). For example, the model can represent one or more trajectory generators that modify a baseline trajectory in different ways and/or generate one or more alternative trajectories to output a set of actions that are usable to define a tree. The tree can, for example, include branches and/or nodes associated with one of the action trajectories in the set of action trajectories for the vehicle to track such that at any subsequent node the vehicle may switch which action reference is being tracked at that point in time. The vehicle computing device can use the tree to determine an action for the vehicle at a future time. By implementing the techniques described herein, a vehicle computing device can predict vehicle actions to control the vehicle with more accuracy and in less time, thereby improving the overall safety of the vehicle. Generally, the model (e.g., a trajectory generator(s)) implemented by the vehicle computing device may provide functionality to determine multiple potential paths for a vehicle to avoid one or more objects in an environment in the future by modifying an input trajectory. In some examples, a single generator or multiple generators can be used to transform, resample, or otherwise modify the input trajectory. For instance, a first generator can generate an additional trajectory that is laterally offset from the input trajectory and a second generator can generate an additional trajectory that includes a scaled velocity relative to the input trajectory. Depending on the functionality performed, a generator can output a single trajectory that is a modified version of the input trajectory and/or a set of trajectories representing various modifications to the input trajectory. In examples when multiple generators are implemented, the generators can be combined or chained together in a variety of different ways to output action references that represent spatial and temporal profiles through the environment. The generators can, for example, be combined in serial and/or in parallel (in any configuration) to generate a variety of trajectories. For example, a transform generator can output, generate, or otherwise determine actions trajectories having a starting point that is offset from a point associated with the input trajectory. A resample generator can, for example, change a number of points and/or a position of a point associated with the input trajectory to generate an output trajectory having a different sampling of points than the input trajectory. By combining the generators in a variety of ways, a variety of action trajectories are made available for consideration during planning operations by a vehicle computing device. The generators are described throughout this disclosure including in relation to FIGS. 2 and 3, and elsewhere. As mentioned, output data from one or more the generators can be used to define a tree. For example, an output from any singl