US-20260126800-A1 - Object Detection for Autonomous Vehicles

US20260126800A1US 20260126800 A1US20260126800 A1US 20260126800A1US-20260126800-A1

Abstract

An example method includes generating, a first bounding shape for an object within an environment of an autonomous vehicle, the first bounding shape indicating a boundary corresponding to a shape of the object. The example method includes identifying an extension of the object outside the boundary corresponding to the shape of the object. The example method includes generating, based on the first bounding shape, a second bounding shape for the object, the extension of the object enclosed in an interior region of the second bounding shape. The example method includes generating, based on the second bounding shape, a motion plan for the autonomous vehicle to control the motion of the autonomous vehicle relative to the second bounding shape. The example method includes providing instructions to control the motion of the autonomous vehicle in accordance with the motion plan.

Inventors

Steven Ziqiu Chen
Nemanja Djuric
Jiaxi Nie

Assignees

AURORA OPERATIONS, INC.

Dates

Publication Date: 20260507
Application Date: 20241101

Claims (20)

1 . A computer-implemented method comprising: generating, based on data indicative of an object within an environment of an autonomous vehicle, a first bounding shape for the object, the first bounding shape indicating a boundary corresponding to a shape of the object; identifying, based on the data indicative of the object and the first bounding shape, an extension of the object outside the boundary corresponding to the shape of the object; generating, based on the first bounding shape, a second bounding shape for the object, the extension of the object enclosed in an interior region of the second bounding shape; generating, based on the second bounding shape, a motion plan for the autonomous vehicle, the motion plan comprising one or more parameters to control the motion of the autonomous vehicle relative to the second bounding shape; and providing one or more instructions to control the motion of the autonomous vehicle in accordance with the one or more parameters of the motion plan.
2 . The computer-implemented method of claim 1 , further comprising: determining, based on the extension, a first portion of the first bounding shape at which the extension is located; performing a transformation on the first portion of the first bounding shape; and generating the second bounding shape to include the first portion of the first bounding shape that has been transformed, such that an outer surface of the extension is included in the interior region of the second bounding shape.
3 . The computer-implemented method of claim 2 , wherein the first portion of the first bounding shape is a first side of the first bounding shape, and wherein the transformation comprises shifting the first side of the first bounding shape away from a centroid of the first bounding shape.
4 . The computer-implemented method of claim 2 , further comprising: determining that the first portion of the first bounding shape is within a field of view of a sensor of the autonomous vehicle.
5 . The computer-implemented method of claim 1 , further comprising: determining a first angle between the autonomous vehicle and a first portion of the first bounding shape of the object at which the extension is located; generating a comparison of the first angle to an angle threshold; and based on the comparison of the first angle to the angle threshold, generating the second bounding shape based on the first bounding shape.
6 . The computer-implemented method of claim 5 , wherein the comparison of the first angle to the angle threshold indicates that the first angle is less than the angle threshold.
7 . The computer-implemented method of claim 6 , further comprising: determining a second angle between the autonomous vehicle and a second portion of the first bounding shape of the object; generating a comparison of the second angle to the angle threshold; and based on the comparison of the second angle to the angle threshold, determining to forgo transforming the second portion of the first bounding shape.
8 . The computer-implemented method of claim 7 , wherein the comparison of the second angle to the angle threshold indicates that the second angle is greater than the angle threshold.
9 . The computer-implemented method of claim 1 , further comprising: determining, based on the first bounding shape, an estimated position of the object within a roadway.
10 . The computer-implemented method of claim 9 , further comprising: generating, also based on the estimated position of the object within the roadway, the motion plan for the autonomous vehicle.
11 . The computer-implemented method of claim 1 , further comprising: determining, based on the data indicative of the object, that the object is not an ephemeral object.
12 . The computer-implemented method of claim 1 , wherein the extension comprises at least one of a protrusion of an item being transported by the object or a protrusion of a component of the object.
13 . The computer-implemented method of claim 1 , wherein the second bounding shape comprises a larger region than the first bounding shape.
14 . The computer-implemented method of claim 1 , further comprising: generating the first bounding shape based on a classification of the object.
15 . The computer-implemented method of claim 1 , further comprising: generating the second bounding box based on a model, the model being trained based on labeled training data, the labeled training data comprising a training object with a training extension, the labeled training data comprising a first training shape representing a canonical shape of the training object and a second training shape representing a shape of the training object that includes the extension of the training object.
16 . An autonomous vehicle (AV) control system comprising: one or more processors; and one or more tangible, non-transitory, computer-readable media that store instructions that are executable by the one or more processors to perform operations comprising: generating, based on data indicative of an object within an environment of an autonomous vehicle, a first bounding shape for the object, the first bounding shape indicating a boundary corresponding to a shape of the object; identifying, based on the data indicative of the object and the first bounding shape, an extension of the object outside the boundary corresponding to the shape of the object; generating, based on the first bounding shape, a second bounding shape for the object, the extension of the object enclosed in an interior region of the second bounding shape; generating, based on the second bounding shape, a motion plan for the autonomous vehicle, the motion plan comprising one or more parameters to control the motion of the autonomous vehicle relative to the second bounding shape; and providing one or more instructions to control the motion of the autonomous vehicle in accordance with the one or more parameters of the motion plan.
17 . The AV control system of claim 16 , wherein the operations further comprise: determining a portion of the first bounding shape at which the extension is located; performing a transformation on the portion of the first bounding shape at which the extension is located; and generating, based on the portion of the first bounding shape that has been transformed, the second bounding shape, such that an outer surface of the extension is included in the interior region of the second bounding shape.
18 . The AV control system of claim 17 , wherein the first portion of the first bounding shape is a first side of the first bounding shape, and wherein the transformation comprises shifting the first side of the first bounding shape away from a centroid of the first bounding shape until an entirety of the extension is enclosed in the interior region of the second bounding shape.
19 . The AV control system of claim 16 , wherein the operations further comprise: determining a first angle between the autonomous vehicle and a first portion of the first bounding shape of the object at which the extension is located; generating a comparison of the first angle to an angle threshold; and based on the comparison of the first angle to the angle threshold, generating the second bounding shape based on the first bounding shape.
20 . One or more tangible, non-transitory, computer readable media storing instructions that are executable by one or more processors to perform operations comprising: generating, based on data indicative of an object within an environment of an autonomous vehicle, a first bounding shape for the object, the first bounding shape indicating a boundary corresponding to a shape of the object; identifying, based on the data indicative of the object and the first bounding shape, an extension of the object outside the boundary corresponding to the shape of the object; generating, based on the first bounding shape, a second bounding shape for the object, the extension of the object enclosed in an interior region of the second bounding shape; generating, based on the second bounding shape, a motion plan for the autonomous vehicle, the motion plan comprising one or more parameters to control the motion of the autonomous vehicle relative to the second bounding shape; and providing one or more instructions to control the motion of the autonomous vehicle in accordance with the one or more parameters of the motion plan.

Description

BACKGROUND A self-driving car may use computer vision techniques to understand the surroundings of the self-driving car and use computers to decide how to drive with respect to the surroundings. SUMMARY The present disclosure is directed to improving the ability of an autonomous vehicle to detect the shapes of objects within the environment of the vehicle and control the motion of the autonomous vehicle through the environment. For example, an autonomous vehicle may process sensor data to detect an object within the surrounding environment (e.g., a pick-up truck in an adjacent lane). The autonomous vehicle may generate a first bounding shape (e.g., bounding box) that corresponds to the shape of the object. For example, the first bounding shape may represent a canonical shape fit tightly to the main volume of the object. The dimensions of the first bounding shape may be based on the classification of the object (e.g., pedestrian, tractor trailer). However, real-world objects may not always conform to canonical shapes. Because the bounding box is tightly fit to the main volume of the object, there may be an extension protruding from the object (e.g., a pole in the truck bed) that extends outside the boundary defining the canonical shape. The technology of the present disclosure allows an autonomous vehicle to better account for such extensions. For example, the autonomous vehicle may process the sensor data and the first bounding shape to determine that there is an extension of the object that extends outside the first bounding shape. To do so, the autonomous vehicle may analyze image pixels to determine that certain colored pixels appear to extend from the object and are present outside the first bounding shape. Additionally, or alternatively, the autonomous vehicle may analyze a LIDAR point cloud return and determine that a certain density of LIDAR points exist for a structure that extends outside the first bounding shape. To account for the extension, the autonomous vehicle may generate a second bounding shape based on the first bounding shape. The second bounding shape may include, for example, a single box/rectangular prism that is axis aligned to the first bounding shape (e.g., the canonical bounding box), but that includes a larger interior region than the first bounding shape. The outermost exterior surface of the extension from the object may be enclosed in the larger, second bounding shape. This helps capture the observed shape of the object, including the actual extremities of the object. In some implementations, the second bounding shape may be oriented slightly differently or offset from the first bounding box to better reflect the observed shape of the object. The autonomous vehicle may generate the second bounding shape by transforming a portion of the first bounding shape. This may include shifting one or more sides of the first bounding shape away from the centroid of the first bounding shape. A side may be shifted until it reaches the outermost surface of the extension. The autonomous vehicle may transform certain portions of the first bounding shape that are “relevant” to the autonomous vehicle. A portion of the first bounding shape may be considered relevant, in the event that the portion includes the extension and is visible to the autonomous vehicle (e.g., within the field of view of a sensor of the autonomous vehicle). In some implementations, the autonomous vehicle may analyze the angle between the autonomous vehicle and a portion of the first bounding shape to help determine whether to perform a transformation. By way of example, the first bounding shape may include a four-sided bounding box that represents the canonical shape of an object. The object may be a pick-up truck that includes a pole extending out of the left side of the truck bed and a piece of lumber extending from the backside of the truck. The autonomous vehicle may be travelling on the diagonal front left side of the truck (e.g., in an adjacent left lane). Accordingly, the angle between the autonomous vehicle and the left side of the truck may be less than an angle threshold, indicating good visibility. The angle between the autonomous vehicle and the backside of the truck may be greater than the angle threshold, indicating poor visibility. Thus, to generate the second bounding shape, the left side of the first bounding shape may be shifted outward until the entirety of the extended pole is enclosed within the region of the bounding shape, while the back side of the first bounding shape, which is less visible, may be unmodified. The second bounding shape may include the transformed version of the first bounding shape. The autonomous vehicle may generate the second bounding shape based on a trained model. The model may be trained based on labeled training data. For example, the training data may include previously captured sensor data. The sensor data may indicate a training object within an environment (e.g., a truck t