US-12626460-B2 - Recursive field networks for object representation

Abstract

Systems, methods, and other embodiments described herein relate to using octrees and trilinear interpretation to generate field-specific representations. In one embodiment, a method includes acquiring a latent vector describing an object. The method includes generating an octree from the latent vector according to a recursive network, the octree representing the object at a desired level-of-detail (LoD). The method includes extracting features from the octree at separate resolutions. The method includes providing a field as a representation of the object according to the features.

Inventors

• Sergey Zakharov
• Katherine Y Liu
• Adrien David GAIDON
• Rares A Ambrus

Assignees

• Toyota Research Institute, Inc.

Dates

Publication Date

20260512

Application Date

20230831

Claims (20)

1. 1 . A modeling system, comprising: one or more processors; a memory communicably coupled to the one or more processors and storing instructions that, when executed by the one or more processors, cause the one or more processors to: acquire a latent vector describing an object; generate an octree from the latent vector according to a recursive network, the octree representing the object at a desired level-of-detail (LoD), including instructions to prune unoccupied voxels as the recursive network expands existing latent nodes according to a prediction of a likelihood of being occupied; extract features from the octree at separate resolutions; and provide a field as a representation of the object according to the features.
2. 2 . The modeling system of claim 1 , wherein the instructions to extract the features include instructions to using trilinear interpolation among neighboring latents in the octree at separate LoDs to estimate the features.
3. 3 . The modeling system of claim 1 , wherein the instructions further include instructions to: concatenate the features across separate LoDs at separate voxels; and decode the features into the field.
4. 4 . The modeling system of claim 1 , wherein the instructions to acquire the latent vector include instructions to receive the latent vector as a request to generate the field, and wherein the octree is a recursive hierarchical representation that integrates representations of the object at multiple different LoDs.
5. 5 . The modeling system of claim 1 , wherein the instructions to generate the octree according to the recursive network include instructions to apply a recursive autodecoder to iteratively divide the octree until reaching a defined LoD.
6. 6 . The modeling system of claim 1 , wherein the latent vector is an abstract representation of the object that is interpretable by a machine-learning algorithm trained on an associated latent space defining aspects of a set of objects.
7. 7 . The modeling system of claim 1 , wherein the instructions to provide the field include instructions to provide the field as one of: a neural radiance field (NeRF), and a signed distance field (SDF).
8. 8 . The modeling system of claim 7 , wherein the instructions to provide the field include instructions to render the object using the field as a portion of a display.
9. 9 . A non-transitory computer-readable medium including instructions that when executed by one or more processors cause the one or more processors to: acquire a latent vector describing an object; generate an octree from the latent vector according to a recursive network, the octree representing the object at a desired level-of-detail (LoD), including instructions to prune unoccupied voxels as the recursive network expands existing latent nodes according to a prediction of a likelihood of being occupied; extract features from the octree at separate resolutions; and provide a field as a representation of the object according to the features.
10. 10 . The non-transitory computer-readable medium of claim 9 , wherein the instructions to extract the features include instructions to using trilinear interpolation among neighboring latents in the octree at separate LoDs to estimate the features.
11. 11 . The non-transitory computer-readable medium of claim 9 , wherein the instructions further include instructions to: concatenate the features across separate LoDs at separate voxels; and decode the features into the field.
12. 12 . The non-transitory computer-readable medium of claim 9 , wherein the instructions to acquire the latent vector include instructions to receive the latent vector as a request to generate the field, and wherein the octree is a recursive hierarchical representation that integrates representations of the object at multiple different LoDs.
13. 13 . The non-transitory computer-readable medium of claim 9 , wherein the instructions to generate the octree according to the recursive network include instructions to apply a recursive autodecoder to iteratively divide the octree until reaching a defined LoD.
14. 14 . A method, comprising: acquiring a latent vector describing an object; generating an octree from the latent vector according to a recursive network, the octree representing the object at a desired level-of-detail (LoD), wherein generating the octree includes pruning unoccupied voxels as the recursive network expands existing latent nodes according to a prediction of a likelihood of being occupied; extracting features from the octree at separate resolutions; and providing a field as a representation of the object according to the features.
15. 15 . The method of claim 14 , wherein extracting the features includes using trilinear interpolation among neighboring latents in the octree at separate LoDs to estimate the features, including sampling the features at different LoDs.
16. 16 . The method of claim 14 , further comprising: concatenating the features across separate LoDs at separate voxels; and decoding the features into the field.
17. 17 . The method of claim 14 , wherein acquiring the latent vector includes receiving the latent vector as a request to generate the field, and wherein the octree is a recursive hierarchical representation that integrates representations of the object at multiple different LoDs.
18. 18 . The method of claim 14 , wherein generating the octree according to the recursive network includes applying a recursive autodecoder to iteratively divide the octree until reaching a defined LoD.
19. 19 . The method of claim 14 , wherein the latent vector is an abstract representation of the object that is interpretable by a machine-learning algorithm trained on an associated latent space defining aspects of a set of objects.
20. 20 . The method of claim 14 , wherein providing the field includes providing the field as one of: a neural radiance field (NeRF), and a signed distance field (SDF), and wherein providing the field includes rendering the object using the field as a portion of a display.

Description

CROSS-REFERENCE TO RELATED APPLICATIONS This application claims benefit of U.S. Provisional Application No. 63/450,482, filed on, Mar. 7, 2023, which is herein incorporated by reference in its entirety. TECHNICAL FIELD The subject matter described herein relates, in general, to representing objects and, more particularly, to using octrees and trilinear interpretation to generate field-specific representations. BACKGROUND Vehicles may be equipped with sensors that facilitate perceiving other vehicles, obstacles, pedestrians, and additional aspects of a surrounding environment. For example, a vehicle may be equipped with a light detection and ranging (LIDAR) sensor that uses light to scan the surrounding environment, while logic associated with the LIDAR analyzes acquired data to detect a presence of objects and other features of the surrounding environment. In further examples, additional/alternative sensors such as cameras may be implemented to acquire information about the surrounding environment from which a system derives awareness about aspects of the surrounding environment. This sensor data can be useful in various circumstances for improving perceptions of the surrounding environment so that systems such as autonomous driving systems can perceive the noted aspects and accurately plan and navigate accordingly. In some instances, a system may subsequently represent the perceived objects in various contexts, such as renderings to a driver, within a simulation, and so on. However, accurately representing objects presents a complex task. For example, encoding shapes using neural architectures may be limited to 3D structure for which training can be especially complex. Thus, such an approach generally loses detail. SUMMARY In one embodiment, example systems and methods relate to a manner of improving 3D object representation and reconstruction. In one embodiment, a modeling system is disclosed. The modeling system includes one or more processors and a memory communicably coupled to the one or more processors. The memory stores instructions that, when executed by the one or more processors, cause the one or more processors to acquire a latent vector describing an object. The instructions include instructions to generate an octree from the latent vector according to a recursive network, the octree representing the object at a desired level-of-detail (LoD). The instructions include instructions to extract features from the octree at separate resolutions. The instructions include instructions to provide a field as a representation of the object according to the features. In one embodiment, a non-transitory computer-readable medium including instructions that, when executed by one or more processors, cause the one or more processors to perform one or more functions is disclosed. The instructions include instructions to acquire a latent vector describing an object. The instructions include instructions to generate an octree from the latent vector according to a recursive network, the octree representing the object at a desired level-of-detail (LoD). The instructions include instructions to extract features from the octree at separate resolutions. The instructions include instructions to provide a field as a representation of the object according to the features. In one embodiment, a method is disclosed. In one embodiment, the method includes acquiring a latent vector describing an object. The method includes generating an octree from the latent vector according to a recursive network, the octree representing the object at a desired level-of-detail (LoD). The method includes extracting features from the octree at separate resolutions. The method includes providing a field as a representation of the object according to the features. BRIEF DESCRIPTION OF THE DRAWINGS The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate various systems, methods, and other embodiments of the disclosure. It will be appreciated that the illustrated element boundaries (e.g., boxes, groups of boxes, or other shapes) in the figures represent one embodiment of the boundaries. In some embodiments, one element may be designed as multiple elements or multiple elements may be designed as one element. In some embodiments, an element shown as an internal component of another element may be implemented as an external component and vice versa. Furthermore, elements may not be drawn to scale. FIG. 1 illustrates one embodiment of a modeling system that is associated with using octrees and trilinear interpolation to generate field-specific representations. FIG. 2 illustrates one example of modeling an object using an octree representing multiple different levels-of-detail. FIG. 3 illustrates a flowchart for one embodiment of a method that is associated with generating an octree from a latent vector. FIG. 4 illustrates a flowchart for one embodiment of a method that is associated with applying tri