KR-102963914-B1 - Image alignment neural network

Abstract

Devices, systems, and techniques for generating a 3D model of an object. In at least one embodiment, a 3D model of an object is generated by one or more neural networks based on a plurality of images of the object.

Inventors

• 에카르트, 벤자민 데이비드
• 위안, 원타오
• 잠파니, 바룬
• 김, 기환
• 카우츠, 잔

Assignees

• 엔비디아 코포레이션

Dates

Publication Date

20260512

Application Date

20201030

Priority Date

20191105

Claims (20)

1. As a processor, A processor comprising a circuit that generates a three-dimensional (3D) model of an object based at least partially on a statistical distribution of multiple points in one or more point clouds generated by one or more neural networks from multiple images representing different views of the object.
2. A processor according to claim 1, wherein the images among the plurality of images include three-dimensional data representing positions on the surface of the object.
3. delete
4. A processor according to claim 1, wherein the statistical distribution corresponds to a Gaussian mixture model, the parameters for the Gaussian mixture model are generated at least partially based on the alignment of the plurality of images, and the alignment is at least partially based on a matching transformation generated from the Gaussian mixture model.
5. A processor according to paragraph 4, wherein the match transformation is generated to be a closed form that enables back-propagation of the match error.
6. In paragraph 4, the above-mentioned matching transformation is a processor that maps points within the plurality of images to a common coordinate system.
7. In claim 1, the one or more neural networks are processors that encode the geometric shape of the object.
8. A processor according to claim 1, wherein the plurality of images include one or more labeled points corresponding to locations on the blocked surface of the object.
9. As a system, A system comprising one or more processors configured to generate a three-dimensional (3D) model of said object based at least partially on a statistical distribution of multiple points in one or more point clouds generated by one or more neural networks from multiple images representing different views of said object.
10. In claim 9, the system comprises a plurality of images including point data representing locations on the surface of the object.
11. In paragraph 9, the above statistical distribution is a system corresponding to a probability model.
12. In claim 11, the system wherein the probability model is calculated at least partially based on a weight matrix output by one or more neural networks.
13. In paragraph 11, a system wherein the matching transformation of one or more point clouds is calculated based at least partially on the probability model.
14. In paragraph 13, a system in which the matching error is backpropagated to one or more of the above neural networks during training.
15. A non-transient machine-readable medium storing a set of instructions, wherein the instructions, when executed by one or more processors, allow one or more processors to at least: A non-transient machine-readable medium that generates a three-dimensional (3D) model of said object based at least partially on a statistical distribution of multiple points in one or more point clouds generated by one or more neural networks from multiple images representing different views of said object.
16. In paragraph 15, the image among the plurality of images comprises information indicating locations on the surface of the object, in a non-transient machine-readable medium.
17. In paragraph 15, an additional set of instructions is stored, and said additional set of instructions, when executed by one or more processors, said one or more processors are allowed to at least: A non-transient machine-readable medium that aligns the plurality of images based at least partially on the above statistical distribution, wherein the statistical distribution corresponds to a Gaussian mixture model.
18. In paragraph 17, the above Gaussian mixture model is calculated at least partially based on a weight matrix output by one or more neural networks, a non-transient machine-readable medium.
19. In paragraph 17, a non-transient machine-readable medium in which a matched transformation is calculated at least partially based on the above-mentioned Gaussian mixture model.
20. In paragraph 19, a non-transient machine-readable medium in which the matching error is backpropagated to one or more of the neural networks during training.

Description

Image alignment neural network Cross-reference regarding related applications This application claims priority based on U.S. Patent Application No. 16/675,120, filed November 5, 2019, titled “IMAGE ALIGNING NEURAL NETWORK,” the entire contents of which are incorporated herein by reference for all purposes. field At least one embodiment relates to processing resources used to perform computer vision tasks using artificial intelligence. For example, at least one embodiment relates to processors used to train neural networks to perform computer vision tasks. Performing computer vision tasks can consume significant amounts of memory, time, or computational resources. Many of these tasks involve aligning visual data, which remains a challenging problem. The amount of memory, time, or computational resources used to perform computer vision tasks can be improved. FIG. 1 illustrates a neural network that performs point cloud matching according to at least one embodiment. FIG. 2 illustrates an example of a system that performs point cloud matching using learned geometric representations according to at least one embodiment. FIG. 3 illustrates an example of a process for training a network to perform point cloud representation using learned geometric representations according to at least one embodiment. FIG. 4 illustrates an example of a process for training a network to generate data for a Gaussian mixture model according to at least one embodiment. FIG. 5 illustrates an example of a solver that obtains a matched transformation according to at least one embodiment. FIG. 6 illustrates an exemplary process for training a neural network according to at least one embodiment. FIG. 7 illustrates an exemplary process for training a neural network to generate a three-dimensional model according to at least one embodiment. FIG. 8a illustrates inference and/or training logic according to at least one embodiment. FIG. 8b illustrates inference and/or training logic according to at least one embodiment. FIG. 9 illustrates the training and deployment of a neural network according to at least one embodiment. FIG. 10 shows an exemplary data center system according to at least one embodiment. FIG. 11a illustrates an example of an autonomous vehicle according to at least one embodiment. FIG. 11b illustrates examples of camera positions and fields of view for the autonomous vehicle of FIG. 11a according to at least one embodiment. FIG. 11c is a block diagram illustrating an exemplary system architecture for an autonomous vehicle of FIG. 11a according to at least one embodiment. FIG. 11d is a drawing illustrating a system for communication between the autonomous vehicle of FIG. 11a and cloud-based server(s) according to at least one embodiment. FIG. 12 is a block diagram illustrating a computer system according to at least one embodiment. FIG. 13 is a block diagram showing a computer system according to at least one embodiment. FIG. 14 illustrates a computer system according to at least one embodiment. FIG. 15 illustrates a computer system according to at least one embodiment. FIG. 16a illustrates a computer system according to at least one embodiment. FIG. 16b illustrates a computer system according to at least one embodiment. FIG. 16c illustrates a computer system according to at least one embodiment. FIG. 16d illustrates a computer system according to at least one embodiment. FIGS. 16e and FIGS. 16f illustrate a shared programming model according to at least one embodiment. FIG. 17 illustrates exemplary integrated circuits and associated graphics processors according to at least one embodiment. FIGS. 18a and FIGS. 18b illustrate exemplary integrated circuits and associated graphics processors according to at least one embodiment. FIGS. 19a and FIGS. 19b illustrate additional exemplary graphics processor logic according to at least one embodiment. FIG. 20 illustrates a computer system according to at least one embodiment. FIG. 21a illustrates a parallel processor according to at least one embodiment. FIG. 21b illustrates a partition unit according to at least one embodiment. FIG. 21c illustrates a processing cluster according to at least one embodiment. FIG. 21d illustrates a graphics multiprocessor according to at least one embodiment. FIG. 22 illustrates a multi-graphics processing unit (GPU) system according to at least one embodiment. FIG. 23 illustrates a graphics processor according to at least one embodiment. FIG. 24 is a block diagram illustrating a processor microarchitecture for a processor according to at least one embodiment. FIG. 25 illustrates a deep learning application processor according to at least one embodiment. FIG. 26 is a block diagram illustrating an exemplary neuromorphic processor according to at least one embodiment. FIG. 27 illustrates at least some parts of a graphics processor according to one or more embodiments. FIG. 28 illustrates at least parts of a graphics processor according to one or more embodim