US-12626096-B2 - Selecting streams for optimized inferencing

Abstract

Apparatuses, systems, and techniques to select streams to run inference based at least in part on heuristics. In at least one embodiment, heuristics are generated based at least in part on information inferred using one or more machine learning models applied to streams.

Inventors

• Tushar Santosh Khinvasara

Assignees

• NVIDIA CORPORATION

Dates

Publication Date

20260512

Application Date

20220315

Claims (20)

1. 1 . A computer-implemented method comprising: generating, using a machine learning model, heuristic information based at least in part on inferred information about a first batch of frames from a set of data streams, the heuristic information comprises skip intervals corresponding to the set of data streams; selecting a subset of streams from the set of streams to generate a second batch of frames based at least in part on the heuristic information; using one or more machine learning models to infer information about the second batch of frames; and updating the heuristic information based at least in part on the inferred information about the second batch of frames.
2. 2 . The method of claim 1 , the computer-implemented method further comprises obtaining one or more configuration parameters, wherein the selecting the subset of streams further based at least in part on the one or more configuration parameters.
3. 3 . The method of claim 2 , wherein the one or more configuration parameters indicate a number of objects detected.
4. 4 . The method of claim 1 , wherein the first batch of frames and second batch of frames each comprise video frames comprising a respective sequence of image frames.
5. 5 . The method of claim 1 , wherein updating the heuristic information depends on a number of objects detected using the second batch of frames.
6. 6 . The method of claim 1 , wherein each of the set of streams is generated by different cameras.
7. 7 . A system comprising: one or more processors; and memory storing instructions that, if executed by the one or more processors, cause the system to: obtain heuristic information based at least in part on information inferred using one or more machine learning models applied to first frames from a set of data streams, the heuristic information comprising skip intervals associated with the set of data streams; determine, based at least in part on the heuristic information, a subset of data streams from the set of data streams to be included in second frames; and update the heuristic information based at least in part on information inferred using the one or more machine learning models applied to the second frames from the subset of data streams.
8. 8 . The system of claim 7 , wherein the memory further stores instructions that, as a result of being executed by the one or more processors, cause the system to obtain one or more configuration parameters, where the determination of the subset of data streams is further based at least in part on the one or more configuration parameters.
9. 9 . The system of claim 8 , wherein the one or more configuration parameters indicate a number of objects detected.
10. 10 . The system of claim 8 , wherein the one or more configuration parameters further indicate at least one of: a resolution of the first frames from the set of data streams, a size of one or more objects represented using the first frames from the set of data streams, or a proximity of one or more region of interest (ROI) depicted in the first frames from the set of data streams.
11. 11 . The system of claim 7 , wherein the memory further stores instructions that, if executed by the one or more processors, cause the system to use one or more secondary machine learning models to infer secondary information about third frames from the set of data streams which are not part of the subset of data streams.
12. 12 . The system of claim 7 , wherein the one or more machine learning models are deployed to perform at least one of: object detection, image segmentation, or object classification.
13. 13 . The system of claim 7 , wherein the updated heuristic information includes additional skip intervals of the subset of the data streams which are different from the one or more skip intervals.
14. 14 . A machine-readable medium having stored thereon a set of instructions, which if performed by one or more processors, cause the one or more processors to: determine heuristic information based at least in part on information inferred from first frames corresponding to a set of data streams, the heuristic information comprising skip intervals of the set of data streams; cause one or more inferencing operations to be performed on second frames from a subset of data streams selected from the set of data streams based at least in part on heuristic information; and update the heuristic information based at least in part on a result of the one or more inferencing operations.
15. 15 . The machine-readable medium of claim 14 , wherein the subset of data streams are selected based at least in part on one or more configuration parameters.
16. 16 . The machine-readable medium of claim 15 , wherein the one or more configuration parameters indicate a number of objects detected.
17. 17 . The machine-readable medium of claim 14 , wherein the selection of the subset of data streams is based at least in part on type of one or more objects detected from the first frames.
18. 18 . The machine-readable medium of claim 14 , wherein the updated heuristic information includes one or more additional skip intervals of the subset of the data streams that is shorter than the skip intervals.
19. 19 . The machine-readable medium of claim 14 , wherein the set of instructions further include instructions which, if performed by the one or more processors, cause third frames from the set of data streams to be skipped according to the skip intervals as a result of selecting the subset of data streams.
20. 20 . The machine-readable medium of claim 14 , wherein the first frames and the second frames are part of a separate batch.

Description

TECHNICAL FIELD At least one embodiment pertains to processors or computing systems used to select streams for inferencing based at least in part on heuristics according to various novel techniques described herein. BACKGROUND Running inference on multiple data streams may be used in different areas. For example, one or more cameras may provide multiple streams of image and/or audio frames based on video monitoring of entry gates, garages, roads, or factory lines to detect, identify, or track one or more objects. To detect, identify, or track one or more objects, systems or applications may use machine learning models to run inference on multiple data streams. Using machine learning models to run inference on multiple data streams may be complex and may require use of significant amounts of computing resources. BRIEF DESCRIPTION OF DRAWINGS FIG. 1 illustrates a system to process multiple data streams, in accordance with at least one embodiment; FIG. 2 illustrates a system to process multiple data streams, in accordance with at least one embodiment; FIG. 3 illustrates an example block diagram of selecting data streams for inferencing based on heuristics, according to at least one embodiment; FIG. 4 illustrates an example procedure of selecting data streams for inferencing based on heuristics and user-defined parameters, according to at least one embodiment; FIG. 5 illustrates an example of selecting data streams for inferencing, according to at least one embodiment; FIG. 6 illustrates an example procedure of processing data streams before inferencing, according to at least one embodiment; FIG. 7A illustrates logic, according to at least one embodiment; FIG. 7B illustrates logic, according to at least one embodiment; FIG. 8 illustrates training and deployment of a neural network, according to at least one embodiment; FIG. 9 illustrates an example data center system, according to at least one embodiment; FIG. 10A illustrates an example of an autonomous vehicle, according to at least one embodiment; FIG. 10B illustrates an example of camera locations and fields of view for the autonomous vehicle of FIG. 10A, according to at least one embodiment; FIG. 10C is a block diagram illustrating an example system architecture for the autonomous vehicle of FIG. 10A, according to at least one embodiment; FIG. 10D is a diagram illustrating a system for communication between cloud-based server(s) and the autonomous vehicle of FIG. 10A, according to at least one embodiment; FIG. 11 is a block diagram illustrating a computer system, according to at least one embodiment; FIG. 12 is a block diagram illustrating a computer system, according to at least one embodiment; FIG. 13 illustrates a computer system, according to at least one embodiment; FIG. 14 illustrates a computer system, according to at least one embodiment; FIG. 15A illustrates a computer system, according to at least one embodiment; FIG. 15B illustrates a computer system, according to at least one embodiment; FIG. 15C illustrates a computer system, according to at least one embodiment; FIG. 15D illustrates a computer system, according to at least one embodiment; FIGS. 15E and 15F illustrate a shared programming model, according to at least one embodiment; FIG. 16 illustrates exemplary integrated circuits and associated graphics processors, according to at least one embodiment; FIGS. 17A-17B illustrate exemplary integrated circuits and associated graphics processors, according to at least one embodiment; FIGS. 18A-18B illustrate additional exemplary graphics processor logic, according to at least one embodiment; FIG. 19 illustrates a computer system, according to at least one embodiment; FIG. 20A illustrates a parallel processor, according to at least one embodiment; FIG. 20B illustrates a partition unit, according to at least one embodiment; FIG. 20C illustrates a processing cluster, according to at least one embodiment; FIG. 20D illustrates a graphics multiprocessor, according to at least one embodiment; FIG. 21 illustrates a multi-graphics processing unit (GPU) system, according to at least one embodiment; FIG. 22 illustrates a graphics processor, according to at least one embodiment; FIG. 23 is a block diagram illustrating a processor micro-architecture for a processor, according to at least one embodiment; FIG. 24 illustrates a deep learning application processor, according to at least one embodiment; FIG. 25 is a block diagram illustrating an example neuromorphic processor, according to at least one embodiment; FIG. 26 illustrates at least portions of a graphics processor, according to one or more embodiments; FIG. 27 illustrates at least portions of a graphics processor, according to one or more embodiments; FIG. 28 illustrates at least portions of a graphics processor, according to one or more embodiments; FIG. 29 is a block diagram of a graphics processing engine of a graphics processor, in accordance with at least one embodiment; FIG. 30 is a block diagram o