US-12619464-B2 - Program flow monitoring and control of an event-triggered system

Abstract

In various examples, a program (e.g., application, algorithm, routine, etc.) may be organized into operational units (e.g., nodes executed by one or more processors), each of which are tasked with executing one or more respective events (e.g., tasks) within the larger program. At least some of the events of the larger program may be successively executed in a flow, one after another, using triggers sent directly from one node to the next. In addition, a manager may exchange communications with the nodes to monitor or assess a status of the system (e.g., determine when a node has completed an event) or to control or trigger a node to initiate an event.

Inventors

• Peter Alexander Boonstoppel
• Michael Cox
• Daniel Perrin

Assignees

• NVIDIA CORPORATION

Dates

Publication Date

20260505

Application Date

20240313

Claims (20)

1. 1 . A method comprising: monitoring, based at least on an execution graph representative of a program, execution of one or more first tasks of a plurality of tasks of the program, the execution graph including a plurality of nodes corresponding to respective tasks of the plurality of tasks and edges defining an execution flow of the plurality of tasks among the plurality of nodes; based at least on the monitoring, determining one or more conditions for triggering one or more nodes of the plurality of nodes in the execution graph are satisfied, the one or more conditions corresponding to completion of one or more portions of the one or more first tasks; and based at least on the determining, transmitting one or more manager-originating triggers to the one or more nodes, wherein the one or more nodes are programmed to enforce, in response to receiving one or more node-originating triggers, an execution order of the plurality of tasks in the execution flow based at least on delaying initiating one or more second tasks of the plurality of tasks until the one or more manager-originating triggers are received.
2. 2 . The method of claim 1 , wherein the one or more conditions include at least one of one or more time-based conditions, one or more resource-based conditions, or one or more status-based conditions.
3. 3 . The method of claim 1 , wherein output of the program is used to direct one or more operations of an autonomous or semi-autonomous machine in an environment.
4. 4 . The method of claim 1 , further comprising prior to initiating the execution flow, programming, based at least on a representation of the execution order, the one or more nodes to, responsive to the receiving of the one or more node-originating triggers, delay the initiating until the one or more manager-originating triggers are received.
5. 5 . The method of claim 1 , wherein the determining the one or more conditions are satisfied uses one or more event reports sent from and generated by at least one of the plurality of nodes, the one or more event reports indicating a start or an end of a task of the one or more first tasks.
6. 6 . The method of claim 1 , wherein the execution is used to control one or more of: an autonomous or semi-autonomous vehicle; artificial intelligence-assisted video monitoring; an artificial intelligence-assisted diagnostic device; or an artificial intelligence-assisted surgical device.
7. 7 . The method of claim 1 , wherein the transmitting is responsive to detecting the one or more conditions are satisfied.
8. 8 . A system comprising: one or more combinations of one or more hardware components to execute operations including: applying sensor data obtained using one or more sensors of a machine in an environment to a program to cause execution of an execution graph representative of the program to detect one or more conditions corresponding to the environment, the execution graph including a plurality of nodes corresponding to respective tasks of a plurality of tasks and edges defining an execution flow of the plurality of tasks among the plurality of nodes; and based at least on a determination, using one or more status indicators of one or more first tasks of the plurality of tasks that correspond to one or more first nodes of the plurality of nodes, that one or more conditions for triggering one or more second nodes of the plurality of nodes in the execution graph are satisfied, providing one or more manager-originating triggers to the one or more second nodes, wherein the one or more second nodes are programmed to enforce, in response to receiving one or more node-originating triggers from one or more other nodes of the plurality of nodes, an execution order of the plurality of tasks in the execution flow based at least on delaying initiation of one or more second tasks of the plurality of tasks until the one or more manager-originating triggers are received, the one or more second tasks using the one or more combinations of the one or more hardware components to control the machine in the environment.
9. 9 . The system of claim 8 , wherein output of the program is reflective of one or more environmental conditions corresponding to the environment in real time.
10. 10 . The system of claim 8 , wherein the one or more conditions include at least one of one or more time-based conditions, one or more resource-based conditions, or one or more status-based conditions.
11. 11 . The system of claim 8 , wherein the execution order is predetermined by an ordered representation of the program.
12. 12 . The system of claim 8 , wherein the determination uses one or more event reports sent from and generated by at least one of the one or more first nodes, the one or more event reports indicating a start or an end of a task of the one or more first tasks.
13. 13 . The system of claim 8 , wherein the machine includes one or more of: an autonomous or semi-autonomous vehicle; an artificial intelligence-assisted diagnostic device; or an artificial intelligence-assisted surgical device.
14. 14 . The system of claim 8 , wherein a node of the plurality of nodes represents a discrete hardware component, a computing device, or a computing system.
15. 15 . The system of claim 8 , wherein the system is comprised in at least one of: a control system for an autonomous or semi-autonomous machine; a perception system for an autonomous or semi-autonomous machine; a system implemented using a robot; a system for performing one or more artificial intelligence-assisted video monitoring operations; a system for performing one or more artificial intelligence-assisted diagnostic device operations; a system for performing one or more artificial intelligence-assisted surgical device operations; or a system for presenting at least one of virtual reality content or augmented reality content.
16. 16 . At least one processor comprising: one or more circuits to provide one or more manager-originating triggers to one or more nodes of a plurality of nodes in an execution graph representative of a program based at least on: monitoring, based at least on the execution graph, execution of one or more first tasks of a plurality of tasks of the program, the execution graph including the plurality of nodes corresponding to respective tasks of the plurality of tasks and edges defining an execution flow of the plurality of tasks among the plurality of nodes; and based at least on the monitoring, determining one or more conditions for triggering the one or more nodes in the execution graph are satisfied, the one or more conditions corresponding to completion of one or more portions of the one or more first tasks, wherein the one or more nodes are programmed to enforce, in response to receiving one or more node-originating triggers from one or more other nodes of the plurality of nodes, an execution order of the plurality of tasks in the execution flow based at least on delaying initiation of one or more second tasks of the plurality of tasks until the one or more manager-originating triggers are received.
17. 17 . The at least one processor of claim 16 , wherein the one or more conditions include at least one of one or more time-based conditions, one or more resource-based conditions, or one or more status-based conditions.
18. 18 . The at least one processor of claim 16 , wherein output of the program is reflective of one or more environmental conditions detected by the program and corresponding to an environment in real time, and the output is used to direct one or more operations of an autonomous or semi-autonomous machine in the environment.
19. 19 . The at least one processor of claim 16 , wherein the delaying causes the one or more nodes to initiate the one or more second tasks after the completion of the one or more first tasks.
20. 20 . The at least one processor of claim 16 , wherein the at least one processor is comprised in at least one of: a control system for an autonomous or semi-autonomous machine; a perception system for an autonomous or semi-autonomous machine; a system implemented using a robot; a system for performing one or more artificial intelligence-assisted video monitoring operations; a system for performing one or more artificial intelligence-assisted diagnostic device operations; a system for performing one or more artificial intelligence-assisted surgical device operations; or a system for presenting at least one of virtual reality content or augmented reality content.

Description

CROSS-REFERENCE TO RELATED APPLICATIONS This application is a continuation of U.S. patent application Ser. No. 16/810,557, filed Mar. 5, 2020, which is incorporated herein by reference in its entirety. BACKGROUND Computer programs (e.g., applications, algorithms, routines, etc.) are often organized into operational units (e.g., nodes or work nodes executed by programmable engines or hardware engines), and each operational unit provides one or more respective events (e.g., tasks) within the larger program. When executing the program (e.g., processing one or more inputs to arrive at an output), the operational units may be successively executed, one after another. In many instances, completing the events in a predetermined order is helpful, such as when the output of a first event is used as an input to a second event. Various scheduling approaches may be used to organize events in an order when a program is executed. For example, some systems may use event-triggered scheduling in which, once an operational unit completes an event, the operational unit triggers the next operational unit to complete a subsequent event, and so on. Other systems may use time-triggered scheduling in which a centralized scheduler starts and monitors the execution of every operational unit based on a pre-determined schedule. Each of these scheduling approaches has its own advantages and disadvantages. For example, the timing of a sequence execution by conventional event-triggered systems is often more dynamic, as compared to a conventional time-triggered system, since the exact timing is variable and is not determined until the actual execution based on various factors (e.g., system load, resource availability, etc.). Conventional event-triggered systems may benefit from high resource utilization, since delay between events can be minimized when operational units directly trigger one another—as opposed to waiting on a central scheduler to trigger events based on a maximum event execution time. However, event-triggered scheduling may experience certain drawbacks. For example, event-triggered systems are often less predictable since schedules may dynamically arise from the actual demand imposed by real-time events. In addition, testing event-triggered systems may be more challenging since the actual load that may be experienced in real-time can be unpredictable, and timeliness is often difficult to verify without testing many scenarios to account for variable runtimes. Furthermore, since operational units directly trigger one another without a central scheduler, an event-triggered system may be more difficult to observe at runtime to ensure correct operation. These various drawbacks—e.g., lower predictability, testability, and observability—make it harder to implement event-triggered systems in various computing environments, such as safety-critical applications that require real-time testability (e.g., autonomous vehicles, semi-autonomous vehicles, artificial intelligence-assisted video monitoring, artificial intelligence-assisted diagnostic and/or surgical devices, etc.). As another example, conventional time-triggered systems—which use a scheduler to trigger operational units based on predetermined timing and to monitor operational-unit status—are often easier to prove and observe, and are more predictable relative to event-triggered systems. However, since the time slot allotted for each event is typically the maximum possible execution time, time-triggered systems often experience less efficient resource utilization and increased runtimes. Furthermore, additional latency is introduced by the sending and receiving of signals between the operational units and the centralized scheduler. These drawbacks of some conventional time-triggered systems—e.g., longer runtime, lower resource utilization, and added latency—may be undesirable in some types of systems, such as those designed to process a condition at an instant or as close as possible to the actual occurrence of the condition (e.g., in “real-time”). However, because conventional time-triggered systems may experience longer runtimes and lower resource utilization, these systems may be incapable of operating in real-time—a requirement of high-performance systems. SUMMARY The present disclosure relates to selectively monitoring and controlling operational units within an event-triggered system. For example, a manager (e.g., module or unit) may be used to monitor and control the event-triggered system and may exchange communications with the operational units. Among other things, the communications may be used to determine a state of the operational unit (e.g., status of an event or task) and/or to trigger an operational unit to initiate an event. As such, each operational unit may be triggered by a previous operational unit, by the manager, or by a combination thereof. In contrast to conventional event-triggered systems, in which operational units directly trigger one another with