US-12626539-B2 - Systems and methods for distributed visualization generation

US12626539B2US 12626539 B2US12626539 B2US 12626539B2US-12626539-B2

Abstract

Provided are systems, methods and computer program code for generating a distributed visualization.

Inventors

Liam Benson
Caleb Foust

Assignees

EMBARK TRUCKS, INC.

Dates

Publication Date: 20260512
Application Date: 20240216

Claims (20)

1 . A visualization generation system, comprising: a memory configured to store data captured by at least a first sensor of a vehicle; and a processor configured to: receive a request to generate a visualization of an operation of the vehicle, the request including a starting timestamp and an ending timestamp, the request further including information specifying a desired service level within which the visualization is to be generated; determine a plurality of worker nodes required to generate the visualization within the desired service level; provision the plurality of worker nodes and provide instructions to each worker node to cause each worker node to generate a portion of the visualization; reassemble the portions of the visualization into a final visualization; and deliver the final visualization to at least a first user.
2 . The visualization system of claim 1 , wherein the request to generate the visualization of the operation of the vehicle further includes information identifying a user interface configuration, wherein delivering the final visualization to the at least first user further includes delivering the final visualization based at least in part on the user interface configuration.
3 . The visualization system of claim 2 , wherein the information identifying a user interface configuration includes information identifying at least a first frame to be displayed in a user interface, the at least first frame displaying a selected type of data associated with the operation of the vehicle.
4 . The visualization system of claim 3 , wherein the information identifying a user interface configuration includes information identifying at least a second frame to be displayed in the user interface, the at least second frame displaying a second selected type of data associated with the operation of the vehicle.
5 . The visualization system of claim 1 , wherein each worker node is provisioned with one or more of a headless browser application, a data server, a Web API and a tile server.
6 . The visualization system of claim 5 , wherein the instructions to each worker node include at least a starting timestamp of a portion of the visualization and an ending timestamp of the portion of the visualization.
7 . The visualization system of claim 2 , wherein the information identifying a user interface configuration is provided in a data storage object.
8 . The visualization system of claim 7 , wherein the data storage object includes information identifying a title, a layout including an arrangement and position of one or more frames, and a source of data for display in each of the one or more frames.
9 . The visualization system of claim 8 , wherein each worker node uses the data storage object to render a user interface in a headless browser.
10 . A method for generating a visualization of a selected segment of a trip of a vehicle, comprising: receiving a request to generate a visualization of the selected segment of the trip by the vehicle, the request including a starting timestamp and an ending timestamp, the request further including information specifying a desired service level within which the visualization is to be generated; determining a plurality of worker nodes required to generate the visualization within the desired service level; provisioning the plurality of worker nodes and providing instructions to each worker node to cause each worker node to generate a portion of the visualization; reassembling the portions of the visualization into a final visualization; and providing the final visualization to at least a first user based at least in part on a user interface configuration.
11 . The method of claim 10 , further comprising: receiving information selecting a trip and a vehicle.
12 . The method of claim 11 , wherein the information selecting a trip and a vehicle and the request to generate a visualization are received from a user operating a user device displaying a desired user interface configuration, and wherein the information identifying the user interface configuration is provided in a data storage object.
13 . The method of claim 10 , wherein the request to generate the visualization of the selected segment of the trip of the vehicle further includes information identifying a user interface configuration, wherein providing the final visualization to the at least first user further includes providing the final visualization based at least in part on the user interface configuration.
14 . The method of claim 13 , wherein the information identifying a user interface configuration includes information identifying at least a first frame, the at least first frame displaying a selected type of data associated with the selected segment of the trip of the vehicle.
15 . The method of claim 14 , wherein the information identifying a user interface configuration includes information identifying at least a second frame, the at least second frame displaying a second selected type of data associated with the selected segment of the trip of the vehicle.
16 . The method of claim 10 , wherein each worker node is provisioned with a headless browser application, a data server, a Web API and a tile server.
17 . The method of claim 16 , wherein the instructions to each worker node include at least a starting timestamp of a portion of the visualization and an ending timestamp of the portion of the visualization.
18 . A non-transitory machine-readable medium storing instructions that, when executed by at least one programmable processor, cause the at least one programmable processor to perform operations comprising: receiving a request to generate a visualization of a selected segment of a trip by a vehicle, the request including a starting timestamp and an ending timestamp, the request further including information specifying a desired service level within which the visualization is to be generated; determining a plurality of worker nodes required to generate the visualization within the desired service level; provisioning the plurality of worker nodes and providing instructions to each worker node to cause each worker node to generate a portion of the visualization; reassembling the portions of the visualization into a final visualization; and providing the final visualization to at least a first user based at least in part on a user interface configuration.
19 . The medium of claim 18 , wherein the request to generate the visualization of the selected segment of the trip of the vehicle further includes information identifying a user interface configuration, wherein providing the final visualization to the at least first user further includes providing the final visualization based at least in part on the user interface configuration.
20 . The medium of claim 19 , wherein the information identifying a user interface configuration includes information identifying at least a first frame, the at least first frame displaying a selected type of data associated with the selected segment of the trip of the vehicle.

Description

CROSS REFERENCE The present application is a non-provisional continuation of and claims priority to U.S. application Ser. No. 18/157,233, which was filed on Jan. 20, 2023, the entire content of which is incorporated by reference herein in its entirety. BACKGROUND Autonomous or semi-autonomous vehicles rely on a substantial amount of sensor and other vehicle data to understand the road, the state of the vehicle and the state of the environment around the vehicle. After the completion of such a vehicle's trip, technicians, analysts and other users frequently have a need to replay visualizations of portions of a trip. These visualizations often include a number of different types of data, from sensors, imaging devices, vehicle systems, or the like and it is difficult to generate videos including all this data. SUMMARY Provided are systems, methods and computer program code for distributed visualization generation of data associated with a portion of an autonomous or semi-autonomous vehicle trip. Some embodiments include a memory configured to store data associated with operation of a vehicle including data captured by at least a first sensor of the vehicle and a processor configured to receive a request to generate a visualization of the operation of the vehicle, the request including a starting timestamp and an ending timestamp, the request further including information identifying a user interface configuration; determining a number of worker nodes required to generate the visualization; provisioning the number of worker nodes and providing instructions to each worker node to cause each worker node to generate a portion of the visualization; reassembling the portions of the visualization into a final visualization; and delivering the visualization to at least a first user. BRIEF DESCRIPTION OF THE DRAWINGS Features and advantages of the example embodiments, and the manner in which the same are accomplished, will become more readily apparent with reference to the following detailed description taken in conjunction with the accompanying drawings. FIG. 1 is a diagram illustrating an example of a communication environment in which a semi-truck may operate pursuant to some embodiments. FIG. 2 is a diagram illustrating a system pursuant to some embodiments. FIG. 3 is a flow diagram illustrating a process pursuant to some embodiments. FIG. 4 is a flow diagram illustrating a further process pursuant to some embodiments. FIGS. 5A and 5B are diagrams illustrating a user interface that may be used to generate a distributed visualization pursuant to some embodiments. FIG. 6 is a diagram illustrating a control system that may be deployed in a vehicle such as the semi-truck depicted in FIGS. 7A-7C, in accordance with an example embodiment. FIGS. 7A-7C are diagrams illustrating exterior views of a semi-truck that may be used in accordance with example embodiments. Throughout the drawings and the detailed description, unless otherwise described, the same drawing reference numerals will be understood to refer to the same elements, features, and structures. The relative size and depiction of these elements may be exaggerated or adjusted for clarity, illustration, and/or convenience. DETAILED DESCRIPTION In the following description, specific details are set forth in order to provide a thorough understanding of the various example embodiments. It should be appreciated that various modifications to the embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments and applications without departing from the spirit and scope of the disclosure. Moreover, in the following description, numerous details are set forth for the purpose of explanation. However, one of ordinary skill in the art should understand that embodiments may be practiced without the use of these specific details. In other instances, well-known structures and processes are not shown or described in order not to obscure the description with unnecessary detail. Thus, the present disclosure is not intended to be limited to the embodiments shown but is to be accorded the widest scope consistent with the principles and features disclosed herein. For convenience and ease of exposition, a number of terms will be used herein. For example, the term “semi-truck” will be used to refer to a vehicle in which systems of the example embodiments may be used. The terms “semi-truck”, “truck”, “tractor”, “vehicle” and “semi” may be used interchangeably herein. Further, as will become apparent to those skilled in the art upon reading the present disclosure, embodiments of the present invention may be used in conjunction with other types of vehicles. In general, embodiments may be used with desirable results in conjunction with any vehicle towing a trailer or carrying cargo over long distances. Features of some embodiments will now be described by first referring to FIG. 1 which depicts features of a syst