US-12620218-B2 - Visualizations of tasks of multiple imaging devices

US12620218B2US 12620218 B2US12620218 B2US 12620218B2US-12620218-B2

Abstract

Systems, methods, and media are described herein for providing one or more visualizations corresponding to one or more tasks executed by a core processor on image data from one or more imaging devices (e.g., a camera). For example, an inspection cycle of the core processor can be initiated for determining or identifying unexpected events associated with the tasks of the core processor. Each inspection cycle may correspond to a particular imaging device. In some embodiments, an inspection cycle includes inspections of a grabbing task, a pre-processing task, a processing task, an idle period, another type of task, or one or more combinations thereof. A visualization that includes the unexpected event and the inspection cycle is provided for display on a user interface. In some embodiments, the visualization includes a standard error bar for the pre-processing task, processing task, idle period, or one or more combinations thereof.

Inventors

Alessandro Francesconi
Gianbattista GUALENI
Marcello CIRCO
Luca Soffritti

Assignees

DATALOGIC IP TECH S.R.L.

Dates

Publication Date: 20260505
Application Date: 20230321

Claims (20)

1 . A system for providing one or more visualizations, the system comprising: a plurality of imaging devices, each having at least one sensor; an electronic display configured to display a graphical user interface; and a data processing engine operably coupled to the plurality of imaging devices and the electronic display, the data processing engine including: at least one central processing unit (CPU) having one or more cores; and one or more computer storage memory having computer-executable instructions stored thereon that, when executed by the one or more cores, cause the one or more cores to: receive image data from the plurality of imaging devices; based on receiving the image data, initiate a first inspection of a first core of the one or more cores, the first inspection corresponding to a first imaging device of the plurality of imaging devices; initiate a second inspection of the first core, the second inspection corresponding to a second imaging device of the plurality of imaging devices, wherein the first inspection and the second inspection are inspection cycles that each include at least a processing task and an idle period; identify at least one unexpected event based on the first inspection and the second inspection of the first core; provide, for display on the graphical user interface, a visualization of the first inspection and the second inspection of the first core depicted on a timeline having a starting point and an ending point; determine a start time corresponding to the processing task and the idle period for the first inspection cycle and the second inspection cycle; determine a standard error metric for each of the start time of the processing task and the idle period for the first inspection cycle and the second inspection cycle, wherein the standard error metrics for each task of the first inspection cycle are based on an average start time for historical processing tasks executed by the first core on the image data received by the first imaging device for a fixed framerate, and wherein the standard error metrics for each task of the second inspection cycle are based on an average start time for historical processing tasks executed by the first core on the image data received by the second imaging device for the fixed framerate; provide, for display on the graphical user interface and based on the standard error metric for the start time of each of the processing task and the idle period for the first inspection cycle and the second inspection cycle, the visualization including a processing standard error bar and an idle period standard error bar on the timeline for the first inspection cycle and the second inspection cycle; and provide, for display on the graphical user interface, the at least one unexpected event depicted on the timeline of the visualization.
2 . The system of claim 1 , wherein a number of the plurality of imaging devices is greater than the number of the one or more cores.
3 . The system of claim 1 , wherein the visualization of the first inspection and the second inspection of the first core is provided based on a time associated with transferring the image data from each of the first imaging device and the second imaging device to the one or more computer storage memory.
4 . The system of claim 1 , wherein the visualization includes pre-processing tasks corresponding to the first inspection and the second inspection depicted on the timeline, wherein the pre-processing tasks include filtering and calibrating the image data received from the first imaging device and the second imaging device.
5 . The system of claim 1 , wherein the one or more cores are further caused to: initiate a plurality of inspections of a second core of the one or more cores, each of the plurality of inspections corresponding to an imaging device of the plurality of imaging devices; identify the at least one unexpected event based on the plurality of inspections of the second core; and provide the visualization including tasks of the first inspection, the second inspection, and the plurality of inspections of the second core depicted on the timeline.
6 . The system of claim 1 , wherein the one or more cores are further caused to: receive additional image data from the first imaging device and the second imaging device; based on receiving the additional image data, initiate a third inspection of the first core, the third inspection corresponding to the additional image data from the first imaging device; based on receiving the additional image data, initiate a fourth inspection of the first core, the fourth inspection corresponding to the additional image data from the second imaging device; determine, based on the third inspection and the fourth inspection, an updated standard error metric for the start time of each of the processing task and the idle period for the first inspection cycle and the second inspection cycle; and provide, for display on the graphical user interface and based on the updated standard error metric for the start time of each of the processing task and the idle period for the first inspection cycle and the second inspection cycle, the visualization including an updated processing standard error bar and an updated idle period standard error bar on the timeline for each of the first inspection cycle and the second inspection cycle.
7 . The system of claim 1 , wherein the standard error metric for each of the start time of the processing task and the idle period of the respective first inspection cycle and the second inspection cycle is calculated as a standard deviation.
8 . The system of claim 1 , wherein the at least one unexpected event is identified based on an idle time duration extending beyond a subsequent grabbing task of a subsequent inspection cycle, a start time of a subsequent grabbing task of a subsequent inspection cycle occurring after a predetermined start time, a start time of a subsequent grabbing task occurring beyond a predetermined time range, an idle time duration exceeding a threshold, or one or more combinations thereof.
9 . A method for providing one or more visualizations, the method comprising: receiving, at a data processing engine operably coupled to a plurality of imaging devices and an electronic display configured to display a user interface, image data from the plurality of imaging devices; based on receiving the image data, initiating a plurality of inspection cycles of a plurality of cores of a central processing unit (CPU), each of the plurality of inspection cycles corresponding to at least one of the plurality of imaging devices, wherein each of the plurality of inspection cycles includes at least a processing task and an idle period subsequent to the processing task for the respective core; providing, for display on the user interface, a visualization of the plurality of inspection cycles of the one or more cores, the visualization including the processing task and the idle period for each respective core depicted on a timeline; determining a duration for the processing task and the idle period for each of the plurality of inspection cycles; determining a standard error metric for the duration of the processing task and the idle period for each of the plurality of inspection cycles, the standard error metric based on an average duration of historical processing tasks and idle periods executed by a corresponding core of the plurality of cores for a fixed framerate; identifying at least one unexpected event based on the standard error metrics for the tasks of the plurality of inspection cycles; providing, for display on the user interface and based on the standard error metric for the duration of the processing task and the idle period for each of the plurality of inspection cycles, the visualization including a processing standard error bar and an idle period standard error bar depicted on the timeline for each of the plurality of inspection cycles along with the at least one identified unexpected event.
10 . The method of claim 9 , wherein the at least one unexpected event is identified based on at least one of the idle period standard error bars of the plurality of inspection cycles.
11 . The method of claim 9 , wherein the at least one unexpected event is identified based on the idle period standard error bar for one of the tasks of one of the plurality of inspection cycles being longer than a predetermined threshold.
12 . The method of claim 9 , wherein the image data received from the plurality of imaging devices corresponds to images that are each captured at a predetermined framerate by each of the plurality of imaging devices.
13 . The method of claim 9 , wherein each of the plurality of inspection cycles includes a grabbing task, a pre-processing task, and a processing task that are each executed by the one or more cores, and wherein the visualization includes the grabbing task, the pre-processing task, and the processing task for each of the plurality of inspection cycles depicted on the timeline.
14 . The method of claim 13 , wherein the grabbing task of a first inspection cycle of the plurality of inspection cycles includes a transfer of the image data from a first imaging device of the plurality of imaging devices to a first core of the one or more cores, and wherein the grabbing task of a second inspection cycle of the plurality of inspection cycles includes a transfer of the image data from a second imaging device of the plurality of imaging devices to memory of the data processing engine.
15 . The method of claim 9 , further comprising dynamically updating the standard error bars within the visualization of the processing task and the idle period for each of the plurality of inspection cycles based on the one or more changes to a corresponding average start time.
16 . One or more non-transitory computer-readable storage media having computer-executable instructions embodied thereon that, when executed by one or more processors, perform a method for providing one or more visualizations, the method comprising: receiving image data from a first imaging device; based on receiving the image data, initiating an inspection cycle of at least one core of a data processing engine based on the at least one core executing a processing task and having an idle period during the inspection cycle; identifying at least one unexpected event based on the inspection cycle and determining standard error metrics related to start times and/or durations of processing tasks and idle periods of historical inspection cycles of the at least one core executing the processing tasks of the historical inspection cycles on historical image data from the first imaging device for a fixed framerate; and causing to display, via a user interface, a visualization including processing tasks and idle periods the inspection cycle on a timeline along with standard error bars based on the standard error metrics for each of the processing tasks and idle periods; and causing to display, via a user interface, the visualization further including the at least one unexpected event on the timeline.
17 . The one or more non-transitory computer-readable storage media of claim 16 , wherein the data processing engine is processing the image data from each of a plurality of imaging devices, including the first imaging device, in parallel using a plurality of cores including the at least one core.
18 . The one or more non-transitory computer-readable storage media of claim 16 , wherein the first imaging device is an optical character recognition device.
19 . The one or more non-transitory computer-readable storage media of claim 16 , wherein the method further comprises determining a start time for the processing task and the idle period for the inspection cycle, and providing the visualization including the processing task and the idle period depicted on the timeline based on the start time for the processing task and the start time for the idle period.
20 . The one or more non-transitory computer-readable storage media of claim 16 , the method further comprising: receiving image data from a second imaging device and a plurality of imaging devices; based on receiving the image data from the second imaging device, initiating a second inspection cycle of the at least one core based on the at least one core executing a processing task and having an idle period during the second inspection cycle; based on receiving the image data from the plurality imaging devices, initiating an inspection cycle corresponding to each of the plurality of imaging devices based on a second core of the data processing engine executing a processing task and having an idle period during each inspection cycle corresponding to each of the plurality of imaging devices; and causing to display the visualization further including the second inspection cycle and each inspection cycle corresponding to each of the plurality of imaging devices.

Description

BACKGROUND Desktop and laptop computers implement algorithms that monitor central processing unit features, such as a quantity that is currently in use (e.g., 17% utilization over 60 seconds). With the advent of mobile devices becoming ubiquitous and providing functions such as still camera, video camera, global positioning system navigation, web browsing, email communications, and texting, processing power required to support such devices and functions has increased. As such, there is a need to provide effective management of processor(s) of various computing devices. SUMMARY This Summary is provided to introduce a selection of concepts in a simplified form that are further described below in the Detailed Description. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used as an aid in determining the scope of the claimed subject matter. Embodiments of the technology disclosed herein correspond to providing one or more visualizations including graphical representations of one or more inspection cycles of a core that executes tasks on image data from one or more imaging devices (e.g., a camera, a computing device having a camera, an optical character recognition device, a medical imaging device). The tasks, for example, may include the transfer of image data from an imaging device (e.g., to a local or external data store), calibrating the image data, filtering the image data, other methods of image data preprocessing, processing the image data, another type of task associated with the image data, or one or more combinations thereof. The inspections cycles includes inspections of the tasks executed by the core. As such, in some embodiments, the one or more visualizations may include one or more inspection cycles and inspections of the tasks for each inspection cycle. In some embodiments, each task may be represented by a different symbol. Additionally, one or more unexpected events can be identified based on one or more inspection cycles of one or more cores. For example, an unexpected event may include jitter in the duration of processing one or more tasks corresponding to one or more cores, processing time above an average processing time of a particular task or a duration for a complete inspection cycle for an individual core, an unexpected delay between the processing of two tasks, another type of unexpected event, or one or more combinations thereof. Based on identifying the unexpected event, the one or more visualizations provided may include the unexpected event. As indicated above, the one or more cores (e.g., of a central processing unit) may receive image data from imaging devices for the execution of the tasks. In some embodiments, a plurality of cores of a data processing engine receive the image data simultaneously. For example, a first plurality of cores of the data processing engine can receive a first set of image data simultaneously and a second plurality of cores of the data processing engine can receive a second set of image data simultaneously. Further, in some embodiments, the number of imaging devices is greater than the number of cores of the data processing engine receiving the image data. Accordingly, the one or more visualizations may include inspection cycles of the first plurality of cores executing tasks on the first set of image data simultaneously. Additionally or alternatively, the one or more visualizations may also include the inspection cycles of the second plurality of cores executing tasks on the second set of image data simultaneously. In some embodiments, the one or more visualizations comprise a graphical representation illustrating a spatial and temporal distribution of image data associated with at least one imaging device and at least one core. The one or more visualizations, for example, may be provided in real-time or near real-time. The visualizations may also be provided in multi-dimensional space (e.g., via an augmented or virtual reality device). In some embodiments, the one or more visualizations are provided via an interactive display. BRIEF DESCRIPTION OF THE DRAWINGS The present disclosure is illustrated by way of example and not limitation in the accompanying figures in which like reference numerals indicate similar elements and in which: FIG. 1 a schematic diagram of an example data processing system for providing a visualization including an inspection cycle and unexpected event, in accordance with an embodiment of the present disclosure; FIG. 2 is an example display of a visualization including inspection cycles of a first core and inspection cycles of a second core, the inspection cycles including tasks associated with each core executing the tasks on image data from an imaging device, in accordance with an embodiment of the present disclosure; FIG. 3 is an example method for providing one or more visualizations for inspection cycles of cores, in accordance with an embodi