US-12628075-B2 - System for facilitating multi-level stream-based edge analytics in multi modal communication and method thereof

US12628075B2US 12628075 B2US12628075 B2US 12628075B2US-12628075-B2

Abstract

A device, system and method for facilitating multi-level stream-based edge analytics in multi-modal communication. The device generates at least one primary data stream and secondary data stream by processing a plurality of data streams received from one or more data sources based on type of a multimodal application. Further, the device determines bandwidth requirements for transmission of the primary data stream and secondary data stream. The device then selects one or more network interfaces from a plurality of network interfaces to transmit at least one primary data stream based on traffic characteristics of the plurality of network interfaces and the bandwidth requirements. The device then transmits the primary data stream to an edge analytic server for edge analytics via the selected one or more network interfaces.

Inventors

Brejesh Lall
Arzad Alam KHERANI
Ashish Singh PATEL
Gaurav SHUKLA

Assignees

INDIAN INSTITUTE OF TECHNOLOGY DELHI

Dates

Publication Date: 20260512
Application Date: 20230825

Claims (11)

1 . A device for facilitating multi-level stream-based edge analytics in multi-modal communication, comprising: a memory; an input/output (I/O) interface communicatively coupled with the memory; and a processing unit communicatively coupled with the memory and the I/O interface, the processing unit configured to: receive a plurality of data streams related to a multimodal application from one or more data sources via the I/O interface; generate at least one primary data stream and at least one secondary data stream by processing the received plurality of data streams based on type of the multimodal application and on input requirements received from an edge analytic server, wherein the primary data stream is sufficient to perform the edge analytics at the edge analytic server, wherein the secondary data stream provides additional information to enhance the edge analytics at the edge analytic server; determine bandwidth requirements for transmission of each of the at least one primary data stream and at least one secondary data stream; select one or more network interfaces from a plurality of network interfaces to transmit the at least one primary data stream based on traffic characteristics including at least latency and uplink speed of the plurality of network interfaces and the bandwidth requirements; and transmit the at least one primary data stream to the edge analytic server for the edge analytics via the selected one or more network interfaces and transmit the at least one secondary data stream to the edge analytic server for aggregation with the at least one primary data stream to perform advanced edge analytics, the secondary data stream being transmitted without meeting a latency requirement.
2 . The device of claim 1 , wherein the plurality of data streams comprises at least one of: video data stream, audio data stream, and haptic data stream.
3 . The device of claim 1 , wherein the one or more data sources comprises one or more of: at least one camera, at least one microphone, at least one temperature sensor, and at least one actuator.
4 . The device of claim 1 , wherein the type of the multimodal application comprises at least one of: remote care, remote detonation, remote operation, remote maintenance, and remote firefighting, wherein the at least one primary data stream and at least one secondary data stream are generated by processing the received plurality of data streams based on input requirements of the edge analytic server to perform the edge analytics based on the type of the multimodal application.
5 . The device of claim 1 , wherein the primary data stream comprises one or more of: low-resolution video data stream, medium resolution video data stream, high-resolution video data stream, low sample rate audio data stream, medium sample rate audio data stream, high sample rate audio data stream and haptic data stream.
6 . A method of facilitating multi-level stream-based edge analytics in multi-modal communication, comprising: receiving a plurality of data streams related to a multimodal application from one or more data sources; generating at least one primary data stream and at least one secondary data stream by processing the received plurality of data streams based on a type of the multimodal application and on input requirements received from an edge analytic server, wherein the primary data stream is sufficient to perform the edge analytics at the edge analytic server, and wherein the secondary data stream provides additional information to enhance the edge analytics at the edge analytic server; determining bandwidth requirements for transmission of each of the at least one primary data stream and at least one secondary data stream; selecting one or more network interfaces from a plurality of network interfaces to transmit the at least one primary data stream based on traffic characteristics including at least latency and uplink speed of the plurality of network interfaces and the bandwidth requirements; and transmitting the at least one primary data stream to the edge analytic server for the edge analytics via the selected one or more network interfaces and transmitting the at least one secondary data stream to the edge analytic server for aggregation with the at least one primary data stream to perform advanced edge analytics, the secondary data stream being transmitted without meeting a latency requirement.
7 . The method of claim 6 , wherein the plurality of data streams comprises at least one of: video data stream, audio data stream, and haptic data stream.
8 . The method of claim 6 , wherein the type of the multimodal application comprises at least one of: remote care, remote detonation, remote operation, remote maintenance, and remote firefighting, wherein the at least one primary data stream and at least one secondary data stream are generated by processing the received plurality of data streams based on input requirements of the edge analytic server to perform the edge analytics based on the type of the multimodal application.
9 . The method of claim 6 , wherein the primary data stream comprises one or more of: low-resolution video data stream, medium resolution video data stream, high-resolution video data stream, low sample rate audio data stream, medium sample rate audio data stream, high sample rate audio data stream and a haptic data stream.
10 . A system for multi-level stream-based edge analytics in multi-modal communication, comprises: an edge analytic server; and a client device configured to communicate with the edge analytic server, via one or more network interfaces, wherein the client device comprises: a memory; an input/output (I/O) interface communicatively coupled with the memory; and a processing unit communicatively coupled with the memory and the I/O interface, the processing unit configured to: receive a plurality of data streams related to a multimodal application from one or more data sources via the I/O interface; generate at least one primary data stream and at least one secondary data stream by processing the received plurality of data streams based on a type of the multimodal application and on input requirements received from the edge analytic server, wherein the primary data stream is sufficient to perform the edge analytics at the edge analytic server, and wherein the secondary data stream provides additional information to enhance the edge analytics at the edge analytic server; determine bandwidth requirements for transmission of each of the at least one primary data stream and at least one secondary data stream; select one or more network interfaces from a plurality of network interfaces to transmit the at least one primary data stream based on traffic characteristics including at least latency and uplink speed of the plurality of network interfaces and the bandwidth requirements; and transmit the at least one primary data stream to the edge analytic server for the edge analytics via the selected one or more network interfaces, and transmit the at least one secondary data stream to the edge analytic server for aggregation with the at least one primary data stream to perform advanced edge analytics, the secondary data stream being transmitted without meeting a latency requirement, wherein the edge analytic server is configured to: receive the at least one primary data stream from the client device; and perform the edge analytics on the received at least one primary data stream.
11 . The system of claim 10 , wherein the edge analytic server is configured to: receive the at least one secondary data stream from the client device; aggregate the at least one primary data stream and at least one secondary data stream; and perform advanced edge analytics on the aggregated data streams.

Description

CROSS-REFERENCED APPLICATION This application claims priority to Indian patent application No. 202311030248, filed on Apr. 27, 2023, which is incorporated herein in its entirety by reference thereto. FIELD OF TECHNOLOGY The present disclosure relates to an edge computing technique in a wireless communication system. In particular, the present disclosure relates to a system and method for facilitating multi-level stream-based edge analytics in a multi-modal communication network. BACKGROUND Edge Computing technology brings data processing closer to the site of data collection, thereby greatly improving the response times and efficient utilization of network resources. However, with the advent of 5G communication, there has been a drastic increase in the amount of data consumed by various end-user devices that affect the performance of edge computing devices. Further, user applications may require different types of real-time data for analysis and processing. These data types, including audio, video, tactile, sensory, kinesthetic, haptics etc. are typically referred as multi-modal data, which is collected from multiple data sources. One such user application that collates multi-modal data from different sources is the tactile application. Tactile applications are currently being used in various use cases such as augmented reality, healthcare, gaming and education. To meet the growing technology needs of users, different techniques such as edge computing and 5G communications are merged to provide a seamless experience. However, the current 5G communication standards impose stringent Quality of Service (QoS) and Quality of Experience (QoE) requirements on the user applications to yield end-to-end latency, reliability, security, and high availability. However, multi-modal applications present a significant challenge in conforming to these requirements due to the large amount of data being transmitted. In view of the above limitations, there arises a need for an edge analytics system that meets the current technology needs for a multi-modal application. SUMMARY The present disclosure overcomes one or more shortcomings of the prior art and provides additional advantages discussed throughout the present disclosure. Additional features and advantages are realized through the techniques of the present disclosure. Other embodiments and aspects of the disclosure are described in detail herein and are considered a part of the disclosure. In one embodiment, a client device for facilitating multi-level stream-based edge analytics in multi-modal communication is disclosed. The client device comprises a memory and an I/O interface communicatively coupled with the memory. The client device further comprises a processing unit communicatively coupled with the memory and the I/O interface. Herein, the processing unit is configured to receive a plurality of data streams related to a multimodal application from one or more data sources via the I/O interface. The processing unit further generates at least one primary data stream and secondary data stream by processing the received plurality of data streams based on type of the multimodal application. Herein, the primary data stream is sufficient to perform the edge analytics at an edge analytic server and the secondary data stream provides additional information to enhance the edge analytics at the edge analytic server. The processing unit is configured to determine bandwidth requirements for transmission of each of the at least one primary data stream and secondary data stream; and select one or more network interfaces from a plurality of network interfaces to transmit at least one primary data stream based on traffic characteristics of the plurality of network interfaces and the bandwidth requirements. Further, the processing unit transmits at least one primary data stream to the edge analytic server for the edge analytics via the selected one or more network interfaces. In another embodiment, the processing unit is configured to further transmit the at least one secondary data stream to the edge analytic server for advanced edge analytics based on aggregation of at least one primary data stream and secondary data stream. In another embodiment, the plurality of data streams comprises at least one of a video data stream, audio data stream, and haptic data stream. In yet another embodiment, one or more data sources comprises one or more of a camera, a microphone, a temperature sensor, and an actuator. In another embodiment, the type of multimodal application comprises at least one of: remote care, remote detonation, remote operation, remote maintenance, and remote firefighting. Further, the primary data stream and secondary data stream are generated by processing the received plurality of data streams based on input requirements of the edge analytic server to perform the edge analytics based on the type of the multimodal application. In another embodiment, the primary data stream