CN-122003909-A - Timely real-time AIML reasoning and troubleshooting in communications

Abstract

Embodiments of the present application relate to apparatus, methods, devices, and computer-readable storage media for timely real-time AIML reasoning and troubleshooting. A method may include obtaining, at a first network entity, information about a trigger time of at least one ML inference and information about a delivery time of the at least one ML inference, determining an inference delay metric based on the obtained information, identifying at least one late-to-ML inference, obtaining trigger time and delivery time information about the at least one identified late-to-ML inference, and transmitting time-related information.

Inventors

• A. Galani
• B. Jiayi Qi
• S. m Wan Jie
• SUN SHUQIANG

Assignees

• 诺基亚通信(上海)股份有限公司
• 诺基亚通信公司

Dates

Publication Date

20260508

Application Date

20231011

Claims (20)

1. 1. An apparatus, comprising: Means for obtaining, at a first network entity, first information regarding a trigger time of the at least one machine-learned ML inference and second information regarding a delivery time of the at least one ML inference, wherein the trigger time of the at least one ML inference comprises a time at which the first network entity requested the at least one ML inference, and the delivery time of the at least one ML inference comprises a time at which an inference output was received by the first network entity, and wherein the first network entity comprises an ML inference consumer; Means for determining, at the first network entity, an inference delay metric based on a value of a trigger time of the at least one ML inference and a value of the corresponding delivery time of the at least one ML inference; means for identifying at least one late-to-ML inference of the at least one ML inference at the first network entity, wherein the at least one late-to-ML inference is identified based on a value of the inference delay metric and an inference delay objective; means for obtaining, at the first network entity, third information regarding a trigger time of the at least one identified late-to-ML reasoning and fourth information regarding a corresponding delivery time of the at least one identified late-to-ML reasoning; means for transmitting from the first network entity to a second network entity at least one of the following information: said first information regarding said trigger time of said at least one ML inference, Said second information regarding said delivery time of said at least one ML inference, Said third information regarding said trigger time of said at least one late-to-ML reasoning, Said fourth information about said delivery time of said at least one ML late inference.
2. 2. The apparatus of claim 1, further comprising: Means for receiving, at the first network entity, a troubleshooting report derived based on the at least one of the information transmitted from the first network entity from the second network entity, wherein the troubleshooting report includes at least one of the following information: fifth information regarding the reason for the at least one late ML reasoning, Sixth information regarding recommendations of an ML model and/or hardware platform for performing the at least one ML inference; Seventh information regarding a recommendation to retrain the ML model for performing the at least one ML inference.
3. 3. The apparatus of any of the preceding claims, further comprising: Means for sending a request from the first network entity to the second network entity for an estimated computational inference delay metric; Means for receiving, at the first network entity, from the second network entity, a calculated inference delay metric for performing at least one candidate ML model and an estimate of at least one candidate hardware platform for the at least one inference, and Means for selecting, at the first network entity, at least one ML model and at least one hardware platform for performing the at least one inference from the at least one candidate ML model and at least one candidate hardware platform based on the received estimated computational inference latency metrics.
4. 4. The apparatus of any preceding claim, wherein the first network entity is included in a user equipment.
5. 5. The apparatus of any preceding claim, wherein the second network entity is comprised in a base station.
6. 6. The apparatus of any of the preceding claims, wherein the means for transmitting further comprises means for transmitting the inferred time delay target from the first network entity to the second network entity.
7. 7. The apparatus of claims 1-3, further comprising: Means for receiving, at the first network entity, eighth information regarding calculated trigger time of the at least one ML inference and ninth information regarding calculated delivery time of the at least one inference from a third network entity, wherein the trigger time of the at least one ML inference includes time at which an inference input arrives at the hardware platform for performing the at least one ML inference, and the calculated delivery time of the at least one ML inference includes time at which an inference output of the at least one inference is obtained at the third network entity, and Means for transmitting from the first network entity to the second network entity at least one of the eighth information regarding the calculated trigger time of the at least one inference and the ninth information regarding the calculated delivery time of the at least one inference.
8. 8. The apparatus of claim 7, wherein the means for transmitting further comprises means for transmitting the inferred time delay target from the first network entity to the third network entity.
9. 9. The apparatus of claim 7 or 8, wherein the first network entity is comprised in a radio unit of a base station, the second network entity is comprised in a distributed unit of the base station, and the third network entity is comprised in a user equipment.
10. 10. An apparatus, comprising: Means for obtaining, at a second network entity, first information regarding a trigger time of at least one machine learning, ML, inference calculation and second information regarding a calculated delivery time of the at least one ML inference, wherein the trigger time of the at least one ML inference calculation includes a time at which an inference input arrives at a hardware platform for performing the at least one ML inference, the calculated delivery time of the at least one ML inference includes a time at which an inference output of the at least one ML inference is obtained at the second network entity, wherein the second network entity includes an ML inference producer; Means for receiving at the second network entity from the first network entity at least one of the following information: third information about the trigger time of the at least one ML inference, Fourth information about the delivery time of the at least one ML inference, Fifth information regarding at least one late-to-ML-inference trigger time of the at least one ML-inference, Said sixth information about the time of arrival of said at least one ML late inference for said at least one ML inference, and The time delay goal is inferred and, Wherein the trigger time of the at least one ML inference comprises a time at which the first network entity requested the at least one ML inference, the arrival time of the at least one ML inference comprises a time at which the first network entity received an inference output, the trigger time of the at least one late ML inference in the at least one ML inference comprises a time at which the first network entity requested the at least one late ML inference of the at least one ML inference, the arrival time of the at least one ML late inference comprises a time at which an inference output provided by the at least one of the at least one ML late inferences was received by the first network entity, and late ML inference means that a difference between its arrival time and its trigger time is greater than the ML inference of the inference delay objective, and wherein the first network entity comprises an ML consumer of the inference Means for sending a troubleshooting report from the second network entity to the first network entity, which is derived based on the information received at the second network entity and information obtained at the second network entity about the calculated trigger time and the delivery time of the at least one inference.
11. 11. The apparatus of claim 10, further comprising: Means for determining, at the second network entity, a latency-related metric based on the at least one information received at the second network entity and at least one of the trigger time and the information regarding the calculation of the at least one inference obtained at the second network entity, and Means for deriving the troubleshooting report based on the latency-related metric at the second network entity.
12. 12. The apparatus of claim 10, further comprising: Means for transmitting from the second network entity to a fourth network entity, the fourth network entity comprising a management service producer, the at least one piece of information received at the second network entity and the information obtained at the second network entity about the calculated trigger time and the delivery time, and Means for receiving, at the second network entity, the troubleshooting report prepared by the fourth network entity from the fourth network entity.
13. 13. The apparatus of any of claims 10 to 12, wherein the troubleshooting report includes at least one of the following information: seventh information regarding the reason for the at least one late ML reasoning, Eighth information regarding recommendations of an ML model and/or hardware platform for performing the at least one ML inference; ninth information regarding a recommendation to retrain the ML model for performing the at least one ML inference.
14. 14. The apparatus of any of claims 10 to 13, further comprising: Means for determining, at the second network entity, a calculated inference delay metric for performing an estimation of at least one ML model and at least one candidate hardware platform for the at least one inference based on the inference delay objective; Means for transmitting the estimated computational inference delay measures from the second network entity to the first network entity.
15. 15. The apparatus of claim 14, further comprising: Means for receiving, at the second network entity, a request from the first network entity for an estimated computational inference delay metric, and Wherein the means for determining the estimated computational inference delay metric further comprises means for determining the estimated computational inference delay metric in response to the request.
16. 16. The apparatus of any of claims 10 to 15, wherein the first network entity is comprised in a user equipment.
17. 17. The apparatus of any of claims 10 to 16, wherein the second network entity is comprised in a base station.
18. 18. An apparatus, comprising: Means for obtaining, at a third network entity, first information regarding calculated trigger times of at least one machine learning, ML, inference and second information regarding calculated delivery times of the at least one inference, wherein the trigger times of the at least one ML inference include times at which inference inputs arrive at a hardware platform for performing the at least one ML inference, the calculated delivery times of the at least one ML inference include times at which inference outputs of the at least one ML inference are obtained at the third network entity, wherein the first network entity includes an ML inference consumer and the third network entity includes an ML inference producer; means for transmitting, from the third network entity to the first network entity, the obtained first information about the calculated trigger time of the at least one inference and the obtained second information about the calculated delivery time of the at least one inference, and Means for receiving, at the third network entity, a troubleshooting report from the first network entity derived based on information obtained at the third network entity about the trigger time of the calculation of the at least one inference and information about the delivery time of the calculation of the at least one inference.
19. 19. The apparatus of claim 18, wherein the troubleshooting report includes at least one of the following information: Third information regarding the reason for the at least one late-to-ML reasoning, Fourth information about recommendations of an ML model and/or hardware platform for performing the at least one ML inference; Fifth information about retraining recommendations of an ML model for performing the at least one ML inference.
20. 20. The apparatus of any of claims 18 to 19, wherein the third network entity is comprised in a user equipment.

Description

Timely real-time AIML reasoning and troubleshooting in communications Technical Field Various example embodiments of this application relate generally to the field of telecommunications and, in particular, relate to methods, apparatus, devices and computer-readable storage media for timely real-time Artificial Intelligence Machine Learning (AIML) reasoning and troubleshooting. Background A communication system may be considered a facility that enables communication sessions between two or more entities, such as user terminals, base stations, and/or other nodes, by providing carriers between various entities in the communication system. The user may access the communication system through an appropriate communication device or terminal. The user's communication equipment may be referred to as User Equipment (UE) or user equipment. The communication device is provided with suitable signal receiving and transmitting means for enabling communication, for example enabling access to a communication network or communication directly with other users. A communication device may access a station, such as a base station in a cell, and transmit and/or receive communications via a modulated carrier. Communication systems and associated devices typically operate in accordance with a given standard or specification which sets out what the various entities associated with the system are permitted to do and how that should be achieved. Communication protocols and/or parameters which should be used for the connection are also typically defined. An example of a communication system is the Universal Mobile Telecommunications System (UMTS) terrestrial radio access network (UTRAN) (3G radio). Other examples of communication systems are the Universal Mobile Telecommunications System (UMTS) radio access technology and the Long Term Evolution (LTE) of the so-called new air interface (NR) network (5G radio). NR is standardized by the third generation partnership project (3 GPP). Other examples of communication systems include 5G enhancements (NR Rel-18 and above) and 6G. Disclosure of Invention This summary is provided to introduce a selection of concepts in a simplified form that are further described below in the detailed description. This summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter. According to a first aspect, an apparatus comprises means for obtaining, at a first network entity, first information regarding a trigger time of at least one ML inference and second information regarding a delivery time of the at least one ML inference, wherein the trigger time of the at least one ML inference comprises a time when the first network entity requested the at least one ML inference and the delivery time of the at least one ML inference comprises a time when an inference output is received by the first network entity, and wherein the first network entity comprises an ML inference consumer; means for determining, at the first network entity, an inference delay metric based on a value of a trigger time of the at least one ML inference and a value of the corresponding arrival time of the at least one ML inference, means for identifying, at the first network entity, at least one late ML inference of the at least one ML inference, wherein the at least one late ML inference is identified based on the value of the inference delay metric and an inference delay objective, means for obtaining, at the first network entity, third information about the at least one identified trigger time of late ML inference and fourth information about the at least one identified corresponding arrival time of late ML inference, means for transmitting, from the first network entity to a second network entity, at least one of the first information about the trigger time of the at least one ML inference, the second information about the arrival time of the at least one ML inference, the third information about the arrival time of the at least one late ML inference, said fourth information regarding said delivery time of said at least one late-to-ML inference. According to an example embodiment of the first aspect, the apparatus may further comprise means for receiving, at the first network entity, from the second network entity, a troubleshooting report derived based on the at least one of the information transmitted from the first network entity, wherein the troubleshooting report comprises at least one of fifth information regarding a cause of the at least one late-to-ML reasoning, sixth information regarding a recommendation of an ML model and/or a hardware platform for performing the at least one ML reasoning, seventh information regarding a recommendation to retrain the ML model for performing the at least one ML reasoning. According to an example embodiment of the first aspect, the apparatus may further comprise means for send