Search

EP-4742612-A1 - METHOD AND APPARATUS FOR MONITORING PERFORMANCE OF ARTIFICIAL INTELLIGENCE (AI) MODEL, AND COMMUNICATION DEVICE, COMMUNICATION SYSTEM AND STORAGE MEDIUM

EP4742612A1EP 4742612 A1EP4742612 A1EP 4742612A1EP-4742612-A1

Abstract

Provided in the present disclosure are a method and apparatus for monitoring the performance of an artificial intelligence (AI) model, and a communication device, a communication system and a storage medium. The method comprises: determining reference channel state information (CSI) and prediction CSI, wherein the reference CSI is used as a reference for monitoring the performance of an AI model, the prediction CSI is predicted by means of the AI model and corresponds to a first prediction moment, and both the reference CSI and the prediction CSI are used for determining performance information of the AI model. Thus, the performance of an AI model for predicting CSI can be effectively monitored.

Inventors

  • LIU, Zhengxuan
  • LIU, MIN

Assignees

  • Beijing Xiaomi Mobile Software Co., Ltd.

Dates

Publication Date
20260513
Application Date
20230705

Claims (20)

  1. A method for monitoring performance of an artificial intelligence (AI) model, comprising: determining reference channel state information (CSI) and prediction CSI, wherein the reference CSI is used as a reference for AI model performance monitoring, the prediction CSI is predicted by the AI model and corresponds to a first prediction time instant, and the reference CSI and the prediction CSI are used together in determining performance information of the AI model.
  2. The method of claim 1, wherein determining the reference CSI comprises at least one of: receiving a downlink reference signal sent by a network device at a second prediction time instant, and determining the reference CSI based on the downlink reference signal, wherein the first prediction time instant may be identical to or differ from the second prediction time instant; or determining the reference CSI corresponding to the first prediction time instant based on a non-AI model mode, wherein the non-AI model mode is a mode of predicting CSI without relying on the AI model.
  3. The method of claim 2, wherein the second prediction time instant is within a prediction window, and in a case where there is one second prediction time instant, the second prediction time instant comprises a first one of time instants within the prediction window, a w d 2 th time instant within the prediction window, a w d 2 th time instant within the prediction window, or a last one of time instants within the prediction window; in a case where there are two second prediction time instants, the second prediction time instants comprise a first one of time instants within the prediction window and a w d 2 th time instant within the prediction window, a first one of time instants within the prediction window and a w d 2 th time instant within the prediction window, a w d 2 th time instant within the prediction window and a last one of time instants within the prediction window, a w d 2 th time instant within the prediction window and a last one of time instants within the prediction window, or a first one of time instants within the prediction window and a last one of time instants within the prediction window; in a case where there are more than two second prediction time instants, the second prediction time instants comprise a ( w d L l − 1 + 1 )th time instant within the prediction window or a ( w d 2 l − 1 + 1 )th time instant within the prediction window, wherein l = 1, ..., L, and L represents a number of second prediction time instants, wherein w d represents a duration of the prediction window, ┌ ┐ represents a ceiling function, and └ ┘ represents a flooring function.
  4. The method of any one of claims 1 to 3, further comprising: determining a monitored value representing performance of the AI model based on the reference CSI and the prediction CSI; and sending first indication information, wherein the first indication information comprises the monitored value and/or information associated with the monitored value.
  5. The method of claim 4, wherein determining the monitored value representing the performance of the AI model based on the reference CSI and the prediction CSI comprises at least one of: determining a first key performance indicator (KPI) between the reference CSI and the prediction CSI, and determining the first KPI as the monitored value, wherein the reference CSI is estimated based on a downlink reference signal; determining a second KPI between the reference CSI and the prediction CSI, and determining the second KPI as the monitored value, wherein the reference CSI is predicted based on a non-AI model mode; determining a first precoding matrix indicator (PMI) corresponding to the reference CSI, determining a second PMI corresponding to the prediction CSI, determining a third KPI between the first PMI and the second PMI, and determining the third KPI as the monitored value, wherein the reference CSI is predicted based on a non-AI model mode; or determining a feature vector based on the reference CSI, determining a fourth KPI between the feature vector and a second PMI, and determining the fourth KPI as the monitored value, wherein the reference CSI is estimated based on a downlink reference signal.
  6. The method of claim 5, wherein there are a plurality of reference CSI, there are a plurality of prediction CSI, different reference CSI correspond to different first prediction time instants or second prediction time instants, different prediction CSI correspond to different first prediction time instants, there are a plurality of first KPIs and/or a plurality of second KPIs and/or a plurality of third KPIs, and there are a plurality of fourth KPIs.
  7. The method of any one of claims 4 to 6, wherein sending the first indication information comprises any one of: sending one first indication information, wherein the one first indication information comprises a preset number of monitored values and/or information associated with each monitored value; or sending a preset number of first indication information, wherein each first indication information comprises one monitored value and/or information associated with the one monitored value.
  8. The method of claim 7, further comprising at least one of: receiving second indication information, wherein the second indication information comprises the preset number; determining the preset number based on a protocol; or reporting the preset number sent to a network device.
  9. The method of any one of claims 4 to 8, wherein the information associated with the monitored value comprises a comparison result between the monitored value and a corresponding monitoring threshold.
  10. The method of claim 9, further comprising at least one of: receiving second indication information, wherein the second indication information comprises the monitoring threshold; determining the monitoring threshold based on a protocol; or reporting the monitoring threshold to a network device.
  11. The method of any one of claims 1 to 3, further comprising: obtaining first processed CSI by processing the reference CSI based on a CSI generation section model, and obtaining second processed CSI by processing the prediction CSI based on the CSI generation section model, wherein the first processed CSI and the second processed CSI are used together in determining the performance information of the AI model; and sending third indication information, wherein the third indication information comprises the first processed CSI and the second processed CSI.
  12. The method of claim 11, wherein there are a plurality of reference CSI, different reference CSI correspond to different first prediction time instants or second prediction time instants, and obtaining the first processed CSI by processing the reference CSI based on the CSI generation section model comprises: obtaining a plurality of first processed CSI by processing the plurality of reference CSI based on the CSI generation section model respectively; or obtaining one first processed CSI by jointly processing the plurality of reference CSI based on the CSI generation section model.
  13. The method of claim 11, wherein there are a plurality of prediction CSI, different prediction CSI correspond to different first prediction time instants, and obtaining the second processed CSI by processing the prediction CSI based on the CSI generation section model comprises: obtaining a plurality of second processed CSI by processing the plurality of prediction CSI based on the CSI generation section model respectively; or obtaining one second processed CSI by jointly processing a plurality of prediction CSI based on the CSI generation section model.
  14. The method of any one of claims 1 to 3, further comprising: determining a first precoding matrix indicator (PMI) corresponding to the reference CSI and a second PMI corresponding to the prediction CSI, wherein the first PMI and the second PMI are used together in determining the performance information of the AI model; and sending third indication information, wherein the third indication information comprises the first PMI and the second PMI.
  15. A method for monitoring performance of an artificial intelligence (AI) model, comprising: determining performance information of the AI model, wherein the performance information is determined from reference channel state information (CSI) and prediction CSI, the reference CSI is used as a reference for AI model performance monitoring, and the prediction CSI is predicted by the AI model and corresponds to a first prediction time instant.
  16. The method of claim 15, wherein the method comprises sending a downlink reference signal to a terminal at a second prediction time instant, wherein the downlink reference signal is used for determining the reference CSI, and the first prediction time instant may be identical to or differ from the second prediction time instant; or the reference CSI is determined based on a non-AI model mode and corresponds to the first prediction time instant, wherein the non-AI model mode is a mode of predicting CSI without relying on the AI model.
  17. The method of claim 16, wherein the second prediction time instant is within a prediction window, and in a case where there is one second prediction time instant, the second prediction time instant comprises a first one of time instants within the prediction window, a w d 2 th time instant within the prediction window, a w d 2 th time instant within the prediction window, or a last one of time instants within the prediction window; in a case where there are two second prediction time instants, the second prediction time instants comprise a first one of time instants within the prediction window and a w d 2 th time instant within the prediction window, a first one of time instants within the prediction window and a w d 2 th time instant within the prediction window, a w d 2 th time instant within the prediction window and a last one of time instants within the prediction window, a w d 2 th time instant within the prediction window and a last one of time instants within the prediction window, or a first one of time instants within the prediction window and a last one of time instants within the prediction window; in a case where there are more than two second prediction time instants, the second prediction time instants comprise a ( w d L l − 1 + 1 )th time instant within the prediction window or a ( w d 2 l − 1 + 1 )th time instant within the prediction window, wherein l = 1, ..., L, and L represents a number of second prediction time instants; wherein w d represents a duration of the prediction window, ┌ ┐ represents a ceiling function, and └ ┘ represents a flooring function .
  18. The method of any one of claims 15 to 17, wherein determining the performance information of the AI model comprises: receiving first indication information, wherein the first indication information comprises a monitored value and/or information associated with the monitored value representing performance of the AI model, and the monitored value is determined from the reference CSI and the prediction CSI; and determining the performance information based on the monitored value and/or the information associated with the monitored value.
  19. The method of claim 18, wherein the monitored value comprises at least one of: a first key performance indicator (KPI) between the reference CSI and the prediction CSI, wherein the reference CSI is estimated based on a downlink reference signal; a second KPI between the reference CSI and the prediction CSI, wherein the reference CSI is predicted based on a non-AI model mode; a third KPI between a first precoding matrix indicator (PMI) corresponding to the reference CSI and a second PMI corresponding to the prediction CSI, wherein the reference CSI is predicted based on a non-AI model mode; or a fourth KPI between a feature vector determined based on the reference CSI and a second PMI, wherein the reference CSI is estimated based on a downlink reference signal.
  20. The method of claim 19, wherein there are a plurality of reference CSI, there are a plurality of prediction CSI, different reference CSI correspond to different first prediction time instants or second prediction time instants, different prediction CSI correspond to different first prediction time instants, there are a plurality of first KPIs and/or a plurality of second KPIs and/or a plurality of third KPIs, and there are a plurality of fourth KPIs.

Description

TECHNICAL FIELD The disclosure relates to a field of communication technologies, in particular to methods for monitoring performance of an artificial intelligence (AI) model, apparatuses for monitoring performance of an AI model, a communication device, a communication system, and a storage medium. BACKGROUND With the development of artificial intelligence (AI) technology, AI has been widely applied to a wireless communication physical layer. For example, channel state information (CSI) may be predicted through AI model inference, i.e., predicting future CSI. The prediction performance of an AI model may surpass that of a non-AI model. SUMMARY Embodiments of the disclosure provide methods for monitoring performance of an artificial intelligent (AI) model, related apparatuses, a device, a chip system, a storage medium, a computer program, and a computer program product, which are applied in the field of communication technologies, to solve a technical problem that the performance of the AI model for CSI prediction cannot be effectively monitored. The disclosure provides methods for monitoring performance of an AI model, apparatuses for monitoring performance of an AI model, a communication device, a communication system, and a storage medium. According to a first aspect, embodiments of the disclosure provide a method for monitoring performance of an AI model. The method includes: determining reference channel state information (CSI) and prediction CSI, in which the reference CSI is used as a reference for AI model performance monitoring, the prediction CSI is predicted by the AI model and corresponds to a first prediction time instant, and the reference CSI and the prediction CSI are used together in determining performance information of the AI model. According to a second aspect, embodiments of the disclosure provide a method for monitoring performance of an AI model. The method includes: determining performance information of the AI model, in which the performance information is determined from reference CSI and prediction CSI, the reference CSI is used as a reference for AI model performance monitoring, and the prediction CSI is predicted by the AI model and corresponds to a first prediction time instant. According to a third aspect, embodiments of the disclosure provide an apparatus for monitoring performance of an AI model. The apparatus includes: a processing module configured to determine reference CSI and prediction CSI, in which the reference CSI is used as a reference for AI model performance monitoring, the prediction CSI is predicted by the AI model and corresponds to a first prediction time instant, and the reference CSI and the prediction CSI are used together in determining performance information of the AI model. According to a fourth aspect, embodiments of the disclosure provide an apparatus for monitoring performance of an AI model. The apparatus includes: a processing module configured to determine performance information of the AI model, in which the performance information is determined from reference CSI and prediction CSI, the reference CSI is used as a reference for AI model performance monitoring, and the prediction CSI is predicted by the AI model and corresponds to a first prediction time instant. According to a fifth aspect, embodiments of the disclosure provide a communication device. The communication device includes: one or more processors. The one or more processors are used to call instructions to cause the communication device to implement the method for monitoring performance of an AI model according to the first aspect or the method for monitoring performance of an AI model according to the second aspect. According to a sixth aspect, embodiments of the disclosure provide a communication system. The communication system includes a network device and a terminal. The terminal is configured to implement the method for monitoring performance of an AI model according to the first aspect, and the network device is configured to implement the method for monitoring performance of an AI model according to the second aspect. According to a seventh aspect, embodiments of the disclosure provide a storage medium having instructions stored thereon. When the instructions are executed by a communication device, the communication device is caused to implement the method for monitoring performance of an AI model according to the first aspect or the method for monitoring performance of an AI model according to the second aspect. BRIEF DESCRIPTION OF THE DRAWINGS In order to clearly illustrate the technical solutions of embodiments of the disclosure or background technologies, descriptions of drawings used in embodiments of the disclosure or the background technologies are given below. FIG. 1 is a schematic structural diagram illustrating a communication system according to an embodiment of the disclosure.FIG. 2 is a schematic diagram illustrating an observation window and a prediction window