EP-4526801-B1 - SPATIAL AREA LAYOUT RECONSTRUCTION BASED ON RADIO FREQUENCY MEASUREMENTS

EP4526801B1EP 4526801 B1EP4526801 B1EP 4526801B1EP-4526801-B1

Inventors

KALATZIS, Dimitrios
OREKONDY, TRIBHUVANESH
Ackermann, Hanno
KARMANOV, Ilia
Ghazvinian Zanjani, Farhad
DIJKMAN, Daniel Hendricus Franciscus
BEHBOODI, Arash
KADAMBI, Shreya
PORIKLI, Fatih Murat

Dates

Publication Date: 20260506
Application Date: 20230505

Claims (15)

A computer-implemented method (900) comprising: receiving (910) an input data set including a plurality of multidimensional samples from a spatial area, each sample of the plurality of multidimensional samples including at least radio frequency wireless channel state information data, localization data, and time data; and training (920) a machine learning model to predict a layout of the spatial area based on the input data set, wherein the predicted layout of the spatial area comprises a plurality of bounding boxes defining different regions of the spatial area, and wherein training the machine learning model comprises: training a first machine learning model to generate a representation of each sample of the plurality of the samples, and training a second machine learning model to generate the plurality of bounding boxes for discrete portions of the spatial area based on the representation of each sample of the plurality of samples.
The method (900) of Claim 1, wherein: the channel state information data comprises power measurements at a given location and time in a three-dimensional space; or the localization data comprises acceleration data and velocity data for a wireless device associated with the channel state information data.
The method (900) of Claim 1, wherein the predicted layout of the spatial area further comprises one or more of a predicted number of regions in the spatial area, a predicted number of openings between regions in the spatial area, predicted coordinates of each region in the spatial area, and predicted coordinates of each opening between regions in the spatial area.
The method (900) of Claim 1, wherein: the first machine learning model comprises a transformer encoder, and training the first machine learning model comprises training the transformer encoder to generate, from the input data set, an output sequence that identifies local correlations within sequences of samples in the input data set.
The method (900) of Claim 4, wherein training the second machine learning model comprises training the second machine learning model to generate: the bounding boxes, wherein each bounding box comprises a set of coordinates in a multi-dimensional space; and a layout of the spatial area from the bounding boxes.
The method (900) of Claim 5, wherein training the second machine learning model further comprises training the second machine learning model to match point sets corresponding to the coordinates of the bounding boxes based on a minimization of one or more of a Chamfer distance between the coordinates of the bounding boxes or a Hungarian loss metric corresponding to the coordinates of each bounding box of the plurality of bounding boxes, in order to generate the layout of the spatial area, optionally wherein training the second machine learning model further comprises training the second machine learning model to minimize a mean intersection over union, IoU, measurement between the bounding boxes.
The method (900) of Claim 1, wherein training the machine learning model (920) further comprises: training the machine learning model to predict the layout of the spatial area based on a predicted distribution of layouts in the spatial area for the input data set; or training the machine learning model to generate the bounding boxes to have non-contiguous coordinates.
The method (900) of Claim 1, wherein training (920) the machine learning model comprises training the machine learning model to predict a distribution of layouts in the spatial area for the input data set based on a joint distribution over parameters of the machine learning model and the predicted distribution of layouts, optionally wherein training (920) the machine learning model comprises training the machine learning model to predict a posterior distribution over weights of the machine learning model, approximated based on a Kullback-Leibler, KL, -divergence measurement of an approximate probability distribution over the weights.
A computer-implemented method (1000) comprising: receiving (1010) an input data set including a plurality of multidimensional samples from a spatial area, each sample including at least radio frequency wireless channel state information data, localization data, and time data; predicting (1020) a layout of the spatial area based on a machine learning model and the received input data set, wherein the predicted layout of the spatial area comprises a plurality of bounding boxes defining different regions of the spatial area, and wherein predicting the layout of the spatial area comprises generating a representation of each sample of the plurality of samples using a first machine learning model; and generating the bounding boxes for discrete portions of the spatial area and the layout of the spatial area using a second machine learning model and the representation of each sample of the plurality of samples; and outputting (1030) the predicted layout of the spatial area.
The method (1000) of Claim 9, wherein the channel state information data comprises power measurements at a given location and time in a three-dimensional space.
The method (1000) of Claim 9, wherein the predicted layout of the spatial area further comprises at least one of a predicted number of regions in the spatial area, a predicted number of openings between regions in the spatial area, predicted coordinates of each region in the spatial area, and predicted coordinates of each opening between regions in the spatial area.
The method (1000) of Claim 9, wherein: the first machine learning model comprises a transformer encoder configured to generate, from the input data set, an output sequence with same dimensions as the input data set that identifies local correlations within sequences of samples in the input data set; and the second machine learning model is configured to generate: the bounding boxes, wherein each bounding box comprises a set of coordinates in a multi-dimensional space, and a layout of the spatial area from the bounding boxes.
A non-transitory computer-readable medium comprising instructions that, when executed by one or more processors of a processing system, cause the processing system to perform: the method (900) of any one of claims 1 to 8; the method (1000) of any one of claims 9 to 12.
A system (1100) comprising: a memory (1124) having executable instructions stored thereon; and a processor (1102, 1104, 1106, 1108) configured to execute the executable instructions in order to cause the system to: receive (910) an input data set including a plurality of multidimensional samples from a spatial area, each sample of the plurality of multidimensional samples including at least radio frequency wireless channel state information data, localization data, and time data; and train (920) a machine learning model to predict a layout of the spatial area based on the input data set, wherein the predicted layout of the spatial area comprises a plurality of bounding boxes defining different regions of the spatial area, and wherein training the machine learning model comprises: training a first machine learning model to generate a representation of each sample of the plurality of the samples, and training a second machine learning model to generate the plurality of bounding boxes for discrete portions of the spatial area based on the representation of each sample of the plurality of samples.
A system (1200) comprising: a memory (1224) having executable instructions stored thereon; and a processor (1202, 1204, 1206, 1208) configured to execute the executable instructions in order to cause the system to: receive (1010) an input data set including a plurality of multidimensional samples from a spatial area, each sample including at least radio frequency wireless channel state information data, localization data, and time data; predict (1020) a layout of the spatial area based on a machine learning model and the received input data set, wherein the predicted layout of the spatial area comprises a plurality of bounding boxes defining different regions of the spatial area, and wherein predicting the layout of the spatial area comprises: generating a representation of each sample of the plurality of samples using a first machine learning model; and generating the bounding boxes for discrete portions of the spatial area and the layout of the spatial area using a second machine learning model and the representation of each sample of the plurality of samples; and output (1030) the predicted layout of the spatial area.

Description

INTRODUCTION Aspects of the present disclosure relate to using machine learning to estimate the layout of a spatial area based on radio frequency measurements. In a wireless communications system, information about the layout of a spatial area in which operations are performed and location estimation (e.g., relative to one or more network entities) within the spatial environment may be used for various purposes. For example, layout information and location estimates can be used to aid in identifying various parameters for subsequent transmissions in the wireless communications system, such as identifying one or more directional beams to use in communicating between a network entity (e.g., a base station) and a user equipment, to identify beamforming patterns to apply to allow for directionality in signal processing, and the like. In another example, location estimation can be used to detect entry and exit of devices into different areas (e.g., defined based on a radius from a given device). Layout information and location estimation can be used for many other purposes as well, such as emergency management within the spatial area, spatial management, and the like. Generally, radio frequency measurements within a spatial area may differ due to various factors within the spatial area. For example, sources of radio frequency interference, such as interfering network entities, may affect radio frequency measurements in some parts of the spatial area. In another example, hard surfaces, such as walls, support columns, or the like may introduce variance, or noise, in radio frequency measurements obtained within the spatial area. Because radio frequency measurements are generally noisy, it may be difficult to accurately estimate the layout of the spatial area based on these radio frequency measurements alone. The paper "Environment Mapping Using Wireless Channel State Information and Deep Learning" by Adrian Donarski et al; XP033872074, describes the use of wireless Channel State Information (CSI) and deep learning for obtaining situational awareness through environment mapping. Algorithms that permit estimation of interior room dimensions using Single-Input Single-Output (SISO) Channel Input Responses (CIRs) are described. The paper "WiCluster: Passive Indoor 2D/3D Positioning using WiFi without Precise Labels" by Ilia Karmanov et al; XP091045871, describes machine learning techniques for passive indoor positioning using Radio Frequency (RF) CSI. The paper "Multi-Faceted Representation Learning with Hybrid Architecture for Time Series Classifiation" by Zhenyu Liu et al; XP081843587, describes a hybrid neural architecture for learning mutli-faceted representations for univariate time series. The paper "See Through Smoke: Robust Indoor Mapping with Low-cost mmWave Radar" by Chris Xiaoxuan Lu et al; XP081659213, describes the design of a single-chip millimeter wave (mmWave) radar based indoor mapping system targeted towards low-visibility environments to assist in emergency response. BRIEF SUMMARY The scope of protection is defined by the scope of the appended claims. Any examples which do not fall within the scope of the claims are not presented as embodiments but as examples which are helpful for understanding the invention. BRIEF DESCRIPTION OF THE DRAWINGS The appended figures depict example features of certain aspects of the present disclosure and are therefore not to be considered limiting of the scope of this disclosure. FIG. 1 illustrates radio frequency measurements in a spatial area and radio frequency measurements captured during traversal of a path in the spatial area.FIG. 2 depicts an example representation of a point along a path in a spatial area, according to aspects of the present disclosure.FIG. 3 depicts an example pipeline for predicting a layout of a spatial area based on a set of multidimensional samples representing data captured while traversing a path through the spatial area and a set predictor model, according to aspects of the present disclosure.FIG. 4 depicts an example of a machine learning model that generates a representation for each sample in a set of multidimensional samples representing data captured while traversing a path through a spatial area, according to aspects of the present disclosure.FIG. 5 depicts an example of a machine learning model that generates bounding boxes from representations of multidimensional samples representing data captured while traversing a path through a spatial area, according to aspects of the present disclosure.FIG. 6 depicts a transformation of bounding boxes representing different portions of a spatial area into a layout of the spatial area, according to aspects of the present disclosure.FIG. 7 depicts an example visual representation of time data used in predicting a layout of a spatial area, according to aspects of the present disclosure.FIG. 8 depicts an example of a machine learning model that predicts a layout of a spatial area based on channe