US-12619873-B2 - Providing unlabelled training data for training a computational model

Abstract

Providing unlabelled training data for training a computational model comprises: obtaining sets of time-aligned unlabelled data, wherein the sets correspond to different ones of a plurality of sensors; marking a first sample, of a first set of the sets, as a positive sample, in dependence on statistical separation information indicating a first statistical similarity of at least a portion of the first set to the at least a portion of the reference set and in dependence on the first sample being time-aligned relative to a reference time; marking a second sample, of a second set of the sets, as a negative sample, in dependence on statistical separation information indicating a second, lower statistical similarity, of at least a portion of the second set to the at least a portion of the reference set, and in dependence on the second sample being time-misaligned relative to the reference time.

Inventors

• Akhil Mathur
• Chulhong Min
• Fahim Kawsar

Assignees

• NOKIA TECHNOLOGIES OY

Dates

Publication Date

20260505

Application Date

20221024

Priority Date

20211112

Claims (20)

1. 1 . A method comprising: obtaining one or more sets of time-aligned unlabeled data corresponding to different ones of a plurality of sensors; obtaining statistical separation information, indicative of statistical separation of at least portions of respective ones of the one or more sets of time-aligned unlabeled data from at least a portion of a reference set associated with a reference class; marking a first sample, of a first set of the one or more sets of time-aligned unlabeled data, as a positive sample, in dependence on the statistical separation information indicating a first statistical similarity of at least a portion of the first set to the at least a portion of the reference set and in dependence on the first sample being time-aligned relative to a reference time, wherein marking the first sample as the positive sample comprises associating the first sample with a positive label for the reference class; marking a second sample, of a second set of the one or more sets of time-aligned unlabeled data, as a negative sample, in dependence on the statistical separation information indicating a second, lower statistical similarity, of at least a portion of the second set to the at least a portion of the reference set, and further in dependence on the second sample being time-misaligned relative to the reference time, wherein marking the second sample as the negative sample comprises associating the second sample with a negative label for the reference class; and providing the positive sample and the negative sample for training a computational model.
2. 2 . The method of claim 1 , wherein (i) the time-aligned unlabeled data of the sets indicate a time-varying context of a common subject, (ii) the first sample and the second sample are within nonoverlapping time windows, and (iii) the plurality of the sensors comprises sensors having a same sensing modality.
3. 3 . The method of claim 1 , wherein the time-aligned unlabeled data comprises one or more of: motion-dependent data; pressure-dependent data; image frame data; audio data; radio signal data; electricity data; or force-dependent data.
4. 4 . The method of claim 1 , wherein the marking of the second sample as the negative sample comprises assigning a training weight to the second set or sample based on the second, lower statistical similarity.
5. 5 . The method of claim 1 , further comprising training the computational model based on one or more positive samples including the positive sample of claim 1 and based on multiple negative samples including the negative sample of claim 1 .
6. 6 . The method of claim 5 , wherein training the computational model comprises executing a loss function to train a feature extractor, wherein the loss function is configured to determine a loss value based on an aggregate of the positive samples and based on an aggregate of the multiple negative samples and based on the reference set.
7. 7 . The method of claim 5 , wherein training the computational model comprises providing further positive and negative samples based on one or more of: different reference times; or different reference sets, and comprises determining the loss value iteratively, based on the further positive and negative samples, until a convergence criterion is satisfied.
8. 8 . The method of claim 1 , wherein training the computational model comprises training a classifier based on a labeled dataset associated with the reference set.
9. 9 . The method of claim 1 , further comprising: obtaining the one or more sets of unlabeled data, each set comprising timestamp information; and time-aligning the sets of unlabeled data based on the timestamp information, to obtain the sets of time-aligned unlabeled data.
10. 10 . The method of claim 1 , further comprising: training the computational model based on the positive sample and the negative sample and the reference set; and associating or sending the trained computational model to a device with which the reference set is associated.
11. 11 . The method of claim 1 , wherein (i) the plurality of sensors are associated with two or more devices, (ii) the one or more sets of time-aligned unlabeled data correspond to two or more devices, (iii) the two or more devices are configured to be time-aligned by a primary reference clock that is synchronizing a network to which the two or more devices are connected, and (iv) the method further comprises: selecting a reference sample from at least the portion of the reference set; generating a transformed version of the reference sample by applying a pre-defined rotation; and providing the positive sample, the negative sample, and the transformed version of the reference sample for training the computational model.
12. 12 . The method of claim 11 , further comprising: providing the positive sample, the negative sample, and the transformed version of the reference sample to a neural network to determine one or more feature embeddings.
13. 13 . A device comprising: at least one processor; and at least one memory including computer program code; the at least one memory storing instructions that, when executed by the at least one processor, cause the device at least to: record data of a sensor associated with the device, wherein the device and one or more other devices associated with a same user are configured to be time-aligned by a primary reference clock that is synchronizing a network to which the device and the one or more other devices are connected; define a set of the sensor data as a reference set; send the reference set to an apparatus for training a computational model; receive from the apparatus a trained computational model, trained based on the sent reference set and one or more of sets of time-aligned unlabeled data from one or more other sensors associated with the one or more other devices; determine an inference based on applying further data from the sensor to the trained computational model; and control a function of the device based on the inference.
14. 14 . The device of claim 13 , further cause to send an indication of a modality of the reference set for training the computational model; and receive from the apparatus the trained computational model, trained based on the sent reference set, the indication of a modality of the reference set and the plurality of sets of time-aligned unlabeled data from one or more of other sensors.
15. 15 . The device of claim 14 , wherein the modality of the reference set comprises one or more of one or more measured physical attributes, features or properties relating to an input used by a specific sensor, or one or more attributes, features or properties relating to a subject being sensed.
16. 16 . The device of claim 15 , wherein the one or more of the other sensors are related to the subject being sensed.
17. 17 . The device of claim 15 , wherein the subject refers to one or more of a device, user, object, space or scene that is being sensed.
18. 18 . The device of claim 13 , further cause to send to the apparatus a current computational model used in the device; or an indication of the current computational model used in the device.
19. 19 . The device of claim 13 , wherein the training is based on: a) a marking of a first sample, of a first set of the one or more of sets, as a positive sample, in dependence on statistical separation information indicating a first statistical similarity of at least a portion of the first set to the at least a portion of the reference set and in dependence on the first sample being time-aligned relative to a reference time; and b) a marking of a second sample, of a second set of the one or more of sets, as a negative sample, in dependence on statistical separation information indicating a second, lower statistical similarity, of at least a portion of the second set to the at least a portion of the reference set, and further in dependence on the second sample being time-misaligned relative to the reference time.
20. 20 . An apparatus comprising: at least one processor; and at least one memory including computer program code; the at least one memory storing instructions that, when executed by the at least one processor, cause the device at least to: obtain one or more sets of time-aligned unlabeled data corresponding to different ones of a plurality of sensors; obtain statistical separation information, indicative of statistical separation of at least portions of respective ones of the one or more sets of time-aligned unlabeled data from at least a portion of a reference set associated with a reference class; mark a first sample, of a first set of the one or more sets of time-aligned unlabeled data, as a positive sample, in dependence on the statistical separation information indicating a first statistical similarity of at least a portion of the first set to the at least a portion of the reference set and in dependence on the first sample being time-aligned relative to a reference time, wherein marking the first sample as the positive sample comprises associating the first sample with a positive label for the reference class; mark a second sample, of a second set of the one or more sets of time-aligned unlabeled data, as a negative sample, in dependence on the statistical separation information indicating a second, lower statistical similarity, of at least a portion of the second set to the at least a portion of the reference set, and further in dependence on the second sample being time-misaligned relative to the reference time, wherein marking the second sample as the negative sample comprises associating the second sample with a negative label for the reference class; and provide the positive sample and the negative sample for training a computational model.

Description

TECHNOLOGICAL FIELD Embodiments of the present disclosure relate to providing unlabelled training data for training a computational model. Some relate to training the computational model. BACKGROUND Supervised machine learning requires labelled data samples to train the computational model. Collection of labelled data for complex models, such as deep learning models, is cumbersome, especially outside of a laboratory. Self-supervised machine learning (SSL) analyses unlabelled input data to derive a supervisory signal. However, pretext tasks for SSL are often hand-crafted (e.g., rotate image by 60 degrees) or rely on pretrained data, because the choice of pretext tasks can have a profound impact on the SSL performance. In addition, for sensor signals (e.g., accelerometer, gyroscope, electrocardiogram, skin conductance, etc.), it is difficult to manually define pretext tasks that could lead to learning of robust feature representations. BRIEF SUMMARY Embodiments of the present disclosure improve upon SSL in a manner that can be described as Group Supervised Learning (GSL), because it leverages the capabilities of a plurality of sensors measuring the same object, for preparing training samples. According to various, but not necessarily all, embodiments there is provided an apparatus comprising means configured to, or means for: obtaining a plurality of sets of time-aligned unlabelled data, wherein the sets correspond to different ones of a plurality of sensors;obtaining statistical separation information, indicative of statistical separation of at least portions of individual ones of the sets from at least a portion of a reference set;marking a first sample, of a first set of the plurality of sets, as a positive sample, in dependence on the statistical separation information indicating a first statistical similarity of at least a portion of the first set to the at least a portion of the reference set and in dependence on the first sample being time-aligned relative to a reference time;marking a second sample, of a second set of the plurality of sets, as a negative sample, in dependence on the statistical separation information indicating a second, lower statistical similarity, of at least a portion of the second set to the at least a portion of the reference set, and further in dependence on the second sample being time-misaligned relative to the reference time; andproviding the positive sample and the negative sample for training a computational model. According to some but not necessarily all examples the time-aligned unlabelled data of the sets indicate a time-varying context of a common subject. According to some but not necessarily all examples the plurality of the sensors comprises sensors having a same sensing modality. According to some but not necessarily all examples the first sample and the second sample are within non-overlapping time windows. According to various, but not necessarily all, embodiments there is provided a system comprising the apparatus, and one or more devices individually associated with different ones of the plurality of sensors. According to various, but not necessarily all, embodiments there is provided a computer-implemented method comprising: obtaining a plurality of sets of time-aligned unlabelled data, wherein the sets correspond to different ones of a plurality of sensors;obtaining statistical separation information, indicative of statistical separation of at least portions of individual ones of the sets from at least a portion of a reference set;marking a first sample, of a first set of the plurality of sets, as a positive sample, in dependence on the statistical separation information indicating a first statistical similarity of at least a portion of the first set to the at least a portion of the reference set and in dependence on the first sample being time-aligned relative to a reference time;marking a second sample, of a second set of the plurality of sets, as a negative sample, in dependence on the statistical separation information indicating a second, lower statistical similarity, of at least a portion of the second set to the at least a portion of the reference set, and further in dependence on the second sample being time-misaligned relative to the reference time; andproviding the positive sample and the negative sample for training a computational model. According to various, but not necessarily all, embodiments there is provided a computer program that, when run on a computer, performs: causing obtaining of a plurality of sets of time-aligned unlabelled data, wherein the sets correspond to different ones of a plurality of sensors;causing obtaining of statistical separation information, indicative of statistical separation of at least portions of individual ones of the sets from at least a portion of a reference set;causing marking of a first sample, of a first set of the plurality of sets, as a positive sample, in dependence on the statistical separation informa