US-12620128-B2 - Method to determine universal heat map

US12620128B2US 12620128 B2US12620128 B2US 12620128B2US-12620128-B2

Abstract

A method for an eye tracking system comprising at least one camera configured to provide a heatmap based on an observation of at least one user and comprising the steps of: defining a region to be analysed; receiving an input in the form of a stimulus whereby the stimulus is positioned within the region; determining a first gaze metric comprising gaze point data and gaze duration data, determined in relation to the region comprising the stimulus and over time; determining a second metric related, comprising data different from the first gaze metric, determined in relation to the region comprising the stimulus and over a duration of time; dividing the region into a plurality of sections; processing data related to gaze metrics by allocating values to the sections, so that a heatmap can be generated based on these allocated values and the sections; and mapping the processed data on a heatmap.

Inventors

Richard Andersson
IHOR LOMACHENKO
YEVHEN DOBRONOHOV
IHOR BIELOUSOV
OLEKSANDRA KUNATOVA

Assignees

TOBII AB

Dates

Publication Date: 20260505
Application Date: 20240408

Claims (12)

1 . A method for an eye tracking system comprising at least one camera, the method being configured to provide a heatmap based on an observation of at least one user and comprising the steps of: defining a region that is to be analysed; receiving an input in the form of a stimulus whereby the stimulus is positioned within the region; determining a first gaze metric comprising gaze point data and gaze duration data of the user, the first gaze metric being determined in relation to the region comprising the stimulus and over a duration of time; determining a second metric related to the user, the second metric comprising data different from the first gaze metric, the second metric being determined in relation to the region comprising the stimulus and over a duration of time, wherein the second metric is at least one parameter of pupil diameter, eye openness, saccade start positions, saccade stop positions, saccade peak velocity positions, saccade peak velocities, a start position of a first saccade and/or a start position of a last saccade or a stop position of a first saccade and/or a stop position of a last saccade; dividing the region into a plurality of sections; processing data related to the first gaze metric and the second metric by allocating values to the sections, so that a heatmap can be generated based on these allocated values and the sections; mapping the processed data to the plurality of sections and generating a heatmap for visually illustrating the processed data; and storing a separate heatmap for each of the parameters of the second metric on a computer readable storage medium for analysis purposes.
2 . The method according to claim 1 , wherein the region is part of a display external to the eye tracking system and each section corresponds to a pixel of the display and wherein the first gaze metric and the second metric are determined in relation to each pixel of the display.
3 . The method according to claim 1 , wherein the second metric further comprises physiological data or behavioural data.
4 . The method according to claim 3 , wherein the region is part of a display and wherein the second metric is any parameter of: cursor position and/or cursor click; and wherein the cursor position and/or cursor click is determined in relation to the region of the display.
5 . The method according to claim 1 , comprising the step of: correlating the parameter of pupil diameter with a diameter of the eye openness, in order to determine mental workload of the user while looking at the region and the stimulus.
6 . The method according to claim 1 , comprising the steps of: detecting hotspots in each heatmap generated in the previous steps; and predicting which second metric has the highest probability for pattern identification based on the detected hotspots.
7 . The method according to claim 1 , comprising the step of performing any of the above steps for at least a second region and/or at least a second stimulus.
8 . A system comprising a processor, a camera and a computer readable storage medium, the computer readable storage medium comprising instructions executable by the processor operative to: define a region that is to be analysed; receive an input in the form of a stimulus whereby the stimulus is positioned within the region; determine a first gaze metric comprising gaze point data and gaze duration data of the user, the first gaze metric being determined in relation to the region comprising the stimulus and over a duration of time; determine a second metric related to the user, said second metric comprising data different from the first gaze metric, the second metric being determined in relation to the region comprising the stimulus and over a duration of time, wherein the second metric is at least one parameter of pupil diameter, eye openness, saccade start positions, saccade stop positions, saccade peak velocity positions, saccade peak velocities, a start position of a first saccade and/or a start position of a last saccade or a stop position of a first saccade and/or a stop position of a last saccade; divide the region into a plurality of sections; process data related to the first gaze metric and the second metric by allocating values to the sections, so that a heatmap can be generated based on these allocated values and the sections; map the processed data to the plurality of sections for visualization in a heatmap; and store a separate heatmap for each of the parameters of the second metric on a computer readable storage medium for analysis purposes.
9 . The system according to claim 8 , wherein the defined region that is to be analysed is part of a display external to the eye tracking system and each section corresponds to a pixel of the display and wherein the first gaze metric and the second metric are determined in relation to each pixel of the display.
10 . A head mounted device comprising a system according to claim 8 .
11 . The head mounted device according to claim 10 , further comprising a display.
12 . A non-transitory computer readable storage medium comprising a computer program, comprising instructions which, when executed by a processor, cause the processor to: define a region that is to be analysed; receive an input in the form of a stimulus whereby the stimulus is positioned within the region; determine a first gaze metric comprising gaze point data and gaze duration data of the user, the first gaze metric being determined in relation to the region comprising the stimulus and over a duration of time; determine a second metric related to the user, said second metric comprising data different from the first gaze metric, the second metric being determined in relation to the region comprising the stimulus and over a duration of time, wherein the second metric is at least one parameter of pupil diameter, eye openness, saccade start positions, saccade stop positions, saccade peak velocity positions, saccade peak velocities, a start position of a first saccade and/or a start position of a last saccade or a stop position of a first saccade and/or a stop position of a last saccade; divide the region into a plurality of sections; process data related to the first gaze metric and the second metric by allocating values to the sections, so that a heatmap can be generated based on these allocated values and the sections; map the processed data to the plurality of sections for visualization in a heatmap; and store a separate heatmap for each of the parameters of the second metric on a computer readable storage medium for analysis purposes.

Description

CROSS-REFERENCE TO RELATED APPLICATION The present application claims priority to Swedish patent application No. 2350413-7, filed 11 Apr. 2023, entitled “METHOD TO DETERMINE UNIVERSAL HEAT MAP,” and is hereby incorporated by reference in its entirety. TECHNICAL FIELD The present invention relates to the field of eye tracking. In particular, the present disclosure relates to the generation of heat maps based on an observation of a user with an eye tracking system. BACKGROUND OF THE INVENTION Efforts have been made to provide heat maps which capture gaze duration and gaze points in relation to an image using an eye tracking system . Such heat maps are used in analysing user behaviour and basically show a representation of visual attention. Such heat maps are used in studies related to safety, marketing, research, product placement and similar purposes. Heat maps can be static or dynamic depending on the requirements. A static heatmap is showing a visualization of user attention independent of the time passed, thus as a summary. A dynamic heat map instead illustrates the behaviour over a period of time. By capturing the behaviour over a period of time, a dynamic heat map may provide more information as to how a user looks at an image. A dynamic heat map may also be called cumulative heat map since gaze duration or gaze points are captured over a period of time. For practical applications, dynamic heat maps are usually preferred. Both static and dynamic heat maps are covered by the disclosure herein. In order to produce a heat map, typically gaze durations and gaze points, and/or the count thereof, are correlated in a map that is similar to a matrix representing a region that is being observed, the region comprising a stimulus. The gaze durations and gaze points are normalized, for example with gaussian kernels, prior to being entered into the map or matrix. For practical purposes, heat maps are often set up to overlay the captured data onto the image that has served as the stimulus. For example, in some versions the gaussian kernels may be cut off at the top in order to let a certain amount of image data pass through the gaze point data. Heat maps can be represented in several ways, for example by illustrating hot spots using a colour scale, in a luminance map where hot spots of the heat map are illustrated with higher illumination, or as focus maps whereby hotspots are displayed sharp while other parts of the heat map are visualized blurry or unsharp. Very often, the gaze data at a pixel level related to the image stimulus is captured via an eye tracking system. A pixel or square at the centre of the gaze, thus the pixel that the user currently fixates, is given a high value in a matrix while the surrounding pixels or squares are given lower values going to 0 (zero). Pixels or squares with the value 0 (zero) remain transparent, while the pixels or squares with a higher value in the matrix receive a more intense or brighter colour in the colour scale. Once the eye tracking system has collected data from a number of users or participants in study, a heat map is produced using this method based on the accumulated values. The resulting heat map presents pixels with no gaze data (zero value) transparent, while the pixels having higher values have a brighter colour, for example from light green to red with yellow in between. The pixels with high attention have thus a warm colour while the pixels with low or no attention fade to transparency, thus showing the image. A heat map may also be established based on a single user or observer. Depending on the objectives of such research, different metrics can be used. Traditionally, the most common metric or type of data collected to form the basis for gaze-related heat maps, has been the duration of the gaze related to a point on the image or just simply the gaze point. Another common metric in gaze detection is fixation count. This metric ensures that every fixation or gaze point gets equal weight regardless of gaze duration. Both gaze duration and count data may be collected as absolute values or as a relative metric, where the input is scaled so that for example gaze fixation data from one user has the same weight as gaze fixation data from another user. Further, various filters and scales can be applied to adjust a heat map based on number of participants, sensitivity, gaze duration and so on. For the purposes of studying user behaviour when confronted with visual stimulus, an image or a region or an observed region may be divided into areas of interest. A resulting heat map then represents the gaze data related to individual areas of interest, as collected using the eye tracking system. The disclosure herein is however not limited to areas of interest. Traditional heat maps, however, have limitations as to their ability to illustrate data beyond gaze point and gaze duration. Current research and applications of heat maps have shown that there is an interest and deman