EP-4742009-A1 - EYE TRACKING METHOD AND DEVICE, AND STORAGE MEDIUM

EP4742009A1EP 4742009 A1EP4742009 A1EP 4742009A1EP-4742009-A1

Abstract

This application provides an eye tracking method, a device, and a storage medium. The method is applied to an electronic device that includes a first infrared light source and a camera module. The camera module includes a receiver and an emitter that is integrated with a second infrared light source. For the electronic device of this structure, a service scenario in which the electronic device is currently located is determined, and then in a case that the electronic device is in a high-precision eye tracking scenario, two infrared light sources are controlled to emit infrared lights to a target object that includes an eye of a user, and the receiver receives infrared reflection lights of a first infrared light and a second infrared light reflected by the target object, to generate a first infrared image of the eye, so that the electronic device can perform eye tracking based on the first infrared image. Because the camera module itself has one infrared light source, high-precision eye tracking can be implemented by only adding at least one additional infrared light source. Therefore, precision of eye tracking is ensured, and a quantity of screen holes of the electronic device can be reduced.

Inventors

MENG, Yuhuang
FENG, XIAOGANG
YUAN, Jiangfeng

Assignees

Honor Device Co., Ltd.

Dates

Publication Date: 20260513
Application Date: 20240802

Claims (13)

An eye tracking method, applied to an electronic device, wherein the electronic device comprises a first infrared light source and a camera module, the camera module comprises an emitter and a receiver, and the emitter comprises a second infrared light source, wherein the method comprises: determining, by the electronic device, a service scenario in which the electronic device is currently located; controlling, in a case that the electronic device is in a first service scenario, the first infrared light source to emit a first infrared light to a target object, and controlling the second infrared light source to emit a second infrared light to the target object, wherein the target object comprises an eye of a user, and the first service scenario is a high-precision eye tracking scenario; receiving, by the receiver, a first infrared reflection light of the first infrared light reflected by the target object, and a second infrared reflection light of the second infrared light reflected by the target object, to generate a first infrared image of the eye; and performing, by the electronic device, eye tracking based on the first infrared image.
The method according to claim 1, wherein the receiving, by the receiver, a first infrared reflection light of the first infrared light reflected by the target object, and a second infrared reflection light of the second infrared light reflected by the target object, to generate a first infrared image of the eye comprises: receiving, by the receiver, the first infrared reflection light of the first infrared light reflected by a cornea and a retina of the eye, and the second infrared reflection light of the second infrared light reflected by the cornea and the retina; wherein the cornea reflects the first infrared reflection light and the second infrared reflection light, and a position of a formed corneal glint remains constant, and the light reflected on the retina identifies a pupil orientation of the eye; and generating the first infrared image of the eye based on the first infrared reflection light and the second infrared reflection light.
The method according to claim 2, wherein the performing, by the electronic device, eye tracking based on the first infrared image comprises: determining, by the electronic device by using a pupil center corneal reflection method, a moving direction of the eye based on a vector between a pupil center and the corneal glint formed by the first infrared reflection light and the second infrared reflection light in the first infrared image at the cornea.
The method according to claim 3, wherein the determining, by the electronic device by using a pupil center corneal reflection method, a moving direction of the eye based on a vector between a pupil center and the corneal glint formed by the first infrared reflection light and the second infrared reflection light in the first infrared image at the cornea comprises: correcting, by the electronic device, a head posture of the user based on 3D depth information provided by the camera module; and determining, by the electronic device by using the pupil center corneal reflection method, the moving direction of the eye based on the vector between the pupil center and the corneal glint formed by the first infrared reflection light and the second infrared reflection light in the first infrared image at the cornea.
The method according to claim 1, wherein the method further comprises: controlling, in a case that the electronic device is in a second service scenario, the second infrared light source to emit the second infrared light to the target object, wherein the second service scenario is a low-precision eye tracking scenario; receiving, by the receiver, a third infrared reflection light of the second infrared light reflected by the target object, to generate a second infrared image of the eye; and performing, by the electronic device, eye tracking based on the second infrared image.
The method according to claim 5, wherein the receiving, by the receiver, a third infrared reflection light of the second infrared light reflected by the target object, to generate a second infrared image of the eye comprises: receiving, by the receiver, a third infrared reflection light of the second infrared light reflected by a face of the user; and generating the second infrared image based on the third infrared reflection light.
The method according to claim 6, wherein the performing, by the electronic device, eye tracking based on the second infrared image comprises: determining, by the electronic device, a key point of an area in which the eye is located in the second infrared image; capturing, by the electronic device, an image area comprising the key from the second infrared image, to obtain an infrared image of the eye; and determining, by the electronic device by using an appearance-based artificial intelligence eye tracking technology, a moving direction of the eye based on an appearance of the face and the eye of the user in the second infrared image and a position of a gaze point of the eye in the infrared image of the eye.
The method according to any one of claims 1 to 7, wherein the first infrared light source and the camera module are disposed below a screen of the electronic device; and the controlling, in a case that the electronic device is in a first service scenario, the first infrared light source to emit a first infrared light to a target object, and controlling the second infrared light source to emit a second infrared light to the target object comprises: controlling, in the case that the electronic device is in the first service scenario, the first infrared light source to emit the first infrared light to the target object at a first intensity, and controlling the second infrared light source to emit the second infrared light to the target object at the first intensity; wherein the first intensity is higher than a second intensity, and the second intensity is a luminous intensity of the first infrared light source and the second infrared light source when the first infrared light source and the camera module are disposed in a through hole provided in the screen.
The method according to any one of claims 1 to 7, wherein a through hole is provided in a screen of the electronic device, and the first infrared light source and the camera module are disposed in the through hole; and the controlling, in a case that the electronic device is in a first service scenario, the first infrared light source to emit a first infrared light to a target object, and controlling the second infrared light source to emit a second infrared light to the target object comprises: controlling, in the case that the electronic device is in the first service scenario, the first infrared light source to emit the first infrared light to the target object at a second intensity, and controlling the second infrared light source to emit the second infrared light to the target object at the second intensity; wherein the second intensity is lower than a first intensity, and the first intensity is a luminous intensity of the first infrared light source and the second infrared light source when the first infrared light source and the camera module are disposed below the screen.
The method according to any one of claims 1 to 7, wherein there is at least one first infrared light source.
The method according to claim 10, wherein a spacing between each first infrared light source and the second infrared light source is greater than or equal to 1 cm, and a distance between any two first infrared light sources is greater than or equal to 1 cm.
An electronic device, wherein the electronic device comprises a memory, a processor, a first infrared light source, and a camera module, the camera module comprises an emitter and a receiver, and the emitter comprises a second infrared light source; the processor is separately coupled to the memory, the first infrared light source, and the camera module; and the memory stores program instructions, and when the program instructions are executed by the processor, the electronic device is enabled to perform the eye tracking method according to any one of claims 1 to 11.
A computer-readable storage medium, comprising a computer program, wherein when the computer program is run on an electronic device, the electronic device is enabled to perform the eye tracking method according to any one of claims 1 to 11.

Description

This application claims priority to Chinese Patent Application No. 202311391303.6, filed with the China National Intellectual Property Administration on October 24, 2023 and entitled "EYE TRACKING METHOD, DEVICE, AND STORAGE MEDIUM", which is incorporated herein by reference in its entirety. TECHNICAL FIELD This application relates to the field of human-computer interaction technologies, and in particular, to an eye tracking method, a device, and a storage medium. BACKGROUND Eye tracking is an application technology that tracks eye movement of a human body by using technologies such as a sensor, a computer, and machine vision. This technology is widely used in fields such as virtual reality (virtual reality, VR), augmented reality (Augmented Reality, AR), medical treatment, education, intelligent driving, and psychological analysis, and is further a current research hotspot in the field of human-computer interaction. Currently, commonly used eye tracking devices include telemetric (screen-type) and head-mounted devices. For the former type of devices, a strip-shaped collection and computing device (including a plurality of infrared light sources and infrared cameras, and the like) needs to be placed (externally placed) above or below an electronic device. For the latter type of devices, a series of hardware is integrated into glasses. Although the two types of devices can better implement an eye tracking function, the devices are usually expensive and inconvenient to carry. In addition, if the telemetric device is integrated into an electronic device such as a mobile phone or a tablet computer, not only hardware power consumption is increased, but also a quantity of holes in a screen is increased. SUMMARY To solve the foregoing technical problem, this application provides an eye tracking method, a device, and a storage medium, which aims to control different infrared light sources to operate based on an actual service scenario of an electronic device by using a camera module that is originally integrated in the electronic device and that can emit an infrared light and then adding at least one additional infrared light source, so as to obtain different infrared images, so that hardware power consumption is considered, and eye tracking with different precision can be implemented. According to a first aspect, this application provides an eye tracking method, applied to an electronic device. The electronic device includes a first infrared light source and a camera module. The camera module includes an emitter and a receiver. The emitter includes a second infrared light source. The eye tracking method includes: determining, by the electronic device, a service scenario in which the electronic device is currently located; controlling, in a case that the electronic device is in a first service scenario, the first infrared light source to emit a first infrared light to a target object, and controlling the second infrared light source to emit a second infrared light to the target object, where the target object includes an eye of a user, and the first service scenario is a high-precision eye tracking scenario; receiving, by the receiver, a first infrared reflection light of the first infrared light reflected by the target object, and a second infrared reflection light of the second infrared light reflected by the target object, to generate a first infrared image of the eye; and performing, by the electronic device, eye tracking based on the first infrared image. A quantity of first infrared light sources may be 1 or more than 1. A quantity of second infrared light sources is 1. The electronic device is a portable device such as a mobile phone or a tablet computer. The camera module is built-in hardware of the electronic device, and the first infrared light source is an additionally added infrared light source. In addition to the second infrared light source, the emitter in the camera module may further include a drive control unit that drives the second infrared light source to operate. The receiver in the camera module may be considered as an image sensor, and may not only receive an infrared reflection light reflected by the target object, but also generate an infrared image of a current object in a current scenario based on the received infrared reflection light. The high-precision eye tracking scenario is a scenario in which a requirement on precision of eye tracking is high, but a power consumption limitation is not high (an eye tracking function does not need to be used for a long time), for example, an eye tracking scenario such as determining a gazed application icon and eye tracking focusing. In this scenario, an infrared camera needs to cooperate with a plurality of infrared light sources to collect an infrared image, and then eye tracking is implemented based on a pupil center corneal reflection (Pupil Center Corneal Reflection, PCCR) method. An operation of determining the service scenario in which the el