EP-4734834-A1 - OPTICAL SENSORS AND AUDIO DEVICE FOR DETERMINING BIOMETRIC INFORMATION

Abstract

A wearable computing device includes one or more optical sensors, an audio device, one or more memories configured to store one or more instructions, and one or more processors configured to execute the one or more instructions stored in the one or more memories to obtain, via the one or more optical sensors, first heart rate information associated with a user wearing the wearable computing device, obtain, via the audio device, second heart rate information associated with the user, and determine biometric information associated with the user based on the first heart rate information and the second heart rate information.

Inventors

• SHIN, DONGEEK
• HSU, JONATHAN
• PATHAK, ANUPAM J.

Assignees

• Google LLC

Dates

Publication Date

20260506

Application Date

20230628

Claims (20)

1. 1. A wearable computing device, comprising: one or more optical sensors; an audio device; one or more memories configured to store one or more instructions; and one or more processors configured to execute the one or more instructions stored in the one or more memories to: obtain, via the one or more optical sensors, first heart rate information associated with a user wearing the wearable computing device, obtain, via the audio device, second heart rate information associated with the user, and determine biometric information associated with the user based on the first heart rate information and the second heart rate information.
2. 2. The wearable computing device of claim 1, the audio device including a transmitter configured to emit a sound wave and a receiver configured to receive a reflected sound wave that corresponds to a sound wave reflected off a first location on the user.
3. 3. The wearable computing device of claim 2, the transmitter emitting an ultrasonic sound wave.
4. 4. The wearable computing device according to claim 2, the one or more optical sensors including one or more photoplethysmography (PPG) sensors, the one or more PPG sensors including a light source configured to emit a light and a photodetector configured to receive reflected light reflected off a body part of the user.
5. 5. The wearable computing device of claim 4, the one or more processors being configured to execute the one or more instructions stored in the one or more memories to: generate a PPG signal to obtain the first heart rate information based on the reflected light, generate a ballistocardiogram (BCG) signal to obtain the second heart rate information based on the reflected sound wave, and determine the biometric information associated with the user based on a first peak associated with the PPG signal and a second peak associated with the BCG signal.
6. 6. The wearable computing device of claim 5, the biometric information including at least one of a pulse transit time, a blood pressure value, or a heart rate.
7. 7. The wearable computing device of claim 5, the one or more processors being configured to execute the one or more instructions stored in the one or more memories to: implement one or more machine-learned models to determine whether the BCG signal generated based on the reflected sound wave is indicative of an expected BCG signal associated with the first location on the user, when a confidence level associated with an output of the one or more machine- learned models is less than a threshold confidence level, exclude the BCG signal as a measurement associated with the second heart rate information for determining the biometric information, and when the confidence level associated with the output of the one or more machine- learned models is greater than the threshold confidence level, include the BCG signal as a measurement associated with the second heart rate information for determining the biometric information.
8. 8. The wearable computing device of claim 1 , the one or more processors being configured to execute the one or more instructions stored in the one or more memories to: determine a distance between the wearable computing device and a body part associated with measuring the second heart rate information, when the distance exceeds a threshold distance value, disable the audio device from obtaining the second heart rate information, and when the distance does not exceed the threshold distance value, enable the audio device to obtain the second heart rate information.
9. 9. The wearable computing device of claim 8, the one or more processors being configured to execute the one or more instructions stored in the one or more memories to: when the distance exceeds the threshold distance value, disable the one or more optical sensors from obtaining the first heart rate information, and when the distance does not exceed the threshold distance value, enable the one or more optical sensors to obtain the first heart rate information.
10. 10. The wearable computing device of claim 1, the one or more processors being configured to execute the one or more instructions stored in the one or more memories to: determine a distance between the wearable computing device and a body part associated with measuring the second heart rate information, when the distance exceeds a threshold distance value, exclude the measurement associated with the second heart rate information for determining the biometric information, and when the distance does not exceed the threshold distance value, include the measurement associated with the second heart rate information for determining the biometric information.
11. 11. The wearable computing device of claim 10, further comprising a motion sensor, the one or more processors being configured to: determine motion information associated with the wearable computing device via the motion sensor, for a first duration of time in which the motion information associated with the wearable computing device exceeds a threshold motion value while the second heart rate information associated with the user is obtained, exclude the second heart rate information corresponding to the first duration of time as second heart rate information for determining the biometric information, and for a second duration of time in which the motion information associated with the wearable computing device does not exceed the threshold motion value while the second heart rate information associated with the user is obtained, include the second heart rate information corresponding to the second duration of time as second heart rate information for determining the biometric information.
12. 12. The wearable computing device of claim 11, the one or more processors being configured to execute the one or more instructions stored in the one or more memories to: determine whether the motion information associated with the wearable computing device exceeds a threshold motion value, when the motion information exceeds the threshold motion value, disable the audio device from obtaining the second heart rate information, and when the motion information does not exceed the threshold motion value, enable the audio device to obtain the second heart rate information.
13. 13. The wearable computing device of claim 1, the one or more processors being configured to obtain, via the one or more optical sensors, the first heart rate information while the second heart rate information is being obtained via the audio device.
14. 14. The wearable computing device of claim 1, the one or more processors being configured to activate the audio device to obtain the second heart rate information in response to the one or more processors detecting the user is engaged in a predetermined activity.
15. 15. The wearable computing device of claim 14, the one or more processors being configured to detect the predetermined activity of the user sleeping.
16. 16. The wearable computing device of claim 1, the wearable computing device including a smartwatch or a tracker.
17. 17. A computer-implemented method, comprising: obtaining, via one or more optical sensor of a wearable computing device, first heart rate information associated with a user wearing the wearable computing device; obtaining, via an audio device of the wearable computing device, second heart rate information associated with the user; and determining biometric information associated with the user based on the first heart rate information and the second heart rate information.
18. 18. The computer-implemented method of claim 17, the first heart rate information being passively obtained via the one or more optical sensors, and the second heart rate information being passively obtained via the audio device.
19. 19. The computer-implemented method of claim 18, obtaining, via the one or more optical sensors of the w earable computing device, the first heart rate information, includes emitting light and detecting reflected light, obtaining, via the audio device of the wearable computing device, the second heart rate information, includes transmitting an audio signal and receiving an audio reflection, determining the biometric information includes: generating a photoplethysmography (PPG) signal based on the reflected light, generating a ballistocardiogram (BCG) signal based on the audio reflection, and determining an interval of time between a first peak associated with the PPG signal and a second peak associated with the BCG signal to determine a pulse transit time.
20. 20. A non-transitory computer-readable medium which stores instructions that are executable by one or more processors of a computing device, the instructions comprising instructions to cause the one or more processors to: obtain, via one or more optical sensors of a wearable computing device, first heart rate information associated w ith a user wearing the wearable computing device; obtain, via an audio device of the wearable computing device, second heart rate information associated with the user; and determine biometric information associated with the user based on the first heart rate information and the second heart rate information.

Description

OPTICAL SENSORS AND AUDIO DEVICE FOR DETERMINING BIOMETRIC INFORMATION FIELD [0001] The disclosure relates generally to computing devices. More particularly, the disclosure relates to wearable computing devices which are used to obtain and store biometric information of a user. BACKGROUND [0002] Some existing methods for determining a pulse transit time metric or blood pressure value can require separate devices (e.g., a smartwatch and a smartphone) to take separate biometric measurements at different locations of the body. Some existing methods also require active user interaction with biometric devices to actively take biometric measurements. Other methods also obtain biometric information for determining a pulse transit time metric at sporadic times, according to an invocation by a user for example, rather than taking continuous and/or passive biometric measurements. SUMMARY [0004] Aspects and advantages of embodiments of the disclosure will be set forth in part in the following description, or can be learned from the description, or can be learned through practice of the example embodiments. [0005] In an example embodiment, a wearable computing device (e.g., a biometric computing device, a smartwatch, a tracker, wearable jewelry having biometric measurement capabilities, and the like) is provided. The wearable computing device includes one or more memories configured to store one or more instructions; and one or more processors configured to execute the one or more instructions stored in the one or more memories to: obtain, via the one or more optical sensors, first heart rate information associated with a user wearing the wearable computing device, obtain, via the audio device, second heart rate information associated w ith the user, and determine biometric information associated with the user based on the first heart rate information and the second heart rate information [0006] In some implementations, the audio device includes a transmitter configured to emit a sound wave and a receiver configured to receive a reflected sound wave that corresponds to a sound wave reflected off a first location on the user. [0007] In some implementations, the transmitter emits an ultrasonic sound wave. [0008] In some implementations, the one or more optical sensors include one or more photoplethysmography (PPG) sensors, the one or more PPG sensors including a light source configured to emit a light and a photodetector configured to receive reflected light that reflected off a body part of the user. [0009] In some implementations, the one or more processors are configured to execute the one or more instructions stored in the one or more memories to: generate a PPG signal to obtain the first heart rate information based on the reflected light, generate a ballistocardiogram (BCG) signal to obtain the second heart rate information based on the reflected sound wave, and determine the biometric information associated with the user based on a first peak associated with the PPG signal and a second peak associated with the BCG signal. [0010] In some implementations, the biometric information includes at least one of a pulse transit time, a blood pressure value, or a heart rate. [0011] In some implementations, the one or more processors are configured to execute the one or more instructions stored in the one or more memories to: implement one or more machine-learned models to determine whether a ballistocardiogram (BCG) signal generated based on the reflected sound wave is indicative of an expected BCG signal associated with the first location on the user, when a confidence level associated with an output of the one or more machine-learned models is less than a threshold confidence level, exclude the BCG signal as a measurement associated with the second heart rate information for determining the biometric information, and when the confidence level associated with the output of the one or more machine-learned models is greater than the threshold confidence level, include the BCG signal as a measurement associated with the second heart rate information for determining the biometric information. [0012] In some implementations, the one or more processors are configured to execute the one or more instructions stored in the one or more memories to: determine a distance between the wearable computing device and a body part associated with measuring the second heart rate information, when the distance exceeds a threshold distance value, disable the audio device from obtaining the second heart rate information, and when the distance does not exceed the threshold distance value, enable the audio device to obtain the second heart rate information. [0013] In some implementations, the one or more processors are configured to execute the one or more instructions stored in the one or more memories to: when the distance exceeds a threshold distance value, disable the one or more optical sensors from obtaining the first heart rate informatio