US-12616417-B1 - Method, apparatus and system for monitoring health through audio data

Abstract

According to some embodiments, an ear-worn device, e.g., a hearing aid, is provided that operates to monitor the health of a wearer of the ear-worn device based on an audio signal detected by the ear-worn device. In some embodiments, a method of monitoring the health of the wearer of the ear-worn device includes: detecting the audio signal with a microphone of the ear-worn device; processing the detected audio signal using a processor of the ear-worn device; and outputting, from an output signal generator of the ear-worn device, a generated output audio signal. In some embodiments, processing the detected audio signal includes: extracting data indicative of a health event associated with the wearer by processing the detected audio signal with a machine learning model; generating the output audio signal based on the detected audio signal; and outputting the data indicative of the health event.

Inventors

• Nicholas Morris
• Igor LOVCHINSKY
• Andrew J. Casper
• Matthew de Jonge

Assignees

• Fortell Research Inc.

Dates

Publication Date

20260505

Application Date

20230123

Claims (20)

1. 1 . A method of monitoring health of a wearer of an ear-worn device based on an audio signal detected by the ear-worn device, the method comprising: detecting the audio signal with a microphone of the ear-worn device; processing the detected audio signal using a processor of the ear-worn device, the processing comprising: extracting data indicative of a health event associated with the wearer of the ear-worn device by processing the detected audio signal with a machine learning model, wherein the machine learning model is configured to output a duration of the health event and/or a severity of the health event; generating an output audio signal based on the detected audio signal; and outputting the data indicative of the health event associated with the wearer of the ear-worn device; and outputting, from an output signal generator of the ear-worn device, the generated output audio signal.
2. 2 . The method of claim 1 , wherein the ear-worn device is a hearing aid.
3. 3 . The method of claim 1 , wherein the health event comprises coughing, sneezing, wheezing, snoring, chewing, swallowing, drinking, and/or speaking.
4. 4 . The method of claim 1 , wherein processing the detected audio signal with the machine learning model comprises processing the detected audio signal using a neural network.
5. 5 . The method of claim 4 , wherein processing the detected audio signal with the machine learning model comprises processing the detected audio signal with a recurrent neural network or a convolutional neural network.
6. 6 . The method of claim 1 , wherein processing the detected audio signal with the machine learning model comprises: detecting the health event by processing the detected audio signal with a detector machine learning model; and after detecting the health event with the detector machine learning model, characterizing the health event by processing the detected audio signal with a characterization machine learning model, wherein the detector machine learning model is different from the characterization machine learning model.
7. 7 . The method of claim 6 , wherein the characterization machine learning model is configured to output the duration of the health event and/or the severity of the health event.
8. 8 . The method of claim 1 , wherein the extracted data indicative of the health event comprises a portion of the detected audio signal corresponding to the health event, an indication that the health event occurred, and/or a characterization of the health event.
9. 9 . The method of claim 1 , wherein outputting the data indicative of the health event comprises storing the data indicative of the health event in a memory of the ear-worn device.
10. 10 . The method of claim 1 , wherein outputting the data indicative of the health event comprises transmitting, to a computing device different from the ear-worn device, the data indicative of the health event.
11. 11 . The method of claim 10 , wherein outputting the data indicative of the health event comprises transmitting the data indicative of the health event using a wireless communication protocol.
12. 12 . The method of claim 1 , wherein generating the output audio signal based on the detected audio signal comprises removing background noise from the detected audio signal.
13. 13 . The method of claim 12 , wherein removing the background noise from the detected audio signal comprises generating a processed audio signal, and wherein processing the detected audio signal with the machine learning model comprises processing the processed audio signal with the machine learning model.
14. 14 . The method of claim 12 , wherein generating the output audio signal based on the detected audio signal further comprises, after removing the background noise from the detected audio signal, amplifying at least one component of the detected audio signal.
15. 15 . The method of claim 1 , wherein generating the output audio signal based on the detected audio signal comprises estimating a signal-to-noise ratio (SNR) of the detected audio signal, and wherein extracting the data indicative of the health event associated with the wearer of the ear-worn device comprises processing the detected audio signal with the machine learning model only when the estimated SNR is within a specified range.
16. 16 . The method of claim 1 , wherein processing the detected audio signal with the machine learning model comprises processing the detected audio signal with a first machine learning model, and wherein generating the output audio signal based on the detected audio signal comprises: isolating a component of the detected audio signal representing a voice of the wearer from among temporally overlapping voice components from multiple speakers by processing the detected audio signal with a second machine learning model using a voice signature of the wearer.
17. 17 . The method of claim 16 , wherein processing the detected audio signal with the first machine learning model comprises processing only the isolated component of the detected audio signal with the first machine learning model.
18. 18 . The method of claim 1 , wherein the machine learning model comprises a first machine learning model, and wherein generating the output audio signal comprises processing the detected audio signal with a second machine learning model.
19. 19 . The method of claim 18 , wherein processing the detected audio signal with the second machine learning model comprises processing the detected audio signal with a neural network.
20. 20 . The method of claim 1 , wherein processing the detected audio signal using the processor of the ear-worn device further comprises computing a Fast Fourier Transform (FFT) of the detected audio signal.

Description

CROSS-REFERENCE TO RELATED APPLICATIONS This application claims the benefit under 35 U.S.C. § 119(e) of U.S. Provisional Application Ser. No. 63/302,474, filed Jan. 24, 2022, and entitled “METHOD, APPARATUS AND SYSTEM FOR MONITORING HEALTH THROUGH AUDIO DATA”, which is hereby incorporated by reference herein in its entirety. BACKGROUND The present application relates to ear-worn devices, such as hearing aids. Hearing aids are used to help those who have trouble hearing to hear better. They are typically positioned in, or near, the ear. BRIEF SUMMARY According to some embodiments, a method of monitoring health of a wearer of an ear-worn device based on an audio signal detected by the ear-worn device is provided. The method comprises: detecting the audio signal with a microphone of the ear-worn device; processing the detected audio signal using a processor of the ear-worn device, the processing comprising: extracting data indicative of a health event associated with the wearer of the ear-worn device by processing the detected audio signal with a machine learning model; generating an output audio signal based on the detected audio signal; and outputting the data indicative of the health event associated with the wearer of the ear-worn device; and outputting, from an output signal generator of the ear-worn device, the generated output audio signal. According to some embodiments, an ear-worn device is provided. The ear-worn device is configured to monitor health of a wearer of the ear-worn device based on an audio signal detected by the ear-worn device. The ear-worn device comprises: a microphone configured to detect the audio signal; a processor; at least one non-transitory computer-readable storage medium storing processor-executable instructions that, when executed by the processor, cause the processor to perform a method comprising: extracting data indicative of a health event associated with the wearer of the ear-worn device by processing the detected audio signal with a machine learning model; generating an output audio signal based on the detected audio signal; and outputting the data indicative of the health event associated with the wearer of the ear-worn device; and an output signal generator configured to output the generated output audio signal. According to some embodiments, at least one non-transitory computer-readable storage medium storing processor-executable instructions that, when executed by a processor, cause the processor to perform a method of monitoring health of a wearer of an ear-worn device based on an audio signal detected by the ear-worn device is provided. The method comprises: extracting data indicative of a health event associated with the wearer of the ear-worn device by processing the detected audio signal with a machine learning model to extract data indicative of a health event associated with the wearer of the ear-worn device; generating an output audio signal based on the detected audio signal; outputting the data indicative of the health event associated with the wearer of the ear-worn device; and transmitting the output audio signal to an output signal generator of the ear-worn device. BRIEF DESCRIPTION OF DRAWINGS Various aspects of the application will be described with reference to the following figures. It should be appreciated that the figures are not necessarily drawn to scale. Items appearing in multiple figures are indicated by the same reference number in all the figures in which they appear. FIG. 1 illustrates detection of a health event using an ear-worn device, according to a non-limiting embodiment of the present application. FIG. 2 illustrates a system with an ear-worn device and a portable electronic device for monitoring the health of a wearer of an ear-worn device, according to a non-limiting embodiment of the present application. FIG. 3A illustrates example components of an ear-worn device that may be configured to detect a health event of a wearer of the ear-worn device, according to a non-limiting embodiment of the present application. FIG. 3B illustrates example components of a variation of the ear-worn device of FIG. 3A that may be configured to detect a heath event of a wearer of the ear-worn device, according to a non-limiting embodiment of the present application. FIG. 3C illustrates example components of a variation of the ear-worn device of FIG. 3A that may be configured to detect a health event of a wearer of the ear-worn device, according to a non-limiting embodiment of the present application. FIG. 4 illustrates example components of an ear-worn device having two microphones, according to a non-limiting embodiment of the present application. FIG. 5A illustrates an example circuit including a health event detector network and a health event characterization network implemented on an ear-worn device, according to a non-limiting embodiment of the present application. FIG. 5B illustrates a variation of the example circuit of FIG. 5A that may inclu