US-12621616-B2 - Method of operating a hearing aid system and a hearing aid system using speech forecasting

Abstract

A method of operating a hearing aid system in order to provide at least one of improved noise reduction and speech intelligibility and a hearing aid system adapted to carry out the method.

Inventors

• Rasmus Malik Hoeegh LINDRUP
• Jens Brehm Bagger NIELSEN
• Asger OUGAARD
• Robert Scholes LYCK

Assignees

• WIDEX A/S

Dates

Publication Date

20260505

Application Date

20220603

Priority Date

20210604

Claims (10)

1. 1 . A method of operating a hearing aid system comprising the steps of: providing an audio signal; providing, using a forecasting neural network architecture comprising a neural network encoder, a forecasting model, and a neural network decoder, a predicted audio signal based on said audio signal and processing said predicted audio signal in order to alleviate a hearing deficit of an individual wearing the hearing aid system and hereby providing a predicted hearing loss compensated electrical output signal, or processing said audio signal in order to alleviate a hearing deficit of an individual wearing the hearing aid system and hereby providing a hearing loss compensated signal and using the forecasting neural network architecture to provide a predicted hearing loss compensated electrical output signal based on said hearing loss compensated signal, wherein the forecasting model is configured to predict future latent encodings associated with said audio signal, the future latent encodings being a pre-determined number of time steps ahead of a current time; and using an electrical-acoustical output transducer of the hearing aid system to generate a delay compensated acoustical output based on the predicted hearing loss compensated electrical output signal.
2. 2 . The method according to claim 1 , wherein said step of using the forecasting neural network architecture comprises the further steps of: passing a frame of the audio signal or of the hearing loss compensated signal through the neural network encoder and hereby obtaining a latent encoding; using the forecasting model to provide a prediction of a future latent encoding based at least on said latent encoding; passing said predicted future latent encoding through the neural network decoder and hereby providing said predicted audio signal or said predicted hearing loss compensated electrical output signal.
3. 3 . The method according to claim 1 , wherein said audio signal is derived from at least one acoustical-electrical input transducer accommodated in the hearing aid system.
4. 4 . The method according to claim 1 , wherein said neural network encoder and said neural network decoder are both part of a variational autoencoder.
5. 5 . The method according to claim 1 , comprising the further step of using a probabilistic forecasting model to provide an estimate of the uncertainty of the forecasting.
6. 6 . The method according to claim 2 , comprising the further steps of: using a probabilistic forecasting model to provide an estimate of the uncertainty of the forecasting; and only using said predicted future latent encoding if the estimated uncertainty is below a given threshold.
7. 7 . The method according to claim 1 , wherein the step of using the forecasting neural network architecture comprises the steps of: detecting speech in the audio signal; using a language model to provide an input to the forecasting model, wherein said input comprises information representing prediction of sentence continuations.
8. 8 . A hearing aid system comprising at least one acoustical-electrical input transducer; a forecasting neural network architecture; a hearing loss compensation processor, and an electrical-acoustical output transducer, and adapted to carry out any one of the methods according to claim 1 .
9. 9 . An internet server comprising a downloadable application that may be executed by a computing device, wherein the downloadable application is adapted to cause the steps of using the forecasting neural network architecture according to claim 1 to be carried out.
10. 10 . A non-transitory computer readable medium carrying instructions which, when executed by a computing device carries out the method steps of claim 1 .

Description

CROSS REFERENCE TO RELATED APPLICATIONS This application is a National Stage of International Application No. PCT/EP2022/065172 filed Jun. 3, 2022, claiming priority based on Denmark Patent Application No. PA202100598 filed Jun. 4, 2021. The present invention relates to a method of operating a hearing aid system. The present invention also relates to a hearing aid system adapted to carry out said method. BACKGROUND OF THE INVENTION An audio device system may comprise one or two audio devices. In this application, an audio device should be understood as a small, battery-powered, microelectronic device designed to be worn in or at an ear of a user. The audio device generally comprises an energy source such as a battery or a fuel cell, at least one microphone, a microelectronic circuit comprising a digital signal processor, and an acoustic output transducer. The audio device is enclosed in a casing suitable for fitting in or at (such as behind) a human ear. If the audio device furthermore is capable of amplifying an ambient sound signal in order to alleviate a hearing deficit the audio device may be considered a personal sound amplification product or a hearing aid. According to variations the mechanical design of an audio device may resemble those of hearing aids and as such traditional hearing aid terminology may be used to describe various mechanical implementations of audio devices that are not hearing aids. As the name suggests, Behind-The-Ear (BTE) hearing aids are worn behind the ear. To be more precise, an electronics unit comprising a housing containing the major electronics parts thereof is worn behind the ear. An earpiece for emitting sound to the hearing aid user is worn in the ear, e.g. in the concha or the ear canal. In a traditional BTE hearing aid, a sound tube is used to convey sound from the output transducer, which in hearing aid terminology is normally referred to as the receiver, located in the housing of the electronics unit and to the ear canal. In more recent types of hearing aids, a conducting member comprising electrical conductors conveys an electric signal from the housing and to a receiver placed in the earpiece in the ear. Such hearing aids are commonly referred to as Receiver-In-The-Ear (RITE) hearing aids. In a specific type of RITE hearing aids the receiver is placed inside the ear canal. This category is sometimes referred to as Receiver-In-Canal (RIC) hearing aids. In-The-Ear (ITE) hearing aids are designed for arrangement in the ear, normally in the funnel-shaped outer part of the ear canal. In a specific type of ITE hearing aids the hearing aid is placed substantially inside the ear canal. This category is sometimes referred to as Completely-In-Canal (CIC) hearing aids or Invisible-In-Canal (IIC). This type of hearing aid requires an especially compact design in order to allow it to be arranged in the ear canal, while accommodating the components necessary for operation of the hearing aid. Generally, a hearing aid system according to the invention is understood as meaning any device which provides an output signal that can be perceived as an acoustic signal by a user or contributes to providing such an output signal, and which has means which are customized to compensate for an individual hearing loss of the user or contribute to compensating for the hearing loss of the user. Within the present context an audio device system may comprise a single audio device (a so called monaural audio device system) or comprise two audio devices, one for each ear of the user (a so called binaural audio device system). Furthermore, the audio device system may comprise at least one additional device (which in the following may also be denoted an external device despite that it is part of the audio device system), such as a smart phone or some other computing device having software applications adapted to interact with other devices of the audio device system. However, the audio device system may also include a remote microphone system (which generally can also be considered a computing device) comprising additional microphones and/or may even include a remote server providing abundant processing resources and generally these additional devices will also include link means adapted to operationally connect to the various other devices of the audio device system. Despite the advantages that contemporary audio device—and especially hearing aid—systems provide, some users may still experience hearing situations that are difficult or maybe even uncomfortable or confusing. A critical element when seeking to alleviate such difficulties is the ability to improve the sound quality. It is therefore a feature of the present invention to provide a method of operating a hearing aid system that provides improved noise reduction. It is another feature of the present invention to provide a hearing aid system adapted to provide such a method of operating a hearing aid system. It is yet another feature of the