EP-4736746-A1 - ELECTRONIC DEVICE AND METHOD FOR CONTROLLING ELECTRONIC DEVICE

Abstract

An electronic device is provided. The electronic device includes a microphone, memory storing one or more computer programs, and one or more processors communicatively coupled to the microphone, and the memory, wherein the one or more computer programs include computer-executable instructions that, when executed by the one or more processors individually or collectively, cause the electronic device to store registration information on breathing sounds of a plurality of users in the memory, based on receiving an audio signal through the microphone, obtain information on a breathing sound of a user based on the audio signal, compare the information on the breathing sound with the registration information, identify at least one user corresponding to the information on the breathing sound among the plurality of users, and based on identifying the at least one user, obtain an analysis result for the sleeping states of each of the at least one user based on information corresponding to each of the at least one user in the information on the breathing sound.

Inventors

• MATUSZEWSKI, MATEUSZ
• PLES, Anna
• SIKORSKI, Olaf
• GORZYNSKI, Kamil
• TKACZUK, Jakub

Assignees

• Samsung Electronics Co., Ltd.

Dates

Publication Date

20260506

Application Date

20241226

Claims (15)

1. An electronic device comprising: a microphone; memory storing one or more computer programs; and one or more processors communicatively coupled to the microphone, and the memory, wherein the one or more computer programs include computer-executable instructions that, when executed by the one or more processors individually or collectively, cause the electronic device to: store registration information on breathing sounds of a plurality of users in the memory, based on receiving an audio signal through the microphone, obtain information on a breathing sound of a user based on the audio signal, compare the information on the breathing sound with the registration information, identify at least one user corresponding to the information on the breathing sound among the plurality of users, and based on identifying the at least one user, obtain an analysis result for sleeping states of each of the at least one user based on information corresponding to each of the at least one user in the information on the breathing sound.
2. The electronic device of claim 1, wherein the one or more computer programs further include computer-executable instructions that, when executed by the one or more processors individually or collectively, cause the electronic device to: based on receiving the audio signal, identify a plurality of segments corresponding to the breathing sound of the user in the audio signal, obtain a plurality of first embedding vectors corresponding to each of the plurality of segments, and based on comparing each of the plurality of first embedding vectors with a plurality of second embedding vectors corresponding to the registration information, identify the at least one user.
3. The electronic device of claim 2, wherein the one or more computer programs further include computer-executable instructions that, when executed by the one or more processors individually or collectively, cause the electronic device to: based on obtaining the plurality of first embedding vectors, identify distances between locations of each of the plurality of first embedding vectors and a centroid location of embedding vectors corresponding to a first user among the plurality of second embedding vectors in a latent space, and based on the identified distances being smaller than a predetermined threshold distance, identify the first user as the at least one user.
4. The electronic device of claim 2, wherein the one or more computer programs further include computer-executable instructions that, when executed by the one or more processor individually or collectively, cause the electronic device to: obtain information on the plurality of segments by inputting the audio signal into a first neural network model trained to distinguish a breathing sound of a user included in an audio signal, obtain the plurality of first embedding vectors by inputting the plurality of segments into a second neural network model trained to convert an input segment into an embedding vector, and obtain the analysis result by inputting the information corresponding to each of the at least one user into a third neural network model trained to identify a sleeping state of a user corresponding to a breathing sound.
5. The electronic device of claim 1, wherein the analysis result comprises at least one of information on whether the at least one user is sleeping, information on sleep quality of the at least one user, or information on health of the at least one user.
6. The electronic device of claim 1, further comprising: a transceiver, wherein the one or more computer programs further include computer-executable instructions that, when executed by the one or more processors individually or collectively, cause the electronic device to: based on the analysis result indicating that the at least one user is sleeping, obtain a control signal for controlling an external device, and transmit the control signal to the external device.
7. The electronic device of claim 1, wherein the one or more computer programs further include computer-executable instructions that, when executed by the one or more processors individually or collectively, cause the electronic device to: based on identifying the at least one user, update the registration information based on the information on the breathing sound.
8. The electronic device of claim 1, further comprising: a display, wherein the one or more computer programs further include computer-executable instructions that, when executed by the one or more processors individually or collectively, cause the electronic device to: based on the at least one user not being identified, display a user interface, and based on receiving a user input for registering the information on the breathing sound through the user interface, add the information on the breathing sound to the registration information.
9. The electronic device of claim 1, further comprising: a sensor, wherein the one or more computer programs further include computer-executable instructions that, when executed by the one or more processors individually or collectively, cause the electronic device to: based on identifying the at least one user, obtain biometric information of the at least one user through the sensor, and obtain the analysis result for the sleeping states of each of the at least one user based on the information corresponding to each of the at least one user and the biometric information.
10. A method performed by an electronic device, the method comprising: storing registration information on breathing sounds of a plurality of users; based on receiving an audio signal, obtaining information on a breathing sound of a user based on the audio signal; comparing the information on the breathing sound with the registration information on breathing sounds of a plurality of users; identifying at least one user corresponding to the information on the breathing sound among the plurality of users; and based on identifying the at least one user, obtaining an analysis result for sleeping states of each of the at least one user based on information corresponding to each of the at least one user in the information on the breathing sound.
11. The method of claim 10, wherein the obtaining of the information on the breathing sound comprises: based on receiving the audio signal, identifying a plurality of segments corresponding to the breathing sound of the user in the audio signal, and obtaining a plurality of first embedding vectors corresponding to each of the plurality of segments, and wherein the identifying of the at least one user comprises: based on comparing each of the plurality of first embedding vectors with a plurality of second embedding vectors corresponding to the registration information, identifying the at least one user.
12. The method of claim 11, wherein the identifying of the at least one user further comprises: based on obtaining the plurality of first embedding vectors, identifying distances between locations of each of the plurality of first embedding vectors and a centroid location of embedding vectors corresponding to a first user among the plurality of second embedding vectors in a latent space; and based on the identified distances being smaller than a predetermined threshold distance, identifying the first user as the at least one user.
13. The method of claim 11, wherein the obtaining of the information on the breathing sound comprises: obtaining information on the plurality of segments by inputting the audio signal into a first neural network model trained to distinguish a breathing sound of a user included in an audio signal, and obtaining the plurality of first embedding vectors by inputting the plurality of segments into a second neural network model trained to convert an input segment into an embedding vector, and wherein the obtaining of the analysis result comprises: obtaining the analysis result by inputting the information corresponding to each of the at least one user into a third neural network model trained to identify a sleeping state of a user corresponding to a breathing sound.
14. The method of claim 10, wherein the analysis result comprises at least one of information on whether the at least one user is sleeping, information on sleep quality of the at least one user, or information on health of the at least one user.
15. The method of claim 10, further comprising: based on the analysis result indicating that the at least one user is sleeping, obtaining a control signal for controlling an external device; and transmitting the control signal to the external device.

Description

[Technical Field] The disclosure relates to an electronic device and a controlling method of the electronic device. More particularly, the disclosure relates to an electronic device that can analyze a breathing sound of a user, and a controlling method thereof. [Background Art] Recently, along with development of fields such as wearable devices, artificial intelligence, the Internet of Things (IoT), etc., development of a technology enabling effective management of a user's health by analyzing a sleeping state of the user is accelerating. In particular, a technology of analyzing a sleeping state of a user by analyzing a breathing sound that is generated according to breathing during the user's sleep has advantages which are that it can provide information on the sleeping state of the user without interfering with the user's sleep by a non-contact method, and does not need an additional sensor of a high price, etc. However, regarding such a conventional technology, a limitation is being pointed out, which is that it is difficult to clearly distinguish breathing sounds of a plurality of users in case the plurality of users are sleeping together in a space, and accordingly, it is also difficult to correctly analyze the sleeping states of each of the plurality of users. The above information is presented as background information only to assist with an understanding of the disclosure. No determination has been made, and no assertion is made, as to whether any of the above might be applicable as prior art with regard to the disclosure. [Disclosure] [Technical Solution] Aspects of the disclosure are to address at least the above-mentioned problems and/or disadvantages and to provide at least the advantages described below. Accordingly, an aspect of the disclosure is to provide an electronic device that can clearly distinguish breathing sounds of a plurality of users, and correctly analyze the sleeping states of each of the plurality of users, and a controlling method thereof. Additional aspects will be set forth in part in the description which follows and, in part, will be apparent from the description, or may be learned by practice of the presented embodiments. In accordance with an aspect of the disclosure, an electronic device is provided. The electronic device includes a microphone, memory storing one or more computer programs, and one or more processors communicatively coupled to the microphone, and the memory, wherein the one or more computer programs include computer-executable instructions that, when executed by the one or more processors individually or collectively, cause the electronic device to store registration information on breathing sounds of a plurality of users in the memory, based on receiving an audio signal through the microphone, obtain information on a breathing sound of a user based on the audio signal, compare the information on the breathing sound with the registration information, identify at least one user corresponding to the information on the breathing sound among the plurality of users, and based on identifying the at least one user, obtain an analysis result for the sleeping states of each of the at least one user based on information corresponding to each of the at least one user in the information on the breathing sound. Meanwhile, the one or more processors, based on receiving the audio signal, identify a plurality of segments corresponding to the breathing sound of the user in the audio signal, obtain a plurality of first embedding vectors corresponding to each of the plurality of segments, and based on comparing each of the plurality of first embedding vectors with a plurality of second embedding vectors corresponding to the registration information, identify the at least one user. Meanwhile, the one or more processors, based on obtaining the plurality of first embedding vectors, identify distances between the locations of each of the plurality of first embedding vectors and the centroid location of embedding vectors corresponding to a first user among the plurality of second embedding vectors in a latent space, and based on the identified distances being smaller than a predetermined threshold distance, identify the first user as the at least one user. Meanwhile, the one or more processors obtain information on the plurality of segments by inputting the audio signal into a first neural network model trained to distinguish a breathing sound of a user included in an audio signal, obtain the plurality of first embedding vectors by inputting the plurality of segments into a second neural network model trained to convert an input segment into an embedding vector, and obtain the analysis result by inputting the information corresponding to each of the at least one user into a third neural network model trained to identify a sleeping state of a user corresponding to a breathing sound. Meanwhile, the analysis result includes at least one of information on whether the at least one u