KR-20260067936-A - ELECTRONIC DEVICE FOR PROVIDING INTERPRETATION FUNCTION, OPERATING METHOD THEREOF, AND STORAGE MEDIUM

KR20260067936AKR 20260067936 AKR20260067936 AKR 20260067936AKR-20260067936-A

Abstract

According to one embodiment, the electronic device may include at least one first microphone, a communication circuit configured to support a short-range communication method, at least one processor, and a memory for storing instructions. According to one embodiment, the instructions may be configured such that, when executed individually or collectively by the at least one processor, the electronic device receives a first voice signal corresponding to a self-utterance from a first wearable electronic device connected to the electronic device through the communication circuit while the interpretation application is executed on the electronic device, the self-utterance is obtained through at least one second microphone of the first wearable electronic device, and in response to the reception of the first voice signal, outputs first interpretation information for the first voice signal using the interpretation application, and while the interpretation application is executed on the electronic device, receives a second voice signal corresponding to a counterparty's utterance through the at least one first microphone, in response to the reception of the second voice signal, obtains second interpretation information for the second voice signal using the interpretation application, and transmits the second interpretation information for the second voice signal to the first wearable electronic device through the communication circuit.

Inventors

김필원
안미경
이니나
이승룡
임연욱

Assignees

삼성전자주식회사

Dates

Publication Date: 20260513
Application Date: 20241204
Priority Date: 20241106

Claims (20)

In the electronic device (101, 201), At least one first microphone (350); A communication circuit (390) configured to support a short-range communication method; At least one processor (320); and It includes a memory (330) for storing instructions, When the above instructions are executed individually or collectively by the at least one processor, the electronic device, While an interpretation application is running on the electronic device, a first voice signal corresponding to a self-utterance is received from a first wearable electronic device (211) connected to the electronic device through the communication circuit, and the self-utterance is obtained through at least one second microphone (351) of the first wearable electronic device. In response to the reception of the first voice signal, first interpretation information for the first voice signal using the interpretation application is output, and While the interpretation application is running on the electronic device, a second voice signal corresponding to the other party's speech is received through the at least one first microphone, and In response to the reception of the second voice signal, second interpretation information for the second voice signal is obtained using the interpretation application, and An electronic device configured to transmit second interpretation information for the second voice signal to the first wearable electronic device through the communication circuit.
In paragraph 1, when the instructions are executed individually or collectively by the at least one processor, the electronic device, An electronic device configured to acquire second interpretation information for the second voice signal in response to receiving the second voice signal through the at least one first microphone while the first voice signal is not received from the first wearable electronic device.
In paragraph 1 or 2, at least one sensor (376) is further included, When the above instructions are executed individually or collectively by the at least one processor, the electronic device, Based on sensor information detected through the above at least one sensor, determining whether the state of the electronic device is a first state in which the electronic device is not exposed to the outside, and An electronic device configured to turn off at least one first microphone while receiving the first voice signal from the first wearable electronic device in response to confirming that the state of the electronic device is the first state.
In any one of paragraphs 1 to 3, at least one first speaker (335) and a display (360) are further included, When the above instructions are executed individually or collectively by the at least one processor, the electronic device, An electronic device configured to output first interpretation information for the first voice signal through the at least one first speaker or the display in response to confirming that the state of the electronic device is not the first state.
In any one of claims 1 to 4, when the instructions are executed individually or collectively by the at least one processor, the electronic device, An electronic device configured to output second interpretation information for the second voice signal through the at least one first speaker or the display in response to confirming that the state of the electronic device is not the first state.
In any one of claims 1 to 5, when the instructions are executed individually or collectively by the at least one processor, the electronic device, Checking whether the second wearable electronic device (212) connected to the electronic device through the communication circuit is being worn by the user, An electronic device configured to receive, through the communication circuit, a third voice signal corresponding to the other party's speech obtained through at least one third microphone (352) of the second wearable electronic device in response to confirming that the second wearable electronic device is being worn by the user.
In any one of claims 1 to 6, when the instructions are executed individually or collectively by the at least one processor, the electronic device, In response to confirming that the second wearable electronic device is connected to the electronic device and the state of the electronic device is the first state while the first voice signal is not received from the first wearable electronic device, the at least one first microphone is turned off, A third voice signal corresponding to the other party's speech obtained through at least one third microphone (352) of the second wearable electronic device is received through the communication circuit, and Obtaining third interpretation information for the third voice signal using the above interpretation application, and An electronic device configured to output third interpretation information for the above third voice signal through at least one first speaker or the display.
In any one of claims 1 to 7, when the instructions are executed individually or collectively by the at least one processor, the electronic device, An electronic device configured to transmit to the second wearable electronic device, in response to confirming that the second wearable electronic device is being worn by the user and that the state of the electronic device is the first state, third interpretation information for the third voice signal is output through at least one third speaker (357) of the second wearable electronic device.
In any one of claims 1 to 8, when the instructions are executed individually or collectively by the at least one processor, the electronic device, An electronic device configured to output information that induces the user to speak the first interpretation information through at least one second speaker (356) of the first wearable electronic device in response to confirming that the second wearable electronic device is not being worn by the user and that the state of the electronic device is the first state.
In any one of claims 1 to 9, when the instructions are executed individually or collectively by the at least one processor, the electronic device, The second voice signal received through the above at least one first microphone is analyzed using an AI (artificial intelligence) model, and Based on the above analysis results, it is confirmed that the second voice signal is a voice signal corresponding to the counterpart's speech, and An electronic device configured to acquire second interpretation information for the second voice signal using the interpretation application in response to confirming that the second voice signal is a voice signal corresponding to the other party's speech.
In a method for providing an interpretation function in an electronic device, While an interpretation application is running on the electronic device, the operation of receiving a first voice signal corresponding to a self-utterance from a first wearable electronic device connected to the electronic device; the self-utterance is acquired through at least one second microphone of the first wearable electronic device, and An operation of outputting first interpretation information for the first voice signal using the interpretation application in response to the reception of the first voice signal; While the interpretation application is running on the electronic device, the operation of receiving a second voice signal corresponding to the other party's speech through at least one first microphone of the electronic device; An operation of obtaining second interpretation information for the second voice signal using the interpretation application in response to the reception of the second voice signal; and A method for providing an interpretation function, comprising the operation of transmitting second interpretation information for the second voice signal to the first wearable electronic device.
In paragraph 11, the operation of obtaining second interpretation information for the second voice signal is, A method for providing an interpretation function, comprising the operation of obtaining second interpretation information for the second voice signal in response to receiving the second voice signal through the at least one first microphone while the first voice signal is not received from the first wearable electronic device.
In claim 11 or 12, an operation of determining whether the state of the electronic device is a first state in which the electronic device is not exposed to the outside, based on sensor information detected through at least one sensor of the electronic device; and A method for providing an interpretation function, further comprising the operation of turning off the at least one first microphone while receiving the first voice signal from the first wearable electronic device in response to confirming that the state of the electronic device is the first state.
In any one of paragraphs 11 to 13, the operation of outputting first interpretation information for the first voice signal is, A method for providing an interpretation function, comprising the operation of outputting first interpretation information for the first voice signal through at least one first speaker of the electronic device or a display of the electronic device in response to confirming that the state of the electronic device is not the first state.
A method for providing an interpretation function, further comprising, in any one of claims 11 to 14, an operation of outputting second interpretation information for the second voice signal through at least one first speaker of the electronic device or a display of the electronic device in response to confirming that the state of the electronic device is not the first state.
In any one of paragraphs 11 through 15, An operation to check whether a second wearable electronic device connected to the above electronic device is being worn by a user; and A method for providing an interpretation function, further comprising the operation of receiving a third voice signal corresponding to the other party's speech obtained through at least one third microphone of the second wearable electronic device in response to confirming that the second wearable electronic device is being worn by the user.
In any one of paragraphs 11 through 16, An operation to turn off the at least one first microphone in response to confirming that the second wearable electronic device is connected to the electronic device and the state of the electronic device is the first state while the first voice signal is not received from the first wearable electronic device; The operation of receiving a third voice signal corresponding to the opponent's speech obtained through at least one third microphone of the second wearable electronic device; The operation of obtaining third interpretation information for the third voice signal using the above interpretation application; and A method for providing an interpretation function, further comprising the operation of outputting third interpretation information for the third voice signal through at least one first speaker of the electronic device or a display of the electronic device.
In any one of paragraphs 11 through 17, A method for providing an interpretation function, further comprising the operation of transmitting to the second wearable electronic device to output third interpretation information for the third voice signal through at least one third speaker of the second wearable electronic device in response to confirming that the second wearable electronic device is being worn by the user and that the state of the electronic device is the first state.
In any one of paragraphs 11 through 18, An electronic device configured to output information that induces the user to speak the first interpretation information through at least one second speaker of the first wearable electronic device in response to confirming that the second wearable electronic device is not being worn by the user and that the state of the electronic device is the first state.
In a non-transient storage medium storing computer-readable instructions, said instructions are configured to cause said electronic device to perform at least one operation when executed by at least one processor (120, 320) of an electronic device (101, 201), said at least one operation, said at least one operation is, While an interpretation application is running on the electronic device, the operation of receiving a first voice signal corresponding to a self-utterance from a first wearable electronic device connected to the electronic device; the self-utterance is acquired through at least one second microphone of the first wearable electronic device, and An operation of outputting first interpretation information for the first voice signal using the interpretation application in response to the reception of the first voice signal; While the interpretation application is running on the electronic device, the operation of receiving a second voice signal corresponding to the other party's speech through at least one first microphone of the electronic device; An operation of obtaining second interpretation information for the second voice signal using the interpretation application in response to the reception of the second voice signal; and A storage medium comprising the operation of transmitting second interpretation information for the second voice signal to the first wearable electronic device.

Description

ELECTRONIC DEVICE FOR PROVIDING INTERPRETATION FUNCTION, OPERATING METHOD THEREOF, AND STORAGE MEDIUM One embodiment disclosed in this document relates to an electronic device for providing an interpretation function, a method of operation thereof, and a storage medium. Various services and additional functions provided through user terminals, such as electronic devices like smartphones, are gradually increasing. In order to enhance the utility value of these electronic devices and satisfy the needs of various users, telecommunications service providers or electronic device manufacturers are competitively developing electronic devices that provide various functions. As artificial intelligence technology advances, users input voice into electronic devices, and the devices perform actions based on that voice input. A voice recognition service is a service that provides various content services to the user in response to the user's voice received by the electronic device. To provide voice recognition services, technologies that recognize and analyze human language (e.g., machine translation, automatic speech recognition, question answering, and/or speech synthesis) are implemented in electronic devices; for example, the electronic device can provide a real-time interpretation function. FIG. 1 is a block diagram of an electronic device in a network environment according to one embodiment. FIG. 2 is a configuration diagram of a system for providing an interpretation function according to one embodiment. FIG. 3 is an internal block diagram of each of an electronic device, a first wearable electronic device, and a second wearable electronic device for providing an interpretation function according to one embodiment. FIG. 4 is an operation flowchart of an electronic device for providing an interpretation function according to one embodiment. FIG. 5 is a diagram showing signals transmitted and received to provide an interpretation function between an electronic device according to one embodiment, a first wearable electronic device, and a second wearable electronic device. FIG. 6 is an operation flowchart of a first wearable electronic device according to one embodiment. FIG. 7 is a diagram illustrating microphone optimization based on the other party's speech according to one embodiment. FIG. 8 is a diagram illustrating a method for outputting interpretation information according to one embodiment. FIG. 9 is an example diagram illustrating a method for outputting interpretation information using a first wearable electronic device in a state where the electronic device according to one embodiment is not exposed to the outside. FIG. 10 is an example diagram illustrating a method of outputting interpretation information using a second wearable electronic device in a state where the electronic device according to one embodiment is not exposed to the outside. FIG. 11 is an example diagram illustrating a method of outputting interpretation information using an electronic device while the electronic device is exposed to the outside according to one embodiment. FIG. 12 is an example diagram illustrating a method of outputting interpretation information using a first wearable electronic device while the electronic device according to one embodiment is exposed to the outside. FIG. 13 is an example diagram illustrating a method for outputting information that induces a user to speak interpretation information using a first wearable electronic device according to one embodiment. FIG. 14 is an example diagram illustrating the operation of an electronic device when a person speaks in the other person's language according to one embodiment. FIG. 15a is an example diagram illustrating a method of sharing interpretation information with a counterparty device according to one embodiment. Figure 15b is a drawing that follows Figure 15a. FIG. 16 is an example diagram illustrating the output direction of interpretation information corresponding to the position of the speaker according to one embodiment. In relation to the description of the drawings, the same or similar reference numerals may be used for identical or similar components. FIG. 1 is a block diagram of an electronic device (101) in a network environment (100) according to one embodiment. Referring to FIG. 1, in the network environment (100), the electronic device (101) may communicate with an electronic device (102) through a first network (198) (e.g., a short-range wireless communication network) or may communicate with at least one of an electronic device (104) or a server (108) through a second network (199) (e.g., a long-range wireless communication network). According to one embodiment, the electronic device (101) may communicate with the electronic device (104) through the server (108). According to one embodiment, the electronic device (101) may include a processor (120), memory (130), input module (150), sound output module (155), display module (160), audio module