KR-20260064420-A - ELECTRONIC DEVICE, METHOD, AND NON-TRANSITORY COMPUTER READABLE STORAGE MEDIUM TO SUMMARIZE TEXT USING SENSOR DATA

KR20260064420AKR 20260064420 AKR20260064420 AKR 20260064420AKR-20260064420-A

Abstract

The electronic device may include a memory for storing instructions, one or more sensors, and at least one processor. The instructions may cause the electronic device to identify an event for outputting a notification for a first text of a first length, determine the length of the text to be used to output the notification using sensor data obtained through one or more sensors, and determine the length of the text to be used to output the notification as a second length, thereby obtaining a second text of a second length from the first text and outputting the obtained second text.

Inventors

방채림
손동일
이상현
정서로
지은혜
최보근

Assignees

삼성전자주식회사

Dates

Publication Date: 20260507
Application Date: 20241217
Priority Date: 20241030

Claims (20)

In an electronic device (100), One or more sensors (211); Memory (206) comprising one or more storage media for storing instructions; and It includes at least one processor (207) including a processing circuitry, and When the above instructions are executed individually or collectively by the at least one processor (207), Identify an event for outputting a notification for a first text (420) of a first length, and Using sensor data (410) obtained through one or more of the above sensors (211), the length of the text to be used to output the notification is determined, and Based on determining the length of the text to be used to output the above notification as a second length, a second text (430) of the second length is obtained from the first text (420), and To output the second text (430) obtained above, causing the above electronic device (100), Electronic device (100).
In claim 1, the second length is, Less than the above first length, Electronic device (100).
In claim 1, When the above instructions are executed individually or collectively by the at least one processor (207), Using a trained model for natural language processing to obtain the second text (430) from the first text (420), causing the above electronic device (100), Electronic device (100).
In claim 3, the trained model is, It is the first trained model, and When the above instructions are executed individually or collectively by the at least one processor (207), Using a second trained model, a prompt including a natural language sentence indicating the length is obtained from the sensor data (410), and By providing the first text (420) and the prompt to the first trained model, the second text (430) is obtained. causing the above electronic device (100), Electronic device (100).
In claim 1, the electronic device (100) is, It further includes output components (212) including a display (208), a speaker (210), and a haptic actuator (209), and When the above instructions are executed individually or collectively by the at least one processor (207), In order to output the second text (430), among the output components (212), control at least one output component indicated by the response obtained by providing the first text (420) to the trained model. causing the above electronic device (100), Electronic device (100).
In claim 1, When the above instructions are executed individually or collectively by the at least one processor (207), Using the sensor data (410) above, a prompt indicating the state of the user of the electronic device (100) is obtained, and Based on the above prompt, to determine the length of the text to be used to output the above notification, causing the above electronic device (100), Electronic device (100).
In claim 1, the one or more sensors (211) are, It includes a microphone (204) and a camera (203), When the above instructions are executed individually or collectively by the at least one processor (207), Based on identifying the above event, the volume of the audio signal obtained through the microphone (204) is identified, and Based on identifying the volume exceeding the threshold volume, the state of the camera (203) is switched from an inactive state to an active state during a time interval, and At least one image is acquired through the camera (203) in the active state during the time interval, and Using the above audio signal and the above at least one image, to determine the length of the text to be used to output the notification, causing the above electronic device (100), Electronic device (100).
In claim 1, When the above instructions are executed individually or collectively by the at least one processor (207), Using the sensor data (410) and information about the software application that causes the notification, the length of the text to be used to output the notification is determined. causing the above electronic device (100), Electronic device (100).
In claim 1, the one or more sensors (211) are, A device comprising at least one of a microphone (204), a camera (203), a GPS (global positioning system), an accelerometer, and a light sensor. Electronic device (100).
In a non-transient computer-readable storage medium storing one or more programs, said one or more programs, When executed by an electronic device (100) having one or more sensors (211), Identify an event for outputting a notification for a first text (420) of a first length, and Using sensor data (410) obtained through one or more of the above sensors (211), the length of the text to be used to output the notification is determined, and Based on determining the length of the text to be used to output the above notification as a second length, a second text (430) of the second length is obtained from the first text (420), and To output the second text (430) obtained above, Instructions that cause the above electronic device (100), Non-transient computer-readable storage media.
In claim 10, the second length is, Less than the above first length, Non-transient computer-readable storage media.
In claim 10, When the above one or more programs are executed by the electronic device (100), Using a trained model for natural language processing to obtain the second text (430) from the first text (420), Instructions that cause the above electronic device (100), Non-transient computer-readable storage media.
In claim 12, the trained model is, It is the first trained model, and When the above one or more programs are executed by the electronic device (100), Using a second trained model, a prompt including a natural language sentence indicating the length is obtained from the sensor data (410), and By providing the first text (420) and the prompt to the first trained model, the second text (430) is obtained. Instructions that cause the above electronic device (100), Non-transient computer-readable storage media.
In claim 10, the electronic device (100) is, It further includes output components (212) including a display (208), a speaker (210), and a haptic actuator (209), and When the above one or more programs are executed by the electronic device (100), In order to output the second text (430), among the output components (212), control at least one output component indicated by the response obtained by providing the first text (420) to the trained model. Instructions that cause the above electronic device (100), Non-transient computer-readable storage media.
In claim 10, When the above one or more programs are executed by the electronic device (100), Using the sensor data (410) above, a prompt indicating the state of the user of the electronic device (100) is obtained, and Based on the above prompt, to determine the length of the text to be used to output the above notification, Instructions that cause the above electronic device (100), Non-transient computer-readable storage media.
In claim 10, the one or more sensors (211) are, It includes a microphone (204) and a camera (203), When the above one or more programs are executed by the electronic device (100), Based on identifying the above event, the volume of the audio signal obtained through the microphone (204) is identified, and Based on identifying the volume exceeding the threshold volume, the state of the camera (203) is switched from an inactive state to an active state during a time interval, and At least one image is acquired through the camera (203) in the active state during the time interval, and Using the above audio signal and the above at least one image, to determine the length of the text to be used to output the notification, Instructions that cause the above electronic device (100), Non-transient computer-readable storage media.
In claim 10, When the above one or more programs are executed by the electronic device (100), Using the sensor data (410) and information about the software application that causes the notification, the length of the text to be used to output the notification is determined. Instructions that cause the above electronic device (100), Non-transient computer-readable storage media.
In claim 10, the one or more sensors (211) are, A device comprising at least one of a microphone (204), a camera (203), a GPS (global positioning system), an accelerometer, and a light sensor. Non-transient computer-readable storage media.
A method to be executed within an electronic device (100) comprising one or more sensors (211), An action for identifying an event to output a notification for a first text (420) of a first length, and Using sensor data (410) obtained through one or more of the above sensors (211), the operation of determining the length of text to be used to output the notification, and Based on determining the length of the text to be used to output the above notification as a second length, the operation of obtaining a second text (430) of the second length from the first text (420), and The operation of outputting the second text (430) obtained above, method.
In claim 19, the second length is, Less than the above first length, method.

Description

Electronic device, method, and non-transient computer-readable storage medium for summarizing text using sensor data The present disclosure relates to an electronic device, a method, and a non-transient computer-readable storage medium for summarizing text using sensor data. The electronic device may include a plurality of sensors, including a camera, a microphone, a light sensor, and an accelerometer. The electronic device may acquire sensor data regarding the surrounding environment of the electronic device through the plurality of sensors. For example, the electronic device may acquire an image of the surrounding environment through the camera. For example, the electronic device may acquire an audio signal regarding the surrounding environment through the microphone. The information described above may be provided as related art for the purpose of aiding understanding of the present disclosure. No claim or determination is made as to whether any of the foregoing can be applied as prior art related to the present disclosure. Figure 1 illustrates an example of an electronic device that provides notifications. Figure 2 is a simplified block diagram of an exemplary electronic device. Figure 3 is a flowchart illustrating the operation of an electronic device that outputs text. FIGS. 4a and 4b illustrate an exemplary operation of an electronic device that obtains a second text from a first text using a trained model. Figure 5 is a flowchart illustrating the operation of an electronic device that determines the length of text using a prompt indicating the user's status. Figure 6 is a flowchart illustrating the operation of an electronic device that activates a camera based on an audio signal. FIG. 7 illustrates examples of operations performed within an electronic device and an external electronic device. FIG. 8 is a block diagram of an electronic device in a network environment according to various embodiments. FIG. 9a illustrates an example of a perspective view of a wearable device. FIG. 9b illustrates an example of one or more pieces of hardware placed within a wearable device. FIGS. 10a and 10b illustrate an example of the appearance of a wearable device. FIG. 11 illustrates an example of a block diagram of a wearable device. Figure 12 shows an example of a block diagram of an electronic device for displaying an image in virtual space. Figure 13 is a schematic diagram of an exemplary AI system. Figure 1 illustrates an example of an electronic device that provides notifications. Referring to FIG. 1, a user (120) may use an electronic device (100) and an external electronic device (110). For example, the electronic device (100) may be referred to as a wearable device or a head-worn electronic device. For example, the electronic device (100) may be used to provide augmented reality (AR) or mixed reality (MR). For example, the electronic device (100) may include AR glasses. For example, the electronic device (100) may include a video see-through (VST) device. For example, the electronic device (100) can identify an event for providing a notification. For example, the electronic device (100) can identify text (140) by said event. For example, the electronic device (100) can display or provide the text (140) through the display of the electronic device (100). According to one embodiment, an electronic device (100) may be connected to or paired with an external electronic device (110). For example, the electronic device (100) may be connected to the external electronic device (110) via WiFi communication techniques, Bluetooth communication techniques, or UWB (ultra-wideband) communication techniques. For example, the external electronic device (110) may identify an event for providing text (140). For example, the external electronic device (110) may transmit various types of data (e.g., text, images, videos, and/or audio) to the electronic device (100). For example, the external electronic device (110) may transmit data representing text (140) to the electronic device (100). For example, the external electronic device (110) may transmit a signal instructing the electronic device (100) to display text (140). For example, the electronic device (100) can receive the data from an external electronic device (110). For example, the electronic device (100) can receive the signal from an external electronic device (110). For example, the electronic device (100) can use the data to display text (140) through the display of the electronic device (100). For example, a user (120) of an electronic device (100) may be in a driving state. For example, a user (120) of an electronic device (100) may be in a state of focusing on the external environment. For example, when the user (120) is in the driving state or when the user (120) is in a state of focusing on the external environment, it may be difficult for the user (120) to perceive notifications or text provided by the electronic device (100). For example, the notification (o