US-12625673-B2 - Smart device configuration guidance via automated assistant interface of separate client device

US12625673B2US 12625673 B2US12625673 B2US 12625673B2US-12625673-B2

Abstract

Methods, apparatus, and computer readable media are described for tailoring instructions for configuring a smart device based on feedback received from the smart device and/or based on feedback received from a device that is separate from the smart device. Some implementations additionally or alternatively relate to utilization of data corresponding to previous installations of similar smart devices to tailor provided instructions in order to mitigate the reoccurrence of past issues during configuration of smart devices. Some implementations additionally or alternatively relate to proactively suggesting and/or providing, to a user, guidance related to a smart device in response to determining the smart device is to be configured by the user and/or in response to determining a complete configuration of the smart device has not occurred.

Inventors

Alexandra Fitzgerald
Jesse Boettcher
Melissa Daniels

Assignees

GOOGLE LLC

Dates

Publication Date: 20260512
Application Date: 20240329

Claims (10)

1 . A method implemented by one or more processors, the method comprising: determining, based on a received query, that a smart device is to be configured by a user, wherein the received query is received prior to any previous utilization of an automated assistant interface of a client device to configure the smart device; providing, by the automated assistant interface of the client device or additional client device of the user, an instruction for capturing an image associated with configuration of the smart device; receiving the image based on the user capturing the image in response to the provided instruction, the image captured via a camera of the client device or an additional client device of the user; determining a characteristic of the configuration based on the received image, wherein determining the characteristic of the configuration based on the received image comprises: determining, based on a QR code in the received image, a model of the smart device; generating, based on the received query and the model of the smart device, a modified query; and providing, by the automated assistant interface, a configuration instruction for configuring the smart device, wherein the configuration instruction is based on processing the modified query.
2 . The method of claim 1 , wherein the determined characteristic is based on a comparison of the received image to other configuration images, the other configuration images associated with previous configurations of similar smart devices.
3 . The method of claim 1 , further comprising: accessing account information to identify applications associated with the user; and determining that a camera application is associated with the user, wherein the instruction identifies the camera application.
4 . The method of claim 1 , wherein the received image corresponds to wires for connecting to the smart device, and the characteristic of the configuration corresponds to a wiring issue identified in the image.
5 . The method of claim 1 , further comprising: determining, based on the received image, an installation location for the smart device, wherein the configuration instruction includes an indication of the installation location.
6 . The method of claim 1 , further comprising: determining, based on the received image, a current location of the smart device; and determining whether to recommend that the user move the smart device to an additional location or to recommend that the user keep the smart device in the current location.
7 . A system comprising one or more processors and memory storing instructions that cause the one or more processors to: determine, based on a received query, that a smart device is to be configured by a user, wherein the received query is received prior to any previous utilization of an automated assistant interface of a client device to configure the smart device; provide, by the automated assistant interface of the client device or additional client device of the user, an instruction for capturing an image associated with configuration of the smart device; receive the image based on the user capturing the image in response to the provided instruction, the image captured via a camera of the client device or an additional client device of the user; determine a characteristic of the configuration based on the received image, wherein in determining the characteristic of the configuration based on the received image one or more of the processors are to: determine, based on a QR code in the received image, a model of the smart device; generate, based on the received query and the model of the smart device, a modified query; and provide, by the automated assistant interface, a configuration instruction for configuring the smart device, wherein the configuration instruction is based on processing the modified query.
8 . The system of claim 7 , wherein the determined characteristic is based on a comparison of the received image to other configuration images, the other configuration images associated with previous configurations of similar smart devices.
9 . The system of claim 7 , further comprising instructions to: access account information to identify applications associated with the user; and determine that a camera application is associated with the user, wherein the instruction identifies the camera application.
10 . The system of claim 7 , wherein the received image corresponds to wires for connecting to the smart device, and the characteristic of the configuration corresponds to a wiring issue identified in the image.

Description

BACKGROUND Automated assistants (also known as “personal assistant modules”, “mobile assistants”, or “chat bots”) may be interacted with by a user via a variety of client devices, such as smart phones, tablet computers, wearable devices, automobile systems, standalone personal assistant devices, and so forth. The automated assistants receive input from the user (e.g., typed and/or spoken natural language input) and respond with responsive content (e.g., visual and/or audible natural language output). An automated assistant interacted with via a client device may be implemented via the client device itself and/or via one or more remote computing devices that are in network communication with the client device (e.g., computing device(s) in “the cloud”). Regardless of the implementation, interactions with the automated assistant can occur via an automated assistant interface, which can be a graphical and/or voice-based interface. SUMMARY Techniques are described herein for using an automated assistant to provide guidance in installing, or otherwise configuring, a smart device. The guidance can be provided via an automated assistant interface of client device(s) that are separate from the smart device, and can be adapted based on various factors, such as feedback generated by the smart device and/or feedback generated by other device(s) during the configuration. Oftentimes upon purchasing a smart device, a user may attempt to install the smart device without assistance and/or by relying on a fixed set of provided instructions. As a result, a user may often install the smart device improperly, install the smart device in a sub-optimal location and/or with sub-optimal parameters, and/or encounter problems during installation. In order to address these and/or other issues, implementations described herein relate to utilizing an automated assistant interface to provide guidance to the user during installation of a smart device. In some implementations, the provided guidance can be dynamically adapted based on feedback received during the installation process. As described herein, the feedback can be generated by the smart device itself, can be image(s) captured via a client device (e.g., a client device providing the automated assistant interface, or a separate client device), and/or other forms of feedback. Furthermore, in some additional or alternative implementations, the automated assistant interface can allow the user to query the automated assistant for various purposes during the installation process, without requiring the user to transition away from the automated assistant interface being utilized to provide the guidance. Various techniques may be utilized to determine that a smart device is to be installed by a user, and to initiate the providing of guidance via an automated assistant interface in response. In some implementations, such a determination may be made based on input that is an explicit self-initiated request of the user (e.g, “Assistant, help me install Smart Device X”). In some implementations, a suggestion to utilize the assistant interface for installation may be presented to the user based on one or more inputs, and the assistant interface may provide the guidance in response to an affirmative response to the suggestion (e.g., a selection of the suggestion). The suggestion may be provided via the automated assistant interface and/or a separate interface. In some of those implementations, the suggestion to install a smart device can be based on: detecting a signal generated by the smart device (e.g., a Bluetooth signal, RFID signal, wireless signal); determining that the user has recently purchased the smart device (e.g., based on an email purchase confirmation); based on captured image(s) that capture the smart device (e.g., an image captured by a camera of a client device that provides the automated assistant interface); based on detecting an issue during configuration of the smart device (e.g., an issue determined based on feedback from the hardware device); etc. As used herein, determining that a smart device is to be installed or otherwise configured by a user can occur prior to any configuration of the smart device by the user and/or can occur during configuration (but before the configuration is complete (as indicated by one or more criteria)). For example, determining that a smart device is to be configured can be based on input(s) received, from the smart device and/or the user, during the configuration of the smart device. As one explicit example, the smart device can provide a temporary beacon signal for introducing itself to electronic device(s) in a home. A client device of the user can receive the beacon signal and, in response to receiving the beacon signal, can cause a suggestion for providing guidance to be provided. In other implementations, guidance may be provided via an automated assistant interface without first providing a suggestion. In some of those other implement