EP-4264599-B1 - ROUTING OF USER COMMANDS ACROSS DISPARATE ECOSYSTEMS

Inventors

• KATHPAL, PRATEEK
• RUBIN, BRIAN ARTHUR

Dates

Publication Date

20260506

Application Date

20211217

Claims (2)

1. A system (10) for routing commands, the system (10) comprising: a recognition module including one or more hardware processors, configured to receive, from a head unit (30) of a vehicle, one or more utterances comprising at least one command, wherein the system is characterized in that the recognition module is further configured to identify a target ecosystem (82) associated with the at least one command, wherein the recognition module includes a natural language understanding, NLU, module (25) that interprets a meaning of the one or more utterances and identifies the target ecosystem (82) based on the interpreted meaning, and wherein the target ecosystem (82) comprises a plurality of integrated smart devices and provides interoperability between the plurality integrated smart devices that are configured to receive and transmit data over a similar protocol or using a similar application program interface; and in that it further comprises: a connection manager (50) configured to transmit the at least one command to the target ecosystem (82), wherein the NLU module (25) identifies the target ecosystem (82) using one or more NLU models accessed by the NLU module (25), and the connection manager (50) receives feedback from the target ecosystem (82) about the at least one command, and the NLU module (25) updates the one or more NLU models based on the feedback.
2. The system (10) of claim 1, wherein the recognition module comprises an automatic speech recognition, ASR, module (60) for transcribing the one or more utterances to text.

Description

CROSS-REFERENCE TO RELATED APPLICATIONS This application claims the benefit of U.S. provisional application Serial No. 63/128,293 filed December 21, 2020. TECHNICAL FIELD Disclosed herein are systems and methods for routing of user commands across disparate Smart Home / IoT ecosystems. BACKGROUND Consumers today are increasingly more connected to their environment whether it be their home, work, or vehicle. For example, smart home devices and systems have become ubiquitous within homes. Multiple types of devices can be included within a smart home system. For example, a typical system can include smart speakers, smart thermostats, smart doorbells, and smart cameras. In such a system, each device can interact with the other devices and be controlled by a user from a single point of control. Connectivity amongst devices and single-point control can typically be accomplished only when each device within a system is manufactured by a single manufacturer or otherwise specifically configured to integrate. The integrated smart devices together with the smart home system can be called a smart home ecosystem or an internet-of-things (IoT) ecosystem. Characteristics of an IoT ecosystem are interoperability between devices configured to receive and transmit data over a similar protocol or using a similar application program interface. IoT ecosystems typically have a shared hub comprising at least a management application and data repository for the data obtained from the devices. Additionally, these ecosystems typically require the devices to execute on a particular operating system such as the Android® or iOS® operating system. IoT ecosystems are designed to restrict the types of devices permitted within the ecosystem. For example, the Google Home ecosystem integrates with Google's Nest products. End users can only achieve interoperability between devices manufactured by the same company, or one designed specifically to function with specific other manufacturers devices. Restricting interoperability in such a way reduces an end user's ability to select a disparate group of devices, instead end users must only buy devices manufactured by the same manufacturer or devices that all use the same communication protocol and/or control application or operating system. It would therefore be advantageous to provide an integration platform within a vehicle, such as a car, enabling end users to interact with their existing smart home ecosystems including ecosystem configuration using a single control application. Document US 9 734 839 B1 discloses a system for routing commands, wherein received speech is mapped to a target application. A connection manager transmits the command to the target application. SUMMARY Described herein are systems and methods for using a connection manager to direct voice or multimodal commands received by an automotive assistant via cloud-based, artificial intelligence for vehicles. The automotive assistant or the cloud-based AI can determine where to route the commands based on an analysis of an utterance and/or other input. The analysis can include speech recognition and natural language understanding. Natural language understanding or natural language processing can be applied to recognized speech to determine a destination ecosystem. Once a target ecosystem is identified, the automotive assistant or the cloud-based AI can identify an IoT ecosystem over which the command should be transmitted and then transmit the command. Transmission can include modifying the command into a format accepted by the target ecosystem, and/or using natural language understanding or natural language processing to modify the content of the command. The cloud-based AI, or a module executing within the cloud-based AI, can receive feedback from the target ecosystem regarding the routed command, and use the received feedback to modify one or more natural language understanding or natural language processing models. Modifying the models can have the effect of modifying a list of target ecosystems associated with the originally received commands. Described herein is a system for routing commands where the system includes a recognition module that receives one or more utterances from a head unit of a vehicle. The one or more utterances can include at least one command. Using this one command, the recognition module can identify a target ecosystem and a connection manager can transmit the at least one command to the target ecosystem. The recognition module can include an automatic speech recognition module that transcribes the one or more utterances into text. The recognition module comprises a natural language understanding (NLU) module that interprets a meaning of the one or more utterances and identifies the target ecosystem based on the interpreted meaning. The NLU module can identify the target ecosystem using one or more NLU models accessed by the NLU module. The connection manager receives feedback from the target ecosys