CN-121986043-A - Electronic device, vehicle and method for transmitting control information

Abstract

An electronic device is disclosed. The electronic device includes a memory circuit, a processing circuit, and an interface. The electronic device is configured to obtain sensor data from one or more sensors. The electronic device is configured to determine one or more physical inputs. The electronic device is configured to determine user data indicative of user behavior, such as by applying a machine learning model to the sensor data and the one or more physical inputs. The electronic device is configured to communicate control information based on the user data.

Inventors

• PETER BLOMKVIST

Assignees

• 索尼集团公司

Dates

Publication Date

20260505

Application Date

20240918

Priority Date

20230928

Claims (20)

1. 1. An electronic device comprising a memory circuit, a processor circuit, and an interface, wherein the electronic device is configured to: Obtaining sensor data from one or more sensors; determining one or more physical inputs; Determining user data indicative of user behavior by applying a machine learning model to the sensor data and the one or more physical inputs, and And transmitting control information based on the user data.
2. 2. The electronic device of claim 1, wherein the sensor data comprises one or more of optical sensor data, head position data, line of sight sensor data, motion data, audio data, and heart rate data.
3. 3. The electronic device of any of the preceding claims, wherein the one or more sensors comprise one or more sensors associated with a vehicle and/or one or more sensors associated with a user device.
4. 4. The electronic device of any one of the preceding claims, wherein the one or more sensors comprise one or more of an optical sensor, a heart rate sensor, and a microphone.
5. 5. The electronic device of any of the preceding claims, wherein the one or more physical inputs comprise one or more of a steering wheel input, a pedal input, an accelerator input, a brake input, and/or a human take over.
6. 6. The electronic device of any of the preceding claims, wherein the user data comprises facial expression, comfort, and/or user profile.
7. 7. The electronic device of any of the preceding claims, wherein the machine learning model is configured to determine the user data based on the sensor data and the one or more physical inputs.
8. 8. The electronic device of any of the preceding claims, wherein the machine learning model comprises one or more of a convolutional neural network, a recurrent neural network, and a multi-modal learning model with a transformer.
9. 9. The electronic device of any of the preceding claims, wherein the electronic device is configured to identify a user based on the sensor data.
10. 10. The electronic device of any of the preceding claims, wherein the electronic device is configured to communicate by displaying a user interface object indicating the control information.
11. 11. The electronic device of claim 10, wherein displaying the user interface object comprises projecting the user interface object on a surface of a vehicle and/or a retina of a user.
12. 12. The electronic device of any of claims 10-11, wherein displaying the user interface object includes causing the user interface object to be displayed on a display device.
13. 13. The electronic device of any of claims 10-12, wherein the user interface object comprises one or more augmented reality objects configured to support navigation and/or provide information of a vehicle surroundings.
14. 14. The electronic device of any of the preceding claims, wherein the electronic device is configured to communicate by transmitting one or more audio signals indicative of the control information.
15. 15. The electronic device of any one of the preceding claims, wherein the electronic device is configured to communicate the control information to a portion of a vehicle to control operation of the vehicle.
16. 16. The electronic device of any of the preceding claims, wherein the control information comprises one or more of control parameters for operating the vehicle, and assistance information to a user.
17. 17. The electronic device of any of the preceding claims, wherein the electronic device is configured to receive feedback from a user via one or more of sensor data, user interface input, and audio signals.
18. 18. The electronic device of any of claims 15-17, wherein the electronic device is configured to adjust operation of a vehicle based on feedback and/or the user data, and/or one or more navigation parameters, and/or energy consumption parameters.
19. 19. The electronic device of any one of the preceding claims, wherein the vehicle is capable of autonomous or semi-autonomous operation.
20. 20. A method performed by an electronic device, the method comprising: Obtaining sensor data from one or more sensors; determining one or more physical inputs; Determining user data indicative of user behavior by applying a machine learning model to the sensor data and the one or more physical inputs, and And transmitting control information based on the user data.

Description

Electronic device, vehicle and method for transmitting control information Technical Field The present disclosure relates to the field of vehicles, such as partially or fully automated vehicles. The present disclosure relates to electronic devices, vehicles, and related methods for communicating control information. Background As vehicles become more automated, there is uncertainty in the automated systems that operate the vehicles due to lack of transparency. Regardless of the level of automation, it is envisaged that there will be human occupants in the vehicle as drivers and/or passengers. For example, an automated vehicle AV is expected to have the potential to improve the safety and comfort of travel. However, there is uncertainty about the intent of the system to support autonomous operation. In addition, there are a variety of different driving styles and/or preferences that, if not followed, may lead to discomfort for the passenger(s) and may lead to distrust of the automated system. Disclosure of Invention Accordingly, there is a need for a vehicle, electronic device, and method for communicating control information of a vehicle system that may alleviate, mitigate, or solve existing drawbacks and may allow for determining user behavior (such as driver and/or passenger behavior reactions), adapting control operations of the vehicle system accordingly, and communicating information about the control operations to a user, for example. An electronic device is disclosed. The electronic device includes a memory circuit, a processor circuit, and an interface. The electronic device is configured to obtain sensor data from one or more sensors. The electronic device is configured to determine one or more physical inputs. The electronic device is configured to determine user data indicative of user behavior, such as by applying a machine learning model to the sensor data and the one or more physical inputs. The electronic device is configured to communicate control information based on user data. A method performed by an electronic device is disclosed. The method includes obtaining sensor data from one or more sensors. The method includes determining one or more physical inputs. The method includes determining user data indicative of user behavior, such as by applying a machine learning model to the sensor data and the one or more physical inputs. The method includes communicating control information based on user data. A vehicle is disclosed. The vehicle includes a memory circuit, a processor circuit, and an interface. A vehicle includes an electronic device according to the present disclosure. An advantage of the present disclosure is that the disclosed electronic device and the disclosed method may assist in human-machine interaction between a user and the electronic device, thereby enabling improved potential comfort, confidence and trust of a vehicle equipped with an automatic capability, such as an automatic driving system of an automatic vehicle, such as a partial or full AV. The disclosed electronic device and the disclosed method may reduce the number of unnecessary manual takeover (e.g., performed by a vehicle driver) by communicating control information based on user data. Further, the disclosed electronic device and the disclosed method may provide an assessment of driver/passenger alertness and/or alertness to ensure that the driver of the vehicle is ready to take control when necessary (such as in the event that evasive action is required). It is to be appreciated that the disclosed techniques allow for automatic adjustment of the operation of the vehicle (such as personalization of the operation and/or parameters thereof). Drawings The above and other features and advantages of the present disclosure will become apparent to those skilled in the art from the following detailed description of an example of the present disclosure with reference to the accompanying drawings, in which: FIG. 1 is a diagram illustrating an example vehicle and an example electronic device according to the present disclosure; FIG. 2 is a diagram illustrating an example vehicle in accordance with this disclosure, an Fig. 3 is a flow chart illustrating an example method performed in an electronic device according to the invention. Detailed Description Various examples and details are described below with reference to the accompanying drawings (when relevant). It should be noted that the figures may or may not be drawn to scale and that elements of similar structure or function are represented by like reference numerals throughout the figures. It should also be noted that the drawings are only intended to facilitate the description of the examples. The drawings are not intended to be an exhaustive description of the disclosure or as a limitation on the scope of the disclosure. Additionally, the illustrated examples need not have all of the illustrated aspects or advantages. Aspects or advantages described in connection wit