CN-121979001-A - Cooking decision method, device, electronic equipment and storage medium

CN121979001ACN 121979001 ACN121979001 ACN 121979001ACN-121979001-A

Abstract

The invention relates to the technical field of intelligent home, and provides a cooking decision method, a cooking decision device, electronic equipment and a storage medium. The cooking decision making method comprises the steps of obtaining multi-mode state data of food to be cooked in a current time step, wherein the multi-mode state data comprise image characteristic data and thermodynamic state data of the food to be cooked, obtaining a joint state embedded vector according to the multi-mode state data through a multi-mode encoder, and determining a cooking decision making task of the next time step by utilizing a cooking decision making determining model based on the joint state embedded vector. The invention can efficiently and accurately make cooking process decisions of the cooking equipment.

Inventors

YU XIAOXI
YU LIMING
Tian Baolei

Assignees

青岛海尔科技有限公司
海尔优家智能科技（北京）有限公司
海尔智家股份有限公司

Dates

Publication Date: 20260505
Application Date: 20251225

Claims (10)

1. A cooking decision making method, characterized by being applied to a cooking apparatus, the method comprising: Acquiring multi-mode state data of food to be cooked in the current time step, wherein the multi-mode state data comprises image characteristic data and thermodynamic state data of the food to be cooked; Obtaining a joint state embedded vector according to the multi-modal state data through a multi-modal encoder, wherein the multi-modal encoder can fuse different modal data and convert the data into a neural network model with a unified representation form; and determining a cooking decision task of the next time step by utilizing a cooking decision determining model based on the joint state embedded vector, wherein the cooking decision determining model is an intelligent algorithm model.
2. The cooking decision method according to claim 1, wherein the obtaining, by the multi-modal encoder, a joint state embedding vector from the multi-modal state data comprises: determining the confidence of each mode data in the multi-mode state data; Obtaining input data according to each mode data and the confidence coefficient thereof; and obtaining the joint state embedded vector by using the multi-mode encoder according to the input data.
3. The cooking decision method according to claim 2, wherein the obtaining input data according to each of the modal data and the confidence level thereof further comprises: Determining a weight value of each mode data according to the confidence coefficient of each mode data and the cooking stage corresponding to the current time step; and obtaining the input data according to each mode data and the weight value thereof.
4. The cooking decision method of claim 1, wherein the multi-modal encoder is a time series model, the method further comprising: Acquiring historical multi-mode state data of a plurality of time steps before the current time step; the obtaining, by the multi-mode encoder, a joint state embedded vector according to the multi-mode state data includes: determining the confidence of each mode data in the multi-mode state data; Determining a weight value of each mode data according to the confidence coefficient of each mode data and the cooking stage corresponding to the current time step; Obtaining first input data according to each mode data and the weight value thereof; determining the confidence degree of each historical mode data in the historical multi-mode state data; determining a weight value of each historical mode data according to the confidence coefficient of each historical mode data and the cooking stage corresponding to the historical time step; obtaining second input data according to each historical mode data and the weight value thereof; And obtaining the joint state embedded vector by utilizing the multi-mode encoder according to the first input data and the second input data.
5. The cooking decision method according to any one of claims 1 to 4, wherein the cooking decision determination model comprises a plurality of decision sub-models and a meta arbiter, wherein each decision sub-model corresponds to a cooking decision proposal; The determining, based on the joint state embedded vector, a cooking decision task of a next time step by using a cooking decision determining model, including: Embedding the joint state into a vector, inputting each decision sub-model, and outputting a cooking decision proposal with confidence coefficient by each decision sub-model to obtain a plurality of cooking decision proposals; determining a cooking decision task of a next time step according to the plurality of cooking decision proposals by using the meta-arbiter.
6. Cooking decision method according to claim 5, characterized in that the meta-arbiter comprises preset rules and decision selection sub-models; Said determining, with said meta-arbiter, a cooking decision task for a next time step based on said plurality of cooking decision proposals, comprising: screening at least one target cooking decision proposal from the plurality of cooking decision proposals according to the confidence degrees corresponding to the plurality of cooking decision proposals by utilizing the preset rule; Determining a cooking decision task for a next time step using a decision selection sub-model according to the at least one target cooking decision proposal.
7. The cooking decision method of claim 1, wherein the method further comprises: if the confidence coefficient of the cooking decision task is larger than a first preset threshold value, executing the cooking decision task; If the confidence coefficient of the cooking decision task is smaller than or equal to the first preset threshold value and larger than the second preset threshold value, executing an active environment perception action, wherein the active environment perception action is adopted for the internal environment state of the cooking equipment and is used for acquiring accurate multi-mode state data or adjusting environment parameters to ensure the autonomous operation of the cooking effect; Triggering an arbitration mechanism if the confidence level of the cooking decision task is less than or equal to the second preset threshold, wherein the arbitration mechanism comprises at least one of the following: Re-executing the step of acquiring multi-mode state data of the food to be cooked in the current time step and the subsequent steps until the confidence level of the cooking decision task is greater than the second preset threshold; And re-executing the step of obtaining the joint state embedded vector and the subsequent steps according to the multi-mode state data by the multi-mode encoder based on the mode data with the highest confidence in the multi-mode state data.
8. A cooking decision making apparatus, characterized in that it is provided in a cooking device, said apparatus comprising: The acquisition module is used for acquiring multi-mode state data of the food to be cooked in the current time step, wherein the multi-mode state data comprises image characteristic data and thermodynamic state data of the food to be cooked; the feature extraction module is used for obtaining a joint state embedded vector according to the multi-mode state data through a multi-mode encoder, wherein the multi-mode encoder can be used for fusing different mode data and converting the data into a neural network model with a unified representation form; and the decision module is used for determining a cooking decision task of the next time step by utilizing a cooking decision determining model based on the joint state embedded vector, wherein the cooking decision determining model is an intelligent algorithm model.
9. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor implements the steps of the cooking decision method according to any one of claims 1 to 7 when the program is executed by the processor.
10. A non-transitory computer readable storage medium, on which a computer program is stored, characterized in that the computer program, when executed by a processor, implements the steps of the cooking decision method according to any one of claims 1 to 7.

Description

Cooking decision method, device, electronic equipment and storage medium Technical Field The invention relates to the technical field of intelligent home, in particular to a cooking decision method, a cooking decision device, electronic equipment and a storage medium. Background The steaming oven technology with cameras has undergone an evolution from "visualization" to "intellectualization" of the cooking process. In early days, the camera mainly realizes remote monitoring and process recording functions, and a user can check food states or share short videos through a mobile phone APP. However, the real technical leap is that the introduction of artificial intelligent image recognition technology enables a system to 'understand' the image content and make cooking decisions, and solves two major pain points of strong dependency of traditional cooking experience and opaque process. At present, the approximate technical scheme in the industry mainly comprises three types of food material identification and automatic program matching, cooking state sensing and real-time adjustment, and safety early warning and interaction assistance, but all have the problems that the image quality is interfered by high-temperature steam, the imaging visual angle is limited, and the like, so that the image preprocessing burden is heavy, the decision misjudgment rate is high, the system robustness is poor and the response delay is caused. These problems limit the stability of automatic cooking and the user experience. Therefore, how to efficiently and accurately make a cooking process decision of the cooking device is a technical problem to be solved. Disclosure of Invention The invention provides a cooking decision method, a device, electronic equipment and a storage medium, which are used for solving the defects in the prior art and efficiently and accurately making a cooking process decision of cooking equipment. The invention provides a cooking decision method, which comprises the following steps: The method comprises the steps of obtaining multi-modal state data of food to be cooked in a current time step, obtaining a multi-modal state data of the food to be cooked in the current time step, wherein the multi-modal state data comprises image characteristic data and thermodynamic state data of the food to be cooked, obtaining a joint state embedded vector through a multi-modal encoder according to the multi-modal state data, converting the multi-modal encoder into a neural network model in a unified representation form, determining a cooking decision task in the next time step by using a cooking decision determining model based on the joint state embedded vector, and determining the cooking decision determining model to be an intelligent algorithm model. The cooking decision method comprises the steps of obtaining a joint state embedded vector according to multi-mode state data through a multi-mode encoder, determining the confidence coefficient of each mode data in the multi-mode state data, obtaining input data according to each mode data and the confidence coefficient of each mode data, and obtaining the joint state embedded vector by utilizing the multi-mode encoder according to the input data. The cooking decision method provided by the invention obtains input data according to each mode data and the confidence coefficient thereof, and further comprises the steps of determining the weight value of each mode data according to the confidence coefficient of each mode data and the cooking stage corresponding to the current time step, and obtaining the input data according to each mode data and the weight value thereof. The method for making the cooking decision comprises the steps of obtaining historical multi-mode state data of a plurality of time steps before a current time step, obtaining a joint state embedded vector according to the multi-mode state data through the multi-mode encoder, determining the confidence coefficient of each mode data in the multi-mode state data, determining the weight value of each mode data according to the confidence coefficient of each mode data and a cooking stage corresponding to the current time step, obtaining first input data according to each mode data and the weight value thereof, determining the confidence coefficient of each historical mode data in the historical multi-mode state data, determining the weight value of each historical mode data according to the confidence coefficient of each historical mode data and the cooking stage corresponding to the historical time step, obtaining second input data according to the confidence coefficient of each historical mode data and the weight value thereof, and obtaining the joint state embedded vector by utilizing the multi-mode encoder according to the first input data and the second input data. The cooking decision determining model comprises a plurality of decision sub-models and a meta arbiter, wherein each decision sub-model correspo