Search

CN-122018349-A - Multi-task cooking navigation method and device and electronic equipment

CN122018349ACN 122018349 ACN122018349 ACN 122018349ACN-122018349-A

Abstract

The invention discloses a multi-task cooking navigation method, a device and electronic equipment, wherein the method comprises the steps of obtaining a target task identifier, a visual mode characteristic and a time sequence mode characteristic in a cooking process and a historical behavior characteristic of a target user; the visual mode features comprise food material image features and temperature image features of a cooking environment, the time sequence mode features comprise gas concentration time sequence features, weight time sequence features of a cooker, temperature time sequence features and firepower time sequence features of a cooker, fusion processing is carried out on the visual mode features and the time sequence mode features based on target task identification to obtain first fusion features, the first fusion features and historical behavior features are input into a cooking multi-task network to obtain a predicted cooking result of a target cooking task, and cooking navigation is carried out based on the predicted cooking result. By utilizing the embodiment of the invention, the precision and accuracy of intelligent cooking navigation can be improved in various cooking scenes.

Inventors

  • MENG XUE

Assignees

  • 宁波方太厨具有限公司

Dates

Publication Date
20260512
Application Date
20260107

Claims (10)

  1. 1. A method of multitasking cooking navigation, the method comprising: The method comprises the steps of acquiring a target task identifier, a visual mode characteristic and a time sequence mode characteristic in a cooking process and a historical behavior characteristic of a target user, wherein the visual mode characteristic comprises a food material image characteristic and a temperature image characteristic of a cooking environment, the time sequence mode characteristic comprises a gas concentration time sequence characteristic, a weight time sequence characteristic of a cooker, a temperature time sequence characteristic and a firepower time sequence characteristic of a cooker, and the target task identifier is used for indicating a target cooking task in at least two cooking tasks; Based on the target task identifier, carrying out fusion processing on the visual mode characteristics and the time sequence mode characteristics to obtain first fusion characteristics; inputting the first fusion characteristic and the historical behavior characteristic into a cooking multi-task network to obtain a predicted cooking result of the target cooking task; and performing cooking navigation based on the predicted cooking result.
  2. 2. The method of claim 1, wherein the fusing the visual modality feature and the temporal modality feature based on the target task identification to obtain a first fused feature comprises: Determining weight information corresponding to each of the visual mode characteristics and the time sequence mode characteristics under the condition that the target task identification indicates that the target cooking task is a cooking step reminding task, wherein the weight information represents the importance degree of each of the visual mode characteristics and the time sequence mode characteristics; And carrying out weighted summation on the visual mode characteristics and the time sequence mode characteristics based on the weight information corresponding to the visual mode characteristics and the time sequence mode characteristics respectively to obtain the first fusion characteristics.
  3. 3. The method of claim 1, wherein the fusing the visual modality feature and the temporal modality feature based on the target task identification to obtain a first fused feature comprises: And under the condition that the target task identification indicates that the target cooking task is a cooking pre-warning task, the visual mode feature and the time sequence mode feature are spliced to obtain the first fusion feature.
  4. 4. The method of claim 1, wherein the fusing the visual modality feature and the temporal modality feature based on the target task identification to obtain a first fused feature comprises: screening to obtain a target image characteristic corresponding to the safety alarm task from the food material image characteristic and the temperature image characteristic, and screening to obtain a target time sequence characteristic corresponding to the safety alarm task from the gas concentration time sequence characteristic, the weight time sequence characteristic, the temperature time sequence characteristic and the firepower time sequence characteristic under the condition that the target task identifier indicates that the target cooking task is the safety alarm task; and carrying out fusion processing on the target image features and the target time sequence features to obtain the first fusion features.
  5. 5. The method of claim 1, wherein the cooking multitasking network comprises at least two gate networks and at least two cooking result prediction networks, the gate networks and the cooking result prediction networks being in one-to-one correspondence, one cooking task for each gate network; The step of inputting the first fusion characteristic and the historical behavior characteristic into a cooking multi-task network to obtain a predicted cooking result of the target cooking task comprises the following steps: Inputting the first fusion feature and the historical behavior feature into a gate network corresponding to the target task identifier to perform feature fusion processing to obtain weight information corresponding to the first fusion feature and the historical behavior feature, and performing weighted summation processing on the first fusion feature and the historical behavior feature based on the weight information corresponding to the first fusion feature and the historical behavior feature to obtain a second fusion feature; and inputting the second fusion characteristic into a cooking result prediction network corresponding to the door network to perform cooking result prediction processing, so as to obtain a predicted cooking result of the target cooking task.
  6. 6. The method of any one of claims 1 to 5, wherein the acquiring visual and temporal modal characteristics of the cooking process comprises: in the cooking process, acquiring a target food material image, a target temperature image of the cooking environment, a gas concentration information sequence, a weight information sequence of a cooker, a temperature information sequence and a firepower information sequence of a cooker; Splicing the target food material image and the target temperature image and extracting spatial features to obtain the visual mode features; And performing splicing and time sequence feature extraction on the gas concentration information sequence, the weight information sequence, the temperature information sequence and the firepower information sequence to obtain the time sequence modal feature.
  7. 7. The method of claim 6, wherein the stitching and timing feature extraction of the gas concentration information sequence, the weight information sequence, the temperature information sequence, and the fire information sequence to obtain the timing modality feature comprises: Acquiring a human sense signal sequence, a touch screen signal sequence, and a vibration information sequence and a pan detection information sequence of the pan; And performing splicing and time sequence feature extraction on the gas concentration information sequence, the weight information sequence, the temperature information sequence, the firepower information sequence, the human sense signal sequence, the touch screen signal sequence, the vibration information sequence and the pot detection information sequence to obtain the time sequence modal feature.
  8. 8. The method of claim 6, wherein the acquiring the target food material image and the target temperature image of the cooking environment, and the gas concentration information sequence, the pan weight information sequence, the temperature information sequence, and the fire information sequence of the cooktop comprise: Acquiring a plurality of initial food material images at preset moments, an initial temperature image of the cooking environment, initial gas concentration information, initial weight information of the cooker, initial temperature information and initial fire information of the cooker; Aligning the initial food material image, the initial temperature image, the gas concentration information, the weight information, the temperature information and the fire information based on each preset moment to obtain the target food material image, the target temperature image, target gas concentration information, target weight information, target temperature information and target fire information; and respectively time sequencing the target gas concentration information, the target weight information, the target temperature information and the target fire information at a plurality of preset moments to obtain the gas concentration information sequence, the weight information sequence, the temperature information sequence and the fire information sequence.
  9. 9. A multi-task cooking navigation device, the device comprising: The system comprises an acquisition module, a target task identification module, a cooking module and a cooking module, wherein the acquisition module is used for acquiring a target task identification, a visual mode characteristic and a time sequence mode characteristic in a cooking process and a historical behavior characteristic of a target user, the visual mode characteristic comprises a food material image characteristic and a temperature image characteristic of a cooking environment, the time sequence mode characteristic comprises a gas concentration time sequence characteristic, a weight time sequence characteristic of a cooker, a temperature time sequence characteristic and a firepower time sequence characteristic of a cooker, and the target task identification is used for indicating a target cooking task in at least two cooking tasks; the fusion module is used for carrying out fusion processing on the visual mode characteristics and the time sequence mode characteristics based on the target task identification to obtain a first fusion characteristic; The prediction module is used for inputting the first fusion characteristic and the historical behavior characteristic into a cooking multi-task network to obtain a predicted cooking result of the target cooking task; and the navigation module is used for carrying out cooking navigation based on the predicted cooking result.
  10. 10. An electronic device for multi-tasking cooking navigation, characterized in that it comprises a processor and a memory in which at least one instruction is stored, said at least one instruction being loaded and executed by said processor to implement the multi-tasking cooking navigation method according to any of the claims 1 to 8.

Description

Multi-task cooking navigation method and device and electronic equipment Technical Field The present invention relates to the field of cooking navigation technologies, and in particular, to a method and an apparatus for multi-task cooking navigation, and an electronic device. Background Along with the increasing demands of people for intelligent kitchen ware, intelligent cooking equipment capable of assisting users in cooking is rapidly developed. Traditional culinary art navigation relies on fixed menu step more, and is weaker to edible material state (such as temperature, ripeness, weight change) perceptibility, leads to guiding big with actual operation deviation, and intelligent culinary art effect is relatively poor, influences user experience. Disclosure of Invention Aiming at the problems in the prior art, the invention discloses a multi-task cooking navigation method, a multi-task cooking navigation device and electronic equipment, which can improve the accuracy and the precision of intelligent cooking navigation under various cooking scenes. The technical scheme disclosed by the invention is as follows: according to an aspect of the disclosed embodiments of the present invention, there is provided a multi-task cooking navigation method including: The method comprises the steps of acquiring a target task identifier, a visual mode characteristic and a time sequence mode characteristic in a cooking process and a historical behavior characteristic of a target user, wherein the visual mode characteristic comprises a food material image characteristic and a temperature image characteristic of a cooking environment, the time sequence mode characteristic comprises a gas concentration time sequence characteristic, a weight time sequence characteristic of a cooker, a temperature time sequence characteristic and a firepower time sequence characteristic of a cooker, and the target task identifier is used for indicating a target cooking task in at least two cooking tasks; Based on the target task identifier, carrying out fusion processing on the visual mode characteristics and the time sequence mode characteristics to obtain first fusion characteristics; inputting the first fusion characteristic and the historical behavior characteristic into a cooking multi-task network to obtain a predicted cooking result of the target cooking task; and performing cooking navigation based on the predicted cooking result. Optionally, based on the target task identifier, performing fusion processing on the visual mode feature and the time sequence mode feature to obtain a first fusion feature includes: Determining weight information corresponding to each of the visual mode characteristics and the time sequence mode characteristics under the condition that the target task identification indicates that the target cooking task is a cooking step reminding task, wherein the weight information represents the importance degree of each of the visual mode characteristics and the time sequence mode characteristics; And carrying out weighted summation on the visual mode characteristics and the time sequence mode characteristics based on the weight information corresponding to the visual mode characteristics and the time sequence mode characteristics respectively to obtain the first fusion characteristics. Optionally, based on the target task identifier, performing fusion processing on the visual mode feature and the time sequence mode feature to obtain a first fusion feature includes: And under the condition that the target task identification indicates that the target cooking task is a cooking pre-warning task, the visual mode feature and the time sequence mode feature are spliced to obtain the first fusion feature. Optionally, based on the target task identifier, performing fusion processing on the visual mode feature and the time sequence mode feature to obtain a first fusion feature includes: screening to obtain a target image characteristic corresponding to the safety alarm task from the food material image characteristic and the temperature image characteristic, and screening to obtain a target time sequence characteristic corresponding to the safety alarm task from the gas concentration time sequence characteristic, the weight time sequence characteristic, the temperature time sequence characteristic and the firepower time sequence characteristic under the condition that the target task identifier indicates that the target cooking task is the safety alarm task; and carrying out fusion processing on the target image features and the target time sequence features to obtain the first fusion features. Optionally, the cooking multitasking network includes at least two gate networks and at least two cooking result prediction networks, the gate networks and the cooking result prediction networks are in one-to-one correspondence, and each gate network corresponds to one cooking task; The step of inputting the first fusion characteristic an