Search

EP-4325119-B1 - COOKING APPARATUS AND CONTROL METHOD THEREFOR

EP4325119B1EP 4325119 B1EP4325119 B1EP 4325119B1EP-4325119-B1

Inventors

  • KA, KEEHWAN
  • RYU, JONGYOUB
  • PARK, Kyoungchoon
  • Song, Minjin
  • LEE, JEONGWON

Dates

Publication Date
20260506
Application Date
20221104

Claims (15)

  1. A cooking apparatus (100) comprising: a cooking chamber (110) configured to receive food; a camera (120) to photograph inside the cooking chamber (110), the camera (120) having a lens; a memory (130, 131, 132) configured to store a plurality of cooking control information used to cook the food; and a processor (140) configured to control the camera (120) to photograph the food inside the cooking chamber (110); wherein the processor (140) is further configured to: extract state information of the food from the photographed image, and determine from the photographed image a contamination degree of the lens of the camera (120), characterized in that the processor (140) is further configured to : correct the state information of the food based on the determined contamination degree; acquire first cooking control information corresponding to the corrected state information; and control the cooking chamber (110) to cook the food according to the acquired first cooking control information.
  2. The cooking apparatus (100) of claim 1, wherein the processor (140) is further configured to: based on the determined contamination degree being equal to or greater than a first threshold, acquire second cooking control information corresponding to a result sensed by at least one another sensor (150) stored in the memory (130, 131, 132); and control the cooking chamber (110) to cook the food according to the acquired second cooking control information.
  3. The cooking apparatus (100) of claim 2, wherein the processor (140) is further configured to: based on the contamination degree being less than the first threshold value and greater than or equal to a second threshold value, acquire cooking control information corresponding to state information corrected based on the contamination degree from the memory (130, 131, 132), and based on the contamination degree being less than the second threshold value, acquire cooking control information corresponding to uncorrected state information from the memory (130, 131, 132).
  4. The cooking apparatus (100) of claim 3, further comprising: a temperature sensor (150); and a timer, wherein the processor (140) is further configured to: based on the determined contamination degree being equal to or greater than the first threshold value, estimate the state information of the food based on a temperature change sensed by the temperature sensor (150) and a time counted by the timer, and acquire cooking control information corresponding to the estimated state information from the memory (130, 131, 132).
  5. The cooking apparatus (100) of claim 1, wherein the processor (140) is further configured to: determine the contamination degree by comparing a size of a region of interest in which the food is identified in the photographed image and a size of a region occluded due to contamination of the lens of the camera (120), and correct the state information by adjusting a state value of at least one of a shape, a color, and a size of the food in proportion to the contamination degree wherein the contamination degree is calculated as a ratio of the size of the occluded region to the size of the region of interest.
  6. The cooking apparatus (100) of claim 1, wherein the memory (130, 131, 132) is configured to store a plurality of food change patterns (910, 920, 1010, 1110, 1120) corresponding to each of the plurality of cooking control information, and wherein the processor (140) is further configured to: acquire a change pattern (910, 920, 1010, 1110, 1120) of the state information by accumulating the state information of the food for a predetermined time; and search, from the memory (130, 131, 132), a food change pattern (910, 920, 1010, 1110, 1120) matching the acquired change pattern (910, 920, 1010, 1110, 1120), and acquire cooking control information corresponding to the searched food change pattern (910, 920, 1010, 1110, 1120).
  7. The cooking apparatus (100) of claim 6, wherein the processor (140) is further configured to, based on the food change pattern (910, 920, 1010, 1110, 1120) matching the acquired change pattern (910, 920, 1010, 1110, 1120) being not searched from the memory (130, 131, 132), search, in the memory (130, 131, 132), a food change pattern (910, 920, 1010, 1110, 1120) matching the acquired change pattern (910, 920, 1010, 1110, 1120) with a variable change pattern (1220).
  8. A method for controlling a cooking apparatus (100) comprising: photographing food inside a cooking chamber (110) through a camera (120) of the cooking apparatus (100); extracting state information of the food from the photographed image photographed by the camera (120); determining from the photographed image a contamination degree of a lens of the camera (120); correcting the state information of the food based on the determined contamination degree; acquiring cooking control information corresponding to the corrected state information from the cooking apparatus (100), the cooking apparatus (100) storing a plurality of cooking control information; and controlling a cooking of the food according to the acquired cooking control information.
  9. The method of claim 8, wherein the acquiring includes, based on the determined contamination degree being equal to or greater than a first threshold, acquiring first cooking control information corresponding to a result sensed by at least one another sensor (150), and wherein the cooking includes cooking the food according to the acquired first cooking control information.
  10. The method of claim 9, wherein the acquiring includes, based on the determined contamination degree being less than the first threshold value and greater than or equal to a second threshold value, acquiring second cooking control information corresponding to state information corrected based on the contamination degree, and based on the determined contamination degree being less than the second threshold value, acquiring third cooking control information corresponding to uncorrected state information.
  11. The method of claim 9, wherein the cooking apparatus (100) further comprises: a temperature sensor (150); and a timer, wherein the acquiring includes, based on the determined contamination degree being equal to or greater than the first threshold value, estimating the state information of the food based on a temperature change sensed by the temperature sensor (150) and a time counted by the timer, and acquiring cooking control information corresponding to the estimated state information.
  12. The method of claim 8, wherein the determining the contamination degree includes calculating the contamination degree by comparing a size of a region of interest in which the food is identified in the photographed image and a size of a region occluded due to contamination of the lens of the camera (120), wherein the correcting includes correcting the state information by adjusting a state value of at least one of a shape, a color, and a size of the food in proportion to the contamination degree, and wherein the contamination degree is calculated as a ratio of the size of the occluded region to the size of the region of interest.
  13. The method of claim 8, wherein the cooking apparatus (100) is configured to store a plurality of food change patterns (910, 920, 1010, 1110, 1120) corresponding to each of the plurality of cooking control information, and wherein the acquiring includes: acquiring a change pattern (910, 920, 1010, 1110, 1120) of the state information by accumulating the state information of the food for a predetermined time, and searching a food change pattern (910, 920, 1010, 1110, 1120) matching the acquired change pattern (910, 920, 1010, 1110, 1120); and acquiring cooking control information corresponding to the searched food change pattern (910, 920, 1010, 1110, 1120).
  14. The method of claim 13, wherein the searching includes, based on the food change pattern (910, 920, 1010, 1110, 1120) matching the acquired change pattern (910, 920, 1010, 1110, 1120) being not searched from the cooking apparatus (100), searching a food change pattern (910, 920, 1010, 1110, 1120) matching the acquired change pattern (910, 920, 1010, 1110, 1120) with a variable change pattern (1220).
  15. A computer program product comprising a memory (130, 131, 132) device that comprises one or more computer executable instructions that when executed by one or more processors (140) cause the one or more processors (140) of a cooking apparatus (100) to perform a method, the method comprising: wherein the program includes: photographing food inside a cooking chamber (110) through a camera (120) of the cooking apparatus (100); extracting state information of the food from the photographed image photographed by the camera (120); determining from the photographed image a contamination degree of a lens of the camera (120); correcting the state information of the food based on the determined contamination degree; acquiring cooking control information corresponding to the corrected state information from the cooking apparatus (100), the cooking apparatus (100) storing a plurality of cooking control information; and controlling a cooking of the food according to the acquired cooking control information.

Description

[Technical Field] The disclosure relates to a cooking apparatus and a control method thereof. More particularly, the disclosure relates to a cooking apparatus for acquiring state information of food from a photographed image of food inside a cooking chamber and cooking food according to the state information of the food, and a control method thereof. [Background Art] A cooking apparatus is an apparatus capable of performing various cooking functions, such as baking, frying, boiling, or the like. Recently, a cooking apparatus equipped with a camera in the cooking apparatus to photograph food to acquire information about the food has been developed. For example, WO 2021/184003 A1 discloses a method for dirty camera detection including: detecting a first predetermined state change event; sampling a set of cavity measurements; optionally determining a set of features of the set of cavity measurements; determining a class label based on the cavity measurements; optionally verifying the classification; and facilitating use of the appliance based on the classification. KR 101 778 561 B1 relates to a pollution detection and notice apparatus of a camera and a control method thereof, comprising: an image analysis portion to distinguish a user area and an analysis area of a boundary of the user area from a partial area of an image filed in a camera module; a pollution detection portion to detect pollution of a lens of the camera module and a protection cover using the image analyzed in the image analysis portion; and a control portion to control driving of a display portion to notify users of occurrence of pollution when pollution is detected in the pollution detection portion. However, the prior cooking apparatus has a problem in that when food is photographed in a state where a camera lens is contaminated due to steam or foreign substances generated inside the cooking chamber, information about the food from the photographed image is unreliable. [Disclosure] [Technical Problem] The present invention is to solve the problem described above, and an object of the present invention is to provide a cooking apparatus for performing automatic cooking by correcting state information of food extracted from a photographed image according to a degree of contamination of a camera lens, and a control method thereof. In addition, an embodiment of acquiring state information of food using sensing information other than the photographed image is included when reliable state information cannot be acquired from the photographed image since the lens is excessively contaminated. [Technical Solution] In order to achieve the above object, according to the present invention, there is provided a cooking apparatus comprising a cooking chamber configured to receive food, a camera to photograph inside the cooking chamber, the camera having a lens, a memory configured to store a plurality of cooking control information used to cook the food and, a processor configured to control the camera to photograph the food inside the cooking chamber. The processor is further configured to extract state information of the food from the photographed image, and determine a contamination degree of the lens of the camera, correct the state information of the food based on the determined contamination degree, acquire cooking first control information corresponding to the corrected state information, and control the cooking chamber to cook the food according to the acquired first cooking control information. The processor is further configured to, based on the determined contamination degree being equal to or greater than a first threshold, acquire second cooking control information corresponding to a result sensed by at least one another sensor stored in the memory, and control the cooking chamber to cook the food according to the acquired second cooking control information. The processor is further configured to, based on the determined contamination degree being less than the first threshold value and greater than or equal to a second threshold value, acquire cooking control information corresponding to state information corrected based on the contamination degree from the memory, and based on the contamination degree being less than the second threshold value, acquire cooking control information corresponding to uncorrected state information from the memory. The apparatus may further comprises a temperature sensor and a timer, wherein the processor is further configured to, based on the contamination degree being equal to or greater than the first threshold value, estimate the state information of the food based on a temperature change sensed by the temperature sensor and a time counted by the timer, and acquire cooking control information corresponding to the estimated state information from the memory. The processor is further configured to, determine the contamination degree by comparing a size of a region of interest in which the food is identified in t