WO2020030462A1

WO2020030462A1 - Cooking system

Info

Publication number: WO2020030462A1
Application number: PCT/EP2019/070338
Authority: WO
Inventors: Klaus Wälzlein; Christoph Luckhardt; Michele Toppano; Thorben SCHIFFLER; Thomas Schulz; Volkmar Bunzel; Constantin KRAUSSE
Original assignee: Electrolux Appliances Aktiebolag
Priority date: 2018-08-10
Filing date: 2019-07-29
Publication date: 2020-02-13
Also published as: EP3608593A1; EP3608593B1

Abstract

A cooking system, comprises an oven (10) having an oven cavity (14) that is accessible via an oven door (16), and an external computing means (32). The oven (10) is provided with at least one optical sensor (28) for obtaining picture data of the oven cavity (14), means for10 detecting door closing and triggering the optical sensor to obtain picture data, processing means for performing a basic picture evaluation to determine that the oven cavity (14) no longer is empty, and communication means for providing picture data to the external computing means (32). The external computing means (32) is adapted to perform a food recognition routine to determine food parameters based on the picture data provided by the communication means, select cooking parameters based on the determined food parameters, and transmit the selected cooking parameters to the oven (10).

Description

Cooking system

The present invention relates to a cooking system and to a method for operating a food processing oven.

In the field of cooking appliances several attempts were made to facilitate the cooking operation for a user by providing assis tance via the internet, such as by providing recipe data from a central database.

In DE 10 2012 204229 A1 there is disclosed a device having a cooking chamber in which there is arranged a camera for record ing an image of the food in the cooking chamber. A recognition unit is provided to recognize the food in the cooking chamber in dependence of the captured image, and a selection unit selects a cooking program from predetermined cooking programs in depend ence of the captured image.

The system known from DE 10 2012 204 229 A1 is disadvantageous in that the food recognition requires sophisticated evaluation means, which requires a considerable amount of computation and data storage, which usually is not available in a household ap pliance. Thus, unless the household appliance is equipped with a fast processing device, which results in high costs of the household appliance, the quality and speed of the food recogni tion will not be satisfactory.

In view of these problems, it is an object of the present inven tion to provide for a cooking system which provides for advances in terms of handling and user comfort at a good cost-benefit ra tio.

In accordance with the present invention the above object is solved by a cooking system as it is defined in claim 1 and fur ther by a method as it is defined in claim 7.

In the cooking system and the corresponding method suggested herein the oven is configured to automatically provide for a picture recognition which is triggered by the closing of the oven door, wherein in contrast to known systems the picture recognition is configured as rather basic picture recognition which merely aims at recognizing whether the oven cavity is empty or might contain a food item. Such basic picture recogni tion does not require sophisticated computation and hence can be implemented with comparable simple components both in terms of hardware and software. Such basic picture recognition thus can be implemented in a fast and effective manner at moderate costs.

The further detailed picture recognition which then is performed to identify the kind of food to be processed, which may include an evaluation of various parameters of the food being processed, such as size, shape, surface structure, color, etc., which pa rameters also may change during the cooking process, such as the degree of browning of the food, such detailed picture recogni tion is not performed at the oven but in an external computing means, for example a cloud-based server system.

Performing the detailed picture recognition in an external com puting means which is not restricted to the inherent constraints of a stand-alone device allows achieving much more accurate re sults at a much higher speed and yet at lower costs. Thus, if the external computing means is for example a central server, such server not only can be provided with much higher computa tion capacity but further can be configured to have access to a large database of pictures of food of various kinds and at vari ous stages of processing. Furthermore, particularly when the ex ternal computing means is configured as an adaptive self-learn ing system, such as a neural network system, configuring the ex ternal computing means as a server system allows the system to be trained by a plurality of users, which further increases the accuracy of the system.

Upon having performed a food recognition routine to determine, based on the picture data provided from the oven, food parame ters of the food being processed, such as the type of food, and the size, shape and condition thereof, the external computing means selects cooking parameters based on the determined food parameters and transmits the selected cooking parameters to the oven .

The external computing means, such as a cloud based server sys tem, thus can be provided with a database that contains cooking parameters that are specifically adapted for the particular oven, and which cooking parameters are designed to reflect the type and state of the particular food being processed, as well as personal preferences of a user, such as a degree of browning or the like.

Preferred embodiments of the present invention are defined in the dependent claims.

In preferred embodiments the communication means of the oven is adapted to communicate with a multi-purpose communication de vice, such as a mobile phone, a tablet, a laptop computer, or a personal computer, which provides for communication with the ex ternal computing means. In such embodiments the multi-purpose communication device can be employed as interfacing device so as to display settings and options to the user, and provide input to the oven and to the external computing means. In such embodi ments the interfacing particularly can be implemented as an ap plication that is executed on the multi-purpose communication device, so as to provide for a convenient means for the user to be presented with operation options, selected operating parame ters and any further appropriate information as regards opera tion of the device. Providing for an application that is run on a multi-purpose communication device provides a comfortable means for controlling the cooking process, which application further can easily be updated via the internet so as to further improve the software.

The external computing means can be configured to transmit food data and/or cooking parameters to the user for confirmation by the user before proceeding to the step of selecting cooking pa rameters based on the determined food parameters and/or the step of transmitting the selected cooking parameters to the oven. Thus, before proceeding further with the operation program, the user may be presented with the determined food data, such as the type of food as determined in the food recognition process, cooking parameters that have been determined based on the deter mined type of food, which cooking parameters may comprise oper ating parameters of the oven, such as heating modes, temperature levels, durations, steam addition, and the like, or preparation parameters for the determined type of food such as a degree of cooking, such as rare or well-done, a degree of browning, a pre determined change in size, etc. While the operation of the device can be configured to be com pletely directed via an external device, such as the above men tioned multipurpose communication device, in preferred embodi ments the oven comprises an interface which is configured for communication with the external computing means and/or the multi-purpose communication device.

Such oven interface can be configured, for example, to start a cooking process upon transmittal of selected cooking parameters to the oven, to assist in a configuration process, to initiate program routines, such as service operations, etc.

In preferred embodiments the external computing means is a cloud-based system, i.e. a remote server system that is accessi ble via the internet and which is configured to communicate with a plurality of users. In contrast to computation means that are provided in the oven as such, a cloud-based system basically is not restricted to any constraints in terms of computing capacity and data volume and hence allows implementing a much more accu rate and fast picture recognition as a stand-alone device that is provided at the oven as such.

In preferred embodiments of the inventive method for operating a food processing oven there is provided, after the step of per forming a food recognition routine on the picture data to deter mine food data, a step of transmitting food data to the user for confirmation before proceeding to the step of selecting cooking parameters based on the determined food data.

In such embodiments the user thus is presented with an interme diate result of the overall evaluation process, such as the type of food to be processed, or suggested operating parameters, be fore these operating parameters are transmitted to the oven. Similarly, the method of the present invention further may com prise a step of transmitting to the user a selection of cooking options and/or optional cooking parameters which is performed after the step of performing a food recognition routine on the picture data to determine food data, so as to first ask the user for confirmation and/or selection of further optional parameters before proceeding with selecting cooking parameters and trans mitting the selected cooking parameters to the oven.

In order to allow the user to check the selected program and op tional settings before starting a cooking operation, the method, after the step of transmitting the selected cooking parameters to the oven, may comprise a step of displaying the selected cooking parameters, wherein the method only proceeds to the step of operating the oven based on the transmitted cooking parame ters upon confirmation by the user. Such confirmation by the user can be effected, for example, by providing a "Start" button either at the oven or at the multi-purpose communication device.

Preferred embodiments of the present invention are described be low by reference of the drawings in which:

Figure 1 is a schematic perspective view of a domestic oven ac cording to an embodiment of the present invention,

Figure 2 illustrates a schematic process diagram of an embodi ment of a method in accordance with the present invention, and

Figure 3 illustrates the process diagram of Figure 3 as a flow diagram. Figure 1 illustrates a schematic perspective view of a domestic cooking oven 10 which comprises a housing 12, an oven cavity 14 inside the housing 12, an oven door 16 for closing a front side of the oven cavity 14, and a front panel 22 which is arranged above the oven door 16. In the front panel 22 there are provided control and display elements for operation of the oven, such as a touchpad panel 24 and a control button 26.

The oven door 16 comprises a door window 18 arranged in a cen tral portion of the oven door 16. A door handle 20 is attached at an outer side of the oven door 16 above the door window 18.

In a central region of the handle 20 there is provided a camera 28 having a camera lens at the rear side of the handle, which camera lens is arranged to have a viewing angle which covers ba sically the entire oven cavity 14.

While a camera or another optical sensor for monitoring the in terior of the oven cavity 14 alternatively could be located within the oven cavity, providing the camera within the handle is advantageous because in this manner the camera is not exposed to the harsh environment of the oven atmosphere which may in volve high temperatures and high humidity, but which also con tains oven fumes and food particles or droplets.

Figure 2 illustrates a schematic process diagram for operation of the oven 10 shown in Figure 1, which process diagram is il lustrated in Figure 3 as a flow diagram

In the embodiment of the method of the present invention illus trated in Figures 2 and 3, the method starts with the user 30 opening the oven door 16 so as to put a food item into the oven cavity 14 (step 1) . The closing of the door is sensed by a door sensor (step 2), such as a switch which is provided in the door frame so as to be contacted by the door when the door is closed, or which is associated to the hinge to detect from the position ing of the hinge when the door is in the fully closed position.

Upon sensing a closing of the door, the camera 28 in the door handle 20 is activated to obtain picture data from within the oven cavity 14 (step 3) . The obtained picture data then is eval uated in a processing unit of the oven 10 so as to determine by some basic picture recognition routine that the oven cavity 14 no longer is empty (step 4) . In an exemplary embodiment, such basic picture recognition routine may comprise comparing a pre stored picture of the empty oven cavity with the picture data obtained after closing the door, wherein it is determined that the oven cavity may contain a food item if the compared picture data does not correspond to each other. The software for con ducting the basic picture evaluation routine can be stored in the memory of the oven controller or in an additional memory that is provided for such purpose.

In case that it is determined that the oven cavity no longer is empty the method proceeds to step 5, in which the obtained pic ture data is transmitted to an external computing means 32, which in Figure 2 is illustrated as a cloud-based server system which is accessible via internet. The obtained picture data is transmitted together with data for identifying the type of oven, such as a type name or model number. The necessary data for identifying the type of oven also can be stored at the cloud- based server system, such as by a registration process which the users performs once upon installing the oven. In such latter case the server can retrieve the required data based on the identification of the user, which identification anyway has to be provided so that the server can respond to the user. Should it be determined during step 4 that the oven cavity 14 still is empty, then the program returns to step 1, i.e. waits for a detection of the closing of the door.

The external computing means 32 processes the picture data ob tained from the oven in a picture recognition routine so as to determine food data including the type of food and various char acteristics such as the size, volume, color, surface texture, etc. of the food (step 6) . Based on the determined food charac teristics and optionally based on settings or options selected by the user, the external computing means selects from a data base operating parameters that specifically are designed for processing the particular food item in the particular oven (step 7) .

The operating parameters that have been determined by the exter nal computing means then are transmitted to the user, which in the process scheme illustrated in Figure 2 is done with the aid of a mobile communication device 34, which in Figure 2 is illus trated as a tablet computer. The data is displayed on the tablet computer wherein optionally also a notification can be output, such as an acoustic tone, so as to alert the user that data has been transmitted (step 8) .

The user now can check the displayed data so as to confirm that the food type and the suggested settings are correct, or to change settings or select additional options (step 9) . The thus confirmed or adapted selections then are transmitted to the ex ternal computing means (step 10), which transmits corresponding operating parameters to the oven (step 11) . The oven then can be operated in accordance with the transmitted operating parameters (step 13) , either directly upon receipt of the operating parameters, or upon the user providing again for some confirmation that the operation of the oven 10 is to be started, such as by pressing the control button 26 which in this case acts as start button for the device (step 12) .

It is to be noted that whereas in Figure 2 an embodiment is il lustrated in which the oven directly communicates with the ex ternal computing means, i.e. the cloud-based server system, such communication usually will be effected via a network interface, such as a home communication system with which the oven communi cates, such as via a LAN or WLAN connection. In a preferred em bodiment of the method and system suggested herein, the communi cation between the oven and the external computing means is ef fected via a multipurpose communication device, in Figure 2 the tablet computer, which thus is operated not only as a user in terface bus also as a communication interface for the oven.

Furthermore, whereas in the embodiment described above the de termination of the operating parameters for conducting a cooking process in conformity with the recognized food and optional user preferences is done in the external computing means, this also could be done at least in part by the multipurpose communication deice, i.e. the tablet computer illustrated in Figure 2, which communicates with the external computing means. For example, whereas the detailed picture recognition may be effected in a cloud based served which is used for providing picture recogni tion for multiple users, the display and selection of further options, such as the suggesting of different recipes for the de tected food or the display of further options for the cooking process, such as the degree of cooking, timer functions, warming functions to be conducted following a cooking process, could be done via an application that is run on the tablet computer, and which uses data that is stored on the tablet computer or which is obtained from the internet, such as from the external compu- ting means.

Whereas in the above described embodiment the food data, oven parameters, and optional settings, is displayed at the tablet computer, such data also can be displayed directly at the touch- pad panel 24 of the oven, which also can be used for displaying the recognized food type, recipes, suggested or selected oven parameters, and for making user selections, such as by display ing a list of options from which the user may chose.

Reference signs

10 oven

12 housing

14 oven cavity

16 oven door

18 door window

20 door handle

22 front panel

24 touchpad panel

26 control button

28 camera

30 user

32 external computing means 34 mobile communication device

Claims

1. A cooking system, comprising:

(a) an oven (10) having an oven cavity (14) that is acces sible via an oven door (16); and

(b) an external computing means (32);

wherein the oven (10) is provided with

(al) at least one optical sensor (28) for obtaining picture data of the oven cavity (14);

(a2) means for detecting door closing and triggering the op tical sensor to obtain picture data;

(a3) processing means for performing a basic picture evalua tion to determine that the oven cavity (14) no longer is empty;

(a4) communication means for providing picture data to the external computing means (32);

wherein the external computing means (32) is adapted to

(bl) perform a food recognition routine to determine food parameters based on the picture data provided by the communication means;

(b2) select cooking parameters based on the determined food parameters; and

(b3) transmit the selected cooking parameters to the oven (10) .

2. The cooking system of claim 1, wherein the communication

means of the oven (10) is adapted to communicate with a multi-purpose communication device (34) which provides for communication with the external computing means (32) .

3. The cooking system of claim 1, wherein the multi-purpose com munication device (34) comprises a mobile phone, a tablet, a laptop computer, or a personal computer.

4. The cooking system of any of the preceding claims, wherein the external computing means (32) is configured to transmit food data and/or cooking parameters to the user (30) for con firmation by the user before proceeding to step (b2) and/or step (b3) .

5. The cooking system of any of the preceding claims, wherein the oven (10) comprises an interface configured for communi cation with the external computing means (32) and/or the multi-purpose communication device (34).

6. The cooking system of any of the preceding claim 1, wherein the external computing means (32) is a cloud-based system.

7. Method for operating a food processing oven (10) having an oven cavity (14) that is accessible via an oven door (16), at least one optical sensor for obtaining picture data from the oven cavity, and means for communicating with an external computing means (32), the method comprising:

(a) monitoring closing of the door (16);

(b) upon detecting closing of the door (16), obtaining pic ture data from within the oven cavity (14);

(c) processing the picture data to determine whether the oven cavity (14) is empty;

(d) upon determining that the oven cavity (14) is not empty, transmitting the picture data to the external computing means ( 32 ) ;

(e) performing a food recognition routine on the picture data to determine food data;

(f) selecting cooking parameters based on the determined food data;

(g) transmitting the selected cooking parameters to the

oven (10) ; and (h) operating the oven (10) based on the transmitted cook ing parameters .

8. The method of claim 7, further comprising after step (e) the step of transmitting food data to the user (30) for confirma tion before proceeding to step (f) .

9. The method of claim 7 or 8, further comprising after step (e) the step of transmitting to the user (30) a selection of cook- ing options and/or optional cooking parameters and proceeding to step (f) upon confirmation and/or selection by the user.

10. The method of any one of claims 7 to 9, further comprising after step (g) the step of displaying the selected cooking parameters and proceeding to step (h) upon confirmation by the user (30) .