CN110598678A - Control method and control device of cooking equipment, cooking equipment and storage medium - Google Patents
Control method and control device of cooking equipment, cooking equipment and storage medium Download PDFInfo
- Publication number
- CN110598678A CN110598678A CN201910963365.7A CN201910963365A CN110598678A CN 110598678 A CN110598678 A CN 110598678A CN 201910963365 A CN201910963365 A CN 201910963365A CN 110598678 A CN110598678 A CN 110598678A
- Authority
- CN
- China
- Prior art keywords
- container
- image
- food
- cooking
- radius
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01G—WEIGHING
- G01G15/00—Arrangements for check-weighing of materials dispensed into removable containers
- G01G15/02—Arrangements for check-weighing of materials dispensed into removable containers with provision for adding or removing a make-up quantity of material to obtain the desired net weight
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/60—Analysis of geometric attributes
- G06T7/62—Analysis of geometric attributes of area, perimeter, diameter or volume
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/10—Terrestrial scenes
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Multimedia (AREA)
- Geometry (AREA)
- Image Analysis (AREA)
- Investigating Or Analysing Materials By Optical Means (AREA)
Abstract
The invention provides a control method and a control device of cooking equipment, the cooking equipment and a storage medium. The control method of the cooking equipment comprises the following steps: acquiring an image in a cooking cavity of a cooking device; and identifying the image, and acquiring the volume of the container in the cooking cavity and the material of the container. The technical scheme of the invention carries out image recognition on the container in the cooking equipment, can automatically and effectively extract the characteristics of the container, and improves the reliability and accuracy of container recognition.
Description
Technical Field
The invention relates to the technical field of cooking equipment, in particular to a control method and a control device of the cooking equipment, the cooking equipment and a storage medium.
Background
Most of image recognition schemes in the related art are models based on deep learning training, recognized objects are obtained through big data training, and the recognized objects are all large and specific models. It has the following disadvantages: (1) because the large models are aimed at, the algorithm is complex, online networking processing is required, and the interaction time is long; (2) since specific models are targeted, reliability and accuracy in identifying the container in the cooking device by using other identification models have influence on specific regional environments.
Disclosure of Invention
The present invention is directed to solving at least one of the problems of the prior art or the related art.
To this end, an aspect of the present invention is to propose a control method of a cooking apparatus.
Another aspect of the present invention is to provide a control apparatus of a cooking apparatus.
Yet another aspect of the present invention is to provide a cooking apparatus.
Yet another aspect of the present invention is to provide a computer-readable storage medium.
In view of the above, according to an aspect of the present invention, there is provided a control method of a cooking apparatus, including: acquiring an image in a cooking cavity of a cooking device; and identifying the image, and acquiring the volume of the container in the cooking cavity and the material of the container.
The invention provides a control method of cooking equipment, wherein a container is placed in a cooking cavity of the cooking equipment, an image of the cooking cavity of the cooking equipment is obtained, the container is displayed in the image, and the image is identified to obtain the volume of the container and the material of the container. The technical scheme of the invention carries out image recognition on the container in the cooking equipment, can automatically and effectively extract the characteristics of the container, and improves the reliability and accuracy of container recognition.
The control method of the cooking device according to the present invention may further have the following technical features:
in the above technical solution, the method further comprises: acquiring the total weight of the container and food contained in the container; acquiring the density of the container according to the material of the container, and calculating the weight of the container by using the density of the container and the volume of the container; the weight of the food is calculated from the total weight and the weight of the container.
In the technical scheme, the weight of the container is obtained through the extracted characteristics of the container, and then the weight of the food is calculated according to the measured total weight of the container and the food, so that the method is more beneficial to independently obtaining the real weight of the food, and the accuracy of intelligently identifying the real weight of the food is further improved.
In any of the above technical solutions, the step of identifying the image and obtaining the volume of the container in the cooking cavity and the material of the container specifically includes: performing background filtering on the image by using the stored empty cooking cavity image; identifying the image to obtain the center point of the container, the radius of the container and the color of the container; acquiring the height of the container; calculating the area of the container according to the center point of the container and the radius of the container, and calculating the volume of the container according to the area of the container and the height of the container; the material of the container is determined according to the color of the container and the stored container use information.
In the technical scheme, the image of the empty cooking cavity is stored in advance, the background in the image is separated from the container by using a picture subtraction algorithm, and the background is effectively filtered. And identifying the image without the background, acquiring the center point of the container, the radius of the container and the color of the container, acquiring the height of the container, and calculating the volume of the container according to the center point of the container, the radius of the container and the height of the container. The material of the container is determined according to the color of the container and the stored container use information, wherein the stored container use information is the use habit information of a user, and the accuracy of obtaining the material of the container is improved by combining the two information.
It should be noted that the height of the container can be obtained by pre-storing historical height data, or can be obtained by an image collected by an image collecting device arranged on the side wall of the cooking cavity.
In any of the above technical solutions, after the step of identifying the image and obtaining the center point of the container, the radius of the container, and the color of the container, the method further includes: gap filling processing is carried out on food contained in the container in the image; identifying the image after gap filling processing to obtain the center point of the food and the radius of the food; calculating food contour information according to the center point of the food and the radius of the food; and according to the food contour information, removing the image of the food from the image.
According to the technical scheme, if food is contained in the container, the container and the food are displayed in the image after the background is removed, and the food in the image is further removed. The method comprises the steps of preprocessing food, filling gaps, considering that the food is different in shape, sliced or sliced, and different in placement, and the food can have gaps, so that influence can be caused when the food is extracted. Then, identifying background characteristic points and foreground characteristic points, identifying container edges according to the center point of the container and the radius of the container, wherein the container edges are the background characteristic points, identifying the image to obtain the center point of the food and the radius of the food, and calculating food contour information according to the center point of the food and the radius of the food, wherein the food contour information is the foreground characteristic points. And finally, matting by utilizing an Onecut algorithm, removing food, obtaining an image of only the remaining container, and verifying the central point of the container, the radius of the container and the color of the container again by identifying the image of only the remaining container, so that the identification accuracy is improved.
In any of the above technical solutions, before the step of identifying the image and obtaining the center point of the container, the radius of the container, and the color of the container, the method further includes: the container in the image is corrected.
In this embodiment, the container in the image is corrected, for example, by correcting the container edge by the picture color near the container edge or by correcting the container edge by the rough shape of the container, and accurate information of the container is further acquired.
In any of the above technical solutions, before the step of identifying the image and obtaining the volume of the container and the material of the container in the cooking cavity, the method further includes: and carrying out noise point removing processing and lamplight interference removing processing on the image.
In the technical scheme, under the environment with lamplight, a white balance algorithm is firstly utilized to process an image, so that the influence caused by lamplight errors is reduced; and then, the image is processed by utilizing a Gaussian filtering algorithm, so that image noise introduced by a camera circuit is reduced.
According to another aspect of the present invention, a control device of a cooking apparatus is provided, comprising a memory and a processor, wherein the memory stores a computer program, and the processor is configured to implement the following steps when executing the computer program: acquiring an image in a cooking cavity of a cooking device; and identifying the image, and acquiring the volume of the container in the cooking cavity and the material of the container.
The invention provides a control device of a cooking device, wherein a container is placed in a cooking cavity of the cooking device, a processor acquires an image in the cooking cavity of the cooking device, the container is displayed in the image, and the image is identified to obtain the volume of the container and the material of the container. The technical scheme of the invention carries out image recognition on the container in the cooking equipment, can automatically and effectively extract the characteristics of the container, and improves the reliability and accuracy of container recognition.
The control device of the cooking apparatus according to the present invention may further have the following technical features:
in the above technical solution, the processor is further configured to, when executing the computer program: acquiring the total weight of the container and food contained in the container; acquiring the density of the container according to the material of the container, and calculating the weight of the container by using the density of the container and the volume of the container; the weight of the food is calculated from the total weight and the weight of the container.
In the technical scheme, the weight of the container is obtained through the extracted characteristics of the container, and then the weight of the food is calculated according to the measured total weight of the container and the food, so that the method is more beneficial to independently obtaining the real weight of the food, and the accuracy of intelligently identifying the real weight of the food is further improved.
In any of the above technical solutions, the processor identifies the image, obtains the volume of the container in the cooking cavity and the material of the container, and specifically includes: performing background filtering on the image by using the stored empty cooking cavity image; identifying the image to obtain the center point of the container, the radius of the container and the color of the container; acquiring the height of the container; calculating the area of the container according to the center point of the container and the radius of the container, and calculating the volume of the container according to the area of the container and the height of the container; the material of the container is determined according to the color of the container and the stored container use information.
In the technical scheme, the image of the empty cooking cavity is stored in advance, the background in the image is separated from the container by using a picture subtraction algorithm, and the background is effectively filtered. And identifying the image without the background, acquiring the center point of the container, the radius of the container and the color of the container, acquiring the height of the container, and calculating the volume of the container according to the center point of the container, the radius of the container and the height of the container. The material of the container is determined according to the color of the container and the stored container use information, wherein the stored container use information is the use habit information of a user, and the accuracy of obtaining the material of the container is improved by combining the two information.
It should be noted that the height of the container can be obtained by pre-storing historical height data, or can be obtained by an image collected by an image collecting device arranged on the side wall of the cooking cavity.
In any of the above technical solutions, the processor, when being configured to execute the computer program, further implements: gap filling processing is carried out on food contained in the container in the image; identifying the image after gap filling processing to obtain the center point of the food and the radius of the food; calculating food contour information according to the center point of the food and the radius of the food; and according to the food contour information, removing the image of the food from the image.
According to the technical scheme, if food is contained in the container, the container and the food are displayed in the image after the background is removed, and the food in the image is further removed. The method comprises the steps of preprocessing food, filling gaps, considering that the food is different in shape, sliced or sliced, and different in placement, and the food can have gaps, so that influence can be caused when the food is extracted. Then, identifying background characteristic points and foreground characteristic points, identifying container edges according to the center point of the container and the radius of the container, wherein the container edges are the background characteristic points, identifying the image to obtain the center point of the food and the radius of the food, and calculating food contour information according to the center point of the food and the radius of the food, wherein the food contour information is the foreground characteristic points. And finally, matting by utilizing an Onecut algorithm, removing food, obtaining an image of only the remaining container, and verifying the central point of the container, the radius of the container and the color of the container again by identifying the image of only the remaining container, so that the identification accuracy is improved.
In any of the above technical solutions, the processor, when being configured to execute the computer program, further implements: the container in the image is corrected.
In this embodiment, the container in the image is corrected, for example, by correcting the container edge by the picture color near the container edge or by correcting the container edge by the rough shape of the container, and accurate information of the container is further acquired.
In any of the above technical solutions, the processor, when being configured to execute the computer program, further implements: and carrying out noise point removing processing and lamplight interference removing processing on the image.
In the technical scheme, under the environment with lamplight, a white balance algorithm is firstly utilized to process an image, so that the influence caused by lamplight errors is reduced; and then, the image is processed by utilizing a Gaussian filtering algorithm, so that image noise introduced by a camera circuit is reduced.
According to still another aspect of the present invention, there is provided a cooking apparatus including: an image capture device configured to capture images within the cooking cavity; and the control device of the cooking equipment according to any one of the above technical schemes is connected with the image acquisition device.
According to the cooking equipment provided by the invention, the image acquisition device and the control device of the cooking equipment in any technical scheme are arranged in the cooking cavity, so that all beneficial technical effects of the control device of the cooking equipment in any technical scheme can be realized, and the description is omitted.
In the above technical solution, the method further comprises: a weight measuring device configured to measure a total weight of the container and the food.
In the technical scheme, the weight of the container is obtained through the extracted characteristics of the container, and then the weight of the food is calculated according to the measured total weight of the container and the food, so that the method is more beneficial to independently obtaining the real weight of the food, and the accuracy of intelligently identifying the real weight of the food is further improved.
According to a further aspect of the present invention, a computer-readable storage medium is proposed, on which a computer program is stored, which computer program, when being executed by a processor, realizes the steps of the control method of a cooking apparatus as in any of the above-mentioned technical solutions.
The present invention provides a computer-readable storage medium, wherein a computer program is executed by a processor to implement the steps of the control method of the cooking device according to any one of the above technical solutions, and therefore the computer-readable storage medium includes all the beneficial effects of the control method of the cooking device according to any one of the above technical solutions.
Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention.
Drawings
The above and/or additional aspects and advantages of the present invention will become apparent and readily appreciated from the following description of the embodiments, taken in conjunction with the accompanying drawings of which:
fig. 1 shows a schematic flow diagram of a method for identifying a cooking appliance container according to a first embodiment of the invention;
fig. 2 shows a schematic flow diagram of a method for identifying a cooking appliance container according to a second embodiment of the invention;
fig. 3 shows a flow chart of a method of identifying a cooking appliance container according to a third embodiment of the invention;
fig. 4 shows a schematic flow diagram of a method for identifying a cooking appliance container according to a fourth embodiment of the invention;
fig. 5 shows a schematic flow chart of an identification method of a cooking appliance container of a fifth embodiment of the invention;
fig. 6 shows a flow chart of an identification method of a cooking appliance container of a sixth embodiment of the invention;
fig. 7 shows a schematic block diagram of a control device of a cooking apparatus of an embodiment of the present invention;
fig. 8 shows a schematic block diagram of a cooking apparatus of an embodiment of the present invention.
Detailed Description
In order that the above objects, features and advantages of the present invention can be more clearly understood, a more particular description of the invention will be rendered by reference to the appended drawings. It should be noted that the embodiments of the present invention and features of the embodiments may be combined with each other without conflict.
In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present invention, however, the present invention may be practiced in other ways than those specifically described herein, and therefore the scope of the present invention is not limited to the specific embodiments disclosed below.
Embodiments of the first aspect of the present invention provide a control method of a cooking apparatus, which is described in detail by the following embodiments.
First embodiment, fig. 1 is a flowchart illustrating a method for controlling a cooking apparatus according to a first embodiment of the present invention. Wherein, the method comprises the following steps:
102, acquiring an image in a cooking cavity of cooking equipment;
and 104, identifying the image, and acquiring the volume of the container in the cooking cavity and the material of the container.
The invention provides a control method of cooking equipment, wherein a container is placed in a cooking cavity of the cooking equipment, an image of the cooking cavity of the cooking equipment is obtained, the container is displayed in the image, and the image is identified to obtain the volume of the container and the material of the container. The technical scheme of the invention carries out image recognition on the container in the cooking equipment, can automatically and effectively extract the characteristics of the container, and improves the reliability and accuracy of container recognition.
Step 104, identifying the image, and acquiring the volume of the container in the cooking cavity and the material of the container, wherein the step specifically comprises the following steps: performing background filtering on the image by using the stored empty cooking cavity image; identifying the image to obtain the center point of the container, the radius of the container and the color of the container; acquiring the height of the container; calculating the area of the container according to the center point of the container and the radius of the container, and calculating the volume of the container according to the area of the container and the height of the container; the material of the container is determined according to the color of the container and the stored container use information.
In this embodiment, the image of the empty cooking cavity is saved in advance, and the background in the image is separated from the container by using a picture subtraction algorithm, so that the background is effectively filtered. And identifying the image without the background, acquiring the center point of the container, the radius of the container and the color of the container, acquiring the height of the container, and calculating the volume of the container according to the center point of the container, the radius of the container and the height of the container. The material of the container is determined according to the color of the container and the stored container use information, wherein the stored container use information is the use habit information of a user, and the accuracy of obtaining the material of the container is improved by combining the two information.
It should be noted that the height of the container can be obtained by pre-storing historical height data, or can be obtained by an image collected by an image collecting device arranged on the side wall of the cooking cavity.
Second embodiment, fig. 2 is a flowchart illustrating a method for controlling a cooking apparatus according to a second embodiment of the present invention. Wherein, the method comprises the following steps:
step 202, acquiring an image in a cooking cavity of the cooking equipment;
step 204, identifying an image, and acquiring the volume of a container in the cooking cavity and the material of the container;
step 206, acquiring the total weight of the container and the food in the container;
step 208, acquiring the density of the container according to the material of the container, and calculating the weight of the container by using the density of the container and the volume of the container;
the weight of the food is calculated 210 from the total weight and the weight of the container.
In the embodiment, the weight of the container is obtained through the extracted features of the container, and then the weight of the food is calculated according to the measured total weight of the container and the food, so that the method is more beneficial to independently obtaining the real weight of the food, and the accuracy of intelligently identifying the real weight of the food is further improved.
Third embodiment, fig. 3 is a flowchart illustrating a method for controlling a cooking apparatus according to a third embodiment of the present invention. Wherein, the method comprises the following steps:
step 302, acquiring an image in a cooking cavity of a cooking device;
step 304, performing background filtering on the image by using the stored empty cooking cavity image;
step 306, the container in the image is corrected. For example, the container edge is corrected by the picture color near the container edge, or the container edge is corrected by the rough shape of the container, so as to further acquire accurate information of the container;
step 308, identifying the image to obtain the center point of the container, the radius of the container and the color of the container; acquiring the height of the container; calculating the area of the container according to the center point of the container and the radius of the container, and calculating the volume of the container according to the area of the container and the height of the container; the material of the container is determined according to the color of the container and the stored container use information.
It should be noted that, after step 308, the method may further include: acquiring the total weight of the container and food contained in the container; acquiring the density of the container according to the material of the container, and calculating the weight of the container by using the density of the container and the volume of the container; the weight of the food is calculated from the total weight and the weight of the container.
Fourth embodiment, fig. 4 is a flowchart illustrating a control method of a cooking apparatus according to a fourth embodiment of the present invention. Wherein, the method comprises the following steps:
step 402, acquiring an image in a cooking cavity of a cooking device;
and step 404, performing denoising point processing and lamplight interference removing processing on the image. Under the environment with lamplight, the white balance algorithm is firstly utilized to process the image, so that the influence caused by lamplight errors is reduced; processing the image by using a Gaussian filtering algorithm, and reducing image noise introduced by a camera circuit;
step 406, performing background filtering on the image by using the stored empty cooking cavity image;
step 408, the container in the image is corrected. For example, the container edge is corrected by the picture color near the container edge, or the container edge is corrected by the rough shape of the container, so as to further acquire accurate information of the container;
step 410, identifying an image to obtain a center point of the container, a radius of the container and a color of the container; acquiring the height of the container; calculating the area of the container according to the center point of the container and the radius of the container, and calculating the volume of the container according to the area of the container and the height of the container; the material of the container is determined according to the color of the container and the stored container use information.
It should be noted that, after step 410, the method may further include: acquiring the total weight of the container and food contained in the container; acquiring the density of the container according to the material of the container, and calculating the weight of the container by using the density of the container and the volume of the container; the weight of the food is calculated from the total weight and the weight of the container.
Fifth embodiment, fig. 5 is a flowchart illustrating a control method of a cooking apparatus according to a fifth embodiment of the present invention. Wherein, the method comprises the following steps:
step 502, acquiring an image in a cooking cavity of a cooking device;
and step 504, performing noise point removal processing and lamplight interference removal processing on the image. Under the environment with lamplight, the white balance algorithm is firstly utilized to process the image, so that the influence caused by lamplight errors is reduced; processing the image by using a Gaussian filtering algorithm, and reducing image noise introduced by a camera circuit;
step 506, performing background filtering on the image by using the stored empty cooking cavity image;
step 508, the container in the image is corrected. For example, the container edge is corrected by the picture color near the container edge, or the container edge is corrected by the rough shape of the container, so as to further acquire accurate information of the container;
step 510, identifying an image to obtain a center point of a container and a radius of the container;
step 512, performing gap filling processing on food contained in the container in the image; identifying the image after gap filling processing to obtain the center point of the food and the radius of the food; calculating food contour information according to the center point of the food and the radius of the food; according to the food contour information, the center point of the container and the radius of the container, removing the image of the food from the image;
step 514, recognizing the image to acquire the center point, the radius of the container and the color of the container again;
step 516, acquiring the height of the container; calculating the area of the container according to the center point of the container and the radius of the container, and calculating the volume of the container according to the area of the container and the height of the container; the material of the container is determined according to the color of the container and the stored container use information.
In this embodiment, if the container contains food, the background-removed image shows the container and the food, and the food in the image is further removed. The method comprises the steps of preprocessing food, filling gaps, considering that the food is different in shape, sliced or sliced, and different in placement, and the food can have gaps, so that influence can be caused when the food is extracted. Then, identifying background characteristic points and foreground characteristic points, identifying container edges according to the center point of the container and the radius of the container, wherein the container edges are the background characteristic points, identifying the image to obtain the center point of the food and the radius of the food, and calculating food contour information according to the center point of the food and the radius of the food, wherein the food contour information is the foreground characteristic points. And finally, matting by utilizing an Onecut algorithm, removing food, obtaining an image of only the remaining container, and verifying the central point of the container, the radius of the container and the color of the container again by identifying the image of only the remaining container, so that the identification accuracy is improved.
It should be noted that, after step 516, the method may further include: acquiring the total weight of the container and food contained in the container; acquiring the density of the container according to the material of the container, and calculating the weight of the container by using the density of the container and the volume of the container; the weight of the food is calculated from the total weight and the weight of the container.
In a sixth embodiment, a method for identifying a container of a cooking device is provided, which enables to extract container characteristics including, but not limited to, shape, size, color, and material inside the cooking device. The weight sensor can be used for subsequent use, for example, the container characteristics are extracted, the container weight is obtained, the weight sensor is matched to obtain the total weight of the container and the food, the weight of the food is corrected, and the weight recognition accuracy can be improved. The image processing algorithm related to this embodiment is described below:
(1) and the background filtering algorithm is used for acquiring a cavity picture without food in advance, processing all pixels of the picture by using a pixel subtraction algorithm, and filtering out the background to obtain the container and the food.
(2) The container identification algorithm is based on an Onecut algorithm, but the traditional Onecut algorithm needs manual interaction to provide foreground points and background characteristic points of an image, and the matting and the target object extraction are realized on the basis. On the basis of an Onecut matting algorithm, the foreground characteristic points and the background characteristic points of the image are automatically calculated by combining the cavity characteristics, the use habits of users and other factors without manual intervention, so that the container contour can be automatically and effectively extracted.
(3) And the Gaussian filtering algorithm is linear smooth filtering, and is used for smoothing signals of a target image, reducing image noise introduced by a camera circuit and improving the accuracy of background removal.
(4) Erosion algorithm, erosion is a process of eliminating boundary points and shrinking boundaries inward to eliminate small and meaningless objects. Because the food shapes are different, the food is sliced or sliced, the placing is different, gaps are formed when the food is placed on a container, the gaps can be effectively filled through the pretreatment of a corrosion algorithm, and the accuracy of extracting the foreground characteristic points is further improved.
(5) And white balance algorithm, wherein white balance is an algorithm for adjusting and calibrating the color temperature of the image, and the color temperature of the image can be accurately restored. Since there are errors in the lighting environment in the cooking cavity of the cooking apparatus and color differences may be caused as the lamp is consumed, it is necessary to reduce the effect of image distortion due to background lighting factors using a white balance algorithm.
(6) And the container edge correction algorithm is used for preliminarily estimating the container edge through the container identification algorithm, then carrying out similarity judgment on the image information near the edge, and further correcting the edge to finish the whole container edge extraction process.
By utilizing the image processing algorithm, the algorithm is organically combined and corrected, the image in a specific environment (such as a cavity of a microwave oven) is automatically segmented, and the environment background and food are removed, so that the container image is extracted.
Fig. 6 shows a flow chart of an identification method of a cooking appliance container according to a sixth embodiment of the invention. Wherein, the method comprises the following steps:
step 602, pre-saving the cavity picture. Because the cavity environment is fixed, the cavity environment characteristics can be processed in advance. Firstly, starting a microwave oven, and under the environment with lamplight, firstly, utilizing a white balance algorithm to process so as to reduce the influence caused by lamplight errors; and then, reducing image noise introduced by a camera circuit by utilizing a Gaussian filtering algorithm, and then taking a photo of the cavity body.
Step 604, acquiring a picture of the cavity filled with food. The cavity is put in a plate filled with food, and then the picture is obtained.
Step 606, gaussian filtering to remove noise. And the Gaussian filtering algorithm is utilized to process the picture, so that the image noise is reduced.
Step 608, white balancing to remove light interference. And the white balance algorithm is utilized to process the picture, so that the interference of the light to the picture is reduced.
Step 610, separate the background from the dish containing the food using a pixel subtraction algorithm. The environment of the cavity is filtered by pixel subtraction, and through the step, the background can be effectively filtered, and the image of only the remaining plate and food is separated.
Step 612, the food is pre-treated to fill the voids. Considering that the food has different shapes, slices or strips and different placement, and the food may have gaps, which may affect the food extraction, the image must be corroded first, and the gaps between the food are filled by corrosion algorithm.
And step 614, identifying the background characteristic points required by the ONECUT algorithm. After filling, the conventional dish is mainly oval, and the food is placed in the middle of the dish. By combining the habits of the users, the shape of the plate is assumed to be elliptical, the center of the plate is calculated through an algorithm, and the long radius and the short radius of the plate can be calculated by using an elliptical calculation formula, so that the edge of the plate can be identified, and the edge of the plate is the background feature point to be input by the Onecut algorithm.
In step 616, foreground feature points required by the ONECUT algorithm are identified. The user habit is considered to be that food is placed in the middle of the plate, and a reasonable radius value is given by utilizing the habit and combining the circle center of the plate, so that the foreground characteristic points can be identified.
At step 618, the OneCut algorithm cuts out the plate. And (5) calculating cutout by utilizing the OneCut, removing food and cutting out a plate.
And step 620, edge correction. And matching and correcting the edge of the plate by using the picture information of the edge of the plate, and further acquiring the accurate edge information of the plate.
Step 622, the plate characteristics are obtained. The area of the plate is calculated according to the edge information and the central point of the plate, and the volume of the plate can be calculated by setting the proper height of the plate. The proper height of the dish can be set according to historical data, and the height of the dish can be obtained by shooting images through a camera arranged on the side edge of the cavity. The material of the dish can be deduced by utilizing the color of the dish and combining the use habit of the user.
The seventh embodiment is that the camera module is installed in the microwave oven (the installation position needs to reduce the shadow of the picture, and the installation position and the lamp tube of the microwave oven are installed on the same horizontal plane, so that the shadow area can be effectively reduced), the camera module is linked with the door closing through software control, and the picture is shot when the microwave oven is closed.
Factory setting: the microwave oven is powered on before leaving the factory, the camera module is controlled by software to photograph the inside of the cavity body, and after the pretreatment of the white balance algorithm and the Gaussian algorithm, the picture of the cavity body is stored in the camera module to complete the factory setting.
The user uses: after the equipment is electrified, the camera module is started; the user puts the plate containing food into cooking equipment, closes the door, and the camera module is automatic to shoot. After photographing, the shape and material of the dish are extracted by combining the identification method of the cooking equipment container. Calculating the weight of the plate according to the shape and the material of the plate, and obtaining the weight of the food by combining the total weight of the plate and the food measured by the weight sensor.
In a second aspect of the present invention, a control device of a cooking apparatus is provided, and fig. 7 shows a schematic block diagram of a control device 700 of a cooking apparatus according to an embodiment of the present invention. Wherein, the control device 700 of the cooking apparatus includes:
a memory 702 and a processor 704, the memory 702 having stored therein computer programs, the processor 704 for implementing when executing the computer programs: acquiring an image in a cooking cavity of a cooking device; and identifying the image, and acquiring the volume of the container in the cooking cavity and the material of the container.
The invention provides a control device 700 of a cooking device, wherein a container is placed in a cooking cavity of the cooking device, a processor 704 acquires an image of the cooking cavity of the cooking device, the container is displayed in the image, and the image is identified to obtain the volume of the container and the material of the container. The technical scheme of the invention carries out image recognition on the container in the cooking equipment, can automatically and effectively extract the characteristics of the container, and improves the reliability and accuracy of container recognition.
In the above embodiments, the processor 704, when executing the computer program, further implements: acquiring the total weight of the container and food contained in the container; acquiring the density of the container according to the material of the container, and calculating the weight of the container by using the density of the container and the volume of the container; the weight of the food is calculated from the total weight and the weight of the container.
In the embodiment, the weight of the container is obtained through the extracted features of the container, and then the weight of the food is calculated according to the measured total weight of the container and the food, so that the method is more beneficial to independently obtaining the real weight of the food, and the accuracy of intelligently identifying the real weight of the food is further improved.
In any of the above embodiments, the processor 704 identifies the image and obtains the volume of the container in the cooking cavity and the material of the container, which specifically includes: performing background filtering on the image by using the stored empty cooking cavity image; identifying the image to obtain the center point of the container, the radius of the container and the color of the container; acquiring the height of the container; calculating the area of the container according to the center point of the container and the radius of the container, and calculating the volume of the container according to the area of the container and the height of the container; the material of the container is determined according to the color of the container and the stored container use information.
In this embodiment, the image of the empty cooking cavity is saved in advance, and the background in the image is separated from the container by using a picture subtraction algorithm, so that the background is effectively filtered. And identifying the image without the background, acquiring the center point of the container, the radius of the container and the color of the container, acquiring the height of the container, and calculating the volume of the container according to the center point of the container, the radius of the container and the height of the container. The material of the container is determined according to the color of the container and the stored container use information, wherein the stored container use information is the use habit information of a user, and the accuracy of obtaining the material of the container is improved by combining the two information.
It should be noted that the height of the container can be obtained by pre-storing historical height data, or can be obtained by an image collected by an image collecting device arranged on the side wall of the cooking cavity.
In any of the above embodiments, the processor 704, when executing the computer program, further implements: gap filling processing is carried out on food contained in the container in the image; identifying the image after gap filling processing to obtain the center point of the food and the radius of the food; calculating food contour information according to the center point of the food and the radius of the food; and according to the food contour information, removing the image of the food from the image.
In this embodiment, if the container contains food, the background-removed image shows the container and the food, and the food in the image is further removed. The method comprises the steps of preprocessing food, filling gaps, considering that the food is different in shape, sliced or sliced, and different in placement, and the food can have gaps, so that influence can be caused when the food is extracted. Then, identifying background characteristic points and foreground characteristic points, identifying container edges according to the center point of the container and the radius of the container, wherein the container edges are the background characteristic points, identifying the image to obtain the center point of the food and the radius of the food, and calculating food contour information according to the center point of the food and the radius of the food, wherein the food contour information is the foreground characteristic points. And finally, matting by utilizing an Onecut algorithm, removing food, obtaining an image of only the remaining container, and verifying the central point of the container, the radius of the container and the color of the container again by identifying the image of only the remaining container, so that the identification accuracy is improved.
In any of the above embodiments, the processor 704, when executing the computer program, further implements: the container in the image is corrected.
In this embodiment, the container in the image is corrected, for example, by correcting the container edge by the picture color near the container edge, or by correcting the container edge by the rough shape of the container, and accurate information of the container is further acquired.
In any of the above embodiments, the processor 704, when executing the computer program, further implements: and carrying out noise point removing processing and lamplight interference removing processing on the image.
In the embodiment, under the environment with lamplight, the white balance algorithm is firstly utilized to process the image, so that the influence caused by lamplight errors is reduced; and then, the image is processed by utilizing a Gaussian filtering algorithm, so that image noise introduced by a camera circuit is reduced.
In a third aspect of the present invention, a cooking apparatus is provided, and fig. 8 shows a schematic block diagram of a cooking apparatus 800 according to an embodiment of the present invention. Wherein, this cooking apparatus 800 includes:
an image capture device 802, the image capture device 802 configured to capture images within the cooking cavity; and a control device 700 of the cooking apparatus according to any of the above embodiments, the control device 700 of the cooking apparatus is connected to the image capturing device 802.
In the cooking apparatus 800 provided by the present invention, the image capturing device 802 and the control device 700 of the cooking apparatus according to any of the above embodiments are disposed in the cooking cavity, so that all beneficial technical effects of the control device 700 of the cooking apparatus according to any of the above embodiments can be achieved, and further description is omitted.
In the above embodiment, the method further includes: a weight measuring device 804, the weight measuring device 804 configured to measure a total weight of the container and the food.
In the embodiment, the weight of the container is obtained through the extracted features of the container, and then the weight of the food is calculated according to the measured total weight of the container and the food, so that the method is more beneficial to independently obtaining the real weight of the food, and the accuracy of intelligently identifying the real weight of the food is further improved.
An embodiment of the fourth aspect of the present invention proposes a computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, implements the steps of the control method of the cooking apparatus according to any one of the above embodiments.
The present invention provides a computer-readable storage medium, which when executed by a processor implements the steps of the control method of a cooking appliance as in any one of the above embodiments, and therefore includes all the advantageous effects of the control method of a cooking appliance as in any one of the above embodiments.
In the description herein, the terms "first" and "second" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance unless explicitly stated or limited otherwise; the terms "connected," "mounted," "secured," and the like are to be construed broadly and include, for example, fixed connections, removable connections, or integral connections; may be directly connected or indirectly connected through an intermediate. The specific meanings of the above terms in the present invention can be understood by those skilled in the art according to specific situations.
In the description herein, the description of the terms "one embodiment," "some embodiments," "specific embodiments," etc., means that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the invention. In this specification, the schematic representations of the terms used above do not necessarily refer to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.
The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention, and various modifications and changes may be made by those skilled in the art. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims (15)
1. A control method of a cooking apparatus, characterized by comprising:
acquiring an image in a cooking cavity of the cooking device;
and identifying the image, and acquiring the volume of the container in the cooking cavity and the material of the container.
2. The control method of a cooking apparatus according to claim 1, further comprising:
obtaining the total weight of the container and the food contained in the container;
acquiring the density of the container according to the material of the container, and calculating the weight of the container by using the density of the container and the volume of the container;
calculating a weight of the food item based on the total weight and the weight of the container.
3. The method for controlling a cooking apparatus according to claim 1, wherein the step of identifying the image and obtaining the volume of the container in the cooking cavity and the material of the container comprises:
background filtering the image using the stored empty cooking cavity image;
identifying the image to obtain a center point of the container, a radius of the container and a color of the container;
acquiring the height of the container;
calculating an area of the container from a center point of the container, a radius of the container, and a volume of the container from the area of the container and a height of the container;
and determining the material of the container according to the color of the container and the stored container use information.
4. The method for controlling a cooking apparatus according to claim 3, further comprising, after the step of recognizing the image, obtaining a center point of the container, a radius of the container, and a color of the container:
performing gap filling processing on food contained in the container in the image;
identifying the image after the gap filling processing to obtain a center point of the food and a radius of the food;
calculating food contour information according to the center point of the food and the radius of the food;
and according to the food contour information, removing the image of the food from the image.
5. The method of controlling a cooking apparatus according to claim 3, further comprising, before the step of recognizing the image to obtain the center point of the container, the radius of the container, and the color of the container:
correcting the container in the image.
6. The method for controlling a cooking apparatus according to any one of claims 1 to 4, further comprising, before the step of identifying the image, and obtaining the volume of the container and the material of the container in the cooking cavity:
and carrying out noise point removal processing and lamplight interference removal processing on the image.
7. A control device for a cooking apparatus, comprising a memory having a computer program stored therein and a processor configured to implement, when executing the computer program:
acquiring an image in a cooking cavity of the cooking device;
and identifying the image, and acquiring the volume of the container in the cooking cavity and the material of the container.
8. The control device of the cooking apparatus of claim 7, wherein the processor, when executing the computer program, further implements:
obtaining the total weight of the container and the food contained in the container;
acquiring the density of the container according to the material of the container, and calculating the weight of the container by using the density of the container and the volume of the container;
calculating a weight of the food item based on the total weight and the weight of the container.
9. The control device of the cooking apparatus according to claim 7, wherein the processor identifies the image and obtains a volume of the container in the cooking cavity and a material of the container, and specifically comprises:
background filtering the image using the stored empty cooking cavity image;
identifying the image to obtain a center point of the container, a radius of the container and a color of the container;
acquiring the height of the container;
calculating an area of the container from a center point of the container, a radius of the container, and a volume of the container from the area of the container and a height of the container;
and determining the material of the container according to the color of the container and the stored container use information.
10. The control device of the cooking apparatus of claim 9, wherein the processor, when executing the computer program, further implements:
performing gap filling processing on food contained in the container in the image;
identifying the image after the gap filling processing to obtain a center point of the food and a radius of the food;
calculating food contour information according to the center point of the food and the radius of the food;
and according to the food contour information, removing the image of the food from the image.
11. The control device of the cooking apparatus of claim 9, wherein the processor, when executing the computer program, further implements:
correcting the container in the image.
12. The control device of a cooking apparatus according to any one of claims 7 to 11, wherein the processor is further configured to, when executing the computer program:
and carrying out noise point removal processing and lamplight interference removal processing on the image.
13. A cooking apparatus, characterized by comprising:
an image capture device configured to capture images within the cooking cavity; and
the control device of the cooking apparatus according to any one of claims 7 to 12, the control device of the cooking apparatus being connected to the image capturing device.
14. The cooking apparatus of claim 13, further comprising:
a weight measuring device configured to measure a total weight of the container and the food.
15. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of a method of controlling a cooking apparatus according to any one of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910963365.7A CN110598678A (en) | 2019-10-11 | 2019-10-11 | Control method and control device of cooking equipment, cooking equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910963365.7A CN110598678A (en) | 2019-10-11 | 2019-10-11 | Control method and control device of cooking equipment, cooking equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110598678A true CN110598678A (en) | 2019-12-20 |
Family
ID=68866457
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910963365.7A Pending CN110598678A (en) | 2019-10-11 | 2019-10-11 | Control method and control device of cooking equipment, cooking equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110598678A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113349645A (en) * | 2020-03-05 | 2021-09-07 | 佛山市云米电器科技有限公司 | Method for acquiring image of article inside oven, oven system, baking equipment and storage medium |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2013188298A (en) * | 2012-03-13 | 2013-09-26 | Asahi Kasei Home Products Kk | Lid material for microwave oven cooking |
| CN103347111A (en) * | 2013-07-27 | 2013-10-09 | 青岛歌尔声学科技有限公司 | Intelligent mobile electronic equipment with size and weight estimation function |
| US20140220196A1 (en) * | 2013-02-06 | 2014-08-07 | Restaurant Technology, Inc. | Temperature Controlled Cooking Device, Control System, and Method |
| CN103983334A (en) * | 2014-05-20 | 2014-08-13 | 联想(北京)有限公司 | Information processing method and electronic equipment |
| CN106372198A (en) * | 2016-08-31 | 2017-02-01 | 乐视控股（北京）有限公司 | Data extraction method based on image recognition technology and mobile terminal thereof |
| CN107944391A (en) * | 2017-11-24 | 2018-04-20 | 广东美的厨房电器制造有限公司 | The definite method, apparatus and storage medium of a kind of food weight |
| CN108140328A (en) * | 2015-09-09 | 2018-06-08 | 菲特利公司 | Using food image identification come the system and method for trophic analysis |
| CN108171722A (en) * | 2017-12-26 | 2018-06-15 | 广东美的厨房电器制造有限公司 | Image extraction method, device and cooking apparatus |
| CN109753977A (en) * | 2017-11-01 | 2019-05-14 | 菜鸟智能物流控股有限公司 | Method, terminal, server and system for measuring article size information |
-
2019
- 2019-10-11 CN CN201910963365.7A patent/CN110598678A/en active Pending
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2013188298A (en) * | 2012-03-13 | 2013-09-26 | Asahi Kasei Home Products Kk | Lid material for microwave oven cooking |
| US20140220196A1 (en) * | 2013-02-06 | 2014-08-07 | Restaurant Technology, Inc. | Temperature Controlled Cooking Device, Control System, and Method |
| CN103347111A (en) * | 2013-07-27 | 2013-10-09 | 青岛歌尔声学科技有限公司 | Intelligent mobile electronic equipment with size and weight estimation function |
| CN103983334A (en) * | 2014-05-20 | 2014-08-13 | 联想(北京)有限公司 | Information processing method and electronic equipment |
| CN108140328A (en) * | 2015-09-09 | 2018-06-08 | 菲特利公司 | Using food image identification come the system and method for trophic analysis |
| CN106372198A (en) * | 2016-08-31 | 2017-02-01 | 乐视控股（北京）有限公司 | Data extraction method based on image recognition technology and mobile terminal thereof |
| CN109753977A (en) * | 2017-11-01 | 2019-05-14 | 菜鸟智能物流控股有限公司 | Method, terminal, server and system for measuring article size information |
| CN107944391A (en) * | 2017-11-24 | 2018-04-20 | 广东美的厨房电器制造有限公司 | The definite method, apparatus and storage medium of a kind of food weight |
| CN108171722A (en) * | 2017-12-26 | 2018-06-15 | 广东美的厨房电器制造有限公司 | Image extraction method, device and cooking apparatus |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113349645A (en) * | 2020-03-05 | 2021-09-07 | 佛山市云米电器科技有限公司 | Method for acquiring image of article inside oven, oven system, baking equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| EP3132380B1 (en) | Method for estimating a quantity of a blood component in a fluid canister | |
| CN103679173B (en) | Method for detecting image salient region | |
| CN106030653B (en) | For generating the image processing system and image processing method of high dynamic range images | |
| CN110378946B (en) | Depth map processing method and device and electronic equipment | |
| CN108229475B (en) | Vehicle tracking method, system, computer device and readable storage medium | |
| EP3358298A1 (en) | Building height calculation method and apparatus, and storage medium | |
| KR20120138627A (en) | A face tracking method and device | |
| CN107231521B (en) | A kind of meter reading identification camera automatic positioning method | |
| CN109658447B (en) | Night image defogging method based on edge detail preservation | |
| US9396411B2 (en) | Method and system for generating intrinsic images using a single reflectance technique | |
| US20110150358A1 (en) | Method and system for factoring an illumination image | |
| CN111161295B (en) | Dish image background stripping method | |
| CN108171722B (en) | Image extraction method and device and cooking utensil | |
| CN109084350A (en) | A kind of kitchen ventilator and oil smoke concentration detection method having filtering functions vision-based detection module | |
| CN108805144A (en) | Shell hole recognition methods based on morphology correction and system, indication of shots equipment | |
| CN112784640A (en) | Menu making method and device and cooking machine | |
| CN110598678A (en) | Control method and control device of cooking equipment, cooking equipment and storage medium | |
| CN109028230A (en) | Have the stove and oil smoke concentration detection method of gesture control vision-based detection function | |
| CN113518182A (en) | Cucumber phenotype characteristic measuring method based on raspberry pie | |
| CN117376717A (en) | Camera disturbance correction method and system | |
| CN112417378A (en) | Eriocheir sinensis quality estimation method based on unmanned aerial vehicle image processing | |
| CN116645418A (en) | Screen button detection method and device based on 2D and 3D cameras and relevant medium thereof | |
| CN113379620B (en) | Optical remote sensing satellite image cloud detection method | |
| CN109344827A (en) | A kind of kitchen ventilator with light compensating apparatus and oil smoke concentration detection method | |
| CN108830238A (en) | The adaptively selected system of lipstick color |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |