CN111797896B - Commodity identification method and device based on intelligent baking - Google Patents
Commodity identification method and device based on intelligent baking Download PDFInfo
- Publication number
- CN111797896B CN111797896B CN202010483443.6A CN202010483443A CN111797896B CN 111797896 B CN111797896 B CN 111797896B CN 202010483443 A CN202010483443 A CN 202010483443A CN 111797896 B CN111797896 B CN 111797896B
- Authority
- CN
- China
- Prior art keywords
- image
- preset
- commodity
- area
- identification
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 40
- 238000001514 detection method Methods 0.000 claims abstract description 85
- 238000013145 classification model Methods 0.000 claims abstract description 40
- 238000013135 deep learning Methods 0.000 claims abstract description 24
- 238000013527 convolutional neural network Methods 0.000 claims abstract description 19
- 230000005856 abnormality Effects 0.000 claims abstract description 9
- 238000012549 training Methods 0.000 claims description 44
- 238000012545 processing Methods 0.000 claims description 28
- 230000002159 abnormal effect Effects 0.000 claims description 25
- 238000011176 pooling Methods 0.000 claims description 18
- 238000002372 labelling Methods 0.000 claims description 14
- 239000000758 substrate Substances 0.000 claims 1
- 238000010586 diagram Methods 0.000 description 8
- 230000008569 process Effects 0.000 description 7
- 235000008429 bread Nutrition 0.000 description 6
- 238000004590 computer program Methods 0.000 description 5
- 238000012937 correction Methods 0.000 description 5
- 230000006870 function Effects 0.000 description 5
- 238000012986 modification Methods 0.000 description 5
- 230000004048 modification Effects 0.000 description 5
- 235000015173 baked goods and baking mixes Nutrition 0.000 description 3
- 238000013138 pruning Methods 0.000 description 3
- 235000003166 Opuntia robusta Nutrition 0.000 description 2
- 244000218514 Opuntia robusta Species 0.000 description 2
- 230000004888 barrier function Effects 0.000 description 2
- 239000003086 colorant Substances 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 230000004075 alteration Effects 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000012217 deletion Methods 0.000 description 1
- 230000037430 deletion Effects 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000003062 neural network model Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 238000007781 pre-processing Methods 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 238000003860 storage Methods 0.000 description 1
- 230000001629 suppression Effects 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0004—Industrial image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/60—Analysis of geometric attributes
- G06T7/62—Analysis of geometric attributes of area, perimeter, diameter or volume
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/90—Determination of colour characteristics
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/25—Determination of region of interest [ROI] or a volume of interest [VOI]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30204—Marker
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
- Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
- Y02P90/30—Computing systems specially adapted for manufacturing
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Data Mining & Analysis (AREA)
- General Engineering & Computer Science (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Computation (AREA)
- Software Systems (AREA)
- Computing Systems (AREA)
- Quality & Reliability (AREA)
- Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Mathematical Physics (AREA)
- Evolutionary Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Multimedia (AREA)
- Geometry (AREA)
- Bioinformatics & Computational Biology (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a commodity identification method and device based on intelligent baking, wherein the method is applied to a server connected with at least one client through a cloud, and comprises the following steps: receiving a first image of a commodity to be identified, which is uploaded by a client; determining whether the first image has commodity overlapping or not by using a preset classification model, wherein the classification model is obtained by deep learning through a convolutional neural network; performing image detection on the first image by using a preset abnormality detection algorithm, and determining whether the first image meets the identification condition according to a detection result; and if the first image has commodity overlapping or does not meet the identification condition, sending an error prompt to the client. The method and the device can solve the problems of low commodity identification efficiency and high cost of the bakery in the prior art.
Description
Technical Field
The invention relates to the technical field of image recognition, in particular to a commodity recognition method and device based on intelligent baking.
Background
The goods of the traditional bakery, such as bread, especially freshly baked bread, are not packaged and price tagged, and therefore the price of the goods cannot be identified by scanning a bar code as in the traditional self-service cashing. In the process of settlement and collection, a cashier is required to calculate the price of the cake in the bread, but the premise is that the cashier is required to remember the price of each commodity, and the requirements and the training cost of the cashier are high.
Especially when the customer needs to check out after selecting baked goods such as bread, the price of the cake is calculated manually, the cake is slow, mistakes are easy to occur, and the problem of long-time queuing of the customer is easy to occur in the consumption peak period; the labor cost is high and the efficiency is low.
Disclosure of Invention
The embodiment of the invention provides a commodity identification method and device based on intelligent baking, which are used for solving the problems of low commodity identification efficiency and high cost of a baking shop in the prior art.
The embodiment of the invention provides a commodity identification method based on intelligent baking, which is applied to a server connected with at least one client through a cloud, and comprises the following steps:
receiving a first image of a commodity to be identified, which is uploaded by a client;
determining whether the first image has commodity overlapping or not by using a preset classification model, wherein the classification model is obtained by deep learning through a convolutional neural network; and
performing image detection on the first image by using a preset abnormality detection algorithm, and determining whether the first image meets the identification condition according to a detection result;
and if the first image has commodity overlapping or does not meet the identification condition, sending an error prompt to the client.
The determining whether the first image has commodity overlapping by using a preset classification model comprises the following steps:
the classification model classifies the first image according to a preset class library; the preset class library comprises an overlapped class library and a non-overlapped class library;
and when the first image belongs to the overlapping class library, determining that commodity overlapping exists in the first image.
The method for obtaining the classification model by deep learning through the convolutional neural network comprises the following steps:
performing two-layer convolution processing on each training image in the training set to obtain a convolution processing result;
carrying out maximum pooling on the convolution processing result to obtain a pooling processing result;
regularizing the pooling treatment result and outputting a classification result;
comparing the classification result with the classification of the training image to determine the classification accuracy of the current training;
and when the classification accuracy is smaller than a preset accuracy threshold, training is circulated again until the classification accuracy reaches the preset accuracy threshold.
The image detection of the first image by using a preset anomaly detection algorithm includes:
determining contour information of an object in the first image;
determining the minimum circumscribed rectangle of each contour according to the contour information;
deleting the minimum circumscribed rectangle with the area smaller than the preset first area threshold value to obtain a candidate minimum circumscribed rectangle;
traversing and calculating the area of the candidate minimum bounding rectangle, and determining the candidate minimum bounding rectangle with the largest area as a target bounding rectangle;
when the area of the target circumscribed rectangle is larger than the preset visible area, the detection result is abnormal;
when the area of the target circumscribed rectangle is not larger than the preset visible area, calculating the absolute value of the difference value between the area of the target circumscribed rectangle and the preset visible area;
when the absolute value of the difference is larger than a preset second area threshold, the detection result is abnormal;
accordingly, the determining whether the first image meets the recognition condition according to the detection result includes:
and when the detection result is abnormal, determining that the first image does not meet the identification condition.
Further, the method further comprises:
if the first image does not have commodity overlapping and meets the identification condition, carrying out commodity identification on the first image through a preset image identification model, and sending an identification result to the client;
accordingly, when the commodity category is newly added, the method further comprises:
receiving a new commodity sample image uploaded by a client;
marking the new commodity sample image by using a preset automatic marking algorithm;
and training the image recognition model by using the marked new commodity sample image to finish adding the new commodity sample.
The marking the new commodity sample image by using a preset automatic marking algorithm comprises the following steps:
determining contour information of an object in the new commodity sample image;
determining the minimum circumscribed rectangle of each contour according to the contour information;
deleting the minimum circumscribed rectangle with the area smaller than the area threshold value of the new commodity to obtain a candidate circumscribed rectangle;
and if the number of the candidate external rectangles is multiple, merging the candidate external rectangles to obtain a sample labeling rectangle.
The embodiment of the invention also provides a commodity identification device based on intelligent baking, which is applied to a server connected with at least one client through a cloud, and comprises the following components: the device comprises a receiving unit, a first detecting unit, a second detecting unit and a feedback unit; wherein,,
the receiving unit is used for receiving the first image of the commodity to be identified, which is uploaded by the client;
the first detection unit is used for determining whether commodity overlapping exists in the first image or not by using a preset classification model, wherein the classification model is obtained by deep learning through a convolutional neural network; and
the second detection unit is used for carrying out image detection on the first image by using a preset abnormality detection algorithm, and determining whether the first image meets the identification condition according to a detection result;
and the feedback unit is used for sending an error prompt to the client if the first image has commodity overlapping or does not meet the identification condition.
The first detection unit is used for determining whether the first image has commodity overlapping or not by using a preset classification model, and is specifically used for:
classifying the first image according to a preset class library by using the classification model; the preset class library comprises an overlapped class library and a non-overlapped class library; and when the first image belongs to the overlapping class library, determining that commodity overlapping exists in the first image.
The first detection unit performs deep learning through a convolutional neural network to obtain a classification model, and is specifically used for:
performing two-layer convolution processing on each training image in the training set to obtain a convolution processing result; carrying out maximum pooling on the convolution processing result to obtain a pooling processing result; regularizing the pooling treatment result and outputting a classification result; comparing the classification result with the classification of the training image to determine the classification accuracy of the current training; and when the classification accuracy is smaller than a preset accuracy threshold, training is circulated again until the classification accuracy reaches the preset accuracy threshold.
The second detection unit is configured to perform image detection on the first image by using a preset anomaly detection algorithm, and is specifically configured to:
determining contour information of an object in the first image; determining the minimum circumscribed rectangle of each contour according to the contour information; deleting the minimum circumscribed rectangle with the area smaller than the preset first area threshold value to obtain a candidate minimum circumscribed rectangle; traversing and calculating the area of the candidate minimum bounding rectangle, and determining the candidate minimum bounding rectangle with the largest area as a target bounding rectangle; when the area of the target circumscribed rectangle is larger than the preset visible area, the detection result is abnormal; when the area of the target circumscribed rectangle is not larger than the preset visible area, calculating the absolute value of the difference value between the area of the target circumscribed rectangle and the preset visible area; when the absolute value of the difference is larger than a preset second area threshold, the detection result is abnormal;
correspondingly, the second detection unit is used for determining whether the first image meets the identification condition according to the detection result, and is specifically configured to:
and when the detection result is abnormal, determining that the first image does not meet the identification condition.
Further, the apparatus further comprises: the identification unit is used for carrying out commodity identification on the first image through a preset image identification model if commodity overlapping does not exist in the first image and identification conditions are met, and sending an identification result to the client;
correspondingly, when the commodity category is newly added, the identification unit is further used for: receiving a new commodity sample image uploaded by a client; marking the new commodity sample image by using a preset automatic marking algorithm; and training the image recognition model by using the marked new commodity sample image to finish adding the new commodity sample.
The identification unit is used for marking the new commodity sample image by using a preset automatic marking algorithm, and is specifically used for:
determining contour information of an object in the new commodity sample image; determining the minimum circumscribed rectangle of each contour according to the contour information; deleting the minimum circumscribed rectangle with the area smaller than the area threshold value of the new commodity to obtain a candidate circumscribed rectangle; and if the number of the candidate external rectangles is multiple, merging the candidate external rectangles to obtain a sample labeling rectangle.
The invention has the following beneficial effects:
according to the commodity identification method and device based on intelligent baking, a server connected with at least one client through a cloud receives a first image of a commodity to be identified uploaded by the client; determining whether the first image has commodity overlapping or not by using a preset classification model, wherein the classification model is obtained by deep learning through a convolutional neural network; performing image detection on the first image by using a preset abnormality detection algorithm, and determining whether the first image meets the identification condition according to a detection result; and if the first image has commodity overlapping or does not meet the identification condition, sending an error prompt to the client. According to the embodiment of the invention, the service end can be connected with a plurality of clients through the cloud, so that the layout quantity of the service end can be effectively reduced, the cost is effectively saved, the commodity overlapping is detected through the classification model obtained through the deep learning of the convolutional neural network, the abnormal condition of the image is detected through the abnormal detection algorithm of the image identification, the problems of overlapping placement, disc placement correction, exceeding of the identification range of the commodity to be identified, barrier shielding and the like in the commodity identification process can be effectively checked, the prompt is carried out for correction aiming at the problems, the model is continuously trained, the identification accuracy is ensured, the commodity identification efficiency of a bakery is improved, and the labor cost is saved.
Drawings
FIG. 1 is a flowchart of a smart bake-based commodity identification method according to an embodiment of the present invention;
fig. 2 is a schematic structural diagram of a smart bake-based commodity identification apparatus according to an embodiment of the present invention.
Detailed Description
Aiming at the problems of low commodity identification efficiency and high cost of a bakery in the prior art, the intelligent baking-based commodity identification method provided by the embodiment of the invention saves cost by adopting a mode that a single server corresponds to multiple clients, and the server detects images of commodities to be identified through a classification model of convolutional neural network deep learning and an anomaly detection algorithm, determines whether the images can accurately identify the commodities according to detection results, and can send error prompts to the clients for correction when the images cannot be identified. The flow of the method of the invention is shown in figure 1, and the execution steps are as follows:
the client is provided with a camera and a lighting platform, when a user selects baked goods and needs to check out, the goods to be identified can be placed on the lighting platform, the client obtains images of the goods to be identified through the camera, and the obtained images of the goods to be identified are recorded as first images for convenience in description; the client is connected with the server through the cloud, and specifically, the client can send the first image to the server through the wireless communication module.
here, when the commodities to be identified are placed in an overlapping manner, the accuracy of commodity identification cannot be guaranteed, so that the situation belongs to an abnormal situation, and a user needs to be prompted to place the commodities to be identified correctly again.
this step is primarily directed to identifying common anomalies, which may include, but are not limited to, camera occlusion, tray not being properly placed, baked items exceeding tray, etc.
When the presence of the abnormal condition is detected, it is determined that the first image does not satisfy the recognition condition.
It should be appreciated that step 102 and step 103 are not strictly sequential in execution.
And 104, if the first image has commodity overlapping or does not meet the identification condition, sending an error prompt to the client.
Optionally, the determining whether the first image has the commodity overlapping in step 102 by using a preset classification model includes:
the classification model classifies the first image according to a preset class library; the preset class library comprises an overlapped class library and a non-overlapped class library;
here, since the first image is a two-dimensional image and has no three-dimensional depth information, the first image needs to be classified by a classification model obtained by performing deep learning through a convolutional neural network, so that the commodity overlapping can be effectively identified.
And when the first image belongs to the overlapping class library, determining that commodity overlapping exists in the first image.
The method for obtaining the classification model by deep learning through the convolutional neural network comprises the following steps:
performing two-layer convolution processing on each training image in the training set to obtain a convolution processing result; carrying out maximum pooling on the convolution processing result to obtain a pooling processing result; regularizing the pooling treatment result and outputting a classification result; comparing the classification result with the classification of the training image to determine the classification accuracy of the current training; and when the classification accuracy is smaller than a preset accuracy threshold, training is circulated again until the classification accuracy reaches the preset accuracy threshold. Preferably, model learning can be performed by using a 13-layer deep learning network, and the specific algorithm implementation operation is as follows:
1) 600 sheets of overlapping bread samples and non-overlapping bread samples were collected, each with a resolution of 640X 480, 500 sheets of each as a training set, totaling 1000 sheets, dividing the 1000 sheets into two categories, overlapping (OL) and non-overlapping (NoOL) categories, and preprocessing the pictures with a resolution of 224X224 as a network input.
2) The training set is brought into the 13-layer deep learning network to carry out the overlay network training, wherein, because the number of layers of the deep learning network is smaller, a shorter training time, such as 5 minutes, can be set according to actual needs;
3) Testing is carried out after training is completed until the accuracy of the overlay LapNet network on whether the pictures are overlapped is up to 95 percent, and the training is finished.
Optionally, in step 103, performing image detection on the first image by using a preset anomaly detection algorithm includes:
determining contour information of an object in the first image; the method for obtaining the profile information is various, and the embodiment of the invention is not limited to this, and is preferably illustrated in a simpler, more convenient and more efficient manner: firstly, carrying out graying treatment on the first image, and converting the first image into a single-channel gray scale image so as to reduce the calculated amount; the gray value of the pixel point in the image is set to be 0 or 255 through binarization, namely the whole image presents obvious black-and-white effect, and the binarized image only comprises two colors: black and white; and determining the contour information of the object according to the binarization result.
Determining the minimum circumscribed rectangle of each contour according to the contour information;
deleting the minimum circumscribed rectangle with the area smaller than the preset first area threshold value to obtain a candidate minimum circumscribed rectangle; here, the preset first area threshold is specifically set according to the smallest commodity in the identifiable commodities, that is, the preset first area threshold is set to be the area of the identifiable smallest commodity, if the area of the smallest circumscribed rectangle is smaller than that of the identifiable smallest commodity, the smallest circumscribed rectangle can be determined to be an invalid circumscribed rectangle, and deletion operation can be performed to reduce subsequent operation amount.
Traversing and calculating the area of the candidate minimum bounding rectangle, and determining the candidate minimum bounding rectangle with the largest area as a target bounding rectangle;
when the area of the target circumscribed rectangle is larger than the preset visible area, the detection result is abnormal; for example, the preset visual area may be set as a dinner plate area, or the like. When the area of the target circumscribed rectangle is larger than the preset visible area, the condition that the baked goods exceed the dinner plate is indicated.
When the area of the target circumscribed rectangle is not larger than the preset visible area, calculating the absolute value of the difference value between the area of the target circumscribed rectangle and the preset visible area;
when the absolute value of the difference is larger than a preset second area threshold, the detection result is abnormal; here, the preset second area threshold value may be set to 1/10, 1/15, or the like of the visible area as needed.
Accordingly, the determining whether the first image meets the recognition condition according to the detection result includes:
and when the detection result is abnormal, determining that the first image does not meet the identification condition.
Further, the method further comprises:
if the first image does not have commodity overlapping and meets the identification condition, carrying out commodity identification on the first image through a preset image identification model, and sending an identification result to the client;
accordingly, when the commodity category is newly added, the method further comprises:
receiving a new commodity sample image uploaded by a client;
marking the new commodity sample image by using a preset automatic marking algorithm;
and training the image recognition model by using the marked new commodity sample image to finish adding the new commodity sample.
Here, the image recognition model selects a deep learning target detection network as a category recognition detection network, which is a YOLOv 3-based detection network, but modifies the backbone network dark net53 of YOLOv3 to a shallower and efficient MobileNet v1 network. The main contribution of MobileNet v1 is to propose to replace the normal convolution with the depth separable convolution, most of its whole network is based on the depth separable convolution, only the previous layers of the network use the normal convolution, the whole network parameter is 2.5 times smaller than GoogLeNet, but the accuracy is higher than GoogLeNet. The DarkNet53 network is composed of 53 layers of neural networks, the DarkNet53 is replaced by a MobileNet v1 network, the network layer is reduced from 53 layers to 21 layers, the network parameter size is also greatly reduced, the network model parameter is reduced by more than 50%, and the running time single frame on the server side can be reduced from 100ms to 32ms. In order to make the time consumption shorter and better, the mobile network can be further optimized, the deep learning network model optimization is generally performed by using operations such as network model sparsification, and structural pruning/sparsification operations can be utilized, namely pruning channels are utilized when an image recognition model is trained, and then precision is recovered through fine tuning; the sparsification is introduced by randomly discarding the channel-wise connection, so that a smaller network can be obtained; channel-wise sparsification is imposed in the training process to optimize the objective function, resulting in a smoother channel pruning process and less loss of accuracy than otherwise. Therefore, a simplified and efficient neural network model spark-mobilet-YOLOv 3 can be obtained as an image recognition model to carry out commodity recognition, for example, the running time on 1050GPU is reduced from 32ms to 20ms, the recognition rate is not greatly reduced while the network model is reduced, and the recognition rate can be basically maintained at about 95% through experiments.
Preferably, the labeling the new commodity sample image by using a preset automatic labeling algorithm includes:
determining contour information of an object in the new commodity sample image; the method for obtaining the profile information is various, and the embodiment of the invention is not limited to this, and is preferably illustrated in a simpler, more convenient and more efficient manner: firstly, carrying out graying treatment on the first image, and converting the first image into a single-channel gray scale image so as to reduce the calculated amount; the gray value of the pixel point in the image is set to be 0 or 255 through binarization, namely the whole image presents obvious black-and-white effect, and the binarized image only comprises two colors: black and white; and determining the contour information of the object according to the binarization result.
Determining the minimum circumscribed rectangle of each contour according to the contour information;
deleting the minimum circumscribed rectangle with the area smaller than the area threshold value of the new commodity to obtain a candidate circumscribed rectangle;
if a plurality of candidate external rectangles are present, the candidate external rectangles are combined to obtain a sample labeling rectangle, and since the newly added commodity types are performed one by one, a sample labeling rectangle is required to be obtained when the sample labeling is performed, if a plurality of candidate external rectangles are present, the plurality of candidate external rectangles still need to be combined to obtain a final sample labeling rectangle, specifically, a non-maximum suppression algorithm can be used for combining, and other combining methods in the prior art are also possible, which is not limited by the embodiment of the present invention.
Based on the same inventive concept, an embodiment of the present invention provides a smart baking-based commodity identification apparatus, which may be applied to a server connected to at least one client through a cloud, and has a structure as shown in fig. 2, including: a receiving unit 21, a first detecting unit 22, a second detecting unit 23, and a feedback unit 24; wherein,,
the receiving unit 21 is configured to receive a first image of a commodity to be identified uploaded by a client;
the first detection unit 22 is configured to determine whether the first image has a commodity overlapping by using a preset classification model, where the classification model is a model obtained by performing deep learning through a convolutional neural network; and
the second detecting unit 23 is configured to perform image detection on the first image by using a preset anomaly detection algorithm, and determine whether the first image meets a recognition condition according to a detection result;
the feedback unit 24 is configured to send an error prompt to the client if the first image has a commodity overlapping or does not satisfy the identification condition.
The first detecting unit 22 determines whether the first image has a commodity overlapping by using a preset classification model, which is specifically configured to:
classifying the first image according to a preset class library by using the classification model; the preset class library comprises an overlapped class library and a non-overlapped class library; and when the first image belongs to the overlapping class library, determining that commodity overlapping exists in the first image.
The first detection unit 22 performs deep learning through a convolutional neural network to obtain a classification model, which is specifically configured to:
performing two-layer convolution processing on each training image in the training set to obtain a convolution processing result; carrying out maximum pooling on the convolution processing result to obtain a pooling processing result; regularizing the pooling treatment result and outputting a classification result; comparing the classification result with the classification of the training image to determine the classification accuracy of the current training; and when the classification accuracy is smaller than a preset accuracy threshold, training is circulated again until the classification accuracy reaches the preset accuracy threshold.
The second detection unit 23 performs image detection on the first image by using a preset anomaly detection algorithm, and is specifically configured to:
determining contour information of an object in the first image; determining the minimum circumscribed rectangle of each contour according to the contour information; deleting the minimum circumscribed rectangle with the area smaller than the preset first area threshold value to obtain a candidate minimum circumscribed rectangle; traversing and calculating the area of the candidate minimum bounding rectangle, and determining the candidate minimum bounding rectangle with the largest area as a target bounding rectangle; when the area of the target circumscribed rectangle is larger than the preset visible area, the detection result is abnormal; when the area of the target circumscribed rectangle is not larger than the preset visible area, calculating the absolute value of the difference value between the area of the target circumscribed rectangle and the preset visible area; when the absolute value of the difference is larger than a preset second area threshold, the detection result is abnormal;
accordingly, the second detecting unit 23 determines, according to the detection result, whether the first image meets the recognition condition, specifically for:
and when the detection result is abnormal, determining that the first image does not meet the identification condition.
Further, the apparatus further comprises: the identification unit is used for carrying out commodity identification on the first image through a preset image identification model if commodity overlapping does not exist in the first image and identification conditions are met, and sending an identification result to the client;
correspondingly, when the commodity category is newly added, the identification unit is further used for: receiving a new commodity sample image uploaded by a client; marking the new commodity sample image by using a preset automatic marking algorithm; and training the image recognition model by using the marked new commodity sample image to finish adding the new commodity sample.
The identification unit is used for marking the new commodity sample image by using a preset automatic marking algorithm, and is specifically used for:
determining contour information of an object in the new commodity sample image; determining the minimum circumscribed rectangle of each contour according to the contour information; deleting the minimum circumscribed rectangle with the area smaller than the area threshold value of the new commodity to obtain a candidate circumscribed rectangle; and if the number of the candidate external rectangles is multiple, merging the candidate external rectangles to obtain a sample labeling rectangle.
It should be understood that the principle and the process of implementing the intelligent baking-based commodity identification apparatus according to the embodiments of the present invention are similar to those of the embodiment shown in fig. 1 and described above, and are not repeated here.
According to the commodity identification method and device based on intelligent baking, a server connected with at least one client through a cloud receives a first image of a commodity to be identified uploaded by the client; determining whether the first image has commodity overlapping or not by using a preset classification model, wherein the classification model is obtained by deep learning through a convolutional neural network; performing image detection on the first image by using a preset abnormality detection algorithm, and determining whether the first image meets the identification condition according to a detection result; and if the first image has commodity overlapping or does not meet the identification condition, sending an error prompt to the client. According to the embodiment of the invention, the service end can be connected with a plurality of clients through the cloud, so that the layout quantity of the service end can be effectively reduced, the cost is effectively saved, the commodity overlapping is detected through the classification model obtained through the deep learning of the convolutional neural network, the abnormal condition of the image is detected through the abnormal detection algorithm of the image identification, the problems of overlapping placement, disc placement correction, exceeding of the identification range of the commodity to be identified, barrier shielding and the like in the commodity identification process can be effectively checked, the prompt is carried out for correction aiming at the problems, the model is continuously trained, the identification accuracy is ensured, the commodity identification efficiency of a bakery is improved, and the labor cost is saved.
Those of ordinary skill in the art will appreciate that: the drawing is a schematic diagram of one embodiment and the modules or flows in the drawing are not necessarily required to practice the invention.
From the above description of embodiments, it will be apparent to those skilled in the art that the present invention may be implemented in software plus a necessary general hardware platform. Based on such understanding, the technical solution of the present invention may be embodied essentially or in a part contributing to the prior art in the form of a software product, which may be stored in a storage medium, such as a ROM/RAM, a magnetic disk, an optical disk, etc., including several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the method described in the embodiments or some parts of the embodiments of the present invention.
In this specification, each embodiment is described in a progressive manner, and identical and similar parts of each embodiment are all referred to each other, and each embodiment mainly describes differences from other embodiments. In particular, for apparatus or system embodiments, since they are substantially similar to method embodiments, the description is relatively simple, with reference to the description of method embodiments in part. The apparatus and system embodiments described above are merely illustrative, wherein the elements illustrated as separate elements may or may not be physically separate, and the elements shown as elements may or may not be physical elements, may be located in one place, or may be distributed over a plurality of network elements. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art will understand and implement the present invention without undue burden.
In addition, in some of the above embodiments and the flows described in the drawings, a plurality of operations appearing in a specific order are included, but it should be clearly understood that the operations may be performed out of the order in which they appear herein or performed in parallel, the sequence numbers of the operations such as 201, 202, 203, etc. are merely used to distinguish between the various operations, and the sequence numbers themselves do not represent any order of execution. In addition, the flows may include more or fewer operations, and the operations may be performed sequentially or in parallel. It should be noted that, the descriptions of "first" and "second" herein are used to distinguish different messages, devices, modules, etc., and do not represent a sequence, and are not limited to the "first" and the "second" being different types.
The apparatus embodiments described above are merely illustrative, wherein the elements illustrated as separate elements may or may not be physically separate, and the elements shown as elements may or may not be physical elements, may be located in one place, or may be distributed over a plurality of network elements. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art will understand and implement the present invention without undue burden.
The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
While alternative embodiments of the present invention have been described, additional variations and modifications in those embodiments may occur to those skilled in the art once they learn of the basic inventive concepts. It is therefore intended that the following appended claims be interpreted as including alternative embodiments and all such alterations and modifications as fall within the scope of the invention.
It will be apparent to those skilled in the art that various modifications and variations can be made to the embodiments of the present invention without departing from the spirit or scope of the embodiments of the invention. Thus, if such modifications and variations of the embodiments of the present invention fall within the scope of the claims and the equivalents thereof, the present invention is also intended to include such modifications and variations.
Claims (12)
1. The commodity identification method based on intelligent baking is characterized by being applied to a server connected with at least one client through a cloud, and comprising the following steps:
receiving a first image of a commodity to be identified, which is uploaded by a client;
determining whether the first image has commodity overlapping or not by using a preset classification model, wherein the classification model is obtained by deep learning through a convolutional neural network; and
performing image detection on the first image by using a preset abnormality detection algorithm, determining whether the first image meets the recognition condition according to a detection result, and determining that the first image does not meet the recognition condition when detecting that an abnormality exists;
if the first image has commodity overlapping or does not meet the identification condition, sending an error prompt to the client;
if the first image does not have commodity overlapping and meets the identification condition, commodity identification is carried out on the first image through a preset image identification model, and an identification result is sent to the client.
2. The method of claim 1, wherein determining whether there is a commercial overlap of the first image using a predetermined classification model comprises:
the classification model classifies the first image according to a preset class library; the preset class library comprises an overlapped class library and a non-overlapped class library;
and when the first image belongs to the overlapping class library, determining that commodity overlapping exists in the first image.
3. The method according to claim 1 or 2, wherein deep learning through a convolutional neural network results in a classification model, comprising:
performing two-layer convolution processing on each training image in the training set to obtain a convolution processing result;
carrying out maximum pooling on the convolution processing result to obtain a pooling processing result;
regularizing the pooling treatment result and outputting a classification result;
comparing the classification result with the classification of the training image to determine the classification accuracy of the current training;
and when the classification accuracy is smaller than a preset accuracy threshold, training is circulated again until the classification accuracy reaches the preset accuracy threshold.
4. The method of claim 1, wherein the performing image detection on the first image using a preset anomaly detection algorithm comprises:
determining contour information of an object in the first image;
determining the minimum circumscribed rectangle of each contour according to the contour information;
deleting the minimum circumscribed rectangle with the area smaller than the preset first area threshold value to obtain a candidate minimum circumscribed rectangle;
traversing and calculating the area of the candidate minimum bounding rectangle, and determining the candidate minimum bounding rectangle with the largest area as a target bounding rectangle;
when the area of the target circumscribed rectangle is larger than the preset visible area, the detection result is abnormal;
when the area of the target circumscribed rectangle is not larger than the preset visible area, calculating the absolute value of the difference value between the area of the target circumscribed rectangle and the preset visible area;
when the absolute value of the difference is larger than a preset second area threshold, the detection result is abnormal;
accordingly, the determining whether the first image meets the recognition condition according to the detection result includes:
and when the detection result is abnormal, determining that the first image does not meet the identification condition.
5. The method of claim 1, wherein the step of determining the position of the substrate comprises,
when the commodity category is newly added, the method further comprises:
receiving a new commodity sample image uploaded by a client;
marking the new commodity sample image by using a preset automatic marking algorithm;
and training the image recognition model by using the marked new commodity sample image to finish adding the new commodity sample.
6. The method of claim 5, wherein labeling the new merchandise sample image using a preset automatic labeling algorithm comprises:
determining contour information of an object in the new commodity sample image;
determining the minimum circumscribed rectangle of each contour according to the contour information;
deleting the minimum circumscribed rectangle with the area smaller than the area threshold value of the new commodity to obtain a candidate circumscribed rectangle;
and if the number of the candidate external rectangles is multiple, merging the candidate external rectangles to obtain a sample labeling rectangle.
7. Wisdom is based on toasting commodity identification device, its characterized in that, the device is applied to the server that is connected with at least one customer end through the high in the clouds, includes: the device comprises a receiving unit, a first detecting unit, a second detecting unit, a feedback unit and an identifying unit; wherein,,
the receiving unit is used for receiving the first image of the commodity to be identified, which is uploaded by the client;
the first detection unit is used for determining whether commodity overlapping exists in the first image or not by using a preset classification model, wherein the classification model is obtained by deep learning through a convolutional neural network; and
the second detection unit is used for carrying out image detection on the first image by using a preset abnormality detection algorithm, and determining whether the first image meets the identification condition according to a detection result; when detecting that an abnormal condition exists, determining that the first image does not meet the identification condition; the feedback unit is used for sending an error prompt to the client if the first image has commodity overlapping or does not meet the identification condition;
and the identification unit is used for carrying out commodity identification on the first image through a preset image identification model if the first image does not have commodity overlapping and meets the identification condition, and sending an identification result to the client.
8. The apparatus according to claim 7, wherein the first detection unit is configured to determine whether the first image has a commodity overlap by using a preset classification model, and is specifically configured to:
classifying the first image according to a preset class library by using the classification model; the preset class library comprises an overlapped class library and a non-overlapped class library; and when the first image belongs to the overlapping class library, determining that commodity overlapping exists in the first image.
9. The apparatus according to claim 7 or 8, wherein the first detection unit performs deep learning through a convolutional neural network to obtain a classification model, and is specifically configured to:
performing two-layer convolution processing on each training image in the training set to obtain a convolution processing result; carrying out maximum pooling on the convolution processing result to obtain a pooling processing result; regularizing the pooling treatment result and outputting a classification result; comparing the classification result with the classification of the training image to determine the classification accuracy of the current training; and when the classification accuracy is smaller than a preset accuracy threshold, training is circulated again until the classification accuracy reaches the preset accuracy threshold.
10. The apparatus according to claim 7, wherein the second detection unit performs image detection on the first image using a preset anomaly detection algorithm, specifically for:
determining contour information of an object in the first image; determining the minimum circumscribed rectangle of each contour according to the contour information; deleting the minimum circumscribed rectangle with the area smaller than the preset first area threshold value to obtain a candidate minimum circumscribed rectangle; traversing and calculating the area of the candidate minimum bounding rectangle, and determining the candidate minimum bounding rectangle with the largest area as a target bounding rectangle; when the area of the target circumscribed rectangle is larger than the preset visible area, the detection result is abnormal; when the area of the target circumscribed rectangle is not larger than the preset visible area, calculating the absolute value of the difference value between the area of the target circumscribed rectangle and the preset visible area; when the absolute value of the difference is larger than a preset second area threshold, the detection result is abnormal;
correspondingly, the second detection unit is used for determining whether the first image meets the identification condition according to the detection result, and is specifically configured to:
and when the detection result is abnormal, determining that the first image does not meet the identification condition.
11. The apparatus of claim 7, wherein the apparatus further comprises:
the identification unit is further configured to: when the commodity type is newly added, receiving a new commodity sample image uploaded by the client; marking the new commodity sample image by using a preset automatic marking algorithm; and training the image recognition model by using the marked new commodity sample image to finish adding the new commodity sample.
12. The device according to claim 11, wherein the identification unit is configured to label the new product sample image by using a preset automatic labeling algorithm, and is specifically configured to: determining contour information of an object in the new commodity sample image; determining the minimum circumscribed rectangle of each contour according to the contour information; deleting the minimum circumscribed rectangle with the area smaller than the area threshold value of the new commodity to obtain a candidate circumscribed rectangle; and if the number of the candidate external rectangles is multiple, merging the candidate external rectangles to obtain a sample labeling rectangle.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010483443.6A CN111797896B (en) | 2020-06-01 | 2020-06-01 | Commodity identification method and device based on intelligent baking |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010483443.6A CN111797896B (en) | 2020-06-01 | 2020-06-01 | Commodity identification method and device based on intelligent baking |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111797896A CN111797896A (en) | 2020-10-20 |
| CN111797896B true CN111797896B (en) | 2023-06-27 |
Family
ID=72806625
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010483443.6A Active CN111797896B (en) | 2020-06-01 | 2020-06-01 | Commodity identification method and device based on intelligent baking |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111797896B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114627315B (en) * | 2022-01-31 | 2023-04-18 | 南通爱米食品有限公司 | Method for identifying baking stage of mass pastry |
Family Cites Families (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP3217324A1 (en) * | 2016-03-07 | 2017-09-13 | Ricoh Company, Ltd. | Hybrid detection recognition system |
| CN108647588A (en) * | 2018-04-24 | 2018-10-12 | 广州绿怡信息科技有限公司 | Goods categories recognition methods, device, computer equipment and storage medium |
| JP7300699B2 (en) * | 2018-11-12 | 2023-06-30 | 株式会社イシダ | Training data generation method, training data generation program, training data generation device, and product identification device |
| CN109816045A (en) * | 2019-02-11 | 2019-05-28 | 青岛海信智能商用***股份有限公司 | A kind of commodity recognition method and device |
| CN109919211A (en) * | 2019-02-26 | 2019-06-21 | 南京旷云科技有限公司 | Commodity recognition method, device, system and computer-readable medium |
| CN110222587A (en) * | 2019-05-13 | 2019-09-10 | 杭州电子科技大学 | A kind of commodity attribute detection recognition methods again based on characteristic pattern |
| CN110458096A (en) * | 2019-08-09 | 2019-11-15 | 广州众聚智能科技有限公司 | A kind of extensive commodity recognition method based on deep learning |
| CN110807808A (en) * | 2019-10-14 | 2020-02-18 | 浙江理工大学 | Commodity identification method based on physical engine and deep full convolution network |
| CN111079699A (en) * | 2019-12-30 | 2020-04-28 | 北京每日优鲜电子商务有限公司 | Commodity identification method and device |
-
2020
- 2020-06-01 CN CN202010483443.6A patent/CN111797896B/en active Active
Non-Patent Citations (1)
| Title |
|---|
| 基于改进Faster RCNN与Grabcut的商品图像检测;胡正委等;《计算机***应用》;20181114(第11期);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111797896A (en) | 2020-10-20 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108320404B (en) | Commodity identification method and device based on neural network and self-service cash register | |
| WO2019228063A1 (en) | Product inspection terminal, method and system, computer apparatus and readable medium | |
| US11593656B2 (en) | Using a first stain to train a model to predict the region stained by a second stain | |
| CN106163381B (en) | Information processing apparatus, information processing system, and program | |
| JP2019035626A (en) | Recognition method of tire image and recognition device of tire image | |
| CN106096932A (en) | The pricing method of vegetable automatic recognition system based on tableware shape | |
| CN103914708A (en) | Food variety detection method and system based on machine vision | |
| US10713541B2 (en) | Systems and methods for occlusion handling in a neural network via activation subtraction | |
| CN108579094A (en) | A kind of user interface detection method and relevant apparatus, system and storage medium | |
| JP7496364B2 (en) | Detecting Cells of Interest in Large Image Datasets Using Artificial Intelligence | |
| CN111415339B (en) | Image defect detection method for complex texture industrial product | |
| US20200292463A1 (en) | Apparatus for optimizing inspection of exterior of target object and method thereof | |
| CN116740728B (en) | Dynamic acquisition method and system for wafer code reader | |
| CN111797896B (en) | Commodity identification method and device based on intelligent baking | |
| CN111340831A (en) | Point cloud edge detection method and device | |
| CN103279944A (en) | Image division method based on biogeography optimization | |
| CN113468914B (en) | Method, device and equipment for determining purity of commodity | |
| CN103714498A (en) | Quick payment system based on image recognition technology and method thereof | |
| CN115512134A (en) | Express item stacking abnormity early warning method, device, equipment and storage medium | |
| CN115760724A (en) | Food quality detection method and system | |
| CN111178405A (en) | Similar object identification method fusing multiple neural networks | |
| CN114332058A (en) | Serum quality identification method, device, equipment and medium based on neural network | |
| CN117437264A (en) | Behavior information identification method, device and storage medium | |
| CN110110594B (en) | Product distribution identification method and device | |
| CN116824624A (en) | Cow identity recognition method, system, equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |