CN115965964A - Egg freshness identification method, system and equipment - Google Patents
Egg freshness identification method, system and equipment Download PDFInfo
- Publication number
- CN115965964A CN115965964A CN202310043469.2A CN202310043469A CN115965964A CN 115965964 A CN115965964 A CN 115965964A CN 202310043469 A CN202310043469 A CN 202310043469A CN 115965964 A CN115965964 A CN 115965964A
- Authority
- CN
- China
- Prior art keywords
- model
- freshness
- egg
- teacher
- knowledge
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 132
- 235000013601 eggs Nutrition 0.000 claims abstract description 238
- 238000004821 distillation Methods 0.000 claims abstract description 60
- 238000013140 knowledge distillation Methods 0.000 claims abstract description 25
- 230000003203 everyday effect Effects 0.000 claims abstract description 14
- 239000011229 interlayer Substances 0.000 claims abstract description 7
- 239000010410 layer Substances 0.000 claims description 89
- 239000013598 vector Substances 0.000 claims description 61
- 230000006870 function Effects 0.000 claims description 45
- 238000012549 training Methods 0.000 claims description 25
- 230000010354 integration Effects 0.000 claims description 19
- 238000004590 computer program Methods 0.000 claims description 16
- 238000003860 storage Methods 0.000 claims description 15
- 238000012545 processing Methods 0.000 claims description 12
- LFQSCWFLJHTTHZ-UHFFFAOYSA-N Ethanol Chemical compound CCO LFQSCWFLJHTTHZ-UHFFFAOYSA-N 0.000 claims 1
- 229910052731 fluorine Inorganic materials 0.000 claims 1
- 125000001153 fluoro group Chemical group F* 0.000 claims 1
- 230000000694 effects Effects 0.000 abstract description 10
- 238000001514 detection method Methods 0.000 abstract description 2
- 235000013305 food Nutrition 0.000 abstract description 2
- 238000012360 testing method Methods 0.000 description 19
- 238000004364 calculation method Methods 0.000 description 13
- 102000002322 Egg Proteins Human genes 0.000 description 11
- 108010000912 Egg Proteins Proteins 0.000 description 11
- 238000010586 diagram Methods 0.000 description 11
- 230000008569 process Effects 0.000 description 11
- 230000000875 corresponding effect Effects 0.000 description 9
- 238000009826 distribution Methods 0.000 description 9
- QCVGEOXPDFCNHA-UHFFFAOYSA-N 5,5-dimethyl-2,4-dioxo-1,3-oxazolidine-3-carboxamide Chemical compound CC1(C)OC(=O)N(C(N)=O)C1=O QCVGEOXPDFCNHA-UHFFFAOYSA-N 0.000 description 8
- 235000014103 egg white Nutrition 0.000 description 8
- 210000000969 egg white Anatomy 0.000 description 8
- 238000004891 communication Methods 0.000 description 7
- 238000005516 engineering process Methods 0.000 description 7
- 230000003044 adaptive effect Effects 0.000 description 5
- 238000012795 verification Methods 0.000 description 5
- 241000272525 Anas platyrhynchos Species 0.000 description 4
- 210000002969 egg yolk Anatomy 0.000 description 4
- 238000005259 measurement Methods 0.000 description 4
- 230000002829 reductive effect Effects 0.000 description 4
- 238000012546 transfer Methods 0.000 description 4
- 230000008859 change Effects 0.000 description 3
- 238000013527 convolutional neural network Methods 0.000 description 3
- 235000013345 egg yolk Nutrition 0.000 description 3
- 238000002474 experimental method Methods 0.000 description 3
- 238000003062 neural network model Methods 0.000 description 3
- 238000005192 partition Methods 0.000 description 3
- 238000004458 analytical method Methods 0.000 description 2
- 230000002238 attenuated effect Effects 0.000 description 2
- 230000008901 benefit Effects 0.000 description 2
- 238000006243 chemical reaction Methods 0.000 description 2
- 238000013145 classification model Methods 0.000 description 2
- 238000010276 construction Methods 0.000 description 2
- 238000013135 deep learning Methods 0.000 description 2
- 230000009977 dual effect Effects 0.000 description 2
- 238000011156 evaluation Methods 0.000 description 2
- 238000005286 illumination Methods 0.000 description 2
- 238000004519 manufacturing process Methods 0.000 description 2
- 238000013507 mapping Methods 0.000 description 2
- 239000011159 matrix material Substances 0.000 description 2
- 238000010295 mobile communication Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 238000005457 optimization Methods 0.000 description 2
- 102000004169 proteins and genes Human genes 0.000 description 2
- 108090000623 proteins and genes Proteins 0.000 description 2
- 208000010470 Ageusia Diseases 0.000 description 1
- 208000019331 Foodborne disease Diseases 0.000 description 1
- 241000287828 Gallus gallus Species 0.000 description 1
- 230000009471 action Effects 0.000 description 1
- 235000019666 ageusia Nutrition 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 230000015556 catabolic process Effects 0.000 description 1
- 230000001413 cellular effect Effects 0.000 description 1
- 238000012512 characterization method Methods 0.000 description 1
- 230000002596 correlated effect Effects 0.000 description 1
- 230000003247 decreasing effect Effects 0.000 description 1
- 238000006731 degradation reaction Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 239000000284 extract Substances 0.000 description 1
- 230000036541 health Effects 0.000 description 1
- 235000006486 human diet Nutrition 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 230000000670 limiting effect Effects 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 230000005055 memory storage Effects 0.000 description 1
- 230000000813 microbial effect Effects 0.000 description 1
- 235000016709 nutrition Nutrition 0.000 description 1
- 230000035764 nutrition Effects 0.000 description 1
- 230000000050 nutritive effect Effects 0.000 description 1
- 238000011022 operating instruction Methods 0.000 description 1
- 238000007781 pre-processing Methods 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
Images
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
- Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
- Y02P90/30—Computing systems specially adapted for manufacturing
Landscapes
- Image Analysis (AREA)
Abstract
The invention discloses an egg freshness identification method, system and device, and relates to the field of food quality detection. The method comprises the steps of continuously collecting egg images every day, and measuring the Hough unit value of eggs; determining the egg freshness grade according to the Hough unit value; constructing an egg freshness identification model based on self-adaptive multi-teacher knowledge distillation; the egg freshness identification model is combined with a label knowledge-based distillation method, an interlayer knowledge-based distillation method and a structured knowledge-based distillation method of 3 teacher models; the method comprises the steps that prediction results of 3 teacher models are integrated, under the help of a truth value label, larger weight is adaptively distributed to the teacher models with smaller prediction loss values, and smaller weight is distributed to the teacher models with larger prediction loss values; the egg image is used as input, the egg freshness grade is used as output, an egg freshness identification model is trained, and the freshness of the egg to be detected is identified. The invention can improve the identification effect of the freshness of the eggs.
Description
Technical Field
The invention relates to the field of food quality detection, in particular to an egg freshness identification method, system and equipment.
Background
The eggs have rich nutritive value, contain high-quality protein necessary for human bodies, have low price and are important sources of high-quality protein in human diet. However, there is a constant degradation of the quality of the eggs during transport and storage. The reduction of the freshness of the eggs not only can cause the loss of taste, quality and nutrition and influence the value of the eggs, but also the microbial pollution generated by the quality change of the eggs can easily cause food-borne disease outbreak and even endanger the life health of consumers. Therefore, how to quickly, accurately and inexpensively identify the freshness of eggs is a common concern for the industry and academia.
In recent years, with the rapid development of computers and information technologies, scholars at home and abroad successfully apply computer vision technology to agricultural product freshness identification. Based on different forms and color information of the air chamber, the yolk and other areas in the egg freshness attenuation process, part of researchers use the air chamber in the egg image as an interested area, and establish a regression model to verify that the air chamber area ratio and the egg freshness are significantly negatively correlated by analyzing the change of the air chamber area ratio (air chamber area ratio), so that the egg freshness is quickly identified. In recent years, researchers provide an egg freshness identification method based on a convolutional neural network, an attention mechanism combining a channel and a space is introduced into GoogLeNet to construct a GoogLeNet-A model, the attention of the model to an air chamber and an egg yolk area in an egg image is enhanced, the egg freshness identification accuracy rate reaches 94.05%, and the application potential of a deep learning technology in egg freshness identification is proved.
The high accuracy of the deep neural network model is realized based on a deeper, wider and more complex network structure, the complexity and the parameters of the network model are increased in geometric multiples while the network performance is continuously improved, and the existing egg freshness identification model is difficult to meet the application in actual production. Therefore, the model needs to be compressed by reducing the model parameters as much as possible on the premise of ensuring the performance of the model. Knowledge distillation obviously improves the performance of a small network by using one or more complex and accurate large networks to guide the simple and small network training, and the models with good classification performance have large parameters and calculation amount, the models with small parameters and calculation amount have poor classification performance, and the egg freshness identification effect is poor.
Disclosure of Invention
The invention aims to provide an egg freshness identification method, an egg freshness identification system and egg freshness identification equipment, which are used for solving the problem of poor egg freshness identification effect.
In order to achieve the purpose, the invention provides the following scheme:
an egg freshness identification method comprises the following steps:
continuously collecting the egg images every day, and determining the Hough unit value of the eggs;
determining the freshness grade of the eggs according to the Hough unit value; the egg freshness grades comprise an AA grade, an A grade and a B grade;
constructing an egg freshness identification model based on self-adaptive multi-teacher knowledge distillation; the egg freshness identification model is a student model; the student model combines 3 teacher models of a distillation method based on label knowledge, a distillation method based on middle layer knowledge and a distillation method based on structured knowledge; the student models integrate the prediction results of the 3 teacher models, and adaptively allocate a larger weight to the teacher model with a smaller prediction loss value and allocate a smaller weight to the teacher model with a larger prediction loss value with the help of the truth value label;
training the egg freshness identification model by taking the egg image as input and the egg freshness grade as output to generate a trained egg freshness identification model;
and identifying the freshness of the eggs to be detected according to the trained egg freshness identification model.
Optionally, the loss function L of the distillation method based on tag knowledge Logits Comprises the following steps:
wherein C is the total number of categories of freshness, C is the freshness category number, C =1,2,3, C; t is a temperature coefficient;
a softening probability vector for the teacher model in the current freshness category; />
A softening probability vector of the student model in the current freshness category; the corner mark t is a teacher model; the corner mark s is a student model.
Optionally, the loss function L of the distillation method based on interlayer knowledge Feature Comprises the following steps:
wherein, f t (x t ) n-2 Feature vectors output for the third layer from the last of the teacher model; f. of s (x s ) m-2 Feature vectors output for the third layer of the student model from the last number; n is the number of the convolution layers of the teacher model; m is the number of convolution layers of the student model; x is the number of t Inputting a feature vector for the last layer of the teacher model; x is the number of s Feature vectors input for the penultimate layer of the student model; w r A fully connected layer, wherein the fully connected layer is used for matching the dimensionality output by the teacher model and the dimensionality output by the last but one layer of the student model; the corner mark t is a teacher model; corner marks is a student model; and | | is a norm operator.
Optionally, the loss function L of the distillation method based on structured knowledge RKD Comprises the following steps:
wherein (x) 1 ,…,x n ) For an n-tuple extracted from the input feature vector X, n =1,2,3.;
a relationship potential function for measuring a relationship attribute of a given n-tuple, the relationship attribute comprising an angle and a Euclidean distance; (f) t (x 1 ),…,f t (x n ) Output feature vectors for the teacher model at any layer; f. of s (x 1 ),…,f s (x n ) Output feature vectors of the student model at any layer; l δ Calculating a Huber loss for the difference between the teacher model and the student model; the corner mark t is a teacher model; the corner mark s is a student model.
Optionally, the cross entropy loss L between the predicted result of the kth teacher model and the true value label teacherk Comprises the following steps:
wherein, y c A one-hot vector for class c freshness; c is the total number of freshness categories, C is the freshness category number, C =1,2,3.., C;
for the probability vector of the kth teacher model in the current freshness category, k =1,2,3, <' > is>
The predicted value for the kth teacher model in the class c freshness category.
Optionally, the weighting w assigned to the predicted loss value of the kth teacher model teacherk Comprises the following steps:
wherein C is the total freshness category number, C is the freshness category number, C =1,2,3.., C; and K is the total number of teacher models.
Optionally, the loss function of the egg freshness identification model is L all Comprises the following steps:
L all ＝α·L student +(1-α)·L teacher
wherein alpha is a hyper-parameter; l is student For the cross-entropy loss between the prediction results of the student model and the true label,
y c a one-hot vector for a freshness of type c, <' >>
For the probability vector of the student model in the class c freshness category, <' >>
The predicted value of the student model in the freshness category of the class C, C is the total number of freshness categories, C is the number of freshness categories, C =1,2,3.., C; l is teacher For integration loss of the teacher model, L teacher ＝w teacher1 ·L Logits +w teacher2 ·L Feature +w teacher3 ·L RKD ，L Logits As a loss function of the distillation method based on the tag knowledge, w teacher1 Weight assigned to 1 st teacher model, L Feature As a loss function of the distillation method based on knowledge of the intermediate layer, w teacher2 Weight assigned to the 2 nd teacher model, L RKD For said distillation based on structured knowledgeLoss function of method, w teacher3 The 3 rd teacher model is assigned a weight.
An egg freshness identification system comprising:
the half unit value measuring module is used for continuously collecting the egg images every day and measuring the half unit value of the egg according to the egg images;
the egg freshness grade determining module is used for determining the egg freshness grade according to the Hough unit value; the egg freshness grades comprise an AA grade, an A grade and a B grade;
the egg freshness identification model building module is used for building an egg freshness identification model based on self-adaptive multi-teacher knowledge distillation; the egg freshness identification model is a student model; the student model combines 3 teacher models of a distillation method based on label knowledge, a distillation method based on middle layer knowledge and a distillation method based on structured knowledge; the student models integrate the prediction results of the 3 teacher models, and adaptively allocate a larger weight to the teacher model with a smaller prediction loss value and allocate a smaller weight to the teacher model with a larger prediction loss value with the help of the truth value label;
the model training module is used for training the egg freshness identification model by taking the egg image as input and the egg freshness grade as output to generate a trained egg freshness identification model;
and the freshness identification module is used for identifying the freshness of the eggs to be detected according to the trained egg freshness identification model.
An electronic device comprises a storage and a processor, wherein the storage is used for storing a computer program, and the processor runs the computer program to enable the electronic device to execute the egg freshness identification method.
A computer-readable storage medium, which stores a computer program, which when executed by a processor implements the egg freshness identification method described above.
According to the specific embodiment provided by the invention, the invention discloses the following technical effects: the invention provides an egg freshness identification method, an egg freshness identification system and egg freshness identification equipment, wherein an egg freshness identification model based on Adaptive Multi-Teacher Knowledge Distillation (A-MKD) is constructed, the model is respectively combined with 3 different Knowledge Distillation strategies such as label Knowledge, middle layer Knowledge and structural Knowledge of 3 Teacher models to improve the feature expression capability of a student model, the student model integrates the prediction results of the 3 Teacher models, a larger weight is adaptively distributed to the Teacher model with a smaller prediction loss value with the help of a truth value label, and a smaller weight is distributed to the Teacher model with a larger prediction loss value, so that the student model can better learn correct Knowledge, the identification effect of the student model is further improved, and the accurate identification of the egg freshness is realized.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings needed to be used in the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings without inventive exercise.
FIG. 1 is a flow chart of an egg freshness identification method provided by the present invention;
FIG. 2 is an exemplary diagram of an enhanced egg image provided by the present invention; FIG. 2a is an illustration of an original image of an egg; FIG. 2b is an exemplary graph of an egg image with enhanced brightness; FIG. 2c is an exemplary image of an egg with reduced brightness; FIG. 2d is an exemplary diagram of a rotated egg image;
FIG. 3 is a schematic diagram of the overall structure of the A-MKD model provided by the present invention;
FIG. 4 is a flow chart of a distillation method based on tag knowledge provided by the present invention;
FIG. 5 is a flow chart of a distillation method based on knowledge of an interlayer provided by the present invention;
FIG. 6 is a flow chart of a distillation method based on structural knowledge provided by the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The invention aims to provide an egg freshness identification method, system and device, which can improve the egg freshness identification effect and realize accurate identification of egg freshness.
In order to make the aforementioned objects, features and advantages of the present invention more comprehensible, the present invention is described in detail with reference to the accompanying drawings and the detailed description thereof.
Example one
Fig. 1 is a flowchart of an egg freshness identification method provided by the present invention, and as shown in fig. 1, the present invention provides an egg freshness identification method, including:
step 101: images of the eggs were collected continuously every day and the hough unit value of the eggs was determined.
In practical application, the egg sample is processed: firstly, collecting images of all eggs every day, and storing the collected images in a computer according to group numbers; then 2 eggs are respectively selected from each group every day, and a Hough unit value is determined through physicochemical experiments, and the freshness grade corresponding to the egg image is determined according to the Hough unit value. The test was continued every day until the Hough Unit value fell below 55, and the test was terminated after the freshness was too low to eat according to the division of fresh eggs and fresh duck eggs (SB/T10638-2011) of the Ministry of commerce.
An egg image acquisition environment is built, and egg images are acquired: the constructed egg image acquisition environment is composed of a smart phone, a camera bellows, an LED lamp and a black partition plate with an oval light hole. Wherein, the black baffle is placed at a height of 10cm from the bottom of the dark box, and the lens is positioned at a vertical position of 30cm above the light hole. And horizontally placing the egg sample on the light holes at a fixed time every day, and acquiring an egg image in a backlight source illumination mode.
The image enhancement method comprises the steps of brightness adjustment and rotation, wherein the brightness conversion coefficient is randomly generated in a range of [0.7,1.3] in the image enhancement method, and the rotation range is randomly generated in a range of [ -30 degrees, 30 degrees ] or a range of [150 degrees, 210 degrees ] respectively.
The measurement of the half unit value of the egg comprises the following steps: firstly, acquiring the mass w of eggs by using an electronic scale; then, the shell of the egg is broken and placed on a white flat base plate, the height of the concentrated egg white at the middle point of the edge of the egg yolk and the edge of the concentrated egg white is measured by an egg white height measuring instrument (the bridle is avoided), 3 points in a regular triangle shape are measured, and the average value is taken to obtain the height H of the concentrated egg white; and finally, determining the Hough unit value of the egg according to a calculation formula of Hough units. The Hough unit calculation formula is as follows:
HU＝100lg(H-1.7w 0.37 +7.57)
step 102: determining the freshness grade of the eggs according to the Hough unit value; the egg freshness grades comprise an AA grade, an A grade and a B grade.
In practical application, the egg freshness grade is divided into three types according to the standard of fresh eggs and fresh duck eggs grading (SB/T10638-2011): AA-grade, A-grade and B-grade eggs, wherein HU is more than or equal to 72; 72 ≧ HU 60 is A-grade egg; 60 ≧ HU 55 is B-grade egg.
Step 103: constructing an egg freshness identification model based on self-adaptive multi-teacher knowledge distillation; the egg freshness identification model is a student model; the student model combines 3 teacher models of a distillation method based on label knowledge, a distillation method based on middle layer knowledge and a distillation method based on structured knowledge; the student models integrate the prediction results of the 3 teacher models, and adaptively allocate a larger weight to the teacher model with a smaller prediction loss value and allocate a smaller weight to the teacher model with a larger prediction loss value with the help of the truth value label. The teacher model is ResNet50, resNet101, and ResNet110, and the student model is ResNet20.
Dividing an egg image data set into a training set, a verification set and a test set, wherein the proportion of the training set, the verification set and the test set is 7:2:1, the training set is used for performing parameter training on an A-MKD model to obtain an egg freshness identification model; the verification set is used for detecting whether the trained model has an over-fitting or under-fitting phenomenon; the test set is used for testing the correctness and the error of the model so as to verify the validity of the model.
In practical applications, knowledge distillation methods based on various forms of knowledge transfer are as follows:
1) Distillation method based on tag knowledge:
in the image classification task, the deep neural network model is usually connected with a Softmax output layer after the fully-connected layer, and an output vector of the fully-connected layer is converted into a probability vector, wherein the probability vector represents the probability that the current sample belongs to each category respectively. The basic concept of the knowledge distillation method based on the label is to directly simulate the final output of a teacher model, and the Softmax layer outputs a more gentle probability distribution by introducing a parameter T, wherein the larger the T is, the more gentle the corresponding probability distribution is, and more 'dark knowledge' is distilled out. The distillation method based on the label knowledge converts discrete label attributes in a data set into continuous probability distribution by introducing soft labels, extracts 'dark knowledge' in a teacher network and transmits the 'dark knowledge' to a student network, thereby improving the performance of a student network model. Loss function L of the distillation method based on tag knowledge Logits Comprises the following steps:
wherein C is the total freshness category number, C is the freshness category number, and C =1,2, 3.; t is a temperature coefficient;
a softening probability vector for the teacher model in the current freshness category; />
A softening probability vector for the student model in the current freshness category; the corner mark t is a teacher model; the corner mark s is a student model.
2) Distillation method based on interlayer knowledge:
in the deep convolutional neural network, the knowledge learned by the network is hierarchical, and the abstraction degree of the knowledge corresponding to the shallow to deep layers is higher and higher. Therefore, the characteristics of the middle layer can also be used as a knowledge carrier for the student network to learn. The distillation based on the knowledge of the middle layer guides the student models according to the loss of the teacher model, and in order to avoid generating excessive calculation cost, the invention takes the third last layer of the teacher model as a guiding layer and the third last layer of the student model as a guided layer. During the training process, a mapping function W needs to be learned r Matching the dimensions of the guided layers of the student model with the guiding layers of the teacher model to obtain the parameter initialization values of the student model at the next stage, and minimizing the Mean Square Error (MSE) value of the teacher model and the student model as the loss L of the distillation method based on the middle layer knowledge Feature Loss function L of the distillation method based on interlayer knowledge Feature Comprises the following steps:
wherein, f t (x t ) n-2 Feature vectors output for the third layer from the last of the teacher model; f. of s (x s ) m-2 Feature vectors output for the third layer of the student model from the last number; n is the number of convolution layers of the teacher model; m is the number of convolution layers of the student model; x is the number of t Inputting a feature vector for the third layer from the last of the teacher model; x is the number of s Inputting a feature vector for the third layer of the student model from the last place; w r And the full connection layer is used for matching the dimensionality output by the teacher model and the dimensionality output by the last but one layer of the student model.
3) Distillation method based on structured knowledge
The traditional knowledge distillation method defaults that samples or characteristics are mutually independent, and a point-to-point knowledge transfer mode is adopted; and the structured knowledge distillation focuses on structural characterization such as the relationship between samples (inter-class relationship) or the context relationship inside the sample characteristics (intra-class relationship). The construction of the structured knowledge has no strict requirement on the specific position of the knowledge, and can be the category of the model output layer or the characteristics of the middle layer.
The invention adopts RKD (Relational Knowledge Distillation) method published in 2019 on CVPR to realize Distillation based on structured Knowledge. The RKD enables student models on the egg image classification task to pay attention to structural association between various types of samples (including intra-class samples and inter-class samples) extracted by a teacher model by establishing angle and distance dual relation measurement between the samples, and therefore more accurate egg freshness identification is achieved.
Loss function L of the distillation method based on structured knowledge RKD Comprises the following steps:
wherein (x) 1 ,…,x n ) For an n-tuple extracted from the input feature vector X, n =1,2,3.;
a relational potential function for measuring relational attributes of a given n-tuple, the relational attributes including angles and euclidean distances; (f) t (x 1 ),…,f t (x n ) Is the output feature vector of the teacher model at any layer; (f) s (x 1 ),…,f s (x n ) Output feature vectors of the student model at any layer; l δ To calculate the Huber loss for the teacher model and the student model differences.
Wherein x represents the vector potential of the relationship attributes (angle and euclidean distance) of the teacher model; y represents the vector potential of the relational attributes (angle and euclidean distance) of the student model.
In practical application, the adaptive model integration method comprises the following steps:
the purpose of model integration is to combine predictions from multiple teacher models to improve robustness and generalizability for a single teacher model, thereby improving the recognition accuracy of the model. Model integration helps to reduce the effects of variance, noise, and bias between predicted and actual values. Most of the current popular methods for model integration are to combine the prediction of multiple teachers with fixed weight distribution or other various label-free schemes, such as calculating weights based on optimization problems or entropy criteria. However, a fixed weight does not distinguish between a high quality teacher and a low quality teacher, and other solutions may mislead students in the case where a low quality teacher predicts the presence.
Based on the analysis, the invention provides a self-adaptive model integration method, which considers the prediction result of the teacher model in self-adaptive integration, adaptively allocates larger weight to the teacher model with smaller prediction loss value and allocates smaller weight to the teacher model with larger prediction loss value with the help of the truth value label, so that the student model learns from a relatively correct direction, and the identification precision of the student model is further improved.
In order to effectively integrate the prediction results of the 3 teacher models, the invention respectively calculates the cross entropy loss between the prediction results of the 3 teacher models and the real label.
(ii) a cross-entropy loss L between the predicted result of the kth teacher model and the true value label teacherk Comprises the following steps:
wherein, y c A one-hot vector for class c freshness;
for the probability vector of the kth teacher model in the current freshness category, k =1,2,3; />
A predicted value of the kth teacher model in the class c freshness category; q (z) c ) Probability vector of freshness of class c; z = [ z ] 1 ,…,z C ]And C is the predicted value of freshness of each type output by the fully-connected layer of the egg freshness identification model, C is the total freshness category number, C is the freshness category number, and C =1,2,3.
In practical application, the predicted loss value of the kth teacher model is assigned with a weight w teacherk Comprises the following steps:
wherein C is the total freshness category number, C is the freshness category number, C =1,2,3.., C; and K is the total number of teacher models.
In practical application, the overall loss of the A-MKD model is composed of the integration loss of 3 teacher models and the prediction loss of the student models, and the loss function of the egg freshness identification model is L all Comprises the following steps:
L all ＝α·L student +(1-α)·L teacher
wherein alpha is a hyperparameter to balance the influence between the knowledge distillation and the cross entropy loss function; l is student For the cross-entropy loss between the prediction results of the student model and the true label,
c is the total number of freshness categories, C is the freshness category number, C =1,2,3.., C;L teacher for integration loss of the teacher model, L teacher ＝w teacher1 ·L Logits +w teacher2 ·L Feature +w teacher3 ·L RKD ，L Logits As a loss function of the distillation method based on the tag knowledge, w teacher1 Weight assigned to 1 st teacher model, L Feature As a loss function of the distillation method based on knowledge of the intermediate layer, w teacher2 Weight assigned to the 2 nd teacher model, L RKD As a loss function, w, of the distillation method based on structured knowledge teacher3 The weight assigned to the 3 rd teacher model.
Step 104: and training the egg freshness identification model by taking the egg image as input and the egg freshness grade as output to generate a trained egg freshness identification model.
In practical application, the training method for the A-MKD model comprises the following steps: in the model training process, the batch size is uniformly set to 16, and the model trains 200 epochs in total. In order to better converge the model, the test adopts the classified cross entropy as a loss function, a model is trained by using a Stochastic Gradient Descent (SGD) method, the learning rate, the weight attenuation and the momentum of 3 training parameters are respectively set to be 0.001, 0.00001 and 0.9, and a learning rate attenuation strategy is set, and the learning rate is attenuated to 80 percent of the original learning rate every 20 epochs. After 200 epochs were trained, the model was saved.
In practical applications, the step 104 further includes:
and evaluating the trained egg freshness identification model.
The model evaluation method comprises 4 indexes such as Precision (Precision), recall (Recall), weighted score (F1-score) and Accuracy (Accuracy), and the calculation formulas are respectively as follows:
and TP, FP, FN and TN are respectively statistics of classification conditions of eggs with different freshness by the classification model in the confusion matrix. TP (True Positive) represents the number of Positive samples and identified as Positive samples for the True value, FP (False Positive) represents the number of Negative samples and identified as Positive samples for the True value, FN (False Negative) represents the number of Positive samples and identified as Negative samples for the True value, TN (True Negative) represents the number of Negative samples and identified as Negative samples for the True value. When freshness identification is carried out, the actual number of classes of samples to be identified is regarded as the positive number of samples, and the sum of all other classes is the negative number of samples.
Step 105: and identifying the freshness of the eggs to be detected according to the trained egg freshness identification model.
The computer vision technology, the digital image processing technology and the deep learning classification recognition method are integrated, and the mobility and the embeddability are realized. The invention provides an A-MKD-based egg freshness identification method, which combines 3 different knowledge distillation strategies of label knowledge, middle layer knowledge, structural knowledge and the like of 3 teacher models to improve the feature expression capability of the models, and provides a self-adaptive model integration method to further improve the identification effect of student models and realize quick, accurate and low-cost egg freshness identification. Compared with the prior art, the invention has the following advantages:
(1) The number of model parameters and the calculation amount are small.
(2) And the model identification precision is high.
(3) The model identification speed is high.
(4) The model is more convenient to deploy and apply.
Example two
Taking the Beijing white 939 egg produced on the same day provided by a certain laying hen farm in Beijing as an example, the invention provides an egg freshness identification method based on adaptive multi-teacher knowledge distillation, which comprises the following steps:
1) And (4) processing an egg sample. The test sample is the Beijing white 939 egg produced on the same day by a certain layer chicken farm in Beijing. Eggs were sent to the laboratory within 2h after collection. Eggs which are free of stain and crack and have too large or too small mass are removed, and 300 fresh eggs with the mass of 45-65 g are selected as test samples. Eggs were numbered and evenly divided into 6 groups and stored in a constant temperature incubator at 20 ℃ and 75% relative humidity.
In the test process, firstly, images of all eggs are collected every day, and the collected images are stored in a computer according to group numbers; then 2 eggs are respectively selected from each group every day to determine a Hough Unit (HU) value, and the freshness grade corresponding to the egg image is determined according to the HU value. The test was continued every day until the HU value decreased to below 55, and the test was terminated after the freshness was too low to be eaten according to the "fresh eggs and fresh duck eggs grading" of the Ministry of commerce (SB/T10638-2011).
2) And establishing an egg image acquisition environment and acquiring egg images. The image acquisition system mainly comprises a glory V20 smart phone, a 40cm multiplied by 40cm dark box, a 10W LED lamp and a black partition plate with an oval light hole. The camera of the smart phone is a CMOS camera with 4800-ten-thousand pixels as effective pixels, the black partition is placed at a height of 10cm from the bottom of the dark box, and the lens is located at a vertical position 30cm above the light hole. And horizontally placing the egg sample on the light holes to acquire an egg image in a backlight source illumination mode.
3) And (4) image preprocessing. A total of 1638 egg images of 3000 pixels by 4000 pixels were acquired for the experiment. In order to simulate different recognition scenes and enhance the robustness of a model, 1638 egg images are increased by 1 time by randomly adopting a brightness adjustment or rotation data enhancement method, the brightness conversion coefficients are randomly generated in a range of [0.7,1.3], and the rotation range is randomly generated in a range of [ -30 degrees, 30 degrees ] or [150 degrees, 210 degrees ] respectively. The enhanced egg image dataset was 3276 in total and the sample image is shown in fig. 2.
4) Egg Hough Unit determination. Firstly, acquiring the mass w of eggs by using an electronic scale; then, the shell of the egg is broken and placed on a white flat base plate, the height of the concentrated egg white at the middle point of the edge of the egg yolk and the edge of the concentrated egg white is measured by an egg white height measuring instrument (the bridle is avoided), 3 points in a regular triangle shape are measured, and the average value is taken to obtain the height H of the concentrated egg white; and finally, calculating the Hough unit value according to a Hough unit calculation formula. The Hough unit calculation formula is as follows:
HU＝100lg(H-1.7w 0.37 +7.57)
5) And determining the freshness of the eggs. Determining the freshness of the eggs according to the standard of fresh eggs and fresh duck egg classification (SB/T10638-2011), namely, the HU is more than or equal to 72, and AA is the grade eggs; 72 ≧ HU 60 is A-grade egg; 60 ≧ HU 55 is B-grade egg. According to the standard, the test lasts for 16d totally, and the HU value of the sample is reduced from the initial 90 or more to 55 or less. The specific changes are as follows: the initial HU values of the samples on the 1 st day of the test are more than 90, the HU values of the eggs are higher than 72 by the 4 th day, and the freshness of the eggs is AA grade; starting from day 5, the HU value of the eggs is continuously reduced from 72 to day 10, the HU value of the eggs is still higher than 60, and the freshness of the eggs is grade A; the HU value of the eggs is reduced to below 60 but still higher than 55 at days 11-15, and the freshness of the eggs is grade B; on day 16 of the test run, the HU value of the eggs dropped below 55 and the test was complete.
6) And dividing the egg image data set. 3276 egg images obtained in the experiment include AA-level egg images 1064, A-level egg images 1102 and B-level egg images 1110 according to SB/T10638-2011 and the measured HU value of the sample. The method comprises the following steps of dividing a 3276 egg image data set into a training set, a verification set and a test set according to the proportion of 7: 2289 egg images in total are obtained by the training set, and the training set respectively comprises 774, 770 and 775 AA, A and B egg images; 658 egg images in the verification set respectively comprise 213, 222 and 223 AA, A and B egg images; a total of 329 egg images were collected for the test set, containing 107, 110 and 112 AA, A and B egg images, respectively. Table 1 is a table of distribution of egg image data sets, as shown in table 1.
TABLE 1
7) And (5) constructing a model. An Adaptive Multi-Teacher Knowledge Distillation (A-MKD) based egg freshness identification model is shown in FIG. 3.
(1) Knowledge distillation method based on multiple knowledge transfer forms
Distillation method based on tag knowledge:
in the image classification task, the deep neural network model is usually connected with a Softmax output layer after the fully-connected layer, and the output vector of the fully-connected layer is converted into a probability vector, wherein the probability vector represents the probability that the current sample belongs to each category respectively. The basic concept of the knowledge distillation method based on the label is to directly simulate the final output of a teacher model, and the Softmax layer outputs a more gentle probability distribution by introducing a parameter T, wherein the larger the T is, the more gentle the corresponding probability distribution is, and more 'dark knowledge' is distilled out. According to the distillation method based on the label knowledge, the soft labels are introduced, discrete label attributes in the data set are converted into continuous probability distribution, the 'dark knowledge' in the teacher network ResNet50 is extracted and transmitted to the student network, and therefore the performance of the student network model is improved. The distillation method based on the tag knowledge is shown in FIG. 4, and the loss function L is Logits Is defined as:
wherein z = [ z ] 1 ,…,z C ]For each type of egg freshness identification model full connection layer outputA predicted value of freshness, C being the total number of categories of freshness, C being the freshness category number, C =1,2,3, C; t is a temperature coefficient; q (z) c And T) is a softening probability vector of the freshness of the class c;
softening probability vector for teacher model in current freshness category, <' >>
A predicted value of the kth teacher model in the class c freshness category; />
Softening probability vector for student model in current freshness category, <' >>
Predicting values of the student models in the class c freshness category; the corner mark t represents a teacher model; the corner mark s is a student model.
Distillation method based on interlayer knowledge:
in the deep convolutional neural network, the knowledge learned by the network is hierarchical, and the abstraction degree of the knowledge corresponding to the shallow to deep layers is higher and higher. Therefore, the characteristics of the middle layer can also be used as a knowledge carrier for the student network to learn. In order to avoid excessive calculation cost, the invention takes the third to last layer of the teacher model ResNet101 as a guiding layer and the third to last layer of the student model as a guided layer, as shown in FIG. 5. During the training process, a mapping function W needs to be learned r Matching the dimensions of the guided layers of the student model with the guiding layers of the teacher model to obtain the parameter initialization values of the student model at the next stage, and minimizing the Mean Square Error (MSE) value of the teacher model and the student model as the loss L of the distillation method based on the middle layer knowledge Feature The calculation formula is as follows:
wherein f is t (x t ) n-2 And f s (x s ) m-2 Feature vectors output at the third layer from the last are respectively a teacher model and a student model; n is the number of convolution layers of the teacher model; m is the number of convolution layers of the student model; x is the number of t Feature vectors input for the third layer from the last of the teacher model; x is the number of s Inputting a feature vector for the third layer from the last of the student model; w r Is a fully connected layer used for matching the output dimensions of the teacher model and the third to last layer of the student model.
Distillation method based on structured knowledge:
the traditional knowledge distillation method defaults that samples or characteristics are mutually independent, and a point-to-point knowledge transfer mode is adopted; and the structured knowledge distillation focuses on structural representation such as the relationship between samples (inter-class relationship) or the context relationship inside sample characteristics (intra-class relationship). The construction of the structured knowledge has no strict requirement on the specific position of the knowledge, and can be the category of the model output layer or the characteristics of the middle layer.
The invention adopts RKD (Relational Knowledge Distillation) method published in 2019 on CVPR to realize Distillation based on structured Knowledge. The RKD enables student models on the egg image classification task to pay attention to structural association between various types of samples (including intra-class samples and inter-class samples) extracted by a teacher model by establishing angle and distance dual relation measurement between the samples, and therefore more accurate egg freshness identification is achieved. The distillation method based on the structured knowledge is shown in FIG. 6, the loss function L of which RKD Is defined as:
wherein (x) 1 ,…,x n ) Is an n-tuple extracted from the input feature vector X;
a relational potential function giving the relational attributes (angle and euclidean distance) of the n-tuple for the measurement; (f) t (x 1 ),…,f t (x n ) Output feature vectors at any layer for the teacher model; (f) s (x 1 ),…,f s (x n ) Output feature vectors at any layer for the student model; l. the δ To calculate the Huber loss for the teacher model versus the student model, the following is defined:
wherein x represents the vector potential of the relationship attributes (angle and euclidean distance) of the teacher model; y represents the vector potential of the relational attributes (angle and euclidean distance) of the student model.
(2) Adaptive model integration method
The purpose of model integration is to combine predictions from multiple teacher models to improve robustness and generalizability to a single teacher model, thereby improving the accuracy of model identification. Model integration helps to reduce the effects of variance, noise, and bias between predicted and actual values. Most of the current popular methods for model integration are to combine the prediction of multiple teachers with fixed weight distribution or other various label-free schemes, such as calculating weights based on optimization problems or entropy criteria. However, a fixed weight does not distinguish between a high-quality teacher and a low-quality teacher, and other solutions may mislead students in the presence of a low-quality teacher prediction.
Based on the analysis, the invention provides a self-adaptive model integration method, which takes the prediction result of a teacher model into consideration in self-adaptive integration, adaptively allocates larger weight to the teacher model with smaller prediction loss value and allocates smaller weight to the teacher model with larger prediction loss value with the help of a truth value label, so that the student models can learn from a relatively correct direction, and the identification precision of the student models is further improved.
In order to effectively integrate the prediction results of 3 teacher models, the invention respectively calculates the cross entropy loss between the prediction results of 3 teacher models and the real label:
wherein, y c A one-hot vector for class c freshness; z = [ z ] 1 ,…,z C ]The predicted value of each type of freshness output by the egg freshness identification model full connection layer, C is the total freshness category number, C is the freshness category number, and C =1,2,3; q (z) c ) Probability vector of freshness of class c;
probability vector for the kth (k =1,2,3) teacher model in the current freshness category, based on the number of teacher models in the class>
The predicted value for the kth teacher model in the class c freshness category. Different weights are assigned according to the loss values of the 3 teacher models:
wherein L is teacherk The larger the value of (A), the corresponding weight w teacherk The smaller.
(3) Overall loss for the A-MKD model
The overall loss of the A-MKD model consists of two parts, namely integration loss of 3 teacher models and prediction loss of student models. The integration loss of the teacher model is:
L teacher ＝w teacher1 ·L Logits +w teacher2 ·L Feature +w teacher3 ·L RKD
L Logits as a loss function of the distillation method based on the tag knowledge, w teacher1 Weight assigned to 1 st teacher model, L Feature As a loss function of the distillation method based on knowledge of the intermediate layer, w teacher2 Weight assigned to the 2 nd teacher model, L RKD As a loss function of the distillation method based on the structured knowledge, w teacher3 The 3 rd teacher model is assigned a weight.
The prediction loss of the student model is the cross entropy loss between the prediction result of the student model and the real label:
the overall loss for the A-MKD model is:
L all ＝α·L student +(1-α)·L teacher
where α is a hyperparameter to balance the influence between the knowledge distillation and the cross entropy loss function.
The training method for the A-MKD model in the step 8) comprises the following steps: the invention takes the value of the hyper-parameter alpha of the A-MKD model as 0.4. In the model training process, the batch size is uniformly set to 16, and the model trains 200 epochs in total. In order to better converge the model, the test adopts the classified cross entropy as a loss function, a model is trained by using a Stochastic Gradient Descent (SGD) method, the learning rate, the weight attenuation and the momentum of 3 training parameters are respectively set to be 0.001, 0.00001 and 0.9, and a learning rate attenuation strategy is set, and the learning rate is attenuated to 80 percent of the original learning rate every 20 epochs. After 200 epochs were trained, the model was saved.
The model evaluation method in the step 9) comprises 4 indexes such as Precision (Precision), recall (Recall), weighted score (F1-score) and Accuracy (Accuracy), and the calculation formulas are respectively as follows:
in the formula, TP, FP, FN and TN are respectively statistics of classification conditions of eggs with different freshness by the classification model in the confusion matrix. Where TP (True Positive) represents the number of Positive samples and also Positive samples are identified, FP (False Positive) represents the number of Negative samples but Positive samples are identified, FN (False Negative) represents the number of Positive samples but Negative samples are identified, and TN (True Negative) represents the number of Negative samples and also Negative samples are identified. When identifying freshness, the actual number of classes of samples to be identified is regarded as positive number of samples, and the sum of all other classes is negative number of samples.
EXAMPLE III
In order to implement the corresponding method of the above-described embodiments to achieve the corresponding functions and technical effects, an egg freshness identification system is provided below.
An egg freshness identification system comprising:
the device comprises a Hough unit value measuring module, a Hough unit value measuring module and a data processing module, wherein the Hough unit value measuring module is used for continuously collecting egg images every day and measuring the Hough unit value of eggs;
the egg freshness grade determining module is used for determining the egg freshness grade according to the Hough unit value; the egg freshness grades comprise an AA grade, an A grade and a B grade;
the egg freshness identification model building module is used for building an egg freshness identification model based on self-adaptive multi-teacher knowledge distillation; the egg freshness identification model is a student model; the student model combines 3 teacher models of a distillation method based on label knowledge, a distillation method based on middle layer knowledge and a distillation method based on structured knowledge; the student models integrate the prediction results of the 3 teacher models, and adaptively allocate a larger weight to the teacher model with a smaller prediction loss value and allocate a smaller weight to the teacher model with a larger prediction loss value with the help of the truth value label;
the model training module is used for training the egg freshness identification model by taking the egg image as input and the egg freshness grade as output to generate a trained egg freshness identification model;
and the freshness identification module is used for identifying the freshness of the eggs to be detected according to the trained egg freshness identification model.
Example four
An embodiment of the present invention provides an electronic device, which includes a memory and a processor, where the memory is used for storing a computer program, and the processor runs the computer program to enable the electronic device to execute the egg freshness identification method provided in the first embodiment.
In practical applications, the electronic device may be a server.
In practical applications, the electronic device includes: at least one processor (processor), memory (memory), bus, and communication Interface (Communications Interface).
Wherein: the processor, the communication interface, and the memory communicate with each other via a communication bus.
A communication interface for communicating with other devices.
The processor is used for executing the program, and specifically can execute the method described in the above embodiment.
In particular, the program may include program code comprising computer operating instructions.
The processor may be a central processing unit CPU or an Application Specific Integrated Circuit ASIC or one or more Integrated circuits configured to implement embodiments of the present invention. The electronic device comprises one or more processors, which can be the same type of processor, such as one or more CPUs; or may be different types of processors such as one or more CPUs and one or more ASICs.
And the memory is used for storing programs. The memory may comprise high-speed RAM memory, and may also include non-volatile memory (non-volatile memory), such as at least one disk memory.
Based on the above description of the embodiments, the present application provides a storage medium having stored thereon computer program instructions executable by a processor to implement the method of any of the embodiments
The egg freshness identification system provided by the embodiment of the application exists in various forms, including but not limited to:
(1) A mobile communication device: such devices are characterized by mobile communications capabilities and are primarily targeted at providing voice, data communications. Such terminals include: smart phones (e.g., iphones), multimedia phones, functional phones, and low-end phones, among others.
(2) Ultra mobile personal computer device: the equipment belongs to the category of personal computers, has the functions of calculation and processing, and generally has the mobile internet access performance. Such terminals include: PDA, MID, and UMPC devices, etc., such as ipads.
(3) A portable entertainment device: such devices can display and play multimedia content. This type of device comprises: audio, video players (e.g., ipods), handheld game consoles, electronic books, and smart toys and portable car navigation devices.
(4) And other electronic equipment with data interaction function.
Thus, particular embodiments of the present subject matter have been described. Other embodiments are within the scope of the following claims. In some cases, the actions recited in the claims can be performed in a different order and still achieve desirable results. In addition, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some embodiments, multitasking and parallel processing may be advantageous.
The systems, devices, modules or units illustrated in the above embodiments may be implemented by a computer chip or an entity, or by a product with certain functions. One typical implementation device is a computer. In particular, the computer may be, for example, a personal computer, a laptop computer, a cellular telephone, a camera phone, a smartphone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.
For convenience of description, the above devices are described as being divided into various units by function, respectively. Of course, the functionality of the units may be implemented in one or more software and/or hardware when implementing the present application. As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
In a typical configuration, a computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.
The memory may include forms of volatile memory in a computer readable medium, random Access Memory (RAM) and/or non-volatile memory, such as Read Only Memory (ROM) or flash memory (flash RAM). Memory is an example of a computer-readable medium.
Computer-readable media, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static Random Access Memory (SRAM), dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), read Only Memory (ROM), electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM),
Digital Versatile Disks (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices
Or any other non-transmission medium, that may be used to store information that may be accessed by a computing device. As defined herein, computer readable media does not include transitory computer readable media (transmyedia) such as modulated data signals and carrier waves.
It should also be noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrases "comprising a," "8230," "8230," or "comprising" does not exclude the presence of other like elements in a process, method, article, or apparatus comprising the element.
The application may be described in the general context of computer-executable instructions, such as program modules, being executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform particular transactions or implement particular abstract data types. The application may also be practiced in distributed computing environments where transactions are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules may be located in both local and remote computer storage media including memory storage devices.
In the present specification, the embodiments are described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments are referred to each other. For the system disclosed by the embodiment, the description is relatively simple because the system corresponds to the method disclosed by the embodiment, and the relevant points can be referred to the method part for description.
The principles and embodiments of the present invention have been described herein using specific examples, which are provided only to help understand the method and the core concept of the present invention; meanwhile, for those skilled in the art, the invention can be implemented in various embodiments and applications based on the idea of the invention
There are changes. In view of the above, the present disclosure should not be construed as limiting the invention.
Claims (10)
1. An egg freshness identification method is characterized by comprising the following steps:
continuously collecting the egg images every day, and determining the Hough unit value of the eggs;
determining the freshness grade of the eggs according to the Hough unit value; the egg freshness grades comprise an AA grade, an A grade and a B grade;
constructing an egg freshness identification model based on self-adaptive multi-teacher knowledge distillation; the egg freshness identification model is a student model; the student model combines 3 teacher models of a distillation method based on label knowledge, a distillation method based on middle layer knowledge and a distillation method based on structured knowledge; the student models integrate the prediction results of the 3 teacher models, and adaptively allocate a larger weight to the teacher model with a smaller prediction loss value and allocate a smaller weight to the teacher model with a larger prediction loss value with the help of the truth value label;
training the egg freshness identification model by taking the egg image as input and the egg freshness grade as output to generate a trained egg freshness identification model;
and identifying the freshness of the eggs to be detected according to the trained egg freshness identification model.
2. Egg freshness identification method according to claim 1, characterized in that the loss function L of the label knowledge based distillation method Logits Comprises the following steps:
wherein z = [ z ] 1 ,…,z C ]The predicted value of each type of freshness output by the full-connection layer of the egg freshness identification model is CA total number of freshness categories, C being a freshness category number, C =1,2,3.., C; t is a temperature coefficient; q (z) c And T) is a softening probability vector of the freshness of the class c;
a softening probability vector for the teacher model in the current freshness category;
a softening probability vector of the student model in the current freshness category; the corner mark t is a teacher model; the corner mark s is a student model.
3. The egg freshness identification method according to claim 1, wherein the loss function L of the distillation method based on interlayer knowledge Feature Comprises the following steps:
wherein f is t (x t ) n-2 Feature vectors output for the third layer from the last of the teacher model; f. of s (x s ) m-2 Feature vectors output for the penultimate layer of the student model; n is the number of the convolution layers of the teacher model; m is the number of convolution layers of the student model; x is the number of t Inputting a feature vector for the third layer from the last of the teacher model; x is a radical of a fluorine atom s Feature vectors input for the penultimate layer of the student model; w r A fully connected layer, wherein the fully connected layer is used for matching the dimensionality output by the teacher model and the dimensionality output by the third last layer of the student model; the corner mark t is a teacher model; the corner mark s is a student model; and | | is a norm operator.
4. Egg freshness identification method according to claim 1, characterized in that the loss function L of the distillation method based on structured knowledge is RKD Comprises the following steps:
wherein x is 1 ,…,x n For n-tuples extracted from the input feature vector X, n =1,2,3.; phi (-) is a relational potential function that measures relational attributes of a given n-tuple, including angle and Euclidean distance; f. of t x 1 ,…,f t x n Outputting the characteristic vector of the teacher model at any layer; f. of s x 1 ,…,f s x n Outputting feature vectors of the student model at any layer; l. the δ Calculating a Huber loss for the difference between the teacher model and the student model; the corner mark t is a teacher model; the corner mark s is a student model.
5. The egg freshness identification method according to claim 1, wherein the cross entropy loss L between the predicted result of the teacher model and the true value tag is k teacherk Comprises the following steps:
wherein, y c A one-hot vector for class c freshness; c is the total number of freshness categories, C is the freshness category number, C =1,2,3.., C;
for the probability vector of the kth teacher model in class c freshness class, k =1,2,3, <' > is>
The predicted value for the kth teacher model in the class c freshness category.
6. The egg freshness identification method according to claim 1, wherein the kth egg freshness identification methodWeight w of predictive loss value assignment for the teacher model teacherk Comprises the following steps:
wherein C is the total freshness category number, C is the freshness category number, C =1,2,3.., C; and K is the total number of the teacher models.
7. The egg freshness identification method according to claim 1, wherein the loss function of the egg freshness identification model is L all Comprises the following steps:
L all ＝α·L student +(1-α)·L teacher
wherein alpha is a hyper-parameter; l is student For the cross-entropy loss between the prediction results of the student model and the true label,
y c a one-hot vector for a freshness of type c, <' >>
For the probability vector of the student model in the class c freshness category, <' >>
The predicted value of the student model in the freshness category of the class C, C is the total number of freshness categories, C is the number of freshness categories, C =1,2,3.., C; l is a radical of an alcohol teacher For integration loss of the teacher model, L teacher ＝w teacher1 ·L Logits +w teacher2 vL Feature +w teacher3 ·L RKD ，L Logits As a loss function of the distillation method based on the tag knowledge, w teacher1 Weight assigned to 1 st teacher model, L Feature As a loss function of the distillation method based on knowledge of the intermediate layer, w teacher2 Weight assigned to the 2 nd teacher model, L RKD As a loss function of the distillation method based on the structured knowledge, w teacher3 The 3 rd teacher model is assigned a weight.
8. An egg freshness identification system, comprising:
the device comprises a Hough unit value measuring module, a Hough unit value measuring module and a data processing module, wherein the Hough unit value measuring module is used for continuously collecting egg images every day and measuring the Hough unit value of eggs;
the egg freshness grade determining module is used for determining the egg freshness grade according to the Hough unit value; the egg freshness grades comprise an AA grade, an A grade and a B grade;
the egg freshness identification model building module is used for building an egg freshness identification model based on self-adaptive multi-teacher knowledge distillation; the egg freshness identification model is a student model; the student model combines 3 teacher models of a distillation method based on label knowledge, a distillation method based on middle layer knowledge and a distillation method based on structured knowledge; the student models integrate the prediction results of the 3 teacher models, and adaptively allocate a larger weight to the teacher model with a smaller prediction loss value and allocate a smaller weight to the teacher model with a larger prediction loss value with the help of the truth value label;
the model training module is used for training the egg freshness identification model by taking the egg image as input and the egg freshness grade as output to generate a trained egg freshness identification model;
and the freshness identification module is used for identifying the freshness of the eggs to be detected according to the trained egg freshness identification model.
9. An electronic device, characterized by comprising a memory for storing a computer program and a processor for executing the computer program to make the electronic device execute the egg freshness identification method according to any one of claims 1 to 7.
10. A computer-readable storage medium, characterized in that it stores a computer program which, when being executed by a processor, implements the egg freshness identification method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310043469.2A CN115965964B (en) | 2023-01-29 | 2023-01-29 | Egg freshness identification method, system and equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310043469.2A CN115965964B (en) | 2023-01-29 | 2023-01-29 | Egg freshness identification method, system and equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115965964A true CN115965964A (en) | 2023-04-14 |
| CN115965964B CN115965964B (en) | 2024-01-23 |
Family
ID=87354866
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310043469.2A Active CN115965964B (en) | 2023-01-29 | 2023-01-29 | Egg freshness identification method, system and equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115965964B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116152240A (en) * | 2023-04-18 | 2023-05-23 | 厦门微图软件科技有限公司 | Industrial defect detection model compression method based on knowledge distillation |
Citations (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112132803A (en) * | 2020-09-21 | 2020-12-25 | 浙江师范大学 | Egg freshness detection method based on convolutional neural network |
| CN112529178A (en) * | 2020-12-09 | 2021-03-19 | 中国科学院国家空间科学中心 | Knowledge distillation method and system suitable for detection model without preselection frame |
| CN112784964A (en) * | 2021-01-27 | 2021-05-11 | 西安电子科技大学 | Image classification method based on bridging knowledge distillation convolution neural network |
| CN113257361A (en) * | 2021-05-31 | 2021-08-13 | 中国科学院深圳先进技术研究院 | Method, device and equipment for realizing self-adaptive protein prediction framework |
| CN113298249A (en) * | 2020-11-16 | 2021-08-24 | 鹏城实验室 | Structured knowledge distillation method, device, equipment and computer readable storage medium |
| CN113627545A (en) * | 2021-08-16 | 2021-11-09 | 山东大学 | Image classification method and system based on isomorphic multi-teacher guidance knowledge distillation |
| CN114049513A (en) * | 2021-09-24 | 2022-02-15 | 中国科学院信息工程研究所 | Knowledge distillation method and system based on multi-student discussion |
| CN114241282A (en) * | 2021-11-04 | 2022-03-25 | 河南工业大学 | Knowledge distillation-based edge equipment scene identification method and device |
| CN114266897A (en) * | 2021-12-24 | 2022-04-01 | 深圳数联天下智能科技有限公司 | Method and device for predicting pox types, electronic equipment and storage medium |
| KR20220096099A (en) * | 2020-12-30 | 2022-07-07 | 성균관대학교산학협력단 | Method and apparatus for learning of teacher assisted attention transfer using total cam information in knowledge distillation |
| CN114898165A (en) * | 2022-06-20 | 2022-08-12 | 哈尔滨工业大学 | Deep learning knowledge distillation method based on model channel cutting |
| US20220351043A1 (en) * | 2021-04-30 | 2022-11-03 | Chongqing University | Adaptive high-precision compression method and system based on convolutional neural network model |
| CN115331285A (en) * | 2022-07-29 | 2022-11-11 | 南京邮电大学 | Dynamic expression recognition method and system based on multi-scale feature knowledge distillation |
| CN115393671A (en) * | 2022-08-25 | 2022-11-25 | 河海大学 | Rock class prediction method based on multi-teacher knowledge distillation and normalized attention |
-
2023
- 2023-01-29 CN CN202310043469.2A patent/CN115965964B/en active Active
Patent Citations (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112132803A (en) * | 2020-09-21 | 2020-12-25 | 浙江师范大学 | Egg freshness detection method based on convolutional neural network |
| CN113298249A (en) * | 2020-11-16 | 2021-08-24 | 鹏城实验室 | Structured knowledge distillation method, device, equipment and computer readable storage medium |
| CN112529178A (en) * | 2020-12-09 | 2021-03-19 | 中国科学院国家空间科学中心 | Knowledge distillation method and system suitable for detection model without preselection frame |
| KR20220096099A (en) * | 2020-12-30 | 2022-07-07 | 성균관대학교산학협력단 | Method and apparatus for learning of teacher assisted attention transfer using total cam information in knowledge distillation |
| CN112784964A (en) * | 2021-01-27 | 2021-05-11 | 西安电子科技大学 | Image classification method based on bridging knowledge distillation convolution neural network |
| US20220351043A1 (en) * | 2021-04-30 | 2022-11-03 | Chongqing University | Adaptive high-precision compression method and system based on convolutional neural network model |
| CN113257361A (en) * | 2021-05-31 | 2021-08-13 | 中国科学院深圳先进技术研究院 | Method, device and equipment for realizing self-adaptive protein prediction framework |
| CN113627545A (en) * | 2021-08-16 | 2021-11-09 | 山东大学 | Image classification method and system based on isomorphic multi-teacher guidance knowledge distillation |
| CN114049513A (en) * | 2021-09-24 | 2022-02-15 | 中国科学院信息工程研究所 | Knowledge distillation method and system based on multi-student discussion |
| CN114241282A (en) * | 2021-11-04 | 2022-03-25 | 河南工业大学 | Knowledge distillation-based edge equipment scene identification method and device |
| CN114266897A (en) * | 2021-12-24 | 2022-04-01 | 深圳数联天下智能科技有限公司 | Method and device for predicting pox types, electronic equipment and storage medium |
| CN114898165A (en) * | 2022-06-20 | 2022-08-12 | 哈尔滨工业大学 | Deep learning knowledge distillation method based on model channel cutting |
| CN115331285A (en) * | 2022-07-29 | 2022-11-11 | 南京邮电大学 | Dynamic expression recognition method and system based on multi-scale feature knowledge distillation |
| CN115393671A (en) * | 2022-08-25 | 2022-11-25 | 河海大学 | Rock class prediction method based on multi-teacher knowledge distillation and normalized attention |
Non-Patent Citations (4)
| Title |
|---|
| JIKUN KANG等: "A Generalized Load Balancing Policy With Multi-Teacher Reinforcement Learning", 2022 IEEE GLOBAL COMMUNICATIONS CONFERENCE: SELECTED AREAS IN COMMUNICATIONS: MACHINE LEARNING FOR COMMUNICATIONS, pages 3096 - 3101 * |
| YUANG LIU等: "Adaptive Multi-Teacher Multi-level Knowledge Distillation", ARXIV:2103.04062V1 [CS.CV] 6 MAR 2021, pages 1 - 10 * |
| 邢志中;张海东;王孟;翟超男;郭小军;陈腾;: "基于计算机视觉和神经网络的鸡蛋新鲜度检测", 江苏农业科学, no. 11, pages 168 - 171 * |
| 邵仁荣: "深度学习中知识蒸馏研究综", 计算机学报, vol. 45, no. 8, pages 1638 - 1673 * |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116152240A (en) * | 2023-04-18 | 2023-05-23 | 厦门微图软件科技有限公司 | Industrial defect detection model compression method based on knowledge distillation |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115965964B (en) | 2024-01-23 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109961089A (en) | Small sample and zero sample image classification method based on metric learning and meta learning | |
| CN106203521B (en) | The SAR image change detection learnt based on disparity map from step | |
| CN115965964B (en) | Egg freshness identification method, system and equipment | |
| CN113239916B (en) | Expression recognition and classroom state evaluation method, device and medium | |
| CN110443262A (en) | Fish body freshness fast non-destructive detection method and its device based on computer vision | |
| CN112016315A (en) | Model training method, text recognition method, model training device, text recognition device, electronic equipment and storage medium | |
| CN116110022B (en) | Lightweight traffic sign detection method and system based on response knowledge distillation | |
| CN111598167B (en) | Small sample image identification method and system based on graph learning | |
| US20200364928A1 (en) | Methods and apparatus to generate a three-dimensional (3d) model for 3d scene reconstruction | |
| CN113554653A (en) | Semantic segmentation method for long-tail distribution of point cloud data based on mutual information calibration | |
| CN112308746A (en) | Teaching state evaluation method and device and electronic equipment | |
| CN116994295B (en) | Wild animal category identification method based on gray sample self-adaptive selection gate | |
| CN114090839A (en) | Method, system, device and storage medium for learner cognitive structure processing | |
| CN113592008A (en) | System, method, equipment and storage medium for solving small sample image classification based on graph neural network mechanism of self-encoder | |
| CN112330512A (en) | Prediction method, system, equipment and storage medium of knowledge distillation learning model | |
| Gong et al. | KDCTime: Knowledge distillation with calibration on InceptionTime for time-series classification | |
| CN109800880B (en) | Self-adaptive learning feature extraction system based on dynamic learning style information and application | |
| CN112183163A (en) | Natural scene text detection method based on full convolution residual error network | |
| CN115438871A (en) | Ice and snow scenic spot recommendation method and system integrating preference and eliminating popularity deviation | |
| CN108921405A (en) | Accurate learning evaluation method and robot system based on big data and artificial intelligence | |
| CN115331021A (en) | Dynamic feature extraction and description method based on multilayer feature self-difference fusion | |
| CN114743619A (en) | Questionnaire quality evaluation method and system for disease risk prediction | |
| CN114359606A (en) | Deep learning-based student classroom behavior detection method, system and terminal | |
| CN117576381B (en) | Target detection training method, electronic device and computer readable storage medium | |
| CN109726690A (en) | Learner behavior image multizone based on DenseCap network describes method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |