CN115017351A - Light-weight industrial picture classification method and system based on federal small sample learning - Google Patents

Light-weight industrial picture classification method and system based on federal small sample learning Download PDF

Info

Publication number
CN115017351A
CN115017351A CN202210611106.XA CN202210611106A CN115017351A CN 115017351 A CN115017351 A CN 115017351A CN 202210611106 A CN202210611106 A CN 202210611106A CN 115017351 A CN115017351 A CN 115017351A
Authority
CN
China
Prior art keywords
small sample
classifier
industrial
training
federal
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202210611106.XA
Other languages
Chinese (zh)
Other versions
CN115017351B (en
Inventor
杨树森
赵聪
赵鹏
孙心悦
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Hangzhou Cumulus Technology Co ltd
Original Assignee
Hangzhou Cumulus Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Hangzhou Cumulus Technology Co ltd filed Critical Hangzhou Cumulus Technology Co ltd
Priority to CN202210611106.XA priority Critical patent/CN115017351B/en
Publication of CN115017351A publication Critical patent/CN115017351A/en
Application granted granted Critical
Publication of CN115017351B publication Critical patent/CN115017351B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
    • G06F16/55—Clustering; Classification
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N20/00—Machine learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
    • Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
    • Y02P90/30—Computing systems specially adapted for manufacturing

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Software Systems (AREA)
  • General Physics & Mathematics (AREA)
  • Physics & Mathematics (AREA)
  • Computing Systems (AREA)
  • Artificial Intelligence (AREA)
  • Evolutionary Computation (AREA)
  • Medical Informatics (AREA)
  • Databases & Information Systems (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Multimedia (AREA)
  • Data Mining & Analysis (AREA)
  • General Engineering & Computer Science (AREA)
  • Mathematical Physics (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
  • Image Analysis (AREA)

Abstract

A lightweight industrial picture classification method and system based on federal small sample learning comprise the following steps: modeling an industrial picture classification system based on a federal small sample learning framework; task for classifying small sample pictures
Figure DDA0003673166430000011
Modeling is carried out; build by embedding module l θ And relation module g φ A small sample classifier f based on a lightweight relational network is formed; picture classification task based on small sample
Figure DDA0003673166430000012
Construction ofA training method of a small sample classifier f; constructing a side cloud collaborative deployment and training method of the small sample classifier based on a small sample classifier training method; and performing model reasoning locally based on the edge cloud collaborative deployment and training method of the small sample classifier. The invention fully utilizes the heterogeneous and mutually isolated limited sample sets of different industrial mechanisms to realize the effective training of the overall small sample picture classifier, is also suitable for external clients holding samples which are not similar during the training period, and supports the industrial mechanisms with limited samples to establish the data-oriented picture classifier.

Description

Light-weight industrial picture classification method and system based on federal small sample learning
Technical Field
The invention belongs to the field of distributed intelligence, and particularly relates to a method and a system for classifying light-weight industrial pictures based on federal small sample learning.
Background
Currently, Deep Convolutional Neural Network (DCNN) based picture classification is widely applied in different industrial scenes. In the existing method, a DCNN classifier which is intensive in calculation is deployed at the cloud, and each industrial mechanism uploads an original picture to be classified. However, this approach may reveal the privacy of the industrial establishment. In addition, cloud brings high network communication burden on large-scale industrial pictures. In order to solve the problem, the federal learning technology is generally concerned, and for the problem of industrial picture classification, different industrial institutions can serve as clients to uniformly coordinate and cooperatively train a global DCNN classifier through the cloud on the premise of not uploading original pictures. However, the sample scarcity problem (including insufficient sample number and sample class isolation) faced by practical industrial institutions has limited the application of existing data-intensive federal learning methods in industrial picture classification scenarios. Aiming at the problem of sample scarcity, each industrial mechanism can adopt the existing single-machine small sample learning technology to obtain a small sample picture classifier based on limited data training locally, but due to strict data isolation among the industrial mechanisms, the model obtained by training cannot be generalized on other types of samples, and under the condition, the industrial mechanism with extremely limited samples cannot establish an available picture classifier. In addition, practical industrial institutions often have heterogeneous and limited computing and network resources, and cannot effectively support the existing resource-intensive DCNN image classification method.
Disclosure of Invention
The invention aims to provide a federate small sample learning-based lightweight industrial picture classification method and system, and aims to solve the problems that an industrial mechanism with extremely limited samples cannot establish an available picture classifier, and an actual industrial mechanism usually has heterogeneous and limited computing and network resources and cannot effectively support the existing resource-intensive DCNN picture classification method.
In order to achieve the purpose, the invention adopts the following technical scheme:
the method for classifying the light-weight industrial pictures based on the federal small sample learning comprises the following steps:
modeling is carried out on an industrial picture classification system based on a federal small sample learning framework, wherein the system comprises a cloud server c and a group of industrial institutions epsilon with the same interest, and each e i E epsilon holds respective local sample sets separately
Figure BDA0003673166410000021
For each industrial mechanism in the constructed system model, classifying the small sample pictures
Figure BDA0003673166410000022
Modeling is performed, each task comprising a support set
Figure BDA0003673166410000023
And query set
Figure BDA0003673166410000024
Build by embedding module l θ And relation module g φ The small sample classifier f based on the lightweight relational network is formed, wherein theta and phi are parameters of functions l and g;
picture classification task based on small sample
Figure BDA0003673166410000025
Constructing a training method of a small sample classifier f;
constructing a side cloud collaborative deployment and training method of the small sample classifier based on a small sample classifier training method, wherein the complete small sample image classifier based on a lightweight relation network is deployed on c, and an embedded module part is arrangedIs deployed at all e i E, on epsilon, training the classifier based on the training method in the step 4 along a discrete time point T of 0,1, … and T, wherein T is the iteration number of a preset model;
edge cloud collaborative deployment and training method based on small sample classifier, and small sample classifier deployed and trained to be constructed
Figure BDA0003673166410000026
After training is completed, each external client is initialized
Figure BDA0003673166410000027
Model (2)
Figure BDA0003673166410000028
Model reasoning is performed locally.
Furthermore, a federal small sample learning technology is utilized to model an industrial picture classification system, privacy of each industrial organization is guaranteed, meanwhile, training of a global small sample picture classifier of an industrial picture island which is heterogeneous and sample-limited is supported, and the obtained global picture classifier is suitable for not only federal model training participation clients, but also external clients holding unseen samples during training.
Further, the specific operation is as follows: the industrial picture classification system modeling based on the federal small sample learning comprises a cloud server c and a group of industrial institutions epsilon-e with the same interest i Wherein
Figure BDA0003673166410000029
Figure BDA00036731664100000210
Each e i E epsilon holds respective local sample sets separately
Figure BDA00036731664100000211
Wherein e is i Hold with
Figure BDA0003673166410000031
The class samples are, for example,
Figure BDA0003673166410000032
Figure BDA0003673166410000033
is e i Class j samples of (1), for all e i ,e j ∈ε，i≠j，
Figure BDA0003673166410000034
c can receive messages sent by epsilon but cannot access the sample set of epsilon, epsilon can receive messages sent by c but the interior of epsilon cannot be communicated, and a federal small sample picture classifier is set to train and participate in a client set
Figure BDA0003673166410000035
External client set
Figure BDA0003673166410000036
Further, the specific operation of modeling the small sample picture classification task T is as follows: c-way K-shot picture classification task on classifier f
Figure BDA0003673166410000037
Including support sets
Figure BDA0003673166410000038
And query set
Figure BDA0003673166410000039
Figure BDA00036731664100000310
The sample containing C x K labeled samples,
Figure BDA00036731664100000311
containing C × K' samples to be classified (, requirement f is based on
Figure BDA00036731664100000312
For is to
Figure BDA00036731664100000313
And (6) classifying.
Furthermore, a small sample image classifier f based on the lightweight relational network is constructed by utilizing a relational network technology.
Further, the specific operation is as follows: constructing a multilayer CNN model f by an embedded module l θ And relation module g φ Composition, where θ, φ are parameters of functions l, g, l θ Extracting a sample feature map of an input image, wherein the sample feature map comprises two convolution blocks, each convolution block comprises 32 convolution layers with 3 multiplied by 3 filters, a batch normalization layer, a ReLU nonlinear layer and a 2 multiplied by 2 maximum pooling layer; g φ And determining whether two input samples belong to the same class or not based on the similarity of the characteristic graphs, wherein the two input samples comprise the two volume blocks, an 8-dimensional full-connection layer based on the ReLU and a 1-dimensional full-connection layer based on the Sigmoid, and for single-channel and three-channel images, the output size H of the last maximum pooling layer is 32 and 32 multiplied by 3.
Further, the training method of the small sample classifier f specifically operates as follows: small sample picture classification task based on
Figure BDA00036731664100000314
For any sample pair
Figure BDA00036731664100000315
Obtaining a feature map l based on an embedding module θ (x a )、l θ (x b ) Computing a relationship score based on the relationship module
Figure BDA00036731664100000316
Wherein,
Figure BDA00036731664100000317
for feature graph join operations, relationship scores for all sample pairs are organized using a relationship matrix, with classifier loss
Figure BDA00036731664100000318
Wherein:
Figure BDA00036731664100000319
updating model parameters theta by random gradient descent * ,φ * 。
Further, a side cloud collaborative deployment and training method of the small sample picture classifier based on the lightweight relational network is constructed by using a side cloud collaborative intelligent technology.
Further, the method comprises the following steps:
step 5.1, setting the current iteration times t ← 0, and initializing a cloud model
Figure BDA0003673166410000041
Wherein theta is init 、φ init Initializing each participating client for random initialization parameters
Figure BDA0003673166410000042
Model parameters of
Figure BDA0003673166410000043
Each participating client e i ∈ε i Organizing local task sets
Figure BDA0003673166410000044
Updating the current iteration time t ← 1;
step 5.2, each participating client e i ∈ε i From
Figure BDA0003673166410000045
Random extraction task
Figure BDA0003673166410000046
Obtaining all sample pairs
Figure BDA0003673166410000047
Characteristic diagram of
Figure BDA0003673166410000048
Upload all featuresSignature and corresponding sample label to c;
step 5.3, for each participating client e i ∈ε i Based on step 4, c calculation
Figure BDA0003673166410000049
Updating
Figure BDA00036731664100000410
Where α is the random gradient descent learning rate, given to each participating client e i ∈ε i Return theta g (t+1)；
Step 5.4, each participating client e i ∈ε i Updating
Figure BDA00036731664100000411
Step 5.5, updating the current iteration time T ← T +1, and if T is less than or equal to T, repeating the steps from 5.2 to 5.4;
step 5.6, outputting the training result
Figure BDA00036731664100000412
Further, a federal small sample learning-based lightweight industrial picture classification system includes:
the system modeling module is used for modeling an industrial picture classification system based on a federal small sample learning framework, and comprises a cloud server c and a group of industrial institutions epsilon with the same interest, and each e i E epsilon holds respective local sample sets separately
Figure BDA00036731664100000413
A small sample picture classification task modeling module used for classifying small sample pictures for each industrial mechanism in the constructed system model
Figure BDA00036731664100000414
Modeling is performed, each task comprising a support set
Figure BDA00036731664100000415
And query set
Figure BDA00036731664100000416
A classifier building module for building a classifier from the embedded module θ And relation module g φ The small sample classifier f based on the lightweight relational network is formed, wherein theta and phi are parameters of functions l and g;
a training module for classifying tasks based on small sample pictures
Figure BDA00036731664100000417
Constructing a training method of a small sample classifier f; constructing an edge cloud collaborative deployment and training method of the small sample classifier based on a small sample classifier training method, wherein the complete small sample image classifier based on a lightweight relational network is deployed on c, and an embedded module is deployed on all e i E, on epsilon, training the classifier based on the training method in the step 4 along a discrete time point T of 0,1, … and T, wherein T is the iteration number of a preset model;
the output reasoning module is used for deploying and training the constructed small sample classifier based on the edge cloud collaborative deployment and training method of the small sample classifier
Figure BDA0003673166410000051
After training is completed, each external client is initialized
Figure BDA0003673166410000052
Model (2)
Figure BDA0003673166410000053
Model reasoning is performed locally.
Compared with the prior art, the invention has the following technical effects:
according to the invention, by establishing an industrial picture classification system model based on federal small sample learning, while not invading the privacy of industrial institutions, the effective training of a global small sample picture classifier is realized by fully utilizing heterogeneous and mutually isolated limited sample sets of different industrial institutions, and the obtained global model is not only suitable for clients participated in federal model training, but also suitable for external clients holding samples which are not classified during training, and supports the industrial institutions with limited samples to establish the data-oriented picture classifier.
Furthermore, by constructing the small sample picture classifier based on the lightweight relational network, the calculation and storage overhead of an industrial mechanism with resource limitation in the model training process is obviously reduced by utilizing a lighter model structure design while the model classification precision is ensured.
Furthermore, by establishing a side cloud collaborative deployment and training method of the small sample picture classifier based on the lightweight relational network, the model splitting precision is guaranteed, and meanwhile, only a communication lightweight embedded module is deployed at the client side of the industrial mechanism, so that the communication overhead of the industrial mechanism with resource limitation in the model training process is remarkably reduced.
Drawings
FIG. 1 is a schematic representation of an implementation of the process herein;
FIG. 2 is a logic flow diagram of the method herein;
Detailed Description
The present invention is described in further detail below with reference to the attached drawing figures.
Referring to fig. 1, the invention provides a lightweight industrial picture classification method based on federal small sample learning, which comprises the following steps:
step 1, modeling a federal small sample industrial picture classification system, wherein the system comprises a cloud server c and a group of global picture classifiers f with the same interests (hope of being cooperatively trained and shared) g ) E (federal learning client) e i Wherein
Figure BDA0003673166410000061
Each e i E epsilon holds respective local sample sets separately
Figure BDA0003673166410000062
Wherein e is i Hold with
Figure BDA0003673166410000063
The class samples are, for example,
Figure BDA0003673166410000064
Figure BDA0003673166410000065
is e i Class j samples above, due to class isolation between actual industrial facilities, for all e i ,e j ∈ε，i≠j，
Figure BDA0003673166410000066
c can receive the message sent by epsilon but cannot access the sample set of epsilon, epsilon can receive the message sent by c but can not intercommunicate inside epsilon, and a federal small sample picture classifier is set to train and participate in the client set
Figure BDA0003673166410000067
External client set
Figure BDA0003673166410000068
Step 2, modeling of small sample picture classification tasks, and C-way K-shot picture classification tasks on a classifier f
Figure BDA0003673166410000069
Including support sets
Figure BDA00036731664100000610
And query set
Figure BDA00036731664100000611
Figure BDA00036731664100000612
Containing C x K labeled samples (K samples for each of the C classes),
Figure BDA00036731664100000613
containing C K 'samples to be classified (K' samples from each of the above C classes), the requirement f is based on
Figure BDA00036731664100000614
To pair
Figure BDA00036731664100000615
Classifying;
step 3, the small sample picture classifier based on the lightweight relational network is a multilayer CNN model f and is formed by an embedded module l θ And relation module g φ Composition, where θ, φ are parameters of functions l, g, l θ Extracting a sample feature map of an input image, wherein the sample feature map comprises two convolution blocks, each convolution block comprises 32 convolution layers with 3 multiplied by 3 filters, a batch normalization layer, a ReLU nonlinear layer and a 2 multiplied by 2 maximum pooling layer; g φ Determining whether two input samples belong to the same class or not based on the similarity (relationship score) of the feature map, wherein the two input samples comprise the two volume blocks, an 8-dimensional full-connection layer based on the ReLU and a 1-dimensional full-connection layer based on the Sigmoid, and for single-channel and three-channel images, the output size H of the last maximum pooling layer is 32 and 32 x 3;
step 4, training the small sample picture classifier based on the lightweight relation network, and classifying the task based on the small sample picture in the step 2
Figure BDA00036731664100000616
For any sample pair
Figure BDA00036731664100000617
Obtaining a feature map l based on an embedding module θ (x a )、l θ (x b ) Computing a relationship score based on the relationship module
Figure BDA00036731664100000618
Wherein,
Figure BDA00036731664100000619
for feature graph join operations, relationship scores for all sample pairs are organized using a relationship matrix, with classifier penalties
Figure BDA00036731664100000620
Wherein:
Figure BDA0003673166410000071
updating model parameters theta by random gradient descent * ,φ * ；
Step 5, edge cloud collaborative deployment and training of the small sample picture classifier based on the lightweight relational network, in order to reduce calculation, communication and storage expenses of a client in the Federal model training process, the complete small sample picture classifier based on the lightweight relational network is deployed on the c, and the embedded module is deployed on all e i E epsilon, training the classifier along the discrete time point T of 0,1, …, T, wherein T is the preset iteration number of the model, and the step 5 comprises the following steps:
step 5.1, setting the current iteration times t ← 0, and initializing a cloud model
Figure BDA0003673166410000072
Wherein theta is init 、φ init Initializing each participating client for random initialization parameters
Figure BDA0003673166410000073
Model parameters of
Figure BDA0003673166410000074
Each participating client e i ∈ε i Organizing local task sets
Figure BDA0003673166410000075
Updating the current iteration time t ← 1;
step 5.2, each participating client e i ∈ε i From
Figure BDA0003673166410000076
Random extraction task
Figure BDA0003673166410000077
Based on step 4, all samples were obtainedBook pair
Figure BDA0003673166410000078
Characteristic diagram of
Figure BDA0003673166410000079
Uploading all feature maps and corresponding sample labels to c;
step 5.3, for each participating client e i ∈ε i Based on step 4, c calculation
Figure BDA00036731664100000710
Updating
Figure BDA00036731664100000711
Wherein alpha is a random gradient descent learning rate, and gives each participating client e i ∈ε i Return theta g (t+1)；
Step 5.4, each participating client e i ∈ε i Updating
Figure BDA00036731664100000712
Step 5.5, updating the current iteration time T ← T +1, and if T is less than or equal to T, repeating the steps from 5.2 to 5.4;
step 5.6, outputting the training result
Figure BDA00036731664100000713
Step 6, initializing each external client
Figure BDA00036731664100000714
Model (2)
Figure BDA00036731664100000715
Model reasoning is performed locally.
Referring to fig. 2, the invention provides a lightweight industrial picture classification method based on federal small sample learning, and a logic architecture body of the method comprises modeling of an industrial picture classification system based on federal small sample learning, construction of a small sample picture classifier based on a lightweight relational network, and edge cloud collaborative deployment and training of the small sample picture classifier based on the lightweight relational network. In order to solve the problem that an industrial mechanism with limited samples cannot effectively establish a data-driven industrial picture classifier, an industrial picture classification system based on federal small sample learning is established, and global small sample picture classifier training of an industrial picture island which is heterogeneous and has limited samples is supported; in order to reduce the calculation and storage cost of an industrial mechanism with limited resources in the model training process, a small sample picture classifier based on a lightweight relational network is constructed; in order to reduce communication overhead of an industrial mechanism with limited resources in the model training process, a side cloud collaborative deployment and training method of a small sample picture classifier based on a lightweight relational network is established.
In another embodiment of the present invention, a light-weight industrial picture classification system based on federal small sample learning is provided, which can be used to implement the light-weight industrial picture classification method based on federal small sample learning described above, and specifically, the system includes:
the system modeling module is used for modeling an industrial picture classification system based on a federal small sample learning framework, and comprises a cloud server c and a group of industrial institutions epsilon with the same interest, and each e i Separately holding respective local sample sets
Figure BDA0003673166410000081
A small sample picture classification task modeling module used for classifying small sample pictures for each industrial mechanism in the constructed system model
Figure BDA0003673166410000088
Modeling is performed, each task comprising a support set
Figure BDA0003673166410000082
And query set
Figure BDA0003673166410000083
A classifier building module for building a classifier from the embedded module θ And relation module g φ The small sample classifier f based on the lightweight relational network is formed, wherein theta and phi are parameters of functions l and g;
a training module for classifying tasks based on small sample pictures
Figure BDA0003673166410000087
Constructing a training method of a small sample classifier f; constructing an edge cloud collaborative deployment and training method of the small sample classifier based on a small sample classifier training method, wherein the complete small sample image classifier based on a lightweight relational network is deployed on c, and an embedded module is deployed on all e i E, on epsilon, training the classifier based on the training method in the step 4 along a discrete time point T of 0,1, … and T, wherein T is the iteration number of a preset model;
the output reasoning module is used for deploying and training the constructed small sample classifier based on the edge cloud collaborative deployment and training method of the small sample classifier
Figure BDA0003673166410000084
After training is completed, each external client is initialized
Figure BDA0003673166410000085
Model (2)
Figure BDA0003673166410000086
Model reasoning is performed locally.
The invention solves the problem that an available data-driven picture classifier cannot be established by a scarce industrial mechanism with sample isolation under the requirement of strong privacy protection. The invention can effectively train the global small sample picture classifier by crossing heterogeneous industrial picture islands with limited samples on the premise of ensuring the privacy of industrial institutions, and the obtained global model is not only suitable for clients participated in the federal model training, but also suitable for external clients holding samples which are not classified during the training. The small sample picture classifier based on the lightweight relational network and the edge cloud collaborative deployment and training method thereof can ensure the classification precision of the model and simultaneously remarkably reduce the calculation, communication and storage expenses of the resource-limited industrial mechanism.
Finally, it should be noted that: the above embodiments are only for illustrating the technical solutions of the present invention and not for limiting the same, and although the present invention is described in detail with reference to the above embodiments, those of ordinary skill in the art should understand that: modifications and equivalents may be made to the embodiments of the invention without departing from the spirit and scope of the invention, which is to be covered by the claims.

Claims (10)

1. The method for classifying the light-weight industrial pictures based on the federal small sample learning is characterized by comprising the following steps of:
modeling an industrial picture classification system based on a federal small sample learning framework, wherein the system comprises a cloud server c and a group of industrial institutions epsilon with same interests, and each e i E epsilon holds respective local sample sets separately
Figure FDA0003673166400000011
For each industrial mechanism in the constructed system model, classifying the small sample pictures
Figure FDA0003673166400000012
Modeling is performed, each task comprising a support set
Figure FDA0003673166400000013
And query set
Figure FDA0003673166400000014
Build by embedding module l θ And relation module g φ The small sample classifier f based on the lightweight relational network is formed, wherein theta and phi are parameters of functions l and g;
picture classification task based on small sample
Figure FDA0003673166400000015
Constructing a training method of a small sample classifier f;
constructing an edge cloud collaborative deployment and training method of the small sample classifier based on a small sample classifier training method, wherein the complete small sample image classifier based on a lightweight relational network is deployed on c, and an embedded module is deployed on all e i E, on epsilon, training the classifier based on the training method in the step 4 along a discrete time point T of 0,1, … and T, wherein T is the iteration number of a preset model;
edge cloud collaborative deployment and training method based on small sample classifier, and small sample classifier deployed and trained to be constructed
Figure FDA0003673166400000016
After training is completed, each external client is initialized
Figure FDA0003673166400000017
Model (2)
Figure FDA0003673166400000018
Model reasoning is performed locally.
2. The lightweight industrial picture classification method based on federal small sample learning as claimed in claim 1, wherein the federal small sample learning technology is used for modeling an industrial picture classification system, privacy of each industrial institution is guaranteed, training of a global small sample picture classifier of an industrial picture island which is heterogeneous and has limited samples is supported, and the obtained global picture classifier is suitable for both federal model training participating clients and external clients holding unseen samples during training.
3. The method for classifying a lightweight industrial picture based on federal small sample learning as claimed in claim 2, wherein the specific operation is: the industrial picture classification system modeling based on the federal small sample learning comprises a cloud server c and a group of industrial institutions with the same interest, wherein the industrial institutions are epsilon or e i Wherein
Figure FDA0003673166400000019
Figure FDA00036731664000000110
Each e i E epsilon holds respective local sample sets separately
Figure FDA00036731664000000111
Wherein e is i Hold with
Figure FDA0003673166400000021
The class samples are, for example,
Figure FDA0003673166400000022
Figure FDA0003673166400000023
is e i Class j samples of (1), for all e i ,e j ∈ε，i≠j，
Figure FDA0003673166400000024
c can receive the message sent by epsilon but cannot access the sample set of epsilon, epsilon can receive the message sent by c but can not intercommunicate inside epsilon, and a federal small sample picture classifier is set to train and participate in the client set
Figure FDA0003673166400000025
External client set
Figure FDA0003673166400000026
4. The method for classifying light-weight industrial pictures based on federal small sample learning as claimed in claim 1, wherein the specific operations for modeling the small sample picture classification task T are as follows: c-way K-shot picture classification task on classifier f
Figure FDA0003673166400000027
Including support sets
Figure FDA0003673166400000028
And query set
Figure FDA0003673166400000029
Figure FDA00036731664000000210
The sample containing C x K labeled samples,
Figure FDA00036731664000000211
containing C × K' samples to be classified (, requirement f is based on
Figure FDA00036731664000000212
To pair
Figure FDA00036731664000000213
And (6) classifying.
5. The method of claim 1, wherein the small sample classifier f based on the lightweight relational network is constructed by using a relational network technology.
6. The method for classifying a lightweight industrial picture based on federal small sample learning as claimed in claim 5, wherein the specific operation is: constructing a multilayer CNN model f by an embedded module l θ And relation module g φ Composition, where θ, φ are parameters of functions l, g, l θ Extracting a sample feature map of an input image, wherein the sample feature map comprises two convolution blocks, each convolution block comprises 32 convolution layers with 3 multiplied by 3 filters, a batch normalization layer, a ReLU nonlinear layer and a 2 multiplied by 2 maximum pooling layer; g φ Determining whether two input samples belong to the same class based on the similarity of the feature maps, including the aboveTwo volume blocks, an 8-dimensional full-connected layer based on ReLU and a 1-dimensional full-connected layer based on Sigmoid, and for single-channel and three-channel images, the output size H of the last maximum pooling layer is 32 and 32 multiplied by 3.
7. The lightweight industrial picture classification method based on federal small sample learning as claimed in claim 1, wherein the training method of the small sample classifier f comprises the following specific operations: small sample picture classification task based on
Figure FDA00036731664000000214
For any sample pair
Figure FDA00036731664000000215
Obtaining a feature map l based on an embedding module θ (x a )、l θ (x b ) Computing a relationship score based on the relationship module
Figure FDA00036731664000000216
Wherein,
Figure FDA00036731664000000217
for feature graph join operations, relationship scores for all sample pairs are organized using a relationship matrix, with classifier loss
Figure FDA00036731664000000218
Wherein:
Figure FDA0003673166400000031
updating model parameters theta by random gradient descent * ,φ * 。
8. The method of claim 1, wherein a side cloud collaborative deployment and training method of a small sample picture classifier based on a lightweight relational network is constructed by using a side cloud collaborative intelligence technology.
9. The method for classifying a lightweight industrial picture based on federal small sample learning as claimed in claim 8, comprising the steps of:
step 5.1, setting the current iteration times t ← 0, and initializing a cloud model
Figure FDA0003673166400000032
Wherein theta is init 、φ init Initializing each participating client for random initialization parameters
Figure FDA0003673166400000033
Model parameters of
Figure FDA0003673166400000034
Each participating client e i ∈ε i Organizing local task sets
Figure FDA0003673166400000035
Updating the current iteration time t ← 1;
step 5.2, each participating client e i ∈ε i From
Figure FDA0003673166400000036
Random extraction task
Figure FDA0003673166400000037
Obtaining all sample pairs
Figure FDA0003673166400000038
Characteristic diagram of
Figure FDA0003673166400000039
Uploading all feature maps and corresponding sample labels to c;
step 5.3, for each participating client e i ∈ε i Based on step 4, c calculation
Figure FDA00036731664000000310
Updating
Figure FDA00036731664000000311
Where α is the random gradient descent learning rate, given to each participating client e i ∈ε i Return theta g (t+1)；
Step 5.4, each participating client e i ∈ε i Updating
Figure FDA00036731664000000312
Step 5.5, updating the current iteration time T ← T +1, and if T is less than or equal to T, repeating the steps from 5.2 to 5.4;
step 5.6, outputting the training result
Figure FDA00036731664000000313
10. Light industry picture classification system based on federal small sample study, its characterized in that includes:
the system modeling module is used for modeling an industrial picture classification system based on a federal small sample learning framework, and comprises a cloud server c and a group of industrial institutions epsilon with the same interest, and each e i E epsilon holds respective local sample sets separately
Figure FDA00036731664000000314
A small sample picture classification task modeling module used for classifying small sample pictures for each industrial mechanism in the constructed system model
Figure FDA0003673166400000041
Modeling is performed, each task comprising a support set
Figure FDA0003673166400000042
And query set
Figure FDA0003673166400000043
A classifier building module for building a classifier from the embedded module θ And relation module g φ The small sample classifier f based on the lightweight relational network is formed, wherein theta and phi are parameters of functions l and g;
a training module for classifying tasks based on small sample pictures
Figure FDA0003673166400000044
Constructing a training method of a small sample classifier f; constructing an edge cloud collaborative deployment and training method of the small sample classifier based on a small sample classifier training method, wherein the complete small sample image classifier based on a lightweight relational network is deployed on c, and an embedded module is deployed on all e i E, on epsilon, training the classifier based on the training method in the step 4 along a discrete time point T of 0,1, … and T, wherein T is the iteration number of a preset model;
the output reasoning module is used for deploying and training the constructed small sample classifier based on the edge cloud collaborative deployment and training method of the small sample classifier
Figure FDA0003673166400000045
After training is completed, each external client is initialized
Figure FDA0003673166400000046
Model (2)
Figure FDA0003673166400000047
Model reasoning is performed locally.
CN202210611106.XA 2022-05-31 2022-05-31 Lightweight industrial picture classification method and system based on federal small sample learning Active CN115017351B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210611106.XA CN115017351B (en) 2022-05-31 2022-05-31 Lightweight industrial picture classification method and system based on federal small sample learning

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210611106.XA CN115017351B (en) 2022-05-31 2022-05-31 Lightweight industrial picture classification method and system based on federal small sample learning

Publications (2)

Publication Number Publication Date
CN115017351A true CN115017351A (en) 2022-09-06
CN115017351B CN115017351B (en) 2024-05-10

Family

ID=83071882

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210611106.XA Active CN115017351B (en) 2022-05-31 2022-05-31 Lightweight industrial picture classification method and system based on federal small sample learning

Country Status (1)

Country Link
CN (1) CN115017351B (en)

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2021189906A1 (en) * 2020-10-20 2021-09-30 平安科技（深圳）有限公司 Target detection method and apparatus based on federated learning, and device and storage medium
WO2021190638A1 (en) * 2020-11-24 2021-09-30 平安科技（深圳）有限公司 Federated modelling method based on non-uniformly distributed data, and related device
CN113469234A (en) * 2021-06-24 2021-10-01 成都卓拙科技有限公司 Network flow abnormity detection method based on model-free federal meta-learning
CN113518007A (en) * 2021-07-06 2021-10-19 华东师范大学 Multi-internet-of-things equipment heterogeneous model efficient mutual learning method based on federal learning

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2021189906A1 (en) * 2020-10-20 2021-09-30 平安科技（深圳）有限公司 Target detection method and apparatus based on federated learning, and device and storage medium
WO2021190638A1 (en) * 2020-11-24 2021-09-30 平安科技（深圳）有限公司 Federated modelling method based on non-uniformly distributed data, and related device
CN113469234A (en) * 2021-06-24 2021-10-01 成都卓拙科技有限公司 Network flow abnormity detection method based on model-free federal meta-learning
CN113518007A (en) * 2021-07-06 2021-10-19 华东师范大学 Multi-internet-of-things equipment heterogeneous model efficient mutual learning method based on federal learning

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
周俊;方国英;吴楠;: "联邦学习安全与隐私保护研究综述", 西华大学学报(自然科学版), no. 04, 10 July 2020 (2020-07-10) *
张婧;袁细国;: "基于小样本学习的高光谱遥感图像分类算法", 聊城大学学报(自然科学版), no. 06, 4 August 2020 (2020-08-04) *

Also Published As

Publication number Publication date
CN115017351B (en) 2024-05-10

Similar Documents

Publication Publication Date Title
CN108647639B (en) Real-time human body skeleton joint point detection method
WO2021190451A1 (en) Method and apparatus for training image processing model
CN111275713B (en) Cross-domain semantic segmentation method based on countermeasure self-integration network
CN112016601B (en) Network model construction method based on knowledge graph enhanced small sample visual classification
US11798145B2 (en) Image processing method and apparatus, device, and storage medium
CN109544442A (en) The image local Style Transfer method of production confrontation network based on dual confrontation
WO2022001805A1 (en) Neural network distillation method and device
CN115457531A (en) Method and device for recognizing text
CN110163801A (en) A kind of Image Super-resolution and color method, system and electronic equipment
CN116091886A (en) Semi-supervised target detection method and system based on teacher student model and strong and weak branches
US20230297844A1 (en) Federated learning using heterogeneous labels
CN114710330B (en) Anomaly detection method based on heterogeneous layered federated learning
WO2022052782A1 (en) Image processing method and related device
Zhang et al. A framework of an intelligent education system for higher education based on deep learning
CN112861659A (en) Image model training method and device, electronic equipment and storage medium
CN108959664A (en) Distributed file system based on picture processor
CN109508640A (en) Crowd emotion analysis method and device and storage medium
CN115713546A (en) Lightweight target tracking algorithm for mobile terminal equipment
CN108537109A (en) Monocular camera sign Language Recognition Method based on OpenPose
CN114298233A (en) Expression recognition method based on efficient attention network and teacher-student iterative transfer learning
CN114463340A (en) Edge information guided agile remote sensing image semantic segmentation method
US20240095539A1 (en) Distributed machine learning with new labels using heterogeneous label distribution
CN110751271A (en) Image traceability feature characterization method based on deep neural network
CN112488132B (en) Semantic feature enhancement-based fine-grained image classification method
CN115017351A (en) Light-weight industrial picture classification method and system based on federal small sample learning

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant