CN112766346A - Multi-example learning method based on graph convolution network - Google Patents
Multi-example learning method based on graph convolution network Download PDFInfo
- Publication number
- CN112766346A CN112766346A CN202110035503.2A CN202110035503A CN112766346A CN 112766346 A CN112766346 A CN 112766346A CN 202110035503 A CN202110035503 A CN 202110035503A CN 112766346 A CN112766346 A CN 112766346A
- Authority
- CN
- China
- Prior art keywords
- graph
- package
- packet
- feature
- feature expression
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 45
- 239000013598 vector Substances 0.000 claims abstract description 11
- 230000007246 mechanism Effects 0.000 claims description 12
- 230000008569 process Effects 0.000 claims description 7
- 238000013527 convolutional neural network Methods 0.000 claims description 5
- 238000002474 experimental method Methods 0.000 description 19
- 230000006870 function Effects 0.000 description 14
- 238000002790 cross-validation Methods 0.000 description 7
- WZUVPPKBWHMQCE-UHFFFAOYSA-N Haematoxylin Chemical compound C12=CC(O)=C(O)C=C2CC2(O)C1C1=CC=C(O)C(O)=C1OC2 WZUVPPKBWHMQCE-UHFFFAOYSA-N 0.000 description 6
- 206010006187 Breast cancer Diseases 0.000 description 5
- 208000026310 Breast neoplasm Diseases 0.000 description 5
- 206010009944 Colon cancer Diseases 0.000 description 5
- 239000013255 MILs Substances 0.000 description 5
- 208000029742 colonic neoplasm Diseases 0.000 description 5
- 238000010586 diagram Methods 0.000 description 5
- 201000010099 disease Diseases 0.000 description 5
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 description 5
- 238000011156 evaluation Methods 0.000 description 5
- 239000011159 matrix material Substances 0.000 description 5
- 238000012360 testing method Methods 0.000 description 5
- 238000012549 training Methods 0.000 description 5
- 230000009466 transformation Effects 0.000 description 5
- 238000013459 approach Methods 0.000 description 4
- 238000013528 artificial neural network Methods 0.000 description 4
- 210000004027 cell Anatomy 0.000 description 4
- 238000011160 research Methods 0.000 description 4
- YQGOJNYOYNNSMM-UHFFFAOYSA-N eosin Chemical compound [Na+].OC(=O)C1=CC=CC=C1C1=C2C=C(Br)C(=O)C(Br)=C2OC2=C(Br)C(O)=C(Br)C=C21 YQGOJNYOYNNSMM-UHFFFAOYSA-N 0.000 description 3
- 210000002919 epithelial cell Anatomy 0.000 description 3
- 238000013507 mapping Methods 0.000 description 3
- 206010012688 Diabetic retinal oedema Diseases 0.000 description 2
- 206010012689 Diabetic retinopathy Diseases 0.000 description 2
- 210000000481 breast Anatomy 0.000 description 2
- 201000011190 diabetic macular edema Diseases 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 230000003211 malignant effect Effects 0.000 description 2
- 238000011176 pooling Methods 0.000 description 2
- ORILYTVJVMAKLC-UHFFFAOYSA-N Adamantane Natural products C1C(C2)CC3CC1CC2C3 ORILYTVJVMAKLC-UHFFFAOYSA-N 0.000 description 1
- 208000009458 Carcinoma in Situ Diseases 0.000 description 1
- 241000406668 Loxodonta cyclotis Species 0.000 description 1
- 238000012614 Monte-Carlo sampling Methods 0.000 description 1
- 206010028980 Neoplasm Diseases 0.000 description 1
- 241000282376 Panthera tigris Species 0.000 description 1
- 241000282485 Vulpes vulpes Species 0.000 description 1
- 201000011510 cancer Diseases 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 239000012141 concentrate Substances 0.000 description 1
- 238000009795 derivation Methods 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000003745 diagnosis Methods 0.000 description 1
- 239000003814 drug Substances 0.000 description 1
- 229940079593 drug Drugs 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 210000002950 fibroblast Anatomy 0.000 description 1
- 201000004933 in situ carcinoma Diseases 0.000 description 1
- 210000004969 inflammatory cell Anatomy 0.000 description 1
- 238000002372 labelling Methods 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 239000003550 marker Substances 0.000 description 1
- 238000010606 normalization Methods 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 238000002360 preparation method Methods 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 238000011524 similarity measure Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000000844 transformation Methods 0.000 description 1
- 238000012795 verification Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
- G06F18/2155—Generating training patterns; Bootstrap methods, e.g. bagging or boosting characterised by the incorporation of unlabelled data, e.g. multiple instance learning [MIL], semi-supervised techniques using expectation-maximisation [EM] or naïve labelling
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Evolutionary Computation (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Software Systems (AREA)
- Mathematical Physics (AREA)
- Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Computing Systems (AREA)
- Molecular Biology (AREA)
- General Health & Medical Sciences (AREA)
- Evolutionary Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Image Analysis (AREA)
Abstract
The invention relates to the field of multi-example learning methods, in particular to a multi-example learning method based on a graph convolution network, which comprises the following steps: s1: establishing a graph structure of the package; s2: extracting initial feature expression of the example; s3: acquiring a feature expression of each example; s4: obtaining an attention weight for each example; s5: obtaining a feature representation of the package; s6: obtaining a class score vector for the packet; the multi-example learning method based on the graph convolution network improves the accuracy of the multi-example learning method for high-resolution medical image classification under the condition of not losing the structural relationship among the original examples.
Description
Technical Field
The invention relates to the field of multi-example learning methods, in particular to a multi-example learning method based on a graph convolution network.
Background
In a typical machine learning problem such as image classification, an image clearly represents a class. However, in many practical application systems, multiple examples are observed, and only one label is given to the entirety of these examples. This scenario is commonly referred to as Multiple Instance Learning (MIL).
The MIL processes an example package, to which a class label is assigned. Therefore, the main goal of MILs is to learn a model that predicts the package label, e.g., medical diagnosis. Another challenge is how to exploit relationships between instances. To address the main task of bag classification, different approaches have been proposed, such as exploiting similarities between bags, and also embedding instances into a compact low-dimensional representation and then passing the low-dimensional representation to a bag-level classifier, in conjunction with the response of the instance-level classifier. Only the last method can provide interpretable results. But the accuracy at the level of the examples of such methods is low. More recently Deep Neural Networks (DNNs) have been applied to MILs. The MIL algorithm based on DNN is greatly improved compared with the existing shallow learning algorithm, and the basic idea is to add a pooling operation in the embedding process of the DNN learning package. Rather than non-trainable pooling, they introduced an attention mechanism for example focusing, trainable attention weights on instances may provide additional information about each instance's contribution to the final decision. This approach provides a reasonable interpretation of the final packet classification, but still treats the example as independent and irrelevant. In recent years, with the development of graph networks, researchers have built graphs by similarity between instances and then have learned package embedding using graph neural networks. Although this method improves accuracy and considers the similarity relationship between instances in the feature value domain, it cannot classify images by using the structural relationship of slices in the original image space domain, and in addition, the attention mechanism they use does not consider the structural relationship between instances.
Disclosure of Invention
The invention aims to provide a multi-example learning method based on a graph convolution network, which can improve the accuracy of the multi-example learning method for high-resolution medical image classification under the condition of not losing the structural relationship among original examples.
In order to achieve the above purpose, the invention adopts the technical scheme that: a multi-example learning method based on a graph convolution network comprises the following steps:
s1: establishing a graph structure of the package: learning each data sample, i.e. packet, using multiple instances, the similarity over a range of feature values establishing a graph structure of the packet;
s2: extracting an initial feature expression of an example: extracting each example initial feature expression in the packet in the step S1 by using a convolutional neural network;
s3: obtain a feature expression for each example: merging the initial feature expression of each example obtained in the step S2 into the graph structure of the package obtained in the step S1 by using a graph volume network to obtain the feature expression of each example;
s4: obtain attention weight for each example: obtaining an attention weight of an example using a learning process of embedding the graph structure into the attention weight using a graph attention mechanism, based on the graph structure of the package obtained in step S1 and the feature expression of the example obtained in step S3;
s5: obtain a feature representation of the package: weighting the attention weight of the example obtained in step S4 and the feature expression of the example obtained in step S3 to obtain a feature representation of the package;
s6: obtain class score vector for packet: the feature of the packet obtained in step S5 is represented as a class score vector of the packet obtained through one full-link layer.
Further, the structure of the convolutional neural network in step S2 is mainly ResNet 18.
The invention has the technical effects that: the invention utilizes the topological relation among examples to establish a graph structure related to a package, then the graph structure is merged into the graph convolution network to learn the embedding of the package, the basic theorem of continuous function on variable arrangement invariance is utilized, the arrangement invariance of the MIL algorithm based on the GCN is summarized, the multi-example learning method based on the graph convolution network is suitable for high-resolution image classification, and the ROI can be presented through attention weight so as to explain the disease category.
Drawings
FIG. 1 is a schematic flow chart diagram of a multi-example learning method based on a graph convolution network according to an embodiment of the present invention;
FIG. 2 is a relational presentation of a multi-example learning reference data set provided by an embodiment of the invention;
fig. 3 is a schematic diagram of obtaining a disease region of interest by using attention weighting according to an embodiment of the present invention.
Detailed Description
Referring to fig. 1-3, Multiple Instance Learning (MIL) processes weakly labeled data as a weakly supervised learning algorithm, where each data sample (commonly referred to as a packet) has multiple instances but only one label. To some extent, MILs can be formulated as a supervised learning task with the package of instances as input and the label at the package level as target. Assume that the set of packets is defined as [ X ]1,X2,...,XN]Each bag XiContaining K instancesThe purpose of MIL is to learn N packets to the corresponding tag [ Y ]1,Y2,...,YN]A mapping function of. Assume that this mapping function is S (X). For a typical two-class MIL problem, a packet is a positive class if it contains an instance of a positive class, otherwise it is a negative sample (equation 1). These assumptions state that the mapping function s (x) for MIL is rank invariant.
Many studies have shown that better performance can be achieved in classification and regression tasks by considering the relationship information between instances in a package. This shows that a better package representation can be obtained by using the structure information between instances in the MIL package. As research in multi-instance learning based on graph neural networks shows, if a packet is considered as a graph, score learning about the packet is consistent with graph classification, so that the score learning about the packet and the graph classification are consistent, and therefore the score learning about the packet and the graph classification establish a link matrix A in the graph by utilizing similarity among instances and collect instancesAnd (4) considering the vertex set V in the diagram, and finally combining the diagram neural network to obtain the class score of the packet. In recent years, with the intensive research of a data learning task of a graph structure, a graph convolution network is researched and found to be capable of simultaneously learning node characteristic information and structure information end to end, and is suitable for being used for learning node characteristic information and structure information end to endNodes and graphs of arbitrary topology are the deep network framework which is preferably based on graphs at present. In this context, we focus on the graph convolution network-based multi-example learning method for package representation learning, and propose a new theoretical point of view to explain the graph structure-based MIL problem in the current research.
It should be noted that the transformation function m (X) of the multi-example learning algorithm based on the graph convolution network can be expressed as the fractional function s (X) of an arbitrary packet X, and since s (X) has the permutation invariance, we only need to prove that m (X) also has the permutation invariance. We derive the following theorem from the basic theorem of continuous functions on the invariance of variable permutations. From theorem 1, M (X) has permutation invariance, so M (X) can be expressed as a fractional function S (X) of a packet.
Theorem 1: if M (X) is represented by a multi-example learning algorithm based on a graph convolution network, M (X) keeps unchanged a permutation transformation pi arbitrarily acting on X, wherein f, sigma are continuous functions, dkRepresenting an exemplary degree, W is a parameter of graph convolution.
And (3) proving that: since permutation transformations arbitrarily acting on X do not change node XkN (k), there is the following derivation process:
i.e., M (X) has alignment invariance.
The main flow of the multi-example learning algorithm based on the graph convolution network is described in fig. 1.
For example initial feature representations that have been given with other strategies (dimension F)1) E.g., MIL reference dataset, as shown in equation 2, using the similarity between the given example initial feature representations to establish a link matrix a in the graph structure of the packagei. For embodiments like the image classification problem, the link matrix A of the package is built using the structural relationship of the example over the original image spatial domain as shown in equation 3iAn example is then given using a trainable convolutional neural network (e.g., ResNet)Initial feature representation with dimension F1。
WhereinRespectively represent packet XiThe m-th, n-th example of (1),to representAndthe similarity measure between them, the measure function used in the embodiment of the present invention is the cosine distance, and ε takes the value of 0.5.
Further extraction of instances by employing a graph-convolution network to blend in a given graph structure and example initial feature representationsIs characterized byDimension of F2. The graph convolution operator used in the graph convolution network is shown in equation 4.
Wherein N isi(k) Presentation bag XiThe set of neighborhood nodes of the kth example of (a),presentation bag XiThe degree of the k-th example in (a),representing the parameter variables to be learned.
As shown in equation 5, the weight coefficient for each example is calculated using a graph attention mechanism incorporating the feature representation of the given graph structure and example into the attention mechanismWeighted summation of example weights and example feature representations to obtain a packet XiIs a characteristic ofi。
Wherein u ∈ RL×1,Representing the parameter variables to be learned. The softmax function is actually a normalization function that normalizes a vector to a sum of 1 in a particular way.
The fully-connected layer is used to convert the feature representation of the packet into a class score vector for the packet, noting this transformation function as σ.
We performed experiments on a number of different data sets to evaluate our invention. We wanted to experimentally validate three research questions, (i) whether our invention achieved the best performance compared to the best method, (ii) whether our invention could provide interpretable results through ROI, and (iii) whether our invention is more applicable to high resolution medical image classification problems.
MIL benchmark datasets all five datasets contain pre-calculated properties and only a few examples and packages, as shown in FIG. 2, MUSK1 and MUSK2 are datasets for drug activity prediction, and FOX, TIGER and ELEPHANT are image datasets. The total number of packets, the total number of instances, the number of positive and negative samples, and the feature vector dimensions that they contain are as shown in fig. 2. To obtain a fair comparison, we used a common evaluation method, i.e., 10-fold cross validation, with five replicates per experiment.
Breast cancer dataset: it consisted of 58 weakly labeled 896 × 768 hematoxylin and eosin (H & E) images. If the picture contains breast cancer cells, it is marked as malignant, otherwise it is benign. We divide each image into 32 x 32 slices, moving by a step size of 32. This resulted in 672 slices per bag. To obtain a fair comparison, we used a common evaluation method, i.e., 5-fold cross-validation.
Colon cancer dataset: it comprises 100H & E images in total, each picture having a resolution of 500 x 500. Images are derived from various tissue appearances of normal and malignant regions. For each image, most of the nuclei of each cell were labeled. A total of 22,444 nuclei have associated class labels, namely epithelial cells, inflammatory cells, fibroblasts and miscellaneous cells. Each packet is composed of slices of size 27 x 27. In addition, if one contains one or more epithelial-like nuclei, a positive case marker is given. Labeling epithelial cells is highly relevant from a clinical point of view, since colon cancer originates in epithelial cells. To obtain a fair comparison, we used a common evaluation method, i.e., 5-fold cross-validation.
Ici ar dataset Part a: the ICIAR dataset consisted of 400 breast microscopic images, classified into 4 categories, normal, benign, invasive and carcinoma in situ. For each class, there were 100 different hematoxylin and eosin stained images, 2048x 1536 pixels in size. We used these two types of images, benign and invasive, to create a two-classification problem to test the algorithm. To obtain a fair comparison, we used a common evaluation method, i.e., 5-fold cross-validation. Each experiment was trained using 160 images, 40 for testing. These slices have been previously cropped from the entire slide tissue image with the label attached, so from the MIL's perspective, a large number of slices (instances) we extract from one image (one bag) remain consistent with the image label. When training the model, randomly turning each image, then cutting out 256x256 slices, and finally randomly cutting out 224x224 slices. When testing the model, each image was cropped to 256x256 slices and finally scaled directly to 224x224 slices.
IDRiD dataset was published in ISBI 2018 as a challenge dataset, one of the purposes of which is an automated disease staging algorithm for the assessment of Diabetic Retinopathy (DR) and Diabetic Macular Edema (DME) using fundus images. The training data contained 413 images and the test data contained 103 images. The size of each image is 4288x2848 pixels. To obtain a fair comparison, we used a common evaluation method, i.e., 5 replicates per experiment. When training the model, randomly turning each image, then cutting out 256x256 slices, and finally randomly cutting out 224x224 slices. When testing the model, each image was cropped to 256x256 slices and finally scaled directly to 224x224 slices.
For experiments on MIL reference datasets, we use the same basic architecture for the deep network as in Attention MIL, but where the fully connected layers except the last one are replaced with convolutional layers, additionally note that the hyper-parameter L in the mechanism is 64. For the experiments on Breast cancer dataset and Colon cancer dataset, we used the same basic architecture for the deep network as the Attention MIL, but where the fully connected layers except the last one were replaced with convolutional layers, and further note that the over-parameter L in the force mechanism is 128. For the experiments on the ICIAR dataset and the IDRiD dataset, we extract feature vectors of slices by using ResNet18 without a full connection layer, then connect a graph convolution layer and a graph attention layer (L ═ 128) to obtain the feature vectors of a packet, and finally connect a full connection layer to obtain the class score of the packet.
For each layer of the network used in this embodiment, we initialize the parameters using a standard regularization strategy, with the bias set to zero. For the parameters in ResNet50, we use the policy of Finetune for initialization. In each experiment, the batch size is set to be 1, the maximum epoch is set to be 100, and finally a model with the minimum loss function on the verification set is obtained.
In the first experiment, our goal was to verify the validity of our invention on the MIL benchmark dataset. In these experiments we use equation 2 to establish the adjacency matrix a in the graph.
As shown in table 1, our method outperforms other MIL algorithms on five data sets, including DNN-based MIL algorithms, traditional non-DNN MIL algorithms, and GNN-based MIL algorithms. For the five MIL reference data sets, the examples in the bag have similarities in the eigenvalue domain. Furthermore, the graph convolution network fuses similar instances to obtain a feature representation for each instance, while the graph attention mechanism fuses instances with complex topologies to obtain a feature representation for a bag. In addition, our approach to creating graphs is the same as that of GNN-based MILs, so the last two rows of table 1 verify that the graph attention mechanism and graph convolution network are more suitable for the MIL problem.
Table 1:Results on classical MIL datasets.Experiments were run five times and an average of the classification accuracy(±astandard error of a mean)is reported.
Fully automated detection of cancer regions in hematoxylin and eosin (H & E) stained full slide images is a highly clinically significant task. Current surveillance methods use pixel-level annotations. However, data preparation requires a lot of time consuming for pathologists, which is highly disturbing for their daily life. Therefore, a solution using weak labels would greatly reduce the workload of the pathologist. In the following experiments, we classified the breast and colon cancer datasets in the hope that the second problem could be verified.
We show the experimental results on the two data sets in tables 2 and 3, respectively. From the experimental results of both tables, our method outperforms the best MIL algorithm based on the attention mechanism. For image classification, we consider slices as examples, images as packages, and each example has a neighboring relationship on the original image. Our method represents this relationship well during image classification. Furthermore, the Attention-based MIL used in GNN-based MILs is the same. The experimental results demonstrate that the graph proposed by us is the best attention mechanism. As can be seen from the last two rows of tables 2 and 3, the experimental result of the method of constructing a map by the structural relationship on the image space domain is superior to that of the method of constructing a map by the similarity on the feature value domain. Therefore, our method of constructing graph structures is applicable to image classification.
Table 2:Results on breast cancer dataset.Experiments are 5-fold cross-validation and an average(±a standard error of the mean)is reported.
Table 3:Results on colon cancer dataset.Experiments are 5-fold cross-validation and an average(±a standard error of the mean)is reported.
Finally, we used a set of images to verify that our method can provide an ROI of the disease. As shown in fig. 3, we show a histopathological image that is segmented into plaques containing single cells. We create a thermodynamic diagram by multiplying the slices by the corresponding attention weights. Although we use image-level labels in the training process, there is a large overlap between the actual values of fig. 3(b) and fig. 3 (c). This result verifies the validity of the attention mechanism.
The two previous experiments demonstrated that our method is suitable for medical image classification and can obtain the region related to the disease, but do not show that our method is more suitable for high resolution medical image classification. To verify the third problem, we performed experiments on IDRiD dataset and ICIAR dataset. From the experiments on these two data sets, we still use equation 3 to build the adjacency matrix a of the graph. For both datasets, we use the convolution layer in ResNet18 to represent the transformation f. All models were trained using the Adam optimization algorithm.
Table 4:Results on IDRiD dataset.*represents the top two teams’results or the IDRiD challenge 2.Experiments are run five times and an average is reported.
Table 5:Results on ICIAR dataset.Experiments are 5-fold cross-validation and an average is reported.
For high resolution image classification, the adjacency between the original pixel information of the image and the patch is very important, and our approach is a good way to integrate this information together. We show the results on the two data sets in tables 4 and 5, respectively. As shown in table 4, our method outperforms other methods, including the best non-MIL method, i.e. they first extract the feature vectors using the pre-trained densnet 121 and then obtain the final result using the integrated algorithm LightGBM. As can be seen from the last three rows of table 5, our method is superior to other methods, including the best MIL algorithm, which concentrates on the most relevant regions of the high resolution image by the monte carlo sampling strategy, and then iterates to obtain the most relevant slices. From the results in the first and last rows of table 5, our method outperforms other non-MIL methods: they used VGG16 and a pre-training strategy on similar datasets as the base model to perform four classification tasks. Therefore, our method is more suitable for high resolution image classification.
The above description is only for the preferred embodiment of the present invention, but the scope of the present invention is not limited thereto, and any changes or substitutions that can be easily conceived by those skilled in the art within the technical scope of the present invention are included in the scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.
Claims (2)
1. A multi-example learning method based on a graph convolution network comprises the following steps:
s1: establishing a graph structure of the package: learning each data sample, i.e. an example in a package, by using multiple examples, and establishing a graph structure of the package according to the similarity of the data samples on the characteristic value range;
s2: extracting an initial feature expression of an example: extracting each example initial feature expression in the packet in the step S1 by using a convolutional neural network;
s3: obtain a feature expression for each example: merging the initial feature expression of each example obtained in the step S2 into the graph structure of the package obtained in the step S1 by using a graph volume network to obtain the feature expression of each example;
s4: obtain attention weight for each example: obtaining an attention weight of an example using a learning process of embedding the graph structure into the attention weight using a graph attention mechanism, based on the graph structure of the package obtained in step S1 and the feature expression of the example obtained in step S3;
s5: obtain a feature representation of the package: weighting the attention weight of the example obtained in step S4 and the feature expression of the example obtained in step S3 to obtain a feature representation of the package;
s6: obtain class score vector for packet: the feature of the packet obtained in step S5 is represented as a class score vector of the packet obtained through one full-link layer.
2. The graph convolution network-based multi-instance learning method of claim 1, wherein: the structure of the convolutional neural network in step S2 is mainly ResNet 18.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202110035503.2A CN112766346A (en) | 2021-01-12 | 2021-01-12 | Multi-example learning method based on graph convolution network |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202110035503.2A CN112766346A (en) | 2021-01-12 | 2021-01-12 | Multi-example learning method based on graph convolution network |
Publications (1)
Publication Number | Publication Date |
---|---|
CN112766346A true CN112766346A (en) | 2021-05-07 |
Family
ID=75701511
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN202110035503.2A Pending CN112766346A (en) | 2021-01-12 | 2021-01-12 | Multi-example learning method based on graph convolution network |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN112766346A (en) |
Citations (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2014124308A2 (en) * | 2013-02-08 | 2014-08-14 | Solan, LLC | Multi-level graphene devices and methods for forming same |
CN106682696A (en) * | 2016-12-29 | 2017-05-17 | 华中科技大学 | Multi-example detection network based on refining of online example classifier and training method thereof |
US20200160177A1 (en) * | 2018-11-16 | 2020-05-21 | Royal Bank Of Canada | System and method for a convolutional neural network for multi-label classification with partial annotations |
CN111488400A (en) * | 2019-04-28 | 2020-08-04 | 北京京东尚科信息技术有限公司 | Data classification method, device and computer readable storage medium |
CN111737552A (en) * | 2020-06-04 | 2020-10-02 | 中国科学院自动化研究所 | Method, device and equipment for extracting training information model and acquiring knowledge graph |
CN111860656A (en) * | 2020-07-22 | 2020-10-30 | 中南民族大学 | Classifier training method, device, equipment and storage medium |
-
2021
- 2021-01-12 CN CN202110035503.2A patent/CN112766346A/en active Pending
Patent Citations (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2014124308A2 (en) * | 2013-02-08 | 2014-08-14 | Solan, LLC | Multi-level graphene devices and methods for forming same |
CN106682696A (en) * | 2016-12-29 | 2017-05-17 | 华中科技大学 | Multi-example detection network based on refining of online example classifier and training method thereof |
US20200160177A1 (en) * | 2018-11-16 | 2020-05-21 | Royal Bank Of Canada | System and method for a convolutional neural network for multi-label classification with partial annotations |
CN111488400A (en) * | 2019-04-28 | 2020-08-04 | 北京京东尚科信息技术有限公司 | Data classification method, device and computer readable storage medium |
CN111737552A (en) * | 2020-06-04 | 2020-10-02 | 中国科学院自动化研究所 | Method, device and equipment for extracting training information model and acquiring knowledge graph |
CN111860656A (en) * | 2020-07-22 | 2020-10-30 | 中南民族大学 | Classifier training method, device, equipment and storage medium |
Non-Patent Citations (4)
Title |
---|
MING TU等: "Multiple instance learning with graph neural networks", 《ARXIV:1906.04881V1》, pages 2 * |
YANGLING MA等: "Multi-Instance Learning by Utilizing Structural Relationship among Instances", 《ARXIV:2102.01889V1》, pages 1 - 22 * |
刘永胜: "基于深度神经网络的弱监督学习方法在图像领域的研究", 《中国博士学位论文全文数据库_信息科技辑》, pages 1 - 4 * |
马阳玲: "融合先验知识的深度学***台》 * |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
Xu et al. | Representation learning on graphs with jumping knowledge networks | |
Chen et al. | Attention embedded lightweight network for maize disease recognition | |
Su et al. | NSCKL: Normalized spectral clustering with kernel-based learning for semisupervised hyperspectral image classification | |
Lu et al. | Learning optimal seeds for diffusion-based salient object detection | |
Elaraby et al. | Classification of citrus diseases using optimization deep learning approach | |
Jia et al. | Object-oriented feature selection of high spatial resolution images using an improved Relief algorithm | |
Zhou et al. | Multi-view spectral clustering with optimal neighborhood Laplacian matrix | |
Nakane et al. | Application of evolutionary and swarm optimization in computer vision: a literature survey | |
Anandhakrishnan et al. | Deep Convolutional Neural Networks for image based tomato leaf disease detection | |
Zuo et al. | FSL-EGNN: Edge-labeling graph neural network for hyperspectral image few-shot classification | |
Qiang et al. | Identification of plant leaf diseases based on inception V3 transfer learning and fine-tuning | |
CN110288088A (en) | Semi-supervised width study classification method based on manifold regularization and broadband network | |
Chen et al. | Hyperspectral remote sensing image classification based on dense residual three-dimensional convolutional neural network | |
CN104966075B (en) | A kind of face identification method and system differentiating feature based on two dimension | |
Lin et al. | EM-ERNet for image-based banana disease recognition | |
CN108268890A (en) | A kind of hyperspectral image classification method | |
CN104281835A (en) | Face recognition method based on local sensitive kernel sparse representation | |
Xiao et al. | Citrus greening disease recognition algorithm based on classification network using TRL-GAN | |
Acharya et al. | Plant Disease detection for paddy crop using Ensemble of CNNs | |
Tu et al. | Discriminant sub-dictionary learning with adaptive multiscale superpixel representation for hyperspectral image classification | |
Huang et al. | Local linear spatial–spectral probabilistic distribution for hyperspectral image classification | |
Sima et al. | Composite kernel of mutual learning on mid-level features for hyperspectral image classification | |
Ye et al. | Supervised functional data discriminant analysis for hyperspectral image classification | |
Wang et al. | Integration of multi-feature fusion and dictionary learning for face recognition | |
Ghojogh | Data Reduction Algorithms in Machine Learning and Data Science |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination |