CN111667453A - Gastrointestinal endoscope image anomaly detection method based on local feature and class mark embedded constraint dictionary learning - Google Patents
Gastrointestinal endoscope image anomaly detection method based on local feature and class mark embedded constraint dictionary learning Download PDFInfo
- Publication number
- CN111667453A CN111667453A CN202010316171.0A CN202010316171A CN111667453A CN 111667453 A CN111667453 A CN 111667453A CN 202010316171 A CN202010316171 A CN 202010316171A CN 111667453 A CN111667453 A CN 111667453A
- Authority
- CN
- China
- Prior art keywords
- dictionary
- matrix
- image
- class
- coding coefficient
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000002496 gastric effect Effects 0.000 title claims abstract description 16
- 238000001514 detection method Methods 0.000 title claims abstract description 8
- 239000011159 matrix material Substances 0.000 claims abstract description 74
- 238000012549 training Methods 0.000 claims abstract description 41
- 238000012360 testing method Methods 0.000 claims abstract description 18
- 238000004422 calculation algorithm Methods 0.000 claims abstract description 17
- 238000007781 pre-processing Methods 0.000 claims abstract description 5
- 239000013598 vector Substances 0.000 claims description 23
- 238000000034 method Methods 0.000 claims description 13
- 230000006870 function Effects 0.000 claims description 11
- 230000004927 fusion Effects 0.000 claims description 7
- 208000025865 Ulcer Diseases 0.000 claims description 6
- 238000010276 construction Methods 0.000 claims description 6
- 231100000397 ulcer Toxicity 0.000 claims description 6
- 201000010099 disease Diseases 0.000 abstract 2
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 abstract 2
- 230000003902 lesion Effects 0.000 abstract 1
- 230000002159 abnormal effect Effects 0.000 description 9
- 238000002474 experimental method Methods 0.000 description 8
- 208000037062 Polyps Diseases 0.000 description 5
- 230000005856 abnormality Effects 0.000 description 4
- 239000002775 capsule Substances 0.000 description 4
- 238000001839 endoscopy Methods 0.000 description 4
- 210000002784 stomach Anatomy 0.000 description 4
- 230000000694 effects Effects 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 210000001035 gastrointestinal tract Anatomy 0.000 description 3
- 238000004364 calculation method Methods 0.000 description 2
- 238000003745 diagnosis Methods 0.000 description 2
- 238000000605 extraction Methods 0.000 description 2
- 239000010794 food waste Substances 0.000 description 2
- 238000012795 verification Methods 0.000 description 2
- 206010017817 Gastric polyps Diseases 0.000 description 1
- 208000018522 Gastrointestinal disease Diseases 0.000 description 1
- 208000032843 Hemorrhage Diseases 0.000 description 1
- 241000167880 Hirundinidae Species 0.000 description 1
- 208000007107 Stomach Ulcer Diseases 0.000 description 1
- 238000009098 adjuvant therapy Methods 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000015572 biosynthetic process Effects 0.000 description 1
- 208000034158 bleeding Diseases 0.000 description 1
- 230000000740 bleeding effect Effects 0.000 description 1
- 238000007635 classification algorithm Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000009795 derivation Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- PCHJSUWPFVWCPO-UHFFFAOYSA-N gold Chemical compound [Au] PCHJSUWPFVWCPO-UHFFFAOYSA-N 0.000 description 1
- 210000000936 intestine Anatomy 0.000 description 1
- 238000010606 normalization Methods 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 230000008855 peristalsis Effects 0.000 description 1
- 238000002203 pretreatment Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000012552 review Methods 0.000 description 1
- 238000004088 simulation Methods 0.000 description 1
- 210000000813 small intestine Anatomy 0.000 description 1
- 239000013589 supplement Substances 0.000 description 1
- 238000003786 synthesis reaction Methods 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0012—Biomedical image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/25—Fusion techniques
- G06F18/253—Fusion techniques of extracted features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/11—Region-based segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/56—Extraction of image or video features relating to colour
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10068—Endoscopic image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30004—Biomedical image processing
- G06T2207/30028—Colon; Small intestine
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30004—Biomedical image processing
- G06T2207/30092—Stomach; Gastric
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Data Mining & Analysis (AREA)
- General Engineering & Computer Science (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Bioinformatics & Computational Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Artificial Intelligence (AREA)
- Life Sciences & Earth Sciences (AREA)
- Multimedia (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Medical Informatics (AREA)
- Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
- Radiology & Medical Imaging (AREA)
- Quality & Reliability (AREA)
- Image Analysis (AREA)
Abstract
A gastrointestinal endoscope image anomaly detection method based on local feature and class mark embedding constraint dictionary learning comprises the steps of preprocessing an original image, extracting an interested region, further extracting color and texture features from image data and fusing, constructing a test set and a training set, establishing a dictionary learning model with an original class mark embedding item and a structural feature constraint item of Profiles, inputting a training set matrix into the model for solving, iteratively updating, respectively training out a dictionary and a coding coefficient matrix, and obtaining classifier parameters by utilizing a coding coefficient matrix and a marking matrix of a training sample; and finally, for the test image, obtaining a sparse coefficient and a prediction label thereof through an orthogonal matching pursuit algorithm, and classifying the multi-class disease images by comparing the label of the reconstructed test set with the label of the original test set. The invention can realize the classification of different lesion images of the gastrointestinal endoscope; can effectively classify endoscope diseases.
Description
Technical Field
The invention relates to an abnormality classification technology of gastrointestinal endoscopy images, in particular to a dictionary learning method based on atomic class label embedding and profiles local feature constraint, which is suitable for gastrointestinal endoscope image abnormality detection.
Background
More and more people suffer from gastrointestinal diseases, and endoscopy is taken as the gold standard of gastrointestinal tract research, is widely applied to early detection and adjuvant therapy of the gastrointestinal tract, and effectively reduces the morbidity and mortality. Conventional hand-held endoscopes are invasive diagnostic devices that have difficulty accurately acquiring the entire gastrointestinal tract. Compared with the traditional endoscopy technology, the wireless capsule endoscope is a new painless and noninvasive technology, can completely enter the small intestine, and reduces the discomfort of patients. The patient swallows the capsule endoscope from the mouth, wirelessly captures color images by gastrointestinal peristalsis for an average duration of about 8 hours, and then wirelessly transmits the images to a data recording device attached to the patient's waist for the clinician to review the images and make a diagnosis.
Although wireless capsule endoscopy is a technological breakthrough, it takes approximately 2 hours for an experienced clinician to analyze approximately 5 million images generated per patient. Abnormal images including polyps, bleeding, ulcers, etc. typically account for less than 5% of all images. In addition, spatial features of the abnormal image (such as shape, texture, size, and contrast with the surrounding environment) may vary, so it may be difficult for a doctor to reliably detect an abnormality in all cases. Therefore, an automatic computer-aided system is designed to assist clinicians in classifying abnormal images, so that the burden of the clinicians is reduced, and the efficiency and the accuracy are improved.
Disclosure of Invention
In order to overcome the defect that the existing multi-anomaly classification result is far unsatisfactory, the invention provides a dictionary learning method of atomic class mark embedding and profiles local feature constraint, which is used for classifying gastrointestinal endoscope image anomalies, and a linear classifier based on a multi-class dictionary is constructed by combining dictionary learning, sparse representation of image features and error comparison of reconstruction of the multi-class images, so that classification of different abnormal images of a gastrointestinal endoscope can be effectively realized.
The technical scheme proposed for solving the technical problems is as follows:
a gastrointestinal endoscope image anomaly detection method based on local feature and class mark embedding constraint dictionary learning comprises the following steps:
step 1: acquiring an endoscope image set which consists of polyps, ulcers and normal images;
step 2: image preprocessing: extracting a tissue area of the whole endoscope image, generating a mask in an invalid area, and removing an image with low quality;
and step 3: extracting image color features: extracting color features of the image by a color histogram method in an HSI color space;
and 4, step 4: extracting image texture features: under the HSI color space, extracting the texture features of the image by adopting a scale-invariant feature transform operator;
and 5: image feature fusion: fusing and normalizing the color and texture characteristics of the image;
step 6: initializing a dictionary and a coding coefficient matrix by using a K-SVD algorithm, wherein the model expression is as follows:
wherein Y is [ Y ]1,…,YC]=[y1,y2,…,yn]∈Rm×nRepresenting a training data set with n training samples, m being the dimension of the training samples, C being the class of the training samples, YiIs a matrix composed of training samples of the ith class; d ═ D1,d2,…,dK]T∈Rm×KRepresenting a dictionary, K is the number of atoms, diIs the ith atom of dictionary D, X ═ X1,x2,…xn]T∈RK×nIs a matrix of coding coefficients, xi=[x1i,x2i,…,xKi]TIs the firsti training samples yiCoding coefficient, T, corresponding to dictionary D0Is a sparse factor, | | x | | Y0Represents a 0 norm representing the number of non-zero elements in the coefficient vector x, | x | | luminance2Represents a 2-norm whose values represent the sum of the squares of the individual elements in the coefficient vector x; solve to obtain an initialization dictionary D0=[D1,D2,…,DC]And initializing the coding coefficient matrix X0=[X1,X2,…,XC];
And 7: according to the step 6, an atom class mark embedding item and a structural feature constraint item of Profiles are introduced, and a dictionary model expression of endoscope image classification is as follows:
whereinThe reconstruction performance of the dictionary is represented,is a regular term of the coding coefficient, λ, α and β are scalar parameters;
7.1 atomic class label insert model construction:
aiming at the ith class training sample, learning a specific class dictionary D by utilizing a K-SVD algorithmiTherefore, the C class training sample is learned to obtain a special class dictionary D0=[D1,D2,…DC]Dictionary D0Containing a C-type atom; if atom di∈DiThen atom diIs defined as bi=[0,0,…,0,1,0,…,0]∈RC,biThe ith element in (1) is 1, and the rest are zero, so the atom class matrix B in the dictionary D is defined as B ═ B1,b2,…,bk]T∈Rk×C(ii) a The weight class index matrix H of the dictionary is defined asBy usingCoding coefficient matrix XTAnd constructing the following class label embedding items by the atom weight class label matrix H:
7.2 construction of structural feature constraint term model of Profiles:
each row vector of the coding coefficient matrix in dictionary learning is defined as a profile, and P ═ X is definedTThen P ═ P1,p2,…,pk]∈Rn×KWherein p isi=[xi1,…,xin]TAtom diThe corresponding profile is piIn order to improve the discrimination performance of the coding coefficient, a discriminant is designed by combining the structural features of profiles with atoms, a neighbor graph M is constructed by utilizing a P matrix, and each P isiRepresenting a vertex; the weight matrix W of the neighbor map M is calculated as follows:
wherein is the parameter, kNN (p)i) Represents piK is close to, wi,jIs piAnd pjReflects the similarity between them, in order to better reflect piAnd pjThe Laplace graph L based on the profiles characteristics is constructed as follows:
the discriminant based on the profiles features is designed as follows:
and 8: optimizing a dictionary model:
8.1 solution update of dictionary D:
assuming that the encoding coefficient matrix X, the Laplace graph L and the matrix U in the objective function are constants, the formula (2) is converted into the formula (7)
And (3) solving by using a Lagrange function to obtain an optimal dictionary D:
D=YXT(XXT+βL+ηI) (8)
where η is a parameter and I is an identity matrix;
8.2 solving and updating the coding coefficient matrix X:
assuming that the dictionary D, the laplacian graph L, and the matrix U in the objective function are constants, equation (2) translates to:
the equation (10) is directly derived to obtain the optimal coding coefficient matrix X:
X=(DTD+αU+γI)-1DTY (10)
wherein γ is a parameter, once a coding coefficient is obtained, a matrix U is obtained according to formula (3), and a laplacian graph L is obtained according to formula (5);
and step 9: constructing a linear classifier: firstly, obtaining classifier parameters G by using a coding coefficient matrix X and a marking matrix H of a training samplex:
Gx=HXT(XXT+Ι)-1(11)
Second, for the test imageComputing coding coefficient vector by orthogonal matching pursuit algorithmAnd a dictionary D, byCalculating to obtain a predicted label vector lxAnd finally, testing the sampleIs a predictive label vector lxThe index corresponding to the maximum element value of (2).
Compared with the prior art, the invention has the beneficial effects that:
a. the invention provides an abnormal image classification algorithm to assist doctors in gastrointestinal abnormality diagnosis, effectively shortens analysis time and reduces workload;
b. the invention adopts a dictionary learning algorithm for classifying gastroscope abnormal images, designs a discriminant based on profiles structural features, and improves the discriminant performance of coding coefficients; distributing a class mark for each atom in the dictionary, and constructing a discriminant by using the atom class mark to improve the discriminability of the dictionary;
c. the invention designs a gastroscope abnormal image dictionary learning classification optimization method, which solves an objective function by adopting an alternative constraint direct derivation method and continuously updates the objective function until the algorithm is converged; obtaining linear classifier parameters by using a coding coefficient matrix and a marking matrix of a training sample, obtaining the relation between a sparse representation coefficient vector and a learning dictionary of a test image by an orthogonal matching pursuit algorithm, then calculating a prediction label vector by using the linear classifier parameters, and finally, the category of the test sample is an index corresponding to the maximum element value in the prediction label vector;
d. the invention integrates the color characteristic and the texture characteristic, and the color characteristic can effectively distinguish gastrointestinal endoscope abnormal images according to the experience of a clinician, but the distinction degree on certain gastrointestinal endoscope abnormal images is still limited. The texture features are fused, so that the classification accuracy can be effectively enhanced;
e. the invention utilizes an image preprocessing method suitable for a capsule endoscope, which comprises region-of-interest extraction and low-quality image rejection, and reduces interference caused by useless information.
Drawings
FIG. 1 is a block diagram of a dictionary learning system of the present invention;
FIG. 2 is a graph showing the effect of pretreatment according to the present invention;
FIG. 3 is a detailed algorithm flow chart of the multi-class dictionary learning and sparse reconstruction of the present invention.
FIG. 4 is a graph of the accuracy of the classification concept of the present invention using fused features with other features at different dictionary sizes.
Detailed Description
The invention is further described below with reference to the accompanying drawings.
Referring to fig. 1 to 4, a gastrointestinal endoscope image anomaly detection method based on local feature and class mark embedding constraint dictionary learning comprises the following steps:
step 1: acquiring an endoscope image set, wherein the endoscope image set consists of three different gastroscope images, namely polyp images, ulcer images and normal images, and the number of the images in each type is the same;
step 2: the images taken directly from the endoscope usually contain a lot of useless information, such as instrument markers, time and patient information, etc. Meanwhile, because a plurality of interference factors exist in the intestines and stomach, such as bubbles, food residues or a plurality of low-quality images caused by shooting reasons, the acquired gastroscope images are preprocessed, the tissue area of the whole endoscopic image is extracted, a mask is generated in an invalid area, and meanwhile, some low-quality images are eliminated;
and step 3: extracting image color features: under the HSI color space, extracting the color characteristics of the image by adopting a color histogram algorithm, quantizing the values of three channels of hue, saturation and lightness respectively, and then performing statistics to form a color histogram as the color characteristics of the image;
and 4, step 4: extracting image texture features: under an HSI color space, searching an extreme point in a space scale by adopting a scale-invariant feature transformation operator, extracting position, scale and rotation invariant information of the extreme point, describing the extreme point by using a group of vectors, and extracting texture features of an image;
and 5: the characteristics of the image cannot be well represented by a single characteristic, the mutual fusion and supplement of the color characteristic and the texture characteristic are more discriminative, and the generated data dimension is high or low due to different characteristic calculation methods, so that the differences are eliminated through normalization, and the data representation is improved; in the new fused features, the number of rows of a matrix of the new fused features represents the feature dimension, the number of columns of the new fused features represents the number of images, namely, each column of vectors represents the features of each image;
step 6: initializing a dictionary and a coding coefficient matrix by using a K-SVD algorithm, wherein the model expression is as follows:
wherein Y is [ Y ═ Y1,y2,…,yn]∈Rm×nRepresenting a training data set with n training samples, m being the dimension of the training samples, D ═ D1,d2,…,dK]T∈Rm×KRepresenting a dictionary, K is the number of atoms, diIs the ith atom of dictionary D, X ═ X1,x2,…xn]T∈RK×nIs a matrix of coding coefficients, xi=[x1i,x2i,…,xKi]TIs the ith training sample yiCoding coefficient, T, corresponding to dictionary D0Is a sparse factor, | | x | | Y0Represents a 0 norm representing the number of non-zero elements in the coefficient vector x, | x | | luminance2Represents a 2-norm whose values represent the sum of the squares of the individual elements in the coefficient vector x; solve to obtain an initialization dictionary D0=[D1,D2,…,DC]And initializing the coding coefficient matrix X0=[X1,X2,…,XC];
And 7: according to the step 6, an atom class mark embedding item and a structural feature constraint item of Profiles are introduced, and a dictionary model expression of endoscope image classification is as follows:
whereinThe reconstruction performance of the dictionary is represented,is a regular term of the coding coefficient, λ, α and β are scalar parameters;
7.1 atomic class label insert model construction:
aiming at the ith class training sample, learning a specific class dictionary D by utilizing a K-SVD algorithmiTherefore, the C class training sample is learned to obtain a special class dictionary D0=[D1,D2,…DC]Dictionary D0Containing a C-type atom; if atom di∈DiThen atom diIs defined as bi=[0,0,…,0,1,0,…,0]∈RC,biThe ith element in (1) is 1, and the rest are zero, so the atom class matrix B in the dictionary D is defined as B ═ B1,b2,…,bk]T∈Rk×C(ii) a The weight class index matrix H of the dictionary is defined asUsing a matrix X of coding coefficientsTAnd constructing the following class label embedding items by the atom weight class label matrix H:
7.2 construction of structural feature constraint term model of Profiles:
each row vector of the coding coefficient matrix in dictionary learning is defined as a profile, and P ═ X is definedTThen P ═ P1,p2,…,pk]∈Rn×KWherein p isi=[xi1,…,xin]TAtom diThe corresponding profile is piIn order to improve the discrimination performance of the coding coefficient, a discriminant is designed by combining the structural features of profiles with atoms, a neighbor graph M is constructed by utilizing a P matrix, and each P isiRepresenting a vertex; the weight matrix W of the neighbor map M is calculated as follows:
wherein is the parameter, kNN (p)i) Represents piK is close to, wi,jIs piAnd pjReflects the similarity between them, in order to better reflect piAnd pjThe Laplace graph L based on the profiles characteristics is constructed as follows:
the discriminant based on the profiles features is designed as follows:
and 8: optimizing a dictionary model:
8.1 solution update of dictionary D:
assuming that the encoding coefficient matrix X, the Laplace graph L and the matrix U in the objective function are constants, the formula (2) is converted into the formula (7)
And (3) solving by using a Lagrange function to obtain an optimal dictionary D:
D=YXT(XXT+βL+ηI) (19)
where η is a parameter and I is an identity matrix;
8.2 solving and updating the coding coefficient matrix X:
assuming that the dictionary D, the laplacian graph L, and the matrix U in the objective function are constants, equation (2) translates to:
the equation (10) is directly derived to obtain the optimal coding coefficient matrix X:
X=(DTD+αU+γI)-1DTY (21)
wherein γ is a parameter, once a coding coefficient is obtained, a matrix U is obtained according to formula (3), and a laplacian graph L is obtained according to formula (5);
and step 9: constructing a linear classifier: firstly, obtaining classifier parameters G by using a coding coefficient matrix X and a marking matrix H of a training samplex:
Gx=HXT(XXT+Ι)-1(22)
Second, for the test imageComputing coding coefficient vector by orthogonal matching pursuit algorithmAnd a dictionary D, byCalculating to obtain a predicted label vector lxAnd finally, testing the sampleIs a predictive label vector lxIs used to determine the index corresponding to the maximum element value of (a).
The effect of the pre-processing is illustrated in fig. 2, which for gastroscopic images taken directly during a gastrointestinal examination, roughly contains two large areas: a background region and a tissue region. The background area often carries patient information, instrument information, time, etc., and the tissue area also has many interference factors, such as bubbles, food residues or dark areas and light spots caused by camera shooting. Firstly, after converting an original image into a gray image, searching a maximum connected area to determine a tissue area, and then cutting out a colorful tissue area from the original image according to boundary points. FIG. (a) is a raw gastroscopic image of the stomach; panel (b) is the extracted tissue region; the pretreatment method can effectively extract the tissue area and reduce the interference of interference factors on feature extraction.
As shown in fig. 3, a detailed algorithm flowchart for dictionary learning is shown. Randomly selecting 80% of images as a training set, using the rest 20% of images as a test set for verification, and initializing to obtain a dictionary D0And a coding coefficient matrix X0Constructing a class mark matrix B of the dictionary by using the class mark matrix of the training sample, calculating an atom weight class mark matrix H and an extension class mark matrix U thereof, and using a profiles matrix P0Computing an initialized Laplace graph matrix L0And outputting the final dictionary D and the encoding coefficient matrix X after multiple iterations, and calculating the accuracy of the classification of the test image through a linear classifier.
To verify the effectiveness of the invention, the following is illustrated by simulation:
the gastroscopic dataset consists of three different categories of gastroscopic sites including gastric polyps, gastric ulcers and normal stomach images, with the same number of images per site, 250 each, for a total of 750. Wherein, 200 images of each type are randomly extracted as test images, and the other 50 images are used as training images. The experimental results take the accuracy as the judgment standard.
In the experiment 1, the classification method provided by the invention is used for classifying gastroscope images, when a dictionary is trained in the experiment, in order to select proper parameters, the training samples and the test samples are randomly used for dictionary learning and classification, the parameters with better classification results are selected, and α is set to 10-4,β=10-2,λ=10-3The size of the dictionary is 200, the average value is obtained after 30 times of experiments, and as shown in table 1, the classification result is the classification result of the endoscope image classification method in the dictionary learning algorithm, and the classification accuracy of polyps, ulcers and normal images and the comprehensive average accuracy of the three types of images are respectively calculated;
categories | Polyp | Ulcer of stomach | Is normal | Synthesis of |
Rate of accuracy | 0.926 | 0.901 | 0.943 | 0.923 |
TABLE 1
By adopting the classification method provided by the invention, the average accuracy of each type of image can reach more than 90%, and meanwhile, the calculation time is rapid, thus proving the effectiveness and reliability of the method.
Experiment 2, single color features or texture features were compared to the fusion features proposed by the present invention. The atomic number of the dictionary in the experiment is 200. As shown in table 2, the endoscope image classification method of the present invention compares with other feature classification results, and calculates the average accuracy rate;
feature(s) | CH | SIFT | CH+SIFT |
Average rate of accuracy | 0.907 | 0.861 | 0.923 |
TABLE 2
Compared with single characteristics, the fusion characteristics provided by the invention obtain better results in the aspect of classification accuracy.
In the experimental verification, the following factors need to be considered:
1) when the test images are distributed, a certain number of samples are randomly selected from each type of samples to serve as a training sample set, the rest samples are used as test samples, the training deviation caused by too few training samples is avoided, and multiple experiments are carried out to obtain an average value, so that a more stable result is obtained;
2) the number of the training samples is far larger than the total class number, so that the high correlation of the training samples is ensured;
3) in the process of training the dictionary, the influence of the dictionary size on the result needs to be considered.
Therefore, the invention also carries out the discussion of the dictionary size, the object of experiment 3 is the same as experiment 2, and the three selected characteristics are tested to respectively obtain the comparison results of the accuracy under different dictionary sizes. As shown in fig. 4, the effect of dictionary size on the result varies from feature to feature. But for the fusion features selected by the invention, higher accuracy can be obtained when the dictionary is smaller. At the same time, complexity and training time may be reduced compared to larger dictionaries.
In addition, fig. 4 further reflects that the fusion feature provided by the present invention can achieve higher accuracy under each dictionary column number.
Claims (1)
1. A gastrointestinal endoscope image anomaly detection method based on local feature and class mark embedding constraint dictionary learning is characterized by comprising the following steps:
step 1: acquiring an endoscope image set which consists of polyps, ulcers and normal images;
step 2: image preprocessing: extracting a tissue area of the whole endoscope image, generating a mask in an invalid area, and removing an image with low quality;
and step 3: extracting image color features: extracting color features of the image by a color histogram method in an HSI color space;
and 4, step 4: extracting image texture features: under the HSI color space, extracting the texture features of the image by adopting a scale-invariant feature transform operator;
and 5: image feature fusion: fusing and normalizing the color and texture characteristics of the image;
step 6: initializing a dictionary and a coding coefficient matrix by using a K-SVD algorithm, wherein the model expression is as follows:
wherein Y is [ Y ]1,…,YC]=[y1,y2,…,yn]∈Rm×nRepresenting a training data set with n training samples, m being the dimension of the training samples, C being the class of the training samples, YiIs a matrix composed of training samples of the ith class; d ═ D1,d2,…,dK]T∈Rm×KRepresenting a dictionary, K is the number of atoms, diIs the ith atom of dictionary D, X ═ X1,x2,…xn]T∈RK×nIs a matrix of coding coefficients, xi=[x1i,x2i,…,xKi]TIs the ith training sample yiCoding coefficient, T, corresponding to dictionary D0Is a sparse factor, | | x | | Y0Represents a 0 norm representing the number of non-zero elements in the coefficient vector x, | x | | luminance2Represents a 2-norm whose values represent the sum of the squares of the individual elements in the coefficient vector x; solve to obtain an initialization dictionary D0=[D1,D2,…,DC]And initializing the coding coefficient matrix X0=[X1,X2,…,XC];
And 7: according to the step 6, an atom class mark embedding item and a structural feature constraint item of Profiles are introduced, and a dictionary model expression of endoscope image classification is as follows:
whereinThe reconstruction performance of the dictionary is represented,is a regular term of the coding coefficient, λ, α and β are scalar parameters;
7.1 atomic class label insert model construction:
aiming at the ith class training sample, learning a specific class dictionary D by utilizing a K-SVD algorithmiTherefore, the C class training sample is learned to obtain a special class dictionary D0=[D1,D2,…DC]Dictionary D0Containing a C-type atom; if atom di∈DiThen atom diIs defined as bi=[0,0,…,0,1,0,…,0]∈RC,biThe ith element in (1) is 1, and the rest are zero, so the atom class matrix B in the dictionary D is defined as B ═ B1,b2,…,bk]T∈Rk×C(ii) a The weight class index matrix H of the dictionary is defined asUsing a matrix X of coding coefficientsTAnd constructing the following class label embedding items by the atom weight class label matrix H:
7.2 construction of structural feature constraint term model of Profiles:
each row vector of the coding coefficient matrix in dictionary learning is defined as a profile, and P ═ X is definedTThen P ═ P1,p2,…,pk]∈Rn×KWherein p isi=[xi1,…,xin]TAtom diThe corresponding profile is piIn order to improve the discrimination performance of the coding coefficient, a discriminant is designed by combining the structural features of profiles with atoms, a neighbor graph M is constructed by utilizing a P matrix, and each P isiRepresenting a vertex; the weight matrix W of the neighbor map M is calculated as follows:
wherein is the parameter, kNN (p)i) Represents piK is close to, wi,jIs piAnd pjReflects the similarity between them, in order to better reflect piAnd pjThe Laplace graph L based on the profiles characteristics is constructed as follows:
the discriminant based on the profiles features is designed as follows:
and 8: optimizing a dictionary model:
8.1 solution update of dictionary D:
assuming that the encoding coefficient matrix X, the Laplace graph L and the matrix U in the objective function are constants, the formula (2) is converted into the formula (7)
And (3) solving by using a Lagrange function to obtain an optimal dictionary D:
D=YXT(XXT+βL+ηI) (8)
where η is a parameter and I is an identity matrix;
8.2 solving and updating the coding coefficient matrix X:
assuming that the dictionary D, the laplacian graph L, and the matrix U in the objective function are constants, equation (2) translates to:
the equation (10) is directly derived to obtain the optimal coding coefficient matrix X:
X=(DTD+αU+γI)-1DTY (10)
wherein γ is a parameter, once a coding coefficient is obtained, a matrix U is obtained according to formula (3), and a laplacian graph L is obtained according to formula (5);
and step 9: constructing a linear classifier: firstly, obtaining classifier parameters G by using a coding coefficient matrix X and a marking matrix H of a training samplex:
Gx=HXT(XXT+Ι)-1(11)
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202010316171.0A CN111667453A (en) | 2020-04-21 | 2020-04-21 | Gastrointestinal endoscope image anomaly detection method based on local feature and class mark embedded constraint dictionary learning |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202010316171.0A CN111667453A (en) | 2020-04-21 | 2020-04-21 | Gastrointestinal endoscope image anomaly detection method based on local feature and class mark embedded constraint dictionary learning |
Publications (1)
Publication Number | Publication Date |
---|---|
CN111667453A true CN111667453A (en) | 2020-09-15 |
Family
ID=72382669
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN202010316171.0A Pending CN111667453A (en) | 2020-04-21 | 2020-04-21 | Gastrointestinal endoscope image anomaly detection method based on local feature and class mark embedded constraint dictionary learning |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN111667453A (en) |
Cited By (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN112487231A (en) * | 2020-12-17 | 2021-03-12 | 中国矿业大学(北京) | Automatic image labeling method based on double-image regularization constraint and dictionary learning |
CN112541470A (en) * | 2020-12-22 | 2021-03-23 | 杭州趣链科技有限公司 | Hypergraph-based face living body detection method and device and related equipment |
CN113793319A (en) * | 2021-09-13 | 2021-12-14 | 浙江理工大学 | Fabric image flaw detection method and system based on class constraint dictionary learning model |
CN113870240A (en) * | 2021-10-12 | 2021-12-31 | 大连理工大学 | Safety valve cavitation phenomenon discrimination method based on image significance detection |
CN113989284A (en) * | 2021-12-29 | 2022-01-28 | 广州思德医疗科技有限公司 | Helicobacter pylori assists detecting system and detection device |
CN114428873A (en) * | 2022-04-07 | 2022-05-03 | 源利腾达(西安)科技有限公司 | Thoracic surgery examination data sorting method |
CN116226778A (en) * | 2023-05-09 | 2023-06-06 | 水利部珠江水利委员会珠江水利综合技术中心 | Retaining wall structure anomaly analysis method and system based on three-dimensional analysis platform |
Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN104166860A (en) * | 2014-07-25 | 2014-11-26 | 哈尔滨工业大学深圳研究生院 | Constraint-based face identification method for single test sample |
CN109376802A (en) * | 2018-12-12 | 2019-02-22 | 浙江工业大学 | A kind of gastroscope organ classes method dictionary-based learning |
CN110648276A (en) * | 2019-09-25 | 2020-01-03 | 重庆大学 | High-dimensional image data dimension reduction method based on manifold mapping and dictionary learning |
-
2020
- 2020-04-21 CN CN202010316171.0A patent/CN111667453A/en active Pending
Patent Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN104166860A (en) * | 2014-07-25 | 2014-11-26 | 哈尔滨工业大学深圳研究生院 | Constraint-based face identification method for single test sample |
CN109376802A (en) * | 2018-12-12 | 2019-02-22 | 浙江工业大学 | A kind of gastroscope organ classes method dictionary-based learning |
CN110648276A (en) * | 2019-09-25 | 2020-01-03 | 重庆大学 | High-dimensional image data dimension reduction method based on manifold mapping and dictionary learning |
Non-Patent Citations (1)
Title |
---|
李争名: "基于鲁棒判别式约束的字典学习算法研究", 《中国优秀博士学位论文全文数据库信息科技辑》 * |
Cited By (11)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN112487231A (en) * | 2020-12-17 | 2021-03-12 | 中国矿业大学(北京) | Automatic image labeling method based on double-image regularization constraint and dictionary learning |
CN112541470A (en) * | 2020-12-22 | 2021-03-23 | 杭州趣链科技有限公司 | Hypergraph-based face living body detection method and device and related equipment |
CN113793319A (en) * | 2021-09-13 | 2021-12-14 | 浙江理工大学 | Fabric image flaw detection method and system based on class constraint dictionary learning model |
CN113793319B (en) * | 2021-09-13 | 2023-08-25 | 浙江理工大学 | Fabric image flaw detection method and system based on category constraint dictionary learning model |
CN113870240A (en) * | 2021-10-12 | 2021-12-31 | 大连理工大学 | Safety valve cavitation phenomenon discrimination method based on image significance detection |
CN113870240B (en) * | 2021-10-12 | 2024-04-16 | 大连理工大学 | Safety valve cavitation phenomenon judging method based on image significance detection |
CN113989284A (en) * | 2021-12-29 | 2022-01-28 | 广州思德医疗科技有限公司 | Helicobacter pylori assists detecting system and detection device |
CN114428873A (en) * | 2022-04-07 | 2022-05-03 | 源利腾达(西安)科技有限公司 | Thoracic surgery examination data sorting method |
CN114428873B (en) * | 2022-04-07 | 2022-06-28 | 源利腾达(西安)科技有限公司 | Thoracic surgery examination data sorting method |
CN116226778A (en) * | 2023-05-09 | 2023-06-06 | 水利部珠江水利委员会珠江水利综合技术中心 | Retaining wall structure anomaly analysis method and system based on three-dimensional analysis platform |
CN116226778B (en) * | 2023-05-09 | 2023-07-07 | 水利部珠江水利委员会珠江水利综合技术中心 | Retaining wall structure anomaly analysis method and system based on three-dimensional analysis platform |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN111667453A (en) | Gastrointestinal endoscope image anomaly detection method based on local feature and class mark embedded constraint dictionary learning | |
Yu et al. | A deep convolutional neural network-based framework for automatic fetal facial standard plane recognition | |
EP3046478B1 (en) | Image analysis techniques for diagnosing diseases | |
CN109544526B (en) | Image recognition system, device and method for chronic atrophic gastritis | |
CN113379693B (en) | Capsule endoscope key focus image detection method based on video abstraction technology | |
JP2022502150A (en) | Devices and methods for diagnosing gastric lesions using deep learning of gastroscopy images | |
Ye et al. | Online tracking and retargeting with applications to optical biopsy in gastrointestinal endoscopic examinations | |
CN109635871B (en) | Capsule endoscope image classification method based on multi-feature fusion | |
CN107851194A (en) | Visual representation study for brain tumor classification | |
JP7333132B1 (en) | Multimodal medical data fusion system based on multiview subspace clustering | |
WO2023207820A1 (en) | Pancreatic postoperative diabetes prediction system based on supervised deep subspace learning | |
CN115019405A (en) | Multi-modal fusion-based tumor classification method and system | |
CN105069131A (en) | Capsule endoscopy image retrieval method based on visual vocabularies and local descriptors | |
Shariaty et al. | Radiomics: extracting more features using endoscopic imaging | |
CN109376802B (en) | Gastroscope organ classification method based on dictionary learning | |
Zimmer et al. | Learning and combining image neighborhoods using random forests for neonatal brain disease classification | |
Van Der Sommen et al. | Computer-aided detection of early cancer in the esophagus using HD endoscopy images | |
CN111383213B (en) | Mammary gland image retrieval method for multi-view discrimination metric learning | |
Sadeghi et al. | A Novel Sep-Unet architecture of convolutional neural networks to improve dermoscopic image segmentation by training parameters reduction | |
Atasoy et al. | Endoscopic video manifolds | |
CN110334582B (en) | Method for intelligently identifying and recording polyp removing video of endoscopic submucosal dissection | |
Sharmila Joseph et al. | Multiclass gastrointestinal diseases classification based on hybrid features and duo feature selection | |
Shen et al. | Locality-constrained dictionary learning classification method of wce images | |
Sharmila et al. | Detection and classification of GI-tract anomalies from endoscopic images using deep learning | |
Fathima et al. | Deep Learning and Machine Learning Approaches for Brain Tumor Detection and Classification |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
RJ01 | Rejection of invention patent application after publication |
Application publication date: 20200915 |
|
RJ01 | Rejection of invention patent application after publication |