CN114360718B - Feature fitting-based PET/CT automatic lung cancer diagnosis and classification system and construction method - Google Patents
Feature fitting-based PET/CT automatic lung cancer diagnosis and classification system and construction method Download PDFInfo
- Publication number
- CN114360718B CN114360718B CN202210231670.9A CN202210231670A CN114360718B CN 114360718 B CN114360718 B CN 114360718B CN 202210231670 A CN202210231670 A CN 202210231670A CN 114360718 B CN114360718 B CN 114360718B
- Authority
- CN
- China
- Prior art keywords
- image
- pet
- feature
- lung cancer
- classification
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
- G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
- G16H50/00—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics
- G16H50/20—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics for computer-aided diagnosis, e.g. based on medical expert systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/11—Region-based segmentation
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Health & Medical Sciences (AREA)
- Data Mining & Analysis (AREA)
- General Physics & Mathematics (AREA)
- Biomedical Technology (AREA)
- Public Health (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Medical Informatics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Engineering & Computer Science (AREA)
- Life Sciences & Earth Sciences (AREA)
- Computational Linguistics (AREA)
- Software Systems (AREA)
- Mathematical Physics (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Computing Systems (AREA)
- Evolutionary Biology (AREA)
- Databases & Information Systems (AREA)
- Molecular Biology (AREA)
- Pathology (AREA)
- Biophysics (AREA)
- Epidemiology (AREA)
- Primary Health Care (AREA)
- Apparatus For Radiation Diagnosis (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a PET/CT automatic lung cancer diagnosis classification system based on feature fitting and a construction method thereof, wherein the system comprises a feature extraction unit, a classification unit and a classification unit, wherein the feature extraction unit is used for extracting and obtaining first features of tumors according to a PET/CT image; the characteristic fitting unit is used for fitting the characteristics in the pathological images corresponding to the PET/CT images according to the PET/CT images to obtain second characteristics of the tumor; and the lung cancer diagnosis and classification unit is used for combining the first characteristic and the second characteristic to carry out lung cancer diagnosis and classification. The invention improves the diagnosis precision obtained by simply using the PET/CT image information characteristics by using the mode of fitting the matched pathological characteristics by using the PET/CT characteristics, is beneficial to the popularization and application of the existing intelligent diagnosis only based on the imaging science, and improves the diagnosis efficiency of clinicians. By the invention, the tumor condition can be better known before the operation of the patient, so that doctors can be better assisted to provide accurate treatment schemes for the patient, and the wound of the patient is reduced.
Description
Technical Field
The invention relates to the field of medical images, in particular to a PET/CT automatic lung cancer diagnosis and classification system based on feature fitting and a construction method.
Background
PET/CT has great value in the diagnosis of benign and malignant pulmonary nodules, the staging of lung cancer and the evaluation after treatment of lung cancer in modern clinical medicine. 18F-FDG (18 fluoro labelled glucose analogue) is currently the most widely used tracer in PET/CT scanning, and can maintain cellular energy supply according to abnormal proliferation of malignant tumors and needs to increase glucose uptake and glycolysis, so that different types of tumors show different degrees of glucose uptake on glucose metabolism imaging images. The tracer decays within the patient and annihilates, producing a pair of 511keV gamma photons with emission directions about 180 ° opposite, and the detector collects information about the location and time at which the gamma photons reach the crystal. The acquired information is reconstructed by using an image reconstruction algorithm and is subjected to post-processing, so that the condition of the metabolism and the ingestion of the reaction tracer in the body of the patient can be obtained. Physicians can reflect the metabolic heterogeneity characteristics in an early and quantitative way through PET imaging, and combine various clinical manifestations of patients to determine further treatment schemes.
Pathological examination is a pathological diagnosis, called biopsy (biopsy), called living body for short, in which a small tissue taken from a diseased part of a patient is prepared into a pathological section, and then the pathological section is stained to determine the nature of the lesion through examination of cell morphology, histopathology and the like under a microscope. That is, a pathomorphological method for examining pathological changes in body organs, tissues or cells is one of the examination methods which has the highest diagnostic accuracy among all examinations and is known as "gold standard". But the way it is checked is invasive.
With the great success of the deep learning technology in the field of computer vision, more and more scholars apply the deep learning technology to medical image analysis, such as classification, detection and segmentation, registration and retrieval, and achieve better effects. Deep learning can automatically learn the salient high-order features of a specific task from a large amount of data so as to complete the specific task, but has certain requirements on the training data volume.
Meanwhile, deep learning also emerges various methods for solving the practical problem, such as multitask learning, knowledge distillation and other methods can be used for reference. Multitask learning (multi task learning): simply speaking, there are multiple objective functions loss to learn simultaneously even though there is multi-task learning. For example, a short video with a big fire at present usually predicts information of multiple dimensions such as how long a user is interested in/not interested in the video, and looks like/not like, and forwards/not forwards the video. The multiple tasks can be learned by establishing a model for each task, and can also be learned by multiple tasks of one model for one-time full learning. Knowledge distillation (knowledge distillation) is a common method for model compression, and is different from pruning and quantification in model compression, and knowledge distillation is to train a small model by constructing a lightweight small model and utilizing supervision information of a large model with better performance so as to achieve better performance and precision. Originally proposed and applied on top of the classification task by Hinton in 2015, this large model was called teacher (teacher model) and the small model was called Student (Student model). The supervisory information from the output of the Teacher model is called knowledge, while the process by which students learn to migrate the supervisory information from Teacher is called Distillation.
For the existing automatic lung cancer diagnosis classification model based on PET/CT, two schemes of a classification model based on image omics and a classification network based on deep learning are mainly provided. Imaging omics is a new emerging research method which is currently emerging, and is proposed by dutch scholars in 2012, and the imaging omics method mainly extracts high-flux image features through images of different modalities (such as CT, MRI, PET and the like), and assists physicians in making the most accurate judgment by means of deeper mining, prediction and analysis of mass image data information, so as to realize the evaluation of tumor heterogeneity and the prognosis evaluation of tumors. Deep learning methods mostly adopt convolutional neural networks to construct specific classification models for tumor classification, such as Resnet, DenseNet and other classification networks and their variants. Compared with biopsy, the two methods can reduce the pain caused by biopsy, improve the working efficiency to a certain extent, reduce the economic burden of patients and provide a healthier and safer way for future patient disease review. Although the existing automatic classification model based on PET/CT for lung cancer diagnosis has achieved better classification accuracy, there is still a long way to go for clinical requirements. Although the existing automatic lung cancer diagnosis classification model based on pathology can obtain higher classification precision, the diagnosis result is slower and traumatic, which is a disadvantage for the rapid diagnosis of lung cancer screening. Therefore, how to combine the advantages of the image and the pathology to shield the disadvantages of the image and the pathology, and introducing the information of the pathological image into the image when the network is trained to be used as prior information during use, so that the accuracy of the image during application is improved, the precision of the PET/CT automatic lung cancer diagnosis is greatly improved, a doctor is better assisted to perform early diagnosis on a patient, and the better help of the patient to make a diagnosis and treatment scheme is also one of the current research hotspots.
Disclosure of Invention
The invention aims to provide a PET/CT automatic lung cancer diagnosis classification system based on feature fitting and a construction method thereof aiming at the defects of the prior art, and the system and the construction method use a feature fitting network to fit out features which are most similar to corresponding pathological features from PET/CT by using multitask learning and distillation learning technology and are superposed on the features extracted by the conventional classification network for assisting the lung cancer diagnosis based on PET/CT images so as to achieve the effects of improving the training efficiency and the upper limit of precision of the PET/CT network.
The purpose of the invention is realized by the following technical scheme:
a PET/CT automatic lung cancer diagnosis and classification system based on feature fitting comprises:
the characteristic extraction unit is used for extracting and obtaining first characteristics of the tumor according to the PET/CT image;
the characteristic fitting unit is used for fitting the characteristics in the pathological images corresponding to the PET/CT images according to the PET/CT images to obtain second characteristics of the tumor;
and the lung cancer diagnosis and classification unit is used for combining the first characteristic and the second characteristic to carry out lung cancer diagnosis and classification.
Further, still include:
the image preprocessing unit is used for removing non-tumor areas in the PET/CT images, operating according to the corresponding PET/CT image labels of professionals, simultaneously acquiring patch images of the tumor areas, and ensuring that the patch size can include the whole tumor area.
Further, the feature fitting unit is a feature fitting convolutional neural network obtained by taking the PET/CT image as input and inputting the feature of the pathological image corresponding to the PET/CT image as a label for training.
Further, the input PET/CT image is extracted by a feature extractor obtained by lung cancer diagnosis classification training based on pathological images. The lung cancer diagnosis classification training based on the pathological images refers to training taking the pathological images as input and lung cancer diagnosis classification as a prediction target.
Further, the extraction method specifically comprises the following steps:
performing operation according to the corresponding pathological image label of the professional, segmenting the input PET/CT image corresponding to the pathological image to obtain a plurality of pathological patch images containing tumor regions;
and taking each pathology patch image as the input of a feature extractor obtained by lung cancer diagnosis classification training based on the pathology image, extracting and obtaining a plurality of features, and selecting one of the plurality of features as the feature of the pathology image.
Further, selecting one of the plurality of features as the feature of the pathological image, and setting different screening schemes according to different situations, wherein a specific screening scheme is preferably as follows: and selecting the feature with the minimum sum of Euclidean distances to all other features as the feature of the pathological image.
Further, the tumor coverage rate of the pathology patch image is greater than or equal to 0.8, that is, the area ratio of the area of the region framed according to the pathology image labels of the professionals to the whole pathology patch image is greater than or equal to 0.8.
Further, the feature extraction unit is a feature extractor obtained by training lung cancer diagnosis classification based on PET/CT images. The training of lung cancer diagnosis classification based on the PET/CT image is training taking the PET/CT image as input and lung cancer diagnosis classification as a prediction target.
A construction method of a PET/CT automatic lung cancer diagnosis and classification system based on feature fitting comprises the following steps:
the method comprises the steps of collecting PET/CT images and pathological images of a patient, obtaining corresponding lung cancer diagnosis classification results, and segmenting the pathological images to obtain a plurality of pathological patch images containing tumor regions.
And taking the pathology patch image corresponding to each feature extractor as the input of the feature extractor obtained by the lung cancer diagnosis classification training based on the pathology images, extracting to obtain a plurality of features, and selecting one of the plurality of features as the feature of the pathology image corresponding to the PET/CT image.
Constructing a convolutional neural network, inputting a PET/CT image, inputting the characteristics of a pathological image corresponding to the PET/CT image as a label for training, and using the trained convolutional neural network as a characteristic fitting unit;
and constructing a feature extractor and a classifier, taking the PET/CT image as the input of the feature extractor and the feature fitting unit, superposing the output features of the feature extractor and the feature fitting unit, then taking the superposed output features as the input of the classifier, training the lung cancer diagnosis classification result of the PET/CT image as a label, taking the trained feature extractor as a feature extraction unit, and taking the classifier as a lung cancer diagnosis classification unit.
The method has the advantages that the PET/CT image and the pathology based on the same case have the same characteristics in a certain space, and the classification characteristics of the pathology are more prominently expressed as the golden standard of the existing clinical diagnosis, so that the PET/CT characteristics are used for fitting the corresponding pathological characteristics, and the diagnosis classification accuracy of the PET/CT image is improved in an assisting manner. Therefore, before pathological examination, more accurate diagnosis information can be provided for a clinician through non-invasive PET/CT images compared with the existing classification scheme, and the diagnosis efficiency of the clinician is improved.
Drawings
FIG. 1 is a schematic diagram of a feature fitting-based PET/CT automatic lung cancer diagnosis system according to the present invention;
FIG. 2 is a schematic diagram showing the structure of a feature extractor obtained from the classification training of pathological image-based lung cancer diagnosis and the structural relationship with the PET/CT automatic lung cancer diagnosis system based on feature fitting according to a preferred embodiment of the present invention;
FIG. 3 is a flow chart of the construction of a PET/CT automatic lung cancer diagnosis system based on feature fitting according to a preferred embodiment of the present invention;
FIG. 4 is a diagram of a neural network structure of a PET/CT automatic lung cancer diagnosis system based on feature fitting according to a preferred embodiment of the present invention.
Detailed Description
Reference will now be made in detail to the exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, like numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present application, as detailed in the appended claims.
The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the application.
As used in this application and the appended claims, the singular forms "a", "an", and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should also be understood that the term "and/or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items.
It is to be understood that although the terms first, second, third, etc. may be used herein to describe various information, such information should not be limited to these terms. These terms are only used to distinguish one type of information from another. For example, without departing from
In the context of the present application, the first information may also be referred to as second information, and similarly, the second information may also be referred to as first information. The word "if" as used herein may be interpreted as "at … …" or "when … …" or "in response to a determination", depending on the context.
The invention provides a PET/CT automatic lung cancer diagnosis and classification system based on feature fitting, as shown in figure 1, comprising:
the characteristic extraction unit is used for extracting and obtaining first characteristics of the tumor according to the PET/CT image;
the characteristic fitting unit is used for fitting the characteristics in the pathological images corresponding to the PET/CT images according to the PET/CT images to obtain second characteristics of the tumor;
and the lung cancer diagnosis and classification unit is used for carrying out diagnosis and classification on the lung cancer by combining the first characteristic and the second characteristic, and the general classification comprises lung adenocarcinoma and lung squamous carcinoma.
The invention uses the PET/CT image to fit the characteristics of the corresponding pathological image, and can provide more accurate diagnosis information for a clinician only through the non-invasive PET/CT image, thereby assisting in improving the diagnosis classification precision of the PET/CT image.
Since the actually extracted features of the tumor in the PET/CT image not only increase the calculation amount but also affect the feature extraction accuracy for the non-tumor region in the PET/CT image, as a preferred embodiment, the PET/CT automatic lung cancer diagnosis and classification system based on feature fitting further includes: and the image preprocessing unit is used for removing non-tumor regions in the PET/CT image. Illustratively, the PET/CT image cut patch may be operated on a tumor mask. Acquiring an circumscribed cube according to the tumor, expanding a plurality of pixels outwards, acquiring a tumor region in the PET/CT image according to the cube, and zeroing a part which is 0 in mask data in the tumor region, namely removing a non-tumor region in the PET/CT image, wherein the part is used as the input of the PET/CT automatic lung cancer diagnosis and classification system. Here, in order to facilitate subsequent training of the deep convolutional neural network model, the size of each patch may be selected to be uniform, and it is ensured that the size may completely contain all tumors in the batch of data.
Preferably, the feature fitting unit is a feature fitting convolutional neural network obtained by taking the PET/CT image as input and inputting the feature of the pathological image corresponding to the PET/CT image as a label for training. The input PET/CT image is obtained by extracting the features of the pathological image through a feature extractor obtained by training lung cancer diagnosis classification based on the pathological image, as shown in fig. 2. The extraction method specifically comprises the following steps:
segmenting the input PET/CT image corresponding to the pathological images to obtain a plurality of pathological patch images containing tumor regions;
and taking each pathology patch image as the input of a feature extractor obtained by lung cancer diagnosis classification training based on the pathology image, extracting to obtain a plurality of features, and selecting one feature from the plurality of features as the feature of the pathology image. The feature of the pathological image may be selected randomly or according to a rule, and preferably, the feature in which the sum of euclidean distances to all other features is the smallest is selected as the feature of the pathological image.
Corresponding to the above PET/CT automatic lung cancer diagnosis and classification system based on feature fitting, the present invention further provides a method for constructing the above PET/CT automatic lung cancer diagnosis and classification system based on feature fitting, which is described in detail below according to embodiments and drawings, wherein the structure of the construction method is shown in fig. 2, the construction process is shown in fig. 3, and the method specifically includes:
the method comprises the following steps: constructing a training data set, and collecting one-to-one matched PET/CT image dataAnd full scan pathology image. Simultaneously acquiring lung cancer diagnosis classification results corresponding to the lung cancer diagnosis classification results as label dataAnd. Here, image labels of the same patient caseAnd pathological labelingAre consistent.
In order to improve the accuracy, as a preferable scheme, the data is preprocessed:
wherein, forData, first for all PET/CT image dataLayer thickness level for uniform imaging: interpolating the image data with larger layer thickness to the layer thickness level with smaller layer thickness of the image data;
second according to the tumor maskPerforming a cut-and-patch operation based on the mask dataAcquiring a circumscribed cube, and acquiring image data according to the cubeAnd in a patch image ofData of maskThe portion of 0 is zeroed. Obtaining a Patch image of the tumor from which the non-tumor region was removed and recording the Patch image asAs input for subsequent classification of the network. Here, in order to facilitate subsequent training of the deep convolutional neural network model, the size of each patch may be selected to be uniform, and it is ensured that the size may completely contain all tumors in the batch of data. The method comprises the following steps: first, all the mask data in the data batch are usedThe maximum tumor size is obtained and then must be an exponential of 2 or an exponential of 7 by 2 depending on the length, width, and height depending on the maximum size. In this embodiment, the calculated patch size is。
For theFirstly, in order to avoid the influence of color inconsistency caused by pathological staining on subsequent results, the color normalization operation is carried out: selecting a pathological image with better dyeing effect as a target pathological image according to the recommendation of a professional doctor in all pathological dataNormalizing the color of the pathology image of all other cases to the target pathology image using a structure-preserving color normalization (SPCN) techniqueThe same color level, thereby eliminating the network performance reduction caused by the inconsistency of pathological staining colors;
secondly, because the pathological image of the full scan is very large and is difficult to be directly input into the convolutional neural network for training, the tumor region is selected according to the tumor mask of the pathological image, and then the cut patch operation is performed, specifically:
mask data from full scan pathology imageFor pathological imagePerforming a cut patch operation, each pathological patch imageMust contain the tumor area, preferably to satisfy the tumor mask dataThe coverage of (a) is 80% or more, namely:
whereinTo representThe area occupied by the doctor in the doctor for marking the tumor,representing the area occupied by the pathology patch image. Simultaneous each pathology patch imageAre of the same size, e.g. allEach pathology patch imageThe label selects the corresponding full-scanning pathological imageTag data ofAs a corresponding label;
Further, in order to ensure the data balance, an overlay-tile strategy (overlay-tile strategy) may be adopted, so that each is enabled to be uniformThe number of cuts remains substantially the same.
The mask can be made by a professional doctor to mark the tumor region, or other conventional mask making methods.
Step two: and training a feature extractor according to a scheme of five-fold cross validation used by a case, and screening to obtain the features of all pathological images. The training of the feature extractor obtained by the pathological image-based lung cancer diagnosis classification training is based on five-fold cross validation of data, five groups of feature extraction parameters are obtained by the five-fold cross validation, and then the features in the five complementary test sets are respectively extracted. And taking the pathological patch image of the test set corresponding to each feature extractor as the input of the feature extractor obtained based on the lung cancer diagnosis classification training of the pathological image, extracting to obtain a plurality of features, and selecting one of the features as the feature of the pathological image corresponding to the PET/CT image. Here, if the number of data sets is small, a scheme such as ten-fold cross validation may be selected for adjustment.
Illustratively, the feature extractor obtained by training lung cancer diagnosis classification based on pathological images refers to a feature extraction structure of a classification network CNN1 obtained by training lung cancer diagnosis classification as a prediction target with the pathological images as input, and the pathological patch images cut in the step one are subjected toAnd label thereforInputting the data into a classification network CNN1 for training, obtaining a better result and storing a network parameter D of CNN11And reserving the feature extraction part to obtain the feature extractor. Because the pathological data are gold standards, a better classification result can be quickly achieved on the lung cancer diagnosis data set, and the classification precision of the test set is required to be more than 95%. Taking ResNet-18 as the main structure of the classification network CNN1 as an example (shown in Table 1 as ResNet-18 network structure), the training specifically comprises the following steps:
TABLE 1 ResNet-18 network architecture
WhereinMNumber representing the tumor diagnostic classification, 2 in this example, lung adenocarcinoma and lung squamous carcinoma, respectively;
1. according to the pathological imagePatch image to be cut outDividing the data into a training set and a test set, and dividing the data of each fold into the training set accounting for 80 percent and the test set accounting for 20 percent according to a five-fold cross validation strategy, wherein the data of the five folds are complementary, namely the data of the test set of the five folds are not overlapped.
2. Samples in the training set (pathology images)Excised pathology patch image) The input is trained in the constructed classification network CNN1 network, and the cross entropy loss function is selected as the loss value of the classification network CNN1 and is recorded as loss _1 for back propagation. The specific calculation method of loss _1 is as follows:
whereinMRepresenting the number of diagnostic categories of the tumor;y ic is a sign function (0 or 1) ifiTrue class of sample equalscTaking 1, otherwise, taking 0;p ic indicating that the input belongs to the category after passing through the network softmax functioncThe prediction probability of (2).
3. Until the loss function is converged or the iteration times are reached, the training is completed, the test set is adopted for verification and testing, and then a group of network parameters D with the best effect is stored1Meanwhile, the classification accuracy of the test set is required to reach the targetMore than 95 percent.
4. Use of network parameter D in a classifying network CNN11Keeping the average pooling layer and the network structure before the average pooling layer as a feature extractor, and outputting the features of the pathology patch image by the feature extractor。
Because the number of the patches cut out from a full-scan pathological image is huge, and the number of the corresponding patches cut out from the PET/CT image is only 1, if the feature fitting is directly carried out, one image feature exists during the feature fittingCorresponding to a plurality of pathological patch image featuresTherefore, a screening operation for pathological characteristics is required. Can randomly select a pathological patch image featureFitting is performed, but this is not an optimal solution, and preferably, features closest to features of other pathology patch images are selected, and the feature screening rule selected in this example is specifically:
for all acquired pathology patch image featuresCalculating Euclidean distance between every two images, and selecting all other pathological patch image featuresThe characteristic of the pathology patch image with the minimum Euclidean distance sum is taken as the characteristic of the pathology imageF PI 。
Step three: constructing a convolution neural network CNN2 and preprocessing the PET/CT tumor patch image in the step oneAs input, the characteristics of the pathological image corresponding to the PET/CT image are inputF PI Training as a label, fitting the characteristics of the corresponding pathological image by using a PET/CT image, fixing the network parameters after obtaining a better result, and obtaining a characteristic fitting unit; the method specifically comprises the following steps:
1. a convolutional neural network CNN2 based on PET/CT is established, the convolutional neural network CNN1 in the present embodiment adopts a feature extraction part of ResNet-18 as a main feature fitting network, and the specific structure is shown in table 2:
TABLE 2 feature fitting network
2. The tumor patch image of the PET/CT preprocessed in the step one is processedThe image is input into a convolutional neural network CNN2 as input, and the PET/CT tumor patch image output by the convolutional neural network CNN2Is characterized by the fitting ofInputting the characteristics of the pathological image corresponding to the PET/CT imageF PI As a label, a scheme of calculating the cosine similarity as a loss function of the convolutional neural network CNN2, which is denoted as loss _2, is selected here for back propagation. The specific calculation method is as follows:
whereinThe dimension representing the output characteristic of the convolutional neural network CNN2,to representThe mean value in the dimension of the object,to representMean in dimension.
3. Until the loss function is converged or the iteration times are reached, the training is completed, and after a better result is obtained, the network parameter D is fixed2And obtaining a feature fitting unit.
Step four: constructing a feature extractor and a classifier, and using the tumor patch image of the PET/CT preprocessed in the step oneThe feature extractor and the feature fitting unit are used as input of the feature extractor and the feature fitting unit, output features of the feature extractor and the feature fitting unit are mixed and then used as input of the classifier, lung cancer diagnosis classification results of the PET/CT images are used as labels to be trained, the trained feature extractor is used as the feature extraction unit, and the classifier is used as the lung cancer diagnosis classification unit.
Wherein, the feature extractor and classifier can be an integrated PET/CT-based lung cancer diagnosis classification convolutional neural network CNN3, which operates in conjunction with the PET/CT-based feature fitting convolutional neural network CNN2, namely: after the feature extraction of the CNN3 is completed, the features extracted by the CNN3 and the features extracted by the CNN2 are mixed and then input into a classification network based on a full connection layer in the CNN3 to perform lung cancer diagnosis and classification, and the feature extraction and classification network parts in the trained lung cancer diagnosis and classification convolutional neural network CNN3 are respectively used as a feature extraction unit and a lung cancer diagnosis and classification unit, which specifically comprises the following steps:
1. establishing a lung cancer diagnosis classification network CNN3 based on PET/CT, wherein the lung cancer diagnosis classification network CNN3 based on PET/CT in the example adopts a DenseNet-121 structure, and the specific structure is as shown in Table 3:
table 3: DenseNet-121 network structure
2. The tumor patch image of the PET/CT preprocessed in the step one is processedAs input to the PET/CT-based lung cancer diagnosis and classification network CNN3 and the feature fitting unit, and extracting features of the PET/CT-based lung cancer diagnosis and classification network CNN3 behind the average pooling layer of the PET/CT-based lung cancer diagnosis and classification network CNN3 networkFeatures extracted by the feature fitting unitPerforming superposition operation, then entering a rear full-connection layer classification network, outputting a lung cancer diagnosis classification result, training a CNN3 network by cooperating with the feature fitting of a feature fitting unit, wherein the loss calculation selects a cross entropy loss function which is consistent with a formula (2) until the loss function is converged or the iteration times are reached, and fixing a network parameter D thereof3Namely, a feature extraction unit and a lung cancer diagnosis classification unit are obtained.
The constructed PET/CT automatic lung cancer diagnosis and classification system based on feature fitting is shown in FIG. 4, and can be used for lung cancer diagnosis and classification, and the classification precision is higher than that of the current system only using image dataThe trained network classification precision is high, and experimental results show that the classification precision is at least improved by 2-3%.
It should be understood that the above examples are only for clarity of illustration and are not intended to limit the embodiments. Other variations and modifications will be apparent to persons skilled in the art in light of the above description. This need not be, nor should all embodiments be exhaustive. And obvious variations or modifications of the invention may be made without departing from the scope of the invention.
Claims (7)
1. A PET/CT automatic lung cancer diagnosis and classification system based on feature fitting is characterized by comprising:
the characteristic extraction unit is used for extracting and obtaining first characteristics of the tumor according to the PET/CT image;
the characteristic fitting unit is used for fitting the characteristics in the pathological images corresponding to the PET/CT images according to the PET/CT images to obtain second characteristics of the tumor; the characteristic fitting unit is a characteristic fitting convolutional neural network obtained by taking the PET/CT image as input and training the input PET/CT image by taking the characteristic of the pathological image corresponding to the PET/CT image as a label; the input PET/CT image corresponding to the pathological image is extracted by a feature extractor obtained by lung cancer diagnosis classification training based on the pathological image;
a lung cancer diagnosis classification unit for classifying the lung cancer diagnosis by combining the first feature and the second feature;
the feature extraction unit and the lung cancer diagnosis classification unit are respectively a feature extraction and classification network part in a lung cancer diagnosis classification convolutional neural network CNN3 obtained by the classification training of lung cancer diagnosis based on PET/CT images;
the lung cancer diagnosis classification convolutional neural network CNN3 is obtained by taking a PET/CT image as the input of a convolutional neural network CNN3 feature extraction part and a feature fitting unit, superposing the output features of the convolutional neural network CNN3 feature extraction part and the feature fitting unit and then taking the superposed output features as the input of a convolutional neural network CNN3 classification network part, and training the lung cancer diagnosis classification result of the PET/CT image as a label.
2. The system of claim 1, further comprising:
and the image preprocessing unit is used for removing non-tumor regions in the PET/CT image.
3. The system according to claim 1, wherein the extraction method is specifically:
segmenting the input PET/CT image corresponding to the pathological images to obtain a plurality of pathological patch images containing tumor regions;
and taking each pathology patch image as the input of a feature extractor obtained by lung cancer diagnosis classification training based on the pathology image, extracting and obtaining a plurality of features, and selecting one of the plurality of features as the feature of the pathology image.
4. The system according to claim 3, characterized in that as a feature of the pathological image is selected from a plurality of features, in particular: and selecting the feature with the minimum sum of Euclidean distances to all other features as the feature of the pathological image.
5. The system of claim 3, wherein a tumor coverage of the pathology patch image is equal to or greater than 0.8.
6. The system according to claim 3, wherein the feature extraction unit is a feature extractor obtained based on a lung cancer diagnosis classification training of PET/CT images.
7. A method for constructing a PET/CT automatic lung cancer diagnosis and classification system based on feature fitting is characterized by comprising the following steps:
acquiring a PET/CT image and a pathological image of a patient, acquiring a corresponding lung cancer diagnosis classification result, and segmenting the pathological image to obtain a plurality of pathological patch images containing tumor regions;
taking each pathology patch image as the input of a feature extractor obtained by lung cancer diagnosis classification training based on pathology images, extracting and obtaining a plurality of features, and selecting one feature from the plurality of features as the feature of the pathology image corresponding to the PET/CT image;
constructing a convolutional neural network, taking the PET/CT image as input, taking the characteristics of the pathological image corresponding to the input PET/CT image as a label for training, and taking the trained convolutional neural network as a characteristic fitting unit;
constructing a lung cancer diagnosis classification convolutional neural network CNN3, taking a PET/CT image as the input of a convolutional neural network CNN3 feature extraction part and a feature fitting unit, overlapping the output features of the convolutional neural network CNN3 feature extraction part and the feature fitting unit, and then taking the overlapped output features as the input of a convolutional neural network CNN3 classification network part, taking the lung cancer diagnosis classification result of the PET/CT image as a label for training, and respectively taking the feature extraction and classification network part in the trained lung cancer diagnosis classification convolutional neural network CNN3 as a feature extraction unit and a lung cancer diagnosis classification unit.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202210231670.9A CN114360718B (en) | 2022-03-10 | 2022-03-10 | Feature fitting-based PET/CT automatic lung cancer diagnosis and classification system and construction method |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202210231670.9A CN114360718B (en) | 2022-03-10 | 2022-03-10 | Feature fitting-based PET/CT automatic lung cancer diagnosis and classification system and construction method |
Publications (2)
Publication Number | Publication Date |
---|---|
CN114360718A CN114360718A (en) | 2022-04-15 |
CN114360718B true CN114360718B (en) | 2022-07-12 |
Family
ID=81094707
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN202210231670.9A Active CN114360718B (en) | 2022-03-10 | 2022-03-10 | Feature fitting-based PET/CT automatic lung cancer diagnosis and classification system and construction method |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN114360718B (en) |
Families Citing this family (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN114796901A (en) * | 2022-05-30 | 2022-07-29 | 北京大学第一医院 | Automatic drawing method, equipment and storage medium for lumbosacral nerve root |
CN117831757B (en) * | 2024-03-05 | 2024-05-28 | 之江实验室 | Pathological CT multi-mode priori knowledge-guided lung cancer diagnosis method and system |
Citations (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN108694718A (en) * | 2018-05-28 | 2018-10-23 | 中山大学附属第六医院 | The same period new chemoradiation therapy curative effect evaluation system and method before rectal cancer |
CN108898160A (en) * | 2018-06-01 | 2018-11-27 | 中国人民解放军战略支援部队信息工程大学 | Breast cancer tissue's Pathologic Grading method based on CNN and image group Fusion Features |
CN108986912A (en) * | 2018-07-12 | 2018-12-11 | 北京三医智慧科技有限公司 | Chinese medicine stomach trouble tongue based on deep learning is as information intelligent processing method |
CN113077434A (en) * | 2021-03-30 | 2021-07-06 | 零氪智慧医疗科技(天津)有限公司 | Method, device and storage medium for lung cancer identification based on multi-modal information |
CN113516210A (en) * | 2021-09-14 | 2021-10-19 | 之江实验室 | Lung adenocarcinoma squamous carcinoma diagnosis model training method and device based on PET/CT |
CN113689382A (en) * | 2021-07-26 | 2021-11-23 | 北京知见生命科技有限公司 | Tumor postoperative life prediction method and system based on medical images and pathological images |
Family Cites Families (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US7298883B2 (en) * | 2002-11-29 | 2007-11-20 | University Of Chicago | Automated method and system for advanced non-parametric classification of medical images and lesions |
US20210093278A1 (en) * | 2019-09-30 | 2021-04-01 | GE Precision Healthcare LLC | Computed tomography medical imaging intracranial hemorrhage model |
-
2022
- 2022-03-10 CN CN202210231670.9A patent/CN114360718B/en active Active
Patent Citations (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN108694718A (en) * | 2018-05-28 | 2018-10-23 | 中山大学附属第六医院 | The same period new chemoradiation therapy curative effect evaluation system and method before rectal cancer |
CN108898160A (en) * | 2018-06-01 | 2018-11-27 | 中国人民解放军战略支援部队信息工程大学 | Breast cancer tissue's Pathologic Grading method based on CNN and image group Fusion Features |
CN108986912A (en) * | 2018-07-12 | 2018-12-11 | 北京三医智慧科技有限公司 | Chinese medicine stomach trouble tongue based on deep learning is as information intelligent processing method |
CN113077434A (en) * | 2021-03-30 | 2021-07-06 | 零氪智慧医疗科技(天津)有限公司 | Method, device and storage medium for lung cancer identification based on multi-modal information |
CN113689382A (en) * | 2021-07-26 | 2021-11-23 | 北京知见生命科技有限公司 | Tumor postoperative life prediction method and system based on medical images and pathological images |
CN113516210A (en) * | 2021-09-14 | 2021-10-19 | 之江实验室 | Lung adenocarcinoma squamous carcinoma diagnosis model training method and device based on PET/CT |
Also Published As
Publication number | Publication date |
---|---|
CN114360718A (en) | 2022-04-15 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN114360718B (en) | Feature fitting-based PET/CT automatic lung cancer diagnosis and classification system and construction method | |
CN111445946B (en) | Calculation method for calculating lung cancer genotyping by using PET/CT (positron emission tomography/computed tomography) images | |
CN109389584A (en) | Multiple dimensioned rhinopharyngeal neoplasm dividing method based on CNN | |
CN113516210B (en) | Lung adenocarcinoma squamous carcinoma diagnosis model training method and device based on PET/CT | |
CN107203989A (en) | End-to-end chest CT image dividing method based on full convolutional neural networks | |
WO2023060944A1 (en) | Liver ct image segmentation system and algorithm based on hybrid supervised learning | |
CN113889261B (en) | Pathological feature assistance-based PET/CT automatic lung cancer diagnosis classification model training method | |
CN115063592B (en) | Multi-scale-based full-scanning pathological feature fusion extraction method and system | |
CN113516653B (en) | Method for identifying glioma recurrence and necrosis through multi-feature fusion calculation | |
CN112200763A (en) | Liver function grading method based on liver CT image | |
CN114398979A (en) | Ultrasonic image thyroid nodule classification method based on feature decoupling | |
CN113011257A (en) | Breast cancer immunohistochemical artificial intelligence interpretation method | |
CN110008836B (en) | Feature extraction method of hyperspectral image of pathological tissue slice | |
CN113538422B (en) | Pathological image automatic classification method based on dyeing intensity matrix | |
CN114565601A (en) | Improved liver CT image segmentation algorithm based on DeepLabV3+ | |
Wu et al. | Transformer-based 3D U-Net for pulmonary vessel segmentation and artery-vein separation from CT images | |
CN114764855A (en) | Intelligent cystoscope tumor segmentation method, device and equipment based on deep learning | |
Do et al. | Supporting thyroid cancer diagnosis based on cell classification over microscopic images | |
CN115132275B (en) | Method for predicting EGFR gene mutation state based on end-to-end three-dimensional convolutional neural network | |
Sun et al. | Liver tumor segmentation and subsequent risk prediction based on Deeplabv3+ | |
CN115471512A (en) | Medical image segmentation method based on self-supervision contrast learning | |
CN111027548A (en) | Brain glioma molecular marker nondestructive prediction method and system based on image omics | |
CN110738649A (en) | training method of Faster RCNN network for automatic identification of stomach cancer enhanced CT images | |
CN115101150A (en) | Specimen collection method for clinical tumor operation in general surgery department | |
Gao et al. | A high-level feature channel attention unet network for cholangiocarcinoma segmentation from microscopy hyperspectral images |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
GR01 | Patent grant | ||
GR01 | Patent grant |