CN111554384A - Adenocarcinoma pathological image analysis method based on prior perception and multitask learning - Google Patents
Adenocarcinoma pathological image analysis method based on prior perception and multitask learning Download PDFInfo
- Publication number
- CN111554384A CN111554384A CN202010405794.5A CN202010405794A CN111554384A CN 111554384 A CN111554384 A CN 111554384A CN 202010405794 A CN202010405794 A CN 202010405794A CN 111554384 A CN111554384 A CN 111554384A
- Authority
- CN
- China
- Prior art keywords
- adenocarcinoma
- benign
- branch
- malignant
- image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 208000009956 adenocarcinoma Diseases 0.000 title claims abstract description 108
- 230000001575 pathological effect Effects 0.000 title claims abstract description 57
- 230000008447 perception Effects 0.000 title claims abstract description 26
- 238000003703 image analysis method Methods 0.000 title claims abstract description 13
- 230000011218 segmentation Effects 0.000 claims abstract description 69
- 230000003211 malignant effect Effects 0.000 claims abstract description 67
- 210000004907 gland Anatomy 0.000 claims abstract description 62
- 238000000034 method Methods 0.000 claims abstract description 47
- 238000010191 image analysis Methods 0.000 claims abstract description 43
- 230000000762 glandular Effects 0.000 claims abstract description 27
- 238000007781 pre-processing Methods 0.000 claims abstract description 23
- 230000007170 pathology Effects 0.000 claims description 57
- 238000012545 processing Methods 0.000 claims description 17
- 238000000605 extraction Methods 0.000 claims description 15
- 230000008569 process Effects 0.000 claims description 11
- 238000005457 optimization Methods 0.000 claims description 10
- 238000010276 construction Methods 0.000 claims description 8
- 238000013508 migration Methods 0.000 claims description 3
- 230000005012 migration Effects 0.000 claims description 3
- 238000011176 pooling Methods 0.000 claims description 3
- 239000013598 vector Substances 0.000 claims description 3
- 238000003745 diagnosis Methods 0.000 abstract description 12
- 210000001519 tissue Anatomy 0.000 description 16
- 238000010827 pathological analysis Methods 0.000 description 5
- 201000011510 cancer Diseases 0.000 description 3
- 238000013135 deep learning Methods 0.000 description 3
- 238000010586 diagram Methods 0.000 description 3
- 230000004069 differentiation Effects 0.000 description 3
- PCHJSUWPFVWCPO-UHFFFAOYSA-N gold Chemical compound [Au] PCHJSUWPFVWCPO-UHFFFAOYSA-N 0.000 description 3
- 238000012549 training Methods 0.000 description 3
- 206010052360 Colorectal adenocarcinoma Diseases 0.000 description 2
- 208000010507 Adenocarcinoma of Lung Diseases 0.000 description 1
- 206010028980 Neoplasm Diseases 0.000 description 1
- 206010060862 Prostate cancer Diseases 0.000 description 1
- 208000000236 Prostatic Neoplasms Diseases 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 201000010897 colon adenocarcinoma Diseases 0.000 description 1
- 208000029742 colonic neoplasm Diseases 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- YQGOJNYOYNNSMM-UHFFFAOYSA-N eosin Chemical compound [Na+].OC(=O)C1=CC=CC=C1C1=C2C=C(Br)C(=O)C(Br)=C2OC2=C(Br)C(O)=C(Br)C=C21 YQGOJNYOYNNSMM-UHFFFAOYSA-N 0.000 description 1
- 201000005249 lung adenocarcinoma Diseases 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
- G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
- G16H30/00—ICT specially adapted for the handling or processing of medical images
- G16H30/40—ICT specially adapted for the handling or processing of medical images for processing medical images, e.g. editing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/11—Region-based segmentation
-
- G—PHYSICS
- G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
- G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
- G16H70/00—ICT specially adapted for the handling or processing of medical references
- G16H70/60—ICT specially adapted for the handling or processing of medical references relating to pathologies
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Health & Medical Sciences (AREA)
- Data Mining & Analysis (AREA)
- General Health & Medical Sciences (AREA)
- General Physics & Mathematics (AREA)
- Public Health (AREA)
- Evolutionary Computation (AREA)
- Medical Informatics (AREA)
- General Engineering & Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Primary Health Care (AREA)
- Epidemiology (AREA)
- Artificial Intelligence (AREA)
- Life Sciences & Earth Sciences (AREA)
- Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
- Radiology & Medical Imaging (AREA)
- Evolutionary Biology (AREA)
- Bioinformatics & Computational Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses an adenocarcinoma pathological image analysis method based on prior perception and multitask learning, which comprises the following steps: for a newly collected adenocarcinoma pathological image, extracting an image block, preprocessing the extracted image block, and sending the image block into a trained adenocarcinoma pathological image analysis model constructed based on a multi-task learning method, wherein the adenocarcinoma pathological image analysis model comprises two parallel branches: the gland structure segmentation branch and the benign and malignant automatic grading branch are used for simultaneously obtaining a gland structure prediction result and a benign and malignant grading result; the segmentation result of the gland structure segmentation branch is used as prior information to be transmitted to the benign and malignant automatic grading branch, and the benign and malignant grading branch is restrained and guided to grade the adenocarcinoma pathological image after paying attention to the semantic content of the gland structure. The invention can realize parallel glandular tissue segmentation and automatic grading, reduces the physical, energy and time cost investment of manual diagnosis of a pathologist, and improves the accuracy of automatic diagnosis.
Description
Technical Field
The invention relates to the technical field of artificial intelligence deep learning methods and medical pathology, in particular to an adenocarcinoma pathology image analysis method, device and system based on prior perception and multitask learning.
Background
With the development of computer vision technology, more and more advanced image processing algorithms are applied to the field of medical images. In the field of digital pathology, deep learning plays an increasingly important role due to its excellent performance in image classification, tissue segmentation, cell detection, and the like. In general, the deep learning method simply implements the processing of image pixels to make the final label prediction. Although it achieves manifestations even exceeding human levels, its application to the clinic lacks explanation of pathological levels. Therefore, this requires a priori knowledge of the pathology to be considered to guide the reasoning of the model.
Adenocarcinoma is the most common cancer. In pathological diagnosis of prostate cancer, lung adenocarcinoma and colon adenocarcinoma, the differentiation degree of glands is an important index for determining the grade of adenocarcinoma and an important standard for manual diagnosis and grading of pathologists. Therefore, the study of adenocarcinomas often relies on an accurate description of the gland. Therefore, a method is needed to incorporate the important index of the gland differentiation degree into the corresponding adenocarcinoma pathological image analysis process so as to obtain higher automatic diagnosis accuracy and more precise diagnosis results.
Disclosure of Invention
The invention aims to provide a method, a device and a system for analyzing adenocarcinoma pathological images based on prior perception and multitask learning, wherein the prior perception and multitask learning method is used for automatically analyzing the adenocarcinoma pathological images, so that parallel glandular tissue segmentation and automatic grading are realized, and the physical, energy and time cost investment of manual diagnosis of pathologists is reduced; meanwhile, in the invention, the gland segmentation result is used as prior information for restricting the automatic grading task prediction, thereby improving the accuracy of automatic diagnosis. Moreover, the priori information provided by the invention is based on the clinical pathological diagnosis gold standard of the adenocarcinoma, and the interpretability of the application to clinical pathology is improved.
In order to achieve the above object, with reference to fig. 1, the present invention provides an adenocarcinoma pathology image analysis method based on prior sensing and multitask learning, where the image analysis method includes:
for a newly collected adenocarcinoma pathological image, extracting an image block, preprocessing the extracted image block, and sending the image block into a trained adenocarcinoma pathological image analysis model constructed based on a multi-task learning method, wherein the adenocarcinoma pathological image analysis model comprises two parallel branches: the gland structure segmentation branch and the benign and malignant automatic grading branch are used for simultaneously obtaining a gland structure prediction result and a benign and malignant grading result;
the segmentation result of the gland structure segmentation branch is used as prior information to be transmitted to the benign and malignant automatic grading branch, and the benign and malignant grading branch is restrained and guided to grade the adenocarcinoma pathology image after paying attention to the semantic content of the gland structure.
As a preferred example, the process of constructing the adenocarcinoma pathology image analysis model includes the following steps:
s1, collecting a certain amount of digital adenocarcinoma pathological images, and manually marking benign and malignant tissue areas and corresponding glandular tissue structures on each adenocarcinoma pathological image;
s2, extracting image blocks aiming at benign and malignant tissue areas of each adenocarcinoma pathological image, and generating a certain amount of sample data after preprocessing;
s3, constructing a multitask CNN model based on prior perception, wherein the multitask CNN model comprises two parallel branches: the gland structure segmentation branch and the benign and malignant automatic grading branch adopt the sample data to train the two branches of the multitask CNN model simultaneously;
s4, the prediction probability of the gland structure segmentation branch is used as prior information to be transmitted to the benign and malignant hierarchical branch, the benign and malignant hierarchical branch is restrained and guided to pay attention to the gland structure semantic content, and optimization of the multitask CNN model is completed.
As a preferred example, in step S2, the extracted image block is preprocessed by using a color migration method and a data enhancement method.
As a preferred example, the result of the segmentation of the glandular structure segmentation branch includes a result of a predicted probability of the glandular structure.
As a preferred example, the adenocarcinoma pathology image analysis model comprises an image receiving unit, a feature extraction unit, a glandular structure segmentation branch and a benign and malignant automatic grading branch;
the image receiving unit is used for receiving the preprocessed newly acquired adenocarcinoma pathological image and transmitting the received image to the feature extraction unit;
the feature extraction unit is constructed based on a ResNet50 backbone network and is used for extracting high-dimensional feature information from an adenocarcinoma pathological image and simultaneously sending the extracted high-dimensional feature information to a gland structure segmentation branch and a benign and malignant automatic grading branch;
the gland structure segmentation branch processes high-dimensional feature information to generate a plurality of corresponding feature maps, the feature maps are subjected to convolution processing through a 3 x 3 convolution layer to obtain small-scale segmentation results which are used as prior information and transmitted to the benign and malignant automatic segmentation branch, and meanwhile, a bilinear interpolation method is adopted to process the small-scale segmentation results to obtain final segmentation results which are consistent with the size of an input image;
the benign and malignant automatic grading branch carries out feature map extraction processing on high-dimensional feature information, meanwhile, gland structure semantic content is extracted from prior information sent by the gland structure segmentation branch, after a basic product is taken, the gland structure semantic content is converted into 2048 vectors through convolution overall pooling, and then grading results are obtained through full-connection layer processing.
The invention also provides an adenocarcinoma pathology image analysis device based on prior perception and multitask learning, which comprises an adenocarcinoma pathology image preprocessing module, an adenocarcinoma pathology image analysis model and an adenocarcinoma pathology image analysis model construction module;
the adenocarcinoma pathological image preprocessing module is used for performing corresponding preprocessing on a newly acquired adenocarcinoma pathological image, and the preprocessing comprises image block extraction and image block enhancement processing;
the adenocarcinoma pathology image analysis model comprises two branches in parallel: the method comprises a gland structure segmentation branch and a benign-malignant automatic grading branch, wherein the segmentation result of the gland structure segmentation branch is transmitted to the benign-malignant automatic grading branch as prior information, and the benign-malignant grading branch is restrained and guided to grade an adenocarcinoma pathological image after paying attention to the semantic content of the gland structure;
the adenocarcinoma pathology image analysis model construction module is used for constructing an adenocarcinoma pathology image analysis model based on a multitask learning method, and comprises the following steps:
the system comprises a sample data generating unit, a data processing unit and a data processing unit, wherein the sample data generating unit is used for acquiring a certain amount of digital adenocarcinoma pathological images, manually marking benign and malignant tissue areas and corresponding glandular tissue structures on each adenocarcinoma pathological image, extracting image blocks aiming at the benign and malignant tissue areas of each adenocarcinoma pathological image, and generating a certain amount of sample data after preprocessing;
a model construction unit, configured to construct a multitask CNN model based on prior sensing, where the multitask CNN model includes two parallel branches: the gland structure segmentation branch and the benign and malignant automatic grading branch adopt the sample data to train the two branches of the multitask CNN model simultaneously;
and the model optimization unit is used for transmitting the prediction probability of the gland structure segmentation branch as prior information to the benign and malignant hierarchical branches, restricting and guiding the benign and malignant hierarchical branches to pay attention to the semantic content of the gland structure, and completing the optimization of the multitask CNN model.
The invention also provides an adenocarcinoma pathology image analysis system based on prior perception and multitask learning, which comprises a processor and a memory which are connected with each other;
the memory has stored therein a computer-implemented program that is executed by the processor to perform the method of adenocarcinoma pathology image analysis based on a priori perception and multitask learning as previously described.
Compared with the prior art, the technical scheme of the invention has the following remarkable beneficial effects:
(1) the implementation of the method can reduce the physical, energy and time cost investment of the manual diagnosis of the pathologist.
(2) The realization of the method has higher automatic diagnosis accuracy.
(3) The method can simultaneously predict the glandular structure and the benign and malignant diagnosis result of the image.
(4) The glandular structure differentiation is an important basis for adenocarcinoma pathological diagnosis, and the prior information utilizes the effective glandular structure, which is highly consistent with the clinical pathological diagnosis basis; so that the method of the invention has strong interpretability when applied to clinical pathology.
(5) The multitask CNN model constructed by the method has high repeatability and robustness, and the difference of different pathologists in diagnosing the same case section is greatly reduced.
It should be understood that all combinations of the foregoing concepts and additional concepts described in greater detail below can be considered as part of the inventive subject matter of this disclosure unless such concepts are mutually inconsistent. In addition, all combinations of claimed subject matter are considered a part of the presently disclosed subject matter.
The foregoing and other aspects, embodiments and features of the present teachings can be more fully understood from the following description taken in conjunction with the accompanying drawings. Additional aspects of the present invention, such as features and/or advantages of exemplary embodiments, will be apparent from the description which follows, or may be learned by practice of specific embodiments in accordance with the teachings of the present invention.
Drawings
The drawings are not intended to be drawn to scale. In the drawings, each identical or nearly identical component that is illustrated in various figures may be represented by a like numeral. For purposes of clarity, not every component may be labeled in every drawing. Embodiments of various aspects of the present invention will now be described, by way of example, with reference to the accompanying drawings, in which:
fig. 1 is a flow chart of the adenocarcinoma pathology image analysis method based on prior perception and multitask learning of the present invention.
Fig. 2 is a schematic structural diagram of an adenocarcinoma pathology image analysis device based on prior perception and multitask learning according to the present invention.
Fig. 3 is a diagram illustrating the comparison result of whether the prior information is adopted to influence the benign and malignant classification of the automatic prediction image.
FIG. 4 is a schematic image of a benign and malignant tumor of adenocarcinoma.
Detailed Description
In order to better understand the technical content of the present invention, specific embodiments are described below with reference to the accompanying drawings.
Detailed description of the preferred embodiment
With reference to fig. 1, the present invention provides an adenocarcinoma pathology image analysis method based on prior sensing and multitask learning, where the image analysis method includes:
for a newly collected adenocarcinoma pathological image, extracting an image block, preprocessing the extracted image block, and sending the image block into a trained adenocarcinoma pathological image analysis model constructed based on a multi-task learning method, wherein the adenocarcinoma pathological image analysis model comprises two parallel branches: and (3) dividing the glandular structure and automatically grading the benign and malignant branches to obtain the glandular structure prediction result and the benign and malignant grading result.
The segmentation result of the gland structure segmentation branch is used as prior information to be transmitted to the benign and malignant automatic grading branch, and the benign and malignant grading branch is restrained and guided to grade the adenocarcinoma pathology image after paying attention to the semantic content of the gland structure.
The invention carries out parallel glandular structure segmentation and benign and malignant grading automatic prediction aiming at adenocarcinoma pathological images including colorectal adenocarcinoma H & E pathological images, and restrains the benign and malignant grading prediction of the images by utilizing the prediction probability result of the glandular structure, thereby not only reducing the physical, energy and time cost input of the pathological doctor in manual diagnosis, but also improving the accuracy of automatic diagnosis; in addition, the gold standard based on clinical pathological diagnosis reuses prior information, so that the interpretability of the gold standard applied to clinical pathology is improved.
In some examples, the process of constructing the adenocarcinoma pathology image analysis model includes the following steps:
s1, a quantity of digitized adenocarcinoma pathology images is acquired, and benign and malignant tissue regions and corresponding glandular tissue structures are manually marked on each adenocarcinoma pathology image. For example, benign and malignant tissue regions and corresponding glandular tissue structures are manually marked by a pathologist on digitized colorectal adenocarcinoma hepatoxylin & Eosin (H & E) pathology images.
And S2, extracting image blocks aiming at benign and malignant tissue areas of each adenocarcinoma pathological image, and generating a certain amount of sample data after preprocessing. Preferably, the extracted image block is preprocessed by image enhancement and the like by adopting a color migration method and a data enhancement method, so that the image is further segmented and graded.
S3, constructing a multitask CNN model based on prior perception, wherein the multitask CNN model comprises two parallel branches: and (3) performing glandular structure segmentation and benign and malignant automatic grading, and simultaneously training two branches of the multitask CNN model by adopting the sample data.
S4, the prediction probability of the gland structure segmentation branch is used as prior information to be transmitted to the benign and malignant hierarchical branch, the benign and malignant hierarchical branch is restrained and guided to pay attention to the gland structure semantic content, and optimization of the multitask CNN model is completed. In some examples, the segmented outcome of the glandular structure segmentation branch includes a predicted probabilistic outcome of the glandular structure.
Referring to fig. 2, the adenocarcinoma pathology image analysis model includes an image receiving unit, a feature extraction unit, a glandular structure segmentation branch, and a benign and malignant automatic grading branch.
The image receiving unit is used for receiving the preprocessed newly acquired adenocarcinoma pathological image and transmitting the received image to the feature extraction unit.
The feature extraction unit is constructed based on a ResNet50 backbone network and is used for extracting high-dimensional feature information from an adenocarcinoma pathology image and sending the extracted high-dimensional feature information to a gland structure segmentation branch and a benign and malignant automatic grading branch at the same time.
The gland structure segmentation branch processes high-dimensional feature information to generate a plurality of corresponding feature maps, the feature maps are subjected to convolution processing through a 3 x 3 convolution layer to obtain small-scale segmentation results, the small-scale segmentation results are used as priori information and are transmitted to the benign and malignant automatic segmentation branch, and meanwhile, a bilinear interpolation method is adopted to process the small-scale segmentation results to obtain final segmentation results consistent with the size of an input image.
The benign and malignant automatic grading branch carries out feature map extraction processing on high-dimensional feature information, meanwhile, gland structure semantic content is extracted from prior information sent by the gland structure segmentation branch, after a basic product is taken, the gland structure semantic content is converted into 2048 vectors through convolution overall pooling, and then grading results are obtained through full-connection layer processing.
For a pathological image, firstly, the pathological image is sent to a ResNet50 backbone network to extract high-dimensional features, then the pathological image is respectively and simultaneously used for gland structure segmentation branches and benign and malignant hierarchical branch tasks, the gland structure prediction probability of the gland structure segmentation branches is used as prior information and is transmitted to the benign and malignant hierarchical branches to restrict the gland structure semantic content which is concerned more, and finally, an image analysis result is obtained. It should be understood that the foregoing procedure is also applicable to the training optimization process of the adenocarcinoma pathology image analysis model.
FIG. 4 is a schematic image of a benign and malignant tumor of adenocarcinoma. Fig. 3 is a diagram illustrating the comparison result of whether the prior information is adopted to influence the benign and malignant classification of the automatic prediction image. Practice proves that the multitask CNN model based on the prior information perception method has better prediction efficiency on image benign and malignant grading.
Detailed description of the invention
The invention also provides an adenocarcinoma pathology image analysis device based on prior perception and multitask learning, which comprises an adenocarcinoma pathology image preprocessing module, an adenocarcinoma pathology image analysis model and an adenocarcinoma pathology image analysis model construction module.
The adenocarcinoma pathological image preprocessing module is used for correspondingly preprocessing a newly acquired adenocarcinoma pathological image, and the preprocessing comprises image block extraction and image block enhancement. Preferably, the adenocarcinoma pathology image preprocessing module can simultaneously preprocess the sample image and the newly acquired image, so as to reduce redundant functional modules.
The adenocarcinoma pathology image analysis model comprises two branches in parallel: the method comprises a gland structure segmentation branch and a benign-malignant automatic grading branch, wherein the segmentation result of the gland structure segmentation branch is transmitted to the benign-malignant automatic grading branch as prior information, and the benign-malignant grading branch is restrained and guided to grade an adenocarcinoma pathological image after the semantic content of the gland structure is concerned.
The adenocarcinoma pathology image analysis model construction module is used for constructing an adenocarcinoma pathology image analysis model based on a multitask learning method, and comprises the following steps:
(1) the sample data generating unit is used for acquiring a certain amount of digital adenocarcinoma pathological images, manually marking benign and malignant tissue areas and corresponding glandular tissue structures on each adenocarcinoma pathological image, extracting image blocks according to the benign and malignant tissue areas of each adenocarcinoma pathological image, and generating a certain amount of sample data after preprocessing.
(2) A model construction unit, configured to construct a multitask CNN model based on prior sensing, where the multitask CNN model includes two parallel branches: and (3) performing glandular structure segmentation and benign and malignant automatic grading, and simultaneously training two branches of the multitask CNN model by adopting the sample data.
(3) And the model optimization unit is used for transmitting the prediction probability of the gland structure segmentation branch as prior information to the benign and malignant hierarchical branches, restricting and guiding the benign and malignant hierarchical branches to pay attention to the semantic content of the gland structure, and completing the optimization of the multitask CNN model.
Detailed description of the preferred embodiment
The invention also refers to an adenocarcinoma pathology image analysis system based on prior perception and multitask learning, said image analysis system comprising a processor and a memory connected to each other.
The memory has stored therein a computer-implemented program that is executed by the processor to perform the method of adenocarcinoma pathology image analysis based on a priori perception and multitask learning as previously described.
In this disclosure, aspects of the present invention are described with reference to the accompanying drawings, in which a number of illustrative embodiments are shown. Embodiments of the present disclosure are not necessarily defined to include all aspects of the invention. It should be appreciated that the various concepts and embodiments described above, as well as those described in greater detail below, may be implemented in any of numerous ways, as the disclosed concepts and embodiments are not limited to any one implementation. In addition, some aspects of the present disclosure may be used alone, or in any suitable combination with other aspects of the present disclosure.
Although the present invention has been described with reference to the preferred embodiments, it is not intended to be limited thereto. Those skilled in the art can make various changes and modifications without departing from the spirit and scope of the invention. Therefore, the protection scope of the present invention should be determined by the appended claims.
Claims (7)
1. An adenocarcinoma pathology image analysis method based on prior perception and multitask learning is characterized in that the image analysis method comprises the following steps:
for a newly collected adenocarcinoma pathological image, extracting an image block, preprocessing the extracted image block, and sending the image block into a trained adenocarcinoma pathological image analysis model constructed based on a multi-task learning method, wherein the adenocarcinoma pathological image analysis model comprises two parallel branches: the gland structure segmentation branch and the benign and malignant automatic grading branch are used for simultaneously obtaining a gland structure prediction result and a benign and malignant grading result;
the segmentation result of the gland structure segmentation branch is used as prior information to be transmitted to the benign and malignant automatic grading branch, and the benign and malignant grading branch is restrained and guided to grade the adenocarcinoma pathology image after paying attention to the semantic content of the gland structure.
2. The method for analyzing the adenocarcinoma pathology image based on prior perception and multitask learning according to claim 1, wherein the process for constructing the adenocarcinoma pathology image analysis model comprises the following steps:
s1, collecting a certain amount of digital adenocarcinoma pathological images, and manually marking benign and malignant tissue areas and corresponding glandular tissue structures on each adenocarcinoma pathological image;
s2, extracting image blocks aiming at benign and malignant tissue areas of each adenocarcinoma pathological image, and generating a certain amount of sample data after preprocessing;
s3, constructing a multitask CNN model based on prior perception, wherein the multitask CNN model comprises two parallel branches: the gland structure segmentation branch and the benign and malignant automatic grading branch adopt the sample data to train the two branches of the multitask CNN model simultaneously;
s4, the prediction probability of the gland structure segmentation branch is used as prior information to be transmitted to the benign and malignant hierarchical branch, the benign and malignant hierarchical branch is restrained and guided to pay attention to the gland structure semantic content, and optimization of the multitask CNN model is completed.
3. The method for analyzing the adenocarcinoma pathology image based on prior perception and multitask learning according to claim 2, wherein in step S2, the extracted image blocks are preprocessed by using a color migration method and a data enhancement method.
4. The method for analyzing pathological image of adenocarcinoma based on prior perception and multitask learning according to any one of claims 1-3, characterized in that the segmentation result of said glandular structure segmentation branch comprises the predicted probability result of glandular structure.
5. The adenocarcinoma pathology image analysis method based on prior perception and multitask learning according to any one of claims 1-3, wherein the adenocarcinoma pathology image analysis model comprises an image receiving unit, a feature extraction unit, a glandular structure segmentation branch and a benign and malignant automatic grading branch;
the image receiving unit is used for receiving the preprocessed newly acquired adenocarcinoma pathological image and transmitting the received image to the feature extraction unit;
the feature extraction unit is constructed based on a ResNet50 backbone network and is used for extracting high-dimensional feature information from an adenocarcinoma pathological image and simultaneously sending the extracted high-dimensional feature information to a gland structure segmentation branch and a benign and malignant automatic grading branch;
the gland structure segmentation branch processes high-dimensional feature information to generate a plurality of corresponding feature maps, the feature maps are subjected to convolution processing through a 3 x 3 convolution layer to obtain small-scale segmentation results which are used as prior information and transmitted to the benign and malignant automatic segmentation branch, and meanwhile, a bilinear interpolation method is adopted to process the small-scale segmentation results to obtain final segmentation results which are consistent with the size of an input image;
the benign and malignant automatic grading branch carries out feature map extraction processing on high-dimensional feature information, meanwhile, gland structure semantic content is extracted from prior information sent by the gland structure segmentation branch, after a basic product is taken, the gland structure semantic content is converted into 2048 vectors through convolution overall pooling, and then grading results are obtained through full-connection layer processing.
6. An adenocarcinoma pathology image analysis device based on prior perception and multitask learning is characterized by comprising an adenocarcinoma pathology image preprocessing module, an adenocarcinoma pathology image analysis model and an adenocarcinoma pathology image analysis model building module;
the adenocarcinoma pathological image preprocessing module is used for performing corresponding preprocessing on a newly acquired adenocarcinoma pathological image, and the preprocessing comprises image block extraction and image block enhancement processing;
the adenocarcinoma pathology image analysis model comprises two branches in parallel: the method comprises a gland structure segmentation branch and a benign-malignant automatic grading branch, wherein the segmentation result of the gland structure segmentation branch is transmitted to the benign-malignant automatic grading branch as prior information, and the benign-malignant grading branch is restrained and guided to grade an adenocarcinoma pathological image after paying attention to the semantic content of the gland structure;
the adenocarcinoma pathology image analysis model construction module is used for constructing an adenocarcinoma pathology image analysis model based on a multitask learning method, and comprises the following steps:
the system comprises a sample data generating unit, a data processing unit and a data processing unit, wherein the sample data generating unit is used for acquiring a certain amount of digital adenocarcinoma pathological images, manually marking benign and malignant tissue areas and corresponding glandular tissue structures on each adenocarcinoma pathological image, extracting image blocks aiming at the benign and malignant tissue areas of each adenocarcinoma pathological image, and generating a certain amount of sample data after preprocessing;
a model construction unit, configured to construct a multitask CNN model based on prior sensing, where the multitask CNN model includes two parallel branches: the gland structure segmentation branch and the benign and malignant automatic grading branch adopt the sample data to train the two branches of the multitask CNN model simultaneously;
and the model optimization unit is used for transmitting the prediction probability of the gland structure segmentation branch as prior information to the benign and malignant hierarchical branches, restricting and guiding the benign and malignant hierarchical branches to pay attention to the semantic content of the gland structure, and completing the optimization of the multitask CNN model.
7. An adenocarcinoma pathology image analysis system based on prior perception and multitask learning, characterized in that the image analysis system comprises a processor and a memory connected to each other;
the memory stores a computer-implemented program, and the processor executes the computer-implemented program stored in the memory to perform the adenocarcinoma pathology image analysis method based on prior perception and multitask learning as claimed in any one of claims 1-5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010405794.5A CN111554384A (en) | 2020-05-14 | 2020-05-14 | Adenocarcinoma pathological image analysis method based on prior perception and multitask learning |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010405794.5A CN111554384A (en) | 2020-05-14 | 2020-05-14 | Adenocarcinoma pathological image analysis method based on prior perception and multitask learning |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111554384A true CN111554384A (en) | 2020-08-18 |
Family
ID=72004720
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010405794.5A Pending CN111554384A (en) | 2020-05-14 | 2020-05-14 | Adenocarcinoma pathological image analysis method based on prior perception and multitask learning |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111554384A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112435243A (en) * | 2020-11-26 | 2021-03-02 | 山东第一医科大学附属省立医院(山东省立医院) | Automatic analysis system and method for full-slice digital pathological image |
| CN112767355A (en) * | 2021-01-20 | 2021-05-07 | 北京小白世纪网络科技有限公司 | Method and device for constructing thyroid nodule Tirads grading automatic identification model |
-
2020
- 2020-05-14 CN CN202010405794.5A patent/CN111554384A/en active Pending
Non-Patent Citations (1)
| Title |
|---|
| CHAOYANG YAN等: ""PRIOR-AWARE CNN WITH MULTI-TASK LEARNING FOR COLON IMAGES ANALYSIS"" * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112435243A (en) * | 2020-11-26 | 2021-03-02 | 山东第一医科大学附属省立医院(山东省立医院) | Automatic analysis system and method for full-slice digital pathological image |
| CN112767355A (en) * | 2021-01-20 | 2021-05-07 | 北京小白世纪网络科技有限公司 | Method and device for constructing thyroid nodule Tirads grading automatic identification model |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111696094B (en) | Immunohistochemical PD-L1 membrane staining pathological section image processing method, device and equipment | |
| CN110428432B (en) | Deep neural network algorithm for automatically segmenting colon gland image | |
| CN110853022B (en) | Pathological section image processing method, device and system and storage medium | |
| CN111598875A (en) | Method, system and device for building thyroid nodule automatic detection model | |
| CN111275686B (en) | Method and device for generating medical image data for artificial neural network training | |
| CN108520215B (en) | Single-sample face recognition method based on multi-scale joint feature encoder | |
| CN114862838A (en) | Unsupervised learning-based defect detection method and equipment | |
| CN111932577B (en) | Text detection method, electronic device and computer readable medium | |
| CN112990214A (en) | Medical image feature recognition prediction model | |
| CN111554384A (en) | Adenocarcinoma pathological image analysis method based on prior perception and multitask learning | |
| CN114445356A (en) | Multi-resolution-based full-field pathological section image tumor rapid positioning method | |
| CN114299082A (en) | New coronary pneumonia CT image segmentation method, device and storage medium | |
| CN114037699B (en) | Pathological image classification method, equipment, system and storage medium | |
| CN115909006A (en) | Mammary tissue image classification method and system based on convolution Transformer | |
| CN114758137A (en) | Ultrasonic image segmentation method and device and computer readable storage medium | |
| CN113177554B (en) | Thyroid nodule identification and segmentation method, system, storage medium and equipment | |
| CN113764101B (en) | Novel auxiliary chemotherapy multi-mode ultrasonic diagnosis system for breast cancer based on CNN | |
| CN113706562A (en) | Image segmentation method, device and system and cell segmentation method | |
| CN113822846A (en) | Method, apparatus, device and medium for determining region of interest in medical image | |
| CN117437423A (en) | Weak supervision medical image segmentation method and device based on SAM collaborative learning and cross-layer feature aggregation enhancement | |
| CN113160175A (en) | Tumor lymphatic vessel infiltration detection method based on cascade network | |
| CN116778164A (en) | Semantic segmentation method for improving deep V < 3+ > network based on multi-scale structure | |
| Alisha et al. | Cervical Cell Nuclei Segmentation On Pap Smear Images Using Deep Learning Technique | |
| Samudrala et al. | Semantic Segmentation in Medical Image Based on Hybrid Dlinknet and Unet | |
| CN113763315B (en) | Slide image information acquisition method, device, equipment and medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20200818 |