CN113591919A

CN113591919A - AI-based analysis method and system for prognosis of postoperative recurrence of early hepatocellular carcinoma

Info

Publication number: CN113591919A
Application number: CN202110731127.0A
Authority: CN
Inventors: 史颖弘; 瞿伟峰; 田孟鑫; 刘卫仁; 唐政; 钱琨; 王治勋; 邹昊; 郭玉成; 裘静韬
Original assignee: Zhongshan Hospital Fudan University
Current assignee: Zhongshan Hospital Fudan University
Priority date: 2021-06-29
Filing date: 2021-06-29
Publication date: 2021-11-02
Anticipated expiration: 2041-06-29
Also published as: CN113591919B

Abstract

The invention relates to the technical field of prognosis intelligent algorithm, in particular to an analysis method and a system for prognosis of early hepatocellular carcinoma postoperative recurrence based on Artificial Intelligence (AI); predicting postoperative recurrence risk and prognosis of early hepatocellular carcinoma by using complete digital hematoxylin-eosin (HE) staining histopathological section, which comprises the following steps: acquiring a full-field digital slide (WS I) image of a specimen to be analyzed; extracting a foreground region of interest (ROI) from the WS I image; the invention identifies different areas of the digital pathological image through deep learning. The postoperative recurrence risk and prognosis of early hepatocellular carcinoma are effectively analyzed directly according to the HE stained section, and the accuracy is very high while various different cell areas are identified, so that the method has great guiding significance for the recurrence of patients.

Description

AI-based analysis method and system for prognosis of postoperative recurrence of early hepatocellular carcinoma

Technical Field

The invention relates to the technical field of prognosis intelligent algorithms, in particular to an AI-based analysis method and system for prognosis of postoperative recurrence of early hepatocellular carcinoma.

Background

Liver cancer is the sixth most common malignancy worldwide, with Hepatocellular carcinoma (HCC) accounting for about 85% of primary liver cancer. Early stage liver cancer is the best indication for radical resection. The 5-year post-operative survival rate of patients in Barcelona client Cancer stage (BCLC stage) 0-A can reach 70%, but half of patients suffer relapse. Therefore, it is of great importance to overcome the difficult problem of recurrence of early liver cancer. The current HCC recurrence prediction model relates to gross morphology, biochemical indexes, gene characteristics and the like of tumors, and the research on the histological level is less.

In recent years, the occurrence of Artificial Intelligence (AI) has increased the reliability and convenience of prognosis models and greatly reduced manpower and material resources. The pathological picture is used as a gold standard for pathological diagnosis and has very important application in clinic and scientific research. With the continuous progress of medical imaging technology, pathological images are digitized, namely, fragmented images are seamlessly joined through a computer to obtain a pathological full-field slice (WSI), so that the problems of time and labor waste, easiness in damage and fading of slides and the like in the traditional microscopic examination are solved. Therefore, it is an urgent requirement to adopt an efficient medical diagnosis algorithm to assist doctors in identifying tumor tissue characteristics and judging patient prognosis.

At present, relevant reports AI can be applied to the identification of pathological sections and the prediction of the postoperative survival rate of patients in HCC. The defects are as follows:

1) the existing algorithms can only identify cancer regions, normal liver tissue, necrotic regions and fibrous tissues, while the important regions of the assembled organs and lymphocytes in the tumor microenvironment are not identified, thereby reducing the model performance.

2) The existing model focuses on the prediction of the overall survival rate, and has insufficient prompting significance on postoperative recurrence risk.

3) Existing algorithms fail to effectively normalize pathological sections from different institutions (e.g., for section validation in The Cancer Genome Atlas, TCGA database), and thus affect model generalization.

4) The specificity and the sensitivity of the existing model need to be improved.

Disclosure of Invention

Solves the technical problem

Aiming at the defects in the prior art, the invention provides an AI-based analysis method and system for prognosis of postoperative recurrence of early hepatocellular carcinoma, which can identify different regions of a digital pathological image through deep learning. The postoperative recurrence risk and prognosis of early hepatocellular carcinoma are effectively analyzed directly according to the HE stained section, so that the prediction precision is improved, and simultaneously, the manpower and material resources are greatly reduced.

Technical scheme

In order to achieve the purpose, the invention is realized by the following technical scheme:

the invention provides an AI-based analysis method for prognosis of postoperative recurrence of early hepatocellular carcinoma, which predicts postoperative recurrence risk and prognosis of early hepatocellular carcinoma by using a complete digital HE stained histopathological section, and comprises the following steps:

acquiring a WSI image of a sample to be analyzed;

and (2) carrying out 6-class liver cancer characteristic tissue labeling on the WSI image by adopting APSP software, wherein the labeling comprises the following steps: tumor area, normal liver tissue, fibrotic area, tract of the assembled organ, lymphatic area and hemorrhage/necrosis area;

extracting a foreground ROI from the WSI image;

cutting the WSI according to the foreground ROI to obtain small pictures, and performing dyeing normalization and data enhancement processing on all the small pictures;

inputting the processed data into a deep learning classification model for calculation and identification, and training to obtain an optimal model for whole image prediction of all WSI images, and performing post-processing optimization on the obtained heat image;

selecting a certain amount of image blocks with the highest prediction probability to perform feature extraction, and meanwhile, counting the ratio of each type as a supplementary feature;

integrating all the extracted features with clinical features of the patient;

screening the integrated features by using Lasso Cox analysis, multiplying the screened features by the weight of the features to obtain clinical scores CS, and simply calculating tissue features to obtain histology scores HS;

the HS and CS were divided into two high-risk and low-risk groups, KM curves were plotted and single-factor, multi-factor Cox analysis was performed for analysis of patient relapse and prognostic variation in the high-and low-risk groups.

In another aspect, the present invention provides an AI-based analysis system for prognosis of postoperative recurrence of early hepatocellular carcinoma, comprising: the data acquisition module is used for acquiring a WSI image of a sample to be analyzed; the data labeling module is used for training the recognition of the liver cancer 6 large-class tissue structure; the data processing module is used for splitting, preprocessing and unifying difference of the WSI image; the model training module is used for constructing a deep learning classification model with the best prediction effect; the operation processing module is used for image data processing, flow execution, data calculation feature integration, screening and index establishment of the WSI image in the analysis step of recurrence prognosis; and the result analysis image generation module is used for automatically drawing the corresponding KM curve visualization index effect and the corresponding parameterized index significance degree according to the data processing result of the operation processing module.

Further, in the step of extracting the foreground ROI, calculating appropriate threshold values for different WSI images by adopting an Otsu threshold value method so as to distinguish the background and the target area of the WSI images; the specific steps of extracting the foreground ROI include:

converting the original color image into a gray image;

determining a threshold value of binary image segmentation by adopting an Otsu threshold value method, and segmenting the gray level image to obtain a mask;

removing a large number of White holes in the binary image by using a morphological White-tophat method to obtain res;

mask-res gets the ROI area of the last whole WSI.

And further, cutting the WSI according to the foreground ROI to obtain small images, and calculating the mean value and the standard deviation of the whole WSI image for the image of which the pixel size is 5000 multiplied by 5000 pixels in each WSI image in the dyeing normalization processing of all the small images, wherein the mean value and the standard deviation are used as the mean value and the standard deviation of all the slices of the WSI image, so that the influence of the difference inside each patch on the dyeing normalization result is reduced. The dyeing normalization treatment specific process comprises the following steps:

reading a WSI image, and cutting a large slice of 5000 multiplied by 5000 pixels according to the foreground ROI to represent the WSI image;

transforming the target slice, the standard picture and the large picture representing the WSI image from an RGB space to an LAB space;

the original slice is then changed from the LAB space back to the RGB space.

Further, the deep learning classification model can identify six major tissue cell regions of tumor tissue, normal liver tissue, fibrosis region, assembler region, lymph region and necrosis region in the WSI image.

Further, the data enhancement processing performs data enhancement on the data with a small number of slices, and specific parameters are as follows: clockwise rotation range [0 degree, 30 degrees ], random horizontal or vertical offset 0.2, cutting range [0,0.2], random horizontal or vertical turning and filling mode is 'reflex'. The data with a large number of slices is subjected to undersampling processing, so that the number of samples of 6 classes of the WSI image is equivalent.

Further, the training set data was loaded into the inclusion-V3 model, with detailed parameters: batch _ size 32, leaving _ rate 1e-5, optimizer: and Adam and epoch are 50, and selecting the model with the best verification set effect as the final model. All 416 WSIs were sliced and predicted, and each WSI generated a heat map with various thresholds of 0.5.

Further, morphological post-processing was performed on each category in the WSI full predicted heat map. Since the lymphocyte accumulation region and the necrotic region are easily fragmented and the regions are not large, a morphological method of filling a cavity and deleting an excessively small region is used; since there is a cavity (normal, no filling) in the region of the junction, the fibrosis, and the tumor, a morphological method of deleting an excessively small region by closed-loop operation is used. Adding the processed class masks according to the area size, the wrapping relation and the importance degree, wherein the sequence is as follows: lymphatic, fibrotic, tumor tissue, necrotic areas, and heat maps obtained after morphological processing.

The parameters of the three morphological methods are as follows:

(1) filling the cavity: remove _ small _ hold in the sketch packet, area _ threshold (area threshold, less than this area for filling) is 64, and connectivity is 1;

(2) and (3) closed operation: closing, kernel ═ disk (1) in the skeleton packet;

(3) deleting the undersized area: remove _ small _ objects, min _ size (area threshold, less than this area for deletion) in the skeleton packet is 4, connectivity is 1;

further, each WSI image selects 10 image blocks with the highest prediction probability, the image group characteristics are extracted, 107 histological characteristics are obtained from each slice, the mean value, the standard deviation, the median and the decile of the characteristics of 10 slices of each type are calculated to serve as each type of characteristics of the WSI image, and meanwhile, the ratio of each type is counted to serve as supplementary characteristics.

Further, primarily screening the integrated and standardized pathological image characteristics and clinical characteristics by adopting a Lasso regression model, selecting an optimal lambda value by using a cross validation strategy with consistency indexes as evaluation indexes, obtaining characteristics with non-zero weight according to the lambda value, obtaining a histological score HS by only multiplying the screened pathological image characteristics by the characteristic weight, and simultaneously adding the clinical characteristics to obtain a clinical score CS. (Lasso Cox uses the glmnet package in C, input is all features, output is time to relapse and status parameters: alpha ═ 1, nfolds ═ 10, type

Advantageous effects

Compared with the known public technology, the technical scheme provided by the invention has the following beneficial effects:

the method effectively preprocesses the WSI image of an analysis sample to present typical tissue areas of 6 liver cancers, inputs the characteristics extracted from the image into a deep learning classification model which is trained, identifies different areas of a digital pathological image through deep learning, obtains histological score HS and clinical score CS by performing characteristic integration and processing on a prediction result, divides the HS and CS into two groups of high-risk and low-risk to draw KM curves, realizes effective analysis on early postoperative hepatocellular carcinoma recurrence risk and prognosis by observing the KM curves directly according to HE staining slices, improves prediction accuracy and greatly reduces manpower and material resources. The six classification models adopted by the pathological image recognition part are one of the AI recognition systems of the hepatocellular carcinoma with the most categories at present. The accuracy is very high while a plurality of different cell areas are identified, and the method has great guiding significance for the relapse of the patient.

Drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below. It is obvious that the drawings in the following description are only some embodiments of the invention, and that for a person skilled in the art, other drawings can be derived from them without inventive effort.

FIG. 1 is a flow chart of the method of the present invention for the analysis of prognosis of postoperative recurrence;

FIG. 2 is a flow chart of the foreground ROI extraction step of the present invention;

FIG. 3 is a schematic flow chart of an embodiment of the present invention;

FIG. 4 is a block diagram of an AI identification process of the present invention;

FIG. 5 is a schematic diagram of HCC six-class tissue identification according to the present invention;

fig. 6 is a KM graph of survival analysis for high and low risk groups according to the present invention.

Detailed Description

In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention. It is to be understood that the embodiments described are only a few embodiments of the present invention, and not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

The present invention will be further described with reference to the following examples.

Example (b):

the invention provides an AI-based analysis method for prognosis of postoperative recurrence of early hepatocellular carcinoma, which predicts postoperative recurrence risk and prognosis of early hepatocellular carcinoma by using a complete digital HE stained histopathological section, and specifically comprises the following steps:

s10, acquiring a WSI image of a sample to be analyzed, wherein the sample is all HCC samples; then selecting a small amount of WSI, and carrying out APSP software on 6 liver cancer tissues: tumor area, paracancerous normal liver tissue, fibrotic area, region of the assembled organ, region of lymphocyte accumulation, hemorrhage/necrosis area are labeled manually.

S20, extracting foreground ROI from the WSI image;

s30, cutting the WSI according to the foreground ROI to obtain small pictures, and performing dyeing normalization and data enhancement processing on all the small pictures;

s40, inputting the processed data into a deep learning classification model for calculation and recognition, using the trained optimal model for whole image prediction of all WSI images, and performing post-processing optimization on the obtained heat map;

s50, selecting a certain amount of image blocks with the highest prediction probability to perform feature extraction, and meanwhile, counting the ratio of each type as a supplementary feature;

s60, integrating all the extracted features with the clinical features of the patient;

s70, screening the integrated features by using Lasso Cox analysis, multiplying the screened features by the weight of the features to obtain clinical scores CS, and simply calculating tissue features to obtain histology scores HS;

s80, dividing HS and CS into two groups with high risk and low risk, drawing KM curve and carrying out single-factor and multi-factor Cox analysis to analyze the relapse and prognosis difference of patients in the high-risk group and the low-risk group.

Wherein, the preparation of the HE stained section is as follows: all liver cancer specimens were fixed with 4% neutral formalin, paraffin-embedded, sectioned at 4 μm thickness, and HE-stained. And finally, digitally scanning the section line into a TIF format file to obtain a pathology full-field section WSI.

Among them, the ROI is called a Region of Interest (Region of Interest), and in a machine vision and image processing problem, a Region to be processed is outlined from a processed image in a manner of a square, a circle, an ellipse, an irregular polygon, or the like. For pathological WSI images, there are marginal white spaces within the image that occur during production, as well as gaps between tissues, and the actual size of each WSI can reach 10^6 pixels in both length and width, which is typically beyond the processing power of modern computers. Therefore, pathological WSIs need to be cut into small pictures for classification, however, a large amount of blank background exists in pathological WSIs, and the extraction of tissue ROI from the WSIs will greatly improve the computer operation efficiency. The extraction of tissue ROI by binarization of WSI using a threshold is a more common method. Considering that in practice, the overall colors of different WSI images may show different colors due to different storage times, different dyeing methods, and the like, the method using the same fixed threshold value cannot extract a suitable ROI for each WSI. Therefore, adaptive methods will be used in this patent to calculate appropriate thresholds for different WSIs to distinguish background regions.

In the step of extracting the foreground ROI at S20, referring to FIG. 2, calculating appropriate thresholds for different WSI images by an Otsu threshold method to distinguish the background and the target region of the WSI images; the specific steps of extracting the foreground ROI include:

s21, converting the original color image into a gray image;

s22, determining a threshold value of binary image segmentation by adopting an Otsu threshold value method, and segmenting the gray-scale image to obtain a mask;

s23, removing a large number of White holes in the binary image by using a morphological White-tophat method to obtain res;

s24, mask-res, obtaining the ROI of the final whole WSI.

The Dajin threshold method is also called the maximum inter-class variance method, and is a self-adaptive threshold determination method, called the Daohu method (OTSU) for short. The algorithm separates the image into background and target portions based on the gray scale characteristics of the image. The larger the inter-class variance between the background and the object, the larger the difference of the parts constituting the image, and the smaller the difference of the parts when part of the foreground is mistaken for the background or part of the background is mistaken for the background. Thus, a segmentation that maximizes the inter-class variance means that the probability of false positives is minimized.

In step S30, the WSI is cut according to the foreground ROI to obtain small images, and in the process of performing the stain normalization on all the small images, the mean and standard deviation of the whole WSI image are calculated for the image cut by 5000 × 5000 pixels in each WSI image, and are used as the mean and standard deviation of all the slices of the WSI image, thereby reducing the influence of the difference inside each patch on the stain normalization result. The dyeing normalization treatment specific process comprises the following steps:

s31, reading the WSI image, and cutting a large slice with 5000 multiplied by 5000 pixels according to the foreground ROI to represent the WSI image;

s32, transforming the target slice, the standard picture and the large picture representing the WSI image from an RGB space to an LAB space; among them, the target slice original is "a slice that needs to be stained", the standard picture target is "a reference picture for staining", and the WSI representative slice WSI _ present is a large slice cut at S31.

The linear transformation equation for the original slice is as follows:

and

is the mean of the three channels in the LAB space,

and

is the standard deviation of the three channels in the LAB space.

And S33, changing the original slice from the LAB space to the RGB space.

In order to improve the traditional Reinhard staining normalization method, 5000 × 5000 images are cut from each WSI image, and the mean value and standard deviation of the whole WSI are calculated and used as the mean value and standard deviation of all sections of the WSI, so that the influence of the difference inside each patch on the staining normalization result is reduced. The improved dyeing normalization method considers the dyeing uniformity from the whole situation, is not influenced by small-range color difference and has stronger robustness.

Six major types of tissue cell regions, i.e., tumor tissue, normal liver tissue, fibrotic region, assembler region, lymphatic region, and necrotic region, in the WSI image can be identified by the deep learning classification model in step S40. Due to the fact that the number of slices is small, morphological characteristics are similar, and the number of blanks in the regions is large in the partial categories, diversity of training samples is increased through Data Augmentation (Data Augmentation) aiming at the regional categories, and robustness of the model is improved. The data enhancement specific parameters are as follows: clockwise rotation range [0 degree, 30 degrees ], random horizontal or vertical offset 0.2, cutting range [0,0.2], random horizontal or vertical turning and filling mode is 'reflex'. The data with a large number of slices is subjected to undersampling processing, so that the number of samples of 6 classes of the WSI image is equivalent.

In S50 in this embodiment, 10 image blocks with the highest prediction probability are selected for each WSI image, the omics features are extracted, 107 histological features are obtained for each slice, the mean, the standard deviation, the median, and the decile of the feature calculation of 10 slices of each category are used as each category of features of the WSI image, and the ratio of each category is counted as a supplementary feature.

Wherein 107 features are extracted from each graph, 428 features are extracted from each class, wherein features of classes which do not exist in the WSI pathological graph are subjected to median filling, and finally each WSI has 2568 features of six classes. And simultaneously counting 6 features of each category in total to supplement the WSI global area statistics. All the features extracted by the pathological WSI are integrated with the clinical features of the patient. And (3) taking the Time To Recurrence (TTR) as a research endpoint, primarily screening the normalized pathological image characteristics and clinical characteristics by using a Lasso regression model, taking a consistency index (C-index) as an evaluation index, and selecting an optimal lambda value by a cross-validation strategy. According to the lambda value, the characteristics with the weight not equal to zero are obtained, and the selected pathological image characteristics are multiplied by the characteristic weight to obtain a Histological Score (HS), and meanwhile, Clinical characteristics are added to obtain a Clinical Score (CS). A Cox model is used for analyzing the risk ratio (HR) and the P value of two indexes, HS and CS are subjected to surv _ cutpoint function to obtain the optimal critical value, and the optimal critical value is divided into two groups of high risk and low risk to draw a Kaplan-Meier curve. The recurrence and the prognosis difference of the patients in the high-risk group and the low-risk group are analyzed through the KM curve, so that the prediction of the course and the outcome (recovery, recurrence, deterioration, death and the like) of the future development of the diseases of the patients is realized.

Referring to fig. 3-6, an embodiment applying the method:

1. 387 early-stage liver cancer patients who performed radical hepatectomy in zhongshan hospital from 1 month to 2007 month 12 in 2006 and 160 patients who performed radical hepatectomy in zhongshan hospital from 10 months to 2015 year 1 in 2014 were collected. The inclusion criteria were: pathologically confirmed HCC; no neoadjuvant anti-tumor therapy is performed; the pre-operative liver function rating is Child-Pugh A-B level; BCLC phase 0-A. Exclusion criteria were: there are other pathological types such as intrahepatic cholangiocarcinoma (ICC) or mixed hepatocellular hepatocarcinoma (CHC); the clinical data of the previous operation treatment is insufficient; death or relapse within 1 month. We randomly divided cases into training sets (383 cases/402 WSIs) and testing sets (164 cases/174 WSIs) in a 7:3 ratio. The external validation set consisted of 147 TNM stage I patients in the cancer genomic map (TCGA) database and 154 corresponding WSIs. And then, completing the construction of a data identification and recurrence depth prediction model, namely a deep learning classification model according to an AI algorithm.

Wherein, the steps in fig. 3 are: 1. 547 patients in Zhongshan Hospital, 576 WSIs were randomly divided into a training set and a test set, 147 patients in the TCGA database, and 154 WSIs were selected as external verification sets. 2. And preprocessing, cutting identification and feature extraction are carried out on the WSI. 3. And performing Lasso analysis on the extracted features and the clinical features to obtain a histological score HS and a clinical score CS. 4. Survival analysis and Cox regression analysis

In fig. 5, 6 major liver cancer-characterized tissues illustrate: NLT normal liver tissue; PA: a sink zone; FI: a fibrotic tissue; LCA: a lymphocyte accumulation zone; TR: a tumor region; H/NA: hemorrhage/necrosis zone

AI recognition effect and model construction

The efficacy of this experiment in identifying six tissues of HCC is shown in figure 3. The overall identification accuracy of the experiment is 94.17%. We then obtained 133 pathological features by Lasso cox analysis, and, in combination with clinical features, we constructed HS and CS by the following two formulas:

x1＝"(-0.122016800369092*m0_shape_Flatness_mean)+(7.75700144152009e- 15*m0_shape_MeshVolume_mean)+(-0.104021271905972*m0_glcm_ClusterShade_val)+(- 0.00728318533363779*m0_glcm_DifferenceVariance_val)+(0.0634685972678304*m0_glcm_MCC_val) +(-0.0720479395240757*m0_gldm_GrayLevelNonUniformity_val)+(- 0.007441131169821*m0_firstorder_RobustMeanAbsoluteDeviation_median)+(0.0237957282652736*m0 _glrlm_HighGrayLevelRunEmphasis_median)+(- 0.0242529667308698*m0_glszm_LowGrayLevelZoneEmphasis_median)+(- 0.0137204628864997*m0_ngtdm_Contrast_median)+(0.0843469888719412*m0_gldm_DependenceNon Uniformity_median)+(-0.026765044562175*m0_firstorder_Median_deciles)+(- 0.0354914627269978*m0_firstorder_RootMeanSquared_deciles)+(- 0.0573236766580267*m0_glcm_DifferenceEntropy_deciles)+(2.69962382794311e- 05*m0_glcm_Imc2_deciles)+(- 0.0345841446130225*m0_glcm_JointAverage_deciles)+(0.102764310269742*m0_glszm_LargeAreaHigh GrayLevelEmphasis_deciles)+(0.0428562349015899*m0_glszm_SizeZoneNonUniformity_deciles)+(- 0.0301566371871564*m0_gldm_LargeDependenceHighGrayLevelEmphasis_deciles)+(- 0.046299665381274*m1_firstorder_90Percentile_val)+(0.000839119908059214*m1_firstorder_Maximu m_val)+(-0.0829140955441657*m1_glcm_InverseVariance_val)+(- 0.0378819905968139*m1_glrlm_GrayLevelVariance_val)+(- 0.147676581527489*m1_glrlm_ShortRunLowGrayLevelEmphasis_val)+(9.86106492097824e- 06*m1_glszm_LargeAreaLowGrayLevelEmphasis_val)+(0.033900989497138*m1_glszm_SmallAreaHig hGrayLevelEmphasis_val)+(-0.0453632949088534*m1_ngtdm_Complexity_val)+(- 0.0202076151839154*m1_gldm_SmallDependenceLowGrayLevelEmphasis_val)+(0.015696516658804* m1_firstorder_Minimum_median)+(- 0.0389092509224935*m1_glszm_SmallAreaEmphasis_median)+(0.318539806931387*m1_glszm_Small AreaLowGrayLevelEmphasis_median)+(- 0.109398600534323*m1_ngtdm_Busyness_median)+(0.0591279489202919*m1_firstorder_Energy_decil es)+(0.144189416420948*m1_firstorder_Median_deciles)+(0.00369010277505741*m1_firstorder_TotalE nergy_deciles)+(0.0383076582570865*m1_glcm_Idm_deciles)+(0.0195905164383039*m1_glrlm_RunEn tropy_deciles)+(-0.0702692563059623*m1_glszm_SmallAreaEmphasis_deciles)+(- 0.0923453799760962*m2_glcm_Imc2_mean)+(0.00342702717007133*m2_glszm_GrayLevelNonUnifor mity_mean)+(- 0.00128111192500525*m2_glszm_LargeAreaEmphasis_mean)+(0.0154366405793262*m2_glszm_SizeZ oneNonUniformity_mean)+(0.00135510273373574*m2_firstorder_Maximum_val)+(- 0.0381878146468702*m2_glcm_Imc2_val)+(- 0.00047769019818483*m2_glszm_GrayLevelVariance_val)+(- 0.00174051524257116*m2_glszm_LowGrayLevelZoneEmphasis_val)+(0.0263128989921181*m2_glszm _SizeZoneNonUniformity_val)+(0.0325931785410028*m2_gldm_DependenceVariance_val)+(0.0058891 902286527*m2_gldm_LargeDependenceLowGrayLevelEmphasis_val)+(0.000278241507637323*m2_gld m_DependenceVariance_median)+(0.0186964378939344*m2_gldm_LargeDependenceLowGrayLevelEm phasis_median)+(0.0642797873239336*m2_gldm_SmallDependenceLowGrayLevelEmphasis_median)+( 0.00463253863909167*m2_firstorder_Uniformity_deciles)+(0.0102635574852423*m2_glcm_Idn_deciles) +(0.00570419161862622*m2_glrlm_RunEntropy_deciles)+(- 0.00642658710799511*m2_glszm_LargeAreaLowGrayLevelEmphasis_deciles)+(0.0052230704895731* m2_gldm_GrayLevelNonUniformity_deciles)+(- 0.0150816804821197*m2_gldm_SmallDependenceHighGrayLevelEmphasis_deciles)+(0.0275087706706 102*m3_firstorder_Minimum_mean)+(- 0.0727078544702491*m3_firstorder_Range_mean)+(0.262108075854777*m3_glcm_MaximumProbabilit y_mean)+(0.0963846481219106*m3_glrlm_GrayLevelNonUniformity_mean)+(- 0.0240827277704738*m3_glrlm_RunEntropy_mean)+(0.0808465990105737*m3_glszm_SmallAreaLow GrayLevelEmphasis_mean)+(0.101730054217782*m3_gldm_DependenceVariance_mean)+(- 0.0641659604047439*m3_glszm_ZoneEntropy_val)+(- 0.156128634477967*m3_ngtdm_Strength_val)+(0.0480761327349284*m3_gldm_DependenceVariance_v al)+(- 0.0158504298440178*m3_glszm_HighGrayLevelZoneEmphasis_median)+(0.239822567610647*m3_glsz m_SizeZoneNonUniformity_median)+(-0.0854227134853446*m3_ngtdm_Strength_median)+(- 0.374990042858843*m3_firstorder_Variance_deciles)+(- 0.0288118947062985*m3_shape_Maximum2DDiameterSlice_deciles)+(0.0412986530691085*m3_glcm_ InverseVariance_deciles)+(0.0545821784331602*m3_ngtdm_Busyness_deciles)+(- 0.0166470771076297*m4_firstorder_RobustMeanAbsoluteDeviation_mean)+(-5.95091408028028e- 05*m4_shape_Elongation_val)+(-0.000327263427893282*m4_shape_Maximum3DDiameter_val)+(- 0.0340888177118691*m4_glrlm_GrayLevelNonUniformity_val)+(- 0.145572056812165*m4_glrlm_RunEntropy_val)+(- 0.00520738835700461*m4_glszm_SmallAreaLowGrayLevelEmphasis_val)+(0.0559264206173697*m4_ glcm_JointEntropy_median)+(0.0814362756256773*m4_glrlm_RunPercentage_median)+(- 0.107068836693169*m4_glszm_SmallAreaLowGrayLevelEmphasis_median)+(- 0.0446803057494133*m4_firstorder_Kurtosis_deciles)+(0.0735869457136152*m4_firstorder_Variance_d eciles)+(0.0905796709524612*m4_glcm_SumEntropy_deciles)+(- 0.00682601410330697*m4_glrlm_RunEntropy_deciles)+(0.0422734191404945*m4_glrlm_ShortRunLow GrayLevelEmphasis_deciles)+(0.121043709580541*m4_glszm_SizeZoneNonUniformity_deciles)+(- 0.0615050877519192*m4_gldm_DependenceNonUniformity_deciles)+(-1.20400145051144e- 07*m4_gldm_DependenceNonUniformityNormalized_deciles)+(- 0.0996393035747283*m4_gldm_SmallDependenceHighGrayLevelEmphasis_deciles)+(- 6.00690899626568e-07*m5_shape_VoxelVolume_mean)+(- 0.123237240690278*m5_glcm_Idmn_mean)+(-0.0885574591978988*m5_shape_Elongation_val)+(- 0.0464775090572624*m5_glrlm_LowGrayLevelRunEmphasis_val)+(- 0.0902622106981713*m5_glrlm_ShortRunEmphasis_val)+(- 0.129049672569143*m5_glrlm_ShortRunLowGrayLevelEmphasis_val)+(0.120089429752362*m5_glszm _GrayLevelNonUniformityNormalized_val)+(0.0882845789655749*m5_ngtdm_Strength_val)+(- 0.0684896372349365*m5_gldm_DependenceEntropy_val)+(- 0.0128537799943673*m5_gldm_LargeDependenceLowGrayLevelEmphasis_val)+(0.44077193209468*m 5_firstorder_Minimum_median)+(-5.36156738191924e- 06*m5_shape_Elongation_median)+(0.109782332532557*m5_glrlm_LongRunEmphasis_median)+(0.037 7897868976266*m5_glrlm_RunEntropy_median)+(0.138719389519223*m5_glrlm_ShortRunLowGrayLe velEmphasis_median)+(- 0.0126701800853986*m5_glszm_SmallAreaHighGrayLevelEmphasis_median)+(- 0.146411819724686*m5_ngtdm_Strength_median)+(- 0.0072636906885224*m5_firstorder_90Percentile_deciles)+(0.0553896879722332*m5_glcm_Correlation _deciles)+(0.0769298932491218*m5_glrlm_GrayLevelNonUniformity_deciles)+(- 0.00566567479165237*m5_glrlm_ShortRunHighGrayLevelEmphasis_deciles)+(0.0283413608763572*m 5_glszm_ZonePercentage_deciles)+(0.0864353494096385*m5_ngtdm_Contrast_deciles)+(0.0972885882 371829*m5_gldm_DependenceVariance_deciles)+(- 0.761630488278793*pred_0)+(0.53192078736907*pred_1)"

x2＝"(- 0.00076463928834012*Sex)+(0.00727032956750945*Age)+(0.298169325397803*HBsAg)+(0.00068424 2884045703*ALT)+(- 0.0152305660276121*ALB)+(0.275821849545985*AFP.CATE)+(0.000627552283166938*GGT)+(0.443 175586385359*Liver_Cirrhosis)+(0.146759389518157*Tumor_Number)+(0.0837535224678497*Tumor_ Siz)+(- 0.033596379247241*Capusle)+(0.541341012706675*Micro_Vascualr_Invasion)+(0.635932388065421*l ymphoid)+(0.0743711973866074*Differentiation)"

HS＝x1

CS＝x1+x2

3. model prognostic analysis

We obtain the optimal cutoff value of the model by a function. Patients with HS.ltoreq. 0.1605954 were enrolled in the low risk group and patients with HS > -0.1605954 were enrolled in the high risk group. We then performed Kaplan-Meier curve analysis on patients in the three data sets (FIG. 4). Patients in the high and low risk groups all have significant differences in Time To Relapse (TTR), time to survival without Relapse (RFS), and overall time to survival (OS). In the training and testing sets, there were 165 patients in the high risk group who had relapsed at the end of the follow-up (76.7%) and 78 in the low risk group (23.5%). The low risk group had 4.83%, 11.85% and 26.00% recurrence probabilities for 1 year, 3 years and 5 years, respectively. The high risk groups were 36.22%, 66.40% and 85.13%, respectively. The RFS rates of the low risk group at 1 year, 3 years and 5 years after operation are respectively 95.17%, 88.15% and 74.00%, and the RFS rates of the high risk group are respectively 62.33%, 32.84% and 14.53%. Median RFS for the high risk group was 19.53 months. Referring to table 1:

TABLE 1 regression analysis of relapse free survival time for high and low risk groups

In conclusion, the accuracy is very high while various different cell regions are identified, and the method has great guiding significance for the relapse of the patient.

The above examples are only intended to illustrate the technical solution of the present invention, but not to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and the modifications or the substitutions do not cause the essence of the corresponding technical solutions to depart from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. An AI-based analysis method for prognosis of postoperative recurrence of early hepatocellular carcinoma is characterized by comprising the following specific steps:

acquiring a WSI image of a sample to be analyzed;

selecting a small amount of WSI, and carrying out APSP software on 6 liver cancer tissues: manually labeling a tumor area, a paracancerous normal liver tissue, a fibrosis area, a junction area, a lymphocyte accumulation area and a hemorrhage/necrosis area;

extracting a foreground ROI from the WSI image;

integrating all the extracted features with clinical features of the patient;

screening the integrated features by using a Lasso Cox model, multiplying the screened features by the weight of the features to obtain a clinical score CS, and simply calculating the tissue features to obtain a histological score HS;

the HS and CS were divided into two high-risk and low-risk groups, KM curves were plotted and single-factor, multi-factor Cox analyses were performed to analyze patient relapse and prognostic variation in the high-and low-risk groups.

2. The AI-based analysis method for prognosis of postoperative recurrence of early hepatocellular carcinoma according to claim 1, wherein in the step of extracting foreground ROI, the Otsu threshold method is used to calculate suitable threshold values for different WSI images to distinguish the background and target regions of the WSI images.

3. The AI-based analysis method for prognosis of postoperative recurrence of early hepatocellular carcinoma as in claim 2, wherein the step of extracting foreground ROI comprises:

converting the original color image into a gray image;

mask-res gets the ROI area of the last whole WSI.

4. The AI-based analysis method for prognosis of postoperative recurrence of early hepatocellular carcinoma according to claim 1, wherein in the normalization process for staining the foreground ROI, the mean and standard deviation of the whole WSI image are calculated for the image cut out by 5000 x 5000 pixels from each WSI image, and used as the mean and standard deviation of all slices of the WSI image, so as to reduce the influence of the difference inside each patch on the normalization result.

5. The AI-based analysis method for prognosis of postoperative recurrence of early hepatocellular carcinoma according to claim 4, wherein the staining normalization procedure comprises:

the original slice is then changed from the LAB space back to the RGB space.

6. The AI-based analysis method for prognosis of postoperative recurrence of early hepatocellular carcinoma in accordance with claim 1, wherein the deep learning classification model can identify six types of histiocyte areas including tumor area, normal liver tissue, fibrosis area, junction area, lymphatic area and hemorrhage/necrosis area in WSI image.

7. The AI-based analysis method for prognosis of postoperative recurrence of early hepatocellular carcinoma as in claim 1, wherein the data enhancement process undersamples the data with large number of slices by rotating, translating, cropping and flipping the data with small number of slices, so that the number of samples in 6 classes of WSI images is equivalent.

8. The AI-based analysis method for prognosis of postoperative recurrence of early hepatocellular carcinoma as in claim 1, wherein the training set data is loaded into the inclusion-V3 model for training, and the model with the best validation set effect is selected as the final model. Meanwhile, all 416 WSIs are sliced and predicted, each WSI generates a heat map, and various thresholds are 0.5. All generated heatmaps were post-processed to optimize results.

9. The AI-based analysis method for prognosis of postoperative recurrence of early hepatocellular carcinoma according to claim 1, wherein each WSI image selects 10 blocks with the highest prediction probability, the image omics features are extracted, 107 histological features are obtained from each slice, the mean, standard deviation, median and decile of the feature calculation of 10 slices of each category are used as each category of features of the WSI image, and simultaneously, the total 6 features of the percentage of each category are counted as the global supplementary features of WSI area statistics.

10. The AI-based analysis method for prognosis of postoperative recurrence of early hepatocellular carcinoma according to claim 1, wherein a Lasso regression model is used to perform a preliminary screening on the integrated and standardized pathological image features and clinical features, the consistency index is used as an evaluation index, a cross validation strategy with ten folds is used to select an optimal lambda value, and the features with non-zero weight are obtained according to the lambda value, and the histological score HS is obtained by multiplying the selected pathological image features by the feature weight, and the clinical score CS is obtained by adding the clinical features.

11. An AI-based analysis system for prognosis of postoperative recurrence of early hepatocellular carcinoma, comprising:

the data acquisition module is used for acquiring a WSI image of a sample to be analyzed;

the data labeling module is used for training the recognition of the liver cancer 6 large-class tissue structure;

the data processing module is used for splitting, preprocessing and unifying difference of the WSI image;

the model training module is used for constructing a deep learning classification model with the best prediction effect;

the operation processing module is used for feature integration, screening and index establishment of the WSI image in the analysis step of recurrence prognosis;

and the result analysis module is used for visualizing the index effect and the significance degree of the parameterized index according to the data processing result of the operation processing module.