CN115266282A - Pathological image automatic identification method based on combination of three methods of HE staining, ki67 and P16 - Google Patents

Pathological image automatic identification method based on combination of three methods of HE staining, ki67 and P16 Download PDF

Info

Publication number
CN115266282A
CN115266282A CN202110484662.0A CN202110484662A CN115266282A CN 115266282 A CN115266282 A CN 115266282A CN 202110484662 A CN202110484662 A CN 202110484662A CN 115266282 A CN115266282 A CN 115266282A
Authority
CN
China
Prior art keywords
immersing
staining
washing
image
1min
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202110484662.0A
Other languages
Chinese (zh)
Other versions
CN115266282B (en
Inventor
庞宝川
郝宗杰
肖笛
龙莉
曹得华
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Wuhan Landing Medical High Tech Co ltd
Original Assignee
Wuhan Landing Medical High Tech Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Wuhan Landing Medical High Tech Co ltd filed Critical Wuhan Landing Medical High Tech Co ltd
Priority to CN202110484662.0A priority Critical patent/CN115266282B/en
Publication of CN115266282A publication Critical patent/CN115266282A/en
Application granted granted Critical
Publication of CN115266282B publication Critical patent/CN115266282B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • GPHYSICS
    • G01MEASURING; TESTING
    • G01NINVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N1/00Sampling; Preparing specimens for investigation
    • G01N1/28Preparing specimens for investigation including physical details of (bio-)chemical methods covered elsewhere, e.g. G01N33/50, C12Q
    • G01N1/30Staining; Impregnating ; Fixation; Dehydration; Multistep processes for preparing samples of tissue, cell or nucleic acid material and the like for analysis
    • GPHYSICS
    • G01MEASURING; TESTING
    • G01NINVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/84Systems specially adapted for particular applications
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/04Architecture, e.g. interconnection topology
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/08Learning methods

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Biomedical Technology (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • Software Systems (AREA)
  • Data Mining & Analysis (AREA)
  • Computational Linguistics (AREA)
  • General Engineering & Computer Science (AREA)
  • Biophysics (AREA)
  • Mathematical Physics (AREA)
  • Evolutionary Computation (AREA)
  • Artificial Intelligence (AREA)
  • Chemical & Material Sciences (AREA)
  • Analytical Chemistry (AREA)
  • Biochemistry (AREA)
  • Immunology (AREA)
  • Pathology (AREA)
  • Investigating Or Analysing Biological Materials (AREA)

Abstract

The invention relates to the field of pathological image processing, and provides an automatic pathological image identification method based on the combination of three methods of HE staining, ki67 and P16, which comprises the following steps: s1, preparing HE staining, ki67 and P16 sample tablets by using continuous pathological tissues respectively; s2, scanning panoramic pictures of the HE staining, ki67 and P16 sample films, and splicing; s3, HE staining, and fusion of Ki67 and P16 sample wafers; s4, identifying an abnormal region and/or a normal region in the fused picture; the pathological image automatic identification is realized through the steps. By adopting the scheme of fusing and identifying the images of three methods of HE dyeing, ki67 and P16, the diagnosis information contained in the fused image can be conveniently and rapidly read by people and artificial intelligence, especially the artificial intelligence can be conveniently identified, and the identification efficiency and precision are improved.

Description

Pathological image automatic identification method based on combination of three methods of HE staining, ki67 and P16
Technical Field
The invention relates to pathological image processing, belongs to the field of machine learning neural network models, and particularly relates to an automatic pathological image identification method based on combination of three methods, namely HE staining, ki67 and P16.
Background
Many studies indicate that abnormal pathological tissues usually have obvious changes, for example, the abnormal pathological tissues have obvious differences from normal pathological tissues in aspects of morphological structures, textural features and the like, and the abnormal pathological tissues usually have large heterogeneity, different sizes and different shapes; at the texture level, chromatin is condensed into clumps due to abnormal tissue division, and the texture is rough, which also leads to an increase in the DNA content inside the tissue. The abnormal characteristics of the pathological tissues provide a pathological basis for the computer to identify abnormal pathological tissues according to the pathological tissues. For example, the application of P16 and Ki67 expression of lehua in cervical lesion diagnosis describes that P16 positive expression is mainly concentrated in the nucleus and cytoplasm parts and shows a brownish yellow granular morphology. Ki67 positive expression is mainly located in the nucleus, and is positive in the form of brown yellow particles.
Many studies are currently carried out to identify abnormal pathological tissues based on their characteristics, which are roughly classified into two methods, one: the method is characterized in that the method is directly based on deep learning technology to extract the characteristics of pathological tissues, a convolutional neural network model is used for automatically extracting the characteristics of pathological tissue images, and then a pathological tissue classifier is constructed, so that abnormal pathological tissues are detected. However, due to the lack of interpretability of deep learning, the significance of the extracted features is not clear, so that the method cannot always maintain good performance and accuracy, for example, a breast cancer histopathological grading method based on the fusion of CNN and imagery omics features, which is described in CN 108898160A. The second method comprises the following steps: the pathological tissues and the overall outline of the pathological tissues are manually segmented, and then various morphological characteristics, textural characteristics and the like of the tissues are extracted. However, due to the problem of slide preparation or staining of pathological tissues, abnormal pathological tissues are often not clearly demarcated from normal pathological tissues, which brings great difficulty to the segmentation of abnormal pathological tissues, and the inaccurate region segmentation can cause the inaccurate extracted characteristics of the pathological tissues, so that the effectiveness of the second method depends heavily on the accurate segmentation of the pathological tissues, and the manual processing efficiency is very low.
Immunohistochemical detection of abnormal pathological tissue usually has a significant negative-positive contrast with respect to normal pathological tissue, so this is a significant feature for distinguishing abnormal pathological tissue. However, in many studies of the automated identification of abnormal pathological tissues, no researcher has used the negative-positive region characteristics of immunohistochemical detection for identifying abnormal pathological tissues. In view of the above, there is an urgent need for an automatic diagnosis method for abnormal pathological tissue, which can effectively mine and utilize abnormal features of pathological tissue, has high interpretability, can always maintain high diagnosis precision, and has high diagnosis efficiency.
Disclosure of Invention
The invention aims to solve the technical problem of providing an automatic pathological image identification method based on combination of three methods of HE dyeing, ki67 and P16, which can greatly improve the interpretability of an artificial intelligent model and improve the performance and the identification accuracy.
In order to solve the technical problems, the technical scheme adopted by the invention is as follows: a pathological image automatic identification method based on combination of three methods of HE staining, ki67 and P16 comprises the following steps:
s1, preparing HE staining, ki67 and P16 sample tablets by using continuous pathological tissues respectively;
s2, scanning panoramic pictures of the HE staining, ki67 and P16 sample films, and splicing;
s3, HE staining, and fusing Ki67 and P16 sample wafers;
s4, identifying an abnormal region and/or a normal region in the fused picture;
the pathological image automatic identification is realized through the steps.
In the step S1, the preferred scheme includes S1.1, material taking, and taking continuous pathological tissues to ensure that images of each continuous pathological tissue are consistent;
s1.2, fixing;
s1.3, dehydrating and transparent;
s1.4, dipping wax and embedding;
s1.5, slicing and pasting; at least three samples of each group of continuous pathological tissues are prepared and used for HE staining and two immunohistochemical examinations of Ki67 and P16 respectively.
In a preferred embodiment, the HE dye is prepared by the following steps:
HE refers to Hematoxylin Eosin (HE) staining solution.
S2.1, immersing the paraffin section into turpentine for 5min;
s2.2, immersing the slices in turpentine for 5min;
s2.3, immersing in absolute ethyl alcohol for 1min;
s2.4, immersing in new absolute ethyl alcohol for 1min;
s2.5, immersing in 95% ethanol for 1min;
s2.6, immersing in new 95% ethanol for 1min;
s2.7, immersing in 85% ethanol for 1min;
s2.8, immersing and washing for 1min, and draining;
s2.9, immersing in hematoxylin for 15min;
s2.10, washing for 1min and draining;
s2.11, immersing in 0.5% hydrochloric acid alcohol for differentiation for 3S;
s2.12, washing for 3 times, and draining;
s2.13, immersing the substrate in 1% ammonia water and returning blue for 2min;
s2.14, washing for 3 times and draining;
s2.15, immersing in 1% eosin for 2min;
s2.16, immersing and washing for 3 times, and draining;
s2.17, immersing in 95% ethanol for 1min;
s2.18, immersing in new 95% ethanol for 1min;
s2.19, immersing in absolute ethyl alcohol for 1min;
s2.20, blowing air to be semi-dry, and turpentine for 2min;
s2.21, immersing the turpentine for 2min;
s2.22, sealing the neutral gum into a piece;
and obtaining the HE dyed sample piece through the steps.
In a preferred embodiment, immunohistochemical P16, ki67 plaques are prepared as follows:
s3.1, immersing the turpentine for 10min; dewaxing;
s3.2, immersing in absolute ethyl alcohol for 1min; a hydration step;
s3.3, immersing in absolute ethyl alcohol for 1min; a hydration step;
s3.4, immersing in 95% ethanol for 1min; a hydration step;
s3.5, immersing in 95% ethanol for several seconds; a hydration step;
s3.6, immersing in 85% ethanol for several seconds; a hydration step;
s3.7, washing with tap water; soaking in water all the time;
s3.8, dividing the wafer into two frames before use, respectively corresponding to the P16 and Ki67 sample wafers, and putting the sample wafers into a pure water box for later use;
s3.9, preparing a repairing liquid:
a) 30ml of EDTA antigen repairing solution +1470ml of purified water;
b) Adding the prepared a) into a pressure cooker, not covering, putting the sample rack into the cooker after boiling with medium fire, covering, and timing for 2min after continuous air blowing; to expose the antigen;
s3.10, cooling the outer wall of the pressure cooker by using cold water;
s3.11, taking out the wafer rack and putting the wafer rack into cold purified water;
s3.12, changing water once; washing to remove the repairing liquid;
s3.13, arranging the sheet in a wet box; separately placing the P16 and Ki67 sample wafers;
s3.14, immersing the glass substrate into PBS (phosphate buffered saline) for washing;
s3.15, assembling a stroke circle;
s3.16, soaking the glass substrate into PBS (phosphate buffered saline) for washing;
s3.17, immersing in a 3% hydrogen peroxide solution for 5min; 3% hydrogen peroxide solution is used as a blocking agent;
s3.18, soaking in PBS for washing for 5min;
s3.19, soaking in PBS and washing for 5min;
s3.20, spin-drying the glass slide, adding 1-2 drops of primary antibody for 40min; at room temperature, the primary antibody, i.e., the first antibody, is a protein that specifically binds to a non-antibody antigen, i.e., a specific antigen; p16 and Ki67 adopt different antigens to obtain different sample wafers;
s3.21, immersing in PBS for washing for 5min;
s3.22, soaking in PBS and washing for 5min;
s3.23, spin-drying the glass slide, adding 1-2 drops of secondary antibody, and keeping for 20min; at room temperature, the secondary antibody is an antibody capable of binding with the antibody, namely the antibody, and the main function of the secondary antibody is to detect the existence of the antibody and amplify the signal of the primary antibody; p16 and Ki67 adopt different antigens to obtain different sample wafers;
s3.24, immersing in PBS for washing for 5min;
s3.25, soaking in PBS and washing for 5min;
s3.26, preparation c) DAB substrate buffer solution: DAB color developing agent 50;
d) Dripping the prepared c) 1-2 drops on a sample glass sheet, and developing for 10min in a dark place;
s3.27, putting on a shelf, and washing with tap water for multiple times to stop color development;
s3.28, immersing the sappanin for 2min;
s3.29, washing with tap water;
s3.30, differentiating for 1-2 times by 0.8% hydrochloric acid alcohol; no more than 10 seconds;
s3.31, washing with tap water;
s3.32, bluing with 1% ammonia water;
s3.33, washing with tap water;
s3.34, immersing in 95% ethanol;
s3.35, soaking in 95% ethanol;
s3.36, immersing in absolute ethyl alcohol;
s3.37, immersing in absolute ethyl alcohol;
s3.38, immersing and blowing the semi-dry sealing sheet;
the immunohistochemical P16 and Ki67 sample with high contrast is obtained through the steps.
In a preferred scheme, the splicing method comprises the following steps:
4.1, making a new blank layer;
4.2, reading in scanned image data;
4.3, reading row and column data corresponding to the image from the scanning equipment, wherein the row and column data are precisely calibrated in the scanning equipment to respectively generate two integer sequences from 0 to the maximum row number and from 0 to the maximum column number, the two integer sequences are the row index number and the column index number of the image in the panoramic data, and the image information corresponding to the initial image at the position 0 and the image position in a certain row and a certain column in the original image are found through the row index number and the column index number;
4.4, reading in the image arrays of the two sequences according to the row index number and the column index number;
4.5, copying the read array image to a corresponding area in a blank layer according to the index position, namely splicing the images into the blank layer one by one to form a required original image;
the automatic splicing of the HE staining and immunohistochemical P16 and Ki67 panoramic images is completed through the steps.
Preferably in step S3: and overlapping HE staining and immunohistochemical P16 and Ki67 picture files corresponding to continuous pathological tissues, setting transparency and/or different overlapping modes for the upper pictures, and obtaining an overlapped image based on the HE staining, the Ki67 and the P16 through the steps.
Preferably in step S3: overlapping by adopting a cv2.Addweighted average value in fusion, and setting discontinuous blocks of each picture as feature points to be aligned one by one;
setting discontinuous or complex texture blocks of each picture as feature point blocks, and aligning the feature point blocks one by one through sequential operations of moving, rotating and zooming; wherein, moving, rotating and zooming are operation sequences, namely, firstly, moving operation is carried out, and whether alignment is carried out is judged; if not, rotating operation is carried out, and whether alignment is carried out is judged; if not, then zooming operation is carried out, and alignment is realized by continuous loop iteration.
Preferably in step S3:
tracking the outline of the block in the HE dyed picture except for the high brightness value by taking the HE dyed picture as a bottom layer to form an outline path of the block;
the picture of Ki67 is positioned above the HE dyed picture and used as a second layer, the transparency is 30-100%, the superposition mode is texture superposition, namely data reflecting contrast difference in the Ki67 are read for superposition, brightness weighted superposition calculation is carried out on the brightness data, and the brightness and the shadow of the brightness channel data are respectively strengthened on pictures of other layers by taking a preset value or a brightness area as a reference so as to strengthen texture expression of a fusion picture;
the picture of P16 is positioned at the top of the picture of Ki67, the transparency is 30-100%, and the superposition mode is color superposition and clear area difference is obtained in a color superposition mode.
Further preferably, in step S3: the picture for copying the Ki67 is positioned above the second layer, the transparency is 30 to 100 percent, and the superposition mode is color superposition.
Preferably in step S4: and performing K clustering analysis on the fused panoramic image, setting the expected label number to be 2, returning a label and a center to separate the abnormal area from the normal area, and giving area proportion data.
According to the pathological image automatic identification method based on the combination of the three methods of HE dyeing, ki67 and P16, the scheme of fusing and identifying the images of the three methods of HE dyeing, ki67 and P16 is adopted, so that the diagnosis information contained in the fused image can be conveniently and rapidly read by people and artificial intelligence, the artificial intelligence can be conveniently identified, and the identification efficiency and accuracy are improved. Compared with single image recognition, the method can identify the abnormal area at one time, and give the occupation ratio between the abnormal area and the normal area.
Drawings
The invention is further illustrated with reference to the following figures and examples:
fig. 1 is an HE-based stained mosaic image of a first example of the invention.
Fig. 2 is a Ki67 based stitched image of the first example of the present invention.
Fig. 3 is a P16-based stitched image of the first example of the present invention.
Fig. 4 is a fused image based on HE, ki67 and P16 according to the first example of the present invention.
Fig. 5 is an image after the automatic recognition mark according to the first embodiment of the present invention.
Fig. 6 is a HE-based stained mosaic image of a second example of the invention.
Fig. 7 is a Ki67 based stitched image of a second example of the present invention.
Fig. 8 is a P16-based stitched image of the second example of the present invention.
Fig. 9 is a fused image based on HE, ki67 and P16 according to a second example of the present invention.
Fig. 10 is an image after the automatic recognition mark according to the second example of the present invention.
Detailed Description
1. The sample preparation steps of the invention are as follows:
1. the steps of preparing the sample wafer are as follows:
1.1, taking materials, and taking continuous pathological tissues to ensure that images of each continuous pathological tissue are basically consistent;
1.2, fixing;
1.3, dehydrating and transparent;
1.4, wax dipping and embedding;
1.5, slicing and pasting. At least three samples of each group of continuous pathological tissues are prepared and used for HE staining and two immunohistochemical examinations of Ki67 and P16 respectively.
2. HE staining
HE refers to Hematoxylin Eosin (HE) staining solution.
2.1, immersing the paraffin sections into turpentine for 5min; dewaxing;
2.2 Soaking the slices in oleum Terebinthinae for 5min; dewaxing;
2.3, immersing in absolute ethyl alcohol for 1min; a hydration step;
2.4, immersing in new absolute ethyl alcohol for 1min; a hydration step;
2.5, soaking in 95% ethanol for 1min; a hydration step;
2.6, immersing in new 95% ethanol for 1min; a hydration step;
2.7, immersing in 85% ethanol for 1min; a hydration step;
2.8, immersing and washing for 1min, and draining; a hydration step;
2.9, immersing in hematoxylin for 15min; a dyeing process;
2.10, washing with water for 1min, and draining; a dyeing process;
2.11, immersing into 0.5 percent hydrochloric acid alcohol for differentiation for 3s; a dyeing process;
2.12, washing for 3 times, and draining; a dyeing process;
2.13, immersing in 1% ammonia water and returning to blue for 2min; a dyeing process;
2.14, washing with water for 3 times, and draining; a dyeing process;
2.15, immersing in 1% eosin for 2min; a dyeing process;
2.16, immersing and washing for 3 times, and draining; a dyeing process;
2.17, soaking in 95% ethanol for 1min; a dyeing process;
2.18, immersing in new 95% ethanol for 1min; a dyeing process;
2.19, immersing in absolute ethyl alcohol for 1min; a dyeing process;
2.20, blowing air to be half-dry, and turpentine for 2min; a sealing step;
2.21, immersing the turpentine for 2min; a step of mounting a sheet;
2.22, sealing neutral gum; and (5) sealing.
HE-stained swatches were obtained by the above procedure, as shown in fig. 1, 6.
3. Immunohistochemistry of P16, ki67
The preparation steps of the immunohistochemical P16 and Ki67 sample tablet are as follows:
3.1, immersing the turpentine for 10min; dewaxing;
3.2, immersing in absolute ethyl alcohol for 1min; a hydration step;
3.3, immersing in absolute ethyl alcohol for 1min; a hydration step;
3.4, soaking in 95% ethanol for 1min; a hydration step;
3.5, immersing in 95% ethanol for several seconds; a hydration step;
3.6, immersing in 85% ethanol for several seconds; a hydration step;
3.7, washing with tap water; soaking in water all the time;
3.8, dividing the wafer into two frames before use, respectively corresponding to the P16 and Ki67 sample wafers, and putting the sample wafers into a pure water box for later use;
3.9, preparation of repair liquid:
a) 30ml of EDTA antigen repairing solution and 1470ml of purified water;
b) Adding the prepared a) into a pressure cooker, without covering, putting the sample rack into the cooker after boiling with medium fire, covering, and timing for 2min after continuous gas emission; to expose the antigen;
3.10, cooling the outer wall of the pressure cooker by cold water;
3.11, taking out the wafer rack and putting the wafer rack into cold purified water;
3.12, changing water once; washing to remove the repairing liquid;
3.13, arranging the sheets in a wet box; separately placing the P16 and Ki67 sample wafers;
3.14, soaking in PBS (phosphate buffered saline) for washing;
3.15, assembling a pen drawing circle;
3.16, soaking in PBS (phosphate buffered saline) for washing;
3.17, immersing in 3% hydrogen peroxide solution for 5min; 3% hydrogen peroxide solution is used as a blocking agent;
3.18, soaking in PBS for washing for 5min;
3.19, soaking in PBS and washing for 5min;
3.20, spin-drying the glass slide, adding primary antibody for 1-2 drops for 40min; at room temperature, the primary antibody, i.e., the primary antibody, is a protein that specifically binds to a non-antibody antigen, i.e., a specific antigen. P16 and Ki67 were obtained in different swatches using different antigens.
3.21, soaking in PBS and washing for 5min;
3.22, soaking in PBS and washing for 5min;
3.23, spin-drying the glass slide, adding 1-2 drops of secondary antibody, and carrying out 20min; at room temperature, the secondary antibody, i.e., the antibody, is an antibody capable of binding to the antibody, i.e., the antibody, and its primary function is to detect the presence of the antibody and amplify the signal of the primary antibody. P16 and Ki67 were obtained in different swatches using different antigens.
3.24, soaking in PBS and washing for 5min;
3.25, soaking in PBS and washing for 5min;
3.26, preparation c) DAB substrate buffer: DAB developer 50. DAB staining solutions are commercially available products, generally comprising three components, which are ready to use at the time of use. Wherein the component A is a peroxidase marker; the component B is DAB substrate buffer solution, and the component C is DAB color developing agent.
During operation, 1 drop of color developing agent is dropped into 1ml of substrate buffer solution, namely 1 drop of component C is dropped into 1ml of component B, and a suction pipe blows and uniformly mixes the components, so that bubbles are prevented from being generated;
d) Dripping 1-2 drops of c) on a sample glass slide, and developing for 10min in a dark place;
3.27, put on shelf, and wash with tap water for multiple times to stop color development.
3.28, immersing in hematoxylin for 2min;
3.29, washing with tap water;
3.30 and 0.8 percent hydrochloric acid alcohol for 1 to 2 times; no more than 10 seconds;
3.31, washing with tap water;
3.32, 1% ammonia returns blue;
3.33, washing with tap water;
3.34, immersing in 95% ethanol;
3.35, soaking in 95% ethanol;
3.36, immersing in absolute ethyl alcohol;
3.37, immersing in absolute ethyl alcohol;
3.38, immersing and blowing the semi-dry sealing piece. Immunohistochemical P16 and Ki67 samples were obtained by the above procedure. The contrast of the sample image obtained in this step is high, which is convenient for subsequent recognition by people or artificial intelligence, as shown in fig. 2, 3, 7, 8.
4. HE staining, immunohistochemistry P16, ki67 were panned and stitched.
The scanning step adopts automatic continuous scanning, and the splicing adopts automatic splicing, for example, a micro-image acquisition device based on a mobile phone and an image splicing and identifying method recorded in CN 201911112866.0. Or the artificial intelligence cloud diagnosis platform recorded in CN 201910964425.7.
Or the following steps are adopted:
4.1, making a new blank layer;
4.2, reading in scanned image data;
4.3, reading row and column data corresponding to the image from the scanning device, wherein the row and column data are precisely calibrated in the scanning device to respectively generate two integer sequences from 0 to the maximum row number and from 0 to the maximum column number, the two integer sequences are the row index number and the column index number of the image in the panoramic data, and then finding the image and the image information corresponding to the initial image at the position 0 and the image position at a certain row and column in the original image through the row index number and the column index number.
4.4, reading in the image arrays of the two sequences according to the row index number and the column index number;
and 4.5, copying the read array images to corresponding areas in a blank layer according to the index positions, namely splicing the images into the blank layer one by one to form a required original image, and completing automatic splicing of the HE dyeing and immunohistochemical P16 and Ki67 panoramic images.
In this example, by accurately calibrating the scanning device, automatic splicing without calculation is realized, and compared with the splicing scheme of CN201911112866.0 after calculation in the prior art, the efficiency of reading and splicing is greatly improved, and a large amount of operation time is reduced.
In a preferred scheme, the panoramic image of the dailies is converted into a BGR channel sequence so as to adapt to an artificial intelligence algorithm model. The data type is converted to float32 type to improve data accuracy.
5. Fusing and labeling the panoramic images of HE staining and immunohistochemical P16 and Ki67
5.1, the data type of center of the HE stained and immunohistochemical P16 and Ki67 panoramic image is changed to uint8, and the processing speed is increased.
5.2, the center is used as an array, the label is used as an array label, and the center [ label ] is saved as a readable general image format, such as TIF and JPG.
Picture files of HE staining, immunohistochemical P16, ki67 were obtained, respectively.
And 5.3, overlapping the image files of the HE staining and immunohistochemical P16 and Ki67 corresponding to the continuous pathological tissues, setting the transparency and/or different overlapping modes of the upper layer of the image, and storing the image files as the image files for fusing the panoramic image. See fig. 4 and 9.
In a preferred scheme, the fusion is carried out by adopting a cv2.Addweighted average value for superposition, and discontinuous blocks of each picture are set as feature points to be aligned one by one. Namely, discontinuous blocks or blocks with rich textures in the image are set as feature points, and the feature point blocks are aligned one by one through sequential operations of moving, rotating and zooming. Wherein, moving, rotating and zooming are operation sequences, namely, firstly, moving operation is carried out, and whether alignment is carried out is judged; if not, performing rotation operation and judging whether the alignment is performed or not; if not, zooming operation is carried out again, and continuous loop iteration is carried out to realize alignment.
In a preferred scheme, an HE dyed picture is used as a bottom layer, block outlines except high-brightness values in the HE dyed picture are tracked, and a block outline path is formed, so that the scheme is convenient for evaluating the occupation ratio of each area through the expression of the whole outline. The high luminance value here is a set value, for example, luminance in BGR mode exceeds 215:215:215, or establishing an artificial intelligence model, taking the HE dyed pictures as a data set, labeling the pictures through labelImg, dividing the data set into a training set and a testing set, and training by adopting a dark 53 network, thereby improving the characteristic extraction efficiency. Darknet-53 is composed of a series of 1 × 1, 3 × 3 convolution blocks, namely, convolitional and Residual blocks, namely, residual, the convolution blocks are composed of convolution layers con2d, batch normalization layers BN and leakage correction linear unit Leaky ReLU layers, meanwhile, in order to adapt to the target detection task Darknet-53, the pooling layers and the full connection layers are removed, and the convolution with the step length of 2 is used for carrying out downsampling, so that the training efficiency is improved.
The picture of Ki67 is located above the HE dyed picture and used as a second layer, the transparency is 30-100%, the superposition mode is texture superposition, namely data reflecting contrast difference in Ki67 are read for superposition, for example, a lab mode is selected, brightness weighting superposition calculation is carried out on brightness channel data, and the brightness and the shadow of the brightness channel data are respectively used for strengthening pictures of other layers by taking a preset value or a brightness area as a reference, namely, the pictures are brighter and darker. To enhance the texture, especially intracellular texture expression, of the fused map. Since Ki67 mainly expresses the activity of hyperplasia, mainly the staining of the cell nucleus. The situation of the cell nucleus can be observed more clearly through the texture.
In a preferred scheme, a picture for copying Ki67 is positioned above the second layer, the transparency is 30-100%, and the superposition mode is color superposition. It is further preferred that only a brownish yellow area is selected in the layer as a color overlay.
The picture of P16 is positioned at the top of the picture of Ki67, the transparency is 30-100%, the superposition mode is color superposition, the positive reaction of P16 is diffuse continuous brown yellow expression, and the expression parts are cytoplasm and cell nucleus. Clear regional differences can be obtained in a color superposition mode. As shown in fig. 4 and 9. The fused images can also express the diagnostic information contained in each image as much as possible.
And 5.4, performing K clustering analysis on the fused panoramic image, setting the number of expected labels to be 2, returning a label and a center, wherein the K clustering is used for separating a detailed abnormal region from a detailed normal region and giving percentage data of each region so as to guide the diagnosis of a doctor. See fig. 5, 10. The fused image is convenient for marking out the abnormal region, and can also be used for conveniently counting the proportion data of the abnormal region and the normal region.
Preferably, the fused panoramic image is subjected to K cluster analysis by using cv2. Kmeans.
The first step is as follows: and determining K values, and clustering into K clusters. In this example 2 is selected.
The second step is that: k data points are randomly selected or somehow selected from the data as the center of the initial classification.
The third step: respectively calculating the distance from each point in the data to each center, and dividing each point into the class closest to the center;
the fourth step: after each center is divided into a plurality of points, the mean value of each class is removed, and a new center is selected.
The fifth step: and comparing the new center with the previous center, and if the distance between the new center and the previous center is smaller than a certain threshold value or the iteration number exceeds a certain threshold value, considering that the clustering is converged and terminating.
And a sixth step: otherwise, the third step to the fifth step are executed continuously until the fifth step is satisfied.
A clustering model is obtained. And establishing a data set, and training a clustering model. The clustering model for labeling the abnormal regions and/or the normal regions is obtained in the above manner.
The above-described embodiments are merely preferred embodiments of the present invention, and should not be construed as limiting the present invention, and features in the embodiments and examples in the present application may be arbitrarily combined with each other without conflict. The protection scope of the present invention is defined by the claims, and includes equivalents of technical features of the claims. I.e., equivalent alterations and modifications within the scope hereof, are also intended to be within the scope of the invention.

Claims (10)

1. A pathological image automatic identification method based on the combination of three methods of HE staining, ki67 and P16 is characterized by comprising the following steps:
s1, preparing HE staining, ki67 and P16 sample tablets by using continuous pathological tissues respectively;
s2, scanning panoramic pictures of the HE staining sample, the Ki67 sample and the P16 sample, and splicing;
s3, HE staining, and fusing Ki67 and P16 sample wafers;
s4, identifying an abnormal region and/or a normal region in the fused picture;
the pathological image automatic identification is realized through the steps.
2. The method for automatically identifying pathological images based on the combination of three methods of HE staining, ki67 and P16 as claimed in claim 1, wherein the method comprises the following steps:
in the step S1, material selection is performed, and continuous pathological tissues are taken so as to ensure that images of all the continuous pathological tissues are consistent;
s1.2, fixing;
s1.3, dehydrating and transparentizing;
s1.4, dipping wax and embedding;
s1.5, slicing and pasting; at least three samples of each group of continuous pathological tissues are prepared and are respectively used for HE staining and two immunohistochemical examinations of Ki67 and P16.
3. The method for automatically identifying pathological images based on the combination of three methods of HE staining, ki67 and P16 as claimed in claim 2, wherein the method comprises the following steps:
the preparation steps of the HE dye are as follows:
HE refers to Hematoxylin Eosin (HE) staining solution;
s2.1, immersing the paraffin sections into turpentine for 5min;
s2.2, immersing the slices in turpentine for 5min;
s2.3, immersing in absolute ethyl alcohol for 1min;
s2.4, immersing in new absolute ethyl alcohol for 1min;
s2.5, immersing in 95% ethanol for 1min;
s2.6, immersing in new 95% ethanol for 1min;
s2.7, immersing in 85% ethanol for 1min;
s2.8, immersing and washing for 1min, and draining;
s2.9, immersing in hematoxylin for 15min;
s2.10, washing for 1min and draining;
s2.11, immersing in 0.5% hydrochloric alcohol for differentiation for 3S;
s2.12, washing for 3 times and draining;
s2.13, immersing the substrate in 1% ammonia water and returning blue for 2min;
s2.14, washing for 3 times and draining;
s2.15, immersing in 1% eosin for 2min;
s2.16, immersing and washing for 3 times, and draining;
s2.17, immersing in 95% ethanol for 1min;
s2.18, immersing in new 95% ethanol for 1min;
s2.19, immersing in absolute ethyl alcohol for 1min;
s2.20, blowing air to be semi-dry, and turpentine for 2min;
s2.21, immersing the turpentine for 2min;
s2.22, sealing a neutral gum;
and obtaining the HE dyed sample piece through the steps.
4. The method for automatically identifying pathological images based on the combination of three methods of HE staining, ki67 and P16 as claimed in claim 2, wherein the method comprises the following steps:
the preparation steps of the immunohistochemical P16 and Ki67 sample tablet are as follows:
s3.1, immersing the turpentine for 10min; dewaxing;
s3.2, immersing in absolute ethyl alcohol for 1min; a hydration step;
s3.3, immersing in absolute ethyl alcohol for 1min; a hydration step;
s3.4, immersing in 95% ethanol for 1min; a hydration step;
s3.5, immersing in 95% ethanol for several seconds; a hydration step;
s3.6, immersing in 85% ethanol for several seconds; a hydration step;
s3.7, washing with tap water; soaking in water all the time;
s3.8, dividing the wafer into two frames before use, respectively corresponding to the P16 and Ki67 sample wafers, and putting the sample wafers into a pure water box for later use;
s3.9, preparing a repairing liquid:
a) 30ml of EDTA antigen repairing solution +1470ml of purified water;
b) Adding the prepared a) into a pressure cooker, without covering, putting the sample rack into the cooker after boiling with medium fire, covering, and timing for 2min after continuous gas emission; to expose the antigen;
s3.10, cooling the outer wall of the pressure cooker by using cold water;
s3.11, taking out the wafer rack and putting the wafer rack into cold purified water;
s3.12, changing water once; washing to remove the repairing liquid;
s3.13, arranging the sheet in a wet box; separately placing the P16 and Ki67 sample wafers;
s3.14, soaking in PBS (phosphate buffered saline) for washing;
s3.15, assembling a stroke circle;
s3.16, soaking the glass substrate into PBS (phosphate buffered saline) for washing;
s3.17, immersing in a 3% hydrogen peroxide solution for 5min; 3% hydrogen peroxide solution is used as a blocking agent;
s3.18, soaking in PBS for washing for 5min;
s3.19, soaking in PBS and washing for 5min;
s3.20, spin-drying the glass slide, adding 1-2 drops of primary antibody for 40min; at room temperature, the primary antibody, i.e., the first antibody, is a protein that specifically binds to a non-antibody antigen, i.e., a specific antigen; p16 and Ki67 adopt different antigens to obtain different sample wafers;
s3.21, soaking in PBS for washing for 5min;
s3.22, soaking in PBS and washing for 5min;
s3.23, spin-drying the glass slide, adding 1-2 drops of secondary antibody, and keeping for 20min; at room temperature, the secondary antibody is an antibody capable of binding with the antibody, namely the antibody, and the main function of the secondary antibody is to detect the existence of the antibody and amplify the signal of the primary antibody; p16 and Ki67 adopt different antigens to obtain different sample wafers;
s3.24, immersing in PBS for washing for 5min;
s3.25, soaking in PBS and washing for 5min;
s3.26, preparation c) DAB substrate buffer solution: DAB color developing agent 50;
d) Dripping 1-2 drops of the prepared c) on a sample glass sheet, and developing for 10min in a dark place;
s3.27, putting on a shelf, and washing with tap water for multiple times to stop color development;
s3.28, immersing the sappanin for 2min;
s3.29, washing with tap water;
s3.30, differentiating for 1-2 times by 0.8% hydrochloric acid alcohol; no more than 10 seconds;
s3.31, washing with tap water;
s3.32, returning blue by 1% ammonia water;
s3.33, washing with tap water;
s3.34, immersing in 95% ethanol;
s3.35, immersing in 95% ethanol;
s3.36, immersing in absolute ethyl alcohol;
s3.37, immersing in absolute ethyl alcohol;
s3.38, immersing and blowing the semi-dry sealing sheet;
the immunohistochemical P16 and Ki67 sample with high contrast is obtained through the steps.
5. The method for automatically identifying pathological images based on the combination of three methods of HE staining, ki67 and P16 as claimed in claim 1, wherein the splicing method in step S2 is as follows:
4.1, manufacturing a new blank layer;
4.2, reading in scanned image data;
4.3, reading row and column data corresponding to the image from the scanning equipment, wherein the row and column data are precisely calibrated in the scanning equipment to respectively generate two integer sequences from 0 to the maximum row number and from 0 to the maximum column number, the two integer sequences are the row index number and the column index number of the image in the panoramic data, and the image information corresponding to the initial image at the position of a certain row and a certain column in the original image are found through the row index number and the column index number;
4.4, reading in the image arrays of the two sequences according to the row index number and the column index number;
4.5, copying the read array image to a corresponding area in a blank layer according to the index position, namely splicing the images into the blank layer one by one to form a required original image;
the automatic splicing of the HE staining and immunohistochemical P16 and Ki67 panoramic images is completed through the steps.
6. The method for automatically identifying pathological images based on the combination of three methods of HE staining, ki67 and P16 as claimed in claim 1, wherein in step S3: and overlapping HE staining and immunohistochemical P16 and Ki67 picture files corresponding to continuous pathological tissues, setting transparency and/or different overlapping modes for the upper pictures, and obtaining an overlapped image based on the HE staining, the Ki67 and the P16 through the steps.
7. The method for automatically identifying pathological images based on the combination of three methods of HE staining, ki67 and P16 as claimed in claim 1, wherein in step S3: overlapping by adopting cv2.Addweighted average values, and setting discontinuous blocks of each picture as feature points to be aligned one by one;
setting discontinuous or complex texture blocks of each picture as feature point blocks, and aligning the feature point blocks one by one through sequential operations of moving, rotating and zooming; the moving, rotating and zooming are operation sequences, namely, the moving operation is firstly carried out, and whether the alignment is carried out or not is judged; if not, rotating operation is carried out, and whether alignment is carried out is judged; if not, zooming operation is carried out again, and continuous loop iteration is carried out to realize alignment.
8. The method for automatically identifying pathological images based on the combination of three methods of HE staining, ki67 and P16 as claimed in claim 1, wherein in step S3:
taking the HE dyed picture as a bottom layer, tracking the outline of the block in the HE dyed picture except the high brightness value to form a block outline path;
the picture of Ki67 is positioned above the HE dyed picture and used as a second layer, the transparency is 30-100%, the superposition mode is texture superposition, namely data reflecting contrast difference in the Ki67 are read for superposition, brightness weighted superposition calculation is carried out on the brightness data, and the brightness and the shadow of the brightness channel data are respectively strengthened on pictures of other layers by taking a preset value or a brightness area as a reference so as to strengthen texture expression of a fusion picture;
the picture of the P16 is positioned at the top of the picture of the Ki67, the transparency is 30 to 100 percent, and the superposition mode is color superposition and the clear area difference is obtained in a color superposition mode.
9. The method for automatically identifying pathological images based on the combination of three methods of HE staining, ki67 and P16 as claimed in claim 8, wherein in step S3: the picture for copying the Ki67 is positioned above the second layer, the transparency is 30 to 100 percent, and the superposition mode is color superposition.
10. The method for automatically identifying pathological images based on the combination of three methods of HE staining, ki67 and P16 as claimed in claim 1, wherein in step S4: and performing K clustering analysis on the fused panoramic image, setting the number of expected labels to be 2, returning a label and a center to separate the abnormal area from the normal area, and giving the proportion data of each area.
CN202110484662.0A 2021-04-30 2021-04-30 Pathological image automatic identification method based on combination of HE (HE) dyeing, ki67 and P16 Active CN115266282B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110484662.0A CN115266282B (en) 2021-04-30 2021-04-30 Pathological image automatic identification method based on combination of HE (HE) dyeing, ki67 and P16

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110484662.0A CN115266282B (en) 2021-04-30 2021-04-30 Pathological image automatic identification method based on combination of HE (HE) dyeing, ki67 and P16

Publications (2)

Publication Number Publication Date
CN115266282A true CN115266282A (en) 2022-11-01
CN115266282B CN115266282B (en) 2023-07-21

Family

ID=83745244

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110484662.0A Active CN115266282B (en) 2021-04-30 2021-04-30 Pathological image automatic identification method based on combination of HE (HE) dyeing, ki67 and P16

Country Status (1)

Country Link
CN (1) CN115266282B (en)

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20080044849A1 (en) * 2006-08-15 2008-02-21 Alfred Bocking Method for cell analysis
CN106373088A (en) * 2016-08-25 2017-02-01 中国电子科技集团公司第十研究所 Quick mosaic method for aviation images with high tilt rate and low overlapping rate
CN107451985A (en) * 2017-08-01 2017-12-08 中国农业大学 A kind of joining method of the micro- sequence image of mouse tongue section
CN110363706A (en) * 2019-06-26 2019-10-22 杭州电子科技大学 A kind of large area bridge floor image split-joint method
CN111180048A (en) * 2019-12-30 2020-05-19 上海研境医疗科技有限公司 Tumor component labeling method, device, equipment and storage medium
CN111766125A (en) * 2020-07-29 2020-10-13 广州金域医学检验中心有限公司 Staining method using fluorescence quenching time difference, automatic staining apparatus, device, and medium

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20080044849A1 (en) * 2006-08-15 2008-02-21 Alfred Bocking Method for cell analysis
CN106373088A (en) * 2016-08-25 2017-02-01 中国电子科技集团公司第十研究所 Quick mosaic method for aviation images with high tilt rate and low overlapping rate
CN107451985A (en) * 2017-08-01 2017-12-08 中国农业大学 A kind of joining method of the micro- sequence image of mouse tongue section
CN110363706A (en) * 2019-06-26 2019-10-22 杭州电子科技大学 A kind of large area bridge floor image split-joint method
CN111180048A (en) * 2019-12-30 2020-05-19 上海研境医疗科技有限公司 Tumor component labeling method, device, equipment and storage medium
CN111766125A (en) * 2020-07-29 2020-10-13 广州金域医学检验中心有限公司 Staining method using fluorescence quenching time difference, automatic staining apparatus, device, and medium

Also Published As

Publication number Publication date
CN115266282B (en) 2023-07-21

Similar Documents

Publication Publication Date Title
US11682192B2 (en) Deep-learning systems and methods for joint cell and region classification in biological images
JP7197584B2 (en) Methods for storing and retrieving digital pathology analysis results
CN110472616B (en) Image recognition method and device, computer equipment and storage medium
CN112435243A (en) Automatic analysis system and method for full-slice digital pathological image
CN112326961B (en) Analysis method and storage device for proportion of PD-L1 positive tumor cells in non-small cell lung cancer
CN113574534A (en) Machine learning using distance-based similarity labels
CN108074243A (en) A kind of cellular localization method and cell segmentation method
CN107328776A (en) A kind of quick determination method of immune chromatography test card
US11959848B2 (en) Method of storing and retrieving digital pathology analysis results
CN113762379B (en) Method for generating training data based on immunohistochemistry and storage device
CN106706643B (en) A kind of liver cancer comparison slice detection method
CN112215217B (en) Digital image recognition method and device for simulating doctor to read film
CN113554638A (en) Method and system for establishing chip surface defect detection model
CN110060229A (en) A kind of cell automatic positioning dividing method of myeloplast
CN111126393A (en) Vehicle appearance refitting judgment method and device, computer equipment and storage medium
CN116057538A (en) Machine learning model for cell localization and classification using exclusive code learning
CN111950544A (en) Method and device for determining interest region in pathological image
CN117670794A (en) TLS pathology detection method, device and medium based on deep learning
CN114387596A (en) Automatic interpretation system for cytopathology smear
Brázdil et al. Automated annotations of epithelial cells and stroma in hematoxylin–eosin‐stained whole‐slide images using cytokeratin re‐staining
CN112270684A (en) Microscopic image immunohistochemical virtual multiple labeling and analyzing method and system
CN115266282A (en) Pathological image automatic identification method based on combination of three methods of HE staining, ki67 and P16
EP3803686A1 (en) Predicting cancer recurrence from spatial multi-parameter cellular and subcellular imaging data
CN114187334B (en) Adjacent slice image superposition and alignment method based on HE staining, Ki67 and P16 combination
CN115049714B (en) In-situ hybridization fluorescence microscopic panorama registration and microenvironment construction method and system

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant