EP3574445A1 - Image segmentation in digital pathology - Google Patents
Image segmentation in digital pathologyInfo
- Publication number
- EP3574445A1 EP3574445A1 EP18701181.2A EP18701181A EP3574445A1 EP 3574445 A1 EP3574445 A1 EP 3574445A1 EP 18701181 A EP18701181 A EP 18701181A EP 3574445 A1 EP3574445 A1 EP 3574445A1
- Authority
- EP
- European Patent Office
- Prior art keywords
- stain
- region
- image
- cytoplasm
- membrane
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Withdrawn
Links
- 230000007170 pathology Effects 0.000 title abstract description 11
- 238000003709 image segmentation Methods 0.000 title abstract description 5
- 210000004027 cell Anatomy 0.000 claims abstract description 116
- 239000012528 membrane Substances 0.000 claims abstract description 90
- 210000000805 cytoplasm Anatomy 0.000 claims abstract description 88
- 238000012545 processing Methods 0.000 claims abstract description 33
- 238000010186 staining Methods 0.000 claims abstract description 22
- 238000004458 analytical method Methods 0.000 claims abstract description 12
- 239000000090 biomarker Substances 0.000 claims abstract description 8
- 238000000034 method Methods 0.000 claims description 72
- 230000008569 process Effects 0.000 claims description 46
- 238000004590 computer program Methods 0.000 claims description 15
- 230000003628 erosive effect Effects 0.000 claims description 9
- 230000000877 morphologic effect Effects 0.000 claims description 9
- 238000001514 detection method Methods 0.000 claims description 6
- 238000003672 processing method Methods 0.000 claims description 4
- 238000003364 immunohistochemistry Methods 0.000 description 16
- WZUVPPKBWHMQCE-UHFFFAOYSA-N Haematoxylin Chemical compound C12=CC(O)=C(O)C=C2CC2(O)C1C1=CC=C(O)C(O)=C1OC2 WZUVPPKBWHMQCE-UHFFFAOYSA-N 0.000 description 10
- 230000011218 segmentation Effects 0.000 description 10
- 230000009977 dual effect Effects 0.000 description 6
- 102000015735 Beta-catenin Human genes 0.000 description 4
- 108060000903 Beta-catenin Proteins 0.000 description 4
- 238000013459 approach Methods 0.000 description 3
- 230000008859 change Effects 0.000 description 3
- 230000001419 dependent effect Effects 0.000 description 3
- 239000000975 dye Substances 0.000 description 3
- 102000039446 nucleic acids Human genes 0.000 description 3
- 108020004707 nucleic acids Proteins 0.000 description 3
- 150000007523 nucleic acids Chemical class 0.000 description 3
- 230000008901 benefit Effects 0.000 description 2
- 238000010191 image analysis Methods 0.000 description 2
- DWCZIOOZPIDHAB-UHFFFAOYSA-L methyl green Chemical compound [Cl-].[Cl-].C1=CC(N(C)C)=CC=C1C(C=1C=CC(=CC=1)[N+](C)(C)C)=C1C=CC(=[N+](C)C)C=C1 DWCZIOOZPIDHAB-UHFFFAOYSA-L 0.000 description 2
- 230000037361 pathway Effects 0.000 description 2
- NTGBUUXKGAZMSE-UHFFFAOYSA-N phenyl n-[4-[4-(4-methoxyphenyl)piperazin-1-yl]phenyl]carbamate Chemical group C1=CC(OC)=CC=C1N1CCN(C=2C=CC(NC(=O)OC=3C=CC=CC=3)=CC=2)CC1 NTGBUUXKGAZMSE-UHFFFAOYSA-N 0.000 description 2
- 230000007704 transition Effects 0.000 description 2
- 238000011179 visual inspection Methods 0.000 description 2
- 230000003936 working memory Effects 0.000 description 2
- HSTOKWSFWGCZMH-UHFFFAOYSA-N 3,3'-diaminobenzidine Chemical compound C1=C(N)C(N)=CC=C1C1=CC=C(N)C(N)=C1 HSTOKWSFWGCZMH-UHFFFAOYSA-N 0.000 description 1
- 206010006187 Breast cancer Diseases 0.000 description 1
- 208000026310 Breast neoplasm Diseases 0.000 description 1
- 102100028914 Catenin beta-1 Human genes 0.000 description 1
- 102000006947 Histones Human genes 0.000 description 1
- 108010033040 Histones Proteins 0.000 description 1
- 101000916173 Homo sapiens Catenin beta-1 Proteins 0.000 description 1
- 108091005461 Nucleic proteins Proteins 0.000 description 1
- 102000013814 Wnt Human genes 0.000 description 1
- 239000000980 acid dye Substances 0.000 description 1
- 230000004913 activation Effects 0.000 description 1
- 230000003044 adaptive effect Effects 0.000 description 1
- 229910052782 aluminium Inorganic materials 0.000 description 1
- -1 aluminum ions Chemical class 0.000 description 1
- 238000011511 automated evaluation Methods 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 230000001413 cellular effect Effects 0.000 description 1
- 230000008045 co-localization Effects 0.000 description 1
- 230000001086 cytosolic effect Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 239000011521 glass Substances 0.000 description 1
- 238000011532 immunohistochemical staining Methods 0.000 description 1
- 238000007689 inspection Methods 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 125000003588 lysine group Chemical group [H]N([H])C([H])([H])C([H])([H])C([H])([H])C([H])([H])C([H])(N([H])[H])C(*)=O 0.000 description 1
- 210000004962 mammalian cell Anatomy 0.000 description 1
- 230000035772 mutation Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 102000004169 proteins and genes Human genes 0.000 description 1
- 238000013077 scoring method Methods 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 230000002195 synergetic effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/11—Region-based segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/187—Segmentation; Edge detection involving region growing; involving region merging; involving connected component labelling
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
- G06V10/267—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion by performing operations on regions, e.g. growing, shrinking or watersheds
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/60—Type of objects
- G06V20/69—Microscopic objects, e.g. biological cells or cellular parts
- G06V20/695—Preprocessing, e.g. image segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/60—Type of objects
- G06V20/69—Microscopic objects, e.g. biological cells or cellular parts
- G06V20/698—Matching; Classification
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N1/00—Sampling; Preparing specimens for investigation
- G01N1/28—Preparing specimens for investigation including physical details of (bio-)chemical methods covered elsewhere, e.g. G01N33/50, C12Q
- G01N1/30—Staining; Impregnating ; Fixation; Dehydration; Multistep processes for preparing samples of tissue, cell or nucleic acid material and the like for analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10024—Color image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20036—Morphological image processing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30004—Biomedical image processing
- G06T2207/30024—Cell structures in vitro; Tissue sections in vitro
Definitions
- the present invention relates to image segmentation in digital pathology, and in particular to a device and an image processing method for detecting compartments of a biological cell.
- Digital pathology enables staining integrated with automated evaluation and analysis.
- the expression and colocalization in membrane, nucleus, and cytoplasm may be quantified. This may be done based on intensity and area of each biomarker of interest across cellular compartments.
- a device for detecting compartments of a biological cell.
- the device comprises an input unit and an image processing unit.
- the input unit is configured to receive at least one digital slide image for providing a first image depicting a biological specimen sample stained with a first stain and a second image depicting a biological specimen sample stained with a second stain of a biological specimen sample.
- the first stain is a stain for selectively staining nuclei in biological cells and the second stain is a stain for selectively staining a biomarker.
- the image processing unit is configured to detect a nucleus region of the biological cell by analyzing a first stain, and to detect a cytoplasm region and a membrane region of the biological cell based on an analysis of variations in stain intensities of the second stain, and a spatial relationship with at least one adjacent biological cell.
- the outlines of nucleus, cytoplasm, and membrane regions can be detected, in particular for the situation where the stain does not enable a clear discrimination of the cytoplasm and the membrane.
- the outlines may be overlaid on the original image to allow a visual inspection by a pathologist.
- the pathologist can relate summary statistics of nucleus, cytoplasm, and membrane IHC stain, e.g. 3,3'-Diaminobenzidine (DAB), levels to the image analysis results.
- DAB 3,3'-Diaminobenzidine
- the compartments of a biological cell can still be detected by deriving the transitions between cell compartments from the location of other cell compartments and the relative layout/position of other cells. This leads to an advantage of streamlining digital pathology workflow, thus resulting in an increased efficiency.
- digital slide image relates to a digital representation of a sample slice.
- the digital sample image may be also referred to as digital slide or digitized glass slide.
- Image data may be created from the sample slide using an image forming device like a scanner.
- a digital slide image may also be acquired from an image management system (IMS).
- IMS image management system
- two sample slides are provided including a first sample slide with the first stain and a second sample slide with the second stain.
- two digital slide images are received, each depicting differently stained biological specimen sample.
- the at least one digital slide image refers to the two digital slide images created from the two sample slides.
- the two digital slide images may also be referred to as the first image with the first stain and the second image with the second stain.
- a single sample slide is provided with the biological specimen sample stained with two stains, i.e. the first stain and the second stain.
- the digital slide image is a RGB dual stain colour image.
- Colour deconvolution may be used to split the original RGB dual stain colour image into the first image with the first stain and the second image with the second stain.
- the at least one digital slide image refers to the single digital slide image created from the single sample slide.
- the first stain is used to identify the nuclei in biological cells.
- the first stain may therefore also be referred to as a nuclear stain.
- the first stain may be an IHC
- Hematoxylin is a common nuclear counterstain in IHC. Oxidized hematoxylin is combined with aluminum ions to form an active metal-dye complex that stains the nuclei of mammalian cells blue by binding to lysine residues on nuclear histones, as opposed to other nuclear dyes that target the nucleic acids.
- Another example is Nuclear fast red, also called Kemechtrot dye, which is also a nuclear stain.
- Methyl green is also a nucleic acid dye that rapidly stains the nuclei green.
- the second stain may be an IHC stain.
- This may be e.g. a DAB stain, which has been used in IHC staining of nucleic acids and proteins.
- the IHC stain may be used for quantifying Beta-Catenin expression.
- spatial relationship relates to the relative layout/position between cells. As mentioned above, even if there is no (or less) discontinuities in staining intensities of the second stain, the compartments of a biological cell can still be detected by deriving the transitions between cell compartments from the location of other cell compartments and the relative layout/position of other cells, i.e. the spatial relationship.
- a region growing method is performed to detect the cytoplasm regions and the membrane regions of the biological cell and the at least one adjacent biological cell in parallel.
- the growing process may be carried out e.g. using a region-based image segmentation method.
- This approach examines neighboring pixels of initial seed points and determines whether the pixel neighbors should be added to the region.
- the regions are then grown from these seed regions to adjacent points depending on a region membership criterion.
- the image processing unit is further configured to perform a first growing process to grow a first area starting from the detected nucleus region to progressively form the cytoplasm region depending on a local variation in stain intensities of the second stain; and to determine the membrane region based on a result of the first growing process.
- the nucleus region detected in the first image is mapped into the second image in order to perform the first growing process.
- the image processing unit is configured to continue the first growing process until a criterion is met; and to define an inner part of the first grown area as the cytoplasm region.
- the criterion comprises: the first grown region touches the at least one adjacent biological cell, or a maximum distance from each nucleus boundary is reached.
- the image processing unit is configured to stop the first growing process and to define the first grown area as the cytoplasm region. In this way, the cytoplasm region can be identified not only for strong variations of staining intensities of the second stain but also for week variations, i.e. less (or no) discontinuities, in staining intensities of the second stain.
- the term "local variation” may refer to the absolute intensity difference between a candidate pixel and the average intensity of the seed region.
- the local variation may also be the absolute intensity difference between a candidate pixel and the running average intensity of the grown region.
- the local variation may also include the difference between the standard deviation in intensity over a specified local neighborhood of a candidate pixel.
- the first grown region is divided into the "inner part” and the "outer part” as mentioned below.
- the inner part forms the cytoplasm region and the outer part forms the membrane region.
- the outer part i.e. the membrane region
- a typical (or average) value of the membrane thickness of the biological cell to be analyzed may be selected as the fixed width.
- the image processing unit is configured to determine an outer part of the first grown region as the membrane region by applying a morphological erosion operation on the first grown region. Or, if the local variation in stain intensities of the second stain is above the predefined threshold, the image processing unit is configured to perform a second growing process to grow a second area to progressively form the membrane region; and to stop the second growing process until the second grown region touches the at least one adjacent biological cell or a maximum distance to the cytoplasm region has been reached; and to define the membrane region as the second grown area.
- the membrane region can be detected also for both situations, i.e. strong and weak variations in staining intensities.
- the maximum distance makes it possible for the detection without there being a single neighboring cell. This situation will e.g. be the case for a small number of cells since most cells have neighbors at close distance. In other words, this ensures that the membrane region does not grow unlimited when neighboring cells are absent.
- a region may be reduced in size using a morphological erosion operation. In case two cells start touching each other already in the first growing step (for cytoplasm), then it may be needed to erode both since there must be a membrane.
- the membrane region i.e. the outer part
- the fixed width may a typical (or average) value of the membrane thickness of the biological cell to be analyzed.
- the biological specimen sample is stained with the first and second stains.
- the image processing unit is configured to split the at least one digital slide image into a first digital image with a first stain and a second digital image with a second stain.
- the nucleus region is detected in the first digital image.
- the cytoplasm region and the membrane region are detected in the second digital image.
- the digital slide image may also be referred to as the original RGB dual stain color image.
- the original RGB dual stain color image may be split using color devolution.
- the device further comprises an output unit.
- the output unit is configured to output a detection result from the image processing unit.
- the output device is a display that allows a visual inspection by a pathologist.
- the detection result may be the segmentation map itself for each detected cell the nucleus, cytoplasm and membrane region.
- Stain statistics e.g. mean and variance
- an image processing method for detecting compartments of a biological cell comprising the following steps: a) receiving at least one digital slide image for providing a first image depicting a biological specimen sample stained with a first stain and a second image depicting a biological specimen sample stained with a second stain of a biological specimen sample; the first stain is a stain for selectively staining nuclei in biological cells and the second stain is a stain for selectively staining a biomarker;
- cytoplasm region and a membrane region of the biological cell based on an analysis of the detected nucleus region, variations in stain intensities of the second stain, and a spatial relationship with at least one adjacent biological cell.
- compartments of a biological cell in an image can be detected, even in the situation when the stain does not enable a clear discrimination of the cytoplasm and the membrane.
- step c) a region growing method is performed to detect the cytoplasm regions and the membrane regions of the biological cell and the at least one adjacent biological cell.
- the digital slide image is split into a first digital image with a first stain and a second digital image with a second stain.
- the nucleus region is detected in the first digital image.
- the cytoplasm region and the membrane region are detected in the second digital image.
- step c) further comprises:
- cl performing a first growing process to grow a first area starting from the detected nucleus region to progressively form the cytoplasm region depending on a local variation in stain intensities of the second stain;
- Step cl) may also be referred to as cytoplasm region identifying step; step c2) may also be referred to as membrane region identifying step.
- step cl) further comprises the following steps.
- cl 1) continuing the first growing process until a criterion is met; and cl2) defining an inner part of the first grown area as the cytoplasm region;
- cl4 defining the first grown area as the cytoplasm region.
- the criterion comprises:
- the first grown region touches the at least one adjacent biological cell; or a maximum distance from each nucleus boundary is reached.
- step c2) the following sub-steps are provided. If the local variation in stain intensities of the second stain is below the predefined threshold, the following sub-step is provided:
- a computer program element which, when being executed by a processing unit, is adapted to perform the method steps indicated above and below.
- a computer readable medium having stored the program element of the above example.
- a device and a method are provided to detect the compartments by detecting the nucleus region by analyzing a first stain in the image that highlights all nuclei and detect the cytoplasm and membrane regions based on an analysis of a variation of a second stain.
- the first stain may be an IHC (counter)stain, e.g. Hematoxylin.
- the second stain may be an IHC stain, e.g. a DAB stain.
- a nucleus area (e.g. using an image with an IHC counterstain) is firstly detected. Then, a first growing process is performed (e.g. using an IHC image)) starting from the detected nucleus to progressively form a cytoplasmic area. If the local variation of IHC (e.g. DAB) level is greater than a predefined threshold, one switches to a second growing process to progressively form a membrane area until contacting a membrane of another cell. If the local variation of DAB level is below the predefined threshold, stop the first process when contacting a membrane of another cell and define the membrane as a part of the cytoplasm area thus formed based on a geometric calculation.
- IHC e.g. DAB
- the solution relies on applying the method on at least two cells in parallel, in order to be able to detect a contact of membranes. In this way, even when the stain does not enable a clear discrimination of the cytoplasm, the compartments of cells can still be identified. A user can thus easily relate summary statistics of nucleus cytoplasm, and membrane (e.g. DAB) stain levels to the image analysis results.
- membrane e.g. DAB
- Fig. 1 shows a device for detecting compartments of a biological cell.
- Fig. 2 shows an example of a situation where the local variation in stain intensities of the second stain is below the predefined threshold.
- Fig. 3 shows an example of a situation where the local variation in stain intensities of the second stain is above the predefined threshold.
- Fig. 4a to Fig. 4c shows segmentation results for three different images from an observed IHC image.
- Fig. 5 shows basic steps of an example of a method.
- Fig. 6 shows a further example of the method.
- Fig. 7 shows another example of the method.
- Fig. 1 shows a device 10 for detecting compartments of a biological cell.
- the device comprises an input unit 12 and an image processing unit 14.
- the input unit 12 is configured to receive at least one digital slide image for providing a first image depicting a biological specimen sample stained with a first stain and a second image depicting a biological specimen sample stained with a second stain of a biological specimen sample.
- the first stain is a stain for selectively staining nuclei in biological cells and the second stain is a stain for selectively staining a biomarker.
- the image processing unit 14 is configured to detect a nucleus region of the biological cell by analyzing a first stain, and to detect a cytoplasm region and a membrane region of the biological cell based on an analysis of variations in stain intensities of the second stain, and a spatial relationship with at least one adjacent biological cell.
- Image data may be created from the sample slide using an image forming device like a scanner.
- a digital slide image may also be acquired from an image management system (IMS).
- IMS image management system
- two sample slides are used to provide two stains, and thus two digital slide images are received, each depicting differently stained biological specimen sample.
- a single sample slide is used to provide two stains, and thus only one digital slide image is received.
- the biological specimen sample is stained with the two stains, i.e. the first stain and the second stain
- the digital slide image is a RGB dual stain colour image.
- the image processing unit 14 is configured to split the at least one digital slide image into a first digital image with a first stain and a second digital image with a second stain.
- colour deconvolution may be used to split the original RGB dual stain colour image.
- the nucleus region is detected in the first digital image.
- the cytoplasm region and the membrane region are detected in the second digital image.
- the first stain may be an IHC counterstain, e.g.
- the second stain may be an IHC stain.
- the device 10 may comprises an output unit 16.
- the output unit 16 is configured to output a detection result from the image processing unit 14.
- the output unit 16 may be a display allowing a user inspection or interaction.
- the detection result may be the segmentation map itself with for each detected cell the nucleus, cytoplasm and membrane region.
- Stain statistics e.g. mean and variance
- the outlines of nucleus, cytoplasm, and membrane regions can be detected, even when the stain does not enable a clear discrimination of the cytoplasm.
- a region growing method is performed to detect the cytoplasm regions and the membrane regions of the biological cell and the at least one adjacent biological cell.
- the image processing unit 14 is further configured to perform a first growing process to grow a first area starting from the detected nucleus region to progressively form the cytoplasm region depending on a local variation in stain intensities of the second stain, and to determine the membrane region based on a result of the first growing process.
- Hematoxylin serves as an example of the first stain
- DAB stain serves as an example of the second stain.
- the first image is not illustrated since it is only used to determine the nucleus region of the biological cell.
- the second image is illustrated for explaining the identification of the cytoplasm and membrane regions.
- Fig. 2 shows an example of a situation 18 where the local variation in stain intensities of the second stain is below the predefined threshold in the second image.
- three cells 20a, 20b, 20c are illustrated that comprise respective nucleus regions 22a, 22b, 22c, respective cytoplasm regions 24a, 24b, 24c, and respective membrane regions 26a, 26b, 26c.
- the boundary between the cells is illustrated as a dotted line.
- the cytoplasm region 24a of the cell 20a contains DAB stain, whereas the cells 20a and 20b are not stained.
- the cytoplasm region 24a contains DAB stains, the DAB stain level does not change between cytoplasm region 24a and the membrane region 26a.
- the image processing unit 14 is configured to continue the first growing process until a criterion is met; and to define an inner part of the first grown area as the cytoplasm region.
- the criterion comprises: the first grown region touches the at least one adjacent biological cell; or a maximum distance from each nucleus boundary is reached.
- the image processing unit 14 is further configured to determine an outer part of the first grown area the membrane region by applying a morphological erosion operation on the first grown region.
- the first grown area starts growing outward from the nucleus boundary of the nucleus region 22a. It is noted that the growing process is also performed on at least two adjacent cells 20b, 20c in parallel. The growing process may stop when the predefined maximum distance from the nucleus from each nucleus boundary is reached. The growing process may also stop when a neighboring cytoplasm region (e.g. 24b, 24c) is touched.
- An inner part of the first grown area can thus be defined as a cytoplasm region 24a of the cell 20a.
- the outer part i.e. the membrane region 26a
- the fixed with may be an average value of the membrane thickness of the biological cell to be analyzed.
- the membrane region 26a can be determined by removing pixels from the current cytoplasm region 24a using morphological erosion.
- the boundary between two cells is determined purely by the geometric configuration of the nuclei and not by the changes in the DAB image. Since all pairs of regions touch, the membrane region for each cell is formed with a fixed width.
- Fig. 3 shows an example of a situation 28 where the local variation in stain intensities of the second stain is above the predefined threshold in the second image.
- the membrane region 26a of the cell 20a is more heavily (darker) stained (indicated with a different pattern from the cytoplasm region 24a) with DAB than the cytoplasm region 24a.
- cytoplasm regions and membrane regions for the cell 20b and 20c can be determined according to Fig. 2.
- the image processing unit 14 is configured to stop the first growing process, and to define the first grown area as the cytoplasm region, i.e. the cytoplasm region 24a.
- the image processing unit 14 is configured to perform a second growing process to grow a second area to progressively form the membrane region, to stop the second growing process until the second grown region touches the at least one adjacent biological cell or a maximum distance to the cytoplasm region has been reached, and to define the membrane region as the second grown area.
- the first grown area starts growing outward from the nucleus boundary and stops growing when the local change in DAB is greater than a predefined threshold.
- the first grown area is defined as the cytoplasm region 24a of the cell 20a.
- a second growing step is now only applied to the cell 20a to determine its outer membrane region.
- the second growing step stops when a neighboring cell (20b, 20c) is touched.
- the second grown area is defined as the membrane region 26a of the cell 20a.
- Fig. 4a to Fig. 4c shows segmentation result for three different images from an observed IHC image.
- the boundary between the cytoplasm region and the membrane region in each cell is illustrated as a dotted line.
- the boundary between the cells and the boundary of the nucleus regions are illustrated as solid lines.
- Fig. 4a shows segmentation of sub-cellular components when DAB staining is absent.
- the segmentation can be performed according to Fig. 2.
- the image processing unit 14 is configured to continue the first growing process until a criterion is met; and to define the first grown area as the cytoplasm region.
- the criterion comprises: the first grown region touches the at least one adjacent biological cell; or a maximum distance from each nucleus boundary is reached.
- the image processing unit 14 is further configured to determine the membrane region based on a morphological erosion operation.
- Fig. 4b shows segmentation of sub-cellular components, which are heavily stained with DAB, and thus the DAB level does not change from the cytoplasm region to the membrane region.
- segmentation can also be performed according to Fig. 2.
- Fig. 4c shows segmentation of sub-cellular components, which show a clear difference in DAB intensity between the cytoplasm region and the membrane region.
- segmentation can be performed according to Fig. 3.
- Fig. 5 shows basic steps of a method 100 of detecting compartments of a biological cell.
- a first step 102 also referred to as step a
- at least one digital slide image is received for providing a first image depicting a biological specimen sample stained with a first stain and a second image depicting a biological specimen sample stained with a second stain of a biological specimen sample.
- the first stain is a stain for selectively staining nuclei in biological cells and the second stain is a stain for selectively staining a biomarker.
- a nucleus region of the biological cell is detected based on an analysis of the first stain.
- a third step 106 also referred to as step c)
- a cytoplasm region and a membrane region of the biological cell are detected based on an analysis of the detected nucleus region, variations in stain intensities of the second stain, and a spatial relationship with at least one adjacent biological cell.
- step c) i.e. in step 106, a region growing method is performed to detect the cytoplasm regions and the membrane regions of the biological cell and the at least one adjacent biological cell.
- the biological specimen sample is stained with the first and second stains.
- step b) i.e. step 104
- the digital slide image is split into a first digital image with a first stain and a second digital image with a second stain.
- step b) i.e. in step 104
- the nucleus region is detected in the first digital image.
- step c) i.e. in step 106
- the cytoplasm region and the membrane region are detected in the second digital image.
- step c) further comprises the step of cl) performing 108 a first growing process to grow a first area starting from the detected nucleus region to progressively form the cytoplasm region depending on a local variation in stain intensities of the second stain, and of c2) determining 110 the membrane region based on a result of the first growing process.
- Fig. 6 shows a further example of the method 100, in which step cl), i.e. step 108, further comprises the following steps.
- cl 1) continuing 112 the first growing process until a criterion is met, and cl2) defining 114 an inner part of the first grown area as the cytoplasm region.
- the criterion comprises: the first grown region touches the at least one adjacent biological cell, or a maximum distance from each nucleus boundary is reached.
- Fig. 7 shows a further example of the method 100, in which step c2), i.e. step 110, further comprises the following steps.
- c21 determining 120 an outer part of the first grown region as the membrane region by applying a morphological erosion operation on the cytoplasm region of the first grown region.
- a computer program or a computer program element is provided that is characterized by being adapted to execute the method steps of the method according to one of the preceding embodiments, on an appropriate system.
- the computer program element might therefore be stored on a computer unit, which might also be part of an embodiment of the present invention.
- This computing unit may be adapted to perform or induce a performing of the steps of the method described above. Moreover, it may be adapted to operate the components of the above described apparatus.
- the computing unit can be adapted to operate automatically and/or to execute the orders of a user.
- a computer program may be loaded into a working memory of a data processor.
- the data processor may thus be equipped to carry out the method of the invention.
- This exemplary embodiment of the invention covers both, a computer program that right from the beginning uses the invention and a computer program that by means of an up-date turns an existing program into a program that uses the invention.
- the computer program element might be able to provide all necessary steps to fulfil the procedure of an exemplary embodiment of the method as described above.
- a computer readable medium such as a CD-ROM
- the computer readable medium has a computer program element stored on it which computer program element is described by the preceding section.
- a computer program may be stored and/or distributed on a suitable medium, such as an optical storage medium or a solid state medium supplied together with or as part of other hardware, but may also be distributed in other forms, such as via the internet or other wired or wireless telecommunication systems.
- a suitable medium such as an optical storage medium or a solid state medium supplied together with or as part of other hardware, but may also be distributed in other forms, such as via the internet or other wired or wireless telecommunication systems.
- the computer program may also be presented over a network like the World Wide Web and can be downloaded into the working memory of a data processor from such a network.
- a medium for making a computer program element available for downloading is provided, which computer program element is arranged to perform a method according to one of the previously described embodiments of the invention.
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Life Sciences & Earth Sciences (AREA)
- Biomedical Technology (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Health & Medical Sciences (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Investigating Or Analysing Biological Materials (AREA)
- Image Processing (AREA)
Abstract
Description
Claims
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
EP17152970 | 2017-01-25 | ||
PCT/EP2018/051771 WO2018138180A1 (en) | 2017-01-25 | 2018-01-25 | Image segmentation in digital pathology |
Publications (1)
Publication Number | Publication Date |
---|---|
EP3574445A1 true EP3574445A1 (en) | 2019-12-04 |
Family
ID=57909469
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
EP18701181.2A Withdrawn EP3574445A1 (en) | 2017-01-25 | 2018-01-25 | Image segmentation in digital pathology |
Country Status (2)
Country | Link |
---|---|
EP (1) | EP3574445A1 (en) |
WO (1) | WO2018138180A1 (en) |
Families Citing this family (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
EP3789960A1 (en) * | 2019-09-05 | 2021-03-10 | Leica Biosystems Imaging, Inc. | User-assisted iteration of cell image segmentation |
Family Cites Families (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
AT411066B (en) * | 2000-10-24 | 2003-09-25 | Steiner Georg E | METHOD AND ARRANGEMENT FOR THE INVESTIGATION OF CELLS |
US9042630B2 (en) * | 2011-10-26 | 2015-05-26 | Definiens Ag | Biomarker evaluation through image analysis |
CA2935473C (en) * | 2014-02-21 | 2022-10-11 | Ventana Medical Systems, Inc. | Medical image analysis for identifying biomarker-positive tumor cells |
-
2018
- 2018-01-25 WO PCT/EP2018/051771 patent/WO2018138180A1/en unknown
- 2018-01-25 EP EP18701181.2A patent/EP3574445A1/en not_active Withdrawn
Also Published As
Publication number | Publication date |
---|---|
WO2018138180A1 (en) | 2018-08-02 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
EP3486836B1 (en) | Image analysis method, apparatus, program, and learned deep learning algorithm | |
CA2746743C (en) | Multi-nucleated cell classification and micronuclei scoring | |
EP3486835A1 (en) | Image analysis method, image analysis apparatus, program, learned deep layer learning algorithm manufacturing method and learned deep layer learning algorithm | |
Gyori et al. | OpenComet: an automated tool for comet assay image analysis | |
US9934571B2 (en) | Image processing device, program, image processing method, computer-readable medium, and image processing system | |
US11854196B2 (en) | System and method for real time assay monitoring | |
EP2610813B1 (en) | Biomarker evaluation through image analysis | |
US11538261B2 (en) | Systems and methods for automated cell segmentation and labeling in immunofluorescence microscopy | |
US11620751B2 (en) | Image analysis algorithms using control slides | |
US10699423B2 (en) | Registration of tissue slice image | |
JP7214756B2 (en) | Quantification of signal in stain aggregates | |
WO2014192184A1 (en) | Image processing device, image processing method, program, and storage medium | |
EP3532984B1 (en) | Apparatus for determining cellular composition information in one or more tissue samples | |
EP3574445A1 (en) | Image segmentation in digital pathology | |
US20150260973A1 (en) | Focus determination apparatus, focus determination method, and imaging apparatus | |
CN115100648B (en) | Method and device for filtering multi-channel fluorescent impurities in cell microscopic image | |
Foucart et al. | Finding the best channel for tissue segmentation in whole-slide images | |
CN108776146B (en) | Method and device for detecting silk-screen lines | |
JPWO2016076104A1 (en) | Image processing method, image processing apparatus, and program | |
Swiderska et al. | Texture and mathematical morphology for hot-spot detection in whole slide images of meningiomas and oligodendrogliomas | |
CN110009616B (en) | Punctate calcification detection method | |
El-Labban et al. | Mitotic phase based detection of chromosome segregation errors in embryonic stem cells |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: UNKNOWN |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: THE INTERNATIONAL PUBLICATION HAS BEEN MADE |
|
PUAI | Public reference made under article 153(3) epc to a published international application that has entered the european phase |
Free format text: ORIGINAL CODE: 0009012 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: REQUEST FOR EXAMINATION WAS MADE |
|
17P | Request for examination filed |
Effective date: 20190826 |
|
AK | Designated contracting states |
Kind code of ref document: A1 Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR |
|
AX | Request for extension of the european patent |
Extension state: BA ME |
|
RAP1 | Party data changed (applicant data changed or rights of an application transferred) |
Owner name: KONINKLIJKE PHILIPS N.V. |
|
DAV | Request for validation of the european patent (deleted) | ||
DAX | Request for extension of the european patent (deleted) | ||
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: THE APPLICATION HAS BEEN WITHDRAWN |
|
18W | Application withdrawn |
Effective date: 20200506 |