WO2013105373A1 - 情報処理装置、撮像制御方法、プログラム、デジタル顕微鏡システム、表示制御装置、表示制御方法及びプログラム - Google Patents
情報処理装置、撮像制御方法、プログラム、デジタル顕微鏡システム、表示制御装置、表示制御方法及びプログラム Download PDFInfo
- Publication number
- WO2013105373A1 WO2013105373A1 PCT/JP2012/081801 JP2012081801W WO2013105373A1 WO 2013105373 A1 WO2013105373 A1 WO 2013105373A1 JP 2012081801 W JP2012081801 W JP 2012081801W WO 2013105373 A1 WO2013105373 A1 WO 2013105373A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- imaging
- failure
- image
- detection unit
- information processing
- Prior art date
Links
- 238000003384 imaging method Methods 0.000 title claims abstract description 472
- 230000010365 information processing Effects 0.000 title claims abstract description 46
- 238000000034 method Methods 0.000 title claims description 54
- 238000001514 detection method Methods 0.000 claims abstract description 143
- 239000002131 composite material Substances 0.000 claims description 84
- 239000000523 sample Substances 0.000 description 47
- 230000008569 process Effects 0.000 description 42
- 238000004891 communication Methods 0.000 description 41
- 238000012545 processing Methods 0.000 description 24
- 238000012854 evaluation process Methods 0.000 description 22
- 230000007547 defect Effects 0.000 description 19
- 238000010586 diagram Methods 0.000 description 16
- 230000002093 peripheral effect Effects 0.000 description 16
- 239000012472 biological sample Substances 0.000 description 14
- 238000003860 storage Methods 0.000 description 12
- 230000008859 change Effects 0.000 description 8
- 238000005286 illumination Methods 0.000 description 8
- 230000004044 response Effects 0.000 description 8
- 238000011156 evaluation Methods 0.000 description 7
- 238000012552 review Methods 0.000 description 5
- 238000012790 confirmation Methods 0.000 description 4
- 238000010827 pathological analysis Methods 0.000 description 4
- 238000002360 preparation method Methods 0.000 description 4
- 238000004904 shortening Methods 0.000 description 4
- 238000012937 correction Methods 0.000 description 3
- 238000004590 computer program Methods 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 238000003708 edge detection Methods 0.000 description 2
- 238000003475 lamination Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 1
- 230000000295 complement effect Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000007257 malfunction Effects 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 229910044991 metal oxide Inorganic materials 0.000 description 1
- 150000004706 metal oxides Chemical class 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 230000002194 synthesizing effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B21/00—Microscopes
- G02B21/36—Microscopes arranged for photographic purposes or projection purposes or digital imaging or video purposes including associated control and data processing arrangements
- G02B21/365—Control or image processing arrangements for digital or video microscopes
- G02B21/367—Control or image processing arrangements for digital or video microscopes providing an output produced by processing a plurality of individual source images, e.g. image tiling, montage, composite images, depth sectioning, image comparison
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B21/00—Microscopes
- G02B21/36—Microscopes arranged for photographic purposes or projection purposes or digital imaging or video purposes including associated control and data processing arrangements
- G02B21/365—Control or image processing arrangements for digital or video microscopes
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
- G06T3/4038—Image mosaicing, e.g. composing plane images from plane sub-images
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/50—Image enhancement or restoration using two or more images, e.g. averaging or subtraction
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/77—Retouching; Inpainting; Scratch removal
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/30—Determination of transform parameters for the alignment of images, i.e. image registration
- G06T7/33—Determination of transform parameters for the alignment of images, i.e. image registration using feature-based methods
- G06T7/337—Determination of transform parameters for the alignment of images, i.e. image registration using feature-based methods involving reference images or patches
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10056—Microscopic image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20212—Image combination
- G06T2207/20221—Image fusion; Image merging
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30004—Biomedical image processing
- G06T2207/30024—Cell structures in vitro; Tissue sections in vitro
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30168—Image quality inspection
Definitions
- the present disclosure relates to an information processing device, an imaging control method, a program, a digital microscope system, a display control device, a display control method, and a program.
- Patent Document 1 discloses a technique for generating high-resolution composite image data by capturing a partial image of a biological sample and synthesizing the partial images.
- an information processing apparatus including a generation unit that generates setting information for setting an imaging condition at the time of re-imaging.
- an imaging control method including generating setting information for setting imaging conditions at the time of reimaging is provided.
- the computer evaluates an image captured using a digital microscope, thereby detecting a failure that requires re-imaging related to the image, and the detection unit detects the failure.
- a program for functioning as a generation unit that generates setting information for setting an imaging condition at the time of re-imaging when detected is provided.
- a digital microscope and a detection unit that detects a failure that requires re-imaging related to the image by evaluating an image captured using the digital microscope, and the detection unit
- a digital microscope system including an information processing apparatus including a generation unit that generates setting information for setting an imaging condition at the time of reimaging when a failure is detected.
- a display control apparatus includes an acquisition unit that acquires setting information for setting, and a display control unit that displays the setting information acquired by the acquisition unit on a display surface.
- the imaging control method, the program, the digital microscope system, the display control device, the display control method, and the program according to the present disclosure it is possible to perform re-imaging with less effort and time. Become.
- FIG. 1 is a block diagram illustrating an example of a configuration of a scanner according to a first embodiment. It is explanatory drawing for demonstrating an example of the imaging by a digital microscope. It is explanatory drawing for demonstrating an example of the bonding of an image. It is explanatory drawing for demonstrating an example of imaging omission. It is explanatory drawing for demonstrating an example of the detection of an imaging omission. It is explanatory drawing for demonstrating an example of the edge which appears in the bonding image at the time of a bonding failure. It is a block diagram which shows an example of a structure of the server which concerns on 1st Embodiment.
- FIG. 1 is an explanatory diagram illustrating an example of a schematic configuration of a digital microscope system 1 according to an embodiment of the present disclosure.
- the digital microscope system 1 includes a scanner 100, a server 200, and a viewer 300.
- the scanner 100 is an information processing apparatus having a digital microscope or connected to the digital microscope.
- the scanner 100 captures an image of a sample using a digital microscope.
- a digital microscope is loaded with a sample preparation.
- the scanner 100 determines an imaging condition, and sets the determined imaging condition in the digital microscope.
- the scanner 100 captures an image of the preparation sample using a digital microscope. More specifically, the scanner 100 determines a sample imaging area as an imaging condition, and sets the determined imaging area in the digital microscope. Further, the scanner 100 divides the imaging area into a plurality of individual areas, determines the imaging order of the plurality of individual areas as imaging conditions, and sets the determined imaging order in the digital microscope.
- the scanner 100 causes the digital microscope to capture an image of the sample in each individual area according to the set imaging order. Further, the scanner 100 generates a composite image by combining the captured images of the individual regions as partial images. Thus, in the scanner 100, an image of one sample, that is, a combined image is generated.
- the server 200 is an information processing apparatus that manages sample images generated by the scanner 100.
- the server 200 stores the composite image generated by the scanner 100 in the database in association with the sample identification information.
- the viewer 300 is an example of a display control device.
- the viewer 300 displays a sample combined image generated by the scanner 100 or stored by the server 200 on the display surface.
- the viewer 300 provides a user interface for enabling the imaging conditions of the scanner 100 to be specified by a user operation. That is, the user can specify the imaging condition of the scanner 100 through an operation on the viewer 300.
- the digital microscope system 1 can perform re-imaging with less effort and time. Thereafter, ⁇ 2. First Embodiment> and ⁇ 3. In the second embodiment, the specific contents will be described.
- the scanner 100 automatically detects a failure that requires re-imaging, and performs re-imaging according to the failure.
- FIG. 2 is a block diagram illustrating an example of the configuration of the scanner 100-1 according to the first embodiment.
- the scanner 100-1 includes an imaging control unit 110, a digital microscope 120, a bonding unit 130, a failure detection unit 140, a setting information generation unit 150, and a communication unit 160.
- Imaging control unit 110 controls imaging using the digital microscope 120.
- the imaging control unit 110 sets imaging conditions designated by the user or automatically determined in the digital microscope 120, and causes the digital microscope 120 to capture an image of the sample according to the imaging conditions.
- the imaging control unit 110 sets an imaging area, an imaging order, illumination brightness, white balance coefficient, and the like as imaging conditions for the entire sample.
- the imaging control unit 110 sets a focus position as an imaging condition for each individual area, for example.
- the imaging control unit 110 sets at least a part of the imaging conditions based on the setting information to the digital microscope 120. Reset to. Then, the imaging control unit 110 causes the digital microscope 120 to re-image the sample image according to the reset imaging conditions.
- the imaging control unit 110 outputs information on the imaging conditions that have been set or reset (hereinafter referred to as “imaging condition information”) to the communication unit 160.
- the digital microscope 120 captures an image according to the image capturing conditions set by the image capturing control unit 110.
- the digital microscope 120 captures an image (hereinafter, referred to as “individual image”) that reflects each individual area within the set imaging area in accordance with the set imaging order.
- individual image an image that reflects each individual area within the set imaging area in accordance with the set imaging order.
- FIG. 3 is an explanatory diagram for explaining an example of imaging by the digital microscope 120.
- an imaging region 20 including the entire biological sample 10 is set.
- the imaging area 20 is divided into a plurality of individual areas 21.
- the imaging order of the individual regions 21 may be an arbitrary order such as a spiral order from the center of the biological sample 10 toward the outside or a zigzag order in which each row is traced alternately from top to bottom and from bottom to top.
- the digital microscope 120 captures an individual image showing each individual region 21 according to the imaging order set in this way.
- the digital microscope 120 captures an image of the expanded area 23 including the individual area 21 as an individual image so that the adjacent individual images can be pasted together.
- the digital microscope 120 outputs the captured individual images to the pasting unit 130 and the failure detecting unit 140.
- the digital microscope 120 includes, for example, an optical system, an image sensor, an image processing circuit, and a drive circuit.
- the optical system can include, for example, an objective lens.
- the imaging element is, for example, a CMOS (Complementary Metal Oxide Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.
- the image processing circuit performs development processing such as demosaic processing and white balance correction.
- the drive circuit moves the relative positions of the objective lens and the sample each time an individual image is captured according to the set imaging region and imaging order.
- the combining unit 130 generates a combined image by combining the individual images captured by the digital microscope 120.
- This point will be described more specifically with reference to FIG.
- FIG. 4 is an explanatory diagram for explaining an example of image combining.
- individual images 30a and 30b captured in adjacent individual regions 21 are shown.
- the individual image 30 is an image of the region 23 including the individual region 21. Therefore, the individual image 30 includes a portion 31 corresponding to the individual region 21 and a portion (hereinafter referred to as “overlapping portion”) 33 that overlaps each other.
- the pasting unit 130 pastes the individual image 30b onto the individual image 30a by appropriately overlapping the overlapping portion 33b with the overlapping portion 33a.
- the bonding unit 130 superimposes the overlapping portion 33b on the overlapping portion 33a using, for example, the luminance value of the overlapping portion 33a and the luminance value of the overlapping portion 33b.
- the pasting unit 130 for example, superimposes the overlapping portion 33b on the overlapping portion 33a, calculates a difference in luminance value between each pixel of the overlapping portion 33a and the corresponding overlapping portion 33b, The absolute value of the difference is summed over a plurality of pixels.
- the pasting unit 130 calculates the total value of the absolute values of the differences in luminance values for each position while changing the position of the overlapping portion 33b.
- the bonding unit 130 determines the position of the overlapping portion 33b when the minimum total value is calculated as the optimum position of the overlapping portion 33b at the time of bonding. In this way, the combining unit 130 sequentially combines the individual images of the adjacent individual regions 21, and finally generates one combined image of the entire sample.
- the combining unit 130 outputs the generated combined image to the communication unit 160.
- the combining unit 130 outputs, for example, the minimum total value for combining individual images to the failure detection unit 140. Note that the combining unit 130 may also output the generated combined image to the failure detection unit 140.
- the failure detection unit 140 evaluates an image captured using the digital microscope 120 to detect a failure requiring re-imaging related to the image.
- “evaluating an image” includes evaluating each individual image, and evaluating a composite image composed of the individual images.
- detection of failure by the failure detection unit 140 will be described using first to fifth examples of specific failure detection.
- the failure detection unit 140 detects a failure related to a composite image generated by combining a plurality of individual images as the failure.
- the failure detection unit 140 detects an imaging omission of an area that should be included in the composite image as a failure related to the composite image.
- this point will be described more specifically with reference to FIGS. 5A and 5B.
- FIG. 5A is an explanatory diagram for explaining an example of imaging omission.
- a biological sample 10 an imaging region 20, a plurality of individual regions 21, and an imaging region of an individual image that includes the individual region 21 are shown.
- the imaging region 20 includes the entire dark-colored portion 11 of the biological sample 10, but cannot include the light-colored portion 13 of the biological sample 10. That is, a part of the sample 10 is not captured although it is an area to be included in the composite image.
- the imaging region is determined by detecting the edge of the biological sample using the entire biological sample image, the light-colored portion 13 may leak from the imaging region.
- it may be difficult to accurately use the sample for example, pathological diagnosis).
- the failure detection unit 140 detects an imaging omission of a region to be included in the composite image by evaluating the direction in which the texture exists in the image. More specifically, the failure detection unit 140 evaluates, for example, the texture existing direction in the individual image of the region located in the peripheral portion of the imaging region 20, that is, the individual image constituting the peripheral portion of the combined image.
- the failure detection unit 140 evaluates, for example, the texture existing direction in the individual image of the region located in the peripheral portion of the imaging region 20, that is, the individual image constituting the peripheral portion of the combined image.
- FIG. 5B is an explanatory diagram for explaining an example of detection of imaging omission.
- one individual image 30 constituting the peripheral portion 35 of the composite image is shown. Since the individual image 30 is an image of the region 23 including the individual region 21, the individual image 30 includes a protruding portion 37 that protrudes from the composite image. In other words, the protruding portion 37 reflects an area outside the imaging area 20. Therefore, if any texture exists in the protruding portion 37 of the individual image 30, it can be said that there is a sample to be imaged also in the direction of the protruding portion 37 leaking from the imaging region 20.
- the failure detection unit 140 can detect whether there is an imaging omission by determining whether a texture exists in the protruding portion 37 of the individual image 30 as an evaluation of the texture presence direction. .
- the failure detection unit 140 detects an edge in the protrusion unit 37 using an edge detection filter, calculates the amount of the detected edge, and determines whether the amount of the edge exceeds a predetermined threshold value. By doing so, it can be determined whether or not a texture exists in the protruding portion 37.
- the failure detection unit 140 may detect an imaging omission using a composite image instead of an individual image. In this case, the failure detection unit 140 detects the imaging omission by evaluating the texture direction in the individual images included in the composite image. Since the combined image does not include the protruding portion 37 of the individual image 30, the failure detection unit 140 determines whether a texture exists in the peripheral portion 35 instead of the protruding portion 37 of the individual image 30, for example. Determine. If any texture exists in the peripheral portion 35, it can be estimated that there is a high possibility that there is a sample to be imaged outside the imaging region 20 in the direction corresponding to the peripheral portion 35.
- the failure detection unit 140 can detect whether there is an imaging omission by determining whether a texture exists in the peripheral portion 35 as an evaluation of the texture presence direction.
- the failure detection unit 140 may determine the direction of the edge of the peripheral portion 35 instead of determining whether or not a texture exists in the peripheral portion 35 or in combination with the determination. That is, if the edge portion 35 has an edge in the outer direction of the composite image (for example, a vertical edge in FIG. 5B), it can be estimated that the edge extends to the protruding portion 37. It can be estimated that there is a high possibility that there is a sample to be imaged outside the imaging region 20 in the direction in which the image is to be captured.
- the failure detection unit 140 detects an imaging omission of an area that should be included in a composite image by evaluating the direction of texture presence in individual images, for example.
- the failure detection unit 140 detects a failure in combining when generating the composite image as a failure related to the composite image.
- a combined image is generated by combining the captured individual images, but the individual images may not be appropriately combined. As a result, an unnatural shift occurs in the whole or part of the combined image, and the sample may not be confirmed accurately.
- the failure detection unit 140 detects the above-mentioned combining failure by evaluating the luminance value of the overlapping portion when the combined image is generated.
- the pasting unit 130 between each pixel of the overlapping portion 33 a and the corresponding pixel of the overlapping portion 33 b, The difference between the luminance values is calculated, and the absolute values of the differences over a plurality of pixels are summed.
- the bonding unit 130 determines the position of the overlapping portion 33b when the minimum total value is calculated as the optimum position of the overlapping portion 33b at the time of bonding.
- the failure detection unit 140 can detect a bonding failure by determining whether or not the minimum total value exceeds a predetermined threshold as an evaluation of the luminance value of the overlapping portion. The failure detection unit 140 acquires the minimum total value from the pasting unit 130.
- the failure detection unit 140 may detect the above-mentioned combining failure by evaluating an edge in a predetermined direction appearing in the combined image.
- this point will be described more specifically with reference to FIG.
- FIG. 6 is an explanatory diagram for explaining an example of an edge appearing in the composite image when the composite fails.
- a composite image 40 is shown. If the combining at the time of generating the combined image 40 fails, the individual images are shifted from each other at the boundary of the combining, so that an edge appears at the boundary of the combining. For example, horizontal edges 41a and vertical edges 41b and 41c appear. Therefore, the failure detection unit 140 detects, for example, an edge in a predetermined direction (for example, the horizontal direction and the vertical direction) that appears in the composite image, and evaluates the length or strength of the detected edge.
- a predetermined direction for example, the horizontal direction and the vertical direction
- the edge when the length of the detected edge is larger than a predetermined ratio of the length of one side of the individual area 21, the edge can be estimated to be an edge generated as a result of the bonding failure.
- the edge even when the detected edge has a strength that cannot be assumed as a unique edge of the sample, the edge can be estimated to be an edge that has occurred as a result of the bonding failure. In such a case, the failure detection unit 140 detects the bonding failure.
- the failure detection unit 140 detects a bonding failure by evaluating, for example, the luminance value of the overlapping portion or the edge in a predetermined direction appearing in the combined image.
- the failure detection unit 140 detects a defect in the focus position as a failure requiring re-imaging related to the individual image or the combined image.
- the imaging control unit 110 sets the focus position to the digital microscope 120, but an individual image with blur may be captured due to an incorrect focus position setting. As a result, it can be difficult to accurately confirm the sample.
- the failure detection unit 140 detects a defect in the focus position by evaluating the contrast of the individual image. An image with blur is low in contrast. Therefore, the failure detection unit 140 can detect a defect in the focus position by calculating the contrast of the individual image and determining whether the calculated contrast exceeds a predetermined threshold. Any method may be used as a contrast calculation method. As an example, the failure detection unit 140 calculates, for each pixel of the individual image, a difference in luminance value from each of the four adjacent pixels, and sums the absolute values of the differences. Then, the failure detection unit 140 further sums up the total value calculated for each pixel in the entire individual image. For example, the total value calculated in this way is used as the contrast of the individual image. As described above, the failure detection unit 140 detects a defect in the focus position.
- the failure detection unit 140 detects a white balance defect as a failure requiring re-imaging related to an individual image or a composite image.
- white balance correction is performed in the development processing of the individual image.
- the color of the individual image may be developed to a color different from the actual sample due to an erroneous white balance. As a result, it can be difficult to accurately confirm the color of the sample.
- the digital microscope 120 captures an image of an area where no sample exists (that is, an image in which nothing is reflected). Then, the failure detection unit 140 compares the color information of the image with the color information prepared in advance as a template. The failure detection unit 140 detects a white balance defect when the difference between the two is larger than a predetermined threshold. In this way, the failure detection unit 140 detects a white balance defect.
- the failure detection unit 140 detects a brightness defect as a failure requiring re-imaging related to an individual image or a composite image.
- the brightness of the illumination is adjusted at the time of imaging, but an individual image that is too bright or an individual image that is too dark may be captured due to incorrect brightness. As a result, it can be difficult to accurately confirm the sample.
- the digital microscope 120 captures an image of an area where no sample exists (that is, an image in which nothing is reflected). Then, the failure detection unit 140 compares the luminance information of the image with the luminance information prepared in advance as a template. And the failure detection part 140 detects the malfunction of a brightness, when both difference is larger than a predetermined threshold value. In this way, the failure detection unit 140 detects a brightness defect.
- failure detection by the failure detection unit 140 has been described above using the first to fifth examples of specific failure detection. Such failure detection makes it possible to automatically detect a failure that requires re-imaging without allowing the user to check the image on the screen.
- the failure detection unit 140 evaluates each image for detection of a failure that requires re-imaging each time a plurality of individual images are captured. By such evaluation, failure can be detected at the time when the individual image is captured, so that it is possible to perform re-imaging immediately after the failure is detected. That is, it is not necessary to re-input the sample into the digital microscope 120 for re-imaging, or to change the setting for items that do not need to be reset. Therefore, in this case, the effort required for re-imaging is almost eliminated, and the time required for re-imaging is extremely reduced.
- the failure detection unit 140 outputs information on the detected failure to the setting information generation unit 150.
- Setting information generation unit 150 When the failure detection unit 140 detects a failure, the setting information generation unit 150 generates setting information for setting an imaging condition for re-imaging. Hereinafter, generation of setting information by the setting information generation unit 150 will be described using the first to fifth examples of failure detection.
- the setting information generation unit 150 When the failure detection unit 140 detects the imaging omission, the setting information generation unit 150 generates imaging region information for resetting the imaging region so as to include an additional region to be newly imaged, for example. .
- the imaging area information is one piece of setting information. More specifically, as described with reference to FIGS. 5A and 5B, the failure detection unit 140 determines which individual region 21 (or individual image 30) when evaluating the texture direction in the individual image. It is known in which direction (for example, left, right, up, down) there is an imaging leak.
- the setting information generation unit 150 acquires a combination of the coordinates of the individual area 21 and the direction of the imaging omission as information related to the imaging omission from the failure detection unit 140, and generates information including the combination as the imaging area information. . With such imaging area information, the imaging control unit 110 can reset the imaging area so as to include an imaging omission area. As a result, the same imaging omission can be avoided during re-imaging.
- the imaging area information may include information that simply instructs to enlarge the imaging area.
- the imaging control unit 110 changes the parameters for determining the imaging region so that the imaging region becomes wider.
- the imaging control unit 110 determines the imaging region by detecting the edge of the sample appearing in the entire sample image. In this case, the imaging control unit 110 lowers the threshold for edge detection. As a result, weaker edges are also detected, so that a wider imaging area is determined and reset in the digital microscope 120.
- the imaging control unit 110 divides the entire sample image into blocks, calculates a variance of luminance values for each block, and determines a block area with variance exceeding the threshold as an imaging area. . In this case, the imaging control unit 110 lowers the threshold value for comparison with the variance. As a result, a larger block area is determined as the imaging area, so that a wider imaging area is reset in the digital microscope 120.
- the setting information generation unit 150 further generates imaging order information for resetting the imaging order of the individual images.
- the imaging order information is one piece of setting information.
- the imaging order information includes, for example, information that instructs to newly set the imaging order.
- individual images of the individual regions 21 adjacent to each other are continuously captured. In this way, by sequentially capturing adjacent individual images and sequentially combining them, the possibility that the combining fails will be reduced.
- the re-imaging when the individual images are first imaged in the original imaging order and the individual images of the enlarged area are additionally imaged, the individual image of the enlarged area is different from the individual image of the original imaging area. The images are taken discontinuously. Therefore, there is a high possibility that the bonding will fail. Therefore, when resetting the imaging region, it is possible to avoid the failure of the pasting during the re-imaging by further generating the imaging sequence information and resetting the imaging sequence.
- the setting information generation unit 150 generates imaging order information for resetting the imaging order of the individual images when the failure detection unit 140 detects the above-described combining failure.
- the imaging order information includes, for example, information instructing to set an imaging order that is different from the imaging order when a bonding failure is detected.
- the different imaging order here may be an imaging order in which the imaging start position is different, or may be an imaging order in which scan patterns (such as a spiral order or a zigzag order) are different. The occurrence of the bonding failure largely depends on what imaging order is set. As described with reference to FIG.
- the individual images 30 are combined using the luminance value of the overlapping portion 33.
- the occurrence of the bonding failure may depend on the imaging order. Therefore, when a bonding failure occurs, the possibility of a bonding failure occurring can be further reduced by performing re-imaging in another imaging order.
- the imaging order information may include information indicating the imaging order itself when a bonding failure is detected.
- the setting information generation unit 150 generates focus position information for resetting the focus position when a defect in the focus position is detected.
- the focus position information is one piece of setting information.
- the focus position information is information including, for example, the coordinates of the individual area corresponding to the individual image in which the defect in the focus position is detected and the set focus position. With such focus position information, the imaging control unit 110 can change the focus position particularly in an individual area with a focus position defect. As a result, it is possible to avoid a similar focus position problem during re-imaging.
- the imaging control unit 110 when determining the focus position in each individual area in the imaging area, the imaging control unit 110, for example, independently measures the focus position in several individual areas and performs interpolation using the measured focus positions. Thus, the focus position of another individual area is determined. In this case, based on the focus position information, the imaging control unit 110 may change the individual area for independently measuring the focus position, or may change the interpolation method. In this case, the focus position information may include information that simply instructs to change the focus position.
- the setting information generation unit 150 generates white balance information for resetting the white balance when a white balance defect is detected.
- the white balance information is one piece of setting information.
- the white balance information includes, for example, color information of an image of an area where no sample exists (that is, color information compared with the color information of the template) when a white balance defect is detected.
- the imaging control unit 110 can adjust the white balance (for example, RGB coefficients in white balance correction) to an appropriate value. As a result, similar white balance problems can be avoided during re-imaging.
- the setting information generation unit 150 should change the RGB coefficient from the color information of the image in the area where the sample does not exist and the color information prepared in advance as a template (for example, R is set to 0. 0).
- White balance information including the information may be generated.
- the setting information generation unit 150 generates brightness information for resetting the brightness of the illumination when a brightness defect is detected.
- the brightness information is one piece of setting information.
- the brightness information includes, for example, brightness information of an image in an area where no sample exists when brightness failure is detected (that is, brightness information compared with the brightness information of the template). With such brightness information, the imaging control unit 110 can adjust the brightness of the illumination to an appropriate value. As a result, the same brightness problem can be avoided during re-imaging.
- the setting information generation unit 150 should change the brightness of the illumination from the luminance information of the image of the region where the sample does not exist and the luminance information prepared in advance as a template (for example, 1.2 Brightness information including the information may be generated.
- the generation of the setting information by the setting information generation unit 150 has been described above using the first to fifth examples of specific failure detection.
- Such setting information enables the scanner 100 to automatically set the imaging conditions for re-imaging without the user having to individually review the imaging conditions for avoiding another failure.
- the setting information generation unit 150 outputs the generated setting information to the imaging control unit 110 and the communication unit 160.
- the failure detection unit 140 detects a failure requiring re-imaging, and the setting information generation unit 150 generates setting information corresponding to the failure.
- the imaging control unit 110 causes the digital microscope 120 to re-image each individual image or to re-image the entire plurality of individual images. Determine. That is, it is determined whether to perform partial re-imaging or total re-imaging. For example, the imaging control unit 110 performs the following determination.
- the imaging control unit 110 can determine that partial re-imaging is to be performed when priority is given to shortening the time required for re-imaging, for example. In the case where priority is given to avoiding a failure in pasting, it may be determined that the entire re-imaging is performed. For example, the imaging control unit 110 determines in advance whether to give priority to shortening the time required for imaging or avoiding a failure in pasting.
- the imaging control unit 110 determines, for example, that the entire re-imaging is performed. Since a failure in combining often covers not only a part of the combined image but the entire combined image, it is desirable to perform re-imaging as a whole. In addition, since the white balance and the brightness are generally set with the same contents not for each individual image but for the whole of the plurality of individual images, the white balance and the brightness similarly affect the plurality of individual images. Therefore, it is desirable to perform overall re-imaging.
- re-imaging in a necessary and sufficient range can be performed by determining the re-imaging range according to the type of failure. In other words, it is possible to perform re-imaging more efficiently while avoiding a second failure in re-imaging.
- the communication unit 160 communicates with the server 200-1 and the viewer 300. For example, the communication unit 160 transmits the combined image from the combining unit 130 to the server 200-1 and the viewer 300. In addition, the communication unit 160 transmits the imaging condition information from the imaging control unit 110 to the server 200-1 and the viewer 300. Also, the setting information from the setting information generation unit 150 is transmitted to the viewer 300.
- the communication unit 160 may receive the imaging condition information from the server 200-1 or the viewer 300 when the imaging condition is determined or changed by the server 200-1 or the viewer 300. Similarly, the communication unit 160 may receive setting information from the server 200-1 or the viewer 300 when the setting information is generated or changed by the server 200-1 or the viewer 300. The communication unit 160 can output the received information to the imaging control unit 110.
- FIG. 7 is a block diagram illustrating an example of the configuration of the server 200-1 according to the first embodiment.
- the server 200-1 includes a communication unit 210, a storage unit 220, and a control unit 230.
- the communication unit 210 communicates with the scanner 100-1 and the viewer 300. For example, the communication unit 210 receives a composite image and imaging condition information from the scanner 100-1. In addition, the communication unit 210 transmits the composite image and the imaging condition information stored in the storage unit 220 to the viewer 300.
- the storage unit 220 stores a composite image and imaging condition information managed by the server 200-1.
- the storage unit 220 may configure a database that stores the combined image of the sample and the imaging condition information when the combined image is generated in association with the identification information of each sample.
- Control unit 230 controls the entire server 200-1. For example, when the composite image and the imaging condition information are received from the scanner 100-1 by the communication unit 210, the control unit 230 causes the storage unit 220 to store the received composite image and the imaging condition information. In addition, the control unit 230 may cause the communication unit 210 to transmit the combined image and the imaging condition information to be presented to the user to the viewer 300 in response to an instruction from the user or automatically.
- FIG. 8 is a block diagram illustrating an example of the configuration of the viewer 300 according to the first embodiment.
- the viewer 300 includes a communication unit 310, an input unit 320, a control unit 330, and a display unit 340.
- the communication unit 310 communicates with the scanner 100-1 and the server 200-1. For example, the communication unit 310 receives a composite image to be presented to the user from the scanner 100-1 or the server 200-1. The communication unit 310 can also receive imaging condition information or setting information associated with the composite image together with the composite image.
- the communication unit 310 scans the imaging condition information indicating the designated imaging condition or the edited setting information when the imaging condition is designated or the setting information is edited by the user via a user interface described later. To 100-1.
- the input unit 320 detects a user operation.
- the input unit 320 may include one or more input devices such as a touch panel, a keyboard, a button, and a pointing device, for example.
- Control unit 330 The control unit 330 controls the entire viewer 300.
- the control unit 330 includes, for example, an acquisition unit 331 and a display control unit 333.
- the acquisition unit 331 acquires a composite image to be presented to the user and imaging condition information when the composite image is generated via the communication unit 310. For example, the acquisition unit 331 acquires the composite image and imaging condition information in response to an instruction from the user or each time a composite image is generated by the scanner 100-1.
- the acquisition unit 331 acquires imaging condition information representing the imaging conditions of the current scanner 100-1 via the communication unit 310. For example, the acquisition unit 331 acquires the imaging condition information in response to an instruction from the user.
- the acquisition unit 331 may acquire setting information via the communication unit 310 under a predetermined condition. For example, the acquisition unit 331 may acquire the setting information when the user instructs the user to present the setting information before starting imaging, or when the setting information to be presented to the user is generated.
- the display control unit 333 displays the combined image on the display surface of the display unit 340.
- the display control unit 333 displays the composite image on the display surface in response to an instruction from the user or every time a composite image is generated.
- the display control unit 333 allows the user to review the imaging conditions, for example, when the composite image is generated.
- the imaging condition information is displayed on the display surface of the display unit 340.
- the display control unit 333 provides, for example, a user interface for enabling the user to specify the imaging condition of the scanner 100-1 so that the user can freely specify the imaging condition. For example, the display control unit 333 presents the current imaging conditions of the scanner 100-1 to the user in response to an instruction from the user. When the user designates an imaging condition by a user operation on the input unit 320, the display control unit 333 transmits imaging condition information representing the designated imaging condition to the scanner 100-1 via the communication unit 310.
- the display control unit 333 may display the acquired setting information on the display surface of the display unit 340.
- the scanner 100-1 automatically detects a failure requiring re-imaging, and re-imaging is also performed automatically. However, even if the setting information for re-imaging is displayed when the user instructs the user to present the setting information before the start of imaging, or when the setting information to be presented to the user is generated, etc. Good.
- the display control unit 333 may present the setting information to the user before re-imaging is executed. Then, the display control unit 333 may make the setting information editable by the user before re-imaging is executed. If the setting information can be edited in this way, the user can edit the setting information and perform more appropriate re-imaging when determining that the setting information is not appropriate.
- the display unit 340 is a display having a display surface.
- the display unit 340 displays a combined image, imaging condition information, setting information, or the like on the display surface in accordance with control by the control unit 330.
- FIG. 9 is a flowchart illustrating an example of a schematic flow of an imaging control process according to the first embodiment.
- steps S401 to S405 are processes related to the entire imaging.
- the imaging control unit 110 of the scanner 100-1 determines an imaging area, and sets the determined imaging area in the digital microscope 120.
- step S ⁇ b> 403 the imaging control unit 110 determines the imaging order of the individual images that reflect the individual areas included in the set imaging area, and sets the determined imaging order in the digital microscope 120.
- step S ⁇ b> 405 the imaging control unit 110 determines other overall imaging conditions such as white balance and illumination brightness, and sets the determined imaging conditions in the digital microscope 120.
- Steps S407 to S415 are processes that are repeated for each individual image.
- step S407 the imaging control unit 110 determines individual imaging conditions for imaging each individual image, such as the focus position, and sets the determined imaging conditions in the digital microscope 120.
- step S409 the digital microscope 120 captures individual images according to the set imaging order.
- step S411 the combining unit 130 combines the captured individual image and the adjacent individual image.
- step S500 the failure detection unit 140 executes an individual image evaluation process regarding the captured individual image.
- step S413 the failure detection unit 140 determines whether a failure requiring re-imaging has been detected in the individual image evaluation process. If the failure is detected, the process proceeds to step S417. Otherwise, the process proceeds to S415.
- step S415 the imaging control unit 110 determines whether imaging of individual images has been completed for all the individual areas included in the imaging area. If the imaging is complete, the process ends. Otherwise, the process returns to step S407.
- step S417 the setting information generation unit 150 generates setting information for setting imaging conditions at the time of reimaging.
- step S419 the imaging control unit 110 determines whether to perform partial re-imaging or overall re-imaging. If overall re-imaging is desired, the process returns to step S401. Otherwise, the process returns to step S407.
- the imaging control unit 110 sets the entire imaging condition and individual imaging conditions based on the generated setting information in steps S401 to S407.
- step S407 the imaging control unit 110 sets individual imaging conditions based on the generated setting information.
- the individual image evaluation process 500 is executed for each type of failure that requires re-imaging. That is, when five types of failure are detected, five individual image evaluation processes 500 are executed.
- the individual image evaluation process 500a related to imaging omission and the individual image evaluation process 500b related to a pasting failure will be described.
- FIG. 10 is a flowchart showing an example of the flow of the individual image evaluation process 500a regarding the imaging omission.
- the individual image evaluation process 500a the direction in which the texture exists in the individual image is evaluated.
- step S510 the failure detection unit 140 determines whether the individual image is an individual image of an area located in the peripheral portion of the imaging region 20 (that is, an individual image constituting the peripheral portion of the combined image). If the individual image is an individual image of a region located in the peripheral portion of the imaging region 20, the process proceeds to step S520. Otherwise, the process ends.
- step S520 the failure detection unit 140 detects the edge of the protruding portion of the individual image (that is, the portion of the individual image that protrudes from the combined image), and calculates the amount of the detected edge.
- step S530 the failure detection unit 140 determines whether the calculated edge amount exceeds a threshold value. That is, the failure detection unit 140 determines whether there is a texture at the protruding portion of the individual image. If the amount of edges exceeds the threshold, the process proceeds to step S540. Otherwise, the process ends.
- step S540 the failure detection unit 140 detects an imaging omission in an area that should be included in the composite image as a failure requiring reimaging. Then, the process ends.
- FIG. 11 is a flowchart showing an example of the flow of the individual image evaluation process 500b regarding the pasting failure.
- the luminance value of the overlapping portion when the composite image is generated is evaluated.
- step S550 the failure detection unit 140 determines a difference in luminance value between overlapping portions of adjacent individual images (that is, a sum of absolute values of differences in luminance values between corresponding pixels in the overlapping portion). Is calculated.
- step S560 the failure detection unit 140 determines whether the difference in luminance value between the overlapping portions exceeds a predetermined threshold. If the difference exceeds a predetermined threshold, the process proceeds to step S570. Otherwise, the process ends.
- step S570 the failure detection unit 140 detects a failure in combining when generating a combined image as a failure requiring re-imaging. Then, the process ends.
- a failure requiring re-imaging is automatically detected in the scanner 100, and based on setting information generated according to the failure.
- Imaging conditions for re-imaging are automatically set. Therefore, the user does not need to perform operations such as confirmation of an image on the screen and resetting of imaging conditions for avoiding a failure again as in the past. Therefore, re-imaging can be performed with less effort and time.
- the server 200-2 automatically detects a failure that requires re-imaging. Then, the server 200-2 prompts the scanner 100-2 to re-image the sample in response to the detection of the failure.
- the configuration of the viewer 300 in the second embodiment may be the same as the configuration of the viewer 300 in the first embodiment.
- FIG. 12 is a block diagram illustrating an example of the configuration of the scanner 100-2 according to the second embodiment.
- the scanner 100-2 includes an imaging control unit 111, a digital microscope 121, a pasting unit 131, and a communication unit 160.
- the imaging control unit 111 controls imaging using the digital microscope 121.
- the imaging control unit 111 sets an imaging condition specified by the user or automatically determined in the digital microscope 121, and causes the digital microscope 121 to capture an image of the sample according to the imaging condition.
- the imaging control unit 111 sets, for example, an imaging region, an imaging order, illumination brightness, a white balance coefficient, and the like as imaging conditions for the entire sample.
- the imaging control unit 111 sets a focus position as an imaging condition for each individual area, for example.
- the imaging control unit 111 sets at least a part of the imaging conditions based on the setting information to a digital microscope. Reset to 121. Then, the imaging control unit 111 causes the digital microscope 121 to re-image the sample image according to the reset imaging conditions.
- the imaging control unit 110 outputs imaging condition information representing the set or reset imaging conditions to the communication unit 160.
- the digital microscope 121 captures an image according to the image capturing conditions set by the image capturing control unit 111. For example, like the digital microscope 120 according to the first embodiment, the digital microscope 121 captures an individual image that reflects each individual area in the set imaging area in accordance with the set imaging order. Then, the digital microscope 121 outputs the captured individual image to the pasting unit 131.
- the combining unit 131 Similar to the combining unit 130 according to the first embodiment, the combining unit 131 generates a combined image by combining the individual images captured by the digital microscope 121. Then, the combining unit 131 outputs the generated combined image to the communication unit 160.
- FIG. 13 is a block diagram illustrating an example of the configuration of the server 200-2 according to the second embodiment.
- the server 200-2 includes a communication unit 210, a storage unit 220, a control unit 230, a failure detection unit 240, and a setting information generation unit 250.
- the failure detection unit 240 detects a failure requiring re-imaging by evaluating a composite image (or a partial image thereof) received from the scanner 100-2 via the communication unit 210.
- the type of failure detected by the failure detection unit 240 may be the same as the type of failure that can be detected by the failure detection unit 140 of the scanner 100-1 according to the first embodiment. That is, the failure detection unit 240 can detect, for example, the imaging omission and the combining failure described above as failures related to the combined image. Further, the failure detection unit 240 can detect the above-described failure in the focus position, failure in white balance, and failure in brightness as other failures.
- the failure detection unit 240 detects an imaging omission, since the protruding portion is not included in the composite image, the failure detection unit 240 is illustrated in FIG. A peripheral portion 35 is used.
- the failure detection unit 240 detects a failure in combining, since the two overlapping portions of the individual images adjacent to each other have already been lost, the edge included in the combined image as shown in FIG. Evaluate the direction.
- the failure detection unit 240 outputs information about the detected failure to the setting information generation unit 250.
- the setting information generation unit 250 When the failure detection unit 240 detects a failure, the setting information generation unit 250 generates setting information for setting an imaging condition for re-imaging. Similar to the setting information generation unit 150, the setting information generation unit 250 detects a failure requiring re-imaging, such as an imaging failure, a pasting failure, a focus position failure, a white balance failure, and a brightness failure. In addition, various setting information is generated.
- a failure requiring re-imaging such as an imaging failure, a pasting failure, a focus position failure, a white balance failure, and a brightness failure.
- various setting information is generated.
- the setting information generation unit 250 When the setting information generation unit 250 generates the setting information, the setting information generation unit 250 transmits the setting information and a re-imaging request for prompting re-imaging to the scanner 100-2 via the communication unit 210.
- the re-imaging request may be once transmitted to the viewer 300 and transferred to the scanner 100-2 after the re-imaging instruction from the user is input to the viewer 300.
- FIG. 14 is a flowchart illustrating an example of a schematic flow of imaging control processing on the server 200-2 side according to the second embodiment.
- step S601 the failure detection unit 240 of the server 200-2 reads a composite image from the storage unit 220. For example, the failure detection unit 240 selects and reads a composite image from the storage unit 220 by automatically selecting an unevaluated composite image or in response to an instruction from the user.
- step S700 the failure detection unit 240 executes a combined image evaluation process.
- step S603 the failure detection unit 240 determines whether a failure requiring reimaging has been detected. If the failure is detected, the process proceeds to step S605. Otherwise, the process ends.
- step S605 the setting information generation unit 250 generates setting information for setting imaging conditions for re-imaging.
- step S607 the setting information generation unit 250 causes the storage unit 220 to store the generated setting information.
- step S609 the setting information generation unit 250 transmits setting information and a re-imaging request to the scanner 100-2 or the viewer 300 via the communication unit 210. Then, the process ends.
- the combined image review process 700 is executed for each type of failure that requires re-imaging. That is, when five types of failure are detected, five combined image evaluation processes 700 are executed.
- a composite image review process 700a related to an imaging omission and a composite image review process 700b related to a composite failure will be described.
- FIG. 15 is a flowchart showing an example of the flow of a combined image evaluation process 700a regarding an imaging omission.
- the texture direction in the composite image is evaluated.
- the failure detection unit 240 detects the edge of the peripheral portion of the composite image, and calculates the amount of the detected edge. For example, the failure detection unit 240 calculates the amount of edge for each individual image (that is, partial image) that constitutes the composite image.
- step S720 the failure detection unit 240 determines whether the calculated edge amount exceeds a threshold value. That is, the failure detection unit 240 determines whether there is a texture at the peripheral portion of the composite image. For example, the failure detection unit 240 makes a determination for each individual image (that is, a partial image) constituting the composite image. If the amount of edges exceeds the threshold, the process proceeds to step S730. Otherwise, the process ends.
- step S730 the failure detection unit 240 detects an imaging omission in an area that should be included in the composite image as a failure requiring reimaging. Then, the process ends.
- FIG. 16 is a flowchart showing an example of the flow of a composite image evaluation process 700b regarding a composite failure.
- an edge in a predetermined direction is evaluated.
- the failure detection unit 240 detects an edge in a predetermined direction included in the composite image, and calculates the length or strength of the detected edge.
- step S750 the failure detection unit 240 determines whether there is an edge in a predetermined direction having a predetermined length or greater than a predetermined strength. If there is an edge in a predetermined direction having a predetermined length or greater than a predetermined strength, the process proceeds to step S760. Otherwise, the process ends.
- step S760 the failure detection unit 240 detects a failure in combining when generating a combined image as a failure requiring re-imaging. Then, the process ends.
- FIG. 17 is a flowchart illustrating an example of a schematic flow of imaging control processing on the scanner 100-2 side according to the second embodiment.
- steps S801 to S807 are processes related to the entire imaging.
- the communication unit 160 of the scanner 100-2 receives setting information from the server 200-2.
- the imaging control unit 111 determines an imaging area based on the setting information, and sets the determined imaging area in the digital microscope 121.
- the imaging control unit 111 determines the imaging order of the individual images that reflect the individual areas included in the set imaging area based on the setting information, and sets the determined imaging order in the digital microscope 121. To do.
- the imaging control unit 111 determines other overall imaging conditions such as white balance and illumination brightness based on the setting information, and sets the determined imaging conditions in the digital microscope 121. To do.
- Steps S809 to S815 are processes that are repeated for each individual image.
- step S809 the imaging control unit 111 determines individual imaging conditions for imaging each individual image, such as the focus position, based on the setting information, and sets the determined imaging conditions in the digital microscope 121.
- step S811 the digital microscope 121 images individual images according to the set imaging order.
- step S813 the combining unit 131 combines the captured individual image and the adjacent individual image.
- step S815 the imaging control unit 111 determines whether imaging of individual images has been completed for all individual areas included in the imaging area. If the imaging has been completed, the process proceeds to step S817. Otherwise, the process returns to step S809.
- step S817 the communication unit 160 transmits the composite image to the server 200-2 and the viewer 300. Then, the process ends.
- the server 200 automatically detects a failure that requires re-imaging, and executes the re-imaging in response to the failure. -2. Therefore, the user does not need to perform operations such as confirmation of an image on the screen and resetting of imaging conditions for avoiding a failure again as in the past. Therefore, re-imaging can be performed with less effort and time. Furthermore, in the second embodiment, even if each individual scanner does not have a failure detection and setting information generation function, the failure detection and setting information generation are performed by storing the composite image on the server. Is possible. Further, it is possible to detect failure and generate setting information afterwards as necessary while shortening the imaging time of the scanner.
- a failure requiring re-imaging is detected by evaluating an image captured using a digital microscope. And when the said failure is detected, the setting information for setting the imaging condition at the time of reimaging is produced
- a failure related to the composite image is detected.
- re-imaging is automatically performed even if a failure unique to the digital microscope system, such as an imaging failure or a bonding failure, occurs. As a result, it is possible to reduce the labor and time for reimaging in the digital microscope system.
- imaging area information is generated as setting information. Based on the imaging area information, it is possible to set the imaging area so as to include an imaging omission area. As a result, the same imaging omission can be avoided during re-imaging.
- imaging order information is generated as setting information. By the imaging order information, it is also possible to avoid a bonding failure at the time of reimaging.
- imaging order information is generated as setting information.
- re-imaging it is determined whether to perform partial re-imaging or overall re-image according to the type of failure detected. In this manner, by determining the re-imaging range according to the type of failure, re-imaging can be performed within a necessary and sufficient range. In other words, it is possible to perform re-imaging more efficiently while avoiding a second failure in re-imaging.
- failure detection and setting information generation are performed, for example, on the scanner side.
- each time a plurality of individual images are captured each image is evaluated to detect a failure that requires re-imaging.
- failure can be detected after imaging, so that re-imaging can be performed immediately. This eliminates the need to re-arrange the imaging target such as a preparation for re-imaging, thereby reducing labor and time required for re-imaging.
- failure detection and setting information generation are performed on the server side, for example.
- failure detection and setting information generation are performed using the composite image stored in the server.
- the digital microscope may not be provided by a scanner, and may be an apparatus different from the scanner.
- the digital microscope is connected to the scanner.
- Two or more devices among the scanner, server, and viewer may be the same device.
- processing steps in the imaging control process of this specification do not necessarily have to be executed in time series in the order described in the flowchart.
- processing steps in the imaging control process may be executed in a different order from the order described as the flowchart, or may be executed in parallel.
- An information processing apparatus comprising: a generation unit configured to generate setting information for setting an imaging condition for re-imaging when the failure is detected by the detection unit.
- a generation unit configured to generate setting information for setting an imaging condition for re-imaging when the failure is detected by the detection unit.
- the detection unit detects a failure associated with a composite image generated by combining a plurality of images as the failure.
- the failure related to the composite image includes an imaging omission of a region to be included in the composite image.
- the detection unit detects the imaging omission by evaluating a texture existing direction in the image.
- the generation unit generates imaging region information for resetting an imaging region so as to include an additional region to be newly imaged when the imaging omission is detected by the detection unit, (2) The information processing apparatus according to any one of (4) to (4). (6) The information processing apparatus according to (5), wherein the generation unit further generates imaging order information for resetting the imaging order of the images when the imaging area is reset. (7) The information processing apparatus according to (2), wherein the failure related to the composite image includes a composite failure when generating the composite image. (8) The information processing apparatus according to (7), wherein the detection unit detects the combining failure by evaluating an edge in a predetermined direction included in the combined image.
- the information processing apparatus (9) The information processing apparatus according to (7), wherein the detection unit detects the combining failure by evaluating a luminance value of an overlapping portion when the combined image is generated. (10) The generation unit generates imaging order information for resetting the imaging order of the images when the detection unit detects the unsuccessful pasting, and any one of (7) to (9) The information processing apparatus according to item. (11) The information processing apparatus according to any one of (2) to (10), wherein the detection unit further detects a failure in a focus position, white balance, or brightness as the failure.
- the information processing apparatus according to the type of the failure detected by the detection unit, an imaging control unit that determines whether to cause the digital microscope to re-image individual images or to re-image a plurality of images as a whole, The information processing apparatus according to any one of (2) to (11), further including: (13) The information processing apparatus according to any one of (2) to (12), wherein the detection unit evaluates each image for detecting the failure each time the plurality of images are captured. (14) The information processing apparatus according to any one of (2) to (12), wherein the detection unit evaluates the composite image or a partial image of the composite image after the composite image is generated. .
- the information processing apparatus includes: A display control unit that presents the setting information generated by the generation unit to a user or allows editing by the user before re-imaging is performed;
- the information processing apparatus according to any one of (1) to (14), further including: (16) Detecting a failure requiring re-imaging associated with the image by evaluating an image captured using a digital microscope; and An imaging control method comprising: generating setting information for setting imaging conditions for re-imaging when the failure is detected by the detection unit.
- a digital microscope A detection unit that detects a failure that requires re-imaging related to the image by evaluating an image that is captured using the digital microscope, and when the failure is detected by the detection unit, during the re-imaging
- An information processing apparatus including a generation unit that generates setting information for setting the imaging conditions of Including digital microscope system.
- a detection unit that detects a failure associated with a composite image generated by combining a plurality of images captured using a digital microscope by evaluating the image;
- An information processing apparatus comprising: a generation unit configured to generate setting information for setting an imaging condition for re-imaging when the failure is detected by the detection unit.
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Chemical & Material Sciences (AREA)
- Analytical Chemistry (AREA)
- Optics & Photonics (AREA)
- Microscoopes, Condenser (AREA)
- Studio Devices (AREA)
- Image Input (AREA)
- Image Processing (AREA)
Abstract
Description
1.デジタル顕微鏡システムの概略的な構成
2.第1の実施形態
2.1.スキャナの構成
2.2.サーバの構成
2.3.ビューアの構成
2.4.処理の流れ
3.第2の実施形態
3.1.スキャナの構成
3.2.サーバの構成
3.3.処理の流れ
4.まとめ
まず、図1を参照して、本開示の実施形態に係るデジタル顕微鏡システム1の概略的な構成について説明する。図1は、本開示の実施形態に係るデジタル顕微鏡システム1の概略的な構成の一例を示す説明図である。図1を参照すると、デジタル顕微鏡システム1は、スキャナ100、サーバ200及びビューア300を含む。
まず、本開示の第1の実施形態を説明する。本開示の第1の実施形態によれば、スキャナ100において、再撮像を要する失敗が自動的に検出され、当該失敗に応じた再撮像が行われる。
図2~図6を参照して、第1の実施形態に係るスキャナ100-1の構成の一例について説明する。図2は、第1の実施形態に係るスキャナ100-1の構成の一例を示すブロック図である。図2を参照すると、スキャナ100-1は、撮像制御部110、デジタル顕微鏡120、貼り合せ部130、失敗検出部140、設定情報生成部150及び通信部160を備える。
撮像制御部110は、デジタル顕微鏡120を用いた撮像を制御する。例えば、撮像制御部110は、ユーザにより指定され、又は自動的に決定される撮像条件をデジタル顕微鏡120に設定し、当該撮像条件に従ってデジタル顕微鏡120にサンプルの画像を撮像させる。撮像制御部110は、例えば、サンプル全体の撮像条件として、撮像領域、撮像順、照明の明るさ及びホワイトバランスの係数等を設定する。また、撮像制御部110は、例えば、個別領域ごとの撮像条件として、フォーカス位置を設定する。
デジタル顕微鏡120は、撮像制御部110により設定された撮像条件に従って画像を撮像する。例えば、デジタル顕微鏡120は、設定された撮影順に従って、設定された撮像領域内の各個別領域を映す画像(以下、「個別画像」と呼ぶ)を撮像する。以下、この点について図3を参照してより具体的に説明する。
貼り合せ部130は、デジタル顕微鏡120により撮像された個別画像を貼り合せることにより、貼り合せ画像を生成する。以下、この点について図4を参照してより具体的に説明する。
失敗検出部140は、デジタル顕微鏡120を用いて撮像される画像を評価することにより、当該画像に関連する再撮像を要する失敗を検出する。ここでの「画像を評価する」とは、個別画像の各々を評価すること、及び、個別画像から構成される貼り合せ画像を評価することを含む。以下に、具体的な失敗検出の第1~第5の例を用いて、失敗検出部140による失敗の検出について説明する。なお、失敗検出の第1の例及び第2の例では、失敗検出部140は、上記失敗として、複数の個別画像を貼り合せることにより生成される貼り合せ画像に関連する失敗を検出する。
設定情報生成部150は、失敗検出部140により失敗が検出された場合に、再撮像の際の撮像条件を設定するための設定情報を生成する。以下に、上記失敗検出の第1~第5の例を用いて、設定情報生成部150による設定情報の生成について説明する。
以上のように、失敗検出部140により、再撮像を要する失敗が検出され、設定情報生成部150により、当該失敗に応じた設定情報が生成される。一方、例えば、撮像制御部110は、失敗検出部140により検出される上記失敗の種類に応じて、デジタル顕微鏡120に個々の個別画像を再撮像させるか複数の個別画像の全体を再撮像させるかを判定する。即ち、部分的な再撮像又は全体的な再撮像のいずれを行うかが判定される。例えば、撮像制御部110は、以下のような判定を行う。
通信部160は、サーバ200-1及びビューア300と通信する。例えば、通信部160は、貼り合せ部130からの貼り合せ画像をサーバ200-1及びビューア300に送信する。また、通信部160は、撮像制御部110からの撮像条件情報をサーバ200-1及びビューア300に送信する。また、設定情報生成部150からの設定情報をビューア300に送信する。
図7を参照して、第1の実施形態に係るサーバ200-1の構成の一例について説明する。図7は、第1の実施形態に係るサーバ200-1の構成の一例を示すブロック図である。図7を参照すると、サーバ200-1は、通信部210、記憶部220及び制御部230を備える。
通信部210は、スキャナ100-1及びビューア300と通信する。例えば、通信部210は、スキャナ100-1から貼り合せ画像及び撮像条件情報を受信する。また、通信部210は、記憶部220に記憶される貼り合せ画像及び撮像条件情報をビューア300に送信する。
記憶部220は、サーバ200-1により管理される貼り合せ画像及び撮像条件情報を記憶する。例えば、記憶部220は、各サンプルの識別情報と関連付けて、当該サンプルの貼り合せ画像及び当該貼り合せ画像が生成された際の撮像条件情報を記憶するデータベースを構成してもよい。
制御部230は、サーバ200-1全体を制御する。例えば、制御部230は、通信部210によりスキャナ100-1から貼り合せ画像及び撮像条件情報が受信されると、受信された貼り合せ画像及び撮像条件情報を記憶部220に記憶させる。また、制御部230は、ユーザからの指示に応じて又は自動的に、ユーザに呈示されるべき貼り合せ画像及び撮像条件情報を通信部210からビューア300へ送信させてもよい。
図8を参照して、第1の実施形態に係るビューア300の構成の一例について説明する。図8は、第1の実施形態に係るビューア300の構成の一例を示すブロック図である。図8を参照すると、ビューア300は、通信部310、入力部320、制御部330及び表示部340を備える。
通信部310は、スキャナ100-1及びサーバ200-1と通信する。例えば、通信部310は、ユーザに呈示されるべき貼り合せ画像を、スキャナ100-1又はサーバ200-1から受信する。通信部310は、貼り合せ画像と共に、当該貼り合せ画像と関連付けられる撮像条件情報又は設定情報をも受信し得る。
入力部320は、ユーザ操作を検出する。当該入力部320は、例えば、タッチパネル、キーボード、ボタン及びポインティングデバイスなどの入力デバイスのうち1つ以上を含み得る。
制御部330は、ビューア300全体を制御する。制御部330は、例えば、取得部331及び表示制御部333を含む。
取得部331は、通信部310を介して、ユーザに呈示されるべき貼り合せ画像及び当該貼り合せ画像が生成された際の撮像条件情報を取得する。取得部331は、例えば、ユーザからの指示に応じて、又はスキャナ100-1で貼り合せ画像が生成される度に、当該貼り合せ画像及び撮像条件情報を取得する。
表示制御部333は、貼り合せ画像を表示部340の表示面に表示させる。例えば、表示制御部333は、ユーザからの指示に応じて、又は貼り合せ画像が生成される度に、貼り合せ画像を上記表示面に表示させる。また、ユーザが貼り合せ画像の閲覧時に再撮像を要する失敗を発見した場合に、ユーザが撮像条件を再検討できるように、表示制御部333は、例えば、当該貼り合せ画像が生成された際の撮像条件情報を表示部340の表示面に表示させる。
表示部340は、表示面を有するディスプレイである。表示部340は、例えば、制御部330による制御に応じて、貼り合せ画像、撮像条件情報、又は設定情報等を表示面に表示する。
次に、図9~図11を参照して、第1の実施形態に係る撮像制御処理の例について説明する。
図9は、第1の実施形態に係る撮像制御処理の概略的な流れの一例を示すフローチャートである。
次に、個別画像評処理500の例について説明する。個別画像評価処理500は、再撮像を要する失敗の種類毎に実行される。即ち、5つの種類の失敗を検出する場合には、5つの個別画像評価処理500が実行される。ここでは、とりわけ、撮像漏れに関する個別画像評価処理500a及び貼り合せ失敗に関する個別画像評価処理500bについて説明する。
続いて、本開示の第2の実施形態を説明する。本開示の第2の実施形態によれば、サーバ200-2において、再撮像を要する失敗が自動的に検出される。そして、サーバ200-2は、当該失敗の検出に応じて、スキャナ100-2にサンプルの再撮像を促す。なお、第2の実施形態におけるビューア300の構成は、第1の実施形態におけるビューア300の構成と同じであってよい。
まず、図12を参照して、第2の実施形態に係るスキャナ100-2の構成の一例について説明する。図12は、第2の実施形態に係るスキャナ100-2の構成の一例を示すブロック図である。図12を参照すると、スキャナ100-2は、撮像制御部111、デジタル顕微鏡121、貼り合せ部131及び通信部160を備える。
撮像制御部111は、デジタル顕微鏡121を用いた撮像を制御する。例えば、撮像制御部111は、ユーザにより指定され、又は自動的に決定される撮像条件をデジタル顕微鏡121に設定し、当該撮像条件に従ってデジタル顕微鏡121にサンプルの画像を撮像させる。撮像制御部111は、第1の実施形態に係る撮像制御部110と同様、例えば、サンプル全体の撮像条件として、撮像領域、撮像順、照明の明るさ及びホワイトバランスの係数等を設定する。また、撮像制御部111は、例えば、個別領域ごとの撮像条件として、フォーカス位置を設定する。
デジタル顕微鏡121は、撮像制御部111により設定された撮像条件に従って画像を撮像する。例えば、デジタル顕微鏡121は、第1の実施形態に係るデジタル顕微鏡120と同様、設定された撮影順に従って、設定された撮像領域内の各個別領域を映す個別画像を撮像する。そして、デジタル顕微鏡121は、撮像された個別画像を貼り合せ部131に出力する。
貼り合せ部131は、第1の実施形態に係る貼り合せ部130と同様、デジタル顕微鏡121により撮像された個別画像を貼り合せることにより、貼り合せ画像を生成する。そして、貼り合せ部131は、生成した貼り合せ画像を通信部160に出力する。
次に、図13を参照して、第2の実施形態に係るサーバ200-2の構成の一例について説明する。図13は、第2の実施形態に係るサーバ200-2の構成の一例を示すブロック図である。図13を参照すると、サーバ200-2は、通信部210、記憶部220、制御部230、失敗検出部240及び設定情報生成部250を備える。
失敗検出部240は、通信部210を介してスキャナ100-2から受信される貼り合せ画像(又はその部分画像)を評価することにより、再撮像を要する失敗を検出する。失敗検出部240により検出される失敗の種類は、第1の実施形態に係るスキャナ100-1の失敗検出部140により検出され得る失敗の種類と同様であってよい。即ち、失敗検出部240は、例えば、貼り合せ画像に関連する失敗として、上述した撮像漏れ及び貼り合せ失敗を検出し得る。また、失敗検出部240は、その他の失敗として、上述したフォーカス位置の不具合、ホワイトバランスの不具合及び明るさの不具合を検出し得る。
設定情報生成部250は、失敗検出部240により失敗が検出された場合に、再撮像の際の撮像条件を設定するための設定情報を生成する。設定情報生成部250は、設定情報生成部150と同様に、撮像漏れ、貼り合せ失敗、フォーカス位置の不具合、ホワイトバランスの不具合及び明るさの不具合等の、再撮像を要する失敗が検出された場合に、各種設定情報を生成する。
次に、図14~図17を参照して、第2の実施形態に係る撮像制御処理の例について説明する。
まず、撮像制御処理のうち、サーバ200-2側で行われる処理の例について説明する。図14は、第2の実施形態に係るサーバ200-2側での撮像制御処理の概略的な流れの一例を示すフローチャートである。
次に、貼り合せ画像評処理700の例について説明する。貼り合せ画像評処理700は、再撮像を要する失敗の種類毎に実行される。即ち、5つの種類の失敗を検出する場合には、5つの貼り合せ画像評処理700が実行される。ここでは、とりわけ、撮像漏れに関する貼り合せ画像評処理700a及び貼り合せ失敗に関する貼り合せ画像評処理700bについて説明する。
次に、撮像制御処理のうち、スキャナ100-2側で行われる処理の例について説明する。図17は、第2の実施形態に係るスキャナ100-2側での撮像制御処理の概略的な流れの一例を示すフローチャートである。
ここまで、図1~17を用いて、本開示に係るデジタル顕微鏡システムの2つの実施形態について説明した。これら実施形態によれば、デジタル顕微鏡を用いて撮像される画像を評価することにより、再撮像を要する失敗が検出される。そして、当該失敗が検出された場合に、再撮像の際の撮像条件を設定するための設定情報が生成される。即ち、再撮像を要する失敗が自動的に検出され、失敗に応じて生成された設定情報に基づいて再撮像の際の撮像条件が自動的に設定される。そのため、ユーザは、従来のように、画面上での画像の確認、及び再度の失敗を回避するための撮像条件の再設定等の作業を行わなくてもよい。よって、より少ない手間及び時間で再撮像を行うことが可能になる。
(1)
デジタル顕微鏡を用いて撮像される画像を評価することにより、前記画像に関連する再撮像を要する失敗を検出する検出部と、
前記検出部により前記失敗が検出された場合に、再撮像の際の撮像条件を設定するための設定情報を生成する生成部と
を備える情報処理装置。
(2)
前記検出部は、前記失敗として、複数の画像を貼り合せることにより生成される貼り合せ画像に関連する失敗を検出する、前記(1)に記載の情報処理装置。
(3)
前記貼り合せ画像に関連する前記失敗は、前記貼り合せ画像に含まれるべき領域の撮像漏れを含む、前記(2)に記載の情報処理装置。
(4)
前記検出部は、前記画像におけるテクスチャの存在方向を評価することにより、前記撮像漏れを検出する、前記(3)に記載の情報処理装置。
(5)
前記生成部は、前記検出部により前記撮像漏れが検出された場合に、新たに撮像されるべき追加領域を含むように撮像領域を再設定するための撮像領域情報を生成する、前記(2)~(4)のいずれか1項に記載の情報処理装置。
(6)
前記生成部は、撮像領域が再設定される場合に、前記画像の撮像順を再設定するための撮像順情報をさらに生成する、前記(5)に記載の情報処理装置。
(7)
前記貼り合せ画像に関連する前記失敗は、前記貼り合せ画像を生成する際の貼り合せ失敗を含む、前記(2)に記載の情報処理装置。
(8)
前記検出部は、前記貼り合せ画像に含まれる所定の方向のエッジを評価することにより、前記貼り合せ失敗を検出する、前記(7)に記載の情報処理装置。
(9)
前記検出部は、前記貼り合せ画像が生成される際の重なり部分の輝度値を評価することにより、前記貼り合せ失敗を検出する、前記(7)に記載の情報処理装置。
(10)
前記生成部は、前記検出部により前記貼り合せ失敗が検出された場合に、前記画像の撮像順を再設定するための撮像順情報を生成する、前記(7)~(9)のいずれか1項に記載の情報処理装置。
(11)
前記検出部は、前記失敗として、フォーカス位置、ホワイトバランス又は明るさの不具合をさらに検出する、前記(2)~(10)のいずれか1項に記載の情報処理装置。
(12)
前記情報処理装置は、前記検出部により検出される前記失敗の種類に応じて、前記デジタル顕微鏡に個々の画像を再撮像させるか複数の画像の全体を再撮像させるかを判定する撮像制御部、をさらに備える、前記(2)~(11)のいずれか1項に記載の情報処理装置。
(13)
前記検出部は、前記複数の画像の各々が撮像される都度、前記失敗の検出のために各画像を評価する、前記(2)~(12)のいずれか1項に記載の情報処理装置。
(14)
前記検出部は、前記貼り合せ画像が生成された後に、前記貼り合せ画像又は前記貼り合せ画像の部分画像を評価する、前記(2)~(12)のいずれか1項に記載の情報処理装置。
(15)
前記情報処理装置は、
前記生成部により生成される前記設定情報を、再撮像が実行される前にユーザに呈示し又はユーザにより編集可能とする表示制御部、
をさらに備える、前記(1)~(14)のいずれか1項に記載の情報処理装置。
(16)
デジタル顕微鏡を用いて撮像される画像を評価することにより、前記画像に関連する再撮像を要する失敗を検出することと、
前記検出部により前記失敗が検出された場合に、再撮像の際の撮像条件を設定するための設定情報を生成することと
を含む撮像制御方法。
(17)
コンピュータを、
デジタル顕微鏡を用いて撮像される画像を評価することにより、前記画像に関連する再撮像を要する失敗を検出する検出部と、
前記検出部により前記失敗が検出された場合に、再撮像の際の撮像条件を設定するための設定情報を生成する生成部と
として機能させるためのプログラム。
(18)
デジタル顕微鏡と、
前記デジタル顕微鏡を用いて撮像される画像を評価することにより、前記画像に関連する再撮像を要する失敗を検出する検出部、及び
前記検出部により前記失敗が検出された場合に、再撮像の際の撮像条件を設定するための設定情報を生成する生成部
を備える情報処理装置と、
を含むデジタル顕微鏡システム。
(19)
デジタル顕微鏡を用いて撮像される画像を評価することにより前記画像に関連する再撮像を要する失敗が検出された場合に生成される、再撮像の際の撮像条件を設定するための設定情報を取得する取得部と、
前記取得部により取得される前記設定情報を表示面に表示させる表示制御部と、
を備える表示制御装置。
(20)
デジタル顕微鏡を用いて撮像される複数の画像を貼り合せることで生成される貼り合せ画像に関連する失敗を、画像を評価することにより検出する検出部と、
前記検出部により前記失敗が検出された場合に、再撮像の際の撮像条件を設定するための設定情報を生成する生成部と
を備える情報処理装置。
10 生体サンプル
20 撮像領域
21 個別領域
30 個別画像
33 重なり部分
35 周縁部分
37 はみ出し部分
40 貼り合せ画像
100 スキャナ
110、111 撮像制御部
120、121 デジタル顕微鏡
130、131 貼り合せ部
140 失敗検出部
150 設定情報生成部
160 通信部
200 サーバ
210 通信部
220 記憶部
230 制御部
240 失敗検出部
250 設定情報生成部
300 ビューア
310 通信部
320 入力部
330 制御部
331 取得部
333 表示制御部
340 表示部
Claims (20)
- デジタル顕微鏡を用いて撮像される画像を評価することにより、前記画像に関連する再撮像を要する失敗を検出する検出部と、
前記検出部により前記失敗が検出された場合に、再撮像の際の撮像条件を設定するための設定情報を生成する生成部と、
を備える情報処理装置。 - 前記検出部は、前記失敗として、複数の画像を貼り合せることにより生成される貼り合せ画像に関連する失敗を検出する、請求項1に記載の情報処理装置。
- 前記貼り合せ画像に関連する前記失敗は、前記貼り合せ画像に含まれるべき領域の撮像漏れを含む、請求項2に記載の情報処理装置。
- 前記検出部は、前記画像におけるテクスチャの存在方向を評価することにより、前記撮像漏れを検出する、請求項3に記載の情報処理装置。
- 前記生成部は、前記検出部により前記撮像漏れが検出された場合に、新たに撮像されるべき追加領域を含むように撮像領域を再設定するための撮像領域情報を生成する、請求項2に記載の情報処理装置。
- 前記生成部は、撮像領域が再設定される場合に、前記画像の撮像順を再設定するための撮像順情報をさらに生成する、請求項5に記載の情報処理装置。
- 前記貼り合せ画像に関連する前記失敗は、前記貼り合せ画像を生成する際の貼り合せ失敗を含む、請求項2に記載の情報処理装置。
- 前記検出部は、前記貼り合せ画像に含まれる所定の方向のエッジを評価することにより、前記貼り合せ失敗を検出する、請求項7に記載の情報処理装置。
- 前記検出部は、前記貼り合せ画像が生成される際の重なり部分の輝度値を評価することにより、前記貼り合せ失敗を検出する、請求項7に記載の情報処理装置。
- 前記生成部は、前記検出部により前記貼り合せ失敗が検出された場合に、前記画像の撮像順を再設定するための撮像順情報を生成する、請求項7に記載の情報処理装置。
- 前記検出部は、前記失敗として、フォーカス位置、ホワイトバランス又は明るさの不具合をさらに検出する、請求項2に記載の情報処理装置。
- 前記情報処理装置は、前記検出部により検出される前記失敗の種類に応じて、前記デジタル顕微鏡に個々の画像を再撮像させるか複数の画像の全体を再撮像させるかを判定する撮像制御部、をさらに備える、請求項2に記載の情報処理装置。
- 前記検出部は、前記複数の画像の各々が撮像される都度、前記失敗の検出のために各画像を評価する、請求項2に記載の情報処理装置。
- 前記検出部は、前記貼り合せ画像が生成された後に、前記貼り合せ画像又は前記貼り合せ画像の部分画像を評価する、請求項2に記載の情報処理装置。
- 前記情報処理装置は、
前記生成部により生成される前記設定情報を、再撮像が実行される前にユーザに呈示し又はユーザにより編集可能とする表示制御部、
をさらに備える、請求項1に記載の情報処理装置。 - デジタル顕微鏡を用いて撮像される画像を評価することにより、前記画像に関連する再撮像を要する失敗を検出することと、
前記検出部により前記失敗が検出された場合に、再撮像の際の撮像条件を設定するための設定情報を生成することと、
を含む撮像制御方法。 - コンピュータを、
デジタル顕微鏡を用いて撮像される画像を評価することにより、前記画像に関連する再撮像を要する失敗を検出する検出部と、
前記検出部により前記失敗が検出された場合に、再撮像の際の撮像条件を設定するための設定情報を生成する生成部と、
として機能させるためのプログラム。 - デジタル顕微鏡と、
前記デジタル顕微鏡を用いて撮像される画像を評価することにより、前記画像に関連する再撮像を要する失敗を検出する検出部、及び
前記検出部により前記失敗が検出された場合に、再撮像の際の撮像条件を設定するための設定情報を生成する生成部
を備える情報処理装置と、
を含むデジタル顕微鏡システム。 - デジタル顕微鏡を用いて撮像される画像を評価することにより前記画像に関連する再撮像を要する失敗が検出された場合に生成される、再撮像の際の撮像条件を設定するための設定情報を取得する取得部と、
前記取得部により取得される前記設定情報を表示面に表示させる表示制御部と、
を備える表示制御装置。 - デジタル顕微鏡を用いて撮像される複数の画像を貼り合せることで生成される貼り合せ画像に関連する失敗を、画像を評価することにより検出する検出部と、
前記検出部により前記失敗が検出された場合に、再撮像の際の撮像条件を設定するための設定情報を生成する生成部と、
を備える情報処理装置。
Priority Applications (6)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
EP12865325.0A EP2804038B1 (en) | 2012-01-11 | 2012-12-07 | Information processing device, imaging control method, program, digital microscope system |
JP2013553213A JP6102749B2 (ja) | 2012-01-11 | 2012-12-07 | 情報処理装置、撮像制御方法、プログラム、デジタル顕微鏡システム、表示制御装置、表示制御方法及びプログラム |
US14/369,227 US10509218B2 (en) | 2012-01-11 | 2012-12-07 | Information processing apparatus, imaging control method, program, digital microscope system, display control apparatus, display control method, and program including detection of a failure requiring reimaging |
CN201280066139.5A CN104081248A (zh) | 2012-01-11 | 2012-12-07 | 信息处理装置、成像控制方法、程序、数字显微镜***、显示控制装置、显示控制方法和程序 |
US16/690,621 US10983329B2 (en) | 2012-01-11 | 2019-11-21 | Information processing apparatus, imaging control method, program, digital microscope system, display control apparatus, display control method, and program including detection of a failure requiring reimaging |
US17/212,918 US11422356B2 (en) | 2012-01-11 | 2021-03-25 | Information processing apparatus, imaging control method, program, digital microscope system, display control apparatus, display control method, and program including detection of a failure requiring reimaging |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
JP2012-003214 | 2012-01-11 | ||
JP2012003214 | 2012-01-11 |
Related Child Applications (2)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US14/369,227 A-371-Of-International US10509218B2 (en) | 2012-01-11 | 2012-12-07 | Information processing apparatus, imaging control method, program, digital microscope system, display control apparatus, display control method, and program including detection of a failure requiring reimaging |
US16/690,621 Continuation US10983329B2 (en) | 2012-01-11 | 2019-11-21 | Information processing apparatus, imaging control method, program, digital microscope system, display control apparatus, display control method, and program including detection of a failure requiring reimaging |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2013105373A1 true WO2013105373A1 (ja) | 2013-07-18 |
Family
ID=48781333
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/JP2012/081801 WO2013105373A1 (ja) | 2012-01-11 | 2012-12-07 | 情報処理装置、撮像制御方法、プログラム、デジタル顕微鏡システム、表示制御装置、表示制御方法及びプログラム |
Country Status (5)
Country | Link |
---|---|
US (3) | US10509218B2 (ja) |
EP (1) | EP2804038B1 (ja) |
JP (1) | JP6102749B2 (ja) |
CN (1) | CN104081248A (ja) |
WO (1) | WO2013105373A1 (ja) |
Cited By (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2016126242A (ja) * | 2015-01-07 | 2016-07-11 | キヤノン株式会社 | 画像処理装置、画像処理方法およびプログラム |
JP2016177037A (ja) * | 2015-03-19 | 2016-10-06 | 株式会社ニコン | 観察装置、観察方法、及びプログラム |
JP2017005555A (ja) * | 2015-06-12 | 2017-01-05 | リコーイメージング株式会社 | 撮影装置、撮影制御装置、撮影制御方法、及び撮影制御プログラム |
JP2017009718A (ja) * | 2015-06-19 | 2017-01-12 | 株式会社島津製作所 | 顕微鏡 |
JP2017102405A (ja) * | 2015-12-04 | 2017-06-08 | オリンパス株式会社 | 顕微鏡、画像貼り合わせ方法、プログラム |
WO2019065050A1 (ja) * | 2017-09-27 | 2019-04-04 | 富士フイルム株式会社 | 観察装置、観察方法および観察プログラム |
JP2021036213A (ja) * | 2019-08-30 | 2021-03-04 | 富士通株式会社 | 撮影漏れ検出装置、及び、撮影漏れ検出方法 |
Families Citing this family (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP5557091B2 (ja) * | 2010-03-30 | 2014-07-23 | 日本電気株式会社 | 画像処理装置、画像読取装置、画像処理方法及び画像処理プログラム |
EP2804038B1 (en) * | 2012-01-11 | 2018-11-14 | Sony Corporation | Information processing device, imaging control method, program, digital microscope system |
DE102016110988A1 (de) * | 2016-06-15 | 2017-12-21 | Sensovation Ag | Verfahren zum digitalen Aufnehmen einer Probe durch ein Mikroskop |
JP6887910B2 (ja) * | 2017-08-01 | 2021-06-16 | 株式会社東芝 | 画像処理装置 |
US10861156B2 (en) * | 2018-02-28 | 2020-12-08 | Case Western Reserve University | Quality control for digital pathology slides |
US10641659B2 (en) * | 2018-08-14 | 2020-05-05 | Shimadzu Corporation | Infrared microscope with adjustable connection optical system |
JP2021135981A (ja) * | 2020-02-28 | 2021-09-13 | 京セラドキュメントソリューションズ株式会社 | データ連携システムおよび処理監視システム |
Citations (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20040105000A1 (en) * | 2002-11-29 | 2004-06-03 | Olymlpus Corporation | Microscopic image capture apparatus |
JP2004191959A (ja) * | 2002-11-29 | 2004-07-08 | Olympus Corp | 顕微鏡画像撮影装置 |
JP2006317406A (ja) * | 2005-05-16 | 2006-11-24 | Olympus Corp | 細胞画像撮像装置、細胞画像撮像方法、細胞画像撮像プログラム、及び細胞観察装置 |
US20070146483A1 (en) * | 2005-12-28 | 2007-06-28 | Moritoshi Ando | Injection apparatus and injection method |
JP2009134100A (ja) * | 2007-11-30 | 2009-06-18 | Sunx Ltd | 拡大観察装置 |
WO2010052929A1 (en) * | 2008-11-10 | 2010-05-14 | Canon Kabushiki Kaisha | Image processing apparatus, image processing method, program, and program recording medium |
JP2010230495A (ja) | 2009-03-27 | 2010-10-14 | Sony Corp | 観察装置 |
Family Cites Families (13)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US7369304B2 (en) | 1999-10-29 | 2008-05-06 | Cytyc Corporation | Cytological autofocusing imaging systems and methods |
US7433025B2 (en) * | 2002-04-10 | 2008-10-07 | Thermo Fisher Scientific (Asheville) Llc | Automated protein crystallization imaging |
US7746404B2 (en) * | 2003-11-10 | 2010-06-29 | Hewlett-Packard Development Company, L.P. | Digital camera with panoramic image capture |
US7792338B2 (en) * | 2004-08-16 | 2010-09-07 | Olympus America Inc. | Method and apparatus of mechanical stage positioning in virtual microscopy image capture |
US7877213B2 (en) * | 2004-09-23 | 2011-01-25 | Agilent Technologies, Inc. | System and methods for automated processing of multiple chemical arrays |
US7336847B2 (en) * | 2004-10-11 | 2008-02-26 | Benq Corporation | Texture error recovery method using edge preserving spatial interpolation |
JP2008535528A (ja) * | 2005-01-18 | 2008-09-04 | トレストル コーポレーション | スライドの可変品質画像を形成するためのシステムおよび方法 |
DE102005017642B4 (de) * | 2005-04-15 | 2010-04-08 | Vistec Semiconductor Systems Jena Gmbh | Verfahren zur Inspektion eines Wafers |
US20060256397A1 (en) * | 2005-05-12 | 2006-11-16 | Lexmark International, Inc. | Method and system for combining images |
US7424218B2 (en) * | 2005-07-28 | 2008-09-09 | Microsoft Corporation | Real-time preview for panoramic images |
US8396269B2 (en) * | 2010-04-08 | 2013-03-12 | Digital Pathco LLC | Image quality assessment including comparison of overlapped margins |
JP5887766B2 (ja) * | 2011-08-31 | 2016-03-16 | 株式会社ニコン | 顕微鏡制御装置、画像処理装置、顕微鏡装置およびプログラム |
EP2804038B1 (en) | 2012-01-11 | 2018-11-14 | Sony Corporation | Information processing device, imaging control method, program, digital microscope system |
-
2012
- 2012-12-07 EP EP12865325.0A patent/EP2804038B1/en active Active
- 2012-12-07 WO PCT/JP2012/081801 patent/WO2013105373A1/ja active Application Filing
- 2012-12-07 US US14/369,227 patent/US10509218B2/en active Active
- 2012-12-07 CN CN201280066139.5A patent/CN104081248A/zh active Pending
- 2012-12-07 JP JP2013553213A patent/JP6102749B2/ja active Active
-
2019
- 2019-11-21 US US16/690,621 patent/US10983329B2/en active Active
-
2021
- 2021-03-25 US US17/212,918 patent/US11422356B2/en active Active
Patent Citations (14)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2004191959A (ja) * | 2002-11-29 | 2004-07-08 | Olympus Corp | 顕微鏡画像撮影装置 |
US20040105000A1 (en) * | 2002-11-29 | 2004-06-03 | Olymlpus Corporation | Microscopic image capture apparatus |
JP2006317406A (ja) * | 2005-05-16 | 2006-11-24 | Olympus Corp | 細胞画像撮像装置、細胞画像撮像方法、細胞画像撮像プログラム、及び細胞観察装置 |
DE602006010018D1 (de) * | 2005-12-28 | 2009-12-10 | Fujitsu Ltd | Injektionsvorrichtung sowie Injektionsmethode |
US20070146483A1 (en) * | 2005-12-28 | 2007-06-28 | Moritoshi Ando | Injection apparatus and injection method |
EP1803806A2 (en) * | 2005-12-28 | 2007-07-04 | Fujitsu Limited | Injection apparatus and injection method |
JP2007175026A (ja) * | 2005-12-28 | 2007-07-12 | Fujitsu Ltd | インジェクション装置およびインジェクション方法 |
JP2009134100A (ja) * | 2007-11-30 | 2009-06-18 | Sunx Ltd | 拡大観察装置 |
WO2010052929A1 (en) * | 2008-11-10 | 2010-05-14 | Canon Kabushiki Kaisha | Image processing apparatus, image processing method, program, and program recording medium |
JP2010110556A (ja) * | 2008-11-10 | 2010-05-20 | Canon Inc | 画像処理装置、画象処理方法、プログラム、及びプログラム記憶媒体 |
KR20110091739A (ko) * | 2008-11-10 | 2011-08-12 | 캐논 가부시끼가이샤 | 화상 처리 장치, 화상 처리 방법, 단층상 촬상 장치 및 프로그램 기억 매체 |
EP2355689A1 (en) * | 2008-11-10 | 2011-08-17 | Canon Kabushiki Kaisha | Image processing apparatus, image processing method, program, and program recording medium |
CN102209488A (zh) * | 2008-11-10 | 2011-10-05 | 佳能株式会社 | 图像处理设备、图像处理方法、程序和程序记录介质 |
JP2010230495A (ja) | 2009-03-27 | 2010-10-14 | Sony Corp | 観察装置 |
Non-Patent Citations (1)
Title |
---|
See also references of EP2804038A4 |
Cited By (9)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2016126242A (ja) * | 2015-01-07 | 2016-07-11 | キヤノン株式会社 | 画像処理装置、画像処理方法およびプログラム |
JP2016177037A (ja) * | 2015-03-19 | 2016-10-06 | 株式会社ニコン | 観察装置、観察方法、及びプログラム |
JP2017005555A (ja) * | 2015-06-12 | 2017-01-05 | リコーイメージング株式会社 | 撮影装置、撮影制御装置、撮影制御方法、及び撮影制御プログラム |
JP2017009718A (ja) * | 2015-06-19 | 2017-01-12 | 株式会社島津製作所 | 顕微鏡 |
JP2017102405A (ja) * | 2015-12-04 | 2017-06-08 | オリンパス株式会社 | 顕微鏡、画像貼り合わせ方法、プログラム |
WO2019065050A1 (ja) * | 2017-09-27 | 2019-04-04 | 富士フイルム株式会社 | 観察装置、観察方法および観察プログラム |
JPWO2019065050A1 (ja) * | 2017-09-27 | 2020-02-27 | 富士フイルム株式会社 | 観察装置、観察方法および観察プログラム |
JP2021036213A (ja) * | 2019-08-30 | 2021-03-04 | 富士通株式会社 | 撮影漏れ検出装置、及び、撮影漏れ検出方法 |
JP7263983B2 (ja) | 2019-08-30 | 2023-04-25 | 富士通株式会社 | 撮影漏れ検出装置、及び、撮影漏れ検出方法 |
Also Published As
Publication number | Publication date |
---|---|
US20210208381A1 (en) | 2021-07-08 |
US11422356B2 (en) | 2022-08-23 |
US10983329B2 (en) | 2021-04-20 |
JPWO2013105373A1 (ja) | 2015-05-11 |
JP6102749B2 (ja) | 2017-03-29 |
EP2804038B1 (en) | 2018-11-14 |
EP2804038A4 (en) | 2015-08-12 |
US10509218B2 (en) | 2019-12-17 |
EP2804038A1 (en) | 2014-11-19 |
US20200088986A1 (en) | 2020-03-19 |
CN104081248A (zh) | 2014-10-01 |
US20140362204A1 (en) | 2014-12-11 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
JP6102749B2 (ja) | 情報処理装置、撮像制御方法、プログラム、デジタル顕微鏡システム、表示制御装置、表示制御方法及びプログラム | |
JP5645051B2 (ja) | 画像処理装置 | |
JP5645052B2 (ja) | 画像処理装置 | |
JP5281972B2 (ja) | 撮像装置 | |
JP5218177B2 (ja) | 画像処理装置および方法 | |
JP5984493B2 (ja) | 画像処理装置、画像処理方法、撮像装置およびプログラム | |
US20080063273A1 (en) | Character extracting apparatus, method, and program | |
US8854448B2 (en) | Image processing apparatus, image display system, and image processing method and program | |
US8774551B2 (en) | Image processing apparatus and image processing method for reducing noise | |
JP2011203811A (ja) | 画像処理装置、画像処理方法、画像処理プログラム、及び複眼デジタルカメラ | |
JP2011048759A (ja) | 情報処理装置、情報処理システム、情報処理方法及び当該情報処理方法をコンピュータに実行させるプログラム | |
WO2011024287A1 (ja) | 情報処理装置、情報処理システム、情報処理方法及び情報処理方法を実行するためのコンピュータプログラム | |
JP2013153429A (ja) | 画像処理装置、画像表示システム、画像処理方法および画像処理プログラム | |
JP5586909B2 (ja) | 情報処理装置、システム、方法及びプログラム | |
JP2017224974A (ja) | 撮像装置および撮像システム並びに外観検査装置および製造方法 | |
JP2017055916A (ja) | 画像生成装置、画像生成方法及びプログラム | |
JP2013148856A (ja) | 撮像装置、表示画面生成装置、撮像システム、及びその制御方法 | |
JP4764169B2 (ja) | 撮影レンズ位置制御装置 | |
JP6409460B2 (ja) | 画像処理装置、および、コンピュータプログラム | |
JP6124676B2 (ja) | 撮像装置、撮像システム、撮像装置の制御方法、プログラム、および、記憶媒体 | |
JP2006094237A (ja) | 画像処理装置、画像処理方法および画像処理プログラム | |
JP2023068507A (ja) | パラメータ調整支援装置及びパラメータ調整支援方法 | |
JP5733706B2 (ja) | 画像処理装置、方法、及びプログラム | |
JP2010136280A (ja) | 画像処理装置、画像処理方法、ソフトウェアと、ソフトウェアを記録する記録媒体 | |
JP2008141229A (ja) | レンズ性能評価装置およびレンズ評価性能方法 |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 12865325 Country of ref document: EP Kind code of ref document: A1 |
|
WWE | Wipo information: entry into national phase |
Ref document number: 14369227 Country of ref document: US |
|
ENP | Entry into the national phase |
Ref document number: 2013553213 Country of ref document: JP Kind code of ref document: A |
|
WWE | Wipo information: entry into national phase |
Ref document number: 2012865325 Country of ref document: EP |
|
NENP | Non-entry into the national phase |
Ref country code: DE |