WO2015104763A1 - 分析システム、分析プログラム及び分析方法 - Google Patents
分析システム、分析プログラム及び分析方法 Download PDFInfo
- Publication number
- WO2015104763A1 WO2015104763A1 PCT/JP2014/006327 JP2014006327W WO2015104763A1 WO 2015104763 A1 WO2015104763 A1 WO 2015104763A1 JP 2014006327 W JP2014006327 W JP 2014006327W WO 2015104763 A1 WO2015104763 A1 WO 2015104763A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- reference position
- moving image
- analysis system
- analysis
- unit
- Prior art date
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0012—Biomedical image inspection
- G06T7/0014—Biomedical image inspection using an image reference approach
- G06T7/0016—Biomedical image inspection using an image reference approach involving temporal comparison
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B21/00—Microscopes
- G02B21/0004—Microscopes specially adapted for specific applications
- G02B21/002—Scanning microscopes
- G02B21/0024—Confocal scanning microscopes (CSOMs) or confocal "macroscopes"; Accessories which are not restricted to use with CSOMs, e.g. sample holders
- G02B21/0028—Confocal scanning microscopes (CSOMs) or confocal "macroscopes"; Accessories which are not restricted to use with CSOMs, e.g. sample holders specially adapted for specific applications, e.g. for endoscopes, ophthalmoscopes, attachments to conventional microscopes
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B21/00—Microscopes
- G02B21/0004—Microscopes specially adapted for specific applications
- G02B21/002—Scanning microscopes
- G02B21/0024—Confocal scanning microscopes (CSOMs) or confocal "macroscopes"; Accessories which are not restricted to use with CSOMs, e.g. sample holders
- G02B21/0052—Optical details of the image generation
- G02B21/0056—Optical details of the image generation based on optical coherence, e.g. phase-contrast arrangements, interference arrangements
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B21/00—Microscopes
- G02B21/0004—Microscopes specially adapted for specific applications
- G02B21/002—Scanning microscopes
- G02B21/0024—Confocal scanning microscopes (CSOMs) or confocal "macroscopes"; Accessories which are not restricted to use with CSOMs, e.g. sample holders
- G02B21/0052—Optical details of the image generation
- G02B21/006—Optical details of the image generation focusing arrangements; selection of the plane to be imaged
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B21/00—Microscopes
- G02B21/24—Base structure
- G02B21/241—Devices for focusing
- G02B21/244—Devices for focusing using image analysis techniques
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B21/00—Microscopes
- G02B21/36—Microscopes arranged for photographic purposes or projection purposes or digital imaging or video purposes including associated control and data processing arrangements
- G02B21/365—Control or image processing arrangements for digital or video microscopes
- G02B21/367—Control or image processing arrangements for digital or video microscopes providing an output produced by processing a plurality of individual source images, e.g. image tiling, montage, composite images, depth sectioning, image comparison
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/50—Depth or shape recovery
- G06T7/55—Depth or shape recovery from multiple images
- G06T7/571—Depth or shape recovery from multiple images from focus
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10016—Video; Image sequence
- G06T2207/10021—Stereoscopic video; Stereoscopic image sequence
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10056—Microscopic image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10141—Special mode during image acquisition
- G06T2207/10148—Varying focus
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30004—Biomedical image processing
- G06T2207/30024—Cell structures in vitro; Tissue sections in vitro
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30004—Biomedical image processing
- G06T2207/30048—Heart; Cardiac
Definitions
- the present technology relates to an analysis system, an analysis program, and an analysis method for analyzing the state of an observation object from a moving image captured by a microscope.
- Non-Patent Document 1 Three-dimensional culture is different from two-dimensional culture in which a cell group forms a monolayer two-dimensionally, and it is possible to obtain a three-dimensionally connected cell group. A group can be obtained.
- three-dimensional recognition In observing such a three-dimensionally cultured cell as an observation object, it is required to realize three-dimensional recognition (three-dimensional recognition) of the state of the observation object.
- a method for realizing three-dimensional recognition there is a method using a scanning microscope system such as a scanning two-photon excitation laser microscope or a confocal laser microscope.
- a scanning microscope system such as a scanning two-photon excitation laser microscope or a confocal laser microscope.
- three-dimensional recognition of an observation object is realized by scanning two-dimensionally while changing the focal position of the laser in the optical axis direction.
- three-dimensional recognition is realized by performing image processing on an image captured using a microscope. For example, it is possible to obtain a three-dimensional image of an observation target by using an autofocus function for detecting a focal plane and a function for generating an omnifocal image.
- the three-dimensionally cultured cells are in a state close to the living body as described above, and if the state can be observed over time, a lot of information can be obtained.
- three-dimensional recognition is performed using a scanning microscope system, since the laser is scanned with dots, it is difficult to increase the speed, and it is unsuitable for observing changes over time of the observation object.
- a three-dimensional image of an observation target is generated using a still image captured at a specific focal length.
- it is possible to observe the instantaneous state of the observation object it is not suitable for observing changes over time of the observation object.
- an object of the present technology is to provide an analysis system, an analysis program, and an analysis method suitable for analyzing a three-dimensional and temporal state of an observation object.
- an analysis system includes a feature amount calculation unit and a reference position calculation unit.
- the feature amount calculation unit includes a plurality of moving images in which samples including an observation target are captured at different focal positions, and a plurality of moving images having the same motion phase in each moving image. The feature amount is calculated.
- the reference position calculation unit calculates a reference position for the observation object in the focal direction based on the feature amount.
- the feature amount calculation unit uses a moving image having the same motion phase as a feature amount calculation target, so that the moving image captured at each focal position is captured at different time zones.
- the reference position calculation unit calculates the reference position, so that the movement of the observation object in the focal direction can be quantitatively evaluated.
- the analysis system can perform analysis by image processing, and since it is not necessary to irradiate the observation target with a laser or the like, the analysis can be performed without inhibiting the physiological state of the observation target. Yes.
- the analysis system includes a motion analysis unit that analyzes motion for each of the plurality of moving images, and a phase matching unit that matches the phases of the plurality of moving images using a motion analysis result by the motion analysis unit. Furthermore, you may comprise.
- the motion analysis unit extracts a pulsation timing due to the pulsation of the observation object for each of the plurality of moving images, and the phase matching unit includes the pulsation timing between the plurality of moving images.
- the phases of the plurality of moving images may be matched so as to match.
- the motion analysis unit can extract a characteristic point of motion as a pulsation timing, and the phase matching unit can synchronize the phases of a plurality of moving images using the pulsation timing. It becomes.
- the feature amount may be a luminance difference
- the reference position calculation unit may calculate the reference position based on the luminance difference
- the observation object included in the moving image can be detected using the luminance difference. Therefore, when the feature amount calculation unit calculates the luminance difference as the feature amount, the reference position calculation unit can calculate the reference position using the feature amount.
- the reference position calculation unit may set a position where the luminance difference is maximum in the focal direction as the reference position.
- the position where the luminance difference is maximum in the focal direction is the position where the focus is best when the moving image is captured, it is possible to grasp the arrangement of the observation object in the focal direction from the position. Therefore, the position where the luminance difference is maximum in the focal direction can be used as the reference position.
- the reference position calculation unit may estimate the reference position based on the correlation of the luminance difference in the focal direction.
- the reference position calculation unit can estimate the reference position using the correlation of the luminance difference.
- the analysis system may further include a result presentation unit that quantifies the movement in the focal direction based on the time variation of the reference position and images it.
- an analysis program includes a feature amount calculation unit and a reference position calculation unit.
- the feature amount calculation unit includes a plurality of moving images in which samples including an observation target are captured at different focal positions, and a plurality of moving images having the same motion phase in each moving image. The feature amount is calculated.
- the reference position calculation unit calculates a reference position for the observation object in the focal direction based on the feature amount.
- the feature amount calculation unit includes a plurality of moving images in which samples including an observation target are captured at different focal positions, A feature amount in each moving image is calculated for a plurality of moving images having the same movement phase.
- the reference position calculation unit calculates a reference position for the observation object in the focal direction based on the feature amount.
- FIG. 1 is a schematic diagram illustrating a configuration of an analysis system 100 according to the present embodiment.
- the analysis system 100 includes a moving image capturing unit 110 and an image processing unit 120.
- the moving image capturing unit 110 may be an optical microscope such as a bright field microscope or a phase contrast microscope
- the image processing unit 120 may be an information processing apparatus such as a personal computer. Note that the image processing unit 120 may be configured integrally with the moving image capturing unit 110.
- FIG. 1 shows an observation sample S.
- the observation sample S is not particularly limited, it can be assumed that cells are accommodated together with the culture solution, and can be, for example, a three-dimensionally cultured cardiomyocyte.
- this cell is assumed to be an observation object and is referred to as an observation object P.
- the analysis system 100 analyzes the observation object P using regular movement (pulsation or the like) of the observation object P. For this reason, cells (including cell groups) and biological tissues that cause regular movement can be used as the observation object P of the analysis system 100.
- the moving image capturing unit 110 captures the observation sample S and generates a plurality of moving images.
- the moving image capturing unit 110 includes at least a microscope optical system and an imaging element such as a CCD (Charge Coupled Device) image sensor, and can capture a moving image of the observation sample S via the microscope optical system.
- CCD Charge Coupled Device
- the moving image capturing unit 110 can change the focal position (the position of the focal plane in the observation sample S) for capturing a moving image.
- FIG. 2 is a schematic diagram illustrating a focal position at which the moving image capturing unit 110 captures a moving image.
- the moving image imaging unit 110 images the observation sample S at a plurality of focal positions (for example, focal positions F1 to F3).
- the focal direction (the optical axis direction of the microscope optical system) is the Z direction
- one direction orthogonal to the Z direction is the X direction
- the direction orthogonal to the Z direction and the X direction is the Y direction.
- the number of focal positions at which a moving image is captured is not particularly limited, and can be changed as appropriate according to the size of the observation object P or the like.
- the interval between the focal positions is preferably a constant interval in the Z direction, and may be, for example, a few ⁇ m interval.
- the accuracy of calculation of a reference Z position improves as the interval between the focal positions becomes smaller. Further, it is meaningful that the distance between the focal positions is less than or equal to the resolution of the microscope optical system. For example, when the resolution of the microscope optical system is 2 ⁇ m, the accuracy is improved even if the distance between the focal positions is 1 ⁇ m.
- the moving image capturing unit 110 captures a moving image for a fixed time for one focal position, moves the focal position in the Z direction, captures a moving image for a fixed time again, and repeats this a predetermined number of times. That is, a moving image captured at each focal position (hereinafter referred to as a captured moving image) is a moving image captured at different times. Note that the moving image capturing unit 110 can move the focal position only in the focal direction (Z direction) and not in other directions (X direction and Y direction).
- FIG. 3 is a schematic diagram showing the positional relationship of the captured moving images, and shows the captured moving images G1 to G3 captured at the focal positions F1 to F3, respectively.
- captured moving images are captured at a plurality of focal positions. That is, the captured moving image is a Z-stack moving image.
- the number of captured moving images is not particularly limited, and can be changed as appropriate according to the size of the observation object P or the like.
- FIG. 4 is an example of the captured moving images G1 to G3.
- the moving image capturing unit 110 outputs the captured moving image and the captured focal position to the image processing unit 120.
- the image processing unit 120 includes a moving image acquisition unit 121, a motion analysis unit 122, a phase matching unit 123, a feature amount calculation unit 124, a reference position calculation unit 125, and a result presentation unit 126.
- Each of these configurations may be a functional configuration realized by cooperation of hardware and software.
- FIG. 5 is a flowchart showing the operation of the image processing unit 120.
- the moving image acquisition unit 121 acquires a plurality of captured moving images captured at different focal positions from the moving image capturing unit 110 (St101). At this time, the moving image acquisition unit 121 acquires the focal position where each captured moving image is captured together with the captured moving image.
- the moving image acquisition unit 121 may acquire a captured moving image via a network or a storage.
- the moving image acquisition unit 121 supplies a plurality of captured moving images and the focal position where each captured moving image is captured to the motion analysis unit 122.
- the motion analysis unit 122 analyzes the motion in each captured moving image (St102). Specifically, the motion analysis unit 122 calculates a motion vector in the XY direction for each pixel of each captured moving image or for each pixel group including a plurality of pixels.
- the motion vector can be calculated by block matching, for example.
- the motion analysis unit 122 can analyze the motion of each captured moving image from the motion amount and motion speed of the calculated motion vector.
- FIG. 6 is a graph showing an example of a motion analysis result for a captured moving image.
- the observation object P has a regular movement (pulsation), and thus the regularity of the movement of the observation object P included in each captured moving image appears in the analysis result.
- the motion analysis unit 122 extracts a characteristic point of motion as a pulsation timing (number in the figure).
- the characteristic points can be, for example, the maximum point and the minimum point of the movement amount as shown in FIG.
- pulsation timings 1 to 5 shown in FIG. 6 are used.
- the number of pulsation timings is not particularly limited.
- the motion analysis unit can extract five pulsation timings.
- the motion analysis unit 122 can determine the pulsation timing according to the type of the observation object P (the type of pulsation).
- the phase matching unit 123 matches the phase of the captured moving image using the motion analysis result by the motion analysis unit 122 (St103).
- FIG. 7 is a schematic diagram showing the phase matching by the phase matching unit 123, and shows the motions g1 to g3 calculated from the captured moving images G1 to G3.
- the moving image capturing unit 110 captures a moving image for a predetermined time at each focal position. For this reason, as shown in FIG. 7A, the imaging time (the extraction times of the motions g1 to g3) of the captured moving images captured at the respective focal positions are different from each other.
- the phase matching unit 123 matches the phase of each captured moving image using the pulsation timing. Specifically, the moving image capturing unit 110 matches the timing of each captured moving image so that the pulsation timings of one captured moving image and another captured moving image are aligned.
- the phase matching unit 123 supplies a plurality of captured moving images whose phases are matched to the feature amount calculation unit 124.
- the feature amount calculation unit 124 calculates a feature amount for each captured moving image (St104).
- the feature amount is an amount representing the feature of the moving image, and can be, for example, a luminance difference (a difference in luminance between adjacent or surrounding pixels).
- FIG. 8 shows luminance difference moving images P1 to P3 which are calculation results of the luminance difference by the feature amount calculation unit 124.
- the luminance difference moving images P1 to P3 are moving images having luminance differences calculated from the captured moving images G1 to G3, respectively. Since the captured moving images G1 to G3 are moving images, the luminance difference also varies with time.
- the calculation of the luminance difference is also called a contrast method, and it is possible to detect a portion with high contrast in the captured moving image.
- the feature amount calculation unit 124 calculates a feature amount from each captured moving image. Note that the feature amount calculation unit 124 may calculate feature amounts other than the luminance difference, or may calculate a plurality of types of feature amounts from one captured moving image.
- the reference position calculation unit 125 calculates a reference position for the observation object in the Z direction (St105). Specifically, the reference position calculation unit 125 specifies a calculation area for each captured moving image.
- FIG. 9 is a schematic diagram showing the calculation area L designated for each of the captured moving images G1 to G3. As shown in the figure, the reference position calculation unit 125 sets a calculation area L in a predetermined area of each of the captured moving images G1 to G3.
- the reference position calculation unit 125 can set the same region in the Z direction (the region having the same X coordinate and Y coordinate) as the calculation region L for each captured moving image as shown in FIG.
- the reference position calculation unit 125 may set a plurality of calculation regions L for each captured moving image, or may set the entire captured image as one calculation region L.
- FIG. 10 shows calculation areas L1 and L2 set for the captured moving images G1 to G3. The imaging magnification in FIG. 10 is 20 times the objective lens, the calculation region L1 is a region including the central part of the cell body of cardiomyocytes (observation target P), and the calculation region L2 is a region including the peripheral part of the cell body. is there.
- the reference position calculation unit 125 obtains a “reference Z position” for each set calculation area L by using the feature amount calculated from the moving image area included therein.
- FIG. 11 is a graph showing the luminance difference in the calculation region L1 for each pulsation timing.
- the horizontal axis of the graph is the Z position, that is, the focal position where the captured moving image is captured.
- the smaller Z position is the lower side (shallow focal position) of the observation object P (see FIG. 2).
- FIG. 12 is a graph showing the luminance difference in the calculation region L2 for each pulsation timing.
- the reference position calculation unit 125 obtains a reference Z position from the feature amount for each calculation region as shown in FIGS.
- the reference position calculation unit 125 can set the Z position where the luminance difference in each calculation region is maximum as the reference Z position.
- FIG. 13 is a diagram showing a method for calculating the reference Z position in the graph shown in FIG.
- the reference position calculation unit 125 can average the luminance difference for each calculation region (low-pass filter), calculate the Z position where the luminance difference is the largest, and set it as the reference Z position. Further, the reference position calculation unit 125 may estimate the Z position from the luminance difference value. The estimation of the Z position will be described later.
- FIG. 14 is a graph showing the luminance difference in the calculation area L1 with shading with respect to the pulsation timing and the Z position
- FIG. 15 shows the luminance difference in the calculation area L2 with respect to the pulsation timing and the Z position. It is a graph expressed by shading. 14 and 15, the reference Z position is indicated by a line M.
- the reference Z position in the calculation area L1 is a deeper Z position than the reference Z position in the calculation area L2.
- luminance difference by the pulsation timing differs in the brightness
- the luminance difference at the pulsation timings 1 and 5 is small, and the luminance difference at the pulsation timing 3 is the largest.
- the luminance difference at the pulsation timings 1 and 5 is large, and the luminance difference at the pulsation timing 3 is the smallest. This also appears as a difference in region shape in FIGS. 14 and 15.
- FIG. 16 is a schematic diagram showing the movement of the observation object P.
- the luminance difference in the calculation area L1 increases and the luminance difference in the calculation area L2 decreases at the pulsation timings 2 to 3. This is because the contrast increases as the central portion (corresponding to the calculation region L1) of the observation target P rises, while the peripheral portion (corresponding to the calculation region L2) of the observation target P is pulled and thinned. The decrease in contrast is considered to be a factor.
- the observation object P moves in the Z direction around the reference Z position.
- the reference position calculation unit 125 calculates the reference Z position, so that it is possible to quantitatively grasp the observation object movement in the Z direction.
- the reference position calculation unit 125 can quantify the movement change in the Z direction by recording the change with time of the reference Z position.
- the reference position calculation unit 125 can also quantify the movement change in the Z direction by tracing a place having the same luminance in the Z direction.
- the reference position calculation unit 125 supplies the reference Z position and the result of quantification to the result presentation unit 126.
- the result presentation unit 126 quantifies the movement of the observation object P in the Z direction, generates an image, and presents it to the user (St106). Specifically, the result presentation unit 126 can generate a result presentation image by superimposing the quantified movement in the Z direction for each region obtained by dividing the captured moving image.
- FIG. 17 is an example of a result presentation image.
- the result presentation image shown in the figure shows a region moving in the Z direction + in white and a region moving in the Z direction-in black. Thereby, for example, it is possible to visualize that the movement direction in the Z direction is different at each pulsation timing between the central portion and the peripheral portion of the cell.
- the result presentation unit 126 may present the feature amount calculated by the feature amount calculation unit 124.
- FIG. 18 shows the difference in luminance calculated from each captured moving image arranged in the XYZ directions along the XYZ coordinates of the captured moving image. As a result, the result presentation unit 126 can present the feature quantity three-dimensionally.
- the analysis system 100 performs the above operation. Information about the movement of the observation target P in the Z direction from the captured moving images captured at different focal positions by the phase matching unit 123 adjusting the phase of the captured moving images using the pulse timing of the captured moving images. Can be obtained.
- the reference position calculation unit 125 calculates the reference Z position, it is possible to quantitatively evaluate the movement of the observation object P in the Z direction. It is suitable for the analysis of the state over time.
- the analysis system 100 can perform the quantitative evaluation in the Z direction by image processing on the captured moving image, and does not need to irradiate a laser or the like. Therefore, the analysis system 100 can perform analysis without inhibiting the physiological state of the observation target P. It is possible.
- FIG. 19 shows an imaging moving image G obtained by imaging colony-like cardiomyocytes.
- the captured moving image G is a moving image captured at a plurality of focal positions
- FIG. 19 is one of the moving images.
- the moving image acquisition unit 121 acquires a plurality of moving images including the captured moving image G, and the phase of the captured moving image is adjusted by the motion analysis unit 122 and the phase matching unit 123.
- the feature amount calculation unit 124 calculates a feature amount for each captured moving image.
- the reference position calculation unit 125 sets a calculation area in the captured moving image G.
- FIG. 19 shows calculation areas L1 to L5. Each calculation area can include a plurality of cells forming a colony.
- the reference position calculation unit 125 obtains a reference Z position for each calculation region by using the feature amount calculated from the pixels included in each calculation region.
- the reference position calculation unit 125 calculates the maximum luminance difference value when the larger one of the focal depth or the focal interval (see FIG. 2) of the objective lens of the moving image capturing unit 110 is smaller than the thickness variation of the observation target P. Can do. On the other hand, the reference position calculation unit 125 has insufficient resolution when the larger one of the focal depth or the focal interval (see FIG. 2) of the objective lens of the moving image capturing unit 110 is larger than the thickness variation of the observation target P.
- the reference position calculation unit 125 can estimate the luminance difference maximum value. For example, when the depth of focus of the objective lens is 3.35 ⁇ m and the interval between the focal positions is 4 ⁇ m, the maximum luminance difference can be calculated when the thickness variation is 4 ⁇ m or more. Is required.
- FIG. 20 is a graph showing the luminance difference at the pulsation timing 1 in the calculation areas L1 to L5.
- the luminance difference maximum value in each of the calculation areas L1 to L5 is indicated by a white circle.
- the reference position calculation unit 125 can set the Z position where the luminance difference is maximum in the calculation regions L1 to L5 as the reference Z position of the calculation region.
- the luminance difference maximum value of the calculation region L1 is indicated as A and the reference Z position is indicated as B, but the same applies to the other calculation regions L.
- FIG. 21 is a graph showing the reference Z position shown in FIG. 20 for each calculation region, and the vertical axis is the reference Z position (B in FIG. 19). It can be seen that there is a difference of 30 ⁇ m in the reference Z position between the calculation area L1 and the calculation area L5.
- the reference position calculation unit 125 can also calculate the thickness variation (Z direction variation) due to the pulsation of the observation object P.
- FIG. 22 shows a luminance difference between pulsation timings 1 to 3 in the calculation area L1
- FIG. 23 shows a luminance difference between pulsation timings 1 to 3 in the calculation area L3.
- a difference in luminance difference due to the pulsation timing is hardly observed, but in the calculation area L3, a difference in luminance difference portion due to the pulsation timing occurs. This is considered because the movement of the observation target P in the Z direction in the calculation area L2 is larger than the movement of the observation target P in the calculation area L1 in the Z direction.
- the reference position calculation unit 125 can estimate the reference Z position not from the luminance difference maximum value but from the luminance difference value itself.
- FIG. 24 is a graph showing estimation of the reference Z position from the luminance difference value.
- the reference position calculation unit 125 has a correlation between the Z position (horizontal axis, B in FIG. 19) where the luminance difference becomes the maximum value and the maximum luminance difference value (vertical axis, A in FIG. 19).
- the reference Z position can be directly estimated from the luminance difference value.
- the feature amount calculation unit 124 calculates the feature amount for the captured moving image whose phase is matched by the phase matching unit 123.
- the phase may not be adjusted by 123.
- the moving image capturing unit 110 captures moving images at different focal positions in accordance with the pulsation timing, so that the captured moving images are in phase. Can be generated. Moreover, you may produce
- An analysis system for each of the plurality of moving images, a motion analysis unit that analyzes motion, An analysis system further comprising: a phase matching unit that matches phases of the plurality of moving images using a motion analysis result by the motion analysis unit.
- the motion analysis unit extracts a pulsation timing due to the pulsation of the observation object for each of the plurality of moving images
- the phase matching unit matches the phases of the plurality of moving images so that the pulsation timings coincide between the plurality of moving images.
- the analysis system according to any one of (1) to (3) above,
- the feature amount is a luminance difference
- the analysis system that calculates the reference position based on the luminance difference.
- the feature amount calculation unit is a plurality of moving images in which a sample including an observation target is captured at different focal positions, and a plurality of moving images having the same phase of movement of each moving image is characterized in each moving image. Calculate the quantity, An analysis method in which the reference position calculation unit calculates a reference position for the observation object in the focal direction based on the feature amount.
Landscapes
- Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Chemical & Material Sciences (AREA)
- Analytical Chemistry (AREA)
- Optics & Photonics (AREA)
- Theoretical Computer Science (AREA)
- Multimedia (AREA)
- General Health & Medical Sciences (AREA)
- Health & Medical Sciences (AREA)
- Radiology & Medical Imaging (AREA)
- Medical Informatics (AREA)
- Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
- Quality & Reliability (AREA)
- Ophthalmology & Optometry (AREA)
- Surgery (AREA)
- Microscoopes, Condenser (AREA)
- Investigating Or Analysing Materials By Optical Means (AREA)
- Image Analysis (AREA)
- Image Processing (AREA)
Abstract
Description
上記特徴量算出部は、観察対象物を含む試料が異なる焦点位置で撮像された複数の動画像であって、各動画像の動きの位相が等しい複数の動画像に対して、各動画像における特徴量を算出する。
上記基準位置算出部は、上記特徴量に基づいて、焦点方向における観察対象物についての基準位置を算出する。
上記特徴量算出部は、観察対象物を含む試料が異なる焦点位置で撮像された複数の動画像であって、各動画像の動きの位相が等しい複数の動画像に対して、各動画像における特徴量を算出する。
上記基準位置算出部は、上記特徴量に基づいて、焦点方向における観察対象物についての基準位置を算出する。
基準位置算出部が、上記特徴量に基づいて、焦点方向における観察対象物についての基準位置を算出する。
なお、ここに記載された効果は必ずしも限定されるものではなく、本開示中に記載されたいずれかの効果であってもよい。
本技術の実施形態に係る分析システムについて説明する。図1は、本実施形態に係る分析システム100の構成を示す模式図である。
上記分析システム100の説明では、細胞の中央部と周縁部によるZ方向の動きの差異について例示したが、分析システム100はコロニー様細胞について適用することも可能である。
上述のように特徴量算出部124は、位相合わせ部123によって位相が合わされた撮像動画像に対して特徴量を算出するとしたが、撮像動画像は位相が合っていればよく、必ずしも位相合わせ部123によって位相が合わされたものでなくてもよい。
観察対象物を含む試料が異なる焦点位置で撮像された複数の動画像であって、各動画像の動きの位相が等しい複数の動画像に対して、各動画像における特徴量を算出する特徴量算出部と、
上記特徴量に基づいて、焦点方向における観察対象物についての基準位置を算出する基準位置算出部と
を具備する分析システム。
上記(1)に記載の分析システムであって、
上記複数の動画像のそれぞれについて、動きを解析する動き解析部と、
上記動き解析部による動き解析結果を利用して、上記複数の動画像の位相を合わせる位相合わせ部と
をさらに具備する分析システム。
上記(1)又は(2)に記載の分析システムであって、
上記動き解析部は、上記複数の動画像のそれぞれについて、観察対象物の拍動に起因する拍動タイミングを抽出し、
上記位相合わせ部は、上記拍動タイミングが上記複数の動画像の間で一致するように上記複数の動画像の位相を合わせる
分析システム。
上記(1)から(3)のいずれか一つに記載の分析システムであって、
上記特徴量は輝度差分であり、
上記基準位置算出部は、上記輝度差分に基づいて上記基準位置を算出する
分析システム。
上記(1)から(4)のいずれか一つに記載の分析システムであって、
上記基準位置算出部は、焦点方向において上記輝度差分が最大となる位置を上記基準位置とする
分析システム。
上記(1)から(5)のいずれか一つに記載の分析システムであって
上記基準位置算出部は、焦点方向における上記輝度差分の相関関係に基づいて上記基準位置を推定する
分析システム。
上記(1)から(6)のいずれか一つに記載の分析システムであって
上記基準位置の時間変動に基づいて上記焦点方向の動きを定量化し、画像化する結果提示部
をさらに具備する分析システム。
観察対象物を含む試料が異なる焦点位置で撮像された複数の動画像であって、各動画像の動きの位相が等しい複数の動画像に対して、各動画像における特徴量を算出する特徴量算出部と、
上記特徴量に基づいて、焦点方向における観察対象物についての基準位置を算出する基準位置算出部と
を具備する分析プログラム。
特徴量算出部が、観察対象物を含む試料が異なる焦点位置で撮像された複数の動画像であって、各動画像の動きの位相が等しい複数の動画像に対して、各動画像における特徴量を算出し、
基準位置算出部が、上記特徴量に基づいて、焦点方向における観察対象物についての基準位置を算出する
分析方法。
110…動画像撮像部
120…画像処理ユニット
121…動画像取得部
122…動き解析部
123…位相合わせ部
124…特徴量算出部
125…基準位置算出部
126…結果提示部
Claims (9)
- 観察対象物を含む試料が異なる焦点位置で撮像された複数の動画像であって、各動画像の動きの位相が等しい複数の動画像に対して、各動画像における特徴量を算出する特徴量算出部と、
前記特徴量に基づいて、焦点方向における観察対象物についての基準位置を算出する基準位置算出部と
を具備する分析システム。 - 請求項1に記載の分析システムであって、
前記複数の動画像のそれぞれについて、動きを解析する動き解析部と、
前記動き解析部による動き解析結果を利用して、前記複数の動画像の位相を合わせる位相合わせ部と
をさらに具備する分析システム。 - 請求項2に記載の分析システムであって、
前記動き解析部は、前記複数の動画像のそれぞれについて、観察対象物の拍動に起因する拍動タイミングを抽出し、
前記位相合わせ部は、前記拍動タイミングが前記複数の動画像の間で一致するように前記複数の動画像の位相を合わせる
分析システム。 - 請求項1に記載の分析システムであって、
前記特徴量は輝度差分であり、
前記基準位置算出部は、前記輝度差分に基づいて前記基準位置を算出する
分析システム。 - 請求項4に記載の分析システムであって、
前記基準位置算出部は、焦点方向において前記輝度差分が最大となる位置を前記基準位置とする
分析システム。 - 請求項4に記載の分析システムであって、
前記基準位置算出部は、焦点方向における前記輝度差分の相関関係に基づいて前記基準位置を推定する
分析システム。 - 請求項1に記載の分析システムであって、
前記基準位置の時間変動に基づいて前記焦点方向の動きを定量化し、画像化する結果提示部
をさらに具備する分析システム。 - 観察対象物を含む試料が異なる焦点位置で撮像された複数の動画像であって、各動画像の動きの位相が等しい複数の動画像に対して、各動画像における特徴量を算出する特徴量算出部と、
前記特徴量に基づいて、焦点方向における観察対象物についての基準位置を算出する基準位置算出部と
を具備する分析プログラム。 - 特徴量算出部が、観察対象物を含む試料が異なる焦点位置で撮像された複数の動画像であって、各動画像の動きの位相が等しい複数の動画像に対して、各動画像における特徴量を算出し、
基準位置算出部が、前記特徴量に基づいて、焦点方向における観察対象物についての基準位置を算出する
分析方法。
Priority Applications (3)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
EP14877880.6A EP3076219B1 (en) | 2014-01-07 | 2014-12-18 | Analysis system, analysis program, and analysis method |
JP2015556639A JP6578950B2 (ja) | 2014-01-07 | 2014-12-18 | 分析システム、分析プログラム及び分析方法 |
US15/106,858 US10535137B2 (en) | 2014-01-07 | 2014-12-18 | Analysis system and analysis method |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
JP2014000843 | 2014-01-07 | ||
JP2014-000843 | 2014-01-07 |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2015104763A1 true WO2015104763A1 (ja) | 2015-07-16 |
Family
ID=53523622
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/JP2014/006327 WO2015104763A1 (ja) | 2014-01-07 | 2014-12-18 | 分析システム、分析プログラム及び分析方法 |
Country Status (4)
Country | Link |
---|---|
US (1) | US10535137B2 (ja) |
EP (1) | EP3076219B1 (ja) |
JP (1) | JP6578950B2 (ja) |
WO (1) | WO2015104763A1 (ja) |
Cited By (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2016052179A1 (ja) * | 2014-09-30 | 2016-04-07 | 富士フイルム株式会社 | 細胞撮像装置および方法 |
WO2017221597A1 (ja) * | 2016-06-21 | 2017-12-28 | ソニー株式会社 | 情報処理装置、情報処理方法、及びプログラム |
CN110866906A (zh) * | 2019-11-12 | 2020-03-06 | 安徽师范大学 | 基于图像边缘提取的三维培养人体心肌细胞搏动检测方法 |
Families Citing this family (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN112513707B (zh) * | 2018-04-17 | 2023-05-26 | 克莫麦特公司 | 对象的描绘 |
Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2003093378A (ja) * | 2001-07-09 | 2003-04-02 | Siemens Ag | コンピュータトモグラフィのための方法およびコンピュータトモグラフィ装置 |
WO2004109228A1 (ja) * | 2003-06-09 | 2004-12-16 | Sumitomo Osaka Cement Co.,Ltd. | 三次元形状測定装置 |
JP2005128086A (ja) | 2003-10-21 | 2005-05-19 | Olympus Corp | 走査型顕微鏡システム |
WO2005122901A1 (ja) * | 2004-06-16 | 2005-12-29 | Hitachi Medical Corporation | 放射線断層像撮像装置 |
JP2012531933A (ja) * | 2009-06-30 | 2012-12-13 | コーニンクレッカ フィリップス エレクトロニクス エヌ ヴィ | 非ecgの生体同期収集による三次元胎児心臓撮像方法を用いた超音波画像診断システム及び方法 |
Family Cites Families (37)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JPH0774856B2 (ja) * | 1986-10-16 | 1995-08-09 | オリンパス光学工業株式会社 | 自動焦点調節方法 |
US7142703B2 (en) * | 2001-07-17 | 2006-11-28 | Cedara Software (Usa) Limited | Methods and software for self-gating a set of images |
JP4497923B2 (ja) * | 2001-12-05 | 2010-07-07 | ザ ジェイ. デビッド グラッドストーン インスティテューツ | ロボット顕微鏡検査システム |
US7756305B2 (en) * | 2002-01-23 | 2010-07-13 | The Regents Of The University Of California | Fast 3D cytometry for information in tissue engineering |
US7194124B2 (en) * | 2002-04-09 | 2007-03-20 | University Of Iowa Research Foundation | Reconstruction and motion analysis of an embryo |
WO2004074500A2 (en) * | 2003-02-21 | 2004-09-02 | University Of Manitoba | Method of monitoring genomic instability using 3d microscopy and analysis |
US7415148B2 (en) * | 2003-08-04 | 2008-08-19 | Raytheon Company | System and method for detecting anomalous targets including cancerous cells |
US20050053267A1 (en) * | 2003-09-05 | 2005-03-10 | Varian Medical Systems Technologies, Inc. | Systems and methods for tracking moving targets and monitoring object positions |
US7388976B2 (en) * | 2004-03-09 | 2008-06-17 | Siemens Medical Solutions Usa, Inc. | Time-based system to link periodic X-ray images |
US7372984B2 (en) * | 2004-05-05 | 2008-05-13 | California Institute Of Technology | Four-dimensional imaging of periodically moving objects via post-acquisition synchronization of nongated slice-sequences |
WO2007037439A1 (ja) * | 2005-09-29 | 2007-04-05 | Olympus Corporation | 焦点位置決定方法、焦点位置決定装置、微弱光検出装置及び微弱光検出方法 |
EP2023127B1 (en) * | 2006-05-31 | 2017-12-20 | Olympus Corporation | Biological specimen imaging method and biological specimen imaging apparatus |
US7916304B2 (en) * | 2006-12-21 | 2011-03-29 | Howard Hughes Medical Institute | Systems and methods for 3-dimensional interferometric microscopy |
WO2008153836A2 (en) * | 2007-05-31 | 2008-12-18 | President And Fellows Of Harvard College | Target-locking acquisition with real-time confocal (tarc) microscopy |
WO2009115108A1 (en) * | 2008-03-19 | 2009-09-24 | Ruprecht-Karls-Universität Heidelberg | A method and an apparatus for localization of single dye molecules in the fluorescent microscopy |
US8184298B2 (en) * | 2008-05-21 | 2012-05-22 | The Board Of Trustees Of The University Of Illinois | Spatial light interference microscopy and fourier transform light scattering for cell and tissue characterization |
JP2012506060A (ja) * | 2008-10-14 | 2012-03-08 | サンフォード−バーナム メディカル リサーチ インスティテュート | 多面画像取得に色収差を使用する自動化された走査型サイトメトリー |
DE102008059788B4 (de) * | 2008-12-01 | 2018-03-08 | Olympus Soft Imaging Solutions Gmbh | Analyse und Klassifizierung insbesondere biologischer oder biochemischer Objekte auf Basis von Zeitreihen-Bildern, anwendbar bei der zytometrischen Time-Lapse-Zellanalyse in der bildbasierten Zytometrie |
CN102365654B (zh) * | 2009-03-27 | 2015-05-13 | 皇家飞利浦电子股份有限公司 | 周期性运动对象的两个图像序列的同步化 |
US8599383B2 (en) * | 2009-05-06 | 2013-12-03 | The Regents Of The University Of California | Optical cytometry |
JP5531710B2 (ja) * | 2010-03-29 | 2014-06-25 | ソニー株式会社 | 記録装置および記録方法 |
US9786052B2 (en) * | 2010-03-29 | 2017-10-10 | Sony Corporation | Image processing apparatus and method for evaluating objects in an image |
US11105686B2 (en) * | 2010-05-10 | 2021-08-31 | University of Pittshurgh-Of the Commonwealth System of Higher Education | Spatial-domain low-coherence quantitative phase microscopy |
US8611621B2 (en) * | 2010-09-08 | 2013-12-17 | Given Imaging Ltd. | System and method for automatic detection of in vivo contraction video sequences |
JP5745919B2 (ja) * | 2011-04-28 | 2015-07-08 | 浜松ホトニクス株式会社 | 細胞解析方法、細胞解析装置、および細胞解析プログラム |
US9557549B2 (en) * | 2011-12-09 | 2017-01-31 | Massachusetts Institute Of Technology | Systems and methods for self-referenced quantitative phase microscopy |
US9025881B2 (en) * | 2012-02-06 | 2015-05-05 | Nanyang Technological University | Methods and apparatus for recovering phase and amplitude from intensity images |
US9930241B2 (en) * | 2012-06-07 | 2018-03-27 | Sony Corporation | Image processing apparatus, image processing program, and image processing method |
US9497379B2 (en) * | 2013-08-22 | 2016-11-15 | California Institute Of Technology | Variable-illumination fourier ptychographic imaging devices, systems, and methods |
EP2915180B1 (en) * | 2012-10-30 | 2018-12-05 | California Institute of Technology | Fourier ptychographic imaging systems, devices, and methods |
EP2929507A1 (en) * | 2012-12-07 | 2015-10-14 | Canon Kabushiki Kaisha | Image generating apparatus and image generating method |
WO2014141647A1 (ja) * | 2013-03-13 | 2014-09-18 | ソニー株式会社 | 情報処理装置、情報処理方法、および情報処理プログラム |
JP6160187B2 (ja) * | 2013-04-09 | 2017-07-12 | ソニー株式会社 | 分析装置、分析プログラム及び分析システム |
US10061111B2 (en) * | 2014-01-17 | 2018-08-28 | The Trustees Of Columbia University In The City Of New York | Systems and methods for three dimensional imaging |
US20170038574A1 (en) * | 2014-02-03 | 2017-02-09 | President And Fellows Of Harvard College | Three-dimensional super-resolution fluorescence imaging using airy beams and other techniques |
US10247672B2 (en) * | 2014-09-29 | 2019-04-02 | Howard Hughes Medical Institute | Non-linear structured illumination microscopy |
US20180262758A1 (en) * | 2017-03-08 | 2018-09-13 | Ostendo Technologies, Inc. | Compression Methods and Systems for Near-Eye Displays |
-
2014
- 2014-12-18 JP JP2015556639A patent/JP6578950B2/ja not_active Expired - Fee Related
- 2014-12-18 WO PCT/JP2014/006327 patent/WO2015104763A1/ja active Application Filing
- 2014-12-18 US US15/106,858 patent/US10535137B2/en active Active
- 2014-12-18 EP EP14877880.6A patent/EP3076219B1/en active Active
Patent Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2003093378A (ja) * | 2001-07-09 | 2003-04-02 | Siemens Ag | コンピュータトモグラフィのための方法およびコンピュータトモグラフィ装置 |
WO2004109228A1 (ja) * | 2003-06-09 | 2004-12-16 | Sumitomo Osaka Cement Co.,Ltd. | 三次元形状測定装置 |
JP2005128086A (ja) | 2003-10-21 | 2005-05-19 | Olympus Corp | 走査型顕微鏡システム |
WO2005122901A1 (ja) * | 2004-06-16 | 2005-12-29 | Hitachi Medical Corporation | 放射線断層像撮像装置 |
JP2012531933A (ja) * | 2009-06-30 | 2012-12-13 | コーニンクレッカ フィリップス エレクトロニクス エヌ ヴィ | 非ecgの生体同期収集による三次元胎児心臓撮像方法を用いた超音波画像診断システム及び方法 |
Non-Patent Citations (1)
Title |
---|
"Three-dimensional cell culture: the missing link in drug discovery", DRUG DISCOV TODAY., vol. 18, no. 5-6, March 2013 (2013-03-01), pages 240 - 9 |
Cited By (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2016052179A1 (ja) * | 2014-09-30 | 2016-04-07 | 富士フイルム株式会社 | 細胞撮像装置および方法 |
US10330583B2 (en) | 2014-09-30 | 2019-06-25 | Fujifilm Corporation | Cell imaging apparatus and method for generating a composite image |
WO2017221597A1 (ja) * | 2016-06-21 | 2017-12-28 | ソニー株式会社 | 情報処理装置、情報処理方法、及びプログラム |
JPWO2017221597A1 (ja) * | 2016-06-21 | 2019-04-11 | ソニー株式会社 | 情報処理装置、情報処理方法、及びプログラム |
JP7036009B2 (ja) | 2016-06-21 | 2022-03-15 | ソニーグループ株式会社 | 情報処理装置、情報処理方法、及びプログラム |
CN110866906A (zh) * | 2019-11-12 | 2020-03-06 | 安徽师范大学 | 基于图像边缘提取的三维培养人体心肌细胞搏动检测方法 |
Also Published As
Publication number | Publication date |
---|---|
JPWO2015104763A1 (ja) | 2017-03-23 |
US10535137B2 (en) | 2020-01-14 |
EP3076219A1 (en) | 2016-10-05 |
EP3076219A4 (en) | 2017-08-30 |
US20170039713A1 (en) | 2017-02-09 |
EP3076219B1 (en) | 2020-04-08 |
JP6578950B2 (ja) | 2019-09-25 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
JP5576739B2 (ja) | 画像処理装置、画像処理方法、撮像装置及びプログラム | |
JP6578950B2 (ja) | 分析システム、分析プログラム及び分析方法 | |
CN104101294B (zh) | 分析装置、分析程序以及分析*** | |
KR102228456B1 (ko) | 깊이 정보를 갖는 영상을 생성하는 영상 획득 장치 및 영상 획득 방법 | |
JP6467813B2 (ja) | 画像処理装置、画像処理プログラム及び画像処理方法 | |
JP5464279B2 (ja) | 画像処理装置、そのプログラム、および画像処理方法 | |
JP2014188223A (ja) | 画像処理装置、内視鏡装置、プログラム及び画像処理方法 | |
JP2011022796A (ja) | 画像処理方法および画像処理装置 | |
JP2010139890A5 (ja) | ||
JP2012028949A (ja) | 画像処理装置及びその制御方法 | |
US20130194403A1 (en) | Endoscope apparatus, image processing method, and information storage device | |
JP2019505836A5 (ja) | ||
JP2015039342A5 (ja) | ||
JP5920464B2 (ja) | 画像処理装置、画像処理プログラム及び画像処理方法 | |
JP2014203038A5 (ja) | ||
FR3057068B1 (fr) | Dispositif d'observation d'un echantillon et procede d'observation d'un echantillon | |
WO2016166871A1 (ja) | 顕微鏡観察システム、顕微鏡観察方法、及び顕微鏡観察プログラム | |
JP2016051167A (ja) | 画像取得装置およびその制御方法 | |
US10721413B2 (en) | Microscopy system, microscopy method, and computer readable recording medium | |
JP6029293B2 (ja) | 走査型電子顕微鏡の画像処理装置、および、走査方法 | |
US10429632B2 (en) | Microscopy system, microscopy method, and computer-readable recording medium | |
WO2013185936A1 (en) | Apparatus and method for estimating a property of a surface using speckle imaging | |
TWI569642B (zh) | 機器視覺中進行影像擷取之方法及裝置 | |
JP2014154981A5 (ja) | 画像処理装置、画像処理方法、撮像装置及びその制御方法 | |
US11256078B2 (en) | Continuous scanning for localization microscopy |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 14877880 Country of ref document: EP Kind code of ref document: A1 |
|
REEP | Request for entry into the european phase |
Ref document number: 2014877880 Country of ref document: EP |
|
WWE | Wipo information: entry into national phase |
Ref document number: 2014877880 Country of ref document: EP |
|
ENP | Entry into the national phase |
Ref document number: 2015556639 Country of ref document: JP Kind code of ref document: A |
|
WWE | Wipo information: entry into national phase |
Ref document number: 15106858 Country of ref document: US |
|
NENP | Non-entry into the national phase |
Ref country code: DE |