CN114265298B - Cell image restoration method for lens-free holographic imaging - Google Patents

Cell image restoration method for lens-free holographic imaging Download PDF

Info

Publication number
CN114265298B
CN114265298B CN202111609687.5A CN202111609687A CN114265298B CN 114265298 B CN114265298 B CN 114265298B CN 202111609687 A CN202111609687 A CN 202111609687A CN 114265298 B CN114265298 B CN 114265298B
Authority
CN
China
Prior art keywords
image
propagation
lens
holographic imaging
domain
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202111609687.5A
Other languages
Chinese (zh)
Other versions
CN114265298A (en
Inventor
黄汐威
韩文韬
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Hangzhou Dianzi University
Original Assignee
Hangzhou Dianzi University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Hangzhou Dianzi University filed Critical Hangzhou Dianzi University
Priority to CN202111609687.5A priority Critical patent/CN114265298B/en
Publication of CN114265298A publication Critical patent/CN114265298A/en
Application granted granted Critical
Publication of CN114265298B publication Critical patent/CN114265298B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Landscapes

  • Holo Graphy (AREA)

Abstract

The invention discloses a cell image restoration method for lens-free holographic imaging. The method comprises the following steps: 1: and back-propagating the target hologram acquired by the image sensor to a plane where the sample is located to obtain a back-propagation diagram. 2: and executing a threshold segmentation algorithm on the back propagation graph to obtain a binary image after threshold segmentation. 3: and executing a minimum circumscribing algorithm on the black part in the binary image, wherein the obtained minimum circumscribing is used as an initial supporting domain. 4: the circle center of the initial supporting domain is fixed, and concentric circles with different sizes are obtained by outwards expanding based on the radius of the initial supporting domain and used as the expanding supporting domain. 5: and respectively carrying out image restoration based on the initial supporting domain and the extended supporting domain to respectively obtain restored images under different supporting domains. 6: and quantifying the fluctuation of the interference fringes of each restored image by using the variance, and taking the restored image with the most gentle fluctuation as the optimal restored image.

Description

Cell image restoration method for lens-free holographic imaging
Technical Field
The invention relates to the technical field of lens-free holographic imaging technology and holographic image restoration, in particular to a cell image restoration method of lens-free holographic imaging.
Background
Currently, point of care (POCT) technology is rapidly evolving. Instant diagnosis and treatment needs a portable, intelligent, rapid and miniaturized diagnosis and treatment detection device. Many instant diagnosis and treatment systems are developed based on micro-fluidic technology, lens-free cell imaging technology and deep learning technology, wherein the lens-free holographic imaging technology is widely applied, and has a large field of view and high numerical aperture, and can record more abundant target (typically blood cell) information (phase information and amplitude information). Such systems typically record image information using CMOS or CCD sensor values, followed by digital image processing techniques to process the image. For a holographic imaging system that records a holographic image of a target, the hologram generally needs to be processed to restore the original appearance of the object, a process commonly referred to as image restoration.
In a lensless holographic imaging system, image restoration is a critical step in order to obtain a high resolution image of the target. A good image restoration algorithm needs to suppress noise and extract useful information as much as possible. The lens-less holographic imaging approach introduces phase loss and twin image disturbance problems during the recording of the holographic image by the image sensor. These two problems seriously affect the resolution of the final recovered image, so that it is generally required for image recovery algorithms to have the ability to recover lost phase and suppress twin image interference, and currently, iterative phase retrieval algorithms are mainly used to solve these two problems. This algorithm is mainly implemented by assuming a light field distribution of an image and propagating this light field between a recording plane (image sensor plane), a real image plane (object plane), a virtual image plane (object plane symmetrically distributed with the sensor plane), which is actually an iterative recursive process, and the iterative process is convergent and converges to the light field distribution of the object. Of course, the number of iterations is relatively large, so some constraints are introduced to speed up the convergence, typically the amplitude of the light field is constrained by the amplitude recorded by the image sensor and the distribution of the light field in the plane of the target is constrained by the area occupied by the target. The area occupied by this object is generally referred to as the support domain. The determination of the support domain is typically using a threshold segmentation algorithm.
However, the current holographic imaging target is generally semitransparent and low-contrast such as cells, and when the threshold segmentation algorithm is used for determining the support domain, the boundary of the target is often difficult to determine, so that a small support domain is segmented. In addition, the propagation distance from the sample to the image sensor is used in the iterative propagation of image recovery, which is often on the order of microns, and lensless holographic imaging systems are generally compact (without the use of very precise optics), which means that an algorithm is required to accurately calculate this distance. Therefore, solving the two problems can effectively improve the image recovery quality of the lens-free holographic microscopic system and accelerate the speed of image recovery.
Disclosure of Invention
The object of the invention is to optimize a method for dividing the support domain and a method for determining the propagation distance during image restoration with a lensless holographic imaging system.
The invention provides a method for progressively exploring the supporting domain to solve the supporting domain information required by image restoration in the process of lens-free holographic imaging of a low-contrast target and an automatic focusing algorithm to determine the distance from the target to an image sensor in the process of lens-free holographic imaging (the distance parameter is one of the key parameters in the image restoration step).
The method for progressively exploring the supporting domain comprises the following steps:
step 1: and back-propagating the target hologram acquired by the image sensor to a plane where the sample is located to obtain a back-propagation diagram.
Step 2: and executing a threshold segmentation algorithm on the back propagation graph to obtain a binary image after threshold segmentation.
Step 3: and executing a minimum circumcircle algorithm on a black part (gray value is 0) in the binary image, wherein the obtained minimum circumcircle is used as an initial support domain.
Step 4: expanding a plurality of concentric circles with different radiuses of the initial support domain on the basis of the initial support domain to serve as an expanded support domain; the radius of each extended support field is greater than the radius of the original support field.
Step 5: and respectively carrying out image restoration based on the initial supporting domain and the extended supporting domain to respectively obtain restored images under different supporting domains.
Step 6: and quantifying the fluctuation of the interference fringes of each restored image by using the variance, and taking the restored image with the most gentle fluctuation as the optimal restored image. The restored image can reflect more details of the cell than the prior art by directly obtaining the image through thresholding.
Further, the back propagation described in step 1 uses the angular spectrum propagation algorithm of plane waves for digital propagation.
Further, the threshold value of the threshold value segmentation algorithm in the step 2 is obtained by adopting an oxford method.
Further, after the step 6 is executed, based on the radius corresponding to the obtained optimal restored image, adding a new expansion supporting domain by the radius step length smaller than that in the step 4; and performing image restoration based on the new extension supporting domains to obtain a restored image. And (3) comparing the fluctuation of the interference fringes of the recovered image with the fluctuation of the interference fringes of the optimal recovered image determined in the step (6) to perform the optimal recovered image.
Further, in the step 4, the radius expansion step length is 1 pixel; the radius extension step size of the extension support field added after the execution of step 6 is 0.1 pixel.
Further, the image recovery in step 5 is performed by using an iterative phase search algorithm.
Further, the specific quantization mode for recovering the fluctuation of the interference fringes of the image in the step 6 is as follows: taking any point of the outer edge of the target in the restored image, making a perpendicular to the outer edge of the restored image passing through the point, and calculating the variance of gray distribution on the perpendicular.
Further, in step 1, the distance of counter-propagation of the target hologram is obtained by an autofocus algorithm; the autofocus algorithm is specifically as follows:
1-1, taking the pure amplitude image as a target, and carrying out lens-free holographic imaging on the pure amplitude image to obtain a corresponding holographic image.
And 1-2, traversing the propagation distances in an estimated range in a certain step length, and back-propagating the hologram according to the propagation distances to obtain a back-propagation diagram corresponding to different propagation distances.
1-3: and (3) executing a threshold segmentation algorithm on each back propagation map obtained in the step (1-2) to obtain a corresponding binary image.
1-4: calculating the number of white pixels (gray value is 255) of the binary image obtained in the step 1-3; the propagation distance corresponding to the binary image with the largest number of white pixels is used as the optimal propagation distance value.
Further, the pure amplitude image described in step 1-1 uses a USAF standard resolution test plate.
Further, the step length of the traverse in the step 1-2 is 1 μm; after obtaining the propagation distance optimum value, fine traversal is performed at both sides thereof with a step size of 0.2 μm, and the propagation distance optimum value is further updated.
Further, the traversing range in step 1-2 is the upper and lower 200 μm interval of the estimated value obtained by the hardware configuration of the original system.
Further, the threshold used in the threshold segmentation algorithm described in step 1-3 is obtained by using the Ojin method.
The beneficial effects of the invention are as follows:
the optimization method provided by the invention is simple to operate and small in calculated amount, and can effectively improve the imaging resolution and robustness of the lens-free holographic imaging system (the image recovery can be better carried out on a target with low contrast). In the imaging process of the lens-free holographic imaging system, an automatic focusing algorithm is firstly used for optimizing the determination of the distance between a target and an image sensor, then an optimized supporting domain is obtained by using a progressive exploration supporting domain method, and finally the image quality of an output image of the whole system is improved.
Drawings
Fig. 1 is a basic structural diagram of lens-less holographic imaging.
Fig. 2 is a flow chart of an autofocus algorithm in accordance with the present invention.
FIG. 3 is a flowchart illustrating the overall steps of the present invention.
FIG. 4 is a flow chart of the method for obtaining the extended support domain in the present invention.
Fig. 5 is a diagram showing the actual operation of the present invention.
Detailed Description
The technical scheme of the invention is further described below with reference to the attached drawings.
As shown in fig. 1, a lens-free holographic imaging system is mainly composed of a light source, a pinhole, a sample plane, and a detection plane. The auto-focusing algorithm described in the present invention is mainly used to determine the micron-scale distance between the sample plane and the detection plane, while the progressive exploration support domain method is mainly used to optimize the determination of the support domain when the hologram back propagates to the sample plane during the system image restoration process. The use of these two optimization methods in lensless holographic imaging is detailed below.
First, before a lens-free holographic system images an object (typically a cell, algae, micromechanical structure, etc.), a USAF standard resolution test plate is placed on the system sample plane (fig. 1), and the hologram of the resolution test plate is recorded. An autofocus algorithm is performed with the hologram as an object.
The flow of the automatic focusing algorithm is shown in FIG. 2, and the distance between the sample plane and the detection plane is obtained according to the hardware parameters of the system (Z 2 ) Estimate of (in general)<1 mm), traversing this estimate by 200 μm up and down in 1 μm steps, and then back-propagating each traversal value. Then, a segmentation result (binary image) of each back propagation graph is obtained by using an oxford method and a threshold segmentation algorithm, and then the sum of pixel points with the gray value of 255 of the binary image is calculated. The maximum point of the final sum value corresponds to the optimal Z 2 Values. Z which can be obtained at 1 μm if higher accuracy is required 2 Is traversed more finely within the neighborhood of (c).
Z obtained using an autofocus algorithm 2 A back propagation map under this condition is obtained, and the following progressive exploration support domain method is performed with the back propagation map.
As shown in fig. 3, the method first performs a threshold segmentation algorithm combined with the oxford method on the target direction propagation diagram to obtain a basic support domain. And then, expanding the support domain outwards to obtain a series of support domains, and then obtaining restored images corresponding to the expanded support domain conditions. And finally, evaluating the quality of the restored image by measuring the gray distribution variance from any point on the target image edge of the restored image to the vertical line on the outer edge of the restored image (the smaller the variance is, the better the image quality is), and the support domain corresponding to the restored image with the best image quality is optimal.
Fig. 4 illustrates the details of the extended support domain in the progressive exploration support domain approach. The method comprises the steps of firstly searching a minimum circumcircle (called an initial support domain) of a basic support domain, and then expanding the radius by a fixed circle center by a certain step length to obtain a series of concentric circles (called an expanded support domain). This step size is typically 1 μm and smaller steps can be used if a higher accuracy support domain is desired.
The following provides specific implementations of the invention:
step 1: and back-propagating the target hologram acquired by the image sensor to a plane where the sample is located to obtain a back-propagation diagram. Counter-propagating uses the angular spectrum propagation algorithm of plane waves for digital propagation.
The counter-propagating distance of the target hologram is obtained by an automatic focusing algorithm; the autofocus algorithm is specifically as follows:
1-1, taking the pure amplitude image as a target, and carrying out lens-free holographic imaging on the pure amplitude image to obtain a corresponding holographic image. The pure amplitude images used USAF standard resolution test plates.
And 1-2, traversing the propagation distances in an estimated range in a certain step length, and back-propagating the hologram according to the propagation distances to obtain a back-propagation diagram corresponding to different propagation distances. The traversing step length is 1 mu m; after obtaining the propagation distance optimum value, fine traversal is performed at both sides thereof with a step size of 0.2 μm, and the propagation distance optimum value is further updated. The traversal range is the upper and lower 200 μm interval of the estimated value obtained by the hardware configuration of the original system.
1-3: and (3) executing a threshold segmentation algorithm on each back propagation map obtained in the step (1-2) to obtain a corresponding binary image. The threshold used by the threshold segmentation algorithm is obtained by using the Ojin method.
1-4: calculating the number of white pixels (gray value is 255) of the binary image obtained in the step 1-3; the propagation distance corresponding to the binary image with the largest number of white pixels is used as the optimal propagation distance value.
Step 2: and executing a threshold segmentation algorithm on the back propagation graph to obtain a binary image after threshold segmentation. The threshold value of the threshold value segmentation algorithm is obtained by adopting an Ojin method.
Step 3: and executing a minimum circumcircle algorithm on a black part (gray value is 0) in the binary image, wherein the obtained minimum circumcircle is used as an initial support domain.
Step 4: the circle center of the initial supporting domain is fixed, and concentric circles with different sizes are obtained by outwards expanding based on the radius of the initial supporting domain and used as the expanding supporting domain. The radius expansion step is 1 pixel.
Step 5: and respectively carrying out image restoration based on the initial supporting domain and the extended supporting domain to respectively obtain restored images under different supporting domains. The image recovery is performed using an iterative phase retrieval algorithm.
Step 6: and quantifying the fluctuation of the interference fringes of each restored image by using the variance, and taking the restored image with the most gentle fluctuation as the optimal restored image. The specific condition of the interference fringes is shown in the figure. The restored image can reflect more details of the cell than the prior art by directly obtaining the image through thresholding. The specific quantification mode for restoring the fluctuation of the interference fringes of the image is as follows: taking any point of the outer edge of the target in the restored image, making a perpendicular to the outer edge of the restored image passing through the point, and calculating the variance of gray distribution on the perpendicular.
Step 7, based on the radius corresponding to the obtained optimal recovery image, adding a new expansion support domain according to the radius step length smaller than that in the step 4; and performing image restoration based on the new extension supporting domains to obtain a restored image. And (3) comparing the fluctuation of the interference fringes of the recovered image with the fluctuation of the interference fringes of the optimal recovered image determined in the step (6) to perform the optimal recovered image. The radius extension step of the extension support field in this step is 0.1 pixel.

Claims (9)

1. A cell image restoration method for lens-free holographic imaging, which is characterized in that: step 1: the target hologram obtained by the image sensor is reversely transmitted to a plane where a sample is located, and a reverse transmission diagram is obtained;
step 2: executing a threshold segmentation algorithm on the back propagation graph to obtain a binary image after threshold segmentation;
step 3: executing a minimum circumscribing algorithm on a black part in the binary image, wherein the obtained minimum circumscribing is used as an initial supporting domain;
step 4: expanding a plurality of concentric circles with different radiuses of the initial support domain on the basis of the initial support domain to serve as an expanded support domain; the radius of each extended support domain is larger than that of the initial support domain;
step 5: image restoration is carried out based on the initial supporting domain and the extended supporting domain respectively, and restored images under different supporting domains are obtained respectively;
step 6: the fluctuation of the interference fringes of each recovery image is quantized by using the variance, the recovery image with the most gentle fluctuation is taken as the optimal recovery image, and the specific quantization mode of the fluctuation of the interference fringes of the recovery image is as follows: taking any point of the outer edge of the target in the restored image, making a perpendicular to the outer edge of the restored image passing through the point, and calculating the variance of gray distribution on the perpendicular.
2. The method for cell image restoration for lens-free holographic imaging of claim 1, wherein: the counter propagation described in the step 1 uses the angular spectrum propagation algorithm of plane wave to make digital propagation; and (3) obtaining the threshold value of the threshold value segmentation algorithm in the step (2) by adopting an Ojin method.
3. The method for cell image restoration for lens-free holographic imaging of claim 1, wherein: after the step 6 is executed, based on the radius corresponding to the obtained optimal recovery image, adding a new expansion support domain by the radius step length smaller than that in the step 4; performing image restoration based on the new extension support domains to obtain a restored image; and (3) comparing the fluctuation of the interference fringes of the recovered image with the fluctuation of the interference fringes of the optimal recovered image determined in the step (6) to perform the optimal recovered image.
4. The method for cell image restoration for lens-free holographic imaging of claim 1, wherein: the radius expansion step length in the step 4 is 1 pixel; the radius extension step size of the extension support field added after the execution of step 6 is 0.1 pixel.
5. The method for cell image restoration for lens-free holographic imaging of claim 1, wherein: and 5, performing image recovery by adopting an iterative phase retrieval algorithm.
6. The method for cell image restoration for lens-free holographic imaging of claim 1, wherein: in the step 1, the counter-propagation distance of the target hologram is obtained by an automatic focusing algorithm; the autofocus algorithm is specifically as follows:
1-1, taking a pure amplitude image as a target, and carrying out lens-free holographic imaging on the pure amplitude image to obtain a corresponding holographic image;
1-2, traversing propagation distances in an estimated range with a certain step length, and back-propagating holograms with the propagation distances to obtain back-propagation maps corresponding to different propagation distances;
1-3: executing a threshold segmentation algorithm on each back propagation graph obtained in the step 1-2 to obtain a corresponding binary image;
1-4: calculating the number of white pixels of the binary image obtained in the step 1-3; the propagation distance corresponding to the binary image with the largest number of white pixels is used as the optimal propagation distance value.
7. The method for cell image restoration for lens-free holographic imaging of claim 6, wherein: the pure amplitude images described in step 1-1 use the USAF standard resolution test plate.
8. The method for cell image restoration for lens-free holographic imaging of claim 6, wherein: the step length of the traversal in the step 1-2 is 1 micrometer; after obtaining the optimum value of the propagation distance, fine traversal is performed on both sides thereof with a step size of 0.2 μm, and the optimum value of the propagation distance is further updated.
9. The method for cell image restoration for lens-free holographic imaging of claim 6, wherein: the traversing range in the step 1-2 is the interval of 200 micrometers above and below the estimated value obtained by the hardware configuration of the original system.
CN202111609687.5A 2021-12-24 2021-12-24 Cell image restoration method for lens-free holographic imaging Active CN114265298B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202111609687.5A CN114265298B (en) 2021-12-24 2021-12-24 Cell image restoration method for lens-free holographic imaging

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202111609687.5A CN114265298B (en) 2021-12-24 2021-12-24 Cell image restoration method for lens-free holographic imaging

Publications (2)

Publication Number Publication Date
CN114265298A CN114265298A (en) 2022-04-01
CN114265298B true CN114265298B (en) 2024-02-09

Family

ID=80830192

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202111609687.5A Active CN114265298B (en) 2021-12-24 2021-12-24 Cell image restoration method for lens-free holographic imaging

Country Status (1)

Country Link
CN (1) CN114265298B (en)

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108508588A (en) * 2018-04-23 2018-09-07 南京大学 A kind of multiple constraint information without lens holographic microphotography phase recovery method and its device
CN112327473A (en) * 2021-01-07 2021-02-05 南京理工大学智能计算成像研究院有限公司 Lensless microscopic imaging system and image reconstruction method based on average projection iteration
CN113359403A (en) * 2021-05-21 2021-09-07 大连海事大学 Automatic focusing method for lens-free digital holographic imaging

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20120071405A (en) * 2009-10-20 2012-07-02 더 리전트 오브 더 유니버시티 오브 캘리포니아 Incoherent lensfree cell holography and microscopy on a chip

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108508588A (en) * 2018-04-23 2018-09-07 南京大学 A kind of multiple constraint information without lens holographic microphotography phase recovery method and its device
CN112327473A (en) * 2021-01-07 2021-02-05 南京理工大学智能计算成像研究院有限公司 Lensless microscopic imaging system and image reconstruction method based on average projection iteration
CN113359403A (en) * 2021-05-21 2021-09-07 大连海事大学 Automatic focusing method for lens-free digital holographic imaging

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
Support-domain constrained phase retrieval algorithms in terahertz in-line digital holography reconstruction of a nonisolated amplitude object;Hu, JQ; Li, Q ; Zhou, Y;APPLIED OPTICS;第55卷(第2期);379-386 *
肖卓磊.基于优化理论的相位恢复算法研究.2021,72-81. *
蒋帅帅.基于细胞图像的计算全息算法研究.2021,2-34. *

Also Published As

Publication number Publication date
CN114265298A (en) 2022-04-01

Similar Documents

Publication Publication Date Title
CN110189255B (en) Face detection method based on two-stage detection
JP2018509678A (en) Target acquisition method and apparatus
Ali et al. Robust focus volume regularization in shape from focus
KR102162451B1 (en) Video interpolation method and video extrapolation method based on learning network
WO2024066035A1 (en) Defect detection method and system based on battery surface image, and related device
CN116579616A (en) Risk identification method based on deep learning
CN115359108A (en) Depth prediction method and system based on defocusing under guidance of focal stack reconstruction
CN115984747A (en) Video saliency target detection method based on dynamic filter
Zhang et al. Automatic focusing method of microscopes based on image processing
Yan et al. Multiscale fusion and aggregation pcnn for 3d shape recovery
CN110942097A (en) Imaging-free classification method and system based on single-pixel detector
CN114265298B (en) Cell image restoration method for lens-free holographic imaging
CN111929688B (en) Method and equipment for determining radar echo prediction frame sequence
CN111833368B (en) Speech restoration method based on phase consistency edge detection
CN117372876A (en) Road damage evaluation method and system for multitasking remote sensing image
CN116259067B (en) Method for high-precision identification of PID drawing symbols
CN117173412A (en) Medical image segmentation method based on CNN and Transformer fusion network
CN115358962B (en) End-to-end visual odometer method and device
CN114372944B (en) Multi-mode and multi-scale fused candidate region generation method and related device
CN116385915A (en) Water surface floater target detection and tracking method based on space-time information fusion
CN114882206A (en) Image generation method, model training method, detection method, device and system
CN114463300A (en) Steel surface defect detection method, electronic device, and storage medium
CN114581876A (en) Method for constructing lane detection model under complex scene and method for detecting lane line
Yaman et al. An automated crack detection method for underwater structures based on multilevel DWT and LPQ feature generation
Tian et al. Intelligent Spot Detection for Degraded Image Sequences Based on Machine Vision

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant