CN110766620B - Confocal endoscope image distortion correction method based on optical fiber probe - Google Patents
Confocal endoscope image distortion correction method based on optical fiber probe Download PDFInfo
- Publication number
- CN110766620B CN110766620B CN201910926443.6A CN201910926443A CN110766620B CN 110766620 B CN110766620 B CN 110766620B CN 201910926443 A CN201910926443 A CN 201910926443A CN 110766620 B CN110766620 B CN 110766620B
- Authority
- CN
- China
- Prior art keywords
- image
- face
- optical fiber
- fiber end
- theoretical
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 239000013307 optical fiber Substances 0.000 title claims abstract description 62
- 238000012937 correction Methods 0.000 title claims abstract description 41
- 239000000523 sample Substances 0.000 title claims abstract description 41
- 238000000034 method Methods 0.000 title claims abstract description 20
- 239000000835 fiber Substances 0.000 claims description 31
- 238000013507 mapping Methods 0.000 claims description 17
- 238000012545 processing Methods 0.000 claims description 3
- 238000003384 imaging method Methods 0.000 description 4
- 238000010586 diagram Methods 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 206010028980 Neoplasm Diseases 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000006835 compression Effects 0.000 description 1
- 238000007906 compression Methods 0.000 description 1
- 238000003745 diagnosis Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 238000003702 image correction Methods 0.000 description 1
- 230000003902 lesion Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 238000012216 screening Methods 0.000 description 1
- 238000003325 tomography Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/80—Geometric correction
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10056—Microscopic image
- G06T2207/10061—Microscopic image from scanning electron microscope
Landscapes
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Microscoopes, Condenser (AREA)
- Endoscopes (AREA)
Abstract
The invention discloses a confocal endoscope image distortion correction method based on an optical fiber probe, which comprises the following steps: acquiring a background image through a confocal endoscope based on a fiber-optic probe; calculating the shape and position of the optical fiber end face in the theoretical image according to the actual image shape and position of the optical fiber end face in the background image; establishing a distortion correction function according to the relation between the actual image shape and position of the optical fiber end face in the background image and the shape and position of the optical fiber end face in the theoretical image; and correcting the transverse distortion of the sample image acquired by the confocal endoscope based on the fiber-optic probe by using the obtained distortion correction function. The image distortion correction method can eliminate the problem of transverse image distortion of the confocal endoscope only according to a single picture on the premise of not increasing the hardware complexity of the confocal endoscope, can adapt to the change of the motion rule of the galvanometer, can improve the quality of the image, and has important clinical value.
Description
Technical Field
The invention relates to the technical field of confocal endoscopic imaging, in particular to a confocal endoscope image distortion correction method based on an optical fiber probe.
Background
The confocal micro-endoscope based on the optical fiber probe is a novel endoscope combining laser confocal micro-technology and traditional endoscope technology, can carry out high-resolution histological diagnosis on living tissues, can realize tomography scanning at a certain depth, and has important effect on screening early tumors and precancerous lesions.
The structure of the confocal laser endoscope based on the fiber probe is shown in fig. 1, and laser emitted by a laser scans the surface of a sample under the deflection action of an X/Y axis scanning galvanometer. Fluorescence excited on the surface of the sample is transmitted to a photoelectric detector through an optical path and converted into an electric signal. The electric signals are collected by a collecting card and then spliced into an image by a computer.
The confocal endoscopic imaging technology based on the optical fiber probe generally adopts a resonance galvanometer as an X-axis scanning galvanometer to improve the image acquisition speed, but the acquired image is transversely distorted due to the nonlinear change of the movement speed of the resonance galvanometer. The velocity change law of the galvanometer resonator is shown in fig. 2, wherein the black dotted line in the graph represents the position of the galvanometer resonator, and the black solid line represents the velocity of the galvanometer resonator. The resonant galvanometer has a velocity of 0 at the edge and a highest velocity in the middle. The image collected based on the confocal endoscope is in compression distortion in the middle of the transverse direction and in tension distortion at two ends of the transverse direction. The image of the confocal endoscope before image correction is raster-imaged on equidistant stripes as shown in fig. 3, and the image shows a distortion mode that two ends are stretched and the middle is compressed. The traditional method for solving the transverse distortion of confocal endoscopic imaging is divided into a hardware method and a software method, wherein one of the software methods is to directly cut off regions with more serious distortion on two sides and leave regions with better middle linearity, but the distortion problem still exists in the middle part of the rest images. Another software method is to directly use a sine curve to represent the motion state of the galvanometer so as to correct the distortion generated by the motion, but in actual use, the galvanometer deviates from the standard sine curve motion due to the influence of the environment, the service life and other reasons, and at this time, if the sine curve correction is still used, the image generates larger distortion. The hardware correction method generally performs non-isochronous sampling by using a pixel clock, but the scanning curve of the galvanometer changes with environmental factors, and the pixel clock cannot follow the change. And the use of a pixel clock increases the hardware cost of the system. In order to solve the problem of image distortion and adapt to the change of the motion rule of a galvanometer on the premise of not increasing the complexity of hardware, the application provides a confocal micro-endoscope image distortion correction method based on an optical fiber probe.
Disclosure of Invention
The present invention is directed to provide a confocal endoscope image distortion correction method based on an optical fiber probe, which is directed to overcome the above-mentioned shortcomings in the prior art.
In order to solve the technical problems, the invention adopts the technical scheme that: a confocal endoscope image distortion correction method based on a fiber probe comprises the following steps:
1) acquiring a background image through a confocal endoscope based on a fiber probe;
2) calculating the shape and position of the optical fiber end face in the theoretical image according to the actual image shape and position of the optical fiber end face in the background image;
3) establishing a distortion correction function according to the relation between the actual image shape and position of the optical fiber end face in the background image and the shape and position of the optical fiber end face in the theoretical image;
4) and correcting the transverse distortion of the sample image acquired by the confocal endoscope based on the optical fiber probe by using the obtained distortion correction function.
Preferably, the step 1) specifically includes: an optical fiber probe of a confocal endoscope is used for collecting an image in an illuminated environment, a light spot area is formed on the image by the end face of the optical fiber probe, and then binarization processing is carried out on the image to obtain a background image with clear light spot area edges.
Preferably, the step 2) specifically includes: by the center coordinate (x) of the actual image of the fiber end face in the background image0,y0) Taking the vertical height value of the actual image of the fiber end face in the background image as the diameter D of the fiber end face in the theoretical image as the center coordinate of the fiber end face in the theoretical image0. The theoretical shape and position of the end face of the optical fiber can also be obtained by calculating the size of the optical fiber and the optical characteristics of a confocal endoscope system.
Preferably, the step 3) includes: calculating the abscissa x of the pixel of the theoretical image edge of the fiber end face in the theoretical image line by lineaAnd the abscissa x of the pixel of the actual image edge of the fiber end face in the background imageb(ii) a For xaAnd xbAnd performing polynomial fitting to obtain a mapping function of the abscissa of each pixel in the actual image before correction and the abscissa of each pixel in the theoretical image after correction.
Preferably, the step 3) is specifically: calculating the pixel abscissa x of the theoretical image edge of the optical fiber end face in the theoretical image line by linea1And xa2，xa1、xa2The values in row i are:
then searching the horizontal coordinate x of the pixel of the actual image edge of the fiber end face in the ith row in the background imageb1(i) And xb2(i) (ii) a N, N is the total number of rows of pixel points in the vertical direction of the image of the fiber end face in the theoretical image; and performing polynomial fitting on the pixel abscissa of the theoretical image edge of the optical fiber end face in the obtained theoretical image and the pixel abscissa of the actual image edge of the optical fiber end face in the background image to obtain the mapping function of each pixel abscissa in the actual image before correction and each pixel abscissa in the theoretical image after correction.
Preferably, in the step 3), a 3 rd order polynomial fitting is performed, and a mapping function of the abscissa of each pixel in the actual image before correction and the abscissa of each pixel in the theoretical image after correction is as
The beneficial effects of the invention are: the image distortion correction method of the invention utilizes the relation between the background image collected by the confocal endoscope under natural light based on the optical fiber and the actual shape of the end face of the optical fiber to fit the mapping function of the pixel abscissa before and after distortion correction, and corrects the transverse distortion of the image through the mapping function; on the premise of not increasing the complexity of hardware, the problem of transverse image distortion of the confocal endoscope can be solved only according to a single picture, the change of the motion law of the galvanometer can be self-adapted, the quality of the image can be improved, and the method has important clinical value.
Drawings
FIG. 1 is a schematic diagram of a prior art fiber-based confocal endoscope;
FIG. 2 is a diagram illustrating the variation of resonant mirror speed in the prior art;
FIG. 3 is a schematic diagram of distortion of an image acquired by a confocal endoscope in the prior art;
FIG. 4 is a background image shape collected in an embodiment of the invention;
FIG. 5 is a theoretical image shape computed from a background image in an embodiment of the present invention;
fig. 6 is a mapping function curve obtained by fitting in an embodiment of the present invention.
Detailed Description
The present invention is further described in detail below with reference to examples so that those skilled in the art can practice the invention with reference to the description.
It will be understood that terms such as "having," "including," and "comprising," as used herein, do not preclude the presence or addition of one or more other elements or groups thereof.
The method for correcting the image distortion of the confocal endoscope based on the optical fiber probe comprises the steps of firstly collecting a background image through the confocal endoscope based on the optical fiber probe; then calculating the shape and position of the optical fiber end face in the theoretical image according to the actual image shape and position of the optical fiber end face in the background image; establishing a distortion correction function according to the relation between the actual image shape and position of the optical fiber end face in the background image and the shape and position of the optical fiber end face in the theoretical image; and finally, correcting the transverse distortion of the sample image acquired by the confocal endoscope based on the optical fiber probe by using the obtained distortion correction function.
The method comprises the following specific steps:
firstly, acquiring a background image:
an optical fiber probe of a confocal endoscope is used for collecting an image in an illuminated environment, ambient light is transmitted to the end face of the other side of an optical fiber through the optical fiber probe, a light spot area is formed on the image by the end face of the optical fiber probe, and then binaryzation processing is carried out on the image to obtain a background image with clear light spot area edges. Background image as shown in fig. 4, the bright area in the center of the background image is the circular end face of the fiber probe. The fiber end face is not rounded in the background image due to distortion.
Secondly, calculating a theoretical image of the end face of the optical fiber:
due to the influence of the transverse distortion of the graph, the imaging shape of the end face of the optical fiber in the background image is a bright spot which is symmetrical up and down and symmetrical left and right. The theoretical image of the fiber end face should be circular and the diameter is the vertical height of the bright spot on the fiber end face in the background image. Therefore, the center coordinate (x) of the actual image of the end face of the optical fiber in the background image is used0,y0) Taking the vertical height value of the actual image of the fiber end face in the background image as the diameter D of the fiber end face in the theoretical image as the center coordinate of the fiber end face in the theoretical image0So as to obtain a theoretical image of the end face of the optical fiber, i.e., a corrected image, as shown in fig. 5. In another embodiment, the theoretical shape and position of the end face of the optical fiber can also be calculated by the size of the optical fiber and the optical characteristics of the confocal endoscopic system.
Thirdly, calculating the actual image and the theoretical image to obtain a mapping function:
calculating the abscissa x of the pixel of the theoretical image edge of the optical fiber end face in the theoretical image line by lineaAnd the abscissa x of the pixel of the actual image edge of the fiber end face in the background imageb(ii) a For xaAnd xbPerforming polynomial fitting to obtain a mapping function x of each pixel abscissa in the actual image before correction and each pixel abscissa in the theoretical image after correctiona＝f(xb)。
More specifically: calculating the pixel abscissa x of the theoretical image edge of the optical fiber end face in the theoretical image line by linea1And xa2，xa1、xa2The values in row i are:
then searching the horizontal seat of the pixel of the actual image edge of the fiber end face in the ith row in the background imageMark xb1(i) And xb2(i) (ii) a Each line has 2 edge points, i is 1,2,3, N is the total number of the lines of the pixel points of the image of the optical fiber end surface in the theoretical image along the vertical direction, i.e. the image is divided into two halves along the vertical diameter of the theoretical image, the total number of the pixel points on the peripheral edge of any half of the image, thereby taking each pixel point of the image edge; and then performing 3-order polynomial fitting on the pixel abscissa of the theoretical image edge of the fiber end face in the obtained theoretical image and the pixel abscissa of the actual image edge of the fiber end face in the background image to obtain a mapping function of each pixel abscissa in the actual image before correction and each pixel abscissa in the theoretical image after correction:
as shown in FIG. 6, the point coordinate in the figure is (x)a，xb) And the solid line is a mapping function curve obtained by fitting.
Fourthly, performing transverse distortion correction on the collected sample image:
the confocal endoscope based on the optical fiber probe is used for collecting the image of the sample, and the image information of the sample is concentrated in a circular area formed by the end face of the optical fiber as the image of the sample is transmitted to the focal plane of the confocal endoscope through the optical fiber. And calculating the coordinates of the pixels in the end face of the optical fiber in the corrected image according to the mapping function obtained in the step three, and mapping the coordinates to the corrected image for display, so as to obtain the image with the transverse distortion corrected.
The invention utilizes the relation between the background image collected by the confocal endoscope under natural light and the actual shape of the end surface of the optical fiber to fit the mapping function of the horizontal coordinates of the pixels before and after distortion correction, and corrects the transverse distortion of the image through the mapping function; the problem of transverse distortion of the confocal endoscope based on the optical fiber is solved on the premise of not increasing the complexity of hardware, the change of the motion rule of the galvanometer can be self-adapted, the quality of images can be improved, and the method has important clinical value.
While embodiments of the invention have been disclosed above, it is not intended to be limited to the details shown in the description and the examples, which are set forth, but are fully applicable to various fields of endeavor as are suited to the particular use contemplated, and further modifications will readily occur to those skilled in the art, since the invention is not limited to the details shown and described without departing from the general concept as defined by the appended claims and their equivalents.
Claims (4)
1. A confocal endoscope image distortion correction method based on a fiber-optic probe is characterized by comprising the following steps:
1) acquiring a background image through a confocal endoscope based on a fiber-optic probe;
2) calculating the shape and position of the optical fiber end face in the theoretical image according to the actual image shape and position of the optical fiber end face in the background image;
3) establishing a distortion correction function according to the relation between the actual image shape and position of the optical fiber end face in the background image and the shape and position of the optical fiber end face in the theoretical image;
4) correcting the transverse distortion of a sample image collected by a confocal endoscope based on the optical fiber probe by using the obtained distortion correction function;
the step 2) specifically comprises the following steps: by the center coordinate (x) of the actual image of the fiber end face in the background image0，y0) Taking the vertical height value of the actual image of the fiber end face in the background image as the diameter D of the fiber end face in the theoretical image as the center coordinate of the fiber end face in the theoretical image0；
The step 3) is specifically as follows: calculating the pixel abscissa x of the theoretical image edge of the optical fiber end face in the theoretical image line by linea1And xa2，xa1、xa2The value in row i is:
then searching the horizontal coordinate x of the pixel of the actual image edge of the fiber end face in the ith row in the background imageb1(i) And xb2(i) (ii) a N, N is the total number of rows of pixel points in the vertical direction of the image of the fiber end face in the theoretical image; and performing polynomial fitting on the pixel abscissa of the theoretical image edge of the optical fiber end face in the obtained theoretical image and the pixel abscissa of the actual image edge of the optical fiber end face in the background image to obtain the mapping function of each pixel abscissa in the actual image before correction and each pixel abscissa in the theoretical image after correction.
2. The confocal endoscope image distortion correction method based on the fiber-optic probe according to claim 1, wherein the step 1) specifically comprises: an optical fiber probe of the confocal endoscope is used for collecting images in an illuminated environment, a light spot area is formed on the images by the end face of the optical fiber probe, and then binaryzation processing is carried out on the images to obtain a background image with clear light spot area edges.
3. The method for image distortion correction of a confocal endoscope based on a fiber-optic probe according to claim 2, wherein the step 3) comprises: calculating the abscissa x of the pixel of the theoretical image edge of the fiber end face in the theoretical image line by lineaAnd the abscissa x of the pixel of the actual image edge of the fiber end face in the background imageb(ii) a For xaAnd xbAnd performing polynomial fitting to obtain a mapping function of each pixel abscissa in the actual image before correction and each pixel abscissa in the theoretical image after correction.
4. The method as claimed in claim 3, wherein the step 3) is performed by 3 degree polynomial fitting, and the mapping function of the abscissa of each pixel in the real image before correction and the abscissa of each pixel in the theoretical image after correction is as follows
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910926443.6A CN110766620B (en) | 2019-09-27 | 2019-09-27 | Confocal endoscope image distortion correction method based on optical fiber probe |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910926443.6A CN110766620B (en) | 2019-09-27 | 2019-09-27 | Confocal endoscope image distortion correction method based on optical fiber probe |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110766620A CN110766620A (en) | 2020-02-07 |
| CN110766620B true CN110766620B (en) | 2022-07-19 |
Family
ID=69330662
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910926443.6A Active CN110766620B (en) | 2019-09-27 | 2019-09-27 | Confocal endoscope image distortion correction method based on optical fiber probe |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110766620B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112785518B (en) * | 2021-01-11 | 2022-08-30 | 飞燕航空遥感技术有限公司 | Method and system for correcting circular dark spots in image |
| CN113837973B (en) * | 2021-11-29 | 2022-02-18 | 中国科学院苏州生物医学工程技术研究所 | Confocal endoscope image correction method and system based on optical fiber probe |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101572828A (en) * | 2009-05-20 | 2009-11-04 | 长春理工大学 | Method for correcting distortion in real time based on GPU camera and video camera |
| CN101729739A (en) * | 2009-11-16 | 2010-06-09 | 潘林岭 | Method for rectifying deviation of image |
| JP2011131023A (en) * | 2009-12-25 | 2011-07-07 | Olympus Corp | Endoscope image processor, endoscope image displaying system, and endoscope image processing method |
| CN108171666A (en) * | 2017-12-27 | 2018-06-15 | 华中科技大学鄂州工业技术研究院 | A kind of sinusoidal fault image antidote for confocal endoscope |
| CN109345467A (en) * | 2018-08-14 | 2019-02-15 | 深圳市麦极客图像技术有限公司 | Image deformation bearing calibration, device, computer equipment and storage medium |
-
2019
- 2019-09-27 CN CN201910926443.6A patent/CN110766620B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101572828A (en) * | 2009-05-20 | 2009-11-04 | 长春理工大学 | Method for correcting distortion in real time based on GPU camera and video camera |
| CN101729739A (en) * | 2009-11-16 | 2010-06-09 | 潘林岭 | Method for rectifying deviation of image |
| JP2011131023A (en) * | 2009-12-25 | 2011-07-07 | Olympus Corp | Endoscope image processor, endoscope image displaying system, and endoscope image processing method |
| CN108171666A (en) * | 2017-12-27 | 2018-06-15 | 华中科技大学鄂州工业技术研究院 | A kind of sinusoidal fault image antidote for confocal endoscope |
| CN109345467A (en) * | 2018-08-14 | 2019-02-15 | 深圳市麦极客图像技术有限公司 | Image deformation bearing calibration, device, computer equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110766620A (en) | 2020-02-07 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109345469B (en) | Speckle denoising method in OCT imaging based on condition generation countermeasure network | |
| US7072047B2 (en) | Method and system for quantitative image correction for optical coherence tomography | |
| AU2007319122B2 (en) | Improvements in or relating to retinal scanning | |
| CN110766620B (en) | Confocal endoscope image distortion correction method based on optical fiber probe | |
| US8155420B2 (en) | System and method for detecting poor quality in 3D reconstructions | |
| CN106226327B (en) | A kind of cambered surface external surface of objects defect inspection method and system | |
| CN102860814B (en) | OCT (Optical Coherence Tomography) synthetic fundus image optic disc center positioning method and equipment | |
| CN103889302A (en) | Endoscopic system | |
| CN102038519B (en) | Space compound imaging method in ultrasonic diagnosis | |
| US10204405B2 (en) | Apparatus and method for parameterizing a plant | |
| CN209951205U (en) | Confocal endoscopic imaging dislocation correction system | |
| CN109645936B (en) | Confocal endoscopic imaging dislocation correction system and method | |
| JP2017068302A (en) | Image creation device and image creation method | |
| CN112155511B (en) | Method for compensating human eye shake in OCT acquisition process based on deep learning | |
| CN115496657A (en) | Pathological section digitalization method and system for low-resolution imaging | |
| CN109820471B (en) | Confocal endoscopic imaging dislocation correction system and method | |
| Feng et al. | Calculating the leaf-area based on non-loss correction algorithm | |
| WO2020103075A1 (en) | Image processing method and device | |
| CN109754365B (en) | Image processing method and device | |
| CN110348569A (en) | Real-time optical chromatography method and system based on convolutional neural networks | |
| US10264960B2 (en) | Information processing apparatus, operation method thereof, and computer program | |
| CN107040745B (en) | method for improving frame frequency of probe type confocal micro endoscope | |
| JPH10162156A (en) | Irradiation filed recognition device and radiation image processor | |
| CN104143196A (en) | Point object detection method based on multiple-linear time difference scanning and expansion sampling | |
| CN109164112A (en) | A kind of cable surface defects detection system based on unmanned plane |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| EE01 | Entry into force of recordation of patent licensing contract |
Application publication date: 20200207 Assignee: SHENYANG SHENDA ENDOSCOPE Co.,Ltd. Assignor: Suzhou Institute of Biomedical Engineering and Technology Chinese Academy of Sciences Contract record no.: X2020980005472 Denomination of invention: Image distortion correction method of confocal endoscope based on optical fiber probe License type: Common License Record date: 20200827 |
|
| EE01 | Entry into force of recordation of patent licensing contract | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |