CN117391955A - Convex set projection super-resolution reconstruction method based on multi-frame optical coherence tomography - Google Patents
Convex set projection super-resolution reconstruction method based on multi-frame optical coherence tomography Download PDFInfo
- Publication number
- CN117391955A CN117391955A CN202311382659.3A CN202311382659A CN117391955A CN 117391955 A CN117391955 A CN 117391955A CN 202311382659 A CN202311382659 A CN 202311382659A CN 117391955 A CN117391955 A CN 117391955A
- Authority
- CN
- China
- Prior art keywords
- image
- resolution
- reference frame
- pixel
- frame
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 78
- 238000012014 optical coherence tomography Methods 0.000 title claims abstract description 78
- 238000006073 displacement reaction Methods 0.000 claims abstract description 55
- 238000012937 correction Methods 0.000 claims abstract description 33
- 230000008569 process Effects 0.000 claims abstract description 21
- 230000015556 catabolic process Effects 0.000 claims abstract description 10
- 238000006731 degradation reaction Methods 0.000 claims abstract description 10
- 238000012216 screening Methods 0.000 claims abstract description 9
- 230000000903 blocking effect Effects 0.000 claims abstract description 3
- 238000003384 imaging method Methods 0.000 claims description 26
- 238000004364 calculation method Methods 0.000 claims description 12
- 239000013598 vector Substances 0.000 claims description 12
- 239000000654 additive Substances 0.000 claims description 9
- 230000000996 additive effect Effects 0.000 claims description 9
- 238000009826 distribution Methods 0.000 claims description 6
- 230000008859 change Effects 0.000 claims description 5
- 238000013519 translation Methods 0.000 claims description 4
- 230000010363 phase shift Effects 0.000 claims description 3
- 230000003321 amplification Effects 0.000 claims description 2
- 238000005520 cutting process Methods 0.000 claims description 2
- 238000003199 nucleic acid amplification method Methods 0.000 claims description 2
- 230000006870 function Effects 0.000 description 43
- 230000003287 optical effect Effects 0.000 description 10
- 230000006872 improvement Effects 0.000 description 8
- 238000013135 deep learning Methods 0.000 description 7
- 238000005516 engineering process Methods 0.000 description 7
- 210000001525 retina Anatomy 0.000 description 7
- 238000012549 training Methods 0.000 description 6
- 230000007547 defect Effects 0.000 description 5
- 238000001228 spectrum Methods 0.000 description 5
- 238000005457 optimization Methods 0.000 description 4
- 230000001427 coherent effect Effects 0.000 description 3
- 230000001419 dependent effect Effects 0.000 description 3
- 238000001514 detection method Methods 0.000 description 3
- 238000004519 manufacturing process Methods 0.000 description 3
- 238000013507 mapping Methods 0.000 description 3
- 239000012528 membrane Substances 0.000 description 3
- 238000012546 transfer Methods 0.000 description 3
- 238000003759 clinical diagnosis Methods 0.000 description 2
- 238000003745 diagnosis Methods 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 201000010099 disease Diseases 0.000 description 2
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 description 2
- 230000004424 eye movement Effects 0.000 description 2
- 230000010354 integration Effects 0.000 description 2
- 238000012804 iterative process Methods 0.000 description 2
- 238000005259 measurement Methods 0.000 description 2
- 238000012545 processing Methods 0.000 description 2
- 230000004044 response Effects 0.000 description 2
- 230000009466 transformation Effects 0.000 description 2
- 208000017442 Retinal disease Diseases 0.000 description 1
- 210000003484 anatomy Anatomy 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 238000011503 in vivo imaging Methods 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 230000001575 pathological effect Effects 0.000 description 1
- 230000007119 pathological manifestation Effects 0.000 description 1
- 230000001737 promoting effect Effects 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 230000002207 retinal effect Effects 0.000 description 1
- 230000003595 spectral effect Effects 0.000 description 1
- 230000001131 transforming effect Effects 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
- G06T3/4053—Scaling of whole images or parts thereof, e.g. expanding or contracting based on super-resolution, i.e. the output image resolution being higher than the sensor resolution
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
- G06T3/4038—Image mosaicing, e.g. composing plane images from plane sub-images
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/50—Image enhancement or restoration using two or more images, e.g. averaging or subtraction
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/30—Determination of transform parameters for the alignment of images, i.e. image registration
- G06T7/33—Determination of transform parameters for the alignment of images, i.e. image registration using feature-based methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/46—Descriptors for shape, contour or point-related descriptors, e.g. scale invariant feature transform [SIFT] or bags of words [BoW]; Salient regional features
- G06V10/462—Salient features, e.g. scale invariant feature transforms [SIFT]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2200/00—Indexing scheme for image data processing or generation, in general
- G06T2200/32—Indexing scheme for image data processing or generation, in general involving image mosaicing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10072—Tomographic images
- G06T2207/10101—Optical tomography; Optical coherence tomography [OCT]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20212—Image combination
- G06T2207/20221—Image fusion; Image merging
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Multimedia (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a convex set projection super-resolution reconstruction method based on multi-frame optical coherence tomography, which mainly comprises two parts, wherein an optical coherence tomography low-resolution image sequence is firstly collected; screening multiple frames with the same section; performing angle correction on the data set; the second section includes: establishing an image degradation model; interpolating the reference frame to a target resolution; defining a closed convex set for each pixel of the high resolution image; calculating residual errors pixel by a degradation model; introducing all reference frame corrections frame by frame; and (5) performing correction circularly for a plurality of times. And reconstructing high-frequency information in the high-resolution image by fusing information in the multi-frame low-resolution image. The operation object of the reconstruction process is image blocks corresponding to a reference frame and a reference frame, different image blocks are registered according to corresponding sub-pixel displacement, firstly, displacement estimation and image reconstruction are carried out in a blocking mode, and then block images reconstructed by super resolution are spliced according to the original position, so that a whole super resolution image is obtained.
Description
Technical Field
The invention belongs to the optical medical image processing technology, and particularly relates to a method for constructing a super-resolution image based on multi-frame optical coherence tomography.
Background
Optical coherence tomography ((Optical Coherence Tomography-OCT)) has been widely used as a detection means for ophthalmic clinical diagnosis of retinal diseases due to its non-invasive, non-contact and application to in vivo imaging. The retina B-scan image obtained by OCT provides thickness and morphology information of each cell layer for clinicians, and has important significance for detecting and evaluating diseases with obvious retina morphology change.
Clinical diagnosis often requires high resolution OCT images to accurately reflect complex membranous pathological structures. Although the resolution of the OCT image at the micron level has played an important role in clinical imaging, if the resolution of the OCT image is further improved, more microstructure information of the membrane layer can be provided, which helps doctors to realize accurate diagnosis and reveals pathological manifestations of specific diseases, thereby greatly promoting the development of clinical medical diagnosis.
OCT system imaging mainly relies on one-dimensional interference signals formed by back-scattered light from different areas of the imaging background and reference beam interference; a two-dimensional B-scan image is formed by demodulating one-dimensional interference signals to form an A-scan image, and stitching a plurality of formed A-scan images in space. The axial resolution is thus dependent on the central wavelength and spectral bandwidth of the interference light source, and the lateral resolution is dependent on the numerical aperture of the focusing objective and the size of the spot projected onto the surface of the object to be measured. Researchers have improved the imaging resolution of OCT systems by improving hardware configurations, including the use of ultra-wideband light sources and complex scanning beam focusing optics. However, the method is limited by diffraction limit and laser technology, the improvement of system hardware is limited in resolution improvement degree, and the complicated optical system and light source increase the manufacturing cost of the OCT system, so that the method cannot be widely applied to market and medical line.
Compared with the improvement of a hardware system, the method for improving the resolution of the OCT image by reconstructing the OCT image is widely promoted in recent years because of the characteristics that the image quality can be improved with smaller cost and the method is not limited by diffraction limit. The existing super-resolution reconstruction methods can be divided into three types: (1) single image-based super-resolution reconstruction: (2) multi-image based super-resolution reconstruction; (3) super-resolution reconstruction based on deep learning. The first type of the method is to calculate a Point Spread Function (PSF) according to parameters of an optical system, and then perform deconvolution operation by using the calculated point spread function to obtain a clear image. However, the original data of OCT has a lot of speckle noise due to the autocorrelation term interference during demodulation, and deconvolving the noisy image will generate a lot of artifacts. And the mature OCT system has high complexity, optical parameters are difficult to obtain by patent protection, and the difficulty of calculating the point spread function is high. While the third class (super-resolution reconstruction based on deep learning) of method can efficiently generate higher-definition images, it relies on correct low-resolution and high-resolution mapping datasets, and its improvement factor to resolution is limited, and the generated high-resolution images have reduced authenticity and cannot be applied to the medical image field. The second proposed scheme of multi-image based super-resolution reconstruction (retinal OCT image) can therefore mitigate the impact of single-image based point spread function estimation errors on the reconstruction results. Furthermore, the multi-image method is robust to low resolution image noise. When noise or less information exists in a certain frame of low-resolution image, other images can still provide effective information to reconstruct lost details, so that the robustness of a reconstruction result is improved.
Therefore, the multi-image-based super-resolution reconstruction method has the advantages of being independent of a data set, being capable of expanding to more application scenes and meeting the requirements of OCT image super-resolution reconstruction in various scenes.
One U.S. patent (US 5978109) discloses a super-resolution scanning optical device that images light from a coherent light source as a fine spot onto a conjugate surface, and a device that scans the conjugate surface as a fine spot. The coherent light source is composed of a first light source and a second light source with opposite phases. The lateral amplitude of the image main lobe of the first light source is offset by the amplitude of the main lobe of the second light source, thereby reducing the width of the main lobe of the image of the first light source. Thereby obtaining super resolution smaller than the diffraction limit without forming slit-like or annular openings. However, the technical characteristics have the defects that the transverse resolution and the axial resolution cannot be improved at the same time, two light sources with strictly opposite phases are needed, and the realization cost and the technical difficulty are high.
An improved OCT imaging system is disclosed in US5994690, which proposes a method of estimating the impulse response from the interferometer output interference signal, which can be obtained from the cross-correlation and auto-correlation data. An auto-correlation power spectrum and a cross-correlation power spectrum are obtained by fourier transforming the auto-correlation data and the cross-correlation data, respectively. And obtaining a transfer function of the OCT and tissue interaction linear translation invariant system by taking the ratio of the cross power spectrum to the self power spectrum, and obtaining the optical impulse response by carrying out Fourier inverse transformation on the transfer function. The coherent demodulation and deconvolution technology are combined, reflection points in the sample, which are closely spaced, are analyzed, and the axial resolution of the OCT system is improved. However, the technical feature is that the system and method cannot simultaneously improve the axial and lateral resolution of the OCT system, and the measurement or estimation of the power spectrum is affected by the characteristics of the optical element, the measurement electronics, the data acquisition system and various noise sources, thereby affecting the estimation and calculation of the transfer function, and therefore cannot guarantee the sharpness of the reconstructed image.
One chinese (CN 105976321) patent discloses a method and apparatus for super-resolution reconstruction of optical coherence tomography. It is proposed to divide a three-dimensional low-resolution OCT image into a plurality of similar frame groups in the time dimension, and divide each OCT image in the plurality of similar frame groups into a plurality of membrane layers, respectively. Each frame of OCT image is divided into a plurality of overlapping image blocks, and a plurality of similar image blocks for each image block are determined within the membrane layer to which each image block corresponds. And obtaining the average image block of each image block according to the determined multiple similar image blocks of each image block. And processing the obtained average image block of each image block through a pre-constructed high-low resolution dictionary pair and a corresponding sparse coefficient mapping equation to obtain a high resolution image of the three-dimensional OCT image. By constructing a plurality of groups of high-low resolution OCT images which are matched with each other as training samples, the accuracy of reconstructing the super-resolution image is improved. However, the technical feature has the defect that the reconstruction accuracy of the method needs to depend on high-low resolution images of a large number of samples as a training set, the method is easily affected by individual variability so that the accuracy is reduced, and similar image blocks used in the process of calculating the average image block are image blocks inside a film layer. Although the reconstruction noise can be suppressed, the edge information is correspondingly weakened, and the high-frequency components in the reconstructed image are reduced.
An OCT super-resolution reconstruction method and device based on constant-change learning and priori guidance are disclosed in China (CN 116128728). Firstly, constructing a first constraint function of a network to be trained according to a constructed OCT imaging model; constructing a second constraint function of the network to be trained based on constant learning; constructing a third constraint function of the network to be trained based on the anatomical priori knowledge of the tissue to be tested; using the collected noise-containing low-resolution image as a training set, and training the neural network to be trained based on the three constraint functions; and performing high signal-to-noise ratio super-resolution reconstruction on the OCT image to be reconstructed based on the trained network. The self-supervision super-resolution reconstruction is realized by utilizing constant-variation learning, and the self-supervision denoising is realized by utilizing the anatomy priori knowledge of the tested tissue, so that the network can realize the super-resolution and the denoising functions of OCT image reconstruction at the same time. However, the technical characteristics have the defects that the reconstruction is carried out based on a self-supervision deep learning method and priori knowledge of a single image, the improvement multiple of the image resolution is limited, the data fidelity of the reconstructed high-resolution image is reduced, and the reconstruction is easily guided by error data, so that the misdiagnosis rate is increased.
One chinese (CN 116630154) patent discloses a deconvolution super-resolution reconstruction method and apparatus for OCT images. And (3) providing the low-resolution OCT image obtained through Fourier transformation as input original data, constructing an optimization function of sparse continuous prior deconvolution calculation, performing initial setting of reconstruction optimization, performing iterative training, and introducing intermediate variables to perform iterative calculation. After the optimization iteration is completed, the final deconvolution super-resolution reconstructed OCT image is output. The method can avoid artifacts generated by deconvolution iterative reconstruction and effectively improve the resolution of the reconstructed image. However, this technical feature has the disadvantage that the Point Spread Function (PSF) used in the process of constructing the optimization function is an estimated value, and the imaging principle of the OCT system is a scanning interference system, and the obtained signal is a one-dimensional signal, so that there is theoretically no two-dimensional PSF available for reconstruction calculation. Inaccurate estimation of the point spread function may cause artifacts in the reconstructed image. And based on deconvolution reconstruction of single images, limited by the amount of information, the reconstruction of high resolution images has reduced fidelity.
In summary, improving the resolution of the OCT system by improving the hardware configuration method has the problems of high cost and complex system. There are further advantages to improving the resolution of OCT systems by means of image reconstruction over improved hardware systems, but the following problems exist that limit their widespread use in the clinical field: (1) The OCT system used clinically has high integration level, large parameter difference and difficult acquisition, so that the difficulty of calculating the point spread function is high, and the complexity is high; (2) The B-scan image is essentially free of two-dimensional point spread functions which can be used for reconstruction calculation, so that inaccurate estimation of the point spread functions can cause artifacts to be generated on a reconstruction result, and the image quality is reduced; (3) The reconstruction method based on single image and deep learning depends on hardware system parameters and correct high-low resolution image pairs, and partial missing information can cause the reduction of the authenticity of the reconstructed image and the increase of the clinical misdiagnosis rate; (4) The reconstruction resolution multiple is limited by the data volume and the model structure, so that the reconstruction resolution multiple has an upper limit, and more detail information cannot be recovered.
Aiming at the problems, the invention provides a convex set projection super-resolution reconstruction method based on multi-frame OCT images, which does not need to calculate hardware system parameters and construct corresponding high-low resolution data sets. The reconstruction multiple depends on the number of frames of the basic image, so that the upper limit of the prior super-resolution multiple can be broken through theoretically, and a clear and real high-resolution image can be obtained.
Disclosure of Invention
The invention aims to provide a convex set projection super-resolution reconstruction method based on multi-frame optical coherence tomography images, which is used for reconstructing high-frequency information in a high-resolution image by fusing different detail information contained in multi-frame low-resolution images, so as to obtain a clear high-resolution image. The chinese interpretation of OCT (Optical Coherence Tomography) is optical coherence tomography.
The proposal provided by the invention comprises the following steps:
(1) The OCT fault scanning low-resolution image sequence is acquired, the scanning mode of the OCT system is changed, so that the OCT system scans the same section in a short time to obtain multi-frame B-ultrasonic scanning (B-scan) images, one frame of images is selected as a reference frame, and other images are selected as reference frames and used for calculating and reconstructing high-resolution images.
(2) Screening multi-frame B-scan images with the same section, comparing reference frames with selected reference frames respectively, and selecting reference frames with large PSNR (peak signal to noise ratio) and SSIM (structural similarity index) values to be fused with the reference frames for reconstruction. The selected reference frames and the filtered reference frames constitute a reconstructed low resolution dataset.
(3) And (3) performing angle correction on the data set by using a Scale Invariant Feature Transform (SIFT) algorithm, adjusting the direction of the reference frame to be consistent with the reference frame, and eliminating the angle influence existing in the subsequent displacement estimation. And detecting all feature points and corresponding feature vectors in the reference frame and the reference frame through a SIFT algorithm, finding out the feature points in the reference frame matched with the feature points in the reference frame through calculating Euclidean distances of the feature vectors in the two images, and establishing a corresponding relation. Screening all the matched characteristic points, selecting the characteristic point with the highest partial matching degree, counting the coordinates of the characteristic point, and calculating the average rotation angle of the two images. The reference frame image is corrected according to the calculated rotation angle.
(4) The reference frame and the reference frame are cut into image blocks, the frequency domain registration method is utilized to calculate displacement between the corresponding image blocks, and the angle correction is carried out based on the SIFT algorithm in the last step, so that the angle influence existing in the calculated displacement is ignored, only displacement in the x and y directions is considered to exist, and the number of pixels is taken as a unit.
Principle of frequency domain registration:
space domain image f 2 (x) Relative to image f 1 The displacement of (u) corresponds to the image F in the frequency domain 2 (u) relative to image F 1 (u) phase shift, whereby the displacement parameter Deltax can be calculated by calculating the phase difference + (F 2 (u)/F 1 (u)) is obtained. The relative displacement of each image block is recorded, and projection super-resolution image reconstruction is carried out on the basis of sub-pixel displacement (delta x, delta y).
(5) The convex set projection image reconstruction process mainly comprises the following six steps:
(5.1) establishing an image degradation model: firstly, an image acquisition model which is connected with an original high-resolution image and a low-resolution observation sequence is established, namely an iterative formula of an unknown high-resolution image can be expressed as follows:
(5.2) interpolating the reference frame to the target resolution: and interpolating the reference frame image by using a linear interpolation method, taking the interpolated reference frame as the initial estimation of the high-resolution image, and carrying out pixel value correction pixel by utilizing prior information provided by the reference frame on the basis.
(5.3) defining a closed convex set for each pixel of the high resolution image: in the actual imaging process, the noise is inevitably interfered, so that the influence of additive noise is introduced on the basis of linear translation fuzzy constraint, and the statistical property of the noise is compared with the prior boundary delta 0 To be connected. In case of additive noiseIs Gaussian distributed and has a variance of sigma v Delta then 0 Should be equal to cσ v (c.gtoreq.0) by a certain statistical confidence. The following closed convex set is defined for each pixel of the reconstructed high resolution image:
wherein,represents x (i) 1 ，i 2 ) And g (m) 1 ，m 2 ) Residual errors between them.
And (5.4) calculating residual errors pixel by pixel according to the degradation model, judging whether the boundary limit is exceeded, and correcting the pixel value of the interpolated reference frame image to limit the pixel value not to exceed a threshold value: and calculating residual errors between the reference frame and the analog low-resolution image subjected to the point spread function downsampling, and correcting the gray value of the pixel point according to the following formula according to whether the residual error value exceeds the boundary limit.
The corrected pixel values need to be satisfied within the image threshold interval [0, 255]
(5.5) introducing all reference frames frame by frame for correction: and sequentially reading all the reference frames, traversing all pixel points in the reference frames, and calculating residual errors. And correcting the residual value on the interpolated and amplified reference frame, wherein the correction position corresponds to:
(5.6) performing correction for a plurality of times: because the supplementary information in the reference frame cannot be completely fused to the high-resolution reference frame by one iteration due to the amplitude limitation of the point spread function, multiple iterations are needed to fully fuse different information of the same imaging object acquired from different reference frame images.
The operation object of the reconstruction process is an image block at the corresponding positions of the reference frame and the reference frame, different image blocks can be registered according to the corresponding sub-pixel displacement, and the pixel value correction position is more accurate. Therefore, displacement estimation and image reconstruction are carried out in blocks, and then block images subjected to super-resolution reconstruction are spliced according to original positions, so that a whole super-resolution image is obtained.
The convex set projection super-resolution reconstruction method based on multi-frame optical coherence tomography has the characteristics and beneficial effects that the convex set projection super-resolution reconstruction method based on multi-frame optical coherence tomography mainly has the following four advantages:
(1) The method can improve the horizontal resolution and the longitudinal resolution of the OCT system at the same time, and does not need to change and upgrade system hardware, thereby reducing the manufacturing cost and the technical requirement of the high-resolution OCT system, and being capable of breaking through diffraction limit and obtaining the resolution of OCT pixels in submicron level.
(2) The super-resolution reconstruction technology based on multi-frame images is different from the deconvolution technology based on single images, the dependence on the point spread function is not high, and accurate calculation of the point spread function is not needed. The difficulty that the parameters of the optical scanning system are difficult to obtain due to high integration level of the OCT system is avoided, and the complexity of calculation is reduced. The high resolution image can be reconstructed by changing only the scan mode of the OCT system, and thus the method can be applied to OCT apparatuses of all parameter models.
(3) The resolution improvement factor depends on the number of frames of the reconstructed data set, and generally follows the principle that the square of the reconstruction factor does not exceed the number of frames of the data set (k 2 N), therefore, the super-resolution reconstruction technology based on the deep learning is limited by model parameters, and the super-resolution reconstruction of any multiple can be realized on the premise of supporting a data set.
(4) The method is based on priori information provided by a plurality of frames of low-resolution images for reconstruction, and the information quantity of a single imaging object is richer than that of super-resolution reconstruction based on single image deconvolution and deep learning, so that the authenticity of the reconstructed image is higher than that of other two methods. The defect that the estimation of the point spread function is inaccurate because the OCT system does not have a two-dimensional space point spread function in the super-resolution technology based on single image deconvolution is overcome. The need for a training set for a large number of correct high-low resolution images by deep learning based super resolution techniques is avoided. High efficiency reconstruction of a true high resolution OCT image can be achieved.
The method provided by the invention can realize high-efficiency reconstruction of a real high-resolution OCT image, and reduce the manufacturing cost of high-resolution OCT equipment. No support of a large number of data sets is required, and no in-depth knowledge of OCT optical scanning system parameters is required, thus reducing computational cost and complexity, and being applicable to OCT systems of all parameters and models. More importantly, the method can theoretically break through the diffraction limit and realize the OCT scanning accuracy of submicron level.
Drawings
FIG. 1 is a block diagram of a computational reconstruction process according to the present invention.
Fig. 2 (a) shows the peak signal-to-noise ratio of the reference frame and the reference frame.
Fig. 2 (b) shows the structural similarity between the reference frame and the reference frame.
Fig. 3 is an image displacement of a reference frame relative to a base frame.
Fig. 4 is an image of SIFT algorithm feature point detection and matching.
Fig. 5 is a reconstructed high resolution image and a reference interpolated high resolution image.
Detailed Description
In order to further clarify the process according to the invention, a detailed description of the technical solution according to the invention is given below with reference to the drawings and by way of specific examples.
The convex set projection super-resolution reconstruction method based on the multi-frame optical coherence tomography image adopts the technical scheme that high-frequency information in a high-resolution image is reconstructed by fusing information contained in a multi-frame low-resolution image so as to obtain a clear high-resolution image. The reconstruction method comprises two parts of motion estimation and projection set projection image reconstruction, wherein the motion estimation comprises 4 steps, and the projection set projection image reconstruction comprises 6 steps.
Motion estimation:
(1) And acquiring an OCT tomographic low-resolution image sequence to change a scanning mode of the optical coherence tomography system so as to enable the OCT tomographic low-resolution image sequence to rapidly scan the same cross section to obtain multi-frame B-scan images, wherein one frame of images is selected as a reference frame, and other images are selected as reference frames and used for calculating and reconstructing high-resolution images.
(2) Screening multi-frame B-scan images with the same section, comparing reference frames with selected reference frames respectively, selecting reference frames with large PSNR and SSIM values to be fused with the reference frames for reconstruction, and forming a reconstructed low-resolution data set by the selected reference frames and the screened reference frames.
(3) And (3) performing angle correction on the data set obtained in the step (2) by using a SIFT algorithm, adjusting the direction of the reference frame to be consistent with the reference frame, and eliminating the angle influence existing in the subsequent displacement estimation. And detecting all feature points and corresponding feature vectors in the reference frame and the reference frame through a SIFT algorithm, finding out the feature points in the reference frame matched with the feature points in the reference frame through calculating Euclidean distances of the feature vectors in the two images, and establishing a corresponding relation. Screening all the matched characteristic points, selecting the characteristic point with the highest matching degree, and counting the coordinates of the characteristic points. And calculating the average rotation angle of the two images, and correcting the reference frame image according to the calculated rotation angle.
(4) The reference frame and the reference frame are cut into image blocks, the displacement is calculated between the corresponding image blocks by using a frequency domain registration method proposed by Vandewalle and the like, and in the step 3, the angle correction is already performed based on the SIFT algorithm, so that the angle influence existing in the calculated displacement is ignored, and only the displacement in the x and y directions is considered to exist, and the number of pixels is taken as a unit.
Vandewalle frequency domain registration principle calculation formula:
space domain image f 2 (x) Relative to image f 1 (x) Corresponding to the displacement of image F in the frequency domain 2 (u) relative to image F 1 Phase shift of (u). The displacement parameter Deltax can be calculated by calculating the phase difference angle (F 2 (u)/F 1 (u)) is obtained. The relative displacement of each image block is recorded, and projection super-resolution image reconstruction is carried out on the basis of sub-pixel displacement (delta x, delta y).
After the above, reconstructing the high resolution image by using a convex set projection method, wherein the convex set projection method is to assume the unknown image as an element suitable for the Hilbert space, each frame of low resolution reference frame is used as a priori knowledge of the unknown image, a closed convex set containing a solution in the Hilbert space is limited and generated, and then a limit of an amplitude boundary is introduced, so that an iteration formula for solving the unknown image is derived, and the super resolution image is calculated by initial estimation iteration.
The convex set projection image reconstruction comprises the following six steps:
(5.1) establishing an image degradation model: firstly, an image acquisition model which is connected with an original high-resolution image and a low-resolution observation sequence is established, and the model is an iterative formula of an unknown high-resolution image and can be expressed as follows:
wherein g l (m 1 ，m 2 ) For the first frame of low resolution image, x is the original high resolution image, h l Representing a spatial Point Spread Function (PSF), called a degradation function, η l Representing additive noise. m is m 1 ×m 2 For low resolution image pixel size, n 1 ×n 2 I for high resolution image pixel size 1 ×i 2 The number of local pixel points of the high-resolution image, the size and the spatial point spread function h l And consistent.
(5.2) interpolating the reference frame to the target resolution: the reference frame image is interpolated by using a linear interpolation method, the interpolated reference frame is used as the initial estimation of the high-resolution image, and pixel value correction is carried out pixel by utilizing prior information (information contained in the low-resolution image) provided by the reference frame.
(5.3) defining a closed convex set for each pixel of the high resolution image: in the actual imaging process, the noise is inevitably interfered, so that the influence of additive noise is introduced on the basis of linear translation fuzzy constraint, and the statistical property of the noise is compared with the prior boundary delta 0 To be connected. If the additive noise is gaussian distributed and its variance is σ v Delta then 0 Equal to csigma v (c.gtoreq.0) by a certain statistical confidence. The following closed convex set is defined for each pixel of the reconstructed high resolution image:wherein (1)>Represents x (i) 1 ，i 2 ) And g (m) 1 ，m 2 ) Residual errors between them. Wherein g (m 1 ，m 2 ) Representing a single pixel in a low resolution reference frame; x (i) 1 ，i 2 ) Representing the sum of g (m 1 ，m 2 ) Corresponding region pixels. Its regional scale is [ M 1 ，M 2 ]Represents the spatial point spread function h (m 1 ，m 2 ；i 1 ，i 2 ) Is a template size of (c). When x (i) 1 ，i 2 ) R represents the pixel point of the real high-resolution image (y) (m 1 ，m 2 ) And noise eta l (m 1 ，m 2 ) The distribution is consistent. Definition of the closed convex set indicates that the reference frame image is represented at pixel points (m 1 ，m 2 ) The absolute value of the difference between the value of the point and the value of the simulated imaging process, which is limited to a preset boundary condition delta 0 ＝cσ v And (3) inner part.
And (5.4) calculating residual errors pixel by pixel according to the degradation model, judging whether the boundary limit is exceeded, and correcting the pixel value of the interpolated reference frame image to limit the pixel value not to exceed a threshold value: the residual between the reference frame and the simulated low resolution image down-sampled by the point spread function (i.e., the low resolution image of the high resolution initial estimate image down-sampled by the point spread function) is calculated. And correcting the gray value of the pixel point according to the following formula according to whether the residual value exceeds the boundary limit.
The corrected pixel values need to be satisfied within the image threshold interval 0, 255,
(5.5) introducing all reference frames frame by frame for correction: reading all reference frames in sequence, traversing all pixel points in the reference frames, calculating residual errors, correcting residual error values on the reference frames subjected to interpolation amplification, wherein correction positions correspond to:
wherein (m) 1 ，m 2 ) For low resolution reference frame pixel coordinate locations, (Δx, Δy) is the sub-pixel displacement estimated for frequency domain registration, (i) 1 ，i 1 ) Estimating image pixel coordinates for the interpolated reference frame with high resolution initial rate, wherein k is interpolation multiple, a rounding function is arranged in a square bracket, and after all the reference frames are corrected according to the rules, one iteration is completed;
(5.6) performing correction for a plurality of times: because the supplementary information in the reference frame cannot be completely fused to the high-resolution reference frame by one iteration due to the amplitude limitation of the point spread function, multiple iterations are needed to fully fuse different information of the same imaging object acquired from different reference frame images.
The operation object of the reconstruction process is an image block at the corresponding position of the reference frame and the reference frame, and different image blocks are registered according to the corresponding sub-pixel displacement, so that the pixel value correction position is more accurate, displacement estimation and image reconstruction are carried out in a blocking mode, and then block images subjected to super-resolution reconstruction are spliced according to the original position, so that a whole super-resolution image is obtained.
In order to obtain a low-resolution data set for reconstruction, the invention changes the scanning mode of the OCT equipment, and scans the same section of retina within 1s to obtain 25B-scan images, wherein the image size is 256 multiplied by 200pix. Since the imaging object of OCT is the retina of a living human eye with motion variation and is affected by random shot noise, the low-resolution image sequence on the time sequence contains different light-dark boundary information of the same imaging object imaged at different positions, which is also the key of being able to perform super-resolution reconstruction. However, the structural similarity of the low resolution image sequences needs to be compared, because the retina of the living human eye is affected by eye movements, and the local traction of the retina may generate structural deformation. The low resolution images taken at this time can no longer be considered as a sequence of images imaging the same imaging subject. It is therefore necessary to screen the obtained low resolution image for structural similarity.
During screening, two indexes of peak signal-to-noise ratio (PSNR) and Structural Similarity Index (SSIM) are selected to compare the distortion condition and similarity of the image sequence, and according to FIG. 2, the change trend of the two indexes is consistent. The peak signal-to-noise ratio is chosen to remove low resolution images that are too much affected by noise. The structural similarity index is chosen to remove low resolution images that are affected by eye movement and alter the imaging target. The 1 st frame low resolution image is selected as a reference frame and the filtered 2-20 frames of images are selected as reference frames to form a reconstructed low resolution data set. And meanwhile, detecting the whole displacement condition of the previous 20 frames of images by using a SIFT algorithm. As shown in fig. 3, the low resolution reference frames numbered 2, 3, 6, 7, 8, 10, 11, 12, 13, 16 and 18 have sub-pixel displacement relation with the reference frames, and other reference frames have no direct sub-pixel displacement relation, so that the accuracy of imaging information of the same imaging object is reduced, and therefore the low resolution images numbered above are preferentially selected for reconstruction in reconstruction calculation.
And carrying out angle correction on the data set by using a SIFT algorithm, adjusting the direction of the reference frame to be consistent with the reference frame, and eliminating the angle influence existing in the subsequent displacement estimation. The feature point parameters detected by the SIFT algorithm comprise a 128-dimensional feature point description vector, feature point coordinates, scale factors and a rotation main direction. As shown in fig. 4, 693 feature points are detected in the reference frame, and 700 feature points are detected in the reference frame. And (3) finding out the characteristic points in the reference frame matched with the characteristic points in the reference frame by calculating the Euclidean distance of the characteristic vectors in the two diagrams, and establishing a corresponding relation. Screening all the matched characteristic points, selecting the characteristic point with the highest partial matching degree, counting the difference between the rotation main directions of the characteristic points, and calculating the average rotation angle of the two images. And rotating the reference frame image according to the calculated average rotation angle for correction. And eliminating the angle influence in the process of estimating displacement by using a frequency domain method.
The reference frames and the reference frames are then cut into image blocks, and the displacement is calculated between the corresponding image blocks by using the frequency domain registration method proposed by vandews et al. The reason why the frequency domain method is adopted to perform displacement estimation instead of directly performing displacement estimation according to the characteristic points detected by the SIFT is that although the SIFT algorithm is resistant to noise interference, the internal points of the retina OCT image layer structure are easily detected as the characteristic points due to the influence of additive noise, so that a mismatching phenomenon is caused, and compared with the frequency domain registration method based on image blocks, the accuracy is reduced. The reason why the cut block images are matched is that pixel value correction can be performed by combining a plurality of sub-pixel displacement vectors on the whole image, and more accurate addition of information to the correction position can be obtained. Therefore, the single reference frame can be locally fine-tuned by using 16 sub-pixel displacement vectors to obtain a clear high-resolution reconstructed image.
Firstly, establishing a mapping model of an original high-resolution image and a low-resolution reference frame image, namely an iteration formula of an unknown high-resolution image:
interpolation of the reference frame to the target resolution is used as an initial estimate of the high resolution image, and pixel value correction fine tuning is performed pixel by pixel on the basis of prior information provided by the reference frame. Interpolation target multiple depends on the number of reconstructed reference frame-based images, following k 2 And n (k is interpolation magnification, n is reference frame image quantity), wherein the number of reference frames is not less than 16 frames for 4 times reconstruction, and therefore 20 frames of reference frame images are adopted, and the number of reference frames is not less than 9 frames for 3 times reconstruction, and therefore 12 frames of reference frame images are adopted, as shown in figure 5.
Defining a closed convex set for each pixel of the high resolution image:
here, a->Represents x (i) 1 ，i 2 ) And g (m) 1 ，m 2 ) Residual errors between them. Wherein g (m 1 ，m 2 ) Representing a single pixel in a low resolution reference frame, x (i 1 ，i 2 ) Representing the sum of g (m 1 ，m 2 ) Corresponding regional pixels with regional scales of [7,7]I.e. the spatial point spread function h (m 1 ，m 2 ；i 1 ，i 2 ) Is a template size of (c). The multiple relation between the scale of the space point spread function and the reconstruction magnification resolution is M 1 ＝M 2 =2k—1, the spatial point spread function is in the form of a two-dimensional gaussian function. Inaccurate modeling of the spatial point spread function may result in inaccurate estimation of the pixel correction value, but the multiple iterative process of the convex set projection algorithm may make up for the defect, which is mainly dependent on the accuracy of a large amount of prior informationAnd is therefore well suited for use in situations where the OCT scanning system itself does not have a two-dimensional spatial point spread function. Here we consider noise η l (m 1 ，m 2 ) The distribution conforms to the normal distribution N (0, 1), so the definition of the closed convex set represents the reference frame image at pixel points (m 1 ，m 2 ) The absolute value of the difference between the value of (c) and the value of the analog imaging process at that point is limited to a preset boundary condition delta 0 ＝σ v In=1 (c=1).
Next, a residual error between the reference frame and the analog low resolution image downsampled by the point spread function (i.e., the low resolution image of the high resolution initial estimate image downsampled by the point spread function) is calculated, the pixel gray value is modified according to the following definition according to whether the residual error exceeds the boundary, and the modified pixel value is defined within the image threshold interval [0, 255 ].
And sequentially reading all the reference frames, traversing all pixel points in the reference frames, and calculating residual errors. And correcting the residual value on the interpolated and amplified reference frame, wherein the correction position corresponds to:
wherein (m) 1 ，m 2 ) For low resolution reference frame pixel coordinate locations, (Δx, Δy) is the sub-pixel displacement estimated for frequency domain registration, (i) 1 ，i 2 ) And estimating image pixel coordinates for the high-resolution initial rate of the interpolated reference frame, wherein k is an interpolation multiple. And correcting all the reference frames according to the rules, and then completing one iteration. And performing multiple iterations to fully fuse different information of the same imaging object acquired on different reference frame images.
The operation object of the reconstruction process is an image block at the corresponding positions of the reference frame and the reference frame, after displacement estimation and image reconstruction are completed based on a single image, the high-resolution block image subjected to super-resolution reconstruction is spliced according to the original position, and a clear and real whole super-resolution image with rich detail information is obtained.
Detailed description of the results obtained by the super-resolution reconstruction method:
referring to fig. 1, this is the computational reconstruction flow of the present invention: firstly, a plurality of frames of OCT B-scan image sequences with the same section are acquired, and one frame is selected as a reference frame. Then, angle correction and displacement estimation are performed. And extracting feature points and feature vectors in the reference frames by using a SIFT algorithm, calculating Euclidean distances of the feature vectors, and obtaining the minimum Euclidean distance as a matching point. And calculating the average rotation angle of the image according to the coordinates of the matching points in the base frame and the reference frame. The reference frame is rotated by the average rotation angle to coincide with the reference frame direction. Then cutting the image into image blocks, and estimating sub-pixel displacement between the image blocks corresponding to the reference frame and the reference frame by using a frequency domain method for image reconstruction.
In the projection image reconstruction process, the reference frame image is interpolated and amplified to the target reconstruction multiple, and then the reference frame image is sequentially read for high-pass filtering treatment. Only the high frequency information of the low resolution reference frame is fused to the final reconstructed high resolution image, sharpening features in the image. A spatial Point Spread Function (PSF) is calculated, and the interpolated and amplified reference frame is downsampled using the point spread function. And calculating residual errors of the downsampled initial frame and other reference frames, correcting partial pixel values of the initial frame according to residual error values, achieving the purpose of fine adjustment to enable the image to be clear, and limiting the adjusted pixel values to be in a range of 0-255. All the reference frames are sequentially read in to perform the calculation, the iterative process is completed after all the reference frames complete the calculation process, the reconstruction is completed after multiple iterations, and the finally reconstructed high-resolution image is output. The reconstructed object is an image block corresponding to the reference frame and the reference frame, and the final whole image is formed by splicing the reconstructed image blocks according to the original position.
Referring to fig. 2 (a) and 2 (b), the peak signal-to-noise ratio and structural similarity of the reference frame compared to the base frame are shown. And taking the acquired first frame image as a reference frame, and taking other frames as reference frames to number in sequence. It can be seen that the peak signal-to-noise ratio and structural similarity from the 21 st frame image are significantly lower than those of the previous 20 frames images, and that the lower the structural similarity of the images, the less information about the same imaging object contained in the low resolution image. The reconstructed image is susceptible to false information and has artifacts, so that only the first 20 frames of images are selected for reconstruction.
Referring to fig. 3, a subpixel displacement estimate of a reference frame relative to a base frame is shown. And taking the reference frame image of the image 1 as an origin, estimating the displacement of the sub-pixels of the 2-25 frames of images, as shown in figure 3. And drawing a circle by taking single pixel displacement as a radius, wherein a sub-pixel displacement relationship exists between an image represented by a serial number in the circle and a reference frame. The image represented by the serial number outside the circle is different from the reference frame by more than one pixel distance, but the displacement comprises a sub-pixel distance, so that the interpolation newly added pixel can be corrected in the reconstruction process.
Referring to fig. 4, the SIFT algorithm feature point detection and matching is shown. Wherein: a) The characteristic point distribution condition detected for the reference frame; b) The characteristic point distribution condition detected for a certain reference frame; c) The characteristic points in the two frames of images are matched. And recording the coordinate positions of the matched characteristic points in the two frames of images, calculating a rotation angle, taking an average value as an angle correction value, and rotating the reference frame to the same direction as the reference frame.
Referring to fig. 5, a high resolution image reconstructed by the convex set projection super resolution reconstruction method based on multi-frame OCT images is shown. Wherein: a) Is a reference frame image; b) A reference frame image amplified by 4 times by bicubic interpolation; c) The image with 4 times of resolution improvement reconstructed based on the method is a reconstruction result based on 20 frames of low-resolution images and subjected to 10 iterations; d) The image with 3 times of resolution improvement reconstructed based on the method is a reconstruction result based on 12 frames of low-resolution images. The low-resolution image is screened, and 12 frames of images with the closest displacement distance to the reference frame and the highest structural similarity are selected for reconstruction. From the reconstruction results, it can be seen that the high resolution image is enlarged compared to bicubic interpolation.
The high-resolution image with the same magnification, which is reconstructed by the method, is clearer, the visual effect is obviously improved, the contrast is enhanced, the boundary is obvious, and the high-frequency information is more abundant. The 3 times reconstruction result is higher in image quality based on the reference frame, so that part of details are obviously enhanced, the reconstruction result is consistent with the reference frame, and the authenticity of the reconstruction result is higher.
Claims (1)
1. The convex set projection super-resolution reconstruction method based on the multi-frame optical coherence tomography image is characterized in that high-frequency information in a high-resolution image is reconstructed by fusing information contained in a multi-frame low-resolution image so as to obtain a clear high-resolution image, and the reconstruction method comprises the following steps of:
(1) Collecting an optical coherence tomography low-resolution image sequence to change a scanning mode of an optical coherence tomography system, so that the optical coherence tomography system can rapidly scan the same cross section to obtain multi-frame B-ultrasonic scanning images, wherein one frame of images is selected as a reference frame, and the other images are selected as reference frames and used for calculating and reconstructing high-resolution images;
(2) Screening multi-frame B ultrasonic scanning images with the same section, comparing reference frames with selected reference frames respectively, selecting the reference frames with large peak signal-to-noise ratio and structural similarity index values to be fused with the reference frames for reconstruction, wherein the selected reference frames and the screened reference frames form a reconstructed low-resolution data set;
(3) Performing angle correction on the data set obtained in the step 2 by using a scale invariance feature matching algorithm, adjusting the direction of a reference frame to be consistent with the reference frame, eliminating angle influence existing in subsequent displacement estimation, detecting all feature points and corresponding feature vectors in the reference frame and the reference frame by using the scale invariance feature matching algorithm, finding feature points in the reference frame matched with the feature points in the reference frame by calculating Euclidean distance of the feature vectors in the two images, establishing a corresponding relation, screening all the matched feature points, selecting the feature point with the highest matching degree, counting the coordinates of the feature points, calculating the average rotation angle of the two images, and correcting the reference frame image according to the calculated rotation angle;
(4) Cutting the reference frame and the reference frame into image blocks, calculating displacement between the corresponding image blocks by using a frequency domain registration method, and performing angle correction based on a scale invariance feature matching algorithm in the step 3, so that the angle influence existing in the calculated displacement is ignored, only displacement in the x and y directions is considered to exist, and the number of pixels is taken as a unit;
frequency domain registration principle calculation formula:
space domain image f 2 (x) Relative to image f 1 (x) Corresponding to the displacement of image F in the frequency domain 2 (u) relative to image F 1 (u) phase shift, the displacement parameter Deltax may be calculated by calculating the phase difference + (F 2 (u)/F 1 (u)) obtaining, recording the relative displacement of each image block, and reconstructing a convex set projection super-resolution image based on the sub-pixel displacement (deltax, deltay);
(5) The convex set projection image reconstruction process mainly comprises the following six steps:
(5.1) establishing an image degradation model: firstly, an image acquisition model which is connected with an original high-resolution image and a low-resolution observation sequence is established, and the model is an iterative formula of an unknown high-resolution image and can be expressed as follows:
wherein g l (m 1 ，m 2 ) Is the firstFrame low resolution image, x (i) 1 ，i 2 ) H is the original high resolution image l (m 1 ，m 2 ；i 1 ，i 2 ) Representing a spatial Point Spread Function (PSF), also known as a degradation function, η l (m 1 ，m 2 ) Representing additive noise; m is m 1 ×m 2 I for low resolution image pixel size 1 ×i 2 The number of local pixel points of the high-resolution image, the size and the space thereofPoint spread function h l Consistent;
(5.2) interpolating the reference frame to the target resolution: interpolation is carried out on the reference frame image by using a linear interpolation method, the interpolated reference frame is used as initial estimation of a high-resolution image, and pixel value correction is carried out pixel by utilizing prior information provided by the reference frame on the basis;
(5.3) defining a closed convex set for each pixel of the high resolution image: in the actual imaging process, the noise is inevitably interfered, so that the influence of additive noise is introduced on the basis of linear translation fuzzy constraint, and the statistical property of the noise is compared with the prior boundary delta 0 In connection, if the additive noise is gaussian distributed and its variance is σ v Delta then 0 Equal to csigma v (c.gtoreq.0), determined by a determined statistical confidence, defining the following closed convex set for each pixel of the reconstructed high resolution image:
wherein,represents x (i) 1 ，i 2 ) And g (m) 1 ，m 2 ) Residual error between, wherein g (m 1 ，m 2 ) Representing a single pixel in a low resolution reference frame; x (i) 1 ，i 2 ) Representing the sum of g (m 1 ，m 2 ) Corresponding regional pixels with regional scale [ M ] 1 ，M 2 ]Represents the spatial point spread function h (m 1 ，m 2 ；i 1 ，i 2 ) When x (i) 1 ，i 2 ) R represents the pixel point of the real high-resolution image (y) (m 1 ，m 2 ) And noise eta l (m 1 ，m 2 ) The distribution is consistent and the definition of the closed convex set indicates that the reference frame image is displayed at the pixel point (m i ，m 2 ) Is a value of (1) and a simulated imaging process, in whichThe absolute value of the difference between the values of the points, which is limited to a preset boundary condition delta 0 ＝cσ v An inner part;
and (5.4) calculating residual errors pixel by pixel according to the degradation model, judging whether the boundary limit is exceeded, and correcting the pixel value of the interpolated reference frame image to limit the pixel value not to exceed a threshold value: calculating residual error between the reference frame and the analog low-resolution image downsampled by the point spread function, correcting the gray value of the pixel point according to the following formula according to whether the residual value exceeds the boundary limit,
the corrected pixel values need to be satisfied within the image threshold interval 0, 255,
(5.5) introducing all reference frames frame by frame for correction: reading all reference frames in sequence, traversing all pixel points in the reference frames, calculating residual errors, correcting residual error values on the reference frames subjected to interpolation amplification, wherein correction positions correspond to:
wherein (m) 1 ，m 2 ) For low resolution reference frame pixel coordinate locations, (Δx, Δy) is the sub-pixel displacement estimated for frequency domain registration, (i) 1 ，i 2 ) Estimating image pixel coordinates for the interpolated reference frame with high resolution initial rate, wherein k is interpolation multiple, a rounding function is arranged in a square bracket, and after all the reference frames are corrected according to the rules, one iteration is completed;
(5.6) performing correction for a plurality of times: because the supplementary information in the reference frame cannot be completely fused to the high-resolution reference frame by one iteration due to the amplitude limitation of the point spread function, multiple iterations are needed to fully fuse different information of the same imaging object acquired from different reference frame images,
the operation object of the reconstruction process is an image block at the corresponding position of the reference frame and the reference frame, and different image blocks are registered according to the corresponding sub-pixel displacement, so that the pixel value correction position is more accurate, displacement estimation and image reconstruction are carried out in a blocking mode, and then block images subjected to super-resolution reconstruction are spliced according to the original position, so that a whole super-resolution image is obtained.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311382659.3A CN117391955A (en) | 2023-10-24 | 2023-10-24 | Convex set projection super-resolution reconstruction method based on multi-frame optical coherence tomography |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311382659.3A CN117391955A (en) | 2023-10-24 | 2023-10-24 | Convex set projection super-resolution reconstruction method based on multi-frame optical coherence tomography |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117391955A true CN117391955A (en) | 2024-01-12 |
Family
ID=89435502
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311382659.3A Pending CN117391955A (en) | 2023-10-24 | 2023-10-24 | Convex set projection super-resolution reconstruction method based on multi-frame optical coherence tomography |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117391955A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117611447A (en) * | 2024-01-24 | 2024-02-27 | 俐玛精密测量技术(苏州)有限公司 | Industrial CT image super-resolution reconstruction method, device and readable storage medium |
| CN117974475A (en) * | 2024-04-02 | 2024-05-03 | 华中科技大学同济医学院附属同济医院 | Focus image fusion method and system under four-dimensional ultrasonic endoscope observation |
-
2023
- 2023-10-24 CN CN202311382659.3A patent/CN117391955A/en active Pending
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117611447A (en) * | 2024-01-24 | 2024-02-27 | 俐玛精密测量技术(苏州)有限公司 | Industrial CT image super-resolution reconstruction method, device and readable storage medium |
| CN117611447B (en) * | 2024-01-24 | 2024-04-26 | 俐玛精密测量技术(苏州)有限公司 | Industrial CT image super-resolution reconstruction method, device and readable storage medium |
| CN117974475A (en) * | 2024-04-02 | 2024-05-03 | 华中科技大学同济医学院附属同济医院 | Focus image fusion method and system under four-dimensional ultrasonic endoscope observation |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111095349B (en) | Reducing noise in images | |
| US9978159B2 (en) | Method and apparatus for motion correction and image enhancement for optical coherence tomography | |
| CN117391955A (en) | Convex set projection super-resolution reconstruction method based on multi-frame optical coherence tomography | |
| CN110390650B (en) | OCT image denoising method based on dense connection and generation countermeasure network | |
| Suhling et al. | Myocardial motion analysis from B-mode echocardiograms | |
| CN109345469A (en) | It is a kind of that speckle denoising method in the OCT image of confrontation network is generated based on condition | |
| Xu et al. | Alignment of 3-D optical coherence tomography scans to correct eye movement using a particle filtering | |
| US20210272291A1 (en) | Method and computer program for segmentation of optical coherence tomography images of the retina | |
| CN112184549B (en) | Super-resolution image reconstruction method based on space-time transformation technology | |
| Pan et al. | OCTRexpert: a feature-based 3D registration method for retinal OCT images | |
| Parvathi et al. | Automatic drusen detection from colour retinal images | |
| CN111815692B (en) | Artifact-free data, method and system for generating artifact-free data, and storage medium | |
| CN112562058B (en) | Method for quickly establishing intracranial vascular simulation three-dimensional model based on transfer learning | |
| Pan et al. | Segmentation guided registration for 3d spectral-domain optical coherence tomography images | |
| CN116630154A (en) | Deconvolution super-resolution reconstruction method and device for optical coherence tomography | |
| Stankiewicz et al. | Matching 3d oct retina images into super-resolution dataset | |
| CN115205241A (en) | Metering method and system for apparent cell density | |
| Nourrit et al. | Blind deconvolution for high-resolution confocal scanning laser ophthalmoscopy | |
| Alkhaldi et al. | Enhancing the standard operating range of a Placido disk videokeratoscope for corneal surface estimation | |
| CN111951315B (en) | Method for registering optical coherence tomography images of skin | |
| Zhao et al. | Perception-oriented generative adversarial network for retinal fundus image super-resolution | |
| Sengupta | Deep Generative Modeling Based Retinal Image Analysis | |
| CN109118499B (en) | Layer segmentation method of coherent light tomography image based on backtracking shortest path algorithm | |
| CN116188309A (en) | OCT image enhancement method based on registration network and application | |
| JP2023552394A (en) | Use of multiple subvolumes, thicknesses, and curvatures for OCT/OCTA data alignment and retinal landmark detection |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |