CN107895349A - A kind of endoscopic video deblurring method based on synthesis - Google Patents
A kind of endoscopic video deblurring method based on synthesis Download PDFInfo
- Publication number
- CN107895349A CN107895349A CN201710993832.1A CN201710993832A CN107895349A CN 107895349 A CN107895349 A CN 107895349A CN 201710993832 A CN201710993832 A CN 201710993832A CN 107895349 A CN107895349 A CN 107895349A
- Authority
- CN
- China
- Prior art keywords
- frame
- fuzzy
- patch
- motion vector
- grid
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 26
- 230000015572 biosynthetic process Effects 0.000 title claims abstract description 15
- 238000003786 synthesis reaction Methods 0.000 title claims abstract description 15
- 238000004422 calculation algorithm Methods 0.000 claims abstract description 41
- 239000013598 vector Substances 0.000 claims description 43
- 239000011159 matrix material Substances 0.000 claims description 25
- 230000000977 initiatory effect Effects 0.000 claims description 5
- 239000000463 material Substances 0.000 claims description 4
- 230000001186 cumulative effect Effects 0.000 claims description 3
- 230000001815 facial effect Effects 0.000 claims description 3
- 230000000644 propagated effect Effects 0.000 claims description 3
- 238000012545 processing Methods 0.000 abstract description 8
- 230000009286 beneficial effect Effects 0.000 abstract description 2
- 238000003759 clinical diagnosis Methods 0.000 abstract description 2
- 230000000694 effects Effects 0.000 description 4
- 210000002784 stomach Anatomy 0.000 description 4
- 238000000605 extraction Methods 0.000 description 3
- 238000004364 calculation method Methods 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 238000011084 recovery Methods 0.000 description 2
- 238000011160 research Methods 0.000 description 2
- 230000002194 synthesizing effect Effects 0.000 description 2
- 238000000205 computational method Methods 0.000 description 1
- 230000008602 contraction Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000003745 diagnosis Methods 0.000 description 1
- 201000010099 disease Diseases 0.000 description 1
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 description 1
- 235000013399 edible fruits Nutrition 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000005669 field effect Effects 0.000 description 1
- 230000002496 gastric effect Effects 0.000 description 1
- 210000001035 gastrointestinal tract Anatomy 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000002572 peristaltic effect Effects 0.000 description 1
- 230000033764 rhythmic process Effects 0.000 description 1
- 239000011435 rock Substances 0.000 description 1
- 238000004088 simulation Methods 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/73—Deblurring; Sharpening
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/50—Image enhancement or restoration using two or more images, e.g. averaging or subtraction
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
- G06T7/215—Motion-based segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
- G06T7/223—Analysis of motion using block-matching
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/30—Determination of transform parameters for the alignment of images, i.e. image registration
- G06T7/38—Registration of image sequences
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10016—Video; Image sequence
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10068—Endoscopic image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20021—Dividing image into blocks, subimages or windows
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20212—Image combination
- G06T2207/20221—Image fusion; Image merging
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30004—Biomedical image processing
- G06T2207/30092—Stomach; Gastric
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30168—Image quality inspection
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Multimedia (AREA)
- Image Analysis (AREA)
- Image Processing (AREA)
Abstract
The invention discloses a kind of endoscopic video deblurring method based on synthesis, fuzzy frame deblurring in video frame number is handled, utilize the alignment algorithm based on grid, articulating frame in video frame number is divided into by several rectangular blocks by grid, under the common constraint on multiple rectangular block summits, aliging for fuzzy frame and articulating frame is realized by calculating multiple homography matrixs corresponding to multiple rectangular blocks, then fuzzy frame is recombined to make fuzzy frame become clear by DPM algorithms.The beneficial effects of the invention are as follows：This programme can carry out deblurring processing to endoscopic video, and so as to recover the part obscured in endoscopic video, so as to improve the reliability of clinical diagnosis, feelings, which are cut, make it that the subsequent treatment of endoscopic video or image is more convenient.
Description
Technical field
The present invention relates to image processing field, is a kind of endoscopic video deblurring method based on synthesis specifically.
Background technology
Blurred picture Producing reason has a lot, such as optical considerations, exercise factor, human factor etc., wherein mostly
Belong to objective or uncontrollable factor.Therefore image deblurring algorithm is significant in real life.
With the development of society, due to larger operating pressure and faster rhythm of life, most people body is all in
Sub-health state.The illness rate of disease of digestive tract especially stomach trouble remains high throughout the year.At present, ESS be most directly and
Effective alimentary canal illness diagnostic mode.However, during surgical cannula, because operative doctor operation causes the fortune of camera lens
It is dynamic, the peristaltic contraction of gastrointestinal wall and have pain perform the operation in patient because not accommodating resistance caused by camera lens significantly rock, cause stomach
There is a part of fuzzy frame in mirror video, and the diagnosis and treatment to clinician judge to generate certain influence.So it is directed to gastroscope video
Deblurring algorithm there is certain Research Significance.
And existing deblurring algorithm is to be directed to normal image mostly, for the effect of this kind of special video of endoscopic video
Fruit is not highly desirable.According to the characteristic of endoscopic video, research it is a set of it is effective be directed to endoscope deblurring algorithm, no
It is only capable of solving the problems, such as video blur, is also beaten for the subsequent treatment of endoscopic images, such as three-dimensional reconstruction, super-resolution reconstruction etc.
Lower solid foundation.
The content of the invention
It is an object of the invention to provide a kind of mould can be carried out to the arbitrarily fuzzy image block of any blurry video frames
Paste is handled to restore the endoscopic video deblurring method based on synthesis of fuzzy frame.
The present invention is achieved through the following technical solutions：A kind of endoscopic video deblurring method based on synthesis, to video
Fuzzy frame deblurring processing in frame number, using the alignment algorithm based on grid, by grid by the articulating frame in video frame number
Several rectangular blocks are divided into, it is more corresponding to multiple rectangular blocks by calculating under the common constraint on multiple rectangular block summits
Individual homography matrix realizes aliging for fuzzy frame and articulating frame, then recombines fuzzy frame by DPM algorithms to make fuzzy frame become clear
It is clear.This programme proposes a kind of printenv motion estimation model based on grid, improves the robust of gastroscope image alignment algorithm
Property, fuzzy frame is restored using the articulating frame neighbouring with fuzzy frame, is advantageous to find most like image in articulating frame
Detailed information, fuzzy frame then is restored by synthesizing these image informations, so as to improve the accuracy of recovery.
Including following steps：
Step S1：Input endoscopic video t (x, y), it is assumed that nth frame is fuzzy frame, select pending fuzzy frame f (x,
Y), in N-10 into N+10 frames, choose with fuzzy frame f (x, y) is neighbouring and picture quality is a best frame as articulating frame g
(x, y)；
Step S2：Initiation parameter；
Step S3：Articulating frame g (x, y) is snapped into fuzzy frame f (x, y) using based on the alignment algorithm of grid；
Step S4：Fuzzy frame f (x, y) content is recombined using DPM algorithms, makes one completely and clearly
Image；
Step S5：Export the image after deblurring.
Described step S3 includes following steps：
Step S31：Block-by-block traversal articulating frame g (x, y) and ambiguous estimation frame f (x, y) point to articulating frame g (x, y) motion
Vector；
Step S32：The grid division on articulating frame g (x, y), picture is divided into A*A grid cell；
Step S33：The grid top that motion vector obtained by estimation in step S31 is propagated in articulating frame g (x, y) successively
Point；
Step S34：All grid vertexes are traveled through successively, judge whether each lattice point possesses more than 1 motion vector, if
It is, using median filter, to determine unique motion vector；Conversely, continue to travel through next lattice point；
Step S35：So far, all lattice points all only have unique motion vector, the motion on four summits of each cell to
A homography matrix can be calculated in amount, with the algorithm of reverse interpolation, calculated by cell and obtain new picture material, finally,
A clear figure to be alignd with fuzzy frame f (x, y) is obtained, is designated as target figure G (x, y).Model based on grid can be preferably
Nonrigid human stomach's inner surface is fitted, reduces the distortion of picture material
Described step S4 includes following steps：
Step S41：Ambiguity in definition frame f (x, y) size is a*b, and two a*b of definition triple channel image array B and C simultaneously will
It is initialized as 0, B matrixes and is used to record the cumulative pixel value after each pixel is processed repeatedly, and C matrixes are each for recording
The processed number of pixel；
Step S42：Block- matching is carried out to fuzzy frame f (x, y) using DPM algorithms, if the sliding window Q for Block- matching is big
Small is m*m, window Q left upper apex and fuzzy frame f (x, y) picture left upper apex is overlapped, the content that window Q is framed
The hunting zone R for being considered as block a patch, patch is the area for centered on current patch and outwards expanding 15~20 pixels
In domain；
Step S43：In target figure G (x, y) region, search and patch most like window Q, and by obtained by
Patch pixel value is added in matrix B relevant position, meanwhile, by each value of the relevant position in Matrix C+1；
Step S44：Whole pictures are traveled through in units of patch, patch moving step lengths are according to depending on patch sizes；
Step S45：The traversal of all patch in fuzzy frame is completed, the value and matrix of each coordinate points in matrix B will be obtained
The value of corresponding coordinate point is divided by C, and obtained value is exactly the respective pixel value of de-blurred image.
In described step S31, during estimating motion vector, judge in the region whether image belongs to texture structure and enrich
Type, if so, carrying out estimating motion vector from Surf image characteristic points, conversely, using light stream point estimation motion vector；Described step
In rapid S33, the motion vector got based on Surf facial feature estimations is broadcast to four lattice points of this feature point place grid, is based on
The motion vector that light stream estimation is got is broadcast to the lattice point covered using R by the circle of radius, and described propagation refers to making to be passed
Broadcast and a little possess identical motion vector with pickup ponints.
In described step S44, adjacent patch pixel Maximum overlap number is 2 or 3.
The present invention compared with prior art, has advantages below and beneficial effect：
This programme can carry out deblurring processing to endoscopic video, so as to recover the portion obscured in endoscopic video
Point, so as to improve the reliability of clinical diagnosis, feelings, which are cut, make it that the subsequent treatment of endoscopic video or image is more convenient.
Brief description of the drawings
Fig. 1 is this programme principle schematic；
Fig. 2 is motion vector estimation and its propagation schematic diagram based on two methods of Surf and light stream；
Fig. 3 is that CPM algorithms and DPM algorithms calculate the distribution map for respectively obtaining SSD values；
Fig. 4 is experimental result schematic diagram.
Embodiment
The present invention is described in further detail with reference to embodiment, but the implementation of the present invention is not limited to this.
Embodiment 1：
As shown in figure 1, in the present embodiment, a kind of endoscopic video deblurring method based on synthesis, in video frame number
Fuzzy frame deblurring processing, using the alignment algorithm based on grid, the articulating frame in video frame number is divided into by grid
Several rectangular blocks, should by the multiple lists calculated corresponding to multiple rectangular blocks under the common constraint on multiple rectangular block summits
Matrix realizes aliging for fuzzy frame and articulating frame, then recombines fuzzy frame by DPM algorithms to make fuzzy frame become clear.We
Case proposes a kind of printenv motion estimation model based on grid, improves the robustness of gastroscope image alignment algorithm, utilizes
Neighbouring articulating frame restores to fuzzy frame with fuzzy frame, is advantageous to find most like image detail letter in articulating frame
Breath, then restores fuzzy frame, so as to improve the accuracy of recovery by synthesizing these image informations.
Embodiment 2：
On the basis of above-described embodiment, in the present embodiment, a kind of endoscopic video deblurring method based on synthesis, bag
Include following steps：
Step S1：Endoscopic video t (x, y) is inputted, endoscopic video t (x, y) includes several frame of video, it is assumed that the
N frames are pending fuzzy frame, choose pending fuzzy frame f (x, y), in N-10 into N+10 frames, selection and fuzzy frame
The frame that f (x, y) is neighbouring and picture quality is best is as articulating frame g (x, y).
Step S2：Initiation parameter.
Step S3：Articulating frame g (x, y) is snapped into fuzzy frame f (x, y) using based on the alignment algorithm of grid.Utilize base
In the alignment algorithm of grid, it is capable of the non-rigid inner surface of more preferable simulation human stomach, can effectively reduces the distortion of image,
So as to improve the picture quality after fuzzy frame is recovered.It includes following steps：
Step S31：As shown in Fig. 2 (a), block-by-block traversal articulating frame g (x, y) and ambiguous estimation frame f (x, y) sensing articulating frames
G (x, y) motion vector.
Step S32：The grid division on articulating frame g (x, y), picture is divided into A*A grid cell.The wide * of grid
A height of M*N, wherein, M, N value are configured in step s 2.
Step S33：The grid top that motion vector obtained by estimation in step S31 is propagated in articulating frame g (x, y) successively
Point.Described propagation refers to making to be transmitted a little possesses identical motion vector with pickup ponints.
Step S34：All grid vertexes are traveled through successively, judge whether each lattice point possesses more than 1 motion vector, if
It is, using median filter, to determine unique motion vector；Conversely, continue to travel through next lattice point.In the present embodiment, intermediate value is used
It is prior art that multiple vectors are synthesized a vector by wave filter, and the concrete principle for misaligning value filter herein is repeated.
Step S35：So far, all lattice points all only have unique motion vector, the motion on four summits of each cell to
A homography matrix can be calculated in amount, with the algorithm of reverse interpolation, calculated by cell and obtain new picture material, finally,
A clear figure to be alignd with fuzzy frame f (x, y) is obtained, is designated as target figure G (x, y).In the present embodiment, described homography matrix
Homography calculating is a basic skills of image alignment in image processing field, and described reverse interpolation is at image
Pixel value is filled out back to the basic skills of image, those skilled in the art can realize above-mentioned according to the record of this programme in reason field
Effect, the calculating process of the specific calculating process of homography matrix and reverse interpolation is not repeated herein.
Step S4：Fuzzy frame f (x, y) content is recombined using DPM algorithms, makes one completely and clearly
Image.The CPM algorithms of traditional deblurring are replaced using DPM algorithms, eliminate the step of obscuring mask is estimated in CPM algorithms, from
And the efficiency of synthesis can be improved, shorten the time spent needed for image procossing.
It includes following steps：
Step S41：Ambiguity in definition frame f (x, y) size is a*b, and two a*b of definition triple channel image array B and C simultaneously will
It is initialized as 0, B matrixes and is used to record the cumulative pixel value after each pixel is processed repeatedly, and C matrixes are each for recording
The processed number of pixel.
Step S42：Block- matching is carried out to fuzzy frame f (x, y) using DPM algorithms, if the sliding window Q for Block- matching is big
Small is m*m, window Q left upper apex and fuzzy frame f (x, y) picture left upper apex is overlapped, the content that window Q is framed
The hunting zone R for being considered as block a patch, patch is the area for centered on current patch and outwards expanding 15~20 pixels
In domain.In the present embodiment, described Block- matching is a kind of method commonly used in image denoising, estimation.By by query block
Matched with adjacent image block, K closest block of Distance query block is found out from these adjacent blocks.For this area
Technical staff for, the effect above can be realized according to the content that this programme is recorded, herein the not detailed process to matching soon
Repeated with operation principle.In the present embodiment, described sliding window Q size is set in step S2 initiation parameters
It is fixed.
Step S43：In target figure G (x, y) region, search and most like sliding window Q patch, and by gained
Pixel value to patch is added in matrix B relevant position, meanwhile, by each value of the relevant position in Matrix C+1.
Step S44：Whole pictures are traveled through in units of patch, patch moving step lengths are according to depending on patch sizes.
Step S45：The traversal of all patch in fuzzy frame is completed, the value and matrix of each coordinate points in matrix B will be obtained
The value of corresponding coordinate point is divided by C, and obtained value is exactly the respective pixel value of de-blurred image.
Step S5：The image after deblurring is exported, as shown in Figure 4.
In the present embodiment, step S41~step S45 mainly aligns it in fuzzy frame f (x, y) with articulating frame g (x, y)
Afterwards, using the articulating frame after alignment, i.e. corresponding clear patch is substituted corresponding in fuzzy frame f (x, y) in target figure G (x, y)
Fuzzy patch, i.e., fuzzy patch pixel value is substituted with clear patch pixel value.
In the present embodiment, described DPM algorithms are described by directly calculating fuzzy patch and clear patch similarities
Fuzzy patch be sliding window Q coverings in the fuzzy frame f (x, y) scope, described clear patch be target figure G (x,
Y) in region with most like sliding window Q patch.By calculating fuzzy patch and clear patch similarities come instead of meter
The similarity of patch after the obscuring mask convolution that the fuzzy patch of calculation and process have been pre-estimated.Referred herein is similar
The judgment criteria of degree is exactly error sum of squares SSD.Fuzzy patch and clear patch can be expressed as N*N matrix, square
Value in battle array is the pixel value of fuzzy patch or clear patch points.SSD calculation formula are as follows：
Wherein, PijRepresent the pixel value at fuzzy patch midpoints (i, j), QijRepresent the picture of clear patch midpoints (i, j)
Element value.
As shown in figure 3, although the SSD values finally calculated using DPM algorithms and CPM algorithms are different, tried to achieve most
The position consistency that small SSD occurs.Accordingly ensure that the image finally synthesized using DPM algorithms is had extremely compared to CPM algorithms
Few identical precision.
Embodiment 3：
As shown in Fig. 2 on the basis of above-described embodiment, in the present embodiment, in described step S31, estimation motion to
During amount, judge whether image belongs to the abundant type of texture structure in the region, if so, estimating to move from Surf image characteristic points
Vector, as shown in Fig. 2 (c),.If conversely, in the region image be not texture-rich but homogenous area, use light stream point
Estimating motion vector, as shown in Fig. 2 (b).
In the present embodiment, described Surf image characteristic points are extracted by using Surf feature point extraction algorithms.
Surf feature point extraction algorithms are a very famous and classical feature point extractions in image procossing and computer vision field
Method, Surf are Speeded Up Robust Features abbreviations.Itd is proposed by Hebert Bay in 2008.This algorithm
The algorithm has been directly invoked in implementation process to extract characteristic point, has been not directed to and this module is designed or optimized.This
The content that art personnel record according to this programme can realize the effect above, not specific to Surf image characteristic points herein
Calculating process is repeated.Light stream point is expressed by optical flow method, is people in the art using light stream point estimation motion vector
The content that the common knowledge and customary means those skilled in the art of member are recorded according to this programme can realize the effect above, this
Place is not repeated the detailed process and computational methods of light stream point estimation motion vector.
In described step S33, the motion vector got based on Surf facial feature estimations is broadcast to net where this feature point
Four lattice points of lattice, the motion vector got based on light stream estimation is broadcast to the lattice point covered using R by the circle of radius, described
Propagation refers to making to be transmitted a little possesses identical motion vector with pickup ponints.Due to estimating to transport using Surf image characteristic points
Moving vector is due to that image belongs to the abundant type of texture structure in region, so as to cause to estimate to move using Surf image characteristic points
Vector has multiple directions, so as to cause grid lattice point to might have multiple motion vectors, now needs to use intermediate value
Wave filter, determine unique motion vector.In the present embodiment, described radius R is in step S2
Set during initiation parameter.
Embodiment 4：
On the basis of above-described embodiment, in the present embodiment, in described step S44, adjacent patch pixel is maximum
Overlapping number is 2 or 3.We assume that sliding window Q size is 21*21, then the region that sliding window Q is initially covered is
Image coordinate abscissa x is in 1-21, and ordinate y is in 1-21 region, if the step-length that slides laterally of window sliding is 21, then
After sliding once, the region of window covering is x in 22-42, and y is in 1-21 region, it is clear that two regions are tightly adjacent and do not had
Have overlapping.If sliding step is 7, then after once sliding, window institute overlay area is x in 8-28, and y is in 1-21 area
Domain, then it is overlapping between 8-21 to slide two front and rear region x.After third time is slided again, window covers x in 15-36
Region, still the region with first time covering overlap, the maximum times of coincidence are 3.After sliding once again, window covering x
In 22 to 42 position, no longer overlapped with the position at the initial place of window.
Because the overlapping adjacent regions twice of sliding window Q are necessary, it is assumed that it is no overlapping, it is front and rear to be used for replacing twice
May it not associated between two clear blocks of blurred block, then obvious gap can be produced between two blocks, so as to cause
The precision of images after processing is poor.So if lap, just by the pixel value on the corresponding articulating frame found every time
An average is done, the pixel value for being used to replace the point obtained every time and overlapping number are recorded, both are done into division
Obtain average, it is possible to eliminate the gap problem of junction.
Because coloured image is divided into tri- passages of RGB, one coloured image of processing needs to do three passages respectively
Single treatment.Assuming that the size of fuzzy frame is a*b, the essence of image is exactly size a*b matrix, the value inside matrix
That is the pixel value of relevant position, it is shown to be exactly a visual picture with computer.It is same by newly-built one
The completely black image array B of size, the pixel value of black is 0, for storing the pixel value newly obtained, makes the position being process multiple times
Tried to achieve pixel value is put to be superimposed.And a newly-built an equal amount of Matrix C again, the value in matrix is relevant position
Overlapping number.So after whole figure has been handled, the value under matrix B and C respective coordinates is divided by, resulting result is exactly most
The image array after deblurring is obtained afterwards.
It is described above, be only presently preferred embodiments of the present invention, any formal limitation not done to the present invention, it is every according to
Any simply modification, the equivalent variations made according to the technical spirit of the present invention to above example, each fall within the protection of the present invention
Within the scope of.
Claims (6)
1. a kind of endoscopic video deblurring method based on synthesis, handling the fuzzy frame deblurring in video frame number, it is special
Sign is：Using the alignment algorithm based on grid, the articulating frame in video frame number is divided into by several rectangular blocks by grid,
Under the common constraint on multiple rectangular block summits, fuzzy frame is realized by calculating multiple homography matrixs corresponding to multiple rectangular blocks
With aliging for articulating frame, then fuzzy frame recombined by DPM algorithms to make fuzzy frame become clear.
A kind of 2. endoscopic video deblurring method based on synthesis according to claim 1, it is characterised in that：Including with
Under several steps：
Step S1：Input endoscopic video t（X, y）, it is assumed that nth frame is fuzzy frame, selectes pending fuzzy frame f（X, y）,
N-10 chooses and fuzzy frame f into N+10 frames（X, y）A neighbouring and best picture quality frame is as articulating frame g（X, y）；
Step S2：Initiation parameter；
Step S3：Using based on the alignment algorithm of grid by articulating frame g（X, y）Snap to fuzzy frame f（X, y）；
Step S4：Fuzzy frame f is recombined using DPM algorithms（X, y）Content, make one it is complete and clearly scheme
Picture；
Step S5：Export the image after deblurring.
A kind of 3. endoscopic video deblurring method based on synthesis according to claim 2, it is characterised in that：Described
Step S3 includes following steps：
Step S31：Block-by-block traversal articulating frame g（X, y）And ambiguous estimation frame f（X, y）Point to articulating frame g（X, y）Motion vector；
Step S32：In articulating frame g（X, y）Upper grid division, picture is divided into A*A grid cell；
Step S33：Estimation gained motion vector in step S31 is propagated into articulating frame g successively（X, y）Interior grid vertex；
Step S34：All grid vertexes are traveled through successively, judge whether each lattice point possesses more than 1 motion vector, if so, making
With median filter, unique motion vector is determined；Conversely, continue to travel through next lattice point；
Step S35：So far, all lattice points all only have unique motion vector, and the motion vector on four summits of each cell can
A homography matrix is calculated, with the algorithm of reverse interpolation, is calculated by cell and obtains new picture material, finally, obtained
One and fuzzy frame f（X, y）The clear figure of alignment, is designated as target figure G（X, y）.
A kind of 4. endoscopic video deblurring method based on synthesis according to claim 3, it is characterised in that：Described
Step S4 includes following steps：
Step S41：Ambiguity in definition frame f（X, y）Size is a*b, defines two a*b triple channel image array B and C and by the beginning of it
Begin to turn to 0, B matrixes for recording the cumulative pixel value after each pixel is processed repeatedly, C matrixes are for recording each pixel
The processed number of point；
Step S42：Using DPM algorithms to fuzzy frame f（X, y）Block- matching is carried out, if the sliding window Q sizes for Block- matching are
M*m, by window Q left upper apex and fuzzy frame f（X, y）Picture left upper apex overlap, the content that window Q is framed is considered as one
Individual block patch, patch hunting zone R are in the region for centered on current patch and outwards expand 15 ~ 20 pixels；
Step S43：In target figure G（X, y）Region in, search and most like window Q block（patch）, and by obtained by
Patch pixel value is added in matrix B relevant position, meanwhile, by each value of the relevant position in Matrix C+1；
Step S44：Whole pictures are traveled through in units of patch, patch moving step lengths are according to depending on patch sizes；
Step S45：The traversal of all patch in fuzzy frame is completed, will be obtained in matrix B in the value and Matrix C of each coordinate points
The value of corresponding coordinate point is divided by, and obtained value is exactly the respective pixel value of de-blurred image.
A kind of 5. endoscopic video deblurring method based on synthesis according to claim 3 or 4, it is characterised in that：Institute
In the step S31 stated, during estimating motion vector, judge in the region whether image belongs to texture structure and enrich type, if so, from
Surf image characteristic points carry out estimating motion vector, conversely, using light stream point estimation motion vector；In described step S33, it is based on
Four lattice points of grid where the motion vector that Surf facial feature estimations are got is broadcast to this feature point, got based on light stream estimation
Motion vector be broadcast to the lattice point covered using R by the circle of radius, described propagation refers to making to be transmitted a little and pickup ponints
Possess identical motion vector.
A kind of 6. endoscopic video deblurring method based on synthesis according to claim 5, it is characterised in that：Described
In step S44, adjacent patch pixel Maximum overlap number is 2 or 3.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710993832.1A CN107895349B (en) | 2017-10-23 | 2017-10-23 | Endoscope video deblurring method based on synthesis |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710993832.1A CN107895349B (en) | 2017-10-23 | 2017-10-23 | Endoscope video deblurring method based on synthesis |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN107895349A true CN107895349A (en) | 2018-04-10 |
| CN107895349B CN107895349B (en) | 2021-06-29 |
Family
ID=61803635
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710993832.1A Active CN107895349B (en) | 2017-10-23 | 2017-10-23 | Endoscope video deblurring method based on synthesis |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107895349B (en) |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109146820A (en) * | 2018-08-29 | 2019-01-04 | 合肥工业大学 | Real-time hysteroscope video enhances processing method and system, storage medium |
| CN109363614A (en) * | 2018-08-29 | 2019-02-22 | 合肥德易电子有限公司 | Intelligent integral robot cavity mirror system with high definition video enhancing processing function |
| CN110349110A (en) * | 2019-07-16 | 2019-10-18 | 天津师范大学 | A kind of fuzzy image enhancement method and application for crossing fusion based on tired frame |
| CN110581934A (en) * | 2019-07-23 | 2019-12-17 | 安徽南瑞继远电网技术有限公司 | Video jitter elimination method and device, terminal equipment and storage medium |
| CN111275626A (en) * | 2018-12-05 | 2020-06-12 | 深圳市炜博科技有限公司 | Video deblurring method, device and equipment based on ambiguity |
| CN111666998A (en) * | 2020-06-03 | 2020-09-15 | 电子科技大学 | Endoscope intelligent intubation decision-making method based on target point detection |
| CN113269682A (en) * | 2021-04-21 | 2021-08-17 | 青岛海纳云科技控股有限公司 | Non-uniform motion blur video restoration method combined with interframe information |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20060257042A1 (en) * | 2005-05-13 | 2006-11-16 | Microsoft Corporation | Video enhancement |
| US20150206289A1 (en) * | 2014-01-21 | 2015-07-23 | Adobe Systems Incorporated | Joint Video Deblurring and Stabilization |
| CN104867111A (en) * | 2015-03-27 | 2015-08-26 | 北京理工大学 | Block-blur-kernel-set-based heterogeneous video blind deblurring method |
-
2017
- 2017-10-23 CN CN201710993832.1A patent/CN107895349B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20060257042A1 (en) * | 2005-05-13 | 2006-11-16 | Microsoft Corporation | Video enhancement |
| US7548659B2 (en) * | 2005-05-13 | 2009-06-16 | Microsoft Corporation | Video enhancement |
| US20150206289A1 (en) * | 2014-01-21 | 2015-07-23 | Adobe Systems Incorporated | Joint Video Deblurring and Stabilization |
| CN104867111A (en) * | 2015-03-27 | 2015-08-26 | 北京理工大学 | Block-blur-kernel-set-based heterogeneous video blind deblurring method |
Non-Patent Citations (5)
| Title |
|---|
| FEITONG TAN ET.AL: "KERNEL-FREE VIDEO DEBLURRING VIA SYNTHESIS", 《ICIP 2016》 * |
| LEI ZHANG ET.AL: "Bundled Kernels for Nonuniform Blind Video Deblurring", 《IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY》 * |
| LINGBING PENG ET.AL: "Endoscopic Video Deblurring via Synthesis", 《VCIP 2017》 * |
| SHUAICHENG LIU ET.AL: "MeshFlow: Minimum Latency Online Video Stabilization", 《ECCV 2016》 * |
| 谯从彬 等: "基于运动分割的视频去模糊", 《计算机辅助设计与图形学学报》 * |
Cited By (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109146820A (en) * | 2018-08-29 | 2019-01-04 | 合肥工业大学 | Real-time hysteroscope video enhances processing method and system, storage medium |
| CN109363614A (en) * | 2018-08-29 | 2019-02-22 | 合肥德易电子有限公司 | Intelligent integral robot cavity mirror system with high definition video enhancing processing function |
| CN109363614B (en) * | 2018-08-29 | 2020-09-01 | 合肥德易电子有限公司 | Intelligent integrated robot cavity mirror system with high-definition video enhancement processing function |
| CN111275626A (en) * | 2018-12-05 | 2020-06-12 | 深圳市炜博科技有限公司 | Video deblurring method, device and equipment based on ambiguity |
| CN111275626B (en) * | 2018-12-05 | 2023-06-23 | 深圳市炜博科技有限公司 | Video deblurring method, device and equipment based on ambiguity |
| CN110349110A (en) * | 2019-07-16 | 2019-10-18 | 天津师范大学 | A kind of fuzzy image enhancement method and application for crossing fusion based on tired frame |
| CN110581934A (en) * | 2019-07-23 | 2019-12-17 | 安徽南瑞继远电网技术有限公司 | Video jitter elimination method and device, terminal equipment and storage medium |
| CN111666998A (en) * | 2020-06-03 | 2020-09-15 | 电子科技大学 | Endoscope intelligent intubation decision-making method based on target point detection |
| CN111666998B (en) * | 2020-06-03 | 2022-04-22 | 电子科技大学 | Endoscope intelligent intubation decision-making method based on target point detection |
| CN113269682A (en) * | 2021-04-21 | 2021-08-17 | 青岛海纳云科技控股有限公司 | Non-uniform motion blur video restoration method combined with interframe information |
Also Published As
| Publication number | Publication date |
|---|---|
| CN107895349B (en) | 2021-06-29 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107895349A (en) | A kind of endoscopic video deblurring method based on synthesis | |
| CN110443842B (en) | Depth map prediction method based on visual angle fusion | |
| KR100415266B1 (en) | Object region information description method, object region information generating device and recording medium | |
| US10068334B2 (en) | Reconstruction of images from an in vivo multi-camera capsule | |
| KR101175097B1 (en) | Panorama image generating method | |
| CN101625768B (en) | Three-dimensional human face reconstruction method based on stereoscopic vision | |
| JP3435084B2 (en) | Optical flow estimation method and image composition method | |
| US20150334276A1 (en) | System and method for displaying an image stream | |
| WO2022148216A1 (en) | Capsule endoscope image recognition method based on deep learning, and device and medium | |
| Phan et al. | Optical flow-based structure-from-motion for the reconstruction of epithelial surfaces | |
| Park et al. | Recent development of computer vision technology to improve capsule endoscopy | |
| CN114049464A (en) | Reconstruction method and device of three-dimensional model | |
| Koppel et al. | Image-based rendering and modeling in video-endoscopy | |
| CN111696167A (en) | Single image super-resolution reconstruction method guided by self-example learning | |
| CN110035687A (en) | The method and device of image mosaic is carried out to the image for using capsule cameras to be captured | |
| Ren et al. | Structure-aware flow generation for human body reshaping | |
| CN110147809A (en) | Image processing method and device, storage medium and vision facilities | |
| Chen et al. | An effective video stitching method | |
| JP2007282906A (en) | Method, apparatus, and program of medical image processing | |
| CN104933692B (en) | A kind of method for reconstructing and device of human face super-resolution | |
| Drozdzal et al. | Aligning endoluminal scene sequences in wireless capsule endoscopy | |
| Zhang et al. | Semantic feature attention network for liver tumor segmentation in large-scale CT database | |
| Loewke et al. | Deformable image mosaicing for optical biopsy | |
| Ye | Image registration and super-resolution mosaicing | |
| Attar et al. | 2D reconstruction of small intestine's interior wall |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |