CN111462004A - Image enhancement method and device, computer equipment and storage medium - Google Patents
Image enhancement method and device, computer equipment and storage medium Download PDFInfo
- Publication number
- CN111462004A CN111462004A CN202010237049.4A CN202010237049A CN111462004A CN 111462004 A CN111462004 A CN 111462004A CN 202010237049 A CN202010237049 A CN 202010237049A CN 111462004 A CN111462004 A CN 111462004A
- Authority
- CN
- China
- Prior art keywords
- image
- sub
- detail
- enhancement
- difference
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 71
- 238000012545 processing Methods 0.000 claims abstract description 26
- 238000005070 sampling Methods 0.000 claims abstract description 11
- 238000001914 filtration Methods 0.000 claims description 15
- 238000004590 computer program Methods 0.000 claims description 11
- 238000000354 decomposition reaction Methods 0.000 claims description 11
- 238000004364 calculation method Methods 0.000 claims description 8
- 238000010586 diagram Methods 0.000 description 8
- 238000005516 engineering process Methods 0.000 description 8
- 230000002708 enhancing effect Effects 0.000 description 6
- 230000006870 function Effects 0.000 description 6
- 238000004891 communication Methods 0.000 description 4
- 238000003384 imaging method Methods 0.000 description 4
- 238000007781 pre-processing Methods 0.000 description 4
- 210000000746 body region Anatomy 0.000 description 3
- 230000000694 effects Effects 0.000 description 3
- 238000003672 processing method Methods 0.000 description 3
- 238000004458 analytical method Methods 0.000 description 2
- 238000003491 array Methods 0.000 description 2
- 230000003247 decreasing effect Effects 0.000 description 2
- 238000013136 deep learning model Methods 0.000 description 2
- 238000000605 extraction Methods 0.000 description 2
- 238000012163 sequencing technique Methods 0.000 description 2
- 230000003044 adaptive effect Effects 0.000 description 1
- 210000003484 anatomy Anatomy 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000006835 compression Effects 0.000 description 1
- 238000007906 compression Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000003745 diagnosis Methods 0.000 description 1
- 238000002059 diagnostic imaging Methods 0.000 description 1
- 239000000835 fiber Substances 0.000 description 1
- 238000009499 grossing Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000003062 neural network model Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000005215 recombination Methods 0.000 description 1
- 230000006798 recombination Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 230000000717 retained effect Effects 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 238000005728 strengthening Methods 0.000 description 1
- 238000012549 training Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- G06T5/70—
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/50—Image enhancement or restoration by the use of more than one image, e.g. averaging, subtraction
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20016—Hierarchical, coarse-to-fine, multiscale or multiresolution image processing; Pyramid transform
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20024—Filtering details
- G06T2207/20032—Median filtering
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20212—Image combination
- G06T2207/20221—Image fusion; Image merging
Abstract
The invention provides an image enhancement method and device, computer equipment and a computer readable storage medium, which solve the problem that the quality of an image acquired by image acquisition equipment in the prior art cannot meet the requirement. The image enhancement method comprises the following steps: acquiring a group of sub-image sequences obtained by down-sampling an acquired original image, wherein the sub-image sequences are sequentially ordered according to the sequence of sequentially reduced resolution; taking the sub-image with the lowest resolution as a reference image to calculate a difference image of the last sub-image; carrying out multi-scale detail enhancement processing on the difference image to obtain a detail enhancement image; calculating a reconstructed image corresponding to the last sub-image based on the detail enhanced image and the last sub-image; and updating the reference image by using the reconstructed image corresponding to the previous sub-image, and repeating the process to calculate the reconstructed image corresponding to the previous sub-image until the reconstructed image corresponding to the original image is obtained.
Description
Technical Field
The present invention relates to the field of image processing technologies, and in particular, to an image enhancement method and apparatus, a computer device, and a computer-readable storage medium.
Background
At present, in order to meet specific requirements of different fields, various dedicated image acquisition devices are developed, such as CT devices, cameras, and the like, however, due to the influence of internal or external factors of these image acquisition devices, images or videos directly acquired by the image acquisition devices are often insufficient to meet the requirements of users on image quality. Therefore, how to improve the image quality becomes a research focus for those skilled in the related art.
Disclosure of Invention
In view of the above, embodiments of the present invention provide an image enhancement method and apparatus, a computer device and a computer-readable storage medium, so as to solve the problem in the prior art that the quality of an image acquired by an image acquisition device cannot meet the requirement.
The invention provides an image enhancement method in a first aspect, which comprises the following steps: acquiring a group of sub-image sequences obtained by down-sampling an acquired original image, and sequentially sequencing the sub-image sequences and the original image according to the sequence of sequentially reduced resolution; taking the sub-image with the lowest resolution as a reference image to calculate a difference image of the last sub-image; carrying out multi-scale detail enhancement processing on the difference image to obtain a detail enhancement image; calculating a reconstructed image corresponding to the last sub-image based on the detail enhanced image and the last sub-image; and updating the reference image by using the reconstructed image corresponding to the previous sub-image, and repeating the process to calculate the reconstructed image corresponding to the previous sub-image until the reconstructed image corresponding to the original image is obtained.
In one embodiment, calculating the difference image of the last sub-image using the lowest resolution sub-image as the reference image comprises: up-sampling the sub-image with the lowest resolution to obtain an intermediate image with the same resolution as the previous sub-image; and (4) making a difference between the intermediate image and the previous sub-image to obtain a difference image.
In one embodiment, performing multi-scale detail enhancement processing on the difference image to obtain a detail enhanced image includes: filtering the difference image by using a plurality of filters to obtain a plurality of filtered images; obtaining a plurality of detail images based on the difference image and the plurality of filtered images; and recombining the multiple detail images according to a preset rule to obtain a detail enhanced image.
In one embodiment, the plurality of filters are all gaussian filters.
In one embodiment, the Gaussian standard deviations of the plurality of Gaussian filters are sequentially 2(i-1)，i＝1～n。
In one embodiment, deriving the plurality of detail images based on the difference image and the plurality of filtered images comprises: numbering the difference image and the plurality of filtered images in sequence; and sequentially carrying out difference on two adjacent images to obtain a plurality of detail images.
In one embodiment, the recombining the plurality of detail images according to a preset rule to obtain a detail enhanced image includes: and superposing the multiple detailed images according to preset weight to obtain a detailed enhanced image.
In one embodiment, computing a reconstructed image corresponding to a previous sub-image based on the detail-enhanced image and the previous sub-image comprises: and overlapping the detail enhanced image and the previous sub-image to obtain a reconstructed image corresponding to the previous sub-image.
In one embodiment, before acquiring the set of sub-image sequences derived from the acquired original image, further comprising: and carrying out multi-scale pyramid decomposition on the acquired original image to obtain a group of sub-image sequences.
In one embodiment, before acquiring the set of sub-image sequences derived from the acquired original image, further comprising: and carrying out median filtering processing on the acquired original image.
A second aspect of the present invention provides an image enhancement apparatus comprising: the acquisition module is used for acquiring a group of sub-image sequences obtained by down-sampling the acquired original image, and the original image and the sub-image sequences are sequentially ordered according to the sequence of sequentially reduced resolution; the calculation module is used for calculating a difference image of a current image based on the acquired reference image, and the current image is selected from an image sequence which is sequentially ordered; the detail enhancement module is used for carrying out multi-scale detail enhancement processing on the difference image to obtain a detail enhancement image; the reconstruction module is used for calculating a reconstructed image corresponding to the current image based on the detail enhanced image and the current image; and the updating module is used for setting the initial value of the reference image as the sub-image with the lowest resolution and gradually updating the reference image by using the reconstructed image corresponding to the current image.
A third aspect of the present invention provides a computer device, comprising a memory, a processor and a computer program stored on the memory and executed by the processor, wherein the processor implements the steps of the image enhancement method according to any of the above embodiments when executing the computer program.
A fourth aspect of the present invention provides a computer-readable storage medium having stored thereon a computer program which, when being executed by a processor, carries out the steps of the image enhancement method of any one of the above-mentioned embodiments.
According to the image enhancement method and device, the computer equipment and the computer readable storage medium provided by the embodiment of the invention, the multi-scale detail enhancement is carried out on the group of sub-image sequences obtained by the original image down-sampling, namely the multi-scale detail enhancement is carried out on the subdivided detail information, so that the detail information is further enriched. Meanwhile, the information after multi-scale detail enhancement is superposed to the corresponding sub-images and is gradually superposed to the original image, so that the loss of potential effective characteristic information in the original image can be effectively reduced. Particularly, the image enhancement method is used as a preprocessing process of a computer vision model, so that the effective feature extraction capability of the model can be effectively improved, the learning complexity of the model is further reduced, and the robustness of the model is improved.
Drawings
Fig. 1 shows a schematic diagram of an exemplary system architecture to which the image enhancement method or apparatus of the present invention may be applied.
Fig. 2 is a flowchart illustrating an image enhancement method according to a first exemplary embodiment of the present invention.
Fig. 3 is a flowchart illustrating an implementation of the image enhancement method shown in fig. 2.
Fig. 4 is a flowchart of a multi-scale detail enhancement processing method according to an embodiment of the present invention.
Fig. 5 is a flowchart illustrating an image enhancement method according to a second embodiment of the present invention.
Fig. 6 is a block diagram illustrating an image enhancement apparatus according to a first embodiment of the present invention.
Fig. 7 is a block diagram of an image enhancement apparatus according to a second embodiment of the present invention.
Fig. 8 is a block diagram of an electronic device according to an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
With the rapid development of computer technology, the combination of computer image processing technology and imaging technology makes the digital image processing technology increasingly and deeply studied and applied in the medical field, and fundamentally changes the traditional way for medical staff to diagnose. At present, the common medical digital imaging devices include CT, MTI, CR, DR, etc., and medical images acquired by these imaging devices can reflect the internal structure or internal function of an anatomical region, thereby providing information for assisting diagnosis and treatment to a doctor in an intuitive form. It is based on the rigor of the medical field that the detail requirements for medical images are extremely strict. However, in the medical image acquisition process, the quality of the acquired medical image is often insufficient to meet the actual requirement due to the influence of various factors such as the acquisition method, the equipment and random interference, and meanwhile, the related art technicians continuously search for improving the quality of the medical image as much as possible.
The image enhancement technology is a basic image processing technology, and mainly solves the problems of blurred image edges and poor contrast, so that the image quality is improved. For example, the patent publication No. 201310503296.4 provides a method for enhancing the contrast of a digital X-ray image, which performs dynamic range expansion on an image of an interested human body region and dynamic range compression on image information of a non-interested human body region according to the characteristics of X-ray imaging, thereby effectively improving the contrast of the image of the interested human body region, and simultaneously performs detail enhancement and multi-scale contrast resolution enhancement on the dynamically adjusted image, and then performs eye viewing effect adjustment on the obtained image, thereby presenting the human body information in front of the eyes of a doctor. For another example, the' 200710067693.6 patent provides a multi-scale adaptive contrast-transformed medical image enhancement method that decomposes a medical image into a set of pyramid-shaped arranged, progressively lower resolution images; adjusting the layering coefficient obtained by decomposition, wherein the adjustment comprises enhancing the contrast of the whole image of each layer and enhancing the contrast of a local area of a detail level image; and recombining the detail level images with the adjusted coefficients into an image with the enhanced original image. However, in the two methods, during the image enhancement process, either a conventional contrast enhancement method based on gray stretching is adopted, or the gray distribution of the original image is directly and dynamically changed to improve the visual viewing effect of human eyes, so that a large amount of detailed information is lost during the image enhancement process. In this case, if the medical image after the image enhancement processing is used for training the deep learning model, it is likely that the model is difficult to converge or a robustness problem occurs.
In view of the foregoing, the present application provides an image enhancement method and apparatus, as well as a computer device and a computer-readable storage medium, which avoid losing potentially important feature information that cannot be recognized by human eyes by enhancing image details. The medical image processed by the image enhancement method or device provided by the application is particularly suitable for image preprocessing of a deep learning model, so that the complexity of model learning can be effectively reduced, and the robustness is improved.
Fig. 1 shows a schematic diagram of an exemplary system architecture to which the image enhancement method or apparatus of the present invention may be applied. As shown in fig. 1, the system architecture 100 includes a terminal device 101, a network 102, and a server 103.
The network 102 serves as a medium for providing a communication link between the terminal device 101 and the server 102. Network 102 includes various types of connections, such as wire, wireless communication links, or fiber optic cables. The terminal device 101 may be various electronic devices having a display screen, including but not limited to various medical imaging devices, such as CT, MTI, CR, DR, etc.; also included are smart phones, tablet computers, laptop computers, desktop computers, and the like. The server 103 may be a server that provides various services. In this way, a user can use the terminal device 101 to interact with the server 103 through the network 102 to receive or send messages.
The image enhancement method provided by the embodiment of the application can be executed by the terminal equipment 101, and the corresponding image enhancement device is arranged in the terminal equipment 101; it may also be performed by the server 102, with the corresponding image enhancement means being provided in the server 102.
For example, a user acquires a medical image by using the terminal device 101, processes the medical image by using the image enhancement method provided by the embodiment of the present application to obtain an enhanced reconstructed image, and then sends the reconstructed image to the server 103. The server 103 directly operates on the reconstructed image, for example, performs image recognition on the reconstructed image by using the trained neural network model, and feeds back the image obtained after the recognition to the terminal device 101. The terminal apparatus 101 may display the obtained recognized image on a display screen of the terminal apparatus 101, or directly print out.
It should be understood that the number of terminal devices 101, networks 102, and servers 103 shown in fig. 1 is merely illustrative. Any number of terminal devices 101, networks 102, and servers 103 may be provided according to actual needs. For example, the server 103 may be a server cluster composed of a plurality of servers.
Fig. 2 is a flowchart illustrating an image enhancement method according to a first exemplary embodiment of the present invention. Fig. 3 is a flowchart illustrating an implementation of the image enhancement method shown in fig. 2. The image enhancement method is suitable for preprocessing of medical images, and as shown in fig. 2 and 3, the image enhancement method 200 includes the following steps:
step 201, acquiring an original image X acquired by the acquisition0Downsampled set of sub-image sequences XiWhere i is 1 to n, and n is a natural number arbitrarily larger than 1 (the same applies hereinafter). Original image X0And a sub-picture sequence XiAnd sequencing the images in sequence according to the sequence of the resolution ratio which is reduced in sequence.
Further, in one embodiment, the sequence of sub-images XiStepwise distribution of resolution, e.g. original image X0Is the first sub-image X14 times the resolution of the first sub-image X1Is the second sub-image X24 times the resolution of (a), and so on, i.e. the resolution of the previous sub-image is 4 times the resolution of the next sub-image.
Step 202, the sub-image X with the lowest resolution is processednComputing the last sub-image X as a reference imagen-1The difference image of (2). For convenience of description, the previous sub-image X will be describedn-1Is recorded as a first difference image I1Subsequently, the previous sub-image X is further processedn-2Is taken as a second difference image I2AnAnd so on.
Last subimage Xn-1Refers to the sub-image X with the lowest resolutionnAdjacent sub-images. And, due to the sub-image sequence XiIn order of decreasing resolution, i.e. the previous sub-picture Xn-1Is higher than the sub-image X with the lowest resolutionnSo as to calculate the previous sub-image Xn-1The difference image of (2) first needs to be the sub-image X with the lowest resolutionnTo and from the previous sub-image Xn-1The resolution ratio of the image is the same, and then the image obtained after the resolution ratio is improved and the next layer of sub-image X are utilizedn-1Calculating a first difference image I1。
In one embodiment, step S202 specifically includes: first, for the sub-image X with the lowest resolutionnUpsampling to obtain the last subimage Xn-1Of the intermediate image of the same resolution. Secondly, the intermediate image and the previous sub-image Xn-1Making difference to obtain first difference image I1。
Step 203, for the first difference image I1And performing multi-scale detail enhancement processing to obtain a first detail enhanced image I x 1.
The purpose of the multi-scale detail enhancement processing is to study the features of the image and the relation among the features on each scale, and analyze the image information on a deep structure through multi-scale decomposition, so that the accuracy of image feature description is enhanced. The multi-scale detail enhancement processing includes a wavelet-based multi-scale analysis method, a multi-scale geometric analysis method, and the like.
Step S204, enhancing the image I1 and the previous sub-image X based on the first detailn-1Computing the last sub-image Xn-1Corresponding first reconstructed image X1.
The first detail enhanced image I1 is the image corresponding to the previous sub-image Xn-1The image after enhancement processing is carried out on the preset detail area, so that the previous sub-image X is enhanced by using the first detail enhanced image I1n-1The first reconstructed image X1 obtained after the reconstruction corresponds to the previous sub-imageXn-1On the basis, the image obtained by strengthening the details of the preset area is obtained.
In one embodiment, step S204 is specifically performed as: enhancing the first detail by image I1 and the previous sub-image Xn-1Superposing to obtain the last sub-image Xn-1X1.
Step S205, using the previous sub-image Xn-1The corresponding first reconstructed image X1 updates the reference image. Then, steps S202 to S205 are repeated to calculate the previous sub-image Xn-2Corresponding second reconstructed image X2 until original image X is obtained0And the corresponding final reconstructed image X X n is the final detail enhanced image of the original image.
According to the image enhancement method provided by the embodiment, multi-scale detail enhancement is performed on a group of sub-image sequences obtained by down-sampling an original image, namely, multi-scale detail enhancement is further performed on subdivided detail information, so that the detail information is enriched. Meanwhile, the information after multi-scale detail enhancement is superposed to the corresponding sub-image and further superposed to the original image, so that the loss of potential effective characteristic information in the original image can be effectively reduced. Particularly, the image enhancement method is used as a preprocessing process of a computer vision model, so that the effective feature extraction capability of the model can be effectively improved, the learning complexity of the model is further reduced, and the robustness of the model is improved.
Fig. 4 is a flowchart of a multi-scale detail enhancement processing method according to an embodiment of the present invention. The method is suitable for the multi-scale detail enhancement processing procedure in the step S203. As shown in fig. 4, the multi-scale detail enhancement processing method 400 includes:
step S401, filtering the difference image with a plurality of filters to obtain a plurality of filtered images.
The plurality of filters means a plurality of different filters, that is, a plurality of filters having different convolution kernels. By using a plurality of filters to pair difference images IiFiltering is carried out, and image information of different frequency bands can be respectively extracted in a targeted manner, so that noise outside the frequency band is filtered out, and a difference value is obtainedImage IiFiltered image B at different scalesi。
In one embodiment, the plurality of filters are all gaussian filters. The gaussian filter is a linear filter, so that the image is filtered by the gaussian filter without introducing other noise.
For example, the Gaussian standard deviations of the Gaussian filters are sequentially 2(k-1)K is 1 to m, and m is a natural number arbitrarily larger than 1. Correspondingly, the Gaussian kernels of the Gaussian filters are G in sequencekThe image to be processed is a first difference image I1. In this case, the plurality of filtered images may be obtained by the following calculation:
B1＝G1*I1，B2＝G2*I1，……，Bk＝Gk*I1
wherein, B1～BkRepresenting a plurality of filtered images.
Step S402, a plurality of detail images are obtained based on the difference image and the plurality of filtered images.
In one embodiment, step S402 is specifically performed as: firstly, numbering a difference image and a plurality of filtered images in sequence; and secondly, sequentially carrying out difference on two adjacent images to obtain a plurality of detail images.
For example, for the above example, after obtaining the plurality of filtered images B1～BkThen, a plurality of detail images can be obtained by the following calculation formula:
D1＝I1-B1，D2＝B1-B2，……，Dk＝Bk-1-Bk
wherein, B1～BkRepresenting a plurality of detail images.
And S403, recombining the multiple detail images according to a preset rule to obtain a detail enhanced image.
The multiple detail images respectively record the first difference image I1By using the detailed information of the multiple images after the enhancement processing of different frequency bandsRecombining the images, which is equivalent to reconstructing the first difference image I1Recombining the enhanced detail information on different frequency bands to obtain a first difference image I1The details of (1) enhance the image.
In one embodiment, step S403 is specifically performed as: and superposing the multiple detailed images according to preset weight to obtain a detailed enhanced image.
For example, for the above example, multiple detail images B are obtained1～BkThe detail-enhanced image may then be obtained by the following calculation:
I*1＝(1-w1×sign(D1))×D1+w2×D2+…+wk×Dk
wherein w1，w2，……，wkFor modulating parameters, typically w1+w2+...+w k1 is ═ 1; sign is a sign function when x>0, sign (x) 1; when x is 0, sign (x) is 0; when x is<0，sign(x)＝-1。
Fig. 5 is a flowchart illustrating an image enhancement method according to a second embodiment of the present invention. As shown in fig. 5, the image enhancement method 500 differs from the image enhancement method 200 shown in fig. 2 only in that the image enhancement method 500 further comprises, before step S201:
step S501, multi-scale pyramid decomposition is carried out on the collected original image to obtain a group of sub-image sequences.
The image pyramid is an efficient and conceptually simple structure that describes an image in multiple resolutions. An image pyramid corresponding to one image is a series of image sequences with gradually changing resolution arranged according to the pyramid shape, and the image sequences are derived from the same original image. The bottom of the pyramid is a high resolution representation of the original image and the top is an approximation of the low resolution of the original image, the higher the image level, the smaller the image, and the lower the resolution.
In this embodiment, step S501 is specifically executed to perform gaussian pyramid decomposition on the acquired original image to obtain a group of sub-image sequences. The gaussian pyramid is one of image pyramids, and is obtained by down-sampling an original image, that is, the bottom layer of the gaussian pyramid is the original image, and the upper layers are sub-images with stepwise reduced resolution, which are obtained by down-sampling the original image.
According to the image enhancement method provided by the embodiment, a group of sub-image sequences is obtained by carrying out multi-scale pyramid decomposition on the original image, so that richer samples are provided for the subsequent multi-scale detail enhancement operation, and the signal-to-noise ratio of the original image is further improved. Meanwhile, pyramid decomposition and multi-scale detail enhancement are fused, so that the image enhancement process is finer in granularity, and the image enhancement effect is better.
In one embodiment, as shown in fig. 5, the image enhancement method 500 further comprises, before step S501:
step S502, the acquired original image is subjected to median filtering processing.
The median filtering is a nonlinear smoothing technology, which sets the gray value of each pixel point as the median of the gray values of all pixel points in the window of the point field. The median filtering is carried out on the original image, so that isolated noise in the image can be filtered out, and the edge information of the image is retained, thereby improving the uniformity of the gray distribution of the image and reducing the complexity of subsequent operation.
The invention also provides an image enhancement device. Fig. 6 is a block diagram of an image enhancement apparatus according to a first embodiment of the present invention. As shown in fig. 6, the image enhancement apparatus 60 includes an acquisition module 61, a calculation module 62, a detail enhancement module 63, a reconstruction module 64, and an update module 65. The obtaining module 61 is configured to obtain a group of sub-image sequences obtained by downsampling an acquired original image, where the original image and the sub-image sequences are sequentially ordered according to a sequence of sequentially decreasing resolutions. The calculation module 62 is configured to calculate a difference image of a current image based on the acquired reference image, where the current image is selected from a sequentially ordered image sequence. The detail enhancement module 63 is configured to perform multi-scale detail enhancement processing on the difference image to obtain a detail enhanced image. The reconstruction module 64 is configured to calculate a reconstructed image corresponding to the current image based on the detail-enhanced image and the current image. The updating module 65 is configured to set the initial value of the reference image as the sub-image with the lowest resolution, and gradually update the reference image with the reconstructed image corresponding to the current image.
In an embodiment, the reconstruction module 64 is specifically configured to superimpose the detail-enhanced image and the previous sub-image to obtain a reconstructed image corresponding to the previous sub-image.
Fig. 7 is a block diagram of an image enhancement apparatus according to a second embodiment of the present invention.
As shown in fig. 7, in the image enhancement device 70, the calculation module 62 specifically includes: an upsampling unit 621 and a difference unit 622. The upsampling unit 621 is configured to upsample the reference image to obtain an intermediate image with the same resolution as the current image. The difference unit 622 is configured to perform a difference between the intermediate image and the current image to obtain a difference image.
The detail enhancement module 63 specifically includes a filtering unit 631, a calculating unit 632, and a recombining unit 633. The filtering unit 631 is configured to perform filtering processing on the difference image by using multiple filters to obtain multiple filtered images. The calculating unit 632 is configured to obtain a plurality of detail images based on the difference image and the plurality of filtered images. The recombination unit 633 is configured to recombine the multiple detail images according to a preset rule, so as to obtain a detail enhanced image.
In one embodiment, the plurality of filters are all gaussian filters.
In one embodiment, the Gaussian standard deviations of the plurality of Gaussian filters are sequentially 2(i-1)，i＝1～n。
In one embodiment, the calculating unit 632 is specifically configured to number the difference image and the plurality of filtered images sequentially; and sequentially carrying out difference on two adjacent images to obtain a plurality of detail images.
In one embodiment, the recombining unit 633 is specifically configured to superimpose the multiple detail images according to a preset weight, so as to obtain the detail enhanced image.
In one embodiment, the image enhancement device 60 further includes a decomposition module 66 and a median filtering module 67. The decomposition module 66 is configured to perform multi-scale pyramid decomposition on the acquired original image to obtain a group of sub-image sequences. The median filtering module 67 is used for performing median filtering on the acquired original image.
The image enhancement device provided by the embodiment of the invention and the image enhancement method provided by the embodiment of the invention belong to the same inventive concept, can execute the image enhancement method provided by any embodiment of the invention, and have the corresponding functional modules and beneficial effects of executing the image enhancement method. For details of the technique not described in detail in this embodiment, reference may be made to the image enhancement method provided in the embodiment of the present invention, and details are not described here again.
The invention also provides the electronic equipment. Fig. 8 is a block diagram of an electronic device according to an embodiment of the present invention. As shown in fig. 8, the electronic device 80 includes one or more processors 81 and memory 82.
The processor 81 may be a Central Processing Unit (CPU) or other form of processing unit having data processing capabilities and/or instruction execution capabilities, and may control other components in the electronic device 80 to perform desired functions.
Memory 82 may include one or more computer program products that may include various forms of computer-readable storage media, such as volatile memory and/or non-volatile memory. The volatile memory may include, for example, Random Access Memory (RAM), cache memory (cache), and/or the like. The non-volatile memory may include, for example, Read Only Memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium and executed by the processor 81 to implement the image enhancement methods of the various embodiments of the present application described above and/or other desired functions. Various contents such as an input signal, a signal component, a noise component, etc. may also be stored in the computer-readable storage medium.
In one example, the electronic device 80 may further include: an input device 83 and an output device 84, which are interconnected by a bus system and/or other form of connection mechanism (not shown).
For example, when the electronic device is the terminal device 101 or the server 102 in fig. 1, the input device 83 may be the microphone or the microphone array described above for capturing the input signal of the sound source. When the electronic device is a stand-alone device, the input means 83 may be a communication network connector for receiving the acquired input signal from the terminal device 101 or the server 102.
The input device 83 may include, for example, a keyboard, a mouse, and the like.
The output device 84 may output various information including the determined distance information, direction information, and the like to the outside. The output devices 84 may include, for example, a display, speakers, a printer, and a communication network and its connected remote output devices, among others.
Of course, for simplicity, only some of the components of the electronic device 80 relevant to the present application are shown in fig. 8, and components such as buses, input/output interfaces, and the like are omitted. In addition, the electronic device 80 may include any other suitable components depending on the particular application.
An embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored, where the computer program, when executed by a processor, implements the image enhancement method according to any of the foregoing embodiments. The computer storage medium may be any tangible medium, such as a floppy disk, a CD-ROM, a DVD, a hard drive, even a network medium, and the like.
It should be understood that although one implementation form of the embodiments of the present invention described above may be a computer program product, the method or apparatus of the embodiments of the present invention may be implemented in software, hardware, or a combination of software and hardware. The hardware portion may be implemented using dedicated logic; the software portions may be stored in a memory and executed by a suitable instruction execution system, such as a microprocessor or specially designed hardware. It will be appreciated by those of ordinary skill in the art that the methods and apparatus described above may be implemented using computer executable instructions and/or embodied in processor control code, such code provided, for example, on a carrier medium such as a disk, CD or DVD-ROM, programmable memory such as read only memory (firmware), or a data carrier such as an optical or electronic signal carrier. The methods and apparatus of the present invention may be implemented in hardware circuitry, such as very large scale integrated circuits or gate arrays, semiconductors such as logic chips, transistors, or programmable hardware devices such as field programmable gate arrays, programmable logic devices, or in software for execution by various types of processors, or in a combination of hardware circuitry and software, such as firmware.
It should be understood that although several modules or units of the apparatus are mentioned in the above detailed description, such division is merely exemplary and not mandatory. Indeed, according to exemplary embodiments of the invention, the features and functions of two or more modules/units described above may be implemented in one module/unit, whereas the features and functions of one module/unit described above may be further divided into implementations by a plurality of modules/units. Furthermore, some of the modules/units described above may be omitted in some application scenarios.
It should be understood that the terms "first", "second", "third" and "fourth" used in the description of the embodiments of the present invention are only used for clearly illustrating the technical solutions, and are not used for limiting the protection scope of the present invention.
The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, equivalents and the like that are within the spirit and principle of the present invention are included in the present invention.
Claims (13)
1. An image enhancement method, comprising:
acquiring a group of sub-image sequences obtained by down-sampling an acquired original image, wherein the sub-image sequences and the original image are sequentially ordered according to the sequence of sequentially reduced resolution;
taking the sub-image with the lowest resolution as a reference image to calculate a difference image of the last sub-image;
carrying out multi-scale detail enhancement processing on the difference image to obtain a detail enhancement image;
calculating a reconstructed image corresponding to the previous sub-image based on the detail enhanced image and the previous sub-image;
and updating the reference image by using the reconstructed image corresponding to the previous sub-image, and repeating the process to calculate the reconstructed image corresponding to the previous sub-image until the reconstructed image corresponding to the original image is obtained.
2. The image enhancement method of claim 1, wherein said taking the sub-image with the lowest resolution as the reference image to calculate the difference image of the last sub-image comprises:
up-sampling the sub-image with the lowest resolution to obtain an intermediate image with the same resolution as that of the previous sub-image;
and obtaining the difference image by taking the difference between the middle image and the previous sub-image.
3. The image enhancement method according to claim 1, wherein the performing the multi-scale detail enhancement processing on the difference image to obtain a detail enhanced image comprises:
filtering the difference image by using a plurality of filters to obtain a plurality of filtered images;
obtaining a plurality of detail images based on the difference image and the plurality of filtered images;
and recombining the multiple detail images according to a preset rule to obtain the detail enhanced image.
4. The image enhancement method of claim 3, wherein the plurality of filters are all Gaussian filters.
5. The image enhancement method according to claim 4, wherein the Gaussian standard deviations of the Gaussian filters are sequentially 2(i-1)，i＝1～n。
6. The method of claim 3, wherein deriving the plurality of detail images based on the difference image and the plurality of filtered images comprises:
sequentially numbering the difference image and the plurality of filtered images;
and sequentially carrying out difference on two adjacent images to obtain the plurality of detail images.
7. The image enhancement method according to claim 3, wherein said recombining the plurality of detail images according to a preset rule to obtain the detail enhanced image comprises:
and superposing the multiple detail images according to preset weight to obtain the detail enhanced image.
8. The image enhancement method according to claim 1, wherein the calculating a reconstructed image corresponding to the previous sub-image based on the detail-enhanced image and the previous sub-image comprises:
and superposing the detail enhanced image and the previous sub-image to obtain a reconstructed image corresponding to the previous sub-image.
9. The image enhancement method of claim 1, prior to said acquiring a set of sub-image sequences derived from the acquired original image, further comprising:
and carrying out multi-scale pyramid decomposition on the acquired original image to obtain the group of sub-image sequences.
10. The image enhancement method of claim 1, prior to said acquiring a set of sub-image sequences derived from the acquired original image, further comprising:
and carrying out median filtering processing on the acquired original image.
11. An image enhancement apparatus, comprising:
the acquisition module is used for acquiring a group of sub-image sequences obtained by down-sampling an acquired original image, and the original image and the sub-image sequences are sequentially ordered according to the sequence of sequentially reduced resolution;
the calculation module is used for calculating a difference image of a current image based on the acquired reference image, and the current image is selected from an image sequence which is sequentially ordered;
the detail enhancement module is used for carrying out multi-scale detail enhancement processing on the difference image to obtain a detail enhancement image;
the reconstruction module is used for calculating a reconstructed image corresponding to the current image based on the detail enhanced image and the current image;
and the updating module is used for setting the initial value of the reference image as the sub-image with the lowest resolution and gradually updating the reference image by using the reconstructed image corresponding to the current image.
12. A computer device comprising a memory, a processor and a computer program stored on the memory for execution by the processor, characterized in that the steps of the image enhancement method as claimed in any one of claims 1 to 10 are implemented when the computer program is executed by the processor.
13. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the steps of the image enhancement method according to any one of claims 1 to 10.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010237049.4A CN111462004B (en) | 2020-03-30 | 2020-03-30 | Image enhancement method and device, computer equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010237049.4A CN111462004B (en) | 2020-03-30 | 2020-03-30 | Image enhancement method and device, computer equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111462004A true CN111462004A (en) | 2020-07-28 |
| CN111462004B CN111462004B (en) | 2023-03-21 |
Family
ID=71685035
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010237049.4A Active CN111462004B (en) | 2020-03-30 | 2020-03-30 | Image enhancement method and device, computer equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111462004B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117058125A (en) * | 2023-09-01 | 2023-11-14 | 无锡维凯科技有限公司 | Detection method and system based on rear cover glass of mobile phone |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP0574969A2 (en) * | 1992-06-19 | 1993-12-22 | Agfa-Gevaert N.V. | A method and apparatus for noise reduction |
| US5461655A (en) * | 1992-06-19 | 1995-10-24 | Agfa-Gevaert | Method and apparatus for noise reduction |
| CN1889125A (en) * | 2006-07-26 | 2007-01-03 | 深圳市嘉易通医疗科技有限公司 | Medical radiation image detail enhancing method |
| GB201710000D0 (en) * | 2015-02-19 | 2017-08-09 | Magic Pony Tech Ltd | Online training of hierarchical algorithms |
| CN108389214A (en) * | 2018-03-06 | 2018-08-10 | 青岛海信医疗设备股份有限公司 | The processing method and processing device of ultrasonoscopy, electronic equipment, storage medium |
| CN109949233A (en) * | 2019-02-18 | 2019-06-28 | 深圳蓝韵医学影像有限公司 | Filter out the method, system, equipment and storage medium of scattered rays in radioscopic image |
| CN110070511A (en) * | 2019-04-30 | 2019-07-30 | 北京市商汤科技开发有限公司 | Image processing method and device, electronic equipment and storage medium |
-
2020
- 2020-03-30 CN CN202010237049.4A patent/CN111462004B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP0574969A2 (en) * | 1992-06-19 | 1993-12-22 | Agfa-Gevaert N.V. | A method and apparatus for noise reduction |
| US5461655A (en) * | 1992-06-19 | 1995-10-24 | Agfa-Gevaert | Method and apparatus for noise reduction |
| CN1889125A (en) * | 2006-07-26 | 2007-01-03 | 深圳市嘉易通医疗科技有限公司 | Medical radiation image detail enhancing method |
| GB201710000D0 (en) * | 2015-02-19 | 2017-08-09 | Magic Pony Tech Ltd | Online training of hierarchical algorithms |
| CN108389214A (en) * | 2018-03-06 | 2018-08-10 | 青岛海信医疗设备股份有限公司 | The processing method and processing device of ultrasonoscopy, electronic equipment, storage medium |
| CN109949233A (en) * | 2019-02-18 | 2019-06-28 | 深圳蓝韵医学影像有限公司 | Filter out the method, system, equipment and storage medium of scattered rays in radioscopic image |
| CN110070511A (en) * | 2019-04-30 | 2019-07-30 | 北京市商汤科技开发有限公司 | Image processing method and device, electronic equipment and storage medium |
Non-Patent Citations (2)
| Title |
|---|
| 曹洪玉;刘冬梅;付秀华;张静;岳鹏飞;: "基于CT图像的超分辨率重构研究" * |
| 李佳田;李显凯;李应芸;钱堂慧;李果家;林艳;: "航空图像光流场的逆向金字塔计算方法" * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117058125A (en) * | 2023-09-01 | 2023-11-14 | 无锡维凯科技有限公司 | Detection method and system based on rear cover glass of mobile phone |
| CN117058125B (en) * | 2023-09-01 | 2024-03-15 | 无锡维凯科技有限公司 | Detection method and system based on rear cover glass of mobile phone |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111462004B (en) | 2023-03-21 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| JP7093886B2 (en) | Image processing methods and devices, electronic devices and storage media | |
| CN109389552B (en) | Image super-resolution algorithm based on context-dependent multitask deep learning | |
| CN110163237B (en) | Model training and image processing method, device, medium and electronic equipment | |
| CN113436076B (en) | Image super-resolution reconstruction method with characteristics gradually fused and electronic equipment | |
| CN111583285A (en) | Liver image semantic segmentation method based on edge attention strategy | |
| CN111462004B (en) | Image enhancement method and device, computer equipment and storage medium | |
| Zhu et al. | Low-light image enhancement network with decomposition and adaptive information fusion | |
| Hai et al. | Advanced retinexnet: a fully convolutional network for low-light image enhancement | |
| CN114187181A (en) | Double-path lung CT image super-resolution method based on residual information refining | |
| CN112200719B (en) | Image processing method, electronic device, and readable storage medium | |
| An et al. | EIEN: endoscopic image enhancement network based on retinex theory | |
| CN110689486A (en) | Image processing method, device, equipment and computer storage medium | |
| Siddiqi et al. | An Ensembled spatial enhancement method for image enhancement in healthcare | |
| CN111161189A (en) | Single image re-enhancement method based on detail compensation network | |
| CN106127712B (en) | Image enhancement method and device | |
| Tian et al. | Retinal fundus image superresolution generated by optical coherence tomography based on a realistic mixed attention GAN | |
| CN116266336A (en) | Video super-resolution reconstruction method, device, computing equipment and storage medium | |
| CN113506226A (en) | Motion blur restoration method and system | |
| CN113128583A (en) | Medical image fusion method and medium based on multi-scale mechanism and residual attention | |
| CN116188346B (en) | Image quality enhancement method and device for endoscope image | |
| Song et al. | NeXtResUNet: A neural network for industrial CT image denoising | |
| CN113555091B (en) | Parkinson sMRI image-based disease classification system, method and device | |
| Cap et al. | A Practical Framework for Unsupervised Structure Preservation Medical Image Enhancement | |
| Vo et al. | Multi-Range Fusion for X-ray Image Enhancement | |
| de Almeida Simões | Image Quality Improvement of Medical Images using Deep Learning for Computer-aided Diagnosis |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information | ||
| CB02 | Change of applicant information |
Address after: Room B401, floor 4, building 1, No. 12, Shangdi Information Road, Haidian District, Beijing 100085 Applicant after: Tuxiang Medical Technology Co.,Ltd. Address before: Room B401, floor 4, building 1, No. 12, Shangdi Information Road, Haidian District, Beijing 100085 Applicant before: INFERVISION |
|
| GR01 | Patent grant | ||
| GR01 | Patent grant |