CN110689502B - Image processing method and related device - Google Patents
Image processing method and related device Download PDFInfo
- Publication number
- CN110689502B CN110689502B CN201910955227.4A CN201910955227A CN110689502B CN 110689502 B CN110689502 B CN 110689502B CN 201910955227 A CN201910955227 A CN 201910955227A CN 110689502 B CN110689502 B CN 110689502B
- Authority
- CN
- China
- Prior art keywords
- image
- matching
- minimum
- images
- processed
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000003672 processing method Methods 0.000 title claims abstract description 21
- 239000013598 vector Substances 0.000 claims abstract description 87
- 238000012545 processing Methods 0.000 claims abstract description 74
- 238000000034 method Methods 0.000 claims abstract description 38
- 230000000903 blocking effect Effects 0.000 claims abstract description 36
- 239000011159 matrix material Substances 0.000 claims description 35
- 230000008569 process Effects 0.000 claims description 14
- 239000000126 substance Substances 0.000 claims description 6
- 238000005259 measurement Methods 0.000 claims description 5
- 230000000875 corresponding effect Effects 0.000 description 30
- 230000004927 fusion Effects 0.000 description 7
- 238000004590 computer program Methods 0.000 description 6
- 238000010586 diagram Methods 0.000 description 5
- 230000009467 reduction Effects 0.000 description 4
- 230000000694 effects Effects 0.000 description 2
- 238000001914 filtration Methods 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 230000006698 induction Effects 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 230000000007 visual effect Effects 0.000 description 2
- 230000002411 adverse Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 239000003086 colorant Substances 0.000 description 1
- 230000008094 contradictory effect Effects 0.000 description 1
- 238000012937 correction Methods 0.000 description 1
- 230000002596 correlated effect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000006073 displacement reaction Methods 0.000 description 1
- 230000005669 field effect Effects 0.000 description 1
- 239000000463 material Substances 0.000 description 1
- 230000000877 morphologic effect Effects 0.000 description 1
- 238000007781 pre-processing Methods 0.000 description 1
- 238000011946 reduction process Methods 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/70—Denoising; Smoothing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
- G06F17/10—Complex mathematical operations
- G06F17/16—Matrix or vector computation, e.g. matrix-matrix or matrix-vector multiplication, matrix factorization
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/22—Matching criteria, e.g. proximity measures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/11—Region-based segmentation
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/80—Camera processing pipelines; Components thereof
- H04N23/81—Camera processing pipelines; Components thereof for suppressing or minimising disturbance in the image signal generation
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/95—Computational photography systems, e.g. light-field imaging systems
- H04N23/951—Computational photography systems, e.g. light-field imaging systems by using two or more images to influence resolution, frame rate or aspect ratio
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Mathematical Physics (AREA)
- Mathematical Optimization (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Pure & Applied Mathematics (AREA)
- Signal Processing (AREA)
- Mathematical Analysis (AREA)
- Multimedia (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Computational Mathematics (AREA)
- Software Systems (AREA)
- Databases & Information Systems (AREA)
- Algebra (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Image Processing (AREA)
Abstract
The embodiment of the application provides an image processing method and a related device. The image processing method of the application shoots N under-exposed images in a left exposure mode to obtain the image with the highest sharpness in the N under-exposed images; the method comprises the steps of carrying out blocking processing on N underexposed images to obtain a first processing image group, wherein the size of a pixel block of the first processing image group is T1, and the first processing image group comprises N processing images; matching each pixel block of other images except the image with the highest sharpness in the first processed image group with the corresponding pixel block on the image with the highest sharpness to obtain N-1 first minimum matching errors, wherein each processed image corresponds to one first minimum matching error; obtaining N-1 first matching vectors corresponding to N-1 first minimum matching errors, wherein each first matching vector corresponds to one first minimum matching error; and (4) superposing the N-1 first matching vectors to the image with the highest sharpness, and outputting the processed image, thereby effectively improving the image quality.
Description
Technical Field
The embodiment of the application relates to the field of image processing, in particular to an image processing method and a related device.
Background
Generally, the image quality is affected by various noises during the generation and transmission of the image, which adversely affects the processing of the subsequent image and the visual effect of the image. The image noise is generated due to various reasons, such as the influence of sensor material properties, working environment, electronic components and circuit structures, and various noises, such as thermal noise caused by resistance, channel thermal noise of a field effect transistor, photon noise, dark current noise, and photo response non-uniformity noise, may be introduced.
Noise often appears as an isolated pixel or block of pixels on an image that causes a strong visual effect. In general, the noise signal is not correlated with the object to be studied, it appears as useless information, disturbing the observable information of the image, and the noise makes the image unclear. In order to suppress noise, improve image quality, and facilitate higher-level processing, it is necessary to perform denoising preprocessing on an image. In the prior art, methods such as spatial domain filtering, transform domain filtering or morphological noise filter are generally adopted to reduce noise of an image, but the noise reduction technology in the prior art cannot effectively process noise of a moving part, and a highlight part is unnatural in the noise reduction process and cannot effectively reduce noise.
Therefore, how to effectively process the noise of the image is an urgent technical problem to be solved.
Disclosure of Invention
In view of the above, an object of the present invention is to provide an image processing method and a related apparatus, which can perform noise reduction processing on each pixel block of an image, thereby effectively improving image quality.
In order to achieve the above object, in a first aspect, an embodiment of the present application provides an image processing method applied to an image processing apparatus, including: shooting N under-exposed images in a left exposure mode to obtain the image with the highest sharpness in the N under-exposed images; the method comprises the steps of carrying out blocking processing on N underexposed images to obtain a first processing image group, wherein the size of a pixel block of the first processing image group is T1, and the first processing image group comprises N processing images; matching each pixel block of other images except the image with the highest sharpness in the first processed image group with the corresponding pixel block on the image with the highest sharpness to obtain N-1 first minimum matching errors, wherein each processed image corresponds to one first minimum matching error; obtaining N-1 first matching vectors corresponding to N-1 first minimum matching errors, wherein each first matching vector corresponds to one first minimum matching error; and superposing the N-1 first matching vectors on the image with the highest sharpness, and outputting the processed image.
In one embodiment, after the step of obtaining N-1 first minimum matching errors, the method further includes: the N under-exposed images are subjected to blocking processing to obtain a second processed image group, the pixel block size of the second processed image group is T2, wherein T2 is smaller than T1, the second processed image group comprises N processed images, each pixel block of the other images except the image with the highest sharpness in the second processed image group is matched with the corresponding pixel block on the image with the highest sharpness respectively to obtain N-1 second minimum matching errors and N-1 second matching vectors, each processed image corresponds to one second minimum matching error, and each second matching vector corresponds to one second minimum matching error; and obtaining the minimum value of the N-1 first minimum matching errors and the N-1 second minimum matching errors, if the minimum value is the second minimum matching error, superposing the N-1 second matching vectors on the image with the highest sharpness by using a weight matrix W, and outputting the processed image, wherein the weight matrix W is a preset matrix model.
In one embodiment, if the minimum value is the first minimum matching error, N-1 first matching vectors are superimposed on the image with the highest sharpness by using a weight matrix W, and the processed image is output, wherein the weight matrix W is a preset matrix model.
In one embodiment, matching each pixel block of the other images except the image with the highest sharpness in the first processed image group with the corresponding pixel block on the image with the highest sharpness to obtain N-1 first minimum matching errors, specifically comprising: respectively matching pixel blocks of other images with pixel blocks in a preset range in the image with the highest sharpness to obtain a plurality of matching errors, wherein the other images comprise N-1 processed images, and each processed image corresponds to the plurality of matching errors; acquiring the minimum value of a plurality of matching errors obtained after each processed image is matched and taking the minimum value as a first minimum matching error, wherein each processed image corresponds to one first minimum matching error; acquiring N-1 first minimum matching errors of N-1 processed images; the matching process specifically comprises the following steps: ω ═ integral | Ii(x+u)-I0(x) | dx; where ω is the first minimum match error, IiIs an underexposed image, I0Is the sharpest image and u is the first matching vector.
In one embodiment, the superimposing N-1 second matching vectors onto the image with the highest sharpness with the weight matrix W, and outputting the processed image specifically includes:
wherein the content of the first and second substances,
is a processed image, IiIs an underexposed image, v is a second matching vector, I0Is the sharpest picture, W ═ Wi(x，v)。
In one of the embodiments, the first and second electrodes are,
wherein the content of the first and second substances,
is an estimate of the reliability of the second matching vector, f (x, v) ═ Ii(x+v)-I0(x)|，
The second minimum match error is corrected by the noise model,
is an estimate of the noise by the noise model,
a, b are parameters of the noise model, obtained by sensor measurement statistics of the image processing apparatus, C1，C2Is a preset value.
In one embodiment, the size of T1 is twice that of T2.
In a second aspect, an embodiment of the present application provides an image processing apparatus, including:
the shooting unit is used for shooting N underexposed images in a leftward exposure mode;
a first acquisition unit configured to acquire a sharpness-highest image of the N underexposed images captured by the capturing unit;
a blocking unit, configured to perform blocking processing on the N underexposed images captured by the capturing unit to obtain a first processed image group, where a pixel block size of the first processed image group is T1, and the first processed image group includes the N processed images;
the matching unit is used for respectively matching each pixel block of other images except the image with the highest sharpness in the first processed image group obtained by the blocking unit with the corresponding pixel block on the image with the highest sharpness obtained by the obtaining unit to obtain N-1 first minimum matching errors, and each processed image corresponds to one first minimum matching error;
the second obtaining unit is used for obtaining N-1 first matching vectors corresponding to N-1 first minimum matching errors, and each first matching vector corresponds to one first minimum matching error;
and the processing unit is used for superposing the N-1 first matching vectors on the image with the highest sharpness acquired by the acquisition unit and outputting the processed image.
In a third aspect, an embodiment of the present application provides an electronic device, where the electronic device includes: a memory storing at least one instruction; and a processor executing instructions stored in the memory to implement the image processing method as provided by the first aspect.
In a fourth aspect, the present application provides a computer-readable storage medium, where at least one instruction is stored, and the at least one instruction is executed by a processor in an electronic device to implement the image processing method according to the first aspect.
According to the method and the device, the underexposed image shot in a leftward exposure mode is subjected to pixel blocking processing, the image subjected to pixel blocking and the image with the highest sharpness are subjected to pixel block matching fusion, the matching vector corresponding to the pixel block with the minimum error is superposed with the image with the highest sharpness to reduce noise, noise of a motion part and a highlight part can be effectively processed, and image quality is effectively improved.
The present application is described in detail below with reference to the attached drawings and specific examples, but the present application is not limited thereto.
Drawings
Fig. 1 is a flowchart illustrating a step of an image processing method according to an embodiment of the present application;
FIG. 2 is a flowchart illustrating another step of an image processing method according to an embodiment of the present disclosure;
fig. 3 is a schematic structural diagram of an image processing apparatus according to an embodiment of the present application;
fig. 4 is a schematic diagram of a hardware structure of an electronic device according to an embodiment of the present application.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more apparent, the present application is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of and not restrictive on the broad application. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
Technical solutions between various embodiments may be combined with each other, but must be realized by those skilled in the art, and when the technical solutions are contradictory or cannot be realized, such a combination should not be considered to exist, and is not within the protection scope of the present application.
Example one
The embodiment provides an image processing method, which shoots N underexposed images in a leftward exposure mode; acquiring the image with the highest sharpness in the N underexposed images; the method comprises the steps of carrying out blocking processing on N underexposed images to obtain a first processing image group, wherein the size of a pixel block of the first processing image group is T1, and the first processing image group comprises N processing images; matching each pixel block of other images except the image with the highest sharpness in the first processed image group with the corresponding pixel block on the image with the highest sharpness to obtain N-1 first minimum matching errors, wherein each processed image corresponds to one first minimum matching error; and acquiring N-1 first matching vectors corresponding to N-1 first minimum matching errors, wherein each first matching vector corresponds to one first minimum matching error, superposing the N-1 first matching vectors on the image with the highest sharpness, and outputting the processed image. The method and the device can be used for performing superposition noise reduction on the matching vector corresponding to the pixel block with the minimum error and the image with the highest sharpness, can effectively process the noise of a motion part and a highlight part, and effectively improve the image quality.
Referring to FIG. 1, a flow chart of steps of an image processing method of the present invention is shown. It is to be understood that the flow charts in the embodiments of the present method are not intended to limit the order in which the steps are performed. The following exemplary description is given by taking an electronic device as an execution subject, and may specifically include the following steps:
and S101, shooting N under-exposed images in a left exposure mode, and acquiring the image with the highest sharpness in the N under-exposed images.
In the implementation of the present application, a set of RAW images is taken by left exposure, and the shuffled RAW images include N underexposed images.
And step S102, carrying out blocking processing on the N underexposed images to obtain a first processing image group, wherein the pixel block size of the first processing image group is T1, and the first processing image group comprises N processing images.
In the implementation of the present application, one pixel has only one possible value of the three primary colors R, G or B. T1 is typically a power of 2 multiple, such as 64, 32, or 16. After the under-exposed image is processed in a blocking mode, the pixel block of the under-exposed image becomes small, so that the error in subsequent matching can be smaller, and the matching can be more accurate.
Step S103, each pixel block of the other images except the image with the highest sharpness in the first processed image group is matched with the corresponding pixel block on the image with the highest sharpness respectively to obtain N-1 first minimum matching errors, and each processed image corresponds to one first minimum matching error.
The matching in this application is induction intensity matching, which is a value of a corresponding position of a sensor, that is, a luminance value of a filter is matched. Matching errors are generated in the matching process, each pixel block generates one matching error in the matching process, and each matching error corresponds to one matching vector.
In one embodiment, each pixel block in each processed image is matched with a pixel block in a preset range in the image with the highest sharpness, for example, the preset range may be 5 pixel blocks or other pixel blocks in a certain range. If each processed image comprises P pixel blocks, the P pixel blocks of one processed image are respectively matched with the pixel blocks within the range of 5 pixels in the image with the highest sharpness, 5P times of matching is needed to obtain 5P matching errors and matching vectors, the minimum value of the 5P matching errors is the first minimum matching error, and N-1 processed images are respectively matched to obtain N-1 first minimum matching errors. If each processed image comprises P pixel blocks, the P pixel blocks of one processed image are respectively matched with all the pixel block blocks in the image with the highest sharpness, P times of matching needs to be carried out, P times of matching are obtained, P times of matching errors and matching vectors are obtained, the minimum value is the first minimum matching error from the P times of matching errors, N-1 processed images are respectively matched, and (N-1) P times of first minimum matching errors can be obtained.
And S104, acquiring N-1 first matching vectors corresponding to N-1 first minimum matching errors, wherein each first matching vector corresponds to one first minimum matching error.
Taking the first minimum matching error obtained in the step S103 as a reference, a first matching vector corresponding to the first minimum matching error is obtained, and each processed image corresponds to one first matching vector.
And step S105, superposing the N-1 first matching vectors to the image with the highest sharpness, and outputting the processed image.
After N-1 first matching vectors are obtained in step S104, the N-1 first matching errors may be superimposed on the sharpness-highest image, and the sharpness-highest image may be subjected to noise reduction processing to output a noise-reduced image.
The method and the device have the advantages that the underexposed image shot in a left exposure mode is subjected to pixel blocking, pixel blocks of a smaller image can be found by performing pixel block matching fusion on the image subjected to pixel blocking and the image with the highest sharpness, a matching vector corresponding to the pixel block with the minimum error is superposed with the image with the highest sharpness, so that image noise is reduced, noise of a motion part and a highlight part can be effectively processed, and image quality is effectively improved.
Example two
Referring to FIG. 2, a flowchart of another step of the image processing method of the present invention is shown. It is to be understood that the flow charts in the embodiments of the present method are not intended to limit the order in which the steps are performed. The following exemplary description is given by taking an electronic device as an execution subject, and may specifically include the following steps:
step S201, shooting N underexposed images in a left exposure mode.
And step S202, acquiring the image with the highest sharpness in the N underexposed images.
Step S203, the N underexposed images are processed in a blocking mode to obtain a first processing image group, the size of a pixel block of the first processing image group is T1, and the first processing image group comprises N processing images.
And step S204, respectively matching pixel blocks of other images except the image with the highest sharpness in the first processed image group with corresponding pixel blocks on the image with the highest sharpness to obtain N-1 first minimum matching errors, wherein each processed image corresponds to one first minimum matching error.
And S205, performing blocking processing on the N under-exposed images to obtain a second processed image group, wherein the pixel block size of the second processed image group is T2, T2 is smaller than T1, and the second processed image group comprises the N processed images.
In one embodiment, T2 is 1/2T1, that is, N under-exposed images are subjected to blocking processing, and the pixel block size of the second processed image group is half that of the first processed image group. In addition, the method and the device can also directly perform the blocking processing of dividing the pixel block of the first processing image group into two parts to obtain the second processing image group, wherein the size of the pixel block of the second processing image group is half of that of the pixel block of the first processing image group, so that after the pixel blocking processing is performed on the processing image twice, the pixel block with a smaller size can be obtained, and the subsequent matching and fusing steps can be more accurate.
And S206, matching each pixel block of other images except the image with the highest sharpness in the second processed image group with the corresponding pixel block on the image with the highest sharpness respectively to obtain N-1 second minimum matching errors and N-1 second matching vectors, wherein each processed image corresponds to one second minimum matching error, and each second matching vector corresponds to one second minimum matching error.
Specifically, the matching operation includes: respectively matching pixel blocks of other images with pixel blocks in a preset range in the image with the highest sharpness to obtain a plurality of matching errors, wherein the other images comprise N-1 processed images, and each processed image corresponds to the plurality of matching errors; acquiring the minimum value of a plurality of matching errors obtained after each processed image is matched and taking the minimum value as a first minimum matching error, wherein each processed image corresponds to one first minimum matching error; acquiring N-1 first minimum matching errors of N-1 processed images;
specifically, the matching operation processes in step S204 and step S206 can be matched through the following formula, where ω ═ ^ jektii(x+u)-I0(x) | dx. Where ω is the first or second match error, IiIs an underexposed image, I0Is the sharpest image and u is the first match vector or the second match vector.
The matching in step S204 and step S206 is induction intensity matching, which is a value of a corresponding position of the sensor, that is, a luminance value of the filter is matched. If each processed image comprises P pixel blocks, the P pixel blocks of one processed image are respectively matched with the pixel blocks within the range of 5 pixels in the image with the highest sharpness, 5P times of matching is needed to obtain 5P matching errors and matching vectors, the minimum value of the 5P matching errors is taken as the minimum matching error, and N-1 processed images are respectively matched to obtain N-1 minimum matching errors. If each processed image comprises P pixel blocks, the P pixel blocks of one processed image are respectively matched with all the pixel block blocks in the image with the highest sharpness, P times of matching needs to be carried out, P times of matching are obtained, P times of matching errors and matching vectors are obtained, the minimum value is taken from the P times of matching errors and is used as the minimum matching error, the N-1 processed images are respectively matched, and the (N-1) P times of minimum matching errors can be obtained. The minimum match error is the first minimum match error in step S204 and the second minimum match error in step S206.
And S207, acquiring the minimum value of the N-1 first minimum matching errors and the N-1 second minimum matching errors.
And S208, if the minimum value is the second minimum matching error, superposing the N-1 second matching vectors to the image with the highest sharpness by using a weight matrix W, and outputting the processed image, wherein the weight matrix W is a preset matrix model.
If the minimum value is the second minimum matching error, the pixel block of the second processed image group after secondary pixel blocking is more similar to the pixel block in the image with the highest sharpness, the matching vectors of the second processed image group are adopted for superposition, the matching and fusion effect is better, the noise of the obtained processed image is smaller, and the image quality is better. The weight matrix W is a preset matrix model, the reliability of the second matching vector can be estimated, and the second matching error can be corrected through a noise model.
And S209, if the minimum value is the first minimum matching error, superposing the N-1 first matching vectors to the image with the highest sharpness by using a weight matrix W, and outputting the processed image, wherein the weight matrix W is a preset matrix model.
If the minimum value is the first minimum matching error, the pixel block of the first processed image group after the first pixel blocking is more similar to the pixel block in the image with the highest sharpness, the matching vectors of the first processed image group are adopted for superposition, the matching and fusion effect is better, the noise of the obtained processed image is smaller, and the image quality is better.
The method and the device have the advantages that the underexposed image shot in a left exposure mode is subjected to pixel blocking, pixel blocks of a smaller image can be found by performing pixel block matching fusion on the image subjected to pixel blocking and the image with the highest sharpness, a matching vector corresponding to the pixel block with the minimum error is superposed with the image with the highest sharpness, so that image noise is reduced, noise of a motion part and a highlight part can be effectively processed, and image quality is effectively improved.
EXAMPLE III
For better understanding of the embodiments of the present application, the following describes an image processing method of the present application with a specific application example, and the image processing method of the present application may include the following steps:
step S301, shooting a group of underexposed images omega with the total number of N in a leftward exposure mode: { I0，I1，...，IN-1}。
Step S302, obtaining the image I with the highest sharpness0。
Step S303, to the underexposed image IiAnd I0Into blocks of pixels of size T1.
Step S304, each IiAbove pixel block and I0Matching the corresponding pixel blocks to obtain a matching vector (u), and calculating a matching error ω, ω ═ ^ integral | Ii(x+u)-I0(x)|dx。
In step S305, the pixel block with size T1 is further subdivided into pixel blocks with size T2, T2 is 1/2T1, and the matching vector v and the matching error C are iteratively calculated0。
Wherein, C0Similar to the calculation formula of step S305.
Step S306, comparing the matching error omega with the matching error C0。
Step S307, if C0Significantly less than ω, then the matching vector v replaces the vector in u.
The specific implementation can be seen in the following formula:
wherein the initial matching vector of each pixel sub-block is v0＝u。
Step S308, for the image IiIs given by superimposing its matching vector to I by the following formula0And obtaining the noise-reduced image.
Wherein the weight matrix WiThe design of (x, v) determines the quality of the image fusion,
is an estimate of the reliability of the second matching vector, f (x, v) ═ Ii(x+v)-I0(x)|，
Is a correction of the second match error by the noise model,
is an estimate of the noise by the noise model,
a, b are parameters of the noise model, obtained by sensor measurement statistics of the image processing apparatus, C1，C2Is a preset value.
In the embodiment of the present application, the position with larger pixel difference f (x, v) has higher probability of unreliable estimated displacement, and when the brightness i (x) is higher, the noise is estimated
The larger the size of the tube is,
the smaller the value of (a), the smaller the superimposition difference width, and the less likely a high optical noise image is formed.
According to the method and the device, the underexposed image shot in a leftward exposure mode is subjected to pixel blocking processing, the image subjected to pixel blocking and the image with the highest sharpness are subjected to pixel block matching fusion, the matching vector corresponding to the pixel block with the minimum error is superposed with the image with the highest sharpness to reduce noise, noise of a motion part and a highlight part can be effectively processed, and image quality is effectively improved.
Example four
Referring to fig. 3, a schematic structural diagram of the image processing apparatus is shown. The image processing apparatus 400 includes a photographing unit 401, a first acquiring unit 402, a blocking unit 403, a matching unit 404, a second acquiring unit 405, and a processing unit 406. The following description will specifically describe the functions of the program modules of the present embodiment:
a shooting unit 401 for shooting N underexposed images by a left exposure manner;
a first acquisition unit 402 for acquiring a sharpness-highest image among the N underexposed images captured by the capturing unit;
a blocking unit 403, configured to perform blocking processing on the N underexposed images captured by the capturing unit to obtain a first processed image group, where a pixel block size of the first processed image group is T1, and the first processed image group includes the N processed images;
a matching unit 404, configured to match each pixel block of the other images, except the image with the highest sharpness, in the first processed image group obtained by the blocking unit with a corresponding pixel block on the image with the highest sharpness obtained by the obtaining unit, respectively, so as to obtain N-1 first minimum matching errors, where each processed image corresponds to one first minimum matching error;
a second obtaining unit 405, configured to obtain N-1 first matching vectors corresponding to N-1 first minimum matching errors, where each first matching vector corresponds to one first minimum matching error;
and the processing unit 405 is configured to superimpose the N-1 first matching vectors onto the image with the highest sharpness acquired by the acquisition unit, and output the processed image.
In one embodiment, after the step of obtaining N-1 first minimum matching errors, the method further includes: the N under-exposed images are subjected to blocking processing to obtain a second processed image group, the pixel block size of the second processed image group is T2, wherein the size of T1 is twice that of T2, the second processed image group comprises N processed images, each pixel block of the other images except the image with the highest sharpness in the second processed image group is respectively matched with the corresponding pixel block on the image with the highest sharpness to obtain N-1 second minimum matching errors and N-1 second matching vectors, each processed image corresponds to one second minimum matching error, and each second matching vector corresponds to one second minimum matching error; and obtaining the minimum value of the N-1 first minimum matching errors and the N-1 second minimum matching errors, if the minimum value is the second minimum matching error, superposing the N-1 second matching vectors on the image with the highest sharpness by using a weight matrix W, and outputting the processed image, wherein the weight matrix W is a preset matrix model. And if the minimum value is the first minimum matching error, superposing the N-1 first matching vectors to the image with the highest sharpness by using a weight matrix W, and outputting the processed image, wherein the weight matrix W is a preset matrix model.
In one embodiment, matching each pixel block of the other images except the image with the highest sharpness in the first processed image group with the corresponding pixel block on the image with the highest sharpness to obtain N-1 first minimum matching errors, specifically comprising: respectively matching pixel blocks of other images with pixel blocks in a preset range in the image with the highest sharpness to obtain a plurality of matching errors, wherein the other images comprise N-1 processed images, and each processed image corresponds to the plurality of matching errors; obtaining the minimum value of a plurality of matching errors obtained after each processed image is matched and taking the minimum value as a first minimum matching error, wherein each processed image corresponds to one first minimum matching error; n-1 first minimum match errors for N-1 processed images are obtained.
The matching process specifically comprises: ω ═ integral | Ii(x+u)-I0(x) | dx; where ω is the first minimum match error, IiIs an underexposed image, I0Is the sharpest image and u is the first matching vector.
In one embodiment, the superimposing process specifically includes:
wherein the content of the first and second substances,
is a processed image, IiIs an underexposed image, v is a second matching vector, I0Is the sharpest picture, W ═ Wi(x，v)。
In one of the embodiments, the first and second electrodes are,
wherein, the first and the second end of the pipe are connected with each other,
is an estimate of the reliability of the second matching vector, f (x, v) ═ Ii(x+v)-I0(x)|，
The second minimum match error is corrected by the noise model,
is an estimate of the noise by the noise model,
a, b are parameters of the noise model, obtained by sensor measurement statistics of the image processing apparatus, C1，C2Is a preset value.
EXAMPLE five
The embodiment of the application provides an electronic device, which can be a camera, a video recorder and other shooting devices, and the internal structure diagram of the electronic device can be as shown in fig. 4. The electronic device includes a processor, a memory, a network interface, and a database connected by a system bus. Wherein the processor of the electronic device is configured to provide computing and control capabilities. The memory of the electronic equipment comprises a nonvolatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of an operating system and computer programs in the non-volatile storage medium. The network interface of the electronic device is used for connecting and communicating with an external terminal through a network. The computer program is executed by a processor to implement an image processing method.
Those skilled in the art will appreciate that the configuration shown in fig. 4 is a block diagram of only a portion of the configuration associated with the present application, and does not constitute a limitation on the electronic device to which the present application is applied, and a particular electronic device may include more or less components than those shown in the drawings, or combine certain components, or have a different arrangement of components.
The application provides an electronic device comprising a memory and a processor, the memory storing a computer program, the processor implementing the following steps when executing the computer program: shooting N under-exposed images in a left exposure mode to obtain the image with the highest sharpness in the N under-exposed images; the method comprises the steps of carrying out blocking processing on N underexposed images to obtain a first processing image group, wherein the size of a pixel block of the first processing image group is T1, and the first processing image group comprises N processing images; matching each pixel block of other images except the image with the highest sharpness in the first processed image group with the corresponding pixel block on the image with the highest sharpness to obtain N-1 first minimum matching errors, wherein each processed image corresponds to one first minimum matching error; obtaining N-1 first matching vectors corresponding to N-1 first minimum matching errors, wherein each first matching vector corresponds to one first minimum matching error; and superposing the N-1 first matching vectors to the image with the highest sharpness, and outputting the processed image.
In one embodiment, after the step of obtaining N-1 first minimum matching errors, the method further includes: the method comprises the steps of performing blocking processing on N under-exposed images to obtain a second processed image group, wherein the size of a pixel block of the second processed image group is T2, the size of T1 is twice that of T2, the second processed image group comprises N processed images, and each pixel block of other images except the image with the highest sharpness in the second processed image group is matched with the corresponding pixel block of the image with the highest sharpness to obtain N-1 second minimum matching errors and N-1 second matching vectors, each processed image corresponds to one second minimum matching error, and each second matching vector corresponds to one second minimum matching error; and obtaining the minimum value of the N-1 first minimum matching errors and the N-1 second minimum matching errors, if the minimum value is the second minimum matching error, superposing the N-1 second matching vectors on the image with the highest sharpness by using a weight matrix W, and outputting the processed image, wherein the weight matrix W is a preset matrix model. And if the minimum value is the first minimum matching error, superposing the N-1 first matching vectors to the image with the highest sharpness by using a weight matrix W, and outputting the processed image, wherein the weight matrix W is a preset matrix model.
In one embodiment, matching each pixel block of the other images except the image with the highest sharpness in the first processed image group with the corresponding pixel block on the image with the highest sharpness to obtain N-1 first minimum matching errors, specifically comprising: respectively matching pixel blocks of other images with pixel blocks in a preset range in the image with the highest sharpness to obtain a plurality of matching errors, wherein the other images comprise N-1 processed images, and each processed image corresponds to the plurality of matching errors; acquiring the minimum value of a plurality of matching errors obtained after each processed image is matched and taking the minimum value as a first minimum matching error, wherein each processed image corresponds to one first minimum matching error; n-1 first minimum match errors for N-1 processed images are obtained.
The matching process specifically comprises: ω ═ integral | Ii(x+u)-I0(x) | dx; where ω is the first minimum match error, IiIs an underexposed image, I0Is the sharpest image and u is the first matching vector.
In one embodiment, the superimposing process specifically includes:
wherein the content of the first and second substances,
is a processed image, IiIs an underexposed image, v is a second matching vector, I0Is the sharpest picture, W ═ Wi(x，v)。
In one of the embodiments, the first and second electrodes are,
wherein the content of the first and second substances,
is an estimate of the reliability of the second matching vector, f (x, v) ═ Ii(x+v)-I0(x)|，
The second minimum match error is corrected by the noise model,
is an estimate of the noise by the noise model,
a, b are parameters of the noise model, obtained by sensor measurement statistics of the image processing apparatus, C1，C2Is a preset value.
EXAMPLE six
The present embodiment also provides a computer-readable storage medium, such as a flash memory, a hard disk, a multimedia card, a card-type memory (e.g., SD or DX memory, etc.), a Random Access Memory (RAM), a Static Random Access Memory (SRAM), a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), a programmable read-only memory (PROM), a magnetic memory, a magnetic disk, an optical disk, a server, an App application mall, etc., on which a computer program is stored, which when executed by a processor implements corresponding functions. The computer-readable storage medium of the embodiment is used for storing an image processing apparatus based on an electronic device, and when being executed by a processor, the computer-readable storage medium implements the image processing method based on the electronic device of the embodiment.
The above-mentioned serial numbers of the embodiments of the present invention are only for description, and do not represent the advantages and disadvantages of the embodiments.
Through the above description of the embodiments, those skilled in the art will clearly understand that the method of the above embodiments can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware, but in many cases, the former is a better implementation manner.
The above description is only a preferred embodiment of the present invention, and is not intended to limit the scope of the present invention, and all equivalent structures or equivalent processes performed by the present invention or directly or indirectly applied to other related technical fields are also included in the scope of the present invention.
Claims (9)
1. An image processing method applied to an image processing apparatus, the method comprising:
shooting N under-exposed images in a left exposure mode, and acquiring the image with the highest sharpness in the N under-exposed images;
performing blocking processing on the N underexposed images to obtain a first processed image group, wherein the size of a pixel block of the first processed image group is T1, and the first processed image group comprises N processed images;
matching each pixel block of other images except the image with the highest sharpness in the first processed image group with the corresponding pixel block on the image with the highest sharpness to obtain N-1 first minimum matching errors, wherein each processed image corresponds to one first minimum matching error;
obtaining N-1 first matching vectors corresponding to the N-1 first minimum matching errors, wherein each first matching vector corresponds to one first minimum matching error;
superposing the N-1 first matching vectors to the image with the highest sharpness, and outputting a processed image;
wherein the matching of each pixel block of the other images except the image with the highest sharpness in the first processed image group with the corresponding pixel block on the image with the highest sharpness respectively to obtain N-1 first minimum matching errors specifically comprises:
the pixel blocks of the other images are respectively matched with the pixel blocks in the preset range in the image with the highest sharpness to obtain a plurality of matching errors, the other images comprise N-1 processing images, and each processing image corresponds to a plurality of matching errors;
acquiring the minimum value of a plurality of matching errors obtained after each processed image is matched and taking the minimum value as the first minimum matching error, wherein each processed image corresponds to one first minimum matching error;
acquiring N-1 first minimum matching errors of the N-1 processed images;
the matching process specifically comprises the following steps: ω ═ integral | Ii(x+u)-I0(x)|dx；
Where ω is the first minimum match error, IiIs the underexposed image, the0Is the sharpest image, u is the first match vector, and x is a block of pixels in the image.
2. The method of claim 1, wherein the step of obtaining N-1 first minimum match errors is followed by the step of:
performing blocking processing on the N under-exposed images to obtain a second processed image group, wherein the size of a pixel block of the second processed image group is T2, T2 is smaller than T1, the second processed image group comprises N processed images, and each pixel block of the other images except the image with the highest sharpness in the second processed image group is respectively matched with the corresponding pixel block on the image with the highest sharpness to obtain N-1 second minimum matching errors and N-1 second matching vectors, each processed image corresponds to one second minimum matching error, and each second matching vector corresponds to one second minimum matching error;
and obtaining the minimum value of N-1 first minimum matching errors and N-1 second minimum matching errors, if the minimum value is the second minimum matching error, superposing the N-1 second matching vectors to the image with the highest sharpness by using a weight matrix W, and outputting the processed image, wherein the weight matrix W is a preset matrix model.
3. The method of claim 2,
and if the minimum value is a first minimum matching error, superposing the N-1 first matching vectors to the image with the highest sharpness by using a weight matrix W, and outputting the processed image, wherein the weight matrix W is a preset matrix model.
4. The method according to claim 2 or 3, wherein the superimposing the N-1 second matching vectors onto the sharpness-highest image with a weight matrix W and outputting the processed image comprises:
wherein, the
Is the processed image, the IiIs the underexposed image, v is the second matching vector, I0Is the sharpness-highest picture, W ═ Wi(x, v), i is the count of underexposed images.
5. The method of claim 4,
wherein the content of the first and second substances,
is an estimate of the reliability of the second matching vector, f (x, v) ═ Ii(x+v)-I0(x)|，
The second minimum match error is corrected by the noise model,
is an estimate of the noise by the noise model,
a, b are parameters of the noise model, obtained by sensor measurement statistics of the image processing apparatus, C1，C2Is a preset value.
6. The method of claim 2, wherein the size of T1 is twice that of T2.
7. An image processing apparatus, characterized in that the apparatus comprises:
the shooting unit is used for shooting N underexposed images in a leftward exposure mode;
a first acquisition unit configured to acquire a sharpness-highest image of the N underexposed images captured by the capturing unit;
a blocking unit, configured to perform blocking processing on the N underexposed images captured by the capturing unit to obtain a first processed image group, where a pixel block size of the first processed image group is T1, and the first processed image group includes the N processed images;
a matching unit, configured to match each pixel block of the other images, except the image with the highest sharpness, in the first processed image group obtained by the blocking unit with a corresponding pixel block on the image with the highest sharpness obtained by the obtaining unit, respectively, so as to obtain N-1 first minimum matching errors, where each processed image corresponds to one first minimum matching error;
a second obtaining unit, configured to obtain N-1 first matching vectors corresponding to the N-1 first minimum matching errors, where each first matching vector corresponds to one first minimum matching error;
the processing unit is used for superposing the N-1 first matching vectors to the image with the highest sharpness acquired by the acquisition unit and outputting a processed image;
wherein the matching of each pixel block of the other images except the image with the highest sharpness in the first processed image group with the corresponding pixel block on the image with the highest sharpness respectively to obtain N-1 first minimum matching errors specifically comprises:
the pixel blocks of the other images are respectively matched with the pixel blocks in the preset range in the image with the highest sharpness to obtain a plurality of matching errors, the other images comprise N-1 processing images, and each processing image corresponds to a plurality of matching errors;
acquiring the minimum value of a plurality of matching errors obtained after each processed image is matched and taking the minimum value as the first minimum matching error, wherein each processed image corresponds to one first minimum matching error;
acquiring N-1 first minimum matching errors of the N-1 processed images;
the matching process specifically comprises the following steps: ω ═ integral | Ii(x+u)-I0(x)|dx；
Where ω is the first minimum match error, IiIs the underexposed image, the0Is the sharpest image, u is the first match vector, and x is a block of pixels in the image.
8. An electronic device, characterized in that the electronic device comprises:
a memory storing at least one instruction; and
a processor executing instructions stored in the memory to implement the image processing method of any one of claims 1 to 6.
9. A computer-readable storage medium having stored therein at least one instruction, the at least one instruction being executable by a processor in an electronic device to implement the image processing method of any one of claims 1-6.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910955227.4A CN110689502B (en) | 2019-10-09 | 2019-10-09 | Image processing method and related device |
| PCT/CN2020/120048 WO2021068904A1 (en) | 2019-10-09 | 2020-10-09 | Image processing method and related apparatus |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910955227.4A CN110689502B (en) | 2019-10-09 | 2019-10-09 | Image processing method and related device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110689502A CN110689502A (en) | 2020-01-14 |
| CN110689502B true CN110689502B (en) | 2022-06-14 |
Family
ID=69111804
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910955227.4A Active CN110689502B (en) | 2019-10-09 | 2019-10-09 | Image processing method and related device |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN110689502B (en) |
| WO (1) | WO2021068904A1 (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110689502B (en) * | 2019-10-09 | 2022-06-14 | 深圳看到科技有限公司 | Image processing method and related device |
Family Cites Families (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8570386B2 (en) * | 2008-12-31 | 2013-10-29 | Stmicroelectronics S.R.L. | Method of merging images and relative method of generating an output image of enhanced quality |
| CN107230192B (en) * | 2017-05-31 | 2020-07-21 | Oppo广东移动通信有限公司 | Image processing method, image processing device, computer-readable storage medium and mobile terminal |
| CN107277387B (en) * | 2017-07-26 | 2019-11-05 | 维沃移动通信有限公司 | High dynamic range images image pickup method, terminal and computer readable storage medium |
| CN107395991B (en) * | 2017-08-31 | 2020-01-10 | Oppo广东移动通信有限公司 | Image synthesis method, image synthesis device, computer-readable storage medium and computer equipment |
| US10200599B1 (en) * | 2017-09-07 | 2019-02-05 | Qualcomm Incorporated | Image capture setting determination in devices having access to multiple cameras |
| CN107909554B (en) * | 2017-11-16 | 2020-05-19 | 深圳市共进电子股份有限公司 | Image noise reduction method and device, terminal equipment and medium |
| CN109978774B (en) * | 2017-12-27 | 2021-06-18 | 展讯通信（上海）有限公司 | Denoising fusion method and device for multi-frame continuous equal exposure images |
| CN109300097B (en) * | 2018-08-16 | 2022-04-01 | 南京理工大学 | Multi-sequence image fusion method based on self-adaptive blocking |
| CN109729279B (en) * | 2018-12-20 | 2020-11-17 | 华为技术有限公司 | Image shooting method and terminal equipment |
| CN109767408B (en) * | 2018-12-29 | 2021-11-12 | 广州方硅信息技术有限公司 | Image processing method, image processing device, storage medium and computer equipment |
| CN110189285B (en) * | 2019-05-28 | 2021-07-09 | 北京迈格威科技有限公司 | Multi-frame image fusion method and device |
| CN110290289B (en) * | 2019-06-13 | 2021-07-09 | Oppo广东移动通信有限公司 | Image noise reduction method and device, electronic equipment and storage medium |
| CN110166707B (en) * | 2019-06-13 | 2020-09-25 | Oppo广东移动通信有限公司 | Image processing method, image processing apparatus, electronic device, and storage medium |
| CN110689502B (en) * | 2019-10-09 | 2022-06-14 | 深圳看到科技有限公司 | Image processing method and related device |
-
2019
- 2019-10-09 CN CN201910955227.4A patent/CN110689502B/en active Active
-
2020
- 2020-10-09 WO PCT/CN2020/120048 patent/WO2021068904A1/en active Application Filing
Non-Patent Citations (1)
| Title |
|---|
| DENOISING OF SHORT EXPOSURE TRANSMISSION ELECTRON MICROSCOPY IMAGES FOR ULTRASTRUCTURAL ENHANCEMENT;Bajic et al;《2018 IEEE 15TH INTERNATIONAL SYMPOSIUM ON BIOMEDICAL IMAGING》;20180407;全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| WO2021068904A1 (en) | 2021-04-15 |
| CN110689502A (en) | 2020-01-14 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US9998666B2 (en) | Systems and methods for burst image deblurring | |
| JP5213670B2 (en) | Imaging apparatus and blur correction method | |
| US9591237B2 (en) | Automated generation of panning shots | |
| WO2018176925A1 (en) | Hdr image generation method and apparatus | |
| CN108833785B (en) | Fusion method and device of multi-view images, computer equipment and storage medium | |
| KR101429371B1 (en) | Algorithms for estimating precise and relative object distances in a scene | |
| US8422827B2 (en) | Image correction apparatus and image correction method | |
| US20090161982A1 (en) | Restoring images | |
| US10306210B2 (en) | Image processing apparatus and image capturing apparatus | |
| CN111062881A (en) | Image processing method and device, storage medium and electronic equipment | |
| JP4847633B2 (en) | Imaging apparatus and image restoration method | |
| JP4856293B2 (en) | Imaging apparatus and image restoration method | |
| CN113344821B (en) | Image noise reduction method, device, terminal and storage medium | |
| EP4089625A1 (en) | Method and apparatus for generating super night scene image, and electronic device and storage medium | |
| WO2019013217A1 (en) | Correction device, imaging device, control method for correction device, and control program | |
| CN112581415A (en) | Image processing method, image processing device, electronic equipment and storage medium | |
| CN105791659B (en) | Image processing method and electronic device | |
| CN110689502B (en) | Image processing method and related device | |
| CN112106352A (en) | Image processing method and device | |
| CN113538318B (en) | Image processing method, device, terminal equipment and readable storage medium | |
| JP2015076754A5 (en) | ||
| JP2005295302A (en) | Camera image processing device | |
| JP6579934B2 (en) | Image processing apparatus, imaging apparatus, image processing method, program, and storage medium | |
| CN113837979B (en) | Live image synthesis method, device, terminal equipment and readable storage medium | |
| CN111835968B (en) | Image definition restoration method and device and image shooting method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |