WO2022222634A1 - Image processing method, image processing apparatus, electronic device, and storage medium - Google Patents
Image processing method, image processing apparatus, electronic device, and storage medium Download PDFInfo
- Publication number
- WO2022222634A1 WO2022222634A1 PCT/CN2022/079998 CN2022079998W WO2022222634A1 WO 2022222634 A1 WO2022222634 A1 WO 2022222634A1 CN 2022079998 W CN2022079998 W CN 2022079998W WO 2022222634 A1 WO2022222634 A1 WO 2022222634A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- image
- polarizer
- bayer
- scene
- pixels
- Prior art date
Links
- 238000003672 processing method Methods 0.000 title claims abstract description 70
- 238000012545 processing Methods 0.000 title claims abstract description 40
- 230000010287 polarization Effects 0.000 claims abstract description 145
- 238000012937 correction Methods 0.000 claims description 52
- 238000006243 chemical reaction Methods 0.000 claims description 43
- 230000004927 fusion Effects 0.000 claims description 21
- 238000004590 computer program Methods 0.000 claims description 16
- 238000003705 background correction Methods 0.000 claims description 15
- 238000001914 filtration Methods 0.000 claims description 13
- 230000008929 regeneration Effects 0.000 claims description 9
- 238000011069 regeneration method Methods 0.000 claims description 9
- 238000000605 extraction Methods 0.000 claims description 5
- 239000004744 fabric Substances 0.000 claims description 3
- 238000010586 diagram Methods 0.000 description 16
- 238000000034 method Methods 0.000 description 15
- 230000006870 function Effects 0.000 description 7
- 230000003287 optical effect Effects 0.000 description 4
- 238000003491 array Methods 0.000 description 2
- 238000007499 fusion processing Methods 0.000 description 2
- 239000000463 material Substances 0.000 description 2
- 230000001360 synchronised effect Effects 0.000 description 2
- 101100248200 Arabidopsis thaliana RGGB gene Proteins 0.000 description 1
- 230000002146 bilateral effect Effects 0.000 description 1
- 208000001970 congenital sucrase-isomaltase deficiency Diseases 0.000 description 1
- 235000019800 disodium phosphate Nutrition 0.000 description 1
- 238000005315 distribution function Methods 0.000 description 1
- 238000009499 grossing Methods 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000008447 perception Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000012549 training Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/50—Image enhancement or restoration using two or more images, e.g. averaging or subtraction
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B5/00—Optical elements other than lenses
- G02B5/30—Polarising elements
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20212—Image combination
- G06T2207/20221—Image fusion; Image merging
Definitions
- the present application relates to the technical field of image processing, and in particular, to an image processing method, an image processing apparatus, an electronic device, and a storage medium.
- polarization sensors are mainly used in industrial cameras.
- the image data output by the polarization sensor is in a non-Bayer array, while the image data processed by electronic devices such as mobile phones is in a Bayer array. That is to say, the image data output by the polarization sensor in the related art is not Can be processed by electronic devices such as mobile phones.
- Embodiments of the present application provide an image processing method, an image processing apparatus, an electronic device, and a storage medium.
- the image processing method of the embodiment of the present application is used for a polarization sensor.
- the polarization sensor includes a polarizer array and a pixel array.
- the polarizer array includes a plurality of polarizer modules, and each of the polarizer modules includes four polarizer groups.
- the four polarizer groups in each polarizer module are arranged in a Bayer array.
- Each of the polarizer groups includes a plurality of polarizers. The polarization directions of the polarizers in each polarizer group are different from each other, and the color components of the polarizers in each polarizer group are the same.
- the pixel array includes a plurality of pixel units, and the plurality of the pixel units and the plurality of the polarizers are arranged in a one-to-one correspondence.
- the pixel unit is used for receiving light transmitted through the polarizer to generate electrical signals.
- the polarization sensor is used to generate a raw image from the electrical signal.
- the pixels of the original image are arranged in a non-Bayer array.
- the image processing method includes: converting the original image according to the current scene to obtain a Bayer image, the pixels of the Bayer image are arranged in a Bayer array; and outputting a target image according to the Bayer image.
- the image processing apparatus of the embodiment of the present application is used for a polarization sensor.
- the polarization sensor includes a polarizer array and a pixel array.
- the polarizer array includes a plurality of polarizer modules, and each of the polarizer modules includes four polarizer groups.
- the four polarizer groups in each polarizer module are arranged in a Bayer array.
- Each of the polarizer groups includes a plurality of polarizers. The polarization directions of the polarizers in each polarizer group are different from each other, and the color components of the polarizers in each polarizer group are the same.
- the pixel array includes a plurality of pixel units, and the plurality of the pixel units and the plurality of the polarizers are arranged in a one-to-one correspondence.
- the pixel unit is used for receiving light transmitted through the polarizer to generate electrical signals.
- the polarization sensor is used to generate a raw image from the electrical signal.
- the pixels of the original image are arranged in a non-Bayer array.
- the image processing device includes a conversion module and an output module.
- the conversion module is configured to convert the original image according to the current scene to obtain a Bayer image, and the pixels of the Bayer image are arranged in a Bayer array.
- the output module is used for outputting the target image according to the Bayer image.
- the electronic device of the embodiments of the present application includes one or more processors and a memory.
- the memory stores a computer program.
- the computer program is executed by the processor, the steps of the image processing method described in the above embodiments are implemented.
- the computer-readable storage medium of the embodiments of the present application stores a computer program thereon, and when the program is executed by the processor, implements the steps of the image processing method described in the above-mentioned embodiments.
- FIG. 1 is a schematic flowchart of an image processing method according to an embodiment of the present application.
- FIG. 2 is a schematic three-dimensional structure diagram of a polarization sensor of an image processing method according to an embodiment of the present application
- FIG. 3 is a schematic diagram of an image processing apparatus according to an embodiment of the present application.
- FIG. 4 is a schematic diagram of an electronic device according to an embodiment of the present application.
- FIG. 5 is a schematic structural diagram of a polarizer module of a polarization sensor of an image processing method according to an embodiment of the present application
- FIG. 6 is a schematic flowchart of an image processing method according to an embodiment of the present application.
- FIG. 7 is a schematic diagram of a scene of an image processing method according to an embodiment of the present application.
- FIG. 8 is a schematic flowchart of an image processing method according to an embodiment of the present application.
- FIG. 9 is a schematic diagram of a scene of an image processing method according to an embodiment of the present application.
- FIG. 10 is a schematic flowchart of an image processing method according to an embodiment of the present application.
- FIG. 11 is a schematic diagram of an image processing apparatus according to an embodiment of the present application.
- FIG. 12 is a schematic diagram of a scene of an image processing method according to an embodiment of the present application.
- FIG. 13 is a schematic flowchart of an image processing method according to an embodiment of the present application.
- FIG. 14 is a schematic diagram of an image processing apparatus according to an embodiment of the present application.
- 15 is a schematic diagram of a scene of an image processing method according to an embodiment of the present application.
- 16 is a schematic flowchart of an image processing method according to an embodiment of the present application.
- 17 is a schematic flowchart of an image processing method according to an embodiment of the present application.
- FIG. 18 is a schematic diagram of a scene of an image processing method according to an embodiment of the present application.
- FIG. 19 is a schematic flowchart of an image processing method according to an embodiment of the present application.
- 20 is a schematic diagram of a scene of an image processing method according to an embodiment of the present application.
- 21 is a schematic flowchart of an image processing method according to an embodiment of the present application.
- 22 is a schematic diagram of a scene of an image processing method according to an embodiment of the present application.
- FIG. 23 is a schematic diagram of a scene of an image processing method according to an embodiment of the present application.
- first and second are only used for description purposes, and cannot be understood as indicating or implying relative importance or implying the number of indicated technical features. Thus, features defined as “first”, “second” may expressly or implicitly include one or more of said features.
- “plurality” means two or more, unless otherwise expressly and specifically defined.
- Embodiments of the present application provide an image processing method for a polarization sensor.
- the polarization sensor includes a polarizer array and a pixel array, the polarizer array includes a plurality of polarizer modules, each polarizer module includes four polarizer groups, and the four polarizer groups in each polarizer module are arranged in a Bayer array, Each polarizer group includes a plurality of polarizers, the polarization directions of each polarizer in each polarizer group are different from each other, the color components of each polarizer in each polarizer group are the same, and the pixel array includes a plurality of pixel units , a plurality of pixel units and a plurality of polarizers are arranged in one-to-one correspondence, the pixel unit is used to receive the light passing through the polarizer to generate an electrical signal, the polarization sensor is used to generate an original image according to the electrical signal, and the pixels of the original image are in a non-Bayer array Arrangement,
- the current scene includes a bright scene
- converting the original image to obtain a Bayer image includes: in the case that the current scene is a bright scene, extracting the same polarizers in each polarizer group The polarizer of the polarization direction corresponds to the pixels of the original image and the extracted pixels are combined to obtain a Bayer image.
- the current scene includes a common scene
- converting the original image to obtain a Bayer image includes: in the case that the current scene is a common scene, fusing the corresponding polarizers in each polarizer group pixels of the original image and combine the fused pixels to obtain a Bayer image.
- the current scene includes an overall dark and partially bright scene
- converting the original image to obtain a Bayer image includes: when the current scene is an overall dark and partially bright scene, extracting The pixels of the original image corresponding to the polarizers with the same polarization direction in each polarizer group and the extracted pixels are combined to obtain a single-polarization image; the pixels of the original image corresponding to each polarizer in each polarizer group are fused and The fused pixels are combined to obtain an unpolarized image; the single-polarized and unpolarized images are fused to obtain a Bayer image.
- fusing the single polarization image and the unpolarized image to obtain the Bayer image includes: filtering the single polarization image and the unpolarized image; and fusing the filtered single polarization image and the unpolarized image to obtain the Bayer image.
- outputting the target image according to the Bayer image includes: sequentially processing the Bayer image through a dead pixel compensation module, a demosaicing module, a color correction module, a gamma correction module, and a color conversion module to output the target image.
- the current scene includes a bright scene
- converting the original image to obtain a Bayer image includes: in the case that the current scene is a bright scene, using a Bayer image regeneration algorithm to generate a Bayer image for the output of the polarization sensor.
- the original image is converted to obtain a Bayer image.
- outputting the target image according to the Bayer image includes: sequentially passing the Bayer image through a camera serial interface decoder, a black level module, a lens shading correction module, a dead pixel compensation module, a demosaicing module, and a color correction module , gamma correction module, color conversion module to output the target image after processing.
- Embodiments of the present application provide an image processing apparatus for a polarization sensor.
- the polarization sensor includes a polarizer array and a pixel array, the polarizer array includes a plurality of polarizer modules, each polarizer module includes four polarizer groups, and the four polarizer groups in each polarizer module are arranged in a Bayer array, Each polarizer group includes a plurality of polarizers, the polarization directions of each polarizer in each polarizer group are different from each other, the color components of each polarizer in each polarizer group are the same, and the pixel array includes a plurality of pixel units , a plurality of pixel units and a plurality of polarizers are arranged in one-to-one correspondence, the pixel unit is used to receive the light passing through the polarizer to generate an electrical signal, the polarization sensor is used to generate an original image according to the electrical signal, and the pixels of the original image are in a non-Bayer array Arranged
- the current scene includes a bright scene
- the conversion module is further configured to extract the pixels of the original image corresponding to the polarizers with the same polarization direction in each polarizer group when the current scene is a bright scene and combine the extracted pixels to obtain a Bayer image.
- the current scene includes an ordinary scene
- the conversion module is further configured to: in the case that the current scene is an ordinary scene, fuse the pixels of the original image corresponding to each polarizer in each polarizer group Pixels are combined to obtain a Bayer image.
- the current scene includes an overall dark scene and a partially bright scene
- the conversion module includes an extraction unit, a first fusion unit, and a second fusion unit.
- the extraction unit is used to extract the pixels of the original image corresponding to the polarizers with the same polarization direction in each polarizer group and combine the extracted pixels to obtain a single polarized image
- the first fusion unit is used to fuse the pixels of the original image corresponding to each polarizer in each polarizer group and combine the fused pixels to obtain an unpolarized image
- the second fusion unit is used to fuse the single-polarized image and unpolarized images to obtain Bayer images.
- the second fusion unit includes a filtering subunit and a fusion subunit.
- the filtering subunit is used to filter the single polarization image and the unpolarized image; the fusion subunit is used to fuse the filtered single polarization image and the unpolarized image to obtain the Bayer image.
- the output module is further configured to sequentially process the Bayer image through a dead pixel compensation module, a demosaic module, a color correction module, a gamma correction module, and a color conversion module to output the target image.
- the current scene includes a bright scene
- the conversion module is further configured to convert the original image output by the polarization sensor using a Bayer image regeneration algorithm to obtain a Bayer image when the current scene is a bright scene.
- the output module is further used for: passing the Bayer image through the camera serial interface decoder, black level module, lens shading correction module, dead pixel compensation module, demosaicing module, color correction module, gamma module in sequence
- the correction module and the color conversion module output the target image after processing.
- Embodiments of the present application provide an electronic device.
- the polarization sensor includes a polarizer array and a pixel array, the polarizer array includes a plurality of polarizer modules, each polarizer module includes four polarizer groups, and the four polarizer groups in each polarizer module are arranged in a Bayer array, Each polarizer group includes a plurality of polarizers, the polarization directions of each polarizer in each polarizer group are different from each other, the color components of each polarizer in each polarizer group are the same, and the pixel array includes a plurality of pixel units , a plurality of pixel units and a plurality of polarizers are arranged in one-to-one correspondence, the pixel unit is used to receive the light passing through the polarizer to generate an electrical signal, the polarization sensor is used to generate an original image according to the electrical signal, and the pixels of the original image are in a non-Bayer array Arrangement, the electronic device includes one or more processor
- the current scene includes a bright scene
- the processor is further configured to extract the pixels of the original image corresponding to the polarizers with the same polarization direction in each polarizer group when the current scene is a bright scene and combine the extracted pixels to obtain a Bayer image.
- the current scene includes an ordinary scene
- the processor is further configured to fuse the pixels of the original image corresponding to each polarizer in each polarizer group and merge the pixels after fusion when the current scene is an ordinary scene. combined to obtain a Bayer image.
- the current scene includes an overall dark and partially bright scene
- the processor is further configured to: in the case that the current scene is an overall dark and partially bright scene, extract each polarizer group with the same polarization direction
- the pixels of the original image corresponding to the polarizers of each polarizer group are combined and the extracted pixels are combined to obtain a single-polarization image; the pixels of the original image corresponding to each polarizer in each polarizer group are fused and the fused pixels are combined to obtain a Unpolarized images; fusing single and unpolarized images to obtain Bayer images.
- the processor is further configured to: filter the single polarization image and the unpolarized image; and fuse the filtered single polarization image and the unpolarized image to obtain a Bayer image.
- the processor is further configured to: output the target image after processing the Bayer image sequentially through a dead pixel compensation module, a demosaicing module, a color correction module, a gamma correction module, and a color conversion module.
- the processor is further configured to: convert the original image output by the polarization sensor using a Bayer image regeneration algorithm to obtain a Bayer image when the current scene is a bright scene.
- the processor is also used to:
- the Bayer image is sequentially processed by the camera serial interface decoder, black level module, lens shading correction module, dead pixel compensation module, demosaicing module, color correction module, gamma correction module, and color conversion module to output the target image.
- Embodiments of the present application also provide a computer-readable storage medium.
- a computer program is stored on the computer-readable storage medium, and when the program is executed by the processor, the steps of the image processing method in any of the above-mentioned embodiments are implemented.
- Polarization sensor 300 includes polarizer array 310 and pixel array 320 .
- the polarizer array 310 includes a plurality of polarizer modules 311 , and each polarizer module 311 includes four polarizer groups 312 .
- the four polarizer groups 312 in each polarizer module 311 are arranged in a Bayer array.
- Each polarizer group 312 includes a plurality of polarizers 314 .
- the polarization directions of each polarizer 314 in each polarizer group 312 are different from each other, and the color components of each polarizer 314 in each polarizer group 312 are the same.
- the pixel array 320 includes a plurality of pixel units 321 .
- the plurality of pixel units 321 and the plurality of polarizers 314 are arranged in a one-to-one correspondence.
- the pixel unit 321 is used for receiving light transmitted through the polarizer 314 to generate electrical signals.
- the polarization sensor 300 is used to generate a raw image from the electrical signal.
- the pixels of the original image are arranged in a non-Bayer array.
- Image processing methods include:
- the image processing method of the embodiment of the present application can be implemented by the image processing apparatus 100 of the embodiment of the present application.
- an image processing method is used for the polarization sensor 300 .
- Polarization sensor 300 includes polarizer array 310 and pixel array 320 .
- the polarizer array 310 includes a plurality of polarizer modules 311 , and each polarizer module 311 includes four polarizer groups 312 .
- the four polarizer groups 312 in each polarizer module 311 are arranged in a Bayer array.
- Each polarizer group 312 includes a plurality of polarizers 314 .
- the polarization directions of each polarizer 314 in each polarizer group 312 are different from each other, and the color components of each polarizer 314 in each polarizer group 312 are the same.
- the pixel array 320 includes a plurality of pixel units 321 .
- the plurality of pixel units 321 and the plurality of polarizers 314 are arranged in a one-to-one correspondence.
- the pixel unit 321 is used for receiving light transmitted through the polarizer 314 to generate electrical signals.
- the polarization sensor 300 is used to generate a raw image from the electrical signal.
- the pixels of the original image are arranged in a non-Bayer array.
- the image processing apparatus 100 includes a conversion module 10 and an output module 30 .
- the conversion module 10 is configured to convert the original image according to the current scene to obtain a Bayer image, and the pixels of the Bayer image are arranged in a Bayer array.
- the output module 30 is used for outputting the target image according to the
- the image processing method of the embodiment of the present application may be implemented by the electronic device 200 of the embodiment of the present application.
- the electronic device 200 includes one or more processors 201 and a memory 202 .
- the memory 202 stores computer programs. When the computer program is executed by the processor 201, the above steps 01 and 03 are implemented.
- the above-mentioned image processing method, image processing apparatus 100 and electronic device 200 can convert the original image arranged in a non-Bayer array output by the polarization sensor 300 into a Bayer image arranged in a Bayer array, so that the original image output by the polarization sensor 300 can be It is processed by the electronic device 200 such as a mobile phone.
- the polarization sensor 300 can receive external light, and generate an original image according to the received light.
- the pixels of the original image are arranged in a non-Bayer array, and the electronic device 200 such as a mobile phone cannot process the original image whose pixels are arranged in a non-Bayer array, and the pixels of the original image need to be converted into a format arranged in a Bayer array.
- the polarization sensor 300 further includes a microlens array 330 disposed directly above the polarizer array 310.
- the microlens array 330 includes a plurality of microlenses 331, and the microlenses 331, the polarizers 314 and the pixel units 321 are in one-to-one correspondence. Please refer to FIG. 5 .
- FIG. 5 FIG.
- the polarizer module 311 may include two first polarizer groups 3122 , one second polarizer group 3124 and one The third polarizer group 3126, the four polarizer groups 312 are arranged in a Bayer array, wherein the color components of all the polarizers 314 in the first polarizer group 3122 are green (Green, G), and the second polarizer group 312 The color components of all polarizers 314 in 3124 are blue (Blue, B), and the color components of all polarizers 314 in the third polarizer group 3126 are red (Red, R).
- the color components of the polarizers 314 are the same, that is, the polarizers 314 can transmit visible light with the same wavelength range, for example, both can transmit red light with a wavelength range of 625-740 nm, or both can transmit green light with a wavelength range of 492-577 nm , or both can transmit blue light in the wavelength range of 440 to 475 nm.
- Two first polarizer groups 3122, one second polarizer group 3124 and one third polarizer group 3126 are arranged in a 2*2 form, and the two first polarizer groups 3122 are arranged along the first diagonal direction D1 cloth, a second polarizer group 3124 and a third polarizer group 3126 are arranged along the second diagonal direction D2, and the first diagonal direction D1 and the second diagonal direction D2 are perpendicular to each other.
- the arrangement of the polarizer modules 311 is GRBG, and in other examples, the arrangement of the polarizer modules 311 can also be RGGB, BGGR or GBRG. In the embodiment of FIG.
- each polarizer group 312 includes four polarizers 314 , and the polarizers of the four polarizers 314 The directions are different from each other and are respectively a first polarization direction, a second polarization direction, a third polarization direction and a fourth polarization direction.
- a pixel unit 321 is disposed under each polarizer 314 , and when external light passes through the polarizer 314 and irradiates the pixel unit 321 , the pixel unit 321 can generate a corresponding electrical signal.
- the first polarization direction, the second polarization direction, the third polarization direction and the fourth polarization direction are 0°, 45°, 90° and 135°, respectively.
- the polarizers 314 in each polarizer group 312 may also be arranged in the form of 3*3 or 4*4. When the polarizers 314 are arranged in the form of 3*3, Each polarizer group 312 includes 9 polarizers 314 with different polarization directions; when the polarizers 314 are arranged in a 4*4 form, each polarizer group 312 includes 16 polarizers 314 with different polarization directions.
- the current scene may include a bright scene, a normal scene, a partial dark scene and an overall bright scene, and a bright scene.
- the normal scene is a scene where the light intensity is between the first intensity value and the second intensity value
- the bright scene is a scene where the light intensity is between the third intensity value and the fourth intensity value
- the bright scene In the scene where the light intensity is greater than the fifth intensity value, the first intensity value is smaller than the second intensity value, the second intensity value is not greater than the third intensity value, the third intensity value is smaller than the fourth intensity value, and the fourth intensity value is not greater than the fifth intensity value intensity value.
- the bright scene may include a cloudy outdoor, and when the light intensity of the current scene is between 2000 lux-3000 lux (eg, 2500 lux), it may be determined that the current scene is a cloudy outdoor.
- the common scene may include an indoor shopping mall, and when the light intensity of the current scene is between 950 lux-1050 lux (eg, 1000 lux), the current scene may be determined to be an indoor shopping mall.
- the partially dark and overall bright scene may include a night scene of street lights or a scene with lights at night. When the light intensity of the current scene is between 300lux and 500lux (for example, 400lux), it may be determined that the current scene is a scene with lights at night. Lighting scene.
- the bright and bright scene may include a sunny outdoor, and when the light intensity of the current scene is greater than 8000 lux (eg, 9000 lux), it may be determined that the current scene is a sunny outdoor.
- the image processing method before step 01, further includes: determining the current scene.
- the electronic device 200 includes a light sensor, and the current scene can be determined according to the measurement value of the light sensor when shooting.
- machine learning may be performed on different shooting environments in advance based on the training model, so that the current scene is automatically determined according to the shooting environment during shooting. In other embodiments, the current scene can also be set by the user.
- the original image is converted according to the current scene to obtain a Bayer image corresponding to the current scene.
- the pixels of the Bayer image are arranged in a Bayer array, and the Bayer image can be further processed by electronic devices 200 such as mobile phones.
- the electronic device 200 such as a mobile phone may include an image pipeline, and the image pipeline processes the Bayer image, thereby outputting the target image.
- the electronic device 200 is a mobile phone.
- the electronic device 200 may include a tablet computer, a smart watch, or other mobile terminals configured with the polarization sensor 300 .
- the current scene includes a bright scene
- step 01 includes:
- the conversion module 10 is configured to extract the pixels of the original image corresponding to the polarizers 314 having the same polarization direction in each polarizer group 312 and combine the extracted pixels when the current scene is a bright scene. Obtain a Bayer image.
- the image processing method of the above embodiment can be implemented by the electronic device 200 of the embodiment of the present application.
- the processor is configured to extract the pixels of the original image corresponding to the polarizers 314 having the same polarization direction in each polarizer group 312 and combine the extracted pixels to obtain Bayer when the current scene is a bright scene image.
- each polarizer group 312 includes polarizers 314 with four polarization directions of 0°, 45°, 90°, and 135°, and when the current scene is a bright scene, extract each polarizer group The pixels of the original image corresponding to the polarizers 314 in the 0° polarization direction in 312, and the extracted pixels are combined according to the combination between the corresponding polarizer groups 312 to generate a 0° polarized image; each polarizer group is extracted.
- each polarizer group is extracted The pixels of the original image corresponding to the polarizers 314 in the 90° polarization direction in 312, and the extracted pixels are combined according to the combination between the corresponding polarizer groups 312 to generate a 90° polarized image; each polarizer group is extracted The pixels of the original image corresponding to the polarizers 314 in the 135° polarization direction in 312, and the extracted pixels are combined according to the combination between the corresponding polarizer groups 312 to generate a 135° polarized image.
- At least one of a ° polarization image, a 45° polarization image, a 90° polarization image, or a 135° polarization image serves as a Bayer image.
- each frame of the obtained Bayer image is a single-polarization image, and the number of pixels of each frame of the obtained Bayer image is one-fourth of the number of pixels of the original image.
- the current scene includes a common scene
- step 01 includes:
- the conversion module 10 is configured to fuse the pixels of the original image corresponding to each polarizer 314 in each polarizer group 312 and combine the fused pixels to obtain a Bayer image when the current scene is an ordinary scene.
- the image processing method of the above embodiment can be implemented by the electronic device 200 of the embodiment of the present application.
- the processor is configured to fuse the pixels of the original image corresponding to each polarizer 314 in each polarizer group 312 and combine the fused pixels to obtain a Bayer image when the current scene is an ordinary scene.
- an unpolarized Bayer image with a high signal-to-noise ratio can be output without using a complex algorithm to process the original image.
- common scenarios include indoor shopping malls.
- the pixel values of the original images corresponding to all the polarizers 314 in one polarizer group 312 are added or averaged to obtain the corresponding polarizer in the Bayer image.
- the pixel values at the positions of the groups 312 are determined by performing the above operations on all the polarizer groups 312 to determine the pixel values corresponding to the positions of each polarizer group 312 in the Bayer image.
- the Bayer image thus obtained has a higher signal-to-noise ratio.
- the obtained Bayer image is an unpolarized image, and the number of pixels of the obtained Bayer image is a quarter of the number of pixels of the original image.
- the current scene includes an overall dark scene and a partially bright scene.
- Step 01 includes:
- the conversion module 10 includes an extraction unit 13 , a first fusion unit 14 , and a second fusion unit 15 .
- the extraction unit 13 is used for extracting the pixels of the original image corresponding to the polarizers 314 having the same polarization direction in each polarizer group 312 and combining the extracted pixels when the current scene is an overall darker and partially brighter scene, to obtain single-polarization images.
- the first fusion unit 14 is configured to fuse the pixels of the original image corresponding to each polarizer 314 in each polarizer group 312 and combine the fused pixels to obtain an unpolarized image.
- the second fusion unit 15 is used to fuse the single-polarization image and the non-polarization image to obtain a Bayer image.
- the image processing method of the above embodiment can be implemented by the electronic device 200 of the embodiment of the present application.
- the processor is configured to extract the pixels of the original image corresponding to the polarizers 314 having the same polarization direction in each polarizer group 312 and combine the extracted pixels when the current scene is an overall darker and partially brighter scene , to obtain a single-polarization image, and for fusing the pixels of the original image corresponding to each polarizer 314 in each polarizer group 312 and combining the fused pixels to obtain an unpolarized image, and for fusing the single-polarization image and unpolarized images to obtain Bayer images.
- the overall dark and partially bright scene may include a street lamp night scene or a scene with lights at night.
- Each polarizer group 312 includes a plurality of polarizers 314 with different polarization directions, and extracting the original images corresponding to the polarizers 314 with the same polarization direction in each polarizer group 312 can obtain multiple frames of single-polarization images.
- the polarization information is different.
- the polarizer group 312 includes polarizers 314 with four polarization directions of 0°, 45°, 90° and 135°
- 4 frames of single-polarization images can be obtained in step 013, which are the 0° polarized image, the 45° polarized image, and the 45° polarized image.
- step 014 the pixel values of the original images corresponding to all the polarizers 314 in one polarizer group 312 are added or averaged to obtain the pixel values corresponding to the position of the polarizer group 312 in the unpolarized image, and for all polarizers 312
- the plate group 312 performs the above operations to determine the pixel value corresponding to the position of each polarizer plate group 312 in the unpolarized image.
- the unpolarized image has one quarter the number of pixels of the original image.
- step 015 when the current scene is an overall dark and partially bright scene, select a frame of images from multiple frames of single-polarization images, so as to fuse a frame of single-polarization images and a frame of unpolarized images to obtain a frame with Bayer image of polarization information.
- the number of pixels of each frame of single-polarization image is one-fourth of the number of pixels of the original image, and the number of pixels of each frame of unpolarized image is also one-fourth of the number of pixels of the original image, in the current scene, the overall darkening and partial polarization
- the number of pixels in each frame of the Bayer image obtained by fusing the single-polarization image and the non-polarization image is a quarter of the pixel number of the original image, and the Bayer image is a fused polarization image.
- step 015 includes:
- 0151 Filter the single-polarization image and the non-polarization image
- 0153 Fuse the filtered single-polarization image and the unpolarized image to obtain a Bayer image.
- the second fusion unit 15 includes a filtering subunit 151 and a fusion subunit 153 .
- the filtering subunit 151 is used for filtering the single-polarization image and the non-polarization image.
- the fusion subunit 153 is used to fuse the filtered single-polarization image and the non-polarization image to obtain a Bayer image.
- the image processing method of the above embodiment can be implemented by the electronic device 200 of the embodiment of the present application.
- the processor is used for filtering the single-polarization image and the non-polarization image, and for fusing the filtered single-polarization image and the non-polarization image to obtain a Bayer image.
- the single-polarization image and the non-polarization image are respectively subjected to bilateral filtering with the unpolarized image as the guide map, so as to perform a smoothing process on the flat area.
- f represents the weight of each coordinate point in the filtering window, the weight of each coordinate point in the filtering window is fixed, and the closer to the center, the greater the weight.
- g represents the weight of the difference between the pixels in other positions and the center pixel, the larger the difference, the smaller the weight.
- f and g can be any weight distribution function, and they can be the same or different.
- f and g are both Gaussian functions, and the Gaussian function is, for example, a, b and c are adjustment coefficients.
- a Fusion algorithm is used to fuse the filtered single-polarization image and the non-polarization image to obtain a Bayer image.
- G_smooth represents the green pixel unit in the filtered single-polarization image, including five pixel points G1, G2, G3, G4, and G5.
- G'_smooth represents the green pixel unit in the filtered unpolarized image, including five pixel points G'1, G'2, G'3, G'4, and G'5.
- the fusion process of the red pixel unit and the blue pixel unit of the filtered single-polarization image and the non-polarization image is basically the same as the fusion process of the green pixel unit. In order to avoid redundancy, details are not repeated here.
- step 03 includes:
- BPC bad pixel correction
- DM demosaic
- CC color correction
- Gamma gamma
- CV color conversion
- the output module 30 is configured to sequentially process the Bayer image through a dead pixel compensation module, a demosaic module, a color correction module, a gamma correction module, and a color conversion module to output the target image.
- the image processing method of the above embodiment can be implemented by the electronic device 200 of the embodiment of the present application.
- the processor is configured to sequentially process the Bayer image through a dead pixel compensation module, a demosaic module, a color correction module, a gamma correction module, and a color conversion module to output the target image.
- the Bayer image enters the image pipeline, and after being processed by the image pipeline, it is converted into the target image and output.
- the target image includes a YUV image.
- the original image (Quadbayer polarized RAW) generated by the polarization sensor (Sensor) 300 is sequentially passed through a camera serial interface decoder (Camera Serial Interface Decoder, CSID), a black level (Optical black, OB) module, After the lens shading correction (LSC) module, it enters the double-rate synchronous dynamic random access memory (Double Data Rate, DDR), and then converts it into a Bayer image in the Binning Algorithm module, and then passes through the dead pixel compensation module in turn.
- the mosaic module, color correction module, gamma correction module, and color conversion module process and output the target image.
- the camera serial interface decoder is used to decode and identify the original image.
- the circuit of the polarization sensor 300 itself will have dark current, so when there is no light irradiation, the pixel unit 321 also has a certain output voltage. Therefore, the influence of the dark current needs to be subtracted, that is, black level correction is performed.
- Lens shading correction is to solve the problem of shadows appearing around the lens due to the optical characteristics of the lens, that is, the lens has uneven optical refraction.
- the double-rate synchronous dynamic random access memory is used to store the original image after successively passing through the camera serial interface decoder, black level module and lens shading correction module.
- the Binning Algorithm module can retrieve the original image from the DDP and execute steps 011, 012, 013, 014, 015, 0151, and 0153 to convert the original image into a Bayer image.
- Bad pixel compensation is bad pixel compensation, which refers to bad pixel correction.
- the demosaicing module can interpolate the Bayer image of the Bayer array into an RGB image, wherein the RGB image can be three frames, namely, the R image, the G image and the B image obtained after interpolation.
- the color saturation of the image after color correction is more obvious, which is more in line with the perception of the human eye.
- Gamma correction can make the image look more in line with the characteristics of the human eye.
- the color conversion module can convert RGB images to YUV images.
- the schematic flowchart includes step 011 and step 031; in some embodiments, the schematic flowchart may include step 012 and step 031; in some embodiments, the schematic flowchart may Including step 013, step 014, step 015 and step 031; in some embodiments, the schematic flowchart may include step 013, step 014, step 0151, step 0153 and step 031, which is not limited herein.
- the current scene includes a bright scene
- step 01 includes:
- the conversion module 10 is configured to convert the original image output by the polarization sensor 300 by using a Bayer image regeneration algorithm to obtain the Bayer image when the current scene is a bright scene.
- the image processing method of the above embodiment can be implemented by the electronic device 200 of the embodiment of the present application.
- the processor is configured to convert the original image output by the polarization sensor 300 by using a Bayer image regeneration algorithm to obtain the Bayer image when the current scene is a bright scene.
- the bright scene may include a sunny day outdoors.
- the Remosaic algorithm can be used to convert the original image into a Bayer image with a higher resolution with a specific polarization direction.
- the Remosaic algorithm can be used to convert the original image into a Bayer image with higher resolution without polarization information.
- the obtained Bayer image is a specific polarized image or a full-size unpolarized image, and the obtained Bayer image has the same number of pixels as the original image, so the Bayer image has a higher resolution.
- step 03 includes:
- the Bayer image is sequentially processed by the camera serial interface decoder, black level module, lens shading correction module, dead pixel compensation module, demosaicing module, color correction module, gamma correction module, and color conversion module, and then output the target image .
- the output module 30 is used to sequentially pass the Bayer image through the camera serial interface decoder, the black level module, the lens shading correction module, the dead pixel compensation module, the demosaicing module, the color correction module, the gamma correction module, and the color conversion module.
- the module outputs the target image after processing.
- the image processing method of the above embodiment can be implemented by the electronic device 200 of the embodiment of the present application.
- the processor is configured to sequentially process the Bayer image through a camera serial interface decoder, a black level module, a lens shading correction module, a dead pixel compensation module, a demosaicing module, a color correction module, a gamma correction module, and a color conversion module Then output the target image.
- the Bayer image enters the image pipeline, and after being processed by the image pipeline, it is converted into the target image and output.
- the original image (Quadbayer polarized RAW) generated by the polarization sensor (Sensor) 300 is converted into a Bayer image of a Bayer array after passing through the Fullsize Algorithm module, and the Bayer image sequentially passes through the camera string.
- Line interface decoder, black level module, lens shading correction module, dead pixel compensation module, demosaicing module, color correction module, gamma correction module, and color conversion module output the target image after processing.
- the Fullsize Algorithm module is used to perform step 016 to convert the original image to a Bayer image. Referring to FIG.
- the original image (Quadbayer polarized RAW) generated by the polarization sensor (Sensor) 300 is converted into a Bayer image of the Bayer array after passing through the Binning Algorithm module, and the Bayer image is sequentially decoded by the camera serial interface
- the output target image is processed by the camera, black level module, lens shading correction module, dead pixel compensation module, demosaicing module, color correction module, gamma correction module, and color conversion module.
- black level module, lens shading correction module, dead pixel compensation module, demosaicing module, color correction module, gamma correction module, and color conversion module reference may be made to the above embodiments.
- the schematic flowchart includes steps 016 and 033; in some embodiments, the schematic flowchart may include steps 011 and 033; in some embodiments, the schematic flowchart may Including step 012 and step 033; in some embodiments, the schematic flow diagram may include step 013, step 014, step 015 and step 033; Step 0153 and Step 033 are not limited here.
- the computer-readable storage medium of the embodiments of the present application stores a computer program thereon, and when the program is executed by a processor, the steps of the image processing method of any of the above-mentioned embodiments are implemented.
- a computer program includes computer program code.
- the computer program code may be in source code form, object code form, an executable file or some intermediate form, or the like.
- Computer-readable storage media may include: any entity or device capable of carrying computer program codes, recording media, U disk, removable hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random storage Access memory (RAM, Random Access Memory), and software distribution media, etc.
- the processor can be a central processing unit, or other general-purpose processors, digital signal processors (Digital Signal Processors, DSPs), application specific integrated circuits (Application Specific Integrated Circuits, ASICs), ready-made programmable gate arrays (Field-Programmable Gate arrays) Array, FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.
- DSPs Digital Signal Processors
- ASICs Application Specific Integrated Circuits
- FPGA Field-Programmable Gate arrays
- FPGA Field-Programmable Gate arrays
- any description of a process or method in the flowcharts or otherwise described herein may be understood to represent a module, segment or portion of code comprising one or more executable instructions for implementing a specified logical function or step of the process , and the scope of the preferred embodiments of the present application includes alternative implementations in which the functions may be performed out of the order shown or discussed, including performing the functions substantially concurrently or in the reverse order depending upon the functions involved, which should It is understood by those skilled in the art to which the embodiments of the present application belong.
Landscapes
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Optics & Photonics (AREA)
- Color Television Image Signal Generators (AREA)
- Image Processing (AREA)
- Studio Devices (AREA)
Abstract
An image processing method, an image processing apparatus (100), an electronic device (200), and a storage medium. The image processing method comprises: converting an original image according to a current scene so as to obtain a Bayer image, the pixels of the Bayer image being arranged in a Bayer array; and outputting a target image according to the Bayer image. An original image outputted by a polarization sensor can be processed by an electronic device such as a mobile phone
Description
优先权信息priority information
本申请请求2021年4月23日向中国国家知识产权局提交的、专利申请号为202110442129.8的专利申请的优先权和权益,并且通过参照将其全文并入此处。This application claims the priority and rights of patent application No. 202110442129.8 filed with the State Intellectual Property Office of China on April 23, 2021, and is hereby incorporated by reference in its entirety.
本申请涉及图像处理技术领域,特别涉及一种图像处理方法、图像处理装置、电子设备及存储介质。The present application relates to the technical field of image processing, and in particular, to an image processing method, an image processing apparatus, an electronic device, and a storage medium.
目前,偏振传感器主要用于工业相机,偏振传感器输出的图像数据呈非拜耳阵列,而手机等电子设备处理的图像数据呈拜耳阵列,也即是说,相关技术中的偏振传感器输出的图像数据不能够被手机等电子设备处理。At present, polarization sensors are mainly used in industrial cameras. The image data output by the polarization sensor is in a non-Bayer array, while the image data processed by electronic devices such as mobile phones is in a Bayer array. That is to say, the image data output by the polarization sensor in the related art is not Can be processed by electronic devices such as mobile phones.
发明内容SUMMARY OF THE INVENTION
本申请的实施方式提供了一种图像处理方法、图像处理装置、电子设备及存储介质。Embodiments of the present application provide an image processing method, an image processing apparatus, an electronic device, and a storage medium.
本申请实施方式的图像处理方法用于偏振传感器。所述偏振传感器包括偏振片阵列和像素阵列。所述偏振片阵列包括多个偏振片模块,每个所述偏振片模块包括四个偏振片组。每个所述偏振片模块中的四个所述偏振片组呈拜耳阵列排布。每个所述偏振片组包括多个偏振片。每个所述偏振片组中的各个所述偏振片的偏振方向互不相同,每个所述偏振片组中的各个所述偏振片的颜色分量相同。所述像素阵列包括多个像素单元,多个所述像素单元和多个所述偏振片一一对应设置。所述像素单元用于接收透过所述偏振片的光线以生成电信号。所述偏振传感器用于根据所述电信号生成原始图像。所述原始图像的像素呈非拜耳阵列排布。所述图像处理方法包括:根据当前场景,对所述原始图像进行转换以获得拜耳图像,所述拜耳图像的像素呈拜耳阵列排布;根据所述拜耳图像输出目标图像。The image processing method of the embodiment of the present application is used for a polarization sensor. The polarization sensor includes a polarizer array and a pixel array. The polarizer array includes a plurality of polarizer modules, and each of the polarizer modules includes four polarizer groups. The four polarizer groups in each polarizer module are arranged in a Bayer array. Each of the polarizer groups includes a plurality of polarizers. The polarization directions of the polarizers in each polarizer group are different from each other, and the color components of the polarizers in each polarizer group are the same. The pixel array includes a plurality of pixel units, and the plurality of the pixel units and the plurality of the polarizers are arranged in a one-to-one correspondence. The pixel unit is used for receiving light transmitted through the polarizer to generate electrical signals. The polarization sensor is used to generate a raw image from the electrical signal. The pixels of the original image are arranged in a non-Bayer array. The image processing method includes: converting the original image according to the current scene to obtain a Bayer image, the pixels of the Bayer image are arranged in a Bayer array; and outputting a target image according to the Bayer image.
本申请实施方式的图像处理装置用于偏振传感器。所述偏振传感器包括偏振片阵列和像素阵列。所述偏振片阵列包括多个偏振片模块,每个所述偏振片模块包括四个偏振片组。每个所述偏振片模块中的四个所述偏振片组呈拜耳阵列排布。每个所述偏振片组包括多个偏振片。每个所述偏振片组中的各个所述偏振片的偏振方向互不相同,每个所述偏振片组中的各个所述偏振片的颜色分量相同。所述像素阵列包括多个像素单元,多个所述像素单元和多个所述偏振片一一对应设置。所述像素单元用于接收透过所述偏振片的光线以生成电信号。所述偏振传感器用于根据所述电信号生成原始图像。所述原始图像的像素呈非拜耳阵列排布。所述图像处理装置包括转换模块和输出模块。转换模块用于根据当前场景,对所述原始图像进行转换以获得拜耳图像,所述拜耳图像的像素呈拜耳阵列排布。输出模块用于根据所述拜耳图像输出目标图像。The image processing apparatus of the embodiment of the present application is used for a polarization sensor. The polarization sensor includes a polarizer array and a pixel array. The polarizer array includes a plurality of polarizer modules, and each of the polarizer modules includes four polarizer groups. The four polarizer groups in each polarizer module are arranged in a Bayer array. Each of the polarizer groups includes a plurality of polarizers. The polarization directions of the polarizers in each polarizer group are different from each other, and the color components of the polarizers in each polarizer group are the same. The pixel array includes a plurality of pixel units, and the plurality of the pixel units and the plurality of the polarizers are arranged in a one-to-one correspondence. The pixel unit is used for receiving light transmitted through the polarizer to generate electrical signals. The polarization sensor is used to generate a raw image from the electrical signal. The pixels of the original image are arranged in a non-Bayer array. The image processing device includes a conversion module and an output module. The conversion module is configured to convert the original image according to the current scene to obtain a Bayer image, and the pixels of the Bayer image are arranged in a Bayer array. The output module is used for outputting the target image according to the Bayer image.
本申请实施方式的电子设备包括一个或多个处理器和存储器。所述存储器存储有计算机程序。所述计算机程序被所述处理器执行的情况下,实现上述实施方式所述的图像处理方法的步骤。The electronic device of the embodiments of the present application includes one or more processors and a memory. The memory stores a computer program. When the computer program is executed by the processor, the steps of the image processing method described in the above embodiments are implemented.
本申请实施方式的计算机可读存储介质,其上存储有计算机程序,所述程序被处理器执行的情况下,实现上述实施方式所述的图像处理方法的步骤。The computer-readable storage medium of the embodiments of the present application stores a computer program thereon, and when the program is executed by the processor, implements the steps of the image processing method described in the above-mentioned embodiments.
本申请的附加方面和优点将在下面的描述中部分给出,部分将从下面的描述中变得明显,或通过本申请的实践了解到。Additional aspects and advantages of the present application will be set forth, in part, from the following description, and in part will become apparent from the following description, or may be learned by practice of the present application.
本申请的上述和/或附加的方面和优点从结合下面附图对实施方式的描述中将变得明显和容易理解,其中:The above and/or additional aspects and advantages of the present application will become apparent and readily understood from the following description of embodiments taken in conjunction with the accompanying drawings, wherein:
图1是本申请实施方式的图像处理方法的流程示意图;1 is a schematic flowchart of an image processing method according to an embodiment of the present application;
图2是本申请实施方式的图像处理方法的偏振传感器的立体结构示意图;2 is a schematic three-dimensional structure diagram of a polarization sensor of an image processing method according to an embodiment of the present application;
图3是本申请实施方式的图像处理装置的示意图;3 is a schematic diagram of an image processing apparatus according to an embodiment of the present application;
图4是本申请实施方式的电子设备的示意图;4 is a schematic diagram of an electronic device according to an embodiment of the present application;
图5是本申请实施方式的图像处理方法的偏振传感器的偏振片模块的结构示意图;5 is a schematic structural diagram of a polarizer module of a polarization sensor of an image processing method according to an embodiment of the present application;
图6是本申请实施方式的图像处理方法的流程示意图;6 is a schematic flowchart of an image processing method according to an embodiment of the present application;
图7是本申请实施方式的图像处理方法的场景示意图;7 is a schematic diagram of a scene of an image processing method according to an embodiment of the present application;
图8是本申请实施方式的图像处理方法的流程示意图;8 is a schematic flowchart of an image processing method according to an embodiment of the present application;
图9是本申请实施方式的图像处理方法的场景示意图;9 is a schematic diagram of a scene of an image processing method according to an embodiment of the present application;
图10是本申请实施方式的图像处理方法的流程示意图;10 is a schematic flowchart of an image processing method according to an embodiment of the present application;
图11是本申请实施方式的图像处理装置的示意图;11 is a schematic diagram of an image processing apparatus according to an embodiment of the present application;
图12是本申请实施方式的图像处理方法的场景示意图;12 is a schematic diagram of a scene of an image processing method according to an embodiment of the present application;
图13是本申请实施方式的图像处理方法的流程示意图;13 is a schematic flowchart of an image processing method according to an embodiment of the present application;
图14是本申请实施方式的图像处理装置的示意图;14 is a schematic diagram of an image processing apparatus according to an embodiment of the present application;
图15是本申请实施方式的图像处理方法的场景示意图;15 is a schematic diagram of a scene of an image processing method according to an embodiment of the present application;
图16本申请实施方式的图像处理方法的流程示意图;16 is a schematic flowchart of an image processing method according to an embodiment of the present application;
图17是本申请实施方式的图像处理方法的流程示意图;17 is a schematic flowchart of an image processing method according to an embodiment of the present application;
图18是本申请实施方式的图像处理方法的场景示意图;18 is a schematic diagram of a scene of an image processing method according to an embodiment of the present application;
图19是本申请实施方式的图像处理方法的流程示意图;19 is a schematic flowchart of an image processing method according to an embodiment of the present application;
图20是本申请实施方式的图像处理方法的场景示意图;20 is a schematic diagram of a scene of an image processing method according to an embodiment of the present application;
图21是本申请实施方式的图像处理方法的流程示意图;21 is a schematic flowchart of an image processing method according to an embodiment of the present application;
图22是本申请实施方式的图像处理方法的场景示意图;22 is a schematic diagram of a scene of an image processing method according to an embodiment of the present application;
图23是本申请实施方式的图像处理方法的场景示意图。FIG. 23 is a schematic diagram of a scene of an image processing method according to an embodiment of the present application.
下面详细描述本申请的实施方式,所述实施方式的实施方式在附图中示出,其中自始至终相同或类似的标号表示相同或类似的元件或具有相同或类似功能的元件。下面通过参考附图描述的实施方式是示例性的,仅用于解释本申请,而不能理解为对本申请的限制。The following describes in detail the embodiments of the present application, the embodiments of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals refer to the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary, only used to explain the present application, and should not be construed as a limitation on the present application.
在本申请的实施方式的描述中,术语“第一”、“第二”仅用于描述目的,而不能理解为指示或暗示相对重要性或者隐含指明所指示的技术特征的数量。由此,限定有“第一”、“第二”的特征可以明示或者隐含地包括一个或者更多个所述特征。在本申请的实施方式的描述中,“多个”的含义是两个或两个以上,除非另有明确具体的限定。In the description of the embodiments of the present application, the terms "first" and "second" are only used for description purposes, and cannot be understood as indicating or implying relative importance or implying the number of indicated technical features. Thus, features defined as "first", "second" may expressly or implicitly include one or more of said features. In the description of the embodiments of the present application, "plurality" means two or more, unless otherwise expressly and specifically defined.
本申请实施方式提供一种用于偏振传感器的图像处理方法。偏振传感器包括偏振片阵列和像素阵列,偏振片阵列包括多个偏振片模块,每个偏振片模块包括四个偏振片组,每个偏振片模块中的四个偏振片组呈拜耳阵列排布,每个偏振片组包括多个偏振片,每个偏振片组中的各个偏振片的偏振方向互不相同,每个偏振片组中的各个偏振片的颜色分量相同,像素阵列包括多个像素单元,多个像素单元和多个偏振片一一对应设置,像素单元用于接收透过偏振片的光线以生成电信号,偏振传感器用于根据电信号生成原始图像,原始图像的像素呈非拜耳阵列排布,图像处理方法包括:根据当前场景,对原始图像进行转 换以获得拜耳图像,拜耳图像的像素呈拜耳阵列排布;根据拜耳图像输出目标图像。Embodiments of the present application provide an image processing method for a polarization sensor. The polarization sensor includes a polarizer array and a pixel array, the polarizer array includes a plurality of polarizer modules, each polarizer module includes four polarizer groups, and the four polarizer groups in each polarizer module are arranged in a Bayer array, Each polarizer group includes a plurality of polarizers, the polarization directions of each polarizer in each polarizer group are different from each other, the color components of each polarizer in each polarizer group are the same, and the pixel array includes a plurality of pixel units , a plurality of pixel units and a plurality of polarizers are arranged in one-to-one correspondence, the pixel unit is used to receive the light passing through the polarizer to generate an electrical signal, the polarization sensor is used to generate an original image according to the electrical signal, and the pixels of the original image are in a non-Bayer array Arrangement, the image processing method includes: converting the original image according to the current scene to obtain a Bayer image, the pixels of the Bayer image are arranged in a Bayer array; and outputting a target image according to the Bayer image.
在某些实施方式中,当前场景包括偏亮场景,根据当前场景,对原始图像进行转换以获得拜耳图像,包括:在当前场景为偏亮场景的情况下,抽取每个偏振片组中具有相同偏振方向的偏振片对应的原始图像的像素并对抽取的像素进行组合,以获得拜耳图像。In some embodiments, the current scene includes a bright scene, and according to the current scene, converting the original image to obtain a Bayer image includes: in the case that the current scene is a bright scene, extracting the same polarizers in each polarizer group The polarizer of the polarization direction corresponds to the pixels of the original image and the extracted pixels are combined to obtain a Bayer image.
在某些实施方式中,当前场景包括普通场景,根据当前场景,对原始图像进行转换以获得拜耳图像,包括:在当前场景为普通场景的情况下,融合每个偏振片组中各个偏振片对应的原始图像的像素并对融合后的像素进行组合,以获得拜耳图像。In some embodiments, the current scene includes a common scene, and according to the current scene, converting the original image to obtain a Bayer image includes: in the case that the current scene is a common scene, fusing the corresponding polarizers in each polarizer group pixels of the original image and combine the fused pixels to obtain a Bayer image.
在某些实施方式中,当前场景包括整体偏暗局部偏亮场景,根据当前场景,对原始图像进行转换以获得拜耳图像,包括:在当前场景为整体偏暗局部偏亮场景的情况下,抽取每个偏振片组中具有相同偏振方向的偏振片对应的原始图像的像素并对抽取的像素进行组合,以获得单偏振图像;融合每个偏振片组中各个偏振片对应的原始图像的像素并对融合后的像素进行组合,以获得无偏振图像;融合单偏振图像和无偏振图像以获得拜耳图像。In some embodiments, the current scene includes an overall dark and partially bright scene, and according to the current scene, converting the original image to obtain a Bayer image includes: when the current scene is an overall dark and partially bright scene, extracting The pixels of the original image corresponding to the polarizers with the same polarization direction in each polarizer group and the extracted pixels are combined to obtain a single-polarization image; the pixels of the original image corresponding to each polarizer in each polarizer group are fused and The fused pixels are combined to obtain an unpolarized image; the single-polarized and unpolarized images are fused to obtain a Bayer image.
在某些实施方式中,融合单偏振图像和无偏振图像以获得拜耳图像,包括:对单偏振图像和无偏振图像进行滤波;融合滤波后的单偏振图像和无偏振图像以获得拜耳图像。In certain embodiments, fusing the single polarization image and the unpolarized image to obtain the Bayer image includes: filtering the single polarization image and the unpolarized image; and fusing the filtered single polarization image and the unpolarized image to obtain the Bayer image.
在某些实施方式中,根据拜耳图像输出目标图像,包括:将拜耳图像依次经过坏点补偿模块、去马赛克模块、色彩校正模块、伽玛校正模块、色彩转换模块处理后输出目标图像。In some embodiments, outputting the target image according to the Bayer image includes: sequentially processing the Bayer image through a dead pixel compensation module, a demosaicing module, a color correction module, a gamma correction module, and a color conversion module to output the target image.
在某些实施方式中,当前场景包括敞亮场景,根据当前场景,对原始图像进行转换以获得拜耳图像,包括:在当前场景为敞亮场景的情况下,采用拜耳图像再生成算法对偏振传感器输出的原始图像进行转换以获得拜耳图像。In some embodiments, the current scene includes a bright scene, and according to the current scene, converting the original image to obtain a Bayer image includes: in the case that the current scene is a bright scene, using a Bayer image regeneration algorithm to generate a Bayer image for the output of the polarization sensor. The original image is converted to obtain a Bayer image.
在某些实施方式中,根据拜耳图像输出目标图像,包括:将拜耳图像依次经过摄像机串行接口解码器、黑电平模块、镜头阴影校正模块、坏点补偿模块、去马赛克模块、色彩校正模块、伽玛校正模块、色彩转换模块处理后输出目标图像。In some embodiments, outputting the target image according to the Bayer image includes: sequentially passing the Bayer image through a camera serial interface decoder, a black level module, a lens shading correction module, a dead pixel compensation module, a demosaicing module, and a color correction module , gamma correction module, color conversion module to output the target image after processing.
本申请实施方式提供一种用于偏振传感器的图像处理装置。偏振传感器包括偏振片阵列和像素阵列,偏振片阵列包括多个偏振片模块,每个偏振片模块包括四个偏振片组,每个偏振片模块中的四个偏振片组呈拜耳阵列排布,每个偏振片组包括多个偏振片,每个偏振片组中的各个偏振片的偏振方向互不相同,每个偏振片组中的各个偏振片的颜色分量相同,像素阵列包括多个像素单元,多个像素单元和多个偏振片一一对应设置,像素单元用于接收透过偏振片的光线以生成电信号,偏振传感器用于根据电信号生成原始图像,原始图像的像素呈非拜耳阵列排布,图像处理装置包括转换模块及输出模块。转换模块用于根据当前场景,对原始图像进行转换以获得拜耳图像,拜耳图像的像素呈拜耳阵列排布;输出模块用于根据拜耳图像输出目标图像。Embodiments of the present application provide an image processing apparatus for a polarization sensor. The polarization sensor includes a polarizer array and a pixel array, the polarizer array includes a plurality of polarizer modules, each polarizer module includes four polarizer groups, and the four polarizer groups in each polarizer module are arranged in a Bayer array, Each polarizer group includes a plurality of polarizers, the polarization directions of each polarizer in each polarizer group are different from each other, the color components of each polarizer in each polarizer group are the same, and the pixel array includes a plurality of pixel units , a plurality of pixel units and a plurality of polarizers are arranged in one-to-one correspondence, the pixel unit is used to receive the light passing through the polarizer to generate an electrical signal, the polarization sensor is used to generate an original image according to the electrical signal, and the pixels of the original image are in a non-Bayer array Arranged, the image processing device includes a conversion module and an output module. The conversion module is used to convert the original image according to the current scene to obtain a Bayer image, and the pixels of the Bayer image are arranged in a Bayer array; the output module is used to output the target image according to the Bayer image.
在某些实施方式中,当前场景包括偏亮场景,转换模块还用于在当前场景为偏亮场景的情况下,抽取每个偏振片组中具有相同偏振方向的偏振片对应的原始图像的像素并对抽取的像素进行组合,以获得拜耳图像。In some embodiments, the current scene includes a bright scene, and the conversion module is further configured to extract the pixels of the original image corresponding to the polarizers with the same polarization direction in each polarizer group when the current scene is a bright scene and combine the extracted pixels to obtain a Bayer image.
在某些实施方式中,当前场景包括普通场景,转换模块还用于:在当前场景为普通场景的情况下,融合每个偏振片组中各个偏振片对应的原始图像的像素并对融合后的像素进行组合,以获得拜耳图像。In some embodiments, the current scene includes an ordinary scene, and the conversion module is further configured to: in the case that the current scene is an ordinary scene, fuse the pixels of the original image corresponding to each polarizer in each polarizer group Pixels are combined to obtain a Bayer image.
在某些实施方式中,当前场景包括整体偏暗局部偏亮场景,转换模块包括抽取单元、第一融合单元及第二融合单元。抽取单元用于在当前场景为整体偏暗局部偏亮场景的情况下,抽取每个偏振片组中具有相同偏振方向的偏振片对应的原始图像的像素并对抽取的像素进行组合,以获得单偏振图像;第一融合单元用于融合每个偏振片组中各个偏振片对应的原始图像的像素并对融合后的像素进行组合,以获得无偏振图像;第二融合单元用于融合单偏振图像和无偏振图像以获得拜耳图像。In some embodiments, the current scene includes an overall dark scene and a partially bright scene, and the conversion module includes an extraction unit, a first fusion unit, and a second fusion unit. The extraction unit is used to extract the pixels of the original image corresponding to the polarizers with the same polarization direction in each polarizer group and combine the extracted pixels to obtain a single polarized image; the first fusion unit is used to fuse the pixels of the original image corresponding to each polarizer in each polarizer group and combine the fused pixels to obtain an unpolarized image; the second fusion unit is used to fuse the single-polarized image and unpolarized images to obtain Bayer images.
在某些实施方式中,第二融合单元包括滤波子单元及融合子单元。滤波子单元用于对单偏振图像和 无偏振图像进行滤波;融合子单元用于融合滤波后的单偏振图像和无偏振图像以获得拜耳图像。In some embodiments, the second fusion unit includes a filtering subunit and a fusion subunit. The filtering subunit is used to filter the single polarization image and the unpolarized image; the fusion subunit is used to fuse the filtered single polarization image and the unpolarized image to obtain the Bayer image.
在某些实施方式中,输出模块还用于将拜耳图像依次经过坏点补偿模块、去马赛克模块、色彩校正模块、伽玛校正模块、色彩转换模块处理后输出目标图像。In some embodiments, the output module is further configured to sequentially process the Bayer image through a dead pixel compensation module, a demosaic module, a color correction module, a gamma correction module, and a color conversion module to output the target image.
在某些实施方式中,当前场景包括敞亮场景,转换模块还用于在当前场景为敞亮场景的情况下,采用拜耳图像再生成算法对偏振传感器输出的原始图像进行转换以获得拜耳图像。In some embodiments, the current scene includes a bright scene, and the conversion module is further configured to convert the original image output by the polarization sensor using a Bayer image regeneration algorithm to obtain a Bayer image when the current scene is a bright scene.
在某些实施方式中,输出模块还用于:将拜耳图像依次经过摄像机串行接口解码器、黑电平模块、镜头阴影校正模块、坏点补偿模块、去马赛克模块、色彩校正模块、伽玛校正模块、色彩转换模块处理后输出目标图像。In some embodiments, the output module is further used for: passing the Bayer image through the camera serial interface decoder, black level module, lens shading correction module, dead pixel compensation module, demosaicing module, color correction module, gamma module in sequence The correction module and the color conversion module output the target image after processing.
本申请实施方式提供一种电子设备。偏振传感器包括偏振片阵列和像素阵列,偏振片阵列包括多个偏振片模块,每个偏振片模块包括四个偏振片组,每个偏振片模块中的四个偏振片组呈拜耳阵列排布,每个偏振片组包括多个偏振片,每个偏振片组中的各个偏振片的偏振方向互不相同,每个偏振片组中的各个偏振片的颜色分量相同,像素阵列包括多个像素单元,多个像素单元和多个偏振片一一对应设置,像素单元用于接收透过偏振片的光线以生成电信号,偏振传感器用于根据电信号生成原始图像,原始图像的像素呈非拜耳阵列排布,电子设备包括一个或多个处理器和存储器,存储器存储有计算机程序,计算机程序被处理器执行的情况下,处理器用于:根据当前场景,对原始图像进行转换以获得拜耳图像,拜耳图像的像素呈拜耳阵列排布;根据拜耳图像输出目标图像。Embodiments of the present application provide an electronic device. The polarization sensor includes a polarizer array and a pixel array, the polarizer array includes a plurality of polarizer modules, each polarizer module includes four polarizer groups, and the four polarizer groups in each polarizer module are arranged in a Bayer array, Each polarizer group includes a plurality of polarizers, the polarization directions of each polarizer in each polarizer group are different from each other, the color components of each polarizer in each polarizer group are the same, and the pixel array includes a plurality of pixel units , a plurality of pixel units and a plurality of polarizers are arranged in one-to-one correspondence, the pixel unit is used to receive the light passing through the polarizer to generate an electrical signal, the polarization sensor is used to generate an original image according to the electrical signal, and the pixels of the original image are in a non-Bayer array Arrangement, the electronic device includes one or more processors and a memory, the memory stores a computer program, and when the computer program is executed by the processor, the processor is used for: according to the current scene, convert the original image to obtain the Bayer image, the Bayer image The pixels of the image are arranged in a Bayer array; the target image is output according to the Bayer image.
在某些实施方式中,当前场景包括偏亮场景,处理器还用于在当前场景为偏亮场景的情况下,抽取每个偏振片组中具有相同偏振方向的偏振片对应的原始图像的像素并对抽取的像素进行组合,以获得拜耳图像。In some embodiments, the current scene includes a bright scene, and the processor is further configured to extract the pixels of the original image corresponding to the polarizers with the same polarization direction in each polarizer group when the current scene is a bright scene and combine the extracted pixels to obtain a Bayer image.
在某些实施方式中,当前场景包括普通场景,处理器还用于在当前场景为普通场景的情况下,融合每个偏振片组中各个偏振片对应的原始图像的像素并对融合后的像素进行组合,以获得拜耳图像。In some embodiments, the current scene includes an ordinary scene, and the processor is further configured to fuse the pixels of the original image corresponding to each polarizer in each polarizer group and merge the pixels after fusion when the current scene is an ordinary scene. combined to obtain a Bayer image.
在某些实施方式中,当前场景包括整体偏暗局部偏亮场景,处理器还用于:在当前场景为整体偏暗局部偏亮场景的情况下,抽取每个偏振片组中具有相同偏振方向的偏振片对应的原始图像的像素并对抽取的像素进行组合,以获得单偏振图像;融合每个偏振片组中各个偏振片对应的原始图像的像素并对融合后的像素进行组合,以获得无偏振图像;融合单偏振图像和无偏振图像以获得拜耳图像。In some embodiments, the current scene includes an overall dark and partially bright scene, and the processor is further configured to: in the case that the current scene is an overall dark and partially bright scene, extract each polarizer group with the same polarization direction The pixels of the original image corresponding to the polarizers of each polarizer group are combined and the extracted pixels are combined to obtain a single-polarization image; the pixels of the original image corresponding to each polarizer in each polarizer group are fused and the fused pixels are combined to obtain a Unpolarized images; fusing single and unpolarized images to obtain Bayer images.
在某些实施方式中,处理器还用于:对单偏振图像和无偏振图像进行滤波;融合滤波后的单偏振图像和无偏振图像以获得拜耳图像。In certain embodiments, the processor is further configured to: filter the single polarization image and the unpolarized image; and fuse the filtered single polarization image and the unpolarized image to obtain a Bayer image.
在某些实施方式中,处理器还用于:将拜耳图像依次经过坏点补偿模块、去马赛克模块、色彩校正模块、伽玛校正模块、色彩转换模块处理后输出目标图像。In some embodiments, the processor is further configured to: output the target image after processing the Bayer image sequentially through a dead pixel compensation module, a demosaicing module, a color correction module, a gamma correction module, and a color conversion module.
在某些实施方式中,处理器还用于:在当前场景为敞亮场景的情况下,采用拜耳图像再生成算法对偏振传感器输出的原始图像进行转换以获得拜耳图像。In some embodiments, the processor is further configured to: convert the original image output by the polarization sensor using a Bayer image regeneration algorithm to obtain a Bayer image when the current scene is a bright scene.
在某些实施方式中,处理器还用于:In certain embodiments, the processor is also used to:
将拜耳图像依次经过摄像机串行接口解码器、黑电平模块、镜头阴影校正模块、坏点补偿模块、去马赛克模块、色彩校正模块、伽玛校正模块、色彩转换模块处理后输出目标图像。The Bayer image is sequentially processed by the camera serial interface decoder, black level module, lens shading correction module, dead pixel compensation module, demosaicing module, color correction module, gamma correction module, and color conversion module to output the target image.
本申请实施方式还提供一种计算机可读存储介质。计算机可读存储介质上存储有计算机程序,程序被处理器执行的情况下,实现上述任一实施方式的图像处理方法的步骤。Embodiments of the present application also provide a computer-readable storage medium. A computer program is stored on the computer-readable storage medium, and when the program is executed by the processor, the steps of the image processing method in any of the above-mentioned embodiments are implemented.
请参阅图1-4,本申请实施方式的图像处理方法用于偏振传感器300。偏振传感器300包括偏振片阵列310和像素阵列320。偏振片阵列310包括多个偏振片模块311,每个偏振片模块311包括四个偏振片组312。每个偏振片模块311中的四个偏振片组312呈拜耳阵列排布。每个偏振片组312包括多个偏振片314。每个偏振片组312中的各个偏振片314的偏振方向互不相同,每个偏振片组312中的各个偏振片314的颜色分量相同。像素阵列320包括多个像素单元321。多个像素单元321和多个偏振片314 一一对应设置。像素单元321用于接收透过偏振片314的光线以生成电信号。偏振传感器300用于根据电信号生成原始图像。原始图像的像素呈非拜耳阵列排布。图像处理方法包括:Referring to FIGS. 1-4 , the image processing method of the embodiment of the present application is used for the polarization sensor 300 . Polarization sensor 300 includes polarizer array 310 and pixel array 320 . The polarizer array 310 includes a plurality of polarizer modules 311 , and each polarizer module 311 includes four polarizer groups 312 . The four polarizer groups 312 in each polarizer module 311 are arranged in a Bayer array. Each polarizer group 312 includes a plurality of polarizers 314 . The polarization directions of each polarizer 314 in each polarizer group 312 are different from each other, and the color components of each polarizer 314 in each polarizer group 312 are the same. The pixel array 320 includes a plurality of pixel units 321 . The plurality of pixel units 321 and the plurality of polarizers 314 are arranged in a one-to-one correspondence. The pixel unit 321 is used for receiving light transmitted through the polarizer 314 to generate electrical signals. The polarization sensor 300 is used to generate a raw image from the electrical signal. The pixels of the original image are arranged in a non-Bayer array. Image processing methods include:
01:根据当前场景,对原始图像进行转换以获得拜耳图像,拜耳图像的像素呈拜耳阵列排布;01: According to the current scene, convert the original image to obtain a Bayer image, and the pixels of the Bayer image are arranged in a Bayer array;
03:根据拜耳图像输出目标图像。03: Output the target image according to the Bayer image.
本申请实施方式的图像处理方法可由本申请实施方式的图像处理装置100实现。具体地,图像处理方法用于偏振传感器300。偏振传感器300包括偏振片阵列310和像素阵列320。偏振片阵列310包括多个偏振片模块311,每个偏振片模块311包括四个偏振片组312。每个偏振片模块311中的四个偏振片组312呈拜耳阵列排布。每个偏振片组312包括多个偏振片314。每个偏振片组312中的各个偏振片314的偏振方向互不相同,每个偏振片组312中的各个偏振片314的颜色分量相同。像素阵列320包括多个像素单元321。多个像素单元321和多个偏振片314一一对应设置。像素单元321用于接收透过偏振片314的光线以生成电信号。偏振传感器300用于根据电信号生成原始图像。原始图像的像素呈非拜耳阵列排布。图像处理装置100包括转换模块10和输出模块30。转换模块10用于根据当前场景,对原始图像进行转换以获得拜耳图像,拜耳图像的像素呈拜耳阵列排布。输出模块30用于根据拜耳图像输出目标图像。The image processing method of the embodiment of the present application can be implemented by the image processing apparatus 100 of the embodiment of the present application. Specifically, an image processing method is used for the polarization sensor 300 . Polarization sensor 300 includes polarizer array 310 and pixel array 320 . The polarizer array 310 includes a plurality of polarizer modules 311 , and each polarizer module 311 includes four polarizer groups 312 . The four polarizer groups 312 in each polarizer module 311 are arranged in a Bayer array. Each polarizer group 312 includes a plurality of polarizers 314 . The polarization directions of each polarizer 314 in each polarizer group 312 are different from each other, and the color components of each polarizer 314 in each polarizer group 312 are the same. The pixel array 320 includes a plurality of pixel units 321 . The plurality of pixel units 321 and the plurality of polarizers 314 are arranged in a one-to-one correspondence. The pixel unit 321 is used for receiving light transmitted through the polarizer 314 to generate electrical signals. The polarization sensor 300 is used to generate a raw image from the electrical signal. The pixels of the original image are arranged in a non-Bayer array. The image processing apparatus 100 includes a conversion module 10 and an output module 30 . The conversion module 10 is configured to convert the original image according to the current scene to obtain a Bayer image, and the pixels of the Bayer image are arranged in a Bayer array. The output module 30 is used for outputting the target image according to the Bayer image.
本申请实施方式的图像处理方法可由本申请实施方式的电子设备200实现。具体地,电子设备200包括一个或多个处理器201和存储器202。存储器202存储有计算机程序。计算机程序被处理器201执行的情况下,实现上述步骤01、步骤03。The image processing method of the embodiment of the present application may be implemented by the electronic device 200 of the embodiment of the present application. Specifically, the electronic device 200 includes one or more processors 201 and a memory 202 . The memory 202 stores computer programs. When the computer program is executed by the processor 201, the above steps 01 and 03 are implemented.
上述图像处理方法、图像处理装置100及电子设备200,能够将偏振传感器300输出的呈非拜耳阵列排布的原始图像转换成呈拜耳阵列排布的拜耳图像,从而偏振传感器300输出的原始图像能够被手机等电子设备200处理。The above-mentioned image processing method, image processing apparatus 100 and electronic device 200 can convert the original image arranged in a non-Bayer array output by the polarization sensor 300 into a Bayer image arranged in a Bayer array, so that the original image output by the polarization sensor 300 can be It is processed by the electronic device 200 such as a mobile phone.
具体地,偏振传感器300能够接收外界的光线,并根据接收到的光线生成原始图像。原始图像的像素呈非拜耳阵列排布,手机等电子设备200不能够处理像素呈非拜耳阵列排布的原始图像,需要将原始图像的像素转换成呈拜耳阵列排布的格式。偏振传感器300还包括微透镜阵列330,微透镜阵列330设置在偏振片阵列310正上方,微透镜阵列330包括多个微透镜331,微透镜331、偏振片314和像素单元321一一对应。请结合图5,图5为偏振传感器300的偏振片模块311的示意图,在某些实施方式中,偏振片模块311可包括两个第一偏振片组3122、一个第二偏振片组3124和一个第三偏振片组3126,四个偏振片组312呈拜耳阵列排布,其中,第一偏振片组3122中的所有偏振片314的颜色分量均为绿色(Green,G),第二偏振片组3124中的所有偏振片314的颜色分量均为蓝色(Blue,B),第三偏振片组3126中的所有偏振片314的颜色分量均为红色(Red,R)。偏振片314的颜色分量相同,即偏振片314能够透过相同波长范围的可见光,例如都能够透过波长范围为625~740nm的红光,或者都能够透过波长范围为492~577nm的绿光,或者都能够透过波长范围为440~475nm的蓝光。两个第一偏振片组3122、一个第二偏振片组3124和一个第三偏振片组3126以2*2的形式排布,两个第一偏振片组3122沿第一对角线方向D1排布,一个第二偏振片组3124和一个第三偏振片组3126沿第二对角线方向D2排布,第一对角线方向D1与第二对角线方向D2相互垂直,在图5的示例中,偏振片模块311的排布为GRBG,在其他示例中,偏振片模块311的排布也可为RGGB、BGGR或者GBRG。在图5的实施方式中,每个偏振片组312中的偏振片314以2*2的形式排布,即每个偏振片组312均包括四个偏振片314,四个偏振片314的偏振方向各不相同,分别为第一偏振方向、第二偏振方向、第三偏振方向和第四偏振方向。每个偏振片314的下方设置一个像素单元321,当外界光线透过偏振片314照射在像素单元321上时,像素单元321能够生成对应的电信号。在一个例子中,第一偏振方向、第二偏振方向、第三偏振方向和第四偏振方向分别为0°、45°、90°和135°。需要指出的是,在其他实施方式中,每个偏振片组312中的偏振片314还可以3*3或4*4的形式排布,当偏振片314以3*3的形式排布时,每个偏振片组312包括 偏振方向不同的9个偏振片314;当偏振片314以4*4的形式排布时,每个偏振片组312包括偏振方向不同的16个偏振片314。Specifically, the polarization sensor 300 can receive external light, and generate an original image according to the received light. The pixels of the original image are arranged in a non-Bayer array, and the electronic device 200 such as a mobile phone cannot process the original image whose pixels are arranged in a non-Bayer array, and the pixels of the original image need to be converted into a format arranged in a Bayer array. The polarization sensor 300 further includes a microlens array 330 disposed directly above the polarizer array 310. The microlens array 330 includes a plurality of microlenses 331, and the microlenses 331, the polarizers 314 and the pixel units 321 are in one-to-one correspondence. Please refer to FIG. 5 . FIG. 5 is a schematic diagram of the polarizer module 311 of the polarization sensor 300 . In some embodiments, the polarizer module 311 may include two first polarizer groups 3122 , one second polarizer group 3124 and one The third polarizer group 3126, the four polarizer groups 312 are arranged in a Bayer array, wherein the color components of all the polarizers 314 in the first polarizer group 3122 are green (Green, G), and the second polarizer group 312 The color components of all polarizers 314 in 3124 are blue (Blue, B), and the color components of all polarizers 314 in the third polarizer group 3126 are red (Red, R). The color components of the polarizers 314 are the same, that is, the polarizers 314 can transmit visible light with the same wavelength range, for example, both can transmit red light with a wavelength range of 625-740 nm, or both can transmit green light with a wavelength range of 492-577 nm , or both can transmit blue light in the wavelength range of 440 to 475 nm. Two first polarizer groups 3122, one second polarizer group 3124 and one third polarizer group 3126 are arranged in a 2*2 form, and the two first polarizer groups 3122 are arranged along the first diagonal direction D1 cloth, a second polarizer group 3124 and a third polarizer group 3126 are arranged along the second diagonal direction D2, and the first diagonal direction D1 and the second diagonal direction D2 are perpendicular to each other. In an example, the arrangement of the polarizer modules 311 is GRBG, and in other examples, the arrangement of the polarizer modules 311 can also be RGGB, BGGR or GBRG. In the embodiment of FIG. 5 , the polarizers 314 in each polarizer group 312 are arranged in the form of 2*2, that is, each polarizer group 312 includes four polarizers 314 , and the polarizers of the four polarizers 314 The directions are different from each other and are respectively a first polarization direction, a second polarization direction, a third polarization direction and a fourth polarization direction. A pixel unit 321 is disposed under each polarizer 314 , and when external light passes through the polarizer 314 and irradiates the pixel unit 321 , the pixel unit 321 can generate a corresponding electrical signal. In one example, the first polarization direction, the second polarization direction, the third polarization direction and the fourth polarization direction are 0°, 45°, 90° and 135°, respectively. It should be noted that, in other embodiments, the polarizers 314 in each polarizer group 312 may also be arranged in the form of 3*3 or 4*4. When the polarizers 314 are arranged in the form of 3*3, Each polarizer group 312 includes 9 polarizers 314 with different polarization directions; when the polarizers 314 are arranged in a 4*4 form, each polarizer group 312 includes 16 polarizers 314 with different polarization directions.
在步骤01中,当前场景可包括偏亮场景、普通场景、局部偏暗整体偏亮场景、敞亮场景。在某些实施方式中,普通场景为光线强度位于第一强度值和第二强度值之间的场景,偏亮场景为光线强度位于第三强度值和第四强度值之间的场景,敞亮场景为光线强度大于第五强度值的场景,第一强度值小于第二强度值,第二强度值不大于第三强度值,第三强度值小于第四强度值,第四强度值不大于第五强度值。在某些实施方式中,偏亮场景可包括阴天室外,在当前场景的光线强度位于2000lux-3000lux之间(例如2500lux)时,可确定当前场景为阴天室外。在某些实施方式中,普通场景可包括室内商场,在当前场景的光线强度位于950lux-1050lux之间(例如1000lux)时,可确定当前场景为室内商场。在某些实施方式中,局部偏暗整体偏亮场景可包括路灯夜景或夜晚有灯光的场景,在当前场景的光线强度位于300lux-500lux之间(例如400lux)时,可确定当前场景为夜晚有灯光的场景。在某些实施方式中,敞亮场景可包括晴天室外,在当前场景的光线强度大于8000lux(例如9000lux)时,可确定当前场景为晴天室外。在某些实施方式中,在步骤01之前,图像处理方法还包括:确定当前场景。在某些实施方式中,电子设备200包括光线传感器,在拍摄时可根据光线传感器的测量值确定当前场景。在某些实施方式中,可基于训练模型预先对不同拍摄环境进行机器学习,从而在拍摄时根据拍摄环境自动确定当前场景。在其他实施方式中,当前场景还可由用户自行设置。在确定当前场景之后,根据当前场景对原始图像进行转换,从而获得与当前场景对应的拜耳图像,拜耳图像的像素呈拜耳阵列排布,拜耳图像可以被手机等电子设备200进一步处理。In step 01, the current scene may include a bright scene, a normal scene, a partial dark scene and an overall bright scene, and a bright scene. In some embodiments, the normal scene is a scene where the light intensity is between the first intensity value and the second intensity value, the bright scene is a scene where the light intensity is between the third intensity value and the fourth intensity value, and the bright scene In the scene where the light intensity is greater than the fifth intensity value, the first intensity value is smaller than the second intensity value, the second intensity value is not greater than the third intensity value, the third intensity value is smaller than the fourth intensity value, and the fourth intensity value is not greater than the fifth intensity value intensity value. In some embodiments, the bright scene may include a cloudy outdoor, and when the light intensity of the current scene is between 2000 lux-3000 lux (eg, 2500 lux), it may be determined that the current scene is a cloudy outdoor. In some embodiments, the common scene may include an indoor shopping mall, and when the light intensity of the current scene is between 950 lux-1050 lux (eg, 1000 lux), the current scene may be determined to be an indoor shopping mall. In some embodiments, the partially dark and overall bright scene may include a night scene of street lights or a scene with lights at night. When the light intensity of the current scene is between 300lux and 500lux (for example, 400lux), it may be determined that the current scene is a scene with lights at night. Lighting scene. In some embodiments, the bright and bright scene may include a sunny outdoor, and when the light intensity of the current scene is greater than 8000 lux (eg, 9000 lux), it may be determined that the current scene is a sunny outdoor. In some embodiments, before step 01, the image processing method further includes: determining the current scene. In some embodiments, the electronic device 200 includes a light sensor, and the current scene can be determined according to the measurement value of the light sensor when shooting. In some embodiments, machine learning may be performed on different shooting environments in advance based on the training model, so that the current scene is automatically determined according to the shooting environment during shooting. In other embodiments, the current scene can also be set by the user. After the current scene is determined, the original image is converted according to the current scene to obtain a Bayer image corresponding to the current scene. The pixels of the Bayer image are arranged in a Bayer array, and the Bayer image can be further processed by electronic devices 200 such as mobile phones.
在步骤03中,手机等电子设备200可包括图像管道,图像管道对拜耳图像进行处理,从而输出目标图像。在图4所示的实施方式中,电子设备200为手机,在其他实施方式中,电子设备200可包括平板电脑、智能手表或者其他配置有偏振传感器300的可移动终端。In step 03, the electronic device 200 such as a mobile phone may include an image pipeline, and the image pipeline processes the Bayer image, thereby outputting the target image. In the embodiment shown in FIG. 4 , the electronic device 200 is a mobile phone. In other embodiments, the electronic device 200 may include a tablet computer, a smart watch, or other mobile terminals configured with the polarization sensor 300 .
请参阅图6和图7,在某些实施方式中,当前场景包括偏亮场景,步骤01包括:Please refer to FIG. 6 and FIG. 7, in some embodiments, the current scene includes a bright scene, and step 01 includes:
011:在当前场景为偏亮场景的情况下,抽取每个偏振片组312中具有相同偏振方向的偏振片314对应的原始图像的像素并对抽取的像素进行组合,以获得拜耳图像。011: When the current scene is a bright scene, extract the pixels of the original image corresponding to the polarizers 314 having the same polarization direction in each polarizer group 312 and combine the extracted pixels to obtain a Bayer image.
上述实施方式的图像处理方法可由本申请实施方式的图像处理装置100实现。具体地,转换模块10用于在当前场景为偏亮场景的情况下,抽取每个偏振片组312中具有相同偏振方向的偏振片314对应的原始图像的像素并对抽取的像素进行组合,以获得拜耳图像。The image processing methods of the above embodiments can be implemented by the image processing apparatus 100 of the embodiments of the present application. Specifically, the conversion module 10 is configured to extract the pixels of the original image corresponding to the polarizers 314 having the same polarization direction in each polarizer group 312 and combine the extracted pixels when the current scene is a bright scene. Obtain a Bayer image.
上述实施方式的图像处理方法可由本申请实施方式的电子设备200实现。具体地,处理器用于在当前场景为偏亮场景的情况下,抽取每个偏振片组312中具有相同偏振方向的偏振片314对应的原始图像的像素并对抽取的像素进行组合,以获得拜耳图像。The image processing method of the above embodiment can be implemented by the electronic device 200 of the embodiment of the present application. Specifically, the processor is configured to extract the pixels of the original image corresponding to the polarizers 314 having the same polarization direction in each polarizer group 312 and combine the extracted pixels to obtain Bayer when the current scene is a bright scene image.
如此,在偏亮场景下,不需要采用复杂的算法处理原始图像,即可输出具有单一偏振方向的拜耳图像。In this way, in a bright scene, a Bayer image with a single polarization direction can be output without using a complex algorithm to process the original image.
具体地,偏亮场景可包括阴天室外。在一个例子中,每个偏振片组312中包括0°、45°、90°和135°四个偏振方向的偏振片314,在当前场景为偏亮场景的情况下,抽取每个偏振片组312中0°偏振方向的偏振片314对应的原始图像的像素,并按照对应的偏振片组312之间的组合方式对抽取的像素进行组合,可生成0°偏振图像;抽取每个偏振片组312中45°偏振方向的偏振片314对应的原始图像的像素,并按照对应的偏振片组312之间的组合方式对抽取的像素进行组合,可生成45°偏振图像;抽取每个偏振片组312中90°偏振方向的偏振片314对应的原始图像的像素,并按照对应的偏振片组312之间的组合方式对抽取的像素进行组合,可生成90°偏振图像;抽取每个偏振片组312中135°偏振方向的偏振片314对应的原始图像的像素,并按照对应的偏振片组312之间的组合方式对抽取的像素进行组合,可生成135°偏振图像,进一步地,可选择0°偏振图像、45°偏振图像、90°偏振图像或者135°偏振图像中的至少一个 作为拜耳图像。在当前场景为偏亮场景的情况下,获得的每帧拜耳图像为单偏振图像,获得的每帧拜耳图像的像素数量是原始图像的像素数量的四分之一。Specifically, the brighter scene may include a cloudy outdoor. In an example, each polarizer group 312 includes polarizers 314 with four polarization directions of 0°, 45°, 90°, and 135°, and when the current scene is a bright scene, extract each polarizer group The pixels of the original image corresponding to the polarizers 314 in the 0° polarization direction in 312, and the extracted pixels are combined according to the combination between the corresponding polarizer groups 312 to generate a 0° polarized image; each polarizer group is extracted. The pixels of the original image corresponding to the polarizers 314 in the 45° polarization direction in 312, and the extracted pixels are combined according to the combination between the corresponding polarizer groups 312 to generate a 45° polarized image; each polarizer group is extracted The pixels of the original image corresponding to the polarizers 314 in the 90° polarization direction in 312, and the extracted pixels are combined according to the combination between the corresponding polarizer groups 312 to generate a 90° polarized image; each polarizer group is extracted The pixels of the original image corresponding to the polarizers 314 in the 135° polarization direction in 312, and the extracted pixels are combined according to the combination between the corresponding polarizer groups 312 to generate a 135° polarized image. Further, 0 can be selected. At least one of a ° polarization image, a 45° polarization image, a 90° polarization image, or a 135° polarization image serves as a Bayer image. When the current scene is a bright scene, each frame of the obtained Bayer image is a single-polarization image, and the number of pixels of each frame of the obtained Bayer image is one-fourth of the number of pixels of the original image.
请参阅图8和图9,在某些实施方式中,当前场景包括普通场景,步骤01包括:Please refer to FIG. 8 and FIG. 9. In some embodiments, the current scene includes a common scene, and step 01 includes:
012:在当前场景为普通场景的情况下,融合每个偏振片组312中各个偏振片314对应的原始图像的像素并对融合后的像素进行组合,以获得拜耳图像。012: When the current scene is an ordinary scene, fuse the pixels of the original image corresponding to each polarizer 314 in each polarizer group 312 and combine the fused pixels to obtain a Bayer image.
上述实施方式的图像处理方法可由本申请实施方式的图像处理装置100实现。具体地,转换模块10用于在当前场景为普通场景的情况下,融合每个偏振片组312中各个偏振片314对应的原始图像的像素并对融合后的像素进行组合,以获得拜耳图像。The image processing methods of the above embodiments can be implemented by the image processing apparatus 100 of the embodiments of the present application. Specifically, the conversion module 10 is configured to fuse the pixels of the original image corresponding to each polarizer 314 in each polarizer group 312 and combine the fused pixels to obtain a Bayer image when the current scene is an ordinary scene.
上述实施方式的图像处理方法可由本申请实施方式的电子设备200实现。具体地,处理器用于在当前场景为普通场景的情况下,融合每个偏振片组312中各个偏振片314对应的原始图像的像素并对融合后的像素进行组合,以获得拜耳图像。The image processing method of the above embodiment can be implemented by the electronic device 200 of the embodiment of the present application. Specifically, the processor is configured to fuse the pixels of the original image corresponding to each polarizer 314 in each polarizer group 312 and combine the fused pixels to obtain a Bayer image when the current scene is an ordinary scene.
如此,在普通场景下,不需要采用复杂的算法处理原始图像,即可输出无偏振的、具有高信噪比的拜耳图像。In this way, in a common scenario, an unpolarized Bayer image with a high signal-to-noise ratio can be output without using a complex algorithm to process the original image.
具体地,普通场景包括室内商场。在某些实施方式中,在当前场景为普通场景的情况下,将一个偏振片组312中所有偏振片314对应的原始图像的像素值相加或求平均,以得到拜耳图像中对应该偏振片组312的位置的像素值,对所有偏振片组312执行上述操作,即可确定拜耳图像中对应每个偏振片组312的位置的像素值,这样得到的拜耳图像具有较高的信噪比。在当前场景为普通场景的情况下,获得的拜耳图像为无偏振图像,获得的拜耳图像的像素数量是原始图像的像素数量的四分之一。Specifically, common scenarios include indoor shopping malls. In some embodiments, when the current scene is an ordinary scene, the pixel values of the original images corresponding to all the polarizers 314 in one polarizer group 312 are added or averaged to obtain the corresponding polarizer in the Bayer image. The pixel values at the positions of the groups 312 are determined by performing the above operations on all the polarizer groups 312 to determine the pixel values corresponding to the positions of each polarizer group 312 in the Bayer image. The Bayer image thus obtained has a higher signal-to-noise ratio. When the current scene is an ordinary scene, the obtained Bayer image is an unpolarized image, and the number of pixels of the obtained Bayer image is a quarter of the number of pixels of the original image.
请参阅图10-12,在某些实施方式中,当前场景包括整体偏暗局部偏亮场景,步骤01括:Please refer to FIGS. 10-12. In some embodiments, the current scene includes an overall dark scene and a partially bright scene. Step 01 includes:
013:在当前场景为整体偏暗局部偏亮场景的情况下,抽取每个偏振片组312中具有相同偏振方向的偏振片314对应的原始图像的像素并对抽取的像素进行组合,以获得单偏振图像;013: In the case where the current scene is an overall dark and partially bright scene, extract the pixels of the original image corresponding to the polarizers 314 having the same polarization direction in each polarizer group 312 and combine the extracted pixels to obtain a single polarized image;
014:融合每个偏振片组312中各个偏振片314对应的原始图像的像素并对融合后的像素进行组合,以获得无偏振图像;014: fuse the pixels of the original image corresponding to each polarizer 314 in each polarizer group 312 and combine the fused pixels to obtain an unpolarized image;
015:融合单偏振图像和无偏振图像以获得拜耳图像。015: Fuse single-polarized and unpolarized images to obtain a Bayer image.
上述实施方式的图像处理方法可由本申请实施方式的图像处理装置100实现。具体地,转换模块10包括抽取单元13、第一融合单元14、第二融合单元15。抽取单元13用于在当前场景为整体偏暗局部偏亮场景的情况下,抽取每个偏振片组312中具有相同偏振方向的偏振片314对应的原始图像的像素并对抽取的像素进行组合,以获得单偏振图像。第一融合单元14用于融合每个偏振片组312中各个偏振片314对应的原始图像的像素并对融合后的像素进行组合,以获得无偏振图像。第二融合单元15用于融合单偏振图像和无偏振图像以获得拜耳图像。The image processing methods of the above embodiments can be implemented by the image processing apparatus 100 of the embodiments of the present application. Specifically, the conversion module 10 includes an extraction unit 13 , a first fusion unit 14 , and a second fusion unit 15 . The extraction unit 13 is used for extracting the pixels of the original image corresponding to the polarizers 314 having the same polarization direction in each polarizer group 312 and combining the extracted pixels when the current scene is an overall darker and partially brighter scene, to obtain single-polarization images. The first fusion unit 14 is configured to fuse the pixels of the original image corresponding to each polarizer 314 in each polarizer group 312 and combine the fused pixels to obtain an unpolarized image. The second fusion unit 15 is used to fuse the single-polarization image and the non-polarization image to obtain a Bayer image.
上述实施方式的图像处理方法可由本申请实施方式的电子设备200实现。具体地,处理器用于在当前场景为整体偏暗局部偏亮场景的情况下,抽取每个偏振片组312中具有相同偏振方向的偏振片314对应的原始图像的像素并对抽取的像素进行组合,以获得单偏振图像,及用于融合每个偏振片组312中各个偏振片314对应的原始图像的像素并对融合后的像素进行组合,以获得无偏振图像,及用于融合单偏振图像和无偏振图像以获得拜耳图像。The image processing method of the above embodiment can be implemented by the electronic device 200 of the embodiment of the present application. Specifically, the processor is configured to extract the pixels of the original image corresponding to the polarizers 314 having the same polarization direction in each polarizer group 312 and combine the extracted pixels when the current scene is an overall darker and partially brighter scene , to obtain a single-polarization image, and for fusing the pixels of the original image corresponding to each polarizer 314 in each polarizer group 312 and combining the fused pixels to obtain an unpolarized image, and for fusing the single-polarization image and unpolarized images to obtain Bayer images.
如此,在当前场景为整体偏暗局部偏亮场景的情况下,融合单偏振图像和无偏振图像,获得具有偏振信息的拜耳图像。In this way, when the current scene is an overall dark scene and a partially bright scene, a single-polarization image and a non-polarization image are fused to obtain a Bayer image with polarization information.
具体地,整体偏暗局部偏亮场景可包括路灯夜景或夜晚有灯光的场景。每个偏振片组312包括多个不同偏振方向的偏振片314,抽取每个偏振片组312中相同偏振方向的偏振片314对应的原始图像可获得多帧单偏振图像,每帧单偏振图像的偏振信息不同。例如,当偏振片组312中包括0°、45°、90°和135°四种偏振方向的偏振片314时,在步骤013中可获得4帧单偏振图像,分别为0°偏振图像、45°偏振图 像、90°偏振图像和135°偏振图像,且每帧单偏振图像的像素数量是原始图像的像素数量的四分之一。Specifically, the overall dark and partially bright scene may include a street lamp night scene or a scene with lights at night. Each polarizer group 312 includes a plurality of polarizers 314 with different polarization directions, and extracting the original images corresponding to the polarizers 314 with the same polarization direction in each polarizer group 312 can obtain multiple frames of single-polarization images. The polarization information is different. For example, when the polarizer group 312 includes polarizers 314 with four polarization directions of 0°, 45°, 90° and 135°, 4 frames of single-polarization images can be obtained in step 013, which are the 0° polarized image, the 45° polarized image, and the 45° polarized image. ° polarized image, 90 ° polarized image and 135 ° polarized image, and the number of pixels of each frame of single polarized image is one-fourth of that of the original image.
在步骤014中,将一个偏振片组312中所有偏振片314对应的原始图像的像素值相加或求平均,以得到无偏振图像中对应该偏振片组312的位置的像素值,对所有偏振片组312执行上述操作,即可确定无偏振图像中对应每个偏振片组312的位置的像素值。无偏振图像的像素数量是原始图像的像素数量的四分之一。In step 014, the pixel values of the original images corresponding to all the polarizers 314 in one polarizer group 312 are added or averaged to obtain the pixel values corresponding to the position of the polarizer group 312 in the unpolarized image, and for all polarizers 312 The plate group 312 performs the above operations to determine the pixel value corresponding to the position of each polarizer plate group 312 in the unpolarized image. The unpolarized image has one quarter the number of pixels of the original image.
在步骤015中,在当前场景为整体偏暗局部偏亮场景的情况下,从多帧单偏振图像中选取一帧图像,从而融合一帧单偏振图像和一帧无偏振图像以获得一帧具有偏振信息的拜耳图像。由于每帧单偏振图像的像素数量是原始图像的像素数量的四分之一,每帧无偏振图像的像素数量也是原始图像的像素数量的四分之一,在当前场景为整体偏暗局部偏亮场景的情况下,融合单偏振图像和无偏振图像获得的每帧拜耳图像的像素数量是原始图像的像素数量的四分之一,拜耳图像为融合偏振图像。In step 015, when the current scene is an overall dark and partially bright scene, select a frame of images from multiple frames of single-polarization images, so as to fuse a frame of single-polarization images and a frame of unpolarized images to obtain a frame with Bayer image of polarization information. Since the number of pixels of each frame of single-polarization image is one-fourth of the number of pixels of the original image, and the number of pixels of each frame of unpolarized image is also one-fourth of the number of pixels of the original image, in the current scene, the overall darkening and partial polarization In the case of a bright scene, the number of pixels in each frame of the Bayer image obtained by fusing the single-polarization image and the non-polarization image is a quarter of the pixel number of the original image, and the Bayer image is a fused polarization image.
请参阅图13-15,在某些实施方式中,步骤015包括:Referring to Figures 13-15, in some embodiments, step 015 includes:
0151:对单偏振图像和无偏振图像进行滤波;0151: Filter the single-polarization image and the non-polarization image;
0153:融合滤波后的单偏振图像和无偏振图像以获得拜耳图像。0153: Fuse the filtered single-polarization image and the unpolarized image to obtain a Bayer image.
上述实施方式的图像处理方法可由本申请实施方式的图像处理装置100实现。具体地,第二融合单元15包括滤波子单元151、融合子单元153。滤波子单元151用于对单偏振图像和无偏振图像进行滤波。融合子单元153用于融合滤波后的单偏振图像和无偏振图像以获得拜耳图像。The image processing methods of the above embodiments can be implemented by the image processing apparatus 100 of the embodiments of the present application. Specifically, the second fusion unit 15 includes a filtering subunit 151 and a fusion subunit 153 . The filtering subunit 151 is used for filtering the single-polarization image and the non-polarization image. The fusion subunit 153 is used to fuse the filtered single-polarization image and the non-polarization image to obtain a Bayer image.
上述实施方式的图像处理方法可由本申请实施方式的电子设备200实现。具体地,处理器用于对单偏振图像和无偏振图像进行滤波,及用于融合滤波后的单偏振图像和无偏振图像以获得拜耳图像。The image processing method of the above embodiment can be implemented by the electronic device 200 of the embodiment of the present application. Specifically, the processor is used for filtering the single-polarization image and the non-polarization image, and for fusing the filtered single-polarization image and the non-polarization image to obtain a Bayer image.
如此,可以获得视觉效果更好的拜耳图像。In this way, a Bayer image with better visual effect can be obtained.
具体地,在步骤0151中,在某些实施方式中,以无偏振图像为导向图,分别对单偏振图像和无偏振图像进行双边滤波,从而将平坦区进行一个平滑的处理。处理过程可由以下公式表示:
其中,k
p=∑
q∈Ωf(||p-q||)g(||I
p′-I
q′||),J
p为输出像素值,k
p为权重总和,Ω为滤波窗口(例如3*3的窗口),p为待滤波像素点在相位图中的坐标,q为滤波窗口内的像素点在相位图中的坐标,I
q为q点对应的像素值,I
p′为导向图中与待滤波像素点对应的像素值,I
q′为导向图中与q点对应的像素值。f表示滤波窗口中每个坐标点的权重,滤波窗口中每个坐标点的权重是固定的,越靠近中心权重越大。g表示其他位置的像素与中心像素差异的权重,差异越大,权重越小。在某些实施方式中,f和g可以为任意的权重分布函数,两者可以相同或者不同,例如,f和g均为高斯函数,高斯函数例如为
a、b和c为调节系数。
Specifically, in step 0151, in some embodiments, the single-polarization image and the non-polarization image are respectively subjected to bilateral filtering with the unpolarized image as the guide map, so as to perform a smoothing process on the flat area. The processing can be represented by the following formula: Among them, k p =∑ q∈Ω f(||pq||)g(||I p ′-I q ′||), J p is the output pixel value, k p is the weight sum, Ω is the filter window ( For example, a 3*3 window), p is the coordinate of the pixel to be filtered in the phase map, q is the coordinate of the pixel in the filter window in the phase map, I q is the pixel value corresponding to point q, and I p ′ is The pixel value corresponding to the pixel to be filtered in the guide map, and I q ′ is the pixel value corresponding to point q in the guide map. f represents the weight of each coordinate point in the filtering window, the weight of each coordinate point in the filtering window is fixed, and the closer to the center, the greater the weight. g represents the weight of the difference between the pixels in other positions and the center pixel, the larger the difference, the smaller the weight. In some embodiments, f and g can be any weight distribution function, and they can be the same or different. For example, f and g are both Gaussian functions, and the Gaussian function is, for example, a, b and c are adjustment coefficients.
在步骤0153中,采用融合(Fusion)算法融合滤波后的单偏振图像和无偏振图像以获得拜耳图像。请结合图15,以绿色像素单元作为示例。G_smooth表示滤波后的单偏振图像中的绿色像素单元,包括G1、G2、G3、G4、G5五个像素点。G'_smooth表示滤波后的无偏振图像中的绿色像素单元,包括G'1、G'2、G'3、G'4、G'5五个像素点。在融合G_smooth和G'_smooth的过程中,融合后中心点的像素值可由以下公式表示:G5_output=G_mean*(G′
5/G′_mean),其中,G5_output表示中心点的像素值,
∑α
n=1,G
n表示Gn点对应的像素值,G′
n表示G'n点对应的像素值。滤波后的单偏振图像和无偏振图像的红色像素单元、蓝色像素单元的融合过程与绿色像素单元的融合过程基本相同,为避免冗余,在此不再赘述。
In step 0153, a Fusion algorithm is used to fuse the filtered single-polarization image and the non-polarization image to obtain a Bayer image. Please refer to Figure 15, taking the green pixel unit as an example. G_smooth represents the green pixel unit in the filtered single-polarization image, including five pixel points G1, G2, G3, G4, and G5. G'_smooth represents the green pixel unit in the filtered unpolarized image, including five pixel points G'1, G'2, G'3, G'4, and G'5. In the process of fusing G_smooth and G'_smooth, the pixel value of the center point after fusion can be represented by the following formula: G5_output=G_mean*(G' 5 /G'_mean), where G5_output represents the pixel value of the center point, Σα n =1, G n represents the pixel value corresponding to the Gn point, and G′ n represents the pixel value corresponding to the G′n point. The fusion process of the red pixel unit and the blue pixel unit of the filtered single-polarization image and the non-polarization image is basically the same as the fusion process of the green pixel unit. In order to avoid redundancy, details are not repeated here.
请参阅图16和图17,在某些实施方式中,步骤03包括:16 and 17, in some embodiments, step 03 includes:
031:将拜耳图像依次经过坏点补偿(Bad pixel correction,BPC)模块、去马赛克(demosaic,DM)模块、色彩校正(Color correction,CC)模块、伽玛(Gamma)校正模块、色彩转换(color conversion,CV)模块处理后输出目标图像。031: Pass the Bayer image through a bad pixel correction (BPC) module, a demosaic (DM) module, a color correction (CC) module, a gamma (Gamma) correction module, and a color conversion (color) module in turn. conversion, CV) module to output the target image after processing.
上述实施方式的图像处理方法可由本申请实施方式的图像处理装置100实现。具体地,输出模块30 用于将拜耳图像依次经过坏点补偿模块、去马赛克模块、色彩校正模块、伽玛校正模块、色彩转换模块处理后输出目标图像。The image processing methods of the above embodiments can be implemented by the image processing apparatus 100 of the embodiments of the present application. Specifically, the output module 30 is configured to sequentially process the Bayer image through a dead pixel compensation module, a demosaic module, a color correction module, a gamma correction module, and a color conversion module to output the target image.
上述实施方式的图像处理方法可由本申请实施方式的电子设备200实现。具体地,处理器用于将拜耳图像依次经过坏点补偿模块、去马赛克模块、色彩校正模块、伽玛校正模块、色彩转换模块处理后输出目标图像。The image processing method of the above embodiment can be implemented by the electronic device 200 of the embodiment of the present application. Specifically, the processor is configured to sequentially process the Bayer image through a dead pixel compensation module, a demosaic module, a color correction module, a gamma correction module, and a color conversion module to output the target image.
如此,拜耳图像进入图像管道,经过图像管道的处理之后,转换为目标图像并输出。In this way, the Bayer image enters the image pipeline, and after being processed by the image pipeline, it is converted into the target image and output.
具体地,目标图像包括YUV图像。在某些实施方式中,偏振传感器(Sensor)300生成的原始图像(Quadbayer polarized RAW)依次经过摄像机串行接口解码器(Camera Serial Interface Decoder,CSID)、黑电平(Optical black,OB)模块、镜头阴影校正(lens shading correction,LSC)模块之后,进入双倍速率同步动态随机存储器(Double Data Rate,DDR),然后在Binning Algorithm模块中转换成拜耳图像之后,再依次经过坏点补偿模块、去马赛克模块、色彩校正模块、伽玛校正模块、色彩转换模块处理并输出目标图像。Specifically, the target image includes a YUV image. In some embodiments, the original image (Quadbayer polarized RAW) generated by the polarization sensor (Sensor) 300 is sequentially passed through a camera serial interface decoder (Camera Serial Interface Decoder, CSID), a black level (Optical black, OB) module, After the lens shading correction (LSC) module, it enters the double-rate synchronous dynamic random access memory (Double Data Rate, DDR), and then converts it into a Bayer image in the Binning Algorithm module, and then passes through the dead pixel compensation module in turn. The mosaic module, color correction module, gamma correction module, and color conversion module process and output the target image.
进一步地,摄像机串行接口解码器用于解码识别原始图像。偏振传感器300的电路本身会存在暗电流,导致在没有光线照射的时候,像素单元321也有一定的输出电压,因此需要减去暗电流带来的影响,也就是进行黑电平校正。镜头阴影校正是为了解决由于镜头的光学特性,即镜头对于光学折射不均匀导致的镜头周围出现阴影的情况。由于镜头本身就是一个凸透镜,由于凸透镜原理,中心的感光必然比周边多,偏振传感器300影像区的边缘区域接收的光强比中心小,所造成的中心和四角亮度不一致的现象。双倍速率同步动态随机存储器用于存储依次经过摄像机串行接口解码器、黑电平模块、镜头阴影校正模块之后的原始图像。请结合图18,Binning Algorithm模块能够从DDP中调取原始图像并执行步骤011、步骤012、步骤013、步骤014、步骤015、步骤0151、步骤0153以将原始图像转换为拜耳图像。坏点补偿即坏像素补偿,是指坏像素校正,在偏振传感器300中的某个像素单元321不能正常工作时,可以利用坏像素校正获得该像素单元321对应的像素值。去马赛克模块可以将拜耳阵列的拜耳图像插值成为RGB图像,其中,RGB图像可以是三帧,即分别为插值后得到的R图像、G图像和B图像。利用颜色校正后的图像的色彩饱和度更加明显,更加符合人眼感官。伽玛校正可使得图像看起来更符合人眼的特性,伽玛校正公式例如为Out=Ingamma或者Out=In1/gamma,其中,In表示输入图像,Out表示输出图像。色彩转换模块可以将RGB图像转换为YUV图像。Further, the camera serial interface decoder is used to decode and identify the original image. The circuit of the polarization sensor 300 itself will have dark current, so when there is no light irradiation, the pixel unit 321 also has a certain output voltage. Therefore, the influence of the dark current needs to be subtracted, that is, black level correction is performed. Lens shading correction is to solve the problem of shadows appearing around the lens due to the optical characteristics of the lens, that is, the lens has uneven optical refraction. Since the lens itself is a convex lens, due to the principle of convex lens, the center must receive more light than the periphery, and the light intensity received by the edge area of the polarization sensor 300 image area is smaller than that of the center, resulting in inconsistent brightness between the center and the four corners. The double-rate synchronous dynamic random access memory is used to store the original image after successively passing through the camera serial interface decoder, black level module and lens shading correction module. Please refer to Figure 18, the Binning Algorithm module can retrieve the original image from the DDP and execute steps 011, 012, 013, 014, 015, 0151, and 0153 to convert the original image into a Bayer image. Bad pixel compensation is bad pixel compensation, which refers to bad pixel correction. When a pixel unit 321 in the polarization sensor 300 cannot work normally, the pixel value corresponding to the pixel unit 321 can be obtained by using bad pixel correction. The demosaicing module can interpolate the Bayer image of the Bayer array into an RGB image, wherein the RGB image can be three frames, namely, the R image, the G image and the B image obtained after interpolation. The color saturation of the image after color correction is more obvious, which is more in line with the perception of the human eye. Gamma correction can make the image look more in line with the characteristics of the human eye. For example, the gamma correction formula is Out=Ingamma or Out=In1/gamma, where In represents the input image and Out represents the output image. The color conversion module can convert RGB images to YUV images.
需要指出的是,在图16的实施方式中,流程示意图中包括步骤011和步骤031;在某些实施方式中,流程示意图可包括步骤012和步骤031;在某些实施方式中,流程示意图可包括步骤013、步骤014、步骤015和步骤031;在某些实施方式中,流程示意图可包括步骤013、步骤014、步骤0151、步骤0153和步骤031,在此不作限定。It should be noted that, in the embodiment of FIG. 16 , the schematic flowchart includes step 011 and step 031; in some embodiments, the schematic flowchart may include step 012 and step 031; in some embodiments, the schematic flowchart may Including step 013, step 014, step 015 and step 031; in some embodiments, the schematic flowchart may include step 013, step 014, step 0151, step 0153 and step 031, which is not limited herein.
请参阅图19和图20,在某些实施方式中,当前场景包括敞亮场景,步骤01包括:Please refer to FIG. 19 and FIG. 20. In some embodiments, the current scene includes a bright scene, and step 01 includes:
016:在当前场景为敞亮场景的情况下,采用拜耳图像再生成(Remosaic)算法对偏振传感器300输出的原始图像进行转换以获得述拜耳图像。016: In the case that the current scene is a bright scene, use a Bayer image regeneration (Remosaic) algorithm to convert the original image output by the polarization sensor 300 to obtain the Bayer image.
上述实施方式的图像处理方法可由本申请实施方式的图像处理装置100实现。具体地,转换模块10用于在当前场景为敞亮场景的情况下,采用拜耳图像再生成算法对偏振传感器300输出的原始图像进行转换以获得述拜耳图像。The image processing methods of the above embodiments can be implemented by the image processing apparatus 100 of the embodiments of the present application. Specifically, the conversion module 10 is configured to convert the original image output by the polarization sensor 300 by using a Bayer image regeneration algorithm to obtain the Bayer image when the current scene is a bright scene.
上述实施方式的图像处理方法可由本申请实施方式的电子设备200实现。具体地,处理器用于在当前场景为敞亮场景的情况下,采用拜耳图像再生成算法对偏振传感器300输出的原始图像进行转换以获得述拜耳图像。The image processing method of the above embodiment can be implemented by the electronic device 200 of the embodiment of the present application. Specifically, the processor is configured to convert the original image output by the polarization sensor 300 by using a Bayer image regeneration algorithm to obtain the Bayer image when the current scene is a bright scene.
如此,在当前场景为敞亮场景的情况下,采用Remosaic算法可以获得分辨率较高的拜耳图像。In this way, when the current scene is a bright scene, a Bayer image with a higher resolution can be obtained by using the Remosaic algorithm.
具体地,敞亮场景可包括晴天室外。在某些实施方式中,在当前场景为敞亮场景的情况下,采用 Remosaic算法可以将原始图像转换为具有特定偏振方向的分辨率较高的拜耳图像。在某些实施方式中,在当前场景为敞亮场景的情况下,采用Remosaic算法可以将原始图像转换为无偏振信息的分辨率较高的拜耳图像。在当前场景为敞亮场景的情况下,获得的拜耳图像为特定偏振图像或全尺寸无偏振图像,获得的拜耳图像的像素数量与原始图像的像素数量相同,从而拜耳图像的分辨率较高。Specifically, the bright scene may include a sunny day outdoors. In some embodiments, when the current scene is a bright scene, the Remosaic algorithm can be used to convert the original image into a Bayer image with a higher resolution with a specific polarization direction. In some embodiments, when the current scene is a bright scene, the Remosaic algorithm can be used to convert the original image into a Bayer image with higher resolution without polarization information. When the current scene is a bright scene, the obtained Bayer image is a specific polarized image or a full-size unpolarized image, and the obtained Bayer image has the same number of pixels as the original image, so the Bayer image has a higher resolution.
请参阅图21-23,在某些实施方式中,步骤03包括:Referring to Figures 21-23, in some embodiments, step 03 includes:
033:将拜耳图像依次经过摄像机串行接口解码器、黑电平模块、镜头阴影校正模块、坏点补偿模块、去马赛克模块、色彩校正模块、伽玛校正模块、色彩转换模块处理后输出目标图像。033: The Bayer image is sequentially processed by the camera serial interface decoder, black level module, lens shading correction module, dead pixel compensation module, demosaicing module, color correction module, gamma correction module, and color conversion module, and then output the target image .
上述实施方式的图像处理方法可由本申请实施方式的图像处理装置100实现。具体地,输出模块30用于将拜耳图像依次经过摄像机串行接口解码器、黑电平模块、镜头阴影校正模块、坏点补偿模块、去马赛克模块、色彩校正模块、伽玛校正模块、色彩转换模块处理后输出目标图像。The image processing methods of the above embodiments can be implemented by the image processing apparatus 100 of the embodiments of the present application. Specifically, the output module 30 is used to sequentially pass the Bayer image through the camera serial interface decoder, the black level module, the lens shading correction module, the dead pixel compensation module, the demosaicing module, the color correction module, the gamma correction module, and the color conversion module. The module outputs the target image after processing.
上述实施方式的图像处理方法可由本申请实施方式的电子设备200实现。具体地,处理器用于将拜耳图像依次经过摄像机串行接口解码器、黑电平模块、镜头阴影校正模块、坏点补偿模块、去马赛克模块、色彩校正模块、伽玛校正模块、色彩转换模块处理后输出目标图像。The image processing method of the above embodiment can be implemented by the electronic device 200 of the embodiment of the present application. Specifically, the processor is configured to sequentially process the Bayer image through a camera serial interface decoder, a black level module, a lens shading correction module, a dead pixel compensation module, a demosaicing module, a color correction module, a gamma correction module, and a color conversion module Then output the target image.
如此,拜耳图像进入图像管道,经过图像管道的处理之后,转换为目标图像并输出。In this way, the Bayer image enters the image pipeline, and after being processed by the image pipeline, it is converted into the target image and output.
具体地,请结合图22,在某些实施方式中,偏振传感器(Sensor)300生成的原始图像(Quadbayer polarized RAW)经过Fullsize Algorithm模块之后,转换成拜耳阵列的拜耳图像,拜耳图像依次经过摄像机串行接口解码器、黑电平模块、镜头阴影校正模块、坏点补偿模块、去马赛克模块、色彩校正模块、伽玛校正模块、色彩转换模块处理后输出目标图像。Fullsize Algorithm模块用于执行步骤016以将原始图像转换为拜耳图像。请结合图23,在某些实施方式中,偏振传感器(Sensor)300生成的原始图像(Quadbayer polarized RAW)经过Binning Algorithm模块之后,转换成拜耳阵列的拜耳图像,拜耳图像依次经过摄像机串行接口解码器、黑电平模块、镜头阴影校正模块、坏点补偿模块、去马赛克模块、色彩校正模块、伽玛校正模块、色彩转换模块处理后输出目标图像。摄像机串行接口解码器、黑电平模块、镜头阴影校正模块、坏点补偿模块、去马赛克模块、色彩校正模块、伽玛校正模块、色彩转换模块的解释说明可以参考上述实施方式。Specifically, referring to FIG. 22 , in some embodiments, the original image (Quadbayer polarized RAW) generated by the polarization sensor (Sensor) 300 is converted into a Bayer image of a Bayer array after passing through the Fullsize Algorithm module, and the Bayer image sequentially passes through the camera string. Line interface decoder, black level module, lens shading correction module, dead pixel compensation module, demosaicing module, color correction module, gamma correction module, and color conversion module output the target image after processing. The Fullsize Algorithm module is used to perform step 016 to convert the original image to a Bayer image. Referring to FIG. 23, in some embodiments, the original image (Quadbayer polarized RAW) generated by the polarization sensor (Sensor) 300 is converted into a Bayer image of the Bayer array after passing through the Binning Algorithm module, and the Bayer image is sequentially decoded by the camera serial interface The output target image is processed by the camera, black level module, lens shading correction module, dead pixel compensation module, demosaicing module, color correction module, gamma correction module, and color conversion module. For the explanation of the camera serial interface decoder, black level module, lens shading correction module, dead pixel compensation module, demosaicing module, color correction module, gamma correction module, and color conversion module, reference may be made to the above embodiments.
需要指出的是,在图21的实施方式中,流程示意图中包括步骤016和步骤033;在某些实施方式中,流程示意图可包括步骤011和步骤033;在某些实施方式中,流程示意图可包括步骤012和步骤033;在某些实施方式中,流程示意图可包括步骤013、步骤014、步骤015和步骤033;在某些实施方式中,流程示意图可包括步骤013、步骤014、步骤0151、步骤0153和步骤033,在此不作限定。It should be noted that, in the embodiment of FIG. 21 , the schematic flowchart includes steps 016 and 033; in some embodiments, the schematic flowchart may include steps 011 and 033; in some embodiments, the schematic flowchart may Including step 012 and step 033; in some embodiments, the schematic flow diagram may include step 013, step 014, step 015 and step 033; Step 0153 and Step 033 are not limited here.
需要指出的是,上述所提到的具体数值只为了作为例子详细说明本申请的实施,而不应理解为对本申请的限制。在其他例子或实施方式或实施例中,可根据本申请来选择其他数值,在此不作具体限定。It should be pointed out that the specific numerical values mentioned above are only used as examples to describe the implementation of the present application in detail, and should not be construed as a limitation on the present application. In other examples or implementations or embodiments, other numerical values may be selected according to the present application, which are not specifically limited herein.
本申请实施方式的计算机可读存储介质,其上存储有计算机程序,程序被处理器执行的情况下,实现上述任一实施方式的图像处理方法的步骤。The computer-readable storage medium of the embodiments of the present application stores a computer program thereon, and when the program is executed by a processor, the steps of the image processing method of any of the above-mentioned embodiments are implemented.
例如,程序被处理器执行的情况下,实现以下图像处理方法的步骤:For example, when the program is executed by the processor, the steps of the following image processing method are implemented:
01:根据当前场景,对原始图像进行转换以获得拜耳图像,拜耳图像的像素呈拜耳阵列排布;01: According to the current scene, convert the original image to obtain a Bayer image, and the pixels of the Bayer image are arranged in a Bayer array;
03:根据拜耳图像输出目标图像。03: Output the target image according to the Bayer image.
可以理解,计算机程序包括计算机程序代码。计算机程序代码可以为源代码形式、对象代码形式、可执行文件或某些中间形式等。计算机可读存储介质可以包括:能够携带计算机程序代码的任何实体或装置、记录介质、U盘、移动硬盘、磁碟、光盘、计算机存储器、只读存储器(ROM,Read-Only Memory)、随机存取存储器(RAM,Random Access Memory)、以及软件分发介质等。处理器可以是中央处理器,还可以是其他通用处理器、数字信号处理器(Digital Signal Processor,DSP)、专用集成电路(Application Specific Integrated Circuit,ASIC)、现成可编程门阵列(Field-Programmable Gate Array,FPGA)或者其他 可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件等。It is understood that a computer program includes computer program code. The computer program code may be in source code form, object code form, an executable file or some intermediate form, or the like. Computer-readable storage media may include: any entity or device capable of carrying computer program codes, recording media, U disk, removable hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random storage Access memory (RAM, Random Access Memory), and software distribution media, etc. The processor can be a central processing unit, or other general-purpose processors, digital signal processors (Digital Signal Processors, DSPs), application specific integrated circuits (Application Specific Integrated Circuits, ASICs), ready-made programmable gate arrays (Field-Programmable Gate arrays) Array, FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.
在本说明书的描述中,参考术语“一个实施例”、“一些实施例”、“示例”、“具体示例”、或“一些示例”等的描述意指结合该实施例或示例描述的具体特征、结构、材料或者特点包含于本申请的至少一个实施例或示例中。在本说明书中,对上述术语的示意性表述不必须针对的是相同的实施例或示例。而且,描述的具体特征、结构、材料或者特点可以在任一个或多个实施例或示例中以合适的方式结合。此外,在不相互矛盾的情况下,本领域的技术人员可以将本说明书中描述的不同实施例或示例以及不同实施例或示例的特征进行结合和组合。In the description of this specification, description with reference to the terms "one embodiment," "some embodiments," "example," "specific example," or "some examples", etc., mean specific features described in connection with the embodiment or example , structure, material or feature is included in at least one embodiment or example of the present application. In this specification, schematic representations of the above terms are not necessarily directed to the same embodiment or example. Furthermore, the particular features, structures, materials or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. Furthermore, those skilled in the art may combine and combine the different embodiments or examples described in this specification, as well as the features of the different embodiments or examples, without conflicting each other.
流程图中或在此以其他方式描述的任何过程或方法描述可以被理解为,表示包括一个或更多个用于实现特定逻辑功能或过程的步骤的可执行指令的代码的模块、片段或部分,并且本申请的优选实施方式的范围包括另外的实现,其中可以不按所示出或讨论的顺序,包括根据所涉及的功能按基本同时的方式或按相反的顺序,来执行功能,这应被本申请的实施例所属技术领域的技术人员所理解。Any description of a process or method in the flowcharts or otherwise described herein may be understood to represent a module, segment or portion of code comprising one or more executable instructions for implementing a specified logical function or step of the process , and the scope of the preferred embodiments of the present application includes alternative implementations in which the functions may be performed out of the order shown or discussed, including performing the functions substantially concurrently or in the reverse order depending upon the functions involved, which should It is understood by those skilled in the art to which the embodiments of the present application belong.
尽管上面已经示出和描述了本申请的实施例,可以理解的是,上述实施例是示例性的,不能理解为对本申请的限制,本领域的普通技术人员在本申请的范围内可以对上述实施例进行变化、修改、替换和变型。Although the embodiments of the present application have been shown and described above, it should be understood that the above embodiments are exemplary and should not be construed as limitations to the present application. Embodiments are subject to variations, modifications, substitutions and variations.
Claims (25)
- 一种图像处理方法,用于偏振传感器,其中,所述偏振传感器包括偏振片阵列和像素阵列,所述偏振片阵列包括多个偏振片模块,每个所述偏振片模块包括四个偏振片组,每个所述偏振片模块中的四个所述偏振片组呈拜耳阵列排布,每个所述偏振片组包括多个偏振片,每个所述偏振片组中的各个所述偏振片的偏振方向互不相同,每个所述偏振片组中的各个所述偏振片的颜色分量相同,所述像素阵列包括多个像素单元,多个所述像素单元和多个所述偏振片一一对应设置,所述像素单元用于接收透过所述偏振片的光线以生成电信号,所述偏振传感器用于根据所述电信号生成原始图像,所述原始图像的像素呈非拜耳阵列排布,所述图像处理方法包括:An image processing method for a polarization sensor, wherein the polarization sensor includes a polarizer array and a pixel array, the polarizer array includes a plurality of polarizer modules, each of the polarizer modules includes four polarizer groups , the four polarizer groups in each of the polarizer modules are arranged in a Bayer array, each of the polarizer groups includes a plurality of polarizers, and each of the polarizers in each of the polarizer groups The polarization directions are different from each other, the color components of the polarizers in each polarizer group are the same, and the pixel array includes a plurality of pixel units, a plurality of the pixel units and a plurality of the polarizers. In a corresponding arrangement, the pixel unit is configured to receive the light passing through the polarizer to generate an electrical signal, the polarization sensor is configured to generate an original image according to the electrical signal, and the pixels of the original image are arranged in a non-Bayer array. cloth, the image processing method includes:根据当前场景,对所述原始图像进行转换以获得拜耳图像,所述拜耳图像的像素呈拜耳阵列排布;According to the current scene, the original image is converted to obtain a Bayer image, and the pixels of the Bayer image are arranged in a Bayer array;根据所述拜耳图像输出目标图像。A target image is output based on the Bayer image.
- 根据权利要求1所述的图像处理方法,其中,所述当前场景包括偏亮场景,所述根据当前场景,对所述原始图像进行转换以获得拜耳图像,包括:The image processing method according to claim 1, wherein the current scene includes a bright scene, and converting the original image to obtain a Bayer image according to the current scene comprises:在所述当前场景为所述偏亮场景的情况下,抽取每个所述偏振片组中具有相同偏振方向的所述偏振片对应的原始图像的像素并对抽取的所述像素进行组合,以获得所述拜耳图像。In the case where the current scene is the bright scene, extracting the pixels of the original image corresponding to the polarizers having the same polarization direction in each of the polarizer groups, and combining the extracted pixels to obtain Obtain the Bayer image.
- 根据权利要求1所述的图像处理方法,其中,所述当前场景包括普通场景,所述根据当前场景,对所述原始图像进行转换以获得拜耳图像,包括:The image processing method according to claim 1, wherein the current scene includes a common scene, and converting the original image according to the current scene to obtain a Bayer image comprises:在所述当前场景为所述普通场景的情况下,融合每个所述偏振片组中各个所述偏振片对应的原始图像的像素并对融合后的像素进行组合,以获得所述拜耳图像。In the case that the current scene is the common scene, the pixels of the original image corresponding to each of the polarizers in each polarizer group are fused, and the fused pixels are combined to obtain the Bayer image.
- 根据权利要求1所述的图像处理方法,其中,所述当前场景包括整体偏暗局部偏亮场景,所述根据当前场景,对所述原始图像进行转换以获得拜耳图像,包括:The image processing method according to claim 1, wherein the current scene includes an overall dark scene and a partially bright scene, and converting the original image to obtain a Bayer image according to the current scene includes:在所述当前场景为所述整体偏暗局部偏亮场景的情况下,抽取每个所述偏振片组中具有相同偏振方向的所述偏振片对应的原始图像的像素并对抽取的所述像素进行组合,以获得单偏振图像;In the case that the current scene is the overall dark and partially bright scene, extract the pixels of the original image corresponding to the polarizers with the same polarization direction in each polarizer group, and analyze the extracted pixels combined to obtain a single polarization image;融合每个所述偏振片组中各个所述偏振片对应的原始图像的像素并对融合后的像素进行组合,以获得无偏振图像;fusing the pixels of the original image corresponding to each of the polarizers in each of the polarizer groups and combining the fused pixels to obtain an unpolarized image;融合所述单偏振图像和所述无偏振图像以获得所述拜耳图像。The single polarization image and the unpolarized image are fused to obtain the Bayer image.
- 根据权利要求4所述的图像处理方法,其中,所述融合所述单偏振图像和所述无偏振图像以获得所述拜耳图像,包括:The image processing method according to claim 4, wherein the fusing the single-polarization image and the non-polarization image to obtain the Bayer image comprises:对所述单偏振图像和所述无偏振图像进行滤波;filtering the single polarization image and the unpolarized image;融合滤波后的所述单偏振图像和所述无偏振图像以获得所述拜耳图像。The filtered single polarization image and the unpolarized image are fused to obtain the Bayer image.
- 根据权利要求2-5中任一项所述的图像处理方法,其中,所述根据所述拜耳图像输出目标图像,包括:The image processing method according to any one of claims 2-5, wherein the outputting the target image according to the Bayer image comprises:将所述拜耳图像依次经过坏点补偿模块、去马赛克模块、色彩校正模块、伽玛校正模块、色彩转换模块处理后输出所述目标图像。The Bayer image is sequentially processed by a dead pixel compensation module, a demosaic module, a color correction module, a gamma correction module, and a color conversion module to output the target image.
- 根据权利要求1所述的图像处理方法,其中,所述当前场景包括敞亮场景,所述根据当前场景,对所述原始图像进行转换以获得拜耳图像,包括:The image processing method according to claim 1, wherein the current scene includes a bright scene, and converting the original image according to the current scene to obtain a Bayer image comprises:在所述当前场景为所述敞亮场景的情况下,采用拜耳图像再生成算法对所述偏振传感器输出的原始图像进行转换以获得所述拜耳图像。When the current scene is the bright scene, a Bayer image regeneration algorithm is used to convert the original image output by the polarization sensor to obtain the Bayer image.
- 根据权利要求2-5、7中任一项所述的图像处理方法,其中,所述根据所述拜耳图像输出目标图像,包括:The image processing method according to any one of claims 2-5 and 7, wherein the outputting the target image according to the Bayer image comprises:将所述拜耳图像依次经过摄像机串行接口解码器、黑电平模块、镜头阴影校正模块、坏点补偿模块、去马赛克模块、色彩校正模块、伽玛校正模块、色彩转换模块处理后输出所述目标图像。The Bayer image is sequentially processed by a camera serial interface decoder, a black level module, a lens shading correction module, a dead pixel compensation module, a demosaicing module, a color correction module, a gamma correction module, and a color conversion module to output the described Bayer image. target image.
- 一种图像处理装置,用于偏振传感器,其中,所述偏振传感器包括偏振片阵列和像素阵列,所述偏振片阵列包括多个偏振片模块,每个所述偏振片模块包括四个偏振片组,每个所述偏振片模块中的四个所述偏振片组呈拜耳阵列排布,每个所述偏振片组包括多个偏振片,每个所述偏振片组中的各个所述偏振片的偏振方向互不相同,每个所述偏振片组中的各个所述偏振片的颜色分量相同,所述像素阵列包括多个像素单元,多个所述像素单元和多个所述偏振片一一对应设置,所述像素单元用于接收透过所述偏振片的光线以生成电信号,所述偏振传感器用于根据所述电信号生成原始图像,所述原始图像的像素呈非拜耳阵列排布,所述图像处理装置包括:An image processing device for a polarization sensor, wherein the polarization sensor includes a polarizer array and a pixel array, the polarizer array includes a plurality of polarizer modules, each of the polarizer modules includes four polarizer groups , the four polarizer groups in each of the polarizer modules are arranged in a Bayer array, each of the polarizer groups includes a plurality of polarizers, and each of the polarizers in each of the polarizer groups The polarization directions are different from each other, the color components of the polarizers in each polarizer group are the same, and the pixel array includes a plurality of pixel units, a plurality of the pixel units and a plurality of the polarizers. In a corresponding arrangement, the pixel unit is configured to receive the light passing through the polarizer to generate an electrical signal, the polarization sensor is configured to generate an original image according to the electrical signal, and the pixels of the original image are arranged in a non-Bayer array. cloth, the image processing device includes:转换模块,用于根据当前场景,对所述原始图像进行转换以获得拜耳图像,所述拜耳图像的像素呈拜耳阵列排布;a conversion module, configured to convert the original image according to the current scene to obtain a Bayer image, the pixels of the Bayer image are arranged in a Bayer array;输出模块,用于根据所述拜耳图像输出目标图像。An output module, configured to output a target image according to the Bayer image.
- 根据权利要求9所述的图像处理装置,其中,所述当前场景包括偏亮场景,所述转换模块还用于:The image processing apparatus according to claim 9, wherein the current scene includes a bright scene, and the conversion module is further configured to:在所述当前场景为所述偏亮场景的情况下,抽取每个所述偏振片组中具有相同偏振方向的所述偏振片对应的原始图像的像素并对抽取的所述像素进行组合,以获得所述拜耳图像。In the case where the current scene is the bright scene, extracting the pixels of the original image corresponding to the polarizers having the same polarization direction in each of the polarizer groups, and combining the extracted pixels to obtain Obtain the Bayer image.
- 根据权利要求9所述的图像处理装置,其中,所述当前场景包括普通场景,所述转换模块还用于:The image processing apparatus according to claim 9, wherein the current scene includes a common scene, and the conversion module is further configured to:在所述当前场景为所述普通场景的情况下,融合每个所述偏振片组中各个所述偏振片对应的原始图像的像素并对融合后的像素进行组合,以获得所述拜耳图像。In the case that the current scene is the common scene, the pixels of the original image corresponding to each of the polarizers in each polarizer group are fused, and the fused pixels are combined to obtain the Bayer image.
- 根据权利要求9所述的图像处理装置,其中,所述当前场景包括整体偏暗局部偏亮场景,所述转换模块包括:The image processing apparatus according to claim 9, wherein the current scene comprises an overall dark and partially bright scene, and the conversion module comprises:抽取单元,用于在所述当前场景为所述整体偏暗局部偏亮场景的情况下,抽取每个所述偏振片组中具有相同偏振方向的所述偏振片对应的原始图像的像素并对抽取的所述像素进行组合,以获得单偏振图像;The extraction unit is configured to extract the pixels of the original image corresponding to the polarizers having the same polarization direction in each of the polarizer groups in the case that the current scene is the overall darker and partially brighter scene, and analyze the The extracted pixels are combined to obtain a single polarization image;第一融合单元,用于融合每个所述偏振片组中各个所述偏振片对应的原始图像的像素并对融合后的像素进行组合,以获得无偏振图像;a first fusion unit, configured to fuse the pixels of the original image corresponding to each of the polarizers in each of the polarizer groups and combine the fused pixels to obtain an unpolarized image;第二融合单元,用于融合所述单偏振图像和所述无偏振图像以获得所述拜耳图像。A second fusion unit, configured to fuse the single-polarization image and the non-polarization image to obtain the Bayer image.
- 根据权利要求12所述的图像处理装置,其中,所述第二融合单元包括:The image processing apparatus according to claim 12, wherein the second fusion unit comprises:滤波子单元,用于对所述单偏振图像和所述无偏振图像进行滤波;a filtering subunit, configured to filter the single-polarization image and the non-polarization image;融合子单元,用于融合滤波后的所述单偏振图像和所述无偏振图像以获得所述拜耳图像。A fusion subunit, configured to fuse the filtered single-polarization image and the non-polarization image to obtain the Bayer image.
- 根据权利要求10-13所述的图像处理装置,其中,所述输出模块还用于:The image processing device according to claims 10-13, wherein the output module is further configured to:将所述拜耳图像依次经过坏点补偿模块、去马赛克模块、色彩校正模块、伽玛校正模块、色彩转换模块处理后输出所述目标图像。The Bayer image is sequentially processed by a dead pixel compensation module, a demosaic module, a color correction module, a gamma correction module, and a color conversion module to output the target image.
- 根据权利要求9所述的图像处理装置,其中,所述当前场景包括敞亮场景,所述转换模块还用于:The image processing apparatus according to claim 9, wherein the current scene includes a bright scene, and the conversion module is further configured to:在所述当前场景为所述敞亮场景的情况下,采用拜耳图像再生成算法对所述偏振传感器输出的原始图像进行转换以获得所述拜耳图像。When the current scene is the bright scene, a Bayer image regeneration algorithm is used to convert the original image output by the polarization sensor to obtain the Bayer image.
- 根据权利要求10-13、15所述的图像处理装置,其中,所述输出模块还用于:The image processing apparatus according to claims 10-13 and 15, wherein the output module is further configured to:将所述拜耳图像依次经过摄像机串行接口解码器、黑电平模块、镜头阴影校正模块、坏点补偿模块、去马赛克模块、色彩校正模块、伽玛校正模块、色彩转换模块处理后输出所述目标图像。The Bayer image is sequentially processed by a camera serial interface decoder, a black level module, a lens shading correction module, a dead pixel compensation module, a demosaicing module, a color correction module, a gamma correction module, and a color conversion module to output the described Bayer image. target image.
- 一种电子设备,其中,偏振传感器包括偏振片阵列和像素阵列,所述偏振片阵列包括多个偏振片模块,每个所述偏振片模块包括四个偏振片组,每个所述偏振片模块中的四个所述偏振片组呈拜耳阵 列排布,每个所述偏振片组包括多个偏振片,每个所述偏振片组中的各个所述偏振片的偏振方向互不相同,每个所述偏振片组中的各个所述偏振片的颜色分量相同,所述像素阵列包括多个像素单元,多个所述像素单元和多个所述偏振片一一对应设置,所述像素单元用于接收透过所述偏振片的光线以生成电信号,所述偏振传感器用于根据所述电信号生成原始图像,所述原始图像的像素呈非拜耳阵列排布,所述电子设备包括一个或多个处理器和存储器,所述存储器存储有计算机程序,所述计算机程序被所述处理器执行的情况下,所述处理器用于:An electronic device, wherein a polarization sensor includes a polarizer array and a pixel array, the polarizer array includes a plurality of polarizer modules, each of the polarizer modules includes four polarizer groups, each of the polarizer modules Four of the polarizer groups are arranged in a Bayer array, each of the polarizer groups includes a plurality of polarizers, and the polarization directions of the polarizers in each of the polarizer groups are different from each other, and each polarizer group has a different polarization direction. The color components of each of the polarizers in each of the polarizer groups are the same, the pixel array includes a plurality of pixel units, and a plurality of the pixel units and a plurality of the polarizers are arranged in a one-to-one correspondence, and the pixel units is used for receiving light transmitted through the polarizer to generate an electrical signal, the polarization sensor is used for generating an original image according to the electrical signal, the pixels of the original image are arranged in a non-Bayer array, and the electronic device includes a or a plurality of processors and a memory, the memory storing a computer program, where the computer program is executed by the processor, the processor is used to:根据当前场景,对所述原始图像进行转换以获得拜耳图像,所述拜耳图像的像素呈拜耳阵列排布;According to the current scene, the original image is converted to obtain a Bayer image, and the pixels of the Bayer image are arranged in a Bayer array;根据所述拜耳图像输出目标图像。A target image is output based on the Bayer image.
- 根据权利要求17所述的电子设备,其中,所述当前场景包括偏亮场景,所述处理器还用于:The electronic device according to claim 17, wherein the current scene includes a bright scene, and the processor is further configured to:在所述当前场景为所述偏亮场景的情况下,抽取每个所述偏振片组中具有相同偏振方向的所述偏振片对应的原始图像的像素并对抽取的所述像素进行组合,以获得所述拜耳图像。In the case where the current scene is the bright scene, extracting the pixels of the original image corresponding to the polarizers having the same polarization direction in each of the polarizer groups, and combining the extracted pixels to obtain Obtain the Bayer image.
- 根据权利要求17所述的图像处理方法,其中,所述当前场景包括普通场景,所述处理器还用于:The image processing method according to claim 17, wherein the current scene includes a common scene, and the processor is further configured to:在所述当前场景为所述普通场景的情况下,融合每个所述偏振片组中各个所述偏振片对应的原始图像的像素并对融合后的像素进行组合,以获得所述拜耳图像。In the case that the current scene is the common scene, the pixels of the original image corresponding to each of the polarizers in each polarizer group are fused, and the fused pixels are combined to obtain the Bayer image.
- 根据权利要求17所述的图像处理方法,其中,所述当前场景包括整体偏暗局部偏亮场景,所述处理器还用于:The image processing method according to claim 17, wherein the current scene comprises an overall dark scene and a partially bright scene, and the processor is further configured to:在所述当前场景为所述整体偏暗局部偏亮场景的情况下,抽取每个所述偏振片组中具有相同偏振方向的所述偏振片对应的原始图像的像素并对抽取的所述像素进行组合,以获得单偏振图像;In the case that the current scene is the overall dark and partially bright scene, extract the pixels of the original image corresponding to the polarizers with the same polarization direction in each polarizer group, and analyze the extracted pixels combined to obtain a single polarization image;融合每个所述偏振片组中各个所述偏振片对应的原始图像的像素并对融合后的像素进行组合,以获得无偏振图像;fusing the pixels of the original image corresponding to each of the polarizers in each of the polarizer groups and combining the fused pixels to obtain an unpolarized image;融合所述单偏振图像和所述无偏振图像以获得所述拜耳图像。The single polarization image and the unpolarized image are fused to obtain the Bayer image.
- 根据权利要求20所述的图像处理方法,其中,所述处理器还用于:The image processing method according to claim 20, wherein the processor is further used for:对所述单偏振图像和所述无偏振图像进行滤波;filtering the single polarization image and the unpolarized image;融合滤波后的所述单偏振图像和所述无偏振图像以获得所述拜耳图像。The filtered single polarization image and the unpolarized image are fused to obtain the Bayer image.
- 根据权利要求18-21中任一项所述的图像处理方法,其中,所述处理器还用于:The image processing method according to any one of claims 18-21, wherein the processor is further configured to:将所述拜耳图像依次经过坏点补偿模块、去马赛克模块、色彩校正模块、伽玛校正模块、色彩转换模块处理后输出所述目标图像。The Bayer image is sequentially processed by a dead pixel compensation module, a demosaic module, a color correction module, a gamma correction module, and a color conversion module to output the target image.
- 根据权利要求17所述的图像处理方法,其中,所述处理器还用于:The image processing method according to claim 17, wherein the processor is further configured to:在所述当前场景为所述敞亮场景的情况下,采用拜耳图像再生成算法对所述偏振传感器输出的原始图像进行转换以获得所述拜耳图像。When the current scene is the bright scene, a Bayer image regeneration algorithm is used to convert the original image output by the polarization sensor to obtain the Bayer image.
- 根据权利要求18-21、23中任一项所述的图像处理方法,其中,所述处理器还用于:The image processing method according to any one of claims 18-21 and 23, wherein the processor is further configured to:将所述拜耳图像依次经过摄像机串行接口解码器、黑电平模块、镜头阴影校正模块、坏点补偿模块、去马赛克模块、色彩校正模块、伽玛校正模块、色彩转换模块处理后输出所述目标图像。The Bayer image is sequentially processed by a camera serial interface decoder, a black level module, a lens shading correction module, a dead pixel compensation module, a demosaicing module, a color correction module, a gamma correction module, and a color conversion module to output the described Bayer image. target image.
- 一种计算机可读存储介质,其上存储有计算机程序,其中,所述程序被处理器执行的情况下,实现权利要求1-8任一项所述的图像处理方法的步骤。A computer-readable storage medium on which a computer program is stored, wherein, when the program is executed by a processor, the steps of the image processing method according to any one of claims 1-8 are implemented.
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202110442129.8 | 2021-04-23 | ||
CN202110442129.8A CN113516607B (en) | 2021-04-23 | 2021-04-23 | Image processing method, image processing apparatus, electronic device, and storage medium |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2022222634A1 true WO2022222634A1 (en) | 2022-10-27 |
Family
ID=78061574
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/CN2022/079998 WO2022222634A1 (en) | 2021-04-23 | 2022-03-09 | Image processing method, image processing apparatus, electronic device, and storage medium |
Country Status (2)
Country | Link |
---|---|
CN (1) | CN113516607B (en) |
WO (1) | WO2022222634A1 (en) |
Families Citing this family (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN113516607B (en) * | 2021-04-23 | 2024-02-13 | Oppo广东移动通信有限公司 | Image processing method, image processing apparatus, electronic device, and storage medium |
CN115118859A (en) * | 2022-06-27 | 2022-09-27 | 联想(北京)有限公司 | Electronic device and processing method |
CN115272138B (en) * | 2022-09-28 | 2023-02-21 | 荣耀终端有限公司 | Image processing method and related device |
Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20110063482A1 (en) * | 2009-09-15 | 2011-03-17 | Samsung Electronics Co., Ltd. | Image sensor for outputting rgb bayer signal through internal conversion and image processing apparatus including the same |
CN110675404A (en) * | 2019-09-03 | 2020-01-10 | RealMe重庆移动通信有限公司 | Image processing method, image processing apparatus, storage medium, and terminal device |
CN110740277A (en) * | 2019-10-29 | 2020-01-31 | Oppo广东移动通信有限公司 | Image sensor, electronic device and imaging method |
CN112291479A (en) * | 2020-11-23 | 2021-01-29 | Oppo(重庆)智能科技有限公司 | Image processing module, image processing method, camera assembly and mobile terminal |
CN113516607A (en) * | 2021-04-23 | 2021-10-19 | Oppo广东移动通信有限公司 | Image processing method, image processing apparatus, electronic device, and storage medium |
Family Cites Families (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP5368369B2 (en) * | 2010-04-28 | 2013-12-18 | 株式会社デンソー | In-vehicle imaging device and in-vehicle image processing device |
JP5764747B2 (en) * | 2011-09-02 | 2015-08-19 | パナソニックIpマネジメント株式会社 | Polarization imaging device and endoscope |
JP6673327B2 (en) * | 2015-02-27 | 2020-03-25 | ソニー株式会社 | Image processing apparatus, image processing method, and imaging device |
WO2017175231A1 (en) * | 2016-04-07 | 2017-10-12 | Carmel Haifa University Economic Corporation Ltd. | Image dehazing and restoration |
CN112351172B (en) * | 2020-10-26 | 2021-09-17 | Oppo广东移动通信有限公司 | Image processing method, camera assembly and mobile terminal |
-
2021
- 2021-04-23 CN CN202110442129.8A patent/CN113516607B/en active Active
-
2022
- 2022-03-09 WO PCT/CN2022/079998 patent/WO2022222634A1/en active Application Filing
Patent Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20110063482A1 (en) * | 2009-09-15 | 2011-03-17 | Samsung Electronics Co., Ltd. | Image sensor for outputting rgb bayer signal through internal conversion and image processing apparatus including the same |
CN110675404A (en) * | 2019-09-03 | 2020-01-10 | RealMe重庆移动通信有限公司 | Image processing method, image processing apparatus, storage medium, and terminal device |
CN110740277A (en) * | 2019-10-29 | 2020-01-31 | Oppo广东移动通信有限公司 | Image sensor, electronic device and imaging method |
CN112291479A (en) * | 2020-11-23 | 2021-01-29 | Oppo(重庆)智能科技有限公司 | Image processing module, image processing method, camera assembly and mobile terminal |
CN113516607A (en) * | 2021-04-23 | 2021-10-19 | Oppo广东移动通信有限公司 | Image processing method, image processing apparatus, electronic device, and storage medium |
Also Published As
Publication number | Publication date |
---|---|
CN113516607B (en) | 2024-02-13 |
CN113516607A (en) | 2021-10-19 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
WO2022222634A1 (en) | Image processing method, image processing apparatus, electronic device, and storage medium | |
US10897609B2 (en) | Systems and methods for multiscopic noise reduction and high-dynamic range | |
TWI526068B (en) | Dual image capture processing | |
KR102480600B1 (en) | Method for low-light image quality enhancement of image processing devices and method of operating an image processing system for performing the method | |
US10325354B2 (en) | Depth assisted auto white balance | |
WO2021047345A1 (en) | Image noise reduction method and apparatus, and storage medium and electronic device | |
CN108712608B (en) | Terminal equipment shooting method and device | |
US9961272B2 (en) | Image capturing apparatus and method of controlling the same | |
CN107707789B (en) | Method, computing device and storage medium for providing a color high resolution image of a scene | |
US10187584B2 (en) | Dynamic range extension to produce high dynamic range images | |
WO2021184362A1 (en) | Photographing device | |
AU2016369789A1 (en) | Image sensor, imaging device, mobile terminal and imaging method | |
CN108419022A (en) | Control method, control device, computer readable storage medium and computer equipment | |
WO2018098981A1 (en) | Control method, control device, electronic device and computer-readable storage medium | |
WO2023016144A1 (en) | Focusing control method and apparatus, imaging device, electronic device, and computer readable storage medium | |
US9654756B1 (en) | Method and apparatus for interpolating pixel colors from color and panchromatic channels to color channels | |
GB2554794B (en) | Monochrome-color mapping using a monochromatic imager and a color map sensor. | |
US20160277721A1 (en) | Color filtered area processing method for improving image processing | |
KR102285756B1 (en) | Electronic system and image processing method | |
CN112714301A (en) | Dual-mode image signal processor and image sensor | |
WO2022032666A1 (en) | Image processing method and related apparatus | |
US12052516B2 (en) | Flexible region of interest color processing for cameras | |
JP2016040870A (en) | Image processing apparatus, image forming method and program | |
US20230017498A1 (en) | Flexible region of interest color processing for cameras | |
WO2021212320A1 (en) | Image zooming-out method and apparatus |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 22790743 Country of ref document: EP Kind code of ref document: A1 |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
122 | Ep: pct application non-entry in european phase |
Ref document number: 22790743 Country of ref document: EP Kind code of ref document: A1 |