WO2022057506A1 - 图像传感器、终端、数据处理方法、装置及存储介质 - Google Patents
图像传感器、终端、数据处理方法、装置及存储介质 Download PDFInfo
- Publication number
- WO2022057506A1 WO2022057506A1 PCT/CN2021/111189 CN2021111189W WO2022057506A1 WO 2022057506 A1 WO2022057506 A1 WO 2022057506A1 CN 2021111189 W CN2021111189 W CN 2021111189W WO 2022057506 A1 WO2022057506 A1 WO 2022057506A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- blue
- spectral region
- pixel unit
- light
- color filter
- Prior art date
Links
- 238000003672 processing method Methods 0.000 title claims abstract description 12
- 230000003595 spectral effect Effects 0.000 claims abstract description 255
- 230000004044 response Effects 0.000 claims description 86
- 238000001228 spectrum Methods 0.000 claims description 63
- 238000002834 transmittance Methods 0.000 claims description 44
- 238000000034 method Methods 0.000 claims description 28
- 238000006243 chemical reaction Methods 0.000 claims description 19
- 238000012545 processing Methods 0.000 claims description 10
- 238000004590 computer program Methods 0.000 claims description 9
- 238000001914 filtration Methods 0.000 claims 9
- 238000010586 diagram Methods 0.000 description 16
- 239000004065 semiconductor Substances 0.000 description 5
- 230000003287 optical effect Effects 0.000 description 4
- 230000005693 optoelectronics Effects 0.000 description 4
- 230000000638 stimulation Effects 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 3
- 230000004075 alteration Effects 0.000 description 2
- 230000000295 complement effect Effects 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 210000001525 retina Anatomy 0.000 description 2
- 238000004088 simulation Methods 0.000 description 2
- 238000012546 transfer Methods 0.000 description 2
- WFZFMHDDZRBTFH-CZEFNJPISA-N 2-[(e)-2-(5-carbamimidoyl-1-benzofuran-2-yl)ethenyl]-1-benzofuran-5-carboximidamide;dihydrochloride Chemical compound Cl.Cl.NC(=N)C1=CC=C2OC(/C=C/C=3OC4=CC=C(C=C4C=3)C(=N)N)=CC2=C1 WFZFMHDDZRBTFH-CZEFNJPISA-N 0.000 description 1
- 235000005811 Viola adunca Nutrition 0.000 description 1
- 240000009038 Viola odorata Species 0.000 description 1
- 235000013487 Viola odorata Nutrition 0.000 description 1
- 235000002254 Viola papilionacea Nutrition 0.000 description 1
- 238000009825 accumulation Methods 0.000 description 1
- 238000003491 array Methods 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 210000004556 brain Anatomy 0.000 description 1
- 239000003990 capacitor Substances 0.000 description 1
- 230000007423 decrease Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000003708 edge detection Methods 0.000 description 1
- 238000005286 illumination Methods 0.000 description 1
- 239000000463 material Substances 0.000 description 1
- 239000011159 matrix material Substances 0.000 description 1
- 229910044991 metal oxide Inorganic materials 0.000 description 1
- 150000004706 metal oxides Chemical class 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 210000005036 nerve Anatomy 0.000 description 1
- 108091008695 photoreceptors Proteins 0.000 description 1
- 230000035945 sensitivity Effects 0.000 description 1
- 239000000758 substrate Substances 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
- H04N25/10—Circuitry of solid-state image sensors [SSIS]; Control thereof for transforming different wavelengths into image signals
- H04N25/11—Arrangement of colour filter arrays [CFA]; Filter mosaics
- H04N25/13—Arrangement of colour filter arrays [CFA]; Filter mosaics characterised by the spectral characteristics of the filter elements
- H04N25/135—Arrangement of colour filter arrays [CFA]; Filter mosaics characterised by the spectral characteristics of the filter elements based on four or more different wavelength filter elements
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
- H04N25/70—SSIS architectures; Circuits associated therewith
- H04N25/702—SSIS architectures characterised by non-identical, non-equidistant or non-planar pixel layout
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/90—Dynamic range modification of images or parts thereof
- G06T5/92—Dynamic range modification of images or parts thereof based on global image properties
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/90—Determination of colour characteristics
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
- H04N25/10—Circuitry of solid-state image sensors [SSIS]; Control thereof for transforming different wavelengths into image signals
- H04N25/11—Arrangement of colour filter arrays [CFA]; Filter mosaics
- H04N25/13—Arrangement of colour filter arrays [CFA]; Filter mosaics characterised by the spectral characteristics of the filter elements
- H04N25/134—Arrangement of colour filter arrays [CFA]; Filter mosaics characterised by the spectral characteristics of the filter elements based on three different wavelength filter elements
Definitions
- the embodiments of the present application relate to the technical field of image acquisition, and in particular, to an image sensor, a terminal, a data processing method, an apparatus, and a storage medium.
- Image sensor is a device developed on the basis of optoelectronic technology that converts optical images into electrical signals.
- the pixel array of the image sensor adopts a Bayer pattern design.
- the Bayer array includes RGB (Red Green Blue, red, green and blue) three photosensitive sub-pixels.
- the Bayer array is formed repeatedly according to a 2*2 array unit.
- the 2*2 array unit includes 1 R photosensitive sub-pixel, One B-sensing sub-pixel and two G-sensing sub-pixels, the R-sensing sub-pixel is sensitive to red light, the B-sensing sub-pixel is sensitive to blue light, and the G-sensing sub-pixel is sensitive to green light.
- Embodiments of the present application provide an image sensor, a terminal, a data processing method, an apparatus, and a storage medium.
- the technical solution is as follows:
- an embodiment of the present application provides an image sensor, the image sensor includes a plurality of pixel units, and the plurality of pixel units includes:
- a fourth pixel element sensitive to the second blue spectral region is sensitive to the second blue spectral region.
- an embodiment of the present application provides a terminal, where the terminal includes the image sensor according to the above aspect.
- an embodiment of the present application provides a data processing method, the method includes:
- the image sensor includes a plurality of pixel units, and the plurality of The pixel units include: a first pixel unit sensitive to the red spectral region, a second pixel unit sensitive to the green spectral region, a third pixel unit sensitive to the first blue spectral region, and a second pixel sensitive to the blue spectral region the fourth pixel unit;
- the red, green and blue responses of the human eye are determined according to the color conversion relationship and the first red component, the first green component, the first blue component and the second blue component.
- an embodiment of the present application provides a data processing apparatus, and the apparatus includes:
- a spectrum acquisition module configured to acquire the spectrum of the first reflected light of the object irradiated by ambient light and reflected into the image sensor; wherein the image sensor includes a plurality of pixel units, and the plurality of pixel units include: a spectrum sensitive to the red spectral region a first pixel unit, a second pixel unit sensitive to the green spectral region, a third pixel unit sensitive to the first blue spectral region, and a fourth pixel unit sensitive to the second blue spectral region;
- a component determination module configured to determine the first red component perceived by the first pixel unit according to the sum of the product of the spectrum of the first reflected light and the wavelength corresponding to the spectral transmittance of the first pixel unit;
- the component determination module is further configured to determine the first green component perceived by the second pixel unit according to the sum of the product of the spectrum of the first reflected light and the wavelength corresponding to the spectral transmittance of the second pixel unit ;as well as,
- the component determination module is further configured to determine the first blue color perceived by the third pixel unit according to the sum of the products of the spectrum of the first reflected light and the wavelength corresponding to the spectral transmittance of the third pixel unit amount;
- the component determination module is further configured to determine the second blue color perceived by the fourth pixel unit according to the sum of the product of the spectrum of the first reflected light and the wavelength corresponding to the spectral transmittance of the fourth pixel unit weight;
- a response determination module configured to determine the red, green and blue responses of the human eye according to the color conversion relationship and the first red component, the first green component, the first blue component and the second blue component.
- an embodiment of the present application provides a computer-readable storage medium, where a computer program is stored in the computer-readable storage medium, and the computer program is loaded and executed by a processor to implement the data processing described in the above aspect method.
- a computer program product comprising computer instructions stored in a computer-readable storage medium.
- the processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs the data processing methods provided in the above aspects.
- FIG. 1 is a schematic diagram of the spectral responses of the red channel of the image sensor and the red cone cells of the human eye in the red band respectively in the related art;
- FIG. 2 is a schematic diagram of the spectral responses of the green channel of the image sensor and the green cone cells of the human eye in the green band respectively in the related art;
- FIG. 3 is a schematic diagram of the spectral responses of the blue channel of the image sensor and the blue cone cells of the human eye in the blue band respectively in the related art;
- FIG. 4 is a schematic diagram of a MacAdam ellipse in the related art
- FIG. 5 is a schematic diagram of an image sensor provided by an embodiment of the present application.
- FIG. 6 is a schematic diagram of the spectral responses corresponding to two blue channels and human eye blue cone cells respectively provided by an embodiment of the present application;
- FIG. 7 is a schematic diagram of a color filter array provided by an embodiment of the present application.
- FIG. 8 is a flowchart of a data processing method provided by an embodiment of the present application.
- FIG. 9 is a block diagram of a data processing apparatus provided by an embodiment of the present application.
- FIG. 1 shows a schematic diagram of the spectral response of the red channel of the image sensor and the red cone cells of the human eye in the red band (red spectral region) in the related art; as shown in Figure 2, it shows the correlation Schematic diagram of the green channel of the image sensor in the technology and the spectral response of the green cone cells of the human eye in the green band (green spectral region); Schematic diagrams of the spectral responses of blue cone cells in the blue band (blue spectral region), respectively.
- the spectral responses of the human eye cone cells in different wavelength bands in the above Figures 1, 2 and 3 are CIE (Commission Internationale de L'Eclariage, International Commission on Illumination) 1931 tristimulus values XYZ, the above spectrum Responses were directed to a 2-degree viewing angle, and other three-stimulus responses to CIE were required for other viewing angles.
- CIE Commission Internationale de L'Eclariage, International Commission on Illumination
- the red channel of the image sensor basically has no response at short wavelengths (450 nm to 500 nm)
- the corresponding CIE1931 tristimulus value X has a response, which can usually be compensated by the response of the blue channel of the sensor.
- two blue spectral regions are introduced in the embodiments of the present application, which are a first blue spectral region and a second blue spectral region, respectively.
- two blue spectral regions are realized by an image sensor, which reduces the deviation between the blue channel of the image sensor and the blue cone cells of the human eye, and improves the color accuracy of the blue region.
- FIG. 5 shows a schematic diagram of an image sensor provided by an embodiment of the present application.
- the image sensor 500 includes a plurality of pixel units, and the plurality of pixel units includes: a first pixel unit 510 sensitive to the red spectral region;
- the second pixel unit 520 is sensitive to the green spectral region;
- the third pixel unit 530 is sensitive to the first blue spectral region;
- the fourth pixel unit 540 is sensitive to the second blue spectral region.
- the image sensor 500 refers to a device for capturing images, and the image sensor 500 can convert optical signals into electrical signals.
- the red spectral region is used to characterize the spectral properties of red light
- the green spectral region is used to characterize the spectral properties of green light
- the first blue spectral region and the second blue spectral region are used to characterize the spectral properties of blue light.
- the first blue spectral region and the second blue spectral region are two different blue spectral regions, and the standard blue spectral region is respectively different from the first blue spectral region and the second blue spectral region. There are overlapping regions.
- the first blue spectral region is located in the long wavelength region of the standard blue spectral region, and the second blue spectral region is located in the low wavelength region of the standard blue spectral region.
- the first blue spectral region includes a wavelength band of 440 nm to 480 nm
- the second blue spectral region includes a band of 400 nm to 440 nm.
- the first blue spectral region and the second blue spectral region have overlapping bands, and the overlapping band belongs to a part of the spectral response band (ie, the standard blue spectral region) corresponding to the blue cone cells of the human eye band.
- the spectral response refers to the ability of the image sensor 500 to respond to light of different wavelengths, which is usually given by a spectral curve, and the spectral response represents the ability of photons of different wavelengths to generate electron-hole pairs.
- Human eye cones are a type of visual cells, located in the retina, because they can receive light stimulation and convert light energy into nerve impulses, so they are also called photoreceptors.
- the standard blue spectral region has overlapping bands with the first band in the first blue spectral region and the second band in the second blue spectral region.
- the spectral response corresponding to the third pixel unit and the spectral response corresponding to the fourth pixel unit can just represent the spectral response corresponding to the blue cone cells of the human eye.
- the response band of the spectral response corresponding to the third pixel unit further includes a third band
- the response band of the spectral response corresponding to the fourth pixel unit further includes a fourth band.
- the third band is larger than the response band of the spectral response corresponding to the blue cone cells of the human eye
- the fourth band is smaller than the response band of the spectral response corresponding to the blue cone cells of the human eye.
- FIG. 6 it shows a schematic diagram of the spectral responses corresponding to the two blue channels and the spectral responses corresponding to the blue cone cells of the human eye provided by an embodiment of the present application.
- the spectral response curve 61 corresponding to the third pixel unit and the spectral response curve 62 corresponding to the fourth pixel unit have an overlapping area (S1 area), the spectral response curve 63 corresponding to the blue cone cells of the human eye, and the spectral response curve 63 corresponding to the third pixel unit.
- the response curve 61 and the spectral response curve 62 corresponding to the fourth pixel unit overlap in the S1 area; the third band in the response band of the spectral response curve 61 corresponding to the third pixel unit is shifted to the long wavelength direction (S2 area); The fourth band in the response band of the spectral response curve 62 corresponding to the four-pixel unit is shifted to the short wavelength direction (S3 region).
- the spectral response corresponding to the blue cone cells of the human eye can be inferred.
- the spectral response corresponding to the fourth pixel unit includes a blue band and a longer blue-green band.
- the spectral response corresponding to the third pixel unit and the spectral response corresponding to the fourth pixel unit caused by the light stimulation shown in L1 can just represent the spectral response corresponding to the blue cone cells of the human eye;
- the ratio of the spectral responses corresponding to the three pixel units increases, as indicated by L2
- the spectral response corresponding to the third pixel unit and the spectral response corresponding to the fourth pixel unit caused by light stimulation which deviate from the sensitivity of the blue cone cells of the human eye It indicates that the actual spectral response corresponding to the blue cone of the human eye is smaller than the spectral response corresponding to the fourth pixel unit, and the spectral response corresponding to the fourth pixel unit needs to be corrected to predict the true blue cone of the human eye
- the corresponding spectral response of the cell When the ratio of the spectral responses corresponding to the three pixel units increases, as indicated by L2, the spectral response corresponding to the third pixel unit and the spectral response corresponding to the
- the ratio of the spectral response corresponding to the fourth pixel unit and the spectral response corresponding to the third pixel unit decreases, as shown in L3, the spectral response corresponding to the third pixel unit and the spectral response corresponding to the fourth pixel unit caused by light stimulation, It comes from the blue wavelength band to which the blue cone cells of the human eye are sensitive. At this time, the spectral response corresponding to the fourth pixel unit can still represent the spectral response corresponding to the blue cone cells of the human eye.
- the pixel unit includes a light sensor and a color filter.
- m color filter arrays are installed in front of the light sensor, the color filter array includes n*n color filters, and the n*n color filters include red color filters and green color filters.
- the number ratio of the first blue color filter and the second blue color filter is the same, m is a positive integer, and n is an integer greater than or equal to 2.
- "greater than or equal to” can be understood as greater than or equal to.
- FIG. 7 which shows a schematic diagram of a color filter array provided by an embodiment of the present application, the color filter array 700 includes 2*2 color filters as an example for description.
- the upper left corner is the red color filter 71
- the upper right corner is the first blue color filter 72
- the lower left corner is the second blue color filter 73
- the lower right corner is the green color filter 74.
- the embodiment of the present application does not limit the arrangement of the respective color filters in the red color filter, the green color filter, the first blue color filter, and the second blue color filter in the color filter array.
- Light sensor is a device developed on the basis of optoelectronic technology that converts optical images into electrical signals, and is divided into vacuum tube light sensors and semiconductor light sensors.
- Solid-state light sensors are highly integrated semiconductor optoelectronic devices, which can complete the conversion, transmission and processing of optoelectronic signals on one device.
- the core of a solid-state light sensor is a charge transfer device, including at least one of the following: CCD (Charge Coupled Device, charge coupled device), CMOS (Complementary Metal Oxide Semiconductor, complementary metal-oxide-semiconductor).
- CCD is composed of metal-oxide-semiconductor (MOS) capacitors arranged in an array on a substrate material, and has the functions of photo-generated charge, accumulation and transfer.
- the CCD can convert optical signals into electrical pulse signals. Each pulse reflects the light receiving condition of a photosensitive element, the level of the pulse amplitude reflects the strength of the light receiving element of the photosensitive element, and the sequence of output pulses can reflect the position of the photosensitive element.
- CMOS chip also includes control circuit, A/D (Analog/Digit, digital/analog) conversion and video signal processing circuit, which can perform complex processing such as brightness, contrast, noise control, and edge detection.
- the color filter includes a first color filter, a second color filter, a third color filter and a fourth color filter; wherein,
- the first color filter is used to filter out all light signals outside the red spectral region, so that only light signals associated with the red spectral region are passed.
- the second color filter is used to filter out all light signals outside the green spectral region, so that only light signals associated with the green spectral region are passed.
- the third color filter is used to filter out all light signals outside the first blue spectral region, so that only light signals associated with the first blue spectral region are passed.
- the fourth color filter is used to filter out all light signals outside the second blue spectral region, so that only light signals associated with the second blue spectral region are passed.
- the first color filter has the same meaning as the red color filter
- the second color filter has the same meaning as the green color filter
- the third color filter has the same meaning as the first blue color filter
- the fourth color filter has the same meaning as the second blue color filter.
- the light sensor includes a first light sensor, a second light sensor, a third light sensor and a fourth light sensor; wherein,
- the first light sensor and the first color filter are used to form a preset first color channel so that only light signals associated with the red spectral region are passed.
- the second light sensor and the second color filter are used to form a preset second color channel so that only light signals associated with the green spectral region are passed.
- the third light sensor and the third color filter are used to form a preset third color channel so that only the light signal associated with the first blue spectral region is passed.
- the fourth light sensor and the fourth color filter are used to form a preset fourth color channel so that only light signals associated with the second blue spectral region are passed.
- the first image is collected under the first color filter
- the second image is collected under the second color filter
- the third image is collected under the third color filter
- the fourth color filter is collected.
- a fourth image is collected, and then the above-mentioned first image, second image, third image and fourth image are processed to obtain a final collected image.
- an embodiment of the present application further provides a terminal, where the terminal includes the image sensor described in the foregoing embodiments.
- a terminal refers to a terminal with an image acquisition function.
- the terminal includes a mobile phone, a tablet computer, a PC (Personal Computer), an intelligent wearable device, an intelligent multimedia device, a camera, etc. The type is not limited.
- FIG. 8 shows a flowchart of a data processing method provided by an embodiment of the present application.
- the method may include the following steps.
- Step 801 acquiring the spectrum of the first reflected light that is reflected from the object illuminated by the ambient light and enters the image sensor.
- the image sensor includes a plurality of pixel units, and the plurality of pixel units include: a first pixel unit sensitive to the red spectral region, a second pixel unit sensitive to the green spectral region, and a first blue spectral region.
- Step 802 Determine the first red component perceived by the first pixel unit according to the sum of the products of the spectrum of the first reflected light and the wavelength corresponding to the spectral transmittance of the first pixel unit.
- the first red component R is determined by the following formula:
- ⁇ ( ⁇ ) represents the spectrum of the first reflected light
- Step 803 Determine the first green component perceived by the second pixel unit according to the sum of the products of the spectrum of the first reflected light and the wavelength corresponding to the spectral transmittance of the second pixel unit.
- the first green component G is determined by the following formula:
- ⁇ ( ⁇ ) represents the spectrum of the first reflected light
- Step 804 Determine the first blue component perceived by the third pixel unit according to the sum of the products of the spectrum of the first reflected light and the wavelength corresponding to the spectral transmittance of the third pixel unit.
- the first blue component B 1 is determined by the following formula:
- ⁇ ( ⁇ ) represents the spectrum of the first reflected light
- Step 805 Determine the second blue component perceived by the fourth pixel unit according to the sum of the products of the spectrum of the first reflected light and the wavelength corresponding to the spectral transmittance of the fourth pixel unit.
- the second blue component B 2 is determined by the following formula:
- ⁇ ( ⁇ ) represents the spectrum of the first reflected light
- steps 802 to 805 may be executed simultaneously, or may be executed sequentially in a certain order, and the present application does not limit the execution sequence of steps 802 to 805 .
- Step 806 Determine the red, green and blue responses of the human eye according to the color conversion relationship and the first red component, the first green component, the first blue component and the second blue component.
- the color conversion relationship is a color conversion matrix.
- the first red component is represented as R
- the first green component is represented as G
- the first blue component is represented as B 1
- the second blue component is represented as B 2 .
- the red, green and blue responses of the human eye can be represented as CIE1931 tristimulus values.
- XYZ at this time, XYZ can be determined by the following formula:
- the execution body of the embodiment of the present application may be a terminal, or may also be an image sensor included in the terminal.
- the terminal also includes an ISP (Image Signal Processing, image signal processor). After the image sensor determines the red, green and blue responses of the human eye, the red, green and blue responses are sent to the ISP, so that the ISP can The above red, green and blue responses are processed to obtain the final color image.
- ISP Image Signal Processing, image signal processor
- the color conversion relationship needs to be determined first.
- the transformation relationship can be determined as follows:
- the color card is an X-Rite 24-color color card.
- the standard light source is D50 light source, and the color temperature of D50 light source is 5000K, which is recognized as the standard color temperature (ICC (International Color Consortium, International Color Association) standard) by the world printing industry.
- ICC International Color Consortium, International Color Association
- the second red component perceived by the first pixel unit is determined according to the sum of the product of the spectrum of the second reflected light and the wavelength corresponding to the spectral transmittance of the first pixel unit.
- the second green component perceived by the second pixel unit is determined according to the sum of the product of the spectrum of the second reflected light and the wavelength corresponding to the spectral transmittance of the second pixel unit.
- the determination method of the second green component is similar to the determination method of the first green component. Therefore, for the determination method of the second green component, reference may be made to the above description, which will not be repeated here.
- the third blue component perceived by the third pixel unit is determined according to the sum of the product of the spectrum of the second reflected light and the wavelength corresponding to the spectral transmittance of the third pixel unit.
- the determination method of the third blue component is similar to the determination method of the first blue component. Therefore, for the determination method of the third blue component, reference may be made to the above description, which will not be repeated here.
- the fourth blue component perceived by the fourth pixel unit is determined according to the sum of the product of the spectrum of the second reflected light and the wavelength corresponding to the spectral transmittance of the fourth pixel unit.
- the determination method of the fourth blue component is similar to the determination method of the second blue component. Therefore, for the determination method of the fourth blue component, reference may be made to the above description, which will not be repeated here.
- the reflection spectrum of X-Rite 24-color color chart under D50 light source is used for simulation.
- Table 1 shows the response of the simulated CIE1931 tristimulus values for the 24-color color chart, the effect of the RGB conversion simulated CIE tristimulus values, and the effect of the RGB 1 B 2 conversion simulated CIE tristimulus values.
- DZ represents the absolute difference between the Z value of RGB or RGB 1 B 2 simulation and the standard CIE1931 Z value. It can be seen from Table 1 that the DZ mean value of the RGB 1 B 2 channel combination is smaller than that of the RGB combination, indicating that the present application can improve the accuracy of the blue-violet region.
- FIG. 9 shows a block diagram of a data processing apparatus provided by an embodiment of the present application.
- the apparatus has the function of implementing the above method example, and the function may be implemented by hardware or by executing corresponding software by hardware.
- the apparatus 900 may include: a spectrum acquisition module 910 , a component determination module 920 and a response determination module 930 .
- the spectrum acquisition module 910 is configured to acquire the spectrum of the first reflected light that is reflected from the object illuminated by the ambient light and enters the image sensor.
- the image sensor includes a plurality of pixel units, and the plurality of pixel units include: a first pixel unit sensitive to the red spectral region, a second pixel unit sensitive to the green spectral region, and sensitive to the first blue spectral region The third pixel unit, the fourth pixel unit sensitive to the second blue spectral region.
- a component determination module 920 configured to determine the first red component perceived by the first pixel unit according to the sum of the products of the spectrum of the first reflected light and the wavelength corresponding to the spectral transmittance of the first pixel unit; and ,
- the component determination module 920 is further configured to determine the first green color perceived by the second pixel unit according to the sum of the products of the spectrum of the first reflected light and the wavelength corresponding to the spectral transmittance of the second pixel unit amount; and,
- the component determination module 920 is further configured to determine the first blue color perceived by the third pixel unit according to the sum of the products of the spectrum of the first reflected light and the wavelength corresponding to the spectral transmittance of the third pixel unit. color components;
- the component determination module 920 is further configured to determine the second blue color perceived by the fourth pixel unit according to the sum of the products of the spectrum of the first reflected light and the wavelength corresponding to the spectral transmittance of the fourth pixel unit. color component.
- a response determination module 930 configured to determine the red, green and blue responses of the human eye according to the color conversion relationship and the first red component, the first green component, the first blue component and the second blue component .
- the apparatus 900 further includes: a relationship determination module (not shown in the figure).
- the relationship determination module is used for:
- the color conversion relationship is determined according to the second red component, the second green component, the third blue component, the fourth blue component and the standard red, green and blue responses corresponding to the color card.
- a computer-readable storage medium is also provided, and a computer program is stored in the computer-readable storage medium, and the computer program is loaded and executed by a processor of a computer device to implement the above data processing method individual steps in the examples.
- a computer program product includes computer instructions stored in a computer-readable storage medium.
- the processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the data processing method provided by the above aspects.
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Spectroscopy & Molecular Physics (AREA)
- Color Television Image Signal Generators (AREA)
- Spectrometry And Color Measurement (AREA)
- Color Image Communication Systems (AREA)
Abstract
本申请实施例提供了一种图像传感器、终端、数据处理方法、装置及存储介质,涉及图像采集技术领域。图像传感器包括多个像素单元,多个像素单元包括:对红色光谱区域敏感的第一像素单元;对绿色光谱区域敏感的第二像素单元;对第一蓝色光谱区域敏感的第三像素单元;对第二蓝色光谱区域敏感的第四像素单元。本申请降低了图像传感器的蓝色通道与人眼蓝视锥细胞的偏差,提升了蓝色区域的颜色准确性。
Description
本申请要求于2020年09月16日提交的申请号为202010973121.X、发明名称为“图像传感器、终端、数据处理方法、装置及存储介质”的中国专利申请的优先权,其全部内容通过引用结合在本申请中。
本申请实施例涉及图像采集技术领域,特别涉及一种图像传感器、终端、数据处理方法、装置及存储介质。
图像传感器是在光电技术基础上发展起来的将光学图像转换成电信号的器件。
在相关技术中,图像传感器的像素阵列采用拜尔阵列(Bayer Pattern)设计。拜耳阵列中包括RGB(Red Green Blue,红绿蓝)三种感光亚像素,拜耳阵列是按照2*2的阵列单元重复形成的,该2*2的阵列单元中包括1个R感光亚像素、1个B感光亚像素和2个G感光亚像素,R感光亚像素对红色光感光,B感光亚像素对蓝色光感光,G感光亚像素对绿色光感光。
发明内容
本申请实施例提供一种图像传感器、终端、数据处理方法、装置及存储介质。所述技术方案如下:
一方面,本申请实施例提供一种图像传感器,所述图像传感器包括多个像素单元,所述多个像素单元包括:
对红色光谱区域敏感的第一像素单元;
对绿色光谱区域敏感的第二像素单元;
对第一蓝色光谱区域敏感的第三像素单元;
对第二蓝色光谱区域敏感的第四像素单元。
另一方面,本申请实施例提供一种终端,所述终端包括如上述方面所述的图像传感器。
另一方面,本申请实施例提供一种数据处理方法,所述方法包括:
获取环境光照射物体经反射进入图像传感器的第一反射光的光谱获取环境光照射物体经反射进入图像传感器的第一反射光的光谱;其中,所述图像传感器包括多个像素单元,所述多个像素单元包括:对红色光谱区域敏感的第一像素单元,对绿色光谱区域敏感的第二像素单元,对第一蓝色光谱区域敏感的第三像素单元,对第二蓝色光谱区域敏感的第四像素单元;
根据所述第一反射光的光谱与所述第一像素单元的光谱透过率对应波长的乘积之和,确定所述第一像素单元感知的第一红色分量;以及,
根据所述第一反射光的光谱与所述第二像素单元的光谱透过率对应波长的乘积之和,确定所述第二像素单元感知的第一绿色分量;以及,
根据所述第一反射光的光谱与所述第三像素单元的光谱透过率对应波长的乘积之和,确定所述第三像素单元感知的第一蓝色分量;以及,
根据所述第一反射光的光谱与所述第四像素单元的光谱透过率对应波长的乘积之和,确定所述第四像素单元感知的第二蓝色分量;
根据颜色转换关系和所述第一红色分量、所述第一绿色分量、所述第一蓝色分量和所述第二蓝色分量,确定人眼的红绿蓝响应。
另一方面,本申请实施例提供一种数据处理装置,所述装置包括:
光谱获取模块,用于获取环境光照射物体经反射进入图像传感器的第一反射光的光谱;其中,所述图像传感器包括多个像素单元,所述多个像素单元包括:对红色光谱区域敏感的 第一像素单元,对绿色光谱区域敏感的第二像素单元,对第一蓝色光谱区域敏感的第三像素单元,对第二蓝色光谱区域敏感的第四像素单元;
分量确定模块,用于根据所述第一反射光的光谱与所述第一像素单元的光谱透过率对应波长的乘积之和,确定所述第一像素单元感知的第一红色分量;以及,
所述分量确定模块,还用于根据所述第一反射光的光谱与所述第二像素单元的光谱透过率对应波长的乘积之和,确定所述第二像素单元感知的第一绿色分量;以及,
所述分量确定模块,还用于根据所述第一反射光的光谱与所述第三像素单元的光谱透过率对应波长的乘积之和,确定所述第三像素单元感知的第一蓝色分量;以及,
所述分量确定模块,还用于根据所述第一反射光的光谱与所述第四像素单元的光谱透过率对应波长的乘积之和,确定所述第四像素单元感知的第二蓝色分量;
响应确定模块,用于根据颜色转换关系和所述第一红色分量、所述第一绿色分量、所述第一蓝色分量和所述第二蓝色分量,确定人眼的红绿蓝响应。
又一方面,本申请实施例提供一种计算机可读存储介质,所述计算机可读存储介质中存储有计算机程序,所述计算机程序由处理器加载并执行以实现如上述方面所述的数据处理方法。
又一方面,提供了一种计算机程序产品,该计算机程序产品包括计算机指令,该计算机指令存储在计算机可读存储介质中。计算机设备的处理器从计算机可读存储介质读取该计算机指令,处理器执行该计算机指令,使得该计算机设备执行上述方面中提供的数据处理方法。
图1是相关技术中图像传感器的红色通道与人眼红视锥细胞分别在红色波段的光谱响应的示意图;
图2是相关技术中图像传感器的绿色通道与人眼绿视锥细胞分别在绿色波段的光谱响应的示意图;
图3是相关技术中图像传感器的蓝色通道与人眼蓝视锥细胞分别在蓝色波段的光谱响应的示意图;
图4是相关技术中麦克亚当椭圆的示意图;
图5是本申请一个实施例提供的图像传感器的示意图;
图6是本申请一个实施例提供的两个蓝色通道与人眼蓝视锥细胞分别对应的光谱响应的示意图;
图7是本申请一个实施例提供的滤色片阵列的示意图;
图8是本申请一个实施例提供的数据处理方法的流程图;
图9是本申请一个实施例提供的数据处理装置的框图。
为使本申请的目的、技术方案和优点更加清楚,下面将结合附图对本申请实施方式作进一步地详细描述。
人眼视网膜上会存在三种视锥细胞分别对长中短波长敏感,即分别对红绿蓝光敏感。如图1所示,其示出了相关技术中图像传感器的红色通道与人眼红视锥细胞分别在红色波段(红色光谱区域)的光谱响应的示意图;如图2所示,其示出了相关技术中图像传感器的绿色通道与人眼绿视锥细胞分别在绿色波段(绿色光谱区域)的光谱响应的示意图;如图3所示,其示出了相关技术中图像传感器的蓝色通道与人眼蓝视锥细胞分别在蓝色波段(蓝色光谱区域)的光谱响应的示意图。需要说明的是,上述图1、图2和图3中人眼视锥细胞在不同波段的光谱响应采用的是CIE(Commission Internationale de L’Eclariage,国际照明委员会)1931三刺激值XYZ,上述光谱响应针对的是2度观察视角,如果针对其它观察视角,需要采用 CIE其他三刺激响应。相关技术中图像传感器的红色通道在短波长(450nm~500nm)基本没有响应,而对应的CIE1931三刺激值X存在响应,这通常可以通过传感器的蓝色通道的响应进行补偿。
由图1、图2和图3可知,传感器的RGB三个通道的光谱响应都与人眼视锥细胞的光谱响应(标准光谱响应)存在差异。由于人眼蓝色视锥细胞的蓝色通道的响应很窄,这一差异会导致最终图像传感器的蓝色通道响应与人眼蓝色视锥细胞的偏差更大,而且经过大脑处理之后,这一色差也更容易被人察觉。如图4所示,其示出了一种macadam(麦克亚当椭圆)的示意图,在xy颜色空间上,椭圆中心与边缘具有恰可察觉(just noticeable)色差(需要说明的是,为了便于说明,椭圆尺寸已经放大了十倍)。由图4可以看出,麦克亚当椭圆左下角蓝紫色区域内的椭圆尺寸明显小于麦克亚当椭圆其他区域内的椭圆尺寸,因此,人眼对该蓝紫色区域更敏锐,微小的色差即可被察觉。其中,x、y计算公式如下所示:
因此,图像传感器蓝色通道对应的光谱响应对最终的颜色准确性有重大影响。为了提升蓝色区域的颜色准确性,本申请实施例中引入了两种蓝色光谱区域,分别为第一蓝色光谱区域和第二蓝色光谱区域。本申请实施例通过图像传感器实现两个蓝色光谱区域,降低了图像传感器的蓝色通道与人眼蓝视锥细胞的偏差,提升了蓝色区域的颜色准确性。
请参考图5,其示出了本申请一个实施例提供的图像传感器的示意图,该图像传感器500包括多个像素单元,上述多个像素单元包括:对红色光谱区域敏感的第一像素单元510;对绿色光谱区域敏感的第二像素单元520;对第一蓝色光谱区域敏感的第三像素单元530;对第二蓝色光谱区域敏感的第四像素单元540。
图像传感器500是指用于采集图像的设备,图像传感器500可以将光信号转换为电信号。红色光谱区域用于表征红色光的光谱特性,绿色光谱区域用于表征绿色光的光谱特性,第一蓝色光谱区域和第二蓝色光谱区域用于表征蓝色光的光谱特性。
在可能的实现方式中,第一蓝色光谱区域和第二蓝色光谱区域是两个不同的蓝色光谱区域,标准蓝色光谱区域分别与第一蓝色光谱区域、第二蓝色光谱区域存在重合区域。
在可能的实现方式中,第一蓝色光谱区域位于标准蓝色光谱区域的长波长区域,第二蓝色光谱区域位于标准蓝色光谱区域的低波长区域。
在可能的实现方式中,第一蓝色光谱区域包括440nm~480nm波段,第二蓝色光谱区域包括400nm~440nm波段。
在可能的实现方式中,第一蓝色光谱区域与第二蓝色光谱区域存在重合波段,该重合波段属于人眼蓝视锥细胞对应的光谱响应波段(即标准蓝色光谱区域)中的一部分波段。光谱响应是指图像传感器500对于不同波长光线的响应能力,通常用光谱曲线给出,光谱响应表示不同波长的光子产生电子-空穴对的能力。人眼视锥细胞是视细胞的一种,位于视网膜内,因为它能接受光刺激,并将光能转换为神经冲动,故亦称光感受器。标准蓝色光谱区域,与 第一蓝色光谱区域中的第一波段,以及第二蓝色光谱区域中的第二波段存在重合波段。
在第一波段和第二波段,第三像素单元对应的光谱响应和第四像素单元对应的光谱响应恰可以表示人眼蓝视锥细胞对应的光谱响应。
在示意性实施例中,第三像素单元对应的光谱响应的响应波段还包括第三波段,第四像素单元对应的光谱响应的响应波段还包括第四波段。第三波段大于人眼蓝视锥细胞对应的光谱响应的响应波段;第四波段小于人眼蓝视锥细胞对应的光谱响应的响应波段。
如图6所示,其示出了本申请一个实施例提供的两个蓝色通道对应的光谱响应与人眼蓝视锥细胞对应的光谱响应的示意图。第三像素单元对应的光谱响应曲线61与第四像素单元对应的光谱响应曲线62存在重合区域(S1区域),人眼蓝视锥细胞对应的光谱响应曲线63,与第三像素单元对应的光谱响应曲线61和第四像素单元对应的光谱响应曲线62在S1区域存在重合;第三像素单元对应的光谱响应曲线61的响应波段中的第三波段向长波长方向(S2区域)偏移;第四像素单元对应的光谱响应曲线62的响应波段中的第四波段向短波长方向(S3区域)偏移。
示例性地,根据第三像素单元对应的光谱响应和第四像素单元对应的光谱响应,可以推断人眼蓝视锥细胞对应的光谱响应。例如,第四像素单元对应的光谱响应包含了蓝色波段以及更长的蓝绿波段。L
1所示光刺激引起的第三像素单元对应的光谱响应和第四像素单元对应的光谱响应恰可以表示人眼蓝视锥细胞对应的光谱响应;当第四像素单元对应的光谱响应与第三像素单元对应的光谱响应的比值增大时,如L
2所示光刺激引起的第三像素单元对应的光谱响应和第四像素单元对应的光谱响应,其偏离了人眼蓝视锥细胞敏感的蓝色波段,表明此时人眼蓝视锥细胞对应的实际光谱响应小于第四像素单元对应的光谱响应,需要对第四像素单元对应的光谱响应进行修正来预测真正的人眼蓝视锥细胞对应的光谱响应。当第四像素单元对应的光谱响应与第三像素单元对应的光谱响应的比值减小时,如L
3所示光刺激引起的第三像素单元对应的光谱响应和第四像素单元对应的光谱响应,其来自于人眼蓝视锥细胞敏感的蓝色波段,此时第四像素单元对应的光谱响应仍然可以表示人眼蓝视锥细胞对应的光谱响应。
综上所述,本申请实施例提供的技术方案中,通过引入两个不同的蓝色光谱区域,从而降低了图像传感器的蓝色通道与人眼蓝视锥细胞的偏差,提升了蓝色区域的颜色准确性。
在示意性实施例中,像素单元包括光传感器和滤色片。
示例性地,光传感器的前面装有m个滤色片阵列,该滤色片阵列中包括n*n个滤色片,该n*n个滤色片中红色滤色片、绿色滤色片、第一蓝色滤色片和第二蓝色滤色片个数比例相同,m为正整数,n为大于等于2的整数。在本申请中,“大于等于”可以理解为大于或等于。如图7所示,其示出了本申请一个实施例提供的滤色片阵列的示意图,以滤色片阵列700包括2*2个滤色片为例进行介绍说明,该滤色片阵列700的左上角为红色滤色片71,右上角为第一蓝色滤色片72,左下角为第二蓝色滤色片73,右下角为绿色滤色片74。本申请实施例对滤色片阵列中红色滤色片、绿色滤色片、第一蓝色滤色片和第二蓝色滤色片中各个滤色片的排列方式不作限定。
光传感器是在光电技术基础上发展起来的将光学图像转换成电信号的器件,分为真空管光传感器和半导体光传感器。固态光传感器是高度集成的半导体光电器件,在一个器件上可 以完成光电信号的转换、传输和处理。固态光传感器的核心是电荷转移器件,包括以下至少一项:CCD(Charge Coupled Device,电荷耦合器件)、CMOS(Complementary Metal Oxide Semiconductor,互补式金属-氧化物-半导体)。
CCD是以阵列形式排列在衬底材料上的金属-氧化物-半导体(Metal Oxide Semiconductor,MOS)电容器组成的,具有光生电荷、积蓄和转移的功能。CCD可以把光信号转换为电脉冲信号。每一个脉冲反应一个光敏元的受光情况,脉冲幅度的高低反映该光敏元受光的强弱,输出脉冲的顺序可以反映光敏元的位置。CMOS芯片除了具有CCD芯片的功能单元,还包括了控制电路、A/D(Analog/Digit,数/模)转换和视频信号处理电路,可以进行亮度、对比度、噪音控制、边缘检测等复杂处理。
在可能的实现方式中,滤色片包括第一滤色片、第二滤色片、第三滤色片和第四滤色片;其中,
第一滤色片,用于滤除红色光谱区域之外的所有光信号,使得仅通过与红色光谱区域相关联的光信号。
第二滤色片,用于滤除绿色光谱区域之外的所有光信号,使得仅通过与绿色光谱区域相关联的光信号。
第三滤色片,用于滤除第一蓝色光谱区域之外的所有光信号,使得仅通过与第一蓝色光谱区域相关联的光信号。
第四滤色片,用于滤除第二蓝色光谱区域之外的所有光信号,使得仅通过与第二蓝色光谱区域相关联的光信号。
需要说明的是,第一滤色片与红色滤色片的含义相同、第二滤色片与绿色滤色片的含义相同、第三滤色片与第一蓝色滤色片的含义相同、第四滤色片与第二蓝色滤色片的含义相同。
在可能的实现方式中,光传感器包括第一光传感器、第二光传感器、第三光传感器和第四光传感器;其中,
第一光传感器和第一滤色片,用于形成预设的第一颜色通道,使得仅通过与红色光谱区域相关联的光信号。
第二光传感器和第二滤色片,用于形成预设的第二颜色通道,使得仅通过与绿色光谱区域相关联的光信号。
第三光传感器和第三滤色片,用于形成预设的第三颜色通道,使得仅通过与第一蓝色光谱区域相关联的光信号。
第四光传感器和第四滤色片,用于形成预设的第四颜色通道,使得仅通过与第二蓝色光谱区域相关联的光信号。
在图像采集过程中,分别在第一滤色片下采集第一图像,在第二滤色片下采集第二图像,在第三滤色片下采集第三图像,在第四滤色片下采集第四图像,然后对上述第一图像、第二图像、第三图像和第四图像进行处理,得到最终采集的图像。
在示意性实施例中,本申请实施例还提供了一种终端,该终端包括如上述实施例所述的图像传感器。示例性地,终端是指具有图像采集功能的终端,例如,终端包括手机、平板电脑、PC(Personal Computer,个人计算机)、智能可穿戴设备、智能多媒体设备、相机等,本申请实施例对终端的类型不作限定。
请参考图8,其示出了本申请一个实施例提供的数据处理方法的流程图。该方法可以包括如下几个步骤。
步骤801,获取环境光照射物体经反射进入图像传感器的第一反射光的光谱。
在本申请实施例中,图像传感器包括多个像素单元,上述多个像素单元包括:对红色光谱区域敏感的第一像素单元,对绿色光谱区域敏感的第二像素单元,对第一蓝色光谱区域敏感的第三像素单元,对第二蓝色光谱区域敏感的第四像素单元。
有关图像传感器的介绍说明可参见上文实施例,此处不再赘述。
步骤802,根据第一反射光的光谱与第一像素单元的光谱透过率对应波长的乘积之和,确定第一像素单元感知的第一红色分量。
在可能的实现方式中,第一红色分量R通过如下公式确定:
步骤803,根据第一反射光的光谱与第二像素单元的光谱透过率对应波长的乘积之和,确定第二像素单元感知的第一绿色分量。
在可能的实现方式中,第一绿色分量G通过如下公式确定:
步骤804,根据第一反射光的光谱与第三像素单元的光谱透过率对应波长的乘积之和,确定第三像素单元感知的第一蓝色分量。
在可能的实现方式中,第一蓝色分量B
1通过如下公式确定:
步骤805,根据第一反射光的光谱与第四像素单元的光谱透过率对应波长的乘积之和,确定第四像素单元感知的第二蓝色分量。
在可能的实现方式中,第二蓝色分量B
2通过如下公式确定:
需要说明的是,步骤802至步骤805可以同时执行,也可以按照某种顺序先后执行,本申请对步骤802至步骤805的执行先后顺序不作限定。
步骤806,根据颜色转换关系和第一红色分量、第一绿色分量、第一蓝色分量和第二蓝色分量,确定人眼的红绿蓝响应。
在可能的实现方式中,颜色转换关系为颜色转换矩阵。第一红色分量表示为R,第一绿色分量表示为G、第一蓝色分量表示为B
1、第二蓝色分量表示为B
2,人眼的红绿蓝响应可表示为CIE1931三刺激值XYZ,此时,XYZ可以通过如下公式确定得到:
本申请实施例的执行主体可以是终端,或者也可以是终端中包括的图像传感器。在可能的实现方式中,终端中还包括ISP(Image Signal Processing,图像信号处理器),图像传感器确定出人眼的红绿蓝响应后,将该红绿蓝响应发送给ISP,以使得ISP对上述红绿蓝响应进行处理,得到最终的彩色图像。
综上所述,本申请实施例提供的技术方案中,通过引入两个不同的蓝色光谱区域,从而降低了图像传感器的蓝色通道与人眼蓝视锥细胞的偏差,提升了蓝色区域的颜色准确性。
在可能的实现方式中,在终端根据颜色转换关系和红色分量、绿色分量、第一蓝色分量和第二蓝色分量,确定人眼的红绿蓝响应之前,需要先确定颜色转换关系,颜色转换关系可通过如下方式确定:
第一、获取标准光源照射色卡后反射进入图像传感器的第二反射光的光谱。
在可能的实现方式中,色卡为爱色丽24色色卡。标准光源为D50光源,D50光源的色温为5000K,被世界印刷业公认为标准色温(ICC(International Color Consortium,国际色彩协会)标准),其条件恰与白天阳光下相近,色彩比较正常。
第二、根据第二反射光的光谱与第一像素单元的光谱透过率对应波长的乘积之和,确定第一像素单元感知的第二红色分量。
由于第二红色分量的确定方式与第一红色分量的确定方式类似,因此,有关第二红色分量的确定方式可参照上文的介绍说明,此处不再赘述。
第三、根据第二反射光的光谱与第二像素单元的光谱透过率对应波长的乘积之和,确定第二像素单元感知的第二绿色分量。
第二绿色分量的确定方式与第一绿色分量的确定方式类似,因此,有关第二绿色分量的确定方式可参照上文的介绍说明,此处不再赘述。
第四、根据第二反射光的光谱与第三像素单元的光谱透过率对应波长的乘积之和,确定第三像素单元感知的第三蓝色分量。
第三蓝色分量的确定方式与第一蓝色分量的确定方式类似,因此,有关第三蓝色分量的确定方式可参照上文的介绍说明,此处不再赘述。
第五、根据第二反射光的光谱与第四像素单元的光谱透过率对应波长的乘积之和,确定第四像素单元感知的第四蓝色分量。
第四蓝色分量的确定方式与第二蓝色分量的确定方式类似,因此,有关第四蓝色分量的确定方式可参照上文的介绍说明,此处不再赘述。
第六、根据第二红色分量、第二绿色分量、第三蓝色分量、第四蓝色分量和色卡对应的标准红绿蓝响应,确定颜色转换关系。
采用爱色丽24色色卡在D50光源下的反射光谱进行模拟。表1示出了24色色卡模拟计 算的CIE1931三刺激值的响应,RGB转换模拟的CIE三刺激值的影响以及RGB
1B
2转换模拟的CIE三刺激值的影响。DZ表示RGB或者RGB
1B
2模拟的Z值与标准的CIE1931Z值的绝对值差距。由表1可知,RGB
1B
2通道组合的DZ均值小于RGB组合,表明本申请可以提升蓝紫色区域的准确性。
表1
下述为本申请装置实施例,可以用于执行本申请方法实施例。对于本申请装置实施例中未披露的细节,请参照本申请方法实施例。
请参考图9,其示出了本申请一个实施例提供的数据处理装置的框图,该装置具有实现上述方法示例的功能,所述功能可以由硬件实现,也可以由硬件执行相应的软件实现。该装置900可以包括:光谱获取模块910、分量确定模块920和响应确定模块930。
光谱获取模块910,用于获取环境光照射物体经反射进入图像传感器的第一反射光的光谱。其中,所述图像传感器包括多个像素单元,所述多个像素单元包括:对红色光谱区域敏感的第一像素单元,对绿色光谱区域敏感的第二像素单元,对第一蓝色光谱区域敏感的第三像素单元,对第二蓝色光谱区域敏感的第四像素单元。
分量确定模块920,用于根据所述第一反射光的光谱与所述第一像素单元的光谱透过率对应波长的乘积之和,确定所述第一像素单元感知的第一红色分量;以及,
所述分量确定模块920,还用于根据所述第一反射光的光谱与所述第二像素单元的光谱透过率对应波长的乘积之和,确定所述第二像素单元感知的第一绿色分量;以及,
所述分量确定模块920,还用于根据所述第一反射光的光谱与所述第三像素单元的光谱透过率对应波长的乘积之和,确定所述第三像素单元感知的第一蓝色分量;以及,
所述分量确定模块920,还用于根据所述第一反射光的光谱与所述第四像素单元的光谱透过率对应波长的乘积之和,确定所述第四像素单元感知的第二蓝色分量。
响应确定模块930,用于根据颜色转换关系和所述第一红色分量、所述第一绿色分量、所述第一蓝色分量和所述第二蓝色分量,确定人眼的红绿蓝响应。
综上所述,本申请实施例提供的技术方案中,通过引入两个不同的蓝色光谱区域,从而降低了图像传感器的蓝色通道与人眼蓝视锥细胞的偏差,提升了蓝色区域的颜色准确性。
在示意性实施例中,所述装置900还包括:关系确定模块(图中未示出)。
所述关系确定模块,用于:
获取标准光源照射色卡后反射进入所述图像传感器的第二反射光的光谱;
根据所述第二反射光的光谱与所述第一像素单元的光谱透过率对应波长的乘积之和,确定所述第一像素单元感知的第二红色分量;以及,
根据所述第二反射光的光谱与所述第二像素单元的光谱透过率对应波长的乘积之和,确定所述第二像素单元感知的第二绿色分量;以及,
根据所述第二反射光的光谱与所述第三像素单元的光谱透过率对应波长的乘积之和,确定所述第三像素单元感知的第三蓝色分量;以及,
根据所述第二反射光的光谱与所述第四像素单元的光谱透过率对应波长的乘积之和,确定所述第四像素单元感知的第四蓝色分量;
根据所述第二红色分量、所述第二绿色分量、所述第三蓝色分量、所述第四蓝色分量和所述色卡对应的标准红绿蓝响应,确定所述颜色转换关系。
需要说明的是,上述实施例提供的装置在实现其功能时,仅以上述各功能模块的划分进行举例说明,实际应用中,可以根据需要而将上述功能分配由不同的功能模块完成,即将设备的内部结构划分成不同的功能模块,以完成以上描述的全部或者部分功能。另外,上述实施例提供的装置与方法实施例属于同一构思,其具体实现过程详见方法实施例,这里不再赘述。
在示例性实施例中,还提供了一种计算机可读存储介质,所述计算机可读存储介质中存储有计算机程序,所述计算机程序由计算机设备的处理器加载并执行以实现上述数据处理方法实施例中的各个步骤。
在示例性实施例中,提供了一种计算机程序产品,该计算机程序产品包括计算机指令,该计算机指令存储在计算机可读存储介质中。计算机设备的处理器从计算机可读存储介质读取该计算机指令,处理器执行该计算机指令,使得该计算机设备执行上述方面提供的数据处理方法。
以上所述仅为本申请的示例性实施例,并不用以限制本申请,凡在本申请的精神和原则之内,所作的任何修改、等同替换、改进等,均应包含在本申请的保护范围之内。
Claims (25)
- 一种图像传感器,其特征在于,所述图像传感器包括多个像素单元,所述多个像素单元包括:对红色光谱区域敏感的第一像素单元;对绿色光谱区域敏感的第二像素单元;对第一蓝色光谱区域敏感的第三像素单元;对第二蓝色光谱区域敏感的第四像素单元。
- 根据权利要求1所述的图像传感器,其特征在于,所述像素单元包括光传感器和滤色片。
- 根据权利要求2所述的图像传感器,其特征在于,所述滤色片包括第一滤色片、第二滤色片、第三滤色片和第四滤色片;其中,所述第一滤色片,用于滤除所述红色光谱区域之外的所有光信号,使得仅通过与所述红色光谱区域相关联的光信号;所述第二滤色片,用于滤除所述绿色光谱区域之外的所有光信号,使得仅通过与所述绿色光谱区域相关联的光信号;所述第三滤色片,用于滤除所述第一蓝色光谱区域之外的所有光信号,使得仅通过与所述第一蓝色光谱区域相关联的光信号;所述第四滤色片,用于滤除所述第二蓝色光谱区域之外的所有光信号,使得仅通过与所述第二蓝色光谱区域相关联的光信号。
- 根据权利要求3所述的图像传感器,其特征在于,所述光传感器包括第一光传感器、第二光传感器、第三光传感器和第四光传感器;其中,所述第一光传感器和所述第一滤色片,用于形成预设的第一颜色通道,使得仅通过与所述红色光谱区域相关联的光信号;所述第二光传感器和所述第二滤色片,用于形成预设的第二颜色通道,使得仅通过与所述绿色光谱区域相关联的光信号;所述第三光传感器和所述第三滤色片,用于形成预设的第三颜色通道,使得仅通过与所述第一蓝色光谱区域相关联的光信号;所述第四光传感器和所述第四滤色片,用于形成预设的第四颜色通道,使得仅通过与所述第二蓝色光谱区域相关联的光信号。
- 根据权利要求1至4任一项所述的图像传感器,其特征在于,所述第一蓝色光谱区域和所述第二蓝色光谱区域是两个不同的蓝色光谱区域,标准蓝色光谱区域分别与所述第一蓝色光谱区域、所述第二蓝色光谱区域存在重合区域。
- 根据权利要求5所述的图像传感器,其特征在于,所述第一蓝色光谱区域位于所述标 准蓝色光谱区域的长波长区域,所述第二蓝色光谱区域位于所述标准蓝色光谱区域的低波长区域。
- 根据权利要求6所述的图像传感器,其特征在于,所述第一蓝色光谱区域包括440nm~480nm波段,所述第二蓝色光谱区域包括400nm~440nm波段。
- 一种终端,其特征在于,所述终端包括如权利要求1至7任一项所述的图像传感器。
- 一种数据处理方法,其特征在于,所述方法包括:获取环境光照射物体经反射进入图像传感器的第一反射光的光谱;其中,所述图像传感器包括多个像素单元,所述多个像素单元包括:对红色光谱区域敏感的第一像素单元,对绿色光谱区域敏感的第二像素单元,对第一蓝色光谱区域敏感的第三像素单元,对第二蓝色光谱区域敏感的第四像素单元;根据所述第一反射光的光谱与所述第一像素单元的光谱透过率对应波长的乘积之和,确定所述第一像素单元感知的第一红色分量;以及,根据所述第一反射光的光谱与所述第二像素单元的光谱透过率对应波长的乘积之和,确定所述第二像素单元感知的第一绿色分量;以及,根据所述第一反射光的光谱与所述第三像素单元的光谱透过率对应波长的乘积之和,确定所述第三像素单元感知的第一蓝色分量;以及,根据所述第一反射光的光谱与所述第四像素单元的光谱透过率对应波长的乘积之和,确定所述第四像素单元感知的第二蓝色分量;根据颜色转换关系和所述第一红色分量、所述第一绿色分量、所述第一蓝色分量和所述第二蓝色分量,确定人眼的红绿蓝响应。
- 根据权利要求9所述的方法,其特征在于,所述根据颜色转换关系和所述红色分量、所述绿色分量、所述第一蓝色分量和所述第二蓝色分量,确定人眼的红绿蓝响应之前,还包括:获取标准光源照射色卡后反射进入所述图像传感器的第二反射光的光谱;根据所述第二反射光的光谱与所述第一像素单元的光谱透过率对应波长的乘积之和,确定所述第一像素单元感知的第二红色分量;以及,根据所述第二反射光的光谱与所述第二像素单元的光谱透过率对应波长的乘积之和,确定所述第二像素单元感知的第二绿色分量;以及,根据所述第二反射光的光谱与所述第三像素单元的光谱透过率对应波长的乘积之和,确定所述第三像素单元感知的第三蓝色分量;以及,根据所述第二反射光的光谱与所述第四像素单元的光谱透过率对应波长的乘积之和,确定所述第四像素单元感知的第四蓝色分量;根据所述第二红色分量、所述第二绿色分量、所述第三蓝色分量、所述第四蓝色分量和所述色卡对应的标准红绿蓝响应,确定所述颜色转换关系。
- 根据权利要求9所述的方法,其特征在于,所述像素单元包括光传感器和滤色片。
- 根据权利要求11所述的方法,其特征在于,所述滤色片包括第一滤色片、第二滤色片、第三滤色片和第四滤色片;其中,所述第一滤色片,用于滤除所述红色光谱区域之外的所有光信号,使得仅通过与所述红色光谱区域相关联的光信号;所述第二滤色片,用于滤除所述绿色光谱区域之外的所有光信号,使得仅通过与所述绿色光谱区域相关联的光信号;所述第三滤色片,用于滤除所述第一蓝色光谱区域之外的所有光信号,使得仅通过与所述第一蓝色光谱区域相关联的光信号;所述第四滤色片,用于滤除所述第二蓝色光谱区域之外的所有光信号,使得仅通过与所述第二蓝色光谱区域相关联的光信号。
- 根据权利要求12所述的方法,其特征在于,所述光传感器包括第一光传感器、第二光传感器、第三光传感器和第四光传感器;其中,所述第一光传感器和所述第一滤色片,用于形成预设的第一颜色通道,使得仅通过与所述红色光谱区域相关联的光信号;所述第二光传感器和所述第二滤色片,用于形成预设的第二颜色通道,使得仅通过与所述绿色光谱区域相关联的光信号;所述第三光传感器和所述第三滤色片,用于形成预设的第三颜色通道,使得仅通过与所述第一蓝色光谱区域相关联的光信号;所述第四光传感器和所述第四滤色片,用于形成预设的第四颜色通道,使得仅通过与所述第二蓝色光谱区域相关联的光信号。
- 根据权利要求9至13任一项所述的方法,其特征在于,所述第一蓝色光谱区域和所述第二蓝色光谱区域是两个不同的蓝色光谱区域,标准蓝色光谱区域分别与所述第一蓝色光谱区域、所述第二蓝色光谱区域存在重合区域。
- 根据权利要求14所述的方法,其特征在于,所述第一蓝色光谱区域位于所述标准蓝色光谱区域的长波长区域,所述第二蓝色光谱区域位于所述标准蓝色光谱区域的低波长区域。
- 根据权利要求15所述的方法,其特征在于,所述第一蓝色光谱区域包括440nm~480nm波段,所述第二蓝色光谱区域包括400nm~440nm波段。
- 一种数据处理装置,其特征在于,所述装置包括:光谱获取模块,用于获取环境光照射物体经反射进入图像传感器的第一反射光的光谱;其中,所述图像传感器包括多个像素单元,所述多个像素单元包括:对红色光谱区域敏感的第一像素单元,对绿色光谱区域敏感的第二像素单元,对第一蓝色光谱区域敏感的第三像素单元,对第二蓝色光谱区域敏感的第四像素单元;分量确定模块,用于根据所述第一反射光的光谱与所述第一像素单元的光谱透过率对应波长的乘积之和,确定所述第一像素单元感知的第一红色分量;以及,所述分量确定模块,还用于根据所述第一反射光的光谱与所述第二像素单元的光谱透过率对应波长的乘积之和,确定所述第二像素单元感知的第一绿色分量;以及,所述分量确定模块,还用于根据所述第一反射光的光谱与所述第三像素单元的光谱透过率对应波长的乘积之和,确定所述第三像素单元感知的第一蓝色分量;以及,所述分量确定模块,还用于根据所述第一反射光的光谱与所述第四像素单元的光谱透过率对应波长的乘积之和,确定所述第四像素单元感知的第二蓝色分量;响应确定模块,用于根据颜色转换关系和所述第一红色分量、所述第一绿色分量、所述第一蓝色分量和所述第二蓝色分量,确定人眼的红绿蓝响应。
- 根据权利要求17所述的装置,其特征在于,所述装置还包括关系确定模块,用于:获取标准光源照射色卡后反射进入所述图像传感器的第二反射光的光谱;根据所述第二反射光的光谱与所述第一像素单元的光谱透过率对应波长的乘积之和,确定所述第一像素单元感知的第二红色分量;以及,根据所述第二反射光的光谱与所述第二像素单元的光谱透过率对应波长的乘积之和,确定所述第二像素单元感知的第二绿色分量;以及,根据所述第二反射光的光谱与所述第三像素单元的光谱透过率对应波长的乘积之和,确定所述第三像素单元感知的第三蓝色分量;以及,根据所述第二反射光的光谱与所述第四像素单元的光谱透过率对应波长的乘积之和,确定所述第四像素单元感知的第四蓝色分量;根据所述第二红色分量、所述第二绿色分量、所述第三蓝色分量、所述第四蓝色分量和所述色卡对应的标准红绿蓝响应,确定所述颜色转换关系。
- 根据权利要求17所述的装置,其特征在于,所述像素单元包括光传感器和滤色片。
- 根据权利要求19所述的装置,其特征在于,所述滤色片包括第一滤色片、第二滤色片、第三滤色片和第四滤色片;其中,所述第一滤色片,用于滤除所述红色光谱区域之外的所有光信号,使得仅通过与所述红色光谱区域相关联的光信号;所述第二滤色片,用于滤除所述绿色光谱区域之外的所有光信号,使得仅通过与所述绿色光谱区域相关联的光信号;所述第三滤色片,用于滤除所述第一蓝色光谱区域之外的所有光信号,使得仅通过与所述第一蓝色光谱区域相关联的光信号;所述第四滤色片,用于滤除所述第二蓝色光谱区域之外的所有光信号,使得仅通过与所述第二蓝色光谱区域相关联的光信号。
- 根据权利要求20所述的装置,其特征在于,所述光传感器包括第一光传感器、第二光传感器、第三光传感器和第四光传感器;其中,所述第一光传感器和所述第一滤色片,用于形成预设的第一颜色通道,使得仅通过与所述红色光谱区域相关联的光信号;所述第二光传感器和所述第二滤色片,用于形成预设的第二颜色通道,使得仅通过与所述绿色光谱区域相关联的光信号;所述第三光传感器和所述第三滤色片,用于形成预设的第三颜色通道,使得仅通过与所述第一蓝色光谱区域相关联的光信号;所述第四光传感器和所述第四滤色片,用于形成预设的第四颜色通道,使得仅通过与所述第二蓝色光谱区域相关联的光信号。
- 根据权利要求17至21任一项所述的装置,其特征在于,所述第一蓝色光谱区域和所述第二蓝色光谱区域是两个不同的蓝色光谱区域,标准蓝色光谱区域分别与所述第一蓝色光谱区域、所述第二蓝色光谱区域存在重合区域。
- 根据权利要求22所述的装置,其特征在于,所述第一蓝色光谱区域位于所述标准蓝色光谱区域的长波长区域,所述第二蓝色光谱区域位于所述标准蓝色光谱区域的低波长区域。
- 根据权利要求23所述的装置,其特征在于,所述第一蓝色光谱区域包括440nm~480nm波段,所述第二蓝色光谱区域包括400nm~440nm波段。
- 一种计算机可读存储介质,其特征在于,所述计算机可读存储介质中存储有计算机程序,所述计算机程序由处理器加载并执行以实现如权利要求9至16任一项所述的数据处理方法。
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
EP21868334.0A EP4213479A4 (en) | 2020-09-16 | 2021-08-06 | IMAGE SENSOR, TERMINAL, DATA PROCESSING METHOD AND APPARATUS, AND RECORDING MEDIUM |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202010973121.XA CN111918005B (zh) | 2020-09-16 | 2020-09-16 | 图像传感器、终端、数据处理方法、装置及存储介质 |
CN202010973121.X | 2020-09-16 |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2022057506A1 true WO2022057506A1 (zh) | 2022-03-24 |
Family
ID=73267911
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/CN2021/111189 WO2022057506A1 (zh) | 2020-09-16 | 2021-08-06 | 图像传感器、终端、数据处理方法、装置及存储介质 |
Country Status (3)
Country | Link |
---|---|
EP (1) | EP4213479A4 (zh) |
CN (1) | CN111918005B (zh) |
WO (1) | WO2022057506A1 (zh) |
Families Citing this family (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN111918005B (zh) * | 2020-09-16 | 2023-05-16 | Oppo广东移动通信有限公司 | 图像传感器、终端、数据处理方法、装置及存储介质 |
Citations (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2006013567A (ja) * | 2004-06-22 | 2006-01-12 | Mitsubishi Electric Corp | 撮像装置及びその信号処理方法 |
CN1862339A (zh) * | 2006-05-25 | 2006-11-15 | 友达光电股份有限公司 | 高色彩表现的液晶显示器 |
US20090295962A1 (en) * | 2008-05-30 | 2009-12-03 | Omnivision | Image sensor having differing wavelength filters |
CN103004218A (zh) * | 2011-05-19 | 2013-03-27 | 松下电器产业株式会社 | 三维摄像装置、摄像元件、透光部、及图像处理装置 |
WO2020112726A1 (en) * | 2018-11-30 | 2020-06-04 | Intuitive Surgical Operations, Inc. | Medical imaging systems and methods |
CN111918005A (zh) * | 2020-09-16 | 2020-11-10 | Oppo广东移动通信有限公司 | 图像传感器、终端、数据处理方法、装置及存储介质 |
Family Cites Families (8)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US7057654B2 (en) * | 2002-02-26 | 2006-06-06 | Eastman Kodak Company | Four color image sensing apparatus |
US7333678B1 (en) * | 2003-05-20 | 2008-02-19 | Micronas Usa, Inc. | Edge adaptive demosaic system and method |
US8139130B2 (en) * | 2005-07-28 | 2012-03-20 | Omnivision Technologies, Inc. | Image sensor with improved light sensitivity |
KR100647402B1 (ko) * | 2005-11-01 | 2006-11-23 | 매그나칩 반도체 유한회사 | 이미지 센서의 화질개선장치 및 그 방법 |
US20080266563A1 (en) * | 2007-04-26 | 2008-10-30 | Redman David J | Measuring color using color filter arrays |
JP5554139B2 (ja) * | 2010-05-11 | 2014-07-23 | パナソニック株式会社 | 複合型撮像素子およびそれを備えた撮像装置 |
CN104412580B (zh) * | 2012-07-06 | 2016-04-06 | 富士胶片株式会社 | 彩色摄像元件以及摄像装置 |
WO2017193738A1 (zh) * | 2016-05-09 | 2017-11-16 | 比亚迪股份有限公司 | 图像传感器、成像方法和成像装置 |
-
2020
- 2020-09-16 CN CN202010973121.XA patent/CN111918005B/zh active Active
-
2021
- 2021-08-06 WO PCT/CN2021/111189 patent/WO2022057506A1/zh unknown
- 2021-08-06 EP EP21868334.0A patent/EP4213479A4/en active Pending
Patent Citations (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2006013567A (ja) * | 2004-06-22 | 2006-01-12 | Mitsubishi Electric Corp | 撮像装置及びその信号処理方法 |
CN1862339A (zh) * | 2006-05-25 | 2006-11-15 | 友达光电股份有限公司 | 高色彩表现的液晶显示器 |
US20090295962A1 (en) * | 2008-05-30 | 2009-12-03 | Omnivision | Image sensor having differing wavelength filters |
CN103004218A (zh) * | 2011-05-19 | 2013-03-27 | 松下电器产业株式会社 | 三维摄像装置、摄像元件、透光部、及图像处理装置 |
WO2020112726A1 (en) * | 2018-11-30 | 2020-06-04 | Intuitive Surgical Operations, Inc. | Medical imaging systems and methods |
CN111918005A (zh) * | 2020-09-16 | 2020-11-10 | Oppo广东移动通信有限公司 | 图像传感器、终端、数据处理方法、装置及存储介质 |
Non-Patent Citations (1)
Title |
---|
See also references of EP4213479A4 |
Also Published As
Publication number | Publication date |
---|---|
CN111918005B (zh) | 2023-05-16 |
CN111918005A (zh) | 2020-11-10 |
EP4213479A4 (en) | 2024-02-21 |
EP4213479A1 (en) | 2023-07-19 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US10257484B2 (en) | Imaging processing device and imaging processing method | |
CN111432099B (zh) | 图像传感器、处理***及方法、电子设备和存储介质 | |
EP4161061A1 (en) | Image acquisition method, camera assembly, and mobile terminal | |
CN205726019U (zh) | 成像***、成像设备和图像传感器 | |
CN108650497B (zh) | 具有透明滤波器像素的成像*** | |
US20080218597A1 (en) | Solid-state imaging device and imaging apparatus | |
US20070057954A1 (en) | Image input device and solid-state image pickup element | |
EP1969558A1 (en) | Method and apparatus providing color interpolation in color filter arrays using edge detection and correction terms | |
CN107547807B (zh) | 用于减少空间闪烁伪影的装置和成像*** | |
EP3021577B1 (en) | Imaging device | |
WO2022057507A1 (zh) | 图像传感器、终端、数据处理方法、装置及存储介质 | |
WO2021073141A1 (zh) | 图像处理方法、图像处理装置以及摄像装置 | |
TWI514894B (zh) | 攝取數位影像中的白平衡校正技術 | |
WO2022057506A1 (zh) | 图像传感器、终端、数据处理方法、装置及存储介质 | |
CN107205139A (zh) | 多通道采集的图像传感器及采集方法 | |
US10924683B2 (en) | Image processing method and imaging device | |
CN101860760A (zh) | 多镜头和多传感器成像***性能的提高 | |
WO2022111015A1 (zh) | 一种图像传感器及成像装置 | |
CN115683327A (zh) | 环境光检测方法以及相关装置 | |
US20140267838A1 (en) | Image Signal Processor Circuits and Methods for Correcting Crosstalk Artifacts | |
Getman et al. | Crosstalk, color tint and shading correction for small pixel size image sensor | |
CN110324541A (zh) | 一种滤波联合去噪插值方法及装置 | |
US8964071B2 (en) | Solid-state imaging device, camera module, and imaging method having first and second green pixel cells | |
CN103327343A (zh) | 一种多光谱图像显示装置 | |
WO2022198436A1 (zh) | 图像传感器、图像数据获取方法、成像设备 |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 21868334 Country of ref document: EP Kind code of ref document: A1 |
|
ENP | Entry into the national phase |
Ref document number: 2021868334 Country of ref document: EP Effective date: 20230412 |
|
NENP | Non-entry into the national phase |
Ref country code: DE |