US9818047B1 - System and method for color enhancement - Google Patents
System and method for color enhancement Download PDFInfo
- Publication number
- US9818047B1 US9818047B1 US14/967,912 US201514967912A US9818047B1 US 9818047 B1 US9818047 B1 US 9818047B1 US 201514967912 A US201514967912 A US 201514967912A US 9818047 B1 US9818047 B1 US 9818047B1
- Authority
- US
- United States
- Prior art keywords
- chrominance
- color
- values
- saturation
- color space
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related, expires
Links
- 238000000034 method Methods 0.000 title claims abstract description 33
- 230000009466 transformation Effects 0.000 claims abstract description 54
- 238000001514 detection method Methods 0.000 claims abstract description 15
- 230000006870 function Effects 0.000 claims description 23
- 230000015654 memory Effects 0.000 claims description 10
- 238000012545 processing Methods 0.000 abstract description 45
- 238000003384 imaging method Methods 0.000 description 6
- 239000003086 colorant Substances 0.000 description 5
- 238000006243 chemical reaction Methods 0.000 description 4
- 230000002708 enhancing effect Effects 0.000 description 4
- 230000007704 transition Effects 0.000 description 4
- 238000004891 communication Methods 0.000 description 3
- 230000009471 action Effects 0.000 description 2
- 239000000654 additive Substances 0.000 description 2
- 230000000996 additive effect Effects 0.000 description 2
- 230000008859 change Effects 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 230000002730 additional effect Effects 0.000 description 1
- 230000004075 alteration Effects 0.000 description 1
- 230000008901 benefit Effects 0.000 description 1
- 230000000977 initiatory effect Effects 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 230000008520 organization Effects 0.000 description 1
- 238000012805 post-processing Methods 0.000 description 1
- 230000000135 prohibitive effect Effects 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 238000000844 transformation Methods 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N9/00—Details of colour television systems
- H04N9/64—Circuits for processing colour signals
- H04N9/68—Circuits for processing colour signals for controlling the amplitude of colour signals, e.g. automatic chroma control circuits
-
- G06K9/6267—
-
- G06K9/4652—
-
- G06K9/4661—
-
- G06K9/52—
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
-
- G06T7/408—
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N1/00—Scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission; Details thereof
- H04N1/46—Colour picture communication systems
- H04N1/56—Processing of colour picture signals
- H04N1/60—Colour correction or control
- H04N1/6027—Correction or control of colour gradation or colour contrast
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/80—Camera processing pipelines; Components thereof
- H04N23/84—Camera processing pipelines; Components thereof for processing colour signals
- H04N23/86—Camera processing pipelines; Components thereof for processing colour signals for controlling the colour saturation of colour signals, e.g. automatic chroma control circuits
-
- G06K2009/4666—
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20021—Dividing image into blocks, subimages or windows
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20048—Transform domain processing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20172—Image enhancement details
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/56—Extraction of image or video features relating to colour
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N1/00—Scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission; Details thereof
- H04N1/46—Colour picture communication systems
- H04N1/56—Processing of colour picture signals
- H04N1/60—Colour correction or control
- H04N1/6002—Corrections within particular colour systems
- H04N1/6005—Corrections within particular colour systems with luminance or chrominance signals, e.g. LC1C2, HSL or YUV
Definitions
- one goal may be to achieve color enhancement of a color image.
- Colors can be represented in different types of color spaces.
- one type of color space may be more appropriate to operate in than another type of color space.
- RGB red, green, blue
- HSV hue, saturation, value
- achieving certain types of high quality color enhancements can be very challenging, and possibly cost prohibitive, especially when operating in the RGB color space.
- it can be very challenging to enhance the skin tone of a person in a color image while also enhancing, for example, color saturation of other portions of the image in an efficient and cost-effective manner.
- this specification discloses an image processing apparatus that includes a zone classifier module, a saturation detection module, a saturation transformation module, and a chrominance enhancement module for performing color enhancement of color images.
- the zone classifier module is configured to map chrominance components of a color pixel to a zone within a color space.
- a chrominance plane of the color space is divided into multiple zones and each zone is assigned a gain value.
- the saturation detection module is configured to generate a saturation value by selecting a maximum absolute value from the chrominance components of the color pixel.
- the saturation transformation module is configured to transform the saturation value to a sigmoidal output value based on a sigmoidal transformation model.
- the chrominance enhancement module is configured to generate enhanced chrominance components for the color pixel based at least in part on the original chrominance components, the saturation value, the sigmoidal output value, and a gain value assigned to a zone of the multiple zones to which the original chrominance components are mapped.
- this specification discloses a method that is performable, for example, by an image processing apparatus to provide color enhancement of color images.
- the method includes generating a chrominance-plane data structure defining a chrominance plane.
- the chrominance-plane data structure stores paired coordinates of chrominance color component values, zone index values defining zones corresponding to angular regions of the chrominance color component values radiating outward from an origin of the chrominance plane, and gain values assigned to the zones.
- the method also includes generating a saturation data structure storing saturation values and the paired coordinates. The saturation values are derived from and related to the paired coordinates within the saturation data structure.
- the method further includes generating a non-linear transformation data structure storing sigmoidal output values and the saturation values.
- the sigmoidal output values are derived from and related to the saturation values within the non-linear transformation data structure.
- the method also includes generating a chrominance enhancement model configured to apply the chrominance-plane data structure, the saturation data structure, and the non-linear transformation data structure to a color image data structure.
- the color image data structure stores pixels of color data.
- An enhanced color image data structure, storing pixels of enhanced color data is generated by the chrominance enhancement model operating on the color image data structure.
- the integrated circuit device includes zone classifier logic, saturation detection logic, saturation transformation logic, and chrominance enhancement logic.
- Zone classifier logic is configured to map chrominance components of pixels of color data stored in an image data structure to zones within a first color space. A chrominance plane of the first color space is divided into the zones and each zone is assigned a gain value.
- Saturation detection logic is configured to generate saturation values by selecting a maximum absolute value from the chrominance components for each pixel of the color data.
- Saturation transformation logic is configured to transform the saturation values to sigmoidal output values based on a sigmoidal transformation model.
- Chrominance enhancement logic is configured to generate enhanced chrominance components for each pixel of the color data based at least in part on the original chrominance components, the saturation values, the sigmoidal output values, and gain values assigned to the zones to which the original chrominance components are mapped.
- FIG. 1 illustrates one embodiment of an image processing device configured to enhance the color of color images.
- FIG. 2 illustrates a color space having Luminance, red chrominance, and blue chrominance color components.
- FIG. 3 illustrates an embodiment of a chrominance plane of the color space of FIG. 2 that is divided into zones.
- FIG. 4 illustrates a graph of a non-linear transformation model which transforms saturation values based on an S-shaped function.
- FIG. 5 Illustrates one embodiment of an architecture of data structures and a chrominance enhancement model configured to enhance color images.
- FIG. 6 illustrates one embodiment of a method, performable by the image processing device of FIG. 1 , for enhancing color in color images.
- FIG. 7 illustrates one embodiment of the image processing device of FIG. 1 implemented as an integrated circuit device.
- Described herein are examples of systems, apparatus, methods, and other embodiments associated with image processing.
- embodiments are described herein that are associated with color image processing for enhancing color components in color images. Enhancement of color components can result in color images that are more visually pleasing to observe, for example, by providing an increase in perceived colorfulness and saturation.
- Such embodiments provide for both the controlled enhancement of the skin tone of people and the enhancement of other objects/scenery in a same color image.
- a combination of zoned gain techniques in a chrominance color plane and non-linear (S-shape) saturation techniques are employed to enhance color images.
- a color image is converted from a red, green, blue (RGB) color space to a luminance, blue-chrominance, red-chrominance (Y, Cb, Cr) color space.
- the zoned gained techniques and the non-linear saturation techniques are applied in the (Y, Cb, Cr) color space to enhance the color image.
- the color image is then converted back to the (R, G, B) color space.
- Embodiments can be applied as one block in an image processing pipeline of multiple blocks (e.g., in a digital camera), or can be applied as a post-processing block in a separate device (e.g., in a digital color printer).
- Pixel data of the enhanced color image can be used to control a color display device or a color printing device, for example, to display or print the enhanced color image.
- an S-shaped curve refers to an S-shaped curve or function, where the “S” shape can take many forms.
- an S-shaped curve is a non-linear curve that has a single transition point somewhere along the curve where the curvature transitions from one direction of curvature to another direction of curvature.
- an S-shaped curve can have the same or different amounts of curvature before and after the transition point.
- sigmoidal output value refers to an output value from a sigmoidal function or model, in response to an input value.
- color pixel refers to the color data associated with a picture element data structure of an image having color components (e.g., red, green, blue color components).
- base chrominance components refer to the original chrominance components of a color pixel before enhancement.
- FIG. 1 illustrates one embodiment of an image processing device 100 configured to enhance color images.
- the image processing device 100 is configured to input RGB color components of color image pixels and output enhanced R′G′B′ color components (where the apostrophe (′) indicates “enhanced”).
- R′G′B′ color components where the apostrophe (′) indicates “enhanced”.
- much of the image processing is performed in the YCbCr color space.
- the RGB color components are first converted to YCbCr color components.
- the chrominance color components Cb and Cr are enhanced to form enhanced color components YCb′Cr′.
- the YCb′Cr′ color components are converted back to the RGB color space as enhanced R′G′B′ color components which are output from the image processing device 100 .
- the image processing device 100 includes color space converter logic 110 , zone classifier logic 120 , saturation detection logic 130 , saturation transformation logic 140 , and chrominance enhancement logic 150 .
- the image processing device 100 is implemented on a chip (i.e., a system-on-chip or SOC configuration) including one or more integrated circuits configured to perform one or more of the functions described herein.
- the logics of the image processing device 100 may be part of an executable algorithm configured to perform the functions of the logics where the algorithm is stored in a non-transitory medium.
- the image processing device 100 is an executable application including algorithms and/or program modules configured to perform the functions of the logics.
- the application is stored in a non-transitory computer storage medium. That is, in one embodiment, the logics of the image processing device 100 are implemented as modules of computer-executable instructions stored on a computer-readable medium.
- the RGB color space is an additive color space based on the RGB color model.
- the RGB color space is defined by the three red, green, and blue additive primary colors, and can produce any color that is the combination defined by the primary colors.
- the YCbCr color space includes a luminance component (Y) and Cb and Cr blue and red chrominance components (sometimes referred to as the blue-difference and red-difference chrominance components).
- FIG. 2 illustrates the YCbCr color space 200 having luminance, red chrominance, and blue chrominance component axes. However, since FIG. 2 is shown in shades of gray instead of colors herein, the chromaticity of the YCbCr color space 200 cannot be fully appreciated in FIG. 2 .
- color space converter logic 110 is configured to convert color components of pixels of color data (e.g., stored in an image data structure) from a first color space (e.g., RGB) to a second color space (e.g., YCbCr), and vice versa.
- first color space e.g., RGB
- second color space e.g., YCbCr
- the following formulas are implemented by color space converter logic 110 to perform the conversions.
- RGB color components (e.g., stored in a first image data structure) are input to color space converter logic 110 .
- Color space converter logic 110 converts the RGB color components to YCbCr color components in accordance with the formulas above (and stores the YCbCr color components in a second image data structure, for example).
- Color space converter logic 110 also outputs the CbCr chrominance components to zone classifier logic 120 , saturation detection logic 130 , and chrominance enhancement logic 150 and retains the luminance component Y.
- color conversion is represented within the image processing device 100 as a color space transformation data structure 510 (see FIG. 5 ) which is discussed in more detail herein with respect to FIG. 5 and FIG. 6 .
- zone classifier logic 120 is configured to receive the YCbCr components of pixels of color data from color space converter logic 110 and map the chrominance components, CbCr, to multiple zones within the YCbCr color space 200 .
- a chrominance plane of the YCbCr color space is divided into the multiple zones and each zone is assigned a gain value.
- FIG. 3 illustrates an embodiment of a chrominance plane 300 of the color space 200 of FIG. 2 that is divided into zones.
- the chrominance values Cb and Cr are normalized to one (1.0).
- the chrominance plane 300 is represented within the image processing device 100 as a chrominance-plane data structure 520 (see FIG.
- FIG. 5 which is discussed in more detail herein with respect to FIG. 5 and FIG. 6 .
- FIG. 3 is shown in shades of gray instead of colors herein, the chrominance of the CbCr chrominance plane cannot be fully appreciated in FIG. 3 .
- each pixel can be classified into an angular zone (z) in one of four (4) quadrants (0-4) of the chrominance plane 300 .
- each quadrant has sixteen (16) angular zones (0-15). Therefore, a total of sixty-four (64) angular zones are defined in the chrominance plane 300 .
- a gain value, g(z) is assigned to each of the angular zones. In this manner, a CbCr pair of chrominance components of a pixel can be associated with a gain value, g(z), based on the chrominance plane 300 .
- each angle radiating radially outward from an origin of the chrominance plane 300 corresponds to a hue. Therefore, each angular zone of the chrominance plane 300 corresponds to an angular region of hues radiating outward from the origin of the chrominance plane 300 .
- gain values may be interpolated between angular zones for a given CbCr pair to obtain a more refined gain value. For a particular angle (hue), however, the gain value is kept the same such that saturation is not corrupted, in accordance with one embodiment.
- a rectangular skin tone zone and a rectangular extended skin tone zone are provided.
- the skin tone zone and the extended skin tone zone represent CbCr color components often associated with the skin tone of people.
- the skin tone zone is assigned its own gain value and the extended skin tone zone is also assigned its own gain value.
- the extended skin tone zone surrounds the skin tone zone.
- a CbCr pair of chrominance components of a pixel classified in the skin tone zone or the extended skin tone zone is associated with the corresponding gain value.
- the skin tone zone is defined by the predefined values skin_Y_th, skin_Cb_high, skin_Cb_low, skin_Cr_high, and skin_Cr_low.
- the current pixel (having color components YCbCr) is classified to be in the skin tone zone when the following conditions are met:
- the extended skin tone zone is defined by the predefined values subSkin_Cb_high, subSkin_Cb_low, subSkin_Cr_high, and subSkin_Cr_low.
- the current pixel is classified to be in the extended skin tone zone if the current pixel is not classified in the skin tone zone and its YCbCr values meet the following conditions:
- saturation detection logic 130 is configured to generate saturation values (SAT) as defined herein by calculating from the base chrominance components CbCr for each pixel of color data.
- ) or SAT SQRT(
- saturation is a distance from the origin of the chrominance plane 300 along a particular hue angle.
- the saturation values are stored within a saturation data structure 530 (see FIG. 5 ) of the image processing device 100 , which is discussed in more detail herein with respect to FIG. 5 and FIG. 6 .
- saturation transformation logic 140 is configured to transform the saturation values “SAT”, from saturation detection logic 130 , to sigmoidal output values “s” based on a non-linear (e.g., sigmoidal) transformation model.
- the non-linear transformation model transforms saturation values “SAT” to sigmoidal output values “s” based on an S-shaped function or S-curve.
- Saturation transformation logic 140 generates the sigmoidal output values “s” based on input saturation values SAT.
- a controls the transition point of curvature in the S-curve and r controls the shape of the S-curve (e.g., see FIG. 4 ).
- the values for “a” and “r” may be predetermined (e.g., based on off-line optimization) or may be selectable by a user of the image processing device 100 , in accordance with various embodiments.
- the sigmoidal output values “s” are stored within a non-linear transformation data structure 540 (see FIG. 5 ) of the image processing device 100 , which is discussed in more detail herein with respect to FIG. 5 and FIG. 6 .
- chrominance enhancement logic 150 is configured to generate enhanced chrominance components (Cb′Cr′) for each pixel of the color data based on the base chrominance components (CbCr), the saturation values (SAT), the sigmoidal output values (s), and gain values (g(z)) assigned to the zones to which the base chrominance components (CbCr) are mapped in the chrominance plane 300 by zone classifier logic 120 .
- chrominance enhancement logic 150 generates the enhanced chrominance components (Cb′Cr′) further based on a global color gain (GCG) value.
- GCG global color gain
- chrominance enhancement logic 150 employs a chrominance enhancement model 550 (see FIG. 5 ) of the image processing device 100 , which is discussed in more detail herein with respect to FIG. 5 and FIG. 6 .
- chrominance enhancement logic 150 generates enhanced chrominance components (Cb′Cr′) based on the following chrominance enhancement model 540 :
- C out SAT+[ s ⁇ SAT]* g ( z )*GCG/64, where “s” is a function of “SAT”; and
- the enhanced chrominance components Cb′Cr′ may be fed back into color space converter logic 110 and converted back to the red, green, blue color space as enhanced color components R′G′B′ which are output from the image processing device 100 . That is, from a black box perspective, RGB color components are input to the image processing device 100 and are processed to generate enhanced R′G′B′ color components which are then output from the image processing device 100 .
- FIG. 5 Illustrates one embodiment of an architecture 500 of data structures and a chrominance enhancement model configured to enhance color images.
- FIG. 6 illustrates one embodiment of a method 600 , performable by the image processing device 100 of FIG. 1 , for enhancing color in color images using the architecture 500 of FIG. 5 .
- Method 600 is implemented to be performed by the image processing device 100 of FIG. 1 , or by another computing device configured with an algorithm of method 600 . That is, method 600 will be described from the perspective that the architecture 500 of FIG. 5 (data structures and model) can be generated by the image processing device 100 or off-line by a computing device. The architecture 500 can then implemented within the image processing device 100 such that the image processing device 100 may receive a color image data structure of image pixel data which can be color enhanced by applying the architecture 500 .
- the image pixel data may be acquired by and/or originate from an image related device such as, for example, a still-image camera device, a video camera device, a scanner, or an image storage device.
- an image related device such as, for example, a still-image camera device, a video camera device, a scanner, or an image storage device.
- the image processing device 100 may be a part of the image related device.
- the image processing device 100 may be separate from the image related device and may be configured to operably connect to the image related device via an operable connection (e.g., via a network connection).
- a color space transformation data structure 510 is generated, allowing transformations between color spaces.
- the color space transformation data structure 510 stores a plurality of color components of a first color space and a plurality of color components of a second color space.
- the color components in the first color space are related to (associated with) the color components of the second color space in a one-to-one correspondence, in accordance with one embodiment.
- the color space transformation data structure 510 stores a plurality of RGB color component values that are mapped to a plurality of YCbCr color component values, and vice versa, in accordance with one embodiment.
- the color space transformation data structure 510 is generated by color space converter logic 110 of the imaging processing device 100 .
- the color space transformation data structure 510 is generated off-line by another computing device.
- a chrominance-plane data structure 520 is generated defining a chrominance plane (e.g., as described with respect to FIG. 3 ).
- the chrominance-plane data structure 520 stores a plurality of paired coordinates of chrominance color component values CbCr, a plurality of zone index values defining a plurality of zones, and a plurality of gain values assigned to the plurality of zones in one-to-one correspondence.
- the plurality of zones correspond to angular regions of the chrominance color component values radiating outward from an origin of the chrominance plane.
- two of the plurality of zones may correspond to a skin tone zone and an extended skin tone zone surrounding the skin tone zone.
- the skin tone zone and the extended skin tone zone are each assigned their own gain value.
- the chrominance-plane data structure 520 is generated by zone classifier logic 120 of the imaging processing device 100 . In another embodiment, the chrominance plane data structure 520 is generated off-line by another computing device.
- a saturation data structure 530 is generated which stores a plurality of saturation values and the plurality of paired coordinates CbCr.
- the plurality of saturation values are derived from and related to (associated with) the plurality of paired coordinates within the saturation data structure 530 in a one-to-one correspondence, in accordance with one embodiment.
- the saturation data structure 530 is generated by saturation detection logic 130 of the imaging processing device 100 , in one embodiment. In another embodiment, the saturation data structure 530 is generated off-line by another computing device.
- a non-linear transformation data structure 540 is generated which stores a plurality of sigmoidal output values and the plurality of saturation values.
- the plurality of sigmoidal output values are derived from and related to (associated with) the plurality of saturation values within the non-linear transformation data structure in a one-to-one-correspondence, in accordance with one embodiment.
- the non-linear transformation data structure 540 is generated by saturation transformation logic 140 of the imaging processing device 100 , in one embodiment. In another embodiment, the non-linear transformation data structure 540 is generated off-line by another computing device.
- a chrominance enhancement model 550 is generated.
- the chrominance enhancement model 550 is configured to apply the color space transformation data structure 510 , the chrominance-plane data structure 520 , the saturation data structure 530 , and the non-linear transformation data structure 540 to the color image data structure.
- the chrominance enhancement model 550 transforms the color image data structure to an enhanced color image data structure or generates an enhanced color image data structure from the color image data structure.
- the chrominance enhancement model 550 is generated by chrominance enhancement logic 150 of the imaging processing device 100 .
- the chrominance enhancement model is generated off-line by another computing device.
- the logics of the image processing device 100 of FIG. 1 are configured to operate on the data structures of FIG. 5 using the chrominance enhancement model 550 of FIG. 5 .
- a pixel from a color image data structure of image pixel data (YCbCr) is input to (e.g., read into) the chrominance enhancement model 550 after being converted from RGB image pixel data using the color space transformation data structure 510 .
- the chrominance components CbCr are used by the chrominance enhancement model 550 to address the chrominance-plane data structure 520 and the saturation data structure 530 .
- Addressing of the chrominance-plane data structure 520 returns a gain value g(z), based on a zone related to the chrominance components CbCr in the chrominance-plane data structure 520 , to the chrominance enhancement model 550 .
- Addressing of the saturation data structure 530 returns a saturation value SAT, related to the chrominance components CbCr in the saturation data structure 530 , to the chrominance enhancement model 550 .
- the returned saturation value SAT is used to address the non-linear transformation data structure 540 .
- Addressing of the non-linear transformation data structure 540 returns a sigmoidal output value “s”, related to the saturation value SAT in the non-linear transformation data structure 540 , to the chrominance enhancement model 550 .
- the chrominance enhancement model 550 operates on the image pixel data YCbCr, the gain value g(z), and the saturation value SAT to generate enhanced chrominance components Cb′Cr′, as shown in FIG. 5 .
- a global color gain (GCG) value may be applied by the chrominance enhancement model 550 to further adjust the enhanced chrominance components Cb′Cr′.
- the original image pixel data component Y may be joined with the enhanced chrominance components Cb′Cr′, forming enhanced image pixel data YCb′Cr′, and the enhanced image pixel data (YCb′Cr′) may be converted to an enhanced image pixel data (R′G′B′) using the color space transformation data structure 510 .
- the data structures of FIG. 5 may be generated and operated upon by the logics of FIG. 1 to enhance pixels in a color image. Color is enhanced in a color image based significantly on a non-linear manipulation of saturation and a distribution of gains across angular zones of hue in the chrominance plane of the YCbCr color space.
- FIG. 7 illustrates one embodiment of the image processing device 100 of FIG. 1 implemented as an integrated circuit device 700 .
- the color space converter logic 110 and the zone classifier logic 120 are each embodied on the same integrated circuit 710 .
- the saturation detection logic 130 , the saturation transformation logic 140 , and the chrominance enhancement logic 150 are each embodied as a separate integrated circuit 720 , 730 , and 740 , respectively.
- circuits are connected via connection paths to communicate signals. While integrated circuits 710 , 720 , 730 , and 740 are illustrated as separate integrated circuits, they may be integrated into a common integrated circuit device 700 . Additionally, integrated circuits 710 , 720 , 730 , and 740 may be combined into fewer integrated circuits or divided into more integrated circuits than illustrated. Similarly, the image processing device 100 of FIG. 1 may be implemented as an integrated circuit device, in accordance with certain embodiments.
- the color space converter logic 110 , the zone classifier logic 120 , the saturation detection logic 130 , the saturation transformation logic 140 , and the chrominance enhancement logic 150 may be combined into a separate application-specific integrated circuit.
- portions of the functionality associated with the color space converter logic 110 , the zone classifier logic 120 , the saturation detection logic 130 , the saturation transformation logic 140 , and the chrominance enhancement logic 150 may be embodied as firmware executable by a processor and stored in a non-transitory memory (e.g., a non-transitory computer storage medium).
- a chrominance-plane data structure is generated that defines a chrominance plane.
- the chrominance-plane data structure stores paired coordinates of chrominance color component values, zone index values defining zones corresponding to angular regions of the chrominance color component values radiating outward from an origin of the chrominance plane, and gain values assigned to the zones.
- a saturation data structure is generated that stores saturation values and the paired coordinates of chrominance color component values. The saturation values are derived from and related to the paired coordinates within the saturation data structure.
- a non-linear transformation data structure is generated that stores sigmoidal output values and the saturation values.
- the sigmoidal output values are derived from and related to the saturation values within the non-linear transformation data structure.
- a chrominance enhancement model is generated which is configured to apply the chrominance-plane data structure, the saturation data structure, and the non-linear transformation data structure to a color image data structure to generate an enhanced color image.
- references to “one embodiment”, “an embodiment”, “one example”, “an example”, and so on, indicate that the embodiment(s) or example(s) so described may include a particular feature, structure, characteristic, property, element, or limitation, but that not every embodiment or example necessarily includes that particular feature, structure, characteristic, property, element or limitation. Furthermore, repeated use of the phrase “in one embodiment” does not necessarily refer to the same embodiment, though it may.
- Computer-readable medium or “computer storage medium”, as used herein, refers to a non-transitory medium that stores instructions and/or data configured to perform one or more of the disclosed functions when executed.
- a computer-readable medium may take forms, including, but not limited to, non-volatile media, and volatile media.
- Non-volatile media may include, for example, optical disks, magnetic disks, and so on.
- Volatile media may include, for example, semiconductor memories, dynamic memory, and so on.
- a computer-readable medium may include, but are not limited to, a floppy disk, a flexible disk, a hard disk, a magnetic tape, other magnetic medium, an application specific integrated circuit (ASIC), a programmable logic device, a compact disk (CD), other optical medium, a random access memory (RAM), a read only memory (ROM), a memory chip or card, a memory stick, solid state storage device (SSD), flash drive, and other media from which a computer, a processor or other electronic device can function with.
- ASIC application specific integrated circuit
- CD compact disk
- RAM random access memory
- ROM read only memory
- memory chip or card a memory chip or card
- SSD solid state storage device
- flash drive and other media from which a computer, a processor or other electronic device can function with.
- Each type of media if selected for implementation in one embodiment, may include stored instructions of an algorithm configured to perform one or more of the disclosed and/or claimed functions.
- Computer-readable media described herein are limited to statutory subject matter under 35 U.
- Logic represents a component that is implemented with computer or electrical hardware, a non-transitory medium with stored instructions of an executable application or program module, and/or combinations of these to perform any of the functions or actions as disclosed herein, and/or to cause a function or action from another logic, method, and/or system to be performed as disclosed herein.
- Equivalent logic may include firmware, a microprocessor programmed with an algorithm, a discrete logic (e.g., ASIC), at least one circuit, an analog circuit, a digital circuit, a programmed logic device, a memory device containing instructions of an algorithm, and so on, any of which may be configured to perform one or more of the disclosed functions.
- logic may include one or more gates, combinations of gates, or other circuit components configured to perform one or more of the disclosed functions. Where multiple logics are described, it may be possible to incorporate the multiple logics into one logic. Similarly, where a single logic is described, it may be possible to distribute that single logic between multiple logics. In one embodiment, one or more of these logics are corresponding structure associated with performing the disclosed and/or claimed functions. Choice of which type of logic to implement may be based on desired system conditions or specifications. For example, if greater speed is a consideration, then hardware would be selected to implement functions. If a lower cost is a consideration, then stored instructions/executable application would be selected to implement the functions. Logic is limited to statutory subject matter under 35 U.S.C. ⁇ 101.
- a “data structure”, as used herein, is an organization of data in a computing system that is stored in a memory, a storage device, or other computerized system.
- a data structure may be any one of, for example, a data field, a data file, a data array, a data record, a database, a data table, a graph, a tree, a linked list, and so on.
- a data structure may be formed from and contain many other data structures (e.g., a database includes many data records). Other examples of data structures are possible as well, in accordance with other embodiments.
- An “operable (or operative) connection”, or a connection by which entities are “operably (or operatively) connected”, is one in which signals, physical communications, and/or logical communications may be sent and/or received.
- An operable connection may include a physical interface, an electrical interface, and/or a data interface.
- An operable connection may include differing combinations of interfaces and/or connections sufficient to allow operable control. For example, two entities can be operably connected to communicate signals to each other directly or through one or more intermediate entities (e.g., processor, operating system, logic, non-transitory computer-readable medium).
- An operable connection may include one entity generating data and storing the data in a memory, and another entity retrieving that data from the memory via, for example, instruction control. Logical and/or physical communication channels can be used to create an operable connection.
- the terms “operable” and “operative”, and there various forms, may be used interchangeably herein.
- illustrated methodologies are shown and described as a series of blocks. The methodologies are not limited by the order of the blocks as some blocks can occur in different orders and/or concurrently with other blocks from that shown and described. Moreover, less than all the illustrated blocks may be used to implement an example methodology. Blocks may be combined or separated into multiple components. Furthermore, additional and/or alternative methodologies can employ additional actions that are not illustrated in blocks. The methods described herein are limited to statutory subject matter under 35 U.S.C. ⁇ 101.
- the phrase “one or more of, A, B, and C” is used herein, (e.g., a data store configured to store one or more of, A, B, and C) it is intended to convey the set of possibilities A, B, C, AB, AC, BC, and/or ABC (e.g., the data store may store only A, only B, only C, A&B, A&C, B&C, and/or A&B&C). It is not intended to require one of A, one of B, and one of C.
- the applicants intend to indicate “at least one of A, at least one of B, and at least one of C”, then the phrasing “at least one of A, at least one of B, and at least one of C” will be used.
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Image Processing (AREA)
Abstract
Description
Y=0.299*R+0.587*G+0.114*B,
Cb=−0.169*R−0.331*G+0.5*B,
Cr=0.5*R−0.419*G−0.0813*B.
R=Y+1.402*Cr,
G=Y−0.344*Cb−0.714*Cr,
B=Y+1.772*Cb.
s=a 1-rSATr,
for 0≦SAT≦a, and
s=1−(1−a)1-r(1-SAT)r,
for a<SAT≦1,
Cout=SAT+[s−SAT]*g(z)*GCG/64,
where “s” is a function of “SAT”; and
Cb′=(Cout/SAT)*Cb
and
Cr′=(Cout/SAT)*Cr.
Claims (20)
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US14/967,912 US9818047B1 (en) | 2015-02-09 | 2015-12-14 | System and method for color enhancement |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US201562113779P | 2015-02-09 | 2015-02-09 | |
US14/967,912 US9818047B1 (en) | 2015-02-09 | 2015-12-14 | System and method for color enhancement |
Publications (1)
Publication Number | Publication Date |
---|---|
US9818047B1 true US9818047B1 (en) | 2017-11-14 |
Family
ID=60255813
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US14/967,912 Expired - Fee Related US9818047B1 (en) | 2015-02-09 | 2015-12-14 | System and method for color enhancement |
Country Status (1)
Country | Link |
---|---|
US (1) | US9818047B1 (en) |
Cited By (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US10516867B2 (en) * | 2017-10-30 | 2019-12-24 | Samsung Display Co., Ltd. | Color conversion device, display device including the same, and method of converting color |
CN113658280A (en) * | 2021-08-20 | 2021-11-16 | 平安科技(深圳)有限公司 | Data augmentation method, device, equipment and storage medium based on artificial intelligence |
CN114120932A (en) * | 2021-11-30 | 2022-03-01 | 中航华东光电有限公司 | Liquid crystal display dimming method combined with image saturation adjustment |
US20220139343A1 (en) * | 2020-11-05 | 2022-05-05 | Lx Semicon Co., Ltd. | Color gamut mapping method and device |
US20220139342A1 (en) * | 2020-11-05 | 2022-05-05 | Lx Semicon Co., Ltd. | Color gamut mapping method and device |
Citations (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20050185839A1 (en) * | 2004-02-19 | 2005-08-25 | Sony Corporation | Image processing apparatus, and image processing method |
US20090195551A1 (en) * | 2008-02-05 | 2009-08-06 | Micron Technology, Inc. | Systems and methods to achieve preferred imager color reproduction |
-
2015
- 2015-12-14 US US14/967,912 patent/US9818047B1/en not_active Expired - Fee Related
Patent Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20050185839A1 (en) * | 2004-02-19 | 2005-08-25 | Sony Corporation | Image processing apparatus, and image processing method |
US7551774B2 (en) * | 2004-02-19 | 2009-06-23 | Sony Corporation | Image processing apparatus, and image processing method |
US20090195551A1 (en) * | 2008-02-05 | 2009-08-06 | Micron Technology, Inc. | Systems and methods to achieve preferred imager color reproduction |
US8130236B2 (en) * | 2008-02-05 | 2012-03-06 | Aptina Imaging Corporation | Systems and methods to achieve preferred imager color reproduction |
Cited By (9)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US10516867B2 (en) * | 2017-10-30 | 2019-12-24 | Samsung Display Co., Ltd. | Color conversion device, display device including the same, and method of converting color |
US20220139343A1 (en) * | 2020-11-05 | 2022-05-05 | Lx Semicon Co., Ltd. | Color gamut mapping method and device |
US20220139342A1 (en) * | 2020-11-05 | 2022-05-05 | Lx Semicon Co., Ltd. | Color gamut mapping method and device |
US11640800B2 (en) * | 2020-11-05 | 2023-05-02 | Lx Semicon Co., Ltd. | Color gamut mapping method and device |
US11887549B2 (en) * | 2020-11-05 | 2024-01-30 | Lx Semicon Co., Ltd. | Color gamut mapping method and device |
CN113658280A (en) * | 2021-08-20 | 2021-11-16 | 平安科技(深圳)有限公司 | Data augmentation method, device, equipment and storage medium based on artificial intelligence |
CN113658280B (en) * | 2021-08-20 | 2023-07-04 | 平安科技(深圳)有限公司 | Data augmentation method, device, equipment and storage medium based on artificial intelligence |
CN114120932A (en) * | 2021-11-30 | 2022-03-01 | 中航华东光电有限公司 | Liquid crystal display dimming method combined with image saturation adjustment |
CN114120932B (en) * | 2021-11-30 | 2022-11-22 | 中航华东光电有限公司 | Liquid crystal display dimming method combined with image saturation adjustment |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US9818047B1 (en) | System and method for color enhancement | |
US10614603B2 (en) | Color normalization for a multi-camera system | |
Afifi et al. | Deep white-balance editing | |
JP4196845B2 (en) | Image processing device | |
JP4117284B2 (en) | Color space conversion method, color image color adjustment device, color adjustment method, and computer-readable recording medium recording the program | |
US10063743B2 (en) | Image processing apparatus, image processing method, and non-transitory computer-readable storage medium | |
CN109934776B (en) | Model generation method, video enhancement method, device and computer-readable storage medium | |
TWI289274B (en) | Method and apparatus for converting from a source color space to a target color space | |
JP5727109B2 (en) | Spectral image processing | |
US8611654B2 (en) | Color saturation-modulated blending of exposure-bracketed images | |
JP6840957B2 (en) | Image similarity calculation device, image processing device, image processing method, and recording medium | |
US9716880B2 (en) | Image calibrating method for stitching images and related camera and image processing system with image calibrating function | |
ES2900490T3 (en) | Color gamut mapping method and system | |
US20120057785A1 (en) | Method and system to modify a color lookup table | |
KR20070090224A (en) | Method of electronic color image saturation processing | |
CN107113411A (en) | A kind of method for displaying image and equipment and its recording medium based on metadata | |
JP5051477B2 (en) | Method for color enhancement with color volume adjustment and variable shift along the luminance axis | |
US8509529B2 (en) | Color-image representative color decision apparatus and method of controlling operation thereof | |
Bianco et al. | Error-tolerant color rendering for digital cameras | |
CN109155845A (en) | Image processing apparatus, image processing method and program | |
US9633461B2 (en) | System and method for color management of mixed content documents | |
US9852531B2 (en) | Electronic apparatus and method for controlling the same | |
JP4826857B2 (en) | Image processing method, image processing apparatus, and image processing program | |
CN115880161A (en) | Image enhancement method, device, equipment and computer readable storage medium | |
JP6544004B2 (en) | Color sample creating apparatus and color sample creating method, and image processing system using color sample |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: MARVELL INTERNATIONAL LTD., BERMUDA Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:MARVELL SEMICONDUCTOR, INC.;REEL/FRAME:043828/0867 Effective date: 20170926 Owner name: MARVELL SEMICONDUCTOR, INC., CALIFORNIA Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:LI, HUAI DONG;HUANG, SAMSON;SIGNING DATES FROM 20151209 TO 20151211;REEL/FRAME:043828/0779 |
|
STCF | Information on status: patent grant |
Free format text: PATENTED CASE |
|
AS | Assignment |
Owner name: CAVIUM INTERNATIONAL, CAYMAN ISLANDS Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:MARVELL INTERNATIONAL LTD.;REEL/FRAME:052918/0001 Effective date: 20191231 |
|
AS | Assignment |
Owner name: MARVELL ASIA PTE, LTD., SINGAPORE Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:CAVIUM INTERNATIONAL;REEL/FRAME:053475/0001 Effective date: 20191231 |
|
FEPP | Fee payment procedure |
Free format text: MAINTENANCE FEE REMINDER MAILED (ORIGINAL EVENT CODE: REM.); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY |
|
LAPS | Lapse for failure to pay maintenance fees |
Free format text: PATENT EXPIRED FOR FAILURE TO PAY MAINTENANCE FEES (ORIGINAL EVENT CODE: EXP.); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY |
|
STCH | Information on status: patent discontinuation |
Free format text: PATENT EXPIRED DUE TO NONPAYMENT OF MAINTENANCE FEES UNDER 37 CFR 1.362 |
|
FP | Lapsed due to failure to pay maintenance fee |
Effective date: 20211114 |