Classifications

• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
  • H04N19/85—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using pre-processing or post-processing specially adapted for video compression
• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T9/00—Image coding
• • G—PHYSICS
  • G11—INFORMATION STORAGE
  • G11B—INFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER
  • G11B27/00—Editing; Indexing; Addressing; Timing or synchronising; Monitoring; Measuring tape travel
  • G11B27/02—Editing, e.g. varying the order of information signals recorded on, or reproduced from, record carriers
  • G11B27/031—Electronic editing of digitised analogue information signals, e.g. audio or video signals
  • G11B27/034—Electronic editing of digitised analogue information signals, e.g. audio or video signals on discs
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
  • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
  • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
  • H04N19/186—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being a colour or a chrominance component
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
  • H04N19/30—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using hierarchical techniques, e.g. scalability
  • H04N19/33—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using hierarchical techniques, e.g. scalability in the spatial domain
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
  • H04N19/46—Embedding additional information in the video signal during the compression process
  • H04N19/463—Embedding additional information in the video signal during the compression process by compressing encoding parameters before transmission
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
  • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
  • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
  • H04N21/234—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
  • H04N21/2343—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements
  • H04N21/234327—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements by decomposing into layers, e.g. base layer and one or more enhancement layers
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
  • H04N19/30—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using hierarchical techniques, e.g. scalability
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
  • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
  • H04N21/25—Management operations performed by the server for facilitating the content distribution or administrating data related to end-users or client devices, e.g. end-user or client device authentication, learning user preferences for recommending movies
  • H04N21/266—Channel or content management, e.g. generation and management of keys and entitlement messages in a conditional access system, merging a VOD unicast channel into a multicast channel
  • H04N21/2662—Controlling the complexity of the video stream, e.g. by scaling the resolution or bitrate of the video stream based on the client capabilities
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
  • H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
  • H04N21/83—Generation or processing of protective or descriptive data associated with content; Content structuring
  • H04N21/845—Structuring of content, e.g. decomposing content into time segments

Definitions

• This disclosure relates generally to an image encoding system and method that provides increased precision, increased dynamic range, and a wider color gamut as compared to many existing image file formats. More particularly, this disclosure relates to an image encoding method that is backwards compatible with existing devices such that the increased precision, dynamic range, and color gamut data does not cause existing devices to fail.
• digital images are expressed in terms of reference values that define the properties of the image.
• properties for each pixel of a digital image may be specified by multiple reference values ⁇ e.g., R or red, G or green, and B or blue values).
• reference values are defined in terms of a color model.
• a color model describes the way that colors can be represented using combinations of reference values.
• the set of colors that can be produced according to a particular color model is a color space.
• the most common color model for producing images on display devices such as television screens, computer monitors, tablets, etc. is the RGB color model.
• the RGB color model defines a set of colors that are produced from combinations of varying levels ⁇ i.e., varying reference values) of red, green, and blue primary colors.
• Outer curved boundary 105 represents the visible spectrum monochromatic colors with wavelengths indicated in nanometers. The colors along outer curved boundary 105 progress through a range of purple, blue, green, yellow, orange, and red with increasing wavelength.
• the chromaticities of the red, green, and blue color primaries for a particular RGB color space ⁇ i.e., the chromaticity where one color channel has a nonzero value and the other two channels have zero values) form the vertices of color triangle 115.
• the gamut of chromaticities that can be represented by the RGB color space are represented by the chromaticities that are within color triangle 115.
• Color triangle 115 corresponds to the sRGB color space, the most common of the RGB color spaces. Vertex
• vertex HOB is the sRGB green primary
• vertex HOC is the sRGB blue primary.
• the D65 white point, the point at which all of the color channels are equal to one, is illustrated at 120.
• typical color spaces such as the sRGB color space encompass substantially less than the full range of
• color spaces are capable of representing only a small portion of the brightness levels that can be perceived by humans.
• These color space limitations have been incorporated into commonly used color spaces by design based on the colors that display media are capable of producing. That is, color spaces need only encompass the colors that can be produced by existing display media such as television displays and computer monitors. In fact, the precision with which colors can be produced (for a given data size) is increased where the color space is limited to only those colors that can be produced.
• a method of encoding an image having extended image content may include obtaining a first image expressed in a first image format and obtaining a second image that corresponds to the first image and is expressed in a second image format.
• Each element of the first image may be defined by reference values in a first range and each element of the second image may be defined by reference values in a second range.
• the first range may be a proper subset of the second range such that the first format and the second format are consistent over the complete range of reference values for the first format.
• the first image may then be subtracted from the second image to obtain a delta image.
• the first image may be encoded in the standard payload portion of an image file and the delta image may be encoded in a metadata portion of the image file.
• the method may be embodied in program code and stored on a non-transitory medium. The stored program code may be executed by one or more processors that are part of, or control, a system that is configured to implement the method.
• a method of decoding an image having extended image content may include decoding a payload portion of the image file to generate a first image.
• the first image may be expressed in a base image format where each image element is defined by reference values in a first range.
• a metadata portion of the image file may be decoded to generate additional image data.
• the additional image data may be combined with the first image to generate a second image.
• the second image may be expressed using an extended image format where each image element is defined by reference values in a second range.
• the first range may be a proper subset of the second range such that the base image format and the extended image format are consistent over the complete range of reference values for the base image format.
• the method may be embodied in program code and stored on a non-transitory medium.
• the stored program code may be executed by one or more processors that are part of, or control, a system that is configured to implement the method.
• Figure 1 illustrates the CIE 1931 color space chromaticity diagram with the sRGB color gamut specified.
• Figure 2 is a block diagram that illustrates the components of an extended image format in accordance with one embodiment.
• Figure 3 is a block diagram that illustrates an operation to extract the additional information encoded in an extended image format in accordance with one embodiment.
• Figure 4 is a block diagram that illustrates an operation to extract the additional information encoded in an extended image format and to separate the extracted information into different channels in accordance with one embodiment.
• Figures 5A through 5C are block diagrams that illustrate processes to encode a base image and extracted image data from an extended image into a widely supported image format in accordance with multiple embodiments.
• Figure 6 is a block diagram that illustrates the encoding of a base image, one or more channels of extracted additional image data, and instructions for reconstructing an extended image in a widely supported image format in accordance with one embodiment.
• Figure 7 is a flow chart that illustrates a process for decoding an extended image that is encoded using a widely supported image format in accordance with one embodiment.
• Figure 8 shows an illustrative electronic device in accordance with one embodiment.
• This disclosure pertains to systems, methods, and computer readable media for encoding an extended image such that it is backwards compatible with existing decoding devices.
• an extended image format is defined that uses channel primaries that match an existing image format. Because the extended image format references the channel primaries of an existing image format, additional image information that is included in the extended image format can be extracted through computationally inexpensive operations as described below.
• numerous specific details are set forth in order to provide a thorough understanding of the inventive concept.
• some of this disclosure's drawings represent structures and devices in block diagram form in order to avoid obscuring the invention. In the interest of clarity, not all features of an actual implementation are described in this specification.
• a base image format may describe an image in terms of reference values 205 that define the properties of image pixels.
• each pixel of an image expressed in an RGB format may include reference values for a red channel, a green channel, and a blue channel.
• the reference values for the red, green, and blue channels define the properties of the pixel within a given color space ⁇ i.e., a color space defined by the red, green, and blue channel primaries).
• the reference values may be described in terms of nominal values that range from 0.0 to 1.0.
• an image pixel having a value of (1.0,0.0,0.0) (expressed as (R, G, B)), would be a pure red pixel with the highest possible brightness ⁇ i.e., a pixel having the properties of the red channel primary).
• each reference value can be expressed as an 8 bit binary number.
• bit depths are possible. For example, 16 bit.
• an extended image format is consistent with the base image format over the range of reference values of the base image format. Therefore, the extended image format references the same primaries as the base image format ⁇ e.g., Red, Green, and Blue).
• the nominal range for reference values 210 may be extended to encode additional image data ⁇ i.e., image data that cannot be represented using the base format such as, for example, gamut, dynamic range, and precision). That is, the range of reference values of the base image format is a proper subset of the range of reference values of the extended image format. Because the extended image format is consistent with the base image format over the range of reference values for the base image format, reference values within this range ⁇ i.e., nominal values of 0.0 to 1.0 in the illustrated embodiment) represent the same color properties in either the base image format or the extended image format.
• the nominal range for each reference value 210 in the extended image format is doubled as compared to the base image format reference value 205 (from 0.0 - 1.0 to -0.75 - 1.25).
• This extended nominal range may be utilized to encode increased brightness and a wider color gamut than can be produced using the base image format.
• negative values may be utilized to encode colors that are outside of the base image format gamut ⁇ i.e., outside of the color space of the base image format).
• a value of (1.0, -0.75, -0.75) may represent a very saturated red that cannot be represented in the base image format.
• Over unity values may be utilized to encode increased brightness that cannot be represented using the base image format ⁇ i.e., outside of the dynamic range of the base image format).
• (1.25, 0.0, 0.0) may represent a much brighter version of the red primary.
• reference values for the extended image format are expressed using 10 bit binary numbers.
• one additional bit may be utilized to double the nominal range as compared to the base image format and another additional bit to double the precision throughout the increased nominal range.
• One advantage of this approach is that when combined with non-linear gamma, the effective linear range of the representation is increased.
• a 2.2 gamma value in a 0.0 to 1.0 representation is still 0.0 to 1.0 linear.
• a 2.2 gamma value in a 0.0 to 1.25 representation is actually 1.6 times the range provided by the 0.0 to 1.0 linear range.
• the illustrated embodiment is provided as an example only. It is not necessary that the additional image information of the extended format be distributed in the manner illustrated in FIG. 2 ⁇ i.e., 75% of the additional range devoted to wider gamut and 25% devoted to increased dynamic range), that the reference values for the base and extended formats be encoded using any particular number of bits, or that the reference values correspond to an RGB color model.
• This disclosure is equally applicable to any extended image format (using any color model) that extends the range of reference values to encode image information that cannot be encoded using a base image format while maintaining consistency over the range of reference values of the base image format.
• Raw image 305 may include image sensor data that has either not been processed at all or has only been minimally processed.
• Raw image 305 may be analogous to a film photography negative in that each may include image information that is not included in a final image format. For example, some of the information captured by an image sensor and included in raw image 305 may represent brightness levels or colors that cannot be encoded in base image 310 or even extended image 315.
• the conversion of raw image 305 to extended image 315 and base image 310 may involve operations that include noise reduction, sharpening, demosaicing, black subtraction, highlight recovery, tone mapping etc., all of which are outside of the scope of the present disclosure.
• the result of these processes will produce base image 310 that is comprised of reference values for a base image format such as reference values 205 and an extended image that is comprised of reference values for an extended image format such as reference values 210.
• Extended image 315 may include image information from raw image 305 that cannot be expressed in base image 310.
• the conversion process between raw image 305 and base and extended images 310 and 315 may result in different reference values even for colors that can be expressed within the base image format range. For example, if raw image 305 depicts a bright yellow object that cannot be entirely reproduced within the base image format of base image 310, rather than clamping the colors that extend outside of the base image color space at the maximum reference values, which results in a flat and unnatural appearance, the conversion process may adjust certain colors that are within the base image color space such that a more natural reproduction of the object is obtained.
• raw image 305 when converting raw image 305 to extended image 315, which is capable of representing a broader color space, it may not be necessary to adjust the object's colors in the same way as for the base image format.
• FIG. 3 illustrates the conversion of raw image 305 to both base image 310 and extended image 315
• raw image 305 may be converted to extended image 315 and base image 310 may be generated from extended image 315.
• base image 310 After each of base image 310 and extended image 315 have been generated, base image 310 may be subtracted from extended image 315 to extract the difference between the two images, expressed as delta image 320.
• extended image 315 is expressed in terms of an extended format that is consistent with the base image format of base image 310 over the full range of reference values for base image 310
• the extraction of delta image 320 may be performed as a simple pixel by pixel difference operation.
• reference values for pixel 310A of base image 310 may be subtracted from corresponding reference values for corresponding pixel 315A of extended image 315.
• the resulting reference values define the properties of pixel 320A of delta image 320.
• the nominal reference values of pixel 315A may be (1.21, -0.4, - 0.3), which specifies a bright, deep red color that is outside of the range of the base image format of base image 310.
• the reference values of corresponding pixel 310A may be (0.98, 0, 0), which represents nearly the brightest, deepest red that can be produced using the base image format.
• the numeric representations of the 10 bit binary expression of the reference values of pixel 315A and the 8 bit binary expression of the reference values of pixel 310A are (1003, 179, 230) and (250, 0, 0), respectively. This results in reference values of (753, 179, 230) for pixel 320A of delta image 320.
• delta image 320 may be separated into delta image channels 322, 324, and 326, which represent additional dynamic range, wider color gamut, and increased precision of extended image 315 with respect to image 310, respectively.
• the number and types of delta image channels generally depends on the differences between the extended image format and the base image format.
• the reference values for each pixel of delta image 320 may be compared to the reference values for corresponding pixels of extended image 315. For example, referring again to the extended format described with respect to FIG.
• the nominal values from -0.75 to 0.0 of the extended image format may be used to express wider color gamut and the nominal values 1.0 to 1.25 used to express increased brightness. These nominal values correspond to numeric values of 0 to 393 and 896 to 1023 in the 10 bit expression of the extended image format of FIG. 2. Accordingly, the portions of reference values that are within these ranges are attributable to the specified properties. Portions of the reference values for pixel 315A that are attributable to increased brightness can be extracted to generate reference values of (107, 0, 0) for pixel 322A of increased brightness delta image channel 322.
• portions of the reference values for pixel 315A that are attributable to wider color gamut can be extracted to generate reference values of (383, 179, 230) for pixel 324A of wide gamut delta image channel 324.
• the resulting reference values (263, 0, 0) represent the increased precision between the base image format and the extended image format and the differences in the conversion process from a raw image to the extended image format and the base image format and can be encoded as the reference values for pixel 326A of increased precision delta image channel 326.
• FIG. 4 depicts the separation of delta image 320 into three different images, more or fewer separate image channels may also be used.
• multiple delta image channels may be encoded, optionally transmitted, and optionally decoded based on a desire to control bandwidth/quality, and the needs of different decoders ⁇ i.e., a device that doesn't support a gamut wider than the nominal range doesn't need the -0.75 - 1.0 channel transmitted to it. Similarly, a device that supports higher precision may subscribe to a delta image channel that provides that extra range). This approach to generating delta image channels is applicable to most existing file formats/image representations.
• the described delta image channels may be stored as explicit metadata, stored implicitly by "stuffing" the data into the file/stream where legal (perhaps after the end of the existing image in accordance with the file format's definition), in a separate but related file (“boxcar”), or file system fork.
• boxcar separate but related file
• the base image may be compressed and encoded using a widely supported image file format ⁇ e.g., JPEG and TIFF).
• the delta image (which may be separated into one or more delta image channels as described above) may be compressed and packaged as metadata that accompanies the base image according to the standards of the selected file format.
• devices that support the extended image format can generate the extended image from the base image and the delta image while devices that do not support the extended image format can simply disregard the delta image metadata and decode the base image in the usual manner.
• base image 510A may be subtracted from extended image 515A to generate delta image 520A.
• the subtraction may be performed as a computationally
• Base image 510A may be compressed (530A) and encoded as the payload portion of image file 540A in accordance with the standards of a widely supported image file format ⁇ e.g., the JPEG File Interchange Format (JFIF)).
• JFIF JPEG File Interchange Format
• Delta image (which may actually be split into several different image channels) may similarly be compressed (530A) and encoded in a metadata portion of image file 540A.
• the JFIF format includes multiple metadata channels.
• one or more delta image channels may be compressed and stored in one or more of these metadata channels.
• additional widely supported image file formats may also enable similar operation. Because the compression and encoding processes are dependent upon the selected file type and are known by those of ordinary skill in the art, these processes are not discussed in detail in this
• delta image channel data is likely to be significantly spatially coherent and, further, may be an unchanging DC level over large areas of the base image (where the raw signal is fully representable by the base).
• the delta image channel may be efficiently encoded using the same compression as the payload ⁇ i.e., base image
• there are other techniques that might further help such as a quadtree representation that would only encode the sparse areas where the delta image channel data is actually modulated.
• specular highlights require significant hyper unity values, and deeply saturated areas negative values.
• delta image channel data may be determined (computed) to include the effect of the lossy compression. This may be achieved, for example by compressing, and decompressing the base image data before performing the above- described subtraction, effectively encoding the compression error in a delta channel and thereby allowing a higher quality image to be decoded than the base. It has been determined that this is an additional use for the delta channel (over gamma, range, and precision).
• base image 510B may first be compressed (530B) and then decompressed (535B) before the image resulting from the compression/decompression is subtracted from extended image 515B.
• the compression/decompression operation ensures that the delta image is calculated from the version of the base image that will be used to reconstruct extended image 515B using image file 540B. This reduces error that may occur during the extended image reconstruction process when using a different version of the base image than that which was used to generate the delta image.
• the compressed base image may be encoded as the payload portion of image file 540B in accordance with the standards of a widely supported image file format.
• delta image 520B may be compressed and encoded in a metadata portion of image file 540B.
• the base image and delta image may need to be converted to the Y'C b C r color model supported by JFIF. Until this point, this conversion process has been assumed to be part of the compression and encoding of the base image and the delta image.
• the color model of the extended image and the base image it may be advantageous to convert the color model of the extended image and the base image and to perform the difference operation (between the extended image and the base image) in the converted color space.
• the payload portion of the selected file format is capable of encoding a wider range of colors than the color space of the base image (as is the case with the Y'C b C r color space of the JPEG standard as compared to the sRGB color space)
• RGB unit cube may be represented in the Y'C b C r 0-1 unit volume.
• Common Y'C b C r to R'G'B' converters typically clamp R'G'B' values to 0.0 - 1.0 so it may be "safe" to encode extended range
• base image 510C and extended image 515C may be converted from a first color model to a second color model (550).
• the second color model may be a required color model for encoding an image using a selected file format ⁇ i.e., the file format of image file 540C).
• base image 510C and extended image 515C are encoded using the base image format and the extended image format described above with respect to FIG.2 and the selected file format requires images to be encoded using the Y'CbQ color model
• base image 510C and extended image 515C are converted from the RGB color model to the JPEG Y'CbQ color model.
• converted base image 512C is subtracted from converted extended image 517C to generate delta image 520C.
• Delta image 520C is expressed in the converted color space. A portion of delta image 520C may be extracted and added back into converted base image 512C to generate adjusted base image 514C.
• the reference values of converted base image 512C may be adjusted towards the reference values of converted extended image 517C by adding a portion of delta image 520C to the extent that the resulting reference values of adjusted base image 514C are within an acceptable range.
• the reference values for a corresponding pixel of delta image 520C may be (- 0.2, -0.056, 0.307). Some or all of these reference values may be added back to the corresponding values of converted base image 514C to the extent that the resulting values are within an acceptable range of nominal reference values ⁇ e.g., 0 to 1).
• this may result in reference values for a corresponding pixel of adjusted base image 514C of (0.093, 0.278, 1) and remaining values for the corresponding pixel of delta image 520C of (0, 0, 0.297). Adjusted base image 514C and the remaining portions of delta image 520C ⁇ i.e., the portions that were not extracted) may then be compressed (530C) and encoded into a payload portion and a metadata portion, respectively, of image file 540C in accordance with the standards of the image file format of file 540C.
• the base image and the delta image may be compressed and encoded in parallel, perhaps using multiple encoding/decoding units.
• the "additional" encoding units don't have to be identical. That is, there may be an opportunity to use a different hardware encoder for the delta channel information (than the base) and thereby permit hardware acceleration for a different codec making the resulting device more versatile.
• the delta channel information could be encoded/decoded in parallel to the base image in a central processing unit (CPU) based codec with the base hardware
• the encoding of both the delta and base images may be CPU-based ⁇ e.g., using different cores of a multicore processing unit).
• the delta image has been separated into multiple delta image channels 620A-C as described above.
• base image 610 and delta image channels 620A-C may be compressed in parallel and encoded in image file 640.
• Base image 610 may be encoded in payload portion 645 of image file 640 and each of delta image channels 620A-C may be encoded in a separate metadata channel 650A-C of image file 640. Although three delta image channels are illustrated in FIG. 6, more or fewer delta image channels may be used.
• a delta image channel may encode the difference between a first compressed version of base image 610 and a second compressed version of delta image 610.
• base image 610 may be compressed in accordance with the standards of the image file format ⁇ i.e., the format of image file 640) and may also be
• the difference between the two compressed versions may be compressed and encoded as one of the delta image channels 620A-C.
• the separation of delta image channels 620A-C into separate metadata channels of image file 640 may enable the selective transmission or usage of the delta image channels.
• a recipient device if it is determined that a recipient device is incapable of utilizing one or more of the delta image channels, only those channels that are useful may be transmitted. For example, if a recipient device is capable of using the precision delta image channel to increase the precision of the displayed image represented by image file 640 but is incapable of utilizing the increased dynamic range, wide gamut, or compression difference channels, the delta image channels that
• the delta image channels may be extracted prior to transmission of image file 640.
• Recognition of downstream decoder capabilities can permit the transmitting station to manage bit-rate and deal with network congestion.
• the receiving decoder may selectively decode deltas based on known circumstances ⁇ e.g., it may choose to not decode negative delta values when a wide gamut display is not available).
• image file 640 includes identifier 655 and instructions 660 that are each stored in separate metadata channels of image file 640.
• Identifier 655 may link originally encoded base image 610 to the delta image channels. This linkage may be used to avoid the application of delta image data to an altered version of base image 610, which could be catastrophic. For example, if image file 640 is modified ⁇ e.g., the representation of base image 610 is rotated 90 degrees), the delta image data should not subsequently be used to attempt to regenerate the extended image.
• identifier 655 may be a hash of all or some portion of original payload portion 645.
• identifier 655 may be a unique identifier that is stored within original payload portion 645 (rather than in a separate metadata channel) and may include a format specific marker such as an extra JPEG restart marker that indicates that the data in payload portion 645 is the original data. Regardless of the specific implementation of identifier 655, any alteration to payload portion 645 would create a mismatch that could be utilized by instructions 660 to abort any subsequent attempt to regenerate the extended image using the payload portion 645 and metadata channels 650A-C. Instructions 660 may also include code that is utilized to reconstruct all or some portion of the extended image using some or all of metadata channels 650A-C.
• instructions 660 of image file 640 may determine which version of the image represented by image file 640 is generated. It may first be determined if the data in payload portion 645 matches identifier 655 (block 705). In one embodiment, it may be determined if a hash of payload 645 matches a hash of the originally encoded data in payload portion 645 that is stored in identifier portion 655. In another embodiment, it may be determined if payload portion 645 includes a unique identifier that was included with the originally encoded payload portion 645.
• payload portion 645 is consistent with the identifier (the "Yes" prong of block 705), it may then be determined if a device that will be used to display the image supports the extended image format (block 710). If payload portion 645 does not match the identifier (the "No" prong of block 705) or if the extended image format is unsupported (the “No” prong of block 710), payload portion 645 may be decompressed (block 715) to generate base image 610. If, however, the extended image version is supported (the "Yes" prong of block 710), payload portion 645 and metadata channels 650A-C may be
• instructions 660 may include code to define necessary conversions and sequences for generating extended image 730 from the base image and some or all of the delta image channels.
• Electronic device 800 may include processor 805, display 810, user interface 815, graphics hardware 820, device sensors 825 ⁇ e.g., proximity sensor/ambient light sensor, accelerometer and/or gyroscope), microphone 830, audio codec(s) 835, speaker(s) 840, communications circuitry 845, digital image capture unit 850, video codec(s) 855, memory 860, storage 865, and communications bus 870.
• Electronic device 800 may be, for example, a digital camera, a personal digital assistant (PDA), personal music player, mobile telephone, server, notebook, laptop, desktop, or tablet computer. More particularly, the disclosed techniques may be executed on a device that includes some or all of the components of device 800.
• PDA personal digital assistant
• Processor 805 may execute instructions necessary to carry out or control the operation of many functions performed by device 800.
• Processor 805 may, for instance, drive display 810 and receive user input from user interface 815.
• User interface 815 can take a variety of forms, such as a button, keypad, dial, a click wheel, keyboard, display screen and/or a touch screen.
• Processor 805 may also, for example, be a system-on-chip such as those found in mobile devices and include a dedicated graphics processing unit (GPU).
• GPU graphics processing unit
• Processor 805 may be based on reduced instruction-set computer (RISC) or complex instruction-set computer (CISC) architectures or any other suitable architecture and may include one or more processing cores.
• Graphics hardware 820 may be special purpose computational hardware for processing graphics and/or assisting processor 805 to process graphics information. In one
• graphics hardware 820 may include a programmable graphics processing unit (GPU).
• Sensor and camera circuitry 850 may capture still and video images that may be processed, at least in part, in accordance with the disclosed techniques by video codec(s) 855 and/or processor 805 and/or graphics hardware 820, and/or a dedicated image processing unit incorporated within circuitry 850. Images so captured may be stored in memory 860 and/or storage 865.
• Memory 860 may include one or more different types of media used by processor 805 and graphics hardware 820 to perform device functions.
• memory 860 may include memory cache, read-only memory (ROM), and/or random access memory (RAM).
• Storage 865 may store media ⁇ e.g., audio, image and video files), computer program instructions or software, preference information, device profile information, and any other suitable data.
• Storage 865 may include one or more non-transitory storage mediums including, for example, magnetic disks (fixed, floppy, and removable) and tape, optical media such as CD-ROMs and digital video disks (DVDs), and semiconductor memory devices such as Electrically Programmable Read-Only Memory (EPROM), and Electrically Erasable Programmable Read-Only Memory (EEPROM).
• Memory 860 and storage 865 may be used to tangibly retain computer program instructions or code organized into one or more modules and written in any desired computer programming language. When executed by, for example, processor 805 such computer program code may implement one or more of the operations described herein.

Landscapes

• Engineering & Computer Science (AREA)
• Multimedia (AREA)
• Signal Processing (AREA)
• Physics & Mathematics (AREA)
• General Physics & Mathematics (AREA)
• Theoretical Computer Science (AREA)
• Databases & Information Systems (AREA)
• Compression Or Coding Systems Of Tv Signals (AREA)
• Compression Of Band Width Or Redundancy In Fax (AREA)
• Processing Of Color Television Signals (AREA)

Priority Applications (5)

Applications Claiming Priority (2)

Publications (1)

Family

ID=51570860

Family Applications (1)

Country Status (7)

Families Citing this family (10)

Citations (1)

Family Cites Families (21)

• 2013
  • 2013-09-30 US US14/042,447 patent/US9036908B2/en active Active
• 2014
  • 2014-08-29 JP JP2016518692A patent/JP2016539526A/ja active Pending
  • 2014-08-29 KR KR1020167007694A patent/KR20160045884A/ko active Application Filing
  • 2014-08-29 WO PCT/US2014/053377 patent/WO2015047658A1/en active Application Filing
  • 2014-08-29 EP EP14767204.2A patent/EP3036743B1/en active Active
  • 2014-08-29 KR KR1020187003575A patent/KR101906779B1/ko active IP Right Grant
  • 2014-08-29 CN CN201480053626.7A patent/CN105580082B/zh active Active
  • 2014-09-12 TW TW103131645A patent/TWI553626B/zh active
  • 2014-09-12 TW TW105120559A patent/TWI567727B/zh active
• 2015
  • 2015-04-10 US US14/683,996 patent/US9204172B2/en active Active
• 2018
  • 2018-04-18 JP JP2018079752A patent/JP6703032B2/ja active Active

Patent Citations (1)

Non-Patent Citations (4)

Also Published As

Similar Documents

Legal Events