MXPA06003283A - Method for simulating film grain by mosaicing pre-computed samples - Google Patents

Abstract

Film grain is simulated in an output image (19) using pre-established blocks of film grain (408) from a pool (408) of pre-established blocks. Successive film grain blocks are selected by matching the average intensity of a block from the pool to the average intensity of a successive one of a set of M x N pixels in an incoming image. Once all of the successive pixel blocks from the image are matched to selected film grain blocks, the selected film grain blocks are"mosaiced", that is composited into a larger image mapped to the incoming image.

Description

European (AT, BE, BG, CH, CY, CZ, DE, DK, EE, ES, Fl, For two-letler codes and other abbreviations. Refer to the "Guid- FR, GB, GR, HU, IE, IT , LU, MC, NL, PL, PT, RO, SE, YES, ane Notes on Codes andAbbres? Aiions "appearing at tlte begin- SK, TR), OAPI (BF, BJ, CF, CG, Cl, CM, GA , GN, GQ, no ofeach regular issue of the PCT Gazetie, GW, ML, MR, NE, SN, TD, TG). Published: METHOD TO SIMULATE THE FILM GRAIN BY DIVIDING IN MOSAICS PRE-COMPUTED SAMPLES CROSS REFERENCE WITH RELATED REQUESTS This application claims priority in accordance with the 35 U.S.C. 119 (e) of Provisional Patent Application Serial No. 60 / 505,146, filed September 23, 2003, which is hereby incorporated by reference in its entirety.

FIELD OF THE INVENTION The invention is related to a technique for similar the film grain in an image.

BACKGROUND OF THE INVENTION The above publications, signed by the present inventors and assigned to the assignee herein, propose the simulation of film grain in a decoded video stream by first filtering the grain of the image before compression. The video stream undergoes compression and then transmission to a decoder for reception along with a message containing information about the film grain that was present in the stream before compression. After receiving the compressed video stream and the grain content message, the decoder decodes the compressed stream and then restores the original grainy appearance of the image by simulating the film grain based on the content of the gain information message. The gain information message may take the form of a complementary improvement information message (SEI) that accompanies the coded video stream. The film grain simulation provides great bit-rate savings for high-quality applications that require film grain conservation. However, this method of film grain simulation increases the complexity of the decoder since the decoder must reproduce and mix the film grain with the decoded video stream as specified by the film grain information message. This is why there is a need for a film grain simulation technique that overcomes the disadvantages of prior art BRIEF DESCRIPTION OF THE INVENTION In short, in accordance with the present principles, a method for simulating the film grain in an input picture block is provided. The method starts at the first compute an average value of at least one image parameter in the input image block. Then, a film grain block of at least one previously established set of film grain blocks whose image parameter is matched more with the image parameter of the input picture block is selected. The selected block is then mixed with the input image block. Selecting a film grain block from at least one set of pre-set film grain blocks for combination with the input image reduces the complexity associated with the simulation of film grain in the decoder, as it was out with the previous methods. In addition, the selection of a film grain block from at least one set of pre-established film grain blocks reduces artifacts that would otherwise arise when switching between independently generated film grain blocks.

BRIEF DESCRIPTION OF THE DRAWINGS Figure 1 illustrates the steps of a prior art method for simulating film grain by tiling the generated film grain samples independently. Figure 2 illustrates the steps of a prior art method for generating blocks of film grain for the method of Figure 1. Figure 3 illustrates a part of the decoder of the prior art for generating an output image containing the film grain. Figure 4 illustrates the steps of a method of conformance to an illustrative embodiment of the present principles to simulate the film grain by dividing the tiles into the pre-computed blocks of film grain. Figure 5 illustrates the steps of a method of conformance to an illustrative embodiment of the present principles for pre-establishing film grain blocks used in the method of Figure 4; and Figure 6 illustrates the steps of a method of conformance with another illustrative embodiment of the present principles to simulate the film grain by dividing the mosaic film grain blocks obtained from the characteristics of the input image.

DETAILED DESCRIPTION OF THE INVENTION In accordance with the present principles, the simulation of film grain in an image occurs by dividing the mosaics into pre-established individual film blocks. The term "dividing into tiles" of the individual film blocks involves the construction of an image composed of mixing the individual film grain blocks that are smaller in size. In order to appreciate the advantages of film grain simulation, to split the pre-established film grain samples into mosaics, a description of the film grain simulation process of the prior art will be very useful. Figure 1 illustrates the steps of a prior art method to simulate the film grain on a block-by-block basis. The method of Figure 1 starts by first starting the system variables during step 10. During step 12, a successive one of a group of blocks of NxM pixels (where N and M are each integers greater than zero) is read from an input image 13. The reading of the blocks during step 12 occurs in a raster scan order. Prior to step 12, the input image 13 typically undergoes filtration to remove or at least attenuate the film grain followed by decomposition to produce non-overlapped blocks of NxM pixels. The initial compression of the image can serve in itself to remove or attenuate the film grain. In some cases, the input image will not have initial film grain, either because the image was captured in digital form and / or generated by computer. Under such circumstances, adding gain where none existed will improve the image. After step 12, a numerical average of the intensity of the image is computed during step 14 only for the read-only block. Then, the selection of the film grain parameter occurs during step 16. The selection is carried out from a group of film grain parameters provided in a film grain message 17 accompanying the input image in accordance with the average intensity value computed during step 14. Typically, the film grain message 17 takes the form of a complementary improvement message (SEI) and, for that reason, the term "SEI message" will appear from now on when making reference to the message that contains the film grain information. Because the film grain characteristics can vary depending on the level of intensity, the selection made during step 16 can produce different pair, film grain parameters depending on the average intensity measured in the extracted picture block. With the use of the film grain parameters selected during step 16, a NxM pixel block of film grain samples is generated during step 18 to be used when creating a film grain image 19. Each of the film grain blocks generated through this process undergoes storage and composition to produce a film grain image that is associated with the size of the original input image. This process of composition constitutes the dividing into mosaics as described. After step 18, a revision is made during step 20 to see if the block selected during step 12 constitutes the last input block 11. When there are remaining blocks, the execution of the program branches to step 12 and to those follow Otherwise, step 22 is presented and the process ends. Figure 2 illustrates the individual steps that collectively comprise step 18 of Figure 1 to generate a block NxM film grain samples. The method of Figure 2 begins after the execution of step 200 during which the initiation of the system variables is performed, then, the generation of a block of NxM random values occurs during step 202 with the use of one or more parameters obtained from message 17 SEI. The block generated during step 204 undergoes a discontinuous cosine transformation (DCT) during step 204 to obtain a corresponding set of NxM frequency coefficients. Other transformations can be used, in addition to the DCT. The coefficients computed during step 204 undergo frequency filtering during step 206, with filtering coefficients computed in accordance with one or more parameters contained in message SEI. Frequency filtering controls the size of the simulated gain. The frequency filtered coefficients undergo an inverted discontinuous cosine (IDCT) transformation during step 208 to produce the film grain image 19. This process requires the specification of the noise deviation as well as the high and low cutoff frequencies that control the filtering process in the frequency domain. For coding applications, the SEI message 17 typically sends such information. After step 208, the process ends. Figure 3 illustrates a schematic block diagram of a portion of the decoder 300 that achieves the mixing of the film grain with the original image. An adder block 310 within the decoder 300 receives the decoded input image 13 as a first input. The film grain image 19 of FIG. 2 undergoes unlocking in an unlocking filter 314 before being received in a second input of the adder block 310. The summing block 310 adds the input image 13 and the film grain image 19 to produce an output image 316 containing the film grain. Since the individual blocks of the film grain have been generated independently, artifacts can be perceived in the transitions between blocks. In this way, the unlocking filter 314 becomes important to reduce the visual artifacts that result from the mosaic division of the film grain. As described, the method of the prior art for film grain simulation requires the generation of the film grain to be mixed with each of the processed pixel blocks of the input image. On the contrary, the simulation of film grain in accordance with the present principles achieves greater efficiency by limiting the generation of film grain by means of a joint creation process where a limited number of film grain blocks are computed. This measure greatly reduces the computational complexity of the film grain simulation process. Figure 4 illustrates the steps of a method according to a first embodiment of the present principles for the simulation of film grain. As described above, the film grain simulation method of Figure 5 advantageously prevents the generation of film grain on the fly by selecting individual pre-computed film grain blocks from a set of such blocks. The method of Figure 4 starts when the system variables first start during step 400. Next, a successive block of NxM pixels is read during step 402 of the input image 13. The reading of the blocks during step 402 occurs in a frame scan order. As described above, the input image 13 will typically have all of its gain removed or attenuated and will have been decomposed into non-overlapped blocks of NxM pixels (where N and M are integers, each greater than zero). The initial compression of the image can by itself serve to remove or attenuate the film grain. In some cases, the input image will not have initial film grain, either because the image was captured in digital form and / or generated by computer. Under such circumstances, adding gain where it did not exist will improve the image. After step 402, the average value of an image parameter, typically, the average intensity value is computed during step 404 for the newly read block during step 402. Next, a block of film grain is selected during step 406 of at least one set of preset film grain blocks, pre-computed in the manner described with respect to Figure 5. Better than relying solely on one set of film grain blocks., there may be multiple sets from which the selection may occur, depending on the average image intensity or dependent on one or more image parameters. When more than one block is available for the same intensity level and color component, a selection criteria must be specified. For example, the pseudo-random selection of blocks in the set can occur to avoid creating patterns when a small number of blocks is available. Transformed copies of the set of available film grain blocks can also be used. After the block selection during step 406, the selected block typically undergoes a transformation during step 410 to produce a film grain image block 19 to be mixed with the film grain image. After step 410, a check is made to determine if additional blocks remain during step 412. When there are additional blocks, the execution of the program branches to step 402 and the steps that follow. Otherwise, the execution of the program ends during step 414. Figure 5 illustrates the steps of a method in accordance with the present principles to generate pre-set film grain blocks 408 of Figure 4. The method of Figure 5 start at first start during step 500 to reset all system variables. Then, the selection of a group of film grain parameters of the SEI message 17 occurs during step 502. Next, a group of K blocks of the film grain samples (where K is an integer greater than zero) is generated for each group of parameters in the SEI message 17 during step 504. The different parameter groups will specify different color components and different intensity levels. A particular implementation, where K = 1, results in the lowest computing requirements, which generates only one block of film grain per group of parameters. However, to avoid creating patterns, it is preferable to use the largest number of blocks or even a different number of blocks for different intensity levels. After step 504, a revision is made during step 506 to determine if additional parameter groups remain. When there are additional groups left, the execution of the program branches to step 502 and the steps that follow. Otherwise, the execution of the program ends during step 508. The process described with respect to Figure 5 for the creation of a set of pre-set film grain blocks is completely dissociated from the film grain mixture that occurs in the decoder 10 as described in Figure 3. Despite the computational advantage of anticipating the generation of film grain, the film grain simulation method of Figure 4 incurs the inconvenience of not being alert as to whether the images input will make use of all the blocks stored in the set. Figure 6 illustrates an alternative embodiment of a film grain simulation method in accordance with the present principles, which makes use of the characteristics of the input image. As will be better understood below, the film grain method of Figure 6 does not create a set of film grain blocks before for all possible intensity levels present in message SEI. Instead, the method of Figure 6 undergoes the creation of a limited set of the film grain blocks and undergoes the updating of the block set depending on the characteristics of the input image. The method of Figure 6 starts first by performing the initiation during step 600 to reset all system variables. Step 602 follows step 600 during which the selection of a block of NxM pixels from the input image 13 occurs. After, the average of at least one image parameter (e.g., the average of the image intensity) of the previously selected image block is computed during step 604. During step 606 a revision is made for the availability of a block of film grain whose average intensity is more equal to the average intensity of a selected picture block. When there is no such block, the execution of the program branches to step 608, a group of parameters of the SEI message 17 is selected. Next, a NxM film grain block is generated from the parameters selected during step 610. The block generated during step 610 introduces the set of blocks 408 of pre-computed film grain. Step 612 proceeds to step 610. Step 612 proceeds to step 606 after the determination that a block of film grain in the set has an average intensity that is very much equal to that of the block of the selected image. During step 612, the most even film grain block of set 408 is selected in the film grain blocks. Then, the selected block typically undergoes a transformation during step 614 to tile the previously selected blocks to produce the film grain image 19. In this way, the transformation block is mixed with the film grain image. In step 616 a determination is made as to whether the block selected during step 602 constitutes the last block. When this is not the case, the execution of the program branches to step 602. Otherwise, the execution of the program ends during step 618. The method described above can calculate when the blocks are not available and not only when the set 408 is empty , but also when the existing blocks that have been used recently to mix. Other criteria can be contemplated to handle the creation of the set and the update process. The method of Figure 6 can progressively create and / or update the set 408 of the pre-set film grain blocks depending on the characteristics of the input images. For example, set 408 can be arranged to arrange a greater number of blocks within the most important intensity levels to avoid the creation of visual patterns. In accordance with this strategy, the distribution of the film grain blocks in the set 408 can progressively equalize the variations in image characteristics related to browning or rinsing.

The foregoing describes a technique for simulating the film grain in an image.

Claims (15)

1. A method for simulating film grain in an input picture block, characterized in that it comprises the steps of: a) computing an average value of at least one image parameter for the block; b) selecting a film grain block from at least one previously established set of film grain blocks whose image parameter is more evenly matched with the image parameter in the input image block; c) mix the selected film grain block with the input image block.

The method according to claim 1, characterized in that it further comprises the step of unlocking the selected film grain block before being mixed with the input image block.

3. The method according to claim 1, characterized in that the previously established film grain blocks are organized in at least one set based on the image intensity.

4. The method according to claim 1, characterized in that it also includes the step of updating the at least one set according to the characteristics of the input map.

5. The method according to claim 3, characterized in that a different film grain block is selected for at least one component of different color.

6. The method according to claim 1, characterized in that it also includes the step of transforming the selected block before the mixing step.

The method according to claim 1, characterized in that it further comprises the step of selecting a block of film grain from among a plurality of sets of the film grain blocks.

8. A method for simulating the film grain in an input image from which the film grain has at least attenuated and decomposed into input picture blocks, characterized in that it comprises: a) selecting one successive input magin blocks; b) computing an average value of at least one input parameter for the succeeding block; c) selecting from among at least one set of pre-set film grain blocks a block of film grain having a mage parameter that is more equal to the average value of at least one block parameter of the block successive; d) repeat steps (a) to (c) for all the pixel blocks in the image; and e) mixing the selected film grain blocks to produce an input image with film grain.

The method according to claim 8, characterized in that the previously established film grain blocks are organized in at least one set based on the image intensity.

10. The method according to claim 8, characterized in that it also includes the step of updating the at least one set of pre-established film grain blocks in accordance with the characteristics of the input image.

The method according to claim 8, characterized in that a different film grain block is selected for at least one component of different color.

The method according to claim 8, characterized in that it also includes the step of transforming the selected block before repeating steps (c) and (d).

The method according to claim 8, characterized in that it further comprises the step of selecting a block of film grain from a plurality of sets of film grain blocks.

The method according to claim 8, characterized in that it further comprises the step of unblocking the film grain block before repeating steps (c) and (d).

15. A method for simulating the film grain in an input block, characterized in that it comprises the steps of: (a) computing an average value of at least one image parameter for the block, (b) selecting a film grain block from at least one previously established set of film grain blocks whose image parameter is equalized more with the image parameter of the input block; (c) mix the selected film grain block with the input picture block.