AU2004218686B2 - A Combined Spatial and Transform Domain Method for Compressing Image Data into a Fixed Size Buffer - Google Patents

Description

S&F Ref: 681244

AUSTRALIA

PATENTS ACT 1990 COMPLETE SPECIFICATION Name and Address of Applicant: Actual Inventor(s): Address for Service: Invention Title: FOR A STANDARD PATENT Canon Kabushiki Kaisha, of 30-2, Shimomaruko 3chome, Ohta-ku, Tokyo, 146, Japan James David Clark, Jimes Philip Andrew Spruson Ferguson St Martins Tower Level 31 Market Street Sydney NSW 2000 (CCN 3710000177) A Combined Spatial and Transform Domain Method for Compressing Image Data into a Fixed Size Buffer ASSOCIATED PROVISIONAL APPLICATION DETAILS [33] Country [31] Applic. No(s) AU 2003905566 [32] Application Date 10 Oct 2003 The following statement is a full description of this invention, including the best method of performing it known to me/us:- 5815c 8 -1-

O

C-)

A COMBINED SPATIAL AND TRANSFORM DOMAIN METHOD FOR 00 0 COMPRESSING IMAGE DATA INTO A FIXED SIZE BUFFER.

\O

00 1,1 Field of the Invention 00 The present invention relates to digital image compression and in particular to image Scompression in raster image processing systems where memory resources are limited.

Background Data compression is used in a wide range of applications that store or transmit vast amounts of data. Examples of such applications are printers, scanners, facsimile machines, image archiving systems, and the Internet. The data is typically transmitted or stored as a document comprising continuous-tone images, text, rendered vector graphics, or a combination of these and/or other data formats. An important reason for compressing a document is to reduce the stored size of the data describing that document, while maintaining the integrity of the reproduction quality of the compressed document, as compared to the original.

In typical printing processes, compressed image data is generated in one pass of the original (source) image pixels. The image data is compressed into a temporary buffer or memory storage before being output as a copy of the original image data. Limitations on memory storage for a printing system can restrict the amount of image data that is buffered during the printing process. Techniques in the prior art for compressing an image in one pass with limited memory size typically rely on embedded coding (that is, the data is encoded in quality levels) of the original image data. If the amount of compressed data exceeds the fixed 681244 -2- O size of the memory storage during the printing of an image, then data from selected quality 00 0 levels is discarded, freeing up memory storage for the remainder of the compression process.

o Prior art techniques for compressing an image into a fixed size memory provide 00 IO either a one-pass spatially embedded compression/decompression system, or a one-pass 00 embedded compression/decompression of transformed data implementing transformation techniques such as the Discrete Cosine Transform (DCT). The spatially embedded technique can generate lossless compression which allows perfect reconstruction of the original image data from the compressed image data if all the compressed image data can fit into the fixedsize buffer. Unlike the spatially embedded technique, embedded compression of transformed data is typically lossy, whereby the reconstruction of the compressed image data is not exactly the same as the original image data.

The effectiveness of different compression methods to compress a particular image varies depending on the content of the image. For example, lossless compression techniques will typically suffer from low compression ratios (as compared to lossy compression techniques) in order to maintain perfect image data reconstruction. As another example, lossy coding such as baseline or progressive JPEG, when applied to text or regions of high contrast (eg. rendered vector graphics), typically results in visually perceptible errors or artefacts between the original and reconstructed image data when compressed at high compression ratios.

An approach of the prior art to overcome the difficulties of achieving good compression for regions containing high contrast data, such as text, and slowly varying data, such as high resolution photographic images, within the one image is to combine two different compression methods within the one compression system. To achieve good compression and 681244 -3- O image quality, many approaches of the prior art either pre-process the input image data before 00 0 encoding, or post-process the decompressed image, or incorporate a combination of the two IND processes. For example, some pre-processing methods of the prior art segment documents 00 INO into regions of text and regions of images, and the data corresponding to each region is 00 compressed separately with an appropriate compression scheme. Such pre-processing techniques can be computationally expensive to implement, and the effectiveness of such techniques typically relies on the type of image that is being segmented.

For transform domain techniques, when compressing some images into a fixed size buffer, the process of deleting scan data can result in large errors in the reconstructed image pixel values, which result in visual artefacts. In particular, the visual quality of images comprising text, vector graphics, or shapes possessing well-defined edges suffers noticeably if a significant number of scans are discarded using transform compression methods.

Summary It is an object of the present invention to substantially overcome, or at least ameliorate, one or more disadvantages of existing arrangements.

According to a first aspect of the present disclosure, there is provided a method of compressing an image into a memory storage of fixed size, said method comprising the steps of: coding said image into a plurality of compressed scans, wherein at least one of said compressed scans comprises spatial domain compressed data, and at least one other of said compressed scans comprises transform domain compressed data; and storing said compressed scans in said memory storage wherein, during the storing of said compressed scans, if it is determined that said memory storage is full, the compressed 681244 -4- 0 scan currently being stored in said memory storage overwrites at least part of a perceptually 00 0 less significant compressed scan already stored in said memory storage.

IN According to another aspect of the present disclosure, there is provided a method of oO 00 00 compressing an image into a memory storage of fixed size, said method comprising the steps 00 5 of: progressively buffering said image as a sequence of bands, each band comprising one or more lines ofpixels; coding said bands into a plurality of compressed scans, wherein at least one of said compressed scans comprises spatial domain compressed data, and at least one other of said compressed scans comprises transform domain compressed data; and storing said compressed scans in said memory storage wherein, during the storing of said compressed scans, if it is determined that said memory storage is full, the compressed scan currently being stored in said memory storage overwrites at least part of a perceptually less significant compressed scan already stored in said memory storage.

According to yet another aspect of the present disclosure, there is provided a system comprising: a memory storage for storing compressed data; a one pass compressor for compressing image data into compressed data, said compressed data being represented as a plurality of scans, said compressor comprising: a first encoder element operating in the spatial domain for producing at least one scan of spatial domain compressed data, and a second encoder element operating in the transform domain for producing at least one scan of transform domain compressed data, and 681244

,I

O a data manager for receiving said compressed data from said one pass compressor, 00 C and for determining which scans of said compressed data to forward to said memory storage based on available storage space in said memory storage.

oO 00 IND Other aspects of the invention are also disclosed.

00 Brief Description of the Drawings One or more embodiments of the present invention will now be described with reference to the drawings, in which: Fig. 1 shows a schematic block diagram of a raster image processing system; Fig. 2 shows a schematic block diagram of the preferred arrangement of the compressor operating within the raster image processing system of Fig. 1; Fig. 3 shows a flow diagram of a method of partitioning a pixel component value of colour raster image data into upper and lower partition values; Figs. 4A and 4B illustrate the method of partitioning the bit-planes of a pixel component by way of an example; Fig. 5 shows a schematic block diagram of the preferred arrangement of a spatial domain encoder within the compressor of Fig. 2; Fig. 6A shows a flow diagram of a method of managing the entropy encoding of difference data; Fig. 6B shows a flow diagram of a step of the method in Fig. 6A in more detail; Fig. 7 illustrates the preferred arrangement of a memory management table used in the raster image processing system of Fig. 1; Fig. 8A shows a flow diagram of a method of managing the storing of encoded image data in a final output buffer of fixed memory size; 681244 -6- O Fig. 8B shows a flow diagram of a step of the method in Fig. 8A in more detail; 00 0 Fig. 9 shows the preferred arrangement of a transform domain encoder used within IND the compressor of Fig 2; 00 00 Fig. 10 illustrates the zigzag ordering of DCT coefficients; 00 Fig. 11 shows the partitioning into spectral bands and bit-planes of a quantised block of uncompressed transform coefficient data in the cyan channel; Fig. 12A shows a flow diagram of a method of managing the entropy encoding of quantised transformed coefficient data; Fig. 12B shows a flow diagram of a step of the method in Fig. 12A in more detail; Fig. 13 shows a schematic block diagram of the preferred arrangement of the decompressor operating within the raster image processing system of Fig. 1; Fig. 14 shows a schematic block diagram of the preferred arrangement of the spatial domain decoder used within the decompressor of Fig. 13; Fig. 15 shows a schematic block diagram of the preferred arrangement of the transform domain decoder used within the decompressor of Fig. 13; Fig. 16 shows a flow diagram of a method of reconstructing bit-planes from upper partitioned pixel data and lower partitioned data; Figs. 17A and 17B illustrate by way of an example the recombining of the upper and lower partition values and into the reconstructed pixel component value; Fig. 18 shows a schematic block diagram of an alternative arrangement of the compressor operating within the raster image processing system of Fig. 1; Fig. 19 illustrates a segment of the spatial domain encoded data formed by the spatial domain encoder from one or more blocks of raster image data; 681244 -7- O Fig. 20 shows a segment of example spatial domain encoded data formed by the 00 0 spatial domain encoder; and INO Fig. 21 is a schematic block diagram of a general-purpose computer upon which 00 ISO elements of the raster image processing system of Fig. 1 can be practiced.

00 Description of Preferred Embodiment Where reference is made in any one or more of the accompanying drawings to steps and/or features, which have the same reference numerals, those steps and/or features have for the purposes of this description the same function(s) or operation(s), unless the contrary intention appears.

When printing a document from a personal computer, the document is typically converted from some page description language to pixel data for reproduction by a printer.

This process is referred to as rendering. Compression is often used in the rendering process in order to facilitate a cost-effective solution, as compressed data requires less memory for storage.

The document data that is compressed during the rendering process often comprises a combination of bi-level data (for example, text), and continuous tone data (for example, natural images). Such document data is referred to hereinafter as either image pixel data or just image data.

A "faithful" or "good quality" reproduction of a document printed from a personal computer is an important feature to the user. For example, it is desirable for the quality of a printed document to be as close as possible to its original form, either represented as an electronic document, as a hardcopy on paper, or viewed on a computer monitor.

681244 -8- O Solutions, performed in hardware and/or software, that are dedicated to compressing 00 0 such combinations of data types typically use different encoding sub-systems for various IND predetermined regions of the image data. However, an increase in design complexity 00 IDintroduced by incorporating multiple encoding sub-systems typically results in an increase in 00 design cost and/or a decrease in speed of processing of the system performing the printing process.

It is desirable to store enough compressed image data during the printing process so that the reproduced image data is exactly the same as, or is as close as possible to, the original image data. The quality of the image data reproduced from the compressed image data that is stored during the printing process will vary depending on the compression method used, the content of the original image, and the size of the buffer used to store the compressed data. The size of the buffer is normally fixed. A technique often used by some printing systems when all of the compressed image data do not fit into the buffer is to discard a portion of the compressed image data, thereby sacrificing the accuracy of the reconstructed image data for a smaller compressed image data size.

A spatial domain encoding method will typically not perform as well as a transform encoding method when compressing natural images into a fixed buffer size, where the size of the buffer is much smaller than the size of the original image data. Further, the encoding of the image pixels using a transform encoding method typically gives a finer degradation of compressed image quality when compressing into a fixed sized buffer, as compared to a spatial encoding method.

When used in a rendering pipeline, the inventors have ascertained that the compression performance of a transform encoding method is acceptable for many types of 681244 -9- O image data. However, for images that contain well-defined edges or text, and do not compress 00 0 completely into the fixed-size buffer, significant visual artefacts often occur. Further, the inventors have ascertained that such image data has certain characteristics that may be 00 0O exploited in order to reduce and limit the maximum pixel error generated during the 00 compression process.

Fig. 1 shows a schematic block diagram of a raster image processing system 100.

The components of the raster image processing system 100 described in detail below may be specially constructed for their respective purposes. In one implementation the raster image processing system 100 is realised using a general purpose computer 2100, such as that shown in Fig. 21 wherein the components of the raster image processing system 100 are implemented as software executing within the computer system 2100. The software may be stored in a computer readable medium, including the storage devices described below, for example. A computer readable medium having such software recorded on it is a computer program product.

The computer system 2100 is formed by a computer module 2101, input devices such as a keyboard 2102 and mouse 2103, and output devices including a printer 2115 and a display device 2114. The computer module 2101 typically includes at least one processor unit 2105, and a memory unit 2106, for example formed from semiconductor random access memory (RAM) and read only memory (ROM). The module2101 also includes an number of input/output interfaces including a video interface2107 that couples to the video display2114, an I/O interface2113 for the keyboard2102 and mouse2103, and an interface 2108 for the printer 2115. A storage device 2109 is provided and typically includes a hard disk drive 2110 and a floppy disk drive 2111. A CD-ROM drive 2112 is typically 681244 O provided as a non-volatile source of data. The components 2105 to 2113 of the computer 00 C module 2101, typically communicate via an interconnected bus 2104 and in a manner which in a conventional mode of operation of the computer system 2100 known to those in 00 ICthe relevant art.

00 Typically, the software for implementing the raster image processing system 100 is resident on the hard disk drive 2110 and read and controlled in its execution by the processor 2105. Intermediate storage of the data generated during encoding, and described in detail below, is accomplished using the semiconductor memory 2106, possibly in concert with the hard disk drive 2110.

In alternate implementations the raster image processing system 100 is realised on other apparatus. For example, the raster image processing system 100 may be realised on a controller, embedded in a printer. The embedded printer controller uses a digital signal processor or an application specific integrated circuit (ASIC), Dynamic Random Access Memory (DRAM) and other components. A host computer transmits a page description language, or some similar high-level page description, such as a display list, to the embedded printer controller. The embedded controller converts the page description, of one or more pages, into a raster image, in raster order, preferably 8 raster lines at a time, compressing the 8-raster lines using the method described herein, into the DRAM memory. Thus the embedded printer controller converts the page description into a compressed raster image in one-pass of raster image, in a manner that does not overflow the fixed size of the DRAM memory. At some later stage the compressed raster image stored in the memory is decompressed and printed by the printer engine in the printer.

681244 -11- O Referring again to Fig. 1, the raster image processing system 100 comprises a 00 O compressor 104, a final output buffer 106 having a fixed size, and a decompressor 108.

IDDuring the encoding process, the compressor 104 accepts colour raster image data 102 as 00 00 input. The raster image data 102 may be generated, for example, from a computer application 00 5 such as a graphics rendering package. The compressor 104 outputs encoded data 105 into the final output buffer 106. During the decoding process, the decompressor 108 accepts encoded data 107 from the final output buffer 106, and outputs reconstructed image data 110.

The colour raster image data 102 is typically represented using four channels, those channels representing each of a cyan component a magenta component a yellow component and a black component Each channel is typically represented using 8 bits of precision. Other colour representations may also be used, such as representing the colour raster image data 102 using one red, one green and one blue component, each having 12 bits of precision.

Fig. 2 shows a schematic block diagram of the preferred arrangement of the compressor 104 operating within the raster image processing system 100 of Fig. 1. The compressor 104 includes a bit-plane splitter 202 for partitioning each colour channel of the colour raster image data 102 into two bit-streams of data, namely an upper partitioned bitstream of data 203 and a lower partitioned bit-stream of data 205. A spatial domain encoder 204 is provided for encoding, in the spatial domain, each colour channel of the upper partitioned bit-stream of data 203 into spatial domain encoded bit-stream data 207. A transform domain encoder 206 is also provided for encoding, in the transform domain, each colour channel of the lower partitioned bit-stream of data 205 into transform domain encoded bit-stream data 209. A scan output manager 208 manages the storing of the encoded bit- 681244 -12- O streams data 207 and 209, together forming the encoded data 105, into the final output buffer 00 C106 of fixed memory size.

The bit-plane splitter 202 preferably buffers the colour raster image data 102 so as to 00 IND operate on 8 by 8 pixel blocks of data of the respective channels. This is typically done by 00 buffering 8 scan-lines of the data of each channel, followed by the sequential processing of each 8 by 8 pixel block in the buffered scan-lines. Each pixel colour component of an 8 by 8 pixel block is partitioned into two streams of contiguous bits.

Fig. 3 shows a flow diagram of a method 300 of partitioning an 8-bit pixel (colour) component value PORIG of the colour raster image data 102 into upper and lower partition values Pup and PLOW, performed by the bit-plane splitter 202. In the preferred implementation upper and lower partition values Pup and PLOW are the 4 most significant bit-planes partition value PUP4, and the 4 least significant bit-planes partition value PLOW4 respectively.

The method 300 starts at step 302 and proceeds to step 304 where the partitioned values PUP4 and PLOW4 are initialised to zero. The upper 4 bits, that is the 4 most significant bits (4 MSB) of the pixel component value PORIG are copied to the 4 least significant bits (4 LSB) of the upper partition value PUP4 in step 306. The bit-plane splitter 202 also, in step 308, copies the lower 4 bits, that is the 4 least significant bits (4 LSB) of the pixel component value PORIG to the 4 least significant bits of the lower partition value PLOW4.

The bit-plane splitter 202 then determines in step 310 whether the least significant bit of the upper partition value PUP4 is equal to 1. If it is determined that the least significant bit of the upper partition value PUP4 is equal to 1 then the lower partition value PLOW4 is point transformed at step 312 to a new lower partition value PLOW4 15 PLOW4. The point transform describes a mapping of one pixel component value to another value, irrespective of 681244 -13- O the values of neighbouring pixel components in the least significant partitioned bit-planes. If 00 0 it is determined in step 310 that the least significant bit of the upper partition value Pup4 is Iequal to 0, or from step 312, the method 300 proceeds to step 314 where it ends.

\O

00 IND In an alternative implementation, the point transformation performed in step 312 may 00 5 be omitted, or replaced by another point transformation method.

The advantage of point transforming in step 312 the lower partition value PLOW4 based on the least significant bit-plane of the upper partition value Pup4 is that better compression results can be produced than for the alternate implementation where no such point transformation step 312 is present. Step 312 "smooths" discontinuities in the lower partition value PLOW4 produced when partitioning the raster image data 102 by the bit-plane splitter 202. For example, consider compressing bi-level image pixel data taking the pixel component values PORIG 0 or 255. The respective lower partition values PLOW4 are 0 and and the lowest significant bit value of the respective upper partition values Pup4 are 0 and 1.

The point transformation of step 312 produces a point transformed lower partition value PLOW4 of 0 for either of the pixel component values PORIG 0 or 255. In this example, the point transformation eliminates all contrast in the lower partition values PLOW4. A constant lower partition value PLOW4 0 for an 8 by 8 pixel block results in highly compressible (and lossless) compressed transform domain scan data 209.

The partition values PUP4 and PLOW4 may be represented by an arbitrary number N of bits. In order to maintain all the information of the pixel component value PORIG, the number N must be greater than or equal to 4. For example, the upper partition value PUP4 may be stored as a 16-bit variable, and the lower partition value PLOW4 may be stored as a variable.

681244 -14- O Figs. 4A and 4B illustrate the method 300 of partitioning the bit-planes of a pixel 00 C component by way of an example. Referring first to Fig. 4A where an example is illustrated of how an 8-bit upper partition value PuP4 is obtained from an 8-bit pixel component value 00 IN PORIG of the colour raster image data 102. The magnitude of each bit-plane is represented by 00 an integer on the horizontal axis labelled Magnitude Bit-plane. The least significant bit (LSB) Sof a pixel is denoted by a Magnitude Bit-plane value of 0. The most significant bit (MSB) of a pixel is denoted by a Magnitude Bit-plane value of 7. In the example the 8-bit pixel component value PORIG is equal to 86. The 4 most significant bit-planes of the pixel component value PORIG are labelled as the "UPPER 4 BITS", whereas the 4 least significant bit-planes of the pixel component value PORIG are labelled as the "LOWER 4 BITS". The values of the magnitude bit-planes for the "UPPER 4 BITS", as read from the most significant to the least significant bit-plane, are 0, 1, 0, and 1.

The pixel component value PORIG is right shifted by 4 and the resulting value copied to the upper partition value PUP4. The 4 most significant bits of Pup4, labelled "PADDED ZEROS", are filled with zeros after shifting. The 4 least significant bits of the upper partition value PUP4, labelled "UPPER 4 BITS", now comprise the 4 most significant bits of the pixel component value PORIG. The upper partition value Pup4 is Referring now to Fig. 4B where an example is illustrated of how an 8-bit lower partition value PLOW4 is obtained from an 8-bit pixel component value PORIG of the colour raster image data 102. In the example the pixel component value PORIG is again equal to 86.

The "UPPER 4 BITS" of the pixel component value PORIG are first masked off by performing an AND operation between the pixel component value PORIG and the value 15 (binary 00001111). In other words, the "UPPER 4 BITS" are set to zero. The result is assigned to the 681244 C0 lower partition value PLOW4, which now has a value equal to 6. Because the least significant 00 0 bit of the upper partition value PUP4 is equal to 1 (see Fig. 4A), the lower partition value PLOW4 IND is point transformed to a new lower partition value PLOW4 15 PLOW4, which equates to 9.

00 00D Referring again to Fig. 2, the partitioned bit-streams of data 203 and 205 resulting from an 8 by 8 pixel block is output to encoders 204 and 206 respectively, with the upper and lower partitioned bit-steams of data 203 and 205 being formed from the upper and lower partition values Pup4 and PLOW4 respectively. In the preferred arrangement, the spatial domain encoder 204 buffers the upper partition values PUP4, and the transform domain encoder 206 buffers the lower partition values PLOW4 before processing as described below.

Fig. 5 shows a schematic block diagram of the preferred arrangement of the spatial domain encoder 204 within the compressor 104 of Fig. 2. The spatial domain encoder 204 encodes the upper partitioned bit-stream of data 203 accepted from the bit-plane splitter 202, and outputs the result 207 to the scan output manager 208. The spatial domain encoder 204 comprises a pixel difference unit 502 and an entropy encoder manager 504.

The pixel difference unit 502 accepts the upper partitioned bit-stream of data 203 as input, taking the difference between successive upper partition values PUP4 in the upper partitioned bit-stream of data 203, and outputs the differences to the entropy encoder manager 504.

The pixel difference unit 502 preferably operates on the 8-bit upper partition values PUP4 resulting from an 8 by 8 pixel block to produce 8-bit difference data Dy. The differenced data Dij may be represented by a 5-bit value, where one bit is used for the sign of the differenced value, and 4-bits are used for the magnitude of the differenced value. If the difference data Dy for each colour channel comprises less than 8 bits of partitioned data, as is 681244 -16- O the case in the preferred implementation, then the difference data Dy may be represented by an 00 0 8-bit variable. For each row and column of the input 8 by 8 block of upper partition values N f{Pup 4 }ij, difference data Di is calculated. The row and column indices of the upper partition 00 IO values {PUP4 ij and difference data Dij are i and j respectively. The 00 difference data Dij is preferably calculated as follows: 0 Do,o {PuP4 }0,0 when i=0 andj=0; Di,o PUP4 }i,0 PUP4 whenj 0 and i and (1) Di, {PuP4 i,j PUP4}i,j-i when i and j Alternative methods of computing the difference data Dij are possible. The blocks of difference data Dj are passed to the entropy encoder manager 504 where they are converted to the descriptors, which in turn are entropy encoded. Preferably the modelling used to convert each 8 by 8 block of difference data Dy to produce descriptors closely resembles the predictive models implemented for the DC and AC Discrete Cosine Transform (DCT) coefficients in the JPEG standard for baseline encoding. That is, the difference data value Doo is considered analogous to the DC coefficient from the DCT based JPEG encoder model, and the remaining difference data values Dy where i 0,j 0 are considered analogous to the AC coefficients thereof. Accordingly, references to the DC difference data and AC difference data hereinafter will correspond to the difference data values Doo and Dy (i 0, j 0) respectively. As in baseline JPEG encoding, the DC difference data Doo is preferably modelled by Differential Pulse Code Modulation (DPCM) coding using the DC difference value Doo of the previous 8 by 8 block of the same colour channel as a 1-D predictor. However, unlike baseline JPEG encoding, the AC difference data Dy is preferably not modelled by rearranging the AC difference data values Dy into a 1-D linear array following a zigzag index sequence. (Zigzag 681244 -17- O ordering is described in detail below). Instead, the AC difference data Di is preferably 00 O arranged into a l-D linear array or vector such that L( 8i j) wherein i and j 1-10 are the row and column indices described above.

00 -0 In what follows the terms descriptor and scan are used interchangeably. Preferably, 00 C 5 the entropy encoder manager 504 performs Huffman entropy encoding, and the scans are defined in accordance with the scan partitioning described with reference to the Scan Table 1 shown below. Each row marked with an asterisk in the Scan number column of the Scan Table 1 specifies one (spatial domain encoding) scan, whereas each row not marked with an asterisk in the scan number column specifies one (transform domain encoding) scan. The entropy encoder manager 504 encodes the scans in row order.

The scan number column of the Scan Table 1 lists scans according to a nominal order of perceptual significance. The component column specifies which colour channel, or component, is encoded in that scan. Component 0 is the cyan channel component 1 is the magenta channel component 2 is yellow channel and component 3 is the black channel The Ss Se column specifies the start (Ss) and end (Se) index, following the sequence of difference data Di described by the 1-D array above, of a contiguous sub-sequence of differenced data DU that is encoded in the scan. For example, the fifth row of Scan Table 1 specifies that in the fifth scan all AC values of the difference data DU of the cyan colour channel is encoded. In another example, the third row of Scan Table 1 specifies that in the third scan the DC difference data Do, 0 of the yellow colour channel be encoded.

681244 -18- Scan number Component Ss Se Ah Al 1* 0 0-0 0,0 2* 1 0-0 0,0 3* 2 0-0 0,0 4* 3 0-0 0,0 0 1-63 0,0 6* 1 1-63 0,0 7* 2 1-63 0,0 8* 3 1-63 0,0 9 0 0-0 0,1 1 0-0 0,1 11 2 0-0 0,1 12 3 0-0 0,1 13 0 1-32 0,1 14 1 1-32 0,1 2 1-32 0,1 16 3 1-32 0,1 17 0 33-63 0,1 18 1 33-63 0,1 19 2 33-63 0,1 3 33-63 0,1 21 0 0-0 22 1 0-0 23 2 0-0 24 3 0-0 0 1-32 26 1 1-32 27 2 1-32 28 3 1-32 29 0 33-63 1 33-63 31 2 33-63 32 3 33-63 Scan Table 1 The Ah Al column specifies the bit precision high and bit precision low parameters respectively as used in the successive approximation mode of progressive JPEG. In the preferred implementation, for all scans relevant to spatial domain encoding, the bit-planes are encoded such that Ah 0 and Al 0 (as used in baseline JPEG). Choosing the values Ah 0 681244 -19- ,a o O and Al 0 ensures that all the bit-plane information of the spatial domain data is retained 00 C during the encoding occurring in the spatial domain encoder 204.

The preferred arrangement of scans of a particular colour channel is such that the 00 00 entropy encoder manager 504 encodes scans of that particular colour channel that differ in 00 5 their perceptual contribution (or significance) to the resultant image. For the spatial domain Sencoding scans in particular (referred to hereinafter as spatial scans), the preferred ordering ranks the DC difference data D 00 as more perceptually significant than the AC difference data Preferably all scans comprising colour channels of the same difference data (DC or AC) contain the same perceptual contribution to the resultant image. Further, all the spatial domain encoding scans have more perceptual significance than the transform domain encoding scans (referred to hereinafter as transform scans), given the information contained in the 4 most significant bit-planes partition value Pup4 is more significant to the resultant image than the information contained in the 4 least significant bit-planes partition value PLOW4.

The entropy encoder manager 504 splits the blocks of differenced data Dj into spatial scans described by the Scan Table 1 and entropy encodes those scans, outputting the encoded data 207 into scan bit-streams such that each scan bit-stream contains a portion of data from each block difference data Di• in raster block order. In the preferred implementation, as in the baseline specification for JPEG, scans are entropy encoded differently according to the type (DC or AC) of that scan. In this case, two entropy encoders are employed to encode scans of the two different spatial scan types: DC or AC. Each spatial scan listed in the Scan Table 1 is encoded using an appropriate entropy encoder. Other entropy encoder procedures are possible.

681244 O Fig. 6A shows a flow diagram of a method 600 performed by the entropy encoder 00 0manager 504 of managing the entropy encoding of difference data DO. Method 600 starts at Istep 602 and proceeds to step 604 where, each time the entropy encoder manager 504 receives

\O

00 IDa block of differenced data Dij of a particular colour channel, the DC spatial scan of that block 00 is entropy encoded.

SFig. 6B shows a flow diagram of step 604 (Fig. 6A) in more detail. Step 604 starts in sub-step 610 and proceeds to sub-step 612 where the entropy encoder manager 504 determines whether a DC spatial scan of the colour channel currently being encoded can be found in the Scan Table 1. If it is determined in sub-step 612 that a DC spatial scan cannot be found, then step 604 ends in sub-step 622. Alternatively, if it is determined in sub-step 612 that a DC spatial scan can be found, then the entropy encoder manager 504 selects this DC spatial scan as a current scan to process and determines in sub-step 614 whether the current scan is active.

If it is determined in sub-step 614 that the current scan is inactive, then step 604 ends in substep 622. A scan is inactive if an active flag entry in a Memory Management Table 700, described below with reference to Fig. 7, corresponding to that scan is set to inactive.

Alternatively, if it is determined in sub-step 614 that the current scan is active (that is the active flag entry in the Memory Management Table 700 is set to active), then the entropy encoder manager 504 encodes in sub-step 616 the current scan using Huffman entropy encoding for DC scans mentioned above, and described in the JPEG standard for baseline encoding. Also in sub-step 616 the encoded scan data is output into a bit-stream that is shifted into an output buffer (not illustrated) of the entropy encoder manager 504. This output buffer is 32 bytes, or some similar size selected to allow a fraction of the output buffer to be 681244 -21-

C-)

O efficiently transferred through the memory sub-system used to access the final output buffer 00 0 106. There is one output (bit) stream corresponding to each encoded scan.

sub-step 616 step 604 proceeds to sub-step 618 where the entropy encoder 00 0O manager 504 determines whether the output buffer contains at least 1 word of encoded scan oO N, 5 data. In the preferred implementation, 1 word is 1 byte of data. If it is determined in sub-step S618 that the output buffer does not contain at least 1 word of encoded scan data, then step 604 ends in sub-step 622.

Alternatively, if it is determined that the output buffer does contain at least 1 word of encoded scan data, then the entropy encoder manager 504 proceeds to sub-step 620 where one word from the output buffer is shifted to the scan output manager 208, and that word is subsequently removed from the output buffer. The order for shifting and removing words from the output buffer follows a first-in, first-out process. That is, for a sequence of words present in the output buffer, the word that was shifted first into the output buffer will be the word that is shifted first to the scan output manager 208. After the entropy encoder manager 504 has shifted the word to the scan output manager 208, step 604 returns to sub-step 618 where the entropy encoder manager 504 again determines whether the output buffer contains at least 1 word of encoded scan data.

After entropy encoding of the DC scan of the current block in step 604 is complete, and referring again to Fig. 6A, the AC spatial scan of that block is entropy encoded in step 606. Step 606 has the same sub-steps as those of step 604 described with reference to Fig. 6B, with the exception that the scan type encoded in step 606 is an AC scan of the current colour channel, and not a DC scan. As a result, AC Huffman entropy encoding for the spatial domain encoding is used to encode the AC scan type.

681244 -22- O Following the entropy encoding of the AC scans in step 606, the method 600 ends in 00 0 step 608.

-D Referring again to Fig. 2, the preferred arrangement of the transform domain encoder 00 206 within the compressor 104 is now described in more detail with reference to Fig. 9. The 00 transform domain encoder 206 encodes the lower partitioned bit-stream of data 205 accepted from the bit-plane splitter 202, and also outputs its result 209 to the scan output manager 208.

The transform domain encoder 206 comprises a Discrete Cosine Transform (DCT) unit 902 and an entropy encoder manager 904.

The DCT unit 902 accepts the lower partitioned bit-stream of data 205 as input, and outputs the result to the entropy encoder manager 904 for encoding. The DCT unit 902 may be any known DCT unit, which operates in conformity with the JPEG standard. Alternatively a lossless DCT may be used. A lossless DCT transforms integer pixels to integer transform coefficients. The DCT unit 902 buffers the lower partitioned bit-stream of data 205 for each channel so as to operate on 8 by 8 blocks of partitioned pixel data.

The transform domain encoder 206 preferably follows a particular mode of progressive JPEG. If a lossless DCT is employed in the DCT unit 902, then the transform domain encoder 206 is not strictly speaking conformant to the JPEG standard. However, a transform domain encoder 206 employing a lossless DCT unit 902 differs only in the transform from JPEG. Other alternatives are also possible. A particular mixture of the progressive modes of successive approximation and spectral approximation is used by the transform domain encoder 206.

In accordance with the JPEG standard, the DCT unit 902 performs a discrete cosine transform on each colour channel of each 8 by 8 pixel block of partitioned pixel data to 681244 -23 O produce transformed data or a transform coefficient block, and the transformed data is 00 0 quantised to produce a quantised block of data. For the purposes of this description, the D quantisation is performed in the DCT unit 902. A table of quantisation values for each colour 00 channel is used during the quantisation performed in the DCT unit 902, and each quantisation 00 5 value q 2 where the exponent n, The row and column indices of the 0 transform coefficient block and the quantisation values are i and j respectively.

The quantised blocks are passed to the entropy encoder manager 904 where they are converted to descriptors, which in turn are entropy encoded in accordance with progressive JPEG. Preferably, the entropy encoder manager 904 performs Huffman entropy encoding, and the scans are defined in accordance with the scan partitioning described in the Scan Table 1 described above. As set out above, each row not marked with an asterisk in the scan number column of the Scan Table 1 specifies one (transform domain encoding) scan. For the transform domain encoded scans, the Ss Se column specifies the start (Ss) and end (Se) index, following a JPEG zigzag sequence of DCT coefficients of a contiguous sub-sequence of DCT coefficients that is encoded in the scan. The JPEG zigzag sequence is illustrated in Fig.

Referring to the Scan Table 1, the thirteenth row for example specifies that in the thirteenth scan DCT coefficients 1 to 32 of the cyan colour channel are encoded, which corresponds to the first 32 AC coefficients (1 to 32) in zigzag sequence illustrated in Fig. The preferred arrangement of transform domain encoded scans (referred to hereinafter as transform scans) of a particular colour channel is such that the entropy encoder manager 904 encodes transform scans of the particular colour channel that differ in their perceptual contribution (or significance) to the resultant image. As an example, a quantised 681244 -24-

,I

O block 1100 of the cyan channel, partitioned into spectral bands and bit-planes, is illustrated in 00 O Fig. 11. The scan partitioning information for the cyan channel is obtained from transform INDscans with a component value 0 in the Scan Table 1. The vertical axis 1110 of the quantised 00 INDblock 1100 describes the magnitude bit-plane of the coefficients. The horizontal axis 1105 of 00 5 the quantised block 1100 describes the zigzag indexing of coefficients in the block 1100. The block 1100 is partitioned into different scans, each scan labelled by a scan number (bold face) from the Scan Table 1.

In the preferred implementation, the ordering of the perceptual significance of transform scans is achieved by arranging transform scans that are comprised of different spectral bands and bit-planes or groups of bit-planes. In this case scan 9 represents the perceptually most significant contribution to the cyan channel and scan 29 the perceptually least significant contribution to the cyan channel, with the other scans ordered in between these two extremes. For the first 3 scans 9, 13 and 17, multiple bit-planes of each coefficient band are effectively coded together. For the remaining scans 21, 25 and 29, the bit-plane is separated into three scans: one for DC coefficients, one for AC coefficients 1 to 32 and one for the remaining AC coefficients 33 to 63. With respect to the preferred ordering, the DC coefficient is the most important, visually, for a given bit-plane or group of bit-planes, then the coefficient band 1 to 32, and then coefficients 33 to 63. Further, bit-planes of higher magnitude are more important, or visually significant, than those bit-planes of lower magnitude.

The entropy encoder manager 904 splits the DCT blocks into transform scans described by the Scan Table 1 and entropy encodes those scans, outputting the encoded data 209 into scan bit-streams such that each scan bit-stream contains a portion of data from each 681244 O quantised DCT block in raster block order. In the preferred implementation, as in the spectral 00oO 0 and successive approximation specification for JPEG, scans are entropy encoded differently 1according to the spectral band type (DC or AC) and bit-plane type (most significant grouped

\O

00 1-10 bit-planes, or refinement singular bit-plane) of that scan. In this case four entropy encoders 00 are employed to encode scans of the four different transform scan types: DC most significant, SDC refinement, AC most significant, and AC refinement. In the preferred implementation each transform scan listed in the Scan Table 1 is encoded using an appropriate entropy encoder. Other entropy encoding procedures are possible.

Fig. 12A shows a flow diagram of a method 1200 performed by the entropy encoder manager 904 of managing the entropy encoding of quantised transformed coefficient data.

Method 1200 starts at step 1202 and proceeds to step 1204 where, each time the entropy encoder manager receives a block of quantised data of a particular colour channel, the DC most significant transform scan of the current quantised block is entropy encoded. The method used for entropy encoding the DC most significant transform scan of that block in step 1204 follows the same steps as the entropy encoding method 604 shown in Fig. 6B, with the notable exception that the scan type to be encoded when performing step 1204 is a DC most significant transform scan of the current colour channel, and not a DC most significant spatial scan. As a result, and referring to sub-step 616 of Fig. 6B, an appropriate Huffman entropy encoder is used to encode the DC most significant transform scan type.

Referring again to Fig. 12A, after step 1204 is completed, the DC refinement transform scans of the current quantised block are entropy encoded in step 1206. Fig. 12B shows a flow diagram of step 1206 in more detail. Step 1206 starts in sub-step 1220 and proceeds to sub-step 1222 where the entropy encoder manager 904 determines whether a next 681244 -26- O DC refinement transform scan of the colour channel under consideration can be found in the 00 0 Scan Table 1. If it is determined in sub-step 1222 that a next DC refinement transform scan of ,O the colour channel under consideration cannot be found, then step 1206 ends at sub-step 1232.

00 Otherwise, if it is determined in sub-step 1222 that a next DC refinement transform scan of the 00 C 5 colour channel under consideration can be found, then the entropy encoder manager 904 Sselects this scan as the current scan and proceeds to sub-step 1224 where the entropy encoder manager 904 determines whether the current scan is active. If it is determined in sub-step 1224 that the current scan is inactive, then step 1206 ends at step 1232.

Alternatively, if it is determined in sub-step 1224 that the current scan is active, then step 1206 proceeds to sub-step 1226 where the entropy encoder manager 904 encodes the current scan using the DC refinement Huffman entropy encoder procedure described in progressive JPEG. The encoded scan data is output into a bit-stream that is shifted into an output buffer (not illustrated) of the entropy encoder manager 904.

In sub-step 1228 the entropy encoder manager 904 then determines whether the output buffer contains at least 1 word of encoded scan data 209. If it is determined in sub-step 1228 that the output buffer contains at least 1 word of encoded scan data 209, then the entropy encoding manager 904 proceeds to sub-step 1230 where one word from the output buffer is emitted to the scan output manager 208, and that word is subsequently removed from the output buffer. Step 1206 then returns to sub-step 1228.

Alternatively, if it is determined in sub-step 1228 that the output buffer does not contain at least 1 word of encoded scan data 209, then the entropy encoding manager 904 returns to sub-step 1222 from where, if a next DC refinement transform scan of the colour channel under consideration can be found in the Scan Table 1, then the process of encoding 681244 -27- O the next (active) scan is repeated. Once all active scans of this scan type have been encoded, 00 the step 1206 ends at step 1232.

IN Referring once again to Fig. 12A, after entropy encoding of the DC transform scans 00 1-10 of the current block are complete, the entropy encoder manager 904 proceeds to encode the 00 AC most significant and AC refinement transform scans in steps 1208 and 1210 respectively.

Steps 1208 and 1210 follow the same sub-steps as that of step 1206 described above with reference to Fig. 12B, with the exception that the transform scan type encoded during step 1208 is an AC most significant scan of the current colour channel, and an AC refinement scan of the current colour channel during step 1210, and not a DC refinement scan as in the case during step 1206. As a result, the AC most significant Huffman entropy encoder procedure and the AC refinement Huffman entropy encoder procedure are used in sub-step 1226 (Fig.

12B) during steps 1208 and 1210 respectively. Method 1200 ends in step 1212.

Referring again to Fig. 2, with the upper and lower partitioned bit-streams of data 203 and 205 encoded by the spatial domain encoder 204 and transform domain encoder 209 respectively to form the encoded bit-streams data 207 and 209 respectively, the scan output manager 208 accepts the encoded data 207 and 209, and manages the storing of the encoded bit-streams data 207 and 209, together forming the encoded data 105, into the final output buffer 106 of fixed memory size. In order to do so, the scan output manager 208 maintains a memory management table 700 in internal memory, with four entries for each scan. Fig. 7 illustrates the preferred arrangement of the memory management table 700. The entries for each scan 702 are: 681244 -28- 0 A linked list root address 704, which is the address in the final output buffer 106 00oO 0(Fig. 1) of the first of a linked list of blocks that contain the entropy encoded data I207 or 209 of this stream;

\O

00 IO A current block address 706, which is the address in the final output buffer 106 of 00 04 5 the last block in the linked list and is the block that is currently being filled; A fill index 708, which is a count of how many words of entropy encoded data 207 or 209 have so far been written into the current block, and is thus also the index at which new data 207 or 209 will be written; and An active flag 710, which indicates whether this entropy encoded stream is still to produce data versus the stream has been stopped.

The scan output manager 208 also holds in a free block register 712 the address in the final output buffer 106 of the first block of a linked list of free blocks that are not currently assigned to any active entropy encoded stream.

Fig. 8A shows a flow diagram of a method 800 performed by the scan output manager 208 of managing the storing of encoded image data 207 and 209 in the final output buffer 106 of fixed memory size. Method 800 starts at step 802 each time an entropy encoder manager 504 or 904 of the spatial domain entropy encoder 204 or the transform domain encoder 206 respectively emits 1 word of entropy encoded data 207 or 209 to the scan output manager 208. Method 800 then proceeds to step 804 where the scan output manager 208 determines whether the fill index entry of a current scan stream in the memory management table 800 is equal to a maximum allowed value.

If it is determined in step 804 that the fill index entry of the current scan stream is not equal to the maximum allowed value, method 800 proceeds to step 816 where the scan output 681244 -29- 0 manager 208 writes a word of entropy encoded data 207 or 209 into the final output buffer 106 00oO 0as encoded data 105. This encoded data 105 is written to that location in the current block indicated by the current block address entry 706 and fill index entry 708 in the memory oO 00 IDmanagement table 700 (Fig. 7) corresponding to the current scan stream. After the encoded 00 data 105 is written, the fill index entry 708 corresponding to the current scan stream is Sincremented in step 818 to account for new data. Method 800 then proceeds to step 820 where method 800 ends. The scan output manager 208 stores in the final output buffer 106 the information indicating for each scan stream its scan stream number and its corresponding linked list root address (if any).

Alternatively, if it is determined in step 804 that the fill index entry of the current scan stream is equal to the maximum allowed value, method 800 proceeds to step 806 where the scan output manager 208 obtains an address of a free block, and if the address of a free block is obtained, determines whether the current scan has been flagged as active. Step 806 is described in more detail below. If the address of a free block is obtained but it is determined that the current scan has been flagged as inactive, then methos 800 ends in step 820.

Alternatively, if the address of a free block is obtained and the current scan has been flagged as active, then the scan output manager 208 determines in step 808 whether the current block address entry 706 of the current scan stream in the memory management table 700 is zero. If the current block address entry 706 of the current scan stream is zero, then, in step 810, the linked list root address entry 704 in the memory management table 700 for the current scan stream is set to the free block address obtained in step 806. Step 812 follows where the current block address entry 706 in the memory management table 700 for the current scan stream is also set to the free block address obtained in step 806.

681244

C-)

O If it is determined in step 808 that the current block address entry 706 of the current 00 0 scan stream is not zero, then method 800 proceeds directly to step 812, where the current NOblock address entry 706 is set to the free block address obtained in step 806.

00 0O Step 814 follows step 812 where the scan output manager 208 updates the free block 00 r, 5 register 712 to the address of the next free block in the linked list of free blocks. If there are no more free blocks available in the linked list of free blocks the scan output manager 208 sets the free block register 712 to zero. The scan output manager 208 then sets the fill index entry 708 of the memory management table 700 to zero. Step 816 follows where a word of entropy encoded data 207 or 209 is written by the scan output manager 208 into the final output buffer 106 as encoded data 105. The fill index entry 708 corresponding to the current scan stream is also incremented in step 818 to account for new data before method 800 ends in step 820.

Fig. 8B shows a flow diagram of step 806 in more detail. After determining in step 804 that the fill index entry 708 of the current scan stream is equal to the maximum allowed value, the scan output manager 208 determines in sub-step 832 of step 806 whether the free block register 712 is zero. If the scan output manager 208 determines that the free block register 712 is not zero, then step 806 proceeds to sub-step 834 where the scan output manager 208 obtains the address of this free block stored in the free block register 712 and ends by passing control to step 808 (Fig. 8A). During sub-step 834 the scan output manager 208 also updates the free block register 712 by assigning the address of the next free block in the linked list of free blocks to the free block register 712. If there is no next free block in the linked list of free blocks, the scan output manger 208 sets the free block register 712 to zero.

Alternatively, if the scan output manager 208 determines in sub-step 832 that the free block register 712 is zero, then step 806 proceeds to sub-step 836 where the scan output 681244 -31- O manager 208 sets the active flag entry 710 in the memory management table 700 of the most 00 C perceptually insignificant of the active scan streams to inactive. Preferably, scans comprising the same scan table values of Ss, Se, Al, and Ah, but different colour component values, are oO 00 INO discarded simultaneously in order to reduce any visual artefacts produced in an image formed 00oO 5 from the encoded data. The scan output manager 208 then, in sub-step 838, assigns the linked list root address 704 of the just previously flagged inactive scan stream to the free block register 712. Once the free block register 712 becomes non-zero, that address is used as the address of the next free block.

Step 806 then proceeds to sub-step 840 where the scan output manger 208 determines whether the just previously flagged inactive scan stream is the current scan stream. If the entropy encoded stream that just became inactive is the current scan stream, the 1 word of entropy encoded data is discarded in sub-step 842, and step 806 ends by passing control to step 820 (Fig. 8A). Otherwise, if it is determined in sub-step 840 that the entropy encoded stream that just became inactive is not the current scan stream, then step 806 returns to substep 832.

The scan output manager 208 is able to detect that the final output buffer 106 is full by determining that the free block register 712 is zero. In this situation, the scan output manager 208 by then assigning in sub-step 838 the linked list root address of the most perceptually insignificant scan stream to the free block register, allows data of a more perceptually significant scan stream to simply overwrite the most perceptually insignificant scan stream. In this way, the compressor 104 of the raster image processing system 100 (Fig.

1) performs a simple one-pass processing for fitting the encoded data 105 into the final output buffer 106 of fixed memory size. The scan output manager 208 allocates the same scan, for 681244 -32- ,a o O example scan 28, of each DCT block to the same linked list. Thus, when the scan output 00 0 manager 208 assigns the linked list root address of the most insignificant scan stream to the 11free block register in sub-step 838 (Fig. 8B), the scan output manager 208 is effectively 00 00 discarding from storage in the final output buffer 106 the same most perceptually insignificant 00 C' 5 scans of all the DCT blocks. Thus, the perceptual degradation introduced to achieve Scompression into a fixed size buffer is applied consistently to the whole image.

As an image is being encoded in substantially raster (or 8-line raster) order, the scan output manager 208 ensures that the encoded image bit stream 105 does not overflow the final output buffer 106. Should the final output buffer 106 become full at any stage before encoding is complete, the scan output manager 208 discards the least perceptually significant scan that remains in the buffer 106, thereby reclaiming buffer space for remaining portions of the encoded image bit-stream 105. In this way several scans may be discarded during encoding. Any scans discarded during encoding are not part of the final encoded image bitstream 105.

In the case where a lossless DCT is employed, if no scans are discarded the encoded image bit-stream represents the colour raster image data 102 without loss. That is the encoded image bit-stream can be decoded to give the original colour raster image data 102.

Alternative transforms may be employed, such as the Discrete Wavelet Transform

(DWT).

Regardless of which transform is employed, should an encoded image bit-stream, consisting of all scans, be too large for the final output buffer 106, the scan manager 208 ensures that the least perceptually significant scans are discarded. In this way the final 681244 0 -33 O encoded image bit-stream 105, consisting of the perceptually most important scans, fits into 00 0 the final output buffer 106, and is formed in one pass in substantially raster order.

ID Further, because the upper 4 and lower 4 bit-planes of the colour raster image data 00 oO IND 102 are separately encoded, an upper limit for the error value is guaranteed between the pixels 00 5 of the original colour raster image data 102 and that of the reconstructed image data 110 after Sdecoding in the case where scan bit-streams are overwritten (discarded) in order to fit the encoded data 105 into the fixed size final output buffer 106. The guarantee of the pixel error upper limit however applies only if all the spatial domain encoded bit-stream data 207 is retained by the scan output manager 208 when storing the encoded data 105, into the final output buffer 106 of fixed memory size.

Referring again to Fig. 1, the decoding of the encoded data 107 from the final output buffer 106 into reconstructed image data 110, with the encoded data 107 being the same as the encoded data 105, is performed by the decompressor 108. The decoding of the encoded data 107 is just the inverse of encoding process performed by the compressor 104 as described above. Therefore, only an abbreviated description of the decompressor 108 follows.

Fig. 13 shows a schematic block diagram of the preferred arrangement of the decompressor 108 operating within the raster image processing system 100 of Fig. 1. The decompressor 108 includes a scan input manager 1302 for retrieving each scan of the encoded data 107 from the final output buffer 106 and splitting the encoded data into spatial scan data 1303 and transform scan data 1305, a spatial domain decoder 1304 for decoding the spatial scan data 1303 to form decoded spatial scan data 1307, a transform domain decoder 1306 for decoding the transform scan data 1305 to form decoded transform scan data 1309, and a bit- 681244 -34c,) O plane splicer 1308 for recombining the decoded scan data 1307 and 1309 into the 00 0reconstructed image data 110.

The scan input manager 1302 first retrieves the linked list information from the final 00 00 output buffer 106. From this information, the scan input manager 1302 is able locate the scans 00 in the final output buffer 106 and output the scans to be decoded by the appropriate decoder S1304 or 1306. That is, the spatial scan data 1303 from the final output buffer 106 is decoded using the spatial domain decoder 1304, and the transform scan data 1305 from the final output buffer 106 is decoded using the transform domain decoder 1306. The encoded image scan data 1303 and 1305 is decoded in substantially raster scan order, producing preferably 8 output pixel lines of the reconstructed image data 110 at a time.

Fig. 14 shows a schematic block diagram of the preferred arrangement of the spatial domain decoder 1304 within the decompressor 108 of Fig. 13. The spatial domain decoder 1304 comprises an entropy decoder manager 1402 and a pixel summation unit 1404. The entropy decoder manager 1402 uses Huffman decoding to decode the spatial scan data 1303, and outputs decoded 8-bit difference data Dij to the pixel summation unit 1404, which sums the decoded 8-bit difference data Dij to produce the decoded spatial scan data 1307.

The entropy decoder manager 1402 decodes each block of spatial scan data 1303 from the compressed scans in block raster order to form decoded difference data Dij. It is noted from the spatial domain encoding described with reference to Fig. 5 that the spatial scan data 1303 relates to the "UPPER 4 BITS", or upper partition values PuP4. Also, as with the spatial domain encoding described with reference to Fig. 5, in the preferred implementation there are two unique Huffman decoding methods for decoding the DC difference data and AC 681244 0 O difference data respectively. A particular Huffman decoding is implemented depending on the 00 0 scan table information for the particular scan being decoded.

The pixel summation unit 1404 preferably operates on a block of decoded 8-bit 00 IN difference data Dij to produce an 8 by 8 block of 8-bit partitioned pixel data {Pup 4 }ij. For 00 each row and column of the 8 by 8 block of difference data Dij, upper partitioned pixel data

S{PUP

4 }ij is calculated. The row and column indices of the partitioned and difference data are i and j respectively. The calculation of the partitioned pixel data {PUP4}ij is as follows: {PUP4 0,0 Do,o when i=0 andj=0; {PUP4}i,0 D, PUP 4 0 when i and j=0; and (2) {PUP4 i,j D,j {PUP 4 }i,j- 1 when i and j Fig. 15 shows a schematic block diagram of the preferred arrangement of the transform domain decoder 1306 within the decompressor 108 of Fig. 13. The transform domain decoder 1308 comprises an entropy decoder manager 1502 and an inverse DCT unit 1504. The entropy decoder manager 1502 accepts the encoded transform scan data 1305 from the scan input manager 1302, and Huffman decodes the transform scan data 1305 to form decoded coefficient data. The inverse DCT unit 1504 combines the coefficient data into a block of quantised transform coefficients and performs inverse quantisation (as necessary) and an inverse discrete cosine transformation on the block, outputting lower partitioned data PLOW4 to the bit-plane splicer 1308.

The entropy decoder manager 1502 operates on the encoded transform data 1305 in block raster order to form the decoded coefficient data. As with the transform domain 681244 -36- O encoding described with reference to Fig. 9, in the preferred implementation there are four 00 0unique Huffman decoding methods for decoding the bit-stream transform scans of data 1305 0according to the particular bit-plane being decoded, namely the DC most significant grouped

\O

00 1O bit-planes, the DC refinement grouped bit-planes, the AC most significant grouped bit-planes, 00 C- 5 and the AC refinement grouped bit-planes. The particular Huffman decoding method is Simplemented depending on the scan table information for that scan being decoded. In the event where perceptually insignificant scans are not present in the final output buffer 106, the inverse DCT unit 1504 pads the decoded transform coefficients corresponding to these scans with zeroes.

Referring again to Fig. 13, the bit-plane splicer 1308 operates on the partitioned pixel data 1307 and 1309 received from the spatial domain decoder 1304 and the transform domain decoder 1306 respectively. The upper partitioned pixel data 1307 and the lower partitioned data 1309 are combined to produce the reconstructed image pixel data 110.

Fig. 16 shows a flow diagram of a method 1600, performed by the bit-plane splicer 1308, of reconstructing the bit-planes from the upper partitioned pixel data 1307 and the lower partitioned data 1309, which comprises the upper and lower partition values PuP4 and PLOW4.

The method 1600 starts at step 1602 and proceeds to step 1604 where the bit-plane splicer 1308 copies the lower 4 bits of the lower partition value PUP4 to the 4 most significant bits of a reconstructed pixel component value PREC. Next, in step 1606 the bit-plane splicer 1308 determines whether the least significant bit of the upper partition value PUP4 is equal to 1. If it is determined that the least significant bit of the upper partition value Pup4 is equal to 1, then the lower partition value PLOW4 is transformed at step 1608 to a new lower partition value PLOW4 15 PLOW4. From step 1608 or if it is determined at step 1606 that the least 681244 -37- O significant bit of the upper partition value PuP4 is equal to 0, the method 1600 proceeds to step 00 1610, where the lower 4 bits of the upper partition value PLOW4 are copied to the 4 least IND significant bits of the reconstructed pixel component value PREC. Method 1600 ends in step 00 INO 1612.

00 C' 5 Figs. 17A and 17B illustrate by way of an example the recombining of the upper and lower partition values PUP4 and PLOW4 into the reconstructed pixel component value PREC.

Referring first to Fig. 17A there is shown an example for obtaining the most significant 4 bits of the reconstructed pixel component value PREc from the upper partition value Pup4. In accordance with the explanation for Figs. 4A and 4B, the magnitude of each bit-plane increases from right to left, starting from 0 for the LSB, to 7 for the MSB. For this example the upper partition value PuP4 5, and hence the 4 least significant bit-planes 1702 of the upper partition value PuP4 are the values 0, 1, 0, and 1. The 4 least significant bit-planes 1702 of the upper partition value PUP4 are left shifted by 4, such that the left-most 4 bits of the reconstructed pixel component value PREC (labelled "UPPER 4 BITS") have the values 0, 1, 0, 1. The right-most 4 bits of the reconstructed pixel component value PREC (labelled "PADDED ZEROS") are set with zeros. Accordingly, the intermediate value of the reconstructed pixel component value PREC is Turning now to Fig. 17B there is shown an example for obtaining the least significant 4 bits of the reconstructed pixel component value PREC from the lower partition value PLOW4.

For this example the lower partition value PLW4 9, and hence the 4 least significant bitplanes 1712 of the lower partition value PLOW4, as read from the most significant to the least significant bit-plane, are the values 1, 0, 0, and 1. If the LSB of the upper partition value PUP4 is equal to 1, (which is the case for the example of Fig. 17A where the upper partition value 681244 -38- O PUP4 5) then the lower partition value PLOW4 is point transformed to a new lower partition 00 C value PLOW4 15 PLOW4. The resulting lower partition value PLOW4 is 6. The 4 least IDsignificant bits ("LOWER 4 BITS") of the new lower partition value PLOW4 are then added to 00 IDthe intermediate value of the reconstructed pixel component value PREC (Fig. 17A). The 00 C' 5 resulting value for the reconstructed pixel component value comprising the decoded "UPPER 4 BITS" and "LOWER 4 BITS" partitioned data is: PREC PREC PLOW4 80 6 86 (3) Referring again to Fig. 2, an advantage of splitting the colour raster image data 102 into two 4-bit partitions, providing that all the spatial domain encoded bit-stream data 207 is retained when compressing the image into the final output buffer 106, is that the pixel error of the reconstructed image data 110 (Fig. as compared to the colour raster image data 102, is at most 15 greyscales or intensity values. For an image comprising 8-bit pixel components, this represents a maximum pixel error of approximately 7% of the maximum intensity value for each colour component. Typically this maximum pixel error does not produce noticeable visual artefacts when compressing most image data. This is particularly true for text and vector graphic images, which are characterised with ringing or speckling around sharp edges when compressed into relatively small buffers using the prior art transform only compression methods.

Splitting the colour raster image data 102 into two 4-bit partitions also provides high compression ratios when applied to text and vector graphic images, while maintaining a desired image quality. In this case, high compression ratios mean more image data can be compressed into the fixed size buffer 106. Or, put another way, the encoded data 105 retains 681244 -39- 0 more of its scan information, and the reconstructed image data 110 will compare more 00 0 favourably to the colour raster image data 102.

When comparing the preferred implementation with purely spatially embedded 00 00 encoding methods, the preferred implementation has the advantage that both text and natural 00 image content can be encoded with excellent compression results. Further, the degradation of Simage quality for the preferred implementation, as least significant scan bit-streams are deleted, is typically "more graceful" than for a purely spatially embedded compression method.

In an alternative implementation, the bit-plane partitioning is altered to better suit "natural images" of continuous tone data. In such an implementation a partitioning of 2 upper bits and 6 lower bits of an 8-bit colour pixel component is used.

Fig. 18 shows a schematic block diagram of an alternative arrangement of the compressor 104' operating within the raster image processing system 100 of Fig. 1. The alternative arrangement of the compressor 104' uses almost the same data flow and many of the same compression components as the preferred arrangement of the compressor 104 described with reference to Fig. 2. Accordingly, the description of the alternative arrangement concentrates on the differences between the preferred and alternative arrangements of the compressor 104.

A buffer (not illustrated) accessed by a block classifier unit 1802 of the compressor 104' preferably buffers a band of 8 lines of colour raster image data 102. Each 8x8 block of data in raster band order is classified by the block classifier unit 1802 as either a spatial block or a transform block. A block is classified as a spatial block if there are at most two different values taken by each pixel component in the block. For example, a block where each pixel 681244 O component has a value of either 0 or 255 is classified as a spatial block. As another example, 00 Ca block where each pixel component has a value of 255 is classified as a spatial block. A block that is not a spatial block is classified as a transform block. Other classification methods

\O

00 IDmay be used in different implementations.

00 N 5 A spatial domain encoder 1804 encodes blocks classified as spatial blocks, while a Stransform domain encoder 1806 encodes blocks classified as transform blocks. Unlike the (-i spatial and transform domain encoders 204 and 206 described with reference to Fig. 2, the spatial and transform domain encoders 1804 and 1806 encode disjoint regions of an image.

That is, a block of raster image data 102 is encoded by either the spatial domain encoder 1804, or the transform domain encoder 1806, but not both. Both the spatial and transform domain encoders 1804 and 1806 encode the block of raster image data 102 in one or more scans, which is output to a scan output manager 1808.

The spatial domain encoder 1804 is now described with reference to Fig. 19, which illustrates a segment 1900 of the spatial domain encoded data 1807 formed by the spatial domain encoder 1804 from one or more blocks of raster image data 102. The first bit 1902 of the segment 1900 is a single bit flag indicating whether the block is a spatial block or a transform block. Preferably a bit flag value of 1 indicates a spatial block, while a bit flag value of 0 indicates a transform block.

A next number of bits 1904 is a variable length code indicating the magnitude of a number of consecutive blocks, or run, of blocks encoded in this spatial domain encoded data segment 1900, and is called the run magnitude code. The possible run magnitude codes are illustrated in Table 2.

681244 -41- Run magnitude Range of number of blocks in run Binary code 0 1 1 2 110 3 [8,15] 1110 N 2 11...10 Table 2 For example, for a run of only 1 block, the run magnitude code is 1. For a run of 8 blocks the run magnitude code is the 4-bit code 1110.

Following the run magnitude code 1904 is another variable length code, called the run value code 1906, indicating the exact run given the run magnitude. For a run with run magnitude n this is an n-bit code, given by the n least significant bits of an unsigned binary representation of the number of blocks in the run. For example, for a run of 3 blocks the run magnitude is 1, while the run-value code is a single 1 bit, indicating that the run is 3 long. The run-value code 1906 is not used (is 0 bits long) when there is only one block in the run.

The bit flag 1902 and block run codes 1904 and 1906 are not strictly part of the spatial domain encoded data 1807, but are included in the (first) spatial domain encoded scan for convenience. For a run of transform domain blocks there are no more bits included in the (spatial domain) current coded segment 1900. Further the run of blocks simply indicates the number of consecutive blocks that are transform domain blocks, and does not indicate that they are the same.

For spatial domain blocks there are further bits, as indicated in Fig. 19. The next 8 bits 1908 following the run-value code 1906 in the segment 1900 is a byte indicating a low signal value, while a further next 8 bits 1910 is another byte indicating a high signal value. A 681244 -42- O final 64 bits 1912 indicate which value, low or high, is taken by each pixel in the 8x8 block, 00 0 for each block in the run. The pixel bits are arranged according in raster pixel order within the IDblock. For example, if the 64 bits are all 0, each pixel takes the low value indicated in bits 00 IND 1908.

00 oO The transform domain encoder 1806 preferably is the same as the transform domain Sencoder 206 (Fig. 2) used in the preferred arrangement of compressor 104. Similarly the scan output manager 1808 operates in the same manner as the scan output manager 208 used in the preferred arrangement of compressor 104.

The operation of the alternative arrangement of the compressor 104' will now be illustrated by way of an example. Consider a single component image that consists of 1 by 6 blocks, each block being 8 by 8 pixels. That is, the dimensions of the image are 8 pixels high and 48 pixels wide.

Further, consider that each pixel in the first two blocks has value 255, while the pixels in the latter four blocks take on a much larger range of values. Because the values of the pixels in the first two blocks have only one value, the first two blocks are classified as spatial blocks, whereas the latter 4 blocks are classified as transform blocks. The first 2 blocks are the same, in terms of pixel values, and thus are coded with a spatial block run-length of 2.

Referring now to Fig. 20 where a segment 2000 of the spatial domain encoded data 1807 formed by the spatial domain encoder 1804 is shown. The first bit 2002 is a bit indicating the first run of blocks is a run of spatial blocks. The next bits 2004 are indicating a runmagnitude of 1. The next bit 2006 coupled with the bits 2004 having values "10" specifies that the run is 2 blocks. The next two bytes (16 bits) 2008 and 2010 specify the low and high values for the block run and have values 0 and 255 respectively. The next 64 bits 2012 681244 -43- ,1 O specify that all pixels in the block run (or for both blocks) take on the high value, which is 00 0 255. Note that since the low value was not used here, another value could have been specified INO in byte 2008.

00 IDIn an alternate implementation a single bit following the run value code can be used 00 that specifies whether there are one or two values taken on by the pixels in the spatial block run. If there is only one value, then only one value is specified in the spatial scan (that is a further 8 bits as opposed to a further 16 bits). The next bit 2014 in the segment 2000 is zero, thus specifying that the following blocks are transform blocks. The run of 4 transform blocks is encoded as a run-magnitude of 2 by bits 2016 having the values "110", and the run-value 2018 of value The actual blocks are encoded in various transform scans according to the transform scans of the first implementation.

Using compressor 104' (Fig. 18) there is only one spatial domain scan while there are multiple transform domain scans. Further, the spatial domain scan is the most significant scan, and cannot be deleted if any non-trivial representation of the compressed image is desired.

An advantage of the spatial domain encoder 1804 is that an upper bound on the compressed size of the spatial domain scan can be determined based only on the number of pixels in the image. The upper bound is calculated as 1 bit for the block classification flag, 1 bit for a run of one block, 16 bits for the low and high signal values, and 64 bits for the pixel bits for each block. This gives a total of 82 bits per block, or approximately 1.28 bits per component pixel. Assuming a buffer that can hold at least 1.28 bits per component pixel, the compressor 104' can compress an image in one pass, while never overflowing the final output buffer 106. Another advantage is that blocks of bi-level data (that is blocks where the data 681244 -44- 0 takes on only one of two levels), such as rendered text blocks, are encoded losslessly. Yet 00 0 another advantage is the decompression performed by decompressor 108 is particularly fast ,in other words computationally inexpensive, for data that is encoded with many (spatial) block 00 IND runs. Further the encoding can be particularly fast for systems that can use rendering 00 knowledge to determine quickly that there are runs of bi-level blocks, and thus encoding quickly such runs.

In alternate implementations of compressor 104' other spatial domain encoders 1804 may be employed, and in particular ones that encode the spatial domain data in more than one scan. The block run code for transform domain blocks can be extended to describe a run of the same transform domain blocks. In this case, only one block in the run is encoded by the transform domain encoder 1806 and included in the transform domain scan bit-streams, for the run of blocks.

The foregoing describes only some embodiments of the present invention, and modifications and/or changes can be made thereto without departing from the scope and spirit of the invention, the embodiments being illustrative and not restrictive.

In the context of this specification, the word "comprising" means "including principally but not necessarily solely" or "having" or "including", and not "consisting only of'. Variations of the word "comprising", such as "comprise" and "comprises" have correspondingly varied meanings.

681244

Claims (14)

1. A method of compressing an image into a memory storage of fixed size, said \O 00 IDmethod comprising the steps of: 00 N 5 coding said image into a plurality of compressed scans, wherein at least one of said compressed scans comprises spatial domain compressed data, and at least one other of said (-i compressed scans comprises transform domain compressed data; and storing said compressed scans in said memory storage wherein, during the storing of said compressed scans, if it is determined that said memory storage is full, the compressed scan currently being stored in said memory storage overwrites at least part of a perceptually less significant compressed scan already stored in said memory storage.

2. A method according to claim 1 wherein said at least one compressed scan comprising spatial domain compressed data and said at least one compressed scan comprising transform domain compressed data encode disjoint regions of said image.

3. A method according to claim 1 further comprising the step of: classifying blocks of image data as either a spatial block or a transform block, wherein spatial blocks are coded in said coding step to form spatial domain compressed data and transform blocks are coded in said coding step to form transform domain compressed data. 681244 -46- c, O 4. A method according to any one of claims 1 to 3 further comprising the initial 00 0 step of progressively buffering said image as a sequence of bands, each band comprising one or more lines ofpixels. \O 00 \O 00

5. A method according to claim 4 wherein said coding comprises sub-steps of: partitioning each band into a set of most significant and a set of least significant bit- planes; transforming said set of least significant bit-planes to produce respective blocks of transform coefficients; and partitioning said blocks of transform coefficients into a plurality of transform domain compressed scans, wherein said plurality of scans comprise a perceptually significant scan and a perceptually insignificant scan and scans of varying perceptual significance there between.

6. A method according to claim 5 wherein each pixel of said image comprises 8 bit-planes, and said bands are partitioned into a set of 4 most significant bit-planes and a set of 4 least significant bit-planes.

7. A method according to claim 5 or 6 wherein the value of each pixel portion belonging to said set of least significant bit-planes is point transformed prior to said transforming step.

8. A method according to claim 7 wherein said point transformation comprises inverting said value of each pixel portion belonging to said set of least significant bit-planes, 681244 -47- O conditionally on the value of the least significant bit of the corresponding pixel portion 00 O belonging to said set of most significant bitplanes. 00 I9. A method according claim 5 wherein said transforming step uses a lossy DCT 00 C 5 transform. A method according to claim 5 wherein said transforming step uses a lossless DCT transform.

11. A method according to claim 5 wherein said coding comprises the further sub- steps of: partitioning said set of most significant bitplanes to produce respective blocks of pixel portions; differencing said pixel portions within each said block to produce a plurality of values such that a first one of said values comprises the magnitude of a first pixel portion in said block and subsequent ones of said values comprise the differences between adjacent pixel portions in said block; arranging said values into a vector array for each block, the first element of said vector array being said magnitude of said first pixel portion and the remaining elements being said differences; and Huffman encoding said vector array as in the encoding of progressive JPEG to produce a plurality of spatial domain compressed scans, wherein said plurality of scans 681244 -48- ,1 O comprise a perceptually significant scan and a perceptually insignificant scan and scans of 00 0 varying perceptual significance there between. \O 00 IN 12. A method according to claim 11 wherein a first said spatial domain compressed 00 scan comprises said encoded first elements and subsequent said spatial domain compressed scans comprise said encoded remaining elements.

13. A method according to claim 12 wherein said plurality of spatial domain compressed scans are assigned a higher perceptual significance than said plurality of transform domain compressed scans.

14. A method according to claim 5 wherein said sub-step of transforming said least significant bit-planes comprises DCT transforming, quantising and Huffman encoding as in the encoding of progressive JPEG. A method according to any one of claims 1 to 14, wherein said method comprises the further step of decoding said compressed scans, and wherein at least one of said compressed scans comprising spatial domain compressed data is decoded losslessly.

16. A method of compressing an image into a memory storage of fixed size, said method comprising the steps of: progressively buffering said image as a sequence of bands, each band comprising one or more lines ofpixels; 681244 -49- O coding said bands into a plurality of compressed scans, wherein at least one of said 00 0compressed scans comprises spatial domain compressed data, and at least one other of said IN compressed scans comprises transform domain compressed data; and oO 00 IDstoring said compressed scans in said memory storage wherein, during the storing of 00 NK 5 said compressed scans, if it is determined that said memory storage is full, the compressed scan currently being stored in said memory storage overwrites at least part of a perceptually less significant compressed scan already stored in said memory storage.

17. A system comprising: a memory storage for storing compressed data; a one pass compressor for compressing image data into compressed data, said compressed data being represented as a plurality of scans, said compressor comprising: a first encoder element operating in the spatial domain for producing at least one scan of spatial domain compressed data, and a second encoder element operating in the transform domain for producing at least one scan of transform domain compressed data, and a data manager for receiving said compressed data from said one pass compressor, and for determining which scans of said compressed data to forward to said memory storage based on available storage space in said memory storage.

18. A system as claimed in claim 17 wherein said memory storage has a fixed size. 681244 O 19. A system as claimed in claim 17 or 18 wherein if said data manager determines 00 that said memory storage is full, then the data manager compressed scan currently being stored in said memory storage overwrites at least part of scans that are perceptually less significant NO 00 IND than the scan currently being forwarded to said memory storage. 00 C~ A computer program for compressing an image into a memory storage of fixed size, said computer program comprising: code for coding said image into a plurality of compressed scans, wherein at least one of said compressed scans comprises spatial domain compressed data, and at least one other of said compressed scans comprises transform domain compressed data; and code for storing said compressed scans in said memory storage wherein, during the storing of said compressed scans, if it is determined that said memory storage is full, the compressed scan currently being stored in said memory storage overwrites at least part of a perceptually less significant compressed scan already stored in said memory storage.

21. A method of compressing an image into a memory storage of fixed size, said method being substantially as described herein with reference to any one of the embodiments shown in the accompanying drawings. 681244 -51- O 22. A system substantially as described herein with reference to any one of the 00 0 embodiments shown in the accompanying drawings. 00 I\ DATED this 7th Day of October 2004 00 CANON KABUSHIKI KAISHA SPatent Attorneys for the Applicant SPRUSON&FERGUSON 681244