EP1166565B1 - Codage d'image par reorganisation de coefficients d'ondelettes - Google Patents
Codage d'image par reorganisation de coefficients d'ondelettes Download PDFInfo
- Publication number
- EP1166565B1 EP1166565B1 EP00916649A EP00916649A EP1166565B1 EP 1166565 B1 EP1166565 B1 EP 1166565B1 EP 00916649 A EP00916649 A EP 00916649A EP 00916649 A EP00916649 A EP 00916649A EP 1166565 B1 EP1166565 B1 EP 1166565B1
- Authority
- EP
- European Patent Office
- Prior art keywords
- coefficients
- encoding
- data
- wavelet
- length
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Lifetime
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/60—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding
- H04N19/63—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding using sub-band based transform, e.g. wavelets
-
- H—ELECTRICITY
- H03—ELECTRONIC CIRCUITRY
- H03M—CODING; DECODING; CODE CONVERSION IN GENERAL
- H03M7/00—Conversion of a code where information is represented by a given sequence or number of digits to a code where the same, similar or subset of information is represented by a different sequence or number of digits
- H03M7/30—Compression; Expansion; Suppression of unnecessary data, e.g. redundancy reduction
- H03M7/40—Conversion to or from variable length codes, e.g. Shannon-Fano code, Huffman code, Morse code
-
- H—ELECTRICITY
- H03—ELECTRONIC CIRCUITRY
- H03M—CODING; DECODING; CODE CONVERSION IN GENERAL
- H03M7/00—Conversion of a code where information is represented by a given sequence or number of digits to a code where the same, similar or subset of information is represented by a different sequence or number of digits
- H03M7/30—Compression; Expansion; Suppression of unnecessary data, e.g. redundancy reduction
- H03M7/46—Conversion to or from run-length codes, i.e. by representing the number of consecutive digits, or groups of digits, of the same kind by a code word and a digit indicative of that kind
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/60—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding
- H04N19/63—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding using sub-band based transform, e.g. wavelets
- H04N19/64—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding using sub-band based transform, e.g. wavelets characterised by ordering of coefficients or of bits for transmission
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
- H04N19/13—Adaptive entropy coding, e.g. adaptive variable length coding [AVLC] or context adaptive binary arithmetic coding [CABAC]
Definitions
- This invention relates generally to the field of image compression and in particular to an improved wavelet coefficient ordering combined with an adaptive run-length encoding mechanism for encoding and decoding image data.
- Digital pictures are used in many applications, such as Web pages, CD-ROM encyclopedias, digital cameras and others. In most cases is necessary to compress the pictures, in order for them to fit into a small amount of storage or to be downloaded in a short amount of time. For example, in a typical digital camera, pictures are taken at a resolution of 1024 ⁇ 768 picture elements (pixels), with a resolution of 12 to 24 bits per pixel. The raw data in each image is therefore around 1.2 to 2.5 megabytes. In order to fit several pictures in a computer diskette, for example, it is necessary to reduce the amount of data used by each picture. The larger the compression ration that is achieved, the more pictures will fit into a diskette or memory card and the faster they can be transferred via bandwidth limited transmission medium such as telephone lines.
- JPEG Joint photographic experts group
- ISO International Standards Organization
- JPEG JPEG independent processing of small 8 ⁇ 8 blocks in JPEG is an advantage from an implementation viewpoint, especially in low-cost hardware.
- JPEG also leads to the main problem with JPEG: blocking artifacts. Because the quantization errors from adjacent blocks are uncorrelated among blocks but correlated within the blocks, the boundaries of the 8 ⁇ 8 blocks becomes visible in the reconstructed image due to the potential difference in encoding between adjacent blocks.
- Such artifacts are referred to as tiling or blocking artifacts which can be reduced (but not completely eliminated) by using transforms with overlapping basis functions.
- An efficient way to remove the blocking artifacts is to replace the block DCT by a wavelet decomposition, which provides an efficient time-frequency representation. Very good compression performance can be obtained by quantizing and encoding wavelet coefficients.
- Some prior wavelet transforms decompose images into coefficients corresponding to 16 subbands. This results in a four by four matrix of subbands, referred to as a big block format, representing spectral decomposition and ordering of channels.
- the letters L and H are used to identifying low pass filtering and high pass filtering respectively for each subband.
- the first subband comprises LL and HL coefficients, where the first letter in each set correspond to horizontal filtering and the second corresponds to vertical filtering. Two stages are used in each subband filtering combination.
- the ordering corresponds to frequencies increasing from left to right and from bottom to top. This ordering is fixed to allow both encoding and decoding to function in a fixed manner.
- Quantization of the coefficients is then performed, followed by some form of compressive encoding of the coefficients, including adaptive Huffman encoding or arithmetic encoding to further compress the image.
- compressive encoding can be quite complex, including zero tree structures, which depend on the data types.
- encoders are fairly complex, and many need to be modified for different images to be compressed, making them difficult to implement in hardware.
- Ordentlich E. et al "A low complexity modeling approach for embedded coding of wavelet compression coefficients" Proceedings DCC '98Data Compression Conference (CAT. No. 98TB100225), Proceedings DCC '98 Data Compression Conference, Snowbird, UT, USA, 30 March-1 April 1998, page 408-417, XP000925096 1998 Los Alamitos, CA, USA, IEEE Comput. Soc, USA ISBN 0-8186-8406-2.
- Ordentlich discloses a two-stage process, in which a set of coefficients is decomposed as a function of previously encoded information outside of the set, into two or more sets of coefficients. The second stage decomposes the two or more sets of coefficients using ordinary context modeling constrained by a selected ordering. (page 410 last paragraph ⁇ page 411 first paragraph.)
- wavelet compression eliminates the blocking and ghost or mosquito effects of JPEG compression
- there is a need for alternative ways to transform images to the frequency domain and compress such transformations including methods that are simple to implement, and may be implemented in either hardware or software.
- Reordering of quantized wavelet coefficients is performed to cluster large and small wavelet coefficients into separate groups without requiring the use of data-dependent data structures.
- the coefficients are then adaptively encoded based on a run-length code which continuously modifies a parameter that controls the codewords uses to represent strings of quantized coefficients, seeking to minimize the number of bits spent in the codewords. Since the ordering pattern is fixed, and the coefficient encoding does not require a modified table for each image, the invention lends itself to easier hardware or software implementations. Further advantages include the elimination of blocking artifacts, and single pass encoding for any desired compression ratio.
- a decoder applies the above in reverse order. Decoding of the encoded coefficients is first performed, followed by an unshuffling of the coefficients. The unshuffled coefficients are then subjected to an inverse wavelet transform to recover the transformed and compressed data, such as image pixels.
- the quantized wavelet coefficients are reordered into blocks such that a matrix of indices contains the coarsest coefficients in the upper left corner, and filling in low high and high low sub bands in larger and larger blocks in an alternating manner, such that low high sub bands comprise the top of the matrix and the high low sub bands comprise the left side of the matrix.
- a matrix of indices contains the coarsest coefficients in the upper left corner
- low high and high low sub bands in larger and larger blocks in an alternating manner, such that low high sub bands comprise the top of the matrix and the high low sub bands comprise the left side of the matrix.
- the encoding of the coefficients is based on a new adaptive kind of run-length encoding.
- the shortest codeword is assigned to represent a run of the most likely character having length of 2 k , where k is a parameter. k is adjusted based on successive characters being encountered. k is increased when the character is the same, and decreased when the character is different.
- adaptive arithmetic coding may also be used.
- the coefficients are encoded as bit planes. This further increases the likelihood that long strings of zeros will be encountered, and further increases the compression ratios which may be achieved.
- a first section describes the operation of a computer system that implements the current invention. This is followed by a high level description of a fixed reordering of quantized wavelet coefficients and adaptive run-length encoding of them. A decoder for such encoded data is also described. Further detail of selected blocks of the high level description is then described by use of flowcharts. This is followed by a general description of the use of such encoders and decoders in an office suite of software applications. A conclusion describes some potential benefits and describes further alternative embodiments.
- Figure 1 provides a brief, general description of a suitable computing environment in which the invention may be implemented.
- the invention will hereinafter be described in the general context of computer-executable program modules containing instructions executed by a personal computer (PC).
- Program modules include routines, programs, objects, components, data structures, etc. that perform particular tasks or implement particular abstract data types.
- Those skilled in the art will appreciate that the invention may be practiced with other computer-system configurations, including hand-held devices, multiprocessor systems, microprocessor-based programmable consumer electronics, network PCs, minicomputers, mainframe computers, and the like which have multimedia capabilities.
- the invention may also be practiced in distributed computing environments where tasks are performed by remote processing devices linked through a communications network.
- program modules may be located in both local and remote memory storage devices.
- Figure 1 shows a general-purpose computing device in the form of a conventional personal computer 20, which includes processing unit 21, system memory 22, and system bus 23 that couples the system memory and other system components to processing unit 21.
- System bus 23 may be any of several types, including a memory bus or memory controller, a peripheral bus, and a local bus, and may use any of a variety of bus structures.
- System memory 22 includes read-only memory (ROM) 24 and random-access memory (RAM) 25.
- ROM read-only memory
- RAM random-access memory
- BIOS basic input/output system
- BIOS 26 stored in ROM 24, contains the basic routines that transfer information between components of personal computer 20. BIOS 26 also contains start-up routines for the system.
- Personal computer 20 further includes hard disk drive 27 for reading from and writing to a hard disk (not shown), magnetic disk drive 28 for reading from and writing to a removable magnetic disk 29, and optical disk drive 30 for reading from and writing to a removable optical disk 31 such as a CD-ROM or other optical medium.
- Hard disk drive 27, magnetic disk drive 28, and optical disk drive 30 are connected to system bus 23 by a hard-disk drive interface 32, a magnetic-disk drive interface 33, and an optical-drive interface 34, respectively.
- the drives and their associated computer-readable media provide nonvolatile storage of computer-readable instructions, data structures, program modules and other data for personal computer 20.
- exemplary environment described herein employs a hard disk, a removable magnetic disk 29 and a removable optical disk 31, those skilled in the art will appreciate that other types of computer-readable media which can store data accessible by a computer may also be used in the exemplary operating environment.
- Such media may include magnetic cassettes, flash-memory cards, digital versatile disks, Bernoulli cartridges, RAMs, ROMs, and the like.
- Program modules may be stored on the hard disk, magnetic disk 29, optical disk 31, ROM 24 and RAM 25.
- Program modules may include operating system 35, one or more application programs 36, other program modules 37, and program data 38.
- a user may enter commands and information into personal computer 20 through input devices such as a keyboard 40 and a pointing device 42.
- Other input devices may include a microphone, joystick, game pad, satellite dish, scanner, or the like.
- These and other input devices are often connected to the processing unit 21 through a serial-port interface 46 coupled to system bus 23; but they may be connected through other interfaces not shown in Figure 1, such as a parallel port, a game port, or a universal serial bus (USB).
- a monitor 47 or other display device also connects to system bus 23 via an interface such as a video adapter 48.
- personal computers typically include other peripheral output devices (not shown) such as speakers and printers.
- Personal computer 20 may operate in a networked environment using logical connections to one or more remote computers such as remote computer 49.
- Remote computer 49 may be another personal computer a server, a router, a network PC, a peer device, or other common network node. It typically includes many or all of the components described above in connection with personal computer 20; however, only a storage device 50 is illustrated in Figure 1.
- the logical connections depicted in Figure 1 include local-area network (LAN) 51 and a wide-area network (WAN) 52.
- LAN local-area network
- WAN wide-area network
- PC 20 When placed in a LAN networking environment, PC 20 connects to local network 51 through a network interface or adapter 53. When used in a WAN networking environment such as the Internet, PC 20 typically includes modem 54 or other means for establishing communications over network 52. Modem 54 may be internal or external to PC 20, and connects to system bus 23 via serial-port interface 46. In a networked environment, program modules, such as those comprising Microsoft® Word which are depicted as residing within 20 or portions thereof may be stored in remote storage device 50. Of course, the network connections shown are illustrative, and other means of establishing a communications link between the computers may be substituted.
- Software may be designed using many different methods, including object oriented programming methods.
- C ++ and Java are two examples of common object oriented computer programming languages that provide functionality associated with object oriented programming.
- Object oriented programming methods provide a means to encapsulate data members (variables) and member functions (methods) that operate on that data into a single entity called a class.
- Object oriented programming methods also provide a means to create new classes based on existing classes.
- An object is an instance of a class.
- the data members of an object are attributes that are stored inside the computer memory, and the methods are executable computer code that act upon this data, along with potentially providing other services.
- the notion of an object is exploited in the present invention in that certain aspects of the invention are implemented as objects in one embodiment.
- An interface is a group of related functions that are organized into a named unit. Each interface may be uniquely identified by some identifier. Interfaces have no instantiation, that is, an interface is a definition only without the executable code needed to implement the methods which are specified by the interface.
- An object may support an interface by providing executable code for the methods specified by the interface. The executable code supplied by the object must comply with the definitions specified by the interface. The object may also provide additional methods. Those skilled in the art will recognize that interfaces are not limited to use in or by an object oriented programming environment.
- FIG. 2 A simplified block diagram of a wavelet transform based image pixel encoder is shown in Figure 2, with a corresponding decoder shown in Figure 3. While the encoder and decoder are described with respect to image pixel data as the respective input and output, other data can also be transformed as desired.
- image pixel data is provided to a wavelet transform block 210, which operates in a known manner to provide wavelet coefficients to a quantization block 220.
- the wavelet coefficients are in a big block format as described in the background section.
- Quantization is performed by means of a uniform quantizer, which is controlled by a quantization step defining threshold T. This results in the representation of each coefficient falling between the steps by the value in the middle of the step.
- the output of block 220 is a series of integer numbers, which are quantized wavelet coefficients.
- the quantizer may be based on normal rounding, or in rounding towards zero (also known as a quantizer with a "dead zone").
- a reordering and blocking function or block 230 groups wavelet coefficients into clusters of like values. It results in a clustering or grouping together of the blocks of frequency coefficients which are most likely to be zero.
- the reordering increases the likelihood of groupings of similar data, in the sense that the data tends to have a monotonically decaying distribution of amplitudes.
- the first blocks tend to have data of larger amplitude, whereas in subsequent blocks the amplitudes of the wavelet coefficients tend to decay.
- the grouping is done by fixing a scanning order, which is data independent. One set of such grouping is shown in Figure 4, for an example with 64 blocks of wavelet coefficients.
- Reordering and blocking block 230 provides a sequence of macroblocks in the scanning order indicated.
- the first block, 0, contains all coefficients of level 0 of the wavelet tree. This corresponds to the coarsest resolution.
- Blocks 0 to 3 comprise all the coefficients of level 1.
- Blocks 0 to 15 comprise all coefficients of level 2, while level 3 comprises blocks 0 to 63. Note that the blocks alternate from low-high and high-low subbands at each level, with low-high being the top of the sequence.
- the bits are then encoded sequentially, starting at the most significant bit.
- An adaptive encoding block 240 receives the macroblocks and encodes them in a lossless manner.
- the clustering of the blocks provide data to compress which has large clusters of zeros. Further reordering the data by encoding on a bit plane basis increases the likelihood of finding large strings of zeros. Starting with the most significant bit for the first bit plane leads to a higher likelihood of a long string of zeros. Further, this also ensures that the most relevant data is encoded first. By the time the third or fourth bit planes are encoded, the odds are about equal for a zero as opposed to a one, and straight binary encoding may be effectively used.
- the encoder is an adaptation of a Golomb-Rice encoder with adaptive run-length modifications.
- a string of 2 k zeros is represented by the codeword consisting of a single bit equal to zero.
- the length of the string of zeros represented by the zero codeword is controlled by the parameter k, which is varied as data is encountered, based on the observed frequency of zeros.
- the parameter k which is varied as data is encountered, based on the observed frequency of zeros.
- a feedback loop 245 is used to represent the backwards adaptive nature of the encoder 240. This encoding provides for efficient compression and fast adaptation to changes in the statistics of the incoming data. Encoder 240 provides a bit stream out which is effectively progressive in that the most relevant information is provided at the beginning of the bit stream. Since the least significant bits are encoded in the last bit plane, for lower resolution bit streams, they may effectively be discarded or not encoded, as represented at a resolution fidelity block 250. This is useful for lower bandwidth transmissions of data.
- Decoding is essentially the reverse of the encoding and data transformations.
- a bit stream of encoded data such as that produced by the encoder of Figure 2 is received at a lossless adaptive decoding block 310.
- the bit stream may be received directly from the decoder, from local storage, or from a remote decoder or storage via one of many viable transmission media such as by satellite transmission, cable transmission or other network.
- Decoding block 310 receives the rules developed during encoding via a feed forward line 315.
- Block 310 essentially receives the string length to be used, and reconstructs the data in accordance with the rules. Again, it operates on a block level, but this is not a requirement of the invention.
- One form of fidelity reduction may be performed at block 310 just by not decoding the last bit in the bit plane. This effectively doubles the step size controlled by the parameter T. It is a simple way to reduce the fidelity of the data.
- Block 330 then is used to unshuffle or reorder the blocks back to the original portions.
- the output of the reorder block 330 is the integer numbers that need to be remultiplied back at block 340 by using the step size which is provided by a header in the received bit stream. This provides reconstructed wavelet coefficients.
- the header also provides information about how big the image size is, and other standard image format data.
- An inverse wavelet transform is then performed in a known manner at 350. It should be noted that the only losses, other than selected desired fidelity or resolution reductions, are incurred in the quantization steps, which is controllable by modification of the T parameter.
- the resolution reduction option block 320 may operate in a few different ways. One way to remove the data is by zeroing the integers involved. A further way to reduce the resolution is to modify the operation of unshuffle block 330, which may be instructed to zero the values at a desired point. By telling both unshuffle block 330, and inverse wavelet transform block 350 where the zeros start, they may be easily modified to eliminate receivededed processing of actual data at such points.
- the adaptive encoding and decoding of the present invention operates very well on data that has clustered zeros with statistics that change.
- This type of data may also be characterized as having a high probability of data with near exponential decay of the probability on either side of the zeros.
- Multimedia data such as static image data and video has this characteristic.
- the transformation of many types of physical data also has this type of characteristic. When capturing physical data, the information normally occurs in just a few places, which means that most of the other data is zero. Symmetry of the data is also a desired characteristic for this type of encoding to work best. In other words, an exponential fall off of both negative and positive values on either side of an information spike is beneficial. Examples of such physical data include ECGs and other biometric type of data.
- the sequence of top left corner indices ( r k , s k ) is defined.
- the parameter J should be chosen such that block zero contains precisely all wavelet coefficients at the coarsest resolution, e.g. all seating function coefficients. Therefore, J should be equal to the number of resolution levels (the tree depth) used in the wavelet transform. It is easy to infer from Figure 4 the sequence of all top left corner indices ( r k , s k ).
- the reason for the alternate scanning of the low-high (LH) and high-low (HL) wavelet coefficients within the same resolution level is simple. Assuming the original image has a particular feature (or no feature) at some spatial location, it is likely that clusters of both the LH and HL subbands, corresponding to that location, will have large (or small) values. Therefore, by ensuring that pairs of blocks from the LH and HL subbands corresponding to the same spatial location appear contiguously in a macroblock or at least proximate or close to each other, we're more likely to create clusters of large and small values. That increases the probability of long runs of zeros in the bit planes of the quantized coefficients.
- a flowchart in Figure 7 describes an algorithm used to write the blocks of coefficients in the order shown in Figure 4.
- the algorithm may be implemented in computer program instructions, or in hardware, firm ware or a combination of all as desired.
- the algorithm is entered at start block 710.
- An input matrix Q containing MxN quantized wavelet coefficients is read at 715.
- the coefficients are such as those provided by quantization block 220.
- a number of wavelet levels is defined at 720 in a known manner as JW.
- a block size is defined as NH x NV, with NH equal to M/(2 JW ) and NV equal to N/(2 JW ).
- the first output block is then written at 730, and IH and IV are initialized as NH and NV respectively for use in defining loops for writing of further blocks, which are larger in size.
- IH and IV are initialized as NH and NV respectively for use in defining loops for writing of further blocks, which are larger in size.
- the matrix Q is 16 by 16, with 4 levels, and a block size of 1. This provides an initial IH and IV of 1.
- the block size is larger, such as 8x8 or 16x16, or even higher.
- a decision block 740 is used to determine if the entire matrix of coefficients has been written by checking to see if IH is less than M. If IH is still less than M, more coefficients need to be written. As seen in Figure 4, the first blocks of coefficients are of dimension 1 by 1, and then they increase to 2 by 2 and 4 by 4 etc.
- the next sets of flowchart blocks are used to write the succeeding blocks by looping from one to a block size parameter NBLK which is set at block 745 as IH/NH.
- a nested loop defined at 750 using I and 755 using J is used to control the order of writing of the output blocks LH and HL at 760. J is incremented at NEXT statement 762, while I is incremented at NEXT statement 764.
- a second set of nested loops is set up at 770 and 775, again using I and J to define positions in which to write an output block at 780.
- This output block corresponds to HH blocks at the same level, which is block 450 for the first time through.
- NEXT J and NEXT I statements complete the nested loop at 782 and 784 respectively.
- the HH block could also have been written at the same time as the LH and HL blocks above since the nested loops are identical.
- IH and IV are incremented as exponents of 2 at 790, and then compared at 740 to see if IH is still less than M. If IH is not less than M, the algorithm is exited at 795, after having provided at complete reordered set of wavelet coefficients in accordance with the present invention.
- bit planes are just the sequences of bits of a particular index in the binary representation (magnitude + sign) of the incoming quantized wavelet coefficients or other data.
- Table 1 shows the bit planes for the sequence of values ⁇ 9, -6, 1, 0, -2, 3, -4, -1, 2 ⁇ .
- bit plane 4 is the sequence ⁇ 100000000
- bit plane 3 is the sequence ⁇ 010000100 ⁇
- bit plane 2 is the sequence ⁇ 010011001 ⁇
- bit plane 1 is the sequence ⁇ 101001010 ⁇ .
- bits of smaller magnitude seem to be more likely to occur, which is also typical of quantized wavelet data and finite alphabet data.
- Bit plane 1 the least significant bit
- the sign bit plane typically have zeros and ones with approximately equal frequency.
- the flow chart in Figure 5 describes the algorithm for efficiently encoding the incoming data through bit planes starting at 505.
- the bit planes are first read from an input buffer x at 510 which contains N numbers.
- the number of bits planes, bmax, is computed at 515, and a significance flag vector sflg is set to all zeros at 520.
- bit plane index variable bit is set equal to bmax, so encoding starts with the most significant bit plane.
- the values of the bits pointed to by the index "bit” form the bit plane vector bp at 530.
- the bits are divided into two subsets as indicated at blocks 535 and 540. x1 correspond to positions for which a "1" entry has not been seen in the higher planes ⁇ those are called significant bits.
- x2 corresponds to positions for which a "1" has already been seen in the higher planes ⁇ those are called refinement bits.
- x1 is encoded with the adaptive run-length Golomb-Rice (ARLGR) encoder which will benefit from the higher frequency of zeros in x1.
- ARLGR adaptive run-length Golomb-Rice
- x2 is encoded with straight binary encoding. This is done by appending the x2 bits to the output stream. Minimal loss in encoding efficiency is encountered because zeros and ones are usually equally likely in x2.
- the sign bits are not referred to as a bit plane because they are not processed as a bit plane.
- the sign bits are sent in the process of coding the x1 vectors of each bit plane.
- the vector x1 as being drawn from the alphabet ⁇ 0, +1, -1 ⁇ , i.e. bit plus sign.
- the vector sflg controls the allocation of bits to x1, and sflg is first initialized to all zeros, and then updated after each bit plane is encoded at 555. Therefore, the decoder can easily track the changes to sflg.
- bit is decremented at 560 and checked to see if the last plane has been decoded at 565. If not, control goes to block 530 for encoding of the next bit plane. If bit was equal to zero, or a higher number if a lower resolution coding is desired, an output buffer containing outputs of all x1 and x2 encodings is written at 570 and the process ends at 575.
- the adaptive Run-length + Golomb-Rice (ARLGR) coder is where the encoding gain resides. h maps long vectors x1 with lots of zeros in a more compact code, with fewer zeros.
- the ARLGR encoder can be used to encoding binary sequences with or without associated sign bits, as shown below. In order to understand the ARGLR encoder, first consider the basics of the run-length encoding and Golomb-Rice coding.
- RL run-length
- RL coding can be applied to the encoding of binary data in which either 0 or 1 is significantly more likely to occur.
- One example is in graphics files, for example, a digitized black drawing on a white background. If white picture elements (pixels) are represented by a bit equal to 0 and black dots by a bit equal to 1, it's clear that zeros are much more likely to occur. In fact, many standard graphics file formats use RL coding.
- Golomb-Rice codes for a source of binary digits are combined with RL codes.
- the code is characterized by a parameter k , which controls the length of the run associated to the codeword 0; this maximum run length is equal to 2 k .
- binary (or binary + sign) vectors are not stationary. Typical examples include data obtained from the physical world, such as quantized wavelet coefficients of pictures or scanned documents. Therefore, we need to adjust the RLGR parameter k over time, to best match the local statistics of the data. Many strategies have been considered, mostly involving dividing the input data in blocks of appropriate length. For each block, P 0 is estimated and then the optimal value of k is computed. An additional code is then sent at the beginning of each block to indicate the value of k that should be used by the decoder.
- the encoder 240 takes a new approach.
- a backward-adaptive strategy is used for changing the RLGR parameter k .
- backward-adaptive it is meant that variations in k are computed based on encoded symbols, not directly on the input data.
- the basic strategy is that the value of k to be used in encoding the next symbol should depend only on previously encoded data. Therefore, all the decoder needs to do to recover the changing values of k is to apply the same adaptation rule as the encoder. Therefore, to simplify decoding it is important that such a rule be as simple as possible to compute.
- the new adaptive Run-Length + Golomb-Rice (ARLGR) encoder 240 uses the following rules for changing the parameter k .
- Several parameters are first defined at block 604.
- a scale factor L is first defined and is used to define kp as L*k.
- kp is an auxiliary parameter whose value moves up or down by an amount Up or Dn respectively to permit fractional moves of k without the use of floating-point arithmetic.
- Uq is defined and used to move kp up if the output code was zero and k was equal to zero.
- An input buffer x is read at 606, and contains M numbers. At 608, k is set to k0, kp is set to L*k and run is set to 0.
- k a good choice for the long-term statistics of the incoming data
- the use of the auxiliary parameter kp and the scale factor L above allows adjustment of k in fractional steps without having to use floating-point arithmetic as indicated above.
- k is first checked at 614 to see if it is equal to zero. If it is, and if symbol is zero, Uq is added to kp at 618. A zero is appended to the output buffer at 620 and if kp is out of range - above kpmax - at 622, it is clipped. At 624, k is set to the largest integer less than kp/L, the scale factor. Xindex is then incremented, and if less than M as determined at 628, the next symbol is selected at 612. If greater than M, the output bit buffet is written to at 630 and the process ends at 640.
- a 1 is appended to the output bit buffer at 642
- a sign bit of symbol is appended to the output bit buffer at 644 if the data has a sign bit, and processing continues at 622 to check to see if kp is within range.
- k is not equal to 1 at block 614, a further check of symbol is performed at 650. If the symbol is not equal to zero, a 1 is appended to the output bit buffer at 652 and a k-bit value of run is appended to the output bit buffer at 654. At 656, Dn is subtracted from kp, and processing continues at 644, where an optional sign bit is appended.
- run is checked at 622 to see if it is equal to runmax. If not, kp is clipped to not exceed kpmax at 622. If run was equal to runmax at 662, a zero is appended to the output bit buffer at 664, and run is set to zero at 666. Finally, Up is added to kp, and processing again reverts to block 622 for clipping of kp, setting of k at 624, incrementing xindex at 626 and checking to see if the last symbol has been processed at 628. If so, the information is written to the ouput bit buffer at 630 and the process is ended at 640.
- a major advantage of the encoder, not shared by the arithmetic encoders, is scalability.
- bit-plane encoding With the described bit-plane encoding, a lower fidelity version of the signal can be easily obtained by stopping the decoding process at a bit plane higher than plane 1. That allows for progressive transmission and reconstruction of the information, and important feature for communication channels such as the Internet.
- Another application of scalability is in digital cameras, for example. If the user wants to take more pictures and is willing to sacrifice quality of pictures already stored, lower bit planes of existing images can be removed to release storage for new pictures.
- each pixel value is replaced by 0 if it has the same color as the same pixel in the row above, or 1 if it has a different color. This is repeated across columns.
- the resulting bits are encoded with the ARLGR encoder 240.
- the ARLGR encoder 240 algorithm outperforms the standard fax encoding algorithm by almost a factor of two. It uses only 55% of the bytes used by the fax algorithm. In fact, the new ARLGR-based encoder even surpassed the state-of-the-art adaptive arithmetic encoder by a small margin for this particular image. In addition, it had the lowest computational complexity. It should be noted that this is just one example, and that the results may vary depending on the image used and tuning of parameters.
- FIG. 8 a block diagram of a suite of office programs is shown generally at 810.
- One particular office suite comprises a plurality of high level applications indicated at 812, including such applications as word processing, email, spreadsheet, presentation tools, photo manipulation programs, and browsers. Supporting these applications are at least two lower level software, hardware or a combination thereof functions at 826 and 818.
- the functions shown include a video in/out function 826 and a fax/scanner function 818. Many other functions may also reside at this level.
- the video function provides the ability to both display video and receive video and image data from external sources.
- the video and fax/scanner functions make use ofthe encoder and decoder described herein and indicated at block 832 to provide encoding and decoding functions as previously described. If raw image or other suitable data is captured in pixel or other form, the encoder 832 is used to encode it. Further, if encoded data is obtained from any source employing the type of encoding described here, the decoder at 832 is called by the application receiving it to transform or decode it to a displayable or useable format.
- the present invention provides an alternative to other forms of coding which removes the blocking artifacts present in JPEG, and is less complex to implement in either software, hardware or hybrid forms as desired.
- the encoder/decoder at 832 is also easy to integrate into such an office suite.
- Reordering of quantized wavelet coefficients is performed to cluster large and small wavelet coefficients into separate groups without requinng the use of data-dependent data structures.
- the coefficients are then adaptively encoded based on a run-length code which continuously modifies a parameter that controls the codewords used to represent strings of quantized coefficients, seeking to minimize the number of bits spent in the codewords. Since the ordering pattern is fixed, and the coefficient encoding does not require a modified table for each image, the invention lends itself to easier hardware or software implementations. Further advantages include the elimination of blocking artifacts, and single pass encoding for any desired compression ratio for image data.
- a decoder which applies the above encoding and blocking in reverse order. Decoding of the encoded coefficients is first performed, followed by an unshuffling of the coefficients. The unshuffled coefficients are then subjected to an inverse wavelet transform to recover the transformed and compressed data, such as image pixels. Adaptive arithmetic coding may also be used in conjunction with the reordering to obtain similar compression benefits, but with slightly higher complexity.
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Compression Or Coding Systems Of Tv Signals (AREA)
- Compression, Expansion, Code Conversion, And Decoders (AREA)
- Compression Of Band Width Or Redundancy In Fax (AREA)
- Dc Digital Transmission (AREA)
Claims (25)
- Méthode de codage comprenant:la réorganisation (230) de coefficients d'ondelettes en blocs dans une matrice (410); etl'exécution d'un codage arithmétique (240) des coefficients réorganisés pour former un train de bits codés,des coefficients basse fréquence sont dans l'un des coins (420, 430, 440, 450) de la matrice,les blocs de coefficients d'ondelettes sont regroupés de façon que les coefficients d'ondelettes au sein de chaque bloc aient des valeurs similaires, etle regroupement est réalisé par la définition d'un ordre de balayage indépendant des données et comprenant un balayage en alternance de coefficients d'ondelettes bas-haut et haut-bas à l'intérieur du même niveau de résolution.
- Méthode selon la revendication 1, dans laquelle la réorganisation (230) des coefficients comprend également l'alternance de sous-bandes bas-haut et haut-bas dans des blocs de plus en plus grands de la matrice (410).
- Méthode selon la revendication 1 ou 2, dans laquelle l'image a une taille de M sur N avec un nombre correspondant de coefficients d'ondelettes.
- Méthode selon la revendication 3, dans laquelle la taille des blocs est de M/2J sur N/2J, J étant sélectionné de façon qu'un bloc de niveau zéro contienne tous les coefficients d'ondelettes à la résolution la plus grossière.
- Méthode selon l'une quelconque des revendications 1 à 4, qui est une méthode pour coder les coefficients d'ondelettes.
- Méthode selon l'une quelconque des revendications 1 à 5, dans laquelle l'exécution du codage arithmétique comprend:le codage (240) des coefficients réorganisés au moyen d'un codeur par longueur de plage adaptatif (832).
- Méthode selon la revendication 6, dans laquelle le codage par longueur de plage (240) des coefficients réorganisés est exécuté par un codeur (832) de Colomb-Rice par longueur de plage adaptatif (545) commandé par un paramètre k qui détermine les longueurs de plage maximales.
- Méthode selon la revendication 7, dans laquelle le paramètre k de codage (240) augmente à chaque fois que le caractère le plus fréquent est rencontré (670).
- Méthode selon la revendication 7, dans laquelle le paramètre k de codage (240) diminue à chaque fois que le caractère le plus fréquent n'est pas rencontré (656).
- Méthode selon l'une quelconque des revendications 1 à 9, qui est une méthode de codage (240) de données d'image et qui comprend également:la production des coefficients par l'intermédiaire d'une transformée en ondelettes (210).
- Méthode selon l'une quelconque des revendications 1 à 10, dans laquelle la réorganisation (230) est sélectionnée pour grouper des coefficients d'ondelettes forts et faibles en groupes séparés (410) sans nécessiter l'utilisation de structures de données dépendantes des données.
- Méthode selon l'une quelconque des revendications 1 à 11, comprenant également:l'initialisation d'une longueur de suite maximale pour un codeur par longueur de plage adaptatif (832);la modification de la longueur de suite maximale en fonction d'un paramètre de commande; etle codage par longueur de plage (240) des symboles pour former un train de bits de sortie comprimé.
- Méthode selon l'une quelconque des revendications 1 à 12, comprenant également:l'initialisation d'une longueur de suite maximale pour un codeur par longueur de plage adaptatif (832);la modification de la longueur de suite maximale en fonction des apparitions du symbole le plus fréquent; etle codage par longueur de plage (240) des symboles pour former un train de bits de sortie comprimé.
- Méthode selon la revendication 13, dans laquelle les symboles sont codés sur la base d'un plan de mémoire.
- Méthode selon la revendication 13 ou 14, dans laquelle les plans de mémoire à bits de poids le plus faible sont codés à l'aide d'un codage binaire (550).
- Méthode selon l'une quelconque des revendications 1 à 13, comprenant également:l'initialisation d'une longueur de suite pour un codeur par longueur de plage adaptatif (832);la modification de la longueur de suite par une puissance k, k étant fonction de l'apparition du symbole le plus fréquent;l'augmentation de k à chaque fois que le symbole le plus fréquent est codé; etla diminution de k à chaque fois qu'un symbole autre que le symbole le plus fréquent est codé, les symboles étant codés sur la base d'un plan de mémoire.
- Méthode selon la revendication 16, dans laquelle k est un nombre entier, et dans laquelle un facteur de mise à l'échelle (L) est utilisé pour incrémenter k d'une fraction à la fois (624).
- Méthode selon l'une quelconque des revendications 1 à 17, qui est une méthode pour coder (240) des données alphabétiques finies.
- Méthode selon l'une quelconque des revendications 1 à 18, dans laquelle les blocs sont groupés de façon que des paires de blocs issus d'une sous-bande bas-haut et haut-bas correspondant à un même emplacement spatial soient proches les unes des autres.
- Méthode de décodage de données d'image sous forme de pixels comprimées, comprenant:la réception d'un train de bits comprimé à l'aide d'un système de codage de Colomb-Rice par longueur de plage adaptatif (240);le décodage du train de bits reçu; etle changement de l'ordre des coefficients pour rétablir un ordre initial résultant de la transformée en ondelettes des données de pixels,le décodage du train de bits reçu comprend la production de coefficients de transformée en ondelettes qui ont été réorganisés pour regrouper les blocs de coefficients d'ondelettes de façon que les coefficients d'ondelettes au sein de chaque bloc aient des valeurs similaires, en définissant un ordre de balayage indépendant des données et comprenant un balayage en alternance de coefficients d'ondelettes bas-haut et haut-bas à l'intérieur d'un même niveau de résolution.
- Méthode selon la revendication 20, comprenant également:la transformation inverse des coefficients d'ondelettes.
- Méthode selon la revendication 20 ou 21, dans laquelle le décodage comprend:le décodage de parties sélectionnées du train de bits reçu pour produire une série de coefficients de transformée en ondelettes de poids le plus fort qui ont été réorganisés pour augmenter la probabilité de groupements de données similaires d'une manière indépendante des données.
- Support lisible par un ordinateur et sur lequel sont stockées des instructions pour amener l'ordinateur à mettre en oeuvre la méthode selon l'une quelconque des revendications 1 à 22.
- Codeur adapté pour mettre en oeuvre la méthode selon l'une quelconque des revendications 1 à 19.
- Décodeur adapté pour mettre en oeuvre la méthode selon l'une quelconque des revendications 20 à 22, ou pour décoder un train de bits produit grâce à la méthode selon l'une quelconque des revendications 1 à 19.
Applications Claiming Priority (7)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US09/276,954 US6850649B1 (en) | 1999-03-26 | 1999-03-26 | Image encoding using reordering and blocking of wavelet coefficients combined with adaptive encoding |
US280135 | 1999-03-26 | ||
US276954 | 1999-03-26 | ||
US09/280,135 US6678419B1 (en) | 1999-03-26 | 1999-03-26 | Reordering wavelet coefficients for improved encoding |
US277255 | 1999-03-26 | ||
US09/277,255 US6477280B1 (en) | 1999-03-26 | 1999-03-26 | Lossless adaptive encoding of finite alphabet data |
PCT/US2000/007832 WO2000059232A1 (fr) | 1999-03-26 | 2000-03-24 | Codage d'image par reorganisation de coefficients d'ondelettes |
Publications (2)
Publication Number | Publication Date |
---|---|
EP1166565A1 EP1166565A1 (fr) | 2002-01-02 |
EP1166565B1 true EP1166565B1 (fr) | 2004-08-04 |
Family
ID=27402856
Family Applications (2)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
EP00916649A Expired - Lifetime EP1166565B1 (fr) | 1999-03-26 | 2000-03-24 | Codage d'image par reorganisation de coefficients d'ondelettes |
EP00918402A Expired - Lifetime EP1188244B1 (fr) | 1999-03-26 | 2000-03-24 | Codage adaptatif sans perte de donnees alphabetiques finies |
Family Applications After (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
EP00918402A Expired - Lifetime EP1188244B1 (fr) | 1999-03-26 | 2000-03-24 | Codage adaptatif sans perte de donnees alphabetiques finies |
Country Status (7)
Country | Link |
---|---|
EP (2) | EP1166565B1 (fr) |
JP (2) | JP4426118B2 (fr) |
KR (1) | KR100733949B1 (fr) |
AT (2) | ATE272925T1 (fr) |
AU (3) | AU3922200A (fr) |
DE (2) | DE60015755T2 (fr) |
WO (3) | WO2000059116A1 (fr) |
Families Citing this family (13)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
AUPR192800A0 (en) | 2000-12-06 | 2001-01-04 | Canon Kabushiki Kaisha | Digital image compression and decompression |
AU770148B2 (en) * | 2000-12-06 | 2004-02-12 | Canon Kabushiki Kaisha | Digital image compression and decompression |
US6798364B2 (en) | 2002-02-05 | 2004-09-28 | Intel Corporation | Method and apparatus for variable length coding |
US20060072834A1 (en) * | 2003-04-17 | 2006-04-06 | Lynch William C | Permutation procrastination |
ES2297083T3 (es) | 2002-09-04 | 2008-05-01 | Microsoft Corporation | Codificacion entropica por adaptacion de la codificacion entre modos por longitud de ejecucion y por nivel. |
US7274995B2 (en) * | 2003-11-19 | 2007-09-25 | Honeywell International Inc. | Apparatus and method for identifying possible defect indicators for a valve |
US7286945B2 (en) * | 2003-11-19 | 2007-10-23 | Honeywell International Inc. | Apparatus and method for identifying possible defect indicators for a valve |
US7689051B2 (en) * | 2004-04-15 | 2010-03-30 | Microsoft Corporation | Predictive lossless coding of images and video |
KR101155525B1 (ko) * | 2005-06-20 | 2012-06-19 | 삼성전자주식회사 | 기록/재생 장치 및 방법 |
WO2008070843A2 (fr) | 2006-12-07 | 2008-06-12 | Qualcomm Incorporated | Commande de débit et compression vidéo par ligne |
US8179974B2 (en) | 2008-05-02 | 2012-05-15 | Microsoft Corporation | Multi-level representation of reordered transform coefficients |
FR2934103B1 (fr) * | 2008-07-16 | 2010-08-27 | Sagem Securite | Procede et systeme de transmission securisee d'une image entre deux points distants |
WO2017046005A1 (fr) * | 2015-09-18 | 2017-03-23 | Koninklijke Philips N.V. | Procédé et appareil pour compression et décompression d'image rapides et efficaces |
Family Cites Families (9)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5321776A (en) * | 1992-02-26 | 1994-06-14 | General Electric Company | Data compression system including successive approximation quantizer |
US5717394A (en) * | 1993-02-10 | 1998-02-10 | Ricoh Company Ltd. | Method and apparatus for encoding and decoding data |
US5381145A (en) * | 1993-02-10 | 1995-01-10 | Ricoh Corporation | Method and apparatus for parallel decoding and encoding of data |
JP3943634B2 (ja) * | 1995-11-02 | 2007-07-11 | キヤノン株式会社 | 情報処理装置及び方法 |
US5818877A (en) * | 1996-03-14 | 1998-10-06 | The Regents Of The University Of California | Method for reducing storage requirements for grouped data values |
IL119523A0 (en) * | 1996-10-30 | 1997-01-10 | Algotec Systems Ltd | Data distribution system |
JP3213582B2 (ja) * | 1997-05-29 | 2001-10-02 | シャープ株式会社 | 画像符号化装置及び画像復号装置 |
WO1998054093A1 (fr) * | 1997-05-30 | 1998-12-03 | Merck & Co., Inc. | Nouveaux inhibiteurs d'angiogenese |
EP0940994B1 (fr) * | 1998-03-06 | 2014-04-16 | Canon Kabushiki Kaisha | Appareil et méthode de traitement d'image, et support d'enregistrement stockant des étapes mettant en oeuvre cette méthode |
-
2000
- 2000-03-24 AU AU39222/00A patent/AU3922200A/en not_active Abandoned
- 2000-03-24 WO PCT/US2000/007955 patent/WO2000059116A1/fr active Search and Examination
- 2000-03-24 DE DE60015755T patent/DE60015755T2/de not_active Expired - Lifetime
- 2000-03-24 AU AU39169/00A patent/AU3916900A/en not_active Abandoned
- 2000-03-24 JP JP2000608507A patent/JP4426118B2/ja not_active Expired - Fee Related
- 2000-03-24 KR KR1020017012203A patent/KR100733949B1/ko active IP Right Grant
- 2000-03-24 EP EP00916649A patent/EP1166565B1/fr not_active Expired - Lifetime
- 2000-03-24 WO PCT/US2000/007831 patent/WO2000059231A1/fr active Application Filing
- 2000-03-24 AT AT00916649T patent/ATE272925T1/de not_active IP Right Cessation
- 2000-03-24 JP JP2000608618A patent/JP4540855B2/ja not_active Expired - Fee Related
- 2000-03-24 DE DE2000612717 patent/DE60012717T2/de not_active Expired - Lifetime
- 2000-03-24 AT AT00918402T patent/ATE282260T1/de not_active IP Right Cessation
- 2000-03-24 WO PCT/US2000/007832 patent/WO2000059232A1/fr active IP Right Grant
- 2000-03-24 AU AU37723/00A patent/AU3772300A/en not_active Abandoned
- 2000-03-24 EP EP00918402A patent/EP1188244B1/fr not_active Expired - Lifetime
Also Published As
Publication number | Publication date |
---|---|
JP4540855B2 (ja) | 2010-09-08 |
JP4426118B2 (ja) | 2010-03-03 |
EP1188244B1 (fr) | 2004-11-10 |
JP2002540711A (ja) | 2002-11-26 |
JP2002540740A (ja) | 2002-11-26 |
ATE272925T1 (de) | 2004-08-15 |
KR20020008133A (ko) | 2002-01-29 |
AU3916900A (en) | 2000-10-16 |
WO2000059116A1 (fr) | 2000-10-05 |
AU3922200A (en) | 2000-10-16 |
DE60015755T2 (de) | 2005-04-07 |
DE60012717D1 (de) | 2004-09-09 |
WO2000059231A1 (fr) | 2000-10-05 |
KR100733949B1 (ko) | 2007-07-02 |
WO2000059232A1 (fr) | 2000-10-05 |
AU3772300A (en) | 2000-10-16 |
DE60015755D1 (de) | 2004-12-16 |
EP1188244A1 (fr) | 2002-03-20 |
DE60012717T2 (de) | 2005-01-13 |
EP1166565A1 (fr) | 2002-01-02 |
ATE282260T1 (de) | 2004-11-15 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US6850649B1 (en) | Image encoding using reordering and blocking of wavelet coefficients combined with adaptive encoding | |
US6477280B1 (en) | Lossless adaptive encoding of finite alphabet data | |
US6678419B1 (en) | Reordering wavelet coefficients for improved encoding | |
US7110609B2 (en) | System and method for progressively transform coding digital data | |
US8565298B2 (en) | Encoder rate control | |
US7016545B1 (en) | Reversible embedded wavelet system implementation | |
Kiely et al. | The ICER progressive wavelet image compressor | |
EP1980022B1 (fr) | Codage de longueur variable pour coefficients faible densité | |
JP2009505494A (ja) | 変換ブロックの効率的なコーディングおよびデコーディング | |
EP1166565B1 (fr) | Codage d'image par reorganisation de coefficients d'ondelettes | |
GB2325584A (en) | Reversible embedded wavelet transform system implementation | |
EP1695559A1 (fr) | Compression de donnees d'image | |
Kanvel et al. | Performance measure of different wavelets for a shuffled image compression scheme | |
Saksena | Wavelet-based image compression | |
KR20000060518A (ko) | 영상 압축 방법 |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PUAI | Public reference made under article 153(3) epc to a published international application that has entered the european phase |
Free format text: ORIGINAL CODE: 0009012 |
|
17P | Request for examination filed |
Effective date: 20010920 |
|
AK | Designated contracting states |
Kind code of ref document: A1 Designated state(s): AT BE CH CY DE DK ES FI FR GB GR IE IT LI LU MC NL PT SE |
|
17Q | First examination report despatched |
Effective date: 20030115 |
|
GRAP | Despatch of communication of intention to grant a patent |
Free format text: ORIGINAL CODE: EPIDOSNIGR1 |
|
GRAS | Grant fee paid |
Free format text: ORIGINAL CODE: EPIDOSNIGR3 |
|
GRAA | (expected) grant |
Free format text: ORIGINAL CODE: 0009210 |
|
AK | Designated contracting states |
Kind code of ref document: B1 Designated state(s): AT BE CH CY DE DK ES FI FR GB GR IE IT LI LU MC NL PT SE |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: IT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRE;WARNING: LAPSES OF ITALIAN PATENTS WITH EFFECTIVE DATE BEFORE 2007 MAY HAVE OCCURRED AT ANY TIME BEFORE 2007. THE CORRECT EFFECTIVE DATE MAY BE DIFFERENT FROM THE ONE RECORDED.SCRIBED TIME-LIMIT Effective date: 20040804 Ref country code: CH Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20040804 Ref country code: ES Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20040804 Ref country code: AT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20040804 Ref country code: NL Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20040804 Ref country code: FI Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20040804 Ref country code: BE Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20040804 Ref country code: LI Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20040804 |
|
REG | Reference to a national code |
Ref country code: GB Ref legal event code: FG4D |
|
REG | Reference to a national code |
Ref country code: CH Ref legal event code: EP |
|
REG | Reference to a national code |
Ref country code: IE Ref legal event code: FG4D |
|
REF | Corresponds to: |
Ref document number: 60012717 Country of ref document: DE Date of ref document: 20040909 Kind code of ref document: P |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: DK Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20041104 Ref country code: GR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20041104 Ref country code: SE Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20041104 |
|
NLV1 | Nl: lapsed or annulled due to failure to fulfill the requirements of art. 29p and 29m of the patents act | ||
REG | Reference to a national code |
Ref country code: CH Ref legal event code: PL |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: CY Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20050324 Ref country code: IE Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20050324 Ref country code: LU Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20050324 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: MC Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20050331 |
|
PLBE | No opposition filed within time limit |
Free format text: ORIGINAL CODE: 0009261 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT |
|
ET | Fr: translation filed | ||
26N | No opposition filed |
Effective date: 20050506 |
|
REG | Reference to a national code |
Ref country code: IE Ref legal event code: MM4A |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: PT Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20050104 |
|
REG | Reference to a national code |
Ref country code: GB Ref legal event code: 732E Free format text: REGISTERED BETWEEN 20100121 AND 20100127 |
|
REG | Reference to a national code |
Ref country code: FR Ref legal event code: TP |
|
REG | Reference to a national code |
Ref country code: FR Ref legal event code: PLFP Year of fee payment: 16 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: GB Payment date: 20150224 Year of fee payment: 16 Ref country code: FR Payment date: 20150224 Year of fee payment: 16 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: DE Payment date: 20150331 Year of fee payment: 16 |
|
REG | Reference to a national code |
Ref country code: DE Ref legal event code: R119 Ref document number: 60012717 Country of ref document: DE |
|
GBPC | Gb: european patent ceased through non-payment of renewal fee |
Effective date: 20160324 |
|
REG | Reference to a national code |
Ref country code: FR Ref legal event code: ST Effective date: 20161130 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: DE Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20161001 Ref country code: GB Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20160324 Ref country code: FR Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20160331 |