AU745942B2 - A system for performing halftoning - Google Patents

Description

S&F Ref: 486564

AUSTRALIA

PATENTS ACT 1990 COMPLETE SPECIFICATION FOR A STANDARD PATENT

ORIGINAL

S

Name and Address of Applicant: Actual Inventor(s): Address for Service: Invention Title: Canon Kabushiki Kaisha 30-2 Shimomaruko 3-chome Ohta-ku Tokyo 146 Japan Mark Pulver Spruson Ferguson St Martins Tower 31 Market Street Sydney NSW 2000 A System for Performing Halftoning

S

C.

S

ASSOCIATED PROVISIONAL APPLICATION DETAILS [33] Country [31] Applic. No(s) AU PP8006 [32] Application Date 4 January 1999 The following statement is a full description of this invention, including the best method of performing it known to me/us:- 5815c -1- A SYSTEM FOR PERFORMING HALFTONING Field of the Invention The present invention relates to the field of image processing as applicable, for instance, to printed documents or video, and in particular to halftoning by means of error diffusion (ED) and dithering using common hardware and a common data path for both functions.

Background of the Invention 10 Halftoning is a technique used in computer graphics systems for increasing the intensity resolution, where the display or hardcopy device in question produces only a limited number of intensity levels. Considering printers for example, availability of only two printed intensity levels are quite common. The range of perceived intensities however can be increased by making use of the spatial integration performed by the human eye. The techniques for exploiting this attribute of the human eye are called halftoning, which effectively trade off spatial resolution for perceived intensity resolution.

One technique used for achieving halftoning is dithering. In one form dithering involves defining an n x m matrix which is tiled across the image whose intensity resolution is to be enhanced. If, say, n m 3, then the resultant 3 x 3 dither matrix can define 10 possible intensity levels, at the price of reducing spatial resolution by one third on each axis.

Error diffusion is another method for performing halftoning, in which the error between the image pixel value and the value actually printed is diffused, or spread, over a number of surrounding pixels.

On the whole, dithering tends to produce more pleasing results in regard to computer generated images or images having relatively low high frequency information, whereas the error diffusion method generally performs better when applied to natural images.

(cFp1580AU CrdCraft~l) (Halflone) :\ELEC\CISRA\CARDCRFT\CRDCRTO1\486564CAPa.oc -2- The two methods generally require differing designs in terms of hardware and/or software, and so for economic reasons usually only one of the two methods is provided in a given system.

It is considered desirable to provide users with the option of selecting between the two halftone methods described, without the full hardware cost of implementing the two methods in parallel in separate hardware.

Summary of the Invention According to a first aspect of the invention, there is provided apparatus for oo 10 halftoning an input image signal to produce a halftone output image signal, said apparatus ":using one of a dither matrix mode and an error diffusion mode, the apparatus comprising: control means producing mode control signals and memory control signals for selecting one of the dither matrix mode and the error diffusion mode; memory means responsive to the memory control signals, said memory means 15 adapted to store memory data being one of dither matrix coefficients and error diffusion terms; and halftone processing means responsive to the mode control signals, and adapted to receive the input image signal and the memory data, thereby producing the halftone S• output image signal, using common hardware and a common data path for both the dither matrix mode and the error diffusion mode.

Brief Description of the Drawings A preferred embodiment of the invention will now be described, by way of example only, with reference to the accompanying drawings, in which: Fig. 1 is a block diagram of a system in accordance with the preferred embodiment; Fig. 2 depicts error from a pixel being diffused in an error diffusion (ED) method; Fig. 3 depicts error accumulation in a pixel in the ED method; (CFP158AU CrdCraft~l) (Halftone) :\ELEC\CISRA\CARDCRF-RCRDCRT01\486564CAPa.doc Fig. 4 illustrates the halftone process hardware in more detail; Fig. 5 illustrates dither matrix information mapped into an external memory; Fig. 6 illustrates a preferred embodiment of a "next line" pipeline process; Fig. 7 depicts a preferred embodiment of a "current line" pipeline process; Fig. 8 depicts the embodiment being practiced using a conventional general purpose computer.

Detailed Description Fig. 1 is a high level description of a preferred embodiment of the invention, and illustrates how an incoming image signal on a bus 100, which signal has a high 8 bit) granularity of intensity resolution, is processed by the system to produce an output signal on a bus 112 to an output device (not shown) which can print, or display, only a relatively limited range 1 bit) of intensity levels. The incoming image signal on bus 100 is input to an input interface 102, which in turn produces a signal P on a bus 104 the signal P S 15 being suitable for processing by a halftone process 106. This halftone process 106 accesses an external random access memory (RAM) 116 via uni-directional buses 114 and •210. The halftone process 106 inputs signal P on bus 104 and after processing, outputs a signal B on a bus 108 to an output interface 110. The output interface 110 adapts signal B on bus 108 to make it suitable for the output device. In the preferred embodiment, the image signal on bus 100 is an 8 bit data signal, where in contrast, the processed halftone signal on bus 112 is a 1 bit signal.

Control and data signals on line 117 from an external host (not shown) select the desired halftoning method of operation, namely the dither matrix mode, or the error diffusion mode. This selection is implemented via a control process 118 which outputs appropriate address data on line 122, and outputs read/write control signals on line 103 to the external RAM 116. The address data on line 122 and the read/write control signals on line 103 are generated in a manner which is dependent upon the halftoning mode of operation selected. Apart from this distinction, both halftoning modes use the same hardware and datapath. The control process 118 also allows the external host (not shown) (CFP158AU CrdCraft~l) (Halftone) I:\ELEC\CISRA\CARDCRFT\CRDCRTO1\486564CAPa.doc to prefill the external RAM 116 with dither matrix data, and to test the external RAM 116. The explanation which immediately follows will consider use of the error diffusion mode. The use of the dither matrix mode will be described thereafter.

Figs 2 and 3 provide background information on error diffusion half-toning in the preferred embodiment. In particular, Fig. 2 illustrates how an outgoing error Eut (ref.

Fig. 4) is distributed when considering a pixel 300. In the present embodiment, the error is distributed to neighbouring pixels 302 and 304 on the same line, and to pixels 306 to 314 on the line below.

Fig. 3 illustrates the incoming error perspective, namely how individual errors 10 generated in the pixels which neighbour pixel 300 are accumulated in pixel 300. The combined operation of a "next line pipeline" process 206, the external RAM 116, and a S"current line pipeline" process 212 (ref. Fig. 4) produces signal Ein on a bus 214 which is derived from the pixel errors of pixels surrounding pixel 300. Pixels 400 to 408 in the line above pixel 300 all contribute to the error diffused to pixel 300. Similarly, on the same line as pixel 300, pixels 410 and 412 will diffuse error into pixel 300. This is described by the following equation where the subscripts refer to pixel coordinates in Fig.

3, Ein is the accumulated error in the pixel 300, and fA to fG are weights by which each error is multiplied means multiply):

E

in(x,y) Eout(x-ly)*fA+Eout(x-2y)*fB+Eout(x+2,y-1)*f f

E

out(x+ly- 1 fD

E

out(x,y-1)*fE Eout(x- fFEout(x-2,y-l)fG [1] In Fig. 4, it is seen that halftone process 106 is made up of a number of subprocesses including a threshold process 200, the "next line pipeline" process 206, and the "current line pipeline" process 212. The image signal P on bus 104 which is output by the input interface 102 (ref: Fig. 1) is input into the threshold process 200, along with signal Ein which is conveyed on bus 214. Thereafter, the threshold process 200 produces two output signals, namely B on bus 108 and Eou t on bus 204, where B is the actual signal output for pixel 300, and Eout is the error between image signal P and the actual output B.

(CFP1580AU CrdCraft~l) (Halftone) :\ELEC\CISRA\CARDCRFTCRDCRT01\486564CAPa.doc In considering a particular pixel 300, threshold process 200 operates on the incoming signal P on bus 104 and on Ein on bus 214 as described in the equations below.

Ein(x,y) P(x,y) 128, B(x,y) 0, Eout(x,y) (Ein(x,y) P(x,y)) Ein(x,y) P(x,y) 128, B(x,y) 1, Eout(x,y) (Ein(x,y) 255 [2] Signal Ein on bus 214 is the error which has been accumulated in relation to the pixel being considered 300 (ref Fig. and P on bus 104 is the image value of that pixel 300. The above equations have the effect that when the sum of the image input signal P 10 on bus 104 plus the accumulated error signal Ein on bus 214 is less than 128, then the actual output signal B on bus 108 which is used for the pixel in question 300 is 0. If the aforementioned condition is not met, then B is 1. In either case, a resulting error from this operation is output as signal Eou t on bus 204 to be diffused among the surrounding pixels 302-314.

S 15 Error signal Eou t on bus 204 which is generated by threshold process 200 is then fed in parallel to the "next line pipeline" process 206, and to the "current line pipeline" process 212. The "next line pipeline" process 206 produces an output signal on bus 114 which is written to the external RAM 116. The partial sum Eout(x+2y-l)*fcEout(x+l,yfD+Eout(x,y-1) E

E

out(x-,y-1)fl)

E

out(x-2y-l)fG (see Eq. 1) which relates to pixels 306 to 314 on the line below the pixel in question 300 (see Fig. 2) is written to external RAM 116. The partial sum Eout(x1,y)*fA+Eout(x-2,y)*fB (see Eq. 1) which relates to pixels 302 304 on the same line as the pixel in question 300 are stored in the "current line pipeline" 212 (ref: Fig. 4).

The external RAM 116 outputs a signal on bus 210 to the "current line pipeline" process 212. The "current line pipeline" process 212, receives Eu t on bus 204 from threshold process 200, and the output from external RAM 116 on bus 210, and processes the signals to produce the output signal Ein on bus 214. This output signal Ein is input into threshold process 200.

(CFP1580AU CrdCraftl) (Halftone) I:\ELEC\CISRA\CARDCRF\CRDCRTO1\486564CAPa.doc Fig. 5 shows that the external RAM 116 (ref: Fig. 4) in the preferred embodiment is a line store, which has sufficient memory locations (708, 710, to contain the partial sum data of Ein from the "next line pipeline" process 206 for each channel (ie colour) of a full line of the input image. This embodiment assumes that input image pixel data is received one line at a time. That is to say, for example, a complete line of data is received for Red, then a complete line for Green, and finally a complete line for Blue.

Fig. 6 provides detail of the preferred embodiment of the "next line pipeline" process 206 (ref: Fig. In a given clock cycle, signal Eou t on bus 204 from threshold process 200 is input in parallel to multipliers 502, 522, 542, 562, and 582. Each multiplier 502, 522, 542, 562 and 582 has an additional signal input in the form of a respective weight fG, fF, fE, fD, and fc.

Multiplier 502 multiplies input signals Eo,, t on bus 204 and fG to produce an output signal on a line 504. This signal on line 504 is input into an adder 508 which in this, a first stage of the next line pipeline 206 is added to 0, the resultant signal on line 510 being input into a register 512. In the next clock cycle, the content of the register 512 is output on to a line 526 into an adder 528 which is in the next stage of the pipeline process 206.

In Fig. 6 therefore the values stored in the registers 512, 532, 552, 572, 592) of the respective pipeline stages are, in each successive clock cycle, each latched into the respective adder in the following pipeline stage. Final latch 592 provides its output signal into the external RAM 116 on line 114 (ref Fig. 4).

Fig. 7 describes the operation of the "current line pipeline" function 212 (ref: Fig.

In a given clock cycle, signal Eo,, t on bus 204 from the threshold process 200 is input in parallel to multipliers 602 and 622.

Multiplier 602 also has weight fB as input, and the multiplier 602 outputs a signal on a line 604 to an adder 608. The signal from the external RAM 116 (ref: Fig. 4) on bus 210 is also input into the adder 608, which outputs a signal on a line 610 into a register 612. As described above in relation to the next line pipeline 206, the value contained in register 612 is latched into adder 628 during the following clock cycle on (CFP1580AU CrdCraftl) (Halftone) 1:\ELEC\CISRA\CARDCRFT\CRDCRTOI\486564CAPa~o (CFP1 S8OAU CrdCraftOl) (Halftone) I:~ELEC\ClSRA~CARDCRFT\CRDCRTO1~486564CAPa.dOC line 626. The following stage of the current line pipeline 212 operates in the same manner. Final latch 632 outputs the value Ein stored therein onto bus 214 to be input into threshold function 200.

With reference to Figs. 6 and 7, weights fA through fG determine how the error associated with pixel 300 (ref: Fig. 2) is to be distributed to neighbouring pixels 302 314 in the present embodiment.

Having discussed the use of the error diffusion method above, we now consider halftoning using the dither matrix method.

Having reference to Fig. 1 and Fig. 4, the control process 118 now generates 10 addressing information for the external RAM 116 in a manner different than that used when the error diffusion mode is selected. For the dither mode, the external RAM 116 is initialised (ie loaded) with data for a dither matrix on line 114. This address data is provided by the host interface (not shown) via line 117 (see Fig. The loading of this dither matrix data is static, being performed only when the dither mode is started. In dither mode operation, the control process 118 does not generate write signals on line 103 :to the external RAM 116, and thus the dither matrix data is preserved in the RAM 116.

Fig. 5 illustrates the manner in which dither matrix data 704 which is to be mapped onto the image 728 as depicted by arrow 730 is loaded into the external RAM 116 when using the dither matrix method. As noted, this loading of data is static, being performed only when the dither mode is selected. In contrast, external RAM 116 is used in a dynamic read/write fashion when the error diffusion method is selected, being written to via bus 114 from the next line pipeline process 206 each clock cycle.

Fig. 5 illustrates how the dither matrix 704 is tiled onto the input image, the mapping being depicted by an arrow 730. It is seen that individual matrix elements of the dither matrix 704 appear repetitively in the image 728. The dither matrix 704 is also shown mapped into the line store 116, the mapping being depicted by an arrow 706.

Individual rows of the dither matrix 704 are shown mapped in contiguous fashion into the line store 116. Thus the three rows of dither matrix 704 are mapped into the line store (CFP158AU CrdCraftl) (Halftone) :\ELECCISRA\CARDCRFT\CRCRT1 W86564CAPa.doc -8memory positions 708 to 724. In the preferred embodiment, the size of the dither matrix 704 is limited only by the length of the line store 116.

Turning to Fig. 7, when the dither matrix method is selected, the multiplier inputs fg and fA are set to 0. This has the effect that the current line pipeline 212 acts as a shift register, with the signal on bus 210 from external RAM 116 being clocked through without change. The resultant output signal Ein on bus 214 is then input into the threshold process 200.

The dither method determines a pixel's final binary output value B on bus 108 (ref. Fig. 4) by thresholding the image pixel value P on bus 104 against a corresponding 10 value Td from the two dimensional dither array 704 (ref: Fig. 5) where the dither matrix array is tiled across the image 728 as shown in Fig. 5. This threshold process is represented by the following equation: P Td, B=O P> Td, B 1 [3] where P is the image pixel value, Td is the dither matrix element corresponding to the pixel in question, and B is the actual pixel value output. In contrast to equation [2] which relates to the error diffusion method, no error terms are generated in equation In order to implement the dither matrix method using the common hardware and data path already described, the threshold terms are rearranged, such that Ein on bus 214 is a modified dither threshold, no longer representing the diffusion error. This modified version of equation above is described by the following equations: P (127- Td)< 128, B 0 P (127- Td)> 1 2 8, B 1 [4] The aforementioned modification gives equation the same form as equation with (127 Td) in equation replacing Ein(x,y) in equation In the present (CFP1580AU CrdCraftl) (Halftone) 1:\ELECCISRA\CARDCRFT\CRDCRTO I 48656CAPa.doc (CFP158OAU CrdCraftOl) (Halftone) l:\ELEC\CISRA\CARDCRFT\CRDCRTO1\486584CAPa.doc embodiment, this is performed by loading the external RAM 116 through bus 114 with the dither matrix values as described above. An arithmetic offset is introduced into the threshold data Td stored in memory so that the various values referred to Equations [2] through range from -127 to +127, rather than from 0 to 254.

Referring to Fig. 5, and noting that dither matrix 704 is loaded into external RAM 116 as shown in memory locations 708 to 724, the address generation relating to external RAM 116 is modified for the dither matrix method as against the address generation relating to the error diffusion method. In the error diffusion mode, the address generation resets on each new scan line, and the address increments with each pixel. In -10 the dither matrix mode, the address for the external RAM 116 is considered as having two fields, namely an X axis field and a Y axis field. The X axis field of the address is incremented modulo the X-dimension of the dither matrix 704, while the Y axis field of the address is incremented modulo the Y-dimension of dither matrix 704 on each new S.scanline.

One embodiment of the system can be implemented using a conventional general-purpose computer 800 as illustrated in Fig. 8. The various processes described ooooo S°are implemented as software executing on the computer 800. In particular, the various process steps are effected by instructions in the software that are carried out by the computer 800. The software can be stored in a computer readable medium, is loaded onto the computer 800 from the medium, and then executed by the computer 800. The use of the computer program product in the computer preferably performs half-tone processing, using either error diffusion or dither methods on the input image signal, and produces a half-tone signal for output to an output device. The computer system 800 includes a computer module 802, an image input card 816, and input devices keyboard 818 and pointing device 820. In addition, the computer system 800 can have any of a number of other output devices including a display 624. The computer system 800 is connected to one or more other computers using an appropriate communication channel such as a modem communications path, a computer network, or the like. The computer network can include a local area network (LAN), a wide area network (WAN), an Intranet, and/or (CFP1580AU CrdCraft~l) (Halftone) [:\ELECCISRACARDCRFT\CRDCRTO I 48654CAPa.doc (CFP158OAU CrdCraftOl) (Halftone) I\ELEC~CISRA~CARDCRFT\CRDCRTO1 ~486564CAPa.doc Internet. Thus, for example, image signals are input via image input card 816, control commands are input via keyboard 818, and the desired half-tone signal is output via image output card 810. The computer 802 itself includes at least one central processing unit 804 (simply referred to as a processor hereinafter), a memory 806 which can include random access memory (RAM) and read-only memory (ROM), an input/output (IO) interface 808, an image input interface 822, and one or more storage devices generally represented by a block 812. The storage device(s) 812 can include one or more of the following: a floppy disk, a hard disk drive, a magneto-optical disk drive, CD-ROM, magnetic tape or any other of a number of non-volatile storage devices well known to those skilled in the art. In the present embodiment, external RAM 116 would be incorporated here. Each of the components 804, 806, 808, 812 and 822, is typically :connected to one or more of the other devices via a bus 814 that in turn can include data, address, and control buses. The image interface 822 is connected to the image input 816 "and video output 810 cards, and provides video input from the image input card 816 to the 15 computer 802 and from the computer 802 to the video output card 810.

o. The method of halftoning an input image signal can preferably be implemented S:"i in dedicated hardware such as one or more integrated circuits performing the functions or sub functions of halftoning an input image signal. Such dedicated hardware may include graphic processors, digital signal processors, or one or more microprocessors and associated memories.

The foregoing describes only some embodiments of the present invention, and modifications and/or changes can be made thereto without departing from the scope and spirit of the invention, the embodiments being illustrative and not restrictive.

In the context of this specification, the word "comprising" means "including principally but not necessarily solely" or "having" or "including" and not "consisting only of'. Variations of the word comprising, such as "comprise" and "comprises" have corresponding meanings.

(cFP1580Au crdcaanol) (Halftone) I:AELEC\CISRA\CARDCRFCRDCRTO Vb\654CAPa.doc EDITORIAL NOTE NO.65532/99 There is no page number 11 contained in this specification.

Claims (9)

1. An apparatus for halftoning an input image signal to produce a halftone output image signal, said apparatus using one of a dither matrix mode and an error diffusion mode, the apparatus comprising: control means producing mode control signals and memory control signals for selecting one of the dither matrix mode and the error diffusion mode; memory means responsive to the memory control signals, said memory means adapted to store memory data being one of dither matrix coefficients and error diffusion terms; and halftone processing means responsive to the mode control signals, and adapted to receive the input image signal and the memory data, thereby producing the halftone output image signal, using common hardware and a common data path for both the dither matrix mode and the error diffusion mode.

2. An apparatus according to claim 1 wherein: :the memory means stores one of: error diffusion terms for an entire line of the output image in the error diffusion mode; and 20 one or more copies of the dither matrix coefficients in the dither matrix mode.

3. An apparatus according to claim 1 wherein: the memory means are a line store.

4. An apparatus according to claim 2, wherein a size of a dither matrix comprising said dither matrix coefficients is one of a size of the line store, and a sub- multiple thereof. 7s 5. An apparatus according to claim 1 wherein: 486564CAP.DOC -13- the halftone processing means include a comparator means adapted to receive both the input image signal and a threshold, said halftone processing means thereby producing the output image signal; and wherein when the error diffusion mode is selected, said halftone processing means producing an error signal for diffusion to as yet unprocessed pixels.

6. An apparatus according to claim 5, wherein a value of said threshold is dependent upon a selection of the error diffusion mode.

7. An apparatus according to claim 1 wherein: the halftone processing means include a comparator means adapted to receive both the input image signal and a threshold, said halftone processing means thereby producing the output image signal; and *.wherein when the dither matrix method is selected, said halftone processing means not producing an error signal. i An apparatus according to claim 7, wherein a value of said threshold is o' 0: dependent upon a selection of the dither matrix mode. o .*oo 20 9. An apparatus according to claim 1 wherein the halftone processing 00.0omeans includes one or more pipeline data paths. An apparatus according to claim 9, wherein said one or more pipeline data paths comprise a current line pipeline, and a next line pipeline, wherein: said next line pipeline comprises a first plurality of cascaded processing elements, each said processing element receiving an associated diffusion weighting coefficient and an error signal for diffusion to as yet unprocessed pixels; and wherein OFIC486564CAP.DOC 486564CAP.OC -14- for each of said first plurality of cascaded processing elements, a product of said error signal and said associated diffusion weighting coefficient is contributed to a next processing element; and wherein for a terminating one of the processing elements, a product of said error signal and said associated diffusion weighting coefficient, and a contribution from preceding processing elements, is output to the line store.

11. An apparatus according to claim 9, wherein said one or more pipeline data paths comprise a current line pipeline, and a next line pipeline, wherein: said current line pipeline comprises a second plurality of cascaded processing elements, each said processing element receiving an associated diffusion weighting coefficient and an error signal for diffusion to as yet unprocessed pixels; and wherein for each of said second plurality of cascaded processing elements, a product of said error signal and said associated diffusion weighting coefficient is contributed to a 15 next processing element; and wherein for a terminating one of the processing elements, a product of said error signal and said associated diffusion weighting coefficient, and a contribution from preceding processing elements, is output to a threshold process. 20 12. An apparatus according to claim 11, wherein in the dither matrix mode, said associated diffusion weighting coefficients are zero.

13. An apparatus according to claim 1, wherein weighting coefficients associated with the error diffusion terms are programmable.

14. An apparatus for producing a halftone image signal for an output device substantially as described herein with reference accompanying Figs. 1, and 4 to 7. DATED this 6 th Day of February 2002 486564CAP.DOC