AU2011203230A1 - Variable patch size alignment hints - Google Patents

Abstract

- 69 VARIABLE PATCH SIZE ALIGNMENT HINTS Disclosed is a method of aligning a candidate image (164) with a reference image (160). 5 The method selects a first alignment patch (1452) in the reference image and a corresponding first alignment patch (1454) in the candidate image and determines an estimated shift (1450) between the first alignment patch (1452) and the corresponding first alignment patch (1454). A second alignment patch (1462) in the reference image is selected together with a corresponding second alignment patch (1464) in the candidate 10 image. A size (1465) of the second alignment patch (1462) and the corresponding second alignment patch (1464) is dependent upon the determined estimated shift (1450). The method determines a further estimated shift (1460) between the second alignment patch (1462) and the corresponding second alignment patch (1464), aligns (240) the candidate image and the reference image based on the determined estimated shift (1450) and the 15 determined further estimated shift (1460). 5404130_1 991865_specijlodge Image Reference Patch Patch extraction extraction Extract Extract alignment alignment feature feature Align 3450 Fig. 34 CAAA I a991865_figs_lodge

Description

S&F Ref: 991865 AUSTRALIA PATENTS ACT 1990 COMPLETE SPECIFICATION FOR A STANDARD PATENT Name and Address Canon Kabushiki Kaisha, of 30-2, Shimomaruko 3 of Applicant : chome, Ohta-ku, Tokyo, 146, Japan Actual Inventor(s): Eric Wai-Shing Chong Address for Service: Spruson & Ferguson St Martins Tower Level 35 31 Market Street Sydney NSW 2000 (CCN 3710000177) Invention Title: Variable patch size alignment hints The following statement is a full description of this invention, including the best method of performing it known to me/us: 5845c(5407461_1) -1 VARIABLE PATCH SIZE ALIGNMENT HINTS TECHNICAL FIELD The current invention relates generally to the image alignment process employed in the real-time assessment of the quality of printed documents, and particularly, to a system for real-time detection of print defects on the printed medium. 5 BACKGROUND There is a general need for measuring the output quality of a printing system. The results from such quality measurement may be used to fine-tune and configure parameters of the printing system for improved performance. Traditionally, this has been performed in an offline fashion through manual inspection of the output print from the print system. 10 With ever increasing printing speeds and volumes, the need for automated real time detection of print defects to maintain print quality has increased. Timely identification of print defects can allow virtually immediate corrective action, such as re printing, to be taken, which in turn reduces waste in paper and ink or toner, while improving efficiency. Some automated detection systems can operate independent of user 15 or operator input, and may be considered automatic detection systems. A number of automatic print defect detection systems have been developed. In some arrangements, these involve the use of an image acquisition device, such as a CCD (charge-coupled device) camera, to capture or scan an image of a document printout (also referred to as an output print). The scan image is then compared to an image (referred to as 20 the original image) of the original source input document. Discrepancies identified during the comparison can be flagged as print defects. Real-time print defect detection for high volume printing is an exceptionally challenging problem because it requires very accurate image alignment between the scan image and the original image, and a very high processing speed. Some existing print 5404130_1 991865_speci-lodge -2 defect detection methods are unsuitable for real-time processing because they are too slow. Other methods are insufficiently accurate and are sensitive to noise and distortion. This either results in a high rate of false defect detections, or a coarse detection method, where only large defects are detected. 5 SUMMARY It is an object of the present invention to substantially overcome, or at least ameliorate, one or more disadvantages of existing arrangements. Disclosed are arrangements, referred to as Adaptive Print Verification (APV) arrangements, which dynamically adapt an image alignment parameter according to local 10 shift measurements between an original source document image and a digitally captured image of the printed source document, in order to establish correspondence between the two images. The arrangements enable accurate image comparison to be performed for the purpose of print defect detection in real-time. According to a first aspect of the present disclosure, there is provided a method of 15 aligning a candidate image with a reference image, the method comprising: (a) selecting a first alignment patch in the reference image and a corresponding first alignment patch in the candidate image; (b) determining an estimated shift between the first alignment patch and the corresponding first alignment patch; 20 (c) selecting a second alignment patch in the reference image and a corresponding second alignment patch in the candidate image, wherein a size of the second alignment patch and the corresponding second alignment patch is dependent upon the determined estimated shift; (d) determining a further estimated shift between the second alignment patch and 25 the corresponding second alignment patch; and 54041301 991865_speci_lodge -3 (e) aligning the candidate image and the reference image based on the determined estimated shift and the determined further estimated shift. According to another aspect of the present disclosure, there is provided an apparatus for implementing the aforementioned method. 5 According to another aspect of the present disclosure, there is provided a computer readable medium having recorded thereon a computer program for implementing the method described above. Other aspects are also disclosed, including a method of print verification which includes the method of aligning. 10 BRIEF DESCRIPTION OF THE DRAWINGS At least one embodiment of the present invention will now be described with reference to the following drawings, in which: Fig. 1 is a top-level flow-chart showing the determination of unexpected differences in a printed page; 15 Fig. 2 is a flow-chart showing the details of step 150 of Fig. 1; Fig. 3 is a diagrammatic overview of the important components of a print system 300 on which the method of Fig. 1 may be practiced; Fig. 4 is a flow-chart showing the details of step 240 of Fig. 2; Fig. 5 is a flow-chart showing the details of step 270 of Fig. 2; 20 Fig. 6 is a flow-chart showing the details of one implementation of step 520 of Fig. 5; Fig.7 shows a graphical view of how the steps of Fig. 2 can be performed in parallel; Fig. 8 is a flow-chart showing the details of one implementation of step 225 of 25 Fig. 2; 5404130_1 991865_speci!lodge -4 Fig. 9 illustrates a typical dot-gain curve 910 of an electrophotographic system; Fig. 10 is a kernel which can be used in the dot-gain model step 810 of Fig. 8; Fig. 11 shows the detail of two bands which could be used as input to the alignment step 240 of Fig. 2; 5 Fig. 12 shows the process of Fig. 8 as modified in an alternate implementation; Fig. 13 shows the process of Fig. 6 as modified in an alternate implementation; Fig. 14 shows an illustration of band-based variable patch size alignment hint; Fig. 15 shows the relationship between maximum expected shift and hint spatial location; 10 Figs. 16A to 16D illustrate band-based dynamic patch selection in image alignment; Fig. 17 is a flow-chart describing the processing steps of performing band-based dynamic patch selection; Fig. 18 shows an expected shift arrangement within an image band; 15 Fig. 19 is intentionally blank; Figs. 20A and 20B collectively form a schematic block diagram representation of an electronic device upon which described arrangements can be practised; Fig. 21 shows the details of the print verification unit 330 of Fig. 3. Fig. 22 is a flow-chart showing the details of step 445 of Fig. 4; 20 Figs. 23A, 23B and 23C illustrate the basic relationship between the size of a patch and its shift relative to a corresponding location; Fig. 24 is a schematic flow diagram of an application of the system; Fig. 25 is a schematic flow diagram of the fine alignment system occurring within the processes of Fig. 24; 25 Fig. 26 is a schematic flow diagram of a patch identification module 54041301 991865_specilodge -5 Fig. 27 illustrates a multi-resolution image representation with alignable patches at a plurality of spatial densities; Fig. 28 illustrates a multi-resolution approach to measure local spatial distortion; Figs. 29A and 29B illustrate a dynamic patch selection method according to local 5 spatial distortion; Fig. 30 illustrates a multi-scale pyramid-based image alignment method; Fig. 31 is a schematic block diagram of an alignment feature extraction process 3100 based on a minimal implementation; Fig. 32 is a schematic block diagram of an alignment feature extraction 10 process 3200 based on a preferred implementation; Fig. 33 is an illustration of the regions in frequency space where image processing artefacts are most prevalent; Fig. 34 is a schematic block diagram of an alignment process 3400; Fig. 35 is an illustration of an image with sub-regions selected for alignment; 15 Fig. 36 is an illustration of the regions in frequency space after modulation corresponding to those in Fig. 33 after modulation; and Fig. 37 is an illustration of the Fourier representation of a downsampled image. DETAILED DESCRIPTION INCLUDING BEST MODE Where reference is made in any one or more of the accompanying drawings to 20 steps and/or features, which have the same reference numerals, those steps and/or features have for the purposes of this description the same function(s) or operation(s), unless the contrary intention appears. It is to be noted that the discussions contained in the "Background" section and that above relating to prior art arrangements relate to discussions of devices which may 25 form public knowledge through their use. Such discussions should not be interpreted as a 5404130_1 991865.specijlodge -6 representation by the present inventor(s) or the patent applicant that such devices in any way form part of the common general knowledge in the art. With reference to Fig. 1, an output print 163 of a print process 130 performed by a print system 300, as seen in Fig. 3, will not, in general, precisely reflect the associated 5 source input document 166. This is because the print process 130, through which the source input document 166 is processed to produce the output print 163, introduces some changes to the source input document 166 by virtue of the physical characteristics of a print engine 329 and associated movement and operations inside the print system 300. Furthermore, if the source input document 166 is compared with a scan image 164 of the 10 output print 163, the physical characteristics of a scan process 140 by which the scan image 164 is formed also contribute changes to the source input document 166. As a result of these cumulative changes, a direct comparison between the source input document 166 and the scan image 164 is unsuitable for assessing the quality of the output print 163. In order to effectively assess the quality of the output print 163, these changes must be 15 accounted for before the source input document 166 is compared to the scan image 164. In going from the source input document 166 to the scan image 164, each pixel in the source input document 166 is expected to undergo changes or transformations in both the intensity domain and in the spatial domain. Intensity transformations are mainly caused by the physical characteristics of the 20 various processes through which the source input document 166 passes in producing the output print 163. Intensity transformations tend to produce expected differences and are dependent on the operating conditions of the print system 300. Those operating conditions are often represented using a set of parameters in the form of a printer model. Spatial transformations are mainly caused by the mechanical movements inside 25 the print system 300. Spatial transformations can severely affect the pixel placement 5404130_1 991865_speci_lodge -7 accuracy on the print medium and presents a major challenge to real-time print output checking. There may also be further differences between the scan image 164 and the source input document 166, which are not accounted for by the transformations that occur within 5 the print process 130 and the scan process 140. Such further differences are referred to as unexpected differences, and these can be amenable to corrective action. The unexpected differences are also referred to as "print defects". The disclosed Adaptive Print Verification (APV) arrangements discriminate between expected and unexpected differences by dynamically adapting to the 10 transformations that occur within the print process 130 and the scan process 140. By generating an "expected print result" in accordance with the transformations, it is possible to check that the output meets expectations with a reduced danger of falsely detecting an otherwise expected change as a defect (known as "false positives"). In one APV arrangement, the output print 163 produced by the print process 130 15 of the print system from the source document 166 is scanned to produce the digital representation 164 (hereinafter referred to as a scan image) of the output print 163. In order to detect print errors in the output print 163, a set of parameters which model characteristics of the print mechanism of the print system are firstly determined, and values for these parameters are determined based on operating condition data for at least a part of 20 the print system 300. This operating condition data may be determined from the print system itself or from other sources, such as for example, external sensors adapted to measure environmental parameters such as the humidity and/or the temperature in which the print system 300 is located. The value associated with each of the parameters is used to generate, by modifying a rendering 160 of the source document 166, an expected digital 25 representation of the output print 163. The expected digital representation takes into 5404130_1 991865_specilodge -8 account the physical characteristics of the print system 300, thereby effectively compensating for output errors associated with operating conditions of the print system (these output errors being expected differences) in the intensity domain. The generated expected digital representation is then aligned to the scan image 164 to remove any spatial 5 transformations caused during printing or scanning. The aligned expected digital representation is compared to the scan image 164 of the output print 163 in order to detect unexpected differences (i.e. differences not attributable to the physical characteristics of the print system or spatial transformations caused during printing or scanning) these being identified as print errors in the output of the print system. 10 Fig. 3 is a diagrammatic overview of the important components of a print system 300 on which the method 100 of Fig. I may be practiced. An expanded depiction is shown in Figs. 20A and 20B. In particular, Fig. 3 is a schematic block diagram of a printer 300 with which the APV arrangements can be practiced. The printer 300 comprises a central processing unit 301 connected to four chromatic image forming units 302, 303, 15 304, and 305. For ease of description, chromatic colourant substances are each referred to simply as the respective colour space -"colourant". In the example depicted in Fig. 3, an image forming unit 302 dispenses cyan colourant from a reservoir 307, an image forming unit 303 dispenses magenta colourant from a reservoir 308, an image forming unit 304 dispenses yellow colourant from a reservoir 309, and an image forming unit 305 dispenses 20 black colourant from a reservoir 310. In this example, there are four chromatic image forming units, creating images with cyan, magenta, yellow, and black (known as a CMYK printing system). Printers with less or more chromatic image forming units and different types of colourants are also available. The central processing unit 301 communicates with the four image forming 25 units 302-305 by a data bus 312. Using the data bus 312, the central processing unit 301 5404130_1 991865_specijlodge -9 can receive data from, and issue instructions to, (a) the image forming units 302-305, as well as (b) an input paper feed mechanism 316, (c) an output visual display and input control user interface 320, and (d) a memory 323 used to store information needed by the printer 300 during its operation. The central processing unit 301 also has a link or 5 interface 322 to a device 321 that acts as a source of data to print. The data source 321 may, for example, be a personal computer, the Internet, a Local Area Network (LAN), or a scanner, etc., from which the central processing unit 301 receives electronic information to be printed, this electronic information being the source document 166 in Fig. 1. The data to be printed may be stored in the memory 323. Alternatively, the data source 321 to be 10 printed may be directly connected to the data bus 312. When the central processing unit 301 receives data to be printed, instructions are sent to the input paper feed mechanism 316. The input paper feed mechanism 316 takes a sheet of paper 319 from an input paper tray 315, and places the sheet of paper 319 on a transfer belt 313. The transfer belt 313 moves in the direction of an arrow 314 (from right 15 to left horizontally in Fig. 3), to cause the sheet of paper 319 to sequentially pass by each of the image forming units 302-305. As the sheet of paper 319 passes under each image forming unit 302, 303, 304, 305, the central processing unit 301 causes the image forming unit 302, 303, 304, or 305 to write an image to the sheet of paper 319 using the particular colourant of the image forming unit in question. After the sheet of paper 319 has passed 20 under all the image forming units 302-305, a full colour image will have been placed on the sheet of paper 319. For the case of a fused toner printer, the sheet of paper 319 then passes by a fuser unit 324 that affixes the colourants to the sheet of the paper 319. The image forming units and the fusing unit 324 are collectively known as a print engine 329. The output print 163 25 of the print engine 329 can then be checked by a print verification unit 330 (also referred to 5404130_1 991865_specilJodge - 10 as a print defect detector system). The sheet of paper 319 is then passed to a paper output tray 317 by an output paper feed mechanism 318. The printer architecture in Fig. 3 is for illustrative purposes only. Many different printer architectures can be adapted for use by the APV arrangements. In one example, the 5 APV arrangements can take the action of sending instructions to the printer 300 to reproduce the output print if one or more errors are detected. Figs. 20A and 20B collectively form a schematic block diagram representation of the print system 300 in more detail, in which the print system is referred to by the reference numeral 2001. Figs. 20A and 20B collectively form a schematic block diagram of a print 10 system 2001 including embedded components, upon which the APV methods to be described are desirably practiced. The print system 2001 in the present APV example is a printer in which processing resources are limited. Nevertheless, one or more of the APV functional processes may alternately be performed on higher-level devices such as desktop computers, server computers, and other such devices with significantly larger processing 15 resources, which are connected to the printer. As seen in Fig. 20A, the print system 2001 comprises an embedded controller 2002. Accordingly, the print system 2001 may be referred to as an "embedded device." In the present example, the controller 2002 has the processing unit (or processor) 301 which is bi-directionally coupled to the internal storage module 323 (see 20 Fig. 3). The storage module 323 may be formed from non-volatile semiconductor read only memory (ROM) 2060 and semiconductor random access memory (RAM) 2070, as seen in Fig. 20B. The RAM 2070 may be volatile, non-volatile or a combination of volatile and non-volatile memory. The print system 2001 includes a display controller 2007 (which is an expanded 25 depiction of the output visual display and input controls 320), which is connected to a 5404130.1 991865_speciJodge -11 video display 2014, such as a liquid crystal display (LCD) panel or the like. The display controller 2007 is configured for displaying graphical images on the video display 2014 in accordance with instructions received from the embedded controller 2002, to which the display controller 2007 is connected. 5 The print system 2001 also includes user input devices 2013 (which is an expanded depiction of the output visual display and input controls 320) which are typically formed by keys, a keypad or like controls. In some implementations, the user input devices 2013 may include a touch sensitive panel physically associated with the display 2014 to collectively form a touch-screen. Such a touch-screen may thus operate as 10 one form of graphical user interface (GUI) as opposed to a prompt or menu driven GUI typically used with keypad-display combinations. Other forms of user input devices may also be used, such as a microphone (not illustrated) for voice commands or a joystick/thumb wheel (not illustrated) for ease of navigation about menus. As seen in Fig. 20A, the print system 2001 also comprises a portable memory 15 interface 2006, which is coupled to the processor 301 via a connection 2019. The portable memory interface 2006 allows a complementary portable memory device 2025 to be coupled to the print system 2001 to act as a source or destination of data or to supplement an internal storage module 323. Examples of such interfaces permit coupling with portable memory devices such as Universal Serial Bus (USB) memory devices, Secure Digital (SD) 20 cards, Personal Computer Memory Card International Association (PCMIA) cards, optical disks and magnetic disks. The print system 2001 also has a communications interface 2008 to permit coupling of the print system 2001 to a computer or communications network 2020 via a connection 2021. The connection 2021 may be wired or wireless. For example, the 25 connection 2021 may be radio frequency or optical. An example of a wired connection 5404130_1 991865_speci_lodge - 12 includes Ethernet. Further, an example of wireless connection includes Bluetooth"T type local interconnection, Wi-Fi (including protocols based on the standards of the IEEE 802.11 family), Infrared Data Association (IrDa) and the like. The source device 321 may, as in the present example, be connected to the processor 301 via the network 2020. 5 The print system 2001 is configured to perform some or all of the APV sub processes in the process 100 in Fig. 1. The embedded controller 2002, in conjunction with the print engine 329 and the print verification unit 330 which are depicted by a special function 2010, is provided to perform that process 100. The special function components 2010 is connected to the embedded controller 2002. 10 The APV methods described hereinafter may be implemented using the embedded controller 2002, where the processes of Figs. 1-2, 4-6, 8, 12-13, 17, 22, and 32-33 may be implemented as one or more APV software application programs 2033 executable within the embedded controller 2002. The APV software application programs 2033 may be functionally distributed 15 among the functional elements in the print system 2001, as shown in the example in Fig. 21 where at least some of the APV software application program is depicted by a reference numeral 2103. The print system 2001 of Fig. 20A implements the described APV methods. In particular, with reference to Fig. 20B, the steps of the described APV methods are effected 20 by instructions in the software 2033 that are carried out within the controller 2002. The software instructions may be formed as one or more code modules, each for performing one or more particular tasks. The software may also be divided into two separate parts, in which a first part and the corresponding code modules performs the described APV methods and a second part and the corresponding code modules manage a user interface 25 between the first part and the user. 5404130_1 991865_speci_lodge - 13 The software 2033 of the embedded controller 2002 is typically stored in the non volatile ROM 2060 of the internal storage module 323. The software 2033 stored in the ROM 2060 can be updated when required from a computer readable medium. The software 2033 can be loaded into and executed by the processor 301. In some instances, 5 the processor 301 may execute software instructions that are located in RAM 2070. Software instructions may be loaded into the RAM 2070 by the processor 301 initiating a copy of one or more code modules from ROM 2060 into RAM 2070. Alternatively, the software instructions of one or more code modules may be pre-installed in a non-volatile region of RAM 2070 by a manufacturer. After one or more code modules have been 10 located in RAM 2070, the processor 301 may execute software instructions of the one or more code modules. The APV application program 2033 is typically pre-installed and stored in the ROM 2060 by a manufacturer, prior to distribution of the print system 2001. However, in some instances, the application programs 2033 may be supplied to the user encoded on one 15 or more CD-ROM (not shown) and read via the portable memory interface 2006 of Fig. 20A prior to storage in the internal storage module 323 or in the portable memory 2025. In another alternative, the software application program 2033 may be read by the processor 301 from the network 2020, or loaded into the controller 2002 or the portable storage medium 2025 from other computer readable media. Computer readable 20 storage media refers to any storage medium that participates in providing instructions and/or data to the controller 2002 for execution and/or processing. Examples of such storage media include floppy disks, magnetic tape, CD-ROM, a hard disk drive, a ROM or integrated circuit, USB memory, a magneto-optical disk, flash memory, or a computer readable card such as a PCMCIA card and the like, whether or not such devices are internal 25 or external of the print system 2001. Examples of computer readable transmission media 5404130_1 991865_specijlodge -14 that may also participate in the provision of software, application programs, instructions and/or data to the print system 2001 include radio or infra-red transmission channels as well as a network connection to another computer or networked device, and the Internet or Intranets including e-mail transmissions and information recorded on Websites and the 5 like. A computer readable medium having such software or computer program recorded on it is a computer program product. The second part of the APV application programs 2033 and the corresponding code modules mentioned above may be executed to implement one or more graphical user interfaces (GUls) to be rendered or otherwise represented upon the display 2014 of 10 Fig. 20A. Through manipulation of the user input device 2013 (e.g., the keypad), a user of the print system 2001 and the application programs 2033 may manipulate the interface in a functionally adaptable manner to provide controlling commands and/or input to the applications associated with the GUI(s). Other forms of functionally adaptable user interfaces may also be implemented, such as an audio interface utilizing speech prompts 15 output via loudspeakers (not illustrated) and user voice commands input via the microphone (not illustrated). Fig. 20B illustrates in detail the embedded controller 2002 having the processor 301 for executing the APV application programs 2033 and the internal storage 323. The internal storage 323 comprises read only memory (ROM) 2060 and 20 random access memory (RAM) 2070. The processor 301 is able to execute the APV application programs 2033 stored in one or both of the connected memories 2060 and 2070. When the electronic device 2002 is initially powered up, a system program resident in the ROM 2060 is executed. The application program 2033 permanently stored in the ROM 2060 is sometimes referred to as "firmware". Execution of the firmware by 5404130_1 991865_specijlodge - 15 the processor 301 may fulfil various functions, including processor management, memory management, device management, storage management and user interface. The processor 301 typically includes a number of functional modules including a control unit (CU) 2051, an arithmetic logic unit (ALU) 2052 and a local or internal 5 memory comprising a set of registers 2054 which typically contain atomic data elements 2056, 2057, along with internal buffer or cache memory 2055. One or more internal buses 2059 interconnect these functional modules. The processor 301 typically also has one or more interfaces 2058 for communicating with external devices via system bus 2081, using a connection 2061. 10 The APV application program 2033 includes a sequence of instructions 2062 through 2063 that may include conditional branch and loop instructions. The program 2033 may also include data, which is used in execution of the program 2033. This data may be stored as part of the instruction or in a separate location 2064 within the ROM 2060 or RAM 2070. 15 In general, the processor 301 is given a set of instructions, which are executed therein. This set of instructions may be organised into blocks, which perform specific tasks or handle specific events that occur in the print system 2001. Typically, the APV application program 2033 waits for events and subsequently executes the block of code associated with that event. Events may be triggered in response to input from a user, via 20 the user input devices 2013 of Fig. 20A, as detected by the processor 301. Events may also be triggered in response to other sensors and interfaces in the print system 2001. The execution of a set of the instructions may require numeric variables to be read and modified. Such numeric variables are stored in the RAM 2070. The disclosed method uses input variables 2071 that are stored in known locations 2072, 2073 in the 25 memory 2070. The input variables 2071 are processed to produce output variables 2077 5404130_1 991865_speci_lodge -16 that are stored in known locations 2078, 2079 in the memory 2070. Intermediate variables 2074 may be stored in additional memory locations in locations 2075, 2076 of the memory 2070. Alternatively, some intermediate variables may only exist in the registers 2054 of the processor 301. 5 The execution of a sequence of instructions is achieved in the processor 301 by repeated application of a fetch-execute cycle. The control unit 2051 of the processor 301 maintains a register called the program counter, which contains the address in ROM 2060 or RAM 2070 of the next instruction to be executed. At the start of the fetch execute cycle, the contents of the memory address indexed by the program counter is loaded into the 10 control unit 2051. The instruction thus loaded controls the subsequent operation of the processor 301, causing for example, data to be loaded from ROM memory 2060 into processor registers 2054, the contents of a register to be arithmetically combined with the contents of another register, the contents of a register to be written to the location stored in another register and so on. At the end of the fetch execute cycle the program counter is 15 updated to point to the next instruction in the system program code. Depending on the instruction just executed this may involve incrementing the address contained in the program counter or loading the program counter with a new address in order to achieve a branch operation. Each step or sub-process in the processes of the APV methods described below is 20 associated with one or more segments of the application program 2033, and is performed by repeated execution of a fetch-execute cycle in the processor 301 or similar programmatic operation of other independent processor blocks in the print system 2001. Returning to Fig. 1, this top-level flow chart shows a process useful for determining if a page contains unexpected differences. In particular, Fig. 1 provides a 25 high-level overview of a process for performing colour imaging according to a preferred 5404130_1 991865_speciodge - 17 APV arrangement running on the printer 300 including the print verification unit 330. The verification unit 330 is shown in more detail in Fig. 21. Fig. 21 shows the details of the print verification unit 330 of Fig. 3, which forms part of the special function module 2010 in Fig. 20A. The unit 330, which performs the 5 noted verification process, employs an image inspection device in the form of an image capture system 2108 to assess the quality of output prints by detecting unexpected print differences generated by the print engine 329 which performs the printing step 130 in the arrangement in Fig. 1. The source input document 166 to the system is, in the present example, a digital document expressed in the form of a page description language (PDL) 10 script, which describes the appearance of document pages. Document pages typically contain text, graphical elements (line-art, graphs, etc.) and digital images (such as photographs). The source input document 166 can also be referred to as a source image, source image data and so on. In a rendering step 120, the source document 166 is rendered using a rasteriser 15 (under control of the CPU 301 executing the APV software application 2033), by processing the PDL, to generate a two-dimensional bitmap image 160 of the source document 166. The rasteriser may be implemented in hardware, software, or a combination of the two. This two dimensional bitmap version 160 of the source document 166 is referred to as the original image 160 hereinafter. In addition, the 20 rasteriser can generate alignment information (also referred to as "alignment hints") that can take the form of a list 162 of regions of the original image 160 with intrinsic alignment structure (referred to as "alignable" regions hereinafter). Each alignment hint corresponds to a region, which is preferably a square patch, with a patch centre (x,y), a patch strength indicating the amount of intrinsic alignment structure within the patch (e.g. high frequency 25 components, edges, corners, etc.), and a patch size, for example corresponding to the width 5404130_1 991865_speci_lodge - 18 and height of the region. There is also desirably established an associated patch separation which defines the minimum distance from neighbouring patches, thereby forming an exclusion zone around each patch where no other patches may appear. The list 162 of alignment hints is sorted in the y-direction, as this is typically the transport direction 314. 5 The rendered original image 160 and the associated list of alignment hints 162 are temporarily stored in the printer memory 323. Upon completing processing in the rendering step 120, the rendered original image 160 is sent to the colour printer process 130. The colour printer process 130 uses the print engine 329, and produces the output print 163 by forming a visible image on a 10 print medium such as the paper sheet 319 using the print engine 329. The rendered original image 160 in the image memory is transferred in synchronism with (a) a sync signal and clock signal (not shown) required for operating the print engine 329, and (b) a transfer request (not shown) of a specific colour component signal or the like, via the bus 312. The rendered original image 160 together with the generated alignment data 162 15 is also sent (a) to a memory 2104 of the print verification unit 330 via the bus 312 and (b) a print verification unit 1/0 unit 2105, as seen in Fig. 21, for use in a subsequent defect detection process 150. The output print 163 (which is on the paper sheet 319 in the described example) that is generated by the colour print process 130 is scanned by an image capturing 20 process 140 using, for example, the image capture system 2108. The image capturing system 2108 may be a colour line scanner for real-time imaging and processing. However, any image capturing device that is capable of digitising and producing high quality digital copy of printouts can be used. Such may include, in some implementations, an imaging device such as a digital camera. 5404130_1 991865_speci_lodge -19 In one APV arrangement as depicted in Fig. 21, the scanner 2108 can be configured to capture an image of the output print 163 from the sheet 319 on a scan-line by scan-line basis, or on a band-by-band basis, where each band comprises a number of scan lines. The captured digital image 164 is sent to the print defect detection process 150 5 (performed by the APV Application Specific Integrated Circuit ASIC 2107 and/or the APV software 2103 application), which aligns and compares the original image 160 and the scan image 164 using the alignment data 162 from the rendering process 120 in order to locate and identify print defects. Upon completion, the print defect detection process 150 outputs a defect map 165 indicating defect types and locations of all detected defects. This is used 10 to make a decision on the quality of the page 163 in a decision step 170, which produces a decision signal 175. This decision signal 175 can then be used to trigger an automatic reprint or alert the user to potential issues with the output printed page 163. In one implementation, the decision signal 175 is set to "1" (error present) if there are more than 10 pixels marked as defective in the defect map 165, or is set to "0" (no error) 15 otherwise. Alternatively, this defective pixel number may be set by the user. Fig. 7 and Fig. 21 show how, in a preferred APV arrangement, the printing process 130, the scanning process 140 and the defect detection process 150 can be arranged in a pipeline. In this arrangement, a section 2111, such as a band of the rendered original image 160, is printed by the print system engine 329 to form a corresponding 20 section of the output print 163 on the paper sheet 319. When the printed section 2111 of the output print 163 moves to a position 2112 under the image capture system 2108, it is scanned according to the scanning process 140 using the image capture system 2108 to form part of the scanned image 164. The scan of section 2112, as a band of scan-lines, is sent to the print defect detection process 150 for alignment and comparison with the 5404130_1 991865_speci_lodge -20 corresponding rendered section (band 2111) of the original image 160 that was sent to the print engine 329. Fig. 7 shows a graphical view of how the steps of Fig. 2 (to be described) can be performed in parallel. In particular, Fig. 7 shows that as the page 319 moves in the feed 5 direction 314, a first section of the rendered original image 160 is printed 710 by the print system engine 329 to form a first section of the printed page 163. As the second section is being printed 715 on the printed page 163, the first section is scanned 720 by the scanning process 140 using the scanner 2108 to form a first section of the scanned image 164. As the third section is being printed 745, the first scanned section is sent 730 to the print 10 defect detection process 150 for alignment and comparison with the corresponding first rendered section that was sent to the print system engine 329, and the second printed section is scanned 725. Subsequent sections of the rendered original image 160 may then be processed in the same manner as shown in Fig. 7. Thus, the pipeline arrangement allows all three processing stages (print, scan, detect) to occur concurrently after the first 15 two sections. Returning to Fig. 1, it is advantageous, during rasterisation in the step 120, to perform image analysis on the rendered original image 160 in order to identify the alignable regions 162 which provide valuable alignment hints to the print defect detection step 150 as shown in Fig. 1. Accurate registration of the original image 160 and the 20 printout scan image 164 enables image quality metric evaluation to be performed on a pixel-to-pixel basis. One of the significant advantages of such an approach is that precise image alignment can be performed without the need to embed special registration marks or patterns explicitly in the source input document 166 and/or the original image 160. The image analysis performed in the rendering step 120 to determine the alignment hints may 25 be based on Harris corners, or other useful image features. 5404130_1 991865_speci_lodge - 21 The process of detecting Harris corners is described in the following example. Given an A4 size document rendered at 300dpi, the rasteriser process in the step 120 generates the original image 160 with an approximate size of 2500 by 3500 pixels. The first step for detecting Harris corners is to determine the gradient or spatial derivatives of a 5 grey-scale version of the original image 160 in both x and y directions, denoted as I, and I,. In practice, this can be approximated by converting the rendered document 160 to grey-scale and applying the Sobel operator to the grey-scale result. To convert the original image 160 to grey-scale, if the original image 160 is an RGB image, the method of Equation [1] can be used: 10 IG R1I , + R 12 1 g + RA3lb [] where IG is the grey-scale output image, 1r, 1g, and lb are the Red, Green, and Blue image components, and the reflectivity constants are defined as R, = 0.2990, R1 = 0.5870, and R, 3 = 0.1140. An 8-bit (0 to 255) encoded CMYK original image 160 can be similarly converted 15 to grey-scale using the following simple approximation of Equation [2]: IG = RH MAX(255-I, -I,,0) + Ry, 2 MAX(255- I,,, -1,,0) [2] + Ry, 3 MAX(255- I, - I,,0) Other conversions may be used if a higher accuracy is required, although it is generally sufficient in this step to use a fast approximation. The Sobel operators use the kernels of Equations [3]: -1 0 1~ S.,= -2 0 2 -- 1 0 1 1 -2 -1 S, = 0 0 0 20 -1 2 [3] 5404130_1 991865_specilodge -22 Edge detection is performed with the operations of Equations [4]: I,=S,*IG I, = S, * I G [4] where - is the convolution operator, IG is the grey-scale image data, S yS are the kernels defined above, and I, and I, are images containing the strength of the edge in the x and y 5 direction respectively. From 1, and I,, three images are produced as represent by Equations [5]: 1"' I,, = I, o I, I,, = I, o Iy [5] where o is a pixel-wise multiplication. This allows a local structure matrix A to be calculated over a neighbourhood 10 around each pixel, using the relationship of Equation [6]: A=Zwtx,y~ 'xy XIy,, I2 , [6] where w(x,y) is a windowing function for spatial averaging over the neighbourhood. In a preferred APV arrangement w(x,y) can be implemented as a Gaussian filter with a standard deviation of 10 pixels. The next step in the process of detecting Harris corners is 15 to form a "cornerness" image by determining the minimum eigenvalue of the local structure matrix at each pixel location. The cornerness image is a 2D map of the likelihood that each pixel is a corner. A pixel is classified as a corner pixel if it is the local maximum (that is, has a higher cornerness value than its 8 neighbours). A list of all the corner points detected, C,,,,,er. together with the strength 20 (cornerness) at that point is created. The list of corner points, C is further filtered by 5404130_1 991865_speci_lodge -23 deleting points which are within S pixels from another, stronger, corner point. In the current APV arrangement, S =64 is used. The list of accepted corners, Cnew' is output from the rendering (rasterising) process 120 to the defect detection step 150 as the list 162 of alignment hints for use in 5 image alignment. Each entry in the list can be described by a data structure comprising three data fields for storing the x-coordinate of the centre of the region (corresponding to the location of the corner), the y-coordinate of the centre of the region, and the corner strength of the region. Alternatively, other suitable methods for determining feature points in the original 10 image 160 such as Gradient Structure Tensor or Scale-Invariant Feature Transform (SIFT) can also be used. In another APV arrangement, the original image 160 is represented as a multi scale image pyramid in the render step 120, prior to determining the alignment hints 162. The image pyramid is a hierarchical structure composed of a sequence of copies of the 15 original image 160 in which both sample density and resolution are decreased in regular steps. This approach allows image alignment to be performed at different resolutions, providing an efficient and effective method for handling output prints 163 on different paper sizes or printout scan images 164 at different resolutions. Fig. 2 is a flow-chart showing the details of step 150 of Fig. 1. The process 150 20 works on bands of the original image 160 and the scan image 164. A band of the scan image 164 corresponds to a band 2112 of the page 319 scanned by the scanner 2108. For some print engines 329, the image is produced in bands 2111, so it is sometimes convenient to define the size of the two bands 2111, 2112 to be the same. A band of the scan image 164, for example, is a number of consecutive image lines stored in the memory 25 buffer 2104. A height of each band (not illustrated) is desirably 256 scanlines in the 5404130_1 991865_specilodge - 24 present APV arrangement example, and the width of each band may be the width of the input image 160. In the case of an A4 original document 160 at 300dpi, the width is 2490 pixels. Image data on the buffer 2104 is updated continuously in a "rolling buffer" arrangement where a fixed number of scanlines are acquired by the scanning sensors 2108 5 in the scanning step 140, and stored in the buffer 2104 by flushing an equal number of scanlines out of the buffer 2104 in a first-in-first-out (FIFO) manner. In one APV arrangement example, the number of scanlines acquired at each scanner sampling instance is 64. As seen in Fig. 2, processing of the step 150 begins at a scanline band retrieval 10 step 210 where the memory buffer 2104 is filled with a band of image data from the scan image 164 fed by the scanning step 140. In one APV arrangement example, the scan band is optionally downsampled in a downsampling step 230, for example using a separable Burt-Adelson filter, to reduce the amount of data to be processed, to thereby output a scan band 235 which is a band of the scan image 164. 15 Around the same time, a band of the original image 160 at the corresponding resolution and location as the scan band, is obtained in an original image band and alignment data retrieval step 220. Furthermore, the list of alignment hints 162 generated during rendering in the step 120 for image alignment is passed to the step 220. Once the corresponding original image band has been extracted in the step 220, a model of the print 20 and capture process (hereafter referred to as a "print/scan model" or merely as a "model") is applied at a model application step 225, which is described in more detail in regard to Fig. 8. The print/scan model applies a set of transforms to the original image 160 to change the image 160 in some of the ways ordinarily changed by the true print and capture processes. These transforms produce an image representing the expected output of a print 5404130_1 991865_speci_lodge - 25 and scan process, referred to as the "expected image". The print/scan model may include many smaller component models. Fig. 8 is a flow-chart showing the details of one example 225a of step 225 of Fig. 2. In the example 225a of Fig. 8, three important effects are modelled, namely a dot 5 gain model 810, an MTF (Modulation Transfer Function) model 820, used for example for blur simulation, and a colour model 830. Each of these smaller models can receive as an input one or more printer operating conditions 840. Printer operating conditions are various aspects of machine state which have an impact on output quality of the output print 163. Since the operating conditions 840 are, in general, time varying, the print/scan 10 model application step 225 will also be time varying, reflecting the time varying nature of the operating conditions 840. In a particular implementation, operating conditions are parameters derived from print system sensor outputs and environmental sensor outputs. These operating condition parameters are used to determine a set of parameters for characterising and modelling the print mechanism. This printer model allows a digital 15 source document to be rendered with an appearance that resembles a printed copy of the document under those operating conditions. Examples of printer operating conditions include the output of a sensor that detects the type of paper 319 held in the input tray 315, the output of a sensor that monitors the age of printing drum(s) in the image forming units 302-305 (the number of pages 20 printed using the current drum (not shown), also known as the drum's "click count"), the output of a sensor that monitors the level and age of toner/ink in the reservoirs 307-3 10, the output of a sensor that measures the internal humidity inside the print engine 329, the output of a sensor that measures the internal temperature in the print system 300, the time since the last page was printed (also known as idle time), the time since the machine last 25 performed a self-calibration, pages printed since last service, and so on. These operating 5404130_1 991865_speci_lodge -26 conditions are measured by a number of operating condition detectors for use in the print process or to aid service technicians, implemented using a combination of sensors (e.g., a toner level sensor for toner level notification in each of the toner reservoirs 307-310, a paper type sensor, or a temperature and humidity sensor), a clock (e.g., to measure the time 5 since the last print), and internal counters (e.g., the number of pages printed since the last service). For example, if the toner level sensor indicates a low level of toner, the printer model is adapted in such a way that the rendered digital document would have an appearance that resembles a printed page with faint colours. Each of the models 810, 820 and 830 will now be described in more detail. 10 In the dot-gain model step 810, the image is adjusted to account for dot-gain. Dot-gain is the process by which the size of printed dots appears larger (known as positive dot-gain) or smaller (known as negative dot-gain) than the ideal size. For example, Fig. 9 shows a typical dot-gain curve graph 900. Fig. 9 illustrates a typical dot-gain curve 910 of an electrophotographic system. 15 The ideal result 920 of printing and scanning different size dots is that the observed (output) dot size will be equal to the expected dot size. A practical result 910 may have the characteristic shown where very small dots (less than 5 pixels at 600 DPI in the example graph 900) are observed as smaller than expected, and larger dots (5 pixels or greater in the example graph 900) are observed as larger than expected. Dot-gain can vary according to 20 paper type, humidity, drum click count, and idle time. Some electrophotographic machines with 4 separate drums can have slightly different dot-gain behaviours for each colour, depending on the age of each drum. Dot-gain is also typically not isotropic, and can be larger in a process direction than in another direction. For example, the dot-gain on an electrophotographic process may be higher in the direction of paper movement. This can 25 be caused by a squashing effect of the rolling parts on the toner. In an inkjet system, dot 5404130_1 991865_speciodge - 27 gain may be higher in the direction of head movement relative to the paper. This can be caused by air-flow effects which can spread a single dot into multiple droplets. In one implementation for an electrophotographic process, an approximate dot gain model can be implemented as a non-linear filter on each subtractive colour channel 5 (e.g., C/M/Y/K channels) as depicted in Equation [7] as follows: Id =I+Mo(I*(soK)) [7] where I is the original image 160 for the colour being processed, K is the dot-gain kernel for the colour being processed, M is a mask image, and 'd is the resulting dot-gained image. The mask image M is defined as a linearly scaled version of the original image 160 10 such that I is white (no ink/toner), and 0 is full coverage of ink/toner. An example dot gain kernel 1000 which can be used for K is shown in Fig. 10. The dot-gain kernel 1000 produces a larger effect in the vertical direction, which is assumed to be the process direction in this case. The effect of the dot-gain kernel K is scaled by a scale factor s, which is defined as follows in Equation [8]: 15 s = d+ MIN0.2, j [8] where d is the drum lifetime, and t is the idle time. Drum lifetime d is a value ranging between 0 (brand new) and I (due for replacement). A typical method for measuring the d factor counts the number of pages that have been printed using the colour of the given drum, and divides this by the expected lifetime in pages. The idle time t of the machine, 20 measured in days, is also included in this model. This is only one possible model for dot-gain which utilises some of the operating conditions 840, and the nature of dot-gain is dependent on the construction of the print engine 329. 5404130_1 991865_speci_lodge - 28 In another implementation for an ink-jet system, a set of dot-gain kernels can be pre-calculated for each type of paper, and a constant scale factor s = 1.0 can be used. The dot-gain of inkjet print systems can vary strongly with paper type. Particularly, plain paper can show a high dot gain due to ink wicking within the paper. Conversely, photo papers 5 (which are often coated with a transparent ink-carrying layer) can show a small but consistent dot-gain due to shadows cast by the ink on the opaque paper surface. Such pre calculated models can be stored in the output checker memory 2104 and accessed according to the type of paper in the input tray 315. Returning to Fig. 8, the next step after the dot-gain model 810 is an MTF model 10 step 820. MTF can be a complex characteristic in a print/scan system, but may be simply approximated with a Gaussian filter operation. As with dot-gain, MTF varies slightly with drum age factor d, and idle time, t. However, the MTF of the print/scan process is generally dominated by the MTF of the capture process, which may not vary with device operating conditions. In one implementation, the MTF filter step is defined as a simple 15 filter, shown in Equation [8A] as follows: I, =l *G, [8A] where G, is a Gaussian kernel with standard deviation a, as is known in the art. In one implementation, a is chosen as a = 0.7 + 0.2d. It is also possible to apply this filter more efficiently using known separable Gaussian filtering methods. 20 Turning to a following colour model application step 830, it is noted that the desired colours of a document can be changed considerably by the process of printing and scanning. In order to detect only the significant differences between two images, it is useful to attempt to match their colours using the colour model step 830. The colour model process assumes that the colour of the original image 160 changes in a way which 25 can be approximated using a simple model. In one APV arrangement, it is assumed that 5404130_1 991865_speci_lodge - 29 the colour undergoes an affine transformation. However, other suitable models can be used, e.g., a gamma correction model, or an nth order polynomial model. If the colour undergoes an affine transformation, in the case of a CMYK source image captured as RGB, it is transformed according to the Equation [9]: [C.l 1crg Rpred A A 1 2

A

1 3 A14 Moi FC1~ M [9] 5 Gpr=d A21 A22 A23 A 24 Y"- + C2 =jA 7""* +C [9] B A32 A33 A34. or C3 K L orig _ Corig _ where (Rprd ,GpredBpred) are the predicted RGB values of the original image 160 after printing in the step 130 and scanning in the step 140 according to this predefined model, (Cg,Mrg,Yrg,K,,g) are the CMYK values of the original image 160, and A and C are the affine transformation parameters. 10 Similarly, an RGB source image captured in RGB undergoes a simpler transformation as shown in Equation [10]: R,,, B Bo B1 R D R, Rpred 811 B 1 2 B i R rii1 Gpred = B 21

B

22 B23 Gong + D 2 =B G +D [10] Bpred LB 3 1

B

32

B

33 B or _D 3 _ Bong J In one implementation example, the parameters A, B and C, D are chosen from a list of pre-determined options. The choice may be made based on the operating conditions 15 of paper type of the page 319, toner/ink types installed in the reservoirs 307-310, and the time which has elapsed since the printer last performed a self-calibration. The parameters A, B and C, D may be pre-determined for the given paper and toner combinations using known colour calibration methods. Once the colour model step 830 has been processed, the model step 225a is 20 complete and the resulting image is the expected image band 227. 5404130_1 991865_specijlodge -30 Returning to Fig. 2, the scan band 235 and the expected image band 227 are then processed by a band alignment step 240 that is performed by the processor 2106 as directed by the APV ASIC 2107 and/or the APV software application program 2103. The step 240 performs image alignment of the scan band 235 and the expected image band 227 5 using the list of alignment regions (i.e. alignment hints) 162. As the model step 225 did not change the coordinate system of the original image band 226, spatially aligning the coordinates of the scan band 235 to the expected band 227 is equivalent to aligning the coordinates to the original image band 226. The purpose of this step 240 is to establish pixel-to-pixel correspondence between 10 the scan band 235 and the expected image band 227 prior to a comparison process in a step 270. It is noted that in order to perform real-time print defect detection, a fast and accurate image alignment method is desirable. A block based correlation technique, where correlation is performed for every block in a regular grid, is inefficient. Furthermore, the block based correlation does not take into account whether or not a block contains image 15 structure that is intrinsically alignable. Inclusion of unreliable correlation results can affect the overall image alignment accuracy. The present APV arrangement example overcomes the above disadvantages of the block based correlation by employing a patch-based sparse image alignment technique that accurately estimates a geometrical transformation between the images using alignable regions. The alignment process 240 will be described in greater 20 detail with reference to Fig. 4 below. In a following step 250, a test is performed by the processor 2106 as directed by the APV ASIC 2107 and/or the APV software application program 2103 to determine if any geometric errors indicating a misalignment condition (e.g. excessive shift, skew, etc.) were detected in the step 240 (the details of this test are described below with reference to 25 Fig. 4). If the result of this test is Yes, processing moves to a defect map output step 295 54041301 991865_specijlodge -31 thereby outputting an indication of detected defects. Otherwise processing continues at a band content comparison step 270. As a result of processing in the step 240, the two image bands are accurately aligned with pixel-to-pixel correspondence. The aligned image bands are further processed 5 by the step 270, performed by the processor 2106 as directed by the APV ASIC 2107 and/or the APV software application program 2103, which compares the contents of the scan band 235 and the expected image band 227 to locate and identify print defects. The step 270 will be described in greater detail with reference to Fig. 5 below. Following the step 270, a check is made at a decision step 280 to determine if any 10 print defects were detected in the step 270. If the result of step 280 is No, processing continues at a step 290. Otherwise processing continues at the step 295. Step 290 determines if there are any new scanlines from the scanner 2108 from the step 140 to be processed. If the result of the step 290 is Yes, processing continues at (returns to) step 210 where the existing band in the buffer is rolled. That is, the top 64 scanlines are removed 15 and the rest of the scanlines in the buffer are moved up by 64 lines, with the final 64 lines replaced by the newly acquired scanlines from the step 140. If the result of the step 290 is No, processing continues at the step 295, where the defect map 165 is updated. The step 295 concludes the detect defects step 150, and control returns to the step 170 in Fig. 1. Returning to Fig. 1, a decision is then made in the decision step 170 as to the 20 acceptability of the output print 163. The defect map 165 may for example be displayed on the video display 2014 for checking by an operator to manually make a decision (step 170) regarding the appropriateness of the printed page 163. When evaluating a colour printer, such as the CMYK printer 300, it is desirable to also measure the alignment of different colour channels. For example, the C (cyan) 25 channel of an output print 163 printed by the cyan image forming unit 302 may be several 5404130_1 991865_specilodge -32 pixels offset from other channels produced by units 303-305 due to mechanical inaccuracy in the printer. This misregistration leads to noticeable visual defects in the output print 163, namely visible lines of white between objects of different colour, or colour fringing that should not be present. Detecting such errors is an important property of a 5 print defect detection system. Colour registration errors can be detected by comparing the relative spatial transformations between the colour channels of the scan band 235 and those of the expected image band 227. This is achieved by first converting the input bands from the RGB colour space to CMYK. The alignment process of the step 240 is then performed 10 between each of the C, M, Y and K channels of the scan band 235 and those of the expected image band 227 in order to produce an affine transformation for each of the C, M, Y and K channels. Each transformation shows the misregistration of the corresponding colour channel relative to the other colour channels. These transformations may be supplied to a field engineer to allow physical correction of the misregistration problems, or 15 alternately, they may be input to the printer for use in a correction circuit that digitally corrects for the printer colour channel misregistration. The sparse image alignment technique employed in the present APV arrangement uses a small number of selected image patches to determine local shift measurements between a pair of input images. In this approach, it is assumed that these local shift 20 measurements are consistent with a global motion model that maps one input image to the other. That is, by measuring these local shifts, it is possible to model the image transformation between the original image 166 and the scan image 164. Furthermore, for real-time processing a band-based arrangement is used to perform the alignment process 240. So instead of aligning the whole original image 166 to the whole scan 5404130_1 991865.speci_lodge - 33 image 164, the alignment process 240 is performed using image bands extracted from the original image 166 and the scan image 164. Fig. 11 shows the detail of two bands which, according to one example, can be input to the alignment step 240 of Fig. 2. In particular, Fig. 11 illustrates examples of the 5 expected image band 227 and the scan image band 235. Relative positions of an example alignable region 1110 in the expected image band 227 and its corresponding region 1120 in the scan image band 235 are shown. Shift estimation is then performed, by the processor 2106 as directed by the APV ASIC 2107 and/or the APV arrangement software program 2103, on the two regions 1110 and 1120 to determine the translation that best 10 relates the two regions 1110 and 1120. A next pair of regions, shown as 1130 and 1140 in Fig. 11, are then selected from the expected image band 227 and the scan image band 235. The region 1130 is another alignable region and the region 1140 is the corresponding region as determined by the transformation between the two images. Shift estimation is then repeated between this new pair of regions 1130 and 1140. These steps are repeated 15 until all the alignable regions 162 within the expected image band 227 have been processed. Fig. 4 depicts the alignment process 240 in greater detail, depicting a flow diagram of the steps for performing the image alignment step 240 in Fig. 2. The step 240 operates on two image bands, those being the scan image band 235 and the expected image 20 band 227, and makes use of the alignment hint data 162 derived in the step 120. In an initial step 410, the list 162 of alignment hints is iterated by checking each hint progressively to identify suitable alignment hints for further processing. If it is determined in a following decision step 420 that the current alignment hint 415 is within the current band, then processing moves step 430. Otherwise processing continues to a transformation 25 deviation step 470. A current alignment hint 415 identified in step 410 is described by a 5404130_1 991865_Speci_lodge -34 data structure comprising four data fields for storing the x and y coordinates of the hint location in the coordinate space of the original image bitmap 160, the corner strength of the hint, and the patch size of the hint. In step 430, an expected shift parameter is obtained. This expected shift 5 parameter is an estimate of the shift between a current best guess location of the current alignment hint 415 in the scan image band 235 and an actual hint location in the scan image band 235. The current best guess location is determined using a transformation derived from a previous alignment operation on a previous image band to transform the x and y coordinates of the current alignment hint 415 in the original image bitmap's 10 coordinate space to its corresponding location (x and y coordinates) in the scan image band 235. The expected shift parameter is used for selecting a suitable patch size, and may also be used for selecting a suitable shift estimation method. In the present APV arrangement, an alignment hint as discussed above includes data which identifies the centre of a corresponding a square image alignment patch as 15 shown in Fig. 11. Each hint corresponds uniquely to a patch area in the original image, centring at (x,y). Accordingly, an important parameter is the patch size Sp (or alignment hint size), which is defined as the width or height of the square image patch. The patch size S, is a key factor in determining the overall alignment speed, efficiency and accuracy. In general, the patch size S, should be at least twice the expected shift value in order to 20 provide sufficient overlapping region during correlation. Figs. 23A to 23C provides an illustration of the basic relationship between the size of a patch and its shift relative to a corresponding location in a region common to each of the candidate scan image 164 and the original reference image 160. The region may be an image band. In Fig. 23A, an alignment hint 2315 is represented by an image patch 2310 in the expected image 25 band 227. The corresponding hint location 2325 in the scan image band 235 is represented 5404130_1 991865_speci_lodge -35 by an image patch 2320. Fig. 23B shows if the shift 2340 is small relative to the patch size 2345, there is a significant amount of overlapping region 2330. It also shows that if the shift 2360 is large relative to the patch size 2345, there is a small overlapping region 2350 which may or may not contain sufficient image structure for shift estimation. 5 Thus, in order to perform shift estimation, the patch size 2370 must be increased to at least twice the shift 2360 as shown in Fig. 23C. Since insufficient image structure within the overlapping region will cause shift estimation to fail, the patch size is generally chosen to be larger than twice the expected shift value. However, choosing an unnecessarily large patch size can lead to slow processing speed. It may also cause shift estimation inaccuracy 10 if significant amount of rotation is present because the basic assumption of shift estimation is that the input patches are related by a pure translation. While most shift estimation methods can tolerate a small degree of rotation, for example less than one degree, shift estimation accuracy drops significantly with increasing angle thereafter. The effect of rotation on shift estimation accuracy is reduced for smaller patches. For this reason, it is 15 advantageous to use as small a patch as possible for shift estimation. In one APV arrangement, the patch size 2345 of the alignment hint location 2315 and its corresponding hint location 2325 in the scan image band 235 is adaptively adjusted according to the current expected shift value, which is derived using feedback from previous shift estimation operations. A small patch size is selected for a small expected 20 shift and a large patch size for a large expected shift. This can significantly improve alignment accuracy and efficiency for aligning rotated images. Shift estimation between a patch 1110 in the expected image band 227 and a patch 1120 in the scan image band 235 is performed under the assumption that the two patches are related by a pure translation. For print verification, this assumption is not strictly true because the scan image band 2112 is 25 obtained using the image capture system 2108 by scanning a portion of the printed 5404130_1 991865_speci_lodge - 36 paper 319 whilst in motion. The relative movement between the image capture system 2108 and the paper feeding mechanism is expected to result in a slight rotation between the expected image band 227 and the scan band 235. Shift estimation is very sensitive to rotation - as the amount of rotation increases, the accuracy of shift estimation 5 deteriorates rapidly until it becomes a complete failure. As the band alignment process 240 progresses down the scan image 164, the expected shift value between a pair of corresponding patches is also expected to reduce. By adopting the variable patch size technique according the expected shift value, the effect of rotation upon shift estimation can be reduced, thus producing more accurate shift estimates. Furthermore, small patches 10 are more efficient to compute, thus improving the overall alignment speed. This variable patch size alignment hint approach is illustrated in Fig. 14. This illustration serves to provide a high level overview of how patch size can change with expected shift magnitude. A scan image 1410 is captured one band at a time using the image capture system 2108. Adjacent bands may contain a partially overlapping 15 region 1425 for improved alignment performance and stability between consecutive bands 1420 and 1430. In a first band 1420, an expected shift value 1450 between a first patch 1452 in an expected image band 227 and its corresponding first patch 1454 in the scan band 1420 is large due to lack of prior alignment information, so a large patch size 1455 is selected. After the first band alignment operation as performed in step 240, a 20 current best guess image transformation that relates the original image 160 and the scan image 164 is derived. This image transformation represents the current best estimate of the global motion model between the two images. In a second scan band 1430, the current best guess image transformation is used to determine corresponding hint locations in the current scan band. Because of the improvement in the image transformation estimate after 25 the first band, the shift measurements in the second band are smaller than those in the first 5404130_1 991865_specijlodge -37 band, which in turn produces a smaller and therefore more refined expected shift value 1460, thus a smaller patch size 1465 is selected for the corresponding patches 1462 and 1464 in the second band 1430. Similarly, in a third band 1440, an expected shift value 1470 is further reduced and a patch size 1475 is chosen for shift estimation between 5 the corresponding patches 1472 and 1474. This illustration shows an incremental band based image alignment technique, where the size of image patches for local shift estimation can be adapted according to the accuracy of the current best guess image transformation. In general, the patch size decreases as the alignment accuracy improves from band to band. 10 In an alternative implementation of APV, the size of image patches for shift estimation can be adapted within a single image band according to prior shift estimation results within the same band. For example, at a given alignment hint location within an image band, any previous shift estimation results in the same neighbourhood as the hint within the same image band may be used to calculate the patch size of the current hint, 15 such that large measured shifts would result in a large patch size, and small measured shifts would results in a small patch size. Thus, whilst the example of Fig. 14 illustrates determining shift estimation for an image divided into a number of bands with alignment patches in each band, shift estimation with multiple alignment patches can be performed within a single band, for example across an entire image, where the whole image is 20 considered one band or region. The variable patch size alignment hint approach is implemented in step 435. For each of the current alignment hint 415, a patch size S, is calculated using the expected shift value 431. In one APV arrangement, the patch size is calculated using the following mathematical Equation [10A]: 25 S, = max(aE,Sm,n) [10A] 5404130_1 991865_specijodge -38 where a is a scaling factor, E is the expected shift value, and Spmin is the minimum patch size. In one implementation a is set to 2.5 and Sp,,in is set to 8 pixels. In step 440, a SxS, patch 441 centred at the current alignment hint location 415 is extracted from the expected image band 227. At the same time, a SpxS, patch 442 centred 5 at the corresponding hint location in the scan image band 235 is extracted. The corresponding hint location is determined using a current best guess transformation derived from a previous alignment operation on a previous document image or band to transform the x and y coordinates of the current alignment hint 415 to its corresponding location (x and y coordinates) in the scan image band 235. 10 In step 445, a shift estimation operation is performed between the alignment hint patch 441 and the corresponding hint patch 442 in the scan image band 235. In one APV arrangement, phase only correlation (hereinafter known as phase correlation) is employed to performed the shift estimation operation. Alternatively, other shift estimation methods such as gradient-based shift estimation may be used. The shift estimation operation in 15 step 445 is further described with reference to Fig. 22. Step 445 begins by applying a window function such as a Hanning window to each of the two patches 441 and 442, and the two windowed patches are then phase correlated in step 2210. The phase correlation of step 2210 produces a raster array 2230 of real values. In a following peak detection step 2220, the location of a highest peak is determined within the raster array 2230, with 20 the location being relative to the centre of the alignment hint patch. A confidence factor for the peak is also determined, defined as the height of the detected peak relative to the height of the second peak, at some suitable minimum distance from the first, in the correlation result. In one implementation, the minimum distance chosen is a radius of 5 pixels. The shift estimation step 445 outputs a displacement vector 446 formed of the 25 highest peak location and the confidence factor. 5404130_1 991865_speci_lodge - 39 After step 445, processing moves to step 447, where the current expected shift value 431 is updated. Due to local distortion or non-translational transformations (e.g. rotation and scaling, etc.) the expected shift may not be uniform across the scan image band 235 (i.e. in the direction of scanning). That is, the expected shift at one end of the 5 scan image band 235 may not be the same as the expected shift at the other end. Thus, it is necessary to determine one expected shift value for each neighbourhood. In one APV arrangement as shown in Fig. 18, the scan image band 235 is partitioned into five equal size partitions or, in this case columns (1820, 1830, 1840, 1850 and 1860), where each column has an expected shift value. Each expected shift value is calculated using 10 information within its neighbourhood. For example, the expected shift for the neighbourhood 1860 is derived using shift measurements from alignment hints 1870. In particular, each expected shift value is derived using the following mathematical Equation [10B]: E,(i) = (1-co)abs(si) + oEi(i) [1OB] 15 where E,(i) is the current expected shift value in the ith column of the scan image band 235, wois a decay rate for controlling the sensitivity of E,(i) to recent shift measurements, s, is the latest shift measurement within the ith column, and E,.i(i) is the previous expected shift value for the ith column. Besides Equation [10B], the current expected shift value E,(i) can be constrained 20 using the spatial location of the current alignment hint if the maximum transformation parameters are known. For example, if the transformation between the original image 160 and the scan image 164 is predominately made up of rotation, scaling and translation (RST), it is possible to estimate the maximum shift at any given location if the maximum rotation, scaling and translation parameters are known. Fig. 15 provides an illustration 25 where the maximum expected rotation 0 is used to determine the maximum shift at 5404130_1 991865_speci lodge - 40 different locations. It shows an original image 1510 that has undergone a rotation of Degrees, and becomes a transformed image 1520. This results in a translation of an alignment hint 1530 in the original image 1510 to its transformed location 1540 in the transformed image 1520. Given the known distance (rt) from the origin of the original 5 image 1510 to the alignment hint location 1530, and the maximum rotation angle 0, a shift t; 1570 representing the maximum possible shift caused by rotation can be determined by Equation [10C]: t, = 2rysin(0/2). [10C] Similarly, an alignment hint 1550 is transformed to a location 1560 in the 10 transformed image 1520, the maximum shift t 2 is thus a function of the maximum rotation angle 0and the distance r 2 . The current expected shift value is constrained according to the following Equation [IOD]: E,(i) = min(E,(i),Ema.(i)) [IOD] where Ema) is the maximum expected shift in the ith column as determined by combining 15 the effects of the maximum transformation parameters such as rotation, scaling and translation. Since the alignment hint patch size is a function of the expected shift value E,(i), and the expected shift value E,(i) may be derived using the maximum transformation parameters (RST), the alignment hint patch size is therefore a function of the maximum transformation parameters (RST). If the transformation is predominately rotation in 20 nature, then the alignment hint patch size may further be a function of the spatial distance of the alignment hint patch from the page centre. After updating the current expected shift value, the location of the peak, the confidence, and the centre of the alignment hint patch are then stored in a system memory location 2104 in a vector displacement storage step 450. If it is determined in a following 25 decision step 460 that more alignment hints exist, then processing moves back to the 5404130_1 991865_speci_lodge -41 step 410, where a next alignment hint 415 is selected. Otherwise processing continues to a transformation derivation step 470. In an alternative APV arrangement, binary correlation may be used in place of phase correlation. 5 The output of the phase correlations is a set of displacement vectors D(n) that represents the transformation that is required to map the pixels of the expected image band 227 to the scan image band 235. Processing in the step 470 determines a transformation from the displacement vectors. In one APV arrangement example, the transformation is an affine transformation 10 with a set of linear transform parameters (b ,b , 2 , , b 22 ,Ax,Ay), that best relates the displacement vectors in the Cartesian coordinate system as follows in Equation [11]: ( ,=bl byl )xn )+AX [11] where (x,,y,) are alignable region centres and (X,,,Y) are affine transformed points. In addition, the points (x.,y.) are displaced by the displacement vectors D(n) to 15 give the displaced points (ij) as follows in Equation [12]: (.,j.) =(x,,,y.)+D(n) [12] The best fitting affine transformation is determined by minimising the error between the displaced coordinates, (4,,), and the affine transformed points(3 ,y,) by changing the affine transform parameters (b 1 , b12, b2, b 2 2 ,Ax, Ay). The error functional 20 to be minimised is the Euclidean norm measure E as follows in Equation [13]: N E =E(j. -y")2 +(j, -yn)2 [13] n=1 The minimising solution is shown in Equations [14]: 5404130_1 991865_speci_lodge -42 b12 =M-14by" 'by', ' b.x b 22 =M' [ [14] With the following relationships of Equations [15]: 'SS" S,, S" Zxnxr? xI.y11 E x M= S,, S,, S, =[X yx. lynyn I YJ S, S, S , 2x n i SS+SS -SS +SS SS -SS Y~ y ))' yY XYY yy 5 M-' = -SS,+SXS, -SS,+SSX SS, -S.S, [15] S S -SS SS, -S.S --SS, +SxS And the relationships of Equation [16]: IMI= det M = -S S,,S, + 2S,S,,S, - SxSS, - SxSxS, + SSS, [16] where the sums are carried out over all displacement vectors with a peak confidence greater than a threshold Pmin. In one implementation, Pmin is 2.0. 10 Following the step 470, the set of linear transform parameters (b,, bb 1 , b 2 , Ax , Ay ) is examined in a geometric error detection step 480 to identify geometric errors such as rotation, scaling, shearing and translation. The set of linear transform parameters (b, , b, 2 b , b 22 , Ax , Ay ) when considered without the translation is a 2x2 matrix as in Equation [17]: 15 A= b,, b 2 1 [171 b 2 b 2 which can be decomposed into individual transformations assuming a particular order of transformations as shown in Equation [18]: 5404130_1 991865_specilodge -43 Lb b 2 _ [S 0 1 00 1 o cos 0 - sin o ~ [18] b b2 0 s, h, 1 sin0 cos _ where scaling is defined as follows [19]: [s, 0 [19] 0 sl where s, and s, specify the scale factor along the x-axis and y-axis, respectively. 5 Shearing is defined as follows [20]: 1 0 or 1 hx] [20] where h, and h, specify the shear factor along the x-axis and y-axis, respectively. Rotation is defined as follows [21]: [cosO -sin[2 sin0 cos9 [ 10 where 0 specifies the angle of rotation. The parameters s,, s,, h,, and 6 can be computed from the above matrix coefficients by the following Equations [22 - 25]: s,= bi+b2 [22] det(A) [23] 15 h - lbub2 +b 2 b 2 2 [24] det(A) tan=- b2- [25] bi In one APV arrangement example, the maximum allowable horizontal or vertical displacement magnitude A. is 4 pixels for images at 300dpi, and the acceptable scale 54041301 991865speci_lodge - 44 factor range (s,,,,s,,) is (0.98, 1.02), the maximum allowable shear factor magnitude h,. is 0.01, and the maximum allowable angle of rotation is 0.1 degree. However, it will be apparent to those skilled in the art that suitable alternative parameters may be used without departing from the scope and spirit of the APV 5 arrangements, such as allowing for greater translation or rotation. If the derived transformation obtained in the step 470 satisfies the above affine transformation criteria, then the scan band 235 is deemed to be free of geometric errors in a following decision step 490, and processing continues at an expected image to scan space mapping step 492. Otherwise processing moves to an end step494 where the step 240 10 terminates and the process 150 in Fig. 2 proceeds to the step 250 in Fig. 2. In the step 492, the set of registration parameters is used to map the expected image band 227 to the scan image space. In particular, the RGB value at coordinate (x., y,) in the transformed image band is the same as the RGB value at coordinate (x, y) in the expected image band 227, where coordinate (x, y) is determined by an inverse of the 15 linear transformation represented by the registration parameters in Equation [26]: (X) _ I b'2 -b '2 (X, - AX) .

[26] y bb 22 -b 2 b 21 -b2 bj )y., - y For coordinates (x, y) that do not correspond to pixel positions, an interpolation scheme (bi-linear interpolation in one arrangement) is used to calculate the RGB value for that position from neighbouring values. Following the step 492, processing terminates at 20 the step 494, and the process 150 in Fig. 2 proceeds to the step 250. In an alternative APV arrangement, in the step 492 the set of registration parameters is used to map the scan image band 235 to the original image coordinate space. As a result of the mapping in the step 492, the expected image band 227 and the scan image band 235 are aligned. 5404130_1 991865_speci_lodge -45 In an alternative APV arrangement, the band alignment step 240 employs a dynamic patch selection approach in which the number of patches selected for shift estimation is adapted dynamically according to local distortion level. For example, a neighbourhood with a low level of distortion is processed using a small number of patches; 5 and one with a high level of distortion is modelled using a large number of patches. This approach has the advantage of optimally processing the least number of patches for establishing image spatial correspondence, thereby improving both the alignment speed and efficiency. This dynamic patch selection approach is graphically illustrated in Figs. 16A to 16D. Fig. 16A shows an original image 1610 with a large number of pre 10 determined alignment hint patches 1615. If the level of distortion within the scan image 1620 is low, as seen in Fig. 16B, then both the patch size and count are reduced as the band alignment process moves from the top to the bottom of the scan image 1620. The variable patch size alignment hints 1625 are derived from those pre-determined hints 1615 in the original image 1610. However if the level of distortion in the scan image 1630 is 15 high, as seen in Fig. 16C, then a large number of alignment hints is selected for shift estimation in order to provide more sample point for modelling the underlying image transformation. This dynamic patch selection approach is most suitable for handling sudden increases in distortion level, which might be caused by non-linear paper feeding speed or drum distortion. The scan image 1640 as depicted in Fig. 16D has a sudden 20 increase in distortion in the middle part of the page, thus additional patches 1645 are selected for shift estimation in that region. Fig. 17 is a flow-chart showing the details of the dynamic patch selection method illustrated in Figs. 16A-16D. Processing begins in step 1710 where the original image 160 is processed to determine as many alignment hints as possible. These alignment hints are 25 then sorted into three groups in step 1720 according to their strength and separation such 5404130_1 991865_speci_lodge - 46 that the strongest and best distributed hints are grouped for low distortion processing, the next group of alignment hints for medium distortion processing, and the final group of alignment hints for high distortion processing. In step 1730, the alignment hints for the current distortion level are selected for shift estimation in step 1740. The level of 5 distortion surrounding each alignment hint patch is determined in step 1750. In a simple implementation, the distortion level is calculated by comparing the estimated shift in each case to a distortion threshold. If the distortion level is higher than the threshold in step 1760, processing moves to step 1770 where the current distortion level is increased. Otherwise processing terminates in step 1780. 10 The dynamic patch selection approach may be extended to a multi-scale image representation which provides an effective trade-off between alignment accuracy and processing time. Fig. 30 shows an image pyramid, which is a stack of low-pass filtered images of an original image at different scales, arranged from the lowest resolution to the highest. This arrangement allows a coarse-to-fine approach to image alignment in which 15 incremental improvement in accuracy is made from one level to the next. Detailed description on how to apply dynamic patch selection for general image alignment is provided below. Fig. 24 illustrates an application of the image alignment system using dynamic patch selection. After a process of printing a test image 2420 onto a physical medium, 20 followed by scanning the physical medium back to a digital image, the resulting image 2430 may be distorted. The image registration system 2440 is used to align the images 2420 and 2430, thereby removing any non-rigid motion distortions that relate the two images. Outputs of the system are the substantially aligned images 2450 or a full warp map that relates the two input images. By accepting a user-specified accuracy 54041301 991865_speci-lodge -47 requirement 2410, the image registration system 2440 tries to adapt its processing to minimize computations while still honouring the required registration accuracy. The process of registering the images 2420 and 2430 performed in step 2440 is now described in more detail with reference to Fig. 25. Step 2440 takes two input images and a 5 threshold 2410 based on the user's specified accuracy requirement. The two input images are denoted as the reference image Ii, 2420, and the target image 12, 2430. In step 2510, a coarse RST alignment process is performed between the reference image I, 2420 and the target image I2, 2430. Based on this coarse alignment, the target image I2, 2430 is warped towards the reference image I, 2420, and the warped target image is converted to a multi 10 resolution image representation in the form of an image pyramid in step 2515. In step 2520, the reference image I, 2420, is analysed at different image resolutions to determine and select image regions with image structures that are likely to produce good alignment results. The selected image regions will be referred to as alignable patches henceforth. Detailed description of the alignable patch generation 15 process will be provided with reference to Fig. 26 below. As a result of processing in step 2520, the reference image I,, 2420, is converted to a multi-resolution image representation in the form of an image pyramid, where each image resolution layer contains a list of alignable patches. In a preferred implementation, an alignable patch is a data structure with a number of data fields for storing the patch location in a Cartesian 20 coordinate system, the size of the patch, an associated patch separation, and a strength parameter for indicating the amount of image structure within the patch area. In step 2530, patches within each image resolution layer are categorised into three groups, namely high density patches, medium density patches and low density patches. Since high density patches will have the shortest patch separation, most patches will be labelled as high 25 density patches, a smaller number of patches will be labelled as medium density patches, a 5404130_1 991865_speci-lodge -48 least number of patches will be labelled as low density patches. The process of categorising the patches are based on their strength and three minimum patch separation distance values. At each image resolution, a minimum patch separation distance is assigned to each patch group such that the minimum separation distance for the high 5 density patches is smaller than the minimum separation distance for the medium density patches; the minimum separation distance for the medium density patches is smaller than the minimum separation distance for the low density patches. If drh, drm, dri are the separation distances for the high, medium and low density patches at resolution r, respectively, the following condition applies: drh< drm< dri. 10 In a preferred implementation, patches within the lowest resolution layer are exempted from the above categorisation process. Thus, only patches from higher resolution layers will have a spatial density label. At the beginning of step 2535, two image pyramids, 2532 and 2517, derived from the reference image 1,, 2420, and the target image I2, 2430 respectively, are now available. 15 In particular, image pyramid 2532 has a number of alignable patches for each resolution layer at various spatial densities as described above. In step 2535, processing begins from the lowest resolution layer, r,, in the reference image pyramid 2532. For each alignable patch in that layer, a corresponding patch can be extracted from the target image pyramid 2517. The corresponding target patch is determined using a warp map which 20 defines the transformation between the two image pyramids. The warp map desirably consists of two components, a linear part that is represented by an affine transformation and a non-linear part which is represented by a sparse vector field. Two sets of image patches are passed onto step 2540 as a result of processing in step 2535. In step 2540, patch correlation is performed for each of the extracted patch pair to determine a shift 5404130_1 991865_speci-lodge - 49 estimate relating the two patches. Preferably, this patch correlation is performed using phase correlation. Alternatively, any other shift estimation algorithm may be applied. The output of step 2540 is a displacement map, consisting of a sparse set of displacement vectors, representing local movements between corresponding image regions 5 in the reference and target images. In step 2550, the warp map modelling the transformation between the reference and target images is updated. As mentioned above, the warp map has two components, namely the affine transformation and the sparse vector field. The linear part can be determined by applying a standard least squares minimisation algorithm such as the Levenberg-Marquardt algorithm to the displacement map to 10 determine the values of the affine transformation parameters. The sparse vector field is determined by subtracting the effect of the affine transformation from the displacement map, so that the sparse vector field represents the residual distortions that remain. In step 2560, a determination is made to check whether the desirable alignment accuracy as specified by the user has been reached. If the result is Yes, processing 15 terminates at step 2599. If the result is No, processing moves to 2570. A test is performed at step 2570 to determine if a higher resolution pyramid layer is available for processing. If the result is No, processing terminates at step 2599. If the result is Yes, processing moves to the next higher resolution pyramid layer in step 2580. In step 2590, a multi resolution approach is used to determine local distortion levels within the current image 20 pyramid layer in the target image pyramid 2517. The process in step 2590 will be described with reference to Fig. 28 below. In step 2595, the results from the local distortion measurements in step 2590 are used to adjust local patch densities in the reference image pyramid 2532, so that a region with high distortion will be allocated with all the patches in the region that have been labelled high density patches. Similarly, a 25 region with a medium level of distortion will be given all the medium density patches 5404130_1 991865_specilodge -50 within that region. If a region has minimal distortion, then only low density patches within the region will be selected for processing. This approach allows the number of patches within any neighbourhood to be adapted according to its local distortion level, so that more patches are selected for highly distorted regions while fewer patches are chosen for low 5 distortion regions. Details of step 2595 will be described further with reference to Figs. 29A and 29B below. After selecting a set of image patch locations, at the current resolution layer in the reference image pyramid 2532, based on local distortion levels in the corresponding resolution layer in the target image pyramid in step 2595, these image patch locations are 10 used to extract a set of reference image patches at the current resolution layer in step 2535. Furthermore, these image patch locations are used together with the current warp map (as determined in step 2550) to determine a corresponding set of patch locations in the current resolution layer in the target image pyramid 2517. This allows a corresponding set of target patches at the current resolution layer to be extracted. As described above in 15 step 2540, these two set of patches will undergo patch correlation to generate a displacement map for the current resolution layer. Processing continues as described above in step 2550, in which the warp map is updated using the current displacement map. Processing in step 2440 continues in the above described manner until either the user specified accuracy level is reached or the highest resolution layer is processed. 20 The process of identifying patch candidates in step 2520 may be performed prior to actual alignment or during alignment. For real-time system, it is advantageous to perform this analysis prior to the actual alignment process. The alignable patch selection process in step 2520 is now described in greater detail with reference to Fig. 26. Processing in step 2520 begins by converting the reference 25 image 2440 to a multi-resolution image pyramid in step 2610. The image pyramid is 5404130_1 991865_specijlodge - 51 created by progressively resampling the reference image 2420 from one (higher) resolution to a lower resolution, thereby generating a stack of images derived from the reference image 2420 at various image resolutions. Desirably, the image pyramid is created by taking the reference image 2420 as the current resolution layer. The current resolution 5 layer is used to generate the next resolution layer by down-sampling with a separable Burt Adelson filter. The down-sampled layer then becomes the current resolution layer, which is downsampled again using the Burt-Adelson filter to create the next resolution layer. This process continues until the desired number of layers is reached, or the minimum image size is reached. 10 In step 2620, each image pyramid layer is analysed in tum to identify strong image patches. The image analysis performed in step 2620 to determine the image patch locations is preferably an assessment based on Harris corners. The process of detecting Harris corners is described as follows. The first step for detecting Harris corners is an assessment of the gradient or spatial derivatives of a grey 15 scale version of an image pyramid layer in both x and y directions, denoted as I., and I,. In practice, this can be approximated by applying the Sobel operator to the image pyramid layer. The Sobel operators use the following kernels: -1 0 1 Sj -2 0 2 -- 1 0 1 -1 -2 -1 S, = 0 0 0 1 2 1 20 Edge detection is performed with the following operations: I,=S, *IG I =S *IG 5404130_1 991865_speci-lodge -52 where . is the convolution operator, 'G is the greyscale image data, S ,,S, are the kernels defined above, and I., and I, are images containing the strength of the edge in the x and y direction respectively. From I, and I,, three images are produced as follows: I = I., o I., IX =I, 0 I, I,, = I, o I, 5 where o-is a pixel-wise multiplication. This allows a local structure matrix A to be calculated over a neighbourhood around each pixel, using the following relationship: =1 ~ A= w(x, y) * 2' where w(x, y) is a ow ing function for spatial averaging over the neighbourhood. In a 10 preferred implementation w(x, y) can be implemented as a Gaussian filter with a standard deviation of 10 pixels. The next step is to form a "cornerness" image by determining the minimum eigenvalue of the local structure matrix at each pixel location. The cornerness image is a 2D map of the likelihood that each pixel is a corner. A pixel is classified as a corner pixel if it is the local maximum (that is, has a higher cornerness value than its 8 15 neighbours). A list of all the corner points detected, Ccorners together with the strength (cornerness) at that point is created. The list of corner points, Ccor,,,' is filtered by deleting points with strength below the user specified threshold 2410. The list of corner points, Ccor,,rs is further filtered by deleting points which are within S pixels from another, 20 stronger, corner point. In a preferred implementation, S = 64 is used. The list of accepted corners, Cnew is output to step 2630 as a list of alignable patches for use in image alignment. Each entry in the list can be described by a data structure comprising three data 5404130_1 991865_speci_lodge - 53 fields for storing the x-coordinate of the centre of the region (corresponding to the location of the corner), the y-coordinate of the centre of the region, and the corner strength of the region. Alternatively, other suitable methods for determining feature points in the original 5 image 160 such as Gradient Structure Tensor or Scale-Invariant Feature Transform (SIFT) can also be used. Fig. 27 provides an illustration of an image pyramid with patches at various spatial densities. The depicted image pyramid has three layers, labelled as 2710, 2720 and 2730. In each layer, there are a number of image patch regions. The figure serves as an 10 illustration only. In a preferred implementation as described above image patch locations are stored in suitable data structures before their extraction in step 2535. In the lowest resolution layer 2710, there is only one type of patch, as indicated by the shading applied to patch 2740. In layers 2720 and 2730, there are three types of patches as described above, as depicted by three different shadings. The high density patches are shown as 15 square blocks with small dots as indicated by 2760 and the medium density patches are shown as square blocks with a simple stripe pattern as indicated by 2750. The distribution of the patches indicates that high density patches have the smallest separation distances between the high density patches, thus allowing more patches to be selected in a given neighbourhood than the other two types of patches. Similarly, there are more medium 20 density patches than low density patches in a given neighbourhood. The basic idea of dynamic patch selection is to pre-select many more patches than generally is needed for image alignment. The additional patches provide an option during alignment to dynamically adapt the number of patches for any neighbourhood based on its distortion level. For example, in a print defect detection system, accurate image alignment is 25 required for detecting small print defects such as missing strokes in printed characters or 5404130_1 991865_speci_lodge - 54 detecting breaks and voids. In this example, the reference image is the original source document that is sent to the printer, and the target image is a digital capture of the printed page. For a high quality printing system, the target image is expected to match the reference image well. However, in a low quality printing system, where the paper feed 5 speed may be non-uniform and the drum rotation speed may not be constant, the printed page, thus the target image, is likely to be distorted. The target image may suffer additional optical distortions during the capturing process. In this case, many displacement vectors will need to be measured in order to accurately model all the different motions within the target image. However, it is very difficult to predetermine a suitable number of 10 patches without any prior knowledge about the distortion rate. By applying dynamic patch selection during alignment, any given neighbourhood within the target image can be measured using an adaptive number of patches. If a neighbourhood is considered to be highly distorted, a high number of patches will be used for measuring all the local movements. If a neighbourhood is considered to be well behaved without much distortion, 15 a small number of patches will be used to measure the local movements. Fig. 28 illustrates the process of determining the level of local distortion as used in step 2590. The figure shows two image pyramids, namely the reference image pyramid and the target image pyramid (for simplicity only two layers are shown here). Starting with a reference image pyramid layer 2810, a patch 2830 is extracted. A corresponding 20 patch 2840 is also extracted in the target image pyramid layer 2820. A patch correlation operation is performed using these two patches to generate a displacement vector, representing the relative movement between the two patches. In the next reference image pyramid layer 2850, region 2870 is a projection of patch 2830. Similarly, region 2880 in the target image pyramid layer 2860 is a projection of patch 2840. Using the same patch 25 size as in 2830, region 2870 can be divided into four patches as shown in Fig. 28. The four 5404130_1 991865_speci_lodge - 55 pairs of patches in regions 2870 and 2880 are processed to generate four displacement vectors. An effective method to measure the distortion level in the immediate neighbourhood surrounding region 2880 is to compare the displacement vectors measured 5 using the four patch pairs in 2870 and 2880. For a region with low distortion, the displacement vectors should provide a high degree of correlation. Based on the degree of correlation between the four displacement vectors, a level of distortion is assigned to a neighbourhood of a predefined size immediately surrounding region 2880. Figs. 29A and 29B illustrate step 2595, in which image patches are selected based 10 on local image distortion levels. Fig. 29A shows two image pyramids, namely the reference image pyramid and the target image pyramid (for simplicity only two layers are shown here). The distortion level for region 2970 in the target pyramid layer 2960 is determined as described above. Based on this distortion level, a neighbourhood, 2990, of predetermined size surrounding the projected region of 2930 may be identified. If the 15 distortion level is medium, then all the medium density patch locations, 2992, within the neighbourhood 2990 are selected for patch extraction in step 2535. If the distortion level is high, as seen in Fig. 29B then all the high density patch locations, 2996 within a neighbourhood 2994corresponding to the neighbourhood 2990 are selected for patch extraction in step 2535. If the distortion in region 2970 is low, then only low density 20 patches within 2990 are selected for patch extraction in step 2535. Fig. 5 is a flow-chart showing the details of the step 270 of Fig. 2. In particular, Fig. 5 depicts the comparison process 270 in more detail, showing a schematic flow diagram of the steps for performing the image comparison. The step 270 operates on two image bands, those being the scan band 235, and the aligned expected image band 502, the 5404130_1 991865_specilodge - 56 latter of which results from processing to be described with reference to in the step 3400 in Fig. 34. Processing in the step 270 operates in a tile raster order, in which tiles are made available for processing from top-to-bottom and left-to-right in the band, one at a time. As 5 seen in Fig. 5, comparing the band contents 270 begins at step 510, where a Q x Q pixel tile is selected from each of the two bands 502, 235, with the tiles having corresponding positions in the respective bands. The two tiles, namely an aligned expected image tile 514 and a scan tile 516 are then processed by a following step 520. In one APV arrangement example, Q is 32 pixels. 10 Step 520 operates to perform a comparison between the tiles 514 and 516, which is conducted by the processor 2106 as directed by the APV ASIC 2107 and/or the APV software application 2103, and which operates to examine a printed region to identify print defects. The step 520 is described in greater detail with reference to Fig. 6. Fig. 6 is a flow-chart showing the details of a preferred implementation 520a of 15 step 520 of Fig. 5. In a first step 610, a scan pixel to be checked is chosen from the scan tile 516. In a next step 620, the minimum difference in a neighbourhood of the chosen scan pixel is determined. To determine the colour difference for a single pixel, a colour difference metric is used. In one implementation, the colour difference metric used is a Euclidean distance in RGB space, which for two pixels p and q is expressed as follows in 20 Equation [27]: D,B(p, q - Spq, Y2 +p, -q, ( pb -q b2 [ 27 ] where P, Pg IPb are the red, green, and blue components of pixel p, and likewise components q,,qg,qb for pixel q. In an alternate implementation, the distance metric used is a Delta E metric, as is known in the art as follows in Equation [28]: 5404130_1 991865_speci_lodge - 57 D&E(p,q) = (IpL* - qL*)2 + (pa. - + (Pb - qb [28] Delta E distance is defined using the L*a*b* colour space, which has a known conversion from the sRGB colour space. For simplicity, it is possible to make the approximation that the RGB values provided by most capture devices (such as the 5 scanner 2108) are sRGB values. The minimum distance between a scan pixel p, at location xy and nearby pixels in the aligned expected image pe, is determined using the chosen metric D according to Equation [29]: Dmi. (p., [x, y= mi (D(p, [x, y] pe [x', y'D) [29] 10 where KB is roughly half the neighbourhood size. In one implementation, KB is chosen as 1 pixel, giving a 3x3 neighbourhood. In a next step 630, a tile defect map is updated at location xy based on the calculated value of Dmin. A pixel is determined to be defective if the Dmin value of the pixel is greater than a certain threshold, Ddefct. In one implementation using D. to 15 calculate Dmin, Ddeect is set as 10. In the next decision step 640, if there are any more pixels left to process in the scan tile, the process returns to the step 610. If no pixels are left to process, the method 520a is completed at the final step 650 and control returns to a step 530 in Fig. 5. Following processing in the step 520, the tile-based defect map created in the 20 comparison step 630 is stored in the defect map 165 in the band defect map updating step 530. In a following decision step 540, a check is made to determine if any print defects existed when updating the band defect map in the step 530. It is noted that the step 530 stores defect location information in a 2-dimensional map, and this allows the user to see where defects occurred in the output print 163. The decision step 540, if it 5404130_1 991865_speci_lodge - 58 returns a "YES" decision, breaks out of the loop once a defect has been detected, and control passes to a termination step 560 as no further processing is necessary. If the result of step 540 is No, processing continues at a following step 550. Otherwise processing continues at the step 560. Step 550 determines if there are any remaining tiles to be 5 processed. If the result of the step 550 is Yes, processing continues at the step 510 by selecting a next set of tiles. If the result of 550 is No, processing terminates at the step 560. Aligning Patches Another way to calculate a shift estimate quickly is to extract some representative 10 part of the image and perform the alignment on those representative features. The selection of the representative feature presents some difficulty. Two straightforward methods in the art are to extract a small sub-image and align on that, or to sub-sample (downsample) the image and align on a smaller image. These methods lose accuracy because information useful for alignment is discarded. Additionally, in the downsampling 15 case, aliasing may further reduce the accuracy. One way of avoiding these problems is to use some measure of the amount of alignable information, for example Harris corner information, and to select patches of the image with a lot of information. However, extracting patches from an image may introduce edge effects, where the edges of the tiles are produce much more prominent features to align on than the information in the image 20 itself. Fig. 33 is a diagram of the frequency domain representation of an image, which may be a sub-image 3510 of another image 3500 seen in Fig. 35. In this representation, the DC frequency, 3310, is at the centre of the image. The frequencies near the axis of the transform, 3340, are where the edge effects mentioned in the background are most 25 prevalent. If the image is downsampled (for example, in step 3120 or in acquiring the 5404130_1 991865_specijlodge - 59 image) then aliasing effects will be strongest in the higher frequencies, 3330. Therefore, the best frequencies, 3320, to align on are those less affected by these effects. Fig. 34 shows a top level view of the alignment process 3400 mentioned above and which provides the aligned expected image band 502 used as an input in step 270 seen 5 in Fig. 5. In many applications, it is desired to determine the shift between an image 3410 and a reference image 3420. Generally, a patch suitable for alignment is first selected, this occurring in steps 3430 from both the image 3410 and the reference image 3420. This can be done using any number of suitable methods, for example, the patches can be selected using Harris corners as discussed above. In cases where the image 3410 and reference 10 image 3420 are quite small, for example, video frames, no patch selection may be required. An alignment feature is extracted in the next step 3440, from each extracted patch. Once these alignment features have been extracted, they can be aligned quickly in an alignment step 3450, due to their small dimension. The method of extracting the features is described below. 15 Fig. 31 is a schematic block diagram of an alignment feature extraction process 3100 based on a minimalist implementation. Step 3110 modulates the image with a complex exponential, and step 3120 downsamples the image. Downsampling an image in the spatial domain reduces the dimensions of the image to a point where it is suitably fast to downsample. This is not the only effect downsampling has on an image. If the 20 image is considered in the frequency domain, a downsampling in the spatial domain is equivalent to cropping out the high frequencies - for example, downsampling an image by 2 times has the effect in the frequency domain of removing all frequencies above 0,/2 (with the exception of those that are aliased into the downsampled image), where 6, is the Nyquist frequency, the highest unambiguously representable frequency in the original 5404130_1 991865_specijlodge -60 image. However, downsampling an image maintains the DC component, 3310, of the image and the axes, which are typically heavily affected by edge effects, 3340. When an image is modulated in the spatial domain by a complex sinusoid of the form e 'xe" , it is equivalent to shifting the signal by 00 in the frequency domain. In one 5 implementation, selecting: FON." ONx _i2 . where ONx and Ny are the horizontal and vertical Nyquist frequencies, translates the origin, 3310, into the middle of the desired region of frequency space, 3320. This results in the situation seen in Fig. 36 where 3610 is the original DC component of the image 10 and 3620 indicates the original Nyquist frequency of the image. Any information to the right of or below the lines 3620 is aliased information from the easier original image that was translated outside the new Nyquist frequency. The new origin of the image coincides with the centre of the desired frequency range, 3320. This image is now complex. This image can then be downsampled by a factor which to excludes the 15 undesirable regions 3330 and 3340 from the image. In one implementation, this factor is 16 times downsampling. The resultant complex image is composed entirely of information from 3320, the most ideal region of frequency space to align on. Since these complex images have been reduced in size quite dramatically, it is possible to quickly perform a correlation or any other shift finding method known in the 20 art, such as gradient based shift estimation, to them quickly. However, the modulation is performed over the full image (3500 or 3510). This is avoided in a preferred implementation, in Fig. 32, described below. 5404130_1 991865_speci_lodge -61 In Fig. 32, a preferred implementation of alignment feature extraction is shown where the image is first downsampled by a factor of 4 in step 3210. This has the effect of removing the potentially aliased information, 3330, as well as increasing the processing speed by reducing the image size. The Fourier representation of this downsampled image 5 is shown in Fig. 37. Once this is done, steps 3110 and 3120 can be performed as before, labelled in Fig. 32 as steps 3210 and 3220 respectively. In the implementation of Fig. 32 however the downsampling factor in step 3120 will be smaller, such as 2 or 4. In another implementation, the rotation and scale of the image with respect to the reference image can be measured. Rotation and scale in the spatial domain corresponds to 10 the same rotation and inverse scaling in the Fourier domain, indicated by the arrows, 3350, in Fig. 33. After the modulation in step 3110 or 3210, the rotation and scale approximate small linear shifts in the Fourier domain, as seen at 3630 in Fig. 36. These can be directly measured by multiplying the two transformed images together, since multiplication in the spatial domain is equivalent to correlation in the Fourier domain. The peak of this 15 correlation may then be determined by any suitable frequency-domain peak-finding method to measure the shift. It is then possible to decompose the shift measured in the Fourier domain into radial and tangential shifts, which correspond to the scale and the rotation respectively. Details of an alternate implementation of the print defect detection system 100 are 20 shown in Fig. 12 and Fig. 13. Fig. 12 shows the process of Fig. 8 as modified in an alternate implementation 225b. In the process 225b, the printer model applied in the model step 225 is not based on current printer operating conditions 840 as in Fig. 7, but rather a fixed basic printer model 1210 is used. Since the printer model 1210 is time independent (i.e. fixed), 25 the print/scan model application step 225 will also be time independent, reflecting the time 5404130_1 991865 specijlodge - 62 independent nature of the printer model 1210. In Fig. 12, each sub-model 810, 820, 830 in the process 225b is therefore as described in the earlier implementation, however the parameters of each of the processes 810, 820, 830 are fixed. For example, the dot-gain model 810 and MTF model 820 may use fixed values of d = 0.2 and i = 0.1. The colour 5 model 830 may make the fixed assumption that plain white office paper is in use. In order to determine the level of unexpected differences (i.e. print defects), the alternate process 225b uses the printer operating conditions 840 as part of step 520b, as shown in Fig. 13. Fig. 13 shows the process of Fig. 6 modified in the alternate 10 implementation 520b. Unlike the step 630 of Fig. 6 which uses a constant value for Ddefect to determine whether or not a pixel is defective, substitute step 1310 of Fig. 13 updates the tile defect map using an adaptive threshold based on the printer operating conditions 840. In one implementation of the system, the value chosen for Ddcret is as follows Equation [30]: 15 Ddeect =10+1 O(MAX(de, d, d,, d, ))+ D, [30] where de, d,,,, dy, dik are the drum age factors for the cyan, magenta, yellow, and black drums respectively, and Dpaper is a correction factor for the type of paper in use. This correction factor may be determined for a given sample paper type by measuring the Delta E value between a standard white office paper and the sample paper type using, for 20 example, a spectrophotometer. INDUSTRIAL APPLICABILITY The arrangements described are applicable to the computer and data processing industries and particularly industries in which printing is an important element. 5404130_1 991865_speci_lodge -63 The foregoing describes only some embodiments of the present invention, and modifications and/or changes can be made thereto without departing from the scope and spirit of the invention, the embodiments being illustrative and not restrictive. In the context of this specification, the word "comprising" means "including 5 principally but not necessarily solely" or "having" or "including", and not "consisting only of'. Variations of the word "comprising", such as "comprise" and "comprises" have correspondingly varied meanings. 54041301 991865_speci_lodge

Claims (24)

1. A method of aligning a candidate image with a reference image, said method comprising the steps of: 5 (a) selecting a first alignment patch in the reference image and a corresponding first alignment patch in the candidate image; (b) determining an estimated shift between the first alignment patch and the corresponding first alignment patch; (c) selecting a second alignment patch in the reference image and a corresponding 10 second alignment patch in the candidate image, wherein a size of the second alignment patch and the corresponding second alignment patch is dependent upon the determined estimated shift; (d) determining a further estimated shift between the second alignment patch and the corresponding second alignment patch; and 15 (e) aligning the candidate image and the reference image based on the determined estimated shift and the determined further estimated shift.

2. The method according to claim 1, wherein the size of the second alignment patch and the corresponding second alignment patch is also a function of the distance of the 20 second alignment patch from the centre the reference image.

3. A method according to claim I wherein the reference image and the candidate image are each formed with a plurality of regions, and the first alignment patches are located in a corresponding one of the regions and the second alignment patches are located 25 in a corresponding another of the regions. 5404130_1 991865_specijlodge - 65

4. A method according to claim 3 wherein the regions comprise image bands.

5. A method according to claim 1 wherein the determining of the further estimated 5 shift comprises estimating an expected shift parameter for a patch in a current partition of the candidate image from the estimated expected shift parameter for a previous patch in a corresponding previous partition of the candidate image and a measured value of shift for the previous patch. 10

6. A method according to claim 5 further comprising using the estimated expected shift parameter to determine a size of the second alignment patches in the current partition.

7. A method according to claim 4 wherein the determining of the further estimated shift comprises estimating an expected shift parameter for a neighbourhood within a band 15 based on at least one of shift measurements and an expected shift parameter of a corresponding neighbourhood in a previous band of the candidate image.

8. A method according to claim 7 wherein the previous band comprises the band immediately preceding a current band of the candidate image. 20

9. A method according to claim 4 wherein the determining of the further estimated shift comprises determining multiple estimate shifts in a band of the candidate image.

10. A method according to claim 5 or 7 wherein the expected shift parameter is 25 constrained by expected transformation parameters determined at least from known 5404130_1 991865_speci_lodge -66 rotation, scaling and translation parameters establishing a maximum expected shift at a given spatial location within the candidate image.

11. A method according to claim 1 wherein a size of the second alignment patch is 5 determined from a size of the first alignment patch and an expected shift in a neighbourhood of the alignment hint associated with the second alignment patch.

12. A method according to claim 1 wherein the determined expected shift is used to determine the shift estimation method used to establish the further estimated shift. 10

13. A method according to claim 12 wherein the shift estimation method is selected from one of: (a) gradient-based shift estimation; and (b) phase correlations. 15

14. A method according to claim 1 wherein the second alignment patches haves a size smaller than the first alignment patches.

15. A method of print verification, said method comprising the steps of: 20 rendering a source input document to form a reference image; printing the reference image onto a hard copy medium; capturing a candidate image from the print of the reference image on the hard copy medium; aligning the captured candidate image with the reference image according to the 25 method of claim 1; 5404130_1 991865_speci_lodge - 67 comparing the aligned images to detect defects in the candidate image; and outputting an indication of detected defects.

16. A method according to claim 15, further comprising: 5 determining a plurality of alignment hints from the reference image; and said aligning comprises using the alignment hints to identify corresponding alignment patches.

17. A method according to claim 15 wherein the alignment hints are determined from 10 assessment of at least one of corners and gradient features of the reference image.

18. A method according to claim 15 wherein the alignment hints are formed during the rendering of the reference image. 15

19. A method according to claim 15, wherein: the candidate image is derived by scanning the printed hard copy medium, the scanning being performed to form the candidate image as a plurality of bands, and the scanning, the aligning and the comparing of the reference image and the candidate image is performed sequentially on a band-by-band basis in a direction of 20 scanning.

20. A method according to claim 19 wherein adjacent bands of the images partially overlap. 5404130_1 991865_specilodge -68

21. A method according to claim 15 further comprising modifying the reference image according to a model representing printer characteristics of print engine by which the printing is performed, and performing the aligning and the comparing between the modified reference image and the candidate image. 5

22. A method of aligning images substantially as described herein with reference to any one of the embodiments as that embodiment is illustrated in the drawings.'

23. A computer readable storage medium having a program recorded thereon, the 10 program being executable by computer apparatus to align images in accordance with the method of any one of claims I to 22.

24. Computerised apparatus adapted to perform the method of any one of claims 1 to 22. 15 Dated this 30th day of JUNE 2011 CANON KABUSHIKI KAISHA Patent Attorneys for the Applicant Spruson&Ferguson 20 5404130_1 991865_speci_lodge