GB2297216A - Digital image interpolator using a plurality of interpolation kernels - Google Patents

Description

2297216 DIGITAL IMAGE INTERPOLATOR USING A PLURALITY OF INTERPOLATION

KERNELS

BACKGROUND OF THE INVENTION 5 Technical Field

This invention relates generally to interpolators for processing digital images, and more particularly to such interpolators for high speed processing of digital images, such as for example x-ray images, to magnify or reduce such images. Background Art

Commonly assigned U.S. Patent No. 5,125,042, which issued to Kerr et al on June 23, 1992, discloses a process of interpolating image data first in a row direction. Multiple rows of interpolated pixels values are then interpolated in a column direction. This requires the use of large depth line buffers at the input to the column interpolation stage. Although the interpolation described in patent does a very good job, the interpolation coefficients are digitized at 256 discrete locations and stored in memory. An improved implementation for "photographic quality" images would utilize a desirable continuous spectrum for the interpolation coefficients.

U.S. Patent No 4,595,958, which issued to Anderson Jr. et al. on June 17, 1986, discloses a process of recording one or more digital images on a hardcopy output medium, such as photographic film or paper. In the process of being recorded, the digital images may be enlarged or reduced by interpolation to fill the available output medium. No specific hardware for performing linear interpolation is described.

U.S. Patent No. 4,578,812, which issued to Yui on March 25, 1986, discloses hardware for performing high speed two-dimensional interpolation on 10297.DOC/2-Jan-96 - 2 a digital image by the method of cubic convolution. Sixteen pixels from the original image surrounding an interpolation site in a two-dimensional array are simultaneously multiplied by sixteen corresponding interpolation coefficients.(also known as weight factors) and the sixteen products are added to produce the interpolated value at the interpolation site. The interpolation coefficients, which represent samples of a two- dimensional cubic convolution interpolation kernel, are stored in a digital memory. The cubic convolution kernel is sampled at a granularity of 32 by 32 samples between original pixels. The samples are stored as 12-bit values. As a result, the total storage requirements for the interpolation coefficients is the product of 32 x 32 x 12 x 16, or 196,608 bits, where the 111611 indicates the sixteen coefficients applied to the 16 pixel values to obtain the interpolated value. The storage requirement is thus 192K bits for the interpolation coefficients.

For high resolution images, such as for example diagnostic x-ray images, it is desirable to sample the cubic convolution kernel at a much finer granularity of, say 256 by 256 samples, and to record the coefficients to a higher accuracy of, say 16 bits, to provide a more accurate interpolation and finer divisions between magnification choices. This would require approximately 16M bits of memory (256 x 256 x 16 x 16) for storing the interpolation coefficients. The provision of such a large amount of memory would, of course, be very costly and difficult to address.

- is known that, for certain Furthermore, it types of images, cubic convolution does not produce an optimum interpolation image. For these images, other 1029'7.DOC/2-Jan-96 - 3 interpolation algorithms, such as linear or replication, are preferred.

DISCLOSURE OF THE INVENTION

It is an object of the present invention to provide apparatus and method for performing cubic convolution interpolation on a digital image that overcomes the shortcomings noted above.

It is a further object of the present invention to provide interpolation apparatus and method that can be employed to perform a plurality of types of interpolation such as cubic convolution, decimation, linear, and replication.

According to these and other objects, and in accordance with the present invention, a digital image processing apparatus and method is provided for performing interpolation of an input digital image,which is expressed as a series of spaced pixel values, the interpolation being effected between the spaced pixel values for presentation at output pixel locations separated by a predetermined increment. A memory stores at least one set of exact coefficients which defines a kernel. A set of exact coefficients is determined at each output pixel location based on (1) spacing between input pixel values, (2) the predetermined increment between output pixel locations, and (3) the desired method of interpolation at each output pixel location. The input digital image pixel values are interpolated by applying the exact coefficients to the series of spaced pixel values.

According to another feature of the present invention, the apparatus performs two-dimensional i-erpolation of an input digital image which is nt expressed as vertically-spaced rows of horizontallyspaced pixel values, the interpolation being effected 35 between the vertically-spaced rows and between the 10297.DOC/2-Jan-96 4 - horizontally-spaced pixel values in the rows for presentation at output pixel locations separated by predetermined vertical and horizontal increments. memory stores a set of exact coefficients equations which define at least one kernel for vertical interpolation and one kernel for horizontal interpolation. A set of exact coefficients is determined at each output pixel location based on (1) vertical spacing between rows of input pixel values, (2) horizontal spacing between input pixel values in the rows, (3) the predetermined vertical increment between rows of output pixel locations, (4) the predetermined horizontal increment between rows of output pixel locations, (5) the desired method of vertical interpolation at each output pixel location, and (6) the desired method of horizontal interpolation at each output pixel location. The output digital image pixel values are calculated by applying the exact coefficients to the series of spaced input pixel values. The plurality of kernel types include kernels for performing cubic convolution, linear, decimation, and replication interpolation.

According to a preferred embodiment of the present invention, the memory stores a plurality of different exact coefficients, each of which defines a different unique kernel type, including kernels for performing cubic convolution, linear, decimation, and replication interpolation. The determining means includes means for selecting one of the stored sets of interpolation coefficients.

The invention, and its objects and advantages, will become more apparent in the detailed description of the preferred embodiments presented below.

10297.!)OC/2-Jan-96 BRIEF DESCRIPTION OF THE DRAWINGS

In the detailed description of the preferred embodiments of the invention presented below, reference is made to the accompanying drawings, in which:

Figure 1 is block diagram of an interpolator according to the invention; Figure 1A is a block diagram of a logic and control unit shown in Figure 1; Figure 2 is a graph showing the cubic convolution interpolation kernel; Figure 3 is a graph useful in describing the method of interpolation by cubic convolution; Figure 4 is a schematic block diagram showing the arrangement of the interpolation coefficient memory; Figure 5 is a graph showing the linear interpolation kernel; Figure 6 is a graph showing the replication interpolation kernel; Figure 7 is a diagram illustrating the layout of one example of an output image; Figure 8 is a schematic block diagram showing the arrangement of the input line controller and multiplexor which are part of the interpolator of Figure 1; Figure 9 is a flow chart illustrating the control of the input line controller of Figure 8; Figure 10 is a flow chart illustrating the control of the vertical interpolator; and Figure 11 is a flow chart illustrating the operation of the horizontal interpolator controller.

BEST MODE FOR CARRYING OUT THE INVENTION

The present description will be directed in

Darticular to elements forming part of, or cooperating more directly with, apparatus in accordance with the 10297.DOC/2-Jan-96 present invention. It is to be understood that elements not specifically shown or described may take various forms well known to those skilled in the art.

Figure 1 is a schematic block diagram showing an interpolator according to the preferred embodiment of the present invention. The interpolator receives digital image pixel data P, for example from a frame store memory (not shown), one row at a time through an input 10, and produces over output 12 interpolated digital image data P" one row at a time. The interpolated digital image data can be supplied to a printer (not shown) or stored for future use, such as for example in a second frame store memory, also not shown.

The interpolator includes a vertical interpolator 14, a horizontal interpolator 16, a vertical interpolation coefficient memory 18 containing a plurality of one dimensional interpolation kernels, a horizontal interpolation coefficient memory 20, also containing a plurality of one dimensional interpolation kernels, a vertical interpolation input buffer 22 and a vertical interpolation output buffer 24.

A logic and control unit 26, which is shown in greater detail in Figure 1A, includes a logic and control section, an input line controller, a vertical interpolation controller, and a horizontal interpolation controller. In operation, logic and control unit 26 receives instructions regarding the number and size of input images to be printed on a particular output format. Values are loaded for input line controller information, horizontal and vertical coefficient data, vertical interpolator output buffer control information, and the number of output pixels and output lines. The input line controller of logic and control unit 26 fills vertical interpolator input 10297.DOC12-Jan-96 7 memory 22 and properly maintains independent read and write pointers along with the control of a multiplexor 28.

Vertical interpolator 14 requests data from the input line controller and requests vertical coefficients from vertical interpolation coefficient memory 18. The vertical interpolator interpolates in a column direction by multiplying a plurality of input columns of pixel data by their respective plurality of interpolation coefficients and sums the products to produce interpolated values P'.

The interpolated pixel values PI from vertical interpolator 14 are supplied to 2-line vertical interpolation output buffers 24 from vertical interpolator 14. The vertical interpolator output is 2-line buffered so that a row of interpolated data can be written into memory while another row is being read out. Double buffering is produced, as is known in the art, by a pair of line buffers and tri-stating the unused buffer output.

Horizontal interpolator 16 acquires rows of interpolated pixel data from vertical interpolation output buffers 24 and delays the data one pixel clock to each stage of the horizontal interpolator with a four pixel pipeline 30. The horizontal interpolator produces rows of horizontally interpolated data by multi-plying the delayed pixel values read out of vertical interpolation output buffers 24 by their respective interpolation coefficient kernel retrieved from horizontal interpolation coefficient memory 20 and then summing the products in a manner similar to the operation performed by vertical interpolator 14.

One dimensional interpolation by use of the coefficients stored in memory 18 and 26 will now be described. Interpolation can be described as:

10297.DOC/2-Jan-96 2 P (X) = I:P C X1Xk k ' 1 k h) where P(x) represents the interpolated (1) value; Cx h Xk) is a value of the interpolation kernel at location (x - xk); h is the sample increment of the interpolated function; xk represents the location of the samples of input data (also called interpolation nodes); and Pk are the values of the input data at the interpolation nodes.

The interpolation kernel is a continuous function that converts sampled data into a continuous function, the resulting continuous function can then be re-sampled to obtain the interpolated sample values. Cubic convolution employs a kernel that is defined by piece-wise cubic polynomials defined on the subintervals (-2,-1), (-1,0), (0,1), and (1,2).

Outside the interval (-2,2) the kernel is zero.

Evaluation of equation (1) for the cubic convolution kernel requires four consecutive data samples from the original image data. Figure 2 is a graph showing a cubic convolution kernel 34 defined on the interval (-2, 2). The scale of the x-axis in Figure 2 is equivalent to the spacing of the data samples in the original image.

Figure 3 illustrates how interpolation kernel 34 is employed to compute an interpolated value at location x. Four consecutive data values from the original image data are shown as A, B, C, and D. To produce an interpolated value at location x, interpolation kernel 34 is centered at x, and the 10297.DOC/2-Jan-96 - 9 values of the kernel are calculated at the locations of the original data samples. These values CAf CBP CCr and CD are the interpolation coefficients. The interpolated value at x is:

P(x)=(ACA) + (B CB) + (C Cc) + (D CD) (2) A value at any location between original sample points B and C can be computed in this manner.

To avoid the necessity of calculating the values of the interpolation coefficients each time an interpolated value is produced, the coefficient values are calculated from the piece-wise cubic polynomials defining the kernel, and stored in vertical and horizontal interpolation coefficient memories 18 and 20, respectively (see Figure 1). The preferred embodiment contains a 'layer' of software that takes the answers to questions and generates the appropriate coefficient values with the proper control bits encoded into the values.

In a preferred mode of practicing the invention, the exact coefficients at each output pixel location are calculated based on (1) spacing between input pixel values, (2) the predetermined increment between output pixel locations, and (3) a kernel. The coefficients are calculated to 16-bit accuracy, and are truncated to twelve bits by the hardware. The coefficients are divided into four groups for horizontal and vertical, representing the four segments of the kernel between (-2,-l), (-1,0), (0,1), and (1,2). The four groups are stored in eight banks of memory, four banks of memory for vertical coefficients, four banks of memory for horizontal coefficients, which can be simultaneously addressed by the logic and control unit. The arrangement of the interpolation 10297.DOC/2-Jan-96 kernel lookup tables is shown in Figure 4, where the first memory bank 36 contains the coefficients that define the kernel on the interval (-2,-l), which is shown graphically in the figure. Similarly, the second memory bank 38 contains the group of coefficients representing the portion of the convolution kernel from (-1,0), and so on for memory banks 40 and 42. As shown in Figure 4, a read request is simultaneously applied to all four memory banks to produce coefficients Cl, C2, C3, and C4- Memory can be used more efficiently when the coefficients repeat in some pattern. A bit in the coefficient memory is fed to logic and control unit 26 to determine when to re-initialize the coefficient memory pointer to the beginning of the coefficients.

The existence of a repeat pattern is determined by the equation:

R = (ratioi)/(integer)(ratioi) == 1 The repeat value is calculated by the equation:

ratio = N/(LCD) where N is the number of unique coefficients and LCD is the least common denominator.

Returning now to Figure 1, the interpolation kernel lookup table may contain kernels for any number of other types of interpolation such as linear and replication, in addition to different types of cubic convolution. In the preferred embodiment of the invention, vertical and horizontal interpolation coefficient memories 18 and 20 contain two cubic convolution kernels, one with a=-l (designated 44) and one with a=-0.5 (designated 46), where "a" is a constraint applied to the interpolation kernel polynomials when the kernel is defined as follows:

10297.!)OC/2-Jan-96 1 (a + 2)lsI 3 - (a + 3)1sl' + 1 0 < ISI < 1 C(S) = aiSI3 - 5alsl' + 8alsi - 4a 1 < [si < 2 0 2 < IsI Vertical and horizontal interpolation coefficient memories 18 and 20 also contain a kernel for linear interpolation (designated 48) and a kernel for replication (designated 50). Linear interpolation kernel 48 is shown in Figure 5, and is graphically represented as a triangular waveform in the interval (-1,1) and zero elsewhere. Replication kernel 50 is shown in Figure 6 and is graphically represented as a step function having a value of one in the interval (-2,-1) and zero elsewhere.

Returning to Figure 1, logic and control circuit 26 selects the memory addressed by vertical and horizontal interpolators 14 and 16, respectively, in response to an operator input specifying the interpolation procedure that is desired. For example, it is known that replication interpolation performs better on binary images such as text. Thus, the nearest neighbor kernel may be selected for performing interpolation on text portions of an image.

The sampling increment "h" of the interpolated function is determined by the user by first determining the magnification factors M. and My defined as the ratios xl/x and yl/y, where x is the number of pixels in the original image in the horizontal direction, and xl is the number of pixels available on the output printer for printing the original image. The number of desired pixels on the output medium is determined by the output format including the width of the output medium and the number 10297.DOC12-Jan-96 - 12 of input images to be printed across the page in the output image. Similarly, y represents the number of pixels in an input image in the vertical direction and y' represents the number of pixels desired to print the image in the vertical direction. The sampling increment h is then determined by:

h. = 1 M.

h, = 1 MY (4a) (4b) It is possible that the sampling increments are not identical due to the fact that the individual pixels may not have the same horizontal and vertical dimensions. Values of h less than one represent magnification, and values greater than one represent reduction.

In a preferred embodiment, the output image format is selectable by the user, and comprises one or more regions. There are two types of regions: image and text. Each text region is specified by the height of the region in pixels and number of images in the region. Minimum borders and separation between the images horizontally and vertically are determined automatically by logic and control unit 26. An example of an output format is shown in Figure 7. The output image 52 includes a first region 54 which is yl-pixels high, having three images 56, 58, and 60, a second region 62, Y2-pixels high having two images 64 and 66, and three text regions 68, 70, and 72, each Y3-pixels high.

Referring to Figure 8, the input line controller shown in Figure 1A as part of logic and control unit 26 loads a new line when vertical 10297.DOC/2-Jan-96 interpolator 14 signals another line is needed before the next line can be processed. The determination of the displacement of the continuous function from the sampled data takes place by selection of the coefficients. A decision to switch from one particular mode of interpolation to another particular mode depends only on the number of loaded lines of data. Specifically, (1) when the first line of data is available, replication is available as an output format; (2) when the second line of data is available, replication or linear interpolation are the available formats; (3) when the third line is available, replication, linear interpolation, or quadratic interpolation are available; and (4) when the fourth line is available and beyond, replication, linear interpolation, quadratic interpolation, or cubic convolution interpolation are available formats. Input line controller 74 continues to fetch lines from the frame store using the algorithm shown in Figure 9 until the vertical interpolator signals an end of data. There are five input line buffers 76-80 allowing the next line to be loaded during one line time without delaying the interpolator.

Referring specifically to Figure 9, the input line controller initializes the buffer pointers at 82. If a line request is not received (test 84), the controller checks to see if there is more data (test 86). If a line request is received, one line of data is inputted into vertical input buffer 22 at function block 88. The buffer pointer is incremented at 90, and the input line controller waits for another line request.

The vertical interpolator control provided by logic and control unit 26 is shown in Figure 10.

During an initialization process, referred to at 10297.DOC/2-Jan-96 - 14 function block 92 of Figure 10, logic and control unit 26 first sends the region parameters for the first region, including the number of lines in the region, number of images in the region, and number of output pixels per image to the vertical interpolator. The logic and control unit then selects, at function block 94, the associated coefficients.

Next, at function block 96, the logic and control unit requests a row of data from vertical interpolation input buffer 22. The logic and control unit continues at 98 by requesting a line from the input buffer until enough lines of data are present to initiate the interpolation. When enough lines of data are present, the initialization process is complete.

When a line is requested from the logic and control unit as determined by function block 100, the logic and control unit signals the vertical interpolator to start interpolating and to request a row of data from the input line controller; block 102.

The four interpolation coefficients Cl-C4 from vertical interpolation coefficient memory 18 are entered in parallel into the vertical interpolator 14. The vertical interpolator computes an interpolated pixel value PI as follows:

P'- (cl P1) + (C2 P2) + (C3 P3) + (C4 P4) Interpolation continues for an entire line and then stops to wait for the next line request. When the next line is requested, the interpolator is restarted.

When at 104, an end of region signal is received from logic and control unit 26, the vertical interpolator controller checks at 106 to see if the last region in the image has been interpolated. If 10297.DOC/2-Jan-96 not, an initialization operation is performed for the next region by returning to function 94, and the process is repeated. If the last region is completed, the process is ended. If at 104, an end of region signal is not received, the next line request is sought at block 102.

The horizontal interpolator 16 will now be described in more detail with reference to Figure 11. The logic and control unit sends the region parameters for the first region, including the number of lines in the region, number of images in the region, and number of output pixels per image to the horizontal interpolator. The horizontal interpolator controller first selects at 110 the interpolation type requested by the user and the associated coefficients. when data is available in the vertical interpolation output buffer 24, the controller requests the number of pixels needed (as determined by the logic and control unit 26 and the needs of the selected coefficient) from the vertical interpolation output buffer 24. This completes initialization by requesting the proper number of pixels.

Horizontal interpolator 16 is enabled, and the horizontal coefficients from coefficient memory 20 are brought together with the pixel data in the interpolator to calculate and then output P". This is repeated for all pixels of the present line, getting additional pixel data as required. At the end of a line, a check is made for an "end of region" signal at 112. If received from the Logic and Control Unit 26, the Horizontal Interpolator Controller checks at 114 to see if the last region in the image has been interpolated. If not, an initialization operation is performed for the next region by returning to function 110, and the process is repeated. If the last region 10297.DOC/2-jan-96 is completed, the process is ended. If at 112, an End-of-Region signal is not received, the next vertical interpolation output buffer is selected at block 116.

In decimation mode, the number of output pixels will be less than the number of input pixels; and a valid data flag is generated to signal to the downstream writer to indicate that such is present. This function is well known in the art, and one can select from several conventional methods of doing so.

Industrial Applicability And Advantages

The interpolator according to the present invention is useful for enlarging and reducing digital images such as diagnostic radiographs. The interpolator is advantageous in that a high resolution interpolation can be practically performed with a minimal size of coefficient memory. The invention has the further advantage that due to the reduced size of coefficient memory over the prior art two-dimensional coefficient memory, several interpolation kernels may be stored in the memory, thereby providing the ability to easily switch between different types of interpolation.

The invention has been described in detail with particular reference to preferred embodiments thereof, but it will be understood that variations and modifications can be effected within the spirit and scope of the invention.

10297.DOC/2-Jan-96 17 -

Claims (6)

CLAIMS:

1. A digital image processing apparatus for performing interpolation of an input digital image which is expressed as a series of spaced pixel values, the interpolation being effected between the spaced pixel values for presentation at output pixel locations separated by a predetermined increment, said apparatus comprising:

a memory for storing at least one set of exact coefficients equation which defines a kernel; means for determining a set of exact coefficients at each output pixel location based on (1) spacing between input pixel values, (2) the predetermined increment between output pixel locations, and (3) a desired method of interpolation at each output pixel location; and means for interpolating the input digital -image pixel values by applying the exact coefficients to the series of spaced pixel values. 20

2. A digital image processing apparatus as defined in Claim 1 wherein: the memory stores a plurality of different sets of exact coefficients, each of which defines a different unique kernel type; and 25 said determining means includes means for selecting one of the stored sets of exact coefficients.

3. A digital image processing apparatus as defined in Claim 2 wherein said different unique kernel types include kernels for performing cubic convolution, linear, decimation, and replication interpolation.

4. A digital image processing apparatus for performing two-dimensional interpolation of an input digital image which is expressed as vertically10297.DOC/2-Jan-96 spaced rows of horizontally-spaced pixel values, the interpolation being effected between the verticallyspaced rows and between the horizontally- spaced pixel values in the rows for presentation at output pixel locations separated by predetermined vertical and horizontal increments, said apparatus comprising:

a memory for storing a set of exact coefficients which define at least one kernel for vertical interpolation and one kernel for horizontal interpolation; means for determining a set of exact coefficients at each output pixel location based on (1) vertical spacing between rows of input pixel values, (2) horizontal spacing between input pixel values in the rows, (3) the predetermined vertical increment between rows of output pixel locations, (4) the predetermined horizontal increment between rows of output pixel locations, (5) a desired method of vertical interpolation at each output pixel location, and (6) a desired method of horizontal interpolation at each output pixel location; and means for interpolating the input digital image pixel values by applying the exact coefficients to the series of spaced pixel values.

5. A digital image processing apparatus as defined in Claim 4 wherein:

the memory stores a plurality of exact coefficients which define a plurality of kernel types for vertical interpolation and a plurality of kernel types for horizontal interpolation; and d determining means includes means for sai selecting one of the stored sets of exact coefficients for vertical interpolation and one of the stored sets of exact coefficients for horizontal interpolation.

10297.DOC/2-Jan-96 - 19

6. A digital image processing apparatus as defined in Claim 4 wherein said interpolationg means includes: vertical interpolation means for multiplying input digital image pixel values of a plurality of successive columns of input digital image pixel values by respective exact coefficients, and summing the products to produce vertically-interpolated pixel values; and horizontal interpolation means for multiplying vertically-interpolated pixel values of a plurality of successive rows of verticallyinterpolated pixel values by respective exact coefficients, and summing the products to produce rows of interpolated pixel values.

10297.DOC/2-jan-96