EP1540595A1

EP1540595A1 - Image transmission system and method for determining regions of interest in an image transmission

Info

Publication number: EP1540595A1
Application number: EP03794831A
Authority: EP
Inventors: Paola Marcella c/o Motorola Ltd. HOBSON; Timor Kadir; John Brady
Original assignee: Motorola Inc
Current assignee: Motorola Solutions Inc
Priority date: 2002-09-13
Filing date: 2003-06-10
Publication date: 2005-06-15
Also published as: GB2393062B; GB0221175D0; HK1063369A1; WO2004025570A1; AU2003242670A1; GB2393062A

Abstract

A method (400) for identifying a feature in an image includes the steps of defining (410) a plurality of saliency values of an image or set of images over multiple parameters and calculating (414) an entropy value for each the saliency values over substantially each of the multiple parameters. One or more peak entropy values are determined (430, 432) for the image or set of images based on the calculated entropy values. A feature of the image or set of images is identified (454), based on the weighted peak entropy value. An image transmission device, image transmission unit and image transmission system are also described. This provides a method and apparatus for extending scale definition to multiple parameters, thereby generalising the structures that can be identified, for example, to identify anisotropic regions.

Description

Image Transmission System And Method For Determining Regions Of Interest In An Image Transmission

Field of the Invention This invention relates to image modelling in an image transmission system. The invention is applicable to, but not limited to, image modelling using multiple parameters to identify anisotropic features.

Background of the Invention

Future generation mobile communication systems are expected to provide the capability for video and image transmission as well as the more conventional voice and data services. As such, video and image services will become more prevalent and improvements in video/image compression technology will likely be needed in order to match the consumer demand within available bandwidth.

Current transmission technologies, which are particularly suited to video applications, focus on interpreting image data at the transmission source. Subsequently, the interpretation data, rather than the image itself, is transmitted and used at the destination communication unit. The interpretation data may or may not be transmitted in compressed form.

Two alternative approaches to image interpretation are known - the ^x image-driven' , or bottom-up, approach, and the model-driven' , or top-down, approach. Model-based techniques tend to produce good results where the range of objects (more generally content) of interest is limited, but where the environment is relatively unconstrained. Image-driven approaches perform better in a relatively constrained environment. In practice, most systems tend to combine elements of the two approaches .

The image-driven approach relies on features in the image, such as edges or corners, to propagate "naturally" and form meaningful descriptions or models of image content. A typical example is ^x figure-ground' image segmentation, where the task is to separate the object of interest in the foreground from the background.

In the model-driven approach, information regarding content expectation is used to extract meaning from images. A typical example is object recognition where an outline Computer-Aided Design (CAD) model is compared to edges found in the image - an approach commonly used in manufacturing line inspection applications.

The key difference between the image-driven and model- driven approaches is in the feature grouping stage. In the image-driven approach, the cues for feature grouping emanate from the image, whereas in the model-driven approach the cues come from the comparison models.

In one variation of an image-driven approach, a number of small salient patches or 'icons' are identified within an image. These icons represent descriptors of areas of interest. In this approach, saliency is defined in terms of local signal complexity or unpredictability or, more specifically, the entropy of local attributes. Icons with a high signal complexity have a flatter intensity distribution and, hence, a higher entropy. Examples of intensity distributions are shown in figure 1, which is discussed later. In more general terms, it is the high complexity of any suitable descriptor that may be used as a measure of local saliency.

Known salient icon selection techniques measure the saliency of icons at the same scale across the entire image. The particular scale selected for use across the whole image may be chosen in several ways. Typically, the smallest scale, at which a maximum (peak) occurs in the average global entropy, is selected. However, the size of image features varies. Therefore, a scale of analysis, that is optimal for a given feature of a given size, might not be optimal for the same feature of a different size.

It is known that scale information is important in the characterisation, analysis, and description of image content. For example, prior to filtering an image, it is necessary to specify the kernel size, or in other words the scale, of a filter to use as well as the frequency response. It is also known that filters are commonly used in image processing for tasks such as edge-detection and anti-aliasing.

Alternatively, scale can be regarded as a measurement to be taken from an image (region) , and hence can be used as a descriptor. The description extracted from the image may be used for subsequent matching or classification of that image (region) . One example may be in segmenting parts of an aerial image into different regions. In order to extract an appropriate scale description, the method must capture the scale behaviour of the most 'dominant', or in other words the most salient, scales in an image . Recent work in the area of matching and registration of aerial images has shown some promising results using a purely image-based approach. This is described in the paper: "Robust Description and Matching of Images. PhD thesis, University of Oxford, 1998", by Sebastien Gilles. Gilles argues that it is possible to sufficiently describe an image by using a number of small (local) salient image patches or icons . The measure of saliency is local entropy; this measures local signal complexity or unpredictability, which in the context of Shannon information theory relates to information content. Salient icons are selected by an algorithm in each of the images to be registered and, by matching these; the approximate global transform between the images may be determined.

Gilles' experiments with a global image matcher, based on the maximisation of mutual information, show that such (global) methods are prone to failure due to local minima in the optimisation function. In order to overcome this problem, Gilles' approach is to use the local salient icon matcher to find the approximate transform, and then to use the global technique to fine-tune this.

The approach is entirely bottom-up and hence no model of expected content is imposed on the image. This generality makes this approach very attractive. A focus of the present invention is to define how those areas of an image (or image sequence) are selected to be representative of the image content (or salient) . Hence, the selected areas can be used as the source for efficient description or content interpretation. Gilles defines saliency in terms of local signal complexity or unpredictability; more specifically he suggests the use of the entropy of local attributes.

Referring now to FIG. 1, local histograms of intensity from various image segments are illustrated. Areas corresponding to high signal complexity are shown to have a flatter distribution, and hence a higher entropy. In general terms, it is the high complexity of any suitable descriptor that may be used as a measure of local saliency. Given a point X=(x,y) in the image and a local neighbourhood R_x, and a descriptor D that takes on values {d ...d_r}, local entropy is defined as:

Blϊ x = - ∑P Jl b&RUtx ^deD [1]

Where pa, R_x is the probability of descriptor 'd' taking on value 'di' in the local region ^λR_x' .

The inventors of the present invention have recognised a problem with the Gilles technique, in that it is necessary to select a scale or size of the local neighbourhood region for the entropy calculation and subsequent description. Gilles' original algorithm measures the saliency of image patches at the same scale across the entire image. Since the size of image features vary, any feature identification technique needs to account for this by selecting an optimum scale (or set of scales) of analysis for a given feature.

Furthermore, the Gilles method considers only the feature-space saliency of an image, whereas the analysis should include the scale dimension as well. That is, the measure of saliency should measure saliency over scale as well as feature-space.

The applicant of the present invention described a mechanism for generating these salient points in co- pending UK patent application GB-A-02367966. In this regard, a Scale Saliency algorithm was introduced as an improvement of the original Gilles algorithm in order to address the aforementioned problem. In order to analyse the scale space behaviour of signals and select appropriate sizes of local scale (the size of the region of interest window used to calculate the entropy) , the method searched for peaks in entropy for increasing scales (at each pixel position) . The method then weighted the entropy value with a scale-normalised measure of the statistical self-dissimilarity at that peak value, as shown in FIG. 2.

A summary of the known Scale Saliency method is described below, where notably the behaviour of an entropy scale measure is considered by varying a single scale parameter.

It is commonly assumed that a feature that is present across a large number of scales is particularly salient. However, saliency is based on complexity, defined in terms of unpredictability. In real-life images, this exists at a small number of scales and spatial locations, and hence is considered relatively rare. If an image was complex and unpredictable at all spatial locations and scales, then it would either be a random image or fractal-like. UK patent application GB-A-02367966 indicated that the width of the entropy plot, taken about different points on an image, could serve as a useful indicator, as a saliency estimate based on peak height alone does not enable a distinction to be made between multiple peaks.

FIG. 2, is a flowchart illustrating the known method for determining icon salience, as described in UK patent application GB-A-02367966.

A region defined by a scale s, is typically initialised to a minimum scale value s in step 200. The region is defined around a pixel at coordinates (x, y) in an image, as shown in step 210. The probability density function (PDF) of the values of the pixels within the region is then estimated, in step 220. The values may be grey- level intensity values, colour levels, or any other characteristic used to define the type of interest desired for image features.

A Parzen window PDF estimator, for example using a Gaussian kernel method or a basic histogram binning method may be used as the PDF estimation algorithm. Once the PDF is calculated, the entropy H_D of region (x, y, s) is then calculated in step 230. Scale s is then increased by the next scale increment in step 240, and the previous steps are repeated for all scales of s between Si and a maximum scale value s in step 250.

Once the entropy has been calculated for all scales between Si and s₂ for each pixel at co-ordinates (x, y) , those regions having a peak entropy relative to the entropy of the immediately preceding and succeeding regions are determined in step 260. The entropy H_D of each peak region is then weighted in accordance with a value W_D(x,y,s), which is proportional to its peak width estimate, in step 270, in order to provide a measure of saliency S.

Steps 200 to 270 are then preferably repeated for all pixels in the image, as shown in step 280. Once the entropy values of regions of peak entropy have been weighted, they are preferably ranked by weighted peak entropy value, thus resulting in a ranking of the pixels by peak saliency, in step 290. In this manner, peak width is used as a weighting factor in the calculation of saliency.

The idea is that since features are considered salient if they are complex or unpredictable in the feature-space, then in the scale dimension self-similarity corresponds to predictability in that dimension. Therefore, unpredictable behaviour over scale should be preferred; that is narrow-peaks in entropy for increasing scales .

The measure for self-similarity that is used is the sum of absolute difference in the histogram of the local descriptor, although there are many alternative ways in which two PDFs may be compared (for example Kullback- Leibler) . The calculation in the continuous case is as follows :

; D{ , ) = £>( ,*) x W£>( , )

[2] Where (x)=(x,y) is the spatial position of the pixel in the image. Where entropy H_D is defined in the continuous case by:

HD{»,X) and where p(d,s,x) is the probability density as a function of scale s, position (x) , and descriptor value (d) , which takes on values in D, the set of all descriptor values .

For a practical implementation, the discrete case of equation [3.1] is required, as shown in equation [3.2] :

The saliency over scale measure, W_D, is defined in the continuous case by:

For a practical implementation, the discrete case of equation [4 . 1 ] is required, as shown in equation [4 . 2 ] :

WD (*, ^X) = ^71 ^X Σ IftlΛX ^~ W,*-!,_*! deo _{[4 - 2 ]}

The absolute partial derivative term is approximated by the sum of absolute difference in the discrete case. The vector of scales at which the entropy peaks, s_p, is defined by:

*{.'^ <^o} [5.1] For a practical implementation, the discrete case of equation [5 . 1] is required, as shown in equation [5.2] : s_p = {s ~ U_D(s- l,x.) < %_D(s,ιή > 'U_D(s + !, )}

[5 . 2 ]

The method generates a 3D space (2 spatial dimensions plus scale) sparsely populated by scalar saliency values.

The above definition does not specify how the probabilities, p(d,s,x), are to be obtained from the image. In the standard embodiment, a circular sampling window is suggested. This is beneficial because it enables a rotationally invariant saliency measure. However, in the general case, any suitable single parameter-sampling window may be used.

Statement of Invention

The inventors of the present invention have recognised that the restriction in the prior art arrangements to a single-parameter scaling function, for example the radius of a circle, biases the method towards isotropically salient features. Furthermore, the method cannot measure local orientation. Also, sub-optimal scales are selected for features exhibiting anisotropy, for example, short sections of line-like features.

Thus, there exists a need in the field of the present invention to provide an image transmission system, an image transmission unit and method of processing an image that further improve salient icon selection techniques, wherein the abovementioned disadvantages may be alleviated. In accordance with a first aspect of the present invention, there is provided a method for identifying a feature in an image, as claimed in Claim 1.

In accordance with a second aspect of the present invention, there is provided a method of image transmission, as claimed in Claim 10.

In accordance with a third aspect of the present invention, there is provided an image transmission device, as claimed in Claim 11.

In accordance with a fourth aspect of the present invention, there is provided an image transmission unit, as claimed in Claim 14.

In accordance with a fifth aspect of the present invention, there is provided an image transmission system, as claimed in Claim 15.

In accordance with a sixth aspect of the present invention, there is provided a storage medium storing processor-implementable instructions for controlling a processor to carry out any of the aforementioned method steps of the first and/or second aspect of the present invention, as claimed in Claim 16.

Further aspects of the present invention are as defined in the dependent Claims.

In summary, the inventive concepts of the present invention, described below, overcome the limitations of the prior art approaches by providing N-parameter sampling windows to account for anisotropic features/regions .

In particular, the inventors propose to generalise the isotropic, single parameter sampling function to N- parameter sampling windows to account for anisotropic features/regions. In this manner, a better description of the image, and therefore a feature in the image, is determined.

Brief Description of the Drawings

FIG. 1 illustrates a series of local histograms of intensity of an image, useful in understanding the context of the present invention; and FIG. 2 is a flowchart illustration of a known method for determining icon salience.

Exemplary embodiments of the present invention will now be described, with reference to the accompanying drawings, in which:

FIG. 3 shows an elliptical function that provides an example of a plurality of parameter functions that can be used in identifying a feature in an image, in accordance with the preferred embodiment of the invention; FIG. 4 shows a flowchart for identifying a feature using a plurality of parameters, in accordance with an enhancement to the preferred embodiment of the invention; FIG. 5 illustrates a device for identifying a feature in an image, in accordance with an embodiment of the invention;

FIG. 6 illustrates salient regions identified using the known isotropic scale saliency method of FIG. 2; FIG. 7 illustrates salient regions identified using the anisotropic scale saliency method of FIG. 4, in accordance with an embodiment of the invention;

FIG. 8 illustrates an original view of a cheetah; and FIG. 9 illustrates a stretched version of the cheetah of FIG. 8 to indicate how the method of FIG. 4 can be used for matching across different views of an object.

Description of Preferred Embodiments

The preferred embodiment of the present invention overcomes the limitations of the aforementioned approaches by providing N-parameter sampling windows to account for anisotropic features/regions.

The preferred approach to resolve the aforementioned problems is to use an ellipse 300 parameterised by two scale parameters 'SI' and 'S2' (one for each axis) and a rotation 'S_theta' _• as illustrated in FIG. 3. However, it is within the contemplation of the invention that any multiple parameter mechanism may benefit from the inventive concepts described herein.

In this manner, a better description of the image, and therefore a feature in the image, is determined.

Furthermore, this approach is able to deal with some of the effects of projective distortion, for example, 3D effects such as perspective. A practical benefit of this approach is that the generalised Scale Saliency algorithm is able to select an improved set of features and scales in cases where the image is of a 3D scene. For example, in an orthographic view, a circle is imaged as a circle. However, in another view, the circle might map onto an ellipse. A yet further benefit of the generalised concept is that orientation information can now be captured.

In an enhanced embodiment of the present invention, the multiple parameter function uses scale, axis ratio, and orientation to parameterise more accurately the ellipse. In this enhanced embodiment, the W_D value has only to be calculated for one parameter instead of two. Indeed, a better parameterisation might be to use scale (s) , ratio

(r=sl/s2) and angle a (equivalent to St_het_a) / where scale is described in UK Patent Application GB-A-02367966. This parameterisation has the useful property that the scale parameter is equivalent to the original isotropic scale, to allow backward compatibility with the single parameter approach.

The preferred embodiment of the present invention is invariant to anisotropic scaling and shear; that is, the full Affine set of transformations. The Affine set is a first order approximation to the full projective set of transformations, as known to those skilled in the art. Consequently, it will not be further described here.

The concept of the present invention can also be applied to the arrangement of UK patent application 0112540.0, (published as GB-A-2375908) which was filed on 23 May 2001. That UK application discloses an arrangement for characterising texture or a texture-like region in an image, and involves obtaining saliency values. The saliency values obtained can be found using the multi- parameter approach of the present invention. In GB-A- 2375908, the single parameter definition of scale (s) would be replaced by the multi-parameter version described in the present application. So the scalar values in GB-A-2375908 would be replaced by the vector S used in the present application. So texture classification is one application of the multi-parameter approach of the present invention. The text of GB-A- 2375908 is hereby incorporated by reference.

The preferred enhanced method of the present invention is described below with reference to the flowchart 400 of FIG. 4A and FIG. 4B. The flowchart focusses on calculating the entropy for regions of pixels, identifying peaks and applying a weighting function.

The enhanced process is carried out for each pixel in the image, where the pixel position is identified by 'x' and 'y' co-ordinates, as shown in step 402. Initially, the rotation (angle) step 'a' of the elliptical function is set to zero, as in step 404. The ratio step (r=Sl/S2) is set to 'rmin/ratiodiv' , as shown in step 406. The variables 'rmin' and 'ratiodiv' specify a number and range of ratios to be tried. For example, the preferred embodiment of the present invention uses rmin = '5', ratiodiv = '20', which results in the ratio sampling from '0.25' to '1' in steps of '0.05'. The scale value S is then set to a minimum (Smin) , as in step 408, and as described in UK Patent Application GB-A-02367966.

The image sampler (IS) samples the local descriptor values at image location I(x,y) using the aforementioned values for 'a', 'r' and 's', as shown in step 410. The preferred arrangement for generating these samples is also described in co-pending UK Patent Application GB-A- 02367966 filed by the same applicant.

A probability density P(d,s) value is set to an estimate of the local probability density function (PDF) from the IS samples, as shown in step 412. The PDF calculation can be performed by any known mechanism, such as a histogram technique, as described in UK Patent Application GB-A-02367966. The entropy (H_D) for that sample is then calculated, for each of the aforementioned values for 'a', 'r' and 's', as shown in step 414. The scale saliency (W_D) for that sample is also calculated for the aforementioned values for 'a', 'r' and 's', as shown in step 416.

The process then moves on to the next scale sample, as in step 418. If the next scale sample is not greater than the maximum scale sample in step 420, the above process repeats from step 410 through step 418. The application of S_maχ is described in UK Patent Application no. GB-A- 02367966. If the next scale sample is greater than the maximum scale sample S_maχ, in step 420, the ratio step is incremented in step 422.

A determination is then made as to whether all ratios have been sampled (i.e. whether the ratio value of Si to

S₂ is '1'), in step 424. If the next ratio value is not greater than '1' in step 424, the above process repeats from step 408 through to step 422. That is, the scale value for the next ratio value is set to a minimum, and a new set of scale values for the new ratio calculated.

If the next ratio value is greater than '1', in step 424, the angle value 'a' is incremented in step 426, in accordance with the selected angle step-size. In this regard, the number of angle steps ( 'number_angles' ) may be selected in order to perform a predetermined number of samples between 0 and π. A higher number of angle steps will give a more accurate result, but will of course be slower.

For this new angle value, a determination is made as to whether the maximum angle 'π' has been sampled, in step 428. If the maximum angle 'π' has not been sampled in step 428, the above process repeats from step 406 through step 426. That is, the scale value and the ratio value are both set to a minimum, and new sets of scale values and ratio values are calculated for this new angle.

The flowchart 400 then moves on to FIG. 4B, once calculations for each value of entropy (H_D) , local descriptor (IS), PDF (P(d,s)) and scale saliency (W_D(a,r,s) have been completed for each value of angle, ratio and scale (a,r,s) . A filter, preferably a three- tap averaging (smoothing) filter is applied to the scale saliency values (W_D(a,r,s)), as shown in step 430. This smoothing step is applied with respect to the scale parameter only, for all angles and ratios, and alleviates some of the potential noise problems. Again, the angle step 'a' of the elliptical function is set to zero in step 432. The ratio step (r= S₁/S₂) is set to 'r_mi_n/ratiodiv' , as shown in step 434. As mentioned above, the variables 'r_mi_n' and 'ratiodiv' specify a number and range of ratios to be tried. The scale value S is then set to a minimum (S_mi_n +1) in step 436.

A determination is then made at the selected scale value (in the first case S_mi_n +1) , to see if there is an entropy peak with respect to scale at this point, as shown in step 438. If there is no entropy peak at this point in step 438, then the scale value is incremented in step 442. If there is an entropy peak at this point in step 438, then Y_D (as specified in equation [6]) is calculated in step 440, and all values of Y_D are stored for later processing. The scale value is then incremented in step 442. If there is not an entropy peak with respect to scale at this point in step 438, the process then moves to step 442 to increment the scale value.

A determination is then made to see whether the selected scale value is a maximum value, in step 444. If the scale value is not at the maximum value in step 444, the above process repeats from step 438 to step 442, with a determination as to whether there is an entropy peak at the current scale value. Otherwise, if the scale value is at the maximum value in step 444, the ratio (r) is incremented in step 446. A determination is then made to see whether the current ratio (r) has reached a value greater than '1', in step 448. If the ratio (r) has not reached a value greater than '1' in step 448, the above process repeats with this incremented value of 'r' at step 436. Otherwise, if the ratio (r) has reached a value greater than '1' in step 448, the angle 'a' is increased in step 450, in the same manner as step 426 above.

For this new angle value, a determination is made as to whether the maximum angle 'π' has been sampled, in step 452. If the maximum angle 'π' has not been sampled in step 452, the above p'rocess repeats from step 434 through step 448. That is, the scale value and the ratio value are both set to a minimum, and new sets of scale values and ratio values are calculated for this new angle. If the maximum angle 'π' has been sampled in step 452, the process ends by determining the feature (isotropic or anisotropic) of the image in step 454.

In essence, the flowchart of FIG. 4A and FIG. 4B describes replacement of the single parameter sampling function with a multiple parameter version. In this case, it's a three-parameter version. The entropy is calculated at each value of each of the three parameters .

However, in the preferred embodiment of the present invention, the scale saliency (W_D) measure is modified such that the partial derivative is taken with respect to only the scale parameter. The inventors have recognised that the rotation angle and ratio cannot be used in this context, as they would measure unpredictability in the rotation angle and ratio. It is not viable to look for peaks over a rotation dimension, as that would bias the method against round items, such as circles, which are not affected by rotation. It is inefficient to look for peaks with respect to ratio, as these do not add to the saliency decision. The peak detection is similarly modified to search for peaks in entropy with respect to scale, but not rotation angle or ratio. The correct orientation is still found because the shape that causes the largest inter-scale saliency measure (W_D) is the one that matches the feature shape .

Specifically, the equations can be modified to the elliptical anisotropic case by replacing the scalar s parameter with a vector, s= (a,r,s) . The vector of scales at which the entropy peaks, s_p, becomes a matrix, S_p with three rows, one for each of the scale variables and as many columns as peaks at that position. The modified equations are as follows:

y_D(s_p,χ) ^ (S_P, ) x w_D(s_p,χ) [6]

7 D{S, ) - /p( ,a,x)log₂i?( _}s,x).dd d£D [7]

WJΓJ(S,X) =4tX / ^■p(d, &,x.) \ Ad d€D [8]

For a practical implementation, the discrete versions of equations [7] , [8] , [9] are required: .2

W_D<S,x) & »,x Pd,s—l,x|

2*-l eo [11]

S_P = {e:fl_D(*-l,x)<fl_D(«,x):>flij(s + l,x)}

[12]

The invention has been described above in terms of a method. However, the invention also comprises an image transmission unit/device functioning in accordance with the invention.

An embodiment of such a device is illustrated in FIG. 5. The device of FIG. 5 serves to rank pixels in an image by degrees of saliency. The device comprises a processor 500 providing a defining function 510 that defines a plurality of regions for each of a plurality of pixels in an image 505. In accordance with the preferred embodiment of the present invention, each of the regions is defined over a different scale 's', at a ratio 'r' and angle 'a' for any one of the pixels at coordinates (x, y) . The processor also includes a calculation function 530 for calculating an entropy value for each of the regions, for each permutation of the respective variables a, r, s .

The processor also includes a peak identification function 560 for identifying any peak entropy regions. The peak entropy regions are regions that include a peak entropy value and a weighting function 570 for weighting the peak entropy value of each of the peak entropy regions by a weight value corresponding to a peak width estimate of the peak entropy value. This is described in UK patent application GB-A-02367966. Finally, the processor includes a feature identification function 590 for identifying a feature in an image based on the determined peaks.

The focus of the preferred embodiment is to find interesting regions in an image in an unsupervised manner. The weighted saliency measure supplies values about how interesting something is. In this regard, the present invention focuses on identifying a feature in an image, which to all intent and purpose is such an interesting region in an image and the two expressions should be deemed synonymous . Once a rank ordered set of weighted saliency measures has been obtained, it is possible to then apply any conventional thresholding technique to select a certain number of regions or points, for example, the 10% most salient points. Alternatively, it is possible to apply clustering by any known technique to group salient points into salient regions .

Also shown in FIG. 5 is an 'acquisition and frame structure' unit 508, which includes a camera for scanning the image. Such a unit is known to the person skilled in the art, and hence is not described in further detail. The PDF estimator 520 provides the probability density function, as described above with reference to the methods of the invention.

Referring now to FIG. 6, an example of the performance of the original isotropic scale saliency algorithm is illustrated. FIG. 7 illustrates an example of the performance of the modified anisotropic scale saliency algorithm. Comparison of the two performances of FIG. 6 and FIG. 7 highlights that the anisotropic version correctly identifies the scales of the ellipses and the circle. In contrast, the isotropic version correctly detects only the circle and finds numerous features along the ellipses.

Although the preferred embodiment of the present invention has been described with respect to a three- parameter sampling window, it is within the contemplation of the invention that the inventive concepts can be applied to any multiple-sampling window or alternative parameterisations . For example, instead of parameterising an ellipse using a major/minor axis ratio, a rotation, and a scale, it is possible to use two scales and a rotation. As another example, any polygon with N sides can be parameterised using N-l terms (for example the corner angles), a rotation and a scale. The W_D term should then be modified to calculate the partial derivative with respect to each of the parameters.

It is envisaged that the inventive concepts hereinbefore described apply to any video or image communication device. The video or image communication device may select salient regions of an image for preferential transmission. Particularly where the transmission is over a radio link to or from a mobile or portable radio (PMR) or mobile telephone, the transmission may comprise the selected regions only. Alternatively, these regions may be transmitted more frequently than other regions. The video or image communication device may form part of a mobile or portable radio (PMR) or cellular telephone. As multimedia communication systems become commodetised in the future, technologies such as those offered by this invention will enable users to efficiently communicate key features of an image, without having to pay for additional and costly bandwidth in order to send the entire image itself. This invention could be incorporated into any mobile image or video transmission device .

It will be understood that the image transmission system, transmission unit and method for identifying a feature within an image, as described above, provides at least the following advantages: (i) Extending the ability to find interesting regions of an image, as described in UK Patent Application GB-A- 02367966, to anisotropic regions. In this regard, scale definition is extended to multiple parameters, thereby generalising the structures that can be identified, (ii) If an ellipse is parameterised using scale, rotation and a ratio of the major and minor axes, the method is invariant to anisotropic scaling and shear. As such, it is compatible with the full Affine set of transformations . (iii) In particular, the method is also useful for matching across different views of an object, as illustrated by the examples shown in FIG. 8 and FIG. 9. FIG. 8 illustrates an original view of the Cheetah, and FIG. 9 illustrates a stretched version. It is noteworthy that the interesting regions (the spots) are still correctly identified in the stretched version of FIG. 9, by utilising the aforementioned inventive concepts.

Method of the invention: In summary, a method for identifying a feature in an image has been provided. The method includes the steps of defining a plurality of saliency values of an image or set of images over multiple parameters, and calculating an entropy value for each of the saliency values over substantially each of the multiple parameters. The method further includes the steps of determining one or more peak entropy values for the image or set of images based on the calculated entropy values; and identifying a feature of said image or set of images based on the determined peak entropy value .

Apparatus of the invention:

Furthermore, an image transmission device has been described that includes a processor having the following functions: a defining function for defining a plurality of saliency values of an image or set of images over multiple parameters; a calculation function for calculating entropy value for each of the saliency values over substantially each of the multiple parameters; a peak entropy identification function for identifying any peak entropy value for the image or set of images based on said calculated entropy values; and a feature identification function identifying a feature in the image or set of images based on the peak entropy value .

An image transmission unit adapted to perform the above method steps has also been provided. In addition, an image transmission system adapted to facilitate any of the above method steps or incorporate the above image transmission device has been provided. Also, a storage medium storing processor-implementable instructions for controlling a processor to carry out any of the above method steps has been provided.

Whilst the specific and preferred implementations of the embodiments of the present invention are described above, it is clear that a skilled artisan could readily apply variations and modifications of such inventive concepts.

Thus, an image transmission system, an image transmission unit and method of identifying an image that further improve salient icon selection techniques have been provided, wherein the abovementioned disadvantages associated with prior art arrangements have been substantially alleviated.

Claims

1. A method (400) for identifying a feature in an image, comprising the following steps: defining a plurality of saliency values of an image or set of images over multiple parameters; calculating an entropy value (414) for each of said saliency values over substantially each of said multiple parameters; determining one or more peak entropy values (438) for said image or set of images based on said calculated entropy values ; and identifying a feature of said image or set of images (454) based on said peak entropy value.

2. The method for identifying a feature in an image according to Claim 1, wherein the step of defining includes the step of sampling a window of an image across multiple parameters such that an anisotropic feature can be identified.

3. The method for identifying a feature in an image according to Claim 1 or Claim 2, wherein the step of defining includes the step of parameterising an ellipse in order to calculate an entropy peak with respect to scale.

4. The method for identifying a feature in an image according to Claim 3 , wherein the step of parameterising an ellipse includes using at least two scale parameters of said ellipse.

5. The method for identifying a feature in an image according to Claim 4, wherein the at least two scale parameters include one or more of : a ratio of the ellipse axes of said ellipse, and/or a rotation, preferably angle (Stheta) of said ellipse, and/or a scale.

6. The method for identifying a feature in an image according to Claim 1 or Claim 2, wherein the step of defining includes using a polygon having N sides that can be parameterised using N-l terms.

7. The method for identifying a feature in an image according to any preceding Claim, wherein said step of calculating an entropy value includes measuring an inter- scale saliency of said image or set of images, the method further characterised by the step of: calculating said inter-scale saliency measurement by taking a partial derivative of a probability density function with respect to scale.

8. The method for identifying a feature in an image according to Claim 7 when dependent upon Claim 4, wherein said step of taking a partial derivative of said inter- scale saliency measurement precludes taking a partial derivative of said rotation angle.

9. The method for identifying a feature in an image according to Claim 7 when dependent upon Claim 4 , wherein said step of determining a peak entropy value precludes taking a partial derivative of said rotation angle and/or ratio .

10. A method of image transmission, comprising a method according to any previous Claim.

11. An image transmission device, comprising a processor (500) that includes the following functions : a defining function for defining a plurality of saliency values of an image or set of images over multiple parameters; a calculation function for calculating entropy value for each of said saliency values over substantially each of said multiple parameters; a peak entropy identification function (560) for identifying any peak entropy value for said image or set of images based on said calculated entropy values; and a feature identification function identifying a feature in said image or set of images based on said peak entropy value.

12. The image transmission device according to Claim 11, further characterised by said plurality of parameters including scale 's', at ratio 'r', and angle 'a', wherein said entropy value is calculated for each permutation of the respective parameters 's', 'a', 'r'.

13. The image transmission device according to Claim 11 or Claim 12, further characterised by said calculation function performing a probability density function to determine entropy values for each of said saliency values over substantially each of said multiple parameters.

14. A mobile or portable telephone incorporating the image transmission device according to any one of claims 11 to 13.

15. A storage medium storing processor-implementable instructions for controlling a processor to carry out the method any of claims 1 to 10.