WO2014175477A1

WO2014175477A1 - Apparatus and method for processing image

Info

Publication number: WO2014175477A1
Application number: PCT/KR2013/003492
Authority: WO
Inventors: Shounan An
Original assignee: Lg Electronics Inc.
Priority date: 2013-04-24
Filing date: 2013-04-24
Publication date: 2014-10-30

Abstract

An image processing apparatus filters an input image by using a plurality of filter intensities on the basis of a guide image to generate a plurality of filtered images. The plurality of filtered images corresponds to the plurality of filter intensities, respectively. The image processing apparatus generates a plurality of difference images on the basis of the plurality of the filtered images and determines characteristic points in the input image on the basis of the plurality of difference images.

Description

APPARATUS AND METHOD FOR PROCESSING IMAGE

The present disclosure relates to an apparatus and method for processing an image, and more particularly, to an apparatus and method for extracting characteristic points in an image.

Recently, in recognizing an object in an image or a video captured by using a camera or a smartphone, a method of extracting characteristic points from the image or the video to recognize the object is used. In particular, in the related art method, a Difference-of-Gaussian (DoG) based characteristic point extracting method is used in extracting characteristic points from the image and the video.

The DoG based characteristic point extracting method is based on a Gaussian function in filtering an image or a video.

However, this DoG based characteristic point extracting method performs characteristic point extraction well in a high contrast region of an image, but has limitation in extracting characteristic points in a low contrast region due to the nonpreservation of edges.

Embodiments provide an apparatus and method for processing an image which easily extracts characteristic points in a low contrast region of an image.

In one embodiment, an apparatus for processing an image, comprises: a plurality of filters for generating a plurality of filtered images respectively, wherein a plurality of filters corresponds to a plurality of filter intensities, each of the plurality of filters filters an input image on the basis of a guide image according to the corresponding filter intensity to generate a corresponding filtered image; a plurality of difference image generators for generating a plurality of difference images on the basis of the plurality of filtered images; and a characteristic point determining unit for determining characteristic points in the input image on the basis of the plurality of difference images.

In another embodiment, a method of processing an image comprising: filtering an input image by using a plurality of filter intensities on the basis of a guide image to generate a plurality of filtered images, wherein the plurality of filtered images corresponds to the plurality of filter intensities, respectively; generating a plurality of difference images on the basis of the plurality of the filtered images; and determining characteristic points in the input image on the basis of the plurality of difference images.

The details of one or more embodiments are set forth in the accompanying drawings and the description below. Other features will be apparent from the description and drawings, and from the claims.

Fig. 1 is a block diagram of a video comparison apparatus according to embodiments.

Fig. 2 is a block diagram of a characteristic point detector according to embodiments.

Fig. 3 is a block diagram of a characteristic point detector according to embodiments.

Fig. 4 illustrates W_k, assuming r=2.

Fig. 5 illustrates a portion of difference images according to embodiments.

Reference will now be made in detail to the embodiments of the present disclosure, examples of which are illustrated in the accompanying drawings.

An apparatus and method for processing an image according to an embodiment will be described in detail with reference to the accompanying drawings. The invention may, however, be embodied in many different forms and should not be construed as being limited to the embodiments set forth herein; rather, that alternate embodiments included in other retrogressive inventions or falling within the spirit and scope of the present disclosure can easily be derived through adding, altering, and changing, and will fully convey the concept of the invention to those skilled in the art.

Referring to Fig. 1, the video comparison apparatus 100 according to embodiments will be described.

The video comparison apparatus 100 according to embodiments includes a camera 110, a reference image storage 120, two characteristic point detectors 130, two descriptor colleting units 140, a matcher 150, and a matching use unit 160. The elements illustrated in Fig. 1 are not essential, and thus the video comparison apparatus 100 may include more or fewer elements than those in Fig. 1.

The video comparison apparatus 100 according to embodiments may obtain a comparison target image C through the camera 110. A video comparison apparatus 100 according to another embodiment may obtain a comparison target image C from a video file.

The reference image storage 120 stores a reference image R. This reference image R may be generated through the camera 110.

The two characteristic point detectors 130 respectively correspond to the comparison image C and the reference image R. The characteristic point detector 130 corresponding to the comparison target image C detects a plurality of characteristic points in the comparison target image C. The characteristic point detector 130 corresponding to the reference image R detects a plurality of characteristic points in the reference image R.

The two descriptor collectors 140 respectively correspond to the comparison image C and the reference image R. The descriptor collector 140 corresponding to the comparison target image C collects from the target image C a plurality of descriptors which respectively correspond to the plurality of characteristic points in the comparison target image C. The descriptor collector 140 corresponding to the reference image R collects from the reference image R a plurality of descriptors which respectively correspond to the plurality of characteristic points in the reference image R. At this time, the descriptor may include characteristic point coordinates, characteristic point color values, color values of pixels surrounding the characteristic points, and the like.

The matcher 150 extracts one or more matching points on the basis of similarities between the plurality of descriptors of the comparison target image C and the plurality of descriptors of the reference image R.

The matching use unit 160 may percept an object on the basis of the matching points. Specifically, the matching use unit 160 may compare the matching points extracted by the matcher 150 with information stored in a memory (not shown). In the memory, matching points of a predetermined object may be stored in advance. The matching use unit 160 compares the matching points extracted by the matcher 150 with the matching points of the predetermined object, stored in the memory. The matching use unit 160 may search the memory for matching points of an object, which are coincident with the matching points extracted by the matcher 150, and determine that the comparison target image C corresponds to which object image.

Fig. 2 is a block diagram of the characteristic point detector 130 according to embodiments.

Referring to Fig. 2, the characteristic point detector 130 according to embodiments will be described.

As described above, the characteristic point detector 130 receives the comparison target image C or the reference image R. Hereinafter, the characteristic point detector 130 corresponding to the comparison target image C will be described, but the characteristic point detector 130 corresponding to the reference image R may also have identical or similar structure to the characteristic point detector 130 corresponding to the comparison target image C.

The characteristic point detector 130 includes an input image obtaining unit 210, a guide image obtaining unit 220, a plurality of characteristic point outputting units 230, and a characteristic point collector 240. The elements illustrated in Fig. 2 are not essential, and thus the characteristic point detector 130 may include more or fewer elements than those in Fig. 2.

The input image obtaining unit 210 obtains an input image from the comparison target image C. The input image according to an embodiment may be identical to, or different from the comparison target image C.

The guide image obtaining unit 210 obtains a guide image I. In an embodiment, the guide image I may be identical to, or different from the input image P. In another embodiment, the video comparison apparatus 100 may include one or more cameras 110. When the video comparison apparatus 100 includes a plurality of cameras 110, any one camera 110 may capture the input image P and another camera 110 may capture the guide image I. In this case, the camera 110 capable of capturing the guide image I may have higher performance than the camera 110 capable of capturing the input image P.

The plurality of characteristic point outputting units 230 correspond to a plurality of image sizes. At this time, the plurality of image sizes may be different from each other. In addition, one of the plurality of image sizes may be the same as that of the input image P, and the rest of the plurality of image sizes may be smaller than that of the input image P. The plurality of image sizes may correspond to a plurality of magnifying powers which are equal to or smaller than 1.

Each of the characteristic point outputting units 230 includes two resizers 231 and a characteristic point extractor 233. The elements illustrated in Fig. 2 are not essential, and thus the characteristic point outputting unit 230 may include more or fewer elements than those in Fig. 2.

Two resizers 231 correspond respectively to the input image and the guide image. The resizer 231 corresponding to the input image resizes the input image size to a corresponding image size to generate a resized input image P_R. The resizer 231 corresponding to the guide image I resizes the guide image size to a corresponding image size to generate a resized guide image I_R. The characteristic point extractor 233 extracts a characteristic point group for a corresponding image size by using the resized input image P_R and the resized guide image I_R.

Like this, the plurality of characteristic point outputting units 230 output a plurality of characteristic points respectively corresponding to the plurality of image sizes.

The characteristic point collector 230 collects a plurality of characteristic point groups respectively corresponding to the plurality of image sizes to determine the plurality of characteristic points of the comparison target image C.

Hereinafter, referring to Fig. 3, the characteristic point extractor 233 according to an embodiment will be described.

Fig. 3 is a block diagram of the characteristic point extractor 233 according to the embodiment.

The characteristic point extractor 233 according to the embodiment includes a plurality of filters 310, a plurality of difference image generator 320, and a characteristic point determining unit 330. For example, the characteristic point extractor 233 may include N filters 310, (N-1) difference image generators 320, and one characteristic point determining unit 330. However, the elements illustrated in Fig. 3 are not essential, and thus the characteristic point extractor 233 may include more or fewer elements than those in Fig. 3.

The N filters 310 respectively correspond to N filter intensities, and each of the filters 310 filters the resized input image P_R on the basis of the resized guide image I_R according to the corresponding filter intensity, to generate filtered image. The N filters 310 generate N filtered images. The N filters 310 are arranged in an ascending order or in a descending order, according to the N filter intensities. The filters 310 will be described later.

The plurality of N filter intensities may be determined by combination of r (a window radius) and ε(a normalization parameter).

Table 1 shows the filter intensities according to combination of r and ε.

Table 1

Filter intensity	Window radius (r)	Normalization parameter (ε)
0	Output an input without change
1	2	0.1²
2	4	0.1²
3	8	0.1²
4	2	0.2²
5	4	0.2²
6	8	0.2²
7	2	0.4²
8	4	0.4²
9	8	0.4²

Hereinafter, referring to Fig. 4, the filer 310 will be described.

Fig. 4 shows Wk, assuming r=2.

The filter 310 filters the resized input image P_R on the basis of the resized guide image I_R.

The filter 310 performs filtering according to the following equation 1.

[equation 1]

Here, q_i denotes a filtered value of a pixel at position i in the resized input image P_R.

I_i denotes a pixel value at position i in the resized guide image I_R.

According to an embodiment, W_k may indicate a square window whose window radius is r, namely, whose side length is 2r. In addition, W_k includes a pixel at position i, and |W| denotes the number of pixels in W_k. Here, the number of windows W_k including a pixel at position i may be plural. According to an embodiment, the number of windows W_k including a pixel at position i may be the same as the number of pixels in W_k. This may be also applied to the resized guide image I_R. That is, the number of the resized guide images I_R may be the same as the number of pixels in W_k.

Fig. 4 shows W_k, assuming r=2.

According to Fig. 4, when r=2, the number of W_k is total 16. That is, different W_ks, W₁ to W₁₆, exist. Thus, according to Equation 1, q_i is a value that (a_k * I_i + b_k) for all W_k are summed and then divided by |W|.

Equation 1 may be represented as equation 2.

[Equation 2]

Here, a_k may be represented as equation 3.

[Equation 3]

Here, u_k denotes an average of values of all pixels included in k-th window W_k in the resized guide image I_R. σ_k ² denotes variance of values of all pixels included in k-th window W_k in the resized guide image IR. ε is a normalization parameter for preventing a_k from getting too great.

According to equation 3, a_k may be obtained by summing values of all pixels included in a predetermined window W_k in the resized guide image I_R, diving the summed result by |W|, namely, the number of pixels in W_k, subtracting a multiplication value of u_k by P_k from the divided result, and dividing the subtracted result by an addition value of σ_k ² and ε.

P_k in equation 3 may be represented as equation 4.

[equation 4]

According to equation 4, P_k is a value obtained by summing values of all pixels included in a predetermined window W_k in the resized input image P_R and dividing the summed value by |W|, namely, the number of pixels in W_k.

In addition, b_k may be represented as equation 5.

[equation 5]

According to equation 5, b_k is a value obtained by subtracting a multiplication value a_k by u_k from P_k.

The filter 310 may obtain the filtered input image P_p by obtaining filtered values q_i for all the pixels in the resized input image P_R.

Referring back to Fig. 3, when the N filters 310 are arranged in an ascending order or in a descending order, k-th difference image generator 320 corresponds to k-th filter 310 (where k=1, …, N-1). That is, a first filter 310 corresponds to a first difference image generator 320, a second filter 310 corresponds to a second difference image generator 320, and an (N-1)-th filter 310 corresponds to an (N-1)-th difference image generator 320. Each difference image generator 320 generates a difference image between a filtered image from a corresponding filter 310 and a filtered image from a filter 310 following the corresponding filter 310. For example, the first difference image generator 320 receives images from the first and second filters 310, the second difference image generator 320 receives images from the second filter 310 and a third filter 310, and the (N-1)th difference image generator 310 receives images from the (N-1)th filter 310 and an N-th filter 310.

The difference image generator 320 generates a difference image by using the received two filtered images, namely, input images P_p filtered by a filter 310 corresponding to the difference image generator 320 and a filter 310 following the filter 310 corresponding the difference image generator 320. The difference image generator 320 obtains a difference between the two filtered input images P_p. Namely, a filtered input image Pp by the filter 310 following the filter 310 corresponding to the difference image generator 320 is subtracted from a filtered input image Pp by the filter 310 corresponding to the difference image generator 320. As a result, each of the plurality of difference image generators 320 generates one difference image.

The characteristic point determining unit 330 receives a plurality of difference images output from the plurality of difference image generator 320 to determine presence of the characteristic points, and then output the result.

The characteristic point determining unit 330 compares a portion or all of the (N-1) difference images with each other to output the characteristic points.

Hereinafter, referring to Fig. 5, the characteristic point determining unit will be described.

Fig. 5 illustrates a portion of the difference images according to embodiments.

In this document, pixels at predetermined positions in three difference images will be exemplified.

The characteristic point determining unit 330 compares a value of a pixel 510 at a position m, desired to confirm presence of a characteristic point, with values of pixels surrounding the pixel 510. And the characteristic point determining unit 330 determines whether the value of the pixel 510 is greater or smaller than any surrounding pixel values.

When the value of the pixel 510 is neither greater nor smaller than any of the surrounding pixel values, the characteristic point determining unit 330 does not determine the pixel 510 to be a characteristic point. When the value of the pixel 510 is greater or smaller than any of the surrounding pixel values, the characteristic point determining unit 330 compares the value of the pixel 510 with values of a pixel at position m and surrounding pixels in a previous difference image 502.

When the value of the pixel 510 is neither greater nor smaller than any values of the pixel at position m and the surrounding pixels in the previous difference image 502, the characteristic point determining unit 330 does not determine the pixel 510 to be a characteristic point. When the value of the pixel 510 is greater or smaller than any values of the pixel at position m and the surrounding pixels in the previous difference image 502, the characteristic point determining unit 330 compares again the value of the pixel 510 with values of a pixel at position m and surrounding pixels in a next difference image 503. When the value of the pixel 510 is neither greater nor smaller than any values of the pixel at position m and the surrounding pixels in the next difference image 503, the characteristic point determining unit 330 does not determine the pixel 510 to be a characteristic point. When the value of the pixel 510 is greater or smaller than any values of the pixel at position m and the surrounding pixels in the next difference image 503, the characteristic point determining unit 330 compares again the value of the pixel 510 with values of a pixel at position m and surrounding pixels in a difference image prior to the previous difference image 502.

Repeating in this way, the characteristic point determining unit 330 compares the value of the pixel 510 with values of pixels at position m and surrounding pixels in all the difference images. As a result, when the value of the pixel 510 is greater or smaller than any values of the pixels at position m and the surrounding pixels in all the difference images, the characteristic point determining unit 330 determines the pixel 510 to be a characteristic point.

In this way, the characteristic point determining unit 330 determines whether a pixel is a characteristic point for pixels at all positions in the difference images to detect a plurality of characteristic points in the difference images.

In the video comparison apparatus 100 as described above, the configuration and the method thereof according to the embodiments are not limitedly applied, but a portion or all of the embodiments can selectively combined to achieve various modifications.

According to the embodiments, characteristic points can be easily extracted in a low contrast region on the basis of a combination of r (a window radius) and ε(a normalization parameter) in an image.

Although embodiments have been described with reference to a number of illustrative embodiments thereof, it should be understood that numerous other modifications and embodiments can be devised by those skilled in the art that will fall within the spirit and scope of the principles of this disclosure. More particularly, various variations and modifications are possible in the component parts and/or arrangements of the subject combination arrangement within the scope of the disclosure, the drawings and the appended claims. In addition to variations and modifications in the component parts and/or arrangements, alternative uses will also be apparent to those skilled in the art.

Claims

1. An apparatus for processing an image, comprising:

a plurality of filters for generating a plurality of filtered images respectively, wherein a plurality of filters corresponds to a plurality of filter intensities, each of the plurality of filters filters an input image on the basis of a guide image according to the corresponding filter intensity to generate a corresponding filtered image;

a plurality of difference image generators for generating a plurality of difference images on the basis of the plurality of filtered images; and

a characteristic point determining unit for determining characteristic points in the input image on the basis of the plurality of difference images.

The apparatus according to claim 1, wherein the plurality of filter intensities correspond to a plurality of window radiuses, respectively.

The apparatus according to claim 1, wherein the plurality of filter intensities correspond to a plurality of normalization parameters, respectively.

The apparatus according to claim 1, wherein the plurality of filter intensities correspond to a plurality of combinations of one or more normalization parameters and one or more window radiuses, respectively.

The apparatus according to claim 1, wherein the characteristic point determining unit compares a value of any one pixel in a current difference image among the plurality of difference images with values of surrounding pixels of the one pixel, to determine whether the one pixel is a characteristic point.

The apparatus according to claim 1, wherein the characteristic point determining unit determines that any one pixel in a current difference image among the plurality of difference images is not a characteristic point, when a value of the one pixel is neither greater nor smaller than any values of surrounding pixels of the one pixel.

The apparatus according to claim 1, further comprising a camera for capturing the input image or the guide image.

The apparatus according to claim 1, wherein the number of the plurality of difference image generators is smaller than the number of the plurality of filters.

The apparatus according to claim 1, wherein each of the difference image generators generates each of the difference images as a difference of a first filtered image received from a filter corresponding to each of the difference image generators, and a second filtered image received from a filter following the filter corresponding to each of the difference image generators.

The apparatus according claim 1, further comprising a matching use unit determining to which object's image the input image correspond on the basis of the characteristic points.

A method of processing an image comprising:

filtering an input image by using a plurality of filter intensities on the basis of a guide image to generate a plurality of filtered images, wherein the plurality of filtered images corresponds to the plurality of filter intensities, respectively;

generating a plurality of difference images on the basis of the plurality of the filtered images; and

determining characteristic points in the input image on the basis of the plurality of difference images.

The method according to claim 11, wherein the plurality of filter intensities correspond to a plurality of window radiuses, respectively.

The method according to claim 11, wherein the plurality of filter intensities correspond to a plurality of normalization parameters, respectively.

The method according to claim 11, wherein the plurality of filter intensities correspond to a plurality of combinations of one or more normalization parameters, and one or more window radiuses, respectively.

The method according to claim 11, wherein the determining of characteristic points further comprises comparing a value of any one pixel in a current difference image among the plurality of difference images with values of surrounding pixels of the one pixel, to determine whether the one pixel is a characteristic point.

The method according to claim 11, wherein the determining of characteristic points further comprises;

comparing a value of any one pixel in a current difference image among the plurality of difference images with values of surrounding pixels of the one pixel; and

determining the one pixel in the current difference image not to be a characteristic point, when the value of the one pixel is neither greater nor smaller than any values of the surrounding pixels, as the compared result.

The method according to claim 11, further comprising:

determining to which object's image the input image correspond on the basis of the characteristic points.

The method according to claim 17, wherein the determining of the input image comprises:

detecting matching points on the basis of the characteristic points; and

searching a memory for matching points of an object, the matching points corresponding to the detected matching points.