CN101263514A - Mutual-rank similarity-space for navigating, visualising and clustering in image databases - Google Patents

Mutual-rank similarity-space for navigating, visualising and clustering in image databases Download PDF

Info

Publication number
CN101263514A
CN101263514A CNA2006800332246A CN200680033224A CN101263514A CN 101263514 A CN101263514 A CN 101263514A CN A2006800332246 A CNA2006800332246 A CN A2006800332246A CN 200680033224 A CN200680033224 A CN 200680033224A CN 101263514 A CN101263514 A CN 101263514A
Authority
CN
China
Prior art keywords
data item
similarity
described method
image
matrix
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CNA2006800332246A
Other languages
Chinese (zh)
Inventor
罗伯特·J·奥卡拉汉
米罗斯瓦夫·博贝尔
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Mitsubishi Electric Corp
Mitsubishi Electric Information Technology Corp
Original Assignee
Mitsubishi Electric Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Mitsubishi Electric Corp filed Critical Mitsubishi Electric Corp
Publication of CN101263514A publication Critical patent/CN101263514A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
    • G06V10/443—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components by matching or filtering
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
    • G06F16/58—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
    • G06F16/583—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content
    • G06F16/5838—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content using colour

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Library & Information Science (AREA)
  • Physics & Mathematics (AREA)
  • Databases & Information Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Multimedia (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
  • Processing Or Creating Images (AREA)
  • Image Analysis (AREA)

Abstract

A method of representing a group of data items comprises, for each of a plurality of data items in the group, determining the similarity between said data item and each of a plurality of other data items in the group, assigning a rank to each pair on the basis of similarity, wherein the ranked similarity values for each of said plurality of data items are associated to reflect the overall relative similarities of data items in the group.

Description

In image data base, navigate, the mutual order similarity space of visual and cluster
Technical field
The present invention relates to effective expression of data item, particularly image collection.It is specifically related to extract the navigation in the image collection of mathematical description of picture material because in such database, can use automatic algorithms analysis, organize, search and browsing data.Digital image set is just becoming more prevalent in specialty and consumer field.Technical progress has made and has caught, storage and transmission of digital image be than cheap more and easily in the past.This makes needs new method to make the user carry out effectively mutual with such set.
Background technology
The method of query image database is known.For example, US-B-6240423 discloses a kind of such method, and wherein Cha Xun result is according to drawing based on the images match in zone with based on the combination of the images match on border.
For the user who begins to learn, being difficult to find intuitively especially, method gets involved such large-capacity data.For example, most consumers is familiar with in the mode of entity their papery photographic prints being organized into photograph album, but it is, this palpable no longer possible alternately for the set of the digital photos in the storer of its personal computer, camera phone or digital camera.At first, be used for the electronic method that each set is navigated paid close attention to this physics, palpable filing experienced carrying out emulation.
People such as Wang (US6,028,603) provide a kind of form with similar photograph album to present the means of image, its by one page of the information with the layout that has defined this page epigraph or more multipage form.Order and layout can change by user's drag-and-drop operation.
Another kind of simple method is from Gargi (US 2002/0140746), and he presents image in the mode of piling up demonstration.When moving past, mouse manifests image.For the user, this folded photo that is similar to from the table picks up.
When the user manually organizes its image collection, some importance are arranged usually for structure.In other words, the layout of its photograph album has certain " meaning " to them.This may relate to the incident related with image, personage or mood, or may for example tell about story.Some electronic navigation instruments have been attempted by allowing the user that image is marked or dividing into groups to simulate and utilize this structure.Some even trial are advised automatically to classification or grouping.
People such as Mojsilovic (US 2003/0123737) disclose a kind of be used for based on the semantic feature that obtains from perception experiment browse, search for, the method for inquiry and visualization digital image collection.It has defined a kind of being used for based on this " feature set fully " relatively measure of the semantic similarity of two images, and a kind of to each image appointment semantic category method for distinguishing.
People such as Rosenzweig (US 2002/0075322) proposed a kind of that be used to browse and retrieve, based on the graphic user interface (GUI) of time shaft, wherein respectively organize image by the size with the group the icon representation that is in proportion.The user activates icon and operates its hierarchy system, and this activation action triggers is carried out the another rank of refinement to the first order.This system can also be to being stored in the image file, for example defining the various metadata of position, personage, incident and decode and obtain described (monopolizing mutually) each group.By activating the image that icon display comprised in the final level/view.
People such as Stavely (US 2003/0086012) have described the another kind of user interface that is used for picture browsing.Utilize simple combination vertical and level input control, its by make each group have " preferably " image allow to organize interior and organize between picture browsing.
Anderson (US 6,538,698) describes a kind of search and browing system of each image being classified and dividing into groups by various criteria for classifications that depend in detail.
Although it is mutual that digital library makes the user can not carry out the physics that photographic prints allowed, the new function that it allows usefulness particularly relates to the function of the automatic analysis of content.Can extract " feature " of token image in many ways.(for example) shape, texture and the color that present in the image can all be described by numerical characteristic, and this makes to come image is compared and index by these attributes.
Above-mentioned automatic classification is assigned an example of such function that only is its permission.Can compare the also feasible structure that may catch and represent entire database to image quantitatively.This is an attracting design, because when the user sets about organizing its photograph album, they often attempt to apply structure.If the image in the set has structure intuitively, it may be the useful part that the user begins.Can also make and search and browse more efficiently utilize or revise it because the user can be known this structure.
Method of the present invention is found the structure of image data base automatically by the right similarity of analysis image.Can utilize this structure in many ways then, comprise it is expressed as the X-Y scheme that the user can interactive navigation.
From the known multiple disposal route that data is projected to lower dimensional space from higher dimensional space of each document, (for example no matter whether be used for expression purely, principal component analysis (PCA) (Principal ComponentAnalysis, PCA)) classification (for example, linear discriminant analysis (Linear DiscriminantAnalysis, LDA)) or visual (for example, laplacian eigenmaps (Laplaciean Eigenmap), multidimensional scaling (MultiDimensional Scaling, MDS)), local projection (the Locality PreservingProjection that keeps, LPP) and self-organization mapping (Self-Organising Map, SOM).In this article, take by matrix relatively is subjected to special concern as the algorithm of importing.For many features, can not point one common its that numerical data is interpreted as in the Ka Dier space (Cartesian space) simply only be suitable for using specific distance measures to compare.Therefore, the algorithm of directly vector data being operated is more useless for our purpose.Technology based on similarity comprises MDS, SOM and laplacian eigenmaps.These have all created the low-dimensional projection of data, and this has reflected measuring similarity separately (wherein, " the best " determined by some cost functions) best.
Rising (US 6,721,759) has described a kind of processing that is used for the classification MDS database of image.This is based on the similarity of use characteristic detectors measure image collection, and the inquiry and the method for new construction more.In order to construct this representation, carry out MDS in top subclass (being called the reference mark) to image.Selected so that the convex closure at approximate number strong point of these points-promptly, the intactly variation that presents in the presentation video.The position initialization of remaining point with respect to the reference mark, and whole set is divided into a plurality of " nodes ", and each node is represented a subclass.Then each node is carried out MDS, arrange with the image of refinement in it.This method has been utilized the efficient aspect of classification tree, to reduce the computation burden (iteration optimization algorithms) of calculating MDS.
The method of Trepess and Thorpe (EP 1 426 882) uses SOM to create the mapping representation of data.Construct hierarchical clustering then so that navigation and demonstration.Can distinguish each cluster by the various characterization information (mark) that obtain automatically from cluster structures.This is used mainly for text document, but this method self is general.Say that in some sense it has reflected the work of Rising: this method carry out mapping then, and at first (globally) calculating mapping of Trepess and Thorpe uses then it to construct level with each other data clusters of level.
Jain and Santini (US 6,121,969) have proposed a kind of Query Result in the image data base to be carried out visualization method.It is display result in three dimensions, and this three-dimensional axle is selected arbitrarily from the set of N dimension.These are corresponding to the various measuring similarities between query image and the database images.Proposed by passing the vision guided navigation in this space, given the user dynamically and the experience of vision.This method because it is not the similarity structure of attempting optimally catching image collection, but provides the similarity of set at the query image of user's selection with two examples are different before.From a plurality of measurements of this similarity, rather than produce a plurality of dimensions from a plurality of mutual similarity of image.
As seeing soon, one of key design that the present invention is contained is: order (rank) structure (rather than similarity structure) is the key property that will keep when performance and organized image databases.Use order to guide cluster for example at people such as Novak (J.Novak, P.Raghavan and A.Tomkins, " Anti-aliasing on the web ", Proc.InternationalWorld Wide Web Conference, pages 30-39,2004) and Fang, (F.M.Fang, " AnAnalytical Study on Image Databases ", Master ' s Thesis, MIT, in June, 1997) mention tout court in the document.These two treatises all the mutual order of object i and j (mutual rank) be defined as i with respect to the order of j and j with respect to the order of i and.
Yet, also do not utilize whole potentiality of the measurement of the type.Especially, aforementioned works has only been considered cluster, handles each then by the comparison to mutual order only isolatedly, makes decision in " greedy (greedy) " mode in the part.The measurement based on order with the overall situation of novelty comes the indication method, is proved to be to manifest the powerful measure of structure.
Each method of prior art all has defective to be solved by this invention:
Simple browsing method had not both utilized the structure of image collection not represent it well yet.
Method based on classification may partly address this problem.They begin to utilize available characteristic information, but because discrete, often be the appointment of exclusive class label and dumb.Automatically classification also is to be difficult to realize famously reliably.
Similarity can be considered and represent to more complicated method, but up to the present, only catch absolute comparison.This method will be caught relativeness between each image in the scope of whole set.
Also do not have in the prior art to calculate and in expression on the embedding time and the design of the joint measure of visual similarity.Binding time and outward appearance provide favourable attribute to visual by this way, comprise making the user be easier to explain consequent arrangement.
Summary of the invention
Each side of the present invention has been proposed in claims.The present invention relates to data item, use device is handled and the corresponding signal of data item.The present invention relates generally to image.The further details of application of the present invention can find in co-pending European application No.05255033.
One aspect of the present invention is: when the structure of the set of presentation video compactly, and relativeness but not the absolute measure of similarity is the key property that will keep.Therefore defined mutual rank matrices as the suitable mode of the data structure being encoded with the form that can carry out mathematical analysis.Item in this matrix is illustrated in the interior right comparison of image of scope of wideer set.Mathematical analysis can comprise based on this information image divide into groups (cluster) maybe is projected to this information the compact representation of the most important aspect that has kept this structure.
Second related aspect is: when whole but not catch this structure when considering described mutual rank measurements most effectively isolatedly.That is when, the angle of non local (by to) is handled when taking the overall situation.
The third aspect is: useful on an equal basis in the context of time and visual information each image in determining set.This means that described relatively the time in tolerance, the time is not taken as independent or amount independently.Therefore resulting cluster or visual representation can represented vision similarity jointly and form in the space of property around.
Description of drawings
Embodiments of the invention will be described with reference to the drawings, in the accompanying drawing:
Fig. 1 is the process flow diagram of first embodiment;
Fig. 2 is the process flow diagram of second embodiment;
Fig. 3 is the process flow diagram of the 3rd embodiment;
Fig. 4 shows browsing apparatus.
Embodiment
In the environment of image retrieval task, common method be provide by its similarity (to a certain extent) ordering, at the sorted lists of Query Result.This has caught the image in the database and the relation of query image well.Its design is, wishes that the user will find interested image, incoherent image to be pulled to the bottom near the top of sorted lists.The present invention with this concept extension to attempt to catch and visible database in image between institute's correlate.
An embodiment of this method is a kind of system, this systematic analysis image, its feature of comparison, produces one group of mutual rank matrices, calculates the mapping expression with its combination and by finding the solution eigenvalue problem.This process illustrates in the process flow diagram of Fig. 1.
Another embodiment shown in Fig. 2.Here, the combination step of in first embodiment mutual rank matrices being carried out is carried out characteristic similarity now.Fig. 3 illustrates the 3rd embodiment, and the stage that some of them are combined in early carries out, and remainder was carried out in the later stage.Select when to conceive irrelevant according to various eigenwert fused data and the present invention.It is the details of specific implementation more exactly.As will be apparent to the skilled person in the art, this selection can be by determining such as the factor of complicacy, feature quantity (dimension) and its independent degree.At the remainder of this instructions, we concentrate on the order shown in Fig. 1 with being without loss of generality.
First step in this system is from image and any some descriptive characteristics of the meta-data extraction that is associated.This feature for example can be the visual descriptor of MPEG-7, this descriptor is described any other visual attributes of color, texture and structure attribute or image, as proposing in MPEG-7 standard ISO/IEC 15938-3 " Information technology-Multimedia content descriptioninterface-Part 3:Visual ".For example, the color descriptor of first image can be illustrated in the position of the average color of image in the given color space.The corresponding color descriptor of second image so can with the color descriptor of first image relatively, be given in the standoff distance in the given color space, and the qualitative assessment of similarity between first and second images therefore.
In other words, for example, with simple range observation or similarity value S, (c1) (a2, b2 c2) compare, wherein with second average color for a1, b1 with first average color
S＝[a 1-a 2]+[b 1-b 2]+[c 1-c 2]
Time is the most important element of metadata, but other information no matter be that the user provides or generation automatically, can be incorporated into.Can be with example this and other mode assembly times and visual information at people such as Cooper " Temporal event clustering for digital photocollections " (Proc.11 ThACM International conference on Multimedia, PP.364-373,2003) in find.
To unique being limited in of descriptive characteristics, these features allow an image and another to compare to produce the similarity value.US-B-6240423 discloses the example that the similarity value between the image is calculated.The measuring similarity that the MPEG-7 standard self has defined descriptor and has been associated.Yet, preferably, these features can also catch picture material some to the significant quality of people.
Second step is to use the cross-matched of descriptive characteristics carries out image.Many examples of descriptive characteristics and its measuring similarity that is associated be known-for example, referring to EP-A-1173827, EP-A-1183624, GB 2351826, GB 2352075, GB 2352076.
Similarly, there are the many technique known that are used to obtain descriptive scalar or vector value (that is, proper vector), can utilize a lot of technique known that this value is compared to determine the similarity of this scalar or vector value, as simple distance metric.
This produces by the matrix S to similarity each feature F FEvery S F(i is at described feature F, similarity between image i and image j j).Therefore matrix is symmetry typically.If for example use the asymmetric tolerance of similarity, then each matrix can not be symmetrical.All images can be included in cross-matched or the subclass.For example, image is cluster in advance, and only handles an image from each cluster, to reduce complexity and redundancy.This can realize with any technology in a plurality of prior art algorithms (for example, k arest neighbors (k-Nearest Neighbours), merging fusion (agglomerativemerging) etc.).
Third step is with similarity matrix S FBe converted to rank matrices R FReplace the similarity value to handle every row independently with some numerical sequences of arranging in order (rank ordinal value).In other words, for each i, for example use N (wherein, N is the quantity of image in the set) to replace maximum similarity S F(i, j), second largest replaces with N-1, and the third-largest replaces with N-2, or the like.After this step, matrix is symmetry no longer, because image i is inequality with respect to the order of i with respect to order and the j of j.The spinoff of this step is that we have calculated the result for retrieval for any image of inquiry in advance.Notice that this is not the unique method that keeps order ordinal number information.Generally speaking, this step can be considered as data, nonlinear, the monotonic transformation of depending on of similarity.Any such conversion can be considered as within the scope of the invention.
The further processing of rank matrices is favourable, although not necessarily.For example, can threshold application remove that deceptive information-for a lot of features, the rank values that surpasses some cut offs becomes nonsensical: image is simple " dissmilarity ", and to keep the rank values that is reducing be nonsensical.Yet the time is not a feature of this situation.Mistiming is consistent to all images with order, and therefore, the rank matrices of this feature does not have threshold value typically.
The 4th step is to make the rank matrices symmetrization for each feature.Any linearity that rank matrices is operated or non-linear, algebraically or statistical function can be used for this purpose.In one embodiment, with rank matrices and its transposition addition, provide an embodiment of mutual rank matrices:
M F＝R F+R F T
In this matrix, in the wideer scope of image collection, each is encoded to the relative similarity between image i and the j.Note M FBe symmetrical.Suitably another example of symmetrization is to select maximal value simply:
M F(i，j)＝max{R F(i，j)，R F(j，i)}
The 5th step is with matrix M FBe combined as the single overall matrix M of mutual rank scores.Exist many possible methods to realize it.In one embodiment, M FBe weighted and sue for peace.This system can comprise the means that some determine weight, and perhaps weight can be fixed in design.When will the commitment in system feature being made up, the various combined methods (discussed in front and by Fig. 2 and 3 illustrations) of wide region can be arranged equally.
In this stage,, can analyze by a plurality of prior art algorithms that are used for cluster and/or expression as matrix M about the affluent resources of the information of database structure.For example, it is right to incorporate the low image of arranging in order mutually of existence iteratively in merging cluster (agglomerative clustering) process.
More useful, matrix M can be analyzed in " overall situation " mode, so that consider some (or perhaps being all) rank measurements mutually simultaneously.This has reduced at the susceptibility of representing described in the single metric (matrix entries) noise, and has caught the bulk properties of data better.The spectral clustering method of knowing from document is an example of such processing, is suitable to the non-local method that it will be apparent to one skilled in the art that any other still.
In a preferred embodiment, rank matrices is embedded in the lower dimensional space by the laplacian eigenmaps method mutually.Be visual purpose, dimension is preferably two, but can be more or less.Perhaps, any amount of dimension can be used for cluster.Additive method also can be carried out embedding.The laplacian eigenmaps method attempts image is embedded in the space as point, makes every among the corresponding M of distance in the space.That is, the image with big rank values mutually is to close to each other, and it is far away to have an image distance of little mutual rank values.
So can obtain following equation as eigenvalue problem:
(D-M)x＝λDx
Wherein D is the diagonal matrix that forms by the row summation to M:
D ( i , i ) = Σ j M ( i , j )
Finding the solution of equation draws N proper vector, x, and this proper vector is the coordinate of image in the similarity space of arranging in order mutually.The importance of each vector (dimension) in catching the structure of set is indicated by corresponding eigenwert.This makes can select several least important dimensions to carry out visual, navigation and cluster.
The example of the map image of one group of data item in 2 dimension spaces that the said method of use shown in Fig. 4 obtains.More specifically, Fig. 4 shows the symbolic representation space on the display 120, and wherein corresponding each data item of symbol (point) here is an image.
The arrangement of symbol in display (that is, relative position and distance between the symbol) has reflected the similarity one or more characteristics (as average color), the corresponding data item based on data item.
The user can use pointing device (pointing device) 130 moving cursor 250 on representation space 10.According to the position of cursor, show one or more images (thumbnail) 270 for the proximity of cursor based on each symbol 260.Its further details is described in our co-pending European application No.05255033 incorporated herein by reference (being entitled as " Method and apparatus for accessing data using a symbolic representationspace ") with relevant method and apparatus.
Each is discussed below improves and substitutes.
When calculating mutual rank matrices, can select the subclass of image.This has reduced the size of matrix and has reduced computation burden.And then hope determines not have the position of image in output region of appearance in initial subset.These can be the remainder of more big collection or the new images of interpolation.According to the foregoing description, with needs add extra row and column in mutual rank matrices and revise existing because when the new image of appearance, the relative order of image will change.Then, mapping will be recomputated fully.Yet, can be similar to and the position of not revising conventional images in the output region this process.People such as Bengio (Y.Bengio, P.Vincent, J.-F.Paiement, O.Delalleau, M.Ouimet and N.Le Roux, " Spectral Clustering and Kernel PCA are LearningEigenfunctions ", Technical Report 1239, D é partement d ' Informatique etRecherche Op é rationnelle, Centre de Recherches Math é matiques, Universitede Montr é al) provided such method, be used for adding extra point to laplacian eigenmaps, with new data projection on the dimension that provides by original decomposition.This is with effective realization in the mutual order similarity space that promotes to owe to sample.
Secondly, the structure of mathematical framework is that it is easy to the imagination and incorporates extraneous information into expression.For example, can use user comment or other label informations to create different expressions (by for example LDA or generalized discriminant analysis (Generalized Discriminant Analysis, GDA)).These will represent better between the tagged class and within structure and relation.It can also be used for the classification appointment that the new images of database is added in suggestion to.This improvement is only to mathematical analysis-rank matrix construction remains unchanged mutually.The output of this improvement system (embedding) will comprise the combined information about vision between the image and time relationship, with and generic attribute.
Any set of that the user may wish to navigate, image or video (typically via key frame (key-frame) or other) is subjected to the influence of this method.Similarly, database records/data items can not belong to image and vision similarity is measured, but any other territory, as audio clips (audio clip) and corresponding measuring similarity.For example, the MPEG-7 standard proposes to be used for the descriptor (ISO/IEC 15938-4 " Information technology-Multimedia content descriptioninterface-Part 4:Audio ") of audio frequency.The audio metadata that can compare two montages is to provide quantitative measuring similarity.As long as the suitable tolerance of similarity is arranged, just can handle to text document.The method that is used to measure the text document similarity is open by people such as Novak (as above).The technical skill that has had this field, (Latent Semantic Indexing, LSI), this is a method well known in the art as potential semantic indexing.Being used to extract the description value of image data item in addition and being used for these description values of comparison is known with the various technology that obtain measuring similarity, will no longer describe in further detail at this.
The invention is not restricted to any specific description value or measuring similarity, and can use any suitable description value or measuring similarity, for example described in the prior or above-mentioned.As example, descriptive characteristics can be color value and corresponding measuring similarity (for example described in the EP-A-1173827), or contour of object and corresponding measuring similarity (for example described in GB2351826 or the GB 2352075) purely.
In this manual, " image " speech is used to describe elementary area, comprise aftertreatment, as filtering, change resolution, up-sampling (upsampling), down-sampling (downsampling), but this speech also is applicable to other similar terms, as the subelement of frame, field, picture or image, frame or zone etc.Term pixel and block of pixels or organize and to exchange use mutually when suitable.In this manual, a zone of term " image " meaning entire image or image is except obviously visible from context.Similarly, entire image can be meaned in the zone of image.Image comprise frame or, and relate to still image and, or the image in the relevant image sets such as the image in the image sequence of film or video.
Image can be gray scale or coloured image, or the multispectral image of other types (for example, IR, UV) or other electromagnetic image or acoustic picture etc.
Term " selecting arrangement " for example can be meant equipment control, that be used to select by the user, as comprises the controller of navigation and selector button, and/or such as representing by the controller on the display of pointer or cursor realization.
The present invention is preferably by handling the data item represented with electronic form and realizing by utilizing appropriate device to handle electric signal.The present invention can for example utilize appropriate software and/or hardware modifications to realize in computer system.For example, the present invention can utilize and have control or treating apparatus (as processor or opertaing device), the data storage device (as storer, magnetic store, CD, DVD etc.), data output device (as display or monitor or printer), data input device (as keyboard) and the image-input device (as scanner) that comprise image memory device or these assemblies with the realizations such as computing machine of any combination of optional feature.Each side of the present invention can provide with software and/or example, in hardware, maybe can provide special-purpose device or special-purpose module, as chip.In devices in accordance with embodiments of the present invention, for example can remotely provide the assembly of system on the internet from other assemblies.

Claims (42)

  1. One kind the expression one group of data item method, this method comprises: in a plurality of data item in this group each, determine the similarity between each data item in a plurality of other data item in described data item and this group, and based on similarity is that each is arranged in order to specifying, and the ordering similarity value of each data item in wherein said a plurality of data item is associated to reflect the overall relative similarity of data item in this group.
  2. 2. method of representing one group of data item based on the overall ordering relative similarity between the data item in the group.
  3. 3. method as claimed in claim 2, described method comprises: by the similarity between specified data item and a plurality of other data item, and determine each data item at least two excessive data items and the similarity between a plurality of other data item, described similarity value is sorted to determine the ordering relative similarity of data item in described group, and based on the similarity of described at least two data item being utilized described overall order similarity value.
  4. 4. as the described method of arbitrary aforementioned claim, wherein said ordering similarity value is arranged to reflect the array of the described overall relative similarity of data item in described group.
  5. 5. as the described method of arbitrary aforementioned claim, described method comprises and obtains matrix array that the item in the wherein said matrix is corresponding to the ordering similarity value between the data item.
  6. 6. method as claimed in claim 5, wherein the matrix entries at i row and the capable place of j is corresponding to the ordering similarity value of i data item and j data item.
  7. 7. as the described method of arbitrary aforementioned claim, described method comprises and obtains matrix array that wherein the item at i row and the capable place of j is corresponding to the similarity between i data item and the j data item.
  8. 8. method as claimed in claim 7, described method comprise by going or by row described similarity value being sorted.
  9. 9. as claim 5,6 or 8 described methods, described method comprises makes described rank matrices symmetrization.
  10. 10. as any one the described method in the claim 5 to 9, described method comprises the matrix entries setting threshold.
  11. 11. as the described method of arbitrary aforementioned claim, wherein based on the characteristic of data item and the similarity of specified data item.
  12. 12. method as claimed in claim 11, wherein the characteristic of data item comprises metadata, time or user data of distributing for example, and/or inherent characteristic, for example color, texture etc.
  13. 13. as the described method of arbitrary aforementioned claim, described method comprises at each characteristic in a plurality of characteristics determines similarity.
  14. 14. method as claimed in claim 13, described method comprises the combination of the similarity of using a plurality of characteristics.
  15. 15. as claim 13 or the described method of claim 14, described method service time and visual characteristic.
  16. 16. as any one the described method in the claim 13 to 15, described method comprises the rank matrices that draws and make up a plurality of characteristics.
  17. 17. as any one the described method in the claim 13 to 15, described method comprises the similarity matrix that draws and make up a plurality of characteristics.
  18. 18. as the described method of arbitrary aforementioned claim, described method for example comprises by selecting subclass, cluster or data item being owed sampling and data item is carried out pre-service.
  19. 19. a method of representing data item, described method comprise the similarity between the specified data item and described similarity are sorted, comprise and utilize the relative order of three or more data item to come together further to handle.
  20. 20. as the described method of arbitrary aforementioned claim, wherein said data item comprises image.
  21. 21. as the described method of arbitrary aforementioned claim, described method comprises the further processing of embedding such as data item, visual and cluster.
  22. 22. method as claimed in claim 21, described method comprise data item is mapped as based on the point in the space of overall ordering similarity value.
  23. 23. method as claimed in claim 22, described method comprise the lower dimensional space that data item is mapped to the expression dimension that for example is lower than described data item.
  24. 24. comprising, method as claimed in claim 23, described method be mapped to two-dimensional space.
  25. 25. as any one described method of claim 23 to 26, wherein in described space the distance between the mapping (enum) data item corresponding to the relative similarity of described data item.
  26. 26. as any one described method of claim 22 to 25, described method comprises the laplacian eigenmaps technology of using.
  27. 27. as the described method of arbitrary aforementioned claim, described method comprises the symbol of demonstration corresponding to data item.
  28. 28. method as claimed in claim 27, wherein in the display the relative arrangement of symbol and/or position corresponding to the relative similarity of each data item.
  29. 29. as the described method of arbitrary aforementioned claim, described method comprises new data item is added to or projects in the described overall expression.
  30. 30. a method of representing data item, described method comprise based on the similarity between time and the visual characteristic specified data item.
  31. 31. one kind to image between the method that sorts of similarity, this method comprises:
    Computed image between the similarity value; Structure similarity matrix, the element representation of this similarity matrix pursue the similarity value; And, the similarity matrix value calculates rank matrices by being analyzed.
  32. 32. also comprising by the row analysis of pursuing of similarity matrix value, method as claimed in claim 31, described method calculate described rank matrices.
  33. 33. as claim 31 or the described method of claim 32, described method also comprises makes described rank matrices symmetry.
  34. 34. method as claimed in claim 33 comprises described rank matrices and its transposition addition, or calculates with respect to the maximal value between the order element of principal diagonal symmetric offset spread.
  35. 35. as any one described method of claim 31 to 34, comprise that also the low-dimensional by described rank matrices embeds, described rank matrices carried out dimensionality reduction.
  36. 36. method as claimed in claim 35 wherein uses the laplacian eigenmaps technology to carry out described dimensionality reduction.
  37. 37. the method for the relation between the specified data Xiang Zuzhong data item comprises the method for arbitrary aforementioned claim.
  38. 38. for example embedding, visual, cluster, searching for and browse, the use of aforementioned any one described method.
  39. 39. an opertaing device, this opertaing device are programmed to carry out aforementioned any one method.
  40. 40. a device, this device are suitable for any one the described method in the enforcement of rights requirement 1 to 38.
  41. 41. a device, this device comprise the memory storage of processor, display device, selecting arrangement and the storing data item of any one the described method that is set to enforcement of rights requirement 1 to 38.
  42. 42. one kind is used for the computer program that enforcement of rights requires any one described method of 1 to 38, or stores the computer-readable recording medium of such computer program.
CNA2006800332246A 2005-08-15 2006-08-14 Mutual-rank similarity-space for navigating, visualising and clustering in image databases Pending CN101263514A (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
EP05255032.4 2005-08-15
EP05255032A EP1755067A1 (en) 2005-08-15 2005-08-15 Mutual-rank similarity-space for navigating, visualising and clustering in image databases

Publications (1)

Publication Number Publication Date
CN101263514A true CN101263514A (en) 2008-09-10

Family

ID=35447182

Family Applications (1)

Application Number Title Priority Date Filing Date
CNA2006800332246A Pending CN101263514A (en) 2005-08-15 2006-08-14 Mutual-rank similarity-space for navigating, visualising and clustering in image databases

Country Status (5)

Country Link
US (1) US20090150376A1 (en)
EP (2) EP1755067A1 (en)
JP (1) JP2009509215A (en)
CN (1) CN101263514A (en)
WO (1) WO2007020423A2 (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108764068A (en) * 2018-05-08 2018-11-06 北京大米科技有限公司 A kind of image-recognizing method and device

Families Citing this family (26)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8639028B2 (en) * 2006-03-30 2014-01-28 Adobe Systems Incorporated Automatic stacking based on time proximity and visual similarity
US8401312B2 (en) 2007-05-17 2013-03-19 Superfish Ltd. Method and a system for organizing an image database
JP5229744B2 (en) 2007-12-03 2013-07-03 国立大学法人北海道大学 Image classification device and image classification program
US8472705B2 (en) * 2008-05-23 2013-06-25 Yahoo! Inc. System, method, and apparatus for selecting one or more representative images
GB0901351D0 (en) * 2009-01-28 2009-03-11 Univ Dundee System and method for arranging items for display
US8209330B1 (en) * 2009-05-29 2012-06-26 Google Inc. Ordering image search results
US20100325552A1 (en) * 2009-06-19 2010-12-23 Sloo David H Media Asset Navigation Representations
AU2010282211B2 (en) * 2009-08-11 2016-09-08 Someones Group Intellectual Property Holdings Pty Ltd Method, system and controller for searching a database
US8352465B1 (en) 2009-09-03 2013-01-08 Google Inc. Grouping of image search results
US8774526B2 (en) * 2010-02-08 2014-07-08 Microsoft Corporation Intelligent image search results summarization and browsing
CN102193934B (en) * 2010-03-11 2013-05-29 株式会社理光 System and method for searching representative image of image set
US8724910B1 (en) 2010-08-31 2014-05-13 Google Inc. Selection of representative images
KR20120028491A (en) * 2010-09-15 2012-03-23 삼성전자주식회사 Device and method for managing image data
US20120294540A1 (en) * 2011-05-17 2012-11-22 Microsoft Corporation Rank order-based image clustering
US20130080950A1 (en) * 2011-09-27 2013-03-28 International Business Machines Corporation Incrementally self-organizing workspace
US8572107B2 (en) * 2011-12-09 2013-10-29 International Business Machines Corporation Identifying inconsistencies in object similarities from multiple information sources
CN102867027A (en) * 2012-08-28 2013-01-09 北京邮电大学 Image data structure protection-based embedded dimension reduction method
US8897556B2 (en) 2012-12-17 2014-11-25 Adobe Systems Incorporated Photo chapters organization
US8983150B2 (en) 2012-12-17 2015-03-17 Adobe Systems Incorporated Photo importance determination
US10353942B2 (en) * 2012-12-19 2019-07-16 Oath Inc. Method and system for storytelling on a computing device via user editing
US9092818B2 (en) 2013-01-31 2015-07-28 Wal-Mart Stores, Inc. Method and system for answering a query from a consumer in a retail store
US9677886B2 (en) * 2013-02-10 2017-06-13 Qualcomm Incorporated Method and apparatus for navigation based on media density along possible routes
JP6561504B2 (en) 2015-03-11 2019-08-21 富士通株式会社 Data arrangement program, data arrangement method, and data arrangement apparatus
CN107169531B (en) * 2017-06-14 2018-08-17 中国石油大学(华东) A kind of image classification dictionary learning method and device based on Laplce's insertion
US10896218B2 (en) * 2017-12-22 2021-01-19 Oracle International Corporation Computerized geo-referencing for images
US20220004921A1 (en) * 2018-09-28 2022-01-06 L&T Technology Services Limited Method and device for creating and training machine learning models

Family Cites Families (20)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5915250A (en) * 1996-03-29 1999-06-22 Virage, Inc. Threshold-based comparison
US6121969A (en) 1997-07-29 2000-09-19 The Regents Of The University Of California Visual navigation in perceptual databases
US6028603A (en) 1997-10-24 2000-02-22 Pictra, Inc. Methods and apparatuses for presenting a collection of digital media in a media container
US6240423B1 (en) * 1998-04-22 2001-05-29 Nec Usa Inc. Method and system for image querying using region based and boundary based image matching
US6538698B1 (en) 1998-08-28 2003-03-25 Flashpoint Technology, Inc. Method and system for sorting images in an image capture unit to ease browsing access
US6512850B2 (en) * 1998-12-09 2003-01-28 International Business Machines Corporation Method of and apparatus for identifying subsets of interrelated image objects from a set of image objects
US6721759B1 (en) 1998-12-24 2004-04-13 Sony Corporation Techniques for spatial representation of data and browsing based on similarity
US6950989B2 (en) 2000-12-20 2005-09-27 Eastman Kodak Company Timeline-based graphical user interface for efficient image database browsing and retrieval
US20090106192A1 (en) * 2001-02-09 2009-04-23 Harris Scott C Visual database for online transactions
US6915489B2 (en) 2001-03-28 2005-07-05 Hewlett-Packard Development Company, L.P. Image browsing using cursor positioning
US7773800B2 (en) * 2001-06-06 2010-08-10 Ying Liu Attrasoft image retrieval
US6943842B2 (en) 2001-11-02 2005-09-13 Hewlett-Packard Development Company, L.P. Image browsing user interface apparatus and method
US20030123737A1 (en) 2001-12-27 2003-07-03 Aleksandra Mojsilovic Perceptual method for browsing, searching, querying and visualizing collections of digital images
CA2397424A1 (en) * 2002-08-09 2004-02-09 Mohammed Lamine Kherfi Content-based image retrieval using positive and negative examples
GB2395807A (en) 2002-11-27 2004-06-02 Sony Uk Ltd Information retrieval
US7532804B2 (en) * 2003-06-23 2009-05-12 Seiko Epson Corporation Method and apparatus for video copy detection
US7697792B2 (en) * 2003-11-26 2010-04-13 Yesvideo, Inc. Process-response statistical modeling of a visual image for use in determining similarity between visual images
JP2005234994A (en) * 2004-02-20 2005-09-02 Fujitsu Ltd Similarity determination program, multimedia data retrieval program, and method and apparatus for similarity determination
JP4465534B2 (en) * 2004-03-31 2010-05-19 パイオニア株式会社 Image search method, apparatus, and recording medium recording program
US7920745B2 (en) * 2006-03-31 2011-04-05 Fujifilm Corporation Method and apparatus for performing constrained spectral clustering of digital image data

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108764068A (en) * 2018-05-08 2018-11-06 北京大米科技有限公司 A kind of image-recognizing method and device

Also Published As

Publication number Publication date
EP1915723A2 (en) 2008-04-30
US20090150376A1 (en) 2009-06-11
WO2007020423A2 (en) 2007-02-22
JP2009509215A (en) 2009-03-05
WO2007020423A3 (en) 2007-05-03
EP1755067A1 (en) 2007-02-21

Similar Documents

Publication Publication Date Title
CN101263514A (en) Mutual-rank similarity-space for navigating, visualising and clustering in image databases
Card Information visualization
Plant et al. Visualisation and browsing of image databases
US9785655B2 (en) Dynamic maps for exploring and browsing of images
US20070098266A1 (en) Cascading cluster collages: visualization of image search results on small displays
JP2009110360A (en) Image processing device and image processing method
US20140250376A1 (en) Summarizing and navigating data using counting grids
Manandhar et al. Learning structural similarity of user interface layouts using graph networks
US20180285965A1 (en) Multi-dimensional font space mapping and presentation
CN114329069A (en) Intelligent system and method for visual search query
Khanwalkar et al. Exploration of large image corpuses in virtual reality
JP2005525606A (en) Hierarchical image feature-based visualization method
Cakmak et al. Multiscale visualization: A structured literature analysis
Barthel et al. Fusing keyword search and visual exploration for untagged videos
Luo et al. Texture Browser: Feature‐based Texture Exploration
EP2465056B1 (en) Method, system and controller for searching a database
Rallabandi et al. Image retrieval system using R-tree self-organizing map
US20070055928A1 (en) User workflow lists to organize multimedia files
Nath et al. A survey on personal image retrieval systems
Hezel et al. ImageX-explore and search local/private images
Zhang et al. Content-based retrieval
Chun et al. Visualization of photo album on mobile devices
Scuturici et al. Topological representation model for image database query
Budikova et al. Visual Exploration of Human Motion Data
Bäuerle et al. Semantic Hierarchical Exploration of Large Image Datasets.

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
C10 Entry into substantive examination
SE01 Entry into force of request for substantive examination
C12 Rejection of a patent application after its publication
RJ01 Rejection of invention patent application after publication

Application publication date: 20080910