CN103970775A - Object spatial position relationship-based medical image retrieval method - Google Patents

Abstract

The invention discloses an object spatial position relationship-based medical image retrieval method, and belongs to the technical field of image processing. Characteristics such as shapes and position relationships of main objects in segmented images are extracted, and the similarity of contents between the two images is calculated by adopting a coarse-to-fine retrieval strategy. The method comprises the following steps of (1) inputting a medical image, and segmenting the medical image to obtain areas occupied by the main objects in the image; (2) extracting the characteristics such as the shapes and the position relationships of the main objects in each area as characteristics of the objects; (3) performing a round of rapid matching according to the characteristics of each simple object; (4) performing size function matching on a relatively small result subset; (5) performing spatial position relationship matching on the image matched in the previous step according to spatial position relationships between objects in example images submitted by a user, and returning the retrieved images to the user for output from the images with greater similarity to the images with lower similarity. According to the method, the images can be retrieved according to any spatial layout and attributes of the objects in the images, and a novel content expression method and a novel calculation method are provided for retrieval by utilizing bottom-layer characteristics and the spatial position relationships.

Description

A kind of medical image search method based on object space position relationship

Technical field

The present invention relates to that image is cut apart and feature extraction is processed, belong to image processing field.

Technical background

Deep development along with modern image and image processing techniques, in recent years can be for expansion rapidly of medical image quantity clinical, that impart knowledge to students and study, the amount of images that for example only University Hospital ofGeneva dept. of radiology produces every day just reaches 12000 width.So in numerous images, how medical worker and related scientific research personnel find needed image to become problem anxious to be resolved quickly and accurately.

Since 20 century 70s, under data base set is unified the common promotion of computer vision two large research fields, image retrieval technologies becomes a very active research field gradually.That early stage image retrieval is used is text marking method (Text-base Image Retrieval, TBIR), popular image indexing system was the object that image is stored in database at that time, with key word or free text, was described.Query manipulation is that the textual description based on this image carries out exact matching or probability match, and the retrieval model of some system also has dictionary support.Yet text based image retrieval technologies exists serious problem completely.Current computer vision and artificial intelligence technology all cannot be carried out autotext mark to image, and must rely on, manually image are made to mark.This work is not only wasted time and energy, and manual mark is inaccurate or incomplete often, also inevitably with subjectivity.

The features such as medical image is different from general pattern, and medical image gray level resolution is high, spatial resolution is high, image similarity is large, containedly contain much information, color type is few, therefore, the image retrieval technologies CBIR based on visual signature (content) is more suitable for.

CBIR technology is the developing history of existing more than ten years so far, due to the wide application prospect of technique and huge commercial value, therefore become rapidly study hotspot, the various contents of utilizing image, as the technology of the Image Visual Feature retrieving images such as color, texture, shape is arisen at the historic moment.Its basic thought is the information such as spatial relationship of the color, texture, shape and the object that comprise according to image, sets up the eigenvector of image, and search method is to carry out similarity matching according to the multidimensional characteristic vector of image.This technology relates to the technology of the various fields such as computer vision, image processing, image understanding, artificial intelligence, machine learning, statistics, database and psychology.Its application relates to multimedia database, military field, building and engineering design, fashion and indoor dress and drills design, tourist industry and advertising, Geographic Information System and remote sensing system, library automation, trade-mark administration, medical image management, public security system and satellite image management etc., is a rising research direction.

At present, Ge great research institution and company, as IBM, MIT, Columbia, Stanford, UIUC, UCSB, Berkeley, Microsoft etc. have released image indexing system separately.Fig. 1 has provided the Basic architecture of a typical CBIR system, system is mainly comprised of image querying subsystem and image data base subsystem two parts, the major function of database subsystem is set up and safeguard that whole image library and associated documents, its core are Visual Feature Retrieval Process and recognition technology.The major function of image querying subsystem is the similarity between image and the query image of user's submission in the mutual and measurement database of being responsible for user.After submit queries image, it is analyzed and extracts eigenvector, according to corresponding similarity measurement criterion, in image library, mate, finally according to similarity order, Query Result is returned to user.

The extraction of the visual signature of image and expression are the bases of CBIR technology.Visual signature can be divided into general visual signature and the visual signature of domain-specific.The former is irrelevant with particular type and the content of image for describing the total feature of all images, mainly comprises color, texture and shape etc.; Latter is based upon on the basis of some priori of institute's Description Image content or hypothesis, is closely related with concrete application, for example, people's facial characteristics or fingerprint characteristic etc.The present invention discusses the search method of the object space position relationship feature based on general.

Summary of the invention

In image retrieval processing procedure, first general people find object by the dividing method based on low-level image feature, and then carry out recognition object further by their features such as shape, texture and topological relation.While having obvious object to occur in image, for example, for a common landscape painting, the content of image can represent by forming the features such as the shape of object (as sky, land, mountain range, buildings etc.) of this width landscape painting and position relationship, as shown in Figure 2.If another piece image also includes these similar objects, we just think that the content of this two width image is similar.

The medical image search method based on object space position relationship that the present invention proposes has been followed principle above just, generally, whole retrieving can be divided into three steps: first, medical image is cut apart, obtained the shared region of main object in image; Then, to features such as each their shape of piece extracted region and position relationships, as the feature of this object; Finally, according to the feature of each object in image, calculate the similarity degree of content between two width images, and realize retrieval.The method can be calculated according to any space layout and the attribute thereof between image object image retrieval, for utilizing low-level image feature and spatial relation to form retrieval, provides new content expression and computing method.

In whole image retrieval process, in order to take into account the requirement of recall precision and retrieval precision, we have adopted by slightly to smart search strategy, as shown in Figure 3 image retrieval flow process.Left half represents: when image is added into image library, and the feature that we need to preserve original image, the object splitting from image simultaneously and extract from object.Right half represents: in retrieval phase, user can submit an example image to, the system auto Segmentation remarkable object in picture of publishing picture, and using the search request of resulting object as user.

In whole retrieving, system is first carried out a Rapid matching according to the feature of each simple object, then in relatively little result subset, carry out scaling function (Size Function) coupling, finally, to submit the spatial relation between object in example image to according to user, the image that previous step coupling is obtained carries out spatial relation coupling, and the image that retrieval is obtained returns to user from big to small according to similarity.To summary of the invention, be described in detail from following four aspects herein:

One, the characteristic matching based on site polling

In a broad sense, when we are using entire image as a large region time, the inquiry of entire image global characteristics also be can be regarded as to site polling.Here the site polling of our indication is sensu lato image querying.We are the generalization of carrying out of site polling, mainly in order to meet the needs of user's inquiry, in user's inquiry, may comprise such region, the visual signature in region is similar with the global characteristics of certain image, user wishes that the region of submitting to illustrates with the global characteristics of this width image, in this case, we need to be using piece image as a region of inquiring about.

Site polling can be divided into single site polling and multizone inquiry, so-called single site polling specifies a region in inquiry, find the image that comprises this region similarly, at this moment do not need the spatial relation in consideration of regional, we inquire about for the some objects that split in image; Multizone inquiry packet containing a plurality of regions, is not only wanted visual signature and the space attribute of consideration of regional, also wants the spatial relation between consideration of regional, and between the image object that we will consider, characteristic matching and the spatial relation that will carry out below mate.

While indicating the space attribute in single region in inquiry, we,, to the centre coordinate in region and minimum frame index, can inquire about absolute spatial position like this.

The locus attribute in region is by (the x of space center in region, y) and minimum frame MBR (the minimum boundary rectangle that region covers determines, position and size are the useful attributes of discriminate regions), here, we come the position of zoning and big or small similarity with such attribute.

1, the space length of interregional central point is provided by Euclidean distance:

$d_{q,t}^s=\sqrt{{(x_q-x_t)}^2{+(y_q-y_t)}^2}$ (formula 1)

2, in many cases, user is also indifferent to the accurate location in searched region, as long as it drops on the region of appointment, like this, user is the spatial dimension in each region in given query neatly, when a target area is dropped in spatial dimension, the space length of target area is appointed as 0, otherwise its distance is Euclidean distance, expression formula is as follows:

(formula 2)

Here, (x _min, y _max) and (x _max, y _min) be respectively the upper left corner and the lower right corner coordinate of spatial dimension.

Space quaternary tree is an effective regional center point indexing means, and quaternary tree is by can quick access 2-D data to group areas.To regional location at (x _t, y _t) inquiry adopt preorder traversal space quaternary tree to obtain nearest group.When instruction manual the inquiry of spatial dimension, need to consider that a plurality of groups so that the distance of the point in scope

Regional center position is not sufficient to represent the locus of the image-region extract, so need to carry out index to their minimum frame MBR, space querying can be specified one for the rectangle searched and be found the target area overlapping.With R-tree, carry out the MBR in index region, R-tree is the general dynamic structure of an index k-D rectangle, k=2 here, and R-tree can regard as by the overlapping hierarchical structure forming of space grouping, can only access the grouping of minority while searching in space.

Space size is that extend in a very important visual characteristic, especially its spatial area and the space in region.The distance of the area between two region q and t is absolute distance:

$d_{q,t}^a=|{\mathrm{area}}_q-{\mathrm{area}}_t|$ (formula 3)

The comparison that extend in the space in region is by the width of MBR and highly determine, width and distance highly between two region q and t are:

$d_{q,t}^m=\sqrt{{(w_q-w_t)}^2+{(h_q-h_t)}^2}$ (formula 4)

These methods and characteristic measure are combined, user can given query construct, like this towards the inquiry of image-region by every kind of visual signature and space characteristics and relation are carried out respectively to query count concurrently, the distance metric in single region is provided by distance metric and the weight thereof of color characteristic, textural characteristics, shape facility, and user also can specify a relative weighting α to each characteristic.

Two, the similarity measurement based on scaling function

Discrete in the situation that, an object Γ can regard as by a point set V={ (x, y) | (x, y) ∈ Γ } form, from V, we can construct a figure G=<V, E>, wherein, V and E are respectively vertex set and Bian Ji.

E＝{e(x ₁，y ₁，x ₂，y ₂)|(x ₁，y ₁)∈V，(x ₂，y ₂)∈V，|x ₁-y ₁|≤1，|x ₂-y ₂|≤1}

E (x wherein ₁, y ₁, x ₂, y ₂) representative connection two point (x ₁, y ₁) and (x ₂, y ₂) limit.Here be to think that the upper adjacent point of Γ has a limit to be connected.In actual applications, we can, by the method for sampling, choose Γ top branch as summit.Like this, we just show an object Γ with a figure.

Then, choose suitable measure function calculate with the following method scaling function value:

Step1：

E←{e(x ₁，y ₁，x ₂，y ₂)|(x ₁，y ₁)∈V or(x ₂，y ₂)∈V}

Step2: remove from G summit and the limit being connected with these summits, form a new figure G:

V←V-V，E←E-E，G←<V，E>

Step3: in new figure G, be calculated to a rare summit (x, y) and meet the number of connected subgraph, and using this number as value.

As can be seen here, scaling function is a kind of character representation method of relevant object shapes, in this method for expressing, has not only comprised the local feature of object, but also the whole topological relation that has comprised object.To measure function distance between two objects " A " and " B " can calculate with the following method:

(formula 5)

Wherein, D is certain suitable region.This method for measuring similarity, its key issue is to choose suitable measure function

Three, between object the expression of spatial relation with mate

In order to describe regional ensemble Ω={ A ₁, A ₂..., A _nspatial relation, the center of gravity in each region, to X-axis and Y-axis projection, is obtained respectively to two set strings in X-axis and Y-axis with as shown in Figure 4.

there is following character:

(1) $X_t^{\mathrm{\&Omega;}}\&NotEqual;\mathrm{\&Phi;},(i=\mathrm{1,2},\&CenterDot;\&CenterDot;\&CenterDot;,s),$ $X_t^{\mathrm{\&Omega;}}\&cap;X_j^{\mathrm{\&Omega;}}=\mathrm{\&Phi;},(i\&NotEqual;j),$ $\{A_1,A_2,\&CenterDot;\&CenterDot;\&CenterDot;,A_n\}=\underset{i=1}{\overset{s}{\mathrm{\&cup;}}}{X}_i^{\mathrm{\&Omega;}},$ That is to say set { A ₁, A ₂..., A _na division.

(2) if | (A _i) _x-(A _j) _x| < ε, ? (A wherein _t) _xrepresent A _ithe projection of center of gravity in X-axis about the coordinate of X-axis, ε is a predetermined constant.This character shows, when the projection coordinate in two regions is more or less the same, should be classified as same class, and this is mainly to consider when actual computation, itself has the existence of error.

(3) if i < j, (A _i) _x< (A _j) _x.

In like manner, also there is similar character.

With these two set strings, we just can describe the relative tertiary location relation between a regional ensemble element easily.To two regional ensemble Ω ₁={ A ₁, A ₂..., A _nand Ω ₂={ B ₁, B ₂..., B _n, if exist corresponding one by one and, under this correspondence: claim Ω ₁with Ω ₂there is identical relative tertiary location relation.At this moment, we can also define their structure distance:

${\mathrm{dis}}_F({\mathrm{\&Omega;}}_1,{\mathrm{\&Omega;}}_2)=\underset{t,j=1}{\overset{n}{\mathrm{\&Sigma;}}}|d(A_i,A_j)-d(F\left(A_i\right),F\left(A_j\right))|$ (formula 6)

D (A wherein _i, A _j) expression A _iwith A _jcenter of gravity between distance.Dis _f(Ω ₁, Ω ₂) can be regarded as Ω ₁with Ω ₂a kind of metric form of similarity of absolute spatial position relation, dis _f(Ω ₁, Ω ₂) less, show Ω ₁with Ω ₂space structure more similar.

More than described the problem of representation of spatial relation between objects in images, for the matching problem of spatial relation, first we illustrate by a concrete instance so.

Suppose that user is marked with A, B, C, tetra-regions of D in the plane of piece image, as shown in Figure 4, the region that certain image in image library comprises is as Fig. 5, and, by Regional Similarity, calculate, determined matching relationship: our problem is: in Fig. 5, and domain of the existence subset whether $\{a,b,c,d\}\&Subset;\left\{\mathrm{1,2,3,4,5,6}\right\},$ Make and a, b, c, d} with A, B, C, does D} have identical relative tertiary location relation? as seen from the figure, the answer of this problem is: a=1, b=2, c=3, d=4.

In the ordinary course of things, this matching problem can be described as:

[known]: Ω ₁={ A ₁, A ₂..., A _n, Ω ₂={ B ₁, B ₂..., B _n,

$A_i\&LeftRightArrow;S_i(i=\mathrm{1,2},\&CenterDot;\&CenterDot;\&CenterDot;n,S_i\&SubsetEqual;{\mathrm{\&Omega;}}_2)$

[solving]: a _i(i=1,2 ... n), it is met: a _i∈ S _i(i=1,2 ... n), and, in mapping under, { a ₁, a ₂..., a _nand { A ₁, A ₂..., A _nthere is an identical relative tertiary location relation.

In order to address this problem, we propose following derivation algorithm:

Step1: input Ω ₁, Ω ₂and matching relationship $A_i\&LeftRightArrow;S_i(i=\mathrm{1,2},\&CenterDot;\&CenterDot;\&CenterDot;n,S_i\&SubsetEqual;{\mathrm{\&Omega;}}_2)$

Step2： ${\mathrm{\&Omega;}}_2\&LeftArrow;\underset{i=1}{\overset{n}{\mathrm{\&cup;}}}{S}_i$

Step3: to S _i(i=1,2 ... n) the element A in _iidentify

Step4: obtain with

Step5: find out in all substrings and require:

$X_i^{{\mathrm{\&Omega;}}_1}\&SubsetEqual;X_{m_i}^{{\mathrm{\&Omega;}}_2}(i=\mathrm{1,2},\&CenterDot;\&CenterDot;\&CenterDot;,s^{{\mathrm{\&Omega;}}_1}).$ If there is no qualified substring, return: problem is without solution.

Step6: to each found out substring

Step6.1： ${\mathrm{\&Omega;}}_2\&LeftArrow;\underset{i=1}{\overset{s^{{\mathrm{\&Omega;}}_1}}{\mathrm{\&cup;}}}{X}_{m_i}^{{\mathrm{\&Omega;}}_2}$

Step6.2: obtain with

Step6.3: find out in all substrings and require:

$Y_i^{{\mathrm{\&Omega;}}_1}\&SubsetEqual;Y_{m_i}^{{\mathrm{\&Omega;}}_2}(i=\mathrm{1,2},\&CenterDot;\&CenterDot;\&CenterDot;,t^{{\mathrm{\&Omega;}}_1})$

Step6.4: to each found out substring

Step6.4.1: to each A _i, find out with make $A_i\&Element;X_{p_i}^{{\mathrm{\&Omega;}}_1}\&cap;Y_{q_i}^{{\mathrm{\&Omega;}}_1}$

Step6.4.2: in choose and be identified as A _iΩ ₂in element a _i

Step6.4.3: output matching

Step7: algorithm finishes

This algorithm is more simple and clear than general spatial relationship matching algorithm, and can effectively realize the search strategy based on objects in images spatial relation.

Four, medical image retrieval algorithm

Based on above-mentioned work, concrete searching algorithm is described below:

If it is Ω that image is cut apart the object set obtaining ₁={ A ₁, A ₂..., A _n}

Step1: input object collection Ω ₁={ A ₁, A ₂..., A _n, determine constant ε ₁and ε ₂

Step2: to each A _i(i=1,2 ... n):

Step2.1: extract A _icharacteristic quantity

Step2.2:ShapeNo _tin ← feature database with A _icharacteristic quantity apart from dis _fbe less than ε ₁the object number of record

Step2.3: by ImageNo _tbe set to empty set

Step2.4: to ShapeNo _tin each object number s _j: by scaling function method, determine itself and A _idistance if from image library, choosing all object number is s _jimage number, and be incorporated to set ImageNo _t, otherwise, from ShapeNo _tin remove s _j

Step2.5: sign mapping A _i→ ShapeNo _t

Step3： $\mathrm{ImageNo}\&LeftArrow;\underset{i=1}{\overset{n}{\mathrm{\&cap;}}}\mathrm{ImageN}{o}_i$

In Step4:ImageNo ← ImageNo with { A ₁, A ₂..., A _nthere is an image of identical relative tertiary location relation

Step5: to each image in ImageNo, calculate itself and { A ₁, A ₂..., A _nstructure distance, and export from small to large the image of some by structure distance.

In this searching algorithm, the elementary retrieval of Step2.2 correspondence based on feature; The similarity matching of Step2.4 correspondence based on scaling function; The coupling of spatial relation between Step4 corresponding objects.

Accompanying drawing explanation:

In order to be illustrated more clearly in the technical scheme of the embodiment of the present invention or prior art, the accompanying drawing that needs to provide in embodiment or description of the Prior Art is done to simple introduction below, the accompanying drawing the following describes is only some embodiments of the present invention.

Fig. 1 is the structural drawing of typical CBIR system.

The image that Fig. 2 is comprised of 6 objects, such as there being the objects such as sky, land, mountain range, buildings.

Fig. 3 is the process flow diagram of example implementation method of the present invention.

Fig. 4 is the two-dimensional ensemble string method for expressing schematic diagram of image object spatial relation in example of the present invention.

Fig. 5 is spatial relation coupling illustration.

Fig. 6 is the CT of the liver image searching result that query case and use case method of the present invention obtain.

Fig. 7 is the chest CT image result for retrieval that query case and use case method of the present invention obtain.

Claims (4)

1. the medical image search method based on object space position relationship, its technical characterictic is:

(1), for the image to be analyzed after cutting apart, the center of gravity of cutting apart each region of gained, to X-axis and Y-axis projection, is obtained respectively to two set strings in X-axis and Y-axis with

(2) propose a kind of simple spatial relationship matching algorithm, this algorithm can calculate the region subset in a sub-picture, and this region subset has identical relative tertiary location relation with the All Ranges in another sub-picture.

(3) an object Γ is regarded as by a point set V={ (x, y) | (x, y) ∈ Γ } form, from figure G=<V of V structure, E>, with the shape facility of scaling function indicated object, in this method for expressing, not only comprised the local feature of object, but also the whole topological relation that has comprised object.

(4), based on above-mentioned work, establishing image, to cut apart the object set obtaining be Ω ₁={ A ₁, A ₂..., A _n; complete, concrete searching algorithm is proposed; realized the coupling of spatial relation between elementary retrieval, the similarity matching based on scaling function and the object based on feature, the image of finally exporting some from small to large by structure distance is as result for retrieval.

2. the medical image search method based on object space position relationship according to claim 1, is characterized in that:

We think that space size is a very important visual characteristic, especially its spatial area and the space extension in region.

The distance of calculating the area between two region q and t is absolute distance:

$d_{q,t}^a=|{\mathrm{area}}_q-{\mathrm{area}}_t|$

The comparison that extend in the space in region is by the width of minimum frame MBR and highly determine, width and distance calculating method highly between two region q and t are:

$d_{q.t}^m=\sqrt{{(w_q-w_t)}^2+{(h_q-h_t)}^2}$

3. the medical image search method based on object space position relationship according to claim 1, is characterized in that:

For two regional ensemble Ω ₁={ A ₁, A ₂..., A _nand Ω ₂={ B ₁, B ₂..., B _n, when it meets the following conditions, between them, there is identical relative tertiary location relation:

(1) there is correspondence one by one

(2) under this correspondence: $F\left(X_i^{{\mathrm{\&Omega;}}_1}\right)=X_i^{{\mathrm{\&Omega;}}_2},$ $F\left(Y_i^{{\mathrm{\&Omega;}}_1}\right)=Y_i^{{\mathrm{\&Omega;}}_2}$

Otherwise the structure distance defining between two regional ensembles is:

${\mathrm{dis}}_F({\mathrm{\&Omega;}}_1,{\mathrm{\&Omega;}}_2)=\underset{i,j=1}{\overset{n}{\mathrm{\&Sigma;}}}|d(A_i,A_j)-d(F\left(A_i\right),F\left(A_j\right))|$

D (A wherein _i, A _j) expression A _iwith A _jcenter of gravity between distance.Dis _f(Ω ₁, Ω ₂) be to Ω ₁with Ω ₂a kind of metric form of similarity of absolute spatial position relation, dis _f(Ω ₁, Ω ₂) less, show Ω ₁with Ω ₂space structure more similar.

4. the medical image search method based on object space position relationship according to claim 1, is characterized in that:

To measure function distance between two objects " A " and " B " can calculate with the following method:

Wherein, D is certain suitable region.Scaling function value calculate with the following method:

Step1：

E←{e(x ₁，y ₁，x ₂，y ₂)|(x ₁，y ₁)∈V or(x ₂，y ₂)∈V}

Step2: remove from G summit and the limit being connected with these summits, form a new figure G:

V←V-V，E←E-E，G←<V，E>

Step3: in new figure G, be calculated to a rare summit (x, y) and meet the number of connected subgraph, and using this number as value.