CN106021406A - Data-driven iterative image online annotation method - Google Patents

Abstract

The invention discloses a data-driven iterative image online annotation method. The method comprises the following steps: 1, data preparation and recovery: making a user feed samples to be annotated into a sample buffer pool successively, performing scale detection on the sample buffer pool, performing feature extraction, and feeding the samples to be annotated into a circular annotation module; and 2, circular annotation: at the start of each round of circulation process, calculating a similarity matrix of features through an online similarity measurement model, optimizing the similarity matrix through constraint propagation, performing data clustering on the similarity matrix by a spectral clustering algorithm, selecting a significant category, pushing the significant category to the user for annotation, cycling constantly till remaining samples are not enough to continue annotation, and exiting the circulation process to obtain a personalized label of an input image region given by the user. Through the technology of the method, rapid and accurate annotation of images can be realized; the annotation efficiency is increased; and the method can adapt to a personalized annotation system of the user.

Description

A kind of iterative image online mask method of data-driven

Technical field

The present invention relates to a kind of image labeling method, belong to technical field of image processing, specifically a kind of data are driven The dynamic online mask method of iterative image.

Background technology

In the last few years, along with developing rapidly of Internet technology, multimedia image and storage device, user touched Amount of images is explosive increase.The most fast and effeciently by substantial amounts of user image data taxonomic organization, to meet user's Other application demands, are important research directions.Traditional images mark, to classify, to be retrieved as application target, is concerned about image The information such as scene, object and the object relationship of entirety or local.Therefore, under the ever-increasing background of user image data, The mark of image is the most flexible, and mark system is various, to adapt to the different application of different user, keeps relatively low user simultaneously Mark burden.

It is true that image labeling technology is always the hot subject of computer vision field, and create substantial amounts of skill Art and method.Typical image labeling method is all to use model-driven mode to realize, i.e. from marking image set learning Go out the relational model between feature space and keyword space, and utilize this model that image to be marked is labeled.As: patent " a kind of Automatic image annotation algorithm " (2013105149427), return mould by the image set training selecting information content abundant Type, then to not marking image classification annotation one by one；Patent " a kind of based on the image automatic annotation method of dependency between word " And " a kind of multi-tag image labeling result fusion method minimized based on order " (2014100085531) (201310375976.2) then by setting up different probabilistic models, Tag Estimation or multi-tag are carried out one by one to not marking image Merge.The performance of this kind of method and effect are largely dependent upon selected hypothesized model and mark training set, except needing choosing Selecting outside suitable disaggregated model, with greater need for manually marking training data in a large number, and its mark system is difficult after model training completes To change.To this end, researcher proposes many new solutions subsequently, on the one hand, for artificial mark training data burden Big problem, it is proposed that the image labeling method of semi-supervised learning, such as: patent is " a kind of based on multi views and semi-supervised learning machine The image labeling method of system " (2014101080605), its main thought is that the low volume data only concentrated training data is marked Then the data not marked substantial amounts of in data set are joined in the training of model by note, by excavating existence between image Neighbor relationships on feature space realizes the effective mark to image, thus greatly reduces artificial mark training dataset Burden；On the other hand, for the diverse problems of mark system, it is proposed that use the image labeling method of Active Learning, such as: literary composition Offer 1:P.Jain and A.Kapoor.Active learning for large multi-class problems.In Proceedings of Computer Vision and Pattern Recognition,pages 762–769.IEEE, 2009,2:A.Kapoor, K.Grauman, R.Urtasun, and T.Darrell.Active learning with gaussian processes for object categorization.In Proceedings of International Conference on Computer Vision,pages 1–8.IEEE,2007.And patent is " a kind of based on Active Learning Image labeling method " (201410106864.1) and " scene image based on Active Learning and multi-tag multi-instance learning mark Method " (201510473322.2), they pick out part representativeness or information content by concentrating from training data on one's own initiative Bigger sample data, for user annotation, decreases the workload of the artificial mark of part, adds somewhat to mark simultaneously The multiformity of system.

For what model driven method existed, data set a priori assumption is marked the dependency of training sample, mark with artificial System is difficult to change the problems such as renewal, and the image labeling of data-driven becomes the new trend in image labeling field in recent years.A kind of It is to use degree of depth thought, such as: patent " a kind of image labeling method learnt based on the multi-modal degree of depth " (201510198325.X) Use with patent " a kind of image automatic annotation method based on degree of depth study with canonical correlation analysis " (201410843484.6) Neutral net obtains the high-level characteristic of image and represents, then sets up sorter model by high-level characteristic, to do not mark image by One carries out differentiating mark；And another kind trend is for the data with certain similarity, utilize the relation between them, use One group of data of its class discovery are labeled, such as: document 3:Y.J.Lee and by the mode of group mark K.Grauman.Object-graphs for context-aware visual category discovery.IEEE Transactions on Pattern Analysis and Machine Intelligence, 34 (2): 346 358,2012, Document 4:D.Dai, M.Prasad, C.Leistner, and L.Van Gool.Ensemble partitioning for unsupervised image categorization.In Proceedings of European Conference on Computer Vision, pages 483 496.Springer, 2012, document 5:D.Liu and T.Chen.Unsupervised image categorization and object localization using topic models and correspondences between images.In Proceedings of International Conference on Computer Vision, pages 1 7.IEEE, 2007, compared with the mode marked one by one, this with The mark that group is form is effectively improved the annotating efficiency of this kind of data.But this kind of method needs also exist for establishing in advance mark Classification, and be only capable of affecting mark system by adjustment cluster numbers.Compared with this kind of method, document 6:M.Wigness, B.A.Draper and J.R.Beveride.Selectively guiding visual concept discovery.WACV 2014, pp.247-254. document 7:C.Galleguillos, B.McFee and G.R.G.Lanckriet.Iterative category discovery via multiple kernel metric learning.Springer IJCV 2014,108 (1-2), pp.115-132. can help user to find potential new mark classification in annotation process, and can be circulated throughout Cheng Zhong, progressively definition mark system so that annotation results is more personalized, variation.

The image labeling method of summary, there is dependence hypothesized model, relies on a large amount of people in the mask method of model-driven Work mark training data, mark system lack the shortcomings such as motility so that it is be not suitable for the mark work of large-scale user images Make；And existing data-driven method, there is also the Dependence Problem to labeled data priori, and carry out for image The optimization of character representation, the sign ability to image.So, above-mentioned image labeling method all can not fully meet image labeling Application demand.

Summary of the invention

Goal of the invention: the technical problem to be solved is for the deficiencies in the prior art, it is provided that a kind of data are driven The dynamic online mask method of iterative image, for supporting the mark of image-region.

In order to solve above-mentioned technical problem, the present invention discloses the online mask method of iterative image of a kind of data-driven, Comprise the following steps:

Step 1, data prepare and feature extraction: send needing the set of image regions i.e. sample set entering circulation mark Enter Sample Buffer pond, and carry out feature extraction,；

Step 2, circulation mark: the feature extracted is carried out Similarity Measure by online measuring similarity model, use The clustering algorithm of constraint propagation optimization clusters not marking image area data, selects notable classification, clusters most excellent Bunch being pushed to user is labeled, and the study that annotation results is used for constraint propagation and measuring similarity model updates, finally Circulation mark obtains user's personalized labels to input picture region；

Step 3, reclaims the image area data that do not marks returned from circulation mark: after circulation annotation process terminates, Reclaim the sample set not being marked to Sample Buffer pond.

Wherein, step 1 comprises the following steps:

Step 1-1, by initialized set of image regions i.e. sample setSend into Sample Buffer pond B_p (subscript p represents that set B is the Buffer Pool that sample prepares),Represent that the c image-region, subscript+expression b are sample areas figure Picture, subscript c represents this sample image region sequence location in set, Sample Buffer pond B_pComprise current all to be marked Image-region, when the sample size of Sample Buffer pond | B_p| more than threshold value T (threshold value generally takes 200), then by sample in Sample Buffer pond This feeding circulation annotation process；

Step 1-2, extracts all sample characteristics preparing to be loaded into circulation annotation process from Sample Buffer pond, including color Histogram feature (RGB/HSV histogram), cold and warm tone feature (warmcold) and location context (locationcontexts)。

Step 2 comprises the following steps:

Step 2-1, carries out Similarity Measure and renewal to the sample characteristics obtained in step 1-2: obtained by equation below The i-th image-region of circulation annotation process must be enteredWith jth image-region(subscript i, j represents any two image The sequence location in region) between characteristic similarity

$S_W(b_i^{+},b_j^{+})=b_i^{{+}^T}{\mathrm{Wb}}_j^{+},$

Wherein W represents tolerance parameter matrix, and the span of i and j is 1～c, then obtains characteristic similarity metric matrixThe quantity of image-region during wherein N is epicycle circulation；

Step 2-2, constraint propagation process: include between image area data connecting and two kinds of relations can not be connected, I.e. represent image-region must be same group or must can not be at same group, by all of image-region annexation YY= {Y_ij}_N×NIt is initialized as two constraint setWithY_ijRepresent i-th image-regionWith jth image-regionBetween Annexation, whenTime Y_ij=1, whenTime Y_ij=-1, otherwise Y_ij=0, i.e.Do not belong to In an any of the above constraint set；

Propagated by the relation between the sample that below equation will mark and do not marked between image area data Relation constraint F^*:

$F^{*}={\left\{f_{ij}^{*}\right\}}_{N\×N},$

Wherein α is the parameter of codomain (0,1),The data of representing matrix the i-th row jth row, I is unit matrix；Wherein by characteristic similarity metric matrixObtaining matrix H, D is diagonal matrix, Wherein (i, i) equal to matrix H the i-th row data sum for diagonal element；

WhenIt it is i-th image-regionK arest neighbors time,Otherwise H_ij=0, S_ijRepresent The similarity data of measuring similarity matrix M the i-th row, jth row, S_iiRepresent measuring similarity matrix M the i-th row, i-th row similar Degrees of data, S_jjRepresent the similarity data of measuring similarity matrix M jth row, jth row, then calculate H=(H+H^T)/2 are to ensure H is symmetrical matrix；

Step 2-3, constrained clustering: the relation constraint that will obtain in step 2-2It is used for adjusting feature phase Like degree metric matrix M, obtain retraining similarityShown in equation below:

$\widetilde{S_W}(b_i^{+},b_j^{+})=\left\{\begin{array}{c}1-(1-f{*}_{ij})(1-H_{ij}),f{*}_{ij}\&GreaterEqual;0\\ (1+f{*}_{ij})H_{ij},f{*}_{ij}<0\end{array}\right.,$

Wherein H_ijStep 2-2 is calculated, the similarity after then constraint being adjustedCarry out spectral clustering, by image Region clustering is NC kind；

Step 2-4, notable class discovery: calculated by below equation in the average class of each clustering cluster c away from:

$d_{\mathrm{int}ra}\left(c\right)=\frac{1}{\left|c\right|\&CenterDot;\left(\right|c|-1)}{\mathrm{\&Sigma;}}_{\begin{array}{c}i,j\&Element;c\\ i\&NotEqual;j\end{array}}d(b_i^{+},b_j^{+}),$

WhereinRepresent i-th image-regionWith jth image-regionIn spectral clustering space Distance, by average class away from minimum clustering cluster c^*Elect optimum cluster bunch as, the notable classification i.e. found in this circulation；

Step 2-5, class members selects: class members therein is done by the notable classification that user obtains according to step 2-4 Select or reject operation, according to the positive sample of the suitable operation of user and negative sample: great majority figure in user admits notable classification As region belongs to same class, then rejecting and be not belonging to this kind of indivedual samples, and confirm to submit classification to, now, negative sample is user The sample rejected, positive sample is remaining sample in notable classification；

In user does not admit notable classification, most image-region belongs to same class, then select to belong to of a sort sample This confirmation is submitted to, and now positive sample is the sample that user selects, and negative sample is remaining sample in notable classification；

Positive sample, after user confirms to submit to, is noted as the label that user selectes, is then admitted to mark sample This concentration, the member relation of the most positive sample is used as the foundation of constraint propagation by step 2-3, optimum in negative sample and cluster result Other clustering cluster samples outside bunch are then sent back to training data and concentrate, and wait next round circulation；

Step 2-6, online multinuclear similarity learns: the target of measuring similarity study is optimization step 2-1 vacuum metrics ginseng Matrix number W so that during circulation mark, meet the sample data of the user view distance on metric space closer to, Simple distance is compared, and the tolerance that study obtains can make clustering cluster bigger, and purity is higher, i.e. user is more easy to admit notable classification and sends out Existing result so that mark burden reduces, owing to measuring similarity model needs iteration in cyclic process to update, uses OASIS algorithm realizes the study of online multinuclear measuring similarity；For tlv triple (x, an x⁺,x^-), wherein x represents any one Sample, x⁺Expression must belong to same category of sample, x with x^-Then represent and must adhere to different classes of sample separately with x, pass through OASIS algorithm optimization metric parameter matrix W makes characteristic similarity meet S_W(x,x⁺)>S_W(x,x^-)+1, i.e. require tlv triple about Bundle belongs to the characteristic similarity between similar sample, it is necessary to adhere to the phase between different classes of sample separately more than tlv triple constraint Add 1 like degree, S_W(x,x⁺) represent sample x and sample x⁺Between characteristic similarity, S_W(x,x^-) represent sample x and sample x^-Between Characteristic similarity, the renewal equation of metric parameter matrix is as follows:

W^t=W^t-1+ τ V,

Whereinl_wFor transfer indfficiency, t represents cycle-index, W^tRepresent t The metric parameter matrix obtained after secondary circulation, W^t-1The metric parameter matrix obtained after representing t-1 circulation, C_pIt is balance parameters, Generally value 0.1, the metric parameter matrix W that study obtains is in the characteristic similarity of step 2-1 calculates and updates；Step 2- In 6, (x, x⁺, x-) and=max (0,1-S_W(x,x⁺)+S_W(x,x^-)), V=x (x⁺,x-)^T。

Step 2-7, circulation mark terminates judging: the image entering circulation annotation process is carried out scale judgement, works as residue When in set of image regions to be marked, sample size is less than cluster numbers NC, or continuous several times is pushed to the notable class of user annotation Other sample size is very few, and user annotation burden is too high, now judges that the remaining sample that do not marks is not enough to find newly to go out significantly Classification gives user annotation, and circulation annotation process terminates, and training dataset (concentrate and comprise warp from Sample Buffer pond by training data Cross after data are loaded into and enter the data of cyclic process) in image-region be recycled in step 1-1 in Sample Buffer pond, etc. Circulation next time annotation process is opened the most afterwards until scale；Otherwise, when data scale is enough, i.e. more than threshold value T, then step 2-is proceeded to 1。

In step 2-7, described continuous several times is pushed to that the notable classification sample size of user annotation is very few refers to continuous 5 times It is pushed to the notable classification sample size of user annotation less equal than 3.

Step 3 includes: after circulation annotation process terminates, reclaim the sample set not being marked I.e. have sample that m is not marked to Sample Buffer pond B_p, it is also possible to Sample Buffer pond B_pPut into incremental image regionI.e. n new samples.

Beneficial effect: the invention have the advantages that first, the present invention can mark image-region progressively, it is not necessary to The all of labeled data of disposable submission, and use special feature combination to characterize image-region.Secondly, the present invention can obtain To meeting the multifarious annotation results of user view, more meet the feature of image-region mark.Finally, the present invention uses in groups Labeling form, annotating efficiency is higher, and along with user's minute mark note image increase, the present invention can increasingly be accurately obtained The mark of user is intended to, and reduces user and bears alternately.

Accompanying drawing explanation

Being the present invention with detailed description of the invention below in conjunction with the accompanying drawings and further illustrate, the present invention's is above-mentioned And/or otherwise advantage will become apparent.

Fig. 1 is the handling process schematic diagram of the present invention.

Fig. 2 is a width sampling art area image.

Fig. 3 a is that the notable class members of User Interface selects exemplary plot.

Fig. 3 b is classification mark exemplary plot after the notable class members of User Interface selects.

Detailed description of the invention

As it is shown in figure 1, the present invention discloses the online mask method of iterative image of a kind of data-driven, specifically include following Step:

Step 1, data prepare and feature extraction: send needing the set of image regions i.e. sample set entering circulation mark Enter Sample Buffer pond, and carry out feature extraction；

Step 2, circulation mark: to enter circulation annotation process image carry out scale judgement, scale enough after, by Line measuring similarity model carries out Similarity Measure to the feature extracted, and uses the clustering algorithm of constraint propagation optimization to not marking View data clusters, and selects notable classification, clusters most excellent bunch and is pushed to user and is labeled, and annotation results is used Study in constraint propagation and measuring similarity model updates, and final circulation mark obtains user's individual character to input picture region Change label；

Step 3, what recovery returned from circulation mark does not marks view data.

The idiographic flow of each step be described below:

Wherein, step 1 comprises the following steps:

Step 1-1, by initialized set of image regions(Represent certain image-region ,+for upper Mark, expression b is area image, and c is subscript herein, represents this image-region sequence location in set) send into Sample Buffer Pond B_p(subscript p represents that set B is the Buffer Pool that sample prepares), B_pComprise current all image-regions to be marked, when sample delays Rush pond sample size | B_p| more than threshold value T (threshold value generally takes 200), then sample in Sample Buffer pond is sent into circulation and marked Journey；

Step 1-2, extracts all sample characteristics preparing to be loaded into circulation annotation process from Sample Buffer pond, due to typically Picture format must be rectangular shape, use roi (region of interest) select need extraction feature any The image-region of shape, the feature extracted here includes document 8:Porebski, A., N.Vandenbroucke, and D.Hamad.“LBP histo-gram selection for supervised color texture classification.”Interna-tional Conference on Image Processing(ICIP),IEEE, Pp.3239 3243, by color histogram feature (RGB/HSV histogram), cold and warm tone feature described in 2013. (warmcold), location context (location contexts) composition.

Step 2 comprises the following steps:

Step 2-1, carries out Similarity Measure and renewal to the sample characteristics obtained in step 1-2: obtained by equation below The image-region of circulation annotation process must be enteredWithBetween (subscript i, j represents the sequence location of any two image-region) Characteristic similarity S_jj:

$S_W(b_i^{+},b_j^{+})=b_i^{{+}^T}{\mathrm{Wb}}_j^{+},$

Wherein W represents tolerance parameter matrix, obtains characteristic similarity distance matrixWherein N is The quantity of image-region in epicycle circulation；

Step 2-2, constraint propagation process: exist between sample data and must connect and can not connect, i.e. represent that sample must Must be same group or must can not be at same group, by all of sample annexation Y={Y_ij}_N×N(Y_ijFor the most different ThisBetween annexation) be initialized as two constraint setWithWhenTime Y_ij=1, whenTime Y_ij=-1, otherwise Y_ij=0, i.e.It is not belonging to an any of the above constraint set；

Propagated by the relation between the sample that below equation will mark and do not marked the relation between view data Constraint F^*:

$F^{*}={\left\{f_{ij}^{*}\right\}}_{N\&times;N},$

Wherein α is the parameter of codomain (0,1), and I is unit matrix；Wherein H is by characteristic similarity degree Moment matrixObtain,

WhenIt isK arest neighbors time,Otherwise H_ij=0, (S_ijRepresent measuring similarity Matrix M the i-th row, the similarity data of jth row, in like manner in S_iiAnd S_jj)

Then H=(H+H is calculated^T)/2 are to ensure that H is symmetrical matrix, and D is then diagonal matrix, wherein diagonal element (i, i) Equal to H the i-th row data sum；

The application uses document 9:Lu, Z., and Ip, H.H.S., " Constrained Spectral Clustering via Exhaustive and Efficient Constraint Propagation.”Computer Vi-sion–ECCV, The efficient constraint propagation utilizing k arest neighbors figure described in Springer Berlin Heidelberg, pp.1-14,2010. is calculated Method, obtains relation constraint；

Step 2-3, constrained clustering: the constraint that will obtain in step 2-2It is used for adjusting characteristic similarity Matrix M, obtains retraining similarityShown in equation below:

$\widetilde{S_W}(b_i^{+},b_j^{+})=\left\{\begin{array}{c}1-(1-f{*}_{ij})(1-H_{ij}),f{*}_{ij}\&GreaterEqual;0\\ (1+f{*}_{ij})H_{ij},f{*}_{ij}<0\end{array}\right.,$

Wherein H_ijBeing calculated in step 2-2, then the application uses document 10:Von Luxburg U. " A tutorial on spectral clustering.”Statistics and computing,2007,17(4):395-416. In spectral clustering (spectral clustering) to constraint adjust after similarityCluster, by image-region Cluster is NC (number of clusters) individual kind；

Step 2-4, notable class discovery: after cluster, calculated by below equation in the average class of each clustering cluster c away from:

$d_{\mathrm{int}ra}\left(c\right)=\frac{1}{\left|c\right|\&CenterDot;\left(\right|c|-1)}{\mathrm{\&Sigma;}}_{\begin{array}{c}i,j\&Element;c\\ i\&NotEqual;j\end{array}}d(b_i^{+},b_j^{+}),$

WhereinRepresent sampleWith sampleDistance in spectral clustering space, by average class away from minimum Clustering cluster c^*Elect optimum cluster bunch as, the notable classification i.e. found in this circulation；

Step 2-5, class members selects: class members therein is done by the notable classification that user obtains according to step 2-4 Select or reject operation: most of image-regions belong to same class in user admits notable classification, then rejecting is not belonging to this kind of Indivedual samples, and confirm submit to classification, now, negative sample is the sample that user rejects, positive sample be in notable classification remain Sample；

In user does not admit notable classification, most image-region belongs to same class, then select to belong to of a sort sample This confirmation is submitted to, and now positive sample is the sample that user selects, and negative sample is remaining sample in notable classification；

Positive sample, after user confirms to submit to, is noted as certain label that user selectes, is then admitted to mark sample This concentration, the member relation of the most positive sample is used as the foundation of constraint propagation by step 2-3, optimum in negative sample and cluster result Other clustering cluster samples outside bunch are then sent back to training data and concentrate, and wait next round circulation；

Step 2-6, online multinuclear similarity learns: the target of measuring similarity study is optimization step 2-1 vacuum metrics ginseng Matrix number W so that during circulation mark, meet the sample data of the user view distance on metric space closer to, Simple distance is compared, and the tolerance that study obtains can make clustering cluster bigger, and purity is higher, i.e. user is more easy to admit notable classification and sends out Existing result so that mark burden reduces, owing to measuring similarity model needs iteration in cyclic process to update, the application makes Use document 11:Chechik, G., Sharma, V., Shalit, U., Bengio, S., " Large scale online learning of image similarity through ranking.”The Journal of Machine Learning OASIS algorithm in Research 11pp.1109-1135,2010. realizes the study of online multinuclear measuring similarity；For one Tlv triple (x, x⁺,x^-), wherein x represents any one sample, x⁺Expression must belong to same category of sample, x with x^-Then represent Different classes of sample must be adhered to separately with x, make characteristic similarity meet S by OASIS algorithm optimization metric parameter matrix W_W (x,x⁺)>S_W(x,x^-)+1, i.e. require that tlv triple constraint belongs to the characteristic similarity between similar sample, it is necessary to more than ternary The similarity that group constraint adheres to separately between different classes of sample adds 1, and its renewal equation is as follows:

W^t=W^t-1+ τ V,

Whereinl_WFor transfer indfficiency, t represents cycle-index,

(x,x⁺,x^-)=max (0,1-S_W(x,x⁺)+S_W(x,x^-)),

C_pIt is balance parameters, V=x (x⁺,x^-)^T, the metric parameter matrix W that obtains of study is for the feature similarity of step 2-1 During degree calculates and updates；

Step 2-7, circulation mark terminates judging: the image entering circulation annotation process is carried out scale judgement, works as residue When in sample set to be marked, sample size is less than cluster numbers NC, or continuous several times (being typically set to 5 times) is pushed to user annotation Notable classification sample size very few (less equal than 3), user annotation burden is too high, now judges remaining not mark sample Being not enough to find newly to go out that notable classification gives user annotation, circulation annotation process terminates, the image-region that training data is concentrated Being recycled in step 1-1 in Sample Buffer pond, the scale of wait opens circulation next time annotation process the most afterwards；Otherwise, number is worked as According to scale enough (i.e. more than threshold value T), then proceed to step 2-1.

The application uses document 12:Galleguillos, C., McFee, B., and Lanckriet, G.R.G., “Iterative category discovery via multiple kernel metric learning.” Internation-al Journal of Computer Vision,Springer Berlin Heidelberg, Class discovery framework in Vol.108.1-2, pp.:115-132,2014. is as circulation mark flow process framework.

Embodiment

In the present embodiment, if Fig. 2 is a width sampled images, it is wherein image target area to be marked in black box, figure 3a is that the notable class members of User Interface selects exemplary plot, left side layout to be to show to be pushed to the aobvious of user in circulation every time Writing classification, the image that user selectes has dark border and occurs in image peripheral, confirms effect with display, and mouse floats to be selected Can present amplification effect time above image, in order to user's careful observation image detail, right side layout is class label page, Yong Hudian Hitting grey box and i.e. can generate a new class label, Fig. 3 b is classification mark after the notable class members of User Interface selects Note exemplary plot, i.e. user, after the member having selected notable classification, are selected the label classification on right side, are then used forward (positive) or reversely (negative) submits to annotation results to complete the mark of this circulation.

Specific implementation process is as follows:

In step one, prepare image-region sample data to be marked.Image-region sample to be marked is put into appointment In file, then perform characteristic extraction procedure, or the image area data that increment adds, equally put into specified folder In.

In step 2, in cyclic process, based on user mutual, complete the image-region sending into circulation annotation process The mark work of data.First characteristic is carried out Similarity Measure, then according to classification constraint, similarity is adjusted, Afterwards the distance metric adjusted is carried out spectral clustering, and pushes notable classification (optimum cluster bunch) and be labeled to user, as Fig. 3 a, in user admits notable classification, most of image-regions belong to same class, the most only need to select and to reject indivedual sample, so The right side class label (having green frame after Xuan Ding as selected prompting) of rear selected interactive interface, uses top on the left of interface Negative submit to button reversely submit annotation results to, now, unselected image-region will be placed into what user specified In the tally set of right side；Otherwise, in user does not admit notable classification, most image belongs to same class, then select and belong to same The sample of class, then selectes the right side class label (having dark border after Xuan Ding as selected prompting) of interactive interface, uses On the left of interface, the positive of top submits to button forward to submit annotation results to, and now, selected image-region will be placed into use In the right side tally set that family is specified.In negative sample and cluster result, optimum bunch of other outer clustering cluster samples will be sent back to train number According to collection, positive sample is first used as constraint propagation and updates, and subsequently into marking sample set, is then used as online multinuclear similarity The training of measurement model updates.

After being circulated throughout of certain number of times, residue sample is not enough to find new classification, and this time circulation annotation process is i.e. For terminating.

The invention provides the online mask method of iterative image of a kind of data-driven, implement the side of this technical scheme Method and approach are a lot, and the above is only the preferred embodiment of the present invention, it is noted that for the common skill of the art For art personnel, under the premise without departing from the principles of the invention, it is also possible to make some improvements and modifications, these improvements and modifications Also should be regarded as protection scope of the present invention.Each ingredient the clearest and the most definite in the present embodiment all can use prior art to be realized.

Claims (6)

1. the online mask method of iterative image of a data-driven, it is characterised in that comprise the following steps:

Step 1, data prepare and feature extraction: send needing the i.e. sample set of set of image regions entering circulation mark into sample This Buffer Pool, and carry out feature extraction；

Step 2, circulation mark: the feature extracted is carried out Similarity Measure by online measuring similarity model, use constraint Propagating the clustering algorithm optimized to cluster not marking image area data, select notable classification, most clustering excellent bunch pushes away Give user to be labeled, and the study that annotation results is used for constraint propagation and measuring similarity model updates, and finally circulates Mark obtains user's personalized labels to input picture region；

Step 3, reclaims the image area data that do not marks returned from circulation mark: after circulation annotation process terminates, reclaim The sample set not being marked is to Sample Buffer pond.

Method the most according to claim 1, it is characterised in that step 1 comprises the following steps:

Step 1-1, by initialized set of image regions i.e. sample setSend into Sample Buffer pond B_p,Table Showing the c image-region, subscript c represents this sample image region sequence location in set, Sample Buffer pond B_pComprise and work as Front all image-regions to be marked, when sample size in Sample Buffer pond | B_p| more than threshold value T, then by sample in Sample Buffer pond This feeding circulation annotation process；

Step 1-2, extracts all sample characteristics preparing to be loaded into circulation annotation process from Sample Buffer pond, including color histogram Figure feature, cold and warm tone feature and location context.

Method the most according to claim 2, it is characterised in that step 2 comprises the following steps:

Step 2-1, carries out Similarity Measure and renewal to the sample characteristics obtained in step 1-2: by equation below obtain into Enter to circulate the i-th image-region of annotation processWith jth image-regionBetween characteristic similarity

$S_W(b_i^{+},b_j^{+})={b_i^{+}}^T{\mathrm{Wb}}_j^{+},$

Wherein W represents tolerance parameter matrix, and the span of i and j is 1～c, then obtains characteristic similarity metric matrixThe quantity of image-region during wherein N is epicycle circulation；

Step 2-2, constraint propagation process: include between image area data connecting and two kinds of relations, i.e. tables can not be connected Show that image-region must be same group or all of image-region annexation Y can not be initialized at same group It is two constraint setWithY_ijRepresent i-th image-regionWith jth image-regionBetween Annexation, whenTime Y_ij=1；WhenTime Y_ij=-1；Otherwise Y_ij=0, i.e.Do not belong to In an any of the above constraint set；

Propagated by the relation between the sample that below equation will mark and do not marked the relation between image area data Constraint F^*:

$F^{*}={\left\{f_{ij}^{*}\right\}}_{N\&times;N},$

Wherein α is the parameter of codomain (0,1),The data of representing matrix the i-th row jth row, I is unit matrix；Wherein by characteristic similarity metric matrixObtaining matrix H, D is diagonal matrix, Wherein (i, i) equal to matrix H the i-th row data sum for diagonal element；

WhenIt it is i-th image-regionK arest neighbors time,Otherwise H_ij=0, S_ijRepresent similar Degree metric matrix M the i-th row, the similarity data of jth row, S_iiRepresent measuring similarity matrix M the i-th row, the similar number of degrees of the i-th row According to, S_jjRepresent the similarity data of measuring similarity matrix M jth row, jth row,

Then H=(H+H is calculated^T)/2W=(W+W^T)/2 with ensure HW be symmetrical matrix be symmetrical matrix；

Step 2-3, constrained clustering: the relation constraint that will obtain in step 2-2It is used for adjusting characteristic similarity degree Moment matrix M, obtains retraining similarityShown in equation below:

$\widetilde{S_W}(b_i^{+},b_j^{+})=\left\{\begin{array}{c}1-(1-f{*}_{ij})(1-H_{ij}),f{*}_{ij}\&GreaterEqual;0\\ (1+f{*}_{ij})H_{ij},f{*}_{ij}<0\end{array}\right.,$

Wherein H_ijW_ijStep 2-2 is calculated, the similarity after then constraint being adjustedCarry out spectral clustering, by image district Territory cluster is NC kind；

Step 2-4, notable class discovery: calculated by below equation in the average class of each clustering cluster c away from:

$d_{\mathrm{int}ra}\left(c\right)=\frac{1}{\left|c\right|\&CenterDot;\left(\right|c|-1)}{\mathrm{\&Sigma;}}_{\underset{i\&NotEqual;j}{i,j\&Element;c}}d(b_i^{+},b_j^{+}),$

WhereinRepresent i-th image-regionWith jth image-regionDistance in spectral clustering space, will be flat All away from minimum clustering cluster c in class^*Elect optimum cluster bunch as, the notable classification i.e. found in this circulation；

Step 2-5, class members selects: class members therein is selected by the notable classification that user obtains according to step 2-4 Or reject operation, according to the positive sample of the suitable operation of user and negative sample, positive sample, after user confirms to submit to, is noted as using The label that family is selected, is then admitted to mark in sample set, and the member relation of the most positive sample is used as about by step 2-3 The foundation that bundle is propagated, in negative sample and cluster result, optimum bunch of other outer clustering cluster samples are then sent back to training dataset In, wait next round circulation；

Step 2-6, online multinuclear similarity learns: for tlv triple (x, an x⁺,x^-), wherein x represents any one sample, x⁺Expression must belong to same category of sample, x with x^-Then represent and must adhere to different classes of sample separately with x, by OASIS algorithm Optimizing metric parameter matrix W makes characteristic similarity meet S_W(x,x⁺)>S_W(x,x^-)+1, i.e. require that tlv triple constraint belongs to same Characteristic similarity between the sample of class, it is necessary to the similarity adhered to separately between different classes of sample more than tlv triple constraint adds 1, S_W(x,x⁺) represent sample x and sample x⁺Between characteristic similarity, S_W(x,x^-) represent sample x and sample x^-Between feature phase Like degree, the renewal equation of metric parameter matrix is as follows:

W^t=W^t-1+ τ V,

Whereinl_WFor transfer indfficiency, t represents cycle-index, W^tRepresent t time and follow The metric parameter matrix obtained after ring, W^t-1The metric parameter matrix obtained after representing t-1 circulation, C_pIt is balance parameters, study The metric parameter matrix W obtained is in the characteristic similarity of step 2-1 calculates and updates；

Step 2-7, circulation mark terminates judging: the image entering circulation annotation process is carried out scale judgement, waits to mark when remaining When in note set of image regions, sample size is less than cluster numbers NC, or continuous several times is pushed to the notable class of user annotation very This quantity is very few, and user annotation burden is too high, now judges that the remaining sample that do not marks is not enough to find newly to go out notable classification Giving user annotation, circulation annotation process terminates, and the image-region that training data is concentrated is recycled to sample in step 1-1 and delays Rushing in pond, the scale of wait opens circulation next time annotation process the most afterwards；Otherwise, when data scale is enough, time i.e. more than threshold value T, Then proceed to step 2-1.

Method the most according to claim 3, it is characterised in that in step 2-6, (x, x⁺,x^-)=max (0,1-S_W(x,x⁺)+ S_W(x,x^-))。

Method the most according to claim 4, it is characterised in that in step 2-6, V=x (x⁺,x^-)^T,.

Method the most according to claim 5, it is characterised in that in step 2-7, described continuous several times is pushed to user annotation Notable classification sample size very few refer to continuous 5 times to be pushed to the notable classification sample size of user annotation less equal than 3.