CN103455639A - Method and device for recognizing microblog burst hotspot events - Google Patents

Abstract

The invention provides a method and a device for recognizing microblog burst hotspot events. The method and the device are used for overcoming shortcomings that variation of trends is analyzed without contents, or events are discovered completely on the basis of the contents and the like in the prior art. The method for recognizing the microblog burst hotspot events includes extracting microblog topic labels of all hotspot events and recording the publish time, author information and a hotspot degree of each topic label; computing three measurement values of each topic label; judging whether each hotspot event is a burst event or not according to magnitudes of the corresponding three measurement values of the hotspot event. Each hotspot degree refers to occurrence numbers of the corresponding topic label within different time periods. The three measurement values of each topic label include an instability degree, an online topic possibility degree and label author information entropy.

Description

A kind of method and device of identifying microblogging burst focus incident

Technical field

The present invention relates to network information Intelligent treatment field, relate in particular to a kind of method and device of identifying microblogging burst focus incident.

Background technology

The main approaches of focus incident discovery technique is the modeling to " sudden ", usually, by analyzing the variation of sequential, detects the burst point in information flow.These class methods are only considered the variation tendency of information, from the angle of mathematics, the integral body of information flow are analyzed.Classical works is that the automat of take that Jon Kleinberg sets up information flow (Email, news etc.) is basic hierarchical model, and information flow is carried out to modeling.Along with the time changes, the nominal growth of observed quantity or minimizing make automat enter different states, and the sequence of these state transitions can be built into tree-like Grade Model, therefore by the variation of tracking mode, the generation of burst phenomenon can be detected.On this basis, Ahmed etc. build a figure by the various states of different content, utilize the annexation completion status of node in figure to shift, thereby find the characteristics of different transfer modes, to find the topic of burst.These methods are all to find the burst point in sequence of values.And the people such as Yang are to online Media, particularly the timing variations pattern of micro-blog (patterns of temporal variation) is analyzed, and adopts clustering algorithm, and similar pattern is got together, and can identify different topics.The time series pattern that the difference of this mode and automaton model is to investigate a plurality of elementary cells (word or phrase), carry out cluster to a plurality of patterns, can identify a plurality of burst topics, and not only utilize numerical information.

For investigating the semanteme in accident, the people such as the people such as Figueiredo, Zubiaga are with word feature or metamessage (meta data, as the classification of an information, time etc.) as the tolerance of information, the introducing of these content characteristics can be weighed the degree of diffusion of information more accurately.Further, Tu etc., Pervin etc., Mathioudakis etc., Pavlyshenko people using word as topic, the component units of event, topic and event are to mean on the vector space of word.Under this expression, different examples is carried out to cluster or subject analysis, can obtain content-based topic detection result.These methods all need text is carried out to participle, utilize afterwards word for feature, adopt the methods such as cluster, topic model or frequent-item, form topic.Therefore the expression of any event or topic, the feature of word in the text that places one's entire reliance upon.If text representation is different, the topic of finding so can disperse.Particularly for same event, if the different angles that people talk the matter over, different factor (as time, place, personage etc.), the method for expressing that the word of take is feature can be identified as different topics, makes an event be dispersed into a plurality of events, has reduced the estimation to its influence degree.

In sum, existing focus incident is found the correlative study work present situation, or breaks away from event content, only analyzes numerical value trend and changes; The text of the content that perhaps places one's entire reliance upon (word), do not have dirigibility.

Summary of the invention

(1) technical matters that will solve

The objective of the invention is, a kind of method and device of identifying microblogging burst focus incident is provided, in order to overcome, prior art breaks away from the variation of content analysis trend or the content that places one's entire reliance upon is carried out the deficiencies such as event discovery.

(2) technical scheme

For solving the problems of the technologies described above, the invention provides a kind of method of identifying microblogging burst focus incident, comprising:

Extract the microblog topic label of all focus incidents, and record issuing time, author information and the popular degree of each topic label; Wherein, described popular degree refers to the occurrence number in different time sections;

For described each topic label, calculate three metrics of described each topic label; Wherein, three metrics are respectively instability degree, online topic possibility degree and label author information entropy;

Judge according to the height of described three metrics whether corresponding focus incident is accident.

Preferably, judge according to the height of described three metrics whether corresponding focus incident is that accident comprises:

Judge whether described instability degree is greater than first threshold, online topic possibility degree whether is less than Second Threshold and whether label author information entropy is greater than the 3rd threshold value;

If judge that corresponding focus incident is as accident;

If not, judge that corresponding focus incident is as non-accident.

Preferably, described instability degree is calculated by following formula:

$\mathrm{Inst}\left(\mathrm{hashtag}\right)=\frac{1}{n}\underset{\tilde{P}\left(x\right)<p,\&ForAll;x}{\mathrm{\&Sigma;}}\mathrm{Inst}\left(x\right),$

Wherein, n is for normalized number of days, the time period that language material covers; refer to the probability of occurrence of point of instability x; P refers to the tolerance probability of prior appointment; Inst(x) refer to the instability degree of point of instability x, defined by following formula:

$\mathrm{Inst}\left(x\right)=\log \left(\frac{1}{\tilde{P}\left(x\right)+\mathrm{\&epsiv;}}\right),$

Wherein, ε > 0, be a little real number, for eliminating zero error.

Preferably, described

by following formula, calculate:

Wherein, 2 μ-x is the symmetric points about μ at x coordinate direction x, and F () means cumulative distribution function, by following formula, calculates:

$F(x;\mathrm{\&mu;},\mathrm{\&sigma;})=\frac{1}{\mathrm{\&sigma;}\sqrt{2\mathrm{\&pi;}}}{\&Integral;}_{-\&infin;}^x\exp (-\frac{{(t-\mathrm{\&mu;})}^2}{2{\mathrm{\&sigma;}}^2})\mathrm{dt}.,$

Wherein, μ is the expectation that distributes, σ ²for variance.

Preferably, described distribution expectation μ, variances sigma ²by following formula, calculate respectively:

${\widehat{\mathrm{\&mu;}}}_i={\stackrel{-}{X}}_i=\frac{1}{7}\underset{k=i-7}{\overset{i-1}{\mathrm{\&Sigma;}}}{x}_k,$

${{\widehat{\mathrm{\&sigma;}}}^2}_i=S_i^2=\frac{1}{7-1}\underset{k=i-7}{\overset{i-1}{\mathrm{\&Sigma;}}}{(x_k-{\stackrel{-}{X}}_i)}^2,$

Wherein, μ is

σ ²be

Preferably, the online topic possibility degree that calculates described topic label comprises:

The use location probability and the character that calculate described topic label form probability;

Described use location probability and character are formed to probability multiplication, obtain the online topic possibility degree of described topic label.

Preferably, the use location probability that calculates described topic label calculates by following formula:

Wherein, Pr _pos(h) being the use location probability of topic label, function | A| is the element number containing set A.

Preferably, the character formation probability that calculates described topic label calculates by following formula:

${\mathrm{Pr}}_{\mathrm{word}}\left(h\right)=1-\frac{N}{L}$

Wherein, Pr _word(h) be the character formation probability of topic label, L is tag length, the word number of N for forming.

Preferably, described label author information entropy calculates by following formula:

$\mathrm{Ent}\left(\mathrm{hashtag}\right)=-\underset{i=1}{\overset{k}{\mathrm{\&Sigma;}}}\frac{c_i}{n}\&CenterDot;\log \left(\frac{c_i}{n}\right),$

Wherein, Ent (hashtag) means the author information entropy of label hashtag, and k means number, c _imean to have issued i bar microblogging, n is ∑ c _i.

For solving the problems of the technologies described above, the present invention also adopts another kind of technical scheme: a kind of device of identifying microblogging burst focus incident is provided, comprises:

Extraction unit, for extracting the microblog topic label of all focus incidents, and record issuing time, author information and the popular degree of each topic label; Wherein, described popular degree refers to the occurrence number in different time sections;

Computing unit, for for described each topic label, calculate three metrics of described each topic label; Wherein, three metrics are respectively instability degree, online topic possibility degree and label author information entropy;

Judging unit, judge for the height according to described three metrics whether corresponding focus incident is accident.

Concrete, judging unit comprises:

Judgment sub-unit, for judging whether described instability degree is greater than first threshold, online topic possibility degree whether is less than Second Threshold and whether label author information entropy is greater than the 3rd threshold value;

The first identifying unit, in the situation that described instability degree is greater than first threshold, online topic possibility degree is less than Second Threshold and label author information entropy is greater than the 3rd threshold value, judges that corresponding focus incident is as accident;

The second identifying unit, be used in the situation that described instability degree is not more than first threshold, online topic possibility degree is not less than Second Threshold and label author information entropy is not more than the 3rd threshold value, judges that corresponding focus incident is as non-accident.

(3) beneficial effect

Be different from background technology, the present invention proposes and take the method for the identification microblogging burst focus incident that the microblog topic label is clue.That this invention closely is connected, has with social event for the microblogging focus incident is sudden, propagate several characteristics such as wide, utilizes the reflection of microblog topic label to text subject, has realized the event discovery procedure based on the microblog topic label.The present invention measures three aspects such as information entropy of the instability degree of popular label, online topic possibility degree and label author, take these three dimensions as basis, realize the division to the microblog topic Label space, realized the discovery of focus incident with this by the classification to popular label.With classic method, compare, the present invention has overcome some shortcomings, as break away from content analysis trend and change, or the content that places one's entire reliance upon is carried out the event discovery, and can adopt this reflection content of label not rely on the characteristics of content, utilize the reflection of label to content of text, theme, be not subject to the restriction of vocabulary, language simultaneously, catch the microblogging of describing same event from multi-angle.

The accompanying drawing explanation

Fig. 1 is one of method schematic diagram of the embodiment of the present invention one identification microblogging burst focus incident;

Fig. 2 be the embodiment of the present invention one identification microblogging burst focus incident the method schematic diagram two;

Fig. 3 is three-view diagram and example tag present position in space of Label space of the present invention;

Fig. 4 is one of module diagram of the embodiment of the present invention two identification microblogging burst focus incidents;

Fig. 5 be the embodiment of the present invention two identification microbloggings burst focus incidents module diagram two.

Embodiment

Below in conjunction with drawings and Examples, the specific embodiment of the present invention is described in further detail.Following examples are used for the present invention is described, but are not used for limiting the scope of the invention.

Embodiment mono-

Refer to Fig. 1, present embodiment provides a kind of method of identifying microblogging burst focus incident.In the present invention, the internet language material can be used as the Data Source that extracts the microblog topic label.At first in step S10, extract the microblog topic label of all focus incidents, and record issuing time, author information and the popular degree of each topic label; Wherein, described popular degree refers to the occurrence number in different time sections.In certain embodiments, the microblog topic label is word or the phrase initial with the # symbol: in the concrete embodiment as Sina's microblogging, and No. # (be a pair of No. # a drawn together microblogging) of microblog topic label for occurring in pairs.In some other embodiment, the microblog topic label can also pass through other label symbol marks.

The microblog topic label has disclosed the theme of microblogging text, event is had to certain susceptibility, popular microblog topic label can reflect hot ticket to a great extent, and we weigh the burst possibility of the corresponding hot ticket of a certain microblog topic label by step S20.

It specifically, in step S20, for described each topic label, calculates three metrics of described each topic label; Wherein, three metrics are respectively instability degree, online topic possibility degree and label author information entropy.In the present invention, the effect of these three metrics is respectively: the instability degree of microblog topic label can be used for weighing the sudden of focus incident, whether online topic possibility degree is popular online topic for the differentiation event, and the label author information drips for removing the microblogging such as commercial paper marketing class.

In the present embodiment, instability degree, online topic possibility degree and label author information entropy mean by Inst, TMP, Ent respectively.Concrete, the microblog topic label hashtag of take is example, below introduces respectively the computation process of above-mentioned three metrics, is specially (1), (2) and (3).

(1) the instability degree of microblog topic label can be used for weighing the sudden of focus incident.In the present embodiment, the instability degree of microblog topic label hashtag is calculated by following formula:

$\mathrm{Inst}\left(\mathrm{hashtag}\right)=\frac{1}{n}\underset{\tilde{P}\left(x\right)<p,\&ForAll;x}{\mathrm{\&Sigma;}}\mathrm{Inst}\left(x\right),$

Wherein, n is for normalized number of days, the time period that language material covers;

refer to the probability of occurrence of point of instability x; P refers to the tolerance probability of prior appointment; Inst(x) refer to the instability degree of point of instability x.Above formula mainly means the instability degree Inst(hashtag of a microblog topic label) by the unstable degree Inst(x of a series of point of instability x) cumulative forming.In the present embodiment, Inst(x) by following formula, defined:

$\mathrm{Inst}\left(x\right)=\log \left(\frac{1}{\tilde{P}\left(x\right)+\mathrm{\&epsiv;}}\right),$

Wherein, ε>0, be a little real number, for eliminating except zero error; refer to the probability of occurrence of point of instability x.In above-mentioned preferred embodiment, weigh the probability that some x occurs

if calculated by establishing probability model:

Wherein, 2 μ-x is the symmetric points of x about μ, and F () means cumulative distribution function, by following formula, calculates:

$F(x;\mathrm{\&mu;},\mathrm{\&sigma;})=\frac{1}{\mathrm{\&sigma;}\sqrt{2\mathrm{\&pi;}}}{\&Integral;}_{-\&infin;}^x\exp (-\frac{{(t-\mathrm{\&mu;})}^2}{2{\mathrm{\&sigma;}}^2})\mathrm{dt}.,$

Wherein, μ is the expectation that distributes, σ ²for variance.

Concrete, when actual computation, the expectation that distributes, variance are the observation sample value (x that adopts 7 skylight openings ₁～x ₇), expectation, the variance of the 8th day made an estimate, expectation μ, variances sigma distribute ²by following formula, calculate respectively:

${\widehat{\mathrm{\&mu;}}}_i={\stackrel{-}{X}}_i=\frac{1}{7}\underset{k=i-7}{\overset{i-1}{\mathrm{\&Sigma;}}}{x}_k,$

${{\widehat{\mathrm{\&sigma;}}}^2}_i={\mathrm{<figure-callout\; id="2"\; label="S\; i"\; filenames="HDA0000389063720000021.png"\; state="\{\{state\}\}">S</figure-callout>}}_i^{}=\frac{1}{7-1}\underset{k=i-7}{\overset{i-1}{\mathrm{\&Sigma;}}}{(x_k-{\stackrel{-}{X}}_i)}^2,$

Wherein, μ is

σ ²be

Based on above-mentioned tolerance, instability degree Inst (hashtag) is higher, means that sudden change (uprushing or the anticlimax) property of label hashtag is stronger, more may reflect the burst focus incident.On the contrary, the instability degree is lower, and label may be more to continue type, and reflection is that continue, stable event.

(2) whether online topic possibility degree is popular online topic for the differentiation event.

As everyone knows, popular online topic and social event have essential distinction, and the operating characteristic of microblog topic label is also different.According to its difference, calculate the online topic possibility degree of described topic label and realize by following steps:

The use location probability and the character that calculate described topic label form probability;

Described use location probability and character are formed to probability multiplication, obtain the online topic possibility degree of described topic label.Concrete computation process is as follows, comprises (21), (22) and (23).

(21) in the present embodiment, the probability that this tolerance of use location probability of topic label appears at beginning of the sentence by topic label h is realized, by following formula, calculates:

Wherein, Pr _pos(h) being the use location probability of topic label, function | A| is the element number (being microblogging quantity) containing set A.

(22) character of topic label forms this tolerance of probability and realizes in the following manner: recording the microblog topic tag length is L, and the word number of composition is N, and the probability be comprised of the word of these numbers forms probability in order to the character that means the topic label.Concrete, the character that calculates described topic label forms probability and calculates by following formula:

${\mathrm{Pr}}_{\mathrm{word}}\left(h\right)=1-\frac{N}{L}$

Wherein, Pr _word(h) be the character formation probability of topic label, L is tag length, the word number of N for forming.In order to understand better this formula, at this, illustrate for example: here according to given English word dictionary, calculate a coupling word number N that label is minimum, for example, dictionary comprises 4 words { art, arthur, thursday, day}, label #arthursday can be divided into: a/r/thursday or arthur/s/day, word number N is 3.

(23) comprehensive above-mentioned calculating, weigh label hashtag and represent that the degree TMP of online topic possibility is: TMP (hashtag)=Pr _pos(hashtag) Prword (hashtag).

TMP is higher for this degree value, and the theme of label reflection may be more online topic; Otherwise may be more the microblogging focus incident.

(3) the label author information drips for removing the microblogging such as commercial paper marketing class.

Consider that the user that label relates to is how many, the present invention weighs user's degree of scatter of a certain label of contribution with information entropy.Remember that the related whole microbloggings of label are total to the n bar, the author who issues these microbloggings is total to k people and has issued respectively c ₁, c ₂..., c _kbar microblogging (∑ c _i=n), label hashtag author entropy Ent () calculates for passing through following formula:

$\mathrm{Ent}\left(\mathrm{hashtag}\right)=-\underset{i=1}{\overset{k}{\mathrm{\&Sigma;}}}\frac{c_i}{n}\&CenterDot;\log \left(\frac{c_i}{n}\right),$

Wherein, Ent (hashtag) means the author information entropy of label hashtag, and k means number, c _imean i bar microblogging, n is ∑ c _i.In the present invention, adopt the degree of scatter tolerance of label author information entropy as label contribution author, entropy is higher, means that the user of corresponding semiotics is overstepping the bounds of propriety loose, affects larger; Otherwise seldom, influence power is limited, does not form important microblogging focus incident for the user who propagates.

Front is high for the burst degree of microblogging focus incident, corresponding to social event, propagate three characteristics such as wide, three tolerance have been proposed, i.e. instability degree Inst, online topic possibility degree TMP and label author's information entropy Ent.Because these three tolerance are separate, therefore the space of microblog topic label can be meaned by these three dimensions, the value of a label on each dimension just reflected its characteristic, discloses its corresponding event type.According to suitable Threshold, each dimension is divided into to height two parts.Wherein, there is the label of higher instability degree, low online topic possibility degree and higher information entropy, represent accident.

In the present embodiment, specifically by step S30, carry out the judgement of accident.In step S30, according to the height of described three metrics, judge whether corresponding focus incident is accident.Refer to Fig. 2, concrete, S30 completes deterministic process by following steps.

S301, judge whether described instability degree is greater than first threshold, online topic possibility degree whether is less than Second Threshold and whether label author information entropy is greater than the 3rd threshold value;

If S302, judge that corresponding focus incident is as accident;

S303, if not, judge that corresponding focus incident is as non-accident.

In order to verify validity and reliability of the present invention, we take the Twitter microblogging and have carried out the related experiment that event is found as application case.Experiment adopts two different time sections, crosses over the different microblogging data set of duration.Data set 1 comprises from the part Twitter micro-blog information in June, 2009 to Dec.Data set 2 comprises from the Twitter micro-blog information in year February in Dec, 2011 to 2012.2,000,000 above bar microbloggings are all arranged the every day of raw data set, through sampling (10,000 of every days), obtain the experimental data collection.The sampling set that comes from 6 months data is designated as Tweet6, and coming from 3 months data samplings is Tweet3.These data sets all comprise the multilingual microbloggings such as English, Japanese, Arabic, Chinese.

At first experiment extracts the popular label (front 1%) of data centralization.Nearly 600 of these labels, quantity accounts for the 0:015% – 0:016% of total amount.From these popular labels, 250 labels of sampling out respectively, amount to 500 and analyzed.Because label itself may have ambiguity, (the non English language text automatic translation is for English) showed in the text sampling that the microblog topic label need to be related to while therefore marking, and asks two makers to be marked the classification of label simultaneously.If annotation results is inconsistent, third maker will be marked this label.If still can not get most results, this label will be dropped.Finally, the Tweets6 data set retains 191 labels, and the Tweets3 data set retains 200 labels.

Assorting process adopts the threshold value of microblog topic label center of distribution point as classification.Instable probability model adopts Gaussian distribution.Existing labeling algorithm relates to angle has nothing in common with each other, and the content normally reflected from label (as music class, sport category etc.) or function (represent descriptor or disclose context) angle is studied, and does not pay close attention to focus incident.The sorting technique of comparatively extensively being quoted (Kwak etc., 2010) in Twitter is selected in experiment.In this sorting technique, label is divided into 4 classes according to the pattern (popularity pattern) of its popularity degree on two dimensions.One of them dimension means that its fashion trend comes from external cause (exogenous) or internal cause (endogenous); Another dimension means that the growth of its trend is in critical point (critical) or sub-critical point (subcritical).Wherein, " external cause+subcritical ", " external cause+critical ", " internal cause+critical " correspond respectively to Twitter tradition idiom, breaking news and continuation news.Because the method does not relate to user profile, therefore there is no classification advertisement marketing class microblogging is distinguished to some extent.To the classification results of this 4 classification problem as following table 1 with as shown in table 2.The document classification algorithm means with Popularity Pattern, and the algorithm that the present invention is based on subspace means with Subspace.The performance data of each classification is listed precision, recall rate and F successively ₁value.From the experimental result of form, can find out that most of classification results all can obtain certain effect.Because the microblog topic tag class is various, the nicety of grading of work on hand is generally limited, as

in 0.4 left and right, and, for nicety of grading, the recall rate of event class, only have respectively 0.13,0.06 Deng precision in people's document.

Following table (i.e. table 1) is the classification results of microblog topic label in Tweets6, and in table, P, R, F1 mean precision, recall rate and F1 value successively.

Table 1

The classification results of microblog topic label in following table (i.e. table 2) Tweets3, in table, P, R, F1 mean precision, recall rate and F1 value successively.

Table 2

Above-mentioned experiment take " on two dimensions, being divided into 4 classes " the present invention is carried out to the principle introduction as example, below in conjunction with concrete data, " on three dimensions, being divided into 8 classes " of the present invention (by three metrics, weighing accidents) set forth.The present invention all has adaptability widely for internet text microblogging language material.But, for the convenience of describing, the Twitter microblogging language material of below take is example, identification focus incident and accident wherein.

The microblogging language material used is as described in testing above, uses respectively the Twitter data set of 6 months and 3 months, and sampling microblogging number is respectively 2,028,794 and 778,395.The microblogging number that wherein comprises label is respectively 212, and 035(10.5%), with 106,327(13.7%), total number of labels is respectively 296,895 and 138,758.

The event recognition algorithm carried out on above-mentioned language material basis, at first calculated three metrics of each label.The result of calculation of some example tag, and residing position is as shown in Figure 3 in Label space.This figure, except embodying the label distribution with temperature, has also indicated two present positions of data concentrated part example tag in Label space.For example, label #sopa(numbers A) instability degree Inst=2.3, online topic possibility degree TMP=0.04, author information entropy Ent=4.9, therefore it is positioned at the upper left corner in the TMP-Inst of three-view diagram view, be arranged in top, the right side () in the Ent-Inst view, be arranged in bottom, the right side () in the Ent-TMP view.

In three dimensions, author information entropy Ent is more responsive to the marketing account.Have the microblog topic label of low entropy, have more concentrated contribution author, usually all propagandized by these accounts, advertisement, marketing characteristic are stronger.Their differences on other two dimensions may cause specifically marketing difference of classification, as described below:

1. have the popular label of low instability degree Inst, quantity is more stable, and the use of label is also comparatively lasting.For example label #abbeydawn is contributed by the robot account of some automatic issuing microblogs, and these microbloggings are all publicizing and supporting Canadian singer Avril Lavigne; Another label #bring1dtonyc is the abbreviation of " bring one day to New York City ", by the robot account, is issued, and is participating in activity that enlarges the city influence power of the U.S..These robot active degrees are not high, operation all the time, though therefore microblogging, the label quantity of issue are not noticeable especially, but comparatively lasting.

2. there is the popular label of higher instability degree, likely embody accident.Yet still only by a small amount of account (propelling movement person) contribution, the event of their representatives is not still true social event due to these labels.As previously mentioned, #property, #praytweets remain by marketing account issue.In addition, label #belieber is also for publicizing the marketing class label of Canadian pop singer Justin Bieber.In fact, this type of marketing originally intention increases singer and its backer's (bean vermicelli) interaction, but the microblogging in data set shows, some marketing accounts are made use of the name of Justin and issued these microbloggings, attempts to allow bean vermicelli concern oneself.This is characteristics of microblogging marketing, but can effectively distinguish this type of marketing content by this method.

Above-mentioned two class labels, have lower author information entropy, according to its instability degree, the kind of advertisement marketing can be divided into to lasting robot releasing advertisements and the sales publicity series advertisements of burst.

Microblog topic label with higher author information entropy, its contribution user is more, affects the crowd large.For distinguishing focus incident and online topic, need to investigate another dimension: online topic possibility degree TMP.For example, the in progress song of label #nowplaying(), #iaintafraidtosay(I ain ' t afraid to say, I do not fear ...) etc. be online topic.It should be noted that in two data sets the instability degree difference of label #nowplaying: in the Tweets6 data set, number of labels increases suddenly in the end one month, and its quantity is comparatively stable in the Tweets3 data set.Therefore in any case, this label only consists of the word of two full spellings, has higher online topic possibility, can identify this label and may be more online topic but not social event.

Other has the label of low online topic possibility, is likely still the traditional custom word in microblogging, #musicmonday that the #followfriday that for example recommends the user to pay close attention to or recommendation user listen to etc.These words, owing to forming tradition, may adopt abbreviation to mean, or appear at the optional position (but not beginning of the sentence) of microblogging text.This operating characteristic and event class label are comparatively close, only adopt aforementioned two kinds of tolerance may not distinguish well this type of label.But their popularity degree has determined the burst degree that it is lower.Therefore, instability degree Inst can distinguish this type of idiom and microblogging focus incident: only have the label of higher instability degree, the microblogging focus incident that just may pay close attention to corresponding to people.These microblog topic labels comprise the #sopa of aforementioned exemplary, also has the abbreviation of #hcr(Health Care Reform, the motion of society of being caused by U.S. medical reform) and #halamadrid(for cheering to Real Madrid football club during football match, particularly during the shooting).

Above-mentioned example explanation is concentrated in real data, and the present invention can identify microblogging burst focus incident and accident.

In conjunction with described above, the present invention proposes and take the method for the identification microblogging burst focus incident that the microblog topic label is clue.That this invention closely is connected, has with social event for the microblogging focus incident is sudden, propagate several characteristics such as wide, utilizes the reflection of microblog topic label to text subject, has realized the event discovery procedure based on the microblog topic label.The present invention measures three aspects such as information entropy of the instability degree of popular label, online topic possibility degree and label author, take these three dimensions as basis, realize the division to the microblog topic Label space, realized the discovery of focus incident with this by the classification to popular label.Experimental result has shown the validity of this invention in multilingual microblogging.In addition, this invention can also identify some advertisement marketing class microbloggings and marketing account, can distinguish popular online topic and true social event.These are all that classic method is not available.With classic method, compare, the present invention has overcome some shortcomings, as break away from content analysis trend and change, or the content that places one's entire reliance upon is carried out the event discovery, and can adopt this reflection content of label not rely on the characteristics of content, utilize the reflection of label to content of text, theme, be not subject to the restriction of vocabulary, language simultaneously, catch the microblogging of describing same event from multi-angle.

Embodiment bis-

Refer to Fig. 4 and Fig. 5, the present embodiment provides a kind of device of identifying microblogging burst focus incident, comprising: extraction unit 41, computing unit 42 and judging unit 43.In the present invention, the internet language material can be used as the Data Source that extracts the microblog topic label.

Extraction unit 41, for extracting the microblog topic label of all focus incidents, and record issuing time, author information and the popular degree of each topic label; Wherein, described popular degree refers to the occurrence number in different time sections.In certain embodiments, the microblog topic label is word or the phrase initial with the # symbol: in the concrete embodiment as Sina's microblogging, and No. # (be a pair of No. # a drawn together microblogging) of microblog topic label for occurring in pairs.In some other embodiment, the microblog topic label can also pass through other label symbol marks.

Computing unit 42, for for described each topic label, calculate three metrics of described each topic label; Wherein, three metrics are respectively instability degree, online topic possibility degree and label author information entropy.In the present invention, the effect of these three metrics is respectively: the instability degree of microblog topic label can be used for weighing the sudden of focus incident, whether online topic possibility degree is popular online topic for the differentiation event, and the label author information drips for removing the microblogging such as commercial paper marketing class.

In the present embodiment, the computation process that computing unit 42 calculates three metrics all, does not repeat them here as shown in the of one as implemented.

Judging unit 43, judge for the height according to described three metrics whether corresponding focus incident is accident.

Concrete, judging unit 43 comprises: judgment sub-unit 431, the first identifying unit 432 and the second identifying unit 433.

Judgment sub-unit 431, for judging whether described instability degree is greater than first threshold, online topic possibility degree whether is less than Second Threshold and whether label author information entropy is greater than the 3rd threshold value.

The first identifying unit 432, in the situation that described instability degree is greater than first threshold, online topic possibility degree is less than Second Threshold and label author information entropy is greater than the 3rd threshold value, judges that corresponding focus incident is as accident.

The second identifying unit 433, be used in the situation that described instability degree is not more than first threshold, online topic possibility degree is not less than Second Threshold and label author information entropy is not more than the 3rd threshold value, judges that corresponding focus incident is as non-accident.

The Data support of each unit of the present embodiment and corresponding technique effect all can be provided by embodiment mono-, do not repeat them here.

The foregoing is only embodiments of the invention; not thereby limit the scope of the claims of the present invention; every equivalent structure or conversion of equivalent flow process that utilizes instructions of the present invention and accompanying drawing content to do; or directly or indirectly be used in other relevant technical fields, all in like manner be included in scope of patent protection of the present invention.

Claims (10)

1. a method of identifying microblogging burst focus incident, is characterized in that, comprising:

Extract the microblog topic label of all focus incidents, and record issuing time, author information and the popular degree of each topic label; Wherein, described popular degree refers to the occurrence number in different time sections;

For described each topic label, calculate three metrics of described each topic label; Wherein, three metrics are respectively instability degree, online topic possibility degree and label author information entropy;

Judge according to the height of described three metrics whether corresponding focus incident is accident.

2. method according to claim 1, is characterized in that, according to the height of described three metrics, judges whether corresponding focus incident is that accident comprises:

Judge whether described instability degree is greater than first threshold, online topic possibility degree whether is less than Second Threshold and whether label author information entropy is greater than the 3rd threshold value;

If judge that corresponding focus incident is as accident;

If not, judge that corresponding focus incident is as non-accident.

3. method according to claim 1, is characterized in that, described instability degree is calculated by following formula:

$\mathrm{Inst}\left(\mathrm{hashtag}\right)=\frac{1}{n}\underset{\tilde{P}\left(x\right)<p,\&ForAll;x}{\mathrm{\&Sigma;}}\mathrm{Inst}\left(x\right),$

Wherein, n is for normalized number of days, the time period that language material covers;

refer to the probability of occurrence of point of instability x; P refers to the tolerance probability of prior appointment; Inst(x) refer to the instability degree of point of instability x, defined by following formula:

$\mathrm{Inst}\left(x\right)=\log \left(\frac{1}{\tilde{P}\left(x\right)+\mathrm{\&epsiv;}}\right),$

Wherein, ε > 0, be a little real number, for eliminating zero error.

4. method according to claim 3, is characterized in that, described by following formula, calculate:

Wherein, 2 μ-x is the symmetric points about μ at x coordinate direction x, and F () means cumulative distribution function, by following formula, calculates:

$F(x;\mathrm{\&mu;},\mathrm{\&sigma;})=\frac{1}{\mathrm{\&sigma;}\sqrt{2\mathrm{\&pi;}}}{\&Integral;}_{-\&infin;}^x\exp (-\frac{{(t-\mathrm{\&mu;})}^2}{2{\mathrm{\&sigma;}}^2})\mathrm{dt}.,$

Wherein, μ is the expectation that distributes, σ ²for variance.

5. method according to claim 4, is characterized in that, described distribution expectation μ, variances sigma ²by following formula, calculate respectively:

${\widehat{\mathrm{\&mu;}}}_i={\stackrel{-}{X}}_i=\frac{1}{7}\underset{k=i-7}{\overset{i-1}{\mathrm{\&Sigma;}}}{x}_k,$

${{\widehat{\mathrm{\&sigma;}}}^2}_i=S_i^2=\frac{1}{7-1}\underset{k=i-7}{\overset{i-1}{\mathrm{\&Sigma;}}}{(x_k-{\stackrel{-}{X}}_i)}^2,$

Wherein, μ is

σ ²be

6. method according to claim 1, is characterized in that, the online topic possibility degree that calculates described topic label comprises:

The use location probability and the character that calculate described topic label form probability;

Described use location probability and character are formed to probability multiplication, obtain the online topic possibility degree of described topic label.

7. method according to claim 6, is characterized in that, the use location probability that calculates described topic label calculates by following formula:

Wherein, Pr _pos(h) being the use location probability of topic label, function | A| is the element number containing set A.

8. method according to claim 6, is characterized in that, the character that calculates described topic label forms probability and calculates by following formula:

${\mathrm{Pr}}_{\mathrm{word}}\left(h\right)=1-\frac{N}{L},$

Wherein, Pr _word(h) be the character formation probability of topic label, L is tag length, the word number of N for forming.

9. method according to claim 1, is characterized in that, described label author information entropy calculates by following formula:

$\mathrm{Ent}\left(\mathrm{hashtag}\right)=-\underset{i=1}{\overset{k}{\mathrm{\&Sigma;}}}\frac{c_i}{n}\&CenterDot;\log \left(\frac{c_i}{n}\right),$

Wherein, Ent (hashtag) means the author information entropy of label hashtag, and k means number, c _imean to have issued i bar microblogging, n is ∑ c _i.

10. a device of identifying microblogging burst focus incident, is characterized in that, comprising:

Extraction unit, for extracting the microblog topic label of all focus incidents, and record issuing time, author information and the popular degree of each topic label; Wherein, described popular degree refers to the occurrence number in different time sections;

Computing unit, for for described each topic label, calculate three metrics of described each topic label; Wherein, three metrics are respectively instability degree, online topic possibility degree and label author information entropy;

Judging unit, judge for the height according to described three metrics whether corresponding focus incident is accident.

Images