CN104933093B - The monitoring of regional public sentiment and decision support system (DSS) based on big data and method - Google Patents

The monitoring of regional public sentiment and decision support system (DSS) based on big data and method Download PDF

Info

Publication number
CN104933093B
CN104933093B CN201510255995.0A CN201510255995A CN104933093B CN 104933093 B CN104933093 B CN 104933093B CN 201510255995 A CN201510255995 A CN 201510255995A CN 104933093 B CN104933093 B CN 104933093B
Authority
CN
China
Prior art keywords
analysis
public
public sentiment
data
regional
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201510255995.0A
Other languages
Chinese (zh)
Other versions
CN104933093A (en
Inventor
刘丽君
李成华
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Anhui Ziyunchuang Intelligent Technology Co.,Ltd.
Original Assignee
WUHAN TIPDM INTELLIGENT TECHNOLOGY Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by WUHAN TIPDM INTELLIGENT TECHNOLOGY Co Ltd filed Critical WUHAN TIPDM INTELLIGENT TECHNOLOGY Co Ltd
Priority to CN201510255995.0A priority Critical patent/CN104933093B/en
Publication of CN104933093A publication Critical patent/CN104933093A/en
Application granted granted Critical
Publication of CN104933093B publication Critical patent/CN104933093B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/90Details of database functions independent of the retrieved data types
    • G06F16/95Retrieval from the web
    • G06F16/955Retrieval from the web using information identifiers, e.g. uniform resource locators [URL]
    • G06F16/9566URL specific, e.g. using aliases, detecting broken or misspelled links
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/90Details of database functions independent of the retrieved data types
    • G06F16/95Retrieval from the web
    • G06F16/958Organisation or management of web site content, e.g. publishing, maintaining pages or automatic linking
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00Handling natural language data
    • G06F40/30Semantic analysis

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Databases & Information Systems (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Health & Medical Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Computational Linguistics (AREA)
  • General Health & Medical Sciences (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)

Abstract

A kind of monitoring of regional public sentiment and decision support system (DSS) based on big data:Information collection memory module carries out structured storage management for the public sentiment source information to acquisition, forms the regional big data public sentiment knowledge base of real-time update;Data preprocessing module forms complete orderly data set for being pre-processed to the data in regional big data public sentiment knowledge base;Big data the analysis of public opinion module carries out the analysis of public opinion for the much-talked-about topic etc. to specified requirements and trend prediction obtains the analysis of public opinion and trend prediction result;Public sentiment monitoring and early warning, decision assistant module are tracked, manage and are dredged for carrying out monitoring in real time to the customization public sentiment excavated and analyzed to the sensitive word in predefined sensitive dictionary;It is notified to policymaker by message, short message, lettergram mode in station by public sentiment is customized;Back Administration Module, for public feelings information Classification Management, user and rights management, keyword management, acquisition management, Content Management, special topic management and analysis report management.

Description

The monitoring of regional public sentiment and decision support system (DSS) based on big data and method
Technical field
The present invention relates to the analysis of public opinion technical field, more particularly to a kind of the monitoring of regional public sentiment and decision based on big data Auxiliary system and method.
Background technology
As Internet technology brings the rise of Internet communication, due to the virtual of network itself, concealment, diversity, The features such as permeability and randomness, more people are ready to show true idea using this channel of network.
Network has been acknowledged as " fourth media " after newspaper, broadcast, TV, becomes the master of reflection Social Public Feelings Want one of carrier.Since internet has the characteristics that virtual, concealment, diversity, permeability and randomness, more and more Netizen gladly by BBS forums, blog, news follow-up post, the channels such as be posted and express viewpoint, propagating thought and strengthened.Net The formation of the free public opinion of network, uncontrolled information are propagated, the lag of mass medium Building of Professional Ethics causes network information flow The out of control of amount generates negative impact to stablizing for society.
If guiding is not good at, negative network public-opinion will form social public security larger threat.Relevant departments are come Say how to reinforce timely monitoring, the effectively guiding, and the positive neutralizing to network public opinion crisis to network public opinion, to safeguarding Social stability promotes national development to have important practical significance.
With the fast development of China Internet, regional public sentiment monitoring has become weight in the analysis of public opinion action The part wanted.Regional public sentiment monitoring and decision assistant method and system of this kind based on big data, seek to use public sentiment skill Art carries out public sentiment monitoring analysis to given area and crowd, finds given area and the public sentiment crisis of crowd, timely processing in advance Crisis.For the public opinion public sentiment that some can have an impact society, public sentiment monitoring is carried out, event can be timely understood Dynamically, these mistakes, unfounded public opinion are correctly guided.Given area society can also be grasped by implementing regional public sentiment monitoring The meeting will of the people, by understanding mood, attitude, view and the opinion and behavior disposition of each stratum common people in this area, then to thing Part makes correct decision.
Foreign study present situation
TDT topic detection and trackings (the Topic Detection and Tracking) system in the U.S. is most well-known net Network the analysis of public opinion system, this concept result from 1996 earliest, at that time U.S. national defense Advanced Research Projects Agency (DARPA) basis Reality demand, propose to develop it is a kind of can in the case of no manual intervention automatic decision news data stream theme need It asks.In 1997, researcher started this demand and carries out Primary Study, and achieves some initial achievements, including establishes one For the beforehand research corpus of TDT researchs, the content of research includes finding the consistent text fragment of inherent theme, that is, provide one The continuous data flow (text or voice) of section, allows system to judge the boundary between two events, and can automatic decision new events Appearance and old affair part reproduction.Since 1998, under DARPA supports, American National Standard technical research institute
(NIST) topic detection and tracking international conference will be held every year, and carries out corresponding system evaluation.
TDT has related generally to 5 subtasks, is respectively:Report cutting, new report identification, association identification, topic detection and Topic Tracking.By complementing each other between this 5 subtasks, organic whole just constitutes, and TDT projects accumulate under study for action Abundant Algorithm of documents categorization found to the topic solved at present in Internet public opinion analysis and tracing problem has done and draws well Lead effect.
The practical effect of existing internet public feelings monitoring system is unsatisfactory, and main cause is existing system to adopting The emotional orientation analysis of the comment text collected is insufficient, does not establish good solution.If do not had in monitoring system Analysis to the Sentiment orientation of comment text will prevent it that cannot be established from effectively being automatically analyzed to internet public feelings Effective and quick public sentiment monitoring and warning mechanism is played, and then not can effectively prevent sprawling of the various negative reports in internet. On the other hand, though the system of public sentiment monitoring analysis is more at present, based on big data specifically for area, the public sentiment of region is supervised Control and decision system do not have but.Existing public sentiment monitoring system is in each impact factor of the regional public sentiment of no targetedly consideration In the case of, can not obtain targetedly, accurate public sentiment monitored results and provide effective information to aid in decision.
Invention content
In order to solve existing public sentiment monitoring system in the feelings without targetedly considering each impact factor of regional public sentiment Under condition, can not obtain targetedly, accurate public sentiment monitored results and the shortcomings that effective information to aid in decision is provided, carry Go out a kind of regional public sentiment monitoring based on big data and decision support system (DSS) and method.
A kind of monitoring of regional public sentiment and decision support system (DSS) based on big data comprising following module:
Information collection memory module, for acquiring the public sentiment source information in specific region in real time, and to the public sentiment source of acquisition Information carries out structured storage management, forms the regional big data public sentiment knowledge base of real-time update;
Data preprocessing module is formed complete for being pre-processed to the data in regional big data public sentiment knowledge base Orderly data set provides available data to be analyzed for subsequent big data the analysis of public opinion module;
Big data the analysis of public opinion module, the characteristics of being used for according to regional public sentiment monitoring analysis, for the relevant influence of public sentiment The factor establishes analytic unit library, analysis model library, carries out data model configuration by configurator in analysis model library, and pass through Data mining algorithm carries out mining analysis, much-talked-about topic, medium type angle of propagation to specified requirements to the data model of configuration Reprinting relational angle, media region angle of distribution between degree, media report importance angle, the just negative sound angle of media, media Degree etc. carries out the analysis of public opinion and trend prediction obtains the analysis of public opinion and trend prediction result;
Public sentiment monitoring and early warning, decision assistant module, for the sensitive word in predefined sensitive dictionary, according to predefined quick The customization public sentiment of the sensitive word various aspects of interest to regional the analysis of public opinion in sense dictionary is oriented excavation, analysis;And According to the analysis of public opinion and trend prediction as a result, sensitive word in predefined sensitivity dictionary to the customization public sentiment excavating and analyze into Row monitoring tracking in real time is managed and is dredged;And for by station in message, short message, lettergram mode will customization public sentiment notify to Policymaker;
Back Administration Module, for public feelings information Classification Management, user and rights management, keyword management, collection tube Reason, Content Management, special topic management and analysis report management.
A kind of monitoring of regional public sentiment and decision assistant method based on big data comprising following steps:
S1, in real time the public sentiment source information in acquisition specific region, and structured storage is carried out to the public sentiment source information of acquisition Management, forms the regional big data public sentiment knowledge base of real-time update;
S2, the data in regional big data public sentiment knowledge base are pre-processed, complete orderly data set is formed, after being Continuous big data the analysis of public opinion module provides available data to be analyzed;
S3, according to regional public sentiment monitoring analysis the characteristics of, for the relevant impact factor of public sentiment, establish analytic unit library, Analysis model library carries out data model configuration by configurator in analysis model library, and by data mining algorithm to configuration Data model carries out mining analysis, to the much-talked-about topic of specified requirements, medium type propagation angle, media report importance angle Degree, the reprinting relational angle between the just negative sound angle of media, media, media region distribution angle etc. carry out the analysis of public opinion and become Gesture is predicted to obtain the analysis of public opinion and trend prediction result;
Sensitive word in S4, predefined sensitive dictionary, according to the sensitive word in predefined sensitive dictionary to regional public sentiment The customization public sentiment for analyzing various aspects of interest is oriented excavation, analysis;And according to the analysis of public opinion and trend prediction as a result, Sensitive word in predefined sensitivity dictionary carries out monitoring tracking in real time, management to the customization public sentiment excavated and analyzed and dredges It leads;And for being notified to policymaker by message, short message, lettergram mode in standing by public sentiment is customized;
S5, to public feelings information Classification Management, user and rights management, keyword management, acquisition management, Content Management, specially Topic management and analysis report management.
The monitoring of regional public sentiment and decision support system (DSS) and method provided by the invention based on big data have beneficial below Effect:
1. stored by information collection, data acquisition, running efficiency of system are improved, solves the scalability of system and steady It is qualitative.It can automatically data acquire, automatic identification language and website coding are supported to a variety of webpage formats, various characters collection The acquisition of coding.By data prediction, it is effectively guaranteed in the case where data volume increasingly increases severely, what system high-speed calculated Efficiency.Meanwhile the mode of big data parallel memorizing, also solve the scalability of system, real-time and stability.
2. improving the promptness to event response, decision assistant support is provided.By big data the analysis of public opinion, solve existing The public feelings information of some the analysis of public opinion system acquisitions works for regional public sentiment monitoring analysis, " nothing caused by specific aim deficiency With information " excessively, caused by when an event initially occurs, fail to find and be pocessed in time, finally make thing State becomes serious, the expensive situation of the cost change of processing.The public sentiment monitoring and early warning function of the present invention, ensure that can be timely The dynamic of understanding event, the most quick negative public sentiment of early warning, correctly guides mistake, unfounded public opinion at the first time.Certainly Plan miscellaneous function can reach the social will of the people of grasp, by the mood, attitude, view, opinion and the row that understand given area resident For tendency, provides auxiliary to various decisions and support.
3. improving coverage rate, comprehensive and accuracy that regional public feelings information obtains.The letter of current public sentiment system acquisition Breath, often more commonization information, inevitably unilateral for doing regional the analysis of public opinion, and the data only collected by tradition Analysis, although ensure that certain fairness, due to not considering that enough impact factors, effect are examined relative to the present invention's The global data for having considered more impact factors still can be far short of what is expected.So the present invention can improve the comprehensive of acquisition of information, carry The coverage rate of high data and the accuracy of analysis.
Description of the drawings
Fig. 1 be the present invention implement based on big data regional public sentiment monitoring and decision support system (DSS) structure diagram;
Fig. 2 is the structure diagram of big data the analysis of public opinion module in Fig. 1;
Fig. 3 be the present invention implement based on big data regional public sentiment monitoring and decision assistant method flow diagram;
Fig. 4 is the sub-process figure of step S3 in Fig. 3.
Specific implementation mode
As shown in Figure 1, a kind of monitoring of regional public sentiment and decision support system (DSS) based on big data, which is characterized in that it is wrapped Include following module:
Information collection memory module, for acquiring the public sentiment source information in specific region in real time, and to the public sentiment source of acquisition Information carries out structured storage management, forms the regional big data public sentiment knowledge base of real-time update.
Optionally, the source of the public sentiment source information in information collection memory module in specific region include news analysis, BBS, blog, polymerization news, mhkc, community network media, microblogging, QQ group, e-newspaper, public number of wechat, news move answer Use program;Acquisition mode includes Meta Search Engine technology, is climbed using the self-defined sources URL of universal search engine and sample frequency, search Take specific public sentiment source information on internet.
The main source of public feelings information under network environment has:News analysis, BBS, blog, polymerization news (RSS) etc..It should The information collection memory module of invention mainly uses Meta Search Engine technology, utilizes the self-defined sources URL of universal search engine and sampling Frequency, extensively search crawl specific public sentiment source information on internet.Access pass-through needs registration, all types of websites logged in, opinion Altar, blog, multi-document summarization acquire comprehensively;It is adopted by the way that Meta Search Engine acquisition, RSS acquisitions and specified sites acquisition etc. are a variety of Collection approach realizes acquisition the whole network covering;It is encoded, is supported to a variety of webpage formats, a variety of words by automatic identification language and website The acquisition of symbol collection coding.
Aim at regional public sentiment monitoring customization, the acquisition oriented in real time, the whole network monitoring, with accurately automatic information collecting engine It is comprehensively three-dimensional to monitor the network medias such as news, forum, mhkc, blog, community in real time for core, while to microblogging, QQ groups, electricity The New Medias such as public number of sub- newpapers and periodicals, wechat, news movement app carry out data acquisition.And it based on the available data that can be integrated, carries The quality and efficiency of high public sentiment supervision.New reprinting is produced for newly generated public feelings information or existing public sentiment, it is new with The variations such as note are acquired, and are fed back, dynamic real-time update at the first time.Based on big data Hadoop parallel memorizing technologies, carry out The structured storage management of the efficient index of data integrates the related data of data with existing and network acquisition, forms real-time update Regional big data public sentiment knowledge base;Given area correlation public sentiment is actively discovered and collects, in conjunction with the available data that can be integrated, branch The regional big data public sentiment monitoring of support and the operation of decision assistant method and system.A variety of retrieval modes can intuitively provide public sentiment prison Control early warning.
Data preprocessing module is formed complete for being pre-processed to the data in regional big data public sentiment knowledge base Orderly data set provides available data to be analyzed for subsequent big data the analysis of public opinion module.
Optionally, carrying out pretreatment to the data in regional big data public sentiment knowledge base in data preprocessing module includes:
IP positioning, network address validity check are carried out to the data in regional big data public sentiment knowledge base, then pass through webpage Parsing, automatic identification, relatedness computation, document No. processing network public-opinion extractive technique carry out public feelings information extraction;Using The automatic duplicate removal of article removes duplicate data automatically with article similarity analysis discriminating step;Pass through text automatic identification and extraction skill Art, title automatic identification and extractive technique intelligently obtain public sentiment text, intelligence abstract and keyword;By garbage information filtering, Stop words filtration step carries out the pretreatment of data.
Junk information can be filtered out, useful information is left.
Big data the analysis of public opinion module, the characteristics of being used for according to regional public sentiment monitoring analysis, for the relevant influence of public sentiment The factor establishes analytic unit library, analysis model library, carries out data model configuration by configurator in analysis model library, and pass through Data mining algorithm carries out mining analysis, much-talked-about topic, medium type angle of propagation to specified requirements to the data model of configuration Reprinting relational angle, media region angle of distribution between degree, media report importance angle, the just negative sound angle of media, media Degree etc. carries out the analysis of public opinion and trend prediction obtains the analysis of public opinion and trend prediction result.
As shown in Fig. 2, optionally, big data the analysis of public opinion module includes:
Information extracting unit the characteristics of being used for according to regional public sentiment monitoring analysis, for the relevant impact factor of public sentiment, leads to Cross Chinese word segmentation, Metadata Extraction, autoabstract carry out information extraction to the much-talked-about topic of specified requirements.
Public sentiment studies and judges unit, for by subject detection, much-talked-about topic extraction, sensitive subjects identification to the information of extraction into Row public sentiment is studied and judged.
Negative judging unit, analysis is passed judgement on for passing through, and studying and judging result to public sentiment is negatively judged.
Automatic taxon classifies automatically to negative judging result for genetic algorithm category analysis related algorithm.
Specific analysis unit, for carrying out specific analysis to automatic taxon.
Hot spot cluster cell, for finding and tracking by automated intelligent cluster, incident analysis network public-opinion hot spot Technology carries out hot spot cluster.
Extensive diagnostic unit, for propagating trend analysis, sentiment classification, media distribution/importance analysis, Area distribution The network public-opinions sentiment classification technologies such as analysis are extended mining analysis, obtain to the probabilistic forecasting of future condition, obtain The analysis of public opinion and trend prediction result.
Big data the analysis of public opinion module can dynamic tracking information, find trend, find hot spot, can with multi-angle formed point Analysis considerably increases precision of analysis and comprehensive.
The realization of big data the analysis of public opinion module is natural language Intelligent treatment technology and big data depth digging technology Commercialization and functionization.The characteristics of module will be according to regional public sentiment monitoring analysis, for given area change in policy, current events political affairs Control, given area economic life, the social people's livelihood, anti-corruption, social morality, the various regional public sentiments such as resident population-employment it is relevant Impact factor establishes analytic unit library, analysis model library, carries out model configuration by analysis model configurator, and pass through data Mining algorithm carries out mining analysis to the data model of configuration, to the much-talked-about topic, medium type propagation angle, matchmaker of specified requirements Body report reprinting relational angle, media region distribution angle between importance angle, the just negative sound angle of media, media etc. into Row the analysis of public opinion and trend prediction.
Big data the analysis of public opinion flow is as shown in Figure 3, studied and judged by information extraction, public sentiment, negatively judged, classified automatically, Specific analysis, hot spot cluster and extensive diagnostic and etc. realize one by one.
Public sentiment monitoring and early warning, decision assistant module, for the sensitive word in predefined sensitive dictionary, according to predefined quick The customization public sentiment of the sensitive word various aspects of interest to regional the analysis of public opinion in sense dictionary is oriented excavation, analysis;And According to the analysis of public opinion and trend prediction as a result, sensitive word in predefined sensitivity dictionary to the customization public sentiment excavating and analyze into Row monitoring tracking in real time is managed and is dredged;And for by station in message, short message, lettergram mode will customization public sentiment notify to Policymaker.
The module will be to emphasis public sentiment by the modes early warning such as message, SMS, mail notification in station, and ultimately forms The analysis of public opinion is reported, is grasped public sentiment dynamic comprehensively for decision-making level, is made right opinion guiding, provide analysis foundation, accomplish in time Monitoring, auxiliary supervision provide aid decision support for the formulation of the relevant policies such as economic support.
Back Administration Module, for public feelings information Classification Management, user and rights management, keyword management, collection tube Reason, Content Management, special topic management and analysis report management.
Back Administration Module includes mainly Classification Management, user and rights management, keyword management, acquisition management, content Management, special topic management and analysis report manage this seven different functions.This seven different functions can be as needed, flexibly matches Current task is set, in backstage automatic running.Wherein analysis report management module supports the post-production of public sentiment to handle, provides carriage The Classification Management of feelings bulletin, chart and history public sentiment bulletin.
The present embodiment can be directed in current existing unitized public sentiment monitoring system, and public sentiment monitors the incisions such as impact factor The no specific aim of point, a kind of limitation of the not accurate enough practicality of analysis result, it is proposed that ground based on big data vertically segmented The method and system of area's public sentiment monitoring and decision assistant, solves the problems, such as that regional public sentiment monitoring is difficult to full automation. The present invention is stored using big data and depth digging technology, to the monitoring of regional public sentiment and decision assistant, to find region in advance Public sentiment crisis, timely processing Crisis, this relatively weak public sentiment monitoring direction have carried out beneficial complement.
As shown in figure 3, the embodiment of the present invention also provide it is a kind of based on big data regional public sentiment monitoring and decision assistant side Method comprising following steps:
S1, in real time the public sentiment source information in acquisition specific region, and structured storage is carried out to the public sentiment source information of acquisition Management, forms the regional big data public sentiment knowledge base of real-time update.
Optionally, the source of the public sentiment source information in step S1 in specific region includes news analysis, BBS, blog, polymerization News, mhkc, community network media, microblogging, QQ groups, e-newspaper, public number of wechat, news mobile applications;Acquisition side Formula includes Meta Search Engine technology, and using the self-defined sources URL of universal search engine and sample frequency, search crawls specific on internet Public sentiment source information.
S2, the data in regional big data public sentiment knowledge base are pre-processed, complete orderly data set is formed, after being Continuous big data the analysis of public opinion module provides available data to be analyzed.
Optionally, carrying out pretreatment to the data in regional big data public sentiment knowledge base in step S2 includes:
IP positioning, network address validity check are carried out to the data in regional big data public sentiment knowledge base, then pass through webpage Parsing, automatic identification, relatedness computation, document No. processing network public-opinion extractive technique carry out public feelings information extraction;Using The automatic duplicate removal of article removes duplicate data automatically with article similarity analysis discriminating step;Pass through text automatic identification and extraction skill Art, title automatic identification and extractive technique intelligently obtain public sentiment text, intelligence abstract and keyword;By garbage information filtering, Stop words filtration step carries out the pretreatment of data.
S3, according to regional public sentiment monitoring analysis the characteristics of, for the relevant impact factor of public sentiment, establish analytic unit library, Analysis model library carries out data model configuration by configurator in analysis model library, and by data mining algorithm to configuration Data model carries out mining analysis, to the much-talked-about topic of specified requirements, medium type propagation angle, media report importance angle Degree, the reprinting relational angle between the just negative sound angle of media, media, media region distribution angle etc. carry out the analysis of public opinion and become Gesture is predicted to obtain the analysis of public opinion and trend prediction result.
Optionally, as shown in figure 4, step S3 includes:
S31, according to regional public sentiment monitoring analysis the characteristics of, for the relevant impact factor of public sentiment, pass through Chinese word segmentation, member Data pick-up, autoabstract carry out information extraction to the much-talked-about topic of specified requirements.
S32, the information progress public sentiment of extraction is studied and judged by subject detection, much-talked-about topic extraction, sensitive subjects identification.
S33, it is analyzed by passing judgement on, result is studied and judged to public sentiment and is negatively judged.
S34, genetic algorithm category analysis related algorithm classify automatically to negative judging result.
S35, specific analysis is carried out to automatic taxon.
S36, find that carrying out hot spot with tracer technique gathers by automated intelligent cluster, incident analysis network public-opinion hot spot Class.
S37, the networks carriages such as trend analysis, sentiment classification, media distribution/importance analysis, Area distribution analysis are propagated Feelings sentiment classification technology is extended mining analysis, obtains the probabilistic forecasting to future condition, obtains the analysis of public opinion and becomes Gesture prediction result.
Sensitive word in S4, predefined sensitive dictionary, according to the sensitive word in predefined sensitive dictionary to regional public sentiment The customization public sentiment for analyzing various aspects of interest is oriented excavation, analysis;And according to the analysis of public opinion and trend prediction as a result, Sensitive word in predefined sensitivity dictionary carries out monitoring tracking in real time, management to the customization public sentiment excavated and analyzed and dredges It leads;And for being notified to policymaker by message, short message, lettergram mode in standing by public sentiment is customized.
S5, to public feelings information Classification Management, user and rights management, keyword management, acquisition management, Content Management, specially Topic management and analysis report management.
The step of method described in conjunction with the examples disclosed in this document or algorithm, can directly be held with hardware, processor The combination of capable software module or the two is implemented.Software module can be placed in random access memory, memory, read-only memory, Electrically programmable ROM, electricity can sassafras except in programming ROM, register, hard disk, moveable magnetic disc, CD-ROM or technical field institute it is public In the storage medium for the arbitrary other forms known.
It is understood that for those of ordinary skill in the art, can be conceived with the technique according to the invention and be done Go out various other corresponding changes and deformation, and all these changes and deformation should all belong to the protection model of the claims in the present invention It encloses.

Claims (6)

1. a kind of monitoring of regional public sentiment and decision support system (DSS) based on big data, which is characterized in that it includes following module:
Information collection memory module, for acquiring the public sentiment source information in specific region in real time, and to the public sentiment source information of acquisition Structured storage management is carried out, the regional big data public sentiment knowledge base of real-time update is formed;
Data preprocessing module is formed complete orderly for being pre-processed to the data in regional big data public sentiment knowledge base Data set, provide available data to be analyzed for subsequent big data the analysis of public opinion module, including to regional big data carriage Data in feelings knowledge base carry out IP positioning, network address validity check, then pass through web analysis, automatic identification, degree of correlation meter It calculates, document No. processing network public-opinion extractive technique carries out public feelings information extraction;It is similar to article using the automatic duplicate removal of article Degree analysis and distinguishing step removes duplicate data automatically;Pass through text automatic identification and extractive technique, title automatic identification and extraction Technical intelligence obtains public sentiment text, intelligence abstract and keyword;Data are carried out by garbage information filtering, stop words filtration step Pretreatment;
Big data the analysis of public opinion module, the characteristics of for according to regional public sentiment monitoring analysis, for the relevant impact factor of public sentiment, Analytic unit library, analysis model library are established, data model configuration is carried out by configurator in analysis model library, and dig by data It digs algorithm and mining analysis is carried out to the data model of configuration, to the much-talked-about topic, medium type propagation angle, media of specified requirements Report that the reprinting relational angle between the just negative sound angle of important angle, media, media, media region distribution angle carry out public sentiment Analysis and trend prediction obtain the analysis of public opinion and trend prediction result;
Public sentiment monitoring and early warning, decision assistant module, for the sensitive word in predefined sensitive dictionary, according to predefined sensitive word The customization public sentiment of the sensitive word various aspects of interest to regional the analysis of public opinion in library is oriented excavation, analysis;And according to The analysis of public opinion and trend prediction are as a result, the sensitive word in predefined sensitivity dictionary carries out in fact the customization public sentiment excavated and analyzed When monitoring tracking, manage and dredge;And for being notified to decision by message, short message, lettergram mode in standing by public sentiment is customized Person;
Back Administration Module, for public feelings information Classification Management, user and rights management, keyword management, acquisition management, content Management, special topic management and analysis report management.
2. the regional public sentiment monitoring based on big data and decision support system (DSS) as described in claim 1, it is characterised in that:
The source of public sentiment source information in information collection memory module in specific region includes news analysis, BBS, blog, polymerization News, mhkc, community network media, microblogging, QQ groups, e-newspaper, public number of wechat, news mobile applications;Acquisition side Formula includes Meta Search Engine technology, using by the self-defined sources URL of search engine and sample frequency, and search crawls specific on internet Public sentiment source information.
3. the regional public sentiment monitoring based on big data and decision support system (DSS) as described in claim 1, it is characterised in that:
Big data the analysis of public opinion module includes:
Information extracting unit, the characteristics of being used for according to regional public sentiment monitoring analysis, for the relevant impact factor of public sentiment, in Literary participle, Metadata Extraction, autoabstract carry out information extraction to the much-talked-about topic of specified requirements;
Public sentiment studies and judges unit, for carrying out carriage to the information of extraction by subject detection, much-talked-about topic extraction, sensitive subjects identification Feelings are studied and judged;
Negative judging unit, analysis is passed judgement on for passing through, and studying and judging result to public sentiment is negatively judged;
Automatic taxon classifies automatically to negative judging result for genetic algorithm category analysis related algorithm;
Specific analysis unit, for carrying out specific analysis to automatic taxon;
Hot spot cluster cell, for passing through automated intelligent cluster, the discovery of incident analysis network public-opinion hot spot and tracer technique Carry out hot spot cluster;
Extensive diagnostic unit, for propagating trend analysis, sentiment classification, media distribution/importance analysis, Area distribution analysis Network public-opinion sentiment classification technology is extended mining analysis, obtains the probabilistic forecasting to future condition, obtains public sentiment point Analysis and trend prediction result.
4. a kind of monitoring of regional public sentiment and decision assistant method based on big data, which is characterized in that it includes the following steps:
S1, in real time the public sentiment source information in acquisition specific region, and structured storage management is carried out to the public sentiment source information of acquisition, Form the regional big data public sentiment knowledge base of real-time update;
S2, the data in regional big data public sentiment knowledge base are pre-processed, forms complete orderly data set, be subsequent Big data the analysis of public opinion module provides available data to be analyzed, include to the data in regional big data public sentiment knowledge base into Row IP positioning, network address validity check, then handle network by web analysis, automatic identification, relatedness computation, document No. Public sentiment extractive technique carries out public feelings information extraction;It is gone automatically using the automatic duplicate removal of article and article similarity analysis discriminating step Fall duplicate data;By text automatic identification and extractive technique, title automatic identification and extractive technique intelligence acquisition public sentiment text, Intelligence abstract and keyword;The pretreatment of data is carried out by garbage information filtering, stop words filtration step;
S3, according to regional public sentiment monitoring analysis the characteristics of, for the relevant impact factor of public sentiment, establish analytic unit library, analysis Model library carries out data model configuration by configurator in analysis model library, and by data mining algorithm to the data of configuration Model carries out mining analysis, to the much-talked-about topic of specified requirements, medium type propagation angle, media report importance angle, matchmaker Reprinting relational angle, media region distribution angle between the just negative sound angle of body, media carry out the analysis of public opinion and trend prediction Obtain the analysis of public opinion and trend prediction result;
Sensitive word in S4, predefined sensitive dictionary, according to the sensitive word in predefined sensitive dictionary to regional the analysis of public opinion The customization public sentiment of various aspects of interest is oriented excavation, analysis;And according to the analysis of public opinion and trend prediction as a result, predetermined Sensitive word in the sensitive dictionary of justice carries out monitoring tracking in real time to the customization public sentiment excavated and analyzed, manages and dredge;And For being notified to policymaker by message, short message, lettergram mode in station by public sentiment is customized;
S5, public feelings information Classification Management, user and rights management, keyword management, acquisition management, Content Management, special topic are managed Reason and analysis report management.
5. the regional public sentiment monitoring based on big data and decision assistant method as claimed in claim 4, it is characterised in that:
The source of public sentiment source information in step S1 in specific region include news analysis, BBS, blog, polymerization news, mhkc, Community network media, microblogging, QQ groups, e-newspaper, public number of wechat, news mobile applications;Acquisition mode is searched including member Rope technology, using the self-defined sources URL of universal search engine and sample frequency, search crawls specific public sentiment source letter on internet Breath.
6. the regional public sentiment monitoring based on big data and decision assistant method as claimed in claim 4, it is characterised in that:
Step S3 includes:
S31, according to regional public sentiment monitoring analysis the characteristics of, for the relevant impact factor of public sentiment, pass through Chinese word segmentation, metadata It extracts, autoabstract carries out information extraction to the much-talked-about topic of specified requirements;
S32, the information progress public sentiment of extraction is studied and judged by subject detection, much-talked-about topic extraction, sensitive subjects identification;
S33, it is analyzed by passing judgement on, result is studied and judged to public sentiment and is negatively judged;
S34, genetic algorithm category analysis related algorithm classify automatically to negative judging result;
S35, specific analysis is carried out to automatic taxon;
S36, it finds to carry out hot spot cluster with tracer technique by automated intelligent cluster, incident analysis network public-opinion hot spot;
S37, trend analysis, sentiment classification, media distribution/importance analysis, Area distribution analysis network public-opinion tendency are propagated Property analytical technology is extended mining analysis, obtains the probabilistic forecasting to future condition, obtains the analysis of public opinion and trend prediction As a result.
CN201510255995.0A 2015-05-19 2015-05-19 The monitoring of regional public sentiment and decision support system (DSS) based on big data and method Active CN104933093B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201510255995.0A CN104933093B (en) 2015-05-19 2015-05-19 The monitoring of regional public sentiment and decision support system (DSS) based on big data and method

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201510255995.0A CN104933093B (en) 2015-05-19 2015-05-19 The monitoring of regional public sentiment and decision support system (DSS) based on big data and method

Publications (2)

Publication Number Publication Date
CN104933093A CN104933093A (en) 2015-09-23
CN104933093B true CN104933093B (en) 2018-08-07

Family

ID=54120261

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201510255995.0A Active CN104933093B (en) 2015-05-19 2015-05-19 The monitoring of regional public sentiment and decision support system (DSS) based on big data and method

Country Status (1)

Country Link
CN (1) CN104933093B (en)

Families Citing this family (54)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105446602B (en) * 2015-11-24 2019-04-16 努比亚技术有限公司 The device and method for positioning article keyword
CN105468768A (en) * 2015-12-07 2016-04-06 临沂大学 System monitoring method of WeChat public sentiment
CN105574090B (en) * 2015-12-10 2017-12-26 北京中科汇联科技股份有限公司 A kind of filtering sensitive words method and system
CN105447202A (en) * 2015-12-31 2016-03-30 宁波公众信息产业有限公司 Internet information collecting system
CN106777124B (en) * 2016-05-26 2018-06-22 中科鼎富(北京)科技发展有限公司 Semantic knowledge method, apparatus and system
CN107544988B (en) * 2016-06-27 2021-03-19 百度在线网络技术(北京)有限公司 Method and device for acquiring public opinion data
CN106331085A (en) * 2016-08-22 2017-01-11 成都天地网络科技有限公司 Operation-based big-data processing system
CN106354769A (en) * 2016-08-22 2017-01-25 成都天地网络科技有限公司 Large data cleaning processing system
CN106354846A (en) * 2016-08-31 2017-01-25 成都广电视讯文化传播有限公司 Intelligent news manuscript selection method and system based on big data
CN106599042B (en) * 2016-11-08 2020-06-23 北京百度网讯科技有限公司 Information pushing method and device based on artificial intelligence
CN106776755A (en) * 2016-11-16 2017-05-31 盐城工学院 A kind of information control system of Subject-oriented
CN107045497A (en) * 2017-05-04 2017-08-15 成都华栖云科技有限公司 A kind of quick newsletter archive content sentiment analysis system and method
CN108985054A (en) * 2017-06-05 2018-12-11 中国电信股份有限公司 Threaten intelligence analysis method and apparatus
CN107273488B (en) * 2017-06-13 2019-08-20 武汉大学 A kind of realistic space activity and cyberspace behavior space-time link evaluation of effect method
CN107203641A (en) * 2017-06-19 2017-09-26 北京易华录信息技术股份有限公司 A kind of method of the collection of Internet traffic public feelings information and processing
CN107330613A (en) * 2017-06-29 2017-11-07 平安万家医疗投资管理有限责任公司 A kind of public sentiment monitoring method, equipment and computer-readable recording medium
CN107391712A (en) * 2017-07-28 2017-11-24 王亚迪 A kind of network public opinion trend prediction analysis method
CN107704621A (en) * 2017-10-27 2018-02-16 西南财经大学 A kind of internet public feelings map visualization methods of exhibiting
CN107908694A (en) * 2017-11-01 2018-04-13 平安科技(深圳)有限公司 Public sentiment clustering method, application server and the computer-readable recording medium of internet news
CN107885873B (en) * 2017-11-28 2021-08-24 百度在线网络技术(北京)有限公司 Method and apparatus for outputting information
CN108052586A (en) * 2017-12-11 2018-05-18 上海壹账通金融科技有限公司 The analysis of public opinion method, system, computer equipment and storage medium
CN108364124B (en) * 2018-01-26 2022-01-07 天津中科智能识别产业技术研究院有限公司 International capacity cooperative risk assessment and decision service system based on big data
CN108287906A (en) * 2018-01-28 2018-07-17 江苏快页信息技术有限公司 A kind of public sentiment monitoring method based on instant messaging social software
CN108446270B (en) * 2018-03-06 2021-06-08 平安科技(深圳)有限公司 Electronic device, early warning method of system sensitive content and storage medium
CN108628994A (en) * 2018-04-28 2018-10-09 广东亿迅科技有限公司 A kind of public sentiment data processing system
CN108804527A (en) * 2018-04-28 2018-11-13 国家计算机网络与信息安全管理中心 Based on wechat region circle of friends data analysis system and method
CN108932291B (en) * 2018-05-23 2022-08-23 福建亿榕信息技术有限公司 Power grid public opinion evaluation method, storage medium and computer
CN109284432A (en) * 2018-08-22 2019-01-29 华东计算技术研究所(中国电子科技集团公司第三十二研究所) Network public opinion analysis system based on big data platform
CN109460922A (en) * 2018-11-13 2019-03-12 电子科技大学 A kind of Internet public opinion analysis and aid decision-making system with power industry feature
CN109543186B (en) * 2018-11-22 2023-12-19 奇安信科技集团股份有限公司 Public opinion information processing method, system, electronic equipment and medium
CN111931098A (en) * 2019-04-28 2020-11-13 北京仝睿科技有限公司 Monitoring object determination method and device and electronic equipment
CN110580565B (en) * 2019-05-27 2020-10-27 光控特斯联(上海)信息科技有限公司 AI heat prediction-based public population dispersion scheduling method and system
CN110502553A (en) * 2019-08-22 2019-11-26 武汉东湖大数据交易中心股份有限公司 A kind of aid decision-making method based on big data
CN110705288A (en) * 2019-09-29 2020-01-17 武汉海昌信息技术有限公司 Big data-based public opinion analysis system
CN110851489A (en) * 2019-10-09 2020-02-28 安徽今日互联科技有限公司 Internet public opinion monitoring system
CN110717111A (en) * 2019-10-15 2020-01-21 深圳迅策科技有限公司 Public opinion analysis method based on internet information
CN111539864B (en) * 2020-03-31 2023-07-11 中国刑事警察学院 Information analysis method and device for treading event based on LBS big data
CN111445369B (en) * 2020-03-31 2023-07-14 中国刑事警察学院 Urban large-scale aggregation activity information early warning method and device based on LBS big data
CN111461553A (en) * 2020-04-02 2020-07-28 上饶市中科院云计算中心大数据研究院 System and method for monitoring and analyzing public sentiment in scenic spot
CN111581480B (en) * 2020-05-12 2023-09-08 杭州风远科技有限公司 News information aggregation analysis method and system, terminal and storage medium
CN111611385A (en) * 2020-05-27 2020-09-01 中航信移动科技有限公司 Flight monitoring and early warning system and method based on public opinion analysis
CN111538888A (en) * 2020-06-05 2020-08-14 国网山东省电力公司检修公司 Network public opinion intensity evolution analysis system based on active monitoring engine and big data
CN112100535A (en) * 2020-09-16 2020-12-18 南京智数云信息科技有限公司 Network public opinion analysis system and method based on DFA algorithm
CN112989034A (en) * 2020-12-16 2021-06-18 中国人民解放军国防科技大学 Social service work quantitative tracking evaluation method based on open source information
CN112650947A (en) * 2020-12-31 2021-04-13 安徽不如信息科技有限公司 Public opinion collection processing system convenient to carry
CN112800308A (en) * 2021-01-30 2021-05-14 贵州工程应用技术学院 Big data-based public opinion monitoring platform
CN113139782A (en) * 2021-03-24 2021-07-20 湖南新浪信息服务有限公司 Intelligent control system for converged media
CN113128217B (en) * 2021-03-26 2024-04-02 航天科工智能运筹与信息安全研究院(武汉)有限公司 Public opinion disposition decision-making method based on network twinning space
CN113220533B (en) * 2021-05-21 2024-05-31 南京诺迈特网络科技有限公司 Network public opinion monitoring method and system
CN113850662A (en) * 2021-08-13 2021-12-28 厦门国际银行股份有限公司 Public opinion early warning processing system and method
CN114386422B (en) * 2022-01-14 2023-09-15 淮安市创新创业科技服务中心 Intelligent auxiliary decision-making method and device based on enterprise pollution public opinion extraction
CN115640463A (en) * 2022-11-18 2023-01-24 太极计算机股份有限公司 Internet public opinion monitoring and analyzing system
CN116128546A (en) * 2023-01-06 2023-05-16 河北科迪新能源科技有限公司 AI public opinion monitoring system and method for external service window in power industry
CN117370621A (en) * 2023-08-28 2024-01-09 郑州大学 Big data-based foreign language speech multilingual public opinion monitoring and early warning system

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101751458A (en) * 2009-12-31 2010-06-23 暨南大学 Network public sentiment monitoring system and method
CN102708096A (en) * 2012-05-29 2012-10-03 代松 Network intelligence public sentiment monitoring system based on semantics and work method thereof
CN103268350A (en) * 2013-05-29 2013-08-28 安徽雷越网络科技有限公司 Internet public opinion information monitoring system and monitoring method
CN103744877A (en) * 2013-12-20 2014-04-23 潘大庆 Public opinion monitoring application system deployed in internet and application method
CN104408157A (en) * 2014-12-05 2015-03-11 四川诚品电子商务有限公司 Funnel type data gathering, analyzing and pushing system and method for online public opinion

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20120323627A1 (en) * 2011-06-14 2012-12-20 Microsoft Corporation Real-time Monitoring of Public Sentiment

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101751458A (en) * 2009-12-31 2010-06-23 暨南大学 Network public sentiment monitoring system and method
CN102708096A (en) * 2012-05-29 2012-10-03 代松 Network intelligence public sentiment monitoring system based on semantics and work method thereof
CN103268350A (en) * 2013-05-29 2013-08-28 安徽雷越网络科技有限公司 Internet public opinion information monitoring system and monitoring method
CN103744877A (en) * 2013-12-20 2014-04-23 潘大庆 Public opinion monitoring application system deployed in internet and application method
CN104408157A (en) * 2014-12-05 2015-03-11 四川诚品电子商务有限公司 Funnel type data gathering, analyzing and pushing system and method for online public opinion

Also Published As

Publication number Publication date
CN104933093A (en) 2015-09-23

Similar Documents

Publication Publication Date Title
CN104933093B (en) The monitoring of regional public sentiment and decision support system (DSS) based on big data and method
CN105577679B (en) A kind of anomalous traffic detection method based on feature selecting and density peaks cluster
Thom et al. Spatiotemporal anomaly detection through visual analysis of geolocated twitter messages
Parekh et al. Studying jihadists on social media: A critique of data collection methodologies
Hussain et al. Web usage mining: A survey on preprocessing of web log file
Wang et al. DUET: Data-driven approach based on latent Dirichlet allocation topic modeling
Chang et al. Extreme user and political rumor detection on twitter
CN104573016A (en) System and method for analyzing vertical public opinions based on industry
CN108733791B (en) Network event detection method
CN104281607A (en) Microblog hot topic analyzing method
CN103544255A (en) Text semantic relativity based network public opinion information analysis method
CN110533212A (en) Urban waterlogging public sentiment monitoring and pre-alarming method based on big data
CN105808722A (en) Information discrimination method and system
CN106649498A (en) Network public opinion analysis system based on crawler and text clustering analysis
CN103885993A (en) Public opinion monitoring method and device for microblog
Sun et al. Efficient event detection in social media data streams
Hromic et al. Graph-based methods for clustering topics of interest in twitter
Jiang et al. An insider threat detection method based on user behavior analysis
Zhang et al. An efficient log parsing algorithm based on heuristic rules
Afyouni et al. Spatio-temporal event discovery in the big social data era
Ofli et al. Using artificial intelligence and social media for disaster response and management: an overview
Arif et al. Social network extraction: a review of automatic techniques
CN116723005A (en) Method and system for tracking malicious code implicit information under polymorphic hiding
Zhao et al. A system to manage and mine microblogging data
Bayat et al. Estimation of Twitter user's nationality based on friends and followers information

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
C10 Entry into substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant
TR01 Transfer of patent right
TR01 Transfer of patent right

Effective date of registration: 20230927

Address after: Room 026, 5th Floor, Building 6, Jinzheng Nanjing Science and Technology Park, No. 6 Fengxin Road, Yuhuatai District, Nanjing City, Jiangsu Province, 210000

Patentee after: Ziqing Jiayuan (Jiangsu) Elderly Care Industry Co.,Ltd.

Address before: A5 North 2-509, No. 999 Gaoxin Avenue, Donghu New Technology Development Zone, Wuhan City, Hubei Province, 430074

Patentee before: WUHAN TIPDM INTELLIGENT TECHNOLOGY Co.,Ltd.

TR01 Transfer of patent right
TR01 Transfer of patent right

Effective date of registration: 20240401

Address after: 242700 No. 8, Zone B, Xiuning High tech Electronic Information Pioneer Park, No. 30, Mount Huangshan North Road, Haiyang Town, Xiuning County, Mount Huangshan City, Anhui Province

Patentee after: Anhui Ziyunchuang Intelligent Technology Co.,Ltd.

Country or region after: China

Address before: Room 026, 5th Floor, Building 6, Jinzheng Nanjing Science and Technology Park, No. 6 Fengxin Road, Yuhuatai District, Nanjing City, Jiangsu Province, 210000

Patentee before: Ziqing Jiayuan (Jiangsu) Elderly Care Industry Co.,Ltd.

Country or region before: China