CN111428106A - Big data-based information collection and analysis system - Google Patents
Big data-based information collection and analysis system Download PDFInfo
- Publication number
- CN111428106A CN111428106A CN202010190796.7A CN202010190796A CN111428106A CN 111428106 A CN111428106 A CN 111428106A CN 202010190796 A CN202010190796 A CN 202010190796A CN 111428106 A CN111428106 A CN 111428106A
- Authority
- CN
- China
- Prior art keywords
- module
- data
- unit
- processing
- information collection
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Withdrawn
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/95—Retrieval from the web
- G06F16/951—Indexing; Web crawling techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/95—Retrieval from the web
- G06F16/953—Querying, e.g. by the use of web search engines
Landscapes
- Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computing Systems (AREA)
- Software Systems (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses an information collecting and analyzing system based on big data, relating to the technical field of data processing and analysis, and aiming at the problem of low efficiency of the existing information processing, the following scheme is proposed, the invention has simple structure and convenient use, and comprises a cloud database which is connected with a plurality of transfer stations, the transfer stations are connected with a plurality of user terminals, and the cloud database is connected with a processing center, can process data for many times, reduce the garbage information in the data, reduce the working strength of hardware equipment of each level, reduce the investment cost of hardware, simultaneously ensures the integrity, the practicability and the safety of the data, ensures the high efficiency and the accuracy of the data processing, in the subsequent data use process, the data can be quickly called, the whole data processing efficiency is improved, and the whole system is convenient to use.
Description
Technical Field
The invention relates to the technical field of data processing and analysis, in particular to an information collecting and analyzing system based on big data.
Background
Big data is a data set which cannot be captured, managed and processed by a conventional software tool within a certain time range, is a massive, high-growth-rate and diversified information asset which needs a new processing mode to have stronger decision-making power, insight discovery power and flow optimization capacity, and the development of big data is a trend of the internet development.
However, data in the big data is complex, so that not only can bad information setting and useless and repeated information exist, but also the processing intensity of devices in different levels is easily increased in the subsequent data processing or process, the hardware investment cost is increased, and the information processing efficiency is reduced.
Disclosure of Invention
The information collection and analysis system based on big data provided by the invention solves the problem of low information processing efficiency.
In order to achieve the purpose, the invention adopts the following technical scheme:
the utility model provides an information collection analytic system based on big data, includes high in the clouds database, high in the clouds database is connected with a plurality of transfer stations, the transfer station is connected with a plurality of user terminal, high in the clouds database is connected with processing center.
Preferably, a temporary storage module, a total storage module and a power supply module are arranged in the cloud database, and a sequencing unit, a calling unit, a grading unit and a plurality of storage units are arranged in the total storage module.
Preferably, the transfer station comprises a receiving module, the receiving module is connected with a prescreening module, and the prescreening module is connected with a sending module.
Preferably, a data acquisition module is arranged in the user terminal, the data acquisition module is connected with a data processing module, and the data processing module is connected with a data transceiver module and a data display module.
Preferably, the processing center comprises a filtering module, a classifying module, a comparing module and a screening module, wherein the comparing module is connected with a selecting module.
Preferably, the prescreening module comprises an arranging unit, a recycling unit, a processing unit and a transmission unit.
Preferably, the data transceiver module includes a data receiving unit, a data transmitting unit, a data buffering unit and a data defining unit.
Preferably, the selecting module comprises a copying unit, a caching unit and a waiting unit, and the screening module comprises a warning unit, a manual spot check unit and a backtracking unit.
The invention has the beneficial effects that:
1: the data can be processed for many times through the multi-level screening processing device, so that the junk information in the data is reduced, the working strength of hardware equipment of each level is reduced, the investment cost of hardware is reduced, and the integrity, the practicability and the safety of the data are guaranteed.
2: through classification, sequencing and comparison of different levels, the high efficiency and accuracy of data processing are guaranteed, meanwhile, in the subsequent data use process, data calling can be rapidly carried out, the efficiency of whole data processing is improved, and the whole system is convenient to use.
Drawings
FIG. 1 is a schematic diagram of a big data-based information collection and analysis system according to the present invention;
FIG. 2 is a schematic diagram of a transfer station of a big data-based information collection and analysis system according to the present invention;
fig. 3 is a schematic diagram of a user terminal of an information collection and analysis system based on big data according to the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments.
Referring to fig. 1-3, an information collection analytic system based on big data, including the high in the clouds database, the high in the clouds database is connected with a plurality of transfer stations, the transfer station is connected with a plurality of user terminals, the high in the clouds database is connected with processing center, the high in the clouds database carries out the classified storage of information, guarantees the realization of information data, the follow user terminal of data produces, in preliminary treatment and the transfer through the transfer station enters into the far-end database, then stores in the high in the clouds database after the processing of processing center again, awaits for use.
The system comprises a cloud database, a temporary storage module, a total storage module and a power supply module, wherein a sorting unit, a calling unit, a grading unit and a plurality of storage units are arranged in the total storage module, the temporary storage module receives information transmitted by a transfer station and temporarily stores the information, a processing center is waited to call the information, the total storage module stores final data and calls subsequent information, the grading unit sorts different information according to preset rules, the sorting unit sorts classified information according to certain specifications, the grading unit searches the graded information of the same type conveniently and quickly, the information data are finally stored in different storage units, and the system is waited for use and calls the information data through the calling unit when the information data are needed to be used.
The transfer station comprises a receiving module, the receiving module is connected with a primary screening module, the primary screening module is connected with a sending module, the receiving module receives data from a user terminal and data called in a total storage module, then the data enter the sending module through the primary screening module and are sent to a temporary storage module or the user terminal, when the data come from the user terminal, the primary screening module conducts preliminary screening on the data, virus information or other redundant information in the data is removed, and the processing intensity of a follow-up device is reduced.
The user terminal is internally provided with a data acquisition module, the data acquisition module is connected with a data processing module, and the data processing module is connected with a data receiving and transmitting module and a data display module.
The processing center includes the filter module, categorised module, contrast module and screening module, contrast module is connected with and selects for use the module, the filter module carries out further screening to the information and distinguishes, guarantee the integrality of data, practicality and security, the filter module is after accomplishing work, carry out different classification through categorised module, then enter into contrast module, select for use the module to transfer the information data that corresponds the type in well storage module and contrast this moment, reduce the pressure of repeated information data to the database, retrench the database simultaneously, promote the efficiency and the ability of the collection processing of data, final data are saved in getting into the total memory module through screening module mark.
The preliminary screening module is including arranging the unit, retrieving the unit, processing unit and transmission unit, arranges the unit and carries out certain buffering and sequencing to the data stream that user terminal gushed into, guarantees going on in order of the processing of data, and processing unit handles the screening to information data, and retrieval unit is kept in a certain time and is deleted the data for direct contrast alleviates processing unit's processing strength, and transmission unit carries out the transmission and the calling of inside data.
The data transceiver module comprises a data receiving unit, a data sending unit, a data caching unit and a data limiting unit, wherein the data receiving unit receives data, the data sending unit sends the data, and the data caching unit caches the data to ensure the data to be orderly processed.
The selecting module comprises a copying unit, a cache unit and a waiting unit, the screening module comprises a warning unit, a manual spot check unit and a backtracking unit, the copying unit copies information data to be compared in the total storage module and waits for the comparison of the comparison module, the cache unit can cache a plurality of data to be compared, the calling speed of the data is increased, the comparison efficiency is ensured, the waiting unit controls the data in the cache unit to ensure the accuracy of the compared data, the manual spot check module in the screening module can further process after the screening of the filtering module, ensure the quality of information processing, meanwhile, the modules are warned and recorded through the warning module, so that the information can cross the primary screening module and the filtering module to be studied and evolved for a long time, and the backtracking unit marks different information streams so as to be convenient for the subsequent examination of the corresponding original user terminal.
The working principle is as follows: in the data collection stage, data is generated by a data acquisition module in a user terminal, different information data is converted into a uniform format by a data processing module and then enters a data transceiver module for transmission, information in a plurality of data transceiver modules is transmitted to a receiving module of a transfer station for receiving, the information enters a primary screening module for primary screening and processing of the data, then enters a sending module for sending, and reaches a temporary storage module for later processing, when a processing center processes the previous data, a filtering module calls the data in the temporary storage module for filtering of the information to remove garbage information in the data, then enters a classification module for sorting and then enters a comparison module for comparison with the data called in a selection module to complete further processing of the data, and finally, performing final processing through a screening module, and entering a storage unit in a main storage module for classified storage.
In the data sending stage, when a data requirement is received, a calling module in a main memory module quickly finds corresponding information data through a sorting module and a grading module, then calls the data, receives the data through a receiving module in a transfer station, then directly reaches a data receiving and sending module through a prescreening module to send the data to a data receiving and sending module in a user terminal, and carries out format conversion on the data through a data processing device to enter a data display module for acute display.
In the description of the present invention, it is to be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", "clockwise", "counterclockwise", and the like, indicate orientations and positional relationships based on those shown in the drawings, and are used only for convenience of description and simplicity of description, and do not indicate or imply that the equipment or element being referred to must have a particular orientation, be constructed and operated in a particular orientation, and thus, should not be considered as limiting the present invention.
Furthermore, the terms "first", "second" and "first" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of the present invention, "a plurality" means two or more unless specifically defined otherwise.
The above description is only for the preferred embodiment of the present invention, but the scope of the present invention is not limited thereto, and any person skilled in the art should be considered to be within the technical scope of the present invention, and the technical solutions and the inventive concepts thereof according to the present invention should be equivalent or changed within the scope of the present invention.
Claims (8)
1. The utility model provides an information collection analytic system based on big data, includes high in the clouds database, its characterized in that, the high in the clouds database is connected with a plurality of transfer stations, the transfer station is connected with a plurality of user terminal, the high in the clouds database is connected with processing center.
2. The big-data-based information collection and analysis system according to claim 1, wherein a temporary storage module, a total storage module and a power supply module are arranged in the cloud database, and a sorting unit, a calling unit, a grading unit and a plurality of storage units are arranged in the total storage module.
3. The big data-based information collection and analysis system according to claim 1, wherein the transfer station comprises a receiving module, the receiving module is connected with a prescreening module, and the prescreening module is connected with a sending module.
4. The big data-based information collection and analysis system according to claim 1, wherein a data collection module is arranged in the user terminal, the data collection module is connected with a data processing module, and the data processing module is connected with a data transceiver module and a data display module.
5. The big data-based information collection and analysis system according to claim 1, wherein the processing center comprises a filtering module, a classifying module, a comparing module and a screening module, and the comparing module is connected with a selecting module.
6. The big data based information collecting and analyzing system of claim 3, wherein the prescreening module comprises an arranging unit, a recycling unit, a processing unit and a transmitting unit.
7. The big data-based information collection and analysis system according to claim 4, wherein the data transceiver module comprises a data receiving unit, a data transmitting unit, a data buffering unit and a data defining unit.
8. The big data-based information collection and analysis system according to claim 5, wherein the selection module comprises a copy unit, a cache unit and a wait unit, and the screening module comprises an alert unit, a manual spot check unit and a backtracking unit.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010190796.7A CN111428106A (en) | 2020-03-18 | 2020-03-18 | Big data-based information collection and analysis system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010190796.7A CN111428106A (en) | 2020-03-18 | 2020-03-18 | Big data-based information collection and analysis system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111428106A true CN111428106A (en) | 2020-07-17 |
Family
ID=71548129
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010190796.7A Withdrawn CN111428106A (en) | 2020-03-18 | 2020-03-18 | Big data-based information collection and analysis system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111428106A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112182066A (en) * | 2020-09-27 | 2021-01-05 | 高维智慧社会信息咨询(江苏)有限公司 | Big data information mining system based on fuzzy theory |
| CN112212916A (en) * | 2020-09-25 | 2021-01-12 | 常熟云开智能科技有限公司 | Be applicable to off-line and high in clouds equipment state acquisition system |
-
2020
- 2020-03-18 CN CN202010190796.7A patent/CN111428106A/en not_active Withdrawn
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112212916A (en) * | 2020-09-25 | 2021-01-12 | 常熟云开智能科技有限公司 | Be applicable to off-line and high in clouds equipment state acquisition system |
| CN112182066A (en) * | 2020-09-27 | 2021-01-05 | 高维智慧社会信息咨询(江苏)有限公司 | Big data information mining system based on fuzzy theory |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111428106A (en) | Big data-based information collection and analysis system | |
| CN105224690B (en) | Generate and select the method and system of the executive plan of the corresponding sentence containing ginseng | |
| CN114238388A (en) | Heterogeneous data collection and retrieval system based on multiple protocols | |
| CN113319007A (en) | Household garbage sorting method based on machine vision recognition | |
| CN113777257A (en) | Water quality online monitoring big data analysis method, system and storage medium | |
| CN111814026A (en) | WeChat applet-based garbage classification system | |
| CN116618333A (en) | Waste sorting system and method based on Internet of Things technology | |
| EP3086238A1 (en) | Visual search method, system and mobile terminal | |
| CN110807174A (en) | Effluent analysis and abnormity identification method for sewage plant group based on statistical distribution | |
| CN216271192U (en) | Dangerous waste integration categorised collection system | |
| CN113051234A (en) | Mobile on-site big data analysis platform | |
| CN114002402A (en) | Water quality monitoring dynamic evaluation and early warning system | |
| CN107103756A (en) | Congestion in road degree judgment means and method based on the public transport operation time | |
| CN114064997A (en) | Artificial intelligence power dispatching decision-making system based on big data | |
| CN113680685A (en) | Garbage classification treatment system based on recoverability | |
| CN113641542A (en) | Detection system and detection method of measurement and control equipment | |
| CN113449326A (en) | Industrial big data analysis system based on multi-source heterogeneous data processing | |
| CN109872078A (en) | A kind of economic management cost control system | |
| CN112668895A (en) | Digital resource quality supervision system | |
| CN117312634B (en) | Artificial intelligence data integration and propagation processing system | |
| CN211768016U (en) | Intelligent garbage clearing and transporting system | |
| CN2646802Y (en) | Speed limiting monitoring apparatus for highway | |
| CN216117822U (en) | Novel large-scale equipment state monitoring wireless terminal | |
| CN114138856A (en) | Efficient data acquisition and data analysis processing system | |
| CN114510402A (en) | System application level performance monitoring system and method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WW01 | Invention patent application withdrawn after publication |
Application publication date: 20200717 |
|
| WW01 | Invention patent application withdrawn after publication |