CN110619572A - Method for monitoring high fault tolerance growth of enterprise public data - Google Patents
Method for monitoring high fault tolerance growth of enterprise public data Download PDFInfo
- Publication number
- CN110619572A CN110619572A CN201910890868.6A CN201910890868A CN110619572A CN 110619572 A CN110619572 A CN 110619572A CN 201910890868 A CN201910890868 A CN 201910890868A CN 110619572 A CN110619572 A CN 110619572A
- Authority
- CN
- China
- Prior art keywords
- data
- monitoring
- information
- enterprise
- dimensions
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2282—Tablespace storage structures; Management thereof
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/95—Retrieval from the web
- G06F16/951—Indexing; Web crawling techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q40/00—Finance; Insurance; Tax strategies; Processing of corporate or income taxes
- G06Q40/03—Credit; Loans; Processing thereof
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Databases & Information Systems (AREA)
- Business, Economics & Management (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Accounting & Taxation (AREA)
- Finance (AREA)
- Software Systems (AREA)
- Development Economics (AREA)
- Economics (AREA)
- Marketing (AREA)
- Strategic Management (AREA)
- Technology Law (AREA)
- General Business, Economics & Management (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The invention discloses a method for monitoring high fault tolerance growth of enterprise public data, which comprises the following steps: storing a list of monitoring enterprises and monitoring data dimensions thereof: storing the information by a database table, and recording monitoring object enterprises of a customer manager and concerned information dimensions; these companies are offered to a crawler crawling queue: generating corresponding crawler programs according to the monitoring dimensions of the company, repeatedly crawling the websites every day by the crawler programs, wherein the crawling frequency is related to the monitoring dimensions; the invention has the beneficial effects that: the method is beneficial to identifying the newly added data of the enterprise, the newly added data of the enterprise can be timely fed back to the client manager, the client manager can be helped to identify whether the enterprise has risks, and the efficiency of the client manager is improved; by the aid of the additionally arranged risk identification module, risk information judgment is conveniently performed on data newly added and fed back to a customer manager every day; through the alarm module who adds, be convenient for report to the police to risk information, make things convenient for timely processing of customer manager.
Description
Technical Field
The invention belongs to the technical field of data monitoring, and particularly relates to a method for monitoring high fault tolerance growth of enterprise public data.
Background
Fault tolerance, which refers to the ability of software to detect and recover from errors occurring in the software or hardware on which an application is running, can be generally measured in terms of reliability, availability, testability, etc. of a system. Data fault tolerance is realized in the way that enterprise data cannot be irretrievable or mistakenly retrieved due to failure of a crawler or other reasons.
The enterprise loan is a borrowing mode according to the specified interest rate and term from a bank or other financial institutions for the production and management of the enterprise; the loan of an enterprise is mainly used for carrying out large-amount long-term investment such as fixed asset purchasing construction, technical transformation and the like.
There is a risk that the financial institution will serve the loan client, either before the loan is identified or slowly after the loan has been made. The former risk can be screened out by carrying out comprehensive background investigation on the enterprise before loan, but the latter risk is often caused by the operation difficulty of the enterprise due to various reasons in the continuous production process, and is difficult to be discovered by financial institutions aiming at the risk or is serious at the time of discovery. The cost of finding problems in advance is very high and the efficiency is very low and not completely accurate, which requires the financial institution to invest very much manpower and time to track each loan client every day, which is possible to avoid these risks in advance. There are the following outstanding problems:
1. each customer manager will be responsible for at least tens of enterprise loan customers; for daily risk identification of each enterprise, a customer manager needs to search for enterprise information on at least more than ten public websites in a targeted manner respectively to check whether corresponding risk information exists, the websites generally comprise an industrial and commercial website, a news public opinion website, a legal information website and the like, and the customer manager needs to spend a great deal of time and energy to complete trivial information collection and comparative analysis, so that the efficiency is reduced;
2. the value of the customer manager should be more reflected in the financial service for the customer, however, the main energy spent on information collection is waste of resources, and the cost is increased;
3. when a large amount of information is collected by a client manager every day, some important information is likely to be missed or identified wrongly due to human negligence or time difference.
Disclosure of Invention
The invention aims to provide a method for monitoring the high fault tolerance growth of enterprise public data, which aims to solve the problem that each customer manager proposed in the background technology is responsible for at least dozens of enterprise loan customers; for daily risk identification of each enterprise, a customer manager needs to search for enterprise information on at least more than ten public websites in a targeted manner respectively to check whether corresponding risk information exists, the websites generally comprise an industrial and commercial website, a news public opinion website, a legal information website and the like, and the customer manager needs to spend a great deal of time and energy to complete trivial information collection and comparative analysis, so that the efficiency is reduced; the value of the customer manager should be more reflected in the financial service for the customer, however, the main energy spent on information collection is waste of resources, and the cost is increased; when a large amount of information is collected by a client manager every day, some important information is likely to be missed or wrongly identified due to human negligence or time difference.
In order to achieve the purpose, the invention provides the following technical scheme: a method for monitoring high fault tolerance growth of enterprise public data comprises the following steps:
the method comprises the following steps: storing a list of monitoring enterprises and monitoring data dimensions thereof: storing the information by a database table, and recording monitoring object enterprises of a customer manager and concerned information dimensions;
step two: these companies are offered to a crawler crawling queue: generating corresponding crawler programs according to the monitoring dimensions of the company, repeatedly crawling the websites every day by the crawler programs, wherein the crawling frequency is related to the monitoring dimensions;
step three: and starting task management: information of enterprises is added into a task manager in a task mode to ensure updating;
step four: single enterprise dimension data growth determination: judging specific growth of single enterprise dimension data through two types;
step five: and feeding back the new data to the user: the newly added data is fed back to the user daily through a channel provided by the customer.
The information identifier is used for distinguishing the crawled repeated data.
As a preferred technical solution of the present invention, in the fourth step, one is that information of a certain dimension is a whole, and some data in the whole is changed in a growth manner; the other is that the information of the dimension is composed in a list type, and the growth mode is embodied in that the information list has new added items on the original basis.
As a preferred embodiment of the present invention, the newly added data is stored in a single form in the event table.
As a preferred technical solution of the present invention, the channel provided by the customer is an email or a short message.
As a preferred technical solution of the present invention, the system further includes a risk identification module, which is configured to perform risk information judgment on data newly added daily and fed back to the customer manager.
As a preferred technical solution of the present invention, the feedback mode is an email or a short message.
As a preferred technical solution of the present invention, the present invention further includes an alarm module, and the alarm module is configured to alarm the risk information.
Compared with the prior art, the invention has the beneficial effects that:
(1) the method is beneficial to identifying the newly added data of the enterprise, the newly added data of the enterprise can be timely fed back to the client manager, the client manager can be helped to identify whether the enterprise has risks, and the efficiency of the client manager is improved;
(2) by the aid of the additionally arranged risk identification module, risk information judgment is conveniently performed on data newly added and fed back to a customer manager every day;
(3) through the alarm module who adds, be convenient for report to the police to risk information, make things convenient for timely processing of customer manager.
Drawings
FIG. 1 is a schematic overall flow diagram of the present invention;
FIG. 2 is a table of monitored enterprises of the present invention storing all the enterprises and dimensions being monitored;
fig. 3 is an event detail table, which stores all the newly added enterprise information.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Referring to fig. 1, fig. 2 and fig. 3, the present invention provides a technical solution: a method for monitoring high fault tolerance growth of enterprise public data comprises the following steps:
the method comprises the following steps: storing a list of monitoring enterprises and monitoring data dimensions thereof: the monitoring enterprise information is stored by a database table, wherein the table storage structure of the monitoring enterprise information is a monitoring enterprise table shown in the specification and the attached figure 2, and the table can record monitoring object enterprises of a client manager and concerned information dimensions;
step two: these companies are offered to a crawler crawling queue: generating corresponding crawler programs according to the monitoring dimensions of the company, wherein the crawler programs can repeatedly crawl the websites every day, and the crawling frequency is related to the monitoring dimensions; for example, the change frequency of the industrial and commercial data is low, the change period is in weeks, and the crawling frequency is once a day for a monitoring company to know whether the corresponding industrial and commercial information changes as early as possible; for example, the change frequency of news data is high, so that data crawling can be performed by taking hours as a periodic unit; the crawled data is likely to be repeatedly processed in the follow-up process;
step three: and starting task management: information updating of enterprises is added into a task manager in a task mode, and the task manager ensures normal operation of updating;
step four: single enterprise dimension data growth determination: specific growth decisions for a single enterprise dimension data fall into two categories;
one is that information of a certain dimension is a whole, certain data in the whole is changed in a growth mode, and the latest data is required to be obtained for the change and is compared with the original data in detail, so that whether information is newly added is determined;
the other is that the information of the dimension is composed of a list type, the growth mode is embodied in that the information list has newly added items on the original basis, the change of the type does not take the full amount of data, but takes the data of the last N days, the value of N is adjusted and set to be 7 through practice, high fault tolerance can be achieved, because a plurality of links possibly cause untimely data in the whole monitoring process, for example, because the crawler does not crawl the data at the moment due to the revision of an information website, or because the task management can not normally operate due to various reasons, the data which fails to grow recently can be ensured to be replenished again through the channel by taking the redundant data of the last N days, and high fault tolerance is further ensured;
the same piece of data crawled from a website may have some small differences due to various reasons, for example, the same news is published on different news websites, but the news contents are the same, and as for how to judge whether the crawled data is the same piece of data, the system is distinguished by an information discriminator, and only the information which does not appear is considered as new data after discrimination; the newly added data is finally stored in a single form in an event table, and the structure of the event table is as shown in an event detail table in the specification and the attached figure 3;
step five: and feeding back the new data to the user: the newly added data system generated by the monitoring method can feed back to the user every day through a channel provided by the customer, such as an email or a short message, and the information every day is fed back to a customer manager in an email or short message mode.
In this embodiment, preferably, the system further includes a risk identification module, and the risk identification module is configured to perform risk information judgment on data newly added daily and fed back to the customer manager.
In this embodiment, preferably, the system further includes an alarm module, and the alarm module is configured to alarm the risk information.
Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that changes, modifications, substitutions and alterations can be made in these embodiments without departing from the principles and spirit of the invention, the scope of which is defined in the appended claims and their equivalents.
Claims (8)
1. A method for monitoring the high fault tolerance growth of enterprise public data is characterized in that: the method comprises the following steps:
the method comprises the following steps: storing a list of monitoring enterprises and monitoring data dimensions thereof: storing the information by a database table, and recording monitoring object enterprises of a customer manager and concerned information dimensions;
step two: these companies are offered to a crawler crawling queue: generating corresponding crawler programs according to the monitoring dimensions of the company, repeatedly crawling the websites every day by the crawler programs, wherein the crawling frequency is related to the monitoring dimensions;
step three: and starting task management: information of enterprises is added into a task manager in a task mode to ensure updating;
step four: single enterprise dimension data growth determination: judging specific growth of single enterprise dimension data through two types;
step five: and feeding back the new data to the user: the newly added data is fed back to the user daily through a channel provided by the customer.
2. The method according to claim 1, wherein the method comprises the following steps: the system also comprises an information discriminator, wherein the discriminator is used for distinguishing the crawled repeated data.
3. The method according to claim 1, wherein the method comprises the following steps: in the fourth step, one is that information of a certain dimension is a whole, and certain data in the whole in a growth mode are changed to a certain extent; the other is that the information of the dimension is composed in a list type, and the growth mode is embodied in that the information list has new added items on the original basis.
4. The method according to claim 3, wherein the method comprises the following steps: the newly added data is stored in a single piece in the event table.
5. The method according to claim 1, wherein the method comprises the following steps: the channel provided by the customer is an email or a short message.
6. The method according to claim 1, wherein the method comprises the following steps: the system also comprises a risk identification module, wherein the risk identification module is used for judging the risk information of the data newly added and fed back to the customer manager every day.
7. The method for monitoring high fault tolerance growth of enterprise public data according to claim 6, wherein: the feedback mode is mail or short message.
8. The method according to claim 7, wherein the method comprises the following steps: the system also comprises an alarm module, and the alarm module is used for alarming the risk information.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910890868.6A CN110619572A (en) | 2019-09-20 | 2019-09-20 | Method for monitoring high fault tolerance growth of enterprise public data |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910890868.6A CN110619572A (en) | 2019-09-20 | 2019-09-20 | Method for monitoring high fault tolerance growth of enterprise public data |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110619572A true CN110619572A (en) | 2019-12-27 |
Family
ID=68923669
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910890868.6A Pending CN110619572A (en) | 2019-09-20 | 2019-09-20 | Method for monitoring high fault tolerance growth of enterprise public data |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110619572A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111754131A (en) * | 2020-06-30 | 2020-10-09 | 苏州朗动网络科技有限公司 | Enterprise information dynamic monitoring method, equipment and medium |
| CN112102076A (en) * | 2020-11-09 | 2020-12-18 | 成都数联铭品科技有限公司 | Comprehensive risk early warning system of platform |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101908194A (en) * | 2010-08-09 | 2010-12-08 | 中国建设银行股份有限公司 | Method for monitoring corporate bank loan |
| CN108763507A (en) * | 2018-05-30 | 2018-11-06 | 北京百度网讯科技有限公司 | Enterprise's incidence relation method for digging and device |
| CN108876228A (en) * | 2018-09-28 | 2018-11-23 | 苏州朗动网络科技有限公司 | Monitoring method, device, computer equipment and the storage medium of business risk |
| CN108932577A (en) * | 2018-04-25 | 2018-12-04 | 广州广电研究院有限公司 | A kind of assessment of business risk and early warning system |
| CN109191279A (en) * | 2018-08-01 | 2019-01-11 | 西安日间结算登记有限公司 | Medium-sized and small enterprises assessing credit risks platform based on supply chain finance on line |
| CN109214915A (en) * | 2018-09-06 | 2019-01-15 | 江西汉辰金融科技集团有限公司 | Borrow risk methods of marking, system and computer readable storage medium |
| CN109325860A (en) * | 2018-08-29 | 2019-02-12 | 中国科学院自动化研究所 | Network public-opinion detection method and system for overseas investment Risk-warning |
| CN109992704A (en) * | 2019-03-12 | 2019-07-09 | 青岛格兰德信用管理咨询有限公司 | A kind of enterprise's public sentiment monitoring system and method based on shot and long term Memory Neural Networks |
-
2019
- 2019-09-20 CN CN201910890868.6A patent/CN110619572A/en active Pending
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101908194A (en) * | 2010-08-09 | 2010-12-08 | 中国建设银行股份有限公司 | Method for monitoring corporate bank loan |
| CN108932577A (en) * | 2018-04-25 | 2018-12-04 | 广州广电研究院有限公司 | A kind of assessment of business risk and early warning system |
| CN108763507A (en) * | 2018-05-30 | 2018-11-06 | 北京百度网讯科技有限公司 | Enterprise's incidence relation method for digging and device |
| CN109191279A (en) * | 2018-08-01 | 2019-01-11 | 西安日间结算登记有限公司 | Medium-sized and small enterprises assessing credit risks platform based on supply chain finance on line |
| CN109325860A (en) * | 2018-08-29 | 2019-02-12 | 中国科学院自动化研究所 | Network public-opinion detection method and system for overseas investment Risk-warning |
| CN109214915A (en) * | 2018-09-06 | 2019-01-15 | 江西汉辰金融科技集团有限公司 | Borrow risk methods of marking, system and computer readable storage medium |
| CN108876228A (en) * | 2018-09-28 | 2018-11-23 | 苏州朗动网络科技有限公司 | Monitoring method, device, computer equipment and the storage medium of business risk |
| CN109992704A (en) * | 2019-03-12 | 2019-07-09 | 青岛格兰德信用管理咨询有限公司 | A kind of enterprise's public sentiment monitoring system and method based on shot and long term Memory Neural Networks |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111754131A (en) * | 2020-06-30 | 2020-10-09 | 苏州朗动网络科技有限公司 | Enterprise information dynamic monitoring method, equipment and medium |
| CN112102076A (en) * | 2020-11-09 | 2020-12-18 | 成都数联铭品科技有限公司 | Comprehensive risk early warning system of platform |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111475804B (en) | Alarm prediction method and system | |
| AU2002359925B2 (en) | Operation managing method and operation managing server | |
| CA3155689A1 (en) | Early-warning method for commodity inventory risk based on a statistical interquartile range, and system and computer-readable storage medium thereof | |
| US20150269242A1 (en) | Email analytics | |
| EP1177503B1 (en) | System for indexing pedestrian traffic | |
| US20100332509A1 (en) | Systems and methods for integrated management of governmental projects | |
| US8275739B2 (en) | User interface display for monitoring a database load engine | |
| US20100257074A1 (en) | Methods and apparatus for early remittance issue detection | |
| CN115310766B (en) | Enterprise management system based on big data analysis | |
| CN110619572A (en) | Method for monitoring high fault tolerance growth of enterprise public data | |
| CN111897806A (en) | Big data offline data quality inspection method and device | |
| CN107480188B (en) | Audit service data processing method and computer equipment | |
| CN111737335A (en) | Product information integration processing method and device, computer equipment and storage medium | |
| CN115038083A (en) | Telecom fraud early warning identification method and system applied to AI operator industry | |
| CN116701525A (en) | Early warning method and system based on real-time data analysis and electronic equipment | |
| CN116821106A (en) | Enterprise data digital management system and method | |
| CN114049036A (en) | Data computing platform, method, device and storage medium | |
| CN112966897A (en) | Multi-dimensional data analysis method based on maintenance platform | |
| CN112883739A (en) | Abnormal warning method and device for rating system, electronic equipment and storage medium | |
| CN113297146A (en) | Processing model and method for local supervision submission data | |
| Raj et al. | On the Impact of ML use cases on Industrial Data Pipelines | |
| US20060253426A1 (en) | Identifying duplicate entries in a historical database | |
| US10460294B1 (en) | Detection of errors in transaction channels using electronic transaction monitoring | |
| JP2020135739A (en) | Failure sign detection system | |
| CN114154094B (en) | Calculation method for untimely updating of website |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20191227 |
|
| RJ01 | Rejection of invention patent application after publication |