CN111061609A - Log monitoring method and system - Google Patents
Log monitoring method and system Download PDFInfo
- Publication number
- CN111061609A CN111061609A CN201911223089.7A CN201911223089A CN111061609A CN 111061609 A CN111061609 A CN 111061609A CN 201911223089 A CN201911223089 A CN 201911223089A CN 111061609 A CN111061609 A CN 111061609A
- Authority
- CN
- China
- Prior art keywords
- log
- monitoring
- file
- log file
- monitoring method
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/3065—Monitoring arrangements determined by the means or processing involved in reporting the monitored data
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Quality & Reliability (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Debugging And Monitoring (AREA)
Abstract
The invention discloses a log monitoring method, which comprises the following steps: monitoring the updated content in the log file in real time, and detecting the set keywords contained in the updated content; analyzing the log file and converting the log file into a specified format; screening the log files according to preset conditions; a receiving party of the log obtains the log file and establishes a processing mode of the log file; the log monitoring method further comprises the following steps: summarizing and counting the types of the log files according to different preset conditions; the invention carries out real-time monitoring on the updated content in the log file and establishes a corresponding processing method, thereby solving the technical problem that the existing method for monitoring the application log can not process the log accurately in real time, thereby processing the log accurately in real time, meeting the real-time requirement of the application level on alarm monitoring and further realizing the improvement of the log monitoring efficiency.
Description
Technical Field
The invention relates to the technical field of computers, in particular to a log monitoring method and system.
Background
In the information society developing at a high speed nowadays, people are increasingly unable to leave intelligent devices such as computers, and with the maturity of computer technologies, although application programs running on the intelligent devices are more and more perfect, errors occur inevitably in the actual running process. Therefore, engineering technicians usually need to record the running state and the operation content of the application program through a log system so as to be conveniently viewed by personnel and used as a basis for debugging equipment. The log records various running states and operation information of the application program, and generates a log file.
But the log is checked and analyzed by using a traditional tool based on a Linux script. These tool approaches have the following disadvantages: the problem cannot be issued in the first time if the operation is not timely; the method is not intuitive and not easy to expand, and the application range is limited to a small number of host computers and log file types. In the existing method for monitoring the application logs, an application server records the application logs, periodically uploads application log files to a remote monitoring server, and then related technical maintenance personnel periodically acquire the log files from the monitoring server for analysis, so that the logs cannot be accurately processed in real time, the real-time requirement of an application level on alarm monitoring is met, and the log monitoring efficiency is low.
Disclosure of Invention
The invention provides a log monitoring method and a log monitoring system, which are used for solving the technical problem that the existing method for monitoring application logs cannot accurately process the logs in real time by monitoring updated contents in a log file in real time and establishing a corresponding processing method, so that the logs are accurately processed in real time, the real-time requirement of an application level on alarm monitoring is met, and the log monitoring efficiency is improved.
In order to solve the above technical problem, an embodiment of the present invention provides a log monitoring method, including:
monitoring the updated content in the log file in real time, and detecting the set keywords contained in the updated content;
analyzing the log file and converting the log file into a specified format;
screening the log files according to preset conditions;
a receiving party of the log obtains the log file and establishes a processing mode of the log file;
the log monitoring method further comprises the following steps:
and summarizing and counting the types of the log files according to different preset conditions.
Preferably, the specified format includes: the log content, the log level, the log generation time, the sender of the log and the application program corresponding to the log.
As a preferred scheme, the log levels are INFO, WARN and ERROR from low to high in sequence; wherein the INFO level is lowest and the ERROR level is highest.
As a preferred scheme, the screening the log files according to preset conditions specifically includes: and screening according to the log content, screening according to the log level, screening according to the log generation time, screening according to a sender of the log and screening according to an application program corresponding to the log.
As a preferred scheme, after the log file is screened, the method further includes: according to the difference between the log content and the log level, the log is selected and sent to different related personnel, so that the personnel who are in service with the opposite port can solve the problem of the log related to the personnel.
Preferably, when the log file is error log monitoring, the error log monitoring is performed in a manner of receiving the log file, determining whether a set warning condition is met, and if so, sending the received log file to a receiving party.
Preferably, when the log file is processed by storing an error log, the error log is stored by storing the error log in a database.
An embodiment of the present invention further provides a log monitoring system, including:
the monitoring unit is used for monitoring the updated content in the log file in real time and detecting the set keywords contained in the updated content;
the analysis unit is used for analyzing the log file and converting the log file into a specified format;
the screening unit is used for screening the log file according to preset conditions;
the processing unit is used for acquiring the log file by a log receiver and establishing a processing mode of the log file;
the log monitoring method further comprises the following steps:
and the statistical unit is used for summarizing and counting the types of the log files according to different preset conditions.
An embodiment of the present invention further provides a computer-readable storage medium, where the computer-readable storage medium includes a stored computer program; wherein the computer program, when running, controls an apparatus in which the computer-readable storage medium is located to perform the log monitoring method as described in any one of the above.
An embodiment of the present invention further provides a terminal device, which includes a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, where the processor implements the log monitoring method according to any one of the above items when executing the computer program.
Compared with the prior art, the embodiment of the invention has the following beneficial effects:
the invention carries out real-time monitoring on the updated content in the log file and establishes a corresponding processing method, thereby solving the technical problem that the existing method for monitoring the application log can not process the log accurately in real time, thereby processing the log accurately in real time, meeting the real-time requirement of the application level on alarm monitoring and further realizing the improvement of the log monitoring efficiency.
Drawings
FIG. 1: the steps of the log monitoring method in the embodiment of the invention are a flow chart;
FIG. 2: a flowchart of log monitoring method steps in another embodiment of the invention;
FIG. 3: the log monitoring system in the embodiment of the invention is a schematic structural diagram.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Typical logs are in a fixed format. In order to facilitate the log to be viewed and analyzed and to have a clear and intuitive impression on the log, the following definitions are made in various examples of the present invention, but the following exemplary definitions of the log do not constitute a limitation to the technical solution of the present invention, and the technical solution disclosed in the present invention may be extended to similar log formats. The log is in row units, and each row has the following information: the generation time, the log level, the thread and class of the executive program, the position of the code and the specific log content.
Referring to fig. 1 and fig. 2, a preferred embodiment of the present invention provides a log monitoring method, including:
step S101, monitoring the updated content in the log file in real time, and detecting the set keywords contained in the updated content.
The log file usually records information by rows, including normal operation information, abnormal information, and failure information of the program, so in this example, the change of the log file is monitored by rows, that is, in the process of recording a new message in the log file, the recorded new message is monitored in real time to find the abnormality or failure of the program. The specific process is as follows:
and acquiring and analyzing a row of log information newly written into the log file, and detecting whether the log information contains the defined keywords in the abnormal and fault information table. If the log information contains the keyword, step S102 is executed, and if the log information does not contain the keyword, no operation is performed, and the monitoring is continued.
And step S102, analyzing the log file and converting the log file into a specified format.
The specified format is specifically as follows: log content, log level, log generation time, log sender and application program corresponding to the log.
S103, screening the log files according to conditions;
because the output of the log is massive, most of the log information personnel are not concerned. The screening step determines whether to process the log information or discard the log information according to a preset condition.
For example, the screening is performed according to the log level, and if the screening log level is the WARN level, only the log files of the WARN level and the ERROR level are screened.
For example, the filtering is performed according to the log generation time, and if the log generation time is set to 2018/05/01 to 2018/06/01, only the log files generated in the time period are filtered.
For example, screening is performed according to a sender of the log, and if the sender of the log is set to be "my-admin", only the log file of which the sender is "my-admin" is screened.
For example, the application program corresponding to the log is screened, and if the application program corresponding to the log is set to be "my-project", only the log file of the application program with the source of "my-project" is screened.
The screening of the log files is convenient for personnel to carry out centralized management on massive log files, and the log files are effectively screened.
And step S104, the receiving party of the log obtains the log file and establishes a processing mode of the log file.
Step S1041, when the processing mode of the log file is error log monitoring, the processing mode of the error log monitoring receives the log file, judges whether the set warning condition is met, and if the set warning condition is met, sends the received log file to related personnel.
Step S1042, when the processing mode of the log file is error log storage, the log file is stored in a database.
The flow of the above sub-steps may be taken as shown in figure 2,
in conclusion, the method manages the log files and checks the analysis logs in a centralized management mode; the join message warning mechanism sends warning messages to related personnel to enable the personnel to know the abnormality of the program at the first time.
Step S105, after the error log is stored, log data are counted periodically. And calculating the weight and the total number of personnel at each level required by the similar error logs. Therefore, personnel at all levels with proper data can be informed to wait before the next similar error comes, and the error log can be solved in time.
The system can periodically collect and count the error logs in the log database according to different conditions, for example, the error logs are grouped according to the log generation time and the log level by week, month and year, and can be subdivided into log corresponding modules. Thereby analyzing the periodicity and corruption of the error log occurrences. And record the weight that the class log is submitted to the technician. For example, the class A person accounts for 60%, the class B person accounts for 30%, and the other persons accounts for 10%. Therefore, when the periodic error log comes, the system can automatically inform the corresponding personnel at all levels in advance according to the weight occupied by the handling personnel, so that the error is handled more leisurely, and the resource waste of the personnel at all levels can be avoided. The personnel at all levels can process more errors according with the self levels, thereby accelerating the speed of solving the errors.
The invention inquires the log file, is used for checking and analyzing the log, and is more visual and efficient: and a log message warning mechanism is established, so that related responsible persons can know abnormal problems in the log file at the first time, and the problems can be processed in time.
Compared with the prior art, the invention has the following advantages and effects: 1. the log file is searched to check and analyze the log, so that the method is more efficient and visual. 2. A message warning mechanism is established, and related personnel can know the abnormity of the program at the first time. 3. The receiver of the log is easy to expand by transmitting the log information through the message.
Referring to fig. 3, an embodiment of the present invention further provides a log monitoring system, including:
the monitoring unit is used for monitoring the updated content in the log file in real time and detecting the set keywords contained in the updated content;
the analysis unit is used for analyzing the log file and converting the log file into a specified format;
the screening unit is used for screening the log file according to preset conditions;
the processing unit is used for acquiring the log file by a log receiver and establishing a processing mode of the log file;
the log monitoring method further comprises the following steps:
and the statistical unit is used for summarizing and counting the types of the log files according to different preset conditions.
An embodiment of the present invention further provides a computer-readable storage medium, where the computer-readable storage medium includes a stored computer program; wherein, when running, the computer program controls the device where the computer readable storage medium is located to execute the log monitoring method according to any of the above embodiments.
The embodiment of the present invention further provides a terminal device, where the terminal device includes a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, and the processor implements the log monitoring method according to any of the above embodiments when executing the computer program.
Preferably, the computer program may be divided into one or more modules/units (e.g., computer program) that are stored in the memory and executed by the processor to implement the invention. The one or more modules/units may be a series of computer program instruction segments capable of performing specific functions, which are used for describing the execution process of the computer program in the terminal device.
The Processor may be a Central Processing Unit (CPU), other general purpose Processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), an off-the-shelf Programmable Gate Array (FPGA) or other Programmable logic device, a discrete Gate or transistor logic device, a discrete hardware component, etc., the general purpose Processor may be a microprocessor, or the Processor may be any conventional Processor, the Processor is a control center of the terminal device, and various interfaces and lines are used to connect various parts of the terminal device.
The memory mainly includes a program storage area and a data storage area, wherein the program storage area may store an operating system, an application program required for at least one function, and the like, and the data storage area may store related data and the like. In addition, the memory may be a high speed random access memory, may also be a non-volatile memory, such as a plug-in hard disk, a Smart Memory Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), and the like, or may also be other volatile solid state memory devices.
It should be noted that the terminal device may include, but is not limited to, a processor and a memory, and those skilled in the art will understand that the terminal device is only an example and does not constitute a limitation of the terminal device, and may include more or less components, or combine some components, or different components.
The above-mentioned embodiments are provided to further explain the objects, technical solutions and advantages of the present invention in detail, and it should be understood that the above-mentioned embodiments are only examples of the present invention and are not intended to limit the scope of the present invention. It should be understood that any modifications, equivalents, improvements and the like, which come within the spirit and principle of the invention, may occur to those skilled in the art and are intended to be included within the scope of the invention.
Claims (10)
1. A log monitoring method, comprising:
monitoring the updated content in the log file in real time, and detecting the set keywords contained in the updated content;
analyzing the log file and converting the log file into a specified format;
screening the log files according to preset conditions;
a receiving party of the log obtains the log file and establishes a processing mode of the log file;
the log monitoring method further comprises the following steps:
and summarizing and counting the types of the log files according to different preset conditions.
2. The log monitoring method of claim 1, wherein the specified format comprises: the log content, the log level, the log generation time, the sender of the log and the application program corresponding to the log.
3. The log monitoring method as claimed in claim 2, wherein the log levels are INFO, WARN, ERROR; wherein the INFO level is lowest and the ERROR level is highest.
4. The log monitoring method according to claim 2, wherein the screening the log file according to a preset condition specifically comprises: and screening according to the log content, screening according to the log level, screening according to the log generation time, screening according to a sender of the log and screening according to an application program corresponding to the log.
5. The log monitoring method of claim 4, after screening the log files, further comprising: according to the difference between the log content and the log level, the log is selected and sent to different related personnel, so that the personnel who are in service with the opposite port can solve the problem of the log related to the personnel.
6. The log monitoring method according to claim 1, wherein when the log file is an error log monitoring, the error log monitoring is performed by receiving the log file, determining whether a set warning condition is satisfied, and if so, transmitting the received log file to a receiving party.
7. The log monitoring method as claimed in claim 1, wherein when the log file is processed in such a manner that the error log is stored, the error log is stored in such a manner that the error log is stored in the database.
8. A log monitoring system, comprising:
the monitoring unit is used for monitoring the updated content in the log file in real time and detecting the set keywords contained in the updated content;
the analysis unit is used for analyzing the log file and converting the log file into a specified format;
the screening unit is used for screening the log file according to preset conditions;
the processing unit is used for acquiring the log file by a log receiver and establishing a processing mode of the log file;
the log monitoring method further comprises the following steps:
and the statistical unit is used for summarizing and counting the types of the log files according to different preset conditions.
9. A computer-readable storage medium, characterized in that the computer-readable storage medium comprises a stored computer program; wherein the computer program, when running, controls an apparatus in which the computer-readable storage medium is located to perform the log monitoring method according to any one of claims 1 to 7.
10. A terminal device comprising a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, the processor implementing the log monitoring method as claimed in any one of claims 1 to 7 when executing the computer program.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911223089.7A CN111061609A (en) | 2019-12-03 | 2019-12-03 | Log monitoring method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911223089.7A CN111061609A (en) | 2019-12-03 | 2019-12-03 | Log monitoring method and system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111061609A true CN111061609A (en) | 2020-04-24 |
Family
ID=70299566
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911223089.7A Pending CN111061609A (en) | 2019-12-03 | 2019-12-03 | Log monitoring method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111061609A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112000806A (en) * | 2020-08-25 | 2020-11-27 | 携程旅游信息技术（上海）有限公司 | Abnormal log monitoring and analyzing method, system, equipment and storage medium |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102981943A (en) * | 2012-10-29 | 2013-03-20 | 新浪技术（中国）有限公司 | Method and system for monitoring application logs |
| CN105159964A (en) * | 2015-08-24 | 2015-12-16 | 广东欧珀移动通信有限公司 | Log monitoring method and system |
| CN105740121A (en) * | 2016-01-26 | 2016-07-06 | 中国银行股份有限公司 | Log text monitoring and early-warning method and apparatus |
| CN105871574A (en) * | 2015-01-21 | 2016-08-17 | ***通信集团浙江有限公司 | Method, device and system for monitoring log based on software development kit |
| CN106385331A (en) * | 2016-09-08 | 2017-02-08 | 努比亚技术有限公司 | Method and system for monitoring alarm based on log |
-
2019
- 2019-12-03 CN CN201911223089.7A patent/CN111061609A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102981943A (en) * | 2012-10-29 | 2013-03-20 | 新浪技术（中国）有限公司 | Method and system for monitoring application logs |
| CN105871574A (en) * | 2015-01-21 | 2016-08-17 | ***通信集团浙江有限公司 | Method, device and system for monitoring log based on software development kit |
| CN105159964A (en) * | 2015-08-24 | 2015-12-16 | 广东欧珀移动通信有限公司 | Log monitoring method and system |
| CN105740121A (en) * | 2016-01-26 | 2016-07-06 | 中国银行股份有限公司 | Log text monitoring and early-warning method and apparatus |
| CN106385331A (en) * | 2016-09-08 | 2017-02-08 | 努比亚技术有限公司 | Method and system for monitoring alarm based on log |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112000806A (en) * | 2020-08-25 | 2020-11-27 | 携程旅游信息技术（上海）有限公司 | Abnormal log monitoring and analyzing method, system, equipment and storage medium |
| CN112000806B (en) * | 2020-08-25 | 2023-06-16 | 携程旅游信息技术（上海）有限公司 | Anomaly log monitoring and analyzing method, system, equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110661659B (en) | Alarm method, device and system and electronic equipment | |
| CN107832196B (en) | Monitoring device and monitoring method for abnormal content of real-time log | |
| CN109947746A (en) | A kind of quality of data management-control method and system based on ETL process | |
| US9891971B1 (en) | Automating the production of runbook workflows | |
| CN110231998B (en) | Detection method and device for distributed timing task and storage medium | |
| CN110149653A (en) | A kind of cloud fault of mobile phone monitoring method and system | |
| CN110929896A (en) | Security analysis method and device for system equipment | |
| CN111782431A (en) | Exception processing method, exception processing device, terminal and storage medium | |
| CN111061609A (en) | Log monitoring method and system | |
| CN112256548B (en) | Abnormal data monitoring method and device, server and storage medium | |
| CN105825641A (en) | Service alarm method and apparatus | |
| CN112087320A (en) | Abnormity positioning method and device, electronic equipment and readable storage medium | |
| CN113472881B (en) | Statistical method and device for online terminal equipment | |
| CN114500178B (en) | Self-operation intelligent Internet of things gateway | |
| CN109240747A (en) | A kind of information inspection method and its relevant apparatus of multistage server management system | |
| CN113760669A (en) | Problem data warning method and device, electronic equipment and storage medium | |
| CN114500316A (en) | Method and system for inspecting equipment of Internet of things | |
| CN110427294B (en) | System integration environment monitoring method, apparatus, readable storage medium and program product | |
| CN113254313A (en) | Monitoring index abnormality detection method and device, electronic equipment and storage medium | |
| CN113676356A (en) | Alarm information processing method and device, electronic equipment and readable storage medium | |
| CN112579391A (en) | Distributed database automatic operation and maintenance method and system based on artificial intelligence | |
| CN113806196B (en) | Root cause analysis method and system | |
| CN116433197B (en) | Information reporting method, device, reporting end and storage medium | |
| CN111105314A (en) | Insurance data clearing system | |
| CN108282378B (en) | Method and device for monitoring network flow |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |