CN113342754A - Method and device for extracting log abnormal data - Google Patents
Method and device for extracting log abnormal data Download PDFInfo
- Publication number
- CN113342754A CN113342754A CN202110771920.3A CN202110771920A CN113342754A CN 113342754 A CN113342754 A CN 113342754A CN 202110771920 A CN202110771920 A CN 202110771920A CN 113342754 A CN113342754 A CN 113342754A
- Authority
- CN
- China
- Prior art keywords
- log data
- data
- abnormal
- log
- extracted
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/17—Details of further file system functions
- G06F16/1734—Details of monitoring file system events, e.g. by the use of hooks, filter drivers, logs
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/18—File system types
- G06F16/1805—Append-only file systems, e.g. using logs or journals to store data
- G06F16/1815—Journaling file systems
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Debugging And Monitoring (AREA)
Abstract
The invention discloses a method and a device for extracting log abnormal data. Wherein, the method comprises the following steps: acquiring original log data; optimizing the original log data to obtain log data to be extracted; extracting abnormal log data from the log data to be extracted according to a preset rule; and displaying the abnormal log data. The method solves the technical problems that in the prior art, original log data cannot be optimized in the process of extracting abnormal log data, so that when the abnormal log data are directly extracted from the original log data, all the abnormal log data cannot be accurately extracted, and the efficiency and accuracy of extracting the abnormal log data are reduced.
Description
Technical Field
The invention relates to the field of log processing, in particular to a method and a device for extracting log abnormal data.
Background
Along with the continuous development of intelligent science and technology, people use intelligent equipment more and more among life, work, the study, use intelligent science and technology means, improved the quality of people's life, increased the efficiency of people's study and work.
At present, when log data is abnormal, original log data needs to be directly checked and extracted, and abnormal data needs to be analyzed and processed through a log analysis function, so that a user using the log can take measures further.
In view of the above problems, no effective solution has been proposed.
Disclosure of Invention
The embodiment of the invention provides a method and a device for extracting abnormal log data, which are used for at least solving the technical problems that in the prior art, the original log data cannot be optimized in the abnormal log data extraction process, so that when the abnormal log data are directly extracted from the original log data, all abnormal log data cannot be accurately extracted, and the efficiency and the accuracy of extracting the abnormal log data are reduced.
According to an aspect of the embodiments of the present invention, there is provided a log abnormal data extraction method, including: acquiring original log data; optimizing the original log data to obtain log data to be extracted; extracting abnormal log data from the log data to be extracted according to a preset rule; and displaying the abnormal log data.
Optionally, before the obtaining the original log data, the method further includes: an abnormal state activation signal is acquired.
Optionally, the optimization process includes: data sorting and data compression.
Optionally, the extracting, according to a preset rule, abnormal log data from the log data to be extracted includes: acquiring a preset rule; and processing the log data to be extracted according to the preset rule to obtain the abnormal log data.
According to another aspect of the embodiments of the present invention, there is also provided a log abnormal data extracting apparatus, including: the acquisition module is used for acquiring original log data; the optimization module is used for optimizing the original log data to obtain log data to be extracted; the extraction module is used for extracting abnormal log data from the log data to be extracted according to a preset rule; and the display module is used for displaying the abnormal log data.
Optionally, the apparatus further comprises: the acquisition module is further used for acquiring the abnormal state activation signal.
Optionally, the optimization process includes: data sorting and data compression.
Optionally, the extracting module includes: an acquisition unit, configured to acquire a preset rule; and the abnormal unit is used for processing the log data to be extracted according to the preset rule to obtain the abnormal log data.
According to another aspect of the embodiments of the present invention, a non-volatile storage medium is further provided, where the non-volatile storage medium includes a stored program, and the program controls, when running, a device in which the non-volatile storage medium is located to execute a log abnormal data extraction method.
According to another aspect of the embodiments of the present invention, there is also provided an electronic device, including a processor and a memory; the memory has stored therein computer readable instructions, and the processor is configured to execute the computer readable instructions, wherein the computer readable instructions when executed perform a method of log anomaly data extraction.
In the embodiment of the invention, the original log data is obtained; optimizing the original log data to obtain log data to be extracted; extracting abnormal log data from the log data to be extracted according to a preset rule; the method for displaying the abnormal log data solves the technical problems that in the process of extracting the abnormal log data in the prior art, the original log data cannot be optimized, all the abnormal log data cannot be accurately extracted when the abnormal data is directly extracted from the original log data, and the efficiency and the accuracy of extracting the abnormal log data are reduced.
Drawings
The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this application, illustrate embodiment(s) of the invention and together with the description serve to explain the invention without limiting the invention. In the drawings:
FIG. 1 is a flow chart of a method for extracting log anomaly data according to an embodiment of the present invention;
fig. 2 is a block diagram of a log abnormal data extracting apparatus according to an embodiment of the present invention.
Detailed Description
In order to make the technical solutions of the present invention better understood, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
It should be noted that the terms "first," "second," and the like in the description and claims of the present invention and in the drawings described above are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used is interchangeable under appropriate circumstances such that the embodiments of the invention described herein are capable of operation in sequences other than those illustrated or described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
In accordance with an embodiment of the present invention, there is provided a method embodiment of a log anomaly data extraction method, it should be noted that the steps illustrated in the flowchart of the figure may be performed in a computer system such as a set of computer-executable instructions, and that while a logical order is illustrated in the flowchart, in some cases the steps illustrated or described may be performed in an order different than here.
Example one
Fig. 1 is a flowchart of a log anomaly data extraction method according to an embodiment of the present invention, as shown in fig. 1, the method includes the following steps:
step S102, obtaining original log data.
Specifically, in order to extract daily log data, the embodiment of the present invention first acquires original log data, and stores and sorts the original log data after acquiring the original log data, so as to facilitate subsequent optimization and extraction.
In addition, when extracting the original log data, the original log calling instruction may be first transmitted through a data instruction transmitting terminal of the ARM9 processor, and corresponding original log data may be called according to the storage condition of the log file storage of the local server, and the original log data may be placed in the temporary cache region, so that the original log data may be read at any time when being subsequently applied.
Optionally, before the obtaining the original log data, the method further includes: an abnormal state activation signal is acquired.
Specifically, in order to save the computing resources, in the embodiment of the present invention, before the original log data is acquired, the abnormal state activation signal may be acquired first, whether an abnormal condition exists is determined, and if the abnormal condition exists, the log function of extracting the abnormal data is started.
For example, when abnormal data is extracted, after log data is acquired in the embodiment of the present invention, in order to save log analysis computing resources, before the log data a is acquired, the central processing unit sends a scan instruction to the log storage layer to scan whether an abnormal state exists, that is, collects abnormal state activation information in the log data, where the abnormal state activation information does not include specific abnormal log data and only exists in the log data as a log tag, so as to facilitate quick acquisition of whether the log has an abnormal condition, and reduce unnecessary abnormal data extraction computation.
And step S104, optimizing the original log data to obtain the log data to be extracted.
Optionally, the optimization process includes: data sorting and data compression.
Specifically, in order to make the subsequent extraction of the abnormal log data more efficient, the original log data needs to be optimized, and the log data after the optimization is transmitted as the log data to be extracted, where the optimization may include: data sorting and data compression.
It should be noted that, the method for performing optimization processing on the original log data may include data sorting and data compression, where the data sorting may be a method of sorting (a +) in the log data out of order by presetting a set priority sorting rule. The data compression can be realized by acquiring the data volume of the original log data and performing compression operation with a proportion of 50% according to the data volume parameter so as to increase the efficiency of data analysis and extraction, and the log data needs to be periodically checked in the data compression process so as to ensure that the original log data cannot be lost in the compression operation.
And step S106, extracting abnormal log data from the log data to be extracted according to a preset rule.
Optionally, the extracting, according to a preset rule, abnormal log data from the log data to be extracted includes: acquiring a preset rule; and processing the log data to be extracted according to the preset rule to obtain the abnormal log data.
Specifically, after the log data to be extracted is obtained, a preset rule needs to be obtained, and the log data to be extracted is processed according to the preset rule to obtain the abnormal log data, wherein the abnormal log data represents the extracted log containing the abnormal data.
And step S108, displaying the abnormal log data.
Specifically, after the abnormal log data is acquired, in order to improve user experience, daily log data can be displayed and output, so that the user can check and analyze the daily log data.
By the embodiment, the technical problems that in the extraction process of abnormal log data in the prior art, the original log data cannot be optimized, all abnormal log data cannot be accurately extracted when the abnormal data is directly extracted from the original log data, and the extraction efficiency and accuracy of the abnormal log data are reduced are solved.
Example two
Fig. 2 is a block diagram of a log abnormal data extracting apparatus according to an embodiment of the present invention, as shown in fig. 2, the apparatus includes:
and an obtaining module 20, configured to obtain the original log data.
Specifically, in order to extract daily log data, the embodiment of the present invention first acquires original log data, and stores and sorts the original log data after acquiring the original log data, so as to facilitate subsequent optimization and extraction.
In addition, when extracting the original log data, the original log calling instruction may be first transmitted through a data instruction transmitting terminal of the ARM9 processor, and corresponding original log data may be called according to the storage condition of the log file storage of the local server, and the original log data may be placed in the temporary cache region, so that the original log data may be read at any time when being subsequently applied.
Optionally, before the obtaining the original log data, the method further includes: an abnormal state activation signal is acquired.
Specifically, in order to save the computing resources, in the embodiment of the present invention, before the original log data is acquired, the abnormal state activation signal may be acquired first, whether an abnormal condition exists is determined, and if the abnormal condition exists, the log function of extracting the abnormal data is started.
For example, when abnormal data is extracted, after log data is acquired in the embodiment of the present invention, in order to save log analysis computing resources, before the log data a is acquired, the central processing unit sends a scan instruction to the log storage layer to scan whether an abnormal state exists, that is, collects abnormal state activation information in the log data, where the abnormal state activation information does not include specific abnormal log data and only exists in the log data as a log tag, so as to facilitate quick acquisition of whether the log has an abnormal condition, and reduce unnecessary abnormal data extraction computation.
And the optimization module 22 is configured to perform optimization processing on the original log data to obtain log data to be extracted.
Optionally, the optimization process includes: data sorting and data compression.
Specifically, in order to make the subsequent extraction of the abnormal log data more efficient, the original log data needs to be optimized, and the log data after the optimization is transmitted as the log data to be extracted, where the optimization may include: data sorting and data compression.
It should be noted that, the method for performing optimization processing on the original log data may include data sorting and data compression, where the data sorting may be a method of sorting (a +) in the log data out of order by presetting a set priority sorting rule. The data compression can be realized by acquiring the data volume of the original log data and performing compression operation with a proportion of 50% according to the data volume parameter so as to increase the efficiency of data analysis and extraction, and the log data needs to be periodically checked in the data compression process so as to ensure that the original log data cannot be lost in the compression operation.
And the extracting module 24 is configured to extract abnormal log data from the log data to be extracted according to a preset rule.
Optionally, the extracting module includes: an acquisition unit, configured to acquire a preset rule; and the abnormal unit is used for processing the log data to be extracted according to the preset rule to obtain the abnormal log data.
Specifically, after the log data to be extracted is obtained, a preset rule needs to be obtained, and the log data to be extracted is processed according to the preset rule to obtain the abnormal log data, wherein the abnormal log data represents the extracted log containing the abnormal data.
And the display module 26 is configured to display the abnormal log data.
Specifically, after the abnormal log data is acquired, in order to improve user experience, daily log data can be displayed and output, so that the user can check and analyze the daily log data.
According to another aspect of the embodiments of the present invention, a non-volatile storage medium is further provided, where the non-volatile storage medium includes a stored program, and the program controls, when running, a device in which the non-volatile storage medium is located to execute a log abnormal data extraction method.
Specifically, the method comprises the following steps: acquiring original log data; optimizing the original log data to obtain log data to be extracted; extracting abnormal log data from the log data to be extracted according to a preset rule; and displaying the abnormal log data.
According to another aspect of the embodiments of the present invention, there is also provided an electronic device, including a processor and a memory; the memory has stored therein computer readable instructions, and the processor is configured to execute the computer readable instructions, wherein the computer readable instructions when executed perform a method of log anomaly data extraction.
Specifically, the method comprises the following steps: acquiring original log data; optimizing the original log data to obtain log data to be extracted; extracting abnormal log data from the log data to be extracted according to a preset rule; and displaying the abnormal log data.
By the embodiment, the technical problems that in the extraction process of abnormal log data in the prior art, the original log data cannot be optimized, all abnormal log data cannot be accurately extracted when the abnormal data is directly extracted from the original log data, and the extraction efficiency and accuracy of the abnormal log data are reduced are solved.
The above-mentioned serial numbers of the embodiments of the present invention are merely for description and do not represent the merits of the embodiments.
In the above embodiments of the present invention, the descriptions of the respective embodiments have respective emphasis, and for parts that are not described in detail in a certain embodiment, reference may be made to related descriptions of other embodiments.
In the embodiments provided in the present application, it should be understood that the disclosed technology can be implemented in other ways. The above-described embodiments of the apparatus are merely illustrative, and for example, the division of the units may be a logical division, and in actual implementation, there may be another division, for example, multiple units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, units or modules, and may be in an electrical or other form.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.
The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a removable hard disk, a magnetic or optical disk, and other various media capable of storing program codes.
The foregoing is only a preferred embodiment of the present invention, and it should be noted that, for those skilled in the art, various modifications and decorations can be made without departing from the principle of the present invention, and these modifications and decorations should also be regarded as the protection scope of the present invention.
Claims (10)
1. A log abnormal data extraction method is characterized by comprising the following steps:
acquiring original log data;
optimizing the original log data to obtain log data to be extracted;
extracting abnormal log data from the log data to be extracted according to a preset rule;
and displaying the abnormal log data.
2. The method of claim 1, wherein prior to said obtaining raw log data, the method further comprises:
an abnormal state activation signal is acquired.
3. The method of claim 1, wherein the optimization process comprises: data sorting and data compression.
4. The method according to claim 1, wherein the extracting abnormal log data from the log data to be extracted according to a preset rule comprises:
acquiring a preset rule;
and processing the log data to be extracted according to the preset rule to obtain the abnormal log data.
5. A log anomaly data extraction device, comprising:
the acquisition module is used for acquiring original log data;
the optimization module is used for optimizing the original log data to obtain log data to be extracted;
the extraction module is used for extracting abnormal log data from the log data to be extracted according to a preset rule;
and the display module is used for displaying the abnormal log data.
6. The apparatus of claim 5, further comprising:
the acquisition module is further used for acquiring the abnormal state activation signal.
7. The apparatus of claim 5, wherein the optimization process comprises: data sorting and data compression.
8. The apparatus of claim 5, wherein the extraction module comprises:
an acquisition unit, configured to acquire a preset rule;
and the abnormal unit is used for processing the log data to be extracted according to the preset rule to obtain the abnormal log data.
9. A non-volatile storage medium, comprising a stored program, wherein the program, when executed, controls an apparatus in which the non-volatile storage medium is located to perform the method of any one of claims 1 to 4.
10. An electronic device comprising a processor and a memory; the memory has stored therein computer readable instructions for execution by the processor, wherein the computer readable instructions when executed perform the method of any one of claims 1 to 4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110771920.3A CN113342754A (en) | 2021-07-08 | 2021-07-08 | Method and device for extracting log abnormal data |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110771920.3A CN113342754A (en) | 2021-07-08 | 2021-07-08 | Method and device for extracting log abnormal data |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113342754A true CN113342754A (en) | 2021-09-03 |
Family
ID=77482983
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110771920.3A Pending CN113342754A (en) | 2021-07-08 | 2021-07-08 | Method and device for extracting log abnormal data |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113342754A (en) |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102831020A (en) * | 2012-07-26 | 2012-12-19 | 中国科学院信息工程研究所 | Log detection method and system |
| CN103914485A (en) * | 2013-01-07 | 2014-07-09 | 上海宝信软件股份有限公司 | System and method for remotely collecting, retrieving and displaying application system logs |
| CN110210512A (en) * | 2019-04-19 | 2019-09-06 | 北京亿阳信通科技有限公司 | A kind of automation daily record method for detecting abnormality and system |
-
2021
- 2021-07-08 CN CN202110771920.3A patent/CN113342754A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102831020A (en) * | 2012-07-26 | 2012-12-19 | 中国科学院信息工程研究所 | Log detection method and system |
| CN103914485A (en) * | 2013-01-07 | 2014-07-09 | 上海宝信软件股份有限公司 | System and method for remotely collecting, retrieving and displaying application system logs |
| CN110210512A (en) * | 2019-04-19 | 2019-09-06 | 北京亿阳信通科技有限公司 | A kind of automation daily record method for detecting abnormality and system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108847977B (en) | Service data monitoring method, storage medium and server | |
| WO2019062081A1 (en) | Salesman profile formation method, electronic device and computer readable storage medium | |
| WO2016070673A1 (en) | Method and device for analyzing user attribute | |
| CN110995273B (en) | Data compression method, device, equipment and medium for power database | |
| CN107404486B (en) | Method, device, terminal equipment and storage medium for analyzing Http data | |
| CN111931809A (en) | Data processing method and device, storage medium and electronic equipment | |
| CN113808037A (en) | Image optimization method and device | |
| CN111092764B (en) | Real-time dynamic affinity relation analysis method and system | |
| JP2021518012A (en) | Data processing methods, equipment and storage media | |
| CN110442439B (en) | Task process processing method and device and computer equipment | |
| CN111915378A (en) | User attribute prediction method, device, computer equipment and storage medium | |
| CN113342754A (en) | Method and device for extracting log abnormal data | |
| CN116668331A (en) | Distributed performance monitoring system and method | |
| CN110909263A (en) | Method and device for determining companion relationship of identity characteristics | |
| CN112860456A (en) | Log processing method and device | |
| CN113312902A (en) | Intelligent auditing and checking method and device for same text | |
| CN106469086B (en) | Event processing method and device | |
| CN113506359A (en) | Animation element acquisition method and device | |
| CN113411828A (en) | Method, device and equipment for sensing call quality and computer readable storage medium | |
| CN111026793A (en) | Data processing method, device, medium and equipment | |
| CN112714057B (en) | Instant message processing method, device, equipment and storage medium | |
| CN113360723A (en) | Data acquisition method and device | |
| CN109684159A (en) | Method for monitoring state, device, equipment and the storage medium of distributed information system | |
| CN114254024A (en) | Data acquisition method, device, platform, equipment and storage medium | |
| CN113590479A (en) | Software comparison test method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20210903 |
|
| RJ01 | Rejection of invention patent application after publication |