CN113535655A - Log analysis method and device - Google Patents
Log analysis method and device Download PDFInfo
- Publication number
- CN113535655A CN113535655A CN202110669836.0A CN202110669836A CN113535655A CN 113535655 A CN113535655 A CN 113535655A CN 202110669836 A CN202110669836 A CN 202110669836A CN 113535655 A CN113535655 A CN 113535655A
- Authority
- CN
- China
- Prior art keywords
- log data
- log
- analysis
- data
- analyzed
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/17—Details of further file system functions
- G06F16/1734—Details of monitoring file system events, e.g. by the use of hooks, filter drivers, logs
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/18—File system types
- G06F16/1805—Append-only file systems, e.g. using logs or journals to store data
- G06F16/1815—Journaling file systems
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Debugging And Monitoring (AREA)
Abstract
The invention discloses a log analysis method and device. Wherein, the method comprises the following steps: acquiring first log data; generating log data to be analyzed according to the first log data; inputting the log data to be analyzed into a log analysis model to generate an analysis result; and displaying the analysis result. The invention solves the technical problems that log data in the prior art are only subjected to log analysis through fixed analysis rules, and cannot be flexibly utilized or changed to a certain extent according to the change of application scene elements, so that the flexibility and the accuracy of log data analysis are reduced.
Description
Technical Field
The invention relates to the field of log analysis, in particular to a log analysis method and device.
Background
Along with the continuous development of intelligent science and technology, people use intelligent equipment more and more among life, work, the study, use intelligent science and technology means, improved the quality of people's life, increased the efficiency of people's study and work.
At present, data log analysis is usually performed through a fixed analysis rule or an analysis program, and log data is processed through the steps of log data acquisition, processing, analysis, display and the like, but the conventional log data only performs log analysis through the fixed analysis rule, and cannot flexibly utilize historical data or perform certain changes according to changes of application scene elements, so that the flexibility and the accuracy of log data analysis are reduced.
In view of the above problems, no effective solution has been proposed.
Disclosure of Invention
The embodiment of the invention provides a log analysis method and a log analysis device, which are used for at least solving the technical problems that log data in the prior art are only subjected to log analysis through fixed analysis rules, and cannot flexibly utilize historical data or change a certain amount according to the change of application scene elements, so that the flexibility and the accuracy of log data analysis are reduced.
According to an aspect of an embodiment of the present invention, there is provided a log analysis method, including: acquiring first log data; generating log data to be analyzed according to the first log data; inputting the log data to be analyzed into a log analysis model to generate an analysis result; and displaying the analysis result.
Optionally, the generating log data to be analyzed according to the first log data includes: optimizing the first log data to generate second log data; and converting the second log data into the log data to be analyzed through a preset rule.
Optionally, before the log data to be analyzed is input into the log analysis model and an analysis result is generated, the method further includes: training the log analysis model through historical data.
Optionally, after displaying the analysis result, the method further includes: and converting the analysis result into the historical data.
According to another aspect of the embodiments of the present invention, there is also provided a log analysis apparatus, including: the acquisition module is used for acquiring first log data; the generating module is used for generating log data to be analyzed according to the first log data; the analysis module is used for inputting the log data to be analyzed into a log analysis model to generate an analysis result; and the display module is used for displaying the analysis result.
Optionally, the generating module includes: the optimization unit is used for optimizing the first log data to generate second log data; and the conversion unit is used for converting the second log data into the log data to be analyzed through a preset rule.
Optionally, the apparatus further comprises: and the training module is used for training the log analysis model through historical data.
Optionally, the apparatus further comprises: and the conversion module is used for converting the analysis result into the historical data.
According to another aspect of the embodiments of the present invention, there is also provided a non-volatile storage medium, which includes a stored program, wherein the program controls a device in which the non-volatile storage medium is located to execute a log analysis method when running.
According to another aspect of the embodiments of the present invention, there is also provided an electronic device, including a processor and a memory; the memory has stored therein computer readable instructions for execution by the processor, wherein the computer readable instructions when executed perform a method of log analysis.
In the embodiment of the invention, the first log data is acquired; generating log data to be analyzed according to the first log data; inputting the log data to be analyzed into a log analysis model to generate an analysis result; the method for displaying the analysis result solves the technical problems that log data in the prior art are only subjected to log analysis through fixed analysis rules, and cannot flexibly utilize historical data or change to a certain extent according to the change of application scene elements, so that the flexibility and the accuracy of log data analysis are reduced.
Drawings
The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this application, illustrate embodiment(s) of the invention and together with the description serve to explain the invention without limiting the invention. In the drawings:
FIG. 1 is a flow chart of a method of log analysis according to an embodiment of the present invention;
fig. 2 is a block diagram of a log analysis apparatus according to an embodiment of the present invention.
Detailed Description
In order to make the technical solutions of the present invention better understood, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
It should be noted that the terms "first," "second," and the like in the description and claims of the present invention and in the drawings described above are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used is interchangeable under appropriate circumstances such that the embodiments of the invention described herein are capable of operation in sequences other than those illustrated or described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
In accordance with an embodiment of the present invention, there is provided a method embodiment of a log analysis method, it being noted that the steps illustrated in the flowchart of the figure may be performed in a computer system such as a set of computer-executable instructions and that, although a logical order is illustrated in the flowchart, in some cases, the steps illustrated or described may be performed in an order different than that presented herein.
Example one
Fig. 1 is a flowchart of a log analysis method according to an embodiment of the present invention, as shown in fig. 1, the method includes the following steps:
in step S102, first log data is acquired.
Specifically, in order to perform model analysis on log data, the embodiment of the present invention first needs to acquire relevant first log data, where the first log data is original log data acquired by an acquisition device, and therefore, the first log data may be stored after being acquired and processed in subsequent optimization and analysis operations.
Furthermore, when the first log data is acquired, data acquisition and arrangement can be carried out on the log generation device through acquisition equipment, server information system audit operation can also be carried out through a remote server, the audit function is started through a CAT (control and authorization) end of the server in the audit operation, data of the information system are monitored and recorded in a log recording mode, when the first log data is required to be acquired, the log data in an audit module in the remote server can be called through a remote server log calling instruction, and the called log data belong to original unprocessed and processed first log data.
And step S104, generating log data to be analyzed according to the first log data.
Specifically, after the first log data is obtained, because the first log data belongs to original log information, that is, log data that has not been processed, the first log data needs to be processed through a preset or optimization process, and the log data to be analyzed is generated for subsequent analysis and display, so as to increase efficiency and accuracy of log data file processing.
Optionally, the generating log data to be analyzed according to the first log data includes: optimizing the first log data to generate second log data; and converting the second log data into the log data to be analyzed through a preset rule.
Specifically, after the first log data is acquired, in order to analyze and process the log data subsequently, optimization processing needs to be performed on the first log data, and the first log data is optimized to generate second log data; and converting the second log data into the log data to be analyzed through a preset rule, wherein the optimization processing can be to scan the first log data, delete wrong and redundant data, and reorder the log data according to the data occurrence sequence, so as to facilitate the subsequent log analysis.
And step S106, inputting the log data to be analyzed into a log analysis model, and generating an analysis result.
Optionally, before the log data to be analyzed is input into the log analysis model and an analysis result is generated, the method further includes: training the log analysis model through historical data.
Specifically, in order to input log data to be analyzed in the model and output an analysis result, the neural network model for log analysis is trained, the log analysis model trained through historical analysis data can accurately output the analysis result, and the accuracy and the output efficiency of the model are increased according to continuous training.
It should be noted that, when the log to be analyzed is input, input and output analysis can be performed according to the log analysis model constructed by the countermeasure network, and meanwhile, when the log analysis model is constructed, the log analysis model can also be repeatedly trained and perfected through similar log analysis historical data, so as to increase the accuracy and reliability of the log analysis model.
And step S108, displaying the analysis result.
Specifically, after the analysis result of the log data is obtained, in order to facilitate subsequent access and application by the user, the analysis result of the log data needs to be displayed, and the display operation may be to send the display data to the user terminal through a remote data transmitting terminal of the processor, or to directly display the analysis result in a local display device.
Optionally, after displaying the analysis result, the method further includes: and converting the analysis result into the historical data.
Specifically, after the analysis result is displayed, the analysis result is used as new training data to train the log analysis neural network model used in the embodiment of the present invention, so as to increase the precision of the log analysis model and continuously improve the output accuracy and efficiency of the model.
Through the embodiment, the technical problems that log data in the prior art are only subjected to log analysis through fixed analysis rules, and cannot be flexibly utilized or certain change is carried out according to the change of application scene elements, so that the flexibility and the accuracy of log data analysis are reduced are solved.
Example two
Fig. 2 is a block diagram of a log analysis apparatus according to an embodiment of the present invention, as shown in fig. 2, the apparatus including:
the obtaining module 20 is configured to obtain the first log data.
Specifically, in order to perform model analysis on log data, the embodiment of the present invention first needs to acquire relevant first log data, where the first log data is original log data acquired by an acquisition device, and therefore, the first log data may be stored after being acquired and processed in subsequent optimization and analysis operations.
Furthermore, when the first log data is acquired, data acquisition and arrangement can be carried out on the log generation device through acquisition equipment, server information system audit operation can also be carried out through a remote server, the audit function is started through a CAT (control and authorization) end of the server in the audit operation, data of the information system are monitored and recorded in a log recording mode, when the first log data is required to be acquired, the log data in an audit module in the remote server can be called through a remote server log calling instruction, and the called log data belong to original unprocessed and processed first log data.
And the generating module 22 is configured to generate log data to be analyzed according to the first log data.
Specifically, after the first log data is obtained, because the first log data belongs to original log information, that is, log data that has not been processed, the first log data needs to be processed through a preset or optimization process, and the log data to be analyzed is generated for subsequent analysis and display, so as to increase efficiency and accuracy of log data file processing.
Optionally, the generating module includes: the optimization unit is used for optimizing the first log data to generate second log data; and the conversion unit is used for converting the second log data into the log data to be analyzed through a preset rule.
Specifically, after the first log data is acquired, in order to analyze and process the log data subsequently, optimization processing needs to be performed on the first log data, and the first log data is optimized to generate second log data; and converting the second log data into the log data to be analyzed through a preset rule, wherein the optimization processing can be to scan the first log data, delete wrong and redundant data, and reorder the log data according to the data occurrence sequence, so as to facilitate the subsequent log analysis.
And the analysis module 24 is configured to input the log data to be analyzed into a log analysis model, and generate an analysis result.
Optionally, the apparatus further comprises: and the training module is used for training the log analysis model through historical data.
Specifically, in order to input log data to be analyzed in the model and output an analysis result, the neural network model for log analysis is trained, the log analysis model trained through historical analysis data can accurately output the analysis result, and the accuracy and the output efficiency of the model are increased according to continuous training.
It should be noted that, when the log to be analyzed is input, input and output analysis can be performed according to the log analysis model constructed by the countermeasure network, and meanwhile, when the log analysis model is constructed, the log analysis model can also be repeatedly trained and perfected through similar log analysis historical data, so as to increase the accuracy and reliability of the log analysis model.
And the display module 26 is used for displaying the analysis result.
Specifically, after the analysis result of the log data is obtained, in order to facilitate subsequent access and application by the user, the analysis result of the log data needs to be displayed, and the display operation may be to send the display data to the user terminal through a remote data transmitting terminal of the processor, or to directly display the analysis result in a local display device.
Optionally, the apparatus further comprises: and the conversion module is used for converting the analysis result into the historical data.
Specifically, after the analysis result is displayed, the analysis result is used as new training data to train the log analysis neural network model used in the embodiment of the present invention, so as to increase the precision of the log analysis model and continuously improve the output accuracy and efficiency of the model.
Specifically, the method comprises the following steps: acquiring first log data; generating log data to be analyzed according to the first log data; inputting the log data to be analyzed into a log analysis model to generate an analysis result; and displaying the analysis result.
According to another aspect of the embodiments of the present invention, there is also provided a non-volatile storage medium, which includes a stored program, wherein the program controls a device in which the non-volatile storage medium is located to execute a log analysis method when running.
Specifically, the method comprises the following steps: acquiring first log data; generating log data to be analyzed according to the first log data; inputting the log data to be analyzed into a log analysis model to generate an analysis result; and displaying the analysis result.
According to another aspect of the embodiments of the present invention, there is also provided an electronic device, including a processor and a memory; the memory has stored therein computer readable instructions for execution by the processor, wherein the computer readable instructions when executed perform a method of log analysis.
The above-mentioned serial numbers of the embodiments of the present invention are merely for description and do not represent the merits of the embodiments.
In the above embodiments of the present invention, the descriptions of the respective embodiments have respective emphasis, and for parts that are not described in detail in a certain embodiment, reference may be made to related descriptions of other embodiments.
In the embodiments provided in the present application, it should be understood that the disclosed technology can be implemented in other ways. The above-described embodiments of the apparatus are merely illustrative, and for example, the division of the units may be a logical division, and in actual implementation, there may be another division, for example, multiple units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, units or modules, and may be in an electrical or other form.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.
The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a removable hard disk, a magnetic or optical disk, and other various media capable of storing program codes.
The foregoing is only a preferred embodiment of the present invention, and it should be noted that, for those skilled in the art, various modifications and decorations can be made without departing from the principle of the present invention, and these modifications and decorations should also be regarded as the protection scope of the present invention.
Claims (10)
1. A log analysis method, comprising:
acquiring first log data;
generating log data to be analyzed according to the first log data;
inputting the log data to be analyzed into a log analysis model to generate an analysis result;
and displaying the analysis result.
2. The method of claim 1, wherein generating log data to be analyzed from the first log data comprises:
optimizing the first log data to generate second log data;
and converting the second log data into the log data to be analyzed through a preset rule.
3. The method of claim 1, wherein before the inputting the log data to be analyzed into a log analysis model and generating an analysis result, the method further comprises:
training the log analysis model through historical data.
4. The method of claim 3, wherein after said presenting the analysis results, the method further comprises:
and converting the analysis result into the historical data.
5. A log analysis apparatus, comprising:
the acquisition module is used for acquiring first log data;
the generating module is used for generating log data to be analyzed according to the first log data;
the analysis module is used for inputting the log data to be analyzed into a log analysis model to generate an analysis result;
and the display module is used for displaying the analysis result.
6. The apparatus of claim 5, wherein the generating module comprises:
the optimization unit is used for optimizing the first log data to generate second log data;
and the conversion unit is used for converting the second log data into the log data to be analyzed through a preset rule.
7. The apparatus of claim 5, further comprising:
and the training module is used for training the log analysis model through historical data.
8. The apparatus of claim 7, further comprising:
and the conversion module is used for converting the analysis result into the historical data.
9. A non-volatile storage medium, comprising a stored program, wherein the program, when executed, controls an apparatus in which the non-volatile storage medium is located to perform the method of any one of claims 1 to 4.
10. An electronic device comprising a processor and a memory; the memory has stored therein computer readable instructions for execution by the processor, wherein the computer readable instructions when executed perform the method of any one of claims 1 to 4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110669836.0A CN113535655A (en) | 2021-06-17 | 2021-06-17 | Log analysis method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110669836.0A CN113535655A (en) | 2021-06-17 | 2021-06-17 | Log analysis method and device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113535655A true CN113535655A (en) | 2021-10-22 |
Family
ID=78096149
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110669836.0A Pending CN113535655A (en) | 2021-06-17 | 2021-06-17 | Log analysis method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113535655A (en) |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111177095A (en) * | 2019-12-10 | 2020-05-19 | 中移（杭州）信息技术有限公司 | Log analysis method and device, computer equipment and storage medium |
| CN112348041A (en) * | 2019-08-07 | 2021-02-09 | 中移(苏州)软件技术有限公司 | Log classification and log classification training method and device, equipment and storage medium |
| CN112395159A (en) * | 2020-11-17 | 2021-02-23 | 华为技术有限公司 | Log detection method, system, device and medium |
| CN112882898A (en) * | 2021-02-24 | 2021-06-01 | 上海浦东发展银行股份有限公司 | Anomaly detection method, system, device and medium based on big data log analysis |
-
2021
- 2021-06-17 CN CN202110669836.0A patent/CN113535655A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112348041A (en) * | 2019-08-07 | 2021-02-09 | 中移(苏州)软件技术有限公司 | Log classification and log classification training method and device, equipment and storage medium |
| CN111177095A (en) * | 2019-12-10 | 2020-05-19 | 中移（杭州）信息技术有限公司 | Log analysis method and device, computer equipment and storage medium |
| CN112395159A (en) * | 2020-11-17 | 2021-02-23 | 华为技术有限公司 | Log detection method, system, device and medium |
| CN112882898A (en) * | 2021-02-24 | 2021-06-01 | 上海浦东发展银行股份有限公司 | Anomaly detection method, system, device and medium based on big data log analysis |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106874134B (en) | Work order type processing method, device and system | |
| CN111722043B (en) | Power equipment fault detection method, device and system | |
| CN109597974B (en) | Report generation method and device | |
| CN116522403B (en) | Interactive information desensitization method and server for focusing big data privacy security | |
| CN109324789A (en) | A kind of software development methodology | |
| CN109508367A (en) | Automatically extract the method, on-line intelligence customer service system and electronic equipment of question and answer corpus | |
| CN112906806A (en) | Data optimization method and device based on neural network | |
| CN112836807A (en) | Data processing method and device based on neural network | |
| CN113535655A (en) | Log analysis method and device | |
| CN113313615A (en) | Method and device for quantitatively grading and grading enterprise judicial risks | |
| CN111782684B (en) | Distribution network electronic handover information matching method and device | |
| CN114090797A (en) | Intelligent recommendation-based component retrieval method and device | |
| CN114863463A (en) | Intelligent auditing and checking method and device for same text | |
| CN112148544B (en) | Terminal device testing method, smart watch and system | |
| CN113312902A (en) | Intelligent auditing and checking method and device for same text | |
| CN106469086B (en) | Event processing method and device | |
| CN112905579A (en) | Log optimization method and system | |
| CN112783920A (en) | Industrial Internet of things data real-time computing method and system based on data arrangement | |
| CN112800035A (en) | GIS (geographic information System) -based power grid data communication sharing system | |
| CN113312422A (en) | Intelligent news media data structuring method and device based on deep learning | |
| CN113609096A (en) | Data processing method and device | |
| CN113806223A (en) | Software evaluation method and device | |
| CN113055843A (en) | Security communication method and device | |
| CN117201021B (en) | Key dynamic switching method, system, device and medium for main service system | |
| CN113506359A (en) | Animation element acquisition method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20211022 |