CN107273269A - Daily record analysis method and device - Google Patents
Daily record analysis method and device Download PDFInfo
- Publication number
- CN107273269A CN107273269A CN201710440027.6A CN201710440027A CN107273269A CN 107273269 A CN107273269 A CN 107273269A CN 201710440027 A CN201710440027 A CN 201710440027A CN 107273269 A CN107273269 A CN 107273269A
- Authority
- CN
- China
- Prior art keywords
- daily record
- parsing
- analytical model
- modell analytical
- detection property
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/3065—Monitoring arrangements determined by the means or processing involved in reporting the monitored data
- G06F11/3072—Monitoring arrangements determined by the means or processing involved in reporting the monitored data where the reporting involves data filtering, e.g. pattern matching, time or event triggered, adaptive or policy-based reporting
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/14—Network analysis or design
- H04L41/145—Network analysis or design involving simulating, designing, planning or modelling of a network
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/02—Network architectures or network communication protocols for network security for separating internal from external traffic, e.g. firewalls
- H04L63/0227—Filtering policies
- H04L63/0236—Filtering by address, protocol, port number or service, e.g. IP-address or URL
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/02—Network architectures or network communication protocols for network security for separating internal from external traffic, e.g. firewalls
- H04L63/0227—Filtering policies
- H04L63/0245—Filtering by information in the payload
Landscapes
- Engineering & Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Signal Processing (AREA)
- Computer Networks & Wireless Communication (AREA)
- Computing Systems (AREA)
- Computer Security & Cryptography (AREA)
- Computer Hardware Design (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Quality & Reliability (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Debugging And Monitoring (AREA)
- Stored Programmes (AREA)
Abstract
The invention discloses a kind of daily record analysis method and device, wherein, method includes：The type of daily record is determined according to default daily record classifying rules；According to the daily record parsing template generation detection property analytic modell analytical model corresponding with the type；The parsing of detection property is carried out to the daily record by the detection property analytic modell analytical model, the detection property analytic modell analytical model is modified according to detection property analysis result, revised daily record analytic modell analytical model is obtained；The daily record is parsed by the revised daily record analytic modell analytical model.Using this programme, it is possible to achieve the automation parsing of daily record, daily record analyzing efficiency is improved.
Description
Technical field
The present invention relates to communication technical field, and in particular to a kind of daily record analysis method and device.
Background technology
Daily record is the log file for recording each generic operation or event, and all kinds of letters can be obtained by the parsing to daily record
Breath.At present, generally daily record is parsed one by one using the method that manually parses, that is, need system maintenance or developer according to
The compiling of parsing demand is corresponding to be parsed code to parse daily record, and then obtains corresponding analysis result.
But, inventor has found that aforesaid way of the prior art at least exists following during the present invention is realized
Defect：Because the species of original log in practical business is varied, parsing demand is complicated and changeable, according to the above method, needs
Corresponding code is compiled to each field of each daily record to parse, so that the automation parsing of daily record can not be realized, and
When daily record quantity is larger, the analyzing efficiency of the above method is very low.
The content of the invention
In view of the above problems, it is proposed that the present invention so as to provide one kind overcome above mentioned problem or at least in part solve on
State the daily record method and device for analyzing of problem.
According to an aspect of the invention, there is provided a kind of daily record analytic method, including：According to default daily record classification gauge
Then determine the type of daily record；According to the daily record parsing template generation detection property analytic modell analytical model corresponding with the type；By described
Detection property analytic modell analytical model carries out the parsing of detection property to the daily record, according to detection property analysis result to the detection property analytic modell analytical model
It is modified, obtains revised daily record analytic modell analytical model；The daily record is carried out by the revised daily record analytic modell analytical model
Parsing.
According to another aspect of the present invention there is provided a kind of daily record resolver, including：Determination type module, suitable for root
The type of daily record is determined according to default daily record classifying rules；Generation module, suitable for being parsed according to the daily record corresponding with the type
Template generation detection property analytic modell analytical model；Detection property parsing module, suitable for being entered by the detection property analytic modell analytical model to the daily record
Row detection property parsing；Correcting module, suitable for being modified according to detection property analysis result to the detection property analytic modell analytical model, is obtained
Revised daily record analytic modell analytical model；Parsing module, suitable for being carried out by the revised daily record analytic modell analytical model to the daily record
Parsing.
According to another aspect of the invention there is provided a kind of terminal, including：Processor, memory, communication interface and communication
Bus, the processor, the memory and the communication interface complete mutual communication by the communication bus；It is described
Memory is used to deposit an at least executable instruction, and the executable instruction makes the above-mentioned daily record analytic method of the computing device
Corresponding operation.
In accordance with a further aspect of the present invention there is provided a kind of computer-readable storage medium, be stored with the storage medium to
A few executable instruction, the executable instruction makes the corresponding operation of for example above-mentioned daily record analytic method of computing device.
The daily record analysis method and device that the present invention is provided, the type of daily record is determined according to default daily record classifying rules；
According to the daily record parsing template generation detection property analytic modell analytical model corresponding with the type；Daily record is entered by detection property analytic modell analytical model
Row detection property parsing, and detection property analytic modell analytical model is modified according to detection property analysis result, obtain revised daily record solution
Analyse model；Daily record is parsed eventually through revised daily record analytic modell analytical model.Using this programme, it is possible to achieve the intelligence of daily record
Can neutralizing analysis, raising daily record analyzing efficiency.
Described above is only the general introduction of technical solution of the present invention, in order to better understand the technological means of the present invention,
And can be practiced according to the content of specification, and in order to allow above and other objects of the present invention, feature and advantage can
Become apparent, below especially exemplified by the embodiment of the present invention.
Brief description of the drawings
By reading the detailed description of hereafter preferred embodiment, various other advantages and benefit is common for this area
Technical staff will be clear understanding.Accompanying drawing is only used for showing the purpose of preferred embodiment, and is not considered as to the present invention
Limitation.And in whole accompanying drawing, identical part is denoted by the same reference numerals.In the accompanying drawings：
Fig. 1 shows a kind of flow chart of the daily record analytic method provided according to one embodiment of the invention；
Fig. 2 shows a kind of flow chart of the daily record analytic method provided according to a further embodiment of the invention；
Fig. 3 shows a kind of structured flowchart of the daily record resolver provided according to one embodiment of the invention；
Fig. 4 shows a kind of structured flowchart of the daily record resolver provided according to a further embodiment of the invention；
Fig. 5 shows a kind of structural representation of the terminal provided according to one embodiment of the invention.
Embodiment
The exemplary embodiment of the disclosure is more fully described below with reference to accompanying drawings.Although showing the disclosure in accompanying drawing
Exemplary embodiment, it being understood, however, that may be realized in various forms the disclosure without should be by embodiments set forth here
Limited.On the contrary, these embodiments are provided to facilitate a more thoroughly understanding of the present invention, and can be by the scope of the present disclosure
Complete conveys to those skilled in the art.
Fig. 1 shows a kind of flow chart of the daily record analytic method provided according to one embodiment of the invention.Such as Fig. 1 institutes
Show, this method includes：
Step S110, the type of daily record is determined according to default daily record classifying rules.
Wherein, the species of daily record is varied, daily record can be classified according to default classifying rules, for example, can be with
Daily record is classified according to journal format, such as the daily record of nginx default forms, JSON format logs, protobuf daily records.This
Embodiment is not limited specific daily record classifying rules, and those skilled in the art can formulate corresponding daily record according to practical business
Classifying rules.
Step S120, according to the daily record parsing template generation detection property analytic modell analytical model corresponding with the type.
Because the features such as same type of log parameters expression, form are essentially identical.By taking apache daily records as an example, its daily record
Form is substantially stationary, and each field corresponds to the information of same type, and the corresponding span of each field is essentially identical, such as
" c-ip " field correspondence client ip address, its data type is character string type., can be with so for same type of daily record
General analytic method is formulated according to its identical feature, that is, obtains corresponding daily record parsing template.
After the type of daily record is determined, initial solution can be gone out according to the daily record parsing structure of transvers plate corresponding with the type
Model is analysed, i.e. detection property analytic modell analytical model.In actual mechanical process, template can be parsed according to daily record and be loaded onto by suitably modified
In operating file, so as to obtain detection property analytic modell analytical model.
Step S130, the parsing of detection property is carried out to daily record by detection property analytic modell analytical model, according to detection property analysis result pair
Detection property analytic modell analytical model is modified, and obtains revised daily record analytic modell analytical model.
The parsing of detection property is carried out to daily record using detection property analytic modell analytical model in step S120, is accorded with for detection property analysis result
The detection analytic modell analytical model of default analysis result requirement is closed directly as revised daily record analytic modell analytical model, step S140 is performed；
It is modified for the detection analytic modell analytical model that detection property analysis result does not meet default analysis result requirement.If for example parsing is lost
Lose, and/or the parsing time does not parse result, and/or analysis result not in default analysis result scope more than predetermined threshold value
Can be to detection property parsing mould during interior (being " 1332,312,133,113 " such as to IP fields analysis result) detection property analysis result
Type is modified.Specific modification method those skilled in the art can voluntarily be set.
Optionally, when the analysis result of a certain field or certain several field in detection property analysis result does not meet default parsing
When as a result, only the corresponding analytic modell analytical model submethod of the field can be modified.Such as when use character string method is to a certain field
When being parsed, its analysis result does not meet default analysis result requirement, then the field can be parsed using shaping method,
And the corresponding resolution rules of field parsing in detection property analytic modell analytical model are modified, so as to obtain revised daily record parsing
Model.
Step S140, is parsed by revised daily record analytic modell analytical model to daily record.
Daily record is parsed by revised daily record analytic modell analytical model final in step S130, so as to obtain parsing knot
Really.Optionally, also further daily record analytic modell analytical model constantly can be corrected according to the analysis result to daily record.
As can be seen here, the daily record analytic method that the present embodiment is provided, by classifying to daily record, due to same type of
Daily record analytic method is essentially identical, so corresponding parsing template is set for the daily record of each type, and according to parsing template life
Into detection property analytic modell analytical model, detection property analytic modell analytical model is the blank of daily record analytic modell analytical model；By detection property analytic modell analytical model to daily record
Analysis result daily record can be modified, so as to obtain more accurate daily record analytic modell analytical model, pass through the daily record and parse mould
Type is parsed to follow-up daily record, so as to avoid the drawbacks of for all daily records need to manually parse one by one, is realized
The automatic parsing of daily record, improves daily record analyzing efficiency.
Fig. 2 shows a kind of flow chart of the daily record analytic method provided according to a further embodiment of the invention.Such as Fig. 2 institutes
Show, this method includes：
Step S210, the parsing configuration parameter that user inputs is received by default parsing with posting port.
User can be by default parsing with posting port input parsing configuration parameter.Wherein, parsing configuration parameter includes treating
Field name, field type and/or field span of parsing etc..For example, parsing configuration parameter can be " visitor-IP；
char；0.0.0.0-255.255.255.255 ", wherein, " visitor-IP " is the field name corresponding to IP address,
" char " represents that the field type is character string type, and " 0.0.0.0-255.255.255.255 " is the span of the field
(span of legal IP address).
Parsing configuration parameter in the present embodiment includes but is not limited to " field name, field type and/or field to be resolved
Span ", it can also include analytic index, and/or deleting for daily record to be resolved selects condition etc..For example, analytic index can be with
For the PV (Page View, page browsing amount) of some page；Or the daily record in January need to be parsed, then it can be deleted in the corresponding time
Select and January is selected in condition, so as in follow-up resolving, only read in the daily record that the log recording time is January and be solved
Analysis.In addition, parsing configuration parameter can also include original log path, so as to voluntarily specify depositing for daily record to be resolved by user
Storage space is put, and improves the flexibility of daily record parsing.
Step S220, sets daily record classifying rules, and according to daily record classifying rules, set and various types of daily records respectively
Corresponding daily record parsing template.
Wherein, daily record classifying rules includes：Classified according to default daily record characteristic of division, and/or pass through engineering
Algorithm is practised to classify to daily record.
Specifically, in a kind of mode classification, daily record can be classified according to default daily record characteristic of division, for example,
Daily record can be classified with journal format, such as be divided into the daily record of nginx default forms or containing nested form JSON daily records
Deng.
Alternatively, in another mode classification, daily record can be classified by machine learning algorithm.For example, can be pre-
Categorical attribute is first set, usual categorical attribute is multiple.As categorical attribute may include that (property value can be safe day to daily record purposes
Will, O＆M daily record etc.), journal format, daily record producer (such as system journal, application log, web daily records) etc., and lead to
Certain sample training (for example, regarding some conventional Log Types as sample) is crossed, using naive Bayesian, and/or decision-making
The specific machine learning algorithm such as tree obtains daily record disaggregated model, so as to classify to daily record.
According to daily record classifying rules, the daily record corresponding with various types of daily records can be set to parse template respectively.Due to
Same type of daily record has many identical features, and its analytic method is basically identical.So for same type of daily record, can
Template is parsed to formulate general daily record according to its identical feature.Wherein, one or more moulds are included in daily record parsing template
Plate parses class.
Step S230, the type of daily record is determined according to default daily record classifying rules.
It can be determined to need the type of the daily record of parsing according to the daily record classifying rules set in step S220.With default day
Will classifying rules is exemplified by being classified by machine learning algorithm, due to using can after machine learning algorithm progress sample training
Obtain daily record disaggregated model, then can be to daily record to be resolved by obtaining its categorical attribute, and further classified mould according to daily record
Type determines the type of daily record.
Step S240, template and parsing configuration parameter generation detection property are parsed according to the daily record corresponding with Log Types
Analytic modell analytical model.
Wherein, detection property analytic modell analytical model is the corresponding daily record parsing template of the Log Types determined in step S230
On, the model of detection property parsing can be carried out to daily record after suitably modified with reference to parsing configuration parameter.If for example, parsing is matched somebody with somebody
The analytic index included in parameter is put for " the PV templates parsing in such daily record analytic modell analytical model then may be selected in the PV " of some page
Class, names there is any discrepancy (to be ordered in such as template parsing class when some of template parsing class name etc. and actual code are operating
Name length can not be limited, but when restricted to name length etc. in actual code operation), can the letter such as suitably modified name
Breath, and be loaded into operating file, so that subsequent step is parsed to the detection of daily record.
Specifically, the type and quantity that the template included in template parses class are parsed according to daily record, the parsing of detection property is set
At least one daily record parsing class included in model.Wherein, phase can be searched in daily record parsing template according to parsing configuration parameter
The template parsing class answered, the number of the template parsing class found is more than or equal to 1, then the daily record parsing in detection property analytic modell analytical model
The number of class is also greater than equal to 1.For example, for parsing visitor's IP address in daily record, its corresponding template parsing
Class can be 1, then the daily record parsing class included in the detection analytic modell analytical model generated based on this is then 1；And for
Each page PV values seniority among brothers and sisters is calculated, then needs multiple template to parse the combination of class to realize, so being wrapped in detection property analytic modell analytical model
Class is parsed containing multiple daily records.
When the daily record parsing class included in detection property analytic modell analytical model is multiple, then further parsed according to daily record in template
Comprising flow setting rule, set detection property analytic modell analytical model in include multiple daily records parsing class between execution logic.Its
In, execution logic includes：Execution sequence between each daily record parsing class, and/or each daily record parse the execution number of times of class.
Specifically, when the daily record parsing class included in detection property analytic modell analytical model is multiple, to ensure daily record detection property solution
The smooth execution of analysis need to set corresponding execution logic for it.For example, in practical business, when pending daily record data amount compared with
When big, to improve log processing efficiency, usually daily record is handled using distributed system, contains multiple daily record solutions so working as
When analysing class, wherein some or multiple classes can be performed one or many parallelizations and handled, and will be per treatment after result protect
Deposit into internal memory or hard disk (because data in EMS memory access speed is very fast, so preferably by result in the present embodiment
Preserve into internal memory), for reduction processing or the parallelization processing of one or more classes next time, specific execution logic sheet
Art personnel can voluntarily be set, and the present invention is not limited.
For example, so that Log Types are the access log in WEB as an example, when parsing each page PV seniority among brothers and sisters, then it is held
Row logic can be：Page PV calculates the parallelization of parsing class and performs the PV values for once calculating each page afterwards, and result is protected
Deposit into internal memory, seniority among brothers and sisters parsing class reads the above results from internal memory and carries out a reductionization processing, finally parses each page
Face PV is ranked.
Step S250, for each field in daily record, a kind of resolution rules are selected from default a variety of resolution rules
Parsed.
After step S240 generations detection property analytic modell analytical model, daily record to be resolved is solved using detection property analytic modell analytical model
Analysis.Wherein, have a variety of to the resolution rules of same log field in detection property analytic modell analytical model, the field in daily record is visited
In the property surveyed resolving, first a kind of resolution rules are selected to parse the field from a variety of resolution rules.If analysis result
For success, the resolution rules currently selected are defined as to the field resolution rules of respective field.If analysis result is failure, root
The resolution rules of respective field are changed according to default a variety of resolution rules, until analysis result is successfully.
For example, there are a variety of resolution rules for plugin_ver fields in detection property analytic modell analytical model, such as with integer
Parsed or parsed or parsed etc. with character type with floating type.Parsed when to plugin_ver fields
When, first the field can be parsed by the way of integer, if successfully resolved, integer parsing is parsed as follow-up daily record
In the field resolution rules；Resolution rules are changed if failure is parsed (as parsed using character type resolution rules or floating type
Rule etc.) field is parsed, until analysis result is successfully.
Step S260, the field scope of each field in daily record, and root are determined according to the analysis result of detection property parsing
The filtering rule for filter false data is generated according to the field scope of field in daily record.
Specifically, after the parsing of detection property is carried out to a number of daily record, it can be determined according to its detection property analysis result
The effective range of field in daily record, and according to the effective range generate filtering rule, wherein filtering rule can be filter out it is non-
Field in effective range.For example, finding only to use character string solution after plugin_ver fields are carried out the parsing of detection property
Successfully resolved during analysis method, the then effective range that can determine whether out the field is the scope where character style, then filtering can be set
Rule is to filter out the log field that the field is non-character form.
Step S270, filtering rule is added in detection property analytic modell analytical model, and each field in the daily record
Field resolution rules to it is described detection property analytic modell analytical model be modified.
Filtering rule in step S260 is added in detection property analytic modell analytical model, and according to field in step S250
Field resolution rules are modified to detection property analytic modell analytical model.And follow-up daily record is solved with revised analytic modell analytical model
Analysis.Optionally, daily record analytic modell analytical model can also be corrected in time according to analysis result in follow-up resolving.
As can be seen here, the daily record analytic method that the present embodiment is provided, first by classifying to daily record, due to same class
The daily record analytic method of type is essentially identical, so corresponding parsing template is set for the daily record of each type, and according to parsing mould
Plate generation detection property analytic modell analytical model, detection property analytic modell analytical model is the blank of daily record analytic modell analytical model；Pass through detection property analytic modell analytical model pair
The analysis result of daily record can be modified to daily record, so as to obtain more accurate daily record analytic modell analytical model, pass through the daily record solution
Analysis model is parsed to follow-up daily record, so that the drawbacks of for all daily records need to manually parse one by one is avoided,
The automatic parsing of daily record is realized, daily record analyzing efficiency is improved；Also, addition and resolution rules by filtering rule is repaiied
Just, the accuracy rate of daily record parsing is improved further；In addition, detection property analytic modell analytical model is by user configuring parameter and daily record solution
The generation of template collective effect is analysed, so final daily record analysis result can more meet user's request, it also avoid needing in the prior art
The communication cost for being parsed and being produced to daily record by exploitation or attendant.
Fig. 3 shows a kind of functional block diagram of the daily record resolver provided according to one embodiment of the invention.Such as Fig. 3 institutes
Show, the device includes：Determination type module 31, generation module 32, detection property parsing module 33, correcting module 34 and parsing
Module 35.
Determination type module 31, the type suitable for determining daily record according to default daily record classifying rules.
Wherein, the species of daily record is varied, daily record can be classified according to default classifying rules, for example, can be with
Daily record is classified according to journal format, such as the daily record of nginx default forms, JSON format logs, protobuf daily records.This
Embodiment is not limited specific daily record classifying rules, and those skilled in the art can formulate corresponding daily record according to practical business
Classifying rules.
Generation module 32, suitable for according to the daily record parsing template generation detection property analytic modell analytical model corresponding with the type.
Because the features such as same type of log parameters expression, form are essentially identical.By taking apache daily records as an example, its daily record
Form is substantially stationary, and each field corresponds to the information of same type, and the corresponding span of each field is essentially identical, such as
" c-ip " field correspondence client ip address, its data type is character string type., can be with so for same type of daily record
General analytic method is formulated according to its identical feature, that is, obtains corresponding daily record parsing template.
After the type of daily record is determined, initial solution can be gone out according to the daily record parsing structure of transvers plate corresponding with the type
Model is analysed, i.e. detection property analytic modell analytical model.In actual mechanical process, template can be parsed according to daily record and be loaded onto by suitably modified
In operating file, so as to obtain detection property analytic modell analytical model.
Detection property parsing module 33, suitable for carrying out the parsing of detection property to daily record by detection property analytic modell analytical model.
Specifically, the parsing of detection property is carried out to daily record to be resolved using detection property analytic modell analytical model, for the parsing of detection property
As a result the detection analytic modell analytical model of default analysis result requirement is met directly as revised daily record analytic modell analytical model.
Correcting module 34, suitable for being modified according to detection property analysis result to detection property analytic modell analytical model, is obtained after amendment
Daily record analytic modell analytical model.
Specifically, the detection analytic modell analytical model for not meeting default analysis result requirement for detection property analysis result need to be to it
It is modified.If for example parsing failure, and/or parsing time do not parse result more than predetermined threshold value, and/or analysis result is not
(being " 1332,312,133,113 " such as to IP fields analysis result) detection property parsing is tied in the range of default analysis result
Detection property analytic modell analytical model can be modified during fruit.Specific modification method those skilled in the art can voluntarily be set.
Optionally, when the analysis result of a certain field or certain several field in detection property analysis result does not meet default parsing
When as a result, only the corresponding analytic modell analytical model submethod of the field can be modified.Such as when use character string method is to a certain field
When being parsed, its analysis result does not meet default analysis result requirement, then the field can be parsed using shaping method,
And the corresponding resolution rules of field parsing in detection property analytic modell analytical model are modified, so as to obtain revised daily record parsing
Model.
Parsing module 35, suitable for being parsed by revised daily record analytic modell analytical model to daily record.
Daily record is parsed by revised daily record analytic modell analytical model, so as to obtain analysis result.Optionally, it can also enter
One step is constantly corrected according to the analysis result to daily record to daily record analytic modell analytical model.
As can be seen here, the daily record resolver that the present embodiment is provided, by classifying to daily record, due to same type of
Daily record analytic method is essentially identical, so corresponding parsing template is set for the daily record of each type, and according to parsing template life
Into detection property analytic modell analytical model, detection property analytic modell analytical model is the blank of daily record analytic modell analytical model；By detection property analytic modell analytical model to daily record
Analysis result daily record can be modified, so as to obtain more accurate daily record analytic modell analytical model, pass through the daily record and parse mould
Type is parsed to follow-up daily record, so as to avoid the drawbacks of for all daily records need to manually parse one by one, is realized
The automatic parsing of daily record, improves daily record analyzing efficiency.
Fig. 4 shows a kind of functional block diagram of the daily record resolver provided according to a further embodiment of the invention.Such as Fig. 4
It is shown, the device on the basis of Fig. 3 shown devices, in addition to：Receiving module 41, setup module 42 and filtering module 43.
Wherein, receiving module 41 receives the parsing configuration parameter that user inputs by default parsing with posting port.
User can be by default parsing with posting port input parsing configuration parameter.Wherein, parsing configuration parameter includes treating
Field name, field type and/or field span of parsing etc..For example, parsing configuration parameter can be " visitor-IP；
char；0.0.0.0-255.255.255.255 ", wherein, " visitor-IP " is the field name corresponding to IP address,
" char " represents that the field type is character string type, and " 0.0.0.0-255.255.255.255 " is the span of the field
(span of legal IP address).
Parsing configuration parameter in the present embodiment includes but is not limited to " field name, field type and/or field to be resolved
Span ", it can also include analytic index, and/or deleting for daily record to be resolved selects condition etc..For example, analytic index can be with
For the PV (Page View, page browsing amount) of some page；Or the daily record in January need to be parsed, then it can be deleted in the corresponding time
Select and January is selected in condition, so as in follow-up resolving, only read in the daily record that the log recording time is January and be solved
Analysis.In addition, parsing configuration parameter can also include original log path, so as to voluntarily specify depositing for daily record to be resolved by user
Storage space is put, and improves the flexibility of daily record parsing.
Setup module 42, suitable for setting daily record classifying rules, and according to daily record classifying rules, is set and all kinds respectively
Daily record it is corresponding daily record parsing template.
Wherein, daily record classifying rules includes：Classified according to default daily record characteristic of division, and/or pass through engineering
Algorithm is practised to classify to daily record.
Specifically, in a kind of mode classification, daily record can be classified according to default daily record characteristic of division, for example,
Daily record can be classified with journal format, such as be divided into the daily record of nginx default forms or containing nested form JSON daily records
Deng.
Alternatively, in another mode classification, daily record can be classified by machine learning algorithm.For example, can be pre-
Categorical attribute is first set, usual categorical attribute is multiple.As categorical attribute may include that (property value can be safe day to daily record purposes
Will, O＆M daily record etc.), journal format, daily record producer (such as system journal, application log, web daily records) etc., and lead to
Certain sample training (for example, regarding some conventional Log Types as sample) is crossed, using naive Bayesian, and/or decision-making
The specific machine learning algorithm such as tree obtains daily record disaggregated model, so as to classify to daily record.
According to daily record classifying rules, the daily record corresponding with various types of daily records can be set to parse template respectively.Due to
Same type of daily record has many identical features, and its analytic method is basically identical.So for same type of daily record, can
Template is parsed to formulate general daily record according to its identical feature.Wherein, one or more moulds are included in daily record parsing template
Plate parses class.
Generation module 32 is further adapted for：According to the daily record parsing template corresponding with Log Types and parsing configuration ginseng
Number generation detection property analytic modell analytical model.
Wherein, detection property analytic modell analytical model is the corresponding daily record parsing of the Log Types determined in determination type module 31
In template, the model of detection property parsing can be carried out to daily record after suitably modified with reference to parsing configuration parameter.If for example, solution
The analytic index included in analysis configuration parameter is " the PV templates in such daily record analytic modell analytical model then may be selected in the PV " of some page
Class is parsed, when template parsing some of class name etc. and the operating name of actual code there is any discrepancy (such as template parsing class
Middle name length can not be limited, but when restricted to name length etc. in actual code operation), can the suitably modified name
Etc. information, and it is loaded into operating file, so that subsequent step is parsed to the detection of daily record.
Generation module 32 is further adapted for：The type and quantity that the template included in template parses class are parsed according to daily record,
At least one daily record parsing class included in detection property analytic modell analytical model is set.
Wherein, corresponding template parsing class can be searched in daily record parsing template, is found according to parsing configuration parameter
The number of template parsing class is more than or equal to 1, then the number of the daily record parsing class in detection property analytic modell analytical model is also greater than equal to 1.Lift
For example, for parsing visitor's IP address in daily record, its corresponding template parsing class can be 1, then as base
The daily record parsing class included in the detection analytic modell analytical model of plinth generation is then 1；And for calculating each page PV values seniority among brothers and sisters, then
Multiple template is needed to parse the combination of class to realize, so parsing class comprising multiple daily records in detection property analytic modell analytical model.
Optionally, when the daily record parsing class included in detection property analytic modell analytical model is multiple, the present apparatus also includes：Logic is set
Module 44, what is included suitable for being parsed according to daily record in the flow setting included in template rule, setting detection property analytic modell analytical model is more
Execution logic between individual daily record parsing class.
Wherein, execution logic includes：Execution sequence between each daily record parsing class, and/or each daily record parsing class
Perform number of times.
Specifically, when the daily record parsing class included in detection property analytic modell analytical model is multiple, to ensure daily record detection property solution
The smooth execution of analysis need to set corresponding execution logic for it.For example, in practical business, when pending daily record data amount compared with
When big, to improve log processing efficiency, usually daily record is handled using distributed system, contains multiple daily record solutions so working as
When analysing class, wherein some or multiple classes can be performed one or many parallelizations and handled, and will be per treatment after result protect
Deposit into internal memory or hard disk (because data in EMS memory access speed is very fast, so preferably by result in the present embodiment
Preserve into internal memory), for reduction processing or the parallelization processing of one or more classes next time, specific execution logic sheet
Art personnel can voluntarily be set, and the present invention is not limited.
For example, so that Log Types are the access log in WEB as an example, when parsing each page PV seniority among brothers and sisters, then it is held
Row logic can be：Page PV calculates the parallelization of parsing class and performs the PV values for once calculating each page afterwards, and result is protected
Deposit into internal memory, seniority among brothers and sisters parsing class reads the above results from internal memory and carries out a reductionization processing, finally parses each page
Face PV is ranked.
Detection property parsing module 33 is further adapted for：For each field in daily record, from default a variety of resolution rules
A kind of middle resolution rules of selection are parsed.
After generation detection property analytic modell analytical model, daily record to be resolved is parsed using detection property analytic modell analytical model.Wherein,
Have a variety of to the resolution rules of same log field in detection property analytic modell analytical model, the field in daily record carries out the parsing of detection property
During, first select a kind of resolution rules to parse the field from a variety of resolution rules., will if analysis result is successfully
The resolution rules currently selected are defined as the field resolution rules of respective field.If analysis result is failure, according to described pre-
If a variety of resolution rules change the resolution rules of respective field, until analysis result is successfully.
For example, there are a variety of resolution rules for plugin_ver fields in detection property analytic modell analytical model, such as with integer
Parsed or parsed or parsed etc. with character type with floating type.Parsed when to plugin_ver fields
When, first the field can be parsed by the way of integer, if successfully resolved, integer parsing is parsed as follow-up daily record
In the field resolution rules；Resolution rules are changed if failure is parsed (as parsed using character type resolution rules or floating type
Rule etc.) field is parsed, until analysis result is successfully.
Filtering module 43, the field model suitable for determining each field in daily record according to the analysis result of detection property parsing
Enclose, and the field scope of each field in daily record generates filtering rule for filter false data.
Specifically, after the parsing of detection property is carried out to a number of daily record, it can be determined according to its detection property analysis result
The effective range of field in daily record, and according to the effective range generate filtering rule, wherein filtering rule can be filter out it is non-
Field in effective range.For example, finding only to use character string solution after plugin_ver fields are carried out the parsing of detection property
Successfully resolved during analysis method, the then effective range that can determine whether out the field is the scope where character style, then filtering can be set
Rule is to filter out the log field that the field is non-character form.
Correcting module 34 is further adapted for：Filtering rule is added in the detection property analytic modell analytical model, and according to daily record
In each field field resolution rules to detection property analytic modell analytical model be modified.
Specifically, filtering rule is added in detection property analytic modell analytical model, and according to field resolution rules to detection property
Analytic modell analytical model is modified.And follow-up daily record is parsed with revised analytic modell analytical model.Optionally, in follow-up parsing
During can also correct daily record analytic modell analytical model in time according to analysis result.
As can be seen here, the daily record resolver that the present embodiment is provided, first by classifying to daily record, due to same class
The daily record analytic method of type is essentially identical, so corresponding parsing template is set for the daily record of each type, and according to parsing mould
Plate generation detection property analytic modell analytical model, detection property analytic modell analytical model is the blank of daily record analytic modell analytical model；Pass through detection property analytic modell analytical model pair
The analysis result of daily record can be modified to daily record, so as to obtain more accurate daily record analytic modell analytical model, pass through the daily record solution
Analysis model is parsed to follow-up daily record, so that the drawbacks of for all daily records need to manually parse one by one is avoided,
The automatic parsing of daily record is realized, daily record analyzing efficiency is improved；Also, addition and resolution rules by filtering rule is repaiied
Just, the accuracy rate of daily record parsing is improved further；In addition, detection property analytic modell analytical model is by user configuring parameter and daily record solution
The generation of template collective effect is analysed, so final daily record analysis result can more meet user's request, it also avoid needing in the prior art
The communication cost for being parsed and being produced to daily record by exploitation or attendant.
A kind of nonvolatile computer storage media is provided according to one embodiment of the invention, the computer storage is situated between
Matter is stored with an at least executable instruction, and the computer executable instructions can perform the daily record solution in above-mentioned any means embodiment
Analysis method.
Fig. 5 shows a kind of structural representation of the terminal provided according to one embodiment of the invention, and the present invention is specific real
Example is applied not limit implementing for terminal.
As shown in figure 5, the terminal can include：Processor (processor) 502, communication interface (Communications
Interface) 504, memory (memory) 506 and communication bus 508.
Wherein：
Processor 502, communication interface 504 and memory 506 complete mutual communication by communication bus 508.
Communication interface 504, communicates for the network element with miscellaneous equipment such as client or other servers etc..
Processor 502, for configuration processor 510, can specifically perform the correlation in above-mentioned daily record analytic method embodiment
Step.
Specifically, program 510 can include program code, and the program code includes computer-managed instruction.
Processor 502 is probably central processor CPU, or specific integrated circuit ASIC (Application
Specific Integrated Circuit), or it is arranged to implement one or more integrated electricity of the embodiment of the present invention
Road.The one or more processors that terminal includes, can be same type of processors, such as one or more CPU；Can also be
Different types of processor, such as one or more CPU and one or more ASIC.
Memory 506, for depositing program 510.Memory 506 may include high-speed RAM memory, it is also possible to also include
Nonvolatile memory (non-volatile memory), for example, at least one magnetic disk storage.
Program 510 specifically can be used for so that processor 502 performs following operation：
The type of daily record is determined according to default daily record classifying rules；
According to the daily record parsing template generation detection property analytic modell analytical model corresponding with the type；
The parsing of detection property is carried out to the daily record by the detection property analytic modell analytical model, according to detection property analysis result to institute
State detection property analytic modell analytical model to be modified, obtain revised daily record analytic modell analytical model；
The daily record is parsed by the revised daily record analytic modell analytical model.
Algorithm and display be not inherently related to any certain computer, virtual system or miscellaneous equipment provided herein.
Various general-purpose systems can also be used together with based on teaching in this.As described above, construct required by this kind of system
Structure be obvious.In addition, the present invention is not also directed to any certain programmed language.It is understood that, it is possible to use it is various
Programming language realizes the content of invention described herein, and the description done above to language-specific is to disclose this hair
Bright preferred forms.
In the specification that this place is provided, numerous specific details are set forth.It is to be appreciated, however, that the implementation of the present invention
Example can be put into practice in the case of these no details.In some instances, known method, structure is not been shown in detail
And technology, so as not to obscure the understanding of this description.
Similarly, it will be appreciated that in order to simplify the disclosure and help to understand one or more of each inventive aspect, exist
Above in the description of the exemplary embodiment of the present invention, each feature of the invention is grouped together into single implementation sometimes
In example, figure or descriptions thereof.However, the method for the disclosure should be construed to reflect following intention：It is i.e. required to protect
The application claims of shield features more more than the feature being expressly recited in each claim.More precisely, such as following
Claims reflect as, inventive aspect is all features less than single embodiment disclosed above.Therefore,
Thus the claims for following embodiment are expressly incorporated in the embodiment, wherein each claim is in itself
All as the separate embodiments of the present invention.
Those skilled in the art, which are appreciated that, to be carried out adaptively to the module in the equipment in embodiment
Change and they are arranged in one or more equipment different from the embodiment.Can be the module or list in embodiment
Member or component be combined into a module or unit or component, and can be divided into addition multiple submodule or subelement or
Sub-component.In addition at least some in such feature and/or process or unit exclude each other, it can use any
Combination is disclosed to all features disclosed in this specification (including adjoint claim, summary and accompanying drawing) and so to appoint
Where all processes or unit of method or equipment are combined.Unless expressly stated otherwise, this specification (including adjoint power
Profit is required, summary and accompanying drawing) disclosed in each feature can or similar purpose identical, equivalent by offer alternative features come generation
Replace.
Although in addition, it will be appreciated by those of skill in the art that some embodiments described herein include other embodiments
In included some features rather than further feature, but the combination of the feature of be the same as Example does not mean in of the invention
Within the scope of and form different embodiments.For example, in the following claims, times of embodiment claimed
One of meaning mode can be used in any combination.
The present invention all parts embodiment can be realized with hardware, or with one or more processor run
Software module realize, or realized with combinations thereof.It will be understood by those of skill in the art that can use in practice
Microprocessor or digital signal processor (DSP) come realize in daily record resolver according to embodiments of the present invention some or
The some or all functions of person's whole part.The present invention is also implemented as perform method as described herein one
Divide or whole equipment or program of device (for example, computer program and computer program product).It is such to realize this hair
Bright program can be stored on a computer-readable medium, or can have the form of one or more signal.It is such
Signal can be downloaded from internet website and obtained, and either provided or provided in any other form on carrier signal.
It should be noted that the present invention will be described rather than limits the invention for above-described embodiment, and ability
Field technique personnel can design alternative embodiment without departing from the scope of the appended claims.In the claims,
Any reference symbol between bracket should not be configured to limitations on claims.Word "comprising" is not excluded the presence of not
Element or step listed in the claims.Word "a" or "an" before element does not exclude the presence of multiple such
Element.The present invention can be by means of including the hardware of some different elements and coming real by means of properly programmed computer
It is existing.In if the unit claim of equipment for drying is listed, several in these devices can be by same hardware branch
To embody.The use of word first, second, and third does not indicate that any order.These words can be explained and run after fame
Claim.
The invention discloses：A1. a kind of daily record analytic method, including：
The type of daily record is determined according to default daily record classifying rules；
According to the daily record parsing template generation detection property analytic modell analytical model corresponding with the type；
The parsing of detection property is carried out to the daily record by the detection property analytic modell analytical model, according to detection property analysis result to institute
State detection property analytic modell analytical model to be modified, obtain revised daily record analytic modell analytical model；
The daily record is parsed by the revised daily record analytic modell analytical model.
A2. the method as described in A1, wherein, it is described the step of determine the type of daily record according to default daily record classifying rules
Before, step is further comprised：The daily record classifying rules is set, and according to the daily record classifying rules, set respectively and each
The daily record that the daily record of type is corresponding parses template；
Wherein, the daily record classifying rules includes：Classified according to default daily record characteristic of division, and/or, pass through machine
Device learning algorithm is classified.
A3. the method as described in A1 or A2, wherein, the basis daily record parsing template generation corresponding with the type is visited
Before the step of property surveyed analytic modell analytical model, further comprise：
The parsing configuration parameter that user inputs is received with posting port by default parsing, then the basis and the type phase
The step of corresponding daily record parsing template generation detection property analytic modell analytical model, specifically includes：According to the daily record solution corresponding with the type
Analyse template and parsing configuration parameter generation detection property analytic modell analytical model.
A4. the method as described in A3, wherein, the parsing configuration parameter includes：Field name to be resolved, field type
And/or field span.
A5. the method as described in A1-A4 is any, wherein, the basis daily record parsing template life corresponding with the type
The step of into detection property analytic modell analytical model, specifically includes：
The type and quantity that the template included in template parses class are parsed according to the daily record, detection property analytic modell analytical model is set
In include at least one daily record parsing class.
A6. the method as described in A5, wherein, when the daily record parsing class included in the detection property analytic modell analytical model is multiple
When, the setting is detected after the step of at least one daily record included in property analytic modell analytical model parses class, is further comprised：
According to the flow setting rule included in daily record parsing template, what is included in setting detection property analytic modell analytical model is more
Execution logic between individual daily record parsing class；Wherein, the execution logic includes：Execution between each daily record parsing class is suitable
Sequence, and/or each daily record parse the execution number of times of class.
A7. the method as described in A1-A6 is any, wherein, it is described that the daily record is entered by the detection property analytic modell analytical model
The step of row detection property is parsed specifically includes：
For each field in the daily record, a kind of resolution rules are selected to be solved from default a variety of resolution rules
Analysis；
If analysis result is successfully, the resolution rules currently selected are defined as to the field resolution rules of respective field；If
Analysis result is failure, then the resolution rules of respective field are changed according to default a variety of resolution rules, until parsing knot
Fruit is successfully.
A8. the method as described in A7, wherein, it is described that detection is carried out to the daily record by the detection property analytic modell analytical model
The step of parsing, further comprises：
The field scope of each field in the daily record is determined according to the analysis result of detection property parsing, and according to described
The field scope of each field in daily record generates the filtering rule for filter false data；
It is then described that the detection property analytic modell analytical model is modified according to detection property analysis result, obtain revised daily record
The step of analytic modell analytical model, specifically includes：
The filtering rule is added in the detection property analytic modell analytical model, and each field in the daily record
Field resolution rules are modified to the detection property analytic modell analytical model.
The invention also discloses：B9. a kind of daily record resolver, including：
Determination type module, the type suitable for determining daily record according to default daily record classifying rules；
Generation module, suitable for according to the daily record parsing template generation detection property analytic modell analytical model corresponding with the type；
Detection property parsing module, suitable for carrying out the parsing of detection property to the daily record by the detection property analytic modell analytical model；
Correcting module, suitable for being modified according to detection property analysis result to the detection property analytic modell analytical model, is corrected
Daily record analytic modell analytical model afterwards；
Parsing module, suitable for being parsed by the revised daily record analytic modell analytical model to the daily record.
B10. the device as described in B9, wherein, described device also includes：
Setup module, suitable for setting the daily record classifying rules, and according to the daily record classifying rules, is set and each respectively
The daily record that the daily record of type is corresponding parses template；
Wherein, the daily record classifying rules includes：Classified according to default daily record characteristic of division, and/or, pass through machine
Device learning algorithm is classified.
B11. the device as described in B9 or B10, wherein, described device also includes：
Receiving module, suitable for receiving the parsing configuration parameter that user inputs with posting port by default parsing；
The generation module is further adapted for：Matched somebody with somebody according to the daily record parsing template corresponding with the type and the parsing
Put parameter generation detection property analytic modell analytical model.
B12. the device as described in B11, wherein, the parsing configuration parameter includes：Field name to be resolved, field type
And/or field span.
B13. the device as described in B9-B12 is any, wherein, the generation module is further adapted for：
The type and quantity that the template included in template parses class are parsed according to the daily record, detection property analytic modell analytical model is set
In include at least one daily record parsing class.
B14. the device as described in B13, wherein, when the daily record parsing class included in the detection property analytic modell analytical model is multiple
When, described device also includes：
Logic setting module, suitable for parsing the flow setting included in template rule according to the daily record, sets detection property
Execution logic between the multiple daily records parsing class included in analytic modell analytical model；Wherein, the execution logic includes：Each daily record solution
The execution sequence between class is analysed, and/or each daily record parses the execution number of times of class.
B15. the device as described in B9-B14 is any, wherein, the detection property parsing module is further adapted for：
For each field in the daily record, a kind of resolution rules are selected to be solved from default a variety of resolution rules
Analysis；
If analysis result is successfully, the resolution rules currently selected are defined as to the field resolution rules of respective field；If
Analysis result is failure, then the resolution rules of respective field are changed according to default a variety of resolution rules, until parsing knot
Fruit is successfully.
B16. the device as described in B15, wherein, described device also includes：
Filtering module, the field model suitable for determining each field in the daily record according to the analysis result of detection property parsing
Enclose, and the field scope of each field in the daily record generates filtering rule for filter false data；
The correcting module is further adapted for：The filtering rule is added in the detection property analytic modell analytical model, and root
The detection property analytic modell analytical model is modified according to the field resolution rules of each field in the daily record.
The invention also discloses：C17. a kind of terminal, including：Processor, memory, communication interface and communication bus, institute
State processor, the memory and the communication interface and mutual communication is completed by the communication bus；
The memory is used to deposit an at least executable instruction, and the executable instruction makes the computing device such as
The corresponding operation of daily record analytic method any one of A1-A8.
The invention also discloses：D18. being stored with a kind of computer-readable storage medium, the storage medium at least one can hold
Row instruction, the executable instruction makes the corresponding operation of daily record analytic method of the computing device as any one of A1-A8.
Claims (10)
1. a kind of daily record analytic method, including：
The type of daily record is determined according to default daily record classifying rules；
According to the daily record parsing template generation detection property analytic modell analytical model corresponding with the type；
The parsing of detection property is carried out to the daily record by the detection property analytic modell analytical model, visited according to detection property analysis result to described
The property surveyed analytic modell analytical model is modified, and obtains revised daily record analytic modell analytical model；
The daily record is parsed by the revised daily record analytic modell analytical model.
2. the method for claim 1, wherein step of the type that daily record is determined according to default daily record classifying rules
Before rapid, further comprise step：The daily record classifying rules is set, and according to the daily record classifying rules, set respectively with
The corresponding daily record parsing template of various types of daily records；
Wherein, the daily record classifying rules includes：Classified according to default daily record characteristic of division, and/or, pass through engineering
Algorithm is practised to be classified.
3. method as claimed in claim 1 or 2, wherein, the basis daily record parsing template generation corresponding with the type
Before the step of detection property analytic modell analytical model, further comprise：
The parsing configuration parameter that user inputs is received with posting port by default parsing, then the basis is corresponding with the type
Daily record parsing template generation detection property analytic modell analytical model the step of specifically include：Mould is parsed according to the daily record corresponding with the type
Plate and parsing configuration parameter generation detection property analytic modell analytical model.
4. method as claimed in claim 3, wherein, the parsing configuration parameter includes：Field name to be resolved, field type
And/or field span.
5. the method as described in claim 1-4 is any, wherein, the basis daily record parsing template life corresponding with the type
The step of into detection property analytic modell analytical model, specifically includes：
The type and quantity that the template included in template parses class are parsed according to the daily record, sets in detection property analytic modell analytical model and wraps
At least one daily record parsing class contained.
6. method as claimed in claim 5, wherein, when the daily record parsing class included in the detection property analytic modell analytical model is multiple
When, the setting is detected after the step of at least one daily record included in property analytic modell analytical model parses class, is further comprised：
The flow setting rule included in template is parsed according to the daily record, the multiple days included in detection property analytic modell analytical model are set
Execution logic between will parsing class；Wherein, the execution logic includes：Execution sequence between each daily record parsing class, and/
Or each daily record parses the execution number of times of class.
7. the method as described in claim 1-6 is any, wherein, it is described that the daily record is entered by the detection property analytic modell analytical model
The step of row detection property is parsed specifically includes：
For each field in the daily record, a kind of resolution rules are selected to be parsed from default a variety of resolution rules；
If analysis result is successfully, the resolution rules currently selected are defined as to the field resolution rules of respective field；If parsing
As a result it is failure, then the resolution rules of respective field is changed according to default a variety of resolution rules, until analysis result is
Success.
8. a kind of daily record resolver, including：
Determination type module, the type suitable for determining daily record according to default daily record classifying rules；
Generation module, suitable for according to the daily record parsing template generation detection property analytic modell analytical model corresponding with the type；
Detection property parsing module, suitable for carrying out the parsing of detection property to the daily record by the detection property analytic modell analytical model；
Correcting module, suitable for being modified according to detection property analysis result to the detection property analytic modell analytical model, obtains revised
Daily record analytic modell analytical model；
Parsing module, suitable for being parsed by the revised daily record analytic modell analytical model to the daily record.
9. a kind of terminal, including：Processor, memory, communication interface and communication bus, the processor, the memory and
The communication interface completes mutual communication by the communication bus；
The memory is used to deposit an at least executable instruction, and the executable instruction makes the computing device such as right will
Ask the corresponding operation of the daily record analytic method any one of 1-7.
10. be stored with an at least executable instruction, the executable instruction in a kind of computer-readable storage medium, the storage medium
Make the corresponding operation of daily record analytic method of the computing device as any one of claim 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710440027.6A CN107273269B (en) | 2017-06-12 | 2017-06-12 | Log analysis method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710440027.6A CN107273269B (en) | 2017-06-12 | 2017-06-12 | Log analysis method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN107273269A true CN107273269A (en) | 2017-10-20 |
| CN107273269B CN107273269B (en) | 2021-04-23 |
Family
ID=60066087
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710440027.6A Active CN107273269B (en) | 2017-06-12 | 2017-06-12 | Log analysis method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107273269B (en) |
Cited By (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108418842A (en) * | 2018-05-31 | 2018-08-17 | 郑州信大天瑞信息技术有限公司 | A kind of intranet security log collection method and system |
| CN108763461A (en) * | 2018-05-28 | 2018-11-06 | 上海七牛信息技术有限公司 | Data processing method, device, system and storage medium |
| CN109325009A (en) * | 2018-09-19 | 2019-02-12 | 亚信科技（成都）有限公司 | The method and device of log parsing |
| CN109347827A (en) * | 2018-10-22 | 2019-02-15 | 东软集团股份有限公司 | Method, apparatus, equipment and the storage medium of attack prediction |
| CN109688027A (en) * | 2018-12-24 | 2019-04-26 | 努比亚技术有限公司 | A kind of collecting method, device, equipment, system and storage medium |
| CN109947715A (en) * | 2018-09-07 | 2019-06-28 | 网联清算有限公司 | Log alarm method and device |
| CN110808965A (en) * | 2019-10-22 | 2020-02-18 | 许继集团有限公司 | Debugging method and device of monitoring system |
| CN111061696A (en) * | 2019-12-17 | 2020-04-24 | 中国银行股份有限公司 | Method and device for analyzing transaction message log |
| CN111367964A (en) * | 2020-02-29 | 2020-07-03 | 上海爱数信息技术股份有限公司 | Method for automatically analyzing log |
| CN115065536A (en) * | 2022-06-16 | 2022-09-16 | 北京天融信网络安全技术有限公司 | Network security data analyzer, analysis method, electronic device and storage medium |
| TWI778698B (en) * | 2021-06-16 | 2022-09-21 | 大陸商深圳富桂精密工業有限公司 | Log analysis system and log analysis method |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103929321A (en) * | 2013-01-15 | 2014-07-16 | 腾讯科技（深圳）有限公司 | Log processing method and device |
| CN104104734A (en) * | 2014-08-04 | 2014-10-15 | 浪潮(北京)电子信息产业有限公司 | Log analysis method and device |
| CN105447099A (en) * | 2015-11-11 | 2016-03-30 | 中国建设银行股份有限公司 | Log structured information extraction method and apparatus |
| CN106168909A (en) * | 2016-06-30 | 2016-11-30 | 北京奇虎科技有限公司 | A kind for the treatment of method and apparatus of daily record |
| CN106656607A (en) * | 2016-12-27 | 2017-05-10 | 上海爱数信息技术股份有限公司 | Equipment log parsing method and system, and server side having system |
| US20170132523A1 (en) * | 2015-11-09 | 2017-05-11 | Nec Laboratories America, Inc. | Periodicity Analysis on Heterogeneous Logs |
-
2017
- 2017-06-12 CN CN201710440027.6A patent/CN107273269B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103929321A (en) * | 2013-01-15 | 2014-07-16 | 腾讯科技（深圳）有限公司 | Log processing method and device |
| CN104104734A (en) * | 2014-08-04 | 2014-10-15 | 浪潮(北京)电子信息产业有限公司 | Log analysis method and device |
| US20170132523A1 (en) * | 2015-11-09 | 2017-05-11 | Nec Laboratories America, Inc. | Periodicity Analysis on Heterogeneous Logs |
| CN105447099A (en) * | 2015-11-11 | 2016-03-30 | 中国建设银行股份有限公司 | Log structured information extraction method and apparatus |
| CN106168909A (en) * | 2016-06-30 | 2016-11-30 | 北京奇虎科技有限公司 | A kind for the treatment of method and apparatus of daily record |
| CN106656607A (en) * | 2016-12-27 | 2017-05-10 | 上海爱数信息技术股份有限公司 | Equipment log parsing method and system, and server side having system |
Cited By (18)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108763461A (en) * | 2018-05-28 | 2018-11-06 | 上海七牛信息技术有限公司 | Data processing method, device, system and storage medium |
| CN108418842A (en) * | 2018-05-31 | 2018-08-17 | 郑州信大天瑞信息技术有限公司 | A kind of intranet security log collection method and system |
| CN109947715B (en) * | 2018-09-07 | 2021-08-27 | 网联清算有限公司 | Log alarm method and device |
| CN109947715A (en) * | 2018-09-07 | 2019-06-28 | 网联清算有限公司 | Log alarm method and device |
| CN109325009A (en) * | 2018-09-19 | 2019-02-12 | 亚信科技（成都）有限公司 | The method and device of log parsing |
| CN109325009B (en) * | 2018-09-19 | 2021-11-30 | 亚信科技（成都）有限公司 | Log analysis method and device |
| CN109347827B (en) * | 2018-10-22 | 2021-06-22 | 东软集团股份有限公司 | Method, device, equipment and storage medium for predicting network attack behavior |
| CN109347827A (en) * | 2018-10-22 | 2019-02-15 | 东软集团股份有限公司 | Method, apparatus, equipment and the storage medium of attack prediction |
| CN109688027A (en) * | 2018-12-24 | 2019-04-26 | 努比亚技术有限公司 | A kind of collecting method, device, equipment, system and storage medium |
| CN110808965A (en) * | 2019-10-22 | 2020-02-18 | 许继集团有限公司 | Debugging method and device of monitoring system |
| CN110808965B (en) * | 2019-10-22 | 2022-11-25 | 许继集团有限公司 | Debugging method and device of monitoring system |
| CN111061696A (en) * | 2019-12-17 | 2020-04-24 | 中国银行股份有限公司 | Method and device for analyzing transaction message log |
| CN111061696B (en) * | 2019-12-17 | 2023-03-31 | 中国银行股份有限公司 | Method and device for analyzing transaction message log |
| CN111367964A (en) * | 2020-02-29 | 2020-07-03 | 上海爱数信息技术股份有限公司 | Method for automatically analyzing log |
| CN111367964B (en) * | 2020-02-29 | 2023-11-17 | 上海爱数信息技术股份有限公司 | Method for automatically analyzing log |
| TWI778698B (en) * | 2021-06-16 | 2022-09-21 | 大陸商深圳富桂精密工業有限公司 | Log analysis system and log analysis method |
| CN115065536A (en) * | 2022-06-16 | 2022-09-16 | 北京天融信网络安全技术有限公司 | Network security data analyzer, analysis method, electronic device and storage medium |
| CN115065536B (en) * | 2022-06-16 | 2023-08-25 | 北京天融信网络安全技术有限公司 | Network security data parser, parsing method, electronic device and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN107273269B (en) | 2021-04-23 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107273269A (en) | Daily record analysis method and device | |
| CN108121795A (en) | User's behavior prediction method and device | |
| CN106919702A (en) | Keyword method for pushing and device based on document | |
| CN110910982A (en) | Self-coding model training method, device, equipment and storage medium | |
| CN106778241A (en) | The recognition methods of malicious file and device | |
| CN110134845A (en) | Project public sentiment monitoring method, device, computer equipment and storage medium | |
| CN104462301A (en) | Network data processing method and device | |
| CN109918296B (en) | Software automation test method and device | |
| CN106960017A (en) | E-book is classified and its training method, device and equipment | |
| CN111291125B (en) | Data processing method and related equipment | |
| US20210209011A1 (en) | Systems and methods for automated testing using artificial intelligence techniques | |
| CN107463935A (en) | Application class methods and applications sorter | |
| US11836331B2 (en) | Mathematical models of graphical user interfaces | |
| CN106897454A (en) | A kind of file classifying method and device | |
| CN112232944B (en) | Method and device for creating scoring card and electronic equipment | |
| CN111353600A (en) | Abnormal behavior detection method and device | |
| CN105989066A (en) | Information processing method and device | |
| Bateman et al. | The The Supervised Learning Workshop: A New, Interactive Approach to Understanding Supervised Learning Algorithms | |
| CN115423040A (en) | User portrait identification method and AI system of interactive marketing platform | |
| CN110019784A (en) | A kind of file classification method and device | |
| CN112508692A (en) | Resource recovery risk prediction method and device based on convolutional neural network and electronic equipment | |
| CN117009232A (en) | Code coverage rate testing method and device | |
| CN116932694A (en) | Intelligent retrieval method, device and storage medium for knowledge base | |
| US8954307B1 (en) | Chained programming language preprocessors for circuit simulation | |
| CN108241643A (en) | The achievement data analysis method and device of keyword |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |