CN113810160A - Intelligent access system of multi-element network equipment - Google Patents
Intelligent access system of multi-element network equipment Download PDFInfo
- Publication number
- CN113810160A CN113810160A CN202111094007.0A CN202111094007A CN113810160A CN 113810160 A CN113810160 A CN 113810160A CN 202111094007 A CN202111094007 A CN 202111094007A CN 113810160 A CN113810160 A CN 113810160A
- Authority
- CN
- China
- Prior art keywords
- data
- analysis
- network
- format
- equipment
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L1/00—Arrangements for detecting or preventing errors in the information received
- H04L1/004—Arrangements for detecting or preventing errors in the information received by using forward error control
- H04L1/0056—Systems characterized by the type of code used
- H04L1/0057—Block codes
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L1/00—Arrangements for detecting or preventing errors in the information received
- H04L1/0078—Avoidance of errors by organising the transmitted data in a format specifically designed to deal with errors, e.g. location
- H04L1/0084—Formats for payload data
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
The invention relates to an intelligent access system of multi-element network equipment, belongs to the technical field of network equipment access, and solves the problems of high difficulty in network security analysis, low efficiency and low accuracy caused by non-uniform network data formats of all network equipment. The intelligent access system comprises: the device access interface component is used for receiving the network data of each network device; the network data comprises equipment type, data format, coding format and data content; the data analysis component is connected with the equipment access interface component and used for analyzing the network data from the equipment access interface component to obtain corresponding analysis data; and the data normalization processing component is connected with the data analysis component and is used for mapping the generated analysis data to the normalized data format based on the mapping relation between the data format and the normalized data format of the network data of each network device to obtain a normalized data set. The system can ensure the consistency of the obtained data in the normalized data set.
Description
Technical Field
The invention relates to the technical field of network equipment access, in particular to an intelligent access system of multi-element network equipment.
Background
Before analyzing the network security, it is a primary task to be able to obtain network data of the relevant network devices. However, since the formats of the network data of the network devices are not uniform, the data quality for the network security analysis is poor, the difficulty of the network security analysis is increased, and the analysis efficiency and the analysis accuracy are reduced.
Therefore, there is a need for an intelligent access system for multiple network devices to solve the above problems in the prior art.
Disclosure of Invention
In view of the foregoing analysis, embodiments of the present invention provide an intelligent access system for multiple network devices, so as to solve the problems of difficult network security analysis, low efficiency and low accuracy caused by non-uniform network data formats of the network devices.
The invention discloses an intelligent access system of multi-element network equipment, which comprises:
the device access interface component is used for receiving the network data of each network device; the network data comprises equipment type, data format, coding format and data content;
the data analysis component is connected with the equipment access interface component and used for analyzing the network data from the equipment access interface component to obtain corresponding analysis data;
and the data normalization processing component is connected with the data analysis component and is used for mapping the generated analysis data to the normalized data format based on the mapping relation between the data format and the normalized data format of the network data of each network device to obtain a normalized data set.
On the basis of the scheme, the invention also makes the following improvements:
further, the device access interface component comprises a plurality of device access interfaces, each device access interface having a unique interface ID; each equipment access interface is used for connecting network equipment which is suitable for the type of the equipment access interface and receiving network data of the network equipment; the network data further includes an interface ID of the device access interface.
Further, the data analysis component is composed of a plurality of data analysis channels; each equipment access interface is uniquely associated with a data analysis channel;
each data analysis channel is used for analyzing the network data received by the associated equipment access interface to obtain corresponding analysis data.
Further, the data normalization processing component is composed of a plurality of normalization processing channels; each normalization processing channel is uniquely associated with one data analysis channel and one equipment access interface;
each normalization processing channel is used for mapping the generated analysis data to the normalization data format based on the mapping relation between the data format of the network data received by the associated equipment access interface and the normalization data format to obtain a normalization data set.
Furthermore, the intelligent access system also comprises an access control component which is used for respectively controlling the starting and the stopping of the equipment access interface, the data analysis channel and the normalization processing channel which are associated with each channel.
Further, the data analysis component also comprises a data analysis method storage module which is used for storing the data analysis method corresponding to each equipment type;
the control component distributes a data analysis method corresponding to the type of equipment to each data analysis channel according to the equipment type of the network data received by each data analysis channel in the data analysis component;
and each data analysis channel analyzes the network data received by the data analysis channel based on the distributed data analysis method to obtain corresponding analysis data.
Further, the data analysis method corresponding to each device type is determined by the following method:
for each network device, obtaining a network data sample of the network device;
the encoding format of the network data samples is read,
if the coding analytic method corresponding to the coding format exists, analyzing the data content in the network data sample by using the coding analytic method to obtain sample analytic data corresponding to the coding analytic method;
the data field format of the network data sample is read,
if the data field format contains an expression, analyzing the data content in the network data sample by using a regular analysis method to obtain sample analysis data corresponding to the regular analysis method;
if the data field format only contains symbols, analyzing the data content in the network data sample by using a symbol analysis method to obtain sample analysis data corresponding to the symbol analysis method;
and comparing the analysis accuracy rates of the sample analysis data obtained by the analysis methods, and selecting the data analysis method with the highest analysis accuracy rate as the data analysis method corresponding to the network data of the network equipment of the type.
Further, the encoding analysis method includes: CEF parsing, XML parsing and JSON parsing;
the regular analytic method comprises the following steps: regular expression analysis, and hook regular analysis;
the symbol resolution method includes: delimiter parsing, key-value pair parsing.
Further, in the data normalization processing component, the mapping relationship is established by:
whether the data format of the network data is consistent with the data field format in the normalized data format,
if the data fields are consistent, directly establishing the mapping relation according to the corresponding relation of the data field formats of the two data fields;
otherwise, processing of adding fields, deleting fields, cutting fields, combining fields or renaming fields is carried out on inconsistent data field formats in the network data, and therefore a one-to-one mapping relation between the data formats of the network data and the normalized data formats is established.
Further, in the data normalization processing component, mapping the generated parsed data into a normalized data format by:
processing analytic data corresponding to the network data according to a processing mode of each data field format in the network data when the mapping relation is established, and mapping the processed analytic data to a normalized data format to obtain a normalized data set;
if the processing of adding the data field format is executed on the inconsistent data field format in the network data, the data corresponding to the added data field format also needs to be supplemented, and the data is mapped to the normalized data format, so that a normalized data set is obtained.
Compared with the prior art, the invention can realize at least one of the following beneficial effects:
the intelligent access system of the multi-element network equipment can design a corresponding data analysis method and a mapping relation for the received network data of different network equipment so as to complete the analysis of original network data and the mapping process to a normalized data set;
the system also independently controls the starting and stopping of the equipment access interface, the data analysis channel and the normalization processing channel which are associated with each channel by arranging the control assembly, so that the network equipment for subsequent analysis can be conveniently selected, and for the network equipment which is not analyzed temporarily, the purposes of reducing energy consumption and saving cost are achieved by stopping the corresponding equipment access interface, the data analysis channel and the normalization processing channel.
In conclusion, the intelligent access system for the multi-element network equipment breaks through key technologies such as a multi-source data acquisition technology and a data preprocessing technology, and realizes intelligent access of the multi-element network equipment; the method provides perfect multivariate data acquisition and processing, and can well realize real-time comprehensive processing of mass data.
Additional features and advantages of the invention will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. The objectives and other advantages of the invention will be realized and attained by the structure particularly pointed out in the written description and drawings.
Drawings
The drawings are only for purposes of illustrating particular embodiments and are not to be construed as limiting the invention, wherein like reference numerals are used to designate like parts throughout.
Fig. 1 is a schematic structural diagram of an intelligent access system based on multiple network devices according to an embodiment of the present invention.
Detailed Description
The accompanying drawings, which are incorporated in and constitute a part of this application, illustrate preferred embodiments of the invention and together with the description, serve to explain the principles of the invention and not to limit the scope of the invention.
The specific embodiment of the invention discloses an intelligent access system of multi-element network equipment, the structural schematic diagram is shown in figure 1, the intelligent access system comprises:
the device access interface component is used for receiving the network data of each network device; the network data comprises equipment type, data format, coding format and data content;
the data analysis component is connected with the equipment access interface component and used for analyzing the network data from the equipment access interface component to obtain corresponding analysis data;
and the data normalization processing component is connected with the data analysis component and is used for mapping the generated analysis data to the normalized data format based on the mapping relation between the data format and the normalized data format of the network data of each network device to obtain a normalized data set.
In this embodiment, each network device uniquely corresponds to one device type; illustratively, the network device in the present embodiment includes a mail system, a host system, a load balancer, a switch, a firewall, a baster, middleware, a network printer, and the like.
Preferably, the items in the network data are described as follows:
a device type for describing a device type of the network device; the device type is the same as the device name of the network device; such as mail systems, host systems, load balancers, switches, firewalls, basters, middleware, network printers, and the like.
A data format for describing a plurality of data field formats in a log file that match the device type;
data content for describing log file information of the network data; it should be noted that the data format and the data content in the network data of each network device are different according to different network devices, and the data format and the data content in the network data of each network device are obtained by using the existing acquisition method, and the data content is not limited in this embodiment.
Encoding formats, encoding formats for describing data content, such as UTF8, CEF, XML, JSOM, etc.
Preferably, the device access interface component comprises a plurality of device access interfaces, each device access interface having a unique interface ID; each equipment access interface is used for connecting network equipment which is suitable for the type of the equipment access interface and receiving network data of the network equipment; at this time, the network data further includes an interface ID of the device access interface, which is used to describe data source information of the network data.
Preferably, the data parsing component is composed of a plurality of data parsing channels; each equipment access interface is uniquely associated with a data analysis channel;
each data analysis channel is used for analyzing the network data received by the associated equipment access interface to obtain corresponding analysis data.
Preferably, the data normalization processing component is composed of a plurality of normalization processing channels; each normalization processing channel is uniquely associated with one and equipment access interface;
each normalization processing channel is used for mapping the generated analysis data to the normalization data format based on the mapping relation between the data format of the network data received by the associated equipment access interface and the normalization data format to obtain a normalization data set.
Preferably, the intelligent access system further includes an access control component, which is used for respectively controlling the activation and deactivation of the device access interface, the data analysis channel and the normalization processing channel associated with each channel. The network equipment which is brought into subsequent analysis is conveniently selected by independently controlling the starting and stopping of the equipment access interface, the data analysis channel and the normalization processing channel which are associated with each path, and for the network equipment which is not analyzed for the moment, the purposes of reducing energy consumption and saving cost are achieved by stopping the corresponding equipment access interface, the data analysis channel and the normalization processing channel.
At this time, the data analysis component further comprises a data analysis method storage module, which is used for storing the data analysis method corresponding to each equipment type; the control component distributes a data analysis method corresponding to the type of equipment to each data analysis channel according to the equipment type of the network data received by each data analysis channel in the data analysis component; and each data analysis channel analyzes the network data received by the data analysis channel based on the distributed data analysis method to obtain corresponding analysis data.
It should be noted that the data analysis method stored in the data analysis method storage module can be determined in an off-line manner and then stored in the data analysis method storage module; preferably, the data analysis method corresponding to each device type is determined by the following method:
for each network device, acquiring a network data sample of the network device in advance; the network data sample is historical data of the network equipment and can be obtained by inquiring the historical data output by the network equipment;
the encoding format of the network data samples is read,
if the coding analytic method corresponding to the coding format exists, analyzing the data content in the network data sample by using the coding analytic method to obtain sample analytic data corresponding to the coding analytic method;
the data field format of the network data sample is read,
if the data field format contains an expression, analyzing the data content in the network data sample by using a regular analysis method to obtain sample analysis data corresponding to the regular analysis method;
if the data field format only contains symbols, analyzing the data content in the network data sample by using a symbol analysis method to obtain sample analysis data corresponding to the symbol analysis method;
and comparing the analysis accuracy rates of the sample analysis data obtained by the analysis methods, and selecting the data analysis method with the highest analysis accuracy rate as the data analysis method corresponding to the network data of the network equipment of the type.
If the maximum value of the analysis accuracy rate is lower than the threshold value of the analysis accuracy rate, generating a self-defined analysis method according to the coding mode and the data format of the network data of the current network equipment; and determining the generated custom analytic method as a data analytic method corresponding to the network data of the network equipment of the kind. The resolution accuracy threshold is adaptively set according to the performance requirements of subsequent network security analysis, and generally exceeds 70%.
The data analysis method involved in the embodiment is specifically divided into:
an encoding analysis method comprising:
CEF analysis method: analyzing the CEF data content according to the data in the CEF format;
XML analysis method: analyzing the data content with XML format;
JSON analysis method: analyzing data with JSON format;
the regular analytic method comprises the following steps:
regular expression analytic method: the method is suitable for the complicated logs which cannot be analyzed in other forms, and the complicated logs are analyzed in a regular expression mode;
a hook canonical analysis method: the method is suitable for complicated logs which cannot be analyzed in other forms, and the logs are analyzed by adopting a GROK expression;
a symbol resolution method comprising:
separator analysis: when the data format is that the content in each piece of data is separated by a separator, the separator pattern is filled in, the program identifies each separator, the content of each piece of data is extracted, for example, in the case of separation, the program identifies ",", and then extracts the data content between the two pieces of data;
key-value pair analysis: separating each piece of data by a field separator and a key value separator, filling the styles of the field separator and the key value separator, and identifying key value pair data by a program according to the two separators;
and the number of the first and second groups,
and the user-defined analysis method is generated according to the coding mode and the data format of the network data of the network equipment.
The custom analytic method may be generated based on the following process:
segmenting the data content of the network data sample according to the encoding mode and the data format of the network data of the current network equipment to obtain the data field content of each data field format corresponding to the encoding mode;
selecting a corresponding parsing sub-rule adapted to the data field content of each data field format; the process is similar to a data analysis method corresponding to each equipment type; in particular, the amount of the solvent to be used,
the data field format of the contents of the data field is read,
if the data field format contains an expression, analyzing the content of the data field by using a regular analysis method to obtain data field content analysis data corresponding to the regular analysis method;
if the data field format only contains symbols, the data field content is analyzed by a symbol analysis method to obtain data field content analysis data corresponding to the symbol analysis method;
and comparing the analysis accuracy of the data field content analysis data obtained by the analysis method, and selecting the data analysis method with the highest analysis accuracy as the data analysis method corresponding to the data field content analysis data.
Mapping the incidence relation among all data field formats in the data format into the incidence relation among corresponding analysis sub-rules;
and integrating all the analysis sub-rules and the association relation thereof to generate a custom analysis method.
Preferably, in the data parsing component, the parsed data is obtained by performing the following operations:
after receiving the network data, reading the equipment type in the network data, and determining a data analysis method corresponding to the equipment type;
and analyzing the corresponding network data based on the determined data analysis method to obtain corresponding analysis data.
Preferably, in the data normalization processing component, the mapping relationship is established by:
whether the data format of the network data is consistent with the data field format in the normalized data format,
if the data fields are consistent, directly establishing the mapping relation according to the corresponding relation of the data field formats of the two data fields;
otherwise, processing of adding fields, deleting fields, cutting fields, combining fields or renaming fields is carried out on inconsistent data field formats in the network data, and therefore a one-to-one mapping relation between the data formats of the network data and the normalized data formats is established.
Wherein the content of the first and second substances,
adding fields, which refer to data field formats that are not added in the data formats of the network data and appear in the data format of the normalized data format data field formats; thus, the corresponding field can be automatically added in the parsed data each time.
Deleting fields, namely deleting data field formats which are not the data field formats of the normalized data formats from the data formats of the network data;
cutting fields, namely cutting corresponding data field formats in the data formats of the network data according to the plurality of data field formats in the normalized data formats if one data field format in the data formats of the network data corresponds to the plurality of data field formats in the normalized data formats;
merging fields, namely merging corresponding data field formats in the data formats of the network data according to one data field format in the normalized data formats if a plurality of data field formats in the data formats of the network data correspond to one data field format in the normalized data formats;
and renaming the field, namely adjusting the data field format in the data format of the network data into the corresponding data field format in the same normalized data format if the data field format in the data format of the network data has the same meaning and the name as the data field format in the normalized data format.
In the data normalization processing component, mapping the generated parsed data into a normalized data format by:
processing analytic data corresponding to the network data according to a processing mode of each data field format in the network data when the mapping relation is established, and mapping the processed analytic data to a normalized data format to obtain a normalized data set;
if the processing of adding the data field format is executed on the inconsistent data field format in the network data, the data corresponding to the added data field format also needs to be supplemented, and the data is mapped to the normalized data format, so that a normalized data set is obtained.
In addition, in order to improve the accuracy of the analysis data, the intelligent access system in this embodiment may further include an analysis data verification component, which is disposed between the data analysis component and the data normalization processing component, and configured to verify the generated analysis data and output the analysis data that passes the verification to the data normalization processing component;
the analysis data checking component can also be composed of a plurality of data checking channels, each data checking channel is associated with one data analysis channel and one normalization processing channel so as to check the analysis data output by the associated data analysis channel and output the analysis data passing the check to the associated normalization processing channel. If the verification fails, the verification failing signal can be fed back to the associated data analysis channel so that the data analysis channel can analyze the data content again, and the re-analysis can be realized based on the original data analysis method or can be realized by other data analysis methods instead.
Illustratively, the parsed data is verified by performing the following operations:
matching and checking the analyzed data and the corresponding data content before analysis, and if the matching is successful, the checking is passed; otherwise, the check fails.
Illustratively, the matching check here refers to: and (4) carrying out consistency judgment on the data content before analysis and the analyzed data after analysis corresponding to each data field format, if the data content before analysis and the analyzed data after analysis are consistent, matching is successful, and the verification is passed.
It should be noted that the specification of the normalized data format is directly related to the purpose of the normalized data set, and this embodiment does not specifically limit this.
Illustratively, the normalized data format may be: resolution rule ID, event summary, event level, application protocol, transport protocol, initial time, source address, source port, destination address, destination port, device address, occurrence time, receive time, send traffic, receive traffic, duration, primitive log, vendor, send packet count, receive packet count, resolution rule name, quintuple, stream ID, end time, data source, bituple, unit, event content, protocol, total traffic, total packet count, alarm, event classification, event name. Accordingly, a specific form of the normalized data set is exemplified by:
{ "resolution rule ID":72190fa7-2a8d-457d-8d83-4985ac8c9b48, "event summary": nta _ flow, "event level": information, "application protocol": rx, "transport protocol": UDP, "initial time": 2020-12-2923:33:05.384, "source address": 10.18.68.102, "source port":2222, "destination address": 10.17.4.23, "destination port":800, "device address": 10.21.172.175, "occurrence time": 2020-12-2923:33:05.384, "reception time": 2020-12-2917:22:29.069, "transmission traffic": 122496, "reception traffic": 131648, "duration": 21090132, "raw log": time "{" estamesp ":1609255985384, @ test _ 1609277075516," sold _ 783-493 b-3527-3 b-3-493 b-3 b-33 b-3643, "flow _ id":455332236222203, "protocol": flow, "" src _ ip ":10.18.68.102," "src _ port":2222, "dst _ ip": 10.17.4.23, "" dst _ port ":800," proto ": UDP," "adjust":0, "ndpi _ app _ proto": "rx," "app _ proto": rx, "" app _ proto _ ts ": unlawn," "app _ proto _ tc": unlawn, "" flow "{" pkts _ top "704," pkts _ top "124," "pkts _ top" 704, "bytes _ top" 122496, "bytes _ top" 131648, "signature": 12-T23: 33 + 5634: 7, "" 34-end ": 3530: 26," throughput "3526" 3645, "" handle _ flag ": 26": 3645, "" handle _ flag _ 12-7 + 357, "" 3526 ": 26,",26, "resolution rule name": nta _ dispatcher, "quintuple": 10.18.68.102_2222_ flow _10.17.4.23_800, "flow ID":455332236222203, "end time": 2020-12-3005:24:35.516, "data source": nta (hansight), "duplet": 10.18.68.102_10.17.4.23, "unit": bytes, "event content": no data, "protocol": flow, "total flow": 254144, "total packet count": 1408, "alarm": false, "event classification": network access/session connection, "event name": network connection }.
In summary, the multivariate network device intelligent access system provided in this embodiment can design a corresponding data analysis method and a mapping relationship for the received network data of different network devices, so as to complete the analysis of the original network data and the mapping process to the normalized data set;
the system also independently controls the starting and stopping of the equipment access interface, the data analysis channel and the normalization processing channel which are associated with each channel by arranging the control assembly, so that the network equipment for subsequent analysis can be conveniently selected, and for the network equipment which is not analyzed temporarily, the purposes of reducing energy consumption and saving cost are achieved by stopping the corresponding equipment access interface, the data analysis channel and the normalization processing channel.
Those skilled in the art will appreciate that all or part of the flow of the method implementing the above embodiments may be implemented by a computer program, which is stored in a computer readable storage medium, to instruct related hardware. The computer readable storage medium is a magnetic disk, an optical disk, a read-only memory or a random access memory.
The above description is only for the preferred embodiment of the present invention, but the scope of the present invention is not limited thereto, and any changes or substitutions that can be easily conceived by those skilled in the art within the technical scope of the present invention are included in the scope of the present invention.
Claims (10)
1. An intelligent access system for multiple network devices, the intelligent access system comprising:
the device access interface component is used for receiving the network data of each network device; the network data comprises equipment type, data format, coding format and data content;
the data analysis component is connected with the equipment access interface component and used for analyzing the network data from the equipment access interface component to obtain corresponding analysis data;
and the data normalization processing component is connected with the data analysis component and is used for mapping the generated analysis data to the normalized data format based on the mapping relation between the data format and the normalized data format of the network data of each network device to obtain a normalized data set.
2. The multiple network device intelligent access system of claim 1,
the device access interface assembly comprises a plurality of device access interfaces, each device access interface having a unique interface ID; each equipment access interface is used for connecting network equipment which is suitable for the type of the equipment access interface and receiving network data of the network equipment; the network data further includes an interface ID of the device access interface.
3. The multiple network device intelligent access system of claim 2,
the data analysis component consists of a plurality of data analysis channels; each equipment access interface is uniquely associated with a data analysis channel;
each data analysis channel is used for analyzing the network data received by the associated equipment access interface to obtain corresponding analysis data.
4. The multiple network device intelligent access system of claim 3,
the data normalization processing component consists of a plurality of normalization processing channels; each normalization processing channel is uniquely associated with one data analysis channel and one equipment access interface;
each normalization processing channel is used for mapping the generated analysis data to the normalization data format based on the mapping relation between the data format of the network data received by the associated equipment access interface and the normalization data format to obtain a normalization data set.
5. The multiple network device intelligent access system of claim 4,
the intelligent access system also comprises an access control component which is used for respectively controlling the starting and the stopping of the equipment access interface, the data analysis channel and the normalization processing channel which are associated with each channel.
6. The multiple network device intelligent access system of claim 5,
the data analysis component also comprises a data analysis method storage module which is used for storing the data analysis method corresponding to each equipment type;
the control component distributes a data analysis method corresponding to the type of equipment to each data analysis channel according to the equipment type of the network data received by each data analysis channel in the data analysis component;
and each data analysis channel analyzes the network data received by the data analysis channel based on the distributed data analysis method to obtain corresponding analysis data.
7. The multiple network device intelligent access system of claim 6,
the data analysis method corresponding to each equipment type is determined by the following method:
for each network device, obtaining a network data sample of the network device;
the encoding format of the network data samples is read,
if the coding analytic method corresponding to the coding format exists, analyzing the data content in the network data sample by using the coding analytic method to obtain sample analytic data corresponding to the coding analytic method;
the data field format of the network data sample is read,
if the data field format contains an expression, analyzing the data content in the network data sample by using a regular analysis method to obtain sample analysis data corresponding to the regular analysis method;
if the data field format only contains symbols, analyzing the data content in the network data sample by using a symbol analysis method to obtain sample analysis data corresponding to the symbol analysis method;
and comparing the analysis accuracy rates of the sample analysis data obtained by the analysis methods, and selecting the data analysis method with the highest analysis accuracy rate as the data analysis method corresponding to the network data of the network equipment of the type.
8. The multiple network device intelligent access system of claim 7,
the encoding analysis method comprises the following steps: CEF parsing, XML parsing and JSON parsing;
the regular analytic method comprises the following steps: regular expression analysis, and hook regular analysis;
the symbol resolution method includes: delimiter parsing, key-value pair parsing.
9. The multiple network device intelligent access system of claim 1 or 4,
in the data normalization processing component, the mapping relationship is established by:
whether the data format of the network data is consistent with the data field format in the normalized data format,
if the data fields are consistent, directly establishing the mapping relation according to the corresponding relation of the data field formats of the two data fields;
otherwise, processing of adding fields, deleting fields, cutting fields, combining fields or renaming fields is carried out on inconsistent data field formats in the network data, and therefore a one-to-one mapping relation between the data formats of the network data and the normalized data formats is established.
10. The multiple network device intelligent access system of claim 9,
in the data normalization processing component, mapping the generated parsed data into a normalized data format by:
processing analytic data corresponding to the network data according to a processing mode of each data field format in the network data when the mapping relation is established, and mapping the processed analytic data to a normalized data format to obtain a normalized data set;
if the processing of adding the data field format is executed on the inconsistent data field format in the network data, the data corresponding to the added data field format also needs to be supplemented, and the data is mapped to the normalized data format, so that a normalized data set is obtained.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111094007.0A CN113810160B (en) | 2021-09-17 | 2021-09-17 | Intelligent access system of multi-element network equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111094007.0A CN113810160B (en) | 2021-09-17 | 2021-09-17 | Intelligent access system of multi-element network equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113810160A true CN113810160A (en) | 2021-12-17 |
| CN113810160B CN113810160B (en) | 2023-07-04 |
Family
ID=78939703
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111094007.0A Active CN113810160B (en) | 2021-09-17 | 2021-09-17 | Intelligent access system of multi-element network equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113810160B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116318969A (en) * | 2023-03-15 | 2023-06-23 | 中国华能集团有限公司北京招标分公司 | Multi-element equipment log access method |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7376969B1 (en) * | 2002-12-02 | 2008-05-20 | Arcsight, Inc. | Real time monitoring and analysis of events from multiple network security devices |
| US10229105B1 (en) * | 2014-09-30 | 2019-03-12 | EMC IP Holding Company LLC | Mobile log data parsing |
| US20190317835A1 (en) * | 2018-04-12 | 2019-10-17 | International Business Machines Corporation | Management of events in event management systems |
| US10782942B1 (en) * | 2019-09-13 | 2020-09-22 | Capital One Services, Llc | Rapid onboarding of data from diverse data sources into standardized objects with parser and unit test generation |
| WO2020211248A1 (en) * | 2019-04-19 | 2020-10-22 | 平安科技（深圳）有限公司 | Living body detection log parsing method and apparatus, storage medium and computer device |
| CN112882991A (en) * | 2019-11-29 | 2021-06-01 | 北京数安鑫云信息技术有限公司 | Log data normalization processing method, device, medium and computer equipment |
| CN112883088A (en) * | 2019-11-29 | 2021-06-01 | 贵州白山云科技股份有限公司 | Data processing method, device, equipment and storage medium |
| CN113268247A (en) * | 2021-06-02 | 2021-08-17 | 哈尔滨工业大学（威海） | HL7 message analysis method based on state machine |
-
2021
- 2021-09-17 CN CN202111094007.0A patent/CN113810160B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7376969B1 (en) * | 2002-12-02 | 2008-05-20 | Arcsight, Inc. | Real time monitoring and analysis of events from multiple network security devices |
| US10229105B1 (en) * | 2014-09-30 | 2019-03-12 | EMC IP Holding Company LLC | Mobile log data parsing |
| US20190317835A1 (en) * | 2018-04-12 | 2019-10-17 | International Business Machines Corporation | Management of events in event management systems |
| WO2020211248A1 (en) * | 2019-04-19 | 2020-10-22 | 平安科技（深圳）有限公司 | Living body detection log parsing method and apparatus, storage medium and computer device |
| US10782942B1 (en) * | 2019-09-13 | 2020-09-22 | Capital One Services, Llc | Rapid onboarding of data from diverse data sources into standardized objects with parser and unit test generation |
| CN112882991A (en) * | 2019-11-29 | 2021-06-01 | 北京数安鑫云信息技术有限公司 | Log data normalization processing method, device, medium and computer equipment |
| CN112883088A (en) * | 2019-11-29 | 2021-06-01 | 贵州白山云科技股份有限公司 | Data processing method, device, equipment and storage medium |
| CN113268247A (en) * | 2021-06-02 | 2021-08-17 | 哈尔滨工业大学（威海） | HL7 message analysis method based on state machine |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116318969A (en) * | 2023-03-15 | 2023-06-23 | 中国华能集团有限公司北京招标分公司 | Multi-element equipment log access method |
| CN116318969B (en) * | 2023-03-15 | 2024-01-26 | 中国华能集团有限公司北京招标分公司 | Multi-element equipment log access method |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113810160B (en) | 2023-07-04 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US6954789B2 (en) | Method and apparatus for monitoring traffic in a network | |
| US8386598B2 (en) | Network monitoring by using packet header analysis | |
| CN112714047B (en) | Industrial control protocol flow based test method, device, equipment and storage medium | |
| CN101841441B (en) | Test method and system for flow control device and data stream playback device | |
| CN113794605B (en) | Method, system and device for detecting kernel packet loss based on eBPF | |
| CN103312565A (en) | Independent learning based peer-to-peer (P2P) network flow identification method | |
| US9398117B2 (en) | Protocol data unit interface | |
| US10091073B2 (en) | Large-scale passive network monitoring using multiple tiers of ordinary network switches | |
| CN110855493A (en) | Application topological graph drawing device for mixed environment | |
| CN112565338A (en) | Method and system for capturing, filtering, storing and analyzing Ethernet message in real time | |
| CN113810160A (en) | Intelligent access system of multi-element network equipment | |
| US20200257602A1 (en) | High Order Layer Intrusion Detection Using Neural Networks | |
| CN102497297A (en) | System and method for realizing deep packet inspection technology based on multi-core and multi-thread | |
| CN113347258A (en) | Method and system for data acquisition, monitoring and analysis under cloud flow | |
| EP3101843A2 (en) | Capturing network data to provide to a data analyser | |
| CN113850069A (en) | Network security data normalization processing method based on multi-element network security equipment | |
| CN108696713A (en) | Safety detecting method, device and the test equipment of code stream | |
| CN111224891A (en) | Traffic application identification system and method based on dynamic learning triples | |
| CN104125106A (en) | Network purity detection device and method based on classified decision tree | |
| Lukashin et al. | Distributed packet trace processing method for information security analysis | |
| CN101127692B (en) | A method and device for identifying and limiting network traffic | |
| CN108650229A (en) | A kind of network application behavior parsing restoring method and system | |
| CN101902758A (en) | Protocol testing based data processing method for wireless network and device thereof | |
| CN113783825B (en) | Message flow statistics method and device | |
| US11122452B2 (en) | System and method for load balancing of network packets received from a MME with smart filtering |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |