CN116915719A

CN116915719A - Service identification method, device, electronic equipment and storage medium

Info

Publication number: CN116915719A
Application number: CN202211640827.XA
Authority: CN
Inventors: 赵雷; 黄丽思; 赵延; 梁燕萍; 余立
Original assignee: China Mobile Communications Group Co Ltd; China Mobile Communications Ltd Research Institute
Current assignee: China Mobile Communications Group Co Ltd; China Mobile Communications Ltd Research Institute
Priority date: 2022-12-19
Filing date: 2022-12-19
Publication date: 2023-10-20

Abstract

The embodiment discloses a service identification method, a device, an electronic device and a computer storage medium, wherein the method comprises the following steps: acquiring an original code stream of network data; determining service characteristic data of the original code stream by analyzing the original code stream; determining the type of the service characteristic data; determining a first identification method for carrying out service identification on the original code stream according to the type of the service characteristic data; and processing the service characteristic data by adopting the first identification method to obtain a service identification result of the original code stream.

Description

Service identification method, device, electronic equipment and storage medium

Technical Field

The application belongs to the technical field of artificial intelligence (Artificial Intelligence, AI), and particularly relates to a service identification method, a service identification device, electronic equipment and a computer storage medium.

Background

In the related art, one popular network traffic service identification scheme is: service identification of the network traffic is carried out in a manual rule checking mode, namely, judging conditions are set based on experience, so that the service type of the network traffic is judged; the scheme for carrying out service identification of network traffic by means of manual rule verification has the problem of low identification accuracy; in addition, the simple and lightweight algorithm model can be used for simply classifying and identifying the data service, however, in the related art, only a single model is adopted for identifying the data service, the method is not suitable for identifying different types of data service, and in the scene of needing to expand the new data service, the model needs to be replaced in a large scale, so that the expansibility is low.

Disclosure of Invention

The embodiment of the application provides a service identification method, a service identification device, electronic equipment and a computer storage medium.

The embodiment of the application provides a service identification method, which comprises the following steps:

acquiring an original code stream of network data;

determining service characteristic data of the original code stream by analyzing the original code stream; determining the type of the service characteristic data;

determining a first identification method for carrying out service identification on the original code stream according to the type of the service characteristic data;

and processing the service characteristic data by adopting the first identification method to obtain a service identification result of the original code stream.

In some embodiments, the determining the service characteristic data of the original code stream by analyzing the original code stream includes: determining the protocol type of the original code stream; and determining service characteristic data of the original code stream according to the protocol type of the original code stream.

It can be seen that, according to the embodiment of the application, the service characteristic data of the original code stream can be determined more accurately according to the protocol type of the original code stream, so that the first identification method for identifying the service of the original code stream can be determined more accurately according to the type of the service characteristic data of the original code stream, and the accuracy of identifying the service of the original code stream can be improved.

In some embodiments, the determining the service characteristic data of the original code stream according to the protocol type of the original code stream includes: in the case that the protocol type of the original code stream is hypertext transfer protocol (Hyper Text Transfer Protocol, HTTP), determining code stream information and a key plaintext field of the original code stream as service feature data of the original code stream, wherein the key plaintext field is a field for representing the service type of the original code stream by using plaintext; under the condition that the protocol type of the original code stream is HTTPS protocol or newly added supportable protocol, at least determining a key plaintext field of the original code stream as service characteristic data of the original code stream; and determining a target field of the original code stream as service characteristic data of the original code stream under the condition that the protocol type of the original code stream is other protocols, wherein the other protocols represent network communication protocols except the HTTP, HTTPS and the newly added supportable protocol, and the target field is a predefined field.

It can be seen that the embodiment of the application can accurately determine the service characteristic data in the original code stream according to the protocol type of the original code stream, and is favorable for accurately determining the first identification method for identifying the service of the original code stream, thereby improving the accuracy of identifying the service of the original code stream.

In some embodiments, the determining the service characteristic data of the original code stream according to the protocol type of the original code stream further includes: and under the condition that the protocol type of the original code stream is HTTPS protocol, determining the content in the original code stream, from which the plaintext information cannot be acquired, as the service characteristic data of the original code stream.

It can be seen that, for the HTTPS protocol, since the content in the original code stream, from which the plaintext information cannot be obtained, can be determined as the service feature data of the original code stream, it is beneficial to more accurately determine the method for identifying the service of the original code stream, so that the accuracy of identifying the service of the original code stream can be improved.

In some embodiments, the determining a first identifying method for identifying the service of the original code stream according to the type of the service feature data includes: in the case that the service feature data is a key plaintext field, determining that the first recognition method includes a method of service recognition using a natural language processing (Natural Language Processing, NLP) model; the key plaintext field is a field for representing the service type of the original code stream by plaintext; in the case that the service feature data is the code stream information of the original code stream, determining the first identification method includes a method of service identification using a convolutional neural network (Convolutional Neural Networks, CNN) model; or determining that the first identification method comprises a method for carrying out service identification by using a CNN model under the condition that the protocol type of the original code stream is HTTPS protocol and the service characteristic data is content in which plaintext information cannot be acquired in the original code stream; judging whether the target field contains the key plaintext field or not under the condition that the service characteristic data is the target field, and obtaining a judging result; determining the first identification method according to the judgment result; the target field is a predetermined field.

It can be seen that the embodiment of the application can more accurately determine the first identification method for identifying the service of the original code stream according to the type of the service characteristic data, thereby being beneficial to improving the accuracy of identifying the service of the original code stream.

In some embodiments, the determining the first identification method according to the determination result includes: determining that the first identification method comprises a method for carrying out service identification by using an NLP model under the condition that the target field contains the key plaintext field; and under the condition that the judging result is that the target field does not contain the key plaintext field, determining the first identification method comprises a method for carrying out service identification by using a preset manual rule.

It can be seen that the embodiment of the application can flexibly determine the method for identifying the service of the original code stream according to the content in the target field, thereby being beneficial to identifying the service of different types of data services.

In some embodiments, the processing the service feature data by using the first identification method to obtain a service identification result of the original code stream includes: in the case that the first recognition method includes a method for performing service recognition by using the NLP model, or the first recognition method includes a method for performing service recognition by using the CNN model, processing the service feature data by using the first recognition method to obtain a preliminary service recognition result and a model prediction probability of the original code stream, where the model prediction probability represents accuracy of obtaining the preliminary service recognition result by using the NLP model or the CNN model; under the condition that the model prediction probability is greater than or equal to a probability threshold value, taking a preliminary service identification result of the original code stream as the service identification result; and under the condition that the model prediction probability is smaller than a probability threshold, the first recognition method is redetermined into a method for carrying out service recognition by using a preset manual rule, and the redetermined first recognition method is adopted to process the service feature data, so that a service recognition result of the original code stream is obtained.

It can be seen that, in the embodiment of the application, under the condition that the model prediction probability is greater than or equal to the probability threshold, a preliminary service identification result obtained by using an NLP model or a CNN model is reserved; under the condition that the model prediction probability is smaller than the probability threshold value, the accuracy of the preliminary service identification result is lower, so that the service identification is performed by using the artificial rule instead, and the accuracy of the service identification is improved.

The embodiment of the application also provides a service identification device, which comprises:

the acquisition module is used for acquiring an original code stream of the network data;

the first processing module is used for determining service characteristic data of the original code stream by analyzing the original code stream; determining the type of the service characteristic data;

the second processing module is used for determining a first identification method for carrying out service identification on the original code stream according to the type of the service characteristic data;

a third processing module, configured to process the service feature data by using the first identification method to obtain a service identification result of the original code stream

The embodiment of the application also provides electronic equipment, which comprises a processor and a memory for storing a computer program capable of running on the processor; wherein the processor is configured to run the computer program to perform any one of the service identification methods described above.

The embodiment of the application also provides a computer storage medium, on which a computer program is stored, which when being executed by a processor, implements any one of the service identification methods.

It can be seen that the embodiment of the application can determine the type of the service characteristic data of the original code stream by analyzing the original code stream, so that the first identification method for carrying out service identification on the original code stream can be accurately determined according to the type of the service characteristic data, thereby being beneficial to improving the accuracy of carrying out service identification on the original code stream; in addition, because the first identification method is determined according to the type of the service characteristic data, the embodiment of the application adopts different service identification methods for network data of different service types, and compared with the scheme of adopting a single model to carry out service identification on the data service in the related technology, the method has wider application range, does not need to carry out large-scale replacement for an algorithm model for a scene needing to carry out expansion on the new network data service, can directly accurately determine the service identification method for the new network data service, and has higher expansibility.

Drawings

FIG. 1 is a flow chart of a business identification method according to an embodiment of the application;

FIG. 2 is a flow chart of service identification using an end-to-end system in an embodiment of the application;

fig. 3 is a schematic structural diagram of a service identifying device according to an embodiment of the present application;

fig. 4 is a schematic structural diagram of an electronic device according to an embodiment of the present application.

Detailed Description

In the related art, a service identification scheme for network traffic is either to perform service identification for network traffic by means of manual rule checking or to simply classify and identify data services by using a simple lightweight algorithm model, and the service identification scheme for network traffic in the related art is exemplarily described below by means of schemes 1 to 4.

Scheme 1 provides a traffic identification method and related equipment, which can divide the service basis field in the network traffic. The main implementation steps of the scheme 1 comprise: and judging whether the network traffic is asymmetric traffic, namely judging whether corresponding deep packet inspection (Deep Packet Inspection DPI, DPI) equipment adopts HTTP, and if the DPI equipment adopts HTTP, transmitting interactive information of the traffic by the DPI equipment, wherein the interactive information can be used for carrying out service identification.

Scheme 2 provides a DPI rule generation method and device; the main implementation steps of the scheme comprise: acquiring internet data, and identifying the internet data based on DPI rules of a DPI rule base; when unidentified data exists in the internet data, analyzing the unidentified data to acquire first characteristics of the unidentified data; compiling and generating DPI rules based on the first characteristics; storing the DPI rules to the DPI rule base.

Scheme 3 provides a network function arrangement method based on service identification in a mobile edge computing platform, wherein the data set in scheme 3 is the service flow which is grabbed according to DPI, and the service flow is utilized and combined with an algorithm to carry out service identification. The main implementation steps of the scheme 3 include: grasping user service request data streams with known service types, and generating a training set, a testing set and a verification set; constructing a Back Propagation (BP) neural network structure, and training the BP neural network; and arranging the mobile edge computing network function according to the service type result of the user service request obtained through training, and realizing the service.

Scheme 4 provides a service identification method, a device, a terminal device and a storage medium. The main implementation steps of the scheme 4 include: acquiring service data to be identified, and extracting identification keywords of corresponding identification elements in the service data to be identified according to preset identification elements; performing matrixing treatment on the identification keywords to obtain the business data matrix to be identified; a K-Nearest Neighbor (KNN) algorithm is adopted to establish a service identification model, and the service to which the service data to be identified belongs is judged; illustratively, a manhattan distance of the traffic data matrix to be identified and each sample data matrix in a pre-established sample database may be calculated; k pieces of sample data with the minimum Manhattan distance are taken, and the service with the highest frequency in the K pieces of sample data is the service to which the service data to be identified belongs.

The scheme 1 and the scheme 2 have the problems of low network resource utilization rate and low expandability. Both schemes only use a single model to analyze specific data fields in network traffic, resulting in excessive data being discarded, resulting in wasted resources. And when the model needs to be expanded aiming at new data service, the model needs to be replaced in a large scale as a whole, and the expansion performance of the original interface is low.

The scheme 3 and the scheme 4 have the problem of lack of pertinence in recognition, and the partial data features of different network flows are large in difference, so that the method is not suitable for carrying out service recognition on different types of data services by adopting the same model. Further, the problem of low service identification performance precision exists in the scheme 3 and the scheme 4, and the performance of the model is greatly influenced based on the current network data magnitude.

Aiming at the technical problems, the technical scheme of the embodiment of the application is provided.

Embodiments of the present application will be described in further detail below with reference to the accompanying drawings and examples. It should be understood that the examples provided herein are merely illustrative of the present application and are not intended to limit the present application. In addition, the embodiments provided below are some of the embodiments for carrying out the present application, but not all of the embodiments for carrying out the present application, and the technical solutions described in the embodiments of the present application may be implemented in any combination without conflict.

It should be noted that, in the embodiments of the present application, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a method or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such method or apparatus. Without further limitation, an element defined by the phrase "comprising one does not exclude the presence of other related elements in a method or apparatus comprising the element (e.g., a step in a method or an element in an apparatus, e.g., an element may be part of a circuit, part of a processor, part of a program or software, etc.).

For example, the service identifying method provided in the embodiment of the present application includes a series of steps, but the service identifying method provided in the embodiment of the present application is not limited to the described steps, and similarly, the service identifying device provided in the embodiment of the present application includes a series of modules, but the device provided in the embodiment of the present application is not limited to the explicitly described modules, and may also include modules that are required to be set when acquiring related information or performing processing based on the information.

The embodiment of the application can apply the deep learning algorithm to the network traffic identification service and can analyze the network data by combining the semantic field information contained in each layer of the network protocol. The embodiment of the application can be realized based on DPI technology, DPI equipment can monitor the flow sending and receiving from a network application layer, and can realize the analysis of network data by analyzing the related network flow data packet and converting the network flow data packet into a binary original code stream.

In the embodiment of the application, the data packet captured by using the DPI technology is stored in the following form: packet header #1, packet information #1, packet header #2, packet information #2, …, packet header #n, packet content #n. Each packet header contains a number of objective parameters such as protocol, host port, transmit-receive address, user information, device information, etc. The data packet information comprises data transmitted by one session, and the network can save the possible operation behavior record of the user and store a plurality of data packet contents after encryption. The packet header and packet information constitute the smallest unit in the network transmission. The payload (payload) data in the embodiments of the present application may be in the form of the packet storage described above, so as to facilitate subsequent analysis.

The embodiment of the application can realize the service identification of network data based on a series of models with mature and excellent performances in the AI field. These models are mainly classified into CNN models and NLP models, etc. In the embodiment of the application, the CNNs part can use mainstream algorithm frameworks, such as network structures of residual error network (ResNet), acceptance v3, VGG-16 and the like. These algorithms are all based on a common infrastructure of input layer-hidden layer-output layer. The NLP part refers to main stream frames such as Self-Attention, fasttext, textCNN, and the like, and the sentences are formed by extracting field words in the network flow, so that semantic information among the sentences is analyzed, and the categories to which the sentences belong are classified.

Fig. 1 is a flowchart of a service identification method according to an embodiment of the present application, as shown in fig. 1, the flowchart may include:

step 101: and acquiring an original code stream of the network data.

Here, the network data may be network traffic data of any one service; in practical application, the DPI device may monitor the original code stream of the network data from the network application layer, where the original code stream grabbed by the DPI device may be stored in a data packet storage form, and the data packet may include a data packet header and data packet information.

Step 102: determining service characteristic data of the original code stream by analyzing the original code stream; the type of the service characteristic data is determined.

Step 103: and determining a first identification method for carrying out service identification on the original code stream according to the type of the service characteristic data.

In the embodiment of the application, for two different original code streams, under the condition that the types of service features are different, the first identification methods adopted for the original code streams can be different. The first recognition method may be a method of performing service recognition using an algorithm model or a method of performing service recognition using a pre-established manual rule, for example; the method for performing service identification using the algorithm model may be a method for performing service identification using a CNN model or a method for performing service identification using an NLP model.

Step 104: and processing the service characteristic data by adopting a first identification method to obtain a service identification result of the original code stream.

Here, the service identification result of the original code stream is used to represent the service type corresponding to the original code stream.

In practical applications, steps 101 to 104 may be implemented based on a processor of an electronic device, where the processor may be at least one of an application specific integrated circuit (Application Specific Integrated Circuit, ASIC), a digital signal processor (Digital Signal Processor, DSP), a digital signal processing device (Digital Signal Processing Device, DSPD), a programmable logic device (Programmable Logic Device, PLD), a field programmable gate array (Field Programmable Gate Array, FPGA), a central processing unit (Central Processing Unit, CPU), a controller, a microcontroller, and a microprocessor.

In some embodiments of the present application, the process of determining the service feature data of the original code stream by analyzing the original code stream may include: determining the protocol type of an original code stream; and determining service characteristic data of the original code stream according to the protocol type of the original code stream.

In the embodiment of the present application, the protocol type of the original code stream may be HTTP or HTTPs, or may be a newly added supportable protocol, for example, the newly added supportable protocol may be a transmission control protocol (Transmission Control Protocol, TCP), a user datagram protocol (User Datagram Protocol, UDP), or the like.

In some embodiments of the present application, the process of determining the service characteristic data of the original code stream according to the protocol type of the original code stream may include:

under the condition that the protocol type of the original code stream is HTTP, determining code stream information and a key plaintext field of the original code stream as service characteristic data of the original code stream, wherein the key plaintext field is a field for representing the service type of the original code stream by using plaintext;

under the condition that the protocol type of the original code stream is HTTPS protocol or newly added supportable protocol, at least determining a key plaintext field of the original code stream as service characteristic data of the original code stream;

And determining the target field of the original code stream as service characteristic data of the original code stream under the condition that the protocol type of the original code stream is other protocols, wherein the other protocols represent network communication protocols except the HTTP protocol, the HTTPS protocol and the newly added supportable protocol, and the target field is a predefined field.

In some embodiments, the code stream information and the key plaintext field of the original code stream may be determined as the traffic feature data of the original code stream for the existing protocols that are supported, such as HTTP; illustratively, the code stream information of the original code stream may be a feature field, where the feature field is a translated 16-ary code stream; the key plaintext field may be understood as a field containing semantic information strongly associated with the current service, such as a host (host) name, source internet protocol (Internet Protocol, IP) information, sink (dest) IP information, etc.

When training a model for carrying out service identification on a code stream, a feature field and a key plaintext field can be used as training data of the model aiming at the supported existing protocol, wherein the feature field can be used as one-dimensional data to be sent into a one-dimensional convolution network for training.

In some embodiments, for HTTPS protocol, since the information in the original code stream is encrypted by the transport layer security protocol (Transport Layer Security, TLS), only the translatable field information such as hostname can be obtained; these translatable field information are key plaintext fields.

In some embodiments, at least the critical plaintext field of the original code stream may be determined as the traffic characteristic data of the original code stream for the newly added supportable protocols such as TCP, UDP, etc.

In some embodiments, for other protocols that are currently unrecognizable and inextensible, the service identification may be performed according to the target field, where the method for performing the service identification according to the target field may include at least one of the following: a method for identifying business by using NLP model and a method for identifying business by using a pre-established manual rule.

In some embodiments of the present application, the process of determining the service characteristic data of the original code stream according to the protocol type of the original code stream further includes: and under the condition that the protocol type of the original code stream is HTTPS protocol, determining the content in the original code stream, from which the plaintext information cannot be acquired, as the service characteristic data of the original code stream.

In some embodiments of the present application, determining a flow of a first identifying method for identifying a service of an original code stream according to a type of service feature data may include:

in the case that the service feature data is a key plaintext field, determining the first recognition method includes a method of performing service recognition using an NLP model.

Under the condition that the service characteristic data is the code stream information of the original code stream, determining a first identification method comprises a method for carrying out service identification by using a CNN model; or under the condition that the protocol type of the original code stream is HTTPS protocol and the service characteristic data is the content of which the plaintext information cannot be acquired in the original code stream, determining a first identification method comprises a method for carrying out service identification by using a CNN model;

Judging whether the target field contains a key plaintext field under the condition that the service characteristic data is the target field, and obtaining a judging result; and determining a first identification method according to the judgment result.

In some embodiments, for a supported existing protocol such as HTTP, since the service feature data includes the code stream information of the original code stream and the key plaintext field, the first recognition method includes a method of performing service recognition using the NLP model and a method of performing service recognition using the CNN model.

In some embodiments, for HTTPS protocol, the translatable field information may be input to the NLP model, and the translatable field information is processed by the NLP model to obtain a service identification result; the content of the original code stream, which cannot acquire the plaintext information, can also be input into the CNN model, and the CNN model is utilized to process the content of the original code stream, which cannot acquire the plaintext information, so as to obtain a service identification result.

In some embodiments, for newly added supportable protocols, such as TCP, UDP, etc., the key plaintext field may be input to the NLP model, and the NLP model is used to process the key plaintext field, so as to obtain the service identification result.

In some embodiments of the present application, in a case that the target field contains the key plaintext field as a result of the determination, determining the first recognition method includes a method for performing service recognition using an NLP model; and under the condition that the target field does not contain the key plaintext field as a judgment result, determining a first identification method comprises a method for carrying out service identification by using a pre-established manual rule.

Here, for other protocols that cannot be identified and cannot be expanded at present, if the target field includes a key plaintext field, the key plaintext field in the target field may be processed by using the NLP model to obtain a service identification result; if the target field does not contain the key plaintext field, service identification can be performed by using a preset manual rule, for example, the target field can be determined according to a preset manual rule table, so that a service identification result is obtained.

In some embodiments of the present application, a process for processing service feature data by using a first identification method to obtain a service identification result of an original code stream may include:

In the case that the first recognition method comprises a method for performing service recognition by using the NLP model, or the first recognition method comprises a method for performing service recognition by using the CNN model, processing service characteristic data by adopting the first recognition method to obtain a preliminary service recognition result and model prediction probability of an original code stream, wherein the model prediction probability represents the accuracy of obtaining the preliminary service recognition result by using the NLP model or the CNN model;

under the condition that the model prediction probability is greater than or equal to a probability threshold value, taking a preliminary service identification result of the original code stream as a service identification result; and under the condition that the model prediction probability is smaller than the probability threshold value, the first recognition method is redetermined into a method for carrying out service recognition by using a pre-established manual rule, and the redetermined first recognition method is adopted to process the service feature data, so that a service recognition result of the original code stream is obtained.

Here, the probability threshold is a preset model threshold parameter, and the probability threshold is between 0 and 1.

According to the embodiment of the application, the data set manufactured from the original code stream can be divided according to the protocol according to the data packet which is captured by DPI equipment and represents the original code stream, and the corresponding service identification method is determined according to different protocols; for different fields of different protocols, different AI models for carrying out service identification are determined, so that training and application of the models are carried out.

In a network traffic service identification scenario, for the case that a protocol in current network traffic is complex and a service scenario is complex, the embodiment of the present application may determine service feature data for service identification according to a determination condition, so as to determine an AI model for service identification, where the determination condition may be at least one of the following: the original code stream comprises a key plaintext field, the original code stream comprises code stream information, and the original code stream comprises a target field; the AI model may be an NLP model or a CNN model. The embodiment of the application can identify the service by adding parallel algorithm middleware and using two different types of algorithm frameworks, can identify the service aiming at various types of data in the original code stream, widens the data coverage rate, and can deeply mine more effective characteristic information in the data. Here, the parallel algorithm middleware may be a plurality of AI models for implementing service recognition in parallel, and the two different types of algorithm frameworks may be an algorithm framework of an NLP model and an algorithm framework of a CNN model.

Because the network data has the characteristics of various data characteristics, wide data sources, manual rule priori and the like, the embodiment of the application can realize service identification in a multi-layer multi-azimuth judgment mode.

The embodiment of the application can realize the service identification of the original code stream based on an end-to-end (end-to-end) system, and fig. 2 is a flow chart of service identification by using the end-to-end system in the embodiment of the application, and referring to fig. 2, the data preprocessing operation is performed on the original code stream, so as to obtain a data set manufactured from the original code stream; and then classifying the protocol of the original code stream according to the protocol field aiming at the data set to obtain the protocol type of the original code stream.

Under the condition that the protocol type of the original code stream is the existing supported protocols such as HTTP, HTTPS and the like, the extraction of the key plaintext field can be carried out aiming at the original code stream; for example, in case that the protocol type of the original code stream is an existing protocol that has been supported by HTTP, HTTPs, or the like, the code stream information may be extracted for the original code stream.

Under the condition that the protocol type of the original code stream is a newly added supportable protocol, aiming at the newly added supportable protocol, the end-to-end system can realize the expansibility of a system interface, and the key plaintext field determined according to the newly added field requirement is determined by providing the interface of the newly added field requirement, namely, after the newly added field requirement is determined, the key plaintext field extraction step is executed. It can be seen that, by the expansibility of the system interface, the embodiment of the application can ensure that the operation of other substructures in the end-to-end system is not affected under the condition of newly increasing supportable protocols.

After determining an AI model adapted to the original code stream according to the protocol type of the original code stream, processing service characteristic data in the original code stream by using the AI model to obtain a preliminary service identification result and model prediction probability of the original code stream; under the condition that the model prediction probability is greater than or equal to a probability threshold value, retaining a preliminary service identification result of the original code stream; and under the condition that the model prediction probability is smaller than the probability threshold value, carrying out service identification based on the manual rule according to the manual rule table to obtain a service identification result. In some embodiments, the manual rule table is a series of manually generated rule tables that may represent expert experience rules that can represent manual suggestions for data classification.

Under the condition that the protocol type of the original code stream is other protocols, service identification based on manual rules can be carried out according to the manual rule table, and a service identification result is obtained.

As can be seen from the overall flow shown in FIG. 2, the service identification method in the embodiment of the application has stronger expansibility, and the algorithm model of the AI model can be a newly added algorithm model aiming at a newly added protocol or other protocols.

According to the embodiment of the application, the protocol classification can be carried out on the original code stream according to the set judging conditions, and the problem of scene division errors caused by the problem of the data source only exists; by setting the probability threshold, the accuracy of service identification can be improved by using artificial rules instead for service identification under the condition of lower accuracy of the primary service identification result. The AI models can not interfere with each other, and the processing process of the service characteristic data of different types can not interfere with each other, so that the parallel processing of the service characteristic data of different types can be realized, and the service identification results corresponding to the service characteristic data of different types can be obtained at the same time.

The training process of the above-described end-to-end system is exemplarily described below. After the original code stream for training the end-to-end system is obtained, data preprocessing operation is firstly carried out on the original code stream, so that a data set manufactured from the original code stream is obtained. Illustratively, each piece of data in the dataset contains a series of fields such as a tag (label) id, feature (feature) field, hostname, protocol, encapsulation length (package length), uniform resource locator (Uniform Resource Locator, URL), etc., and table 1 is detailed information of the fields in the data.

TABLE 1

Field name	Format of the form	Meaning of
			label id	int	Data tag, belonging to service class
feature	int	Hexadecimal code stream
			host	string	Host name
Protocol(s)	string	Protocol class
			Packaging length	list	Length of each transport packet of a piece of data
URL	string	Linking

When the data set is read, the protocol type of the original code stream is firstly divided according to the protocol, namely the protocol type of the original code stream in the data set can comprise a plurality of parts such as a protocol 1, a protocol 2, a protocol N and the like. The available fields for each protocol are not unique and therefore the adaptable AI model is likewise not unique. For example, the HTTP protocol also contains translatable features fields and critical plaintext fields, so both CNN and NLP models are applicable. The prediction precision of different models obtained under the condition can be compared transversely, and the service class is judged from multiple dimensions, so that the service identification performance is better.

In the embodiment of the application, the key plaintext fields are defined as some fields with obvious semantic information in the database, and can intuitively show the specific information of the current service class. These fields may be used as training data for some language models, such as algorithms BERT, fastText.

The embodiment of the application can set different judging conditions so as to pertinently take different contents in the original code stream as service characteristic data, thereby training an end-to-end system, wherein the different contents in the original code stream can be contents such as code stream information, key plaintext fields, target fields and the like.

In the embodiment of the application, the AI model adaptation is a model adaptation mechanism designed for different protocols, field information can be utilized to the maximum extent, and based on priori knowledge, the characteristic information contained in different fields is not suitable for training by uniformly adopting a single model, and the Payload data contains obvious data differences divided according to the protocols. For example, the HTTP protocol includes unencrypted code stream information and key plaintext fields; the code stream information in the HTTPS is encrypted, so that a key plaintext field is selected to train language models such as NLP and the like.

After the end-to-end system is trained by loading the model framework, the trained end-to-end system can be utilized to carry out service identification on the original code stream. Illustratively, after determining an AI model adapted to the original code stream according to the protocol type of the original code stream, the AI model may be utilized to process service feature data in the original code stream, so as to obtain a preliminary service identification result and a model prediction probability of the original code stream; under the condition that the model prediction probability is smaller than the probability threshold value, the service identification accuracy of the AI model is insufficient to meet the accuracy requirement of service identification, so that the service identification result can be obtained by using the service identification based on the manual rule, and the service identification result is covered on the primary service identification result. It should be noted that, the manual rule cannot completely replace the algorithm model, and is only used as an auxiliary tool to participate in the service identification of the data.

Table 2 shows the training accuracy, test accuracy, and model volume of various AI models, and it can be seen from Table 2 that for the feature field, the training and application can be performed using models such as acceptance and ResNet, and for the host field, the training and application can be performed using models such as FastText and textCNN. Models such as FastText, textCNN and the like generally belong to lighter models with relatively simple structures, have higher reasoning speed, and have the problem of smaller model data volume.

TABLE 2

Model	Data field	Training accuracy	Test accuracy	Model volume
					Inception	feature	60％	40％	400MB
ResNet	feature	86％	78％	1.1GB
					FastText	host et al	94％	87％	20MB
TextCNN	host et al	88％	82％	100MB

Taking the residual network as an example, table 3 shows the recognition coverage and recognition accuracy of residual network models corresponding to different probability thresholds, where the recognition coverage represents the ratio of the preliminary service recognition result higher than the probability threshold after the probability threshold is set in all the preliminary service recognition results, and the recognition accuracy represents the accuracy of the preliminary service recognition result higher than the probability threshold.

TABLE 3 Table 3

Probability threshold	Identification coverage (%)	Recognition accuracy (%)
			0.3	87	70
0.5	79	69
			0.7	61	76
0.9	42	83
			0.99	31	85

In summary, according to the embodiment of the application, the AI model adapted to the original code stream can be determined according to the protocol type of the original code stream; for different protocol types, the final service identification result can be determined by combining an AI model and manual rules; and switching to use manual rules to carry out risk avoidance on the output service identification result under the condition that the model prediction probability is smaller than the probability threshold value, so as to ensure the reasoning precision of the whole framework.

Compared with a scheme of carrying out service identification only by means of manual rule verification in the related art, the embodiment of the application can apply the AI model to the scheme of service identification, thereby improving the accuracy of service identification, for example, compared with the scheme of carrying out service identification only by means of manual rule verification in the related art, the scheme of service identification in the embodiment of the application can improve the identification accuracy by 10% -20%.

Compared with the scheme that service identification is carried out only through manual rule verification in the related art, the embodiment of the application can organically combine manual rules and the AI model, the manual rules and the AI model complement each other, most of original code streams can be accurately identified by adopting the AI model, and under the condition that the accuracy of a preliminary service identification result obtained by adopting the AI model is low, the preliminary service identification result obtained by adopting the AI model is an abnormal result, and the service identification based on the manual rules can be used for the abnormal result to obtain the service identification result.

Compared with a lightweight machine learning model, the embodiment of the application can accurately predict the service type of network data and can support larger data volume by using the deep learning AI model. In addition, the embodiment of the application provides expansibility with higher degree of freedom for newly added service and scene; the coupling degree between all the substructures in the end-to-end system is low, and the reliability of the model structure is high.

The embodiment of the application can be applied to the scene of data mining and deep learning by using DPI equipment, can realize service identification of network data, and can be widely applied to specific data extraction and algorithm training.

It will be appreciated by those skilled in the art that in the above-described method of the specific embodiments, the written order of steps is not meant to imply a strict order of execution but rather should be construed according to the function and possibly inherent logic of the steps.

On the basis of the service identification method provided by the embodiment, the embodiment of the application also provides a service identification device.

Fig. 3 is a schematic structural diagram of a service identifying device according to an embodiment of the present application, as shown in fig. 3, the device may include an obtaining module 300, a first processing module 301, a second processing module 302, and a third processing module 303; wherein,,

an acquisition module 300, configured to acquire an original code stream of network data;

the first processing module 301 is configured to determine service feature data of the original code stream by analyzing the original code stream; determining the type of the service characteristic data;

the second processing module 302 is configured to determine, according to the type of the service feature data, a first identification method for performing service identification on the original code stream;

and a third processing module 303, configured to process the service feature data by using the first identification method, so as to obtain a service identification result of the original code stream.

In some embodiments, the first processing module 301 is configured to determine, by analyzing the original code stream, service feature data of the original code stream, including: determining the protocol type of the original code stream; and determining service characteristic data of the original code stream according to the protocol type of the original code stream.

In some embodiments, the first processing module 301 is configured to determine service characteristic data of the original code stream according to a protocol type of the original code stream, and includes:

and determining a target field of the original code stream as service characteristic data of the original code stream under the condition that the protocol type of the original code stream is other protocols, wherein the other protocols represent network communication protocols except the HTTP, HTTPS and the newly added supportable protocol, and the target field is a predefined field.

In some embodiments, the first processing module 301 is configured to determine service characteristic data of the original code stream according to a protocol type of the original code stream, and further includes:

and under the condition that the protocol type of the original code stream is HTTPS protocol, determining the content in the original code stream, from which the plaintext information cannot be acquired, as the service characteristic data of the original code stream.

In some embodiments, the second processing module 302 is configured to determine, according to the type of the service feature data, a first identifying method for identifying a service of the original code stream, where the first identifying method includes:

under the condition that the service characteristic data is a key plaintext field, determining the first identification method comprises a method for carrying out service identification by using an NLP model; the key plaintext field is a field for representing the service type of the original code stream by plaintext;

under the condition that the service characteristic data is the code stream information of the original code stream, determining the first identification method comprises a method for carrying out service identification by using a CNN model; or determining that the first identification method comprises a method for carrying out service identification by using a CNN model under the condition that the protocol type of the original code stream is HTTPS protocol and the service characteristic data is content in which plaintext information cannot be acquired in the original code stream;

Judging whether the target field contains the key plaintext field or not under the condition that the service characteristic data is the target field, and obtaining a judging result; determining the first identification method according to the judgment result; the target field is a predetermined field.

In some embodiments, the second processing module 302 is configured to determine the first identification method according to the determination result, and includes:

when the judging result is that the target field contains the key plaintext field, determining the first identification method comprises a method for carrying out service identification by using an NLP model;

and under the condition that the judging result is that the target field does not contain the key plaintext field, determining the first identification method comprises a method for carrying out service identification by using a preset manual rule.

In some embodiments, the third processing module 303 is configured to process the service feature data by using the first identifying method to obtain a service identifying result of the original code stream, and includes:

in the case that the first recognition method includes a method for performing service recognition by using the NLP model, or the first recognition method includes a method for performing service recognition by using the CNN model, processing the service feature data by using the first recognition method to obtain a preliminary service recognition result and a model prediction probability of the original code stream, where the model prediction probability represents accuracy of obtaining the preliminary service recognition result by using the NLP model or the CNN model;

Under the condition that the model prediction probability is greater than or equal to a probability threshold value, taking a preliminary service identification result of the original code stream as the service identification result; and under the condition that the model prediction probability is smaller than a probability threshold, the first recognition method is redetermined into a method for carrying out service recognition by using a preset manual rule, and the redetermined first recognition method is adopted to process the service feature data, so that a service recognition result of the original code stream is obtained.

In practical applications, the acquiring module 300, the first processing module 301, the second processing module 302, and the third processing module 303 may be implemented based on a processor of an electronic device.

It should be noted that the description of the above device embodiments is similar to the description of the method embodiments described above, with similar advantageous effects as the method embodiments. For technical details not disclosed in the embodiments of the apparatus of the present application, please refer to the description of the embodiments of the method of the present application.

It should be noted that, in the embodiment of the present application, if the method is implemented in the form of a software functional module, and sold or used as a separate product, the method may also be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the embodiments of the present application may be embodied essentially or in part in the form of a software product stored in a storage medium, including instructions for causing a computer device (which may be a terminal, a server, etc.) to perform all or part of the methods described in the embodiments of the present application. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read Only Memory (ROM), a magnetic disk, an optical disk, or other various media capable of storing program codes. Thus, embodiments of the application are not limited to any specific combination of hardware and software.

Correspondingly, the embodiment of the application further provides a computer program product, which comprises computer executable instructions for implementing any one of the service identification methods provided by the embodiment of the application.

Accordingly, an embodiment of the present application further provides a computer storage medium, where computer executable instructions are stored on the computer storage medium, where the computer executable instructions are configured to implement any one of the service identification methods provided in the foregoing embodiments.

An embodiment of the present application further provides an electronic device, fig. 4 is a schematic diagram of a composition structure of the electronic device provided in the embodiment of the present application, and as shown in fig. 4, the electronic device 40 may include:

a memory 401 for storing executable instructions;

and a processor 402, configured to implement any one of the service identification methods when executing the executable instructions stored in the memory 401.

The processor 402 may be at least one of ASIC, DSP, DSPD, PLD, FPGA, CPU, a controller, a microcontroller, and a microprocessor.

The computer readable storage medium/Memory may be a Read Only Memory (ROM), a programmable Read Only Memory (Programmable Read-Only Memory, PROM), an erasable programmable Read Only Memory (Erasable Programmable Read-Only Memory, EPROM), an electrically erasable programmable Read Only Memory (Electrically Erasable Programmable Read-Only Memory, EEPROM), a magnetic random access Memory (Ferromagnetic Random Access Memory, FRAM), a Flash Memory (Flash Memory), a magnetic surface Memory, an optical disk, or a Read Only optical disk (Compact Disc Read-Only Memory, CD-ROM); but may also be various terminals such as mobile phones, computers, tablet devices, personal digital assistants, etc., that include one or any combination of the above-mentioned memories.

In some embodiments, the functions or modules included in the apparatus provided by the embodiments of the present application may be used to perform the methods described in the foregoing method embodiments, and specific implementations thereof may refer to descriptions of the foregoing method embodiments, which are not repeated herein for brevity.

The foregoing description of various embodiments is intended to highlight differences between the various embodiments, which may be the same or similar to each other by reference, and is not repeated herein for the sake of brevity.

The methods disclosed in the method embodiments provided by the application can be arbitrarily combined under the condition of no conflict to obtain a new method embodiment.

The features disclosed in the embodiments of the products provided by the application can be combined arbitrarily under the condition of no conflict to obtain new embodiments of the products.

The features disclosed in the embodiments of the method or the device provided by the application can be arbitrarily combined under the condition of no conflict to obtain a new embodiment of the method or the device.

From the above description of the embodiments, it will be clear to those skilled in the art that the above-described embodiment method may be implemented by means of software plus a necessary general hardware platform, but of course may also be implemented by means of hardware, but in many cases the former is a preferred embodiment. Based on such understanding, the technical solution of the present application may be embodied essentially or in a part contributing to the prior art in the form of a software product stored in a storage medium (e.g. ROM/RAM, magnetic disk, optical disk) comprising instructions for causing a terminal (which may be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to perform the method according to the embodiments of the present application.

The embodiments of the present application have been described above with reference to the accompanying drawings, but the present application is not limited to the above-described embodiments, which are merely illustrative and not restrictive, and many forms may be made by those having ordinary skill in the art without departing from the spirit of the present application and the scope of the claims, which are to be protected by the present application.

Claims

1. A method of service identification, the method comprising:

acquiring an original code stream of network data;

2. The method according to claim 1, wherein said determining the traffic characteristic data of the original code stream by analyzing the original code stream comprises:

Determining the protocol type of the original code stream; and determining service characteristic data of the original code stream according to the protocol type of the original code stream.

3. The method according to claim 2, wherein said determining service characteristic data of said original code stream according to a protocol type of said original code stream comprises:

under the condition that the protocol type of the original code stream is a hypertext transfer protocol (HTTP), determining code stream information and a key plaintext field of the original code stream as service characteristic data of the original code stream, wherein the key plaintext field is a field for representing the service type of the original code stream by using plaintext;

4. The method of claim 3, wherein said determining service characteristic data of said original code stream according to a protocol type of said original code stream further comprises:

5. The method according to any one of claims 1 to 4, wherein the determining a first identifying method for identifying the service of the original code stream according to the type of the service feature data includes:

under the condition that the service characteristic data is a key plaintext field, determining the first identification method comprises a method for carrying out service identification by using a natural language processing NLP model; the key plaintext field is a field for representing the service type of the original code stream by plaintext;

under the condition that the service characteristic data is the code stream information of the original code stream, determining the first identification method comprises a method for carrying out service identification by using a convolutional neural network CNN model; or determining that the first identification method comprises a method for carrying out service identification by using a CNN model under the condition that the protocol type of the original code stream is HTTPS protocol and the service characteristic data is content in which plaintext information cannot be acquired in the original code stream;

6. The method of claim 5, wherein the determining the first recognition method according to the determination result includes:

7. The method of claim 5, wherein the processing the service feature data by using the first recognition method to obtain the service recognition result of the original code stream includes:

8. A service identification device, the device comprising:

and the third processing module is used for processing the service characteristic data by adopting the first identification method to obtain a service identification result of the original code stream.

9. An electronic device comprising a processor and a memory for storing a computer program capable of running on the processor; wherein,,

the processor is configured to run the computer program to perform the service identification method of any one of claims 1 to 7.

10. A computer storage medium having stored thereon a computer program, which when executed by a processor implements the service identification method of any of claims 1 to 7.