CN116634046A - Message processing method and device, electronic equipment and storage medium - Google Patents

Message processing method and device, electronic equipment and storage medium Download PDF

Info

Publication number
CN116634046A
CN116634046A CN202310770363.2A CN202310770363A CN116634046A CN 116634046 A CN116634046 A CN 116634046A CN 202310770363 A CN202310770363 A CN 202310770363A CN 116634046 A CN116634046 A CN 116634046A
Authority
CN
China
Prior art keywords
message
analysis
group
protocol type
data
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202310770363.2A
Other languages
Chinese (zh)
Inventor
唐奇敏
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Anbotong Jin'an Technology Co ltd
Original Assignee
Beijing Anbotong Jin'an Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Anbotong Jin'an Technology Co ltd filed Critical Beijing Anbotong Jin'an Technology Co ltd
Priority to CN202310770363.2A priority Critical patent/CN116634046A/en
Publication of CN116634046A publication Critical patent/CN116634046A/en
Pending legal-status Critical Current

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L69/00—Network arrangements, protocols or services independent of the application payload and not provided for in the other groups of this subclass
    • H04L69/22—Parsing or analysis of headers
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L43/00—Arrangements for monitoring or testing data switching networks
    • H04L43/18—Protocol analysers
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D30/00—Reducing energy consumption in communication networks
    • Y02D30/50—Reducing energy consumption in communication networks in wire-line communication networks, e.g. low power modes or reduced link rate

Abstract

The embodiment of the invention relates to the technical field of computers and discloses a message processing method, a device, electronic equipment and a storage medium, wherein the method comprises the following steps: when a plurality of data messages are received, five-tuple information of each data message is obtained; creating a message group formed by a plurality of data messages based on five-tuple information of each data message; analyzing the data messages in the message group, and determining the analysis protocol type of the message group, wherein the analysis protocol type comprises a standard analysis protocol type and a custom analysis protocol type; and analyzing the message group based on a target message analysis rule corresponding to the analysis protocol type, and determining the message protocol of the message group. By applying the technical scheme of the invention, the analysis efficiency of the data message can be improved.

Description

Message processing method and device, electronic equipment and storage medium
Technical Field
The embodiment of the invention relates to the technical field of computers, in particular to a message processing method, a message processing device, electronic equipment and a storage medium.
Background
The application layer is the highest layer of the OSI (Open System Interconnection Reference Model, open communication system interconnection reference model) reference model, which serves users, and has the user interface functions of network transport, which mainly functions to adhere to various application layer protocols, thereby generating data, which is then sent down to the transport layer. By using the application layer protocol, the data message transmitted by the application layer can be analyzed, and the message content of the data message can be identified.
The formats of the data messages are various, for some data messages of the custom protocol, the data messages are generally analyzed first, then the data messages are developed independently, the message formats are summarized, and finally the protocol analysis is carried out according to the summarized message formats, so that the message content of the data messages is obtained. However, the method adopting the customized development flow has the advantages of complex message processing flow and low universality, so that the protocol analysis efficiency of the data message is low.
Disclosure of Invention
In view of the above problems, embodiments of the present invention provide a method, an apparatus, an electronic device, and a storage medium for processing a packet, which are used to solve the problem in the prior art that the protocol parsing efficiency of a data packet is low.
According to an aspect of an embodiment of the present invention, there is provided a method for processing a message, including: when a plurality of data messages are received, five-tuple information of each data message is obtained; creating a message group formed by a plurality of data messages based on five-tuple information of each data message; analyzing the data messages in the message group, and determining the analysis protocol type of the message group, wherein the analysis protocol type comprises a standard analysis protocol type and a custom analysis protocol type; and analyzing the message group based on a target message analysis rule corresponding to the analysis protocol type, and determining the message protocol of the message group.
In some embodiments, the five-tuple information includes a source IP address, a source port, a destination IP address, a destination port, and a transport layer protocol of the data packet.
In some embodiments, the creating a packet group including a plurality of data packets based on five-tuple information of each of the data packets includes: and determining a plurality of data messages with the same five-tuple information as the message group based on the five-tuple information of each data message.
In some embodiments, the analyzing the data packets in the packet group to determine the parsing protocol type of the packet group includes: and adopting a flow classification algorithm to match the five-tuple information of any data message in the message group with the five-tuple information in the configuration information, and determining the analysis protocol type of the message group, wherein the flow classification algorithm comprises an afbv algorithm.
In some embodiments, the method further comprises: matching the IP address of the data message in the message group with the IP address in the configuration information by adopting a bit vector method, and determining the matching result of the data message in the message group and the IP address of the configuration information; the IP address of the data message comprises a source IP address and a destination IP address, and the IP address in the configuration information comprises a source configuration IP address and a destination configuration IP address.
In some embodiments, the method further comprises: if the analysis protocol type is the custom analysis protocol type, extracting analysis characteristics of the message group; matching the analysis characteristics of the message group with the preset analysis characteristics corresponding to the custom analysis protocol type; and when the analysis characteristics of the message group are matched with any preset analysis characteristics corresponding to the custom analysis protocol type, determining the analysis rule corresponding to any preset analysis characteristics as the target message analysis rule.
In some embodiments, the parsing the packet based on the target packet parsing rule corresponding to the parsing protocol type, to determine a packet protocol of the packet includes: if the analysis protocol type is the standard analysis protocol type, determining a standard message analysis rule as the target message analysis rule, analyzing the message group according to the standard message analysis rule, and determining a message protocol of the message group; if the analysis protocol type is the custom analysis protocol type, determining a custom message analysis rule as the target message analysis rule, analyzing the message group according to the custom message analysis rule, and determining the message protocol of the message group.
In some embodiments, before parsing the packet based on the target packet parsing rule corresponding to the parsing protocol type, the method further includes: and loading the target message analysis rule into a configuration linked list so as to read and update the target message analysis rule through the configuration linked list.
In some embodiments, the method further comprises: when the analysis protocol type of the message group is determined to be the self-defined analysis protocol type, analyzing the service index of the data message in the message group; the service index comprises any one or more of response time, message transmission delay, message response time and message transmission performance.
In some embodiments, the method further comprises: matching the effective load of the message group with the effective load in the target message analysis rule, and determining the index analysis condition of the message group; and when the message group is determined to meet the index analysis condition, analyzing the service index of the data message in the message group.
In some embodiments, the matching the payload of the packet with the payload in the target packet parsing rule, to determine an indicator parsing condition of the packet, further includes: and (3) adopting an av algorithm or hyperscan algorithm to quickly match the effective load of the message group with the effective load in the target message analysis rule so as to determine the index analysis condition.
According to another aspect of an embodiment of the present invention, there is provided a message processing apparatus, including: the acquisition module is used for acquiring five-tuple information of each data message when receiving a plurality of data messages; the creation module is used for creating a message group formed by a plurality of data messages based on five-tuple information of each data message; the analysis module is used for analyzing the data messages in the message group and determining the analysis protocol type of the message group, wherein the analysis protocol type comprises a standard analysis protocol type and a custom analysis protocol type; and the analysis module is used for analyzing the message group based on the target message analysis rule corresponding to the analysis protocol type and determining the message protocol of the message group.
In some embodiments, the five-tuple information includes a source IP address, a source port, a destination IP address, a destination port, and a transport layer protocol of the data packet.
In some embodiments, the creating module is configured to determine, based on five-tuple information of each of the data packets, a plurality of data packets having the same five-tuple information as the packet group.
In some embodiments, the analysis module is configured to match quintuple information of any data packet in the packet group with quintuple information in the configuration information by using a flow classification algorithm, and determine an parsing protocol type of the packet group, where the flow classification algorithm includes an afbv algorithm.
In some embodiments, the analyzing module is further configured to match an IP address of a data packet in the packet group with an IP address in the configuration information by using a bit vector method, and determine a matching result of the data packet in the packet group with the IP address of the configuration information; the IP address of the data message comprises a source IP address and a destination IP address, and the IP address in the configuration information comprises a source configuration IP address and a destination configuration IP address.
In some embodiments, the parsing module is further configured to extract parsing features of the packet group if the parsing protocol type is the custom parsing protocol type; matching the analysis characteristics of the message group with the preset analysis characteristics corresponding to the custom analysis protocol type; and when the analysis characteristics of the message group are matched with any preset analysis characteristics corresponding to the custom analysis protocol type, determining the analysis rule corresponding to any preset analysis characteristics as the target message analysis rule.
In some embodiments, the parsing module is configured to determine a standard message parsing rule as the target message parsing rule if the parsing protocol type is the standard parsing protocol type, parse the message group according to the standard message parsing rule, and determine a message protocol of the message group; if the analysis protocol type is the custom analysis protocol type, determining a custom message analysis rule as the target message analysis rule, analyzing the message group according to the custom message analysis rule, and determining the message protocol of the message group.
In some embodiments, before parsing the packet based on the target packet parsing rule corresponding to the parsing protocol type, the parsing module is further configured to load the target packet parsing rule into a configuration linked list, so as to read and update the target packet parsing rule through the configuration linked list.
In some embodiments, the parsing module is further configured to analyze a traffic indicator of a data packet in the packet when determining that the parsing protocol type of the packet is the custom parsing protocol type; the service index comprises any one or more of response time, message transmission delay, message response time and message transmission performance.
In some embodiments, the parsing module is further configured to match a payload of the packet with a payload in the target packet parsing rule, and determine an indicator parsing condition of the packet; and when the message group is determined to meet the index analysis condition, analyzing the service index of the data message in the message group.
In some embodiments, the parsing module is further configured to quickly match the payload of the packet with the payload in the target packet parsing rule by using an av algorithm or a hyperscan algorithm, so as to determine the indicator parsing condition.
According to another aspect of an embodiment of the present invention, there is provided an electronic apparatus including: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to perform the operations of the message processing method of any one of the above, via execution of the executable instructions.
According to yet another aspect of an embodiment of the present invention, there is provided a computer readable storage medium having stored therein at least one executable instruction that, when executed on an electronic device, causes the electronic device to perform the operations of the message processing method as set forth in any one of the above.
In summary, according to the method, the device, the electronic equipment and the storage medium for processing the message provided by the embodiment of the invention, when a plurality of data messages are received, five-tuple information of each data message can be obtained, a message group formed by the plurality of data messages is created based on the five-tuple information of each data message, the data messages in the message group are analyzed, the analysis protocol type of the message group is determined, the message group is analyzed based on the target message analysis rule corresponding to the analysis protocol type, and the message protocol of the message group is determined. By the method, the analysis flow of the data message can be unified, the protocol analysis of the data message is realized, and the analysis efficiency of the data message can be improved by creating and analyzing the message group.
The foregoing description is only an overview of the technical solutions of the embodiments of the present invention, and may be implemented according to the content of the specification, so that the technical means of the embodiments of the present invention can be more clearly understood, and the following specific embodiments of the present invention are given for clarity and understanding.
Drawings
The drawings are only for purposes of illustrating embodiments and are not to be construed as limiting the invention. Also, like reference numerals are used to designate like parts throughout the figures. In the drawings:
FIG. 1 is a schematic flow chart of a message processing method according to the present invention;
FIG. 2 is a schematic diagram of a configuration interface provided by the present invention;
FIG. 3 is a schematic diagram of a field configuration interface provided by the present invention;
FIG. 4 is a schematic diagram showing a sub-flow of a message processing method according to the present invention;
FIG. 5 is a schematic diagram illustrating a sub-flow of another message processing method according to the present invention;
FIG. 6 is a schematic diagram illustrating a sub-flow of another message processing method according to the present invention;
FIG. 7 is a schematic diagram of a business index interface provided by the present invention;
Fig. 8 is a schematic structural diagram of a message processing apparatus according to the present invention;
fig. 9 shows a schematic structural diagram of an electronic device provided by the invention.
Detailed Description
Exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. While exemplary embodiments of the present invention are shown in the drawings, it should be understood that the present invention may be embodied in various forms and should not be limited to the embodiments set forth herein.
In one approach of the related art, a network packet analysis tool, such as wireshark software, may be used to perform protocol parsing on the data packets. Specifically, for a data message of a custom protocol, an analysis script of the data message is written through lua (a light and small script language), and then the script is stored under a specific installation directory of the wirehierarchy, so that the wirehierarchy can identify and analyze the data message according to rules set in the analysis script.
However, although the analysis script written by lua is very convenient to use, and does not need to be compiled, and the requirements can be met in the process of functional test, in the scene of a large number of data messages, because the wireshark is a function-oriented plug-in unit, and the analysis script of the data message of the custom protocol usually has a plurality of processes, the analysis script can appear that script loading is slow, the device performance is reduced, the package is lost, and the correctness of other services is affected when the data message is analyzed by the analysis script.
In view of one or more of the foregoing problems, fig. 1 shows a flowchart of a method for processing a message, which may be performed by an electronic device according to an embodiment of the present invention. As shown in fig. 1, the method comprises the steps of:
step 110: and when a plurality of data messages are received, acquiring five-tuple information of each data message.
The quintuple information is a basic attribute of the data message, and may include a source IP address, a source port, a destination IP address, a destination port, and a transport layer protocol of the data message.
When a plurality of data messages are received, the data messages can be read and analyzed to obtain five-tuple information of the data messages. For example, according to the message format of the data message, the data of a specific byte in the data message can be read and parsed to obtain the five-tuple information of the data message.
Step 120: based on five-tuple information of each data message, a message group formed by a plurality of data messages is created.
After five-tuple information of each data message is obtained, the plurality of data messages can be divided into one or more message groups according to the five-tuple information. By the method, when the data message is processed subsequently, the message group can be analyzed by taking the message group as a unit, and the processing efficiency of the data message can be improved.
In some embodiments, step 120 may be implemented by:
based on the five-tuple information of each data message, determining a plurality of data messages with the same five-tuple information as a message group.
According to the five-tuple information of each data message, the data messages with the same five-tuple information can be screened out from a plurality of data messages, and the data messages with the same five-tuple information are determined as a message group.
By the method, a plurality of data messages can be grouped according to the five-tuple information of the data messages, so that the data messages with the same five-tuple information are divided into the same message group.
Step 130: and analyzing the data messages in the message group to determine the analysis protocol type of the message group.
The parsing protocol type comprises a standard parsing protocol type and a custom parsing protocol type. The standard parsing protocol type refers to a data message type conforming to a standard protocol, for example, HTTP (Hyper Text Transfer Protocol ) of an application layer of the web may be stored as an RFC document (Request For Comments, a series of files arranged by numbers) in a public domain, and a browser can access web data conforming to the document standard to obtain a corresponding web page.
The custom parsing protocol type refers to a data message type conforming to the custom protocol, and can be configured by a developer according to actual requirements, and the custom parsing protocol type is equivalent to a special protocol and cannot be applied to the public field.
For example, the characteristics in the packet may be parsed, and the feature value obtained by parsing may be matched with the preset characteristics, so as to determine the parsing protocol type of the packet. That is, by analyzing the data packets in the packet, the parsing protocol type of the packet can be determined.
In some embodiments, step 130 may be implemented by:
and matching the five-tuple information of any data message in the message group with the five-tuple information in the configuration information by adopting a stream classification algorithm, and determining the analysis protocol type of the message group.
The flow classification algorithm may include an afbv algorithm, among others. The configuration information may be information preconfigured by a developer, and may include various information required for parsing the data packet, for example, may include preset quintuple information.
And (3) through a stream classification algorithm, the five-tuple information of any data message in the message group is quickly matched with the five-tuple information in the configuration information, so that whether the data message in the message group is consistent with the five-tuple information in the configuration information can be determined, and the analysis protocol type of the message group is determined.
For example, assuming that the quintuple information in the configuration information is the quintuple information of the data message conforming to the custom parsing protocol type, if any data message in the message group is consistent with the quintuple information in the configuration information, the parsing protocol type of the message group is described as the custom parsing protocol type. Correspondingly, if any data message in the message group is consistent with the five-tuple information in the configuration information, the analysis protocol type of the message group is the standard analysis protocol type.
Meanwhile, since the message group is created by the data messages with the same five-tuple information, the five-tuple information of any one data message in the message group is matched with the five-tuple information in the configuration information, so that the analysis protocol type of the whole message group can be determined without matching the five-tuple information of each data message in the message group with the five-tuple information in the configuration information. That is, when determining the parsing protocol type of the packet, the parsing protocol type of the whole packet can be determined by only performing one-time matching on the five-tuple information of the data packet in the packet.
By adopting the afbv algorithm to match five-tuple information of any data message in the message group, the matching efficiency of the message group can be improved, and the matching time can be reduced.
Further, in some embodiments, for IP address information in the quintuple information, matching can be performed by:
and matching the IP address of the data message in the message group with the IP address in the configuration information by adopting a bit vector method, and determining the matching result of the IP address of the data message in the message group and the configuration information.
The IP address of the data message comprises a source IP address and a destination IP address, and the IP address in the configuration information comprises a source configuration IP address and a destination configuration IP address. The bit vector method may be a range supported bit vector method.
By adopting the bit vector method, the source IP address of the data message in the message group and the source configuration IP address in the configuration information can be matched, the destination IP address of the data message in the message group and the destination configuration IP address in the configuration information can be matched, and the IP address matching result of the two can be determined.
Specifically, when the bit vector method is adopted to match the IP addresses, for example, the source IP addresses of the data messages in the message group, may be precoded and stored in the memory. Secondly, the fields of the packet heads of the data messages are used as addresses of the read memories, and the range matching result is obtained through the Boolean operation of the pipeline.
In some embodiments, a bit vector (AFBV) method of any type field may be further used to match quintuple information of any data packet in the packet group with quintuple information in the configuration information, so as to determine an parsing protocol type of the packet group. The vector can effectively support flow classification of multi-dimensional fields, including exact matches, longest prefix matches, distance matches, and arbitrary wildcard matches.
In this embodiment, the configuration information may be information related to parsing the data packet configured by the developer, for example, the developer may set five-tuple information, transmission protocol, and other information in the configuration interface shown in fig. 2. The five-tuple information and the transmission protocol in the configuration information may be all set, or only a part of them may be set, for example, only the transmission protocol may be set, only the five-tuple information may be set, or only a part of the five-tuple information may be set, which is not limited in this embodiment.
Therefore, when the five-tuple information and the transmission protocol of the data message are inconsistent with the five-tuple information and the transmission protocol in the configuration information, the data message is not consistent with the protocol analysis rule corresponding to the configuration information, and the analysis flow corresponding to the configuration information is not entered.
Step 140: and analyzing the message group based on a target message analysis rule corresponding to the analysis protocol type, and determining the message protocol of the message group.
The target message parsing rule refers to a parsing rule of a data message corresponding to a message group, and may be preconfigured by a developer. The target message parsing rule may be set as a field configuration rule, and the fields in the target message parsing rule may be set according to the client and the response end, where the name of each field and the alias of the field may be set freely, and meanwhile, the value length of each field may be set, where the value length refers to the length of the field, and the offset is used to locate the position of the field in the data packet.
In some embodiments, a developer may configure field information corresponding to the parsing rule of the target message, such as field attribution, field name, field alias, extraction mode, offset, and value length of a certain field, in a field configuration interface as shown in fig. 3.
After determining the analysis protocol type of the message group, the message group can be analyzed according to the target message analysis rule corresponding to the analysis protocol type of the message group, and the message protocol of the message group is determined. For example, according to the field information in the target message parsing rule, the message content of the specific field of the data message in the message group can be obtained, and then the message protocol of the message group is determined according to the message protocol corresponding to the message content in the target message parsing rule.
By the method, the message group can be analyzed according to the target message analysis rule corresponding to the analysis protocol type, so that the message protocol of the message group is determined, the analysis flow of the data message is unified, and the analysis of the data message is more efficient.
For a standard parsing protocol type or a custom parsing protocol type, there may be multiple target message parsing rules corresponding to the standard parsing protocol type, and in order to determine the target message parsing rules corresponding to the parsing protocol type, in some embodiments, referring to fig. 4, the following method may be executed:
step 410: and if the analysis protocol type is the custom analysis protocol type, extracting analysis characteristics of the message group.
The parsing feature of the packet may be the content of some fields in the packet, for example, may be the content of a specific field in one or more data packets in the packet, or may also be the identification information about all data packets in the packet written in the packet, where the identification information may be the information written in the packet when the packet is created, and may be used to uniquely identify the packet.
When the analysis protocol type is a custom analysis protocol type, the analysis characteristics of the message group can be read from the message group according to the corresponding reading rule.
Step 420: and matching the analysis characteristics of the message group with the preset analysis characteristics corresponding to the custom analysis protocol type.
For example, the parsing feature of the packet may be compared with a preset parsing feature corresponding to a custom parsing protocol type.
Step 430: when the analysis characteristics of the message group are matched with any preset analysis characteristics corresponding to the custom analysis protocol type, determining the analysis rule corresponding to any preset analysis characteristics as a target message analysis rule.
When the analysis characteristics of the message group are the same as the preset analysis characteristics corresponding to the custom analysis protocol type, the match between the analysis characteristics and the preset analysis characteristics is determined, and at the moment, the analysis rule corresponding to any preset analysis characteristic can be determined as a target message analysis rule.
In some embodiments, when the number of parsing features of the packet is more than one, each parsing feature of the packet may be compared with each preset parsing feature corresponding to a custom parsing protocol type. At this time, the custom parsing protocol type may correspond to a plurality of feature groups, and when each parsing feature of the message group is respectively matched with a preset parsing feature corresponding to a certain feature group, it is indicated that the message group is matched with the feature group, so that the parsing rule corresponding to the feature group may be determined as the target message parsing rule.
By the method, the analysis characteristics of the message group can be extracted, the analysis characteristics of the message group are matched with the preset analysis characteristics corresponding to the custom analysis protocol type, and the target message analysis rule corresponding to the message group is determined.
In some embodiments, referring to fig. 5, step 140 may be implemented by either step 510 or step 520:
step 510: if the analysis protocol type is the standard analysis protocol type, determining the standard message analysis rule as a target message analysis rule, analyzing the message group according to the standard message analysis rule, and determining the message protocol of the message group.
When the analysis protocol type is determined to be the standard analysis protocol type, the data message in the message group is described to follow the standard analysis protocol, so that the standard message analysis rule can be determined to be the target message analysis rule, the message group is analyzed according to the standard message analysis rule, and the message protocol of the message group is determined.
Step 520: if the analysis protocol type is the custom analysis protocol type, determining the custom message analysis rule as a target message analysis rule, analyzing the message group according to the custom message analysis rule, and determining the message protocol of the message group.
When the analysis protocol type is determined to be the custom analysis protocol type, the data message in the message group is described to follow the custom analysis protocol, so that the custom message analysis rule can be determined to be a target message analysis rule, the message group is analyzed according to the custom message analysis rule, and the message protocol of the message group is determined.
Through the steps 510 and 520, the message group can be parsed by using the corresponding message parsing rule based on the parsing protocol type of the message group, so as to identify the message protocol of the message group, and the protocol analysis of the data message conforming to the standard parsing protocol and the custom parsing protocol can be realized.
In some embodiments, prior to step 140, the following method may also be performed:
and loading the target message analysis rule into a configuration linked list so as to read and update the target message analysis rule through the configuration linked list.
The configuration linked list is a discontinuous and non-sequential storage structure on a physical storage unit, and the logic sequence of the data elements is realized through the pointer link sequence in the configuration linked list.
By loading the target message analysis rule into the configuration linked list, the deletion, correction and check can be completed through the configuration linked list, so that the maintenance of the target message analysis rule is very convenient and takes effect in real time.
To determine the transmission performance of the data messages in the message group, in some embodiments, the following method may be further performed:
and when the analysis protocol type of the message group is determined to be the custom analysis protocol type, analyzing the service index of the data message in the message group.
The service index comprises any one or more of response time, message transmission delay, message response time and message transmission performance.
For example, when the parsing protocol type of the packet is determined to be the custom parsing protocol type, the time information of the data packet in the packet is read, and the response time, the packet transmission delay and the packet response time are calculated according to the time information, so as to determine the service indexes such as the packet transmission performance.
In some embodiments, the time information of the data message in the message group may be recorded in another transmission file during the transmission process of the data message, and at this time, the time information of the data message in the message group may be obtained by reading the transmission file, so as to determine the service index of the data message in the message group.
By the method, when the analysis protocol type of the message group is determined to be the custom analysis protocol type, the business index of the data message in the message group can be analyzed, so that the monitoring of the transmission process of the data message is realized.
In some embodiments, referring to FIG. 6, the following method may also be performed:
step 610: and matching the effective load of the message group with the effective load in the target message analysis rule, and determining the index analysis condition of the message group.
Where the payload of a group of messages refers to a portion of the transmission data of the actual intended message that does not include any headers or metadata only to facilitate the transfer of the payload. The index analysis condition refers to a condition required to be met by the business index of the analysis message group, and can be set by a developer according to actual requirements, such as a URL or a common character string.
By matching the payload of the packet with the payload in the target packet parsing rule, the indicator parsing condition of the packet can be selected from the target packet parsing rule.
Step 620: and when the message group is determined to meet the index analysis condition, analyzing the service index of the data message in the message group.
After determining the index analysis condition of the packet, the packet can be analyzed to determine whether the packet meets the index analysis condition, and if so, the business index of the data packet in the packet can be analyzed.
Referring to fig. 7, when the packet group meets the index analysis condition, the user may select the index analysis condition in the index analysis interface, so as to output the service index, such as response time and transaction time, of the data packet in the packet group.
For example, when determining whether the packet group meets the index analysis condition, a URL of a data packet in the packet group may be obtained, the URL is compared with a preset URL, and if the two URLs are consistent, it is determined that the packet group meets the index analysis condition.
By the method, the index analysis condition of the message group can be determined, and when the message group meets the index analysis condition, the business index of the data message in the message group is analyzed, so that the analysis of the transmission performance of the data message is realized.
In order to increase the efficiency of determining the index analysis condition, in some embodiments, step 610 may further include the following methods:
and (3) adopting an av algorithm or hyperscan algorithm to quickly match the effective load of the message group with the effective load in the target message analysis rule so as to determine the index analysis condition.
The av algorithm and hyperscan algorithm are two different fast matching algorithms, respectively.
And (3) utilizing an av algorithm or hyperscan algorithm to quickly match the effective load of the message group with the effective load in the target message analysis rule, and determining the index analysis condition corresponding to the message group. The matching performance of the hyperscan algorithm is better than that of the av algorithm under the condition of sufficient memory through analysis.
In summary, according to the message processing method in this embodiment, when a plurality of data messages are received, five-tuple information of each data message may be obtained, a message group formed by the plurality of data messages is created based on the five-tuple information of each data message, the data messages in the message group are analyzed, an analysis protocol type of the message group is determined, the message group is analyzed based on a target message analysis rule corresponding to the analysis protocol type, and a message protocol of the message group is determined. By the method, the analysis flow of the data message can be unified, the protocol analysis of the data message is realized, and the analysis efficiency of the data message can be improved by creating and analyzing the message group.
Fig. 8 is a schematic structural diagram of a message processing apparatus according to an embodiment of the present invention. As shown in fig. 8, the message processing apparatus 800 includes: the obtaining module 810 is configured to obtain five-tuple information of each data packet when receiving a plurality of data packets; a creating module 820, configured to create a packet group formed by a plurality of data packets based on five-tuple information of each data packet; the analysis module 830 is configured to analyze the data packets in the packet group, determine an analysis protocol type of the packet group, where the analysis protocol type includes a standard analysis protocol type and a custom analysis protocol type; the parsing module 840 is configured to parse the packet based on a target packet parsing rule corresponding to the parsing protocol type, and determine a packet protocol of the packet.
In some embodiments, the five-tuple information includes a source IP address, a source port, a destination IP address, a destination port, and a transport layer protocol of the data packet.
In some embodiments, the creating module 820 is configured to determine, based on five-tuple information of each data packet, a plurality of data packets having the same five-tuple information as a packet group.
In some embodiments, the analysis module 830 is configured to match five-tuple information of any data packet in the packet group with five-tuple information in the configuration information by using a flow classification algorithm, and the flow classification algorithm includes an afbv algorithm.
In some embodiments, the analysis module 830 is further configured to match an IP address of a data packet in the packet group with an IP address in the configuration information by using a bit vector method, and determine an IP address matching result of the data packet in the packet group and the configuration information; the IP address of the data message comprises a source IP address and a destination IP address, and the IP address in the configuration information comprises a source configuration IP address and a destination configuration IP address.
In some embodiments, the parsing module 840 is further configured to extract parsing characteristics of the packet if the parsing protocol type is a custom parsing protocol type; matching the analysis characteristics of the message group with the preset analysis characteristics corresponding to the custom analysis protocol type; when the analysis characteristics of the message group are matched with any preset analysis characteristics corresponding to the custom analysis protocol type, determining the analysis rule corresponding to any preset analysis characteristics as a target message analysis rule.
In some embodiments, the parsing module 840 is configured to determine the standard message parsing rule as the target message parsing rule if the parsing protocol type is the standard parsing protocol type, parse the message group according to the standard message parsing rule, and determine the message protocol of the message group; if the analysis protocol type is the custom analysis protocol type, determining the custom message analysis rule as a target message analysis rule, analyzing the message group according to the custom message analysis rule, and determining the message protocol of the message group.
In some embodiments, before parsing the packet based on the target packet parsing rule corresponding to the parsing protocol type, the parsing module 840 is further configured to load the target packet parsing rule into the configuration linked list, so as to read and update the target packet parsing rule through the configuration linked list.
In some embodiments, the parsing module 840 is further configured to analyze a traffic index of a data packet in the packet when determining that the parsing protocol type of the packet is a custom parsing protocol type; the service index comprises any one or more of response time, message transmission delay, message response time and message transmission performance.
In some embodiments, the parsing module 840 is further configured to match the payload of the packet with the payload in the target packet parsing rule, and determine an index parsing condition of the packet; and when the message group is determined to meet the index analysis condition, analyzing the service index of the data message in the message group.
In some embodiments, the parsing module 840 is further configured to quickly match the payloads of the packet group with the payloads in the target packet parsing rule by using an av algorithm or a hyperscan algorithm to determine the index analysis condition.
The specific details of each module in the above apparatus are already described in the method section embodiments, and the details of the undisclosed solution may be referred to the method section embodiments, so that they will not be described in detail.
Fig. 9 shows a schematic structural diagram of an electronic device according to an embodiment of the present invention, which is not limited to the specific implementation of the electronic device according to the embodiment of the present invention.
As shown in fig. 9, the electronic device may include: a processor 902, a communication interface (Communications Interface), a memory 906, and a communication bus 908.
Wherein: processor 902, communication interface 904, and memory 906 communicate with each other via a communication bus 908. A communication interface 904 for communicating with network elements of other devices, such as clients or other servers. The processor 902 is configured to execute the program 910, and may specifically perform the relevant steps in the foregoing embodiments of the method for processing a message.
In particular, the program 910 may include program code including computer-executable instructions.
The processor 902 may be a central processing unit, CPU, or a specific integrated circuit ASIC (Application Specific Integrated Circuit), or one or more integrated circuits configured to implement embodiments of the present invention. The one or more processors included in the electronic device may be the same type of processor, such as one or more CPUs; but may also be different types of processors such as one or more CPUs and one or more ASICs.
A memory 906 for storing a program 910. Memory 906 may comprise high-speed RAM memory or may also include non-volatile memory (non-volatile memory), such as at least one disk memory.
The program 910 may be specifically invoked by the processor 902 to cause the electronic device to:
when a plurality of data messages are received, five-tuple information of each data message is obtained, a message group formed by the plurality of data messages is created based on the five-tuple information of each data message, the data messages in the message group are analyzed, the analysis protocol type of the message group is determined, the analysis protocol type comprises a standard analysis protocol type and a custom analysis protocol type, the message group is analyzed based on a target message analysis rule corresponding to the analysis protocol type, and the message protocol of the message group is determined.
The embodiment of the invention provides a computer readable storage medium, which stores at least one executable instruction, and when the executable instruction runs on electronic equipment/message processing device, the electronic equipment/message processing device executes the message processing method in any method embodiment.
The executable instructions may be specifically configured to cause the electronic device/message processing apparatus to:
when a plurality of data messages are received, five-tuple information of each data message is obtained, a message group formed by the plurality of data messages is created based on the five-tuple information of each data message, the data messages in the message group are analyzed, the analysis protocol type of the message group is determined, the analysis protocol type comprises a standard analysis protocol type and a custom analysis protocol type, the message group is analyzed based on a target message analysis rule corresponding to the analysis protocol type, and the message protocol of the message group is determined.
The algorithms or displays presented herein are not inherently related to any particular computer, virtual system, or other apparatus. In addition, embodiments of the present invention are not directed to any particular programming language.
In the description provided herein, numerous specific details are set forth. It will be appreciated, however, that embodiments of the invention may be practiced without such specific details. Similarly, in the above description of exemplary embodiments of the invention, various features of embodiments of the invention are sometimes grouped together in a single embodiment, figure, or description thereof for the purpose of streamlining the disclosure and aiding in the understanding of one or more of the various inventive aspects. Wherein the claims following the detailed description are hereby expressly incorporated into this detailed description, with each claim standing on its own as a separate embodiment of this invention.
Those skilled in the art will appreciate that the modules in the apparatus of the embodiments may be adaptively changed and disposed in one or more apparatuses different from the embodiments. The modules or units or components of the embodiments may be combined into one module or unit or component and, furthermore, they may be divided into a plurality of sub-modules or sub-units or sub-components. Except that at least some of such features and/or processes or elements are mutually exclusive.
It should be noted that the above-mentioned embodiments illustrate rather than limit the invention, and that those skilled in the art will be able to design alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The word "comprising" does not exclude the presence of elements or steps not listed in a claim. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The invention may be implemented by means of hardware comprising several distinct elements, and by means of a suitably programmed computer. In the unit claims enumerating several means, several of these means may be embodied by one and the same item of hardware. The use of the words first, second, third, etc. do not denote any order. These words may be interpreted as names. The steps in the above embodiments should not be construed as limiting the order of execution unless specifically stated.

Claims (14)

1. A method for processing a message, the method comprising:
when a plurality of data messages are received, five-tuple information of each data message is obtained;
creating a message group formed by a plurality of data messages based on five-tuple information of each data message;
analyzing the data messages in the message group, and determining the analysis protocol type of the message group, wherein the analysis protocol type comprises a standard analysis protocol type and a custom analysis protocol type;
and analyzing the message group based on a target message analysis rule corresponding to the analysis protocol type, and determining the message protocol of the message group.
2. The method of claim 1, wherein the five-tuple information comprises a source IP address, a source port, a destination IP address, a destination port, and a transport layer protocol of the data message.
3. The method according to claim 1, wherein creating a packet group consisting of a plurality of data packets based on five-tuple information of each of the data packets comprises:
and determining a plurality of data messages with the same five-tuple information as the message group based on the five-tuple information of each data message.
4. The method of claim 1, wherein analyzing the data packets in the packet group to determine the parsing protocol type of the packet group comprises:
and adopting a flow classification algorithm to match the five-tuple information of any data message in the message group with the five-tuple information in the configuration information, and determining the analysis protocol type of the message group, wherein the flow classification algorithm comprises an afbv algorithm.
5. The method according to claim 4, wherein the method further comprises:
matching the IP address of the data message in the message group with the IP address in the configuration information by adopting a bit vector method, and determining the matching result of the data message in the message group and the IP address of the configuration information;
the IP address of the data message comprises a source IP address and a destination IP address, and the IP address in the configuration information comprises a source configuration IP address and a destination configuration IP address.
6. The method according to claim 1, wherein the method further comprises:
if the analysis protocol type is the custom analysis protocol type, extracting analysis characteristics of the message group;
Matching the analysis characteristics of the message group with the preset analysis characteristics corresponding to the custom analysis protocol type;
and when the analysis characteristics of the message group are matched with any preset analysis characteristics corresponding to the custom analysis protocol type, determining the analysis rule corresponding to any preset analysis characteristics as the target message analysis rule.
7. The method according to any one of claims 1-6, wherein the parsing the packet based on the target packet parsing rule corresponding to the parsing protocol type to determine a packet protocol of the packet includes:
if the analysis protocol type is the standard analysis protocol type, determining a standard message analysis rule as the target message analysis rule, analyzing the message group according to the standard message analysis rule, and determining a message protocol of the message group;
if the analysis protocol type is the custom analysis protocol type, determining a custom message analysis rule as the target message analysis rule, analyzing the message group according to the custom message analysis rule, and determining the message protocol of the message group.
8. The method according to any one of claims 1-6, wherein before parsing the group of messages based on a target message parsing rule corresponding to the parsing protocol type, the method further comprises:
and loading the target message analysis rule into a configuration linked list so as to read and update the target message analysis rule through the configuration linked list.
9. The method according to any one of claims 1-6, further comprising:
when the analysis protocol type of the message group is determined to be the self-defined analysis protocol type, analyzing the service index of the data message in the message group;
the service index comprises any one or more of response time, message transmission delay, message response time and message transmission performance.
10. The method according to claim 9, wherein the method further comprises:
matching the effective load of the message group with the effective load in the target message analysis rule, and determining the index analysis condition of the message group;
and when the message group is determined to meet the index analysis condition, analyzing the service index of the data message in the message group.
11. The method of claim 10, wherein said matching the payloads of the packet group with the payloads of the target packet parsing rule to determine the indicator parsing condition of the packet group further comprises:
and (3) adopting an av algorithm or hyperscan algorithm to quickly match the effective load of the message group with the effective load in the target message analysis rule so as to determine the index analysis condition.
12. A message processing apparatus, the apparatus comprising:
the acquisition module is used for acquiring five-tuple information of each data message when receiving a plurality of data messages;
the creation module is used for creating a message group formed by a plurality of data messages based on five-tuple information of each data message;
the analysis module is used for analyzing the data messages in the message group and determining the analysis protocol type of the message group, wherein the analysis protocol type comprises a standard analysis protocol type and a custom analysis protocol type;
and the analysis module is used for analyzing the message group based on the target message analysis rule corresponding to the analysis protocol type and determining the message protocol of the message group.
13. An electronic device, comprising:
a processor; and
a memory for storing executable instructions of the processor;
wherein the processor is configured to perform the operations of the message processing method of any of claims 1-11 via execution of the executable instructions.
14. A computer readable storage medium, wherein at least one executable instruction is stored in the storage medium, which when executed on an electronic device, causes the electronic device to perform the operations of the message processing method according to any one of claims 1-11.
CN202310770363.2A 2023-06-27 2023-06-27 Message processing method and device, electronic equipment and storage medium Pending CN116634046A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202310770363.2A CN116634046A (en) 2023-06-27 2023-06-27 Message processing method and device, electronic equipment and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202310770363.2A CN116634046A (en) 2023-06-27 2023-06-27 Message processing method and device, electronic equipment and storage medium

Publications (1)

Publication Number Publication Date
CN116634046A true CN116634046A (en) 2023-08-22

Family

ID=87592197

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202310770363.2A Pending CN116634046A (en) 2023-06-27 2023-06-27 Message processing method and device, electronic equipment and storage medium

Country Status (1)

Country Link
CN (1) CN116634046A (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117278660A (en) * 2023-11-21 2023-12-22 华信咨询设计研究院有限公司 Protocol analysis method for flow filtering based on DPDK technology
CN117472387A (en) * 2023-12-26 2024-01-30 深圳麦格米特电气股份有限公司 Method and device for dynamically analyzing data and cloud platform

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117278660A (en) * 2023-11-21 2023-12-22 华信咨询设计研究院有限公司 Protocol analysis method for flow filtering based on DPDK technology
CN117278660B (en) * 2023-11-21 2024-03-29 华信咨询设计研究院有限公司 Protocol analysis method for flow filtering based on DPDK technology
CN117472387A (en) * 2023-12-26 2024-01-30 深圳麦格米特电气股份有限公司 Method and device for dynamically analyzing data and cloud platform
CN117472387B (en) * 2023-12-26 2024-04-16 深圳麦格米特电气股份有限公司 Method and device for dynamically analyzing data and cloud platform

Similar Documents

Publication Publication Date Title
CN109688202B (en) Interface data processing method and device, computing equipment and storage medium
CN103888490B (en) A kind of man-machine knowledge method for distinguishing of full automatic WEB client side
CN116634046A (en) Message processing method and device, electronic equipment and storage medium
US6665634B2 (en) Test system for testing dynamic information returned by a web server
CN112148674B (en) Log data processing method, device, computer equipment and storage medium
CN109842629A (en) The implementation method of custom protocol based on protocol analysis frame
CN111813701B (en) HTTP-based interface testing method and device, computer equipment and storage medium
US20120278489A1 (en) Extracting web services from resources using a web services resources programming model
CN111935081B (en) Data packet desensitization method and device
CN112732572A (en) Service testing method, device and system, storage medium and electronic device
CN108900554B (en) HTTP asset detection method, system, device and computer medium
JPWO2018131199A1 (en) Coupling device, coupling method and coupling program
CN104573520A (en) Method and device for detecting permanent type cross site scripting vulnerability
CN112448969A (en) Link tracking method, device, system, equipment and readable storage medium
CN113032655A (en) Method for extracting and fixing dark network electronic data
EP2847976A1 (en) Method and apparatus
US8230002B2 (en) Method and system for automatic setup in web-based applications
US9300677B2 (en) Data security system
CN113055420B (en) HTTPS service identification method and device and computing equipment
WO2016058401A1 (en) Hypertext transfer protocol data restoring method and device
CN106055571A (en) Method and system for website identification
CN110764994A (en) Page element packaging method and device, electronic equipment and storage medium
CN115543479A (en) Interface calling analysis method and device suitable for dynamic parameters
CN113179317B (en) Test system and method for content rewriting device
CN114172980A (en) Method, system, device, equipment and medium for identifying type of operating system

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination