CN108400938A - A kind of data flow processing method and device - Google Patents
A kind of data flow processing method and device Download PDFInfo
- Publication number
- CN108400938A CN108400938A CN201810072594.5A CN201810072594A CN108400938A CN 108400938 A CN108400938 A CN 108400938A CN 201810072594 A CN201810072594 A CN 201810072594A CN 108400938 A CN108400938 A CN 108400938A
- Authority
- CN
- China
- Prior art keywords
- matching
- message information
- message
- matching result
- result
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L45/00—Routing or path finding of packets in data switching networks
- H04L45/74—Address processing for routing
- H04L45/745—Address table lookup; Address filtering
- H04L45/74591—Address table lookup; Address filtering using content-addressable memories [CAM]
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
The present invention provides a kind of data flow processing method and devices, including：Obtain data flow, wherein the data flow includes at least one message；For message described in each, it is performed both by：Judge whether current message is IP packet, if so, extracting message information from the current message；The message information is replicated according to pre-set matching threshold, obtains at least two message informations；Determine the matching order of each message information；According to the matching order of each message information, each message information is matched successively；Determine the corresponding matching result of each message information；The corresponding matching result of each message information is analyzed.Scheme provided by the invention can reduce matching delay.
Description
Technical field
The present invention relates to digital communication technology field, more particularly to a kind of data flow processing method and device.
Background technology
With the development of digital communication technology, matched demand is carried out repeatedly increasingly to data stream using Different Rule
It is high.
Currently, when being matched respectively to data stream using Different Rule, after the completion of preceding primary matching, need to matching
As a result it is analyzed, and time matching after carrying out is decided whether according to matching result.
But the matching delay of this method is larger.
Invention content
An embodiment of the present invention provides a kind of data flow processing method and devices, can reduce matching delay.
In a first aspect, an embodiment of the present invention provides a kind of data flow processing methods, including：
Obtain data flow, wherein the data flow includes at least one message；
For message described in each, it is performed both by：Judge whether current message is IP packet, if so, from described current
Message information is extracted in message；The message information is replicated according to pre-set matching threshold, obtains at least two
Message information；
Determine the matching order of each message information；
According to the matching order of each message information, each message information is matched successively；
Determine the corresponding matching result of each message information；
The corresponding matching result of each message information is analyzed.
Preferably,
The matching order according to each message information successively matches each message information, packet
It includes：
According to the matching order of each message information, the matching order and matching rule of pre-set message information
Correspondence, determine the corresponding matching rule of each message information, successively utilize corresponding matching rule, to each institute
Message information is stated to be matched.
Preferably,
It is described that the corresponding matching result of each message information is analyzed, including：
A1：Using the matching result to make number one as current matching result, wherein each message information is corresponding
Matching result is arranged according to the matching order；
A2：Judge whether the current matching result meets pre-set matching requirement, if so, A3 is executed, otherwise,
Execute A4；
A3：The matching result after the current matching result will be come to delete；
A4：Judge whether the current matching result rolls into last place, if not, the current matching result will be come
The matching result of latter position is updated to current matching as a result, and executing A2.
Preferably,
The message information, including：Five-tuple information.
Second aspect, an embodiment of the present invention provides a kind of data stream processing devices, including：
Processing unit, for obtaining data flow, wherein the data flow includes at least one message；For each
The message, is performed both by：Judge whether current message is IP packet, if so, extracting message information from the current message；
The message information is replicated according to pre-set matching threshold, obtains at least two message informations；
Sequencing unit, the matching order for determining each message information that the processing unit obtains；
TCAM (Ternary Content Addressable Memory, three-state content addressing memory) is used for basis
The matching order for each message information that the sequencing unit determines, successively matches each message information；
Alignment unit, for determining the corresponding matching result of each message information；
Analytic unit, for analyzing the corresponding matching result of each message information.
Preferably,
The TCAM is used for the matching of the matching order, pre-set message information according to each message information
The correspondence of sequence and matching rule, determines the corresponding matching rule of each message information, utilizes corresponding successively
With rule, each message information is matched.
Preferably,
The analytic unit is used for A1：Using the matching result to make number one as current matching result, wherein each
The corresponding matching result of the message information is arranged according to the matching order；A2：Judge whether the current matching result is full
Otherwise the pre-set matching requirement of foot, executes A4 if so, executing A3；A3：After coming the current matching result
Matching result delete；A4：The matching result for coming the latter position of current matching result is updated to current matching as a result, simultaneously
Execute A2.
Preferably,
The message information, including：Five-tuple information.
The third aspect, an embodiment of the present invention provides a kind of readable mediums, including execute instruction, when the place of storage control
When being executed instruction described in reason device execution, the storage control executes the method described in any of the above-described embodiment.
Fourth aspect, an embodiment of the present invention provides a kind of storage controls, including：Processor, memory and bus；
The memory is executed instruction for storing, and the processor is connect with the memory by the bus, when
When the storage control operation, the processor executes the described of memory storage and executes instruction, so that the storage
Controller executes the method described in any of the above-described embodiment.
An embodiment of the present invention provides a kind of data flow processing method and devices, wherein this method is multiple to needing to carry out
Matched message information is replicated, is sorted, and is matched successively according to determining matching order.In the method, it matches
As a result the delay interval being sequentially output waits for the delay of previous matching result much smaller than rear matching in the prior art, can be smaller
Or time matching needs to wait for the delay time of previous matching result after eliminating, and is carried out to front and back matching relationship decoupling.
Description of the drawings
In order to more clearly explain the embodiment of the invention or the technical proposal in the existing technology, to embodiment or will show below
There is attached drawing needed in technology description to be briefly described, it should be apparent that, the accompanying drawings in the following description is the present invention
Some embodiments for those of ordinary skill in the art without creative efforts, can also basis
These attached drawings obtain other attached drawings.
Fig. 1 is a kind of flow chart of data flow processing method provided by one embodiment of the present invention；
Fig. 2 is a kind of structural schematic diagram of data stream processing device provided by one embodiment of the present invention.
Specific implementation mode
In order to make the object, technical scheme and advantages of the embodiment of the invention clearer, below in conjunction with the embodiment of the present invention
In attached drawing, technical scheme in the embodiment of the invention is clearly and completely described, it is clear that described embodiment is
A part of the embodiment of the present invention, instead of all the embodiments, based on the embodiments of the present invention, those of ordinary skill in the art
The every other embodiment obtained without making creative work, shall fall within the protection scope of the present invention.
As shown in Figure 1, an embodiment of the present invention provides a kind of data flow processing method, this method may include following step
Suddenly：
Step 101：Obtain data flow, wherein data flow includes at least one message；
Step 102：For each message, it is performed both by：Judge whether current message is IP packet, if so, executing step
Rapid 103；
Step 103：Message information is extracted from current message；
Step 104：Message information is replicated according to pre-set matching threshold, obtains at least two messages letter
Breath；
Step 105：Determine the matching order of each message information；
Step 106：According to the matching order of each message information, each message information is matched successively；
Step 107：Determine the corresponding matching result of each message information；
Step 108：The corresponding matching result of each message information is analyzed.
This method is replicated to needing to carry out repeatedly matched message information, is sorted, and according to determining matching order
It is matched successively.In the method, the delay interval that matching result is sequentially output is much smaller than rear matching etc. in the prior art
Wait for the delay of previous matching result, can it is smaller or after eliminating time matching need to wait for the delay time of previous matching result, it is right
Front and back matching relationship carries out decoupling.
Wherein, data flow can be the data packet of network flow, can also be the authentication data stream of Verification System.
In one embodiment of the invention, according to the matching order of each message information, successively to each message information
It is matched, including：
According to the matching order of each message information, pair of the matching order of pre-set message information and matching rule
Should be related to, determine the corresponding matching rule of each message information, successively utilize corresponding matching rule, to each message information into
Row matching.
The matching process is realized by TCAM, wherein multiple regions divided inside TCAM, correspond to different matching rules,
And then meets the needs of Different matching rule.
In practical application scene, whether secondary match carries out after the previous matching result decision of TCAM, front and back matching process
There are close coupling relationships, to enable multiple matching efficiency to be more nearly single match efficiency, to improve entire analysis system
Performance, this method to front and back matching process carry out it is decoupling.
In one embodiment of the invention, the corresponding matching result of each message information is analyzed, including：
A1：Using the matching result to make number one as current matching result, wherein the corresponding matching of each message information
As a result it is arranged according to matching order；
A2：Judge whether current matching result meets pre-set matching requirement, if so, executing A3, otherwise, executes
A4；
A3：The matching result after current matching result will be come to delete；
A4：Judge whether current matching result rolls into last place, if not, the latter position of current matching result will be come
Matching result is updated to current matching as a result, and executing A2.
By taking two message informations as an example, two matching results can successively be received by matching process, if previous result table
It time no longer needs to match after bright, if time matched after then system directly abandons as a result, previous result demonstrates the need for time matching after carrying out,
Time matching result after then system can directly acquire, time matched decision information after whether being carried out without return.
In one embodiment of the invention, message information, including：Five-tuple information.
Wherein, five-tuple information includes：Source IP address, source port, purpose IP address, destination interface and transport layer protocol.
It should be noted that according to actual needs, the message information of extraction is in addition to that can also be message for five-tuple information
In the other informations such as data.
In one embodiment of the invention, by taking the message information of extraction is five-tuple information as an example, to data source processing
Method is described in detail, and this method includes：
Obtain data flow, wherein data flow includes three messages, and an IP packet is filtered out from three messages.
IP packet is parsed, the message after parsing isolates five-tuple information, and one is carried out to five-tuple information
Secondary duplication, and matched number is needed in extension bits label, then TCAM is inputted by certain delay is front and back.
The five-tuple information of front and back sequence, which is sequentially input to TCAM, to be matched, and the five-tuple information initially entered is according to certainly
Body label is matched in the first formula area, and the five-tuple information entered below is according to self marker in the progress of Second Rule area
Match, obtains matching result twice.
Matching result is aligned with corresponding five-tuple information holding, then carries out matching result analysis.If occurring five
When tuple information or matching result are lost, not influence subsequent match, reset signal is triggered, by the matching result and corresponding five
Tuple information abandons.
In matching result analytic process, handle first previous matched as a result, being made whether time matching after needing to carry out
Necessity, time matching after if desired carrying out, then time matched result is analyzed after directly reading；If secondary after need not carrying out
Match, then read rear secondary matching result and does discard processing.
As shown in Fig. 2, an embodiment of the present invention provides a kind of data stream processing devices, including：
Processing unit 201, for obtaining data flow, wherein data flow includes at least one message；For each report
Text is performed both by：Judge whether current message is IP packet, if so, extracting message information from current message；According to setting in advance
The matching threshold set replicates message information, obtains at least two message informations；
Sequencing unit 202, the matching order for determining each message information that processing unit obtains；
TCAM 203, the matching order of each message information for being determined according to sequencing unit 202, successively to each report
Literary information is matched；
Alignment unit 204, for determining the corresponding matching result of each message information；
Analytic unit 205, for analyzing the corresponding matching result of each message information.
In one embodiment of the invention, 203 TCAM, for according to the matching order of each message information, set in advance
The matching order for the message information set and the correspondence of matching rule determine the corresponding matching rule of each message information, according to
It is secondary to utilize corresponding matching rule, each message information is matched.
In one embodiment of the invention, analytic unit is used for A1：Using the matching result to make number one as current
Matching result, wherein the corresponding matching result of each message information is arranged according to matching order；A2：Judge current matching result
Whether meet pre-set matching requirement, if so, executing A3, otherwise, executes A4；A3：To come current matching result it
Matching result afterwards is deleted；A4：The matching result for coming the latter position of current matching result is updated to current matching as a result, and holding
Row A2.
In one embodiment of the invention, message information, including：Five-tuple information.
An embodiment of the present invention provides a kind of readable mediums, including execute instruction, when the processor of storage control executes
When executing instruction, method that storage control executes any of the above-described embodiment.
An embodiment of the present invention provides a kind of storage controls, including：Processor, memory and bus；
Memory is executed instruction for storing, and processor is connect with memory by bus, when storage control is run,
Processor executes executing instruction for memory storage, so that the method that storage control executes any of the above-described embodiment.
The contents such as the information exchange between each unit, implementation procedure in above-mentioned apparatus, due to implementing with the method for the present invention
Example is based on same design, and particular content can be found in the narration in the method for the present invention embodiment, and details are not described herein again.
To sum up, each embodiment of the present invention at least has the following effects that：
1, in embodiments of the present invention, this method is replicated to needing to carry out repeatedly matched message information, is sorted, and
It is matched successively according to determining matching order.In the method, the delay interval that matching result is sequentially output is much smaller than existing
Time matching waits for the delay of previous matching result after having in technology, can it is smaller or after eliminating time matching need to wait for previous matching
As a result delay time carries out front and back matching relationship decoupling, raising Data Stream Processing efficiency.
It should be noted that herein, such as first and second etc relational terms are used merely to an entity
Or operation is distinguished with another entity or operation, is existed without necessarily requiring or implying between these entities or operation
Any actual relationship or order.Moreover, the terms "include", "comprise" or its any other variant be intended to it is non-
It is exclusive to include, so that the process, method, article or equipment including a series of elements includes not only those elements,
But also include other elements that are not explicitly listed, or further include solid by this process, method, article or equipment
Some elements.In the absence of more restrictions, the element limited by sentence " including a 〃 〃 ", it is not excluded that
There is also other identical factors in the process, method, article or apparatus that includes the element.
One of ordinary skill in the art will appreciate that：Realize that all or part of step of above method embodiment can pass through
The relevant hardware of program instruction is completed, and program above-mentioned can be stored in computer-readable storage medium, the program
When being executed, step including the steps of the foregoing method embodiments is executed；And storage medium above-mentioned includes：ROM, RAM, magnetic disc or light
In the various media that can store program code such as disk.
Finally, it should be noted that：The foregoing is merely presently preferred embodiments of the present invention, is merely to illustrate the skill of the present invention
Art scheme, is not intended to limit the scope of the present invention.Any modification for being made all within the spirits and principles of the present invention,
Equivalent replacement, improvement etc., are included within the scope of protection of the present invention.
Claims (10)
1. a kind of data flow processing method, which is characterized in that including：
Obtain data flow, wherein the data flow includes at least one message；
For message described in each, it is performed both by：Judge whether current message is IP packet, if so, from the current message
Middle extraction message information；The message information is replicated according to pre-set matching threshold, obtains at least two messages
Information；
Determine the matching order of each message information；
According to the matching order of each message information, each message information is matched successively；
Determine the corresponding matching result of each message information；
The corresponding matching result of each message information is analyzed.
2. data flow processing method according to claim 1, which is characterized in that
The matching order according to each message information successively matches each message information, including：
According to the matching order, the matching order of pre-set message information and pair of matching rule of each message information
It should be related to, determine the corresponding matching rule of each message information, corresponding matching rule be utilized successively, to each report
Literary information is matched.
3. data flow processing method according to claim 1, which is characterized in that
It is described that the corresponding matching result of each message information is analyzed, including：
A1：Using the matching result to make number one as current matching result, wherein the corresponding matching of each message information
As a result it is arranged according to the matching order；
A2：Judge whether the current matching result meets pre-set matching requirement, if so, executing A3, otherwise, executes
A4；
A3：The matching result after the current matching result will be come to delete；
A4：Judge whether the current matching result rolls into last place, if not, it is latter to come the current matching result
The matching result of position is updated to current matching as a result, and executing A2.
4. according to any data flow processing method in claim 1-3, which is characterized in that
The message information, including：Five-tuple information.
5. a kind of data stream processing device, which is characterized in that including：
Processing unit, for obtaining data flow, wherein the data flow includes at least one message；For described in each
Message is performed both by：Judge whether current message is IP packet, if so, extracting message information from the current message；According to
Pre-set matching threshold replicates the message information, obtains at least two message informations；
Sequencing unit, the matching order for determining each message information that the processing unit obtains；
The matching of three-state content addressing memory TCAM, each message information for being determined according to the sequencing unit are suitable
Sequence successively matches each message information；
Alignment unit, for determining the corresponding matching result of each message information；
Analytic unit, for analyzing the corresponding matching result of each message information.
6. data stream processing device according to claim 5, which is characterized in that
The TCAM, for the matching order according to the matching order of each message information, pre-set message information
With the correspondence of matching rule, the corresponding matching rule of each message information is determined, advised successively using corresponding matching
Then, each message information is matched.
7. data stream processing device according to claim 5, which is characterized in that
The analytic unit is used for A1：Using the matching result to make number one as current matching result, wherein each described
The corresponding matching result of message information is arranged according to the matching order；A2：It is pre- to judge whether the current matching result meets
Otherwise the matching requirement being first arranged, executes A4 if so, executing A3；A3：After the current matching result will be come
It is deleted with result；A4：The matching result for coming the latter position of current matching result is updated to current matching as a result, and executing
A2。
8. according to any data stream processing device in claim 5-7, which is characterized in that
The message information, including：Five-tuple information.
9. a kind of readable medium, which is characterized in that including executing instruction, refer to when the processor of storage control executes the execution
When enabling, the storage control perform claim requires any method in 1-4.
10. a kind of storage control, which is characterized in that including：Processor, memory and bus；
The memory is executed instruction for storing, and the processor is connect with the memory by the bus, when described
When storage control is run, the processor executes the described of memory storage and executes instruction, so that the storage controls
Device perform claim requires any method in 1-4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810072594.5A CN108400938A (en) | 2018-01-25 | 2018-01-25 | A kind of data flow processing method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810072594.5A CN108400938A (en) | 2018-01-25 | 2018-01-25 | A kind of data flow processing method and device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN108400938A true CN108400938A (en) | 2018-08-14 |
Family
ID=63094879
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810072594.5A Pending CN108400938A (en) | 2018-01-25 | 2018-01-25 | A kind of data flow processing method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108400938A (en) |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1674557A (en) * | 2005-04-01 | 2005-09-28 | 清华大学 | Parallel IP packet sorter matched with settling range based on TCAM and method thereof |
| CN101035061A (en) * | 2006-03-09 | 2007-09-12 | 中兴通讯股份有限公司 | Segmented coded expansion method for realizing the match of the three-folded content addressable memory range |
| CN102801659A (en) * | 2012-08-15 | 2012-11-28 | 成都卫士通信息产业股份有限公司 | Implementation method and device for security gateway based on stream strategy |
| CN102970242A (en) * | 2012-11-09 | 2013-03-13 | 深圳市共进电子股份有限公司 | Method for achieving load balancing |
| CN103345479A (en) * | 2013-06-18 | 2013-10-09 | 苏州雄立科技有限公司 | Novel work mode for TCAM |
| CN103997469A (en) * | 2014-05-27 | 2014-08-20 | 华为技术有限公司 | Network processor configuration method and network processor |
| CN104468381A (en) * | 2014-12-01 | 2015-03-25 | 国家计算机网络与信息安全管理中心 | Implementation method for multi-field rule matching |
| US20170134279A1 (en) * | 2008-09-09 | 2017-05-11 | At&T Intellectual Property I, L.P. | Systems and Methods for Optimized Route Caching |
| CN107342926A (en) * | 2017-06-13 | 2017-11-10 | 国家计算机网络与信息安全管理中心 | A kind of method of multi-service Rapid matching distribution |
-
2018
- 2018-01-25 CN CN201810072594.5A patent/CN108400938A/en active Pending
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1674557A (en) * | 2005-04-01 | 2005-09-28 | 清华大学 | Parallel IP packet sorter matched with settling range based on TCAM and method thereof |
| CN101035061A (en) * | 2006-03-09 | 2007-09-12 | 中兴通讯股份有限公司 | Segmented coded expansion method for realizing the match of the three-folded content addressable memory range |
| US20170134279A1 (en) * | 2008-09-09 | 2017-05-11 | At&T Intellectual Property I, L.P. | Systems and Methods for Optimized Route Caching |
| CN102801659A (en) * | 2012-08-15 | 2012-11-28 | 成都卫士通信息产业股份有限公司 | Implementation method and device for security gateway based on stream strategy |
| CN102970242A (en) * | 2012-11-09 | 2013-03-13 | 深圳市共进电子股份有限公司 | Method for achieving load balancing |
| CN103345479A (en) * | 2013-06-18 | 2013-10-09 | 苏州雄立科技有限公司 | Novel work mode for TCAM |
| CN103997469A (en) * | 2014-05-27 | 2014-08-20 | 华为技术有限公司 | Network processor configuration method and network processor |
| CN104468381A (en) * | 2014-12-01 | 2015-03-25 | 国家计算机网络与信息安全管理中心 | Implementation method for multi-field rule matching |
| CN107342926A (en) * | 2017-06-13 | 2017-11-10 | 国家计算机网络与信息安全管理中心 | A kind of method of multi-service Rapid matching distribution |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US20230275835A1 (en) | Apparatus and method of generating lookups and making decisions for packet modifying and forwarding in a software-defined network engine | |
| US20210006638A1 (en) | Storing packet data in mirror buffer | |
| US11425058B2 (en) | Generation of descriptive data for packet fields | |
| US9154418B1 (en) | Efficient packet classification in a network device | |
| US10805437B2 (en) | Compiler and hardware interactions to remove action dependencies in the data plane of a network forwarding element | |
| US20150016460A1 (en) | Using Headerspace Analysis to Identify Flow Entry Reachability | |
| CN103004158A (en) | Network device with a programmable core | |
| US20120219000A1 (en) | Network switch with mutually coupled look-up engine and network processor | |
| CN111897863B (en) | Multi-source heterogeneous data fusion and convergence method | |
| CN106790170B (en) | Data packet filtering method and device | |
| CN109802960A (en) | Firewall policy processing method and processing device, computer equipment and storage medium | |
| CN107392436A (en) | A kind of method and apparatus for extracting enterprise's incidence relation information | |
| CN114172854B (en) | Report Wen Jingxiang, mirror image configuration method, virtual switch and mirror image configuration device | |
| CN107885885A (en) | A kind of data lead-in method and device | |
| CN111404768A (en) | DPI recognition realization method and equipment | |
| CN103984633B (en) | A kind of bank main passes down the automatization test system of operation | |
| CN107729486B (en) | Video searching method and device | |
| KR102365658B1 (en) | Method for classifying traffic and apparatus thereof | |
| CN108400938A (en) | A kind of data flow processing method and device | |
| KR20030042800A (en) | Parallel lookup engine and method for fast packet forwarding in network router | |
| CN107086960A (en) | A kind of message transmitting method and device | |
| CN108965093A (en) | A kind of VLAN allocation method and device | |
| CN115514683A (en) | Method and device for determining packet loss reason, exchange chip and storage medium | |
| CN102843269B (en) | A kind of method and system for simulating microcode business processing flow | |
| JP2000040085A (en) | Method and device for post-processing for japanese morpheme analytic processing |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20180814 |
|
| RJ01 | Rejection of invention patent application after publication |