CN1838624B - High-performance network data processing platform system and processing method - Google Patents
High-performance network data processing platform system and processing method Download PDFInfo
- Publication number
- CN1838624B CN1838624B CN200610039900A CN200610039900A CN1838624B CN 1838624 B CN1838624 B CN 1838624B CN 200610039900 A CN200610039900 A CN 200610039900A CN 200610039900 A CN200610039900 A CN 200610039900A CN 1838624 B CN1838624 B CN 1838624B
- Authority
- CN
- China
- Prior art keywords
- data
- packet
- board
- processing
- access board
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
- 238000012545 processing Methods 0.000 title claims description 76
- 238000003672 processing method Methods 0.000 title claims description 8
- 238000000034 method Methods 0.000 claims abstract description 10
- 230000008569 process Effects 0.000 claims abstract description 7
- 238000007726 management method Methods 0.000 claims description 6
- 238000007689 inspection Methods 0.000 claims description 3
- 230000007246 mechanism Effects 0.000 claims description 3
- 238000012360 testing method Methods 0.000 claims description 3
- 230000009471 action Effects 0.000 claims description 2
- 238000012550 audit Methods 0.000 claims description 2
- 230000015572 biosynthetic process Effects 0.000 claims description 2
- 238000012216 screening Methods 0.000 claims description 2
- 102100029368 Cytochrome P450 2C18 Human genes 0.000 abstract 2
- 101000919360 Homo sapiens Cytochrome P450 2C18 Proteins 0.000 abstract 2
- 238000005516 engineering process Methods 0.000 description 12
- 230000006870 function Effects 0.000 description 5
- 230000008901 benefit Effects 0.000 description 4
- 238000001914 filtration Methods 0.000 description 4
- 230000000875 corresponding effect Effects 0.000 description 3
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 238000013461 design Methods 0.000 description 3
- 238000010586 diagram Methods 0.000 description 3
- 230000008859 change Effects 0.000 description 2
- 238000004891 communication Methods 0.000 description 2
- 230000006855 networking Effects 0.000 description 2
- 238000004458 analytical method Methods 0.000 description 1
- 238000004883 computer application Methods 0.000 description 1
- 230000002596 correlated effect Effects 0.000 description 1
- 238000013497 data interchange Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 239000000835 fiber Substances 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 238000012552 review Methods 0.000 description 1
- 230000000153 supplemental effect Effects 0.000 description 1
- 238000003786 synthesis reaction Methods 0.000 description 1
- 230000009897 systematic effect Effects 0.000 description 1
Landscapes
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
The high-performance network data process platform system comprises: a network processor, and an exchange and support system of CPCI2.16 for system data with a CPCI 2.16 cabinet composed by a redundant power and a self-check unit for system hardware fault; wherein, selecting multiple x86 process boards of CPCI 2.16 to form the data access and pre-process board with high-speed network processor; the data process and storage board integrated with an tri-content addressable memory chip supported high-speed lookup, a coprocessor supported content pattern check, match and label, and an interface control chip of CPCI2.16; and the fault self-check board, respectively.
Description
Technical field
The present invention relates to the combination technique that high-performance hardware platform and high-speed data are handled, comprise system and processing method, by network data is unified Collecting and dealing, thereby realize the shunting equilibrium of data, belong to applications of computer network field.
Background technology
After the gigabit networking technology is constantly popularized, propose stern challenge for the product of traditional x86 framework to network processes and safety product performance demands.Safety products such as fire compartment wall based on the asic chip technology begin to occur.Safety products such as ASIC fire compartment wall carry out hardware-accelerated processing by custom-designed asic chip logic, because of its outstanding advantage that possesses at aspect of performance, be fit to very much the pattern that is applied to simple, to the processing of the higher big flow of carrier class of throughput and time delay index request.The limitation of aspects such as though the hardware security product based on the asic chip technology has advantageous advantage on performance, this technology exists R﹠D costs higher, limited flexibility system, can't support too many function, and the model change cycle is long.
NP (Network processor) can be described as between x86 and ASIC technology between the two, NP is the processor that designs for the network device processing network traffics specially, its architecture and instruction set handle scheduling algorithm for fire compartment wall packet filtering commonly used, forwarding, Hash table and special optimization has all been carried out in operation, can finish the operation commonly used of TCP/IP stack efficiently, and network traffics are carried out concurrent processing fast.Use NP development difficulty and flexibility all between ASIC and x86 framework.The main flow of COTS hardware platform was used and also to be based on the second generation and to meet cPCI2.16 standard PICMG platform today.
CN01126714.3 mass data processing method and system relate to data acquisition and the treatment technology in the computer application system, be specifically related to application systems software when handling mass data according to certain feature highly effective gathering related data and the method analyzed according to the preliminary treatment that these features are correlated with.Specifically may further comprise the steps: the source image data takes place from data in application system, in the tables of data of typing acquired original layer; Feature according to initial data is carried out the classification first time to data; Attribution rule according to data carries out secondary classification to classified data; In the tables of data with sorted data typing intermediate data layer; Application system directly from middle tables of data image data carry out further computing; The present invention carries out twice processing to data.
CN200510004126.7 data processing equipment and program and the method used therewith, in data processing equipment, when carrying out with the communicating by letter of a plurality of access request source, described data processing equipment can improve the response of communication, wherein, have only an access request source to have the mandate of writing at most, and other access request does not have the mandate of writing.Management equipment sends the strong connection request that requires to obtain to write mandate in SAM to Secure Application Module (SAM).When determining not when other management equipment is distributed strong connection request, under the weak state that is connected that keeps with described management equipment, SAM distributes strong the connection to management equipment, and wherein, described weak connection has read authority.
Constantly change in the network security threats situation, the application and the data security requirements of support constantly promote, need in a large number to use the message data content carry out analytical review the time, can cause using separately the systematic function of asic chip speed technology or NP sharply to descend.Therefore, rely on the network throughput performance of asic chip speed technology or NP linear speed merely, be not sufficient to create the linear speed safety product hardware platform of safety assurance ability requirement.
Summary of the invention
Main purpose of the present invention is to provide an effective high-speed data processing platform of unification for multiple network safety means, server cluster under the high speed network environment, this plateform system has made full use of the advantage of various hardware data processing units and has been effective combination, has realized the high efficiency of data processing under the gigabit networking environment.
Content of the present invention is achieved in that high-performance network data processing platform system and processing method, utilize technology such as existing network processor, CPCI2.16, the advantage of the abundant various technology of analysis-by-synthesis, the high-performance network data processing platform system of a practicality of structure.
General high-speed data processing platform system, form by following part: network processing unit, meet CPCI2.16 system data exchange back-up system, adopt the cabinet of CPCI2.16 to exchange back-up system as system data, comprise redundant power, system hardware fault self-checking unit, adopt polylith to support the x86 disposable plates composition data of CPCI2.16 to insert and the preliminary treatment integrated circuit board, data processing and storage integrated circuit board, system hardware fault self-checking unit, the data access board adopts the express network processor, and the Ternary Content Addressable Memory chip of the high zoom table of integrated support carries out pattern examination with supporting to content, the harmonizing processor chip of coupling and mark, CPCI2.16 interface control chip composition data is handled and the storage integrated circuit board, and based on the controller of a plurality of chip collaborative works of coordination of FPGA exploitation.Make up the high-performance network data processing platform system of a practicality.
High-performance network data processing platform system and processing method, it is characterized in that the access board of general high-speed data processing platform system is connected on enterprise's backbone network, addressable data-interface is provided, after all data that need handle converge by access board, handle balanced data processing that is transmitted to the back and storage integrated circuit board through Hash, realize the surface speed forwarding of data and the data balancing between a plurality of processing unit, and the mark of packet being done according to access board, proceed check to handle, and the packet after will handling is beamed back access board and is transmitted.
Processing method of the present invention, harmonizing processor chip, a kind of Ternary Content Addressable Memory and a kind of general hardware platform that utilization can be carried out the network processor chip of express network processing data packets, content is mated fast, the comprehensive system that forms, support 2,3,4 layers of gigabit Ethernet exchange of linear speed, a plurality of gigabit Ethernet mouths can be provided, can realize the able to programme of gigabit wire speed.Its QoS mechanism comprises grading control, and priority management and bandwidth guarantee that chip internal comprises MACs based on shared memory architecture, address search CAM, entry address table, bandwidth allocator, formation machine, switch and buffer memory etc.Described data insert and the preliminary treatment integrated circuit board, and have the gigabit wire speed access and transmit disposal ability, and integrated a plurality of kilomega optic fiber and RJ45 network interface.
System data exchange back-up system of the present invention comprises: redundant power, system data interchange channel and other supplemental support unit.
Described data processing and storage integrated circuit board, the processing unit that comprises a plurality of isomeries, they adopt different chip technology and hardware structure, can be used to finish different network data processing tasks, but, a plurality of processing unit primordials are in the cooperation group of planes of Ethernet, the data processing of application system is carried out parallel computation as required on cluster, realize load balancing, manage throughout and realize communication efficiently between the unit and handle synchronization mechanism efficiently, the data that reduce between each parallel processing element rely on.
Described data insert and pre-processed board, the function of data access and preliminary treatment (data distribution) is provided, make rational division of work between access board and the processing unit, the strong point of dealing with performance separately of two kinds of hardware of performance, improve system-computed usefulness to greatest extent, for the processing requirements of gigabit wire speed provides enough performance guarantees.The performing step of this function is as follows:
Step 601: access board inserts express network;
Step 602: all packets that need handle exchange through access board;
Step 603: access board carries out simple rule inspection and preliminary screening test to the message that receives according to the built-in linear strategy of access board, abandon the packet that does not satisfy the built-in strategy of access board, comprise various anomaly sxtructures, illegal and unsupported packet, confirm next step operation of packet;
Step 604: after 603 processing, remaining packet is handed to the rule list of access board and is handled.Rule at data on the access board comprises 4 kinds of processing modes, promptly directly abandons, directly transmits, and the corresponding module that does not process and submit to the X86 disposable plates is handled.Except needs were submitted to the packet that the X86 disposable plates handles, other all packets were according to according to processing rule, selected directly to abandon, do not handle or carried out surface speed forwarding;
Step 605: to the packet that needs are further handled, access board carries out HASH to it according to the hash algorithm that presets and calculates, load balancing be transmitted to the polylith data processing and memory plane sticks into capable follow-up work;
Step 607: data processing and memory plane stick into row further rule check, and finish action such as audit;
Step 608: after data processing and the work of storage integrated circuit board are finished, packet is beamed back access board transmit.
Described high-speed data processing platform system has higher H A function.At first, system provides real-time hardware fault selftest module, and is to send warning in fault, is convenient to fault and in time gets rid of; Secondly, adopt the Redundancy Design of power supply redundancy design and access integrated circuit board.Form a group of planes between each similar disposable plates with reciprocity computing capability, redundancy backup each other, the fault of monolithic integrated circuit board can not have influence on the normal operation of system; In addition, also can utilize the access integrated circuit board of backup to be connected, replace traditional two-node cluster hot backup mode with spare line.
Adopt the cabinet of CPCI2.16 to exchange back-up system as system data, redundant power, system hardware fault self-checking unit, the exchanges data support that provides bandwidth to reach as high as 32Gbitps are provided for it;
The data access board adopts the express network processor, and the Ternary Content Addressable Memory chip of the high zoom table of integrated support and support content is carried out harmonizing processor chip, the CPCI2.16 interface control chip of pattern examination, coupling and mark, and based on the controller of a plurality of chip collaborative works of coordination of FPGA exploitation.
This access board is connected on enterprise's backbone network, addressable data-interface externally is provided, after all data that need handle converge by access board, handle balanced data processing that is transmitted to the back and storage integrated circuit board through Hash, realize the surface speed forwarding of data and the data balancing between a plurality of processing unit.
Data processing and storage integrated circuit board adopt polylith to support the x86 disposable plates of CPCI2.16, according to the mark that access board is done packet, proceed to check and handle, and the packet after will handling are beamed back access board and are transmitted.
Description of drawings
Fig. 1 is the inside connection diagram of high-speed data processing platform system.
Fig. 2 is the mutual schematic diagram of high-speed data processing platform system hardware.
Fig. 3 carries out the flow chart of data processing for high-speed data processing platform system.
Fig. 4 is a high-speed data processing platform system hardware schematic diagram
Embodiment
The present invention will be further described below in conjunction with the drawings and specific embodiments:
As Fig. 1, whole system is handled memory cell by CPCI2.16 cabinet (comprising power supply and backup battery, exchanges data back-up system, system failure self-test unit), access and front-end processing unit (and backup units), X86 and is formed.
When the high-speed data processing platform is applied in the real network, the network interface that offers the user can be 1 (server cluster) or a plurality of (network security products), can be configured and manage the high-speed data processing platform by this interface user, this interface also is the packet access point of processing platform simultaneously, directly link to each other with the access database of data processing platform (DPP), all data that need handle are all submitted to data processing platform (DPP) by this interface.
It also can be polylith that the access database can be one, carry out status poll and switching by internal system exchanges data back-up system based on CPCI2.16, simultaneously also by CPCI2.16 with after connect X86 data processing memory plane and be connected, carry out forwarding of data and processing.
As Fig. 3, the concrete flow chart of data processing of this plateform system is as follows:
1, network packet is sent to the external interface of data processing platform (DPP);
2, insert the data processing plate data are carried out the analysis of first level, the legitimacy of judgment data, judge that content mainly comprises: whether whether packet be normal packet, be to satisfy the filtering rule that inserts the data processing plate;
If 3 packets do not satisfy the legitimacy filtering rule that inserts database, then this packet is dropped;
If 4 packets satisfy the legitimacy filtering rule that inserts database, then source address, source port, destination address, destination interface, source MAC, the target MAC (Media Access Control) address according to this packet carries out Hash for hexa-atomic group, with packet be forwarded to hash value coupling after connect on the data processing memory plane and handle;
5, connect the data processing memory plane after and receive after insert the data that database forwards, carry out corresponding data processing, and write down corresponding daily record;
6, if desired result is returned, connect the data processing memory plane after then and return results is beamed back inserted database and transmit.
Claims (1)
1. high-performance network data processing platform processing method, the access and the pre-processed board of general high-speed data processing platform system are connected on enterprise's backbone network, addressable data-interface is provided, after all data that need handle converge by access board, handle balanced data processing that is transmitted to the back and storage integrated circuit board through Hash, realize the surface speed forwarding of data and the data balancing between a plurality of processing unit, and the mark of packet being done according to access board, proceed to check and handle, and the packet after will handling is beamed back access board and is transmitted, it is characterized in that adopting the network processor chip of carrying out the express network processing data packets, the harmonizing processor chip that content is mated fast, adopt the comprehensive system that forms of a kind of Ternary Content Addressable Memory and a kind of general hardware platform, support linear speed 2,3,4 layers of gigabit Ethernet exchange, a plurality of gigabit Ethernet mouths are provided, realize the able to programme of gigabit wire speed; Its QoS mechanism comprises grading control, and priority management and bandwidth guarantee that described chip internal comprises MACs, address search CAM, entry address table, bandwidth allocator, formation machine, switch and buffer memory based on shared memory architecture;
Described data insert and pre-processed board, provide data to insert and data distribution preliminary treatment, make rational division of work between access board and the processing unit, the strong point of dealing with performance separately of two kinds of hardware of performance, for the processing requirements of gigabit wire speed provides enough performance guarantees, performing step is as follows:
Step 601: access board inserts express network;
Step 602: all packets that need handle exchange through access board;
Step 603: access board carries out simple rule inspection and preliminary screening test to the message that receives according to the built-in linear strategy of access board, abandon the packet that does not satisfy the built-in strategy of access board, comprise various anomaly sxtructures, illegal and unsupported packet, confirm next step operation of packet;
Step 604: after 603 processing, remaining packet is handed to the rule list of access board and is handled; Rule at data on the access board comprises 4 kinds of processing modes, promptly directly abandons, directly transmits, and the corresponding module that does not process and submit to the X86 disposable plates is handled; Except needs were submitted to the packet that the X86 disposable plates handles, other all packets were according to processing rule, selected directly to abandon, do not handle or carried out surface speed forwarding;
Step 605: to the packet that needs are further handled, access board carries out HASH to it according to the hash algorithm that presets and calculates, load balancing be transmitted to the polylith data processing and memory plane sticks into capable follow-up work;
Step 607: data processing and memory plane stick into further rule inspection of row, and finish the audit action;
Step 608: after data processing and the work of storage integrated circuit board are finished, packet is beamed back access board transmit.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN200610039900A CN1838624B (en) | 2006-04-26 | 2006-04-26 | High-performance network data processing platform system and processing method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN200610039900A CN1838624B (en) | 2006-04-26 | 2006-04-26 | High-performance network data processing platform system and processing method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN1838624A CN1838624A (en) | 2006-09-27 |
| CN1838624B true CN1838624B (en) | 2010-05-12 |
Family
ID=37015883
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN200610039900A Expired - Fee Related CN1838624B (en) | 2006-04-26 | 2006-04-26 | High-performance network data processing platform system and processing method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN1838624B (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| TWI527409B (en) * | 2008-05-30 | 2016-03-21 | 馬維爾國際股份有限公司 | A network processor unit and a method for a network processor unit |
| CN101707629B (en) * | 2009-11-13 | 2012-11-14 | 国网电力科学研究院 | Synchronous communication method of mirror images for self organization information of safety and stability control device of electric network |
| CN111446246B (en) * | 2016-03-07 | 2023-04-07 | 杭州海存信息技术有限公司 | Memory with data analysis function |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1637720A (en) * | 2004-01-09 | 2005-07-13 | 索尼株式会社 | Data processing device, program and method |
-
2006
- 2006-04-26 CN CN200610039900A patent/CN1838624B/en not_active Expired - Fee Related
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1637720A (en) * | 2004-01-09 | 2005-07-13 | 索尼株式会社 | Data processing device, program and method |
Non-Patent Citations (1)
| Title |
|---|
| 刘刚.基于网络处理器的千兆防火墙设计与实现.东华大学硕士学位论文.2004,第3-12,39页. * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN1838624A (en) | 2006-09-27 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN101388844B (en) | Data flow processing method and system | |
| CN100369423C (en) | Network simulation detection system and method | |
| CN104811396A (en) | Load balance (LB) method and system | |
| CN101197851B (en) | Method and system for implementing control of plane centralized type data plane distribution | |
| US9300574B2 (en) | Link aggregation emulation for virtual NICs in a cluster server | |
| CN100596351C (en) | Firewall method and system based on high-speed network data processing platform | |
| CN103368777B (en) | A kind of processing data packets plate and processing method | |
| CN102209024A (en) | Method and system of virtual machine migration | |
| US20140189094A1 (en) | Resilient duplicate link aggregation emulation | |
| JP2004213125A (en) | High-availability disk controller and failure processing method therefor, and high-availability disk subsystem | |
| CN105071994B (en) | A kind of mass data monitoring system | |
| CN100531085C (en) | Method for accessing into control address table spacing using Ethernet switch medium | |
| CN102546813A (en) | High-performance cluster computing system based on x86PC framework | |
| CN101277214A (en) | Method and system for managing blade type server | |
| CN1838624B (en) | High-performance network data processing platform system and processing method | |
| CN101217472B (en) | A modularized switch message route method | |
| CN101488101A (en) | CPCI redundancy stand-by system | |
| CN103530247B (en) | The priority concocting method of bus access between a kind of node based on multiserver | |
| CN207304610U (en) | A kind of high bit rate base band data real time processing system of remote sensing load | |
| CN106612335A (en) | Method of adopting Docker container to realize IoT (Internet of things) information exchange and communication | |
| Nooruzzaman et al. | Hyperscale data center networks with transparent HyperX architecture | |
| CN103246262B (en) | Comprehensive service system for network data analysis | |
| CN102904803B (en) | A kind of message transmitting method and equipment | |
| CN203204423U (en) | Comprehensive service system for network data analysis | |
| Liu et al. | Burstbalancer: Do less, better balance for large-scale data center traffic |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20100512 |
|
| CF01 | Termination of patent right due to non-payment of annual fee |