CN110611624A - Massive market quotation data acceleration system and acceleration method based on FPGA - Google Patents

Massive market quotation data acceleration system and acceleration method based on FPGA Download PDF

Info

Publication number
CN110611624A
CN110611624A CN201810621561.1A CN201810621561A CN110611624A CN 110611624 A CN110611624 A CN 110611624A CN 201810621561 A CN201810621561 A CN 201810621561A CN 110611624 A CN110611624 A CN 110611624A
Authority
CN
China
Prior art keywords
fpga
data
market data
market
quotation
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201810621561.1A
Other languages
Chinese (zh)
Inventor
方逸洲
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shanghai Instrument Electric (group) Co Ltd Central Research Institute
Original Assignee
Shanghai Instrument Electric (group) Co Ltd Central Research Institute
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shanghai Instrument Electric (group) Co Ltd Central Research Institute filed Critical Shanghai Instrument Electric (group) Co Ltd Central Research Institute
Priority to CN201810621561.1A priority Critical patent/CN110611624A/en
Publication of CN110611624A publication Critical patent/CN110611624A/en
Pending legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q40/00—Finance; Insurance; Tax strategies; Processing of corporate or income taxes
    • G06Q40/04—Trading; Exchange, e.g. stocks, commodities, derivatives or currency exchange
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L49/00—Packet switching elements
    • H04L49/10—Packet switching elements characterised by the switching fabric construction
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L49/00—Packet switching elements
    • H04L49/30—Peripheral units, e.g. input or output ports
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L69/00—Network arrangements, protocols or services independent of the application payload and not provided for in the other groups of this subclass
    • H04L69/16—Implementation or adaptation of Internet protocol [IP], of transmission control protocol [TCP] or of user datagram protocol [UDP]
    • H04L69/163—In-band adaptation of TCP data exchange; In-band control procedures
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L69/00—Network arrangements, protocols or services independent of the application payload and not provided for in the other groups of this subclass
    • H04L69/22—Parsing or analysis of headers

Landscapes

  • Engineering & Computer Science (AREA)
  • Signal Processing (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Business, Economics & Management (AREA)
  • Accounting & Taxation (AREA)
  • Finance (AREA)
  • Computer Security & Cryptography (AREA)
  • Technology Law (AREA)
  • Strategic Management (AREA)
  • Physics & Mathematics (AREA)
  • General Business, Economics & Management (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Marketing (AREA)
  • Economics (AREA)
  • Development Economics (AREA)
  • Data Exchanges In Wide-Area Networks (AREA)

Abstract

The invention relates to a massive quotation data acceleration system and method based on FPGA (field programmable gate array), wherein the acceleration system comprises a trading place quotation server, a TAP (test access port) switch and an FPGA quotation data processing module, the FPGA quotation data processing module is connected with the trading place quotation server through the TAP switch, and the FPGA quotation data processing module comprises a data monitoring unit and an FPGA board card; in the acceleration method, a data monitoring unit of an FPGA market data processing module monitors market data of a TAP switch, the monitored market data is subjected to mirror image processing and then transmitted to an FPGA board card, and the FPGA board card analyzes and processes the market data and then sends the market data. Compared with the prior art, the method has the advantages of realizing the acceleration of the market quotation of the hardware, realizing the extremely-fast analysis and the like.

Description

Massive market quotation data acceleration system and acceleration method based on FPGA
Technical Field
The invention relates to a big data technology, in particular to a massive market data acceleration system and method based on an FPGA (field programmable gate array).
Background
The securities trading market is complicated and changeable instantly, the fluctuation and the profit and the loss of wind and cloud are just between the moments, and the time is more precious than the gold. Therefore, the real-time requirement of the market for the market quotation system is more and more outstanding, the market pulsation can be grasped only by obtaining the market quotation in advance, the trend wind direction is known, and the success is in the millisecond period. In real disk transaction, the speed of the market system directly determines the transaction rate of the customers and the amount of income, so most of the practical mechanism customers on the market use the market accelerating system to improve the speed of receiving the market, the existing market accelerating system is mostly realized based on a pure software system, however, the pure software mode updates the disk port data, brings a great amount of delay, is difficult to meet the market demand of real-time updating,
so how to design a faster market acceleration system allows a well-designed trading strategy to be better implemented. The required information can be made to stand out in a complicated information flow and reach the eye, and the market is the most direct market demand at present when people are fast and one step at any time and the machine is occupied first in minutes, seconds and seconds.
Disclosure of Invention
The invention aims to overcome the defects in the prior art and provide a massive market data acceleration system based on an FPGA.
The purpose of the invention can be realized by the following technical scheme:
the massive quotation data acceleration system based on the FPGA comprises a trading exchange quotation server, a TAP switch and an FPGA quotation data processing module, wherein the FPGA quotation data processing module is connected with the trading exchange quotation server through the TAP switch, and comprises a data monitoring unit and an FPGA board card.
Furthermore, the FPGA board card comprises a PHY layer, an MAC layer and a network cross-layer analysis layer which are sequentially arranged.
Furthermore, the FPGA board card further comprises a PCIE output port, and the network cross-layer analysis layer is connected with the client terminal through the PCIE output port.
Further, the network cross-layer resolution layer comprises:
the network data filter is used for grabbing a market data frame with a set network data header from the MAC layer;
and the format conversion unit is used for carrying out format conversion on the market data frame.
Furthermore, the FPGA board card is provided with a ten-gigabit network card input end.
An acceleration method implemented by using the FPGA-based massive market data acceleration system specifically comprises the following steps:
the TAP switch calls the market data from the exchange market server and releases the market data, the data monitoring unit of the FPGA market data processing module monitors the market data of the TAP switch, the monitored market data is subjected to mirror image processing and then transmitted to the FPGA board card, and the FPGA board card analyzes and processes the market data and then sends the market data.
Further, the analyzing and processing of the market data by the FPGA board specifically includes:
the market data reaches an MAC layer through a PHY layer, and a required network data frame is obtained through network cross-layer analysis and filtration;
the filtration is specifically as follows: and grabbing a market data frame with a set network data header from the MAC layer.
Compared with the prior art, the invention has the following beneficial effects:
a. the invention realizes the acceleration of mass market quotation data on hardware by arranging the FPGA market quotation data processing module;
b. the FPGA board card comprises a network cross-layer analysis layer, is characterized by effectively specifying a network data head through a network data filter, captures a required network data frame, discards an unnecessary network data frame, greatly improves the analysis rate, realizes the extremely-fast analysis of market data, and solves the problem of TCP analysis delay;
c. the FPGA market data processing module provided by the invention realizes market acceleration in a network audition mode, does not influence the existing network topology structure, and has the advantages of simple realization mode and low cost.
Drawings
FIG. 1 is a schematic structural view of the present invention;
FIG. 2 is a schematic diagram of the data capture principle employed by the present invention;
FIG. 3 is a schematic diagram of a market analysis framework based on FPGA in an embodiment of the present invention;
FIG. 4 is a Level2 market data capture analysis in an embodiment of the invention.
Detailed Description
The invention is described in detail below with reference to the figures and specific embodiments. The present embodiment is implemented on the premise of the technical solution of the present invention, and a detailed implementation manner and a specific operation process are given, but the scope of the present invention is not limited to the following embodiments.
As shown in fig. 1, the present invention provides a massive quotation data acceleration system based on an FPGA, which includes a source end 2 and an FPGA end 1, wherein the source end 1 includes a trading exchange quotation server 22 and a TAP switch 21, the FPGA end 1 includes an FPGA quotation data processing module 11, the FPGA quotation data processing module 11 is connected with the trading exchange quotation server 22 through the TAP switch 21, and the FPGA quotation data processing module includes a data monitoring unit and an FPGA board card.
The FPGA board card comprises a PHY layer, an MAC layer and a network cross-layer analysis layer which are sequentially arranged. The FPGA board card further comprises a gigabit network card input end and a PCIE output port, and the network cross-layer analysis layer is connected with the client terminal through the PCIE output port and the PCIE fast interface 12.
The network cross-layer analysis layer comprises a network data filter and a format conversion unit, the network data filter is used for capturing a market data frame with a set network data header from the MAC layer, and the format conversion unit is used for carrying out format conversion on the market data frame.
Because the market data is a typical TCP/IP packet, if the complete TCP/IP protocol stack is needed to be realized in the FPGA to solve the market data, the TCP/IP protocol stack can completely solve the market data but will bring delay of TCP analysis. It can be seen that the network packet at the MAC layer is determined for a particular market data interface before the payload (payload). So as long as a network data filter is designed in the FPGA, the filter is characterized by a specific network data header to filter the desired network frame.
The acceleration method realized by the FPGA-based massive market data acceleration system specifically comprises the following steps: the TAP switch calls the market data from the exchange market server and releases the market data, the data monitoring unit of the FPGA market data processing module monitors the market data of the TAP switch, the monitored market data is subjected to mirror image processing and then transmitted to the FPGA board card, and the FPGA board card analyzes and processes the market data and then sends the market data. The analysis treatment specifically comprises the following steps: the market data reaches an MAC layer through a PHY layer, and a required network data frame is obtained through network cross-layer analysis and filtration; the filtration is specifically as follows: and grabbing a market data frame with a set network data header from the MAC layer.
As shown in fig. 1, the source end 2 further includes a plurality of policy machines 24, and the policy machines 24 are connected to the FPGA market data processing module 11 through the TAP switch 21. The source end 2 further includes a remote operation and maintenance server 23, and the remote operation and maintenance server 23 provides a bandwidth of 1Gbit/s and is connected to the FPGA end 1.
Examples
In this embodiment, taking the analysis of the market Level2 market at the place of the exchange as an example, a specific connection analysis framework is shown in fig. 3, and based on the fast analysis principle of the market of the FPGA, the analysis is mainly realized by monitoring the network channel data of normal VSS and VDE, the original connection channel of VSS and VDE is not changed, the network topology is shown in fig. 3, a PC-a machine and an FPGA board card are connected under the same TAP switch, and it is assumed that VSS is installed on the PC-a machine to realize normal customer market service, meanwhile, network monitoring is realized by port mirroring of the TAP switch, network data sent from VDE to a VSS program is copied and sent to the input end of a tera network card of the FPGA board card.
The operation flow is as follows:
firstly, TCP connection is established between a VSS program on a PC-A and a vertical 2 market data service program VDE (dealer end), and after the connection is normal, market network data sent by the VDE arrives at a TAP switch of a dealer machine room, so that on one hand, VSS receives normal market data, and meanwhile, an FPGA board card also receives data of a mirror image port, and the data of the mirror image port are the same as the data received by VSS and arrive at the same time.
The data enters a PHY layer (PCS/PMA) of the FPGA through the SFP + to reach an MAC layer, then is analyzed in a cross-layer mode through a network, the quotation data is unpacked in parallel by utilizing the parallel processing advantages of the FPGA, the data at a disk port is unpacked, and is directly sent to a client transaction terminal through a fast interface of the DMA + PCIE, so that the extremely fast quotation data is realized.
Fig. 4 is an analysis of the MAC layer packet of the market at the upper exchange Level2, and it can be seen that the first 54 bytes of data are used for representing the network information and the address of the market source and the market publishing port of the Level 2. Accordingly, with reference to the principle of fig. 4, the market data filter characteristic values for Level2 implemented in the FPGA are:
[00:50:56:92:75:4d:00:00:5e:00:01:64:08:00:45:00:05:d0:e5:e2:40:00:37:06:d9:3b:72:50:9b:8b:0a:be:66:70:21:99:07:7d:81:48:5e:be:f1:d6:a8:7e:50:10:30:00:c4:eb:00:00]
then, when receiving the market data, each time receiving a network data packet, comparing the first 54 bytes of the network data packet with the characteristic value in the filter, if the data packet is the same, then the needed market data packet is obtained, otherwise, the data packet is discarded.
This achieves several benefits: firstly, the analysis of a TCP/IP protocol layer is saved, so that much time is saved, and the time delay of TCP decompression is reduced; secondly, can customize and customize to a certain special type market quotation data, for example only accept the mainboard market quotation, or other types of market quotations, like this, can save the unpacking of other types of market quotations, can save a lot of time again, reduced the time delay again, it is very efficient to the structure customer of the securities dealer.
The foregoing detailed description of the preferred embodiments of the invention has been presented. It should be understood that numerous modifications and variations could be devised by those skilled in the art in light of the present teachings without departing from the inventive concepts. Therefore, the technical solutions available to those skilled in the art through logic analysis, reasoning and limited experiments based on the prior art according to the concept of the present invention should be within the scope of protection defined by the claims.

Claims (7)

1. The massive quotation data acceleration system based on the FPGA is characterized by comprising a trading place quotation server, a TAP switch and an FPGA quotation data processing module, wherein the FPGA quotation data processing module is connected with the trading place quotation server through the TAP switch, and comprises a data monitoring unit and an FPGA board card.
2. The FPGA-based massive market data acceleration system according to claim 1, wherein the FPGA board card comprises a PHY layer, an MAC layer and a network cross-layer analysis layer which are sequentially arranged.
3. The FPGA-based massive market data acceleration system of claim 2, characterized in that the FPGA board card further comprises a PCIE output port, and the network cross-layer analysis layer is connected with the client terminal through the PCIE output port.
4. The FPGA-based massive market data acceleration system of claim 2, wherein the network cross-layer parsing layer comprises:
the network data filter is used for grabbing a market data frame with a set network data header from the MAC layer;
and the format conversion unit is used for carrying out format conversion on the market data frame.
5. The FPGA-based massive market data acceleration system of claim 1, characterized in that the FPGA board card is provided with a ten-gigabit network card input.
6. An acceleration method implemented by using the FPGA-based massive market data acceleration system according to claim 1, the method comprising:
the TAP switch calls the market data from the exchange market server and releases the market data, the data monitoring unit of the FPGA market data processing module monitors the market data of the TAP switch, the monitored market data is subjected to mirror image processing and then transmitted to the FPGA board card, and the FPGA board card analyzes and processes the market data and then sends the market data.
7. The acceleration method implemented by the FPGA-based massive market data acceleration system according to claim 6, wherein the analysis processing of the market data by the FPGA board card specifically comprises:
the market data reaches an MAC layer through a PHY layer, and a required network data frame is obtained through network cross-layer analysis and filtration;
the filtration is specifically as follows: and grabbing a market data frame with a set network data header from the MAC layer.
CN201810621561.1A 2018-06-15 2018-06-15 Massive market quotation data acceleration system and acceleration method based on FPGA Pending CN110611624A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201810621561.1A CN110611624A (en) 2018-06-15 2018-06-15 Massive market quotation data acceleration system and acceleration method based on FPGA

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201810621561.1A CN110611624A (en) 2018-06-15 2018-06-15 Massive market quotation data acceleration system and acceleration method based on FPGA

Publications (1)

Publication Number Publication Date
CN110611624A true CN110611624A (en) 2019-12-24

Family

ID=68888411

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201810621561.1A Pending CN110611624A (en) 2018-06-15 2018-06-15 Massive market quotation data acceleration system and acceleration method based on FPGA

Country Status (1)

Country Link
CN (1) CN110611624A (en)

Cited By (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111489258A (en) * 2020-03-31 2020-08-04 深圳华云信息***有限公司 Data processing system, method, electronic device, and storage medium
CN112333119A (en) * 2020-11-27 2021-02-05 深圳华云信息***有限公司 Data transmission system and method
CN112347020A (en) * 2020-10-26 2021-02-09 东方证券股份有限公司 FAST market analysis system and method based on CGRA
CN113671880A (en) * 2021-08-24 2021-11-19 中科亿海微电子科技（苏州）有限公司 Financial data acceleration system and method
CN114024910A (en) * 2021-10-29 2022-02-08 上海广策信息技术有限公司 Extremely-low-delay reliable communication system and method for financial transaction system
CN114328348A (en) * 2021-12-17 2022-04-12 广东浪潮智慧计算技术有限公司 FPGA acceleration board card and market data processing method thereof
CN115687708A (en) * 2022-09-19 2023-02-03 中科驭数（北京）科技有限公司 Transaction market data processing method and device and data processing board card
CN117271402A (en) * 2023-11-22 2023-12-22 中科亿海微电子科技（苏州）有限公司 FPGA-based low-latency PCIe DMA data transmission method

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP1356640A1 (en) * 2001-01-31 2003-10-29 TELDIX GmbH Modular and scalable switch and method for the distribution of fast ethernet data frames
CN103425746A (en) * 2013-07-18 2013-12-04 大连理工大学 Real-time financial index market information parallel computing method based on FPGA
CN105989539A (en) * 2015-09-22 2016-10-05 盛立金融软件开发（杭州）有限公司 Financial trading condition acquisition system and method
US9501795B1 (en) * 2010-08-23 2016-11-22 Seth Gregory Friedman Validating an electronic order transmitted over a network between a client server and an exchange server with a hardware device
CN106296397A (en) * 2015-05-26 2017-01-04 南京艾科朗克信息科技有限公司 Forward quotations add speed system and accelerated method
CN107392768A (en) * 2017-08-03 2017-11-24 武汉旷腾信息技术有限公司 A kind of futures trading system and method based on FPGA

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP1356640A1 (en) * 2001-01-31 2003-10-29 TELDIX GmbH Modular and scalable switch and method for the distribution of fast ethernet data frames
US9501795B1 (en) * 2010-08-23 2016-11-22 Seth Gregory Friedman Validating an electronic order transmitted over a network between a client server and an exchange server with a hardware device
CN103425746A (en) * 2013-07-18 2013-12-04 大连理工大学 Real-time financial index market information parallel computing method based on FPGA
CN106296397A (en) * 2015-05-26 2017-01-04 南京艾科朗克信息科技有限公司 Forward quotations add speed system and accelerated method
CN105989539A (en) * 2015-09-22 2016-10-05 盛立金融软件开发（杭州）有限公司 Financial trading condition acquisition system and method
CN107392768A (en) * 2017-08-03 2017-11-24 武汉旷腾信息技术有限公司 A kind of futures trading system and method based on FPGA

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
梁陇成等: "基于ZigBee的无线实时多通道数据包***设计与研究", 《网络安全技术与应用》, no. 03, 15 March 2015 (2015-03-15) *

Cited By (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111489258A (en) * 2020-03-31 2020-08-04 深圳华云信息***有限公司 Data processing system, method, electronic device, and storage medium
CN112347020A (en) * 2020-10-26 2021-02-09 东方证券股份有限公司 FAST market analysis system and method based on CGRA
CN112333119A (en) * 2020-11-27 2021-02-05 深圳华云信息***有限公司 Data transmission system and method
CN113671880A (en) * 2021-08-24 2021-11-19 中科亿海微电子科技（苏州）有限公司 Financial data acceleration system and method
CN114024910A (en) * 2021-10-29 2022-02-08 上海广策信息技术有限公司 Extremely-low-delay reliable communication system and method for financial transaction system
CN114328348A (en) * 2021-12-17 2022-04-12 广东浪潮智慧计算技术有限公司 FPGA acceleration board card and market data processing method thereof
CN115687708A (en) * 2022-09-19 2023-02-03 中科驭数（北京）科技有限公司 Transaction market data processing method and device and data processing board card
CN115687708B (en) * 2022-09-19 2023-08-22 中科驭数（北京）科技有限公司 Transaction quotation data processing method and device and data processing board card
CN117271402A (en) * 2023-11-22 2023-12-22 中科亿海微电子科技（苏州）有限公司 FPGA-based low-latency PCIe DMA data transmission method
CN117271402B (en) * 2023-11-22 2024-01-30 中科亿海微电子科技（苏州）有限公司 FPGA-based low-latency PCIe DMA data transmission method

Similar Documents

Publication Publication Date Title
CN110611624A (en) Massive market quotation data acceleration system and acceleration method based on FPGA
CN110517136A (en) A kind of quotation accelerated processing method and system
US8848528B1 (en) Network data flow collection and processing
CN106227780B (en) A kind of the automation screenshot evidence collecting method and system of magnanimity webpage
US20130195457A1 (en) Method and system for performing distributed deep-packet inspection
CN101163051A (en) Network card transmission speed testing system and method
CN111371640A (en) SDN controller-based traffic collection analysis method and system
CN105023185B (en) A kind of futures exchange disk mouth data real time parsing system based on FPGA
CN110838930B (en) Method and device for generating service logic topology
CN110061999A (en) A kind of network data security analysis ancillary equipment based on ZYNQ
CN101051948A (en) System and method for realizing multiple link point-to-point silent interception by port image
CN102546542A (en) Electronic system and embedded device and transit device of electronic system
WO2014015665A1 (en) Data processing method, device, and system
CN109302685B (en) Remote test data processing method for small satellite in transmitting field
CN106936648A (en) The failure monitoring method and system of a kind of IT system
CN111741007B (en) Financial business real-time monitoring system and method based on network layer message analysis
CN110297785B (en) Financial data flow control device and method based on FPGA
CN110798345A (en) Network flow monitoring and analyzing equipment based on ZYNQ
CN107181701B (en) The collection method and device of common gateway interface data
CN106230660B (en) The method and device of sFlow sampling
JP2002324014A (en) Monitor and control system
CN103067394A (en) Transmission control protocol (TCP) connecting establishment method and establishment device based on one-way data package
CN108400905B (en) Method for processing end-to-end flow analysis of distributed storage
CN114095383A (en) Network flow sampling method and system and electronic equipment
CN201369744Y (en) Terminal security device for the separation of inner net and outer net of system

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination