CN110740184B - Transaction strategy testing system based on micro-service architecture - Google Patents
Transaction strategy testing system based on micro-service architecture Download PDFInfo
- Publication number
- CN110740184B CN110740184B CN201911010054.5A CN201911010054A CN110740184B CN 110740184 B CN110740184 B CN 110740184B CN 201911010054 A CN201911010054 A CN 201911010054A CN 110740184 B CN110740184 B CN 110740184B
- Authority
- CN
- China
- Prior art keywords
- test
- service module
- strategy
- transaction
- tested
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q40/00—Finance; Insurance; Tax strategies; Processing of corporate or income taxes
- G06Q40/04—Trading; Exchange, e.g. stocks, commodities, derivatives or currency exchange
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/50—Network services
- H04L67/56—Provisioning of proxy services
- H04L67/568—Storing data temporarily at an intermediate stage, e.g. caching
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/50—Network services
- H04L67/60—Scheduling or organising the servicing of application requests, e.g. requests for application data transmissions using the analysis and optimisation of the required network resources
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Business, Economics & Management (AREA)
- Accounting & Taxation (AREA)
- Finance (AREA)
- Development Economics (AREA)
- Economics (AREA)
- Marketing (AREA)
- Strategic Management (AREA)
- Technology Law (AREA)
- Physics & Mathematics (AREA)
- General Business, Economics & Management (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Debugging And Monitoring (AREA)
Abstract
The invention discloses a transaction strategy testing system based on a micro-service architecture, which comprises: the system comprises a front-end service module, a test service module and a database service module, wherein the test service module comprises a test task scheduler and a plurality of test task executors deployed in a cluster; the front-end service module is used for receiving a plurality of strategy test requests initiated by the user terminal and sending the strategy test requests to the test service module; the test service module is used for sending each strategy test request transmitted by the front-end service module to a test task executor through the test task scheduler, and each test task executor executes a test on the transaction strategy to be tested according to the test parameter information of the transaction strategy to be tested; the database service module is used for storing the strategy information of the transaction strategy to be tested, market quotation data and the test result of each test task executor; the front-end service module is also used for outputting the test result. The invention can reduce the system coupling and meet the requirement of high-concurrency strategy test.
Description
Technical Field
The invention relates to the field of internet, in particular to a transaction strategy testing system based on a micro-service architecture.
Background
This section is intended to provide a background or context to the embodiments of the invention that are recited in the claims. The description herein is not admitted to be prior art by inclusion in this section.
As is well known, in the quantitative transaction of financial products, the investor often needs to create a transaction strategy to execute the quantitative transaction. After a transaction strategy is created, historical return test or simulated transaction test is often required to be executed on the transaction strategy, and only when a test result meets a certain condition, the transaction strategy can be put on a real disk to carry out real transaction.
At present, an existing transaction strategy testing system is based on a B/S architecture system, and cannot respond to high-concurrency strategy testing requests, so that the strategy testing efficiency is low.
Disclosure of Invention
The embodiment of the invention provides a transaction strategy testing system based on a micro-service architecture, which is used for solving the technical problem that the existing transaction strategy testing system is based on a B/S architecture system, cannot cope with high concurrent strategy testing requests and causes low strategy testing efficiency, and comprises the following components: the system comprises a front-end service module, a test service module and a database service module, wherein the test service module comprises a test task scheduler and a plurality of test task executors deployed in a cluster;
the system comprises a front-end service module, a test service module and a data processing module, wherein the front-end service module is connected with a user terminal and is used for receiving a plurality of strategy test requests initiated by the user terminal and sending the strategy test requests to the test service module, wherein each strategy test request comprises strategy information of a transaction strategy to be tested and test parameter information for executing a test on the transaction strategy to be tested;
the test service module is connected with the front-end service module and used for sending each strategy test request transmitted by the front-end service module to one test task executor through the test task scheduler, wherein each test task executor executes a test on a transaction strategy to be tested according to the test parameter information of the transaction strategy to be tested;
the database service module is respectively connected with the front-end service module and the testing service module and is used for storing strategy information of the transaction strategy to be tested, market quotation data required by each testing task executor for testing the transaction strategy to be tested and a testing result obtained by each testing task executor for testing the transaction strategy to be tested;
the front-end service module is also used for outputting a test result of the transaction strategy to be tested.
In the embodiment of the invention, based on a micro-service architecture, the front-end service function, the test service function and the database service function of a transaction strategy test system are decomposed and respectively realized by an independent front-end service module, a test service module and a database service module, after a plurality of strategy test requests initiated by a user terminal are received by the front-end service module, a plurality of test task executors are established by the test service module connected with the front-end service module, each strategy test request is sent to one test task executor to execute the test function, because the database service module is respectively connected with the front-end service module and the test service module, the front-end service module can store the strategy information of a transaction strategy to be tested in the database service module, each test task executor in the test service module can acquire the strategy information of the transaction strategy to be tested from the database module, and executing a test on the transaction strategy to be tested according to market quotation data stored in the database service module, and storing a test result in the database service module after executing the test on the transaction strategy to be tested so that the front-end service module can obtain the test result of the transaction strategy to be tested.
According to the embodiment of the invention, based on the micro-service architecture, each sub-service function of the transaction strategy test system is decomposed and realized by each independent module, so that the coupling of the system can be reduced, and more flexible service support is provided. In addition, the test service module in the embodiment of the invention adopts a plurality of test task executors deployed in a cluster to test a plurality of strategy test requests, so that the requirement of high concurrent strategy test can be met, and the transaction strategy test system has strong scalability.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts. In the drawings:
fig. 1 is a schematic diagram of a transaction policy testing system based on a microservice architecture according to an embodiment of the present invention;
FIG. 2 is an interaction diagram of a transaction policy testing system based on a micro service architecture according to an embodiment of the present invention;
fig. 3 is a schematic diagram of a message queue based on a Redis list according to an embodiment of the present invention;
FIG. 4 is a flowchart of a method for performing historical backtesting and simulated transaction testing on transaction policies, according to an embodiment of the present invention;
FIG. 5 is a schematic diagram of an alternative transaction policy testing system based on micro-service architecture according to an embodiment of the present invention;
FIG. 6 is a schematic diagram of an alternative transaction policy return system based on a microservice architecture according to an embodiment of the present invention;
FIG. 7 is a flow chart of a specific implementation of history retrieval according to an embodiment of the present invention;
fig. 8 is a flowchart illustrating an embodiment of a real-time simulated transaction test according to the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention more apparent, the embodiments of the present invention are further described in detail below with reference to the accompanying drawings. The exemplary embodiments and descriptions of the present invention are provided to explain the present invention, but not to limit the present invention.
In the description of the present specification, the terms "comprising," "including," "having," "containing," and the like are used in an open-ended fashion, i.e., to mean including, but not limited to. Reference to the description of the terms "one embodiment," "a particular embodiment," "some embodiments," "for example," etc., means that a particular feature, structure, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the application. In this specification, the schematic representations of the terms used above do not necessarily refer to the same embodiment or example. Furthermore, the particular features, structures, or characteristics described may be combined in any suitable manner in any of several embodiments or examples. The sequence of steps involved in the various embodiments is provided to illustrate the practice of the present application, and the sequence of steps is not limited thereto and can be adjusted as needed.
The embodiment of the invention provides a transaction strategy testing system based on a micro-service architecture, which is used for solving the technical problem that the existing transaction strategy testing system is based on a B/S architecture system, cannot respond to high-concurrency strategy testing requests and has low strategy testing efficiency.
Fig. 1 is a schematic diagram of a transaction policy testing system based on a microservice architecture according to an embodiment of the present invention, as shown in fig. 1, the system may include: the system comprises a front-end service module 10, a test service module 20 and a database service module 30, wherein the test service module 20 comprises a test task scheduler 201 and a plurality of test task executors 202 deployed in a cluster;
the front-end service module 10 is connected to a user terminal (omitted in fig. 1), and is configured to receive multiple policy test requests initiated by the user terminal, and send the multiple policy test requests to the test service module 20, where each policy test request includes policy information of a transaction policy to be tested, and test parameter information for performing a test on the transaction policy to be tested;
the test service module 20 is connected to the front-end service module 10, and configured to send each policy test request transmitted by the front-end service module 10 to one test task executor 202 through the test task scheduler 201, where each test task executor 202 executes a test on a transaction policy to be tested according to test parameter information of the transaction policy to be tested;
the database service module 30 is connected to the front-end service module 10 and the test service module 20, and is configured to store policy information of the transaction policy to be tested, market quotation data required by each test task executor 202 to execute a test on the transaction policy to be tested, and a test result obtained by each test task executor 202 to execute a test on the transaction policy to be tested;
the front-end service module 10 is further configured to output a test result of the transaction policy to be tested.
It should be noted that, in the embodiment of the present invention, the transaction policy to be tested may be a transaction policy created by a user locally in a user terminal through a code writing manner or a visual configuration manner. Preferably, the transaction strategy to be tested in the embodiment of the present invention may be a quantitative strategy for trading financial products (e.g., products such as stocks, funds, futures, foreign exchange, precious metals, options, etc.).
As an optional embodiment, in the embodiment of the present invention, the test service module 20 uses a Docker to construct each test task executor 202 deployed in a cluster, so that each test task executor 202 exclusively owns one Docker container. In the embodiment of the invention, the function of the test task executor 202 for executing the test on the transaction strategy is virtualized into the Docker container, so that the test services of the test task executors 202 are isolated from each other and do not affect each other, and the Docker container can be newly added with a processing node by copying and applied to the test service module 20, and the cluster deploys the test task executors 202, so that the cluster has good expandability, and the computing nodes in the cluster can be rapidly increased by copying the mirror image file.
It should be further noted that, in the embodiment of the present invention, the front-end service module, the test service module, and the database service module refer to modules that independently run each sub-service, and may be on one device or broadcast on different devices, and whether on one device or on different devices, compatibility and low-coupling connection between the modules are ensured, so as to ensure a stable and efficient operation mechanism of the entire system.
In the following, the embodiment of the present invention is described by taking an example in which the front-end service module, the test service module, and the database service module are respectively operated on different servers. Fig. 2 is an interaction schematic diagram of a transaction policy testing system based on a microservice architecture provided in an embodiment of the present invention, and as shown in fig. 2, a user may submit a transaction policy to be tested (that is, send a policy testing request) to a front-end server through an application based on a client on a user terminal or an application based on a browser access, and the front-end server stores policy information of the transaction policy to be tested in a text database of a database server. A user calls a cluster test service system (comprising a test task scheduler and a plurality of test task executors deployed in a cluster) through a front-end server, and the cluster test service system and the front-end server both follow the RESTful architecture design style, so that the interconnection between the front-end server and the cluster test service system can be ensured.
The front-end server sends the received strategy test request from the user terminal to a test task scheduler on the cluster test service system, the strategy test request is distributed to each test task executor through the test task scheduler, and after each test task executor executes test on the distributed transaction strategy to be tested, the test result is stored in a database server; and the database server returns the test result to the front-end server based on the query request of the front-end server.
As can be seen from the above, the transaction policy testing system based on micro service architecture provided in the embodiments of the present invention is implemented by the front end service module, the test service module, and the database service module, which are independent from each other, respectively, and the front end service module, the test service module, and the database service module are configured to, after receiving a plurality of policy test requests initiated by the user terminal through the front end service module, create a plurality of test task executors through the test service module connected to the front end service module, and send each policy test request to one test task executor to execute the test function, because the database service module is connected to the front end service module and the test service module, respectively, the front end service module can store policy information of the transaction policy to be tested in the database service module, and each test task executor in the test service module, the method can acquire the strategy information of the transaction strategy to be tested from the database module, test the transaction strategy to be tested according to market data stored in the database service module, and store the test result in the database service module after testing the transaction strategy to be tested so that the front-end service module can acquire the test result of the transaction strategy to be tested.
According to the transaction strategy testing system based on the micro-service architecture, provided by the embodiment of the invention, based on the micro-service architecture, each sub-service function of the transaction strategy testing system is decomposed and realized by each independent module, so that the coupling of the system can be reduced, and more flexible service support is provided. In addition, the test service module in the embodiment of the invention adopts a plurality of test task executors deployed in a cluster to test a plurality of strategy test requests, so that the requirement of high concurrent strategy test can be met, and the transaction strategy test system has strong scalability.
Although the test service module in the embodiment of the present invention tests the multiple policy test requests through the multiple test task executors deployed in the cluster, in a scenario of a high concurrent policy test request, or in a situation that the number of policy test requests received by the front-end server module exceeds the number of test task executors deployed by the test service module cluster, such a high concurrent situation may cause the test service module to be down or restarted, and thus, as an optional implementation manner, in the transaction policy test system based on the micro-service architecture provided in the embodiment of the present invention, the front-end service module 10 may further send each policy test request to the test service module 20 through a Redis message queue. Fig. 3 is a schematic diagram of a message queue based on a Redis list according to an embodiment of the present invention, and as shown in fig. 3, the message queue is implemented using the Redis list according to the embodiment of the present invention, and the Redis list is implemented using a bidirectional linked list, and head and tail nodes are stored, so that inserting and fetching elements on both sides of the head and tail of the list are very fast.
It should be noted that each test task executor deployed by the cluster in the test service module 20 occupies a lot of physical resources, and the more the number of the test task executors deployed by the cluster is, the more the resources are occupied, so that when the test execution of the transaction policy to be tested by each test task executor 202 in the test service module 20 is finished, as an optional implementation manner, in the transaction policy test system based on the micro-service architecture provided in the embodiment of the present invention, the test task scheduler 201 is further configured to release the physical resources occupied by each test task executor 202 when each test task executor 202 stores the test result of the transaction policy in the database service module 30.
Through the above embodiment, after each test task executor executes a test on a transaction policy to be tested, the test result is stored in the database server module 30, so that the front-end service module 10 directly queries the test result from the database server module 30, and therefore, the test task scheduler 201 timely releases the physical resources occupied by each test task executor 202 under the condition that each test task executor 202 stores the test result of the transaction policy in the database server module 30, so as to process other policy test requests.
Preferably, each test task executor 202 may store the test results of the transaction policy in the database service module 30 through a Kafka message queue. The embodiment of the invention writes the strategy test result into the text database by adding the message queue Kafka, thereby being capable of keeping the long-term stable performance of the system.
When each test task executor 202 executes a test on a transaction policy to be tested, because the market situation data of each tick, the transaction policy to be tested generates a test result (i.e., a policy transaction result), therefore, for one test of the transaction policy, generally, millions or even tens of millions of data are transmitted between the test task executor 202 and the database service module 30, by setting a Kafka message queue between the test task executor 202 and the database service module 30, the embodiment of the present invention not only supports high throughput, but also the test service module 20 and the database service module 30 are independent from each other, wherein the test task executor 202 only needs to be responsible for transmitting the test result of the transaction policy to the Kafka message queue; and the database service module 30 only needs to service to read the test results of each transaction policy from the Kafka message queue.
Optionally, in this embodiment of the present invention, the database service module 30 may further be configured to store the test result of the transaction policy in a text database, so that the storage of the test result data is completed in an intuitive document manner. Preferably, the text database employed may be a database based on distributed file storage, mongoDB database.
As an optional embodiment, in the transaction policy test system based on the micro service architecture provided in the embodiment of the present invention, the policy test request may be a history return test request or a simulated transaction test request; the historical return test request is used for requesting to execute historical return test on the transaction strategy to be tested based on historical market data; the simulated transaction test request is used for requesting to execute a simulated transaction test on the transaction strategy to be tested based on the real-time market quotation data, wherein when the strategy test request is a historical return test request, the trigger mode of the test task executor 202 is event trigger; when the policy test request is a simulated transaction test request, the trigger mode of the test task executor 202 is event trigger.
It should be noted that, when the test task executor performs the history retest on the transaction policy to be tested, since the data used for performing the history retest on the transaction policy to be tested is the historical market quotation data in a certain retest time period selected by the user, the start of the test task executor is triggered by an operation event (for example, the user selects the retest time period and triggers the test of the test task executor by clicking a retest button or other control), which is called event trigger; when the test task executor executes the simulated transaction test on the transaction strategy to be tested, the data used for executing the simulated transaction test on the transaction strategy to be tested is the real-time market quotation data, so that the test of the test task executor is started by triggering the simulated transaction test of the test task executor by time, which is called time triggering.
In an alternative embodiment, in the transaction policy testing system based on the microservice architecture provided in the embodiment of the present invention, the database service module 30 may be further configured to store the real-time market quotation data in a Redis database, and store the historical market quotation data in a time-series database, or store the historical market quotation data in a relational database in a partition manner.
It should be noted that, since the policy backtesting is performed on the historical market data in the accessed data backtesting time period, the policy backtesting is automatically stopped after the execution of one backtesting time period, and the simulated transaction test is performed on the real-time market data accessed by the simulated transaction, if the policy backtesting is not stopped. In order to implement automatic interruption of the simulated transaction test, in the embodiment of the present invention, in each policy test request received by the front-end service module 10, if the policy test request is a history retest request, the test parameter information may at least include a retest time period for performing history retest on the transaction policy to be tested; if the policy test request is a simulated transaction test request, the test parameter information may at least include a simulation run time length for executing a simulated transaction test on the transaction policy to be tested, so that the retest task executor 202 may stop the historical retest according to the retest time length or stop the simulated transaction test according to the simulation run time length.
In an optional embodiment, when the policy test request is a simulated transaction test request, the test task executor 202 in the test service module 20 executes a simulated transaction test on the transaction policy to be tested, and reads the real-time market quotation data directly from the Redis database.
In another optional implementation, when the policy test request is a history return test request, the test task scheduler 201 reads historical market quotation data from the time series database or the relational database according to the test parameter information of the transaction policy to be tested, and caches the historical market quotation data to the test service module 20; the test task executor 202 in the test service module 20 performs history retest on the transaction policy to be tested, and performs history retest on the transaction policy to be tested according to the history market quotation data cached in the test service module 20. By adopting a data caching mode, market quotation data required by historical return test is cached, so that the read-write operation on a database in the historical return test process can be reduced, and the test efficiency of the transaction strategy test can be improved.
In any of the above embodiments, each test task executor 202 may read market quotation data required for executing the test on the transaction policy through the RabbitMQ message queue.
Further, after reading the market quotation data of each tick, each test task executor 202 returns a confirmation message to the RabbitMQ message queue; the RabbitMQ message queue clears market quotation data of the corresponding tick according to the received confirmation message returned by each test task executor 202.
Fig. 4 is a flowchart of a method for performing a history retest and a simulated transaction test on a transaction policy according to an embodiment of the present invention, as shown in fig. 4, which mainly includes the following parts:
market quotation access:
the data engine of the policy testing system receives all real-time market data distributed by the production system data engine module. Optionally, market quotation data can be transmitted between the production system data engine and the data engine of the strategy test system through a RabbitMQ message queue.
(II) market quotation storage:
after the data engine of the policy testing system obtains the market quotation data, the real-time market quotation data (for example, the market quotation data of the latest tick) is stored in a Redis database, and the historical market quotation data is stored in the following two ways:
(1) stored to a time series database (e.g., dopinndb).
(2) The data is stored in a non-time-series database, for example, in a manner of relational database (Mysql) + partition. The partitioning principle includes two dimensions: partitioned by product ID or by time.
And (III) strategy testing:
(1) the user selects real-time simulation through a front-end interface, and simultaneously selects a strategy to be tested:
a strategy simulation transaction system (test task executor) initiates market data subscription to a data engine of the strategy test system according to a strategy to be tested selected by a front end, and the data engine of the strategy test system reads the latest data of a requested product from a Redis database after receiving a subscription request and sends the latest data to a RabbitMQ queue connected with the strategy simulation transaction system (test task executor);
(2) the user selects the historical retest through the front-end interface, and simultaneously selects the strategy and the retest date range (calendar form):
the strategy simulation transaction system (test task executor) initiates subscription of historical market data to a backtesting engine of the strategy test system according to the strategy to be tested selected by the front end and the backtesting time period of the historical backtesting, and after receiving a subscription request, the backtesting engine reads corresponding historical data from a time sequence database or a relational database, sends the historical data to a RabbitMQ queue communicated with the strategy simulation transaction system (test task executor), and temporarily caches the market data at the moment;
fourthly, the strategy simulation trading system (test task executor) reads market quotation data in the RabbitMQ queue and outputs strategy test results (for example, strategy income information);
and (V) the user checks the strategy test result through the front-end interface.
Fig. 5 is a schematic diagram of an alternative transaction policy testing system based on a micro-service architecture according to an embodiment of the present invention, where the system may be used for both historical backtesting of policies and simulated transaction testing of policies. Market quotation data are accessed into a Redis database of a database service module, and historical market quotation data are stored in a time sequence database or a relational database in a partitioning mode. After receiving the strategy test request, the front-end service module calls a test service, sends the strategy test request to a test task scheduler in the test service module through a message queue, sends the strategy test request to each test task executor deployed in the cluster through the test task scheduler, and concurrently processes a plurality of strategy test requests, and after each test task executor tests the transaction strategy to be tested, stores the test result into a text database of the database service module through a Kafka message queue.
Optionally, in order to reduce the read-write operation on the database, when the policy test request is a history retest request, the test task scheduler reads the history market quotation data from the time series database or the relational database according to the test parameter information of the transaction policy to be tested, and caches the history market quotation data to the test service module, so that the test task executor performing history retest on the transaction policy to be tested in the test service module performs history retest on the transaction policy to be tested according to the history market quotation data cached in the test service module.
Fig. 6 is a schematic diagram of an optional transaction policy review system based on a micro-service architecture according to an embodiment of the present invention, as shown in fig. 6, a user accesses a front-end service module through a user terminal, submits a policy, and clicks to start review; the front-end service module sends each strategy test request to the return test scheduler through the distributed cache Redis queue, and if the operation is overtime or the return test scheduler returns error reporting information, the front-end service module sends a prompt message of test start failure to the user terminal; if the retest scheduler returns a prompt message that the retest start is successful, the front-end service module receives the message and starts polling the database to obtain a retest result.
The backtesting scheduler creates a backtesting example according to the strategy text identification, the backtesting time interval, the transaction target equal-backtesting related parameters transmitted from the front end; and the retest scheduler reads market quotation data from the quotation database and caches the market quotation data to the retest service module according to marks such as product labels, market and date, so that the database reading operation in the retest process is reduced, and the retest efficiency is improved. The backtest scheduler creates a backtest on a backtest cluster (Docker cluster), which reads the backtest instance context data and reads market quotation data from a cache or database service module of the backtest service module.
The retest cluster executes a retest instance through a driving type batch processing engine to complete a retest process; and stores the return test results or return test reports to a text database in the database service module by Kafka.
The front-end service module scans and reads the text database at regular time and inquires the execution result of the retest instance; and performing visual rendering according to the retest result, and displaying the result to a user through a user terminal to finish the retest.
Fig. 7 is a flowchart of a specific implementation of history retrieval according to an embodiment of the present invention, as shown in fig. 7, including the following steps:
(1) and starting a retest engine, starting a thrift service, and waiting for a data subscription request of the transaction strategy to be tested.
(2) Starting a test task executor, initiating a login request, and defining a queue after receiving the login request by a retest engine as follows:
md_(senderCompID):(senderSubID)。
(3) and starting a test task executor and sending a data subscription request to the retest engine.
(4) After the backtesting engine receives the subscription request, taking the mdStreamID in the request parameters as a key binding queue:
md_(senderCompID):(senderSubID)。
(5) and the backtesting engine circularly reads the market data publish from the time sequence database to the corresponding exchange according to the time and the subscription parameters specified in the parameter configuration interface. The per-packet acknowledgement mechanism of MQs needs to be considered here.
(6) The test task executor tests the strategy, outputs a test result strategy receiving message for processing, and sends the quotation information to the simulated transaction system through the TD.
(7) And after the backtesting engine confirms that the data is sent, unbinding the key and then deleting the queue.
(8) And the test task executor tests the strategy and stores the test result into the database.
FIG. 8 is a flowchart illustrating an embodiment of a real-time simulated transaction testing process; as shown in fig. 8, the method comprises the following steps:
(1) the data engine starts a thrift service and waits for a data subscription request of a transaction policy to be tested.
(2) Starting a test task executor, initiating a login request to a data engine, and defining a queue md (sendCompID): sendSubID after the policy end receives the login request.
(3) The transaction to be tested sends a data subscription request to the data engine.
(4) After receiving the data subscription request, the data engine takes the mdstreamID in the request parameters as a key binding queue: md _ (senderCompsiD): sendersubID.
(5) Once the data engine configures the time and the subscription parameters specified in the interface according to the parameters, the data engine publishes the time and the subscription parameters to the corresponding exchange.
(6) And monitoring the mq by the test task executor, and receiving and executing a simulated transaction test on the transaction strategy to be tested once a new message exists.
(7) And the test task executor performs simulation transaction test on the strategy and stores the test result into the database.
(8) And if the test task executor cancels the subscription of certain data, the data engine unbinds the key, logs out and outputs a simulation transaction test result.
To sum up, the embodiment of the present invention provides a transaction policy testing system based on a micro-service architecture, which decomposes each sub-service function of the transaction policy testing system based on the micro-service architecture, and is implemented by each independent module, so that the coupling of the system can be reduced, and more flexible service support can be provided. In the embodiment of the invention, the test service module adopts a plurality of test task executors deployed in a cluster to test a plurality of strategy test requests, so that the requirement of high concurrent strategy test can be met, and the transaction strategy test system has strong scalability. Optionally, the Docker distributed cluster deployment is adopted, not only are the sub-services isolated from each other, but also the expansion of cluster nodes can be realized by copying container files, and the stability and high availability of the system can be ensured.
As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on a variety of computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
The above-mentioned embodiments are intended to illustrate the objects, technical solutions and advantages of the present invention in further detail, and it should be understood that the above-mentioned embodiments are only exemplary embodiments of the present invention, and are not intended to limit the scope of the present invention, and any modifications, equivalent substitutions, improvements and the like made within the spirit and principle of the present invention should be included in the scope of the present invention.
Claims (8)
1. A transaction strategy testing system based on micro service architecture is characterized by comprising: the system comprises a front-end service module, a test service module and a database service module, wherein the test service module comprises a test task scheduler and a plurality of test task executors deployed in a cluster;
the system comprises a front-end service module, a test service module and a front-end service module, wherein the front-end service module is connected with a user terminal and is used for receiving a plurality of strategy test requests initiated by the user terminal and sending the strategy test requests to the test service module, and each strategy test request comprises strategy information of a transaction strategy to be tested and test parameter information for testing the transaction strategy to be tested;
the test service module is connected with the front-end service module and used for sending each strategy test request transmitted by the front-end service module to one test task executor through a test task scheduler, wherein each test task executor executes a test on a transaction strategy to be tested according to test parameter information of the transaction strategy to be tested;
the database service module is respectively connected with the front-end service module and the testing service module and is used for storing strategy information of the transaction strategy to be tested, data required by each testing task executor to execute testing on the transaction strategy to be tested and a testing result of each testing task executor;
the front-end service module is further used for outputting a test result of the transaction policy, the test service module is a container cluster managed by a container management tool, each container in the cluster is an independent test task executor, and each test task instance monopolizes one test task executor;
the strategy test request is a history return test request or a simulation transaction test request; the historical return test request is used for requesting to execute historical return test on the transaction strategy to be tested based on historical market data; the simulated transaction test request is used for requesting to execute a simulated transaction test on a transaction strategy to be tested based on real-time market quotation data, wherein the trigger mode of the test task executor is event trigger under the condition that the strategy test request is a historical return test request; when the strategy test request is a simulated transaction test request, the trigger mode of the test task executor is time trigger;
the database service module is also used for storing the real-time market quotation data into a Redis database, and storing the historical market quotation data into a time sequence database or a relational database in a partitioning mode;
when the strategy test request is a simulated transaction test request, a test task executor which executes a simulated transaction test on the transaction strategy to be tested in the test service module directly reads real-time market quotation data from the Redis database;
under the condition that the strategy test request is a historical return test request, the test task scheduler reads historical market quotation data from the time sequence database or the relational database according to test parameter information of a transaction strategy to be tested and caches the historical market quotation data to the test service module; the test service module is used for executing the historical retest to the transaction strategy to be tested, and the test service module is used for executing the historical retest to the transaction strategy to be tested according to the historical market quotation data cached in the test service module.
2. The system of claim 1, wherein the front end service module sends respective policy test requests to the test service module through a Redis message queue.
3. The system of claim 1, wherein the test task scheduler is further configured to release physical resources occupied by each test task executor if each test task executor stores a test result of a transaction policy in the database service module.
4. The system of claim 3, wherein each test task executor stores test results of a transaction policy in the database service module via a Kafka message queue.
5. The system of claim 4, wherein the database service module is further configured to store the test results of the transaction policy in a text database.
6. The system of claim 1, wherein each test task executor reads market quotation data required to perform testing on trading strategies through a RabbitMQ message queue.
7. The system of claim 6, wherein each test task executor returns a confirmation message to the RabbitMQ message queue after reading market quotation data for each tick; and the RabbitMQ message queue clears the market quotation data of the corresponding tick according to the received confirmation message returned by each test task executor.
8. The system of claim 1, wherein the transaction policies to be tested are transaction policies created locally at the user terminal by code writing or visual configuration.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911010054.5A CN110740184B (en) | 2019-10-23 | 2019-10-23 | Transaction strategy testing system based on micro-service architecture |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911010054.5A CN110740184B (en) | 2019-10-23 | 2019-10-23 | Transaction strategy testing system based on micro-service architecture |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110740184A CN110740184A (en) | 2020-01-31 |
| CN110740184B true CN110740184B (en) | 2022-09-20 |
Family
ID=69270980
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911010054.5A Active CN110740184B (en) | 2019-10-23 | 2019-10-23 | Transaction strategy testing system based on micro-service architecture |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110740184B (en) |
Families Citing this family (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111737125B (en) * | 2020-06-19 | 2024-01-30 | 中国工商银行股份有限公司 | Method, device and server for generating quotation data of quantized transaction |
| CN113300900A (en) * | 2020-06-28 | 2021-08-24 | 阿里巴巴集团控股有限公司 | Method, device and system for testing service on cloud and method and device for testing container |
| CN112596885A (en) * | 2020-12-25 | 2021-04-02 | 网易（杭州）网络有限公司 | Task scheduling method, device, equipment and storage medium |
| CN112866256A (en) * | 2021-01-22 | 2021-05-28 | 中信银行股份有限公司 | Data processing method, device and storage medium |
| CN112905486B (en) * | 2021-03-26 | 2022-07-08 | 建信金融科技有限责任公司 | Service integration test method, device and system |
| CN114723566B (en) * | 2022-06-10 | 2022-09-02 | 高盈国际创新科技(深圳)有限公司 | Financial transaction data processing method and system |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110349022A (en) * | 2019-06-28 | 2019-10-18 | 北京奇才天下科技有限公司 | A kind of automated testing method, device and the electronic equipment of the virtual credit card transaction scene based on micro services |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9111019B2 (en) * | 2008-09-30 | 2015-08-18 | Interactive TKO, Inc. | Modeling and testing interactions between components of a software system |
| CN207490985U (en) * | 2017-09-07 | 2018-06-12 | 厦门宽投信息科技有限公司 | A kind of electronic transaction strategy test equipment |
| CN107578334B (en) * | 2017-09-07 | 2022-07-12 | 张毅 | Execution method of electronic transaction strategy and distributed transaction system |
| CN108765149B (en) * | 2018-05-11 | 2021-10-19 | 南京工程学院 | Cluster-based quantization strategy retest system and method |
-
2019
- 2019-10-23 CN CN201911010054.5A patent/CN110740184B/en active Active
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110349022A (en) * | 2019-06-28 | 2019-10-18 | 北京奇才天下科技有限公司 | A kind of automated testing method, device and the electronic equipment of the virtual credit card transaction scene based on micro services |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110740184A (en) | 2020-01-31 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110740184B (en) | Transaction strategy testing system based on micro-service architecture | |
| US8306996B2 (en) | Processing model-based commands for distributed applications | |
| US7949999B1 (en) | Providing support for multiple interface access to software services | |
| CN108874558B (en) | Message subscription method of distributed transaction, electronic device and readable storage medium | |
| US7359824B2 (en) | Systems and methods for a distributed execution environment with per-command environment management | |
| CN109471796A (en) | Interface test method, device, computer equipment and storage medium | |
| US20180113746A1 (en) | Software service execution apparatus, system, & method | |
| US20100153492A1 (en) | Systems and Methods For Intregrating Services | |
| AU2018309008B2 (en) | Writing composite objects to a data store | |
| WO2002033636A9 (en) | Apparatus, methods and articles of manufacture for constructing and executing computerized transaction processes and programs | |
| EP2098982A1 (en) | Distributed business process tracking | |
| CN111813868B (en) | Data synchronization method and device | |
| CN111367792A (en) | Test method, test device, storage medium and electronic equipment | |
| EP3069272B1 (en) | Managing job status | |
| US7152189B2 (en) | Testing distributed services by using multiple boots to timeshare a single computer | |
| US10621163B2 (en) | Tracking and reusing function results | |
| CN112825525A (en) | Method and apparatus for processing transactions | |
| US20070074225A1 (en) | Apparatus, method and computer program product providing integration environment having an integration control functionality coupled to an integration broker | |
| US11341022B2 (en) | Runtime performance introspection | |
| CN112181443B (en) | Automatic service deployment method and device and electronic equipment | |
| CN109525642B (en) | LIMS system client data automatic reporting method under user mechanism | |
| CN114153427A (en) | Optimization method and system of continuous integration assembly line | |
| CN101303751B (en) | Alternating processing method, system, and computer program product | |
| US20120296951A1 (en) | System and method to execute steps of an application function asynchronously | |
| CN114374677B (en) | Cross-platform online publishing method and device, computing equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |