CN110570307A - Whole-market multi-variety fund financing management system based on intelligent strategy platform - Google Patents

Whole-market multi-variety fund financing management system based on intelligent strategy platform Download PDF

Info

Publication number
CN110570307A
CN110570307A CN201910844652.6A CN201910844652A CN110570307A CN 110570307 A CN110570307 A CN 110570307A CN 201910844652 A CN201910844652 A CN 201910844652A CN 110570307 A CN110570307 A CN 110570307A
Authority
CN
China
Prior art keywords
strategy
container
market
platform
task
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201910844652.6A
Other languages
Chinese (zh)
Other versions
CN110570307B (en
Inventor
吴永涛
邓红帅
孙涛
杨峰
刘建芳
李开太
宁俊军
梁建
毕晓建
郑达
郭信
胥善治
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Shenzhou Tongdao Intelligent Information Technology Co ltd
Original Assignee
Beijing Shenzhou Tongdao Intelligent Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Shenzhou Tongdao Intelligent Technology Co Ltd filed Critical Beijing Shenzhou Tongdao Intelligent Technology Co Ltd
Priority to CN201910844652.6A priority Critical patent/CN110570307B/en
Publication of CN110570307A publication Critical patent/CN110570307A/en
Application granted granted Critical
Publication of CN110570307B publication Critical patent/CN110570307B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06QINFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q40/00Finance; Insurance; Tax strategies; Processing of corporate or income taxes
    • G06Q40/04Trading; Exchange, e.g. stocks, commodities, derivatives or currency exchange

Abstract

The invention discloses a whole-market multi-variety financial resource management system based on an intelligent strategy platform, which comprises: the trading platform is responsible for receiving and sending a ticket reporting instruction, executing algorithm trading and carrying out a function of receiving and jointing a market ticket reporting; the data platform is responsible for service market access and production of related derivative market; the strategy platform is responsible for strategy production and strategy on-line operation with high performance, and comprises an off-line strategy production management subsystem and an on-line strategy operation management subsystem, wherein the strategy platform adopts a machine learning algorithm to calculate the weight of the capital proportion of each strategy in the combination, selects the weight of the investment combination in each period and maximizes the final income; the strategy platform comprises an online strategy container cluster supporting single-node multi-container and horizontal expansion; and in the online operation link, a CEP event processing engine is adopted and set up on an Apama container to transfer the core service.

Description

whole-market multi-variety fund financing management system based on intelligent strategy platform
Technical Field
The invention belongs to the technical field of financial information, and particularly relates to a whole-market multi-variety financial resource management system based on an intelligent strategy platform.
Technical Field
At present, the financial industry has entered the era of large asset management, and when the scale of asset management reaches the heaven volume, the competitive barrier between financial industries is broken, on one hand, different financial institutions are allowed to develop homogeneous asset management business, on the other hand, the extension of the original system business is continuously expanded, the connotation is continuously enriched, and the business cooperativity between financial license plates is obviously enhanced. Asset management is typically a knowledge-intensive, talent-intensive industry, highly dependent on human capital with expertise and capabilities. The financial information system service relates to wide scope, strong specialization and complex structure, and the exterior needs to get through the barriers among links of different fields, different institutions and different values; the inner part needs to open barriers among a product interface, a business interface and an organization interface; in terms of products and services, competitive products need to be designed, products and customers with different risk degrees need to be priced differently, and risks with different types, attributes and characteristics need to be scientifically and effectively segmented and recombined to achieve reasonable matching between the risks and benefits. The processing, analysis and decision of mass financial data are no longer accessible by pure manpower, the financial industry needs more intelligent and automatic information tools, dependence on manpower is reduced, and efficiency is improved. Therefore, a need exists for an advanced, safe and efficient information technology system that closely integrates with the overall process of asset management to lead the innovation of the asset management industry.
Disclosure of Invention
The invention aims to provide a whole-market multi-variety financial resource management system based on an intelligent strategy platform, which is used as an intelligent quantitative transaction platform and can be used for acquiring, processing and storing financial market data, producing, retesting, optimizing and storing strategies, arranging, deploying, operating, controlling and managing strategy instances, further obtaining transaction target, transaction signals and excellent net value curves and realizing full-dimensional intelligent resource management service.
The technical scheme is as follows:
A full market multi-variety financial asset management system based on an intelligent policy platform, comprising:
The trading platform is responsible for receiving and sending a ticket reporting instruction, executing algorithm trading and carrying out a function of receiving and jointing a market ticket reporting;
The data platform is responsible for service market access and production of related derivative market;
the strategy platform is responsible for strategy production and strategy on-line operation with high performance, and comprises an off-line strategy production management subsystem and an on-line strategy operation management subsystem, wherein the strategy platform adopts a machine learning algorithm to calculate the weight of the capital proportion of each strategy in the combination, selects the weight of the investment combination in each period and maximizes the final income; the strategy platform comprises an online strategy container cluster supporting single-node multi-container and horizontal expansion; and in the online operation link, a CEP event processing engine is adopted and set up on an Apama container to transfer the core service.
Preferably, the offline policy production management subsystem comprises 3 sub-modules: the strategy production module is used for realizing the functions of flow control, report display and list query; the strategy retest optimization module is used for realizing the establishment and the operation of a strategy retest task, the scheduling of a parallel computing task and the injection of a strategy into a retest container; the test-back container module is used for operating the strategy, sending quotation to the strategy, calculating fund and position taking when operating the strategy and providing strategy API common strategy calling during test-back; and during the operation of the online strategy operation management subsystem, the factor calculation module establishes network link with the test container, receives strategy signals from the test container, calculates strategy evaluation factors and evaluation single curves according to the signals, and calculates comprehensive evaluation comprehensive curves of the strategies on a plurality of varieties after all strategy instances are operated.
Preferably, the strategy platform comprises a product module, an option module, a factor module and a strategy-api module, the interaction principle of each module is that an upstream module calls a downstream module to adopt dobbo service, and the downstream module informs the upstream module to adopt an MQ mode; the product module is responsible for image evaluation main flow control, service query and metadata management, and realizes retest by calling option through dubbo when image retest is involved in the image evaluation process; the optimization is responsible for backtesting, optimizing and scheduling backtesting tasks of the image and the strategy, and the backtesting time needs to acquire a container, inject the strategy and send a strategy initial instruction by calling an infrastructure through a dubbo; the Infrastructure is responsible for interacting with the policy container, including the management of the online policy container and the offline policy container; the stream-api is responsible for strategy online and online strategy start and stop; the factor is responsible for receiving orders from the offline container corrlator during retest, calculating strategy factors and scores, and informing the option or the product module of completion of calculation by sending MQ messages.
Preferably, the policy platform comprises a callback container cluster: the cluster comprises 6 machines, each node is provided with a container process, and horizontal expansion is supported; the policy platform includes a factor computation cluster: and (3) single-machine multi-instance deployment, wherein one of the single-machine multi-instance deployment deploys 4 services, the other one deploys 2 services, and the number of the services and the number of the callback containers are 1: 1.
Preferably, the image evaluation process of the policy platform: the Manager receives an image evaluation task starting request, calls the image evaluation task starting request through the dubbo and transmits the image evaluation task starting request to the product module; the product starts image evaluation and optimizes step by step, and each image of each layer needs to call option for retesting; and the Optize receives the image retest request, creates an image retest task, realizes parameter opening through permutation and combination, and starts the retest task.
Preferably, the task of starting the test-back of the strategy platform comprises: acquiring available strategy container resources, injecting a strategy into a container, and sending a starting instruction to the container; when the strategy in the container return test enters into operation, an order signal is sent to a Factor, and the Factor receives the order and starts to calculate the strategy performance; after the Factor finishes calculating the strategy performance, a measurement completion notice is sent back to the Optize, and the Optize carries out task ending processing according to the notice, wherein the task ending processing comprises task body modification and container resource release.
Preferably, the policy container asynchronous event-driven process of the policy platform: if the environment is a survey environment, directly reading the local Sqlite file sending market through a monitor inside the container; if the container is an online container, subscribing market information from the Data-Source; asynchronous event delivery: the quotation/signal is circulated in the strategy container in an asynchronous event mode; the online strategy supports the complete Bar, CurrentBar, Tick full-period and full-variety market subscription and drive; reading historical market data in the strategy by adopting local memory cache to reduce a large amount of IO operations, and compressing the operation time of the strategy to microsecond level; after the strategy in the container generates signals, whether the container is a retest or an online container is judged according to environment variables, if the container is a retest task, the signals are sent to a Factor module for performance calculation, and if the container is an online container, the signals are directly sent to a trading module and then forwarded to a trading market.
Preferably, the solution of under-fitting: carrying out retest aiming at the whole market, selecting a variety with better fluidity to carry out retest, and avoiding the problem of under-fitting through image evaluation and strategy screening; solution of overfitting: in the image evaluation process, the maximum layer number of the strategy image is limited to 3 levels, a plain parameter and a limiting parameter island are adopted, in addition, the separate test of the inside and the outside of a sample is adopted, the strategy is trained in the sample, and the test of the outside of the sample is used for verifying the effectiveness of the strategy generated in the sample.
Preferably, the automated production process of the strategy is as follows: and creating a total scheduling task in the product, receiving page data, storing the page data into mongo, starting dimension element arrangement, sending an MQ notification to the product module after the task is successful, and automatically constructing the whole task to be stopped if a certain step fails to be executed.
Drawings
FIG. 1 is a simplified logical architecture diagram of a muji system.
FIG. 2 is a schematic diagram of the physical architectural deployment of the on-line run portion of the muji system.
Fig. 3 is a transaction statement flow diagram.
Fig. 4 is a flow chart of moving bins and changing months.
fig. 5, 6 and 7 are the flow charts of the bill-in, bill-withdrawing and bill-checking of the transaction micro-service subsystem, respectively.
Fig. 8 is a flow chart of bin difference automatic monitoring.
FIG. 9 is an algorithm sheet process flow diagram.
FIG. 10 is a data platform hardware deployment architecture diagram.
FIG. 11 is a functional flow diagram of a data platform.
FIG. 12 is a business flow diagram of a data platform.
FIG. 13 is a logical block diagram of a policy platform.
FIG. 14 is a timing diagram of image evaluation of a policy platform.
FIG. 15 is a policy container-asynchronous event driven flow diagram.
FIG. 16 is a policy platform parallel computing task scheduling schematic.
FIG. 17 is a policy screening flow diagram.
FIG. 18 is a policy automation production flow diagram.
FIG. 19 is a diagram of a strategic production parallel computing architecture.
FIG. 20 is a policy run linear extension architecture diagram.
Detailed Description
in order to make those skilled in the art better understand the technical solution of the present invention, the following describes in detail a full market multi-item fund financing system based on an intelligent policy platform according to an embodiment of the present invention. The following examples are intended to illustrate the invention only and are not intended to limit the scope of the invention.
The whole-market multi-variety fund financing management system based on the intelligent strategy platform is named as a wooden chicken (muji) system in the embodiment. The market-wide concept includes domestic and foreign markets, and the trading of a wide variety of financial products including, but not limited to, foreign exchange, futures, stocks, bonds, and digital currency products.
as shown in fig. 1, the woodchicken (muji) system is mainly composed of three parts: the trading platform is responsible for receiving and sending a ticket reporting instruction, executing algorithm trading and carrying out a function of receiving and jointing a market ticket reporting; the data platform is responsible for service market access and production of related derivative market; and the strategy platform is responsible for strategy production and high-performance strategy online operation. As shown in fig. 2, a CEP event processing engine is used in the whole online operation link of the wooden chicken, and is built on an Apama container to transfer the core service. The data platform provides market pushing service and a basic surface data/historical market query function; the strategy online container receives the market data, and the strategy carries out the execution of strategy idea based on market events to generate transaction signals; when the transaction container receives the transaction signal, a series of logic processing is carried out, such as generating a master order, splitting the order by an algorithm, and finally reporting the order to the market; after the market is committed, a return event is pushed to a transaction channel, and then the return event enters a transaction container to maintain order data; and finally, the calculation related to position taking and fund is completed. Table 1 shows the development environment and the operating environment of the woodchicken (muji) system.
Table 1
I. A transaction platform: the transaction platform plays multiple roles of transaction signal processing, algorithm order splitting, market docking, fund calculation and risk control in the wooden chicken system. And the online transaction part of the whole wooden chicken system is supported, and the core functions of algorithm transaction, risk control and market routing distribution are completed. High safety, namely fund safety, data safety, strategy safety and system safety; high availability, namely avoiding single point of failure, realizing isolation of accounts and isolation of strategies; high performance, performance of data acquisition, performance of data processing, performance of policy execution, high concurrency; high accuracy, namely the accuracy of data processing, the accuracy of transaction signals and the accuracy of fund management; the system has high stability, no fault exists when the system runs for a long time, and the system can be recovered in a very short time through a perfect fault recovery mechanism if the system runs for a long time. Supporting the whole market, namely supporting the online transaction of the financial targets of the domestic market and the overseas market; the whole variety supports the multi-level financial transaction targets of stocks, futures, options, bonds, foreign exchange, funds and the like; based on artificial intelligence technology, all transaction processes are completed by adopting a mode of combining programmed transaction and algorithm transaction; the method has the advantages of full fund, free adaptation to products with any fund amount, output of excellent net value curve and realization of the goal of managing value-added service. Table 2 shows the modules of the trading platform.
Table 2
Transaction nouns:
Example of the strategy: a policy instance is the specific execution of a policy (policy combination), and is essentially a diversified combination of policy subunit instances, which comprises three states of to-be-started, run and terminated.
And (4) trading units, namely the whole process from building the bins to emptying the bins of a single trading variety in a strategy (strategy combination) example is called as a trading unit, and the middle process comprises operations of adding the bins, reducing the bins and the like.
And (4) moving warehouses and changing months, namely after the futures main power contracts are switched, closing warehouses for contracts in the wooden chicken system, and carrying out the process of opening warehouses of old contracts and new contracts.
And the strategy bin is formed by accumulating the positions theoretically held by the strategy examples according to the strategy signals.
And the real disk bin is used for calculating the position actually held by the current strategy instance according to the accumulation of the sub-list return reports.
The bin difference is the difference value between the strategy bin and the real disk bin, and the bin difference is the strategy bin and the real disk bin; when the bin difference is positive, the sent entrusting instruction is consistent with the strategy signal instruction in direction; and when the bin difference is negative, the issued entrusting instruction is opposite to the strategy signal instruction in direction.
And (4) finishing the algorithm list, namely that the mother list is in a finished state, and all the son lists have returns (including successful transaction, successful list withdrawal or rejected lists).
and automatic alignment refers to the process of executing transaction signals, and the position of the real disk bin and the position of the strategy bin are automatically aligned.
Non-automatic alignment corresponds to an automatic alignment task, as distinguished from manual alignment. Refers to a transaction mode that adopts an algorithm to perform automatic entrusted transaction without bin alignment.
Manual alignment, which is distinguished from automatic transaction (including automatic alignment and non-automatic alignment), refers to a transaction behavior in which manual intervention is used to make a policy bin and a real bin consistent. The manual alignment function is active in the policy instance running state (both in-service and suspended).
and (4) a price-tracking order, namely, after the single-period time of the entrustment is not committed, the entrustment order is removed, a new entrustment price is adopted to re-report the order (tracking order), and price-tracking orders, price-tracking orders and latest price tracking orders can be further classified according to the price newly reported into price-tracking orders, price-tracking orders and latest price tracking orders. Sometimes the consideration is also referred to as an opponent's consideration.
Price-seeking: if a purchase is to be made, the price chase is made using a sales price.
price hanging and order tracing: if the purchase is to be made, the purchase price is used for buying the additional price.
Latest price tracking: the latest price tracking is bought or sold using the latest price (LastPrice).
Price limit instruction: the order is reported in a limited price instruction specified by the trade, and the order is required to specify the order price.
market price instruction: the market price specified by the exchange is reported, the entrusting order does not need to specify the entrusting price, and the exchange system can automatically convert the market price instruction into the price of the fluctuation and drop board in the entrusting direction to participate in the transaction. Buying the market price list will be converted into buying at stop price and selling the market price list will be converted into selling at stop price.
FOK: if all deals are done immediately, or else, the deals are automatically cancelled, that is, the deals must be done at the appointed price and the entrusted quantity, otherwise, the deals are automatically cancelled by the system. The result is either a total deal or a total withdrawal.
FAK: the remaining amount of the immediate deal is automatically withdrawn, which means that the deal is made at the designated price and the remaining order is automatically withdrawn by the system.
And (4) transaction day: the future in China currently follows a daily settlement system, except for the gold institute, the market is closed at 15:00 pm, the market is closed at 15:15 of the gold institute, and settlement is performed after the market is closed. The trading day refers to the trading time period belonging to the same settlement, and the night trading day and the natural day are different. Such as a night disk on a 5-month 30-day (thursday) evening, which would be attributed a transaction day of 5-month 31.
Bin all the year round: taking the position after opening the position on the current trading day. Yesterhouse: taken position opened on a non-current trading day. Plain and current: and (5) leveling the taken position opened on the same day.
Flat yesterday: the taken positions opened on the non-current day are leveled, the current domestic futures major businessmen and Zheng businessmen, and the gold institute cannot distinguish the previous taken positions, and the yesterday taken positions can be leveled off by using the leveled taken positions. The previous place and the previous place can distinguish the previous bin from the previous bin, and the previous bin must be leveled until the previous bin is leveled.
The insurance types are classified into hedging value (short for hedging), arbitrage and speculation, and generally, the transactions belong to speculation; the hedging is mainly used for production and operation of some enterprises; arbitrage is mainly used for arbitrage trading strategies of some institutions, and some trading exchanges (such as gold) can relax the supervision requirements of maximum withdrawal number for arbitrage traders.
The minimum change unit of the contract price refers to the point number of the minimum change of each price, for example, the minimum change unit of the corn is 1, the minimum change unit of the coke is 0.5, and the minimum change unit of the IF is 0.2.
Hop count: for the smallest unit of variation, for example, 5 hops for corn corresponds to a price of 5 × 1 — 5; the price of coke 5 jump is 5 x 0.5-2.5; the IF5 hops correspond to a price of 1.
In the future market, traders can participate in the buying and selling of the futures contract only by paying a small amount of funds as a financial guarantee for fulfilling the futures contract according to a certain ratio of the price of the futures contract, and the funds are the futures deposit.
And (4) freezing the deposit, namely when the opening entrusting order reports that the deposit has not been committed, the collected deposit is called the freezing deposit.
As shown in fig. 3, the transaction statement flow of the transaction container in the transaction platform is as follows: s1: after the transaction signal reaches the transaction container, signal initialization and strategy bin association are realized through logic processing of a signal management module; s2: after the signals are dequeued from the signal management module, the signals enter an order management module to be filtered and processed to generate a master order; s3: the master list enters the algorithm module, is routed through the algorithm, firstly is processed by the preposed processing logic, then is routed to the corresponding list splitting algorithm for splitting the list, and generates a list splitting end signal for the master list after all the hand numbers are split to the algorithm post-module; s4: the sub-sheets generated in the sheet removing process are directly sent to a fund checking module to carry out wind planning and fund checking logic, and the sub-sheets are sent to a market route after the checking is passed, otherwise, the fund checking rejection is carried out; s5: the transaction routing module defaults to send a request to a transaction channel gateway in a java plug-in form and sends the request to a corresponding transaction market through a route; s6: the return sent back from the market is firstly received by a client end arranged in a trading channel, then basic service of container management is called to send the return back to a trading container, the return entering the trading container is received by a sub-order state monitor and triggers corresponding state circulation, the state processing of closing order/rejecting order is carried out, and finally fund calculation and the state propulsion of the mother order are triggered.
In the transaction report flow S2 of the transaction container in the transaction platform, the order management module performs three-layer filtering: processing the signals of the type under the master force mapping to obtain a corresponding master force contract; aligning the bin, and processing the logic of automatic bin alignment or non-automatic bin alignment to correct the number of reported single hands; and (4) if the strong flat type secondary confirmation is not consistent with the bin position of the real disk, reinitializing the number of reported hands and finally generating a mother list. After the transaction statement process S6 of the transaction container in the transaction platform, the transaction report triggers the fund calculation: and calculating layer by layer in the form of java plug-ins, namely funds of position taking and position taking dimensions of the strategy instance dimension, then funds of the strategy instance dimension and finally funds of the account dimension. After the transaction report process S6 of the transaction container in the transaction platform, the state of the master bill is promoted after the corresponding action of the master bill is triggered, and if the master bill is finished, the result is fed back to the signal management module to complete the promotion of the signal state.
A transaction signal received by a transaction container in a transaction platform is initially INIT after being put in a storage of a signal management module of the container; each transaction signal after entering the warehouse is positioned in a logic queue, and the logic main key of the queue is stratInstId + symbol + position direction; after the signals with non-trading time or high priority are enqueued, part of the signals are directly pushed to the UNDO state; combining if multiple identical signal types are at INIT, the combined signal being pushed to merge; the signal is normally dequeued and executed, and the state flow is transferred to the wait algorithm for execution; the signal enters the algorithm to execute the statement splitting, and the state is changed into RUNNING; after the signal in the RUNNING state is terminated by the signal with high priority level, the signal is converted into UNDO; advancing the abnormal condition to UNKNOW; under normal conditions, the signal execution is changed to DONE.
As shown in fig. 4, the warehouse-moving and month-changing process of the trading platform is as follows: s1: the transaction container monitors a moving warehouse and changing month signal S2 sent by the transaction timing task system: if the execution signal with low priority exists, the bin shifting and moon changing signal can stop the execution signal with lower priority than the execution signal, and then enters algorithm execution after receiving the termination report, otherwise, directly enters the algorithm. S3: after algorithm transaction is entered, algorithm calculation is started, and if the average old hand number is 0, the strategy bin is directly updated to be a new mastery contract; if the average old number of hands is more than 0, sending an invoice according to the buying and selling quantity and the price, and establishing monitoring on the current order return. After the warehouse moving and month changing process S3 of the trading platform, if the return is successful, updating the transaction record; if the report fails, sending a request for removing the order; and if the report is rejected, triggering a wind control alarm monitoring mechanism and sending an alarm signal.
Fig. 5, 6 and 7 are the flow charts of the bill-in, bill-withdrawing and bill-checking of the transaction micro-service subsystem, respectively. The transaction platform comprises a transaction micro-service subsystem, and the order reporting process comprises the following steps: s1: the client side with signals in the market routing module of the transaction container dynamically routes the request to the corresponding transaction channel routing gateway; s2: the channel route receives the report request, carries out secondary service routing and forwards the request to a CTP channel or an IB channel; s3: after receiving the request, the corresponding channel firstly checks whether the channel is established, if not, the channel is initialized and established; s4: checking the login state of the user, and if the user does not log in, re-logging in; s5: converting parameters to parameters required by a corresponding market and converting the report request to redis to trigger asynchronous database dropping; s6: finally, the request is sent to the market and waits for the subsequent return of the market. After the report process S6 of the transaction micro-service subsystem receives the market report, the order information is firstly obtained from redis, then the event corresponding to the transaction container is constructed, and the basic service of container management is called to send the report to the transaction container.
The order removing operation of the transaction micro-service subsystem comprises the steps that a transaction container is not committed to trigger an order removing, or a manual order is manually removed by a page, or a timing order removing after a disk, and the order removing process comprises the following steps: s1: after receiving the order removing request, the OTC forwards the request to a corresponding CTP channel or IB channel according to the corresponding brokerID; s2: performing data package of the order removal after parameter verification; s3: checking the login state of the user, and if the user does not log in, re-logging in; s4: after the account login connection is obtained, a bill removing request is directly sent; s5: and after synchronously collecting the receipt withdrawal report, carrying out subsequent state change.
The transaction micro-service subsystem triggers the order checking service when the order is sent out or removed, and the order checking process comprises the following steps: s1: firstly, after receiving a ticket checking request, the OTC forwards the request to a CTP channel or an IB channel according to a corresponding order number; s2: assembling necessary parameters of the order checking request from the cache and the database; s3: checking the login state of the user, and if the user does not log in, re-logging in; s4: and returning the query result.
The suspension and resumption of the policy instance of the transaction microservice subsystem may be interfaced by a back-end manager. If the function is pause: firstly, the cocker management platform initiates a pause event request, the state of the strategy instance is verified through the transaction API service, and the verification is sent to a transaction container through the basic API service. The transaction container performs corresponding business execution according to the function whether the front end selects the automatic leveling function or not: if auto-leveling is selected, a turbo-leveling process is initiated. If auto-leveling is not selected, the policy instance is suspended directly. If the function is the recovery function: firstly, a recovery event request is initiated by a wooden chicken management platform, and the request is sent to a container to start operations such as moving a warehouse, changing a month and the like through transaction API service, basic API service and the like.
The bin monitoring operation of the transaction micro-service subsystem is as follows: obtaining a bin difference by obtaining strategy bin data; aligning the bin by acquiring available trading containers and sending trading signals; the bin difference reason is obtained by obtaining non-deals or partial deals by time range.
The method comprises the following steps that a wind control system of a trading platform monitors four risk conditions outside a supervision index range in real time in the execution process of each strategy instance signal: there are real-time signals, four types per minute, hour, day: the wind control system checks the opening list, and if the abnormal behavior supervision indexes of the instance to which the opening list belongs interfere, the report is refused; the 2-minute timed task and the return of the deal in the disk trigger a wind-raising risk calculation interface of a strategy instance and an account, calculate a wind-raising risk level, and if the strategy instance level wind-raising risk level achieves intervention, call a trade-api interface to suspend and level the strategy instance; calling a wind control plug-in before the order is reported by the transaction container to calculate whether the dimension available wind is sufficient or not, and if not, directly rejecting the order; the timed task system scans the policy instances and the account dimension risk index monitoring table, and alarms risk items when the policy instances and the account dimensions reach reminding, early warning and intervention levels; and the wind control system accumulates the early warning values of the strategy instance and the account dimensionality on the withdrawal order entrusted return and the closing return, and triggers an alarm after reaching the early warning level.
The transaction platform comprises a timing task subsystem, the timing task can make daily settlement at 8 o' clock 35 before the disk is loaded every night, and the daily settlement is mainly used for counting various fund data and guaranteeing normal transaction of the next transaction day. Because the counter does not give the daily statement and cannot check the account in a certain day, the daily statement fails in the day, but the daily statement is automatically supplemented by the task in the following period and is completed correctly. The process is as follows: s1: in order to ensure the performance of daily knot, the account is divided into pieces, and the pieces are used as an entrance for daily knot; s2: inquiring all policy instances of non-INIT states under the account as daily ending conditions; s3: combining the position taking record of the day before success and the transaction record of the day by the strategy example day, and making a matching composite disc to obtain position taking data and strategy example fund data; s4: then updating the fund and the position taken by the double-disk to a database, calculating and updating various funds of the account, and recording corresponding fund snapshots after all the funds are calculated; s5: finally, carrying out daily cleaning. The commission fee, the balance of the warehouse, the profit and the loss and the like are all cleared, and the fund of the next transaction day is ensured to be normal.
The muji system carries out route forwarding of global market counters through a uniform channel gateway. The password of the fund account is stored in a database in a symmetric encryption mode. The online transaction process monitoring of the transaction platform comprises a trade-server monitoring device, a trade-schedule monitoring device, a trade-risk-server monitoring device, a futures transaction container, a futures online container, a foreign exchange transaction container, a foreign exchange online container and the like. The transaction platform adopts a log4j2 asynchronous log printing mode, log monitoring alarm contents comprise ctp client monitoring (ctp interface timeout and system abnormity), order management alarm, algorithm transaction alarm, snapshot data alarm, transaction unit processing abnormity, database monitoring, user non-login, all forward normal message notification, self-transaction times, frequent order reporting and withdrawing times, large order reporting and withdrawing times and wind research.
And II, a data platform, as shown in FIG. 11, recording global market quotation data in real time, synthesizing K lines and indexes through stream processing, pushing real-time quotations in real time, providing a data service interface, and cleaning and processing historical quotation data.
As shown in fig. 10, the flow of implementing the multi-path CTP market quotation access processing by the data platform is as follows: basic market data are accessed from a data source, and different network lines are used by a receiving client, so that the stability of the data is ensured to the maximum extent; each client sends the received quotations to a quotation processing service through the netty; monitoring whether each server side has quotations or not, and if one or more client sides have no quotations, sending alarm information; monitoring basic plane information sent by a server, judging whether the basic plane information is consistent or not, and sending alarm information if the basic plane information is inconsistent; and correspondingly processing the given alarm information, such as processing the client application problem, replacing the front of the futures company and the like. And optimizing the market quotations of the market quotation data received from different clients according to the time priority and traffic priority principle, and performing subsequent market quotation processing by using the optimized market quotations through the data platform.
Access of IB quotations: and starting two IB gateways to acquire market data, wherein the two IB-clients respectively receive the market data of the two gateways, perform market screening in the market processing service, and select the market of the main market source to perform subsequent K line synthesis and other processing. IB multi-activity market monitoring process: the timing task is 1 time per second, the time of avoiding the opening of the disk and the time of starting the application are considered, when the main market source or the standby market source exceeds 5 seconds and has no market, the alarm is given, and the alarm control frequency is controlled (only one alarm is given in five minutes); if the main quotation source is abnormal and the standby quotation source is normal, the quotation sources need to be switched, the main quotation source in the cache is set as the code of the standby quotation source, and the code of the main quotation source is stored in the cache (redis) after switching.
The real-time K line synthesis process comprises the following steps: receiving the optimized Tick quotation, checking the transaction time, performing the following steps at the transaction time, sending Tick data to mq according to whether the Tick data is processed by Zheng businessmen, acquiring the Tick of the previous cache, acquiring a K line set in the cache, synthesizing low-frequency K lines according to the low-frequency K line period, synthesizing high-frequency K lines according to the high-frequency K line period, asynchronously warehousing the Tick data, and updating a redis cache K line set. A real-time K line segmentation process: executing processing every minute by a timing task, acquiring contract information, segmenting K lines in a cache according to whether the time for segmenting the K lines is reached, traversing subscribed contracts, executing a subsequent K line segmenting program in a multithread mode, acquiring contract transaction time periods, acquiring K line set cache corresponding to the contracts, calculating accumulated transaction time, traversing K line periods, judging whether the K lines are required to be segmented, backing up the K lines in the cache if the K lines are required to be segmented, emptying the cache, asynchronously warehousing the K lines, and pushing the K lines to a policy container.
And (3) market subscription process: storing the subscription push relationship between the contract and the container by using a td _ describe _ relation table; initializing the relationship between a subscription contract loaded from a newly-built server and a container ip and port; monitoring subscription information through mq after starting; after receiving the subscription message, maintaining the subscription relationship (including subscription and unsubscription) in the memory; persisting the subscription relationship to the database. And a market pushing process: judging whether to push processInBar or not when synthesizing the K line; cutting the K line to judge whether the bar is pushed; and if the bar needs to be pushed, pushing the bar to the target container through the tcp connection according to the subscription relation.
And (3) index synthesis process: inquiring contract variety data; receiving tick quotations from mq, judging whether a first tick is present or not, and setting the timing time t1 as a timestamp of the tick if the first tick is present; comparing the words not in the first stroke with the timing time t 1; if the difference is greater than 500ms, generating an index tick according to the cached tick data; caching tick data, synthesizing a K line according to an index contract tick, warehousing the K line data, and pushing the K line data to a strategy platform.
the processing flow of the global market quotation of the data platform is as follows: accessing basic market data of various markets around the world from different data suppliers; pushing the basic quotation data to an mq cluster, and landing the basic quotation data; each market processing application subscribes basic market data and processed market data from the mq cluster to perform market processing, such as variety index synthesis, cross-season, cross-variety arbitrage market processing, cross-market derived data processing and the like; and pushing the processed market data to the mq cluster, and landing the processed market data.
The derived data reflects the change rule between indexes of two or more varieties or basic contract quotations, and the derived data processing aims to provide market support for arbitrage trading and mainly comprises three forms: cross-term arbitrage derivative data: reflecting the change of price gaps among different delivery periods of the same commodity in the same market; cross-breed arbitrage derived data: reflecting the variation of price difference between different commodity varieties in the same market (generally referring to varieties with relativity, such as corn and corn starch, soybean oil, soybean meal and the like); cross-market arbitrage derivative data: reflecting the change of price gaps of the same commodity variety in different markets, wherein the data platform derivative data processing flow is as follows: creating a derived data definition, warehousing the derived data definition, calculating a public transaction time period, subscribing a basic contract contained in the derived data definition in an mq cluster, acquiring a basic contract tick quotation, calculating a price difference by using a rule engine Aviator according to a calculation formula in the derived data definition, generating a derived data tick, combining the derived data tick with a K line, cutting the K line by the derived data tick, and pushing the K line to a strategy container to drive the strategy to operate.
Historical data cleaning process: loading a cleaning target after starting, and inquiring variety and contract data; cleaning tick data according to the transaction date; and cleaning the K line data according to the transaction date, and writing the abnormal quotation information into a log file. Historical data processing: loading variety and contract information; loading a basic contract tick to generate an index tick; generating a high-frequency index K line and a low-frequency index K line by using an index tick; and (5) warehousing the K lines.
The data platform provides other platform data query services through the dubbo interface. High concurrency, high efficiency and real-time processing, high availability, stable system, reasonable data storage structure and data interaction design guarantee data processing performance and accuracy.
Strategy platform: as shown in fig. 12 and 13, muji-manager is the portal of the whole woodchicken system, and all front-end access requests are forwarded to the services of the background via muji-manager. The policy platform contains two main subsystems: 1. performing offline policy production management; 2. and (5) performing online policy operation management. The off-line strategy production mainly comprises 3 sub-modules: 1. a strategy production module (the main functions are flow control, report display, list query and the like); 2. a strategy retest optimization module (the main functions are the establishment and the operation of a strategy retest task, the scheduling of a parallel computing task and the injection of a strategy into a retest container); 3. during the operation of the online strategy, the factor calculation module establishes a network link with the measuring container, receives strategy signals from the measuring container, calculates strategy evaluation factors and scores (single curves) according to the signals, and calculates comprehensive scores (comprehensive curves) of the strategies on a plurality of varieties after all strategy examples are operated.
Setting a dimension rule in the strategy platform: a series of rules are defined under a certain dimension to describe the market, and indexes such as MACD, MA, RSI and the like are a set of rules, and meta-rules of different dimensions are combined and crossed with each other, so that better description capability is generated. Dimension element: in brief, a dimension element is a section of atomic logic (non-subdividable) for describing a market, belongs to a specific dimension element rule, has a multi/null direction, different elements under the same dimension element rule are mutually exclusive, and all elements are added to cover all market states on a time sequence. Drawing: and superposing permutation and combination of different periods through a plurality of dimension elements. To more accurately describe the market from multiple dimensions, the formula: an image is generated from a permutation and combination of 1 or more dimensions (meta) periods.
The policy platform includes a plurality of policy subunits: the system comprises an image subunit, an opening subunit, a leveling subunit, a fund management subunit, a wind control subunit and the like, wherein each type of subunit only can form strategies with different types of subunits, and a plurality of strategy subunits are arranged and combined to form a plurality of strategies and formulas: the image subunit, the open bin subunit, the flat bin subunit, the fund management subunit and the wind control subunit.
The business process of the strategy platform comprises the following steps: and (5) managing the dimension elements: uploading the uploaded wiki codes to a system, and supporting later maintenance; arranging dimension elements: selecting a plurality of dimension elements according to the dimensions, and carrying out full-permutation and combination to generate an image; like evaluation: performing retest, evaluation and screening on a batch of images generated by one-time dimension element arrangement to find images and parameters meeting conditions; and (3) policy subunit management: uploading the code of the strategy subunit to a system, and supporting later maintenance; strategy arrangement: the selected strategy subunits are arranged completely to generate strategies; strategy screening: according to a batch of strategies generated by strategy arrangement, carrying out one-by-one retest and screening according to specified screening rules, and taking the strategies meeting the conditions as effective strategies to carry out warehousing operation; strategy adaptation: aiming at the warehousing strategy passing the screening, parameter adaptation is carried out, and the best transaction variety corresponding to the effective parameter group is found; setting to be on-line: setting the strategy example (strategy + parameter + variety) meeting the conditions as a strategy to be online; strategy online: and pushing the strategy to be online from the strategy library to the online, and entering a real disk or simulation disk running state.
the strategy platform mainly comprises a product/option/factor/muji-stream-api. In order to avoid circular dependence, the interaction principle of each module is that an upstream module calls a downstream module to adopt dobbo service, and the downstream module informs the upstream module to adopt an MQ mode. The product module is responsible for image evaluation main flow control, service query and metadata management, and when the image return time is involved in the image evaluation process, the return test can be realized by calling option through dubbo; the optimization is responsible for backtesting, optimizing and scheduling backtesting tasks of the image and the strategy, and the backtesting time needs to acquire a container, inject the strategy and send a strategy initial instruction by calling an infrastructure through a dubbo; the Infrastructure is responsible for interacting with the policy container, including the management of the online policy container and the offline policy container; muji-stream-api: the system is responsible for strategy online and online strategy start and stop; factor: and the system is responsible for receiving orders from the offline container corrlator during retesting, calculating strategy factors and scores, and informing the option or product module of completion of calculation by sending MQ messages.
And (3) testing container clusters: the return measurement container runs the return measurement task concurrently, the resource consumption is large, a cluster mode is adopted, the cluster comprises 6 machines, each node is provided with a container process, and horizontal expansion is supported.
Factor calculation clustering: and the occupied resources are more, the single machine is deployed with multiple instances, one of the single machine is deployed with 4 services, the other one is deployed with 2 services, and the number of the services and the number of the test containers are 1: 1.
The online policy container cluster: and an online strategy is operated, a single-node multi-container is supported, and horizontal expansion is supported.
Like the evaluation process:
The Manager receives an image evaluation task starting request, calls the image evaluation task starting request through the dubbo and transmits the image evaluation task starting request to the product module;
the product starts image evaluation and optimizes step by step, and each image of each layer needs to call option for retesting;
The Opticle receives the image retest request, creates an image retest task, explodes the parameters (permutation and combination), and starts the retest task;
Initiating a retest task includes: acquiring available strategy container resources, injecting a strategy into a container, and sending a starting instruction to the container; when the strategy in the container return test enters into operation, an order signal is sent to a Factor, and the Factor receives the order and starts to calculate the strategy performance; after the Factor finishes calculating the strategy performance, a measurement completion notice is sent back to the Optize, and the Optize carries out task ending processing according to the notice, wherein the task ending processing comprises task body modification and container resource release.
As shown in fig. 15, policy container-asynchronous event driven process: different market signal driving modes are adopted for different environments: if the environment is a survey environment, directly reading the local Sqlite file sending market through a monitor inside the container, and if the environment is an online container, subscribing the market from the Data-Source; asynchronous event delivery: the quotation/signal is circulated in the strategy container in an asynchronous event mode; the online strategy supports the conclusion of market quotation subscription and driving of Bar, CurrentBar and Tick full cycle/full variety; the historical market data is read from the interior of the strategy and cached by using a local memory, so that a large number of IO operations are reduced, and the operation time of the strategy is compressed to microsecond level; after the strategy in the container generates the signal, whether the signal is a retest or an online is judged according to the environment variable, if the signal is a retest task, the signal is sent to the Factor module for performance calculation, and if the signal is an online container, the signal is directly sent to the trading module and then forwarded to the trading market.
The parallel computing task scheduling process of the strategy module comprises the following steps: one request submitted by a user corresponds to a parent task, the parent task does not directly enter a task waiting queue, but is firstly split into atomic tasks with relatively uniform granularity, and the atomic tasks enter the waiting queue to wait for being scheduled; the task waiting queue is sorted according to the task priority set by a user, the higher priority close to the Head is firstly operated, and the same priority is sorted according to the time sequence; the scheduling module circularly obtains tasks corresponding to the Head of the waiting queue, then obtains available computing resources, and if available idle resources exist, the Head tasks are moved to a running task list and enter a running state; the completed task is removed from the running task list.
Particularly, a task scheduling module of the optimal subsystem manages the running and resource allocation of the atomic task, a tree structure is constructed in the task creating process, a root node represents a task which can be seen by a user, and leaf nodes are the atomic task; the atomic tasks are sorted according to the priority, enter a task waiting queue, a task scheduling module polls from the head of the waiting queue at regular time, if available computing resources are found, the resources are distributed to the tasks, the head tasks are moved to a running list, and task retesting running is started; the Optimize receives the notice from the Factor subsystem, starts the processing flow of task completion, if the task is successfully finished, the resources are released, and the task is notified to schedule and start the next task, and in addition, some business processing flows are provided; if the task fails, resources are released, the task is retried for 3 times, if the task fails to be retried for 3 times, the task is considered to really fail in business, then a failure processing flow is entered, and the failure of the atomic task affects the state of a parent task, including the state of a root node of a task tree and the state of a sibling task.
The strategy module forms a tree structure of the portrait through dimension element arrangement, and each node on the tree structure performs full-market retest evaluation by adopting a layer-by-layer optimization mode. And (3) image evaluation step-by-step optimization process: the image evaluation is to evaluate each node of a multi-layer tree structure of the image generated by one-time dimension element arrangement; each node represents an image, each image has a score and an optimal parameter corresponding to the score after being evaluated, only the image with the score larger than 0.6 is considered as a valid image in service, and the parameter with the highest score is used as the optimal parameter of the image. In order to reduce the calculation amount, the optimal parameters corresponding to the upper node (image) are inherited to the next image, and the lower image is subjected to exploding (parameter arrangement-Cartesian product) back evaluation on the vitamin elements of the layer on the basis of the optimal parameters of the upper image. The evaluation criteria for images at different levels are different, and for one level of images, the second level of images is the unit time loss according to net profit. In order to prevent the problem that the generated effective image is over-fitted, a comprehensive curve of all varieties is adopted to carry out a platform in the evaluation process, and meanwhile, a parameter plain algorithm is also applied to parameter optimization, so that the effective image can be evaluated to participate in strategy arrangement.
The cluster performance optimization scheme has the technical effects that:
resources are dynamically managed and distributed through a task scheduling engine and a resource pool management module, a large task is divided into small retest tasks, and a plurality of small retest tasks run on different machines at the same time. A cluster consisting of 6 servers is adopted for parallel backtesting, and the cluster throughput is 185.53 kbar/s. Using six years data, using double average dimension elements, cycle 15 minutes, only one layer of image, each image explosion 492 sets of parameters, on 6 servers to perform the whole variety image evaluation, 28536 examples, average time consumption 7.2 minutes, according to the result estimation, using 6 years data to produce, can test 570.72 ten thousand strategies each day.
The step-by-step optimization algorithm scheme has the technical effects that:
The dimension element arrangement arranges a plurality of dimension elements into a multi-stage tree structure in a permutation and combination mode, evaluates from the root of the tree step by step, searches for an optimal parameter, and transmits the optimal parameter to the next-stage image. The method has the advantages that a large number of invalid operations are reduced, computing resources are saved, the double-average line with 3 periods is taken as an example, each node is 100 groups of parameters, after arrangement, the first layer is 2 nodes, the second layer is 4 nodes, and the third layer is 8 nodes.
Adopting full-array mixed-level optimizing operation quantity: 100 × 2+ (100 × 4) + (100 × 8) ═ 8040200;
Adopting step-by-step optimization operation: 100 x 2+100 x 4+100 x 8 ═ 1400;
Compared with the prior art, the fully-arranged mixed-level optimizing computation amount is more than 5700 times of the step-by-step optimizing, and the system computation amount after the step-by-step optimizing is only 1/5700 of the full arrangement, so that the improvement effect is obvious.
And (3) under-fitting solution: under-fitting means that the sample data size of general backtesting is too small, so that the fitting strategy cannot meet the requirements, the error is large, the solution is a large time range, the time range of the backtesting of large samples is 2012-01-01 to 2019-3-31 years aiming at the national futures market currently, and the data is nearly 7 years; the other method is to conduct retest for the whole market, such as image evaluation and strategy screening of domestic futures, and select 29 varieties with good liquidity to conduct retest, so that the problem of under-fitting can be effectively avoided.
Solution of overfitting: the overfitting refers to that the strategy overfitts historical data without generalization capability, the maximum layer number of the strategy image is limited to 3 levels in the image evaluation process, meanwhile, plain parameters and parameter isolated islands are adopted, in addition, separate tests of the inside and outside of a sample are adopted, a strategy is trained in the sample, and the test of the outside of the sample is used for verifying the effectiveness of the strategy generated in the sample.
Parameter plain algorithm: determining a target factor: determining a factor from the existing evaluation factors as a basic evaluation factor (such as net profit, annual profitability and the like) of the plain algorithm; determining the neighbor length: newly increased, the adjacent neighbor length is a positive integer. The parameter set range is used for determining the parameter plain, the default value is 2, and the business personnel can modify the parameter set range. Setting a parameter group with M parameters, finding out a neighboring parameter group (+ N-N steps, when taking the neighboring parameter of the parameter, other parameters in the parameter group are fixed, each 1 parameter has 2N neighboring parameter groups and a target factor value (not including the parameter itself), M parameters has M × 2N neighboring parameter groups, and finally adding the parameter itself, the parameter group and the corresponding target factor value are total M × 2N +1 parameter groups. For example: in the double-average strategy, a Fast parameter group (1-50) corresponds to 50 parameters, and the step length is 1; the Slow parameter group (0-200) corresponds to 21 parameters, and the step length is 10; if the parameter set Fast 6 and Slow 40 are selected, the parameter set takes values in the ranges of (6,20), (6,30), (6,40), (6,50), (6,60), (4,40), (5,40), (7,40), and (8, 40).
The strategy production process adopts a multi-level report form: as a large amount of intermediate temporary data are generated in the process of strategy production, a large amount of storage space is occupied, reports of different levels are adopted according to different business requirements in order to save the consumption of storage resources, and a background program stores data according to the levels of the reports. Storage and computation quantity order: easy report > general report > scoring report; for low level reports, if the user needs to see more data, the high level report data can be seen using the report expansion function.
A strategy comprehensive scoring formula: f is n + m/100;
Wherein, n is annual lever-free profit/maximum withdrawal (representing profit capacity, the higher profit capacity is stronger at the same risk level), m is (PR-Q)/R (representing effectiveness, P-win ratio, R-profit-loss ratio, Q is 1-P), and the comprehensive score f is more than 0.6, which is regarded as an effective strategy.
Stock multi-factor calculation:
Net worth performance:
annual rate of return: (netValue (end) -1) ^ (250/length (netValue)), netValue (end) is the net value sequence end value, and length (netValue) is the net value sequence length;
the ratio of the kama: annual rate of return/historical maximum withdrawal;
The sharp ratio: (annual profitability-0.03)/annual fluctuation rate; the annual fluctuation rate std (r _ i) × v 250.
And IC (information coefficient) is that IC (t) is corr (F (t), R (t +1)), and the correlation coefficient (the prediction capability of the reflection factor to the future profit) of the current-period factor value sequence F (t) and the next-period yield sequence R (t +1) is a sequence value.
IC mean is the mean value of the IC sequence;
IC std.: standard deviation of IC sequence;
Ann.ICIR:
Ann.ICIR=(mean(IC))/(std(IC))√C
wherein C is the number of times of adjusting the bin in one year.
The hand-off rate turnover is that the number of stocks called out by the ith group in the next period/the total number of stocks in the ith group in the current period;
Annual hand change rate Ann. Turnover, hand change rate Turnover and annual number of times to change bins C.
As shown in fig. 18, automated production of policies: and creating a total scheduling task in the product, receiving page data, storing the page data into mongo, starting dimension element arrangement, sending an MQ notification to the product module after the task is successful, and automatically constructing the whole task to be stopped if a certain step fails to be executed. Because the manual operation has a plurality of steps from the beginning of uploading the dimension elements to the end of generating the effective strategy, the operation is more complicated, the function of one line can be added in the automatic production process of the strategy, the dimension element arrangement, the image evaluation, the strategy arrangement and the strategy screening process are spliced into one process, and the operation steps are simplified.
The policy platform employs a machine learning algorithm to compute a weight for the proportion of funds for each policy in the combination. The goal of the online portfolio is to select portfolio weights at each time period to maximize its ultimate wealth.
As shown in fig. 19, in order to prevent overfitting during large-scale policy production (parallel computation) -policy retest, the retest is performed in a long-period and full-market manner when the policy is retested, the computation is huge, and a single machine cannot meet the retest performance requirement, so that a multi-machine parallel/single-machine multi-thread retest architecture is designed to support the retest of a large number of policy instances. And (3) a strategy retest task generates too many strategy instances which exceed the upper limit of the running capacity of one machine, the task is automatically split into subtasks, and the splitting granularity of the subtasks is based on the upper limit of the strategy instances which are simultaneously run by a single machine.
technical effect comparative example:
The scheme is as follows: the method has the advantages that single-machine multithreading is adopted, each machine runs 200 threads simultaneously, no shared data and no concurrent lock exist between every two threads, a large amount of IO operations are reduced by adopting local memory cache common data and a K-line condition optimization algorithm, CPU occupation is close to 100% after a task is started, time slices are concentrated on user time, and multi-core CPU performance is utilized to the maximum extent.
As a result: the six-year data of the domestic futures market is used for one-minute data in the past, parallel return measurement is carried out on 6 servers, the average time consumption of a single instance is 0.5 second, the average throughput of a single machine is 47.83 ten thousand bar/second, and compared with an industrially mainstream return measurement platform Xquant developed by a large merchant, the single machine performance is 12.5 times that of the Xquant.
As shown in fig. 20, large scale strategy runs (linear expansion): the policy for simultaneous operation of a single policy container is limited, generally, a process can be guaranteed within 1000 policies, and assuming that an average policy can allocate 10 ten thousand of funds, this limits the platform to manage only 1 hundred million-scale funds (1000 × 10 ten-1 million), but if 100 ten thousand policies are to be operated, the policy container is required to support the cluster and have the capability of linear horizontal expansion.
The present invention is not limited to the above-described examples, and various changes can be made without departing from the spirit and scope of the present invention within the knowledge of those skilled in the art.

Claims (9)

1. A full market multi-variety financial asset management system based on an intelligent policy platform, comprising:
The trading platform is responsible for receiving and sending a ticket reporting instruction, executing algorithm trading and carrying out a function of receiving and jointing a market ticket reporting;
the data platform is responsible for service market access and production of related derivative market;
The strategy platform is responsible for strategy production and strategy on-line operation with high performance, and comprises an off-line strategy production management subsystem and an on-line strategy operation management subsystem, wherein the strategy platform adopts a machine learning algorithm to calculate the weight of the capital proportion of each strategy in the combination, selects the weight of the investment combination in each period and maximizes the final income; the strategy platform comprises an online strategy container cluster supporting single-node multi-container and horizontal expansion; and in the online operation link, a CEP event processing engine is adopted and set up on an Apama container to transfer the core service.
2. the intelligent policy platform based whole market multi-variety financial asset management system according to claim 1, wherein the off-line policy production management subsystem comprises 3 sub-modules: the strategy production module is used for realizing the functions of flow control, report display and list query; the strategy retest optimization module is used for realizing the establishment and the operation of a strategy retest task, the scheduling of a parallel computing task and the injection of a strategy into a retest container; the test-back container module is used for operating the strategy, sending quotation to the strategy, calculating fund and position taking when operating the strategy and providing strategy API common strategy calling during test-back; and during the operation of the online strategy operation management subsystem, the factor calculation module establishes network link with the test container, receives strategy signals from the test container, calculates strategy evaluation factors and evaluation single curves according to the signals, and calculates comprehensive evaluation comprehensive curves of the strategies on a plurality of varieties after all strategy instances are operated.
3. The intelligent strategy platform based whole market multi-variety financial resource management system according to claim 2, wherein the strategy platform comprises a product module, an option module, a factor module and a stream-api module, each module interaction principle is that an upstream module calls a downstream module to adopt dobbo service, and the downstream module informs the upstream module to adopt MQ mode; the product module is responsible for image evaluation main flow control, service query and metadata management, and realizes retest by calling option through dubbo when image retest is involved in the image evaluation process; the optimization is responsible for backtesting, optimizing and scheduling backtesting tasks of the image and the strategy, and the backtesting time needs to acquire a container, inject the strategy and send a strategy initial instruction by calling an infrastructure through a dubbo; the Infrastructure is responsible for interacting with the policy container, including the management of the online policy container and the offline policy container; the stream-api is responsible for strategy online and online strategy start and stop; the factor is responsible for receiving orders from the offline container corrlator during retest, calculating strategy factors and scores, and informing the option or the product module of completion of calculation by sending MQ messages.
4. the intelligent policy platform based whole market multi-item financial asset management system according to claim 3, wherein the policy platform comprises a survey container cluster: the cluster comprises 6 machines, each node is provided with a container process, and horizontal expansion is supported; the policy platform includes a factor computation cluster: and (3) single-machine multi-instance deployment, wherein one of the single-machine multi-instance deployment deploys 4 services, the other one deploys 2 services, and the number of the services and the number of the callback containers are 1: 1.
5. The intelligent policy platform based whole market multi-variety financial asset management system according to claim 4, wherein the policy platform image evaluation process: the Manager receives an image evaluation task starting request, calls the image evaluation task starting request through the dubbo and transmits the image evaluation task starting request to the product module; the product starts image evaluation and optimizes step by step, and each image of each layer needs to call option for retesting; and the Optize receives the image retest request, creates an image retest task, realizes parameter opening through permutation and combination, and starts the retest task.
6. The intelligent policy platform based whole market multi-variety financial resource management system according to claim 5, wherein the start-up of the policy platform for the return test task comprises: acquiring available strategy container resources, injecting a strategy into a container, and sending a starting instruction to the container; when the strategy in the container return test enters into operation, an order signal is sent to a Factor, and the Factor receives the order and starts to calculate the strategy performance; after the Factor finishes calculating the strategy performance, a measurement completion notice is sent back to the Optize, and the Optize carries out task ending processing according to the notice, wherein the task ending processing comprises task body modification and container resource release.
7. the intelligent policy platform based whole market multi-item financial asset management system according to claim 6, wherein the policy container asynchronous event driven process of the policy platform: if the environment is a survey environment, directly reading the local Sqlite file sending market through a monitor inside the container; if the container is an online container, subscribing market information from the Data-Source; asynchronous event delivery: the quotation/signal is circulated in the strategy container in an asynchronous event mode; the online strategy supports the complete Bar, CurrentBar, Tick full-period and full-variety market subscription and drive; reading historical market data in the strategy by adopting local memory cache to reduce a large amount of IO operations, and compressing the operation time of the strategy to microsecond level; after the strategy in the container generates signals, whether the container is a retest or an online container is judged according to environment variables, if the container is a retest task, the signals are sent to a Factor module for performance calculation, and if the container is an online container, the signals are directly sent to a trading module and then forwarded to a trading market.
8. the intelligent policy platform based market wide multi-item financial asset management system according to claim 7, wherein the under-fit resolution: carrying out retest aiming at the whole market, selecting a variety with better fluidity to carry out retest, and avoiding the problem of under-fitting through image evaluation and strategy screening; solution of overfitting: in the image evaluation process, the maximum layer number of the strategy image is limited to 3 levels, a plain parameter and a limiting parameter island are adopted, in addition, the separate test of the inside and the outside of a sample is adopted, the strategy is trained in the sample, and the test of the outside of the sample is used for verifying the effectiveness of the strategy generated in the sample.
9. The intelligent policy platform based whole market multi-variety financial asset management system according to claim 8, wherein the automated production process of the policy comprises: and creating a total scheduling task in the product, receiving page data, storing the page data into mongo, starting dimension element arrangement, sending an MQ notification to the product module after the task is successful, and automatically constructing the whole task to be stopped if a certain step fails to be executed.
CN201910844652.6A 2019-09-06 2019-09-06 Whole market multi-variety gold financing management system based on intelligent strategy platform Active CN110570307B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201910844652.6A CN110570307B (en) 2019-09-06 2019-09-06 Whole market multi-variety gold financing management system based on intelligent strategy platform

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201910844652.6A CN110570307B (en) 2019-09-06 2019-09-06 Whole market multi-variety gold financing management system based on intelligent strategy platform

Publications (2)

Publication Number Publication Date
CN110570307A true CN110570307A (en) 2019-12-13
CN110570307B CN110570307B (en) 2024-01-05

Family

ID=68778313

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201910844652.6A Active CN110570307B (en) 2019-09-06 2019-09-06 Whole market multi-variety gold financing management system based on intelligent strategy platform

Country Status (1)

Country Link
CN (1) CN110570307B (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111652682A (en) * 2020-05-29 2020-09-11 平安证券股份有限公司 Business data matching method and device, electronic equipment and computer storage medium
CN112734177A (en) * 2020-12-28 2021-04-30 四川新网银行股份有限公司 Wind control system and method for intelligent shunting automatic decision

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106022916A (en) * 2016-04-27 2016-10-12 南京大学 Quantitative transaction method based on virtualization and backtesting library
CN106934716A (en) * 2017-03-10 2017-07-07 燧石科技(武汉)有限公司 Based on the multimode automated transaction system that network distribution type is calculated
US20170213126A1 (en) * 2016-01-27 2017-07-27 Bonsai AI, Inc. Artificial intelligence engine configured to work with a pedagogical programming language to train one or more trained artificial intelligence models
CN107797797A (en) * 2017-11-03 2018-03-13 上海宽全智能科技有限公司 Quantify back to survey with quantifying method of commerce and device, storage medium, equipment and system

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20170213126A1 (en) * 2016-01-27 2017-07-27 Bonsai AI, Inc. Artificial intelligence engine configured to work with a pedagogical programming language to train one or more trained artificial intelligence models
CN106022916A (en) * 2016-04-27 2016-10-12 南京大学 Quantitative transaction method based on virtualization and backtesting library
CN106934716A (en) * 2017-03-10 2017-07-07 燧石科技(武汉)有限公司 Based on the multimode automated transaction system that network distribution type is calculated
CN107797797A (en) * 2017-11-03 2018-03-13 上海宽全智能科技有限公司 Quantify back to survey with quantifying method of commerce and device, storage medium, equipment and system

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111652682A (en) * 2020-05-29 2020-09-11 平安证券股份有限公司 Business data matching method and device, electronic equipment and computer storage medium
CN111652682B (en) * 2020-05-29 2024-04-30 平安证券股份有限公司 Service data matching method and device, electronic equipment and computer storage medium
CN112734177A (en) * 2020-12-28 2021-04-30 四川新网银行股份有限公司 Wind control system and method for intelligent shunting automatic decision
CN112734177B (en) * 2020-12-28 2023-07-21 四川新网银行股份有限公司 Wind control method for intelligent diversion automatic decision

Also Published As

Publication number Publication date
CN110570307B (en) 2024-01-05

Similar Documents

Publication Publication Date Title
CN110634071B (en) Full market multi-variety intelligent fund financing system based on automatic quantitative trading platform
CN110619580B (en) Whole market multi-variety gold financing system based on intelligent data processing platform
CN110580614B (en) Full market multi-variety gold financing management system based on mass policy intelligent processing platform
US10713594B2 (en) Systems, methods, and apparatuses for implementing machine learning model training and deployment with a rollback mechanism
US11631136B2 (en) Methods and systems for low latency generation and distribution of quote price direction estimates
CN110533540A (en) A kind of whole city multi items finance money guard system based on intelligence dimension Meta-Policy platform
CN112365355B (en) Method, device and readable medium for calculating foundation valuation and risk index in real time
CN115310766B (en) Enterprise management system based on big data analysis
CN110610422B (en) Full market multi-variety gold financing management system based on intelligent effective strategy determination platform
CN110570307B (en) Whole market multi-variety gold financing management system based on intelligent strategy platform
CN110619581B (en) Full market multi-variety intelligent gold financing management system based on automatic quantification micro-service subsystem
US20220172298A1 (en) Utilizing a machine learning model for predicting issues associated with a closing process of an entity
Mondal et al. Role of machine learning in ETL automation
CN110619507A (en) Whole-market multi-variety intelligent fund financing management system
Deldari et al. A survey on preemptible IaaS cloud instances: challenges, issues, opportunities, and advantages
Wan et al. A financial transaction methods based on mapreduce technology and blockchain
CN110428331A (en) Quantitative transaction data processing system and method based on cloud platform
CN109146672A (en) It is a kind of that matching process and device are brought together based on production model
US20210166177A1 (en) Adjusting a master build plan based on events using machine learning
KR100936272B1 (en) Method for managing bond trading system based on independent function modules
Suryawanshi et al. A Stochastic Programming Approach to Design Perishable Product Supply Chain Network Under Different Disruptions
Meng et al. Financial Analysis System Design for Institutional Users Based on Deep Learning
US20240012820A1 (en) Transaction exchange platform with classification microservice to generate alternative workflows
US20240012676A1 (en) Transaction exchange platform to determine secondary workflows for transaction object processing
Sison et al. Survey of Machine Learning and Deep Learning Techniques for Travel Demand Forecasting

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
TA01 Transfer of patent application right

Effective date of registration: 20191204

Address after: 100094 101-06, floor 1, building 1, No. 103, Beiqing Road, Haidian District, Beijing

Applicant after: Beijing Shenzhou Tongdao Intelligent Information Technology Co.,Ltd.

Address before: 1413, floor 14, building 3, yard 8, Chama street, Xicheng District, Beijing 100055

Applicant before: Beijing Shenzhou Tongdao Intelligent Technology Co.,Ltd.

TA01 Transfer of patent application right
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant